2023-11-27T14:53:57.268 INFO:root:teuthology version: 0.0.1.dev205+g2442239 2023-11-27T14:53:57.268 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/misc}} --name choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi --owner scheduled_choffman@teuthology --verbose --archive /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467810 -- /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467810/orig.config.yaml 2023-11-27T14:53:57.325 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-27T14:53:57.408 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467810 branch: wip-choffman-62793 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/misc}} email: null first_in_suite: false job_id: '7467810' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-choffman-62793 ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 1c9565146c8647d4e00eb6c7ae626dc192680e7b subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 1c9565146c8647d4e00eb6c7ae626dc192680e7b kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - wsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-62793 sha1: 5e4a743ad7fd1422ebd6c4dc950a21a303279207 owner: scheduled_choffman@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8430 sha1: 1c9565146c8647d4e00eb6c7ae626dc192680e7b sleep_before_teardown: 0 suite: fs suite_branch: wip-62793 suite_path: /home/teuthworker/src/github.com_chrisphoffman_ceph_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa suite_relpath: qa suite_repo: https://github.com/chrisphoffman/ceph.git suite_sha1: 5e4a743ad7fd1422ebd6c4dc950a21a303279207 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi052.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi102.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi150.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - fs/misc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 2442239d2653456406c25ae0c71b689c8f2657b6 timestamp: 2023-11-27_14:43:37 tube: smithi user: choffman verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-27T14:53:57.408 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_chrisphoffman_ceph_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa; will attempt to use it 2023-11-27T14:53:57.409 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_chrisphoffman_ceph_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks 2023-11-27T14:53:57.409 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-27T14:53:57.410 INFO:teuthology.task.internal:Checking packages... 2023-11-27T14:53:57.433 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '1c9565146c8647d4e00eb6c7ae626dc192680e7b' 2023-11-27T14:53:57.434 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-27T14:53:57.434 INFO:teuthology.packaging:ref: None 2023-11-27T14:53:57.434 INFO:teuthology.packaging:tag: None 2023-11-27T14:53:57.434 INFO:teuthology.packaging:branch: wip-choffman-62793 2023-11-27T14:53:57.434 INFO:teuthology.packaging:sha1: 1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-27T14:53:57.434 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-choffman-62793 2023-11-27T14:53:57.581 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-3906.g1c956514 2023-11-27T14:53:57.582 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-27T14:53:57.588 INFO:teuthology.task.internal:no buildpackages task found 2023-11-27T14:53:57.589 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-27T14:53:57.602 INFO:teuthology.task.internal:Saving configuration 2023-11-27T14:53:57.623 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-27T14:53:57.629 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-27T14:53:57.655 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi052.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467810', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-27 14:48:52.493518', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-27T14:53:57.680 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi102.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467810', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-27 14:48:52.495442', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-27T14:53:57.703 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467810', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-27 14:48:52.497152', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-27T14:53:57.703 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-27T14:53:57.709 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-11-27T14:53:57.710 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-11-27T14:53:57.710 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-11-27T14:53:57.710 INFO:teuthology.run_tasks:Running task console_log... 2023-11-27T14:53:57.811 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd4cbb1aaf0>, signals=[15]) 2023-11-27T14:53:57.812 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-27T14:53:57.818 INFO:teuthology.task.internal:Opening connections... 2023-11-27T14:53:57.818 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi052.front.sepia.ceph.com 2023-11-27T14:53:57.819 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-27T14:53:57.888 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi102.front.sepia.ceph.com 2023-11-27T14:53:57.890 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-27T14:53:57.957 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi150.front.sepia.ceph.com 2023-11-27T14:53:57.957 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-27T14:53:58.026 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-27T14:53:58.033 DEBUG:teuthology.orchestra.run.smithi052:> uname -m 2023-11-27T14:53:58.054 INFO:teuthology.orchestra.run.smithi052.stdout:x86_64 2023-11-27T14:53:58.055 DEBUG:teuthology.orchestra.run.smithi052:> cat /etc/os-release 2023-11-27T14:53:58.115 INFO:teuthology.orchestra.run.smithi052.stdout:NAME="Red Hat Enterprise Linux" 2023-11-27T14:53:58.115 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION="8.6 (Ootpa)" 2023-11-27T14:53:58.115 INFO:teuthology.orchestra.run.smithi052.stdout:ID="rhel" 2023-11-27T14:53:58.115 INFO:teuthology.orchestra.run.smithi052.stdout:ID_LIKE="fedora" 2023-11-27T14:53:58.115 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION_ID="8.6" 2023-11-27T14:53:58.115 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM_ID="platform:el8" 2023-11-27T14:53:58.115 INFO:teuthology.orchestra.run.smithi052.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-27T14:53:58.116 INFO:teuthology.orchestra.run.smithi052.stdout:ANSI_COLOR="0;31" 2023-11-27T14:53:58.116 INFO:teuthology.orchestra.run.smithi052.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-27T14:53:58.116 INFO:teuthology.orchestra.run.smithi052.stdout:HOME_URL="https://www.redhat.com/" 2023-11-27T14:53:58.116 INFO:teuthology.orchestra.run.smithi052.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-27T14:53:58.116 INFO:teuthology.orchestra.run.smithi052.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-27T14:53:58.116 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T14:53:58.116 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-27T14:53:58.116 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-27T14:53:58.116 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-27T14:53:58.116 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-27T14:53:58.117 INFO:teuthology.lock.ops:Updating smithi052.front.sepia.ceph.com on lock server 2023-11-27T14:53:58.136 DEBUG:teuthology.orchestra.run.smithi102:> uname -m 2023-11-27T14:53:58.157 INFO:teuthology.orchestra.run.smithi102.stdout:x86_64 2023-11-27T14:53:58.157 DEBUG:teuthology.orchestra.run.smithi102:> cat /etc/os-release 2023-11-27T14:53:58.217 INFO:teuthology.orchestra.run.smithi102.stdout:NAME="Red Hat Enterprise Linux" 2023-11-27T14:53:58.217 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION="8.6 (Ootpa)" 2023-11-27T14:53:58.217 INFO:teuthology.orchestra.run.smithi102.stdout:ID="rhel" 2023-11-27T14:53:58.218 INFO:teuthology.orchestra.run.smithi102.stdout:ID_LIKE="fedora" 2023-11-27T14:53:58.218 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION_ID="8.6" 2023-11-27T14:53:58.218 INFO:teuthology.orchestra.run.smithi102.stdout:PLATFORM_ID="platform:el8" 2023-11-27T14:53:58.218 INFO:teuthology.orchestra.run.smithi102.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-27T14:53:58.218 INFO:teuthology.orchestra.run.smithi102.stdout:ANSI_COLOR="0;31" 2023-11-27T14:53:58.218 INFO:teuthology.orchestra.run.smithi102.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-27T14:53:58.219 INFO:teuthology.orchestra.run.smithi102.stdout:HOME_URL="https://www.redhat.com/" 2023-11-27T14:53:58.219 INFO:teuthology.orchestra.run.smithi102.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-27T14:53:58.219 INFO:teuthology.orchestra.run.smithi102.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-27T14:53:58.219 INFO:teuthology.orchestra.run.smithi102.stdout: 2023-11-27T14:53:58.219 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-27T14:53:58.219 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-27T14:53:58.219 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-27T14:53:58.219 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-27T14:53:58.219 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server 2023-11-27T14:53:58.245 DEBUG:teuthology.orchestra.run.smithi150:> uname -m 2023-11-27T14:53:58.264 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2023-11-27T14:53:58.265 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2023-11-27T14:53:58.325 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="Red Hat Enterprise Linux" 2023-11-27T14:53:58.325 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="8.6 (Ootpa)" 2023-11-27T14:53:58.325 INFO:teuthology.orchestra.run.smithi150.stdout:ID="rhel" 2023-11-27T14:53:58.325 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE="fedora" 2023-11-27T14:53:58.325 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="8.6" 2023-11-27T14:53:58.325 INFO:teuthology.orchestra.run.smithi150.stdout:PLATFORM_ID="platform:el8" 2023-11-27T14:53:58.325 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-27T14:53:58.325 INFO:teuthology.orchestra.run.smithi150.stdout:ANSI_COLOR="0;31" 2023-11-27T14:53:58.325 INFO:teuthology.orchestra.run.smithi150.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-27T14:53:58.326 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://www.redhat.com/" 2023-11-27T14:53:58.326 INFO:teuthology.orchestra.run.smithi150.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-27T14:53:58.326 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-27T14:53:58.326 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-27T14:53:58.326 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-27T14:53:58.326 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-27T14:53:58.326 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-27T14:53:58.326 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-27T14:53:58.326 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2023-11-27T14:53:58.344 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-27T14:53:58.355 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-27T14:53:58.361 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-27T14:53:58.361 DEBUG:teuthology.orchestra.run.smithi052:> test '!' -e /home/ubuntu/cephtest 2023-11-27T14:53:58.363 DEBUG:teuthology.orchestra.run.smithi102:> test '!' -e /home/ubuntu/cephtest 2023-11-27T14:53:58.365 DEBUG:teuthology.orchestra.run.smithi150:> test '!' -e /home/ubuntu/cephtest 2023-11-27T14:53:58.384 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-27T14:53:58.391 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-27T14:53:58.392 DEBUG:teuthology.orchestra.run.smithi052:> test -z $(ls -A /var/lib/ceph) 2023-11-27T14:53:58.423 DEBUG:teuthology.orchestra.run.smithi102:> test -z $(ls -A /var/lib/ceph) 2023-11-27T14:53:58.425 DEBUG:teuthology.orchestra.run.smithi150:> test -z $(ls -A /var/lib/ceph) 2023-11-27T14:53:58.463 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-27T14:53:58.541 INFO:teuthology.run_tasks:Running task kernel... 2023-11-27T14:53:58.555 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-11-27T14:53:58.555 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2023-11-27T14:53:58.555 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2023-11-27T14:53:58.555 DEBUG:teuthology.orchestra.run.smithi052:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-27T14:53:58.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T14:53:58.575 DEBUG:teuthology.orchestra.run.smithi052:> uname -r 2023-11-27T14:53:58.636 INFO:teuthology.orchestra.run.smithi052.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-27T14:53:58.636 INFO:teuthology.task.kernel:Running kernel on smithi052: 4.18.0-372.9.1.el8.x86_64 2023-11-27T14:53:58.636 DEBUG:teuthology.orchestra.run.smithi052:> sudo yum install -y kernel 2023-11-27T14:53:59.949 INFO:teuthology.orchestra.run.smithi052.stdout:Updating Subscription Management repositories. 2023-11-27T14:53:59.949 INFO:teuthology.orchestra.run.smithi052.stdout:Unable to read consumer identity 2023-11-27T14:54:01.409 INFO:teuthology.orchestra.run.smithi052.stdout:Copr repo for python3-asyncssh owned by ceph 15 kB/s | 3.5 kB 00:00 2023-11-27T14:54:02.268 INFO:teuthology.orchestra.run.smithi052.stdout:Extra Packages for Enterprise Linux 19 MB/s | 16 MB 00:00 2023-11-27T14:54:05.960 INFO:teuthology.orchestra.run.smithi052.stdout:lab-extras 451 kB/s | 24 kB 00:00 2023-11-27T14:54:06.720 INFO:teuthology.orchestra.run.smithi052.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-11-27T14:54:06.775 INFO:teuthology.orchestra.run.smithi052.stdout:Dependencies resolved. 2023-11-27T14:54:06.776 INFO:teuthology.orchestra.run.smithi052.stdout:Nothing to do. 2023-11-27T14:54:06.776 INFO:teuthology.orchestra.run.smithi052.stdout:Complete! 2023-11-27T14:54:06.895 DEBUG:teuthology.orchestra.run.smithi052:> echo no | sudo yum reinstall kernel || true 2023-11-27T14:54:07.244 INFO:teuthology.orchestra.run.smithi052.stdout:Updating Subscription Management repositories. 2023-11-27T14:54:07.245 INFO:teuthology.orchestra.run.smithi052.stdout:Unable to read consumer identity 2023-11-27T14:54:07.523 INFO:teuthology.orchestra.run.smithi052.stdout:Last metadata expiration check: 0:00:02 ago on Mon 27 Nov 2023 02:54:05 PM UTC. 2023-11-27T14:54:07.568 INFO:teuthology.orchestra.run.smithi052.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-27T14:54:07.629 INFO:teuthology.orchestra.run.smithi052.stderr:Error: No packages marked for reinstall. 2023-11-27T14:54:07.676 DEBUG:teuthology.orchestra.run.smithi052:> sudo yum reinstall -y kernel || true 2023-11-27T14:54:08.026 INFO:teuthology.orchestra.run.smithi052.stdout:Updating Subscription Management repositories. 2023-11-27T14:54:08.026 INFO:teuthology.orchestra.run.smithi052.stdout:Unable to read consumer identity 2023-11-27T14:54:08.298 INFO:teuthology.orchestra.run.smithi052.stdout:Last metadata expiration check: 0:00:03 ago on Mon 27 Nov 2023 02:54:05 PM UTC. 2023-11-27T14:54:08.343 INFO:teuthology.orchestra.run.smithi052.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-27T14:54:08.364 INFO:teuthology.orchestra.run.smithi052.stderr:Error: No packages marked for reinstall. 2023-11-27T14:54:08.411 DEBUG:teuthology.orchestra.run.smithi052:> rpm -q kernel | sort -rV | head -n 1 2023-11-27T14:54:08.494 INFO:teuthology.orchestra.run.smithi052.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-11-27T14:54:08.494 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-11-27T14:54:08.495 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-11-27T14:54:08.495 DEBUG:teuthology.parallel:result is None 2023-11-27T14:54:08.495 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-27T14:54:08.504 INFO:teuthology.task.internal:Creating test directory... 2023-11-27T14:54:08.504 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-27T14:54:08.506 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-27T14:54:08.509 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-27T14:54:08.529 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-27T14:54:08.535 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-27T14:54:08.543 INFO:teuthology.task.internal:Creating archive directory... 2023-11-27T14:54:08.543 DEBUG:teuthology.orchestra.run.smithi052:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-27T14:54:08.563 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-27T14:54:08.569 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-27T14:54:08.836 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-27T14:54:08.844 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-27T14:54:08.844 DEBUG:teuthology.orchestra.run.smithi102:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-27T14:54:08.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T14:54:08.863 DEBUG:teuthology.orchestra.run.smithi150:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-27T14:54:08.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T14:54:08.882 DEBUG:teuthology.orchestra.run.smithi052:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-27T14:54:08.884 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-27T14:54:08.906 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-27T14:54:08.916 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-27T14:54:08.936 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-27T14:54:08.945 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-27T14:54:08.960 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-27T14:54:08.984 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-27T14:54:09.001 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-27T14:54:09.002 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-27T14:54:09.010 INFO:teuthology.task.internal:Configuring sudo... 2023-11-27T14:54:09.010 DEBUG:teuthology.orchestra.run.smithi052:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-27T14:54:09.012 DEBUG:teuthology.orchestra.run.smithi102:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-27T14:54:09.029 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-27T14:54:09.073 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-27T14:54:09.083 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-27T14:54:09.083 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-27T14:54:09.085 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-27T14:54:09.102 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-27T14:54:09.132 DEBUG:teuthology.orchestra.run.smithi052:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-27T14:54:09.186 DEBUG:teuthology.orchestra.run.smithi052:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-27T14:54:09.237 DEBUG:teuthology.orchestra.run.smithi052:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-27T14:54:09.322 DEBUG:teuthology.orchestra.run.smithi052:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-27T14:54:09.363 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2023-11-27T14:54:09.363 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-27T14:54:09.431 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-27T14:54:09.477 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-27T14:54:09.533 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-27T14:54:09.622 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-27T14:54:09.667 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2023-11-27T14:54:09.667 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-27T14:54:09.740 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-27T14:54:09.787 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-27T14:54:09.845 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-27T14:54:09.930 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-27T14:54:09.974 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-27T14:54:09.974 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-27T14:54:10.043 DEBUG:teuthology.orchestra.run.smithi052:> sudo service rsyslog restart 2023-11-27T14:54:10.045 DEBUG:teuthology.orchestra.run.smithi102:> sudo service rsyslog restart 2023-11-27T14:54:10.047 DEBUG:teuthology.orchestra.run.smithi150:> sudo service rsyslog restart 2023-11-27T14:54:10.095 INFO:teuthology.orchestra.run.smithi052.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-27T14:54:10.110 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-27T14:54:10.152 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-27T14:54:10.539 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-27T14:54:10.547 INFO:teuthology.task.internal:Starting timer... 2023-11-27T14:54:10.547 INFO:teuthology.run_tasks:Running task pcp... 2023-11-27T14:54:10.562 INFO:teuthology.run_tasks:Running task selinux... 2023-11-27T14:54:10.570 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-11-27T14:54:10.571 DEBUG:teuthology.orchestra.run.smithi052:> sudo service auditd rotate 2023-11-27T14:54:10.633 INFO:teuthology.orchestra.run.smithi052.stdout:Rotating logs: 2023-11-27T14:54:10.634 DEBUG:teuthology.orchestra.run.smithi102:> sudo service auditd rotate 2023-11-27T14:54:10.703 INFO:teuthology.orchestra.run.smithi102.stdout:Rotating logs: 2023-11-27T14:54:10.705 DEBUG:teuthology.orchestra.run.smithi150:> sudo service auditd rotate 2023-11-27T14:54:10.761 INFO:teuthology.orchestra.run.smithi150.stdout:Rotating logs: 2023-11-27T14:54:10.762 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-27T14:54:10.763 DEBUG:teuthology.orchestra.run.smithi052:> /usr/sbin/getenforce 2023-11-27T14:54:10.795 INFO:teuthology.orchestra.run.smithi052.stdout:Permissive 2023-11-27T14:54:10.795 DEBUG:teuthology.orchestra.run.smithi102:> /usr/sbin/getenforce 2023-11-27T14:54:10.831 INFO:teuthology.orchestra.run.smithi102.stdout:Permissive 2023-11-27T14:54:10.831 DEBUG:teuthology.orchestra.run.smithi150:> /usr/sbin/getenforce 2023-11-27T14:54:10.860 INFO:teuthology.orchestra.run.smithi150.stdout:Permissive 2023-11-27T14:54:10.861 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi052.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi102.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi150.front.sepia.ceph.com': 'permissive'} 2023-11-27T14:54:10.861 DEBUG:teuthology.orchestra.run.smithi052:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-27T14:54:10.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T14:54:10.891 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-27T14:54:10.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T14:54:10.920 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-27T14:54:10.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T14:54:10.948 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-27T14:54:10.948 DEBUG:teuthology.orchestra.run.smithi052:> sudo /usr/sbin/setenforce permissive 2023-11-27T14:54:10.976 DEBUG:teuthology.orchestra.run.smithi102:> sudo /usr/sbin/setenforce permissive 2023-11-27T14:54:11.003 DEBUG:teuthology.orchestra.run.smithi150:> sudo /usr/sbin/setenforce permissive 2023-11-27T14:54:11.030 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-27T14:54:11.040 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-11-27T14:54:11.040 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-27T14:54:11.057 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-27T14:54:11.058 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi052.front.sepia.ceph.com,smithi102.front.sepia.ceph.com,smithi150.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-27T15:05:04.189 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi052.front.sepia.ceph.com'), Remote(name='ubuntu@smithi102.front.sepia.ceph.com'), Remote(name='ubuntu@smithi150.front.sepia.ceph.com')] 2023-11-27T15:05:04.190 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2023-11-27T15:05:04.191 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-27T15:05:04.261 DEBUG:teuthology.orchestra.run.smithi052:> true 2023-11-27T15:05:04.328 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi052.front.sepia.ceph.com' 2023-11-27T15:05:04.329 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2023-11-27T15:05:04.330 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-27T15:05:04.398 DEBUG:teuthology.orchestra.run.smithi102:> true 2023-11-27T15:05:04.468 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi102.front.sepia.ceph.com' 2023-11-27T15:05:04.468 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2023-11-27T15:05:04.469 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-27T15:05:04.537 DEBUG:teuthology.orchestra.run.smithi150:> true 2023-11-27T15:05:04.608 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2023-11-27T15:05:04.609 INFO:teuthology.run_tasks:Running task clock... 2023-11-27T15:05:04.620 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-27T15:05:04.620 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-27T15:05:04.620 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-27T15:05:04.623 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-27T15:05:04.623 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-27T15:05:04.625 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-27T15:05:04.625 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-27T15:05:04.656 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-27T15:05:04.658 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-27T15:05:04.673 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-27T15:05:04.674 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-27T15:05:04.681 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-27T15:05:04.696 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-27T15:05:04.747 INFO:teuthology.orchestra.run.smithi102.stderr:sudo: ntpd: command not found 2023-11-27T15:05:04.752 INFO:teuthology.orchestra.run.smithi052.stderr:sudo: ntpd: command not found 2023-11-27T15:05:04.760 INFO:teuthology.orchestra.run.smithi102.stdout:506 Cannot talk to daemon 2023-11-27T15:05:04.765 INFO:teuthology.orchestra.run.smithi052.stdout:506 Cannot talk to daemon 2023-11-27T15:05:04.774 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-27T15:05:04.776 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: ntpd: command not found 2023-11-27T15:05:04.780 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-27T15:05:04.788 INFO:teuthology.orchestra.run.smithi150.stdout:506 Cannot talk to daemon 2023-11-27T15:05:04.788 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-27T15:05:04.794 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-27T15:05:04.802 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-27T15:05:04.816 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-27T15:05:04.863 INFO:teuthology.orchestra.run.smithi102.stderr:bash: ntpq: command not found 2023-11-27T15:05:04.867 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-27T15:05:04.867 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2023-11-27T15:05:04.867 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:05:04.867 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:05:04.868 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:05:04.868 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:05:04.869 INFO:teuthology.orchestra.run.smithi052.stderr:bash: ntpq: command not found 2023-11-27T15:05:04.871 INFO:teuthology.orchestra.run.smithi052.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-27T15:05:04.871 INFO:teuthology.orchestra.run.smithi052.stdout:=============================================================================== 2023-11-27T15:05:04.871 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:05:04.871 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:05:04.871 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:05:04.872 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:05:04.893 INFO:teuthology.orchestra.run.smithi150.stderr:bash: ntpq: command not found 2023-11-27T15:05:04.897 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-27T15:05:04.897 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2023-11-27T15:05:04.898 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:05:04.898 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:05:04.898 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:05:04.898 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:05:04.899 INFO:teuthology.run_tasks:Running task pexec... 2023-11-27T15:05:04.909 INFO:teuthology.task.pexec:Executing custom commands... 2023-11-27T15:05:04.910 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi052.front.sepia.ceph.com 2023-11-27T15:05:04.910 DEBUG:teuthology.orchestra.run.smithi052:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-11-27T15:05:04.910 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi102.front.sepia.ceph.com 2023-11-27T15:05:04.911 DEBUG:teuthology.orchestra.run.smithi102:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-11-27T15:05:04.911 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi150.front.sepia.ceph.com 2023-11-27T15:05:04.911 DEBUG:teuthology.orchestra.run.smithi150:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-11-27T15:05:05.281 INFO:teuthology.orchestra.run.smithi102.stdout:Updating Subscription Management repositories. 2023-11-27T15:05:05.304 INFO:teuthology.orchestra.run.smithi052.stdout:Updating Subscription Management repositories. 2023-11-27T15:05:05.317 INFO:teuthology.orchestra.run.smithi150.stdout:Updating Subscription Management repositories. 2023-11-27T15:05:06.923 INFO:teuthology.orchestra.run.smithi052.stdout:Last metadata expiration check: 0:01:52 ago on Mon 27 Nov 2023 03:03:14 PM UTC. 2023-11-27T15:05:06.996 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:02:02 ago on Mon 27 Nov 2023 03:03:04 PM UTC. 2023-11-27T15:05:07.022 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:02:03 ago on Mon 27 Nov 2023 03:03:03 PM UTC. 2023-11-27T15:05:08.715 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2023-11-27T15:05:08.717 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2023-11-27T15:05:08.717 INFO:teuthology.orchestra.run.smithi150.stdout: Package Architecture Version Repository Size 2023-11-27T15:05:08.717 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2023-11-27T15:05:08.717 INFO:teuthology.orchestra.run.smithi150.stdout:Resetting modules: 2023-11-27T15:05:08.717 INFO:teuthology.orchestra.run.smithi150.stdout: container-tools 2023-11-27T15:05:08.717 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-27T15:05:08.717 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2023-11-27T15:05:08.717 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2023-11-27T15:05:08.717 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-27T15:05:08.831 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2023-11-27T15:05:08.832 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2023-11-27T15:05:08.832 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2023-11-27T15:05:08.832 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2023-11-27T15:05:08.833 INFO:teuthology.orchestra.run.smithi102.stdout:Resetting modules: 2023-11-27T15:05:08.833 INFO:teuthology.orchestra.run.smithi102.stdout: container-tools 2023-11-27T15:05:08.833 INFO:teuthology.orchestra.run.smithi102.stdout: 2023-11-27T15:05:08.833 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2023-11-27T15:05:08.833 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2023-11-27T15:05:08.833 INFO:teuthology.orchestra.run.smithi102.stdout: 2023-11-27T15:05:08.910 INFO:teuthology.orchestra.run.smithi052.stdout:Dependencies resolved. 2023-11-27T15:05:08.911 INFO:teuthology.orchestra.run.smithi052.stdout:================================================================================ 2023-11-27T15:05:08.911 INFO:teuthology.orchestra.run.smithi052.stdout: Package Architecture Version Repository Size 2023-11-27T15:05:08.911 INFO:teuthology.orchestra.run.smithi052.stdout:================================================================================ 2023-11-27T15:05:08.911 INFO:teuthology.orchestra.run.smithi052.stdout:Resetting modules: 2023-11-27T15:05:08.911 INFO:teuthology.orchestra.run.smithi052.stdout: container-tools 2023-11-27T15:05:08.911 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:05:08.911 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction Summary 2023-11-27T15:05:08.912 INFO:teuthology.orchestra.run.smithi052.stdout:================================================================================ 2023-11-27T15:05:08.912 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:05:09.690 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:02:05 ago on Mon 27 Nov 2023 03:03:04 PM UTC. 2023-11-27T15:05:09.825 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:02:06 ago on Mon 27 Nov 2023 03:03:03 PM UTC. 2023-11-27T15:05:09.922 INFO:teuthology.orchestra.run.smithi052.stdout:Last metadata expiration check: 0:01:55 ago on Mon 27 Nov 2023 03:03:14 PM UTC. 2023-11-27T15:05:18.999 INFO:teuthology.orchestra.run.smithi052.stdout:Complete! 2023-11-27T15:05:19.477 INFO:teuthology.orchestra.run.smithi052.stdout:Updating Subscription Management repositories. 2023-11-27T15:05:19.864 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2023-11-27T15:05:20.336 INFO:teuthology.orchestra.run.smithi150.stdout:Updating Subscription Management repositories. 2023-11-27T15:05:20.721 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2023-11-27T15:05:21.176 INFO:teuthology.orchestra.run.smithi102.stdout:Updating Subscription Management repositories. 2023-11-27T15:05:21.256 INFO:teuthology.orchestra.run.smithi052.stdout:Last metadata expiration check: 0:02:06 ago on Mon 27 Nov 2023 03:03:14 PM UTC. 2023-11-27T15:05:22.251 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:02:17 ago on Mon 27 Nov 2023 03:03:04 PM UTC. 2023-11-27T15:05:22.750 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:02:19 ago on Mon 27 Nov 2023 03:03:03 PM UTC. 2023-11-27T15:05:23.235 INFO:teuthology.orchestra.run.smithi052.stdout:Dependencies resolved. 2023-11-27T15:05:23.336 INFO:teuthology.orchestra.run.smithi052.stdout:============================================================================================================ 2023-11-27T15:05:23.337 INFO:teuthology.orchestra.run.smithi052.stdout: Package Arch Version Repository Size 2023-11-27T15:05:23.337 INFO:teuthology.orchestra.run.smithi052.stdout:============================================================================================================ 2023-11-27T15:05:23.337 INFO:teuthology.orchestra.run.smithi052.stdout:Installing group/module packages: 2023-11-27T15:05:23.338 INFO:teuthology.orchestra.run.smithi052.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-11-27T15:05:23.338 INFO:teuthology.orchestra.run.smithi052.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-11-27T15:05:23.338 INFO:teuthology.orchestra.run.smithi052.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-11-27T15:05:23.339 INFO:teuthology.orchestra.run.smithi052.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-11-27T15:05:23.339 INFO:teuthology.orchestra.run.smithi052.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-11-27T15:05:23.339 INFO:teuthology.orchestra.run.smithi052.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-11-27T15:05:23.339 INFO:teuthology.orchestra.run.smithi052.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-11-27T15:05:23.339 INFO:teuthology.orchestra.run.smithi052.stdout:Installing dependencies: 2023-11-27T15:05:23.340 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-11-27T15:05:23.340 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-11-27T15:05:23.340 INFO:teuthology.orchestra.run.smithi052.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-11-27T15:05:23.340 INFO:teuthology.orchestra.run.smithi052.stdout:Installing module profiles: 2023-11-27T15:05:23.340 INFO:teuthology.orchestra.run.smithi052.stdout: container-tools/common 2023-11-27T15:05:23.340 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:05:23.340 INFO:teuthology.orchestra.run.smithi052.stdout:Enabling module streams: 2023-11-27T15:05:23.340 INFO:teuthology.orchestra.run.smithi052.stdout: container-tools rhel8 2023-11-27T15:05:23.341 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:05:23.341 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction Summary 2023-11-27T15:05:23.341 INFO:teuthology.orchestra.run.smithi052.stdout:============================================================================================================ 2023-11-27T15:05:23.341 INFO:teuthology.orchestra.run.smithi052.stdout:Install 10 Packages 2023-11-27T15:05:23.341 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:05:23.342 INFO:teuthology.orchestra.run.smithi052.stdout:Total download size: 18 M 2023-11-27T15:05:23.345 INFO:teuthology.orchestra.run.smithi052.stdout:Installed size: 65 M 2023-11-27T15:05:23.345 INFO:teuthology.orchestra.run.smithi052.stdout:Downloading Packages: 2023-11-27T15:05:23.513 INFO:teuthology.orchestra.run.smithi052.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 234 kB/s | 41 kB 00:00 2023-11-27T15:05:23.546 INFO:teuthology.orchestra.run.smithi052.stdout:(2/10): python3-pyxdg-0.25-16.el8.noarch.rpm 452 kB/s | 94 kB 00:00 2023-11-27T15:05:23.605 INFO:teuthology.orchestra.run.smithi052.stdout:(3/10): python3-pytoml-0.1.14-5.git7dea353.el8. 93 kB/s | 25 kB 00:00 2023-11-27T15:05:23.639 INFO:teuthology.orchestra.run.smithi052.stdout:(4/10): cockpit-podman-49.1-1.module+el8.6.0+16 5.6 MB/s | 524 kB 00:00 2023-11-27T15:05:23.664 INFO:teuthology.orchestra.run.smithi052.stdout:(5/10): udica-0.2.6-4.module+el8.6.0+16771+28df 327 kB/s | 49 kB 00:00 2023-11-27T15:05:23.789 INFO:teuthology.orchestra.run.smithi052.stdout:(6/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.7 MB/s | 213 kB 00:00 2023-11-27T15:05:23.864 INFO:teuthology.orchestra.run.smithi052.stdout:(7/10): python3-podman-4.0.1-1.module+el8.6.0+1 663 kB/s | 149 kB 00:00 2023-11-27T15:05:24.032 INFO:teuthology.orchestra.run.smithi052.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 19 MB/s | 8.0 MB 00:00 2023-11-27T15:05:24.106 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2023-11-27T15:05:24.148 INFO:teuthology.orchestra.run.smithi052.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 20 MB/s | 7.0 MB 00:00 2023-11-27T15:05:24.198 INFO:teuthology.orchestra.run.smithi150.stdout:============================================================================================================ 2023-11-27T15:05:24.198 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2023-11-27T15:05:24.199 INFO:teuthology.orchestra.run.smithi150.stdout:============================================================================================================ 2023-11-27T15:05:24.199 INFO:teuthology.orchestra.run.smithi150.stdout:Installing group/module packages: 2023-11-27T15:05:24.199 INFO:teuthology.orchestra.run.smithi150.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-11-27T15:05:24.199 INFO:teuthology.orchestra.run.smithi150.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-11-27T15:05:24.199 INFO:teuthology.orchestra.run.smithi150.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-11-27T15:05:24.199 INFO:teuthology.orchestra.run.smithi150.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-11-27T15:05:24.199 INFO:teuthology.orchestra.run.smithi150.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-11-27T15:05:24.199 INFO:teuthology.orchestra.run.smithi150.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-11-27T15:05:24.200 INFO:teuthology.orchestra.run.smithi150.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-11-27T15:05:24.200 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2023-11-27T15:05:24.200 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-11-27T15:05:24.200 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-11-27T15:05:24.200 INFO:teuthology.orchestra.run.smithi150.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-11-27T15:05:24.200 INFO:teuthology.orchestra.run.smithi150.stdout:Installing module profiles: 2023-11-27T15:05:24.200 INFO:teuthology.orchestra.run.smithi150.stdout: container-tools/common 2023-11-27T15:05:24.200 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-27T15:05:24.201 INFO:teuthology.orchestra.run.smithi150.stdout:Enabling module streams: 2023-11-27T15:05:24.201 INFO:teuthology.orchestra.run.smithi150.stdout: container-tools rhel8 2023-11-27T15:05:24.201 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-27T15:05:24.201 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2023-11-27T15:05:24.201 INFO:teuthology.orchestra.run.smithi150.stdout:============================================================================================================ 2023-11-27T15:05:24.201 INFO:teuthology.orchestra.run.smithi150.stdout:Install 10 Packages 2023-11-27T15:05:24.201 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-27T15:05:24.202 INFO:teuthology.orchestra.run.smithi052.stdout:(10/10): toolbox-0.0.99.3-0.6.module+el8.6.0+16 6.6 MB/s | 2.2 MB 00:00 2023-11-27T15:05:24.203 INFO:teuthology.orchestra.run.smithi052.stdout:-------------------------------------------------------------------------------- 2023-11-27T15:05:24.203 INFO:teuthology.orchestra.run.smithi052.stdout:Total 21 MB/s | 18 MB 00:00 2023-11-27T15:05:24.203 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 18 M 2023-11-27T15:05:24.203 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 65 M 2023-11-27T15:05:24.203 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2023-11-27T15:05:24.406 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction check 2023-11-27T15:05:24.426 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction check succeeded. 2023-11-27T15:05:24.426 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction test 2023-11-27T15:05:24.462 INFO:teuthology.orchestra.run.smithi150.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 360 kB/s | 94 kB 00:00 2023-11-27T15:05:24.487 INFO:teuthology.orchestra.run.smithi150.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 87 kB/s | 25 kB 00:00 2023-11-27T15:05:24.513 INFO:teuthology.orchestra.run.smithi150.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 131 kB/s | 41 kB 00:00 2023-11-27T15:05:24.607 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2023-11-27T15:05:24.622 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction test succeeded. 2023-11-27T15:05:24.626 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction 2023-11-27T15:05:24.638 INFO:teuthology.orchestra.run.smithi150.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 281 kB/s | 49 kB 00:00 2023-11-27T15:05:24.700 INFO:teuthology.orchestra.run.smithi102.stdout:============================================================================================================ 2023-11-27T15:05:24.700 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2023-11-27T15:05:24.700 INFO:teuthology.orchestra.run.smithi102.stdout:============================================================================================================ 2023-11-27T15:05:24.700 INFO:teuthology.orchestra.run.smithi102.stdout:Installing group/module packages: 2023-11-27T15:05:24.700 INFO:teuthology.orchestra.run.smithi102.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-11-27T15:05:24.700 INFO:teuthology.orchestra.run.smithi102.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-11-27T15:05:24.700 INFO:teuthology.orchestra.run.smithi102.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-11-27T15:05:24.700 INFO:teuthology.orchestra.run.smithi102.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-11-27T15:05:24.700 INFO:teuthology.orchestra.run.smithi102.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-11-27T15:05:24.701 INFO:teuthology.orchestra.run.smithi102.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-11-27T15:05:24.701 INFO:teuthology.orchestra.run.smithi102.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-11-27T15:05:24.701 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2023-11-27T15:05:24.701 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-11-27T15:05:24.701 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-11-27T15:05:24.701 INFO:teuthology.orchestra.run.smithi102.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-11-27T15:05:24.701 INFO:teuthology.orchestra.run.smithi102.stdout:Installing module profiles: 2023-11-27T15:05:24.701 INFO:teuthology.orchestra.run.smithi102.stdout: container-tools/common 2023-11-27T15:05:24.701 INFO:teuthology.orchestra.run.smithi102.stdout: 2023-11-27T15:05:24.701 INFO:teuthology.orchestra.run.smithi102.stdout:Enabling module streams: 2023-11-27T15:05:24.701 INFO:teuthology.orchestra.run.smithi102.stdout: container-tools rhel8 2023-11-27T15:05:24.701 INFO:teuthology.orchestra.run.smithi102.stdout: 2023-11-27T15:05:24.701 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2023-11-27T15:05:24.701 INFO:teuthology.orchestra.run.smithi102.stdout:============================================================================================================ 2023-11-27T15:05:24.702 INFO:teuthology.orchestra.run.smithi102.stdout:Install 10 Packages 2023-11-27T15:05:24.702 INFO:teuthology.orchestra.run.smithi102.stdout: 2023-11-27T15:05:24.702 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 18 M 2023-11-27T15:05:24.702 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 65 M 2023-11-27T15:05:24.702 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2023-11-27T15:05:24.738 INFO:teuthology.orchestra.run.smithi150.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 2.0 MB/s | 524 kB 00:00 2023-11-27T15:05:24.764 INFO:teuthology.orchestra.run.smithi150.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.2 MB/s | 149 kB 00:00 2023-11-27T15:05:24.802 INFO:teuthology.orchestra.run.smithi102.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 250 kB/s | 25 kB 00:00 2023-11-27T15:05:24.881 INFO:teuthology.orchestra.run.smithi150.stdout:(7/10): buildah-1.26.2-1.module+el8.6.0+16771+2 22 MB/s | 8.0 MB 00:00 2023-11-27T15:05:24.886 INFO:teuthology.orchestra.run.smithi102.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 223 kB/s | 41 kB 00:00 2023-11-27T15:05:24.906 INFO:teuthology.orchestra.run.smithi150.stdout:(8/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.2 MB/s | 213 kB 00:00 2023-11-27T15:05:24.911 INFO:teuthology.orchestra.run.smithi102.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 450 kB/s | 94 kB 00:00 2023-11-27T15:05:24.953 INFO:teuthology.orchestra.run.smithi102.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 328 kB/s | 49 kB 00:00 2023-11-27T15:05:25.028 INFO:teuthology.orchestra.run.smithi102.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 3.6 MB/s | 524 kB 00:00 2023-11-27T15:05:25.056 INFO:teuthology.orchestra.run.smithi150.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 24 MB/s | 7.0 MB 00:00 2023-11-27T15:05:25.115 INFO:teuthology.orchestra.run.smithi150.stdout:(10/10): toolbox-0.0.99.3-0.6.module+el8.6.0+16 9.4 MB/s | 2.2 MB 00:00 2023-11-27T15:05:25.115 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2023-11-27T15:05:25.116 INFO:teuthology.orchestra.run.smithi150.stdout:Total 20 MB/s | 18 MB 00:00 2023-11-27T15:05:25.149 INFO:teuthology.orchestra.run.smithi052.stdout: Preparing : 1/1 2023-11-27T15:05:25.162 INFO:teuthology.orchestra.run.smithi102.stdout:(6/10): buildah-1.26.2-1.module+el8.6.0+16771+2 32 MB/s | 8.0 MB 00:00 2023-11-27T15:05:25.212 INFO:teuthology.orchestra.run.smithi102.stdout:(7/10): python3-podman-4.0.1-1.module+el8.6.0+1 580 kB/s | 149 kB 00:00 2023-11-27T15:05:25.246 INFO:teuthology.orchestra.run.smithi102.stdout:(8/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 980 kB/s | 213 kB 00:00 2023-11-27T15:05:25.316 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2023-11-27T15:05:25.320 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-11-27T15:05:25.335 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2023-11-27T15:05:25.335 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2023-11-27T15:05:25.387 INFO:teuthology.orchestra.run.smithi102.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 12 MB/s | 2.2 MB 00:00 2023-11-27T15:05:25.496 INFO:teuthology.orchestra.run.smithi102.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 21 MB/s | 7.0 MB 00:00 2023-11-27T15:05:25.498 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2023-11-27T15:05:25.498 INFO:teuthology.orchestra.run.smithi102.stdout:Total 23 MB/s | 18 MB 00:00 2023-11-27T15:05:25.518 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2023-11-27T15:05:25.522 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2023-11-27T15:05:25.537 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-27T15:05:25.695 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2023-11-27T15:05:25.713 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2023-11-27T15:05:25.714 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2023-11-27T15:05:25.734 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-11-27T15:05:25.896 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2023-11-27T15:05:25.899 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2023-11-27T15:05:25.950 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-11-27T15:05:26.040 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2023-11-27T15:05:26.230 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-11-27T15:05:26.336 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-11-27T15:05:26.404 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-27T15:05:26.411 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2023-11-27T15:05:26.614 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-11-27T15:05:26.632 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-11-27T15:05:26.813 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-27T15:05:26.835 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-11-27T15:05:26.990 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-11-27T15:05:27.150 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-11-27T15:05:27.200 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-11-27T15:05:27.203 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-11-27T15:05:27.583 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-11-27T15:05:27.955 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-11-27T15:05:28.089 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-11-27T15:05:28.321 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-11-27T15:05:28.370 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-11-27T15:05:28.518 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-11-27T15:05:28.678 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-27T15:05:28.813 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-11-27T15:05:28.954 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-27T15:05:28.954 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-11-27T15:05:28.954 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-27T15:05:28.954 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-11-27T15:05:28.954 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-11-27T15:05:28.954 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-11-27T15:05:28.954 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-11-27T15:05:28.954 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-11-27T15:05:28.954 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-11-27T15:05:28.955 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-11-27T15:05:29.047 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-11-27T15:05:29.211 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-11-27T15:05:29.251 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-11-27T15:05:29.397 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-27T15:05:29.473 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-11-27T15:05:29.660 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-27T15:05:29.661 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-11-27T15:05:29.661 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-27T15:05:29.661 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-11-27T15:05:29.661 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-11-27T15:05:29.661 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-11-27T15:05:29.661 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-11-27T15:05:29.661 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-11-27T15:05:29.662 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-11-27T15:05:29.662 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-11-27T15:05:29.678 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-11-27T15:05:29.864 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-27T15:05:30.155 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-11-27T15:05:30.155 INFO:teuthology.orchestra.run.smithi052.stdout:Last metadata expiration check: 0:02:15 ago on Mon 27 Nov 2023 03:03:14 PM UTC. 2023-11-27T15:05:30.164 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-27T15:05:30.164 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-11-27T15:05:30.164 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-27T15:05:30.164 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-11-27T15:05:30.164 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-11-27T15:05:30.164 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-11-27T15:05:30.164 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-11-27T15:05:30.164 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-11-27T15:05:30.164 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-11-27T15:05:30.164 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-11-27T15:05:30.822 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-11-27T15:05:30.822 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:02:26 ago on Mon 27 Nov 2023 03:03:04 PM UTC. 2023-11-27T15:05:31.293 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-11-27T15:05:31.293 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:02:27 ago on Mon 27 Nov 2023 03:03:03 PM UTC. 2023-11-27T15:05:40.003 INFO:teuthology.orchestra.run.smithi052.stdout:Installed products updated. 2023-11-27T15:05:40.222 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:05:40.222 INFO:teuthology.orchestra.run.smithi052.stdout:Installed: 2023-11-27T15:05:40.222 INFO:teuthology.orchestra.run.smithi052.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:05:40.222 INFO:teuthology.orchestra.run.smithi052.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-11-27T15:05:40.222 INFO:teuthology.orchestra.run.smithi052.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:05:40.223 INFO:teuthology.orchestra.run.smithi052.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-11-27T15:05:40.223 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-11-27T15:05:40.223 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-11-27T15:05:40.223 INFO:teuthology.orchestra.run.smithi052.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:05:40.223 INFO:teuthology.orchestra.run.smithi052.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:05:40.223 INFO:teuthology.orchestra.run.smithi052.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-11-27T15:05:40.223 INFO:teuthology.orchestra.run.smithi052.stdout: yajl-2.1.0-10.el8.x86_64 2023-11-27T15:05:40.223 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:05:40.223 INFO:teuthology.orchestra.run.smithi052.stdout:Complete! 2023-11-27T15:05:40.357 DEBUG:teuthology.parallel:result is None 2023-11-27T15:05:41.081 INFO:teuthology.orchestra.run.smithi150.stdout:Installed products updated. 2023-11-27T15:05:41.282 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-27T15:05:41.282 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2023-11-27T15:05:41.282 INFO:teuthology.orchestra.run.smithi150.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:05:41.283 INFO:teuthology.orchestra.run.smithi150.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-11-27T15:05:41.283 INFO:teuthology.orchestra.run.smithi150.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:05:41.283 INFO:teuthology.orchestra.run.smithi150.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-11-27T15:05:41.283 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-11-27T15:05:41.283 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-11-27T15:05:41.283 INFO:teuthology.orchestra.run.smithi150.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:05:41.283 INFO:teuthology.orchestra.run.smithi150.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:05:41.283 INFO:teuthology.orchestra.run.smithi150.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-11-27T15:05:41.283 INFO:teuthology.orchestra.run.smithi150.stdout: yajl-2.1.0-10.el8.x86_64 2023-11-27T15:05:41.283 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-27T15:05:41.283 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2023-11-27T15:05:41.439 DEBUG:teuthology.parallel:result is None 2023-11-27T15:05:41.768 INFO:teuthology.orchestra.run.smithi102.stdout:Installed products updated. 2023-11-27T15:05:41.966 INFO:teuthology.orchestra.run.smithi102.stdout: 2023-11-27T15:05:41.966 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2023-11-27T15:05:41.966 INFO:teuthology.orchestra.run.smithi102.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:05:41.966 INFO:teuthology.orchestra.run.smithi102.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-11-27T15:05:41.966 INFO:teuthology.orchestra.run.smithi102.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:05:41.966 INFO:teuthology.orchestra.run.smithi102.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-11-27T15:05:41.967 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-11-27T15:05:41.967 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-11-27T15:05:41.967 INFO:teuthology.orchestra.run.smithi102.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:05:41.967 INFO:teuthology.orchestra.run.smithi102.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:05:41.967 INFO:teuthology.orchestra.run.smithi102.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-11-27T15:05:41.967 INFO:teuthology.orchestra.run.smithi102.stdout: yajl-2.1.0-10.el8.x86_64 2023-11-27T15:05:41.967 INFO:teuthology.orchestra.run.smithi102.stdout: 2023-11-27T15:05:41.967 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2023-11-27T15:05:42.108 DEBUG:teuthology.parallel:result is None 2023-11-27T15:05:42.108 INFO:teuthology.run_tasks:Running task install... 2023-11-27T15:05:42.117 DEBUG:teuthology.task.install:project ceph 2023-11-27T15:05:42.117 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1c9565146c8647d4e00eb6c7ae626dc192680e7b'}} 2023-11-27T15:05:42.117 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1c9565146c8647d4e00eb6c7ae626dc192680e7b'} 2023-11-27T15:05:42.118 INFO:teuthology.task.install:Using flavor: default 2023-11-27T15:05:42.123 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-27T15:05:42.123 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-27T15:05:42.123 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1c9565146c8647d4e00eb6c7ae626dc192680e7b', 'tag': None, 'wait_for_package': False} 2023-11-27T15:05:42.124 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-27T15:05:42.126 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1c9565146c8647d4e00eb6c7ae626dc192680e7b', 'tag': None, 'wait_for_package': False} 2023-11-27T15:05:42.126 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-27T15:05:42.127 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1c9565146c8647d4e00eb6c7ae626dc192680e7b', 'tag': None, 'wait_for_package': False} 2023-11-27T15:05:42.127 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-27T15:05:42.268 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/ 2023-11-27T15:05:42.268 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3906.g1c956514 2023-11-27T15:05:42.273 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/ 2023-11-27T15:05:42.273 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3906.g1c956514 2023-11-27T15:05:42.363 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/ 2023-11-27T15:05:42.363 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3906.g1c956514 2023-11-27T15:05:42.371 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-27T15:05:42.372 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2023-11-27T15:05:42.372 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-27T15:05:42.378 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-27T15:05:42.378 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-27T15:05:42.378 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-27T15:05:42.398 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-27T15:05:42.399 DEBUG:teuthology.orchestra.run.smithi102:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1c9565146c8647d4e00eb6c7ae626dc192680e7b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-27T15:05:42.409 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-27T15:05:42.409 DEBUG:teuthology.orchestra.run.smithi150:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1c9565146c8647d4e00eb6c7ae626dc192680e7b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-27T15:05:42.465 DEBUG:teuthology.orchestra.run.smithi102:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-27T15:05:42.467 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-27T15:05:42.467 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2023-11-27T15:05:42.467 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-27T15:05:42.479 DEBUG:teuthology.orchestra.run.smithi150:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-27T15:05:42.502 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-27T15:05:42.502 DEBUG:teuthology.orchestra.run.smithi052:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1c9565146c8647d4e00eb6c7ae626dc192680e7b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-27T15:05:42.541 DEBUG:teuthology.orchestra.run.smithi102:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-27T15:05:42.558 DEBUG:teuthology.orchestra.run.smithi150:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-27T15:05:42.575 DEBUG:teuthology.orchestra.run.smithi052:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-27T15:05:42.607 INFO:teuthology.orchestra.run.smithi102.stdout:check_obsoletes = 1 2023-11-27T15:05:42.608 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum clean all 2023-11-27T15:05:42.624 INFO:teuthology.orchestra.run.smithi150.stdout:check_obsoletes = 1 2023-11-27T15:05:42.626 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum clean all 2023-11-27T15:05:42.658 DEBUG:teuthology.orchestra.run.smithi052:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-27T15:05:42.727 INFO:teuthology.orchestra.run.smithi052.stdout:check_obsoletes = 1 2023-11-27T15:05:42.728 DEBUG:teuthology.orchestra.run.smithi052:> sudo yum clean all 2023-11-27T15:05:42.977 INFO:teuthology.orchestra.run.smithi102.stdout:Updating Subscription Management repositories. 2023-11-27T15:05:42.999 INFO:teuthology.orchestra.run.smithi150.stdout:Updating Subscription Management repositories. 2023-11-27T15:05:43.124 INFO:teuthology.orchestra.run.smithi052.stdout:Updating Subscription Management repositories. 2023-11-27T15:05:43.886 INFO:teuthology.orchestra.run.smithi102.stdout:44 files removed 2023-11-27T15:05:43.937 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-27T15:05:44.046 INFO:teuthology.orchestra.run.smithi052.stdout:44 files removed 2023-11-27T15:05:44.100 DEBUG:teuthology.orchestra.run.smithi052:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-27T15:05:44.109 INFO:teuthology.orchestra.run.smithi150.stdout:44 files removed 2023-11-27T15:05:44.164 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-27T15:05:44.266 INFO:teuthology.orchestra.run.smithi102.stdout:Updating Subscription Management repositories. 2023-11-27T15:05:44.452 INFO:teuthology.orchestra.run.smithi052.stdout:Updating Subscription Management repositories. 2023-11-27T15:05:44.496 INFO:teuthology.orchestra.run.smithi150.stdout:Updating Subscription Management repositories. 2023-11-27T15:05:45.598 INFO:teuthology.orchestra.run.smithi102.stdout:ceph packages for x86_64 264 kB/s | 80 kB 00:00 2023-11-27T15:05:45.925 INFO:teuthology.orchestra.run.smithi052.stdout:ceph packages for x86_64 244 kB/s | 80 kB 00:00 2023-11-27T15:05:45.932 INFO:teuthology.orchestra.run.smithi102.stdout:ceph noarch packages 31 kB/s | 9.4 kB 00:00 2023-11-27T15:05:45.974 INFO:teuthology.orchestra.run.smithi150.stdout:ceph packages for x86_64 221 kB/s | 80 kB 00:00 2023-11-27T15:05:46.251 INFO:teuthology.orchestra.run.smithi052.stdout:ceph noarch packages 32 kB/s | 9.4 kB 00:00 2023-11-27T15:05:46.333 INFO:teuthology.orchestra.run.smithi102.stdout:ceph source packages 3.9 kB/s | 1.5 kB 00:00 2023-11-27T15:05:46.350 INFO:teuthology.orchestra.run.smithi150.stdout:ceph noarch packages 27 kB/s | 9.4 kB 00:00 2023-11-27T15:05:46.502 INFO:teuthology.orchestra.run.smithi052.stdout:ceph source packages 6.5 kB/s | 1.5 kB 00:00 2023-11-27T15:05:46.509 INFO:teuthology.orchestra.run.smithi102.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2023-11-27T15:05:46.600 INFO:teuthology.orchestra.run.smithi150.stdout:ceph source packages 6.5 kB/s | 1.5 kB 00:00 2023-11-27T15:05:46.702 INFO:teuthology.orchestra.run.smithi052.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2023-11-27T15:05:46.767 INFO:teuthology.orchestra.run.smithi150.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2023-11-27T15:05:47.539 INFO:teuthology.orchestra.run.smithi052.stdout:Extra Packages for Enterprise Linux 20 MB/s | 16 MB 00:00 2023-11-27T15:05:47.912 INFO:teuthology.orchestra.run.smithi102.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2023-11-27T15:05:48.118 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2023-11-27T15:05:51.262 INFO:teuthology.orchestra.run.smithi052.stdout:lab-extras 395 kB/s | 24 kB 00:00 2023-11-27T15:05:51.443 INFO:teuthology.orchestra.run.smithi102.stdout:lab-extras 415 kB/s | 24 kB 00:00 2023-11-27T15:05:51.660 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 448 kB/s | 24 kB 00:00 2023-11-27T15:05:52.339 INFO:teuthology.orchestra.run.smithi052.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 47 MB/s | 47 MB 00:01 2023-11-27T15:05:52.578 INFO:teuthology.orchestra.run.smithi102.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 50 MB/s | 53 MB 00:01 2023-11-27T15:05:52.796 INFO:teuthology.orchestra.run.smithi150.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 45 MB/s | 47 MB 00:01 2023-11-27T15:06:00.930 INFO:teuthology.orchestra.run.smithi150.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 49 MB/s | 53 MB 00:01 2023-11-27T15:06:01.082 INFO:teuthology.orchestra.run.smithi102.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 39 MB/s | 47 MB 00:01 2023-11-27T15:06:01.674 INFO:teuthology.orchestra.run.smithi052.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 29 MB/s | 53 MB 00:01 2023-11-27T15:06:12.028 INFO:teuthology.orchestra.run.smithi102.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-27T15:06:12.028 INFO:teuthology.orchestra.run.smithi102.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-27T15:06:12.042 INFO:teuthology.orchestra.run.smithi102.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-27T15:06:12.043 INFO:teuthology.orchestra.run.smithi102.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-27T15:06:12.045 INFO:teuthology.orchestra.run.smithi102.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-27T15:06:12.046 INFO:teuthology.orchestra.run.smithi102.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-27T15:06:12.048 INFO:teuthology.orchestra.run.smithi102.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-27T15:06:12.051 INFO:teuthology.orchestra.run.smithi102.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-27T15:06:12.052 INFO:teuthology.orchestra.run.smithi102.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-27T15:06:12.061 INFO:teuthology.orchestra.run.smithi150.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-27T15:06:12.062 INFO:teuthology.orchestra.run.smithi150.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-27T15:06:12.076 INFO:teuthology.orchestra.run.smithi150.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-27T15:06:12.077 INFO:teuthology.orchestra.run.smithi150.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-27T15:06:12.079 INFO:teuthology.orchestra.run.smithi150.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-27T15:06:12.080 INFO:teuthology.orchestra.run.smithi150.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-27T15:06:12.082 INFO:teuthology.orchestra.run.smithi150.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-27T15:06:12.085 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-27T15:06:12.086 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-27T15:06:12.181 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2023-11-27T15:06:12.194 INFO:teuthology.orchestra.run.smithi102.stdout:=========================================================================================================================== 2023-11-27T15:06:12.194 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2023-11-27T15:06:12.194 INFO:teuthology.orchestra.run.smithi102.stdout:=========================================================================================================================== 2023-11-27T15:06:12.194 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2023-11-27T15:06:12.195 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-27T15:06:12.195 INFO:teuthology.orchestra.run.smithi102.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-27T15:06:12.195 INFO:teuthology.orchestra.run.smithi102.stdout: ceph x86_64 2:18.0.0-3906.g1c956514.el8 ceph 6.4 k 2023-11-27T15:06:12.196 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base x86_64 2:18.0.0-3906.g1c956514.el8 ceph 5.3 M 2023-11-27T15:06:12.196 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 848 k 2023-11-27T15:06:12.196 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3906.g1c956514.el8 ceph 165 k 2023-11-27T15:06:12.196 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr x86_64 2:18.0.0-3906.g1c956514.el8 ceph 1.5 M 2023-11-27T15:06:12.197 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 127 k 2023-11-27T15:06:12.197 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 1.7 M 2023-11-27T15:06:12.197 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 7.4 M 2023-11-27T15:06:12.197 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 47 k 2023-11-27T15:06:12.197 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw x86_64 2:18.0.0-3906.g1c956514.el8 ceph 13 M 2023-11-27T15:06:12.198 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test x86_64 2:18.0.0-3906.g1c956514.el8 ceph 41 M 2023-11-27T15:06:12.198 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-volume noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 265 k 2023-11-27T15:06:12.198 INFO:teuthology.orchestra.run.smithi102.stdout: cephadm noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 83 k 2023-11-27T15:06:12.198 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-mirror x86_64 2:18.0.0-3906.g1c956514.el8 ceph 258 k 2023-11-27T15:06:12.199 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-top noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 18 k 2023-11-27T15:06:12.199 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-27T15:06:12.199 INFO:teuthology.orchestra.run.smithi102.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-27T15:06:12.199 INFO:teuthology.orchestra.run.smithi102.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-27T15:06:12.199 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel x86_64 2:18.0.0-3906.g1c956514.el8 ceph 31 k 2023-11-27T15:06:12.199 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 771 k 2023-11-27T15:06:12.199 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel x86_64 2:18.0.0-3906.g1c956514.el8 ceph 139 k 2023-11-27T15:06:12.199 INFO:teuthology.orchestra.run.smithi102.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-27T15:06:12.199 INFO:teuthology.orchestra.run.smithi102.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-27T15:06:12.199 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-27T15:06:12.199 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-27T15:06:12.200 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-27T15:06:12.200 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-27T15:06:12.200 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs x86_64 2:18.0.0-3906.g1c956514.el8 ceph 197 k 2023-11-27T15:06:12.200 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados x86_64 2:18.0.0-3906.g1c956514.el8 ceph 371 k 2023-11-27T15:06:12.200 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 364 k 2023-11-27T15:06:12.200 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw x86_64 2:18.0.0-3906.g1c956514.el8 ceph 107 k 2023-11-27T15:06:12.200 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 94 k 2023-11-27T15:06:12.200 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.4 M 2023-11-27T15:06:12.200 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 180 k 2023-11-27T15:06:12.200 INFO:teuthology.orchestra.run.smithi102.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-27T15:06:12.201 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrading: 2023-11-27T15:06:12.201 INFO:teuthology.orchestra.run.smithi102.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-27T15:06:12.201 INFO:teuthology.orchestra.run.smithi102.stdout: librados2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.7 M 2023-11-27T15:06:12.201 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.6 M 2023-11-27T15:06:12.201 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2023-11-27T15:06:12.201 INFO:teuthology.orchestra.run.smithi102.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-27T15:06:12.201 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common x86_64 2:18.0.0-3906.g1c956514.el8 ceph 22 M 2023-11-27T15:06:12.201 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 23 k 2023-11-27T15:06:12.201 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds x86_64 2:18.0.0-3906.g1c956514.el8 ceph 2.3 M 2023-11-27T15:06:12.201 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 240 k 2023-11-27T15:06:12.201 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon x86_64 2:18.0.0-3906.g1c956514.el8 ceph 4.6 M 2023-11-27T15:06:12.201 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 18 M 2023-11-27T15:06:12.201 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 15 k 2023-11-27T15:06:12.202 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux x86_64 2:18.0.0-3906.g1c956514.el8 ceph 24 k 2023-11-27T15:06:12.202 INFO:teuthology.orchestra.run.smithi102.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-27T15:06:12.202 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-27T15:06:12.202 INFO:teuthology.orchestra.run.smithi102.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-27T15:06:12.202 INFO:teuthology.orchestra.run.smithi102.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-27T15:06:12.202 INFO:teuthology.orchestra.run.smithi102.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-27T15:06:12.202 INFO:teuthology.orchestra.run.smithi102.stdout: libcephsqlite x86_64 2:18.0.0-3906.g1c956514.el8 ceph 137 k 2023-11-27T15:06:12.202 INFO:teuthology.orchestra.run.smithi102.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-27T15:06:12.202 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-27T15:06:12.202 INFO:teuthology.orchestra.run.smithi102.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-27T15:06:12.202 INFO:teuthology.orchestra.run.smithi102.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-27T15:06:12.203 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-27T15:06:12.203 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-27T15:06:12.203 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 473 k 2023-11-27T15:06:12.203 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-27T15:06:12.203 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 6.5 M 2023-11-27T15:06:12.203 INFO:teuthology.orchestra.run.smithi102.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-27T15:06:12.203 INFO:teuthology.orchestra.run.smithi102.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-27T15:06:12.203 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-27T15:06:12.203 INFO:teuthology.orchestra.run.smithi102.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-27T15:06:12.203 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-27T15:06:12.203 INFO:teuthology.orchestra.run.smithi102.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-27T15:06:12.203 INFO:teuthology.orchestra.run.smithi102.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-27T15:06:12.204 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-27T15:06:12.204 INFO:teuthology.orchestra.run.smithi102.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-27T15:06:12.204 INFO:teuthology.orchestra.run.smithi102.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-27T15:06:12.204 INFO:teuthology.orchestra.run.smithi102.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-27T15:06:12.204 INFO:teuthology.orchestra.run.smithi102.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-27T15:06:12.204 INFO:teuthology.orchestra.run.smithi102.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-27T15:06:12.204 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-27T15:06:12.204 INFO:teuthology.orchestra.run.smithi102.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-27T15:06:12.204 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-27T15:06:12.204 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 45 k 2023-11-27T15:06:12.204 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common x86_64 2:18.0.0-3906.g1c956514.el8 ceph 119 k 2023-11-27T15:06:12.204 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-27T15:06:12.205 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-27T15:06:12.205 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-27T15:06:12.205 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-27T15:06:12.205 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-27T15:06:12.205 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-27T15:06:12.205 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-27T15:06:12.205 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-27T15:06:12.205 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-27T15:06:12.205 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-27T15:06:12.205 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-27T15:06:12.205 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-27T15:06:12.206 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-27T15:06:12.206 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-27T15:06:12.206 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-27T15:06:12.206 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-27T15:06:12.206 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-27T15:06:12.206 INFO:teuthology.orchestra.run.smithi102.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-27T15:06:12.206 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-27T15:06:12.206 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-27T15:06:12.206 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-27T15:06:12.206 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-27T15:06:12.206 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-27T15:06:12.206 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-27T15:06:12.207 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-27T15:06:12.207 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-27T15:06:12.207 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-27T15:06:12.207 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-27T15:06:12.207 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-27T15:06:12.207 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-27T15:06:12.207 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-27T15:06:12.207 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-27T15:06:12.207 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-27T15:06:12.207 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-27T15:06:12.207 INFO:teuthology.orchestra.run.smithi102.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-27T15:06:12.207 INFO:teuthology.orchestra.run.smithi102.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-27T15:06:12.207 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-27T15:06:12.207 INFO:teuthology.orchestra.run.smithi102.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-27T15:06:12.208 INFO:teuthology.orchestra.run.smithi102.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-27T15:06:12.208 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-27T15:06:12.208 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-27T15:06:12.208 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-27T15:06:12.208 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-27T15:06:12.208 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-27T15:06:12.208 INFO:teuthology.orchestra.run.smithi102.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-27T15:06:12.208 INFO:teuthology.orchestra.run.smithi102.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-27T15:06:12.208 INFO:teuthology.orchestra.run.smithi102.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-27T15:06:12.208 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-27T15:06:12.208 INFO:teuthology.orchestra.run.smithi102.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-27T15:06:12.208 INFO:teuthology.orchestra.run.smithi102.stdout:Enabling module streams: 2023-11-27T15:06:12.208 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql 10 2023-11-27T15:06:12.209 INFO:teuthology.orchestra.run.smithi102.stdout: 2023-11-27T15:06:12.209 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2023-11-27T15:06:12.209 INFO:teuthology.orchestra.run.smithi102.stdout:=========================================================================================================================== 2023-11-27T15:06:12.209 INFO:teuthology.orchestra.run.smithi102.stdout:Install 128 Packages 2023-11-27T15:06:12.209 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrade 3 Packages 2023-11-27T15:06:12.209 INFO:teuthology.orchestra.run.smithi102.stdout: 2023-11-27T15:06:12.209 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 197 M 2023-11-27T15:06:12.210 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2023-11-27T15:06:12.217 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2023-11-27T15:06:12.230 INFO:teuthology.orchestra.run.smithi150.stdout:=========================================================================================================================== 2023-11-27T15:06:12.230 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2023-11-27T15:06:12.230 INFO:teuthology.orchestra.run.smithi150.stdout:=========================================================================================================================== 2023-11-27T15:06:12.230 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2023-11-27T15:06:12.230 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-27T15:06:12.230 INFO:teuthology.orchestra.run.smithi150.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-27T15:06:12.230 INFO:teuthology.orchestra.run.smithi150.stdout: ceph x86_64 2:18.0.0-3906.g1c956514.el8 ceph 6.4 k 2023-11-27T15:06:12.230 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base x86_64 2:18.0.0-3906.g1c956514.el8 ceph 5.3 M 2023-11-27T15:06:12.231 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 848 k 2023-11-27T15:06:12.231 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3906.g1c956514.el8 ceph 165 k 2023-11-27T15:06:12.231 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr x86_64 2:18.0.0-3906.g1c956514.el8 ceph 1.5 M 2023-11-27T15:06:12.231 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 127 k 2023-11-27T15:06:12.231 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 1.7 M 2023-11-27T15:06:12.231 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 7.4 M 2023-11-27T15:06:12.231 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 47 k 2023-11-27T15:06:12.231 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw x86_64 2:18.0.0-3906.g1c956514.el8 ceph 13 M 2023-11-27T15:06:12.231 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test x86_64 2:18.0.0-3906.g1c956514.el8 ceph 41 M 2023-11-27T15:06:12.231 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 265 k 2023-11-27T15:06:12.231 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 83 k 2023-11-27T15:06:12.231 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror x86_64 2:18.0.0-3906.g1c956514.el8 ceph 258 k 2023-11-27T15:06:12.231 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 18 k 2023-11-27T15:06:12.231 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-27T15:06:12.232 INFO:teuthology.orchestra.run.smithi150.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-27T15:06:12.232 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-27T15:06:12.232 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel x86_64 2:18.0.0-3906.g1c956514.el8 ceph 31 k 2023-11-27T15:06:12.232 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 771 k 2023-11-27T15:06:12.232 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel x86_64 2:18.0.0-3906.g1c956514.el8 ceph 139 k 2023-11-27T15:06:12.232 INFO:teuthology.orchestra.run.smithi150.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-27T15:06:12.232 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-27T15:06:12.232 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-27T15:06:12.232 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-27T15:06:12.232 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-27T15:06:12.232 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-27T15:06:12.232 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs x86_64 2:18.0.0-3906.g1c956514.el8 ceph 197 k 2023-11-27T15:06:12.233 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados x86_64 2:18.0.0-3906.g1c956514.el8 ceph 371 k 2023-11-27T15:06:12.233 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 364 k 2023-11-27T15:06:12.233 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw x86_64 2:18.0.0-3906.g1c956514.el8 ceph 107 k 2023-11-27T15:06:12.233 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 94 k 2023-11-27T15:06:12.233 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.4 M 2023-11-27T15:06:12.233 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 180 k 2023-11-27T15:06:12.233 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-27T15:06:12.233 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrading: 2023-11-27T15:06:12.233 INFO:teuthology.orchestra.run.smithi150.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-27T15:06:12.233 INFO:teuthology.orchestra.run.smithi150.stdout: librados2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.7 M 2023-11-27T15:06:12.233 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.6 M 2023-11-27T15:06:12.233 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2023-11-27T15:06:12.234 INFO:teuthology.orchestra.run.smithi150.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-27T15:06:12.234 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common x86_64 2:18.0.0-3906.g1c956514.el8 ceph 22 M 2023-11-27T15:06:12.234 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 23 k 2023-11-27T15:06:12.234 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds x86_64 2:18.0.0-3906.g1c956514.el8 ceph 2.3 M 2023-11-27T15:06:12.234 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 240 k 2023-11-27T15:06:12.234 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon x86_64 2:18.0.0-3906.g1c956514.el8 ceph 4.6 M 2023-11-27T15:06:12.234 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 18 M 2023-11-27T15:06:12.234 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 15 k 2023-11-27T15:06:12.234 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux x86_64 2:18.0.0-3906.g1c956514.el8 ceph 24 k 2023-11-27T15:06:12.234 INFO:teuthology.orchestra.run.smithi150.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-27T15:06:12.234 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-27T15:06:12.235 INFO:teuthology.orchestra.run.smithi150.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-27T15:06:12.235 INFO:teuthology.orchestra.run.smithi150.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-27T15:06:12.235 INFO:teuthology.orchestra.run.smithi150.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-27T15:06:12.235 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite x86_64 2:18.0.0-3906.g1c956514.el8 ceph 137 k 2023-11-27T15:06:12.235 INFO:teuthology.orchestra.run.smithi150.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-27T15:06:12.235 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-27T15:06:12.235 INFO:teuthology.orchestra.run.smithi150.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-27T15:06:12.235 INFO:teuthology.orchestra.run.smithi150.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-27T15:06:12.235 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-27T15:06:12.235 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-27T15:06:12.235 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 473 k 2023-11-27T15:06:12.235 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-27T15:06:12.236 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 6.5 M 2023-11-27T15:06:12.236 INFO:teuthology.orchestra.run.smithi150.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-27T15:06:12.236 INFO:teuthology.orchestra.run.smithi150.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-27T15:06:12.236 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-27T15:06:12.236 INFO:teuthology.orchestra.run.smithi150.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-27T15:06:12.236 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-27T15:06:12.236 INFO:teuthology.orchestra.run.smithi150.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-27T15:06:12.236 INFO:teuthology.orchestra.run.smithi150.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-27T15:06:12.236 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-27T15:06:12.236 INFO:teuthology.orchestra.run.smithi150.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-27T15:06:12.236 INFO:teuthology.orchestra.run.smithi150.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-27T15:06:12.236 INFO:teuthology.orchestra.run.smithi150.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-27T15:06:12.237 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-27T15:06:12.237 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-27T15:06:12.237 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-27T15:06:12.237 INFO:teuthology.orchestra.run.smithi150.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-27T15:06:12.237 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-27T15:06:12.237 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 45 k 2023-11-27T15:06:12.237 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common x86_64 2:18.0.0-3906.g1c956514.el8 ceph 119 k 2023-11-27T15:06:12.237 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-27T15:06:12.237 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-27T15:06:12.237 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-27T15:06:12.237 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-27T15:06:12.237 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-27T15:06:12.238 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-27T15:06:12.238 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-27T15:06:12.238 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-27T15:06:12.238 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-27T15:06:12.238 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-27T15:06:12.238 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-27T15:06:12.238 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-27T15:06:12.238 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-27T15:06:12.238 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-27T15:06:12.238 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-27T15:06:12.238 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-27T15:06:12.238 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-27T15:06:12.239 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-27T15:06:12.239 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-27T15:06:12.239 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-27T15:06:12.239 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-27T15:06:12.239 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-27T15:06:12.239 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-27T15:06:12.239 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-27T15:06:12.239 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-27T15:06:12.239 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-27T15:06:12.239 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-27T15:06:12.239 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-27T15:06:12.239 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-27T15:06:12.240 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-27T15:06:12.240 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-27T15:06:12.240 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-27T15:06:12.240 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-27T15:06:12.240 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-27T15:06:12.240 INFO:teuthology.orchestra.run.smithi150.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-27T15:06:12.240 INFO:teuthology.orchestra.run.smithi150.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-27T15:06:12.240 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-27T15:06:12.240 INFO:teuthology.orchestra.run.smithi150.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-27T15:06:12.240 INFO:teuthology.orchestra.run.smithi150.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-27T15:06:12.240 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-27T15:06:12.240 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-27T15:06:12.241 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-27T15:06:12.241 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-27T15:06:12.241 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-27T15:06:12.241 INFO:teuthology.orchestra.run.smithi150.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-27T15:06:12.241 INFO:teuthology.orchestra.run.smithi150.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-27T15:06:12.241 INFO:teuthology.orchestra.run.smithi150.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-27T15:06:12.241 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-27T15:06:12.241 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-27T15:06:12.241 INFO:teuthology.orchestra.run.smithi150.stdout:Enabling module streams: 2023-11-27T15:06:12.241 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql 10 2023-11-27T15:06:12.241 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-27T15:06:12.241 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2023-11-27T15:06:12.242 INFO:teuthology.orchestra.run.smithi150.stdout:=========================================================================================================================== 2023-11-27T15:06:12.242 INFO:teuthology.orchestra.run.smithi150.stdout:Install 128 Packages 2023-11-27T15:06:12.242 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrade 3 Packages 2023-11-27T15:06:12.242 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-27T15:06:12.242 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 197 M 2023-11-27T15:06:12.243 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2023-11-27T15:06:12.373 INFO:teuthology.orchestra.run.smithi102.stdout:(1/131): ceph-18.0.0-3906.g1c956514.el8.x86_64. 55 kB/s | 6.4 kB 00:00 2023-11-27T15:06:12.489 INFO:teuthology.orchestra.run.smithi150.stdout:(1/131): ceph-18.0.0-3906.g1c956514.el8.x86_64. 35 kB/s | 6.4 kB 00:00 2023-11-27T15:06:12.589 INFO:teuthology.orchestra.run.smithi102.stdout:(2/131): ceph-fuse-18.0.0-3906.g1c956514.el8.x8 3.8 MB/s | 848 kB 00:00 2023-11-27T15:06:12.631 INFO:teuthology.orchestra.run.smithi102.stdout:(3/131): ceph-immutable-object-cache-18.0.0-390 3.9 MB/s | 165 kB 00:00 2023-11-27T15:06:12.957 INFO:teuthology.orchestra.run.smithi102.stdout:(4/131): ceph-mds-18.0.0-3906.g1c956514.el8.x86 7.0 MB/s | 2.3 MB 00:00 2023-11-27T15:06:13.023 INFO:teuthology.orchestra.run.smithi150.stdout:(2/131): ceph-base-18.0.0-3906.g1c956514.el8.x8 7.5 MB/s | 5.3 MB 00:00 2023-11-27T15:06:13.073 INFO:teuthology.orchestra.run.smithi150.stdout:(3/131): ceph-immutable-object-cache-18.0.0-390 3.2 MB/s | 165 kB 00:00 2023-11-27T15:06:13.124 INFO:teuthology.orchestra.run.smithi102.stdout:(5/131): ceph-mgr-18.0.0-3906.g1c956514.el8.x86 9.3 MB/s | 1.5 MB 00:00 2023-11-27T15:06:13.173 INFO:teuthology.orchestra.run.smithi150.stdout:(4/131): ceph-fuse-18.0.0-3906.g1c956514.el8.x8 1.2 MB/s | 848 kB 00:00 2023-11-27T15:06:13.324 INFO:teuthology.orchestra.run.smithi150.stdout:(5/131): ceph-mds-18.0.0-3906.g1c956514.el8.x86 9.1 MB/s | 2.3 MB 00:00 2023-11-27T15:06:17.437 INFO:teuthology.orchestra.run.smithi102.stdout:(6/131): ceph-mon-18.0.0-3906.g1c956514.el8.x86 7.0 MB/s | 4.6 MB 00:00 2023-11-27T15:06:17.438 INFO:teuthology.orchestra.run.smithi102.stdout:(7/131): ceph-base-18.0.0-3906.g1c956514.el8.x8 3.2 MB/s | 5.3 MB 00:01 2023-11-27T15:06:17.438 INFO:teuthology.orchestra.run.smithi102.stdout:(8/131): ceph-osd-18.0.0-3906.g1c956514.el8.x86 7.8 MB/s | 18 MB 00:02 2023-11-27T15:06:17.438 INFO:teuthology.orchestra.run.smithi102.stdout:(9/131): ceph-selinux-18.0.0-3906.g1c956514.el8 365 kB/s | 24 kB 00:00 2023-11-27T15:06:17.438 INFO:teuthology.orchestra.run.smithi150.stdout:(6/131): ceph-mon-18.0.0-3906.g1c956514.el8.x86 9.1 MB/s | 4.6 MB 00:00 2023-11-27T15:06:17.438 INFO:teuthology.orchestra.run.smithi150.stdout:(7/131): ceph-mgr-18.0.0-3906.g1c956514.el8.x86 1.5 MB/s | 1.5 MB 00:01 2023-11-27T15:06:17.438 INFO:teuthology.orchestra.run.smithi150.stdout:(8/131): ceph-osd-18.0.0-3906.g1c956514.el8.x86 5.5 MB/s | 18 MB 00:03 2023-11-27T15:06:17.438 INFO:teuthology.orchestra.run.smithi150.stdout:(9/131): ceph-selinux-18.0.0-3906.g1c956514.el8 324 kB/s | 24 kB 00:00 2023-11-27T15:06:17.440 INFO:teuthology.orchestra.run.smithi052.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-27T15:06:17.440 INFO:teuthology.orchestra.run.smithi052.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-27T15:06:17.440 INFO:teuthology.orchestra.run.smithi052.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-27T15:06:17.440 INFO:teuthology.orchestra.run.smithi052.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-27T15:06:17.440 INFO:teuthology.orchestra.run.smithi052.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-27T15:06:17.440 INFO:teuthology.orchestra.run.smithi052.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-27T15:06:17.440 INFO:teuthology.orchestra.run.smithi052.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-27T15:06:17.440 INFO:teuthology.orchestra.run.smithi052.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-27T15:06:17.441 INFO:teuthology.orchestra.run.smithi052.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-27T15:06:17.441 INFO:teuthology.orchestra.run.smithi052.stdout:Dependencies resolved. 2023-11-27T15:06:17.441 INFO:teuthology.orchestra.run.smithi052.stdout:=========================================================================================================================== 2023-11-27T15:06:17.441 INFO:teuthology.orchestra.run.smithi052.stdout: Package Arch Version Repository Size 2023-11-27T15:06:17.441 INFO:teuthology.orchestra.run.smithi052.stdout:=========================================================================================================================== 2023-11-27T15:06:17.441 INFO:teuthology.orchestra.run.smithi052.stdout:Installing: 2023-11-27T15:06:17.441 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-27T15:06:17.441 INFO:teuthology.orchestra.run.smithi052.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-27T15:06:17.441 INFO:teuthology.orchestra.run.smithi052.stdout: ceph x86_64 2:18.0.0-3906.g1c956514.el8 ceph 6.4 k 2023-11-27T15:06:17.441 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-base x86_64 2:18.0.0-3906.g1c956514.el8 ceph 5.3 M 2023-11-27T15:06:17.441 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-fuse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 848 k 2023-11-27T15:06:17.441 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3906.g1c956514.el8 ceph 165 k 2023-11-27T15:06:17.442 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr x86_64 2:18.0.0-3906.g1c956514.el8 ceph 1.5 M 2023-11-27T15:06:17.442 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 127 k 2023-11-27T15:06:17.442 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 1.7 M 2023-11-27T15:06:17.442 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 7.4 M 2023-11-27T15:06:17.442 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-rook noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 47 k 2023-11-27T15:06:17.442 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-radosgw x86_64 2:18.0.0-3906.g1c956514.el8 ceph 13 M 2023-11-27T15:06:17.442 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-test x86_64 2:18.0.0-3906.g1c956514.el8 ceph 41 M 2023-11-27T15:06:17.442 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-volume noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 265 k 2023-11-27T15:06:17.442 INFO:teuthology.orchestra.run.smithi052.stdout: cephadm noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 83 k 2023-11-27T15:06:17.442 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-mirror x86_64 2:18.0.0-3906.g1c956514.el8 ceph 258 k 2023-11-27T15:06:17.442 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-top noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 18 k 2023-11-27T15:06:17.442 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-27T15:06:17.442 INFO:teuthology.orchestra.run.smithi052.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-27T15:06:17.443 INFO:teuthology.orchestra.run.smithi052.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-27T15:06:17.443 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs-devel x86_64 2:18.0.0-3906.g1c956514.el8 ceph 31 k 2023-11-27T15:06:17.443 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 771 k 2023-11-27T15:06:17.443 INFO:teuthology.orchestra.run.smithi052.stdout: librados-devel x86_64 2:18.0.0-3906.g1c956514.el8 ceph 139 k 2023-11-27T15:06:17.443 INFO:teuthology.orchestra.run.smithi052.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-27T15:06:17.443 INFO:teuthology.orchestra.run.smithi052.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-27T15:06:17.443 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-27T15:06:17.443 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-27T15:06:17.443 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-27T15:06:17.444 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-27T15:06:17.444 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cephfs x86_64 2:18.0.0-3906.g1c956514.el8 ceph 197 k 2023-11-27T15:06:17.444 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rados x86_64 2:18.0.0-3906.g1c956514.el8 ceph 371 k 2023-11-27T15:06:17.444 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rbd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 364 k 2023-11-27T15:06:17.444 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rgw x86_64 2:18.0.0-3906.g1c956514.el8 ceph 107 k 2023-11-27T15:06:17.444 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-fuse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 94 k 2023-11-27T15:06:17.444 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-mirror x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.4 M 2023-11-27T15:06:17.444 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-nbd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 180 k 2023-11-27T15:06:17.444 INFO:teuthology.orchestra.run.smithi052.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-27T15:06:17.444 INFO:teuthology.orchestra.run.smithi052.stdout:Upgrading: 2023-11-27T15:06:17.444 INFO:teuthology.orchestra.run.smithi052.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-27T15:06:17.444 INFO:teuthology.orchestra.run.smithi052.stdout: librados2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.7 M 2023-11-27T15:06:17.444 INFO:teuthology.orchestra.run.smithi052.stdout: librbd1 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.6 M 2023-11-27T15:06:17.445 INFO:teuthology.orchestra.run.smithi052.stdout:Installing dependencies: 2023-11-27T15:06:17.445 INFO:teuthology.orchestra.run.smithi052.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-27T15:06:17.445 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-common x86_64 2:18.0.0-3906.g1c956514.el8 ceph 22 M 2023-11-27T15:06:17.445 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 23 k 2023-11-27T15:06:17.445 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mds x86_64 2:18.0.0-3906.g1c956514.el8 ceph 2.3 M 2023-11-27T15:06:17.445 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 240 k 2023-11-27T15:06:17.445 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mon x86_64 2:18.0.0-3906.g1c956514.el8 ceph 4.6 M 2023-11-27T15:06:17.445 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-osd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 18 M 2023-11-27T15:06:17.445 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 15 k 2023-11-27T15:06:17.445 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-selinux x86_64 2:18.0.0-3906.g1c956514.el8 ceph 24 k 2023-11-27T15:06:17.445 INFO:teuthology.orchestra.run.smithi052.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-27T15:06:17.445 INFO:teuthology.orchestra.run.smithi052.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-27T15:06:17.445 INFO:teuthology.orchestra.run.smithi052.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-27T15:06:17.445 INFO:teuthology.orchestra.run.smithi052.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-27T15:06:17.446 INFO:teuthology.orchestra.run.smithi052.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-27T15:06:17.446 INFO:teuthology.orchestra.run.smithi052.stdout: libcephsqlite x86_64 2:18.0.0-3906.g1c956514.el8 ceph 137 k 2023-11-27T15:06:17.446 INFO:teuthology.orchestra.run.smithi052.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-27T15:06:17.446 INFO:teuthology.orchestra.run.smithi052.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-27T15:06:17.446 INFO:teuthology.orchestra.run.smithi052.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-27T15:06:17.446 INFO:teuthology.orchestra.run.smithi052.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-27T15:06:17.446 INFO:teuthology.orchestra.run.smithi052.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-27T15:06:17.446 INFO:teuthology.orchestra.run.smithi052.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-27T15:06:17.446 INFO:teuthology.orchestra.run.smithi052.stdout: libradosstriper1 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 473 k 2023-11-27T15:06:17.446 INFO:teuthology.orchestra.run.smithi052.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-27T15:06:17.446 INFO:teuthology.orchestra.run.smithi052.stdout: librgw2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 6.5 M 2023-11-27T15:06:17.446 INFO:teuthology.orchestra.run.smithi052.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-27T15:06:17.447 INFO:teuthology.orchestra.run.smithi052.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-27T15:06:17.447 INFO:teuthology.orchestra.run.smithi052.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-27T15:06:17.447 INFO:teuthology.orchestra.run.smithi052.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-27T15:06:17.447 INFO:teuthology.orchestra.run.smithi052.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-27T15:06:17.447 INFO:teuthology.orchestra.run.smithi052.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-27T15:06:17.447 INFO:teuthology.orchestra.run.smithi052.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-27T15:06:17.447 INFO:teuthology.orchestra.run.smithi052.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-27T15:06:17.447 INFO:teuthology.orchestra.run.smithi052.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-27T15:06:17.447 INFO:teuthology.orchestra.run.smithi052.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-27T15:06:17.447 INFO:teuthology.orchestra.run.smithi052.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-27T15:06:17.447 INFO:teuthology.orchestra.run.smithi052.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-27T15:06:17.447 INFO:teuthology.orchestra.run.smithi052.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-27T15:06:17.447 INFO:teuthology.orchestra.run.smithi052.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-27T15:06:17.448 INFO:teuthology.orchestra.run.smithi052.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-27T15:06:17.448 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-27T15:06:17.448 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-argparse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 45 k 2023-11-27T15:06:17.448 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-common x86_64 2:18.0.0-3906.g1c956514.el8 ceph 119 k 2023-11-27T15:06:17.448 INFO:teuthology.orchestra.run.smithi052.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-27T15:06:17.448 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-27T15:06:17.448 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-27T15:06:17.448 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-27T15:06:17.448 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-27T15:06:17.448 INFO:teuthology.orchestra.run.smithi052.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-27T15:06:17.448 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-27T15:06:17.448 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-27T15:06:17.448 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-27T15:06:17.448 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-27T15:06:17.449 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-27T15:06:17.449 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-27T15:06:17.449 INFO:teuthology.orchestra.run.smithi052.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-27T15:06:17.449 INFO:teuthology.orchestra.run.smithi052.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-27T15:06:17.449 INFO:teuthology.orchestra.run.smithi052.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-27T15:06:17.449 INFO:teuthology.orchestra.run.smithi052.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-27T15:06:17.449 INFO:teuthology.orchestra.run.smithi052.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-27T15:06:17.449 INFO:teuthology.orchestra.run.smithi052.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-27T15:06:17.449 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-27T15:06:17.449 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-27T15:06:17.449 INFO:teuthology.orchestra.run.smithi052.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-27T15:06:17.449 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-27T15:06:17.449 INFO:teuthology.orchestra.run.smithi052.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-27T15:06:17.449 INFO:teuthology.orchestra.run.smithi052.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-27T15:06:17.450 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-27T15:06:17.450 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-27T15:06:17.450 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-27T15:06:17.450 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-27T15:06:17.450 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-27T15:06:17.450 INFO:teuthology.orchestra.run.smithi052.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-27T15:06:17.450 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-27T15:06:17.450 INFO:teuthology.orchestra.run.smithi052.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-27T15:06:17.450 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-27T15:06:17.450 INFO:teuthology.orchestra.run.smithi052.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-27T15:06:17.450 INFO:teuthology.orchestra.run.smithi052.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-27T15:06:17.471 INFO:teuthology.orchestra.run.smithi052.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-27T15:06:17.471 INFO:teuthology.orchestra.run.smithi052.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-27T15:06:17.472 INFO:teuthology.orchestra.run.smithi052.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-27T15:06:17.472 INFO:teuthology.orchestra.run.smithi052.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-27T15:06:17.472 INFO:teuthology.orchestra.run.smithi052.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-27T15:06:17.472 INFO:teuthology.orchestra.run.smithi052.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-27T15:06:17.472 INFO:teuthology.orchestra.run.smithi052.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-27T15:06:17.472 INFO:teuthology.orchestra.run.smithi052.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-27T15:06:17.472 INFO:teuthology.orchestra.run.smithi052.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-27T15:06:17.472 INFO:teuthology.orchestra.run.smithi052.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-27T15:06:17.472 INFO:teuthology.orchestra.run.smithi052.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-27T15:06:17.472 INFO:teuthology.orchestra.run.smithi052.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-27T15:06:17.472 INFO:teuthology.orchestra.run.smithi052.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-27T15:06:17.472 INFO:teuthology.orchestra.run.smithi052.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-27T15:06:17.472 INFO:teuthology.orchestra.run.smithi052.stdout:Enabling module streams: 2023-11-27T15:06:17.473 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql 10 2023-11-27T15:06:17.473 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:06:17.473 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction Summary 2023-11-27T15:06:17.473 INFO:teuthology.orchestra.run.smithi052.stdout:=========================================================================================================================== 2023-11-27T15:06:17.473 INFO:teuthology.orchestra.run.smithi052.stdout:Install 128 Packages 2023-11-27T15:06:17.473 INFO:teuthology.orchestra.run.smithi052.stdout:Upgrade 3 Packages 2023-11-27T15:06:17.473 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:06:17.473 INFO:teuthology.orchestra.run.smithi052.stdout:Total download size: 197 M 2023-11-27T15:06:17.473 INFO:teuthology.orchestra.run.smithi052.stdout:Downloading Packages: 2023-11-27T15:06:17.473 INFO:teuthology.orchestra.run.smithi052.stdout:(1/131): ceph-18.0.0-3906.g1c956514.el8.x86_64. 19 kB/s | 6.4 kB 00:00 2023-11-27T15:06:17.473 INFO:teuthology.orchestra.run.smithi052.stdout:(2/131): ceph-fuse-18.0.0-3906.g1c956514.el8.x8 877 kB/s | 848 kB 00:00 2023-11-27T15:06:17.473 INFO:teuthology.orchestra.run.smithi052.stdout:(3/131): ceph-immutable-object-cache-18.0.0-390 903 kB/s | 165 kB 00:00 2023-11-27T15:06:17.473 INFO:teuthology.orchestra.run.smithi052.stdout:(4/131): ceph-base-18.0.0-3906.g1c956514.el8.x8 2.7 MB/s | 5.3 MB 00:02 2023-11-27T15:06:17.473 INFO:teuthology.orchestra.run.smithi052.stdout:(5/131): ceph-mgr-18.0.0-3906.g1c956514.el8.x86 4.1 MB/s | 1.5 MB 00:00 2023-11-27T15:06:17.474 INFO:teuthology.orchestra.run.smithi052.stdout:(6/131): ceph-mds-18.0.0-3906.g1c956514.el8.x86 2.0 MB/s | 2.3 MB 00:01 2023-11-27T15:06:17.474 INFO:teuthology.orchestra.run.smithi052.stdout:(7/131): ceph-mon-18.0.0-3906.g1c956514.el8.x86 5.0 MB/s | 4.6 MB 00:00 2023-11-27T15:06:18.950 INFO:teuthology.orchestra.run.smithi102.stdout:(10/131): ceph-common-18.0.0-3906.g1c956514.el8 3.2 MB/s | 22 MB 00:06 2023-11-27T15:06:19.142 INFO:teuthology.orchestra.run.smithi102.stdout:(11/131): cephfs-mirror-18.0.0-3906.g1c956514.e 1.3 MB/s | 258 kB 00:00 2023-11-27T15:06:19.250 INFO:teuthology.orchestra.run.smithi102.stdout:(12/131): libcephfs-devel-18.0.0-3906.g1c956514 286 kB/s | 31 kB 00:00 2023-11-27T15:06:19.654 INFO:teuthology.orchestra.run.smithi102.stdout:(13/131): libcephfs2-18.0.0-3906.g1c956514.el8. 1.9 MB/s | 771 kB 00:00 2023-11-27T15:06:19.767 INFO:teuthology.orchestra.run.smithi102.stdout:(14/131): libcephsqlite-18.0.0-3906.g1c956514.e 1.2 MB/s | 137 kB 00:00 2023-11-27T15:06:19.868 INFO:teuthology.orchestra.run.smithi102.stdout:(15/131): librados-devel-18.0.0-3906.g1c956514. 1.4 MB/s | 139 kB 00:00 2023-11-27T15:06:19.904 INFO:teuthology.orchestra.run.smithi052.stdout:(8/131): ceph-radosgw-18.0.0-3906.g1c956514.el8 5.3 MB/s | 13 MB 00:02 2023-11-27T15:06:20.013 INFO:teuthology.orchestra.run.smithi052.stdout:(9/131): ceph-selinux-18.0.0-3906.g1c956514.el8 225 kB/s | 24 kB 00:00 2023-11-27T15:06:20.043 INFO:teuthology.orchestra.run.smithi102.stdout:(16/131): libradosstriper1-18.0.0-3906.g1c95651 2.6 MB/s | 473 kB 00:00 2023-11-27T15:06:20.510 INFO:teuthology.orchestra.run.smithi102.stdout:(17/131): ceph-radosgw-18.0.0-3906.g1c956514.el 2.0 MB/s | 13 MB 00:06 2023-11-27T15:06:20.600 INFO:teuthology.orchestra.run.smithi150.stdout:(10/131): ceph-common-18.0.0-3906.g1c956514.el8 2.6 MB/s | 22 MB 00:08 2023-11-27T15:06:20.602 INFO:teuthology.orchestra.run.smithi102.stdout:(18/131): python3-ceph-argparse-18.0.0-3906.g1c 489 kB/s | 45 kB 00:00 2023-11-27T15:06:20.740 INFO:teuthology.orchestra.run.smithi102.stdout:(19/131): python3-ceph-common-18.0.0-3906.g1c95 860 kB/s | 119 kB 00:00 2023-11-27T15:06:20.800 INFO:teuthology.orchestra.run.smithi150.stdout:(11/131): cephfs-mirror-18.0.0-3906.g1c956514.e 1.3 MB/s | 258 kB 00:00 2023-11-27T15:06:20.942 INFO:teuthology.orchestra.run.smithi150.stdout:(12/131): libcephfs-devel-18.0.0-3906.g1c956514 219 kB/s | 31 kB 00:00 2023-11-27T15:06:20.961 INFO:teuthology.orchestra.run.smithi102.stdout:(20/131): python3-cephfs-18.0.0-3906.g1c956514. 894 kB/s | 197 kB 00:00 2023-11-27T15:06:21.270 INFO:teuthology.orchestra.run.smithi102.stdout:(21/131): python3-rados-18.0.0-3906.g1c956514.e 1.2 MB/s | 371 kB 00:00 2023-11-27T15:06:21.325 INFO:teuthology.orchestra.run.smithi150.stdout:(13/131): libcephfs2-18.0.0-3906.g1c956514.el8. 2.0 MB/s | 771 kB 00:00 2023-11-27T15:06:21.442 INFO:teuthology.orchestra.run.smithi150.stdout:(14/131): libcephsqlite-18.0.0-3906.g1c956514.e 1.1 MB/s | 137 kB 00:00 2023-11-27T15:06:21.545 INFO:teuthology.orchestra.run.smithi102.stdout:(22/131): python3-rbd-18.0.0-3906.g1c956514.el8 1.3 MB/s | 364 kB 00:00 2023-11-27T15:06:21.559 INFO:teuthology.orchestra.run.smithi150.stdout:(15/131): librados-devel-18.0.0-3906.g1c956514. 1.2 MB/s | 139 kB 00:00 2023-11-27T15:06:21.654 INFO:teuthology.orchestra.run.smithi102.stdout:(23/131): python3-rgw-18.0.0-3906.g1c956514.el8 985 kB/s | 107 kB 00:00 2023-11-27T15:06:21.768 INFO:teuthology.orchestra.run.smithi150.stdout:(16/131): libradosstriper1-18.0.0-3906.g1c95651 2.2 MB/s | 473 kB 00:00 2023-11-27T15:06:21.779 INFO:teuthology.orchestra.run.smithi102.stdout:(24/131): rbd-fuse-18.0.0-3906.g1c956514.el8.x8 756 kB/s | 94 kB 00:00 2023-11-27T15:06:21.913 INFO:teuthology.orchestra.run.smithi052.stdout:(10/131): ceph-common-18.0.0-3906.g1c956514.el8 2.8 MB/s | 22 MB 00:07 2023-11-27T15:06:22.068 INFO:teuthology.orchestra.run.smithi150.stdout:(17/131): ceph-radosgw-18.0.0-3906.g1c956514.el 1.7 MB/s | 13 MB 00:07 2023-11-27T15:06:22.105 INFO:teuthology.orchestra.run.smithi052.stdout:(11/131): cephfs-mirror-18.0.0-3906.g1c956514.e 1.3 MB/s | 258 kB 00:00 2023-11-27T15:06:22.221 INFO:teuthology.orchestra.run.smithi052.stdout:(12/131): libcephfs-devel-18.0.0-3906.g1c956514 266 kB/s | 31 kB 00:00 2023-11-27T15:06:22.427 INFO:teuthology.orchestra.run.smithi150.stdout:(18/131): python3-ceph-argparse-18.0.0-3906.g1c 125 kB/s | 45 kB 00:00 2023-11-27T15:06:22.597 INFO:teuthology.orchestra.run.smithi052.stdout:(13/131): libcephfs2-18.0.0-3906.g1c956514.el8. 2.0 MB/s | 771 kB 00:00 2023-11-27T15:06:22.619 INFO:teuthology.orchestra.run.smithi150.stdout:(19/131): python3-ceph-common-18.0.0-3906.g1c95 620 kB/s | 119 kB 00:00 2023-11-27T15:06:22.634 INFO:teuthology.orchestra.run.smithi102.stdout:(25/131): librgw2-18.0.0-3906.g1c956514.el8.x86 2.5 MB/s | 6.5 MB 00:02 2023-11-27T15:06:22.706 INFO:teuthology.orchestra.run.smithi052.stdout:(14/131): libcephsqlite-18.0.0-3906.g1c956514.e 1.2 MB/s | 137 kB 00:00 2023-11-27T15:06:22.777 INFO:teuthology.orchestra.run.smithi150.stdout:(20/131): python3-cephfs-18.0.0-3906.g1c956514. 1.2 MB/s | 197 kB 00:00 2023-11-27T15:06:22.806 INFO:teuthology.orchestra.run.smithi052.stdout:(15/131): librados-devel-18.0.0-3906.g1c956514. 1.4 MB/s | 139 kB 00:00 2023-11-27T15:06:23.019 INFO:teuthology.orchestra.run.smithi150.stdout:(21/131): python3-rados-18.0.0-3906.g1c956514.e 1.5 MB/s | 371 kB 00:00 2023-11-27T15:06:23.039 INFO:teuthology.orchestra.run.smithi052.stdout:(16/131): libradosstriper1-18.0.0-3906.g1c95651 2.0 MB/s | 473 kB 00:00 2023-11-27T15:06:23.088 INFO:teuthology.orchestra.run.smithi102.stdout:(26/131): ceph-test-18.0.0-3906.g1c956514.el8.x 5.8 MB/s | 41 MB 00:06 2023-11-27T15:06:23.113 INFO:teuthology.orchestra.run.smithi102.stdout:(27/131): rbd-nbd-18.0.0-3906.g1c956514.el8.x86 376 kB/s | 180 kB 00:00 2023-11-27T15:06:23.163 INFO:teuthology.orchestra.run.smithi102.stdout:(28/131): ceph-grafana-dashboards-18.0.0-3906.g 308 kB/s | 23 kB 00:00 2023-11-27T15:06:23.253 INFO:teuthology.orchestra.run.smithi150.stdout:(22/131): python3-rbd-18.0.0-3906.g1c956514.el8 1.5 MB/s | 364 kB 00:00 2023-11-27T15:06:23.289 INFO:teuthology.orchestra.run.smithi102.stdout:(29/131): ceph-mgr-cephadm-18.0.0-3906.g1c95651 727 kB/s | 127 kB 00:00 2023-11-27T15:06:23.361 INFO:teuthology.orchestra.run.smithi150.stdout:(23/131): python3-rgw-18.0.0-3906.g1c956514.el8 987 kB/s | 107 kB 00:00 2023-11-27T15:06:23.432 INFO:teuthology.orchestra.run.smithi052.stdout:(17/131): ceph-osd-18.0.0-3906.g1c956514.el8.x8 2.7 MB/s | 18 MB 00:06 2023-11-27T15:06:23.462 INFO:teuthology.orchestra.run.smithi150.stdout:(24/131): rbd-fuse-18.0.0-3906.g1c956514.el8.x8 945 kB/s | 94 kB 00:00 2023-11-27T15:06:23.472 INFO:teuthology.orchestra.run.smithi102.stdout:(30/131): ceph-mgr-dashboard-18.0.0-3906.g1c956 5.5 MB/s | 1.7 MB 00:00 2023-11-27T15:06:23.507 INFO:teuthology.orchestra.run.smithi052.stdout:(18/131): python3-ceph-argparse-18.0.0-3906.g1c 596 kB/s | 45 kB 00:00 2023-11-27T15:06:23.556 INFO:teuthology.orchestra.run.smithi102.stdout:(31/131): ceph-mgr-modules-core-18.0.0-3906.g1c 2.8 MB/s | 240 kB 00:00 2023-11-27T15:06:23.591 INFO:teuthology.orchestra.run.smithi052.stdout:(19/131): python3-ceph-common-18.0.0-3906.g1c95 1.4 MB/s | 119 kB 00:00 2023-11-27T15:06:23.614 INFO:teuthology.orchestra.run.smithi102.stdout:(32/131): rbd-mirror-18.0.0-3906.g1c956514.el8. 1.9 MB/s | 3.4 MB 00:01 2023-11-27T15:06:23.640 INFO:teuthology.orchestra.run.smithi102.stdout:(33/131): ceph-mgr-rook-18.0.0-3906.g1c956514.e 567 kB/s | 47 kB 00:00 2023-11-27T15:06:23.708 INFO:teuthology.orchestra.run.smithi052.stdout:(20/131): python3-cephfs-18.0.0-3906.g1c956514. 1.6 MB/s | 197 kB 00:00 2023-11-27T15:06:23.723 INFO:teuthology.orchestra.run.smithi102.stdout:(34/131): ceph-volume-18.0.0-3906.g1c956514.el8 3.1 MB/s | 265 kB 00:00 2023-11-27T15:06:23.740 INFO:teuthology.orchestra.run.smithi102.stdout:(35/131): ceph-prometheus-alerts-18.0.0-3906.g1 116 kB/s | 15 kB 00:00 2023-11-27T15:06:23.807 INFO:teuthology.orchestra.run.smithi102.stdout:(36/131): cephadm-18.0.0-3906.g1c956514.el8.noa 1.0 MB/s | 83 kB 00:00 2023-11-27T15:06:23.832 INFO:teuthology.orchestra.run.smithi102.stdout:(37/131): cephfs-top-18.0.0-3906.g1c956514.el8. 195 kB/s | 18 kB 00:00 2023-11-27T15:06:23.841 INFO:teuthology.orchestra.run.smithi052.stdout:(21/131): python3-rados-18.0.0-3906.g1c956514.e 2.7 MB/s | 371 kB 00:00 2023-11-27T15:06:23.950 INFO:teuthology.orchestra.run.smithi052.stdout:(22/131): python3-rbd-18.0.0-3906.g1c956514.el8 3.3 MB/s | 364 kB 00:00 2023-11-27T15:06:23.962 INFO:teuthology.orchestra.run.smithi150.stdout:(25/131): librgw2-18.0.0-3906.g1c956514.el8.x86 3.0 MB/s | 6.5 MB 00:02 2023-11-27T15:06:24.008 INFO:teuthology.orchestra.run.smithi052.stdout:(23/131): python3-rgw-18.0.0-3906.g1c956514.el8 1.8 MB/s | 107 kB 00:00 2023-11-27T15:06:24.062 INFO:teuthology.orchestra.run.smithi150.stdout:(26/131): rbd-nbd-18.0.0-3906.g1c956514.el8.x86 1.8 MB/s | 180 kB 00:00 2023-11-27T15:06:24.067 INFO:teuthology.orchestra.run.smithi052.stdout:(24/131): rbd-fuse-18.0.0-3906.g1c956514.el8.x8 1.6 MB/s | 94 kB 00:00 2023-11-27T15:06:24.146 INFO:teuthology.orchestra.run.smithi150.stdout:(27/131): ceph-grafana-dashboards-18.0.0-3906.g 277 kB/s | 23 kB 00:00 2023-11-27T15:06:24.246 INFO:teuthology.orchestra.run.smithi150.stdout:(28/131): ceph-mgr-cephadm-18.0.0-3906.g1c95651 1.2 MB/s | 127 kB 00:00 2023-11-27T15:06:24.508 INFO:teuthology.orchestra.run.smithi102.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 87 kB/s | 59 kB 00:00 2023-11-27T15:06:24.846 INFO:teuthology.orchestra.run.smithi150.stdout:(29/131): ceph-mgr-dashboard-18.0.0-3906.g1c956 2.8 MB/s | 1.7 MB 00:00 2023-11-27T15:06:24.892 INFO:teuthology.orchestra.run.smithi102.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 282 kB/s | 306 kB 00:01 2023-11-27T15:06:24.934 INFO:teuthology.orchestra.run.smithi052.stdout:(25/131): rbd-mirror-18.0.0-3906.g1c956514.el8. 4.0 MB/s | 3.4 MB 00:00 2023-11-27T15:06:25.009 INFO:teuthology.orchestra.run.smithi052.stdout:(26/131): rbd-nbd-18.0.0-3906.g1c956514.el8.x86 2.3 MB/s | 180 kB 00:00 2023-11-27T15:06:25.038 INFO:teuthology.orchestra.run.smithi150.stdout:(30/131): rbd-mirror-18.0.0-3906.g1c956514.el8. 2.2 MB/s | 3.4 MB 00:01 2023-11-27T15:06:25.059 INFO:teuthology.orchestra.run.smithi052.stdout:(27/131): ceph-grafana-dashboards-18.0.0-3906.g 463 kB/s | 23 kB 00:00 2023-11-27T15:06:25.117 INFO:teuthology.orchestra.run.smithi102.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 123 kB/s | 75 kB 00:00 2023-11-27T15:06:25.118 INFO:teuthology.orchestra.run.smithi052.stdout:(28/131): ceph-mgr-cephadm-18.0.0-3906.g1c95651 2.1 MB/s | 127 kB 00:00 2023-11-27T15:06:25.205 INFO:teuthology.orchestra.run.smithi150.stdout:(31/131): ceph-mgr-modules-core-18.0.0-3906.g1c 1.4 MB/s | 240 kB 00:00 2023-11-27T15:06:25.460 INFO:teuthology.orchestra.run.smithi052.stdout:(29/131): ceph-mgr-dashboard-18.0.0-3906.g1c956 5.0 MB/s | 1.7 MB 00:00 2023-11-27T15:06:25.525 INFO:teuthology.orchestra.run.smithi102.stdout:(41/131): ceph-mgr-diskprediction-local-18.0.0- 3.3 MB/s | 7.4 MB 00:02 2023-11-27T15:06:25.544 INFO:teuthology.orchestra.run.smithi052.stdout:(30/131): librgw2-18.0.0-3906.g1c956514.el8.x86 2.6 MB/s | 6.5 MB 00:02 2023-11-27T15:06:25.581 INFO:teuthology.orchestra.run.smithi150.stdout:(32/131): ceph-test-18.0.0-3906.g1c956514.el8.x 4.8 MB/s | 41 MB 00:08 2023-11-27T15:06:25.606 INFO:teuthology.orchestra.run.smithi150.stdout:(33/131): ceph-mgr-rook-18.0.0-3906.g1c956514.e 118 kB/s | 47 kB 00:00 2023-11-27T15:06:25.625 INFO:teuthology.orchestra.run.smithi102.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 86 kB/s | 44 kB 00:00 2023-11-27T15:06:25.631 INFO:teuthology.orchestra.run.smithi150.stdout:(34/131): ceph-prometheus-alerts-18.0.0-3906.g1 292 kB/s | 15 kB 00:00 2023-11-27T15:06:25.677 INFO:teuthology.orchestra.run.smithi052.stdout:(31/131): ceph-mgr-modules-core-18.0.0-3906.g1c 1.8 MB/s | 240 kB 00:00 2023-11-27T15:06:25.690 INFO:teuthology.orchestra.run.smithi150.stdout:(35/131): cephadm-18.0.0-3906.g1c956514.el8.noa 1.4 MB/s | 83 kB 00:00 2023-11-27T15:06:25.740 INFO:teuthology.orchestra.run.smithi150.stdout:(36/131): cephfs-top-18.0.0-3906.g1c956514.el8. 358 kB/s | 18 kB 00:00 2023-11-27T15:06:25.769 INFO:teuthology.orchestra.run.smithi052.stdout:(32/131): ceph-mgr-rook-18.0.0-3906.g1c956514.e 518 kB/s | 47 kB 00:00 2023-11-27T15:06:25.799 INFO:teuthology.orchestra.run.smithi150.stdout:(37/131): ceph-volume-18.0.0-3906.g1c956514.el8 1.3 MB/s | 265 kB 00:00 2023-11-27T15:06:25.918 INFO:teuthology.orchestra.run.smithi102.stdout:(43/131): python3-asyncssh-2.7.0-2.el8.noarch.r 464 kB/s | 476 kB 00:01 2023-11-27T15:06:26.093 INFO:teuthology.orchestra.run.smithi102.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 71 kB/s | 33 kB 00:00 2023-11-27T15:06:26.124 INFO:teuthology.orchestra.run.smithi150.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 183 kB/s | 59 kB 00:00 2023-11-27T15:06:26.187 INFO:teuthology.orchestra.run.smithi052.stdout:(33/131): ceph-test-18.0.0-3906.g1c956514.el8.x 6.6 MB/s | 41 MB 00:06 2023-11-27T15:06:26.199 INFO:teuthology.orchestra.run.smithi150.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 667 kB/s | 306 kB 00:00 2023-11-27T15:06:26.211 INFO:teuthology.orchestra.run.smithi052.stdout:(34/131): ceph-prometheus-alerts-18.0.0-3906.g1 33 kB/s | 15 kB 00:00 2023-11-27T15:06:26.241 INFO:teuthology.orchestra.run.smithi150.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 641 kB/s | 75 kB 00:00 2023-11-27T15:06:26.253 INFO:teuthology.orchestra.run.smithi052.stdout:(35/131): ceph-volume-18.0.0-3906.g1c956514.el8 3.9 MB/s | 265 kB 00:00 2023-11-27T15:06:26.287 INFO:teuthology.orchestra.run.smithi052.stdout:(36/131): cephadm-18.0.0-3906.g1c956514.el8.noa 1.1 MB/s | 83 kB 00:00 2023-11-27T15:06:26.312 INFO:teuthology.orchestra.run.smithi052.stdout:(37/131): cephfs-top-18.0.0-3906.g1c956514.el8. 307 kB/s | 18 kB 00:00 2023-11-27T15:06:26.316 INFO:teuthology.orchestra.run.smithi150.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 583 kB/s | 44 kB 00:00 2023-11-27T15:06:26.334 INFO:teuthology.orchestra.run.smithi102.stdout:(45/131): python3-beautifulsoup4-4.6.3-2.el8.1. 229 kB/s | 185 kB 00:00 2023-11-27T15:06:26.385 INFO:teuthology.orchestra.run.smithi102.stdout:(46/131): python3-certifi-2018.10.15-7.el8.noar 35 kB/s | 16 kB 00:00 2023-11-27T15:06:26.484 INFO:teuthology.orchestra.run.smithi150.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.1 MB/s | 185 kB 00:00 2023-11-27T15:06:26.551 INFO:teuthology.orchestra.run.smithi150.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 494 kB/s | 33 kB 00:00 2023-11-27T15:06:26.584 INFO:teuthology.orchestra.run.smithi150.stdout:(44/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.2 MB/s | 476 kB 00:00 2023-11-27T15:06:26.618 INFO:teuthology.orchestra.run.smithi150.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 244 kB/s | 16 kB 00:00 2023-11-27T15:06:26.646 INFO:teuthology.orchestra.run.smithi052.stdout:(38/131): ceph-mgr-diskprediction-local-18.0.0- 6.2 MB/s | 7.4 MB 00:01 2023-11-27T15:06:26.760 INFO:teuthology.orchestra.run.smithi150.stdout:(46/131): ceph-mgr-diskprediction-local-18.0.0- 3.9 MB/s | 7.4 MB 00:01 2023-11-27T15:06:26.785 INFO:teuthology.orchestra.run.smithi150.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 860 kB/s | 173 kB 00:00 2023-11-27T15:06:26.844 INFO:teuthology.orchestra.run.smithi150.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 190 kB/s | 11 kB 00:00 2023-11-27T15:06:26.877 INFO:teuthology.orchestra.run.smithi102.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 220 kB/s | 173 kB 00:00 2023-11-27T15:06:26.902 INFO:teuthology.orchestra.run.smithi150.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 301 kB/s | 18 kB 00:00 2023-11-27T15:06:26.928 INFO:teuthology.orchestra.run.smithi150.stdout:(50/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.2 MB/s | 384 kB 00:00 2023-11-27T15:06:26.963 INFO:teuthology.orchestra.run.smithi052.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 90 kB/s | 59 kB 00:00 2023-11-27T15:06:26.977 INFO:teuthology.orchestra.run.smithi102.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 137 kB/s | 81 kB 00:00 2023-11-27T15:06:26.986 INFO:teuthology.orchestra.run.smithi150.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 837 kB/s | 49 kB 00:00 2023-11-27T15:06:27.053 INFO:teuthology.orchestra.run.smithi150.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 884 kB/s | 59 kB 00:00 2023-11-27T15:06:27.078 INFO:teuthology.orchestra.run.smithi150.stdout:(53/131): python3-google-auth-1.1.1-10.el8.noar 255 kB/s | 81 kB 00:00 2023-11-27T15:06:27.112 INFO:teuthology.orchestra.run.smithi150.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.0 MB/s | 60 kB 00:00 2023-11-27T15:06:27.171 INFO:teuthology.orchestra.run.smithi150.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 267 kB/s | 16 kB 00:00 2023-11-27T15:06:27.238 INFO:teuthology.orchestra.run.smithi150.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 504 kB/s | 34 kB 00:00 2023-11-27T15:06:27.244 INFO:teuthology.orchestra.run.smithi102.stdout:(49/131): python3-cherrypy-18.4.0-1.el8.noarch. 423 kB/s | 384 kB 00:00 2023-11-27T15:06:27.261 INFO:teuthology.orchestra.run.smithi102.stdout:(50/131): python3-jaraco-6.2-6.el8.noarch.rpm 29 kB/s | 11 kB 00:00 2023-11-27T15:06:27.355 INFO:teuthology.orchestra.run.smithi150.stdout:(57/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.0 MB/s | 283 kB 00:00 2023-11-27T15:06:27.378 INFO:teuthology.orchestra.run.smithi102.stdout:(51/131): python3-jaraco-functools-2.0-4.el8.no 44 kB/s | 18 kB 00:00 2023-11-27T15:06:27.421 INFO:teuthology.orchestra.run.smithi150.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.0 MB/s | 196 kB 00:00 2023-11-27T15:06:27.447 INFO:teuthology.orchestra.run.smithi150.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 700 kB/s | 64 kB 00:00 2023-11-27T15:06:27.480 INFO:teuthology.orchestra.run.smithi150.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 320 kB/s | 19 kB 00:00 2023-11-27T15:06:27.505 INFO:teuthology.orchestra.run.smithi052.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 87 kB/s | 75 kB 00:00 2023-11-27T15:06:27.505 INFO:teuthology.orchestra.run.smithi150.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 408 kB/s | 24 kB 00:00 2023-11-27T15:06:27.539 INFO:teuthology.orchestra.run.smithi150.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 498 kB/s | 29 kB 00:00 2023-11-27T15:06:27.564 INFO:teuthology.orchestra.run.smithi150.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 469 kB/s | 27 kB 00:00 2023-11-27T15:06:27.680 INFO:teuthology.orchestra.run.smithi052.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 249 kB/s | 44 kB 00:00 2023-11-27T15:06:27.698 INFO:teuthology.orchestra.run.smithi150.stdout:(64/131): python3-kubernetes-11.0.0-6.el8.noarc 1.5 MB/s | 1.2 MB 00:00 2023-11-27T15:06:27.703 INFO:teuthology.orchestra.run.smithi102.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 110 kB/s | 49 kB 00:00 2023-11-27T15:06:27.724 INFO:teuthology.orchestra.run.smithi150.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 1.4 MB/s | 255 kB 00:00 2023-11-27T15:06:27.749 INFO:teuthology.orchestra.run.smithi150.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.3 MB/s | 251 kB 00:00 2023-11-27T15:06:27.774 INFO:teuthology.orchestra.run.smithi150.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 809 kB/s | 61 kB 00:00 2023-11-27T15:06:27.820 INFO:teuthology.orchestra.run.smithi102.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 133 kB/s | 59 kB 00:00 2023-11-27T15:06:27.875 INFO:teuthology.orchestra.run.smithi150.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 183 kB/s | 23 kB 00:00 2023-11-27T15:06:27.900 INFO:teuthology.orchestra.run.smithi150.stdout:(69/131): python3-webtest-2.0.33-1.el8.noarch.r 486 kB/s | 85 kB 00:00 2023-11-27T15:06:27.925 INFO:teuthology.orchestra.run.smithi150.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-11-27T15:06:27.972 INFO:teuthology.orchestra.run.smithi052.stdout:(42/131): gperftools-libs-2.7-9.el8.x86_64.rpm 182 kB/s | 306 kB 00:01 2023-11-27T15:06:27.997 INFO:teuthology.orchestra.run.smithi052.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 584 kB/s | 185 kB 00:00 2023-11-27T15:06:28.064 INFO:teuthology.orchestra.run.smithi052.stdout:(44/131): python3-asyncssh-2.7.0-2.el8.noarch.r 432 kB/s | 476 kB 00:01 2023-11-27T15:06:28.076 INFO:teuthology.orchestra.run.smithi150.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 762 kB/s | 110 kB 00:00 2023-11-27T15:06:28.089 INFO:teuthology.orchestra.run.smithi052.stdout:(45/131): python3-cachetools-3.1.1-4.el8.noarch 282 kB/s | 33 kB 00:00 2023-11-27T15:06:28.123 INFO:teuthology.orchestra.run.smithi052.stdout:(46/131): python3-certifi-2018.10.15-7.el8.noar 129 kB/s | 16 kB 00:00 2023-11-27T15:06:28.137 INFO:teuthology.orchestra.run.smithi102.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 139 kB/s | 60 kB 00:00 2023-11-27T15:06:28.226 INFO:teuthology.orchestra.run.smithi150.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 362 kB/s | 54 kB 00:00 2023-11-27T15:06:28.240 INFO:teuthology.orchestra.run.smithi052.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 982 kB/s | 173 kB 00:00 2023-11-27T15:06:28.393 INFO:teuthology.orchestra.run.smithi150.stdout:(73/131): python3-scipy-1.0.0-21.module+el8.5.0 28 MB/s | 14 MB 00:00 2023-11-27T15:06:28.407 INFO:teuthology.orchestra.run.smithi052.stdout:(48/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.2 MB/s | 384 kB 00:00 2023-11-27T15:06:28.418 INFO:teuthology.orchestra.run.smithi150.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.3 MB/s | 259 kB 00:00 2023-11-27T15:06:28.443 INFO:teuthology.orchestra.run.smithi150.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-11-27T15:06:28.513 INFO:teuthology.orchestra.run.smithi102.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 409 kB/s | 283 kB 00:00 2023-11-27T15:06:28.524 INFO:teuthology.orchestra.run.smithi052.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 39 kB/s | 11 kB 00:00 2023-11-27T15:06:28.541 INFO:teuthology.orchestra.run.smithi052.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 131 kB/s | 18 kB 00:00 2023-11-27T15:06:28.546 INFO:teuthology.orchestra.run.smithi102.stdout:(56/131): python3-kubernetes-11.0.0-6.el8.noarc 917 kB/s | 1.2 MB 00:01 2023-11-27T15:06:28.551 INFO:teuthology.orchestra.run.smithi150.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 291 kB/s | 39 kB 00:00 2023-11-27T15:06:28.563 INFO:teuthology.orchestra.run.smithi102.stdout:(57/131): python3-portend-2.6-1.el8.noarch.rpm 37 kB/s | 16 kB 00:00 2023-11-27T15:06:28.575 INFO:teuthology.orchestra.run.smithi052.stdout:(51/131): python3-google-auth-1.1.1-10.el8.noar 180 kB/s | 81 kB 00:00 2023-11-27T15:06:28.602 INFO:teuthology.orchestra.run.smithi150.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 4.2 MB/s | 688 kB 00:00 2023-11-27T15:06:28.702 INFO:teuthology.orchestra.run.smithi150.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 3.5 MB/s | 538 kB 00:00 2023-11-27T15:06:28.803 INFO:teuthology.orchestra.run.smithi150.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 783 kB/s | 157 kB 00:00 2023-11-27T15:06:28.816 INFO:teuthology.orchestra.run.smithi052.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 177 kB/s | 49 kB 00:00 2023-11-27T15:06:28.828 INFO:teuthology.orchestra.run.smithi150.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.5 MB/s | 187 kB 00:00 2023-11-27T15:06:28.850 INFO:teuthology.orchestra.run.smithi052.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 213 kB/s | 59 kB 00:00 2023-11-27T15:06:28.903 INFO:teuthology.orchestra.run.smithi150.stdout:(81/131): thrift-0.13.0-2.el8.x86_64.rpm 1.5 MB/s | 1.7 MB 00:01 2023-11-27T15:06:28.979 INFO:teuthology.orchestra.run.smithi150.stdout:(82/131): automake-1.16.1-7.el8.noarch.rpm 4.0 MB/s | 713 kB 00:00 2023-11-27T15:06:28.988 INFO:teuthology.orchestra.run.smithi102.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 71 kB/s | 34 kB 00:00 2023-11-27T15:06:29.047 INFO:teuthology.orchestra.run.smithi052.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 262 kB/s | 60 kB 00:00 2023-11-27T15:06:29.097 INFO:teuthology.orchestra.run.smithi102.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 120 kB/s | 64 kB 00:00 2023-11-27T15:06:29.104 INFO:teuthology.orchestra.run.smithi150.stdout:(83/131): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2023-11-27T15:06:29.121 INFO:teuthology.orchestra.run.smithi150.stdout:(84/131): python3-jsonpatch-1.21-2.el8.noarch.r 122 kB/s | 27 kB 00:00 2023-11-27T15:06:29.146 INFO:teuthology.orchestra.run.smithi150.stdout:(85/131): flex-2.6.1-9.el8.x86_64.rpm 1.9 MB/s | 320 kB 00:00 2023-11-27T15:06:29.179 INFO:teuthology.orchestra.run.smithi150.stdout:(86/131): python3-prettytable-0.7.2-14.el8.noar 583 kB/s | 44 kB 00:00 2023-11-27T15:06:29.201 INFO:teuthology.orchestra.run.smithi052.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 807 kB/s | 283 kB 00:00 2023-11-27T15:06:29.256 INFO:teuthology.orchestra.run.smithi102.stdout:(60/131): python3-routes-2.4.1-12.el8.noarch.rp 277 kB/s | 196 kB 00:00 2023-11-27T15:06:29.259 INFO:teuthology.orchestra.run.smithi052.stdout:(56/131): python3-kubernetes-11.0.0-6.el8.noarc 1.6 MB/s | 1.2 MB 00:00 2023-11-27T15:06:29.263 INFO:teuthology.orchestra.run.smithi150.stdout:(87/131): python3-werkzeug-0.12.2-4.el8.noarch. 3.8 MB/s | 457 kB 00:00 2023-11-27T15:06:29.339 INFO:teuthology.orchestra.run.smithi150.stdout:(88/131): python3-numpy-1.14.3-10.el8.x86_64.rp 17 MB/s | 3.7 MB 00:00 2023-11-27T15:06:29.364 INFO:teuthology.orchestra.run.smithi150.stdout:(89/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.2 MB/s | 225 kB 00:00 2023-11-27T15:06:29.385 INFO:teuthology.orchestra.run.smithi052.stdout:(57/131): python3-portend-2.6-1.el8.noarch.rpm 46 kB/s | 16 kB 00:00 2023-11-27T15:06:29.389 INFO:teuthology.orchestra.run.smithi150.stdout:(90/131): uuid-1.6.2-43.el8.x86_64.rpm 510 kB/s | 64 kB 00:00 2023-11-27T15:06:29.389 INFO:teuthology.orchestra.run.smithi102.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 47 kB/s | 19 kB 00:00 2023-11-27T15:06:29.427 INFO:teuthology.orchestra.run.smithi052.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.1 MB/s | 196 kB 00:00 2023-11-27T15:06:29.456 INFO:teuthology.orchestra.run.smithi150.stdout:(91/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.5 MB/s | 103 kB 00:00 2023-11-27T15:06:29.473 INFO:teuthology.orchestra.run.smithi102.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 64 kB/s | 24 kB 00:00 2023-11-27T15:06:29.481 INFO:teuthology.orchestra.run.smithi150.stdout:(92/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 888 kB/s | 126 kB 00:00 2023-11-27T15:06:29.548 INFO:teuthology.orchestra.run.smithi150.stdout:(93/131): python3-jsonpointer-1.10-11.el8.noarc 108 kB/s | 20 kB 00:00 2023-11-27T15:06:29.632 INFO:teuthology.orchestra.run.smithi102.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 78 kB/s | 29 kB 00:00 2023-11-27T15:06:29.632 INFO:teuthology.orchestra.run.smithi150.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 3.9 MB/s | 709 kB 00:00 2023-11-27T15:06:29.666 INFO:teuthology.orchestra.run.smithi150.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 1.6 MB/s | 198 kB 00:00 2023-11-27T15:06:29.685 INFO:teuthology.orchestra.run.smithi052.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 72 kB/s | 19 kB 00:00 2023-11-27T15:06:29.710 INFO:teuthology.orchestra.run.smithi052.stdout:(60/131): python3-repoze-lru-0.7-6.el8.noarch.r 66 kB/s | 34 kB 00:00 2023-11-27T15:06:29.736 INFO:teuthology.orchestra.run.smithi052.stdout:(61/131): python3-rsa-4.9-2.el8.noarch.rpm 183 kB/s | 64 kB 00:00 2023-11-27T15:06:29.761 INFO:teuthology.orchestra.run.smithi052.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 317 kB/s | 24 kB 00:00 2023-11-27T15:06:29.773 INFO:teuthology.orchestra.run.smithi102.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 72 kB/s | 27 kB 00:00 2023-11-27T15:06:29.774 INFO:teuthology.orchestra.run.smithi150.stdout:(96/131): openblas-threads-0.3.15-3.el8.x86_64. 17 MB/s | 4.9 MB 00:00 2023-11-27T15:06:29.824 INFO:teuthology.orchestra.run.smithi150.stdout:(97/131): jq-1.6-3.el8.x86_64.rpm 1.2 MB/s | 202 kB 00:00 2023-11-27T15:06:29.828 INFO:teuthology.orchestra.run.smithi052.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 249 kB/s | 29 kB 00:00 2023-11-27T15:06:29.853 INFO:teuthology.orchestra.run.smithi052.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 235 kB/s | 27 kB 00:00 2023-11-27T15:06:29.887 INFO:teuthology.orchestra.run.smithi052.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 2.0 MB/s | 255 kB 00:00 2023-11-27T15:06:29.924 INFO:teuthology.orchestra.run.smithi150.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.3 MB/s | 354 kB 00:00 2023-11-27T15:06:29.987 INFO:teuthology.orchestra.run.smithi052.stdout:(66/131): python3-webtest-2.0.33-1.el8.noarch.r 854 kB/s | 85 kB 00:00 2023-11-27T15:06:30.013 INFO:teuthology.orchestra.run.smithi052.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 385 kB/s | 61 kB 00:00 2023-11-27T15:06:30.017 INFO:teuthology.orchestra.run.smithi150.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 12 MB/s | 4.7 MB 00:00 2023-11-27T15:06:30.092 INFO:teuthology.orchestra.run.smithi150.stdout:(100/131): postgresql-contrib-10.21-2.module+el 4.7 MB/s | 808 kB 00:00 2023-11-27T15:06:30.154 INFO:teuthology.orchestra.run.smithi052.stdout:(68/131): python3-webob-1.8.5-1.el8.1.noarch.rp 770 kB/s | 251 kB 00:00 2023-11-27T15:06:30.159 INFO:teuthology.orchestra.run.smithi150.stdout:(101/131): postgresql-server-10.21-2.module+el8 15 MB/s | 5.1 MB 00:00 2023-11-27T15:06:30.174 INFO:teuthology.orchestra.run.smithi102.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 367 kB/s | 255 kB 00:00 2023-11-27T15:06:30.206 INFO:teuthology.orchestra.run.smithi052.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 105 kB/s | 23 kB 00:00 2023-11-27T15:06:30.216 INFO:teuthology.orchestra.run.smithi102.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 139 kB/s | 61 kB 00:00 2023-11-27T15:06:30.217 INFO:teuthology.orchestra.run.smithi150.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 7.5 MB/s | 1.5 MB 00:00 2023-11-27T15:06:30.238 INFO:teuthology.orchestra.run.smithi052.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 833 kB/s | 70 kB 00:00 2023-11-27T15:06:30.242 INFO:teuthology.orchestra.run.smithi150.stdout:(103/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 311 kB/s | 47 kB 00:00 2023-11-27T15:06:30.301 INFO:teuthology.orchestra.run.smithi150.stdout:(104/131): libselinux-devel-2.9-5.el8.x86_64.rp 1.4 MB/s | 200 kB 00:00 2023-11-27T15:06:30.316 INFO:teuthology.orchestra.run.smithi102.stdout:(67/131): python3-webob-1.8.5-1.el8.1.noarch.rp 367 kB/s | 251 kB 00:00 2023-11-27T15:06:30.402 INFO:teuthology.orchestra.run.smithi150.stdout:(105/131): python3-jwt-1.6.1-2.el8.noarch.rpm 231 kB/s | 43 kB 00:00 2023-11-27T15:06:30.427 INFO:teuthology.orchestra.run.smithi150.stdout:(106/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.2 MB/s | 220 kB 00:00 2023-11-27T15:06:30.447 INFO:teuthology.orchestra.run.smithi052.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 529 kB/s | 110 kB 00:00 2023-11-27T15:06:30.494 INFO:teuthology.orchestra.run.smithi150.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 948 kB/s | 87 kB 00:00 2023-11-27T15:06:30.527 INFO:teuthology.orchestra.run.smithi150.stdout:(108/131): keyutils-libs-devel-1.5.10-9.el8.x86 213 kB/s | 48 kB 00:00 2023-11-27T15:06:30.547 INFO:teuthology.orchestra.run.smithi052.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 542 kB/s | 54 kB 00:00 2023-11-27T15:06:30.569 INFO:teuthology.orchestra.run.smithi150.stdout:(109/131): python3-requests-oauthlib-1.0.0-1.el 300 kB/s | 43 kB 00:00 2023-11-27T15:06:30.595 INFO:teuthology.orchestra.run.smithi150.stdout:(110/131): python3-pycparser-2.14-14.el8.noarch 1.1 MB/s | 109 kB 00:00 2023-11-27T15:06:30.600 INFO:teuthology.orchestra.run.smithi102.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 60 kB/s | 23 kB 00:00 2023-11-27T15:06:30.625 INFO:teuthology.orchestra.run.smithi102.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-11-27T15:06:30.692 INFO:teuthology.orchestra.run.smithi102.stdout:(70/131): python3-webtest-2.0.33-1.el8.noarch.r 165 kB/s | 85 kB 00:00 2023-11-27T15:06:30.703 INFO:teuthology.orchestra.run.smithi150.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.1 MB/s | 560 kB 00:00 2023-11-27T15:06:30.728 INFO:teuthology.orchestra.run.smithi150.stdout:(112/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.1 MB/s | 229 kB 00:00 2023-11-27T15:06:30.731 INFO:teuthology.orchestra.run.smithi052.stdout:(73/131): python3-scipy-1.0.0-21.module+el8.5.0 26 MB/s | 14 MB 00:00 2023-11-27T15:06:30.770 INFO:teuthology.orchestra.run.smithi150.stdout:(113/131): libaio-devel-0.3.112-1.el8.x86_64.rp 107 kB/s | 19 kB 00:00 2023-11-27T15:06:30.775 INFO:teuthology.orchestra.run.smithi052.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.1 MB/s | 259 kB 00:00 2023-11-27T15:06:30.801 INFO:teuthology.orchestra.run.smithi102.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 267 kB/s | 47 kB 00:00 2023-11-27T15:06:30.815 INFO:teuthology.orchestra.run.smithi052.stdout:(75/131): thrift-0.13.0-2.el8.x86_64.rpm 2.2 MB/s | 1.7 MB 00:00 2023-11-27T15:06:30.859 INFO:teuthology.orchestra.run.smithi102.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 1.2 MB/s | 200 kB 00:00 2023-11-27T15:06:30.871 INFO:teuthology.orchestra.run.smithi150.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 3.9 MB/s | 559 kB 00:00 2023-11-27T15:06:30.896 INFO:teuthology.orchestra.run.smithi150.stdout:(115/131): python3-oauthlib-2.1.0-1.el8.noarch. 806 kB/s | 155 kB 00:00 2023-11-27T15:06:30.913 INFO:teuthology.orchestra.run.smithi150.stdout:(116/131): libverto-devel-0.3.0-5.el8.x86_64.rp 126 kB/s | 18 kB 00:00 2023-11-27T15:06:30.915 INFO:teuthology.orchestra.run.smithi052.stdout:(76/131): socat-1.7.4.1-1.el8.x86_64.rpm 1.7 MB/s | 323 kB 00:00 2023-11-27T15:06:30.918 INFO:teuthology.orchestra.run.smithi102.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 365 kB/s | 43 kB 00:00 2023-11-27T15:06:30.957 INFO:teuthology.orchestra.run.smithi052.stdout:(77/131): python3-markupsafe-0.23-19.el8.x86_64 214 kB/s | 39 kB 00:00 2023-11-27T15:06:30.985 INFO:teuthology.orchestra.run.smithi102.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.7 MB/s | 220 kB 00:00 2023-11-27T15:06:30.999 INFO:teuthology.orchestra.run.smithi052.stdout:(78/131): bison-3.0.4-10.el8.x86_64.rpm 3.7 MB/s | 688 kB 00:00 2023-11-27T15:06:31.005 INFO:teuthology.orchestra.run.smithi150.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.1 MB/s | 238 kB 00:00 2023-11-27T15:06:31.055 INFO:teuthology.orchestra.run.smithi150.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 3.2 MB/s | 605 kB 00:00 2023-11-27T15:06:31.058 INFO:teuthology.orchestra.run.smithi052.stdout:(79/131): python3-jinja2-2.10.1-3.el8.noarch.rp 3.7 MB/s | 538 kB 00:00 2023-11-27T15:06:31.100 INFO:teuthology.orchestra.run.smithi052.stdout:(80/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.1 MB/s | 157 kB 00:00 2023-11-27T15:06:31.113 INFO:teuthology.orchestra.run.smithi150.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 12 MB/s | 2.3 MB 00:00 2023-11-27T15:06:31.118 INFO:teuthology.orchestra.run.smithi102.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 240 kB/s | 48 kB 00:00 2023-11-27T15:06:31.135 INFO:teuthology.orchestra.run.smithi102.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 578 kB/s | 87 kB 00:00 2023-11-27T15:06:31.139 INFO:teuthology.orchestra.run.smithi150.stdout:(120/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 540 kB/s | 72 kB 00:00 2023-11-27T15:06:31.192 INFO:teuthology.orchestra.run.smithi052.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 5.2 MB/s | 713 kB 00:00 2023-11-27T15:06:31.217 INFO:teuthology.orchestra.run.smithi052.stdout:(82/131): oniguruma-6.8.2-2.el8.x86_64.rpm 862 kB/s | 187 kB 00:00 2023-11-27T15:06:31.261 INFO:teuthology.orchestra.run.smithi102.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 301 kB/s | 43 kB 00:00 2023-11-27T15:06:31.286 INFO:teuthology.orchestra.run.smithi102.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 725 kB/s | 109 kB 00:00 2023-11-27T15:06:31.290 INFO:teuthology.orchestra.run.smithi150.stdout:(121/131): fuse-2.9.7-15.el8.x86_64.rpm 354 kB/s | 83 kB 00:00 2023-11-27T15:06:31.306 INFO:teuthology.orchestra.run.smithi150.stdout:(122/131): elfutils-libelf-devel-0.186-1.el8.x8 311 kB/s | 60 kB 00:00 2023-11-27T15:06:31.317 INFO:teuthology.orchestra.run.smithi052.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 212 kB/s | 27 kB 00:00 2023-11-27T15:06:31.348 INFO:teuthology.orchestra.run.smithi150.stdout:(123/131): libuuid-devel-2.32.1-35.el8.x86_64.r 470 kB/s | 98 kB 00:00 2023-11-27T15:06:31.365 INFO:teuthology.orchestra.run.smithi150.stdout:(124/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 763 kB/s | 58 kB 00:00 2023-11-27T15:06:31.367 INFO:teuthology.orchestra.run.smithi052.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 884 kB/s | 44 kB 00:00 2023-11-27T15:06:31.412 INFO:teuthology.orchestra.run.smithi102.stdout:(79/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.4 MB/s | 560 kB 00:00 2023-11-27T15:06:31.443 INFO:teuthology.orchestra.run.smithi052.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 14 MB/s | 4.8 MB 00:00 2023-11-27T15:06:31.453 INFO:teuthology.orchestra.run.smithi102.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.2 MB/s | 229 kB 00:00 2023-11-27T15:06:31.460 INFO:teuthology.orchestra.run.smithi052.stdout:(86/131): flex-2.6.1-9.el8.x86_64.rpm 1.3 MB/s | 320 kB 00:00 2023-11-27T15:06:31.465 INFO:teuthology.orchestra.run.smithi150.stdout:(125/131): libcom_err-devel-1.45.6-4.el8.x86_64 242 kB/s | 39 kB 00:00 2023-11-27T15:06:31.507 INFO:teuthology.orchestra.run.smithi150.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 1.2 MB/s | 171 kB 00:00 2023-11-27T15:06:31.541 INFO:teuthology.orchestra.run.smithi150.stdout:(127/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 3.3 MB/s | 644 kB 00:00 2023-11-27T15:06:31.610 INFO:teuthology.orchestra.run.smithi052.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.5 MB/s | 225 kB 00:00 2023-11-27T15:06:31.620 INFO:teuthology.orchestra.run.smithi102.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 90 kB/s | 19 kB 00:00 2023-11-27T15:06:31.646 INFO:teuthology.orchestra.run.smithi102.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 807 kB/s | 155 kB 00:00 2023-11-27T15:06:31.683 INFO:teuthology.orchestra.run.smithi150.stdout:(128/131): NetworkManager-initscripts-updown-1. 635 kB/s | 138 kB 00:00 2023-11-27T15:06:31.718 INFO:teuthology.orchestra.run.smithi052.stdout:(88/131): python3-numpy-1.14.3-10.el8.x86_64.rp 11 MB/s | 3.7 MB 00:00 2023-11-27T15:06:31.752 INFO:teuthology.orchestra.run.smithi052.stdout:(89/131): python3-werkzeug-0.12.2-4.el8.noarch. 1.4 MB/s | 457 kB 00:00 2023-11-27T15:06:31.778 INFO:teuthology.orchestra.run.smithi052.stdout:(90/131): uuid-1.6.2-43.el8.x86_64.rpm 383 kB/s | 64 kB 00:00 2023-11-27T15:06:31.796 INFO:teuthology.orchestra.run.smithi102.stdout:(83/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 119 kB/s | 18 kB 00:00 2023-11-27T15:06:31.817 INFO:teuthology.orchestra.run.smithi150.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 5.9 MB/s | 798 kB 00:00 2023-11-27T15:06:31.829 INFO:teuthology.orchestra.run.smithi102.stdout:(84/131): python3-cryptography-3.2.1-5.el8.x86_ 2.6 MB/s | 559 kB 00:00 2023-11-27T15:06:31.869 INFO:teuthology.orchestra.run.smithi052.stdout:(91/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 840 kB/s | 126 kB 00:00 2023-11-27T15:06:31.923 INFO:teuthology.orchestra.run.smithi052.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 713 kB/s | 103 kB 00:00 2023-11-27T15:06:31.930 INFO:teuthology.orchestra.run.smithi102.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.4 MB/s | 605 kB 00:00 2023-11-27T15:06:31.962 INFO:teuthology.orchestra.run.smithi052.stdout:(93/131): python3-jsonpointer-1.10-11.el8.noarc 95 kB/s | 20 kB 00:00 2023-11-27T15:06:31.967 INFO:teuthology.orchestra.run.smithi150.stdout:(130/131): librbd1-18.0.0-3906.g1c956514.el8.x8 8.4 MB/s | 3.6 MB 00:00 2023-11-27T15:06:32.005 INFO:teuthology.orchestra.run.smithi102.stdout:(86/131): thrift-0.13.0-2.el8.x86_64.rpm 1.0 MB/s | 1.7 MB 00:01 2023-11-27T15:06:32.031 INFO:teuthology.orchestra.run.smithi102.stdout:(87/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.2 MB/s | 238 kB 00:00 2023-11-27T15:06:32.037 INFO:teuthology.orchestra.run.smithi052.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 4.1 MB/s | 709 kB 00:00 2023-11-27T15:06:32.112 INFO:teuthology.orchestra.run.smithi052.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 1.3 MB/s | 198 kB 00:00 2023-11-27T15:06:32.148 INFO:teuthology.orchestra.run.smithi102.stdout:(88/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 11 MB/s | 2.3 MB 00:00 2023-11-27T15:06:32.190 INFO:teuthology.orchestra.run.smithi102.stdout:(89/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 394 kB/s | 72 kB 00:00 2023-11-27T15:06:32.196 INFO:teuthology.orchestra.run.smithi052.stdout:(96/131): openblas-threads-0.3.15-3.el8.x86_64. 18 MB/s | 4.9 MB 00:00 2023-11-27T15:06:32.223 INFO:teuthology.orchestra.run.smithi102.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 797 kB/s | 60 kB 00:00 2023-11-27T15:06:32.229 INFO:teuthology.orchestra.run.smithi052.stdout:(97/131): jq-1.6-3.el8.x86_64.rpm 1.7 MB/s | 202 kB 00:00 2023-11-27T15:06:32.265 INFO:teuthology.orchestra.run.smithi102.stdout:(91/131): fuse-2.9.7-15.el8.x86_64.rpm 362 kB/s | 83 kB 00:00 2023-11-27T15:06:32.290 INFO:teuthology.orchestra.run.smithi102.stdout:(92/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 977 kB/s | 98 kB 00:00 2023-11-27T15:06:32.315 INFO:teuthology.orchestra.run.smithi102.stdout:(93/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 625 kB/s | 58 kB 00:00 2023-11-27T15:06:32.357 INFO:teuthology.orchestra.run.smithi102.stdout:(94/131): libcom_err-devel-1.45.6-4.el8.x86_64. 420 kB/s | 39 kB 00:00 2023-11-27T15:06:32.371 INFO:teuthology.orchestra.run.smithi052.stdout:(98/131): openblas-0.3.15-3.el8.x86_64.rpm 14 MB/s | 4.7 MB 00:00 2023-11-27T15:06:32.399 INFO:teuthology.orchestra.run.smithi102.stdout:(95/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 5.8 MB/s | 644 kB 00:00 2023-11-27T15:06:32.405 INFO:teuthology.orchestra.run.smithi052.stdout:(99/131): librdkafka-0.11.4-3.el8.x86_64.rpm 1.7 MB/s | 354 kB 00:00 2023-11-27T15:06:32.425 INFO:teuthology.orchestra.run.smithi102.stdout:(96/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 1.5 MB/s | 171 kB 00:00 2023-11-27T15:06:32.433 INFO:teuthology.orchestra.run.smithi150.stdout:(131/131): librados2-18.0.0-3906.g1c956514.el8. 4.0 MB/s | 3.7 MB 00:00 2023-11-27T15:06:32.438 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2023-11-27T15:06:32.438 INFO:teuthology.orchestra.run.smithi150.stdout:Total 9.7 MB/s | 197 MB 00:20 2023-11-27T15:06:32.508 INFO:teuthology.orchestra.run.smithi102.stdout:(97/131): NetworkManager-initscripts-updown-1.3 915 kB/s | 138 kB 00:00 2023-11-27T15:06:32.533 INFO:teuthology.orchestra.run.smithi102.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.0 MB/s | 110 kB 00:00 2023-11-27T15:06:32.639 INFO:teuthology.orchestra.run.smithi052.stdout:(100/131): postgresql-server-10.21-2.module+el8 12 MB/s | 5.1 MB 00:00 2023-11-27T15:06:32.659 INFO:teuthology.orchestra.run.smithi102.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 360 kB/s | 54 kB 00:00 2023-11-27T15:06:32.843 INFO:teuthology.orchestra.run.smithi102.stdout:(100/131): python3-scipy-1.0.0-21.module+el8.5. 31 MB/s | 14 MB 00:00 2023-11-27T15:06:32.868 INFO:teuthology.orchestra.run.smithi102.stdout:(101/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 776 kB/s | 259 kB 00:00 2023-11-27T15:06:32.893 INFO:teuthology.orchestra.run.smithi102.stdout:(102/131): socat-1.7.4.1-1.el8.x86_64.rpm 1.3 MB/s | 323 kB 00:00 2023-11-27T15:06:32.914 INFO:teuthology.orchestra.run.smithi052.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 3.0 MB/s | 1.5 MB 00:00 2023-11-27T15:06:32.939 INFO:teuthology.orchestra.run.smithi052.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 156 kB/s | 47 kB 00:00 2023-11-27T15:06:32.973 INFO:teuthology.orchestra.run.smithi052.stdout:(103/131): postgresql-contrib-10.21-2.module+el 1.3 MB/s | 808 kB 00:00 2023-11-27T15:06:32.977 INFO:teuthology.orchestra.run.smithi102.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 6.3 MB/s | 538 kB 00:00 2023-11-27T15:06:33.002 INFO:teuthology.orchestra.run.smithi102.stdout:(104/131): python3-markupsafe-0.23-19.el8.x86_6 244 kB/s | 39 kB 00:00 2023-11-27T15:06:33.080 INFO:teuthology.orchestra.run.smithi052.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 304 kB/s | 43 kB 00:00 2023-11-27T15:06:33.115 INFO:teuthology.orchestra.run.smithi052.stdout:(105/131): libselinux-devel-2.9-5.el8.x86_64.rp 995 kB/s | 200 kB 00:00 2023-11-27T15:06:33.144 INFO:teuthology.orchestra.run.smithi102.stdout:(105/131): bison-3.0.4-10.el8.x86_64.rpm 2.4 MB/s | 688 kB 00:00 2023-11-27T15:06:33.149 INFO:teuthology.orchestra.run.smithi052.stdout:(106/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.2 MB/s | 220 kB 00:00 2023-11-27T15:06:33.249 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2023-11-27T15:06:33.266 INFO:teuthology.orchestra.run.smithi052.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 580 kB/s | 87 kB 00:00 2023-11-27T15:06:33.291 INFO:teuthology.orchestra.run.smithi052.stdout:(108/131): keyutils-libs-devel-1.5.10-9.el8.x86 228 kB/s | 48 kB 00:00 2023-11-27T15:06:33.325 INFO:teuthology.orchestra.run.smithi052.stdout:(109/131): python3-requests-oauthlib-1.0.0-1.el 243 kB/s | 43 kB 00:00 2023-11-27T15:06:33.350 INFO:teuthology.orchestra.run.smithi052.stdout:(110/131): python3-pycparser-2.14-14.el8.noarch 1.3 MB/s | 109 kB 00:00 2023-11-27T15:06:33.370 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2023-11-27T15:06:33.370 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2023-11-27T15:06:33.403 INFO:teuthology.orchestra.run.smithi102.stdout:(106/131): python3-mako-1.0.6-13.el8.noarch.rpm 369 kB/s | 157 kB 00:00 2023-11-27T15:06:33.428 INFO:teuthology.orchestra.run.smithi102.stdout:(107/131): oniguruma-6.8.2-2.el8.x86_64.rpm 440 kB/s | 187 kB 00:00 2023-11-27T15:06:33.434 INFO:teuthology.orchestra.run.smithi052.stdout:(111/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.6 MB/s | 229 kB 00:00 2023-11-27T15:06:33.459 INFO:teuthology.orchestra.run.smithi052.stdout:(112/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.1 MB/s | 560 kB 00:00 2023-11-27T15:06:33.462 INFO:teuthology.orchestra.run.smithi102.stdout:(108/131): automake-1.16.1-7.el8.noarch.rpm 2.2 MB/s | 713 kB 00:00 2023-11-27T15:06:33.476 INFO:teuthology.orchestra.run.smithi052.stdout:(113/131): libaio-devel-0.3.112-1.el8.x86_64.rp 149 kB/s | 19 kB 00:00 2023-11-27T15:06:33.529 INFO:teuthology.orchestra.run.smithi102.stdout:(109/131): python3-jsonpatch-1.21-2.el8.noarch. 265 kB/s | 27 kB 00:00 2023-11-27T15:06:33.593 INFO:teuthology.orchestra.run.smithi052.stdout:(114/131): python3-oauthlib-2.1.0-1.el8.noarch. 976 kB/s | 155 kB 00:00 2023-11-27T15:06:33.658 INFO:teuthology.orchestra.run.smithi052.stdout:(115/131): python3-cryptography-3.2.1-5.el8.x86 2.8 MB/s | 559 kB 00:00 2023-11-27T15:06:33.662 INFO:teuthology.orchestra.run.smithi102.stdout:(110/131): python3-babel-2.5.1-7.el8.noarch.rpm 18 MB/s | 4.8 MB 00:00 2023-11-27T15:06:33.687 INFO:teuthology.orchestra.run.smithi102.stdout:(111/131): python3-prettytable-0.7.2-14.el8.noa 277 kB/s | 44 kB 00:00 2023-11-27T15:06:33.710 INFO:teuthology.orchestra.run.smithi052.stdout:(116/131): libverto-devel-0.3.0-5.el8.x86_64.rp 76 kB/s | 18 kB 00:00 2023-11-27T15:06:33.713 INFO:teuthology.orchestra.run.smithi102.stdout:(112/131): flex-2.6.1-9.el8.x86_64.rpm 1.2 MB/s | 320 kB 00:00 2023-11-27T15:06:33.744 INFO:teuthology.orchestra.run.smithi052.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 3.9 MB/s | 605 kB 00:00 2023-11-27T15:06:33.786 INFO:teuthology.orchestra.run.smithi052.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.8 MB/s | 238 kB 00:00 2023-11-27T15:06:33.813 INFO:teuthology.orchestra.run.smithi102.stdout:(113/131): python3-werkzeug-0.12.2-4.el8.noarch 3.6 MB/s | 457 kB 00:00 2023-11-27T15:06:33.836 INFO:teuthology.orchestra.run.smithi052.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 786 kB/s | 72 kB 00:00 2023-11-27T15:06:33.838 INFO:teuthology.orchestra.run.smithi102.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 1.8 MB/s | 225 kB 00:00 2023-11-27T15:06:33.878 INFO:teuthology.orchestra.run.smithi052.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 899 kB/s | 83 kB 00:00 2023-11-27T15:06:33.922 INFO:teuthology.orchestra.run.smithi102.stdout:(115/131): python3-numpy-1.14.3-10.el8.x86_64.r 14 MB/s | 3.7 MB 00:00 2023-11-27T15:06:33.961 INFO:teuthology.orchestra.run.smithi052.stdout:(121/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 9.3 MB/s | 2.3 MB 00:00 2023-11-27T15:06:33.970 INFO:teuthology.orchestra.run.smithi102.stdout:(116/131): uuid-1.6.2-43.el8.x86_64.rpm 409 kB/s | 64 kB 00:00 2023-11-27T15:06:33.987 INFO:teuthology.orchestra.run.smithi052.stdout:(122/131): elfutils-libelf-devel-0.186-1.el8.x8 398 kB/s | 60 kB 00:00 2023-11-27T15:06:34.012 INFO:teuthology.orchestra.run.smithi052.stdout:(123/131): libuuid-devel-2.32.1-35.el8.x86_64.r 734 kB/s | 98 kB 00:00 2023-11-27T15:06:34.014 INFO:teuthology.orchestra.run.smithi102.stdout:(117/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 720 kB/s | 126 kB 00:00 2023-11-27T15:06:34.062 INFO:teuthology.orchestra.run.smithi052.stdout:(124/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 574 kB/s | 58 kB 00:00 2023-11-27T15:06:34.073 INFO:teuthology.orchestra.run.smithi102.stdout:(118/131): python3-jsonpointer-1.10-11.el8.noar 133 kB/s | 20 kB 00:00 2023-11-27T15:06:34.087 INFO:teuthology.orchestra.run.smithi052.stdout:(125/131): libcom_err-devel-1.45.6-4.el8.x86_64 384 kB/s | 39 kB 00:00 2023-11-27T15:06:34.123 INFO:teuthology.orchestra.run.smithi102.stdout:(119/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 676 kB/s | 103 kB 00:00 2023-11-27T15:06:34.157 INFO:teuthology.orchestra.run.smithi102.stdout:(120/131): libtool-2.4.6-25.el8.x86_64.rpm 4.9 MB/s | 709 kB 00:00 2023-11-27T15:06:34.179 INFO:teuthology.orchestra.run.smithi052.stdout:(126/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 3.8 MB/s | 644 kB 00:00 2023-11-27T15:06:34.236 INFO:teuthology.orchestra.run.smithi052.stdout:(127/131): NetworkManager-initscripts-updown-1. 930 kB/s | 138 kB 00:00 2023-11-27T15:06:34.265 INFO:teuthology.orchestra.run.smithi102.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 25 MB/s | 4.9 MB 00:00 2023-11-27T15:06:34.271 INFO:teuthology.orchestra.run.smithi052.stdout:(128/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 819 kB/s | 171 kB 00:00 2023-11-27T15:06:34.290 INFO:teuthology.orchestra.run.smithi102.stdout:(122/131): libpq-13.5-1.el8.x86_64.rpm 1.2 MB/s | 198 kB 00:00 2023-11-27T15:06:34.391 INFO:teuthology.orchestra.run.smithi102.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 20 MB/s | 4.7 MB 00:00 2023-11-27T15:06:34.413 INFO:teuthology.orchestra.run.smithi052.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 5.5 MB/s | 798 kB 00:00 2023-11-27T15:06:34.424 INFO:teuthology.orchestra.run.smithi102.stdout:(124/131): jq-1.6-3.el8.x86_64.rpm 1.2 MB/s | 202 kB 00:00 2023-11-27T15:06:34.450 INFO:teuthology.orchestra.run.smithi102.stdout:(125/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.2 MB/s | 354 kB 00:00 2023-11-27T15:06:34.452 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2023-11-27T15:06:34.456 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2023-11-27T15:06:34.550 INFO:teuthology.orchestra.run.smithi102.stdout:(126/131): postgresql-contrib-10.21-2.module+el 6.3 MB/s | 808 kB 00:00 2023-11-27T15:06:34.608 INFO:teuthology.orchestra.run.smithi102.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 9.5 MB/s | 1.5 MB 00:00 2023-11-27T15:06:34.647 INFO:teuthology.orchestra.run.smithi052.stdout:(130/131): librbd1-18.0.0-3906.g1c956514.el8.x8 8.7 MB/s | 3.6 MB 00:00 2023-11-27T15:06:34.705 INFO:teuthology.orchestra.run.smithi052.stdout:(131/131): librados2-18.0.0-3906.g1c956514.el8. 7.0 MB/s | 3.7 MB 00:00 2023-11-27T15:06:34.709 INFO:teuthology.orchestra.run.smithi052.stdout:-------------------------------------------------------------------------------- 2023-11-27T15:06:34.709 INFO:teuthology.orchestra.run.smithi052.stdout:Total 9.4 MB/s | 197 MB 00:20 2023-11-27T15:06:34.734 INFO:teuthology.orchestra.run.smithi102.stdout:(128/131): postgresql-server-10.21-2.module+el8 15 MB/s | 5.1 MB 00:00 2023-11-27T15:06:34.867 INFO:teuthology.orchestra.run.smithi102.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 5.9 MB/s | 798 kB 00:00 2023-11-27T15:06:35.009 INFO:teuthology.orchestra.run.smithi102.stdout:(130/131): librados2-18.0.0-3906.g1c956514.el8. 8.0 MB/s | 3.7 MB 00:00 2023-11-27T15:06:35.568 INFO:teuthology.orchestra.run.smithi102.stdout:(131/131): librbd1-18.0.0-3906.g1c956514.el8.x8 3.7 MB/s | 3.6 MB 00:00 2023-11-27T15:06:35.570 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2023-11-27T15:06:35.571 INFO:teuthology.orchestra.run.smithi102.stdout:Total 8.4 MB/s | 197 MB 00:23 2023-11-27T15:06:35.581 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction check 2023-11-27T15:06:35.714 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction check succeeded. 2023-11-27T15:06:35.714 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction test 2023-11-27T15:06:36.433 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2023-11-27T15:06:36.553 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2023-11-27T15:06:36.553 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2023-11-27T15:06:36.862 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction test succeeded. 2023-11-27T15:06:36.866 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction 2023-11-27T15:06:37.625 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2023-11-27T15:06:37.629 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2023-11-27T15:06:39.159 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2023-11-27T15:06:39.338 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-27T15:06:39.517 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-27T15:06:39.722 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-11-27T15:06:39.882 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-27T15:06:39.998 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-27T15:06:40.241 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-27T15:06:40.639 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-27T15:06:40.857 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librados2-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-27T15:06:41.251 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-27T15:06:41.457 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-27T15:06:41.584 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-27T15:06:41.748 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-27T15:06:41.847 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-27T15:06:41.855 INFO:teuthology.orchestra.run.smithi052.stdout: Preparing : 1/1 2023-11-27T15:06:42.007 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 9/134 2023-11-27T15:06:42.027 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-27T15:06:42.191 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-27T15:06:42.244 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-27T15:06:42.422 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-11-27T15:06:42.451 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-27T15:06:42.579 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-27T15:06:42.692 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-27T15:06:42.846 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-27T15:06:42.930 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2023-11-27T15:06:42.963 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-27T15:06:43.066 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-27T15:06:43.138 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-27T15:06:43.207 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 14/134 2023-11-27T15:06:43.311 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 14/134 2023-11-27T15:06:43.330 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-27T15:06:43.444 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-27T15:06:43.456 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 15/134 2023-11-27T15:06:43.524 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 15/134 2023-11-27T15:06:43.532 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-27T15:06:43.645 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-27T15:06:43.665 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-27T15:06:43.696 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : librados2-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-27T15:06:43.782 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-27T15:06:43.915 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-27T15:06:43.956 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-27T15:06:44.096 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librados2-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-27T15:06:44.287 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-27T15:06:44.305 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-27T15:06:44.419 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-27T15:06:44.524 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librados2-2:18.0.0-3906.g1c956514.el8.x86_64 5/134 2023-11-27T15:06:44.577 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-27T15:06:44.629 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-27T15:06:44.730 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-27T15:06:44.748 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-27T15:06:44.901 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:18.0.0-3906.g1c956514.el8.x86_64 5/134 2023-11-27T15:06:44.922 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 9/134 2023-11-27T15:06:45.075 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-27T15:06:45.128 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-27T15:06:45.202 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-27T15:06:45.343 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-27T15:06:45.358 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-27T15:06:45.390 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-27T15:06:45.489 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-27T15:06:45.531 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-11-27T15:06:45.678 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-27T15:06:45.749 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-27T15:06:45.854 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-11-27T15:06:45.939 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-27T15:06:46.079 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 14/134 2023-11-27T15:06:46.135 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-27T15:06:46.194 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 14/134 2023-11-27T15:06:46.238 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-11-27T15:06:46.362 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 15/134 2023-11-27T15:06:46.393 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-27T15:06:46.453 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 15/134 2023-11-27T15:06:46.504 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-11-27T15:06:46.587 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-27T15:06:46.707 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-11-27T15:06:46.737 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-27T15:06:46.786 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-27T15:06:46.920 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-27T15:06:46.935 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-11-27T15:06:47.054 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-27T15:06:47.128 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-11-27T15:06:47.190 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 14/134 2023-11-27T15:06:47.288 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 14/134 2023-11-27T15:06:47.369 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-11-27T15:06:47.448 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 15/134 2023-11-27T15:06:47.560 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-27T15:06:47.588 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 15/134 2023-11-27T15:06:47.607 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-27T15:06:47.745 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-27T15:06:47.771 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-common-2:18.0.0-3906.g1c956514.el8. 26/134 2023-11-27T15:06:47.835 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-11-27T15:06:47.909 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-argparse-2:18.0.0-3906.g1c956514.el 27/134 2023-11-27T15:06:48.002 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-11-27T15:06:48.196 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-11-27T15:06:48.323 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-27T15:06:48.384 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-27T15:06:48.466 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-27T15:06:48.561 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-11-27T15:06:48.648 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-27T15:06:48.830 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_6 28/134 2023-11-27T15:06:49.145 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-11-27T15:06:49.262 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-27T15:06:49.349 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-11-27T15:06:49.382 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-11-27T15:06:49.431 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-27T15:06:49.585 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-11-27T15:06:49.650 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-11-27T15:06:49.810 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-11-27T15:06:49.856 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-27T15:06:50.028 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-27T15:06:50.042 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-11-27T15:06:50.198 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-11-27T15:06:50.244 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-11-27T15:06:50.482 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-11-27T15:06:50.718 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-27T15:06:50.886 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-ceph-common-2:18.0.0-3906.g1c956514.el8. 26/134 2023-11-27T15:06:50.912 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-11-27T15:06:51.033 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-ceph-argparse-2:18.0.0-3906.g1c956514.el 27/134 2023-11-27T15:06:51.204 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-11-27T15:06:51.379 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-11-27T15:06:51.599 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-27T15:06:51.611 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-27T15:06:51.796 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-common-2:18.0.0-3906.g1c956514.el8. 26/134 2023-11-27T15:06:51.926 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-argparse-2:18.0.0-3906.g1c956514.el 27/134 2023-11-27T15:06:51.940 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-11-27T15:06:52.002 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_6 28/134 2023-11-27T15:06:52.150 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-27T15:06:52.195 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_6 28/134 2023-11-27T15:06:52.343 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-11-27T15:06:52.357 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-27T15:06:52.389 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-27T15:06:52.536 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 36/134 2023-11-27T15:06:52.619 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-11-27T15:06:52.709 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librados-devel-2:18.0.0-3906.g1c956514.el8.x86_6 37/134 2023-11-27T15:06:52.854 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-11-27T15:06:52.926 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-11-27T15:06:53.006 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-11-27T15:06:53.119 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-27T15:06:53.224 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-11-27T15:06:53.383 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-11-27T15:06:53.593 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-11-27T15:06:53.807 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-11-27T15:06:54.149 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-27T15:06:54.170 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-11-27T15:06:54.653 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-11-27T15:06:54.973 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-11-27T15:06:55.002 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-27T15:06:55.176 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-11-27T15:06:55.195 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-11-27T15:06:55.297 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-11-27T15:06:55.393 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-11-27T15:06:55.484 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-27T15:06:55.537 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-11-27T15:06:55.697 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-11-27T15:06:55.699 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-27T15:06:55.833 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-11-27T15:06:55.892 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 36/134 2023-11-27T15:06:55.921 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-11-27T15:06:56.065 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-11-27T15:06:56.065 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librados-devel-2:18.0.0-3906.g1c956514.el8.x86_6 37/134 2023-11-27T15:06:56.114 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-11-27T15:06:56.244 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-11-27T15:06:56.309 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-11-27T15:06:56.317 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-27T15:06:56.413 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-11-27T15:06:56.483 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-11-27T15:06:56.524 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-27T15:06:56.580 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-11-27T15:06:56.666 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-11-27T15:06:56.711 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 36/134 2023-11-27T15:06:56.749 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-11-27T15:06:56.903 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librados-devel-2:18.0.0-3906.g1c956514.el8.x86_6 37/134 2023-11-27T15:06:56.955 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-11-27T15:06:56.979 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-11-27T15:06:57.046 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-11-27T15:06:57.156 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-11-27T15:06:57.180 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-11-27T15:06:57.241 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-11-27T15:06:57.380 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-11-27T15:06:57.449 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-11-27T15:06:57.493 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-11-27T15:06:57.543 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-11-27T15:06:57.694 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-11-27T15:06:57.728 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-11-27T15:06:57.874 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-11-27T15:06:57.895 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-11-27T15:06:57.900 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-11-27T15:06:58.045 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-11-27T15:06:58.080 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-27T15:06:58.089 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-11-27T15:06:58.159 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-27T15:06:58.221 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-11-27T15:06:58.295 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-11-27T15:06:58.301 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-11-27T15:06:58.394 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-11-27T15:06:58.552 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-11-27T15:06:58.567 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-11-27T15:06:58.586 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-11-27T15:06:58.771 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-11-27T15:06:58.808 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-11-27T15:06:58.864 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-11-27T15:06:58.941 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-11-27T15:06:58.971 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-27T15:06:59.038 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-27T15:06:59.086 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-11-27T15:06:59.158 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-11-27T15:06:59.250 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-11-27T15:06:59.259 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-11-27T15:06:59.300 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-11-27T15:06:59.422 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-11-27T15:06:59.452 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-11-27T15:06:59.545 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-11-27T15:06:59.596 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-27T15:06:59.633 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-11-27T15:06:59.736 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-11-27T15:06:59.765 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-27T15:06:59.807 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-11-27T15:07:00.005 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-27T15:07:00.120 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-11-27T15:07:00.154 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-11-27T15:07:00.245 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-27T15:07:00.375 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-27T15:07:00.399 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-11-27T15:07:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-27T15:07:00.515 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-11-27T15:07:00.653 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-11-27T15:07:00.695 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-27T15:07:00.732 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-11-27T15:07:00.884 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-11-27T15:07:00.889 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-27T15:07:00.983 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-11-27T15:07:01.043 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-11-27T15:07:01.088 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-27T15:07:01.187 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-11-27T15:07:01.251 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-11-27T15:07:01.265 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-27T15:07:01.410 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-11-27T15:07:01.418 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-27T15:07:01.484 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-11-27T15:07:01.563 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-11-27T15:07:01.634 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-27T15:07:01.747 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-11-27T15:07:01.754 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-11-27T15:07:01.848 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-27T15:07:01.958 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-11-27T15:07:01.978 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-11-27T15:07:02.059 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-27T15:07:02.119 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-27T15:07:02.204 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-27T15:07:02.218 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-11-27T15:07:02.286 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-27T15:07:02.416 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-11-27T15:07:02.437 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-11-27T15:07:02.494 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-27T15:07:02.582 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-11-27T15:07:02.630 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-11-27T15:07:02.659 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-27T15:07:02.770 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-27T15:07:02.866 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-27T15:07:02.947 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-11-27T15:07:02.951 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-27T15:07:03.225 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-11-27T15:07:03.233 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-27T15:07:03.444 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-27T15:07:03.476 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-11-27T15:07:03.574 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-27T15:07:03.697 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-11-27T15:07:03.740 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-27T15:07:03.896 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-27T15:07:03.933 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-27T15:07:03.958 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-27T15:07:04.065 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-27T15:07:04.199 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-27T15:07:04.204 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-27T15:07:04.354 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-27T15:07:04.382 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-27T15:07:04.454 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-27T15:07:04.555 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-27T15:07:04.619 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-27T15:07:04.700 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-27T15:07:04.764 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-27T15:07:04.783 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-27T15:07:04.971 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-27T15:07:04.981 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-27T15:07:05.037 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-27T15:07:05.188 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-27T15:07:05.199 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-27T15:07:05.303 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-27T15:07:05.400 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-27T15:07:05.416 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-27T15:07:05.623 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-27T15:07:05.648 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-27T15:07:05.807 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-27T15:07:05.852 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-27T15:07:05.985 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-27T15:07:06.036 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-27T15:07:06.098 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-27T15:07:06.197 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-27T15:07:06.256 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-27T15:07:06.262 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 92/134 2023-11-27T15:07:06.330 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 92/134 2023-11-27T15:07:06.458 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-27T15:07:06.496 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 93/134 2023-11-27T15:07:06.643 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-27T15:07:06.915 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-27T15:07:07.170 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-27T15:07:07.353 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-27T15:07:07.458 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-27T15:07:07.603 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-27T15:07:07.750 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-27T15:07:07.995 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-27T15:07:08.305 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-27T15:07:08.570 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-27T15:07:08.734 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-27T15:07:08.776 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-27T15:07:09.032 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-27T15:07:09.279 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-27T15:07:09.568 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-27T15:07:09.604 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-27T15:07:09.734 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 92/134 2023-11-27T15:07:09.813 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 92/134 2023-11-27T15:07:09.918 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-27T15:07:09.985 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 93/134 2023-11-27T15:07:10.100 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-27T15:07:10.178 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-27T15:07:10.612 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-27T15:07:10.970 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-27T15:07:11.162 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 92/134 2023-11-27T15:07:11.214 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-27T15:07:11.242 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 92/134 2023-11-27T15:07:11.412 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 93/134 2023-11-27T15:07:11.477 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 95/134 2023-11-27T15:07:11.558 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 95/134 2023-11-27T15:07:11.711 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 96/134 2023-11-27T15:07:13.837 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-27T15:07:14.076 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-27T15:07:14.701 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-27T15:07:15.153 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-27T15:07:15.213 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 95/134 2023-11-27T15:07:15.284 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 95/134 2023-11-27T15:07:15.425 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 96/134 2023-11-27T15:07:15.695 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-27T15:07:16.286 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-27T15:07:16.562 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 95/134 2023-11-27T15:07:16.685 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 95/134 2023-11-27T15:07:16.828 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 96/134 2023-11-27T15:07:30.621 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 96/134 2023-11-27T15:07:30.621 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /sys 2023-11-27T15:07:30.621 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /proc 2023-11-27T15:07:30.621 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /dev 2023-11-27T15:07:30.621 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /run 2023-11-27T15:07:30.621 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /mnt 2023-11-27T15:07:30.622 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /var/tmp 2023-11-27T15:07:30.622 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /home 2023-11-27T15:07:30.622 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /tmp 2023-11-27T15:07:30.622 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /dev 2023-11-27T15:07:30.622 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-27T15:07:30.976 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 97/134 2023-11-27T15:07:32.544 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 97/134 2023-11-27T15:07:32.724 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 98/134 2023-11-27T15:07:32.943 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 98/134 2023-11-27T15:07:33.125 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 99/134 2023-11-27T15:07:33.578 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 99/134 2023-11-27T15:07:33.767 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 100/134 2023-11-27T15:07:33.784 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 100/134 2023-11-27T15:07:35.090 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 96/134 2023-11-27T15:07:35.090 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /sys 2023-11-27T15:07:35.090 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /proc 2023-11-27T15:07:35.090 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /dev 2023-11-27T15:07:35.090 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /run 2023-11-27T15:07:35.090 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /mnt 2023-11-27T15:07:35.091 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /var/tmp 2023-11-27T15:07:35.091 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /home 2023-11-27T15:07:35.091 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /tmp 2023-11-27T15:07:35.091 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /dev 2023-11-27T15:07:35.091 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:07:35.482 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 97/134 2023-11-27T15:07:35.487 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephadm-2:18.0.0-3906.g1c956514.el8.noarch 101/134 2023-11-27T15:07:35.537 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 96/134 2023-11-27T15:07:35.537 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /sys 2023-11-27T15:07:35.537 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /proc 2023-11-27T15:07:35.537 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /dev 2023-11-27T15:07:35.537 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /run 2023-11-27T15:07:35.538 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /mnt 2023-11-27T15:07:35.538 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /var/tmp 2023-11-27T15:07:35.538 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /home 2023-11-27T15:07:35.538 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /tmp 2023-11-27T15:07:35.538 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /dev 2023-11-27T15:07:35.538 INFO:teuthology.orchestra.run.smithi102.stdout: 2023-11-27T15:07:35.694 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephadm-2:18.0.0-3906.g1c956514.el8.noarch 101/134 2023-11-27T15:07:35.839 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.e 102/134 2023-11-27T15:07:35.892 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 97/134 2023-11-27T15:07:36.056 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3906.g1c956514. 103/134 2023-11-27T15:07:36.249 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 104/134 2023-11-27T15:07:36.464 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 104/134 2023-11-27T15:07:36.782 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 105/134 2023-11-27T15:07:37.146 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 97/134 2023-11-27T15:07:37.310 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 98/134 2023-11-27T15:07:37.455 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 97/134 2023-11-27T15:07:37.541 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 98/134 2023-11-27T15:07:37.651 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 98/134 2023-11-27T15:07:37.723 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 99/134 2023-11-27T15:07:37.747 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 105/134 2023-11-27T15:07:37.868 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 98/134 2023-11-27T15:07:37.908 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 106/134 2023-11-27T15:07:37.999 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 106/134 2023-11-27T15:07:38.042 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 99/134 2023-11-27T15:07:38.205 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 99/134 2023-11-27T15:07:38.345 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el 107/134 2023-11-27T15:07:38.373 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 100/134 2023-11-27T15:07:38.392 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 100/134 2023-11-27T15:07:38.492 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 99/134 2023-11-27T15:07:38.552 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 108/134 2023-11-27T15:07:38.587 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 108/134 2023-11-27T15:07:38.725 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 100/134 2023-11-27T15:07:38.733 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 109/134 2023-11-27T15:07:38.742 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 100/134 2023-11-27T15:07:38.744 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 109/134 2023-11-27T15:07:39.994 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-2:18.0.0-3906.g1c956514.el8.x86_64 110/134 2023-11-27T15:07:40.199 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: cephadm-2:18.0.0-3906.g1c956514.el8.noarch 101/134 2023-11-27T15:07:40.203 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 111/134 2023-11-27T15:07:40.237 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 111/134 2023-11-27T15:07:40.383 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3906.g1c956 112/134 2023-11-27T15:07:40.424 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3906.g1c956 112/134 2023-11-27T15:07:40.429 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephadm-2:18.0.0-3906.g1c956514.el8.noarch 101/134 2023-11-27T15:07:40.530 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: cephadm-2:18.0.0-3906.g1c956514.el8.noarch 101/134 2023-11-27T15:07:40.566 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 113/134 2023-11-27T15:07:40.679 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.e 102/134 2023-11-27T15:07:40.720 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephadm-2:18.0.0-3906.g1c956514.el8.noarch 101/134 2023-11-27T15:07:40.873 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.e 102/134 2023-11-27T15:07:40.911 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 113/134 2023-11-27T15:07:40.913 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3906.g1c956514. 103/134 2023-11-27T15:07:41.048 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3906.g1c956514. 103/134 2023-11-27T15:07:41.074 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 114/134 2023-11-27T15:07:41.121 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 104/134 2023-11-27T15:07:41.258 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 104/134 2023-11-27T15:07:41.347 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 104/134 2023-11-27T15:07:41.473 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 104/134 2023-11-27T15:07:41.693 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 105/134 2023-11-27T15:07:41.759 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 105/134 2023-11-27T15:07:42.715 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 105/134 2023-11-27T15:07:42.716 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 105/134 2023-11-27T15:07:42.858 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 106/134 2023-11-27T15:07:42.872 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 106/134 2023-11-27T15:07:42.948 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 106/134 2023-11-27T15:07:42.967 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 106/134 2023-11-27T15:07:43.312 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el 107/134 2023-11-27T15:07:43.321 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el 107/134 2023-11-27T15:07:43.540 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 108/134 2023-11-27T15:07:43.541 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 108/134 2023-11-27T15:07:43.575 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 108/134 2023-11-27T15:07:43.579 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 108/134 2023-11-27T15:07:43.741 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 109/134 2023-11-27T15:07:43.746 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 109/134 2023-11-27T15:07:43.753 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 109/134 2023-11-27T15:07:43.759 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 109/134 2023-11-27T15:07:44.978 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-2:18.0.0-3906.g1c956514.el8.x86_64 110/134 2023-11-27T15:07:45.011 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 114/134 2023-11-27T15:07:45.117 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-2:18.0.0-3906.g1c956514.el8.x86_64 110/134 2023-11-27T15:07:45.195 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 111/134 2023-11-27T15:07:45.229 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 111/134 2023-11-27T15:07:45.287 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 115/134 2023-11-27T15:07:45.338 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 111/134 2023-11-27T15:07:45.374 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 111/134 2023-11-27T15:07:45.391 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3906.g1c956 112/134 2023-11-27T15:07:45.433 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3906.g1c956 112/134 2023-11-27T15:07:45.448 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-11-27T15:07:45.530 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3906.g1c956 112/134 2023-11-27T15:07:45.574 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3906.g1c956 112/134 2023-11-27T15:07:45.616 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 113/134 2023-11-27T15:07:45.638 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-11-27T15:07:45.746 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 113/134 2023-11-27T15:07:45.961 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 113/134 2023-11-27T15:07:46.091 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 114/134 2023-11-27T15:07:46.113 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 113/134 2023-11-27T15:07:46.271 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 114/134 2023-11-27T15:07:46.511 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-11-27T15:07:46.813 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-11-27T15:07:46.985 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 119/134 2023-11-27T15:07:47.203 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-27T15:07:47.369 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_ 121/134 2023-11-27T15:07:47.516 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-27T15:07:49.729 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-27T15:07:49.981 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-27T15:07:49.995 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 114/134 2023-11-27T15:07:49.999 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-27T15:07:50.148 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 124/134 2023-11-27T15:07:50.422 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 125/134 2023-11-27T15:07:50.434 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 114/134 2023-11-27T15:07:50.718 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 126/134 2023-11-27T15:07:50.737 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 115/134 2023-11-27T15:07:50.886 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 115/134 2023-11-27T15:07:50.945 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-11-27T15:07:50.983 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-27T15:07:51.148 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-27T15:07:51.154 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-11-27T15:07:51.156 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-27T15:07:51.196 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-11-27T15:07:51.352 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-11-27T15:07:51.418 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 117/134 2023-11-27T15:07:51.539 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-11-27T15:07:51.620 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-11-27T15:07:51.712 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-11-27T15:07:51.839 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-11-27T15:07:51.852 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-11-27T15:07:51.852 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-27T15:07:51.873 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-11-27T15:07:52.051 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-27T15:07:52.051 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-27T15:07:52.062 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-11-27T15:07:52.100 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-11-27T15:07:52.302 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-11-27T15:07:52.304 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-27T15:07:52.407 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-11-27T15:07:52.522 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-27T15:07:52.536 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_ 121/134 2023-11-27T15:07:52.600 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 119/134 2023-11-27T15:07:52.691 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-27T15:07:52.766 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-27T15:07:52.942 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_ 121/134 2023-11-27T15:07:53.045 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-27T15:07:53.045 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-2:18.0.0-3906.g1c956514.el8.x86_64 1/134 2023-11-27T15:07:53.045 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 2/134 2023-11-27T15:07:53.045 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 3/134 2023-11-27T15:07:53.045 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 4/134 2023-11-27T15:07:53.045 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3906.g1c956 5/134 2023-11-27T15:07:53.045 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-27T15:07:53.045 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-27T15:07:53.046 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-27T15:07:53.046 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 9/134 2023-11-27T15:07:53.046 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 10/134 2023-11-27T15:07:53.046 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 11/134 2023-11-27T15:07:53.046 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 12/134 2023-11-27T15:07:53.046 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 13/134 2023-11-27T15:07:53.046 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_ 14/134 2023-11-27T15:07:53.046 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 15/134 2023-11-27T15:07:53.046 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 16/134 2023-11-27T15:07:53.046 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados-devel-2:18.0.0-3906.g1c956514.el8.x86_6 17/134 2023-11-27T15:07:53.046 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 18/134 2023-11-27T15:07:53.047 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 19/134 2023-11-27T15:07:53.047 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3906.g1c956514.el 20/134 2023-11-27T15:07:53.047 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-common-2:18.0.0-3906.g1c956514.el8. 21/134 2023-11-27T15:07:53.047 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_6 22/134 2023-11-27T15:07:53.047 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 23/134 2023-11-27T15:07:53.047 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 24/134 2023-11-27T15:07:53.047 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 25/134 2023-11-27T15:07:53.047 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 26/134 2023-11-27T15:07:53.047 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 27/134 2023-11-27T15:07:53.047 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 28/134 2023-11-27T15:07:53.048 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3906.g1c956514. 29/134 2023-11-27T15:07:53.048 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 30/134 2023-11-27T15:07:53.048 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 31/134 2023-11-27T15:07:53.048 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 32/134 2023-11-27T15:07:53.048 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el 33/134 2023-11-27T15:07:53.048 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 34/134 2023-11-27T15:07:53.048 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.e 35/134 2023-11-27T15:07:53.048 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 36/134 2023-11-27T15:07:53.048 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephadm-2:18.0.0-3906.g1c956514.el8.noarch 37/134 2023-11-27T15:07:53.048 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 38/134 2023-11-27T15:07:53.048 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-27T15:07:53.048 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-27T15:07:53.049 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-27T15:07:53.049 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-27T15:07:53.049 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-27T15:07:53.049 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-27T15:07:53.049 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-27T15:07:53.049 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-27T15:07:53.049 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-27T15:07:53.049 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-27T15:07:53.049 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-27T15:07:53.049 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-27T15:07:53.049 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-27T15:07:53.050 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-27T15:07:53.050 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-27T15:07:53.050 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-27T15:07:53.050 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-27T15:07:53.050 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-27T15:07:53.050 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-27T15:07:53.050 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-27T15:07:53.050 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-27T15:07:53.050 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-27T15:07:53.051 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-27T15:07:53.051 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-27T15:07:53.051 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-27T15:07:53.051 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-27T15:07:53.051 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-27T15:07:53.051 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-27T15:07:53.051 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-27T15:07:53.051 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-27T15:07:53.051 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-27T15:07:53.051 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-27T15:07:53.051 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-27T15:07:53.052 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-11-27T15:07:53.052 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-11-27T15:07:53.052 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-11-27T15:07:53.052 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-11-27T15:07:53.052 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-11-27T15:07:53.052 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-11-27T15:07:53.052 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-11-27T15:07:53.052 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-11-27T15:07:53.052 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-11-27T15:07:53.052 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-11-27T15:07:53.052 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-11-27T15:07:53.054 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-11-27T15:07:53.054 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-11-27T15:07:53.054 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-11-27T15:07:53.054 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-11-27T15:07:53.054 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-11-27T15:07:53.054 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-11-27T15:07:53.054 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-11-27T15:07:53.055 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-11-27T15:07:53.055 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-11-27T15:07:53.055 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-11-27T15:07:53.055 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-11-27T15:07:53.055 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-11-27T15:07:53.055 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-11-27T15:07:53.055 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-11-27T15:07:53.055 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-11-27T15:07:53.055 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-11-27T15:07:53.055 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-11-27T15:07:53.055 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-11-27T15:07:53.056 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-11-27T15:07:53.056 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-11-27T15:07:53.056 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-11-27T15:07:53.056 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-11-27T15:07:53.056 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-11-27T15:07:53.056 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-11-27T15:07:53.056 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-11-27T15:07:53.056 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-11-27T15:07:53.056 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-11-27T15:07:53.056 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-11-27T15:07:53.056 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-11-27T15:07:53.057 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-11-27T15:07:53.057 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-11-27T15:07:53.057 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-11-27T15:07:53.057 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-11-27T15:07:53.057 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-11-27T15:07:53.057 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-11-27T15:07:53.057 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-11-27T15:07:53.057 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-11-27T15:07:53.057 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-27T15:07:53.057 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-11-27T15:07:53.057 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-27T15:07:53.057 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-11-27T15:07:53.057 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-11-27T15:07:53.058 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-11-27T15:07:53.058 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-11-27T15:07:53.058 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-11-27T15:07:53.058 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-27T15:07:53.058 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:18.0.0-3906.g1c956514.el8.x86_64 129/134 2023-11-27T15:07:53.058 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-27T15:07:53.058 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 131/134 2023-11-27T15:07:53.058 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-27T15:07:53.058 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-27T15:07:53.113 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-27T15:07:55.021 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-27T15:07:55.323 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-27T15:07:55.343 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-27T15:07:55.466 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-27T15:07:55.466 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:55 ago on Mon 27 Nov 2023 03:06:00 PM UTC. 2023-11-27T15:07:55.502 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-27T15:07:55.572 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 124/134 2023-11-27T15:07:55.729 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-27T15:07:55.748 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-27T15:07:55.822 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 125/134 2023-11-27T15:07:55.901 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 124/134 2023-11-27T15:07:56.143 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 126/134 2023-11-27T15:07:56.203 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 125/134 2023-11-27T15:07:56.517 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-27T15:07:56.534 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 126/134 2023-11-27T15:07:56.764 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-11-27T15:07:56.846 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-11-27T15:07:56.855 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-27T15:07:56.986 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-27T15:07:56.996 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-27T15:07:57.022 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-11-27T15:07:57.035 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-11-27T15:07:57.206 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-27T15:07:57.214 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-27T15:07:57.216 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-11-27T15:07:57.341 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-11-27T15:07:57.341 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-27T15:07:57.394 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-11-27T15:07:57.486 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-11-27T15:07:57.576 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-27T15:07:57.576 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-27T15:07:57.711 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-11-27T15:07:57.727 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-11-27T15:07:57.727 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-27T15:07:57.795 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-27T15:07:57.924 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-27T15:07:57.924 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-27T15:07:58.013 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-27T15:07:58.152 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-27T15:07:58.378 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-27T15:07:58.527 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-27T15:07:58.527 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-2:18.0.0-3906.g1c956514.el8.x86_64 1/134 2023-11-27T15:07:58.527 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 2/134 2023-11-27T15:07:58.527 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 3/134 2023-11-27T15:07:58.528 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 4/134 2023-11-27T15:07:58.528 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3906.g1c956 5/134 2023-11-27T15:07:58.528 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-27T15:07:58.528 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-27T15:07:58.528 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-27T15:07:58.528 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 9/134 2023-11-27T15:07:58.528 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 10/134 2023-11-27T15:07:58.528 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 11/134 2023-11-27T15:07:58.528 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 12/134 2023-11-27T15:07:58.528 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 13/134 2023-11-27T15:07:58.528 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_ 14/134 2023-11-27T15:07:58.529 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 15/134 2023-11-27T15:07:58.529 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 16/134 2023-11-27T15:07:58.529 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados-devel-2:18.0.0-3906.g1c956514.el8.x86_6 17/134 2023-11-27T15:07:58.529 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 18/134 2023-11-27T15:07:58.529 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 19/134 2023-11-27T15:07:58.529 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3906.g1c956514.el 20/134 2023-11-27T15:07:58.529 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-common-2:18.0.0-3906.g1c956514.el8. 21/134 2023-11-27T15:07:58.529 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_6 22/134 2023-11-27T15:07:58.529 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 23/134 2023-11-27T15:07:58.529 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 24/134 2023-11-27T15:07:58.529 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 25/134 2023-11-27T15:07:58.529 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 26/134 2023-11-27T15:07:58.530 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 27/134 2023-11-27T15:07:58.530 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 28/134 2023-11-27T15:07:58.538 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3906.g1c956514. 29/134 2023-11-27T15:07:58.538 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 30/134 2023-11-27T15:07:58.538 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 31/134 2023-11-27T15:07:58.538 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 32/134 2023-11-27T15:07:58.539 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el 33/134 2023-11-27T15:07:58.539 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 34/134 2023-11-27T15:07:58.539 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.e 35/134 2023-11-27T15:07:58.539 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 36/134 2023-11-27T15:07:58.539 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephadm-2:18.0.0-3906.g1c956514.el8.noarch 37/134 2023-11-27T15:07:58.539 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 38/134 2023-11-27T15:07:58.539 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-27T15:07:58.539 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-27T15:07:58.539 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-27T15:07:58.539 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-27T15:07:58.539 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-27T15:07:58.539 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-27T15:07:58.540 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-27T15:07:58.540 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-27T15:07:58.540 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-27T15:07:58.540 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-27T15:07:58.540 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-27T15:07:58.540 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-27T15:07:58.540 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-27T15:07:58.540 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-27T15:07:58.540 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-27T15:07:58.540 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-27T15:07:58.540 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-27T15:07:58.540 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-27T15:07:58.540 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-27T15:07:58.541 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-27T15:07:58.541 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-27T15:07:58.541 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-27T15:07:58.541 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-27T15:07:58.541 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-27T15:07:58.541 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-27T15:07:58.541 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-27T15:07:58.541 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-27T15:07:58.541 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-27T15:07:58.541 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-27T15:07:58.541 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-27T15:07:58.541 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-27T15:07:58.541 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-27T15:07:58.542 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-27T15:07:58.542 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-11-27T15:07:58.542 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-11-27T15:07:58.542 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-11-27T15:07:58.542 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-11-27T15:07:58.542 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-11-27T15:07:58.542 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-11-27T15:07:58.542 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-11-27T15:07:58.542 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-11-27T15:07:58.542 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-11-27T15:07:58.542 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-11-27T15:07:58.542 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-11-27T15:07:58.542 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-11-27T15:07:58.543 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-11-27T15:07:58.543 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-11-27T15:07:58.543 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-11-27T15:07:58.543 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-11-27T15:07:58.543 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-11-27T15:07:58.543 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-11-27T15:07:58.543 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-11-27T15:07:58.543 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-11-27T15:07:58.543 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-11-27T15:07:58.543 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-11-27T15:07:58.543 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-11-27T15:07:58.543 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-11-27T15:07:58.543 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-11-27T15:07:58.544 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-11-27T15:07:58.544 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-11-27T15:07:58.544 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-11-27T15:07:58.544 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-11-27T15:07:58.544 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-11-27T15:07:58.544 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-11-27T15:07:58.544 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-11-27T15:07:58.544 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-11-27T15:07:58.544 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-11-27T15:07:58.544 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-11-27T15:07:58.544 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-11-27T15:07:58.544 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-11-27T15:07:58.544 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-11-27T15:07:58.545 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-11-27T15:07:58.545 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-11-27T15:07:58.545 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-11-27T15:07:58.545 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-11-27T15:07:58.545 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-11-27T15:07:58.545 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-11-27T15:07:58.545 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-11-27T15:07:58.545 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-11-27T15:07:58.545 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-11-27T15:07:58.545 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-11-27T15:07:58.545 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-11-27T15:07:58.545 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-11-27T15:07:58.545 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-11-27T15:07:58.546 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-11-27T15:07:58.546 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-11-27T15:07:58.546 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-11-27T15:07:58.546 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-11-27T15:07:58.546 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-11-27T15:07:58.546 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-11-27T15:07:58.546 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:18.0.0-3906.g1c956514.el8.x86_64 129/134 2023-11-27T15:07:58.546 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-27T15:07:58.546 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 131/134 2023-11-27T15:07:58.546 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-27T15:07:58.546 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-27T15:07:58.917 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-27T15:07:58.917 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-2:18.0.0-3906.g1c956514.el8.x86_64 1/134 2023-11-27T15:07:58.917 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 2/134 2023-11-27T15:07:58.917 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 3/134 2023-11-27T15:07:58.918 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 4/134 2023-11-27T15:07:58.918 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3906.g1c956 5/134 2023-11-27T15:07:58.918 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-27T15:07:58.918 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-27T15:07:58.918 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-27T15:07:58.918 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 9/134 2023-11-27T15:07:58.918 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 10/134 2023-11-27T15:07:58.918 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 11/134 2023-11-27T15:07:58.918 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 12/134 2023-11-27T15:07:58.918 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 13/134 2023-11-27T15:07:58.918 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_ 14/134 2023-11-27T15:07:58.918 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 15/134 2023-11-27T15:07:58.918 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 16/134 2023-11-27T15:07:58.918 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librados-devel-2:18.0.0-3906.g1c956514.el8.x86_6 17/134 2023-11-27T15:07:58.919 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 18/134 2023-11-27T15:07:58.919 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 19/134 2023-11-27T15:07:58.919 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3906.g1c956514.el 20/134 2023-11-27T15:07:58.919 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-ceph-common-2:18.0.0-3906.g1c956514.el8. 21/134 2023-11-27T15:07:58.919 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_6 22/134 2023-11-27T15:07:58.919 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 23/134 2023-11-27T15:07:58.919 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 24/134 2023-11-27T15:07:58.919 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 25/134 2023-11-27T15:07:58.919 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 26/134 2023-11-27T15:07:58.919 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 27/134 2023-11-27T15:07:58.919 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 28/134 2023-11-27T15:07:58.919 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3906.g1c956514. 29/134 2023-11-27T15:07:58.928 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 30/134 2023-11-27T15:07:58.928 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 31/134 2023-11-27T15:07:58.928 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 32/134 2023-11-27T15:07:58.928 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el 33/134 2023-11-27T15:07:58.928 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 34/134 2023-11-27T15:07:58.928 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.e 35/134 2023-11-27T15:07:58.928 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 36/134 2023-11-27T15:07:58.928 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephadm-2:18.0.0-3906.g1c956514.el8.noarch 37/134 2023-11-27T15:07:58.928 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 38/134 2023-11-27T15:07:58.928 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-27T15:07:58.928 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-27T15:07:58.928 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-27T15:07:58.929 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-27T15:07:58.929 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-27T15:07:58.929 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-27T15:07:58.929 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-27T15:07:58.929 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-27T15:07:58.929 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-27T15:07:58.929 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-27T15:07:58.929 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-27T15:07:58.929 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-27T15:07:58.929 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-27T15:07:58.929 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-27T15:07:58.929 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-27T15:07:58.929 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-27T15:07:58.930 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-27T15:07:58.930 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-27T15:07:58.930 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-27T15:07:58.930 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-27T15:07:58.930 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-27T15:07:58.930 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-27T15:07:58.930 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-27T15:07:58.930 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-27T15:07:58.930 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-27T15:07:58.930 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-27T15:07:58.930 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-27T15:07:58.930 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-27T15:07:58.931 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-27T15:07:58.931 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-27T15:07:58.931 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-27T15:07:58.931 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-27T15:07:58.931 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-27T15:07:58.931 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-11-27T15:07:58.931 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-11-27T15:07:58.931 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-11-27T15:07:58.931 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-11-27T15:07:58.931 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-11-27T15:07:58.931 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-11-27T15:07:58.931 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-11-27T15:07:58.932 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-11-27T15:07:58.932 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-11-27T15:07:58.932 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-11-27T15:07:58.932 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-11-27T15:07:58.932 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-11-27T15:07:58.932 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-11-27T15:07:58.932 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-11-27T15:07:58.932 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-11-27T15:07:58.932 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-11-27T15:07:58.932 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-11-27T15:07:58.932 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-11-27T15:07:58.932 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-11-27T15:07:58.933 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-11-27T15:07:58.933 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-11-27T15:07:58.933 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-11-27T15:07:58.933 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-11-27T15:07:58.933 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-11-27T15:07:58.933 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-11-27T15:07:58.933 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-11-27T15:07:58.933 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-11-27T15:07:58.933 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-11-27T15:07:58.933 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-11-27T15:07:58.933 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-11-27T15:07:58.933 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-11-27T15:07:58.933 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-11-27T15:07:58.934 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-11-27T15:07:58.934 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-11-27T15:07:58.934 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-11-27T15:07:58.934 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-11-27T15:07:58.934 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-11-27T15:07:58.934 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-11-27T15:07:58.934 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-11-27T15:07:58.934 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-11-27T15:07:58.934 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-11-27T15:07:58.934 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-11-27T15:07:58.934 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-11-27T15:07:58.934 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-11-27T15:07:58.935 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-11-27T15:07:58.935 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-11-27T15:07:58.935 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-11-27T15:07:58.935 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-11-27T15:07:58.935 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-27T15:07:58.935 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-11-27T15:07:58.935 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-27T15:07:58.935 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-11-27T15:07:58.935 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-11-27T15:07:58.935 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-11-27T15:07:58.935 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-11-27T15:07:58.935 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-11-27T15:07:58.935 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-27T15:07:58.936 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librados2-2:18.0.0-3906.g1c956514.el8.x86_64 129/134 2023-11-27T15:07:58.936 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-27T15:07:58.936 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 131/134 2023-11-27T15:07:58.936 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-27T15:07:58.936 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-27T15:08:01.034 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-27T15:08:01.034 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:02:01 ago on Mon 27 Nov 2023 03:06:00 PM UTC. 2023-11-27T15:08:01.514 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-27T15:08:01.514 INFO:teuthology.orchestra.run.smithi052.stdout:Last metadata expiration check: 0:02:01 ago on Mon 27 Nov 2023 03:06:00 PM UTC. 2023-11-27T15:08:02.201 INFO:teuthology.orchestra.run.smithi150.stdout:Installed products updated. 2023-11-27T15:08:02.418 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-27T15:08:02.418 INFO:teuthology.orchestra.run.smithi150.stdout:Upgraded: 2023-11-27T15:08:02.418 INFO:teuthology.orchestra.run.smithi150.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-27T15:08:02.419 INFO:teuthology.orchestra.run.smithi150.stdout: librados2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:02.419 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:02.419 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2023-11-27T15:08:02.419 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-27T15:08:02.419 INFO:teuthology.orchestra.run.smithi150.stdout: automake-1.16.1-7.el8.noarch 2023-11-27T15:08:02.419 INFO:teuthology.orchestra.run.smithi150.stdout: bison-3.0.4-10.el8.x86_64 2023-11-27T15:08:02.419 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:02.419 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:02.419 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:02.419 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:02.419 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:08:02.419 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:02.420 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:02.420 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:02.420 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:08:02.420 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:08:02.420 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:08:02.420 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:08:02.420 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:08:02.420 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:02.420 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:02.420 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:08:02.420 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:02.420 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:02.421 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:02.421 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:08:02.421 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:08:02.421 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:02.421 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:08:02.421 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-27T15:08:02.421 INFO:teuthology.orchestra.run.smithi150.stdout: flex-2.6.1-9.el8.x86_64 2023-11-27T15:08:02.421 INFO:teuthology.orchestra.run.smithi150.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-27T15:08:02.421 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-27T15:08:02.421 INFO:teuthology.orchestra.run.smithi150.stdout: jq-1.6-3.el8.x86_64 2023-11-27T15:08:02.421 INFO:teuthology.orchestra.run.smithi150.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-27T15:08:02.422 INFO:teuthology.orchestra.run.smithi150.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-27T15:08:02.422 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-27T15:08:02.422 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:02.422 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:02.422 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:02.422 INFO:teuthology.orchestra.run.smithi150.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-27T15:08:02.422 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-27T15:08:02.422 INFO:teuthology.orchestra.run.smithi150.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-27T15:08:02.422 INFO:teuthology.orchestra.run.smithi150.stdout: libpq-13.5-1.el8.x86_64 2023-11-27T15:08:02.422 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-27T15:08:02.422 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-27T15:08:02.423 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:02.423 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:02.423 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-27T15:08:02.423 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:02.423 INFO:teuthology.orchestra.run.smithi150.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-27T15:08:02.423 INFO:teuthology.orchestra.run.smithi150.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-27T15:08:02.423 INFO:teuthology.orchestra.run.smithi150.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-27T15:08:02.423 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-27T15:08:02.423 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-27T15:08:02.423 INFO:teuthology.orchestra.run.smithi150.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-27T15:08:02.423 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-27T15:08:02.423 INFO:teuthology.orchestra.run.smithi150.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-27T15:08:02.424 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-27T15:08:02.424 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-27T15:08:02.424 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-27T15:08:02.424 INFO:teuthology.orchestra.run.smithi150.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-27T15:08:02.424 INFO:teuthology.orchestra.run.smithi150.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-27T15:08:02.424 INFO:teuthology.orchestra.run.smithi150.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-27T15:08:02.424 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-27T15:08:02.424 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-27T15:08:02.424 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-27T15:08:02.424 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-27T15:08:02.424 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-27T15:08:02.425 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-27T15:08:02.425 INFO:teuthology.orchestra.run.smithi150.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-27T15:08:02.425 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-27T15:08:02.425 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:02.425 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:02.425 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:02.425 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-27T15:08:02.425 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-27T15:08:02.425 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-27T15:08:02.425 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-27T15:08:02.425 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-27T15:08:02.426 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-27T15:08:02.426 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-27T15:08:02.426 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-27T15:08:02.426 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-27T15:08:02.426 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-27T15:08:02.426 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-27T15:08:02.426 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-27T15:08:02.426 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-27T15:08:02.426 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-27T15:08:02.426 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-27T15:08:02.427 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-27T15:08:02.427 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-27T15:08:02.427 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-27T15:08:02.427 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-27T15:08:02.427 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-27T15:08:02.427 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-27T15:08:02.427 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-27T15:08:02.427 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend-2.6-1.el8.noarch 2023-11-27T15:08:02.427 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-27T15:08:02.427 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-27T15:08:02.427 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-27T15:08:02.427 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-27T15:08:02.428 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-27T15:08:02.428 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-27T15:08:02.428 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:02.428 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:02.428 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-27T15:08:02.428 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-27T15:08:02.428 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:02.428 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-27T15:08:02.428 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-27T15:08:02.428 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-27T15:08:02.429 INFO:teuthology.orchestra.run.smithi150.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-27T15:08:02.429 INFO:teuthology.orchestra.run.smithi150.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-27T15:08:02.429 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-27T15:08:02.429 INFO:teuthology.orchestra.run.smithi150.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-27T15:08:02.429 INFO:teuthology.orchestra.run.smithi150.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-27T15:08:02.430 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-27T15:08:02.430 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-27T15:08:02.430 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-27T15:08:02.430 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-27T15:08:02.430 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-27T15:08:02.430 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:02.430 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:02.430 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:02.430 INFO:teuthology.orchestra.run.smithi150.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-27T15:08:02.430 INFO:teuthology.orchestra.run.smithi150.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-27T15:08:02.431 INFO:teuthology.orchestra.run.smithi150.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-27T15:08:02.431 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-27T15:08:02.431 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-27T15:08:02.431 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-27T15:08:02.431 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-27T15:08:02.431 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2023-11-27T15:08:02.602 DEBUG:teuthology.parallel:result is None 2023-11-27T15:08:07.013 INFO:teuthology.orchestra.run.smithi052.stdout:Installed products updated. 2023-11-27T15:08:07.254 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:08:07.254 INFO:teuthology.orchestra.run.smithi052.stdout:Upgraded: 2023-11-27T15:08:07.254 INFO:teuthology.orchestra.run.smithi052.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-27T15:08:07.254 INFO:teuthology.orchestra.run.smithi052.stdout: librados2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.254 INFO:teuthology.orchestra.run.smithi052.stdout: librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.255 INFO:teuthology.orchestra.run.smithi052.stdout:Installed: 2023-11-27T15:08:07.255 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-27T15:08:07.255 INFO:teuthology.orchestra.run.smithi052.stdout: automake-1.16.1-7.el8.noarch 2023-11-27T15:08:07.255 INFO:teuthology.orchestra.run.smithi052.stdout: bison-3.0.4-10.el8.x86_64 2023-11-27T15:08:07.255 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.255 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.255 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.255 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.255 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-grafana-dashboards-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:08:07.256 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-immutable-object-cache-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.256 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.256 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.256 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:08:07.256 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:08:07.256 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:08:07.256 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:08:07.256 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:08:07.256 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.256 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.257 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:08:07.257 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.257 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.257 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.257 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:08:07.257 INFO:teuthology.orchestra.run.smithi052.stdout: cephadm-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:08:07.257 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.257 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:08:07.257 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-27T15:08:07.257 INFO:teuthology.orchestra.run.smithi052.stdout: flex-2.6.1-9.el8.x86_64 2023-11-27T15:08:07.257 INFO:teuthology.orchestra.run.smithi052.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-27T15:08:07.258 INFO:teuthology.orchestra.run.smithi052.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-27T15:08:07.258 INFO:teuthology.orchestra.run.smithi052.stdout: jq-1.6-3.el8.x86_64 2023-11-27T15:08:07.258 INFO:teuthology.orchestra.run.smithi052.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-27T15:08:07.258 INFO:teuthology.orchestra.run.smithi052.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-27T15:08:07.258 INFO:teuthology.orchestra.run.smithi052.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-27T15:08:07.258 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.258 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.258 INFO:teuthology.orchestra.run.smithi052.stdout: libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.258 INFO:teuthology.orchestra.run.smithi052.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-27T15:08:07.258 INFO:teuthology.orchestra.run.smithi052.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-27T15:08:07.259 INFO:teuthology.orchestra.run.smithi052.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-27T15:08:07.259 INFO:teuthology.orchestra.run.smithi052.stdout: libpq-13.5-1.el8.x86_64 2023-11-27T15:08:07.259 INFO:teuthology.orchestra.run.smithi052.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-27T15:08:07.259 INFO:teuthology.orchestra.run.smithi052.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-27T15:08:07.259 INFO:teuthology.orchestra.run.smithi052.stdout: librados-devel-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.259 INFO:teuthology.orchestra.run.smithi052.stdout: libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.259 INFO:teuthology.orchestra.run.smithi052.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-27T15:08:07.259 INFO:teuthology.orchestra.run.smithi052.stdout: librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.259 INFO:teuthology.orchestra.run.smithi052.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-27T15:08:07.260 INFO:teuthology.orchestra.run.smithi052.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-27T15:08:07.260 INFO:teuthology.orchestra.run.smithi052.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-27T15:08:07.260 INFO:teuthology.orchestra.run.smithi052.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-27T15:08:07.260 INFO:teuthology.orchestra.run.smithi052.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-27T15:08:07.260 INFO:teuthology.orchestra.run.smithi052.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-27T15:08:07.260 INFO:teuthology.orchestra.run.smithi052.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-27T15:08:07.260 INFO:teuthology.orchestra.run.smithi052.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-27T15:08:07.260 INFO:teuthology.orchestra.run.smithi052.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-27T15:08:07.260 INFO:teuthology.orchestra.run.smithi052.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-27T15:08:07.260 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-27T15:08:07.261 INFO:teuthology.orchestra.run.smithi052.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-27T15:08:07.261 INFO:teuthology.orchestra.run.smithi052.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-27T15:08:07.261 INFO:teuthology.orchestra.run.smithi052.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-27T15:08:07.261 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-27T15:08:07.261 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-27T15:08:07.261 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-27T15:08:07.261 INFO:teuthology.orchestra.run.smithi052.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-27T15:08:07.261 INFO:teuthology.orchestra.run.smithi052.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-27T15:08:07.261 INFO:teuthology.orchestra.run.smithi052.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-27T15:08:07.261 INFO:teuthology.orchestra.run.smithi052.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-27T15:08:07.261 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-27T15:08:07.262 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-argparse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.262 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.262 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.262 INFO:teuthology.orchestra.run.smithi052.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-27T15:08:07.262 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-27T15:08:07.262 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-27T15:08:07.262 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-27T15:08:07.262 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-27T15:08:07.262 INFO:teuthology.orchestra.run.smithi052.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-27T15:08:07.263 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-27T15:08:07.263 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-27T15:08:07.263 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-27T15:08:07.263 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-27T15:08:07.263 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-27T15:08:07.263 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-27T15:08:07.263 INFO:teuthology.orchestra.run.smithi052.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-27T15:08:07.263 INFO:teuthology.orchestra.run.smithi052.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-27T15:08:07.263 INFO:teuthology.orchestra.run.smithi052.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-27T15:08:07.263 INFO:teuthology.orchestra.run.smithi052.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-27T15:08:07.264 INFO:teuthology.orchestra.run.smithi052.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-27T15:08:07.264 INFO:teuthology.orchestra.run.smithi052.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-27T15:08:07.264 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-27T15:08:07.264 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-27T15:08:07.264 INFO:teuthology.orchestra.run.smithi052.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-27T15:08:07.264 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-27T15:08:07.264 INFO:teuthology.orchestra.run.smithi052.stdout: python3-portend-2.6-1.el8.noarch 2023-11-27T15:08:07.264 INFO:teuthology.orchestra.run.smithi052.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-27T15:08:07.264 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-27T15:08:07.265 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-27T15:08:07.265 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-27T15:08:07.265 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-27T15:08:07.265 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-27T15:08:07.265 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.265 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.265 INFO:teuthology.orchestra.run.smithi052.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-27T15:08:07.265 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-27T15:08:07.265 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.265 INFO:teuthology.orchestra.run.smithi052.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-27T15:08:07.265 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-27T15:08:07.266 INFO:teuthology.orchestra.run.smithi052.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-27T15:08:07.266 INFO:teuthology.orchestra.run.smithi052.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-27T15:08:07.266 INFO:teuthology.orchestra.run.smithi052.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-27T15:08:07.266 INFO:teuthology.orchestra.run.smithi052.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-27T15:08:07.266 INFO:teuthology.orchestra.run.smithi052.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-27T15:08:07.266 INFO:teuthology.orchestra.run.smithi052.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-27T15:08:07.266 INFO:teuthology.orchestra.run.smithi052.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-27T15:08:07.266 INFO:teuthology.orchestra.run.smithi052.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-27T15:08:07.266 INFO:teuthology.orchestra.run.smithi052.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-27T15:08:07.267 INFO:teuthology.orchestra.run.smithi052.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-27T15:08:07.267 INFO:teuthology.orchestra.run.smithi052.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-27T15:08:07.267 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.267 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.267 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.267 INFO:teuthology.orchestra.run.smithi052.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-27T15:08:07.267 INFO:teuthology.orchestra.run.smithi052.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-27T15:08:07.267 INFO:teuthology.orchestra.run.smithi052.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-27T15:08:07.267 INFO:teuthology.orchestra.run.smithi052.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-27T15:08:07.267 INFO:teuthology.orchestra.run.smithi052.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-27T15:08:07.268 INFO:teuthology.orchestra.run.smithi052.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-27T15:08:07.268 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:08:07.268 INFO:teuthology.orchestra.run.smithi052.stdout:Complete! 2023-11-27T15:08:07.437 DEBUG:teuthology.parallel:result is None 2023-11-27T15:08:07.566 INFO:teuthology.orchestra.run.smithi102.stdout:Installed products updated. 2023-11-27T15:08:07.779 INFO:teuthology.orchestra.run.smithi102.stdout: 2023-11-27T15:08:07.779 INFO:teuthology.orchestra.run.smithi102.stdout:Upgraded: 2023-11-27T15:08:07.779 INFO:teuthology.orchestra.run.smithi102.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-27T15:08:07.780 INFO:teuthology.orchestra.run.smithi102.stdout: librados2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.780 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.780 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2023-11-27T15:08:07.780 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-27T15:08:07.780 INFO:teuthology.orchestra.run.smithi102.stdout: automake-1.16.1-7.el8.noarch 2023-11-27T15:08:07.780 INFO:teuthology.orchestra.run.smithi102.stdout: bison-3.0.4-10.el8.x86_64 2023-11-27T15:08:07.780 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.780 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.780 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.780 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.780 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:08:07.781 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.781 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.781 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.781 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:08:07.781 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:08:07.781 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:08:07.781 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:08:07.781 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:08:07.781 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.781 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.781 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:08:07.781 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.782 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.782 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.782 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:08:07.782 INFO:teuthology.orchestra.run.smithi102.stdout: cephadm-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:08:07.782 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.782 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:08:07.782 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-27T15:08:07.782 INFO:teuthology.orchestra.run.smithi102.stdout: flex-2.6.1-9.el8.x86_64 2023-11-27T15:08:07.782 INFO:teuthology.orchestra.run.smithi102.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-27T15:08:07.782 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-27T15:08:07.782 INFO:teuthology.orchestra.run.smithi102.stdout: jq-1.6-3.el8.x86_64 2023-11-27T15:08:07.782 INFO:teuthology.orchestra.run.smithi102.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-27T15:08:07.783 INFO:teuthology.orchestra.run.smithi102.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-27T15:08:07.783 INFO:teuthology.orchestra.run.smithi102.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-27T15:08:07.783 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.783 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.783 INFO:teuthology.orchestra.run.smithi102.stdout: libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.783 INFO:teuthology.orchestra.run.smithi102.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-27T15:08:07.783 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-27T15:08:07.783 INFO:teuthology.orchestra.run.smithi102.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-27T15:08:07.783 INFO:teuthology.orchestra.run.smithi102.stdout: libpq-13.5-1.el8.x86_64 2023-11-27T15:08:07.783 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-27T15:08:07.783 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-27T15:08:07.783 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.784 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.784 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-27T15:08:07.784 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.784 INFO:teuthology.orchestra.run.smithi102.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-27T15:08:07.784 INFO:teuthology.orchestra.run.smithi102.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-27T15:08:07.784 INFO:teuthology.orchestra.run.smithi102.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-27T15:08:07.784 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-27T15:08:07.784 INFO:teuthology.orchestra.run.smithi102.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-27T15:08:07.784 INFO:teuthology.orchestra.run.smithi102.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-27T15:08:07.784 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-27T15:08:07.784 INFO:teuthology.orchestra.run.smithi102.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-27T15:08:07.784 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-27T15:08:07.784 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-27T15:08:07.785 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-27T15:08:07.785 INFO:teuthology.orchestra.run.smithi102.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-27T15:08:07.785 INFO:teuthology.orchestra.run.smithi102.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-27T15:08:07.785 INFO:teuthology.orchestra.run.smithi102.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-27T15:08:07.785 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-27T15:08:07.785 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-27T15:08:07.785 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-27T15:08:07.785 INFO:teuthology.orchestra.run.smithi102.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-27T15:08:07.785 INFO:teuthology.orchestra.run.smithi102.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-27T15:08:07.785 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-27T15:08:07.785 INFO:teuthology.orchestra.run.smithi102.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-27T15:08:07.785 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-27T15:08:07.786 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.786 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.786 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.786 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-27T15:08:07.786 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-27T15:08:07.786 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-27T15:08:07.786 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-27T15:08:07.786 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-27T15:08:07.786 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-27T15:08:07.786 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-27T15:08:07.786 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-27T15:08:07.787 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-27T15:08:07.787 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-27T15:08:07.787 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-27T15:08:07.787 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-27T15:08:07.787 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-27T15:08:07.787 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-27T15:08:07.787 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-27T15:08:07.787 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-27T15:08:07.787 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-27T15:08:07.787 INFO:teuthology.orchestra.run.smithi102.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-27T15:08:07.787 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-27T15:08:07.787 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-27T15:08:07.788 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-27T15:08:07.788 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-27T15:08:07.788 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend-2.6-1.el8.noarch 2023-11-27T15:08:07.788 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-27T15:08:07.788 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-27T15:08:07.788 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-27T15:08:07.788 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-27T15:08:07.788 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-27T15:08:07.788 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-27T15:08:07.788 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.788 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.789 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-27T15:08:07.789 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-27T15:08:07.789 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.789 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-27T15:08:07.789 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-27T15:08:07.789 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-27T15:08:07.789 INFO:teuthology.orchestra.run.smithi102.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-27T15:08:07.789 INFO:teuthology.orchestra.run.smithi102.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-27T15:08:07.789 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-27T15:08:07.789 INFO:teuthology.orchestra.run.smithi102.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-27T15:08:07.789 INFO:teuthology.orchestra.run.smithi102.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-27T15:08:07.790 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-27T15:08:07.790 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-27T15:08:07.790 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-27T15:08:07.790 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-27T15:08:07.790 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-27T15:08:07.790 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.790 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.790 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:08:07.790 INFO:teuthology.orchestra.run.smithi102.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-27T15:08:07.790 INFO:teuthology.orchestra.run.smithi102.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-27T15:08:07.790 INFO:teuthology.orchestra.run.smithi102.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-27T15:08:07.791 INFO:teuthology.orchestra.run.smithi102.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-27T15:08:07.791 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-27T15:08:07.791 INFO:teuthology.orchestra.run.smithi102.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-27T15:08:07.791 INFO:teuthology.orchestra.run.smithi102.stdout: 2023-11-27T15:08:07.791 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2023-11-27T15:08:07.997 DEBUG:teuthology.parallel:result is None 2023-11-27T15:08:07.998 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-27T15:08:08.228 DEBUG:teuthology.orchestra.run.smithi052:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-27T15:08:08.256 INFO:teuthology.orchestra.run.smithi052.stdout:18.0.0-3906.g1c956514.el8 2023-11-27T15:08:08.257 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3906.g1c956514.el8 2023-11-27T15:08:08.257 INFO:teuthology.task.install:The correct ceph version 18.0.0-3906.g1c956514 is installed. 2023-11-27T15:08:08.258 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-27T15:08:08.471 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-27T15:08:08.496 INFO:teuthology.orchestra.run.smithi102.stdout:18.0.0-3906.g1c956514.el8 2023-11-27T15:08:08.496 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3906.g1c956514.el8 2023-11-27T15:08:08.496 INFO:teuthology.task.install:The correct ceph version 18.0.0-3906.g1c956514 is installed. 2023-11-27T15:08:08.496 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-27T15:08:08.637 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-27T15:08:08.666 INFO:teuthology.orchestra.run.smithi150.stdout:18.0.0-3906.g1c956514.el8 2023-11-27T15:08:08.666 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3906.g1c956514.el8 2023-11-27T15:08:08.666 INFO:teuthology.task.install:The correct ceph version 18.0.0-3906.g1c956514 is installed. 2023-11-27T15:08:08.667 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-27T15:08:08.667 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2023-11-27T15:08:08.667 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-27T15:08:08.717 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2023-11-27T15:08:08.717 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-27T15:08:08.760 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-27T15:08:08.760 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-27T15:08:08.809 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-27T15:08:08.810 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2023-11-27T15:08:08.810 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/daemon-helper 2023-11-27T15:08:08.841 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-27T15:08:08.913 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2023-11-27T15:08:08.913 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/daemon-helper 2023-11-27T15:08:08.941 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-27T15:08:09.008 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-27T15:08:09.008 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/daemon-helper 2023-11-27T15:08:09.040 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-27T15:08:09.111 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-27T15:08:09.111 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2023-11-27T15:08:09.111 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-27T15:08:09.143 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-27T15:08:09.214 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2023-11-27T15:08:09.214 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-27T15:08:09.241 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-27T15:08:09.308 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-27T15:08:09.308 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-27T15:08:09.337 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-27T15:08:09.406 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-27T15:08:09.406 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2023-11-27T15:08:09.407 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/stdin-killer 2023-11-27T15:08:09.440 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-27T15:08:09.511 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2023-11-27T15:08:09.512 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/stdin-killer 2023-11-27T15:08:09.539 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-27T15:08:09.606 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-27T15:08:09.607 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/stdin-killer 2023-11-27T15:08:09.636 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-27T15:08:09.705 INFO:teuthology.run_tasks:Running task cephadm... 2023-11-27T15:08:09.779 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '1c9565146c8647d4e00eb6c7ae626dc192680e7b', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2023-11-27T15:08:09.779 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-27T15:08:09.780 INFO:tasks.cephadm:Cluster fsid is c6c7df88-8d36-11ee-95a2-87774f69a715 2023-11-27T15:08:09.780 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-11-27T15:08:09.780 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.52', 'mon.b': '172.21.15.102', 'mon.c': '172.21.15.150'} 2023-11-27T15:08:09.780 INFO:tasks.cephadm:First mon is mon.a on smithi052 2023-11-27T15:08:09.780 INFO:tasks.cephadm:First mgr is x 2023-11-27T15:08:09.780 INFO:tasks.cephadm:Normalizing hostnames... 2023-11-27T15:08:09.781 DEBUG:teuthology.orchestra.run.smithi052:> sudo hostname $(hostname -s) 2023-11-27T15:08:09.814 DEBUG:teuthology.orchestra.run.smithi102:> sudo hostname $(hostname -s) 2023-11-27T15:08:09.842 DEBUG:teuthology.orchestra.run.smithi150:> sudo hostname $(hostname -s) 2023-11-27T15:08:09.873 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-11-27T15:08:09.873 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-27T15:08:10.088 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1c9565146c8647d4e00eb6c7ae626dc192680e7b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/74215/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-3906-g1c956514', 'node_name': '172.21.2.6+braggi06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-3906.g1c956514'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-11-16 20:52:31.578811', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-choffman-62793', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-11-27T15:08:10.229 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-choffman-62793, sha1 1c9565146c8647d4e00eb6c7ae626dc192680e7b from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-27T15:08:10.229 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/x86_64/flavors/default/cephadm 2023-11-27T15:08:10.229 DEBUG:teuthology.orchestra.run.smithi052:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-11-27T15:08:10.772 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-rw-r--. 1 ubuntu ubuntu 384133 Nov 27 15:08 /home/ubuntu/cephtest/cephadm 2023-11-27T15:08:10.773 DEBUG:teuthology.orchestra.run.smithi102:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-11-27T15:08:11.488 INFO:teuthology.orchestra.run.smithi102.stdout:-rw-rw-r--. 1 ubuntu ubuntu 384133 Nov 27 15:08 /home/ubuntu/cephtest/cephadm 2023-11-27T15:08:11.488 DEBUG:teuthology.orchestra.run.smithi150:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-11-27T15:08:11.992 INFO:teuthology.orchestra.run.smithi150.stdout:-rw-rw-r--. 1 ubuntu ubuntu 384133 Nov 27 15:08 /home/ubuntu/cephtest/cephadm 2023-11-27T15:08:11.993 DEBUG:teuthology.orchestra.run.smithi052:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-11-27T15:08:12.015 DEBUG:teuthology.orchestra.run.smithi102:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-11-27T15:08:12.037 DEBUG:teuthology.orchestra.run.smithi150:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-11-27T15:08:12.072 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b on all hosts... 2023-11-27T15:08:12.072 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b pull 2023-11-27T15:08:12.074 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b pull 2023-11-27T15:08:12.077 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b pull 2023-11-27T15:08:16.641 INFO:teuthology.orchestra.run.smithi150.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b... 2023-11-27T15:08:16.764 INFO:teuthology.orchestra.run.smithi052.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b... 2023-11-27T15:08:16.993 INFO:teuthology.orchestra.run.smithi102.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b... 2023-11-27T15:08:41.603 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2023-11-27T15:08:41.603 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_version": "ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev)", 2023-11-27T15:08:41.603 INFO:teuthology.orchestra.run.smithi150.stdout: "image_id": "ace6904b066bfa00ae45df809dc67d592e0953ee68f7eed3c36f7d4e23e52db2", 2023-11-27T15:08:41.604 INFO:teuthology.orchestra.run.smithi150.stdout: "repo_digests": [ 2023-11-27T15:08:41.604 INFO:teuthology.orchestra.run.smithi150.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cb1d293acfc56ca3305212d5789a9da86fb3fb7a4af319d5f427084581ab00ca" 2023-11-27T15:08:41.604 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2023-11-27T15:08:41.604 INFO:teuthology.orchestra.run.smithi150.stdout:} 2023-11-27T15:08:48.315 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2023-11-27T15:08:48.316 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_version": "ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev)", 2023-11-27T15:08:48.316 INFO:teuthology.orchestra.run.smithi102.stdout: "image_id": "ace6904b066bfa00ae45df809dc67d592e0953ee68f7eed3c36f7d4e23e52db2", 2023-11-27T15:08:48.316 INFO:teuthology.orchestra.run.smithi102.stdout: "repo_digests": [ 2023-11-27T15:08:48.316 INFO:teuthology.orchestra.run.smithi102.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cb1d293acfc56ca3305212d5789a9da86fb3fb7a4af319d5f427084581ab00ca" 2023-11-27T15:08:48.316 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2023-11-27T15:08:48.316 INFO:teuthology.orchestra.run.smithi102.stdout:} 2023-11-27T15:08:53.528 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:08:53.528 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_version": "ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev)", 2023-11-27T15:08:53.528 INFO:teuthology.orchestra.run.smithi052.stdout: "image_id": "ace6904b066bfa00ae45df809dc67d592e0953ee68f7eed3c36f7d4e23e52db2", 2023-11-27T15:08:53.528 INFO:teuthology.orchestra.run.smithi052.stdout: "repo_digests": [ 2023-11-27T15:08:53.529 INFO:teuthology.orchestra.run.smithi052.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cb1d293acfc56ca3305212d5789a9da86fb3fb7a4af319d5f427084581ab00ca" 2023-11-27T15:08:53.529 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2023-11-27T15:08:53.529 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:08:53.548 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /etc/ceph 2023-11-27T15:08:53.579 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /etc/ceph 2023-11-27T15:08:53.615 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph 2023-11-27T15:08:53.653 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 777 /etc/ceph 2023-11-27T15:08:53.679 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 777 /etc/ceph 2023-11-27T15:08:53.710 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 777 /etc/ceph 2023-11-27T15:08:53.740 INFO:tasks.cephadm:Writing seed config... 2023-11-27T15:08:53.741 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-11-27T15:08:53.742 INFO:tasks.cephadm: override: [client] debug client = 20 2023-11-27T15:08:53.742 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-11-27T15:08:53.742 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-11-27T15:08:53.742 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-11-27T15:08:53.742 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-11-27T15:08:53.742 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-11-27T15:08:53.742 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-11-27T15:08:53.742 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-11-27T15:08:53.742 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-11-27T15:08:53.742 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-11-27T15:08:53.743 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-11-27T15:08:53.743 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-11-27T15:08:53.743 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-11-27T15:08:53.743 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-11-27T15:08:53.743 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-11-27T15:08:53.743 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-11-27T15:08:53.743 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2023-11-27T15:08:53.743 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-11-27T15:08:53.743 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-11-27T15:08:53.743 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-11-27T15:08:53.743 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-11-27T15:08:53.743 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2023-11-27T15:08:53.743 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-11-27T15:08:53.743 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-11-27T15:08:53.744 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-11-27T15:08:53.744 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-11-27T15:08:53.744 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-11-27T15:08:53.744 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-11-27T15:08:53.744 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-11-27T15:08:53.744 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2023-11-27T15:08:53.744 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-11-27T15:08:53.744 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-11-27T15:08:53.744 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-11-27T15:08:53.744 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-11-27T15:08:53.744 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-11-27T15:08:53.744 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-11-27T15:08:53.745 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-11-27T15:08:53.745 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-11-27T15:08:53.745 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-11-27T15:08:53.745 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-11-27T15:08:53.745 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-11-27T15:08:53.745 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-11-27T15:08:53.745 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-11-27T15:08:53.745 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2023-11-27T15:08:53.746 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2023-11-27T15:08:53.746 DEBUG:teuthology.orchestra.run.smithi052:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-11-27T15:08:53.763 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = c6c7df88-8d36-11ee-95a2-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-11-27T15:08:53.763 DEBUG:teuthology.orchestra.run.smithi052:mon.a> sudo journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mon.a.service 2023-11-27T15:08:53.804 DEBUG:teuthology.orchestra.run.smithi052:mgr.x> sudo journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mgr.x.service 2023-11-27T15:08:53.831 INFO:journalctl@ceph.mon.a.smithi052.stdout:-- Logs begin at Mon 2023-11-27 14:52:43 UTC. -- 2023-11-27T15:08:53.832 INFO:tasks.cephadm:Bootstrapping... 2023-11-27T15:08:53.832 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b -v bootstrap --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.52 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-11-27T15:08:53.858 INFO:journalctl@ceph.mgr.x.smithi052.stdout:-- Logs begin at Mon 2023-11-27 14:52:43 UTC. -- 2023-11-27T15:08:54.089 INFO:teuthology.orchestra.run.smithi052.stdout:-------------------------------------------------------------------------------- 2023-11-27T15:08:54.089 INFO:teuthology.orchestra.run.smithi052.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b', '-v', 'bootstrap', '--fsid', 'c6c7df88-8d36-11ee-95a2-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.52', '--skip-admin-label'] 2023-11-27T15:08:54.238 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stdout 4.1.1 2023-11-27T15:08:54.238 INFO:teuthology.orchestra.run.smithi052.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-11-27T15:08:54.239 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying podman|docker is present... 2023-11-27T15:08:54.386 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stdout 4.1.1 2023-11-27T15:08:54.386 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying lvm2 is present... 2023-11-27T15:08:54.386 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying time synchronization is in place... 2023-11-27T15:08:54.393 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-11-27T15:08:54.393 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-11-27T15:08:54.399 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-11-27T15:08:54.399 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout inactive 2023-11-27T15:08:54.405 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout enabled 2023-11-27T15:08:54.411 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout active 2023-11-27T15:08:54.411 INFO:teuthology.orchestra.run.smithi052.stdout:Unit chronyd.service is enabled and running 2023-11-27T15:08:54.412 INFO:teuthology.orchestra.run.smithi052.stdout:Repeating the final host check... 2023-11-27T15:08:54.562 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stdout 4.1.1 2023-11-27T15:08:54.562 INFO:teuthology.orchestra.run.smithi052.stdout:podman (/bin/podman) version 4.1.1 is present 2023-11-27T15:08:54.562 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl is present 2023-11-27T15:08:54.562 INFO:teuthology.orchestra.run.smithi052.stdout:lvcreate is present 2023-11-27T15:08:54.568 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-11-27T15:08:54.568 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-11-27T15:08:54.574 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-11-27T15:08:54.574 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout inactive 2023-11-27T15:08:54.580 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout enabled 2023-11-27T15:08:54.586 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout active 2023-11-27T15:08:54.586 INFO:teuthology.orchestra.run.smithi052.stdout:Unit chronyd.service is enabled and running 2023-11-27T15:08:54.586 INFO:teuthology.orchestra.run.smithi052.stdout:Host looks OK 2023-11-27T15:08:54.586 INFO:teuthology.orchestra.run.smithi052.stdout:Cluster fsid: c6c7df88-8d36-11ee-95a2-87774f69a715 2023-11-27T15:08:54.587 INFO:teuthology.orchestra.run.smithi052.stdout:Acquiring lock 140098383907304 on /run/cephadm/c6c7df88-8d36-11ee-95a2-87774f69a715.lock 2023-11-27T15:08:54.587 INFO:teuthology.orchestra.run.smithi052.stdout:Lock 140098383907304 acquired on /run/cephadm/c6c7df88-8d36-11ee-95a2-87774f69a715.lock 2023-11-27T15:08:54.588 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying IP 172.21.15.52 port 3300 ... 2023-11-27T15:08:54.588 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying IP 172.21.15.52 port 6789 ... 2023-11-27T15:08:54.588 INFO:teuthology.orchestra.run.smithi052.stdout:Base mon IP(s) is [172.21.15.52:3300, 172.21.15.52:6789], mon addrv is [v2:172.21.15.52:3300,v1:172.21.15.52:6789] 2023-11-27T15:08:54.591 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2023-11-27T15:08:54.591 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.52 metric 100 2023-11-27T15:08:54.594 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-11-27T15:08:54.594 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-11-27T15:08:54.594 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1717sec hoplimit 64 pref medium 2023-11-27T15:08:54.596 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-11-27T15:08:54.596 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-11-27T15:08:54.597 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-11-27T15:08:54.597 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2023-11-27T15:08:54.597 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15de/64 scope link 2023-11-27T15:08:54.597 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-11-27T15:08:54.597 INFO:teuthology.orchestra.run.smithi052.stdout:Mon IP `172.21.15.52` is in CIDR network `172.21.0.0/20` 2023-11-27T15:08:54.598 INFO:teuthology.orchestra.run.smithi052.stdout:Mon IP `172.21.15.52` is in CIDR network `172.21.0.0/20` 2023-11-27T15:08:54.598 INFO:teuthology.orchestra.run.smithi052.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-11-27T15:08:54.598 INFO:teuthology.orchestra.run.smithi052.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-11-27T15:08:54.599 INFO:teuthology.orchestra.run.smithi052.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b... 2023-11-27T15:08:55.042 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stdout ace6904b066bfa00ae45df809dc67d592e0953ee68f7eed3c36f7d4e23e52db2 2023-11-27T15:08:55.043 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b... 2023-11-27T15:08:55.043 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Getting image source signatures 2023-11-27T15:08:55.043 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Copying blob sha256:eb1dc24ed1b729f25bfe21673f4e5cc46a126932dc459841b1c31fbebaca605e 2023-11-27T15:08:55.043 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Copying blob sha256:837e66e3ff89804ed1bb42786f80e6c23534ff65fe48526d5007449f4e1f79b0 2023-11-27T15:08:55.043 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Copying config sha256:ace6904b066bfa00ae45df809dc67d592e0953ee68f7eed3c36f7d4e23e52db2 2023-11-27T15:08:55.044 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Writing manifest to image destination 2023-11-27T15:08:55.044 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Storing signatures 2023-11-27T15:08:55.894 INFO:teuthology.orchestra.run.smithi052.stdout:ceph: stdout ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev) 2023-11-27T15:08:55.895 INFO:teuthology.orchestra.run.smithi052.stdout:Ceph version: ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev) 2023-11-27T15:08:55.895 INFO:teuthology.orchestra.run.smithi052.stdout:Extracting ceph user uid/gid from container image... 2023-11-27T15:08:56.741 INFO:teuthology.orchestra.run.smithi052.stdout:stat: stdout 167 167 2023-11-27T15:08:56.741 INFO:teuthology.orchestra.run.smithi052.stdout:Creating initial keys... 2023-11-27T15:08:57.489 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph-authtool: stdout AQAJsWRlVDriCBAAbGU+gFhNYHLzDzKnOCbA5Q== 2023-11-27T15:08:58.236 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph-authtool: stdout AQAJsWRlEXsPNBAAAS0tQ1GyMwNYjhvr20+SxQ== 2023-11-27T15:08:59.087 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph-authtool: stdout AQAKsWRlD+8oJBAAy5NjIIGmvHCv6njQe/RBIA== 2023-11-27T15:08:59.088 INFO:teuthology.orchestra.run.smithi052.stdout:Creating initial monmap... 2023-11-27T15:08:59.844 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-11-27T15:08:59.844 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-11-27T15:08:59.845 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to c6c7df88-8d36-11ee-95a2-87774f69a715 2023-11-27T15:08:59.845 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-11-27T15:08:59.845 INFO:teuthology.orchestra.run.smithi052.stdout:monmaptool for a [v2:172.21.15.52:3300,v1:172.21.15.52:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-11-27T15:08:59.845 INFO:teuthology.orchestra.run.smithi052.stdout:setting min_mon_release = pacific 2023-11-27T15:08:59.845 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/monmaptool: set fsid to c6c7df88-8d36-11ee-95a2-87774f69a715 2023-11-27T15:08:59.845 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-11-27T15:08:59.845 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:08:59.845 INFO:teuthology.orchestra.run.smithi052.stdout:Creating mon... 2023-11-27T15:09:00.702 INFO:teuthology.orchestra.run.smithi052.stdout:create mon.a on 2023-11-27T15:09:00.802 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-11-27T15:09:00.890 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-11-27T15:09:00.985 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-c6c7df88-8d36-11ee-95a2-87774f69a715.target → /etc/systemd/system/ceph-c6c7df88-8d36-11ee-95a2-87774f69a715.target. 2023-11-27T15:09:00.985 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-c6c7df88-8d36-11ee-95a2-87774f69a715.target → /etc/systemd/system/ceph-c6c7df88-8d36-11ee-95a2-87774f69a715.target. 2023-11-27T15:09:01.111 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mon.a 2023-11-27T15:09:01.112 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mon.a.service: Unit ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mon.a.service not loaded. 2023-11-27T15:09:01.201 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c6c7df88-8d36-11ee-95a2-87774f69a715.target.wants/ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mon.a.service → /etc/systemd/system/ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@.service. 2023-11-27T15:09:01.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:01 smithi052 systemd[1]: Starting Ceph mon.a for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:09:02.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:02 smithi052 podman[98510]: 2023-11-27T15:09:03.418 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-11-27T15:09:03.418 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout disabled 2023-11-27T15:09:03.426 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-11-27T15:09:03.426 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout inactive 2023-11-27T15:09:03.426 INFO:teuthology.orchestra.run.smithi052.stdout:firewalld.service is not enabled 2023-11-27T15:09:03.426 INFO:teuthology.orchestra.run.smithi052.stdout:Not possible to enable service . firewalld.service is not available 2023-11-27T15:09:03.427 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for mon to start... 2023-11-27T15:09:03.427 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for mon... 2023-11-27T15:09:03.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:03 smithi052 bash[98510]: 7370e38860bc6676fcf816fcd3076f5906ae7e3d2191cd578103a400ca502063 2023-11-27T15:09:03.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:03 smithi052 ceph-mon[98550]: mkfs c6c7df88-8d36-11ee-95a2-87774f69a715 2023-11-27T15:09:03.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:03 smithi052 ceph-mon[98550]: mon.a is new leader, mons a in quorum (ranks 0) 2023-11-27T15:09:03.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:03 smithi052 systemd[1]: Started Ceph mon.a for c6c7df88-8d36-11ee-95a2-87774f69a715. 2023-11-27T15:09:04.462 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:04 smithi052 ceph-mon[98550]: mon.a is new leader, mons a in quorum (ranks 0) 2023-11-27T15:09:04.462 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:04 smithi052 ceph-mon[98550]: monmap e1: 1 mons at {a=[v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0]} removed_ranks: {} 2023-11-27T15:09:04.462 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:04 smithi052 ceph-mon[98550]: fsmap 2023-11-27T15:09:04.462 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:04 smithi052 ceph-mon[98550]: osdmap e1: 0 total, 0 up, 0 in 2023-11-27T15:09:04.462 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:04 smithi052 ceph-mon[98550]: mgrmap e1: no daemons active 2023-11-27T15:09:04.462 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:04 smithi052 ceph-mon[98550]: from='client.? 172.21.15.52:0/1636661653' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-11-27T15:09:04.481 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout cluster: 2023-11-27T15:09:04.481 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout id: c6c7df88-8d36-11ee-95a2-87774f69a715 2023-11-27T15:09:04.481 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-11-27T15:09:04.482 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2023-11-27T15:09:04.482 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout services: 2023-11-27T15:09:04.482 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.757706s) 2023-11-27T15:09:04.482 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-11-27T15:09:04.482 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-11-27T15:09:04.482 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2023-11-27T15:09:04.482 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout data: 2023-11-27T15:09:04.483 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-11-27T15:09:04.483 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-11-27T15:09:04.483 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-11-27T15:09:04.483 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout pgs: 2023-11-27T15:09:04.483 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2023-11-27T15:09:04.483 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.950+0000 7f1d6dc3b700 1 Processor -- start 2023-11-27T15:09:04.483 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.950+0000 7f1d6dc3b700 1 -- start start 2023-11-27T15:09:04.483 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.950+0000 7f1d6dc3b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d6807bdb0 0x7f1d6807a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:04.484 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.950+0000 7f1d6dc3b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6807a7f0 con 0x7f1d6807bdb0 2023-11-27T15:09:04.484 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.950+0000 7f1d677fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d6807bdb0 0x7f1d6807a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:04.484 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.951+0000 7f1d677fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d6807bdb0 0x7f1d6807a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48688/0 (socket says 172.21.15.52:48688) 2023-11-27T15:09:04.484 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.951+0000 7f1d677fe700 1 -- 172.21.15.52:0/3061843843 learned_addr learned my addr 172.21.15.52:0/3061843843 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:04.484 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.951+0000 7f1d677fe700 1 -- 172.21.15.52:0/3061843843 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d6807a930 con 0x7f1d6807bdb0 2023-11-27T15:09:04.484 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.951+0000 7f1d677fe700 1 --2- 172.21.15.52:0/3061843843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d6807bdb0 0x7f1d6807a2b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1d580098a0 tx=0x7f1d58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9385e5652919610f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:04.484 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.951+0000 7f1d667fc700 1 -- 172.21.15.52:0/3061843843 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d58004160 con 0x7f1d6807bdb0 2023-11-27T15:09:04.484 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.951+0000 7f1d667fc700 1 -- 172.21.15.52:0/3061843843 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d580042c0 con 0x7f1d6807bdb0 2023-11-27T15:09:04.485 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.951+0000 7f1d667fc700 1 -- 172.21.15.52:0/3061843843 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d580045a0 con 0x7f1d6807bdb0 2023-11-27T15:09:04.485 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.952+0000 7f1d6dc3b700 1 -- 172.21.15.52:0/3061843843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d6807bdb0 msgr2=0x7f1d6807a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:04.485 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.952+0000 7f1d6dc3b700 1 --2- 172.21.15.52:0/3061843843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d6807bdb0 0x7f1d6807a2b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1d580098a0 tx=0x7f1d58009bb0 comp rx=0 tx=0).stop 2023-11-27T15:09:04.485 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.952+0000 7f1d6dc3b700 1 -- 172.21.15.52:0/3061843843 shutdown_connections 2023-11-27T15:09:04.485 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.952+0000 7f1d6dc3b700 1 --2- 172.21.15.52:0/3061843843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d6807bdb0 0x7f1d6807a2b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:04.485 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.952+0000 7f1d6dc3b700 1 -- 172.21.15.52:0/3061843843 >> 172.21.15.52:0/3061843843 conn(0x7f1d68101170 msgr2=0x7f1d68103580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:04.485 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.952+0000 7f1d6dc3b700 1 -- 172.21.15.52:0/3061843843 shutdown_connections 2023-11-27T15:09:04.486 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.952+0000 7f1d6dc3b700 1 -- 172.21.15.52:0/3061843843 wait complete. 2023-11-27T15:09:04.486 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.952+0000 7f1d6dc3b700 1 Processor -- start 2023-11-27T15:09:04.486 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.953+0000 7f1d6dc3b700 1 -- start start 2023-11-27T15:09:04.486 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.953+0000 7f1d6dc3b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d6807bdb0 0x7f1d6819b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:04.486 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.953+0000 7f1d6dc3b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6819b760 con 0x7f1d6807bdb0 2023-11-27T15:09:04.486 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.953+0000 7f1d677fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d6807bdb0 0x7f1d6819b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:04.486 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.953+0000 7f1d677fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d6807bdb0 0x7f1d6819b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48690/0 (socket says 172.21.15.52:48690) 2023-11-27T15:09:04.487 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.953+0000 7f1d677fe700 1 -- 172.21.15.52:0/1636661653 learned_addr learned my addr 172.21.15.52:0/1636661653 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:04.487 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.953+0000 7f1d677fe700 1 -- 172.21.15.52:0/1636661653 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d58009550 con 0x7f1d6807bdb0 2023-11-27T15:09:04.487 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.953+0000 7f1d677fe700 1 --2- 172.21.15.52:0/1636661653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d6807bdb0 0x7f1d6819b220 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1d58004000 tx=0x7f1d58004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:04.487 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.954+0000 7f1d64ff9700 1 -- 172.21.15.52:0/1636661653 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d58011630 con 0x7f1d6807bdb0 2023-11-27T15:09:04.487 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.954+0000 7f1d64ff9700 1 -- 172.21.15.52:0/1636661653 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d58011790 con 0x7f1d6807bdb0 2023-11-27T15:09:04.487 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.954+0000 7f1d6dc3b700 1 -- 172.21.15.52:0/1636661653 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d6807ae30 con 0x7f1d6807bdb0 2023-11-27T15:09:04.488 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.954+0000 7f1d64ff9700 1 -- 172.21.15.52:0/1636661653 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d58011a30 con 0x7f1d6807bdb0 2023-11-27T15:09:04.488 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.954+0000 7f1d6dc3b700 1 -- 172.21.15.52:0/1636661653 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d6807aca0 con 0x7f1d6807bdb0 2023-11-27T15:09:04.488 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.954+0000 7f1d64ff9700 1 -- 172.21.15.52:0/1636661653 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1d58022070 con 0x7f1d6807bdb0 2023-11-27T15:09:04.488 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.954+0000 7f1d64ff9700 1 -- 172.21.15.52:0/1636661653 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1d5801c450 con 0x7f1d6807bdb0 2023-11-27T15:09:04.488 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.955+0000 7f1d6dc3b700 1 -- 172.21.15.52:0/1636661653 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d6807afc0 con 0x7f1d6807bdb0 2023-11-27T15:09:04.488 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:03.957+0000 7f1d64ff9700 1 -- 172.21.15.52:0/1636661653 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f1d5801b450 con 0x7f1d6807bdb0 2023-11-27T15:09:04.489 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.089+0000 7f1d6dc3b700 1 -- 172.21.15.52:0/1636661653 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f1d6819be60 con 0x7f1d6807bdb0 2023-11-27T15:09:04.489 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.089+0000 7f1d64ff9700 1 -- 172.21.15.52:0/1636661653 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f1d5801b5f0 con 0x7f1d6807bdb0 2023-11-27T15:09:04.489 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.090+0000 7f1d6dc3b700 1 -- 172.21.15.52:0/1636661653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d6807bdb0 msgr2=0x7f1d6819b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:04.489 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.090+0000 7f1d6dc3b700 1 --2- 172.21.15.52:0/1636661653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d6807bdb0 0x7f1d6819b220 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1d58004000 tx=0x7f1d58004750 comp rx=0 tx=0).stop 2023-11-27T15:09:04.489 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.090+0000 7f1d6dc3b700 1 -- 172.21.15.52:0/1636661653 shutdown_connections 2023-11-27T15:09:04.489 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.090+0000 7f1d6dc3b700 1 --2- 172.21.15.52:0/1636661653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d6807bdb0 0x7f1d6819b220 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:04.489 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.090+0000 7f1d6dc3b700 1 -- 172.21.15.52:0/1636661653 >> 172.21.15.52:0/1636661653 conn(0x7f1d68101170 msgr2=0x7f1d68102ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:04.490 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.090+0000 7f1d6dc3b700 1 -- 172.21.15.52:0/1636661653 shutdown_connections 2023-11-27T15:09:04.490 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.090+0000 7f1d6dc3b700 1 -- 172.21.15.52:0/1636661653 wait complete. 2023-11-27T15:09:04.490 INFO:teuthology.orchestra.run.smithi052.stdout:mon is available 2023-11-27T15:09:04.490 INFO:teuthology.orchestra.run.smithi052.stdout:Assimilating anything we can from ceph.conf... 2023-11-27T15:09:05.507 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:05 smithi052 ceph-mon[98550]: from='client.? 172.21.15.52:0/4232767187' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-11-27T15:09:05.507 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:05 smithi052 ceph-mon[98550]: from='client.? 172.21.15.52:0/4232767187' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-11-27T15:09:05.522 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2023-11-27T15:09:05.529 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout [global] 2023-11-27T15:09:05.529 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout fsid = c6c7df88-8d36-11ee-95a2-87774f69a715 2023-11-27T15:09:05.529 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.52:3300,v1:172.21.15.52:6789] 2023-11-27T15:09:05.529 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-11-27T15:09:05.529 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-11-27T15:09:05.529 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-11-27T15:09:05.529 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-11-27T15:09:05.529 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2023-11-27T15:09:05.529 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout [mgr] 2023-11-27T15:09:05.530 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-11-27T15:09:05.530 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2023-11-27T15:09:05.530 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout [osd] 2023-11-27T15:09:05.530 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-11-27T15:09:05.530 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-11-27T15:09:05.530 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-11-27T15:09:05.530 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-11-27T15:09:05.530 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-11-27T15:09:05.530 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.985+0000 7fc9f2fb7700 1 Processor -- start 2023-11-27T15:09:05.530 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.985+0000 7fc9f2fb7700 1 -- start start 2023-11-27T15:09:05.530 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.985+0000 7fc9f2fb7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9ec105ac0 0x7fc9ec105ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:05.530 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.985+0000 7fc9f2fb7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec106410 con 0x7fc9ec105ac0 2023-11-27T15:09:05.531 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.985+0000 7fc9f0d53700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9ec105ac0 0x7fc9ec105ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:05.531 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.985+0000 7fc9f0d53700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9ec105ac0 0x7fc9ec105ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48692/0 (socket says 172.21.15.52:48692) 2023-11-27T15:09:05.531 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.985+0000 7fc9f0d53700 1 -- 172.21.15.52:0/1898706869 learned_addr learned my addr 172.21.15.52:0/1898706869 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:05.531 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.985+0000 7fc9f0d53700 1 -- 172.21.15.52:0/1898706869 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ec106550 con 0x7fc9ec105ac0 2023-11-27T15:09:05.531 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.986+0000 7fc9f0d53700 1 --2- 172.21.15.52:0/1898706869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9ec105ac0 0x7fc9ec105ed0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00098a0 tx=0x7fc9e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2d21c55357347d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:05.531 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.986+0000 7fc9eb7fe700 1 -- 172.21.15.52:0/1898706869 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9e0004160 con 0x7fc9ec105ac0 2023-11-27T15:09:05.531 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.986+0000 7fc9eb7fe700 1 -- 172.21.15.52:0/1898706869 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e00042c0 con 0x7fc9ec105ac0 2023-11-27T15:09:05.531 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.986+0000 7fc9eb7fe700 1 -- 172.21.15.52:0/1898706869 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9e00045a0 con 0x7fc9ec105ac0 2023-11-27T15:09:05.531 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.986+0000 7fc9f2fb7700 1 -- 172.21.15.52:0/1898706869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9ec105ac0 msgr2=0x7fc9ec105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:05.531 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.986+0000 7fc9f2fb7700 1 --2- 172.21.15.52:0/1898706869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9ec105ac0 0x7fc9ec105ed0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00098a0 tx=0x7fc9e0009bb0 comp rx=0 tx=0).stop 2023-11-27T15:09:05.531 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.986+0000 7fc9f2fb7700 1 -- 172.21.15.52:0/1898706869 shutdown_connections 2023-11-27T15:09:05.531 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.986+0000 7fc9f2fb7700 1 --2- 172.21.15.52:0/1898706869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9ec105ac0 0x7fc9ec105ed0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:05.531 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.986+0000 7fc9f2fb7700 1 -- 172.21.15.52:0/1898706869 >> 172.21.15.52:0/1898706869 conn(0x7fc9ec101170 msgr2=0x7fc9ec103580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:05.532 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.986+0000 7fc9f2fb7700 1 -- 172.21.15.52:0/1898706869 shutdown_connections 2023-11-27T15:09:05.532 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.986+0000 7fc9f2fb7700 1 -- 172.21.15.52:0/1898706869 wait complete. 2023-11-27T15:09:05.532 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.987+0000 7fc9f2fb7700 1 Processor -- start 2023-11-27T15:09:05.532 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.987+0000 7fc9f2fb7700 1 -- start start 2023-11-27T15:09:05.532 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.988+0000 7fc9f2fb7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9ec105ac0 0x7fc9ec199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:05.532 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.988+0000 7fc9f2fb7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec199570 con 0x7fc9ec105ac0 2023-11-27T15:09:05.532 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.988+0000 7fc9f0d53700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9ec105ac0 0x7fc9ec199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:05.532 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.988+0000 7fc9f0d53700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9ec105ac0 0x7fc9ec199030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48694/0 (socket says 172.21.15.52:48694) 2023-11-27T15:09:05.532 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.988+0000 7fc9f0d53700 1 -- 172.21.15.52:0/4232767187 learned_addr learned my addr 172.21.15.52:0/4232767187 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:05.532 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.988+0000 7fc9f0d53700 1 -- 172.21.15.52:0/4232767187 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e0009550 con 0x7fc9ec105ac0 2023-11-27T15:09:05.532 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.988+0000 7fc9f0d53700 1 --2- 172.21.15.52:0/4232767187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9ec105ac0 0x7fc9ec199030 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec1067f0 tx=0x7fc9e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:05.532 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.988+0000 7fc9e9ffb700 1 -- 172.21.15.52:0/4232767187 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9e0011630 con 0x7fc9ec105ac0 2023-11-27T15:09:05.533 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.988+0000 7fc9e9ffb700 1 -- 172.21.15.52:0/4232767187 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e0011790 con 0x7fc9ec105ac0 2023-11-27T15:09:05.533 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.988+0000 7fc9f2fb7700 1 -- 172.21.15.52:0/4232767187 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9ec107010 con 0x7fc9ec105ac0 2023-11-27T15:09:05.533 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.988+0000 7fc9e9ffb700 1 -- 172.21.15.52:0/4232767187 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9e0011a30 con 0x7fc9ec105ac0 2023-11-27T15:09:05.533 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.989+0000 7fc9f2fb7700 1 -- 172.21.15.52:0/4232767187 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9ec106e80 con 0x7fc9ec105ac0 2023-11-27T15:09:05.533 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.989+0000 7fc9e9ffb700 1 -- 172.21.15.52:0/4232767187 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc9e0022070 con 0x7fc9ec105ac0 2023-11-27T15:09:05.533 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.989+0000 7fc9e9ffb700 1 -- 172.21.15.52:0/4232767187 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc9e001c450 con 0x7fc9ec105ac0 2023-11-27T15:09:05.533 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.990+0000 7fc9f2fb7700 1 -- 172.21.15.52:0/4232767187 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9ec1071a0 con 0x7fc9ec105ac0 2023-11-27T15:09:05.533 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:04.992+0000 7fc9e9ffb700 1 -- 172.21.15.52:0/4232767187 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7fc9e001b450 con 0x7fc9ec105ac0 2023-11-27T15:09:05.533 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:05.118+0000 7fc9f2fb7700 1 -- 172.21.15.52:0/4232767187 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fc9ec02cfe0 con 0x7fc9ec105ac0 2023-11-27T15:09:05.533 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:05.122+0000 7fc9e9ffb700 1 -- 172.21.15.52:0/4232767187 <== mon.0 v2:172.21.15.52:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc9e001b5f0 con 0x7fc9ec105ac0 2023-11-27T15:09:05.533 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:05.122+0000 7fc9e9ffb700 1 -- 172.21.15.52:0/4232767187 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fc9e0024630 con 0x7fc9ec105ac0 2023-11-27T15:09:05.533 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:05.123+0000 7fc9f2fb7700 1 -- 172.21.15.52:0/4232767187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9ec105ac0 msgr2=0x7fc9ec199030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:05.534 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:05.123+0000 7fc9f2fb7700 1 --2- 172.21.15.52:0/4232767187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9ec105ac0 0x7fc9ec199030 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec1067f0 tx=0x7fc9e0004750 comp rx=0 tx=0).stop 2023-11-27T15:09:05.534 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:05.123+0000 7fc9f2fb7700 1 -- 172.21.15.52:0/4232767187 shutdown_connections 2023-11-27T15:09:05.534 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:05.123+0000 7fc9f2fb7700 1 --2- 172.21.15.52:0/4232767187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9ec105ac0 0x7fc9ec199030 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:05.534 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:05.123+0000 7fc9f2fb7700 1 -- 172.21.15.52:0/4232767187 >> 172.21.15.52:0/4232767187 conn(0x7fc9ec101170 msgr2=0x7fc9ec18d520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:05.534 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:05.123+0000 7fc9f2fb7700 1 -- 172.21.15.52:0/4232767187 shutdown_connections 2023-11-27T15:09:05.534 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:05.123+0000 7fc9f2fb7700 1 -- 172.21.15.52:0/4232767187 wait complete. 2023-11-27T15:09:05.534 INFO:teuthology.orchestra.run.smithi052.stdout:Generating new minimal ceph.conf... 2023-11-27T15:09:06.536 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:06 smithi052 ceph-mon[98550]: from='client.? 172.21.15.52:0/3906965074' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:09:06.573 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.022+0000 7f925e0dc700 1 Processor -- start 2023-11-27T15:09:06.573 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.022+0000 7f925e0dc700 1 -- start start 2023-11-27T15:09:06.573 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.022+0000 7f925e0dc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92581064f0 0x7f92581088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:06.574 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.022+0000 7f925e0dc700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9258079e00 con 0x7f92581064f0 2023-11-27T15:09:06.574 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.022+0000 7f92577fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92581064f0 0x7f92581088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:06.574 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.022+0000 7f92577fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92581064f0 0x7f92581088d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48696/0 (socket says 172.21.15.52:48696) 2023-11-27T15:09:06.574 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.022+0000 7f92577fe700 1 -- 172.21.15.52:0/2466048636 learned_addr learned my addr 172.21.15.52:0/2466048636 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:06.574 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.023+0000 7f92577fe700 1 -- 172.21.15.52:0/2466048636 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9258108e10 con 0x7f92581064f0 2023-11-27T15:09:06.574 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.023+0000 7f92577fe700 1 --2- 172.21.15.52:0/2466048636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92581064f0 0x7f92581088d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f924c0098a0 tx=0x7f924c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84061c45ca3e0f8e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:06.574 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.023+0000 7f92567fc700 1 -- 172.21.15.52:0/2466048636 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f924c004160 con 0x7f92581064f0 2023-11-27T15:09:06.574 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.023+0000 7f92567fc700 1 -- 172.21.15.52:0/2466048636 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f924c0042c0 con 0x7f92581064f0 2023-11-27T15:09:06.574 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.023+0000 7f92567fc700 1 -- 172.21.15.52:0/2466048636 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f924c011a00 con 0x7f92581064f0 2023-11-27T15:09:06.574 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.024+0000 7f925e0dc700 1 -- 172.21.15.52:0/2466048636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92581064f0 msgr2=0x7f92581088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:06.574 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.024+0000 7f925e0dc700 1 --2- 172.21.15.52:0/2466048636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92581064f0 0x7f92581088d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f924c0098a0 tx=0x7f924c009bb0 comp rx=0 tx=0).stop 2023-11-27T15:09:06.574 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.024+0000 7f925e0dc700 1 -- 172.21.15.52:0/2466048636 shutdown_connections 2023-11-27T15:09:06.574 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.024+0000 7f925e0dc700 1 --2- 172.21.15.52:0/2466048636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92581064f0 0x7f92581088d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:06.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.024+0000 7f925e0dc700 1 -- 172.21.15.52:0/2466048636 >> 172.21.15.52:0/2466048636 conn(0x7f92580ffed0 msgr2=0x7f9258102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:06.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.024+0000 7f925e0dc700 1 -- 172.21.15.52:0/2466048636 shutdown_connections 2023-11-27T15:09:06.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.024+0000 7f925e0dc700 1 -- 172.21.15.52:0/2466048636 wait complete. 2023-11-27T15:09:06.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.024+0000 7f925e0dc700 1 Processor -- start 2023-11-27T15:09:06.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.025+0000 7f925e0dc700 1 -- start start 2023-11-27T15:09:06.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.025+0000 7f925e0dc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92581064f0 0x7f9258199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:06.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.025+0000 7f925e0dc700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9258199cb0 con 0x7f92581064f0 2023-11-27T15:09:06.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.025+0000 7f92577fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92581064f0 0x7f9258199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:06.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.025+0000 7f92577fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92581064f0 0x7f9258199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48698/0 (socket says 172.21.15.52:48698) 2023-11-27T15:09:06.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.025+0000 7f92577fe700 1 -- 172.21.15.52:0/3906965074 learned_addr learned my addr 172.21.15.52:0/3906965074 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:06.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.025+0000 7f92577fe700 1 -- 172.21.15.52:0/3906965074 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924c009550 con 0x7f92581064f0 2023-11-27T15:09:06.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.025+0000 7f92577fe700 1 --2- 172.21.15.52:0/3906965074 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92581064f0 0x7f9258199770 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f924c009fb0 tx=0x7f924c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:06.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.025+0000 7f9254ff9700 1 -- 172.21.15.52:0/3906965074 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f924c003930 con 0x7f92581064f0 2023-11-27T15:09:06.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.025+0000 7f925e0dc700 1 -- 172.21.15.52:0/3906965074 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9258109d40 con 0x7f92581064f0 2023-11-27T15:09:06.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.025+0000 7f9254ff9700 1 -- 172.21.15.52:0/3906965074 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f924c003a90 con 0x7f92581064f0 2023-11-27T15:09:06.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.025+0000 7f9254ff9700 1 -- 172.21.15.52:0/3906965074 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f924c019610 con 0x7f92581064f0 2023-11-27T15:09:06.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.026+0000 7f925e0dc700 1 -- 172.21.15.52:0/3906965074 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9258109bb0 con 0x7f92581064f0 2023-11-27T15:09:06.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.026+0000 7f9254ff9700 1 -- 172.21.15.52:0/3906965074 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f924c019bf0 con 0x7f92581064f0 2023-11-27T15:09:06.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.026+0000 7f9254ff9700 1 -- 172.21.15.52:0/3906965074 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f924c0243f0 con 0x7f92581064f0 2023-11-27T15:09:06.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.027+0000 7f925e0dc700 1 -- 172.21.15.52:0/3906965074 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9258062120 con 0x7f92581064f0 2023-11-27T15:09:06.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.029+0000 7f9254ff9700 1 -- 172.21.15.52:0/3906965074 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f924c003c00 con 0x7f92581064f0 2023-11-27T15:09:06.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.156+0000 7f925e0dc700 1 -- 172.21.15.52:0/3906965074 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f925819a1f0 con 0x7f92581064f0 2023-11-27T15:09:06.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.157+0000 7f9254ff9700 1 -- 172.21.15.52:0/3906965074 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f924c01a9d0 con 0x7f92581064f0 2023-11-27T15:09:06.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.157+0000 7f925e0dc700 1 -- 172.21.15.52:0/3906965074 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92581064f0 msgr2=0x7f9258199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:06.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.157+0000 7f925e0dc700 1 --2- 172.21.15.52:0/3906965074 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92581064f0 0x7f9258199770 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f924c009fb0 tx=0x7f924c011ad0 comp rx=0 tx=0).stop 2023-11-27T15:09:06.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.157+0000 7f925e0dc700 1 -- 172.21.15.52:0/3906965074 shutdown_connections 2023-11-27T15:09:06.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.157+0000 7f925e0dc700 1 --2- 172.21.15.52:0/3906965074 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92581064f0 0x7f9258199770 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:06.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.157+0000 7f925e0dc700 1 -- 172.21.15.52:0/3906965074 >> 172.21.15.52:0/3906965074 conn(0x7f92580ffed0 msgr2=0x7f9258101530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:06.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.158+0000 7f925e0dc700 1 -- 172.21.15.52:0/3906965074 shutdown_connections 2023-11-27T15:09:06.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:06.158+0000 7f925e0dc700 1 -- 172.21.15.52:0/3906965074 wait complete. 2023-11-27T15:09:06.577 INFO:teuthology.orchestra.run.smithi052.stdout:Restarting the monitor... 2023-11-27T15:09:06.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:06 smithi052 systemd[1]: Stopping Ceph mon.a for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:09:07.253 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:06 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mon-a[98526]: 2023-11-27T15:09:06.915+0000 7fc6fa630700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:09:07.253 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:06 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mon-a[98526]: 2023-11-27T15:09:06.915+0000 7fc6fa630700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-11-27T15:09:07.613 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:07 smithi052 bash[98886]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mon-a 2023-11-27T15:09:07.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:07 smithi052 systemd[1]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mon.a.service: Succeeded. 2023-11-27T15:09:07.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:07 smithi052 systemd[1]: Stopped Ceph mon.a for c6c7df88-8d36-11ee-95a2-87774f69a715. 2023-11-27T15:09:07.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:07 smithi052 systemd[1]: Starting Ceph mon.a for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:09:08.167 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:07 smithi052 podman[99016]: 2023-11-27T15:09:08.168 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 bash[99016]: 39f489ecf31e274595e9ab270c70cbafdbab1140864cbbb144c7db717c4e1a21 2023-11-27T15:09:08.168 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: set uid:gid to 167:167 (ceph:ceph) 2023-11-27T15:09:08.168 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev), process ceph-mon, pid 7 2023-11-27T15:09:08.168 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: pidfile_write: ignore empty --pid-file 2023-11-27T15:09:08.168 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: load: jerasure load: lrc 2023-11-27T15:09:08.168 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: RocksDB version: 7.9.2 2023-11-27T15:09:08.168 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Git sha 0 2023-11-27T15:09:08.168 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Compile date 2023-11-16 19:40:53 2023-11-27T15:09:08.169 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: DB SUMMARY 2023-11-27T15:09:08.169 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: DB Session ID: 3I8RSVXUDYKZ5RHCXTBW 2023-11-27T15:09:08.169 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: CURRENT file: CURRENT 2023-11-27T15:09:08.169 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: IDENTITY file: IDENTITY 2023-11-27T15:09:08.169 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-11-27T15:09:08.169 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-11-27T15:09:08.169 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 98693 ; 2023-11-27T15:09:08.169 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.error_if_exists: 0 2023-11-27T15:09:08.169 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.create_if_missing: 0 2023-11-27T15:09:08.170 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.paranoid_checks: 1 2023-11-27T15:09:08.170 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.flush_verify_memtable_count: 1 2023-11-27T15:09:08.170 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-11-27T15:09:08.170 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-11-27T15:09:08.170 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.env: 0x55cd9cd05720 2023-11-27T15:09:08.170 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.fs: PosixFileSystem 2023-11-27T15:09:08.170 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.info_log: 0x55cd9f0ef3c0 2023-11-27T15:09:08.170 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.max_file_opening_threads: 16 2023-11-27T15:09:08.170 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.statistics: (nil) 2023-11-27T15:09:08.171 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.use_fsync: 0 2023-11-27T15:09:08.171 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.max_log_file_size: 0 2023-11-27T15:09:08.171 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-11-27T15:09:08.171 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.log_file_time_to_roll: 0 2023-11-27T15:09:08.171 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.keep_log_file_num: 1000 2023-11-27T15:09:08.171 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.recycle_log_file_num: 0 2023-11-27T15:09:08.171 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.allow_fallocate: 1 2023-11-27T15:09:08.171 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.allow_mmap_reads: 0 2023-11-27T15:09:08.171 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.allow_mmap_writes: 0 2023-11-27T15:09:08.171 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.use_direct_reads: 0 2023-11-27T15:09:08.171 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-11-27T15:09:08.171 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.create_missing_column_families: 0 2023-11-27T15:09:08.171 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.db_log_dir: 2023-11-27T15:09:08.172 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.wal_dir: 2023-11-27T15:09:08.172 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.table_cache_numshardbits: 6 2023-11-27T15:09:08.172 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.WAL_ttl_seconds: 0 2023-11-27T15:09:08.172 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.WAL_size_limit_MB: 0 2023-11-27T15:09:08.172 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-11-27T15:09:08.172 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-11-27T15:09:08.172 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.is_fd_close_on_exec: 1 2023-11-27T15:09:08.172 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.advise_random_on_open: 1 2023-11-27T15:09:08.172 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.db_write_buffer_size: 0 2023-11-27T15:09:08.172 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.write_buffer_manager: 0x55cd9e3603c0 2023-11-27T15:09:08.173 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-11-27T15:09:08.173 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-11-27T15:09:08.173 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.use_adaptive_mutex: 0 2023-11-27T15:09:08.173 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.rate_limiter: (nil) 2023-11-27T15:09:08.173 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-11-27T15:09:08.173 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.wal_recovery_mode: 2 2023-11-27T15:09:08.173 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.enable_thread_tracking: 0 2023-11-27T15:09:08.173 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.enable_pipelined_write: 0 2023-11-27T15:09:08.173 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.unordered_write: 0 2023-11-27T15:09:08.173 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-11-27T15:09:08.174 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-11-27T15:09:08.174 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-11-27T15:09:08.174 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-11-27T15:09:08.174 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.row_cache: None 2023-11-27T15:09:08.174 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.wal_filter: None 2023-11-27T15:09:08.174 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-11-27T15:09:08.174 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.allow_ingest_behind: 0 2023-11-27T15:09:08.174 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.two_write_queues: 0 2023-11-27T15:09:08.174 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.manual_wal_flush: 0 2023-11-27T15:09:08.174 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.wal_compression: 0 2023-11-27T15:09:08.175 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.atomic_flush: 0 2023-11-27T15:09:08.175 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-11-27T15:09:08.175 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.persist_stats_to_disk: 0 2023-11-27T15:09:08.175 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.write_dbid_to_manifest: 0 2023-11-27T15:09:08.175 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.log_readahead_size: 0 2023-11-27T15:09:08.175 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-11-27T15:09:08.175 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.best_efforts_recovery: 0 2023-11-27T15:09:08.175 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-11-27T15:09:08.175 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-11-27T15:09:08.175 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.allow_data_in_errors: 0 2023-11-27T15:09:08.176 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.db_host_id: __hostname__ 2023-11-27T15:09:08.176 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.enforce_single_del_contracts: true 2023-11-27T15:09:08.176 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.max_background_jobs: 2 2023-11-27T15:09:08.176 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.max_background_compactions: -1 2023-11-27T15:09:08.176 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.max_subcompactions: 1 2023-11-27T15:09:08.176 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-11-27T15:09:08.176 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-11-27T15:09:08.176 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.delayed_write_rate : 16777216 2023-11-27T15:09:08.176 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.max_total_wal_size: 0 2023-11-27T15:09:08.178 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-11-27T15:09:08.178 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.stats_dump_period_sec: 600 2023-11-27T15:09:08.178 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.stats_persist_period_sec: 600 2023-11-27T15:09:08.178 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-11-27T15:09:08.178 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.max_open_files: -1 2023-11-27T15:09:08.178 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.bytes_per_sync: 0 2023-11-27T15:09:08.178 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.wal_bytes_per_sync: 0 2023-11-27T15:09:08.179 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.strict_bytes_per_sync: 0 2023-11-27T15:09:08.179 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.compaction_readahead_size: 0 2023-11-27T15:09:08.179 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.max_background_flushes: -1 2023-11-27T15:09:08.179 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Compression algorithms supported: 2023-11-27T15:09:08.179 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-11-27T15:09:08.179 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: kZSTD supported: 0 2023-11-27T15:09:08.179 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: kXpressCompression supported: 0 2023-11-27T15:09:08.179 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: kLZ4HCCompression supported: 1 2023-11-27T15:09:08.179 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: kZlibCompression supported: 1 2023-11-27T15:09:08.179 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: kSnappyCompression supported: 1 2023-11-27T15:09:08.179 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: kLZ4Compression supported: 1 2023-11-27T15:09:08.179 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: kBZip2Compression supported: 0 2023-11-27T15:09:08.180 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Fast CRC32 supported: Supported on x86 2023-11-27T15:09:08.180 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: DMutex implementation: pthread_mutex_t 2023-11-27T15:09:08.180 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-27T15:09:08.180 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-11-27T15:09:08.180 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-11-27T15:09:08.180 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.merge_operator: 2023-11-27T15:09:08.180 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.compaction_filter: None 2023-11-27T15:09:08.180 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.compaction_filter_factory: None 2023-11-27T15:09:08.180 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.sst_partitioner_factory: None 2023-11-27T15:09:08.180 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.memtable_factory: SkipListFactory 2023-11-27T15:09:08.180 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.table_factory: BlockBasedTable 2023-11-27T15:09:08.180 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55cd9f0ef4e0) 2023-11-27T15:09:08.180 INFO:journalctl@ceph.mon.a.smithi052.stdout: cache_index_and_filter_blocks: 1 2023-11-27T15:09:08.181 INFO:journalctl@ceph.mon.a.smithi052.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-11-27T15:09:08.181 INFO:journalctl@ceph.mon.a.smithi052.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-11-27T15:09:08.181 INFO:journalctl@ceph.mon.a.smithi052.stdout: pin_top_level_index_and_filter: 1 2023-11-27T15:09:08.181 INFO:journalctl@ceph.mon.a.smithi052.stdout: index_type: 0 2023-11-27T15:09:08.181 INFO:journalctl@ceph.mon.a.smithi052.stdout: data_block_index_type: 0 2023-11-27T15:09:08.181 INFO:journalctl@ceph.mon.a.smithi052.stdout: index_shortening: 1 2023-11-27T15:09:08.181 INFO:journalctl@ceph.mon.a.smithi052.stdout: data_block_hash_table_util_ratio: 0.750000 2023-11-27T15:09:08.181 INFO:journalctl@ceph.mon.a.smithi052.stdout: checksum: 4 2023-11-27T15:09:08.181 INFO:journalctl@ceph.mon.a.smithi052.stdout: no_block_cache: 0 2023-11-27T15:09:08.181 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_cache: 0x55cd9e401350 2023-11-27T15:09:08.181 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_cache_name: BinnedLRUCache 2023-11-27T15:09:08.181 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_cache_options: 2023-11-27T15:09:08.181 INFO:journalctl@ceph.mon.a.smithi052.stdout: capacity : 536870912 2023-11-27T15:09:08.182 INFO:journalctl@ceph.mon.a.smithi052.stdout: num_shard_bits : 4 2023-11-27T15:09:08.182 INFO:journalctl@ceph.mon.a.smithi052.stdout: strict_capacity_limit : 0 2023-11-27T15:09:08.182 INFO:journalctl@ceph.mon.a.smithi052.stdout: high_pri_pool_ratio: 0.000 2023-11-27T15:09:08.182 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_cache_compressed: (nil) 2023-11-27T15:09:08.182 INFO:journalctl@ceph.mon.a.smithi052.stdout: persistent_cache: (nil) 2023-11-27T15:09:08.182 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_size: 4096 2023-11-27T15:09:08.182 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_size_deviation: 10 2023-11-27T15:09:08.182 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_restart_interval: 16 2023-11-27T15:09:08.182 INFO:journalctl@ceph.mon.a.smithi052.stdout: index_block_restart_interval: 1 2023-11-27T15:09:08.182 INFO:journalctl@ceph.mon.a.smithi052.stdout: metadata_block_size: 4096 2023-11-27T15:09:08.182 INFO:journalctl@ceph.mon.a.smithi052.stdout: partition_filters: 0 2023-11-27T15:09:08.182 INFO:journalctl@ceph.mon.a.smithi052.stdout: use_delta_encoding: 1 2023-11-27T15:09:08.183 INFO:journalctl@ceph.mon.a.smithi052.stdout: filter_policy: bloomfilter 2023-11-27T15:09:08.183 INFO:journalctl@ceph.mon.a.smithi052.stdout: whole_key_filtering: 1 2023-11-27T15:09:08.183 INFO:journalctl@ceph.mon.a.smithi052.stdout: verify_compression: 0 2023-11-27T15:09:08.183 INFO:journalctl@ceph.mon.a.smithi052.stdout: read_amp_bytes_per_bit: 0 2023-11-27T15:09:08.183 INFO:journalctl@ceph.mon.a.smithi052.stdout: format_version: 5 2023-11-27T15:09:08.183 INFO:journalctl@ceph.mon.a.smithi052.stdout: enable_index_compression: 1 2023-11-27T15:09:08.183 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_align: 0 2023-11-27T15:09:08.183 INFO:journalctl@ceph.mon.a.smithi052.stdout: max_auto_readahead_size: 262144 2023-11-27T15:09:08.183 INFO:journalctl@ceph.mon.a.smithi052.stdout: prepopulate_block_cache: 0 2023-11-27T15:09:08.183 INFO:journalctl@ceph.mon.a.smithi052.stdout: initial_auto_readahead_size: 8192 2023-11-27T15:09:08.183 INFO:journalctl@ceph.mon.a.smithi052.stdout: num_file_reads_for_auto_readahead: 2 2023-11-27T15:09:08.183 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.write_buffer_size: 33554432 2023-11-27T15:09:08.183 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.max_write_buffer_number: 2 2023-11-27T15:09:08.184 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.compression: NoCompression 2023-11-27T15:09:08.184 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.bottommost_compression: Disabled 2023-11-27T15:09:08.184 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.prefix_extractor: nullptr 2023-11-27T15:09:08.184 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-11-27T15:09:08.184 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.num_levels: 7 2023-11-27T15:09:08.184 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-11-27T15:09:08.184 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-11-27T15:09:08.184 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-11-27T15:09:08.184 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-11-27T15:09:08.184 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-11-27T15:09:08.184 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-11-27T15:09:08.184 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-11-27T15:09:08.185 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-11-27T15:09:08.185 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-11-27T15:09:08.185 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-11-27T15:09:08.186 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-11-27T15:09:08.186 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-11-27T15:09:08.186 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.compression_opts.window_bits: -14 2023-11-27T15:09:08.186 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.compression_opts.level: 32767 2023-11-27T15:09:08.186 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.compression_opts.strategy: 0 2023-11-27T15:09:08.186 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-11-27T15:09:08.186 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-11-27T15:09:08.186 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-11-27T15:09:08.186 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-11-27T15:09:08.186 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.compression_opts.enabled: false 2023-11-27T15:09:08.186 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-11-27T15:09:08.187 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-11-27T15:09:08.187 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-11-27T15:09:08.187 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-11-27T15:09:08.187 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.target_file_size_base: 67108864 2023-11-27T15:09:08.187 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.target_file_size_multiplier: 1 2023-11-27T15:09:08.187 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-11-27T15:09:08.187 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-11-27T15:09:08.187 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-11-27T15:09:08.187 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-11-27T15:09:08.187 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-11-27T15:09:08.187 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-11-27T15:09:08.187 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-11-27T15:09:08.187 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-11-27T15:09:08.188 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-11-27T15:09:08.188 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-11-27T15:09:08.188 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-11-27T15:09:08.188 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-11-27T15:09:08.188 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-11-27T15:09:08.188 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.arena_block_size: 1048576 2023-11-27T15:09:08.188 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-11-27T15:09:08.188 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-11-27T15:09:08.188 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.disable_auto_compactions: 0 2023-11-27T15:09:08.188 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-11-27T15:09:08.188 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-11-27T15:09:08.188 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-11-27T15:09:08.188 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-11-27T15:09:08.189 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-11-27T15:09:08.189 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-11-27T15:09:08.189 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-11-27T15:09:08.189 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-11-27T15:09:08.189 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-11-27T15:09:08.189 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-11-27T15:09:08.189 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.table_properties_collectors: 2023-11-27T15:09:08.189 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.inplace_update_support: 0 2023-11-27T15:09:08.189 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.inplace_update_num_locks: 10000 2023-11-27T15:09:08.189 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-11-27T15:09:08.189 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-11-27T15:09:08.189 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.memtable_huge_page_size: 0 2023-11-27T15:09:08.190 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.bloom_locality: 0 2023-11-27T15:09:08.190 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.max_successive_merges: 0 2023-11-27T15:09:08.190 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.optimize_filters_for_hits: 0 2023-11-27T15:09:08.190 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.paranoid_file_checks: 0 2023-11-27T15:09:08.190 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.force_consistency_checks: 1 2023-11-27T15:09:08.190 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.report_bg_io_stats: 0 2023-11-27T15:09:08.190 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.ttl: 2592000 2023-11-27T15:09:08.190 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.periodic_compaction_seconds: 0 2023-11-27T15:09:08.190 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-11-27T15:09:08.190 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-11-27T15:09:08.190 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.enable_blob_files: false 2023-11-27T15:09:08.190 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.min_blob_size: 0 2023-11-27T15:09:08.191 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.blob_file_size: 268435456 2023-11-27T15:09:08.191 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.blob_compression_type: NoCompression 2023-11-27T15:09:08.191 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.enable_blob_garbage_collection: false 2023-11-27T15:09:08.191 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-11-27T15:09:08.191 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-11-27T15:09:08.191 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-11-27T15:09:08.191 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.blob_file_starting_level: 0 2023-11-27T15:09:08.191 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-11-27T15:09:08.191 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-11-27T15:09:08.191 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-11-27T15:09:08.191 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 3f12d391-9eae-4c15-9729-a880b90073b7 2023-11-27T15:09:08.191 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701097748103089, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-11-27T15:09:08.191 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-11-27T15:09:08.192 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701097748104161, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93946, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 314, "table_properties": {"data_size": 91945, "index_size": 283, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13658, "raw_average_key_size": 50, "raw_value_size": 84486, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 273, "num_filter_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1701097748, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "3f12d391-9eae-4c15-9729-a880b90073b7", "db_session_id": "3I8RSVXUDYKZ5RHCXTBW", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-11-27T15:09:08.192 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701097748104233, "job": 1, "event": "recovery_finished"} 2023-11-27T15:09:08.192 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-11-27T15:09:08.192 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-11-27T15:09:08.192 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55cd9e49e000 2023-11-27T15:09:08.192 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: DB pointer 0x55cd9e48a000 2023-11-27T15:09:08.192 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-11-27T15:09:08.192 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-11-27T15:09:08.192 INFO:journalctl@ceph.mon.a.smithi052.stdout: ** DB Stats ** 2023-11-27T15:09:08.192 INFO:journalctl@ceph.mon.a.smithi052.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-27T15:09:08.192 INFO:journalctl@ceph.mon.a.smithi052.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-11-27T15:09:08.192 INFO:journalctl@ceph.mon.a.smithi052.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-27T15:09:08.193 INFO:journalctl@ceph.mon.a.smithi052.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-27T15:09:08.193 INFO:journalctl@ceph.mon.a.smithi052.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-11-27T15:09:08.193 INFO:journalctl@ceph.mon.a.smithi052.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-27T15:09:08.193 INFO:journalctl@ceph.mon.a.smithi052.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-27T15:09:08.193 INFO:journalctl@ceph.mon.a.smithi052.stdout: 2023-11-27T15:09:08.193 INFO:journalctl@ceph.mon.a.smithi052.stdout: ** Compaction Stats [default] ** 2023-11-27T15:09:08.193 INFO:journalctl@ceph.mon.a.smithi052.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-27T15:09:08.193 INFO:journalctl@ceph.mon.a.smithi052.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-11-27T15:09:08.193 INFO:journalctl@ceph.mon.a.smithi052.stdout: L0 2/0 93.58 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 135.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-27T15:09:08.193 INFO:journalctl@ceph.mon.a.smithi052.stdout: Sum 2/0 93.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 135.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-27T15:09:08.193 INFO:journalctl@ceph.mon.a.smithi052.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 135.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-27T15:09:08.193 INFO:journalctl@ceph.mon.a.smithi052.stdout: 2023-11-27T15:09:08.194 INFO:journalctl@ceph.mon.a.smithi052.stdout: ** Compaction Stats [default] ** 2023-11-27T15:09:08.194 INFO:journalctl@ceph.mon.a.smithi052.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-27T15:09:08.194 INFO:journalctl@ceph.mon.a.smithi052.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-11-27T15:09:08.194 INFO:journalctl@ceph.mon.a.smithi052.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 135.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-27T15:09:08.194 INFO:journalctl@ceph.mon.a.smithi052.stdout: 2023-11-27T15:09:08.194 INFO:journalctl@ceph.mon.a.smithi052.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-11-27T15:09:08.194 INFO:journalctl@ceph.mon.a.smithi052.stdout: 2023-11-27T15:09:08.194 INFO:journalctl@ceph.mon.a.smithi052.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-27T15:09:08.194 INFO:journalctl@ceph.mon.a.smithi052.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-11-27T15:09:08.194 INFO:journalctl@ceph.mon.a.smithi052.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-11-27T15:09:08.194 INFO:journalctl@ceph.mon.a.smithi052.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-11-27T15:09:08.194 INFO:journalctl@ceph.mon.a.smithi052.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-11-27T15:09:08.194 INFO:journalctl@ceph.mon.a.smithi052.stdout: AddFile(Keys): cumulative 0, interval 0 2023-11-27T15:09:08.195 INFO:journalctl@ceph.mon.a.smithi052.stdout: Cumulative compaction: 0.00 GB write, 28.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-27T15:09:08.195 INFO:journalctl@ceph.mon.a.smithi052.stdout: Interval compaction: 0.00 GB write, 28.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-27T15:09:08.195 INFO:journalctl@ceph.mon.a.smithi052.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-11-27T15:09:08.195 INFO:journalctl@ceph.mon.a.smithi052.stdout: Block cache BinnedLRUCache@0x55cd9e401350#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-11-27T15:09:08.195 INFO:journalctl@ceph.mon.a.smithi052.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-11-27T15:09:08.195 INFO:journalctl@ceph.mon.a.smithi052.stdout: 2023-11-27T15:09:08.195 INFO:journalctl@ceph.mon.a.smithi052.stdout: ** File Read Latency Histogram By Level [default] ** 2023-11-27T15:09:08.195 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: starting mon.a rank 0 at public addrs [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] at bind addrs [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid c6c7df88-8d36-11ee-95a2-87774f69a715 2023-11-27T15:09:08.195 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: mon.a@-1(???) e1 preinit fsid c6c7df88-8d36-11ee-95a2-87774f69a715 2023-11-27T15:09:08.195 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: mon.a@-1(???).mds e1 new map 2023-11-27T15:09:08.195 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: mon.a@-1(???).mds e1 print_map 2023-11-27T15:09:08.195 INFO:journalctl@ceph.mon.a.smithi052.stdout: e1 2023-11-27T15:09:08.195 INFO:journalctl@ceph.mon.a.smithi052.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-11-27T15:09:08.196 INFO:journalctl@ceph.mon.a.smithi052.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-27T15:09:08.196 INFO:journalctl@ceph.mon.a.smithi052.stdout: legacy client fscid: -1 2023-11-27T15:09:08.196 INFO:journalctl@ceph.mon.a.smithi052.stdout: 2023-11-27T15:09:08.196 INFO:journalctl@ceph.mon.a.smithi052.stdout: No filesystems configured 2023-11-27T15:09:08.196 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-11-27T15:09:08.196 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-11-27T15:09:08.196 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-11-27T15:09:08.196 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-11-27T15:09:08.196 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-11-27T15:09:08.196 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: expand_channel_meta expand map: {default=false} 2023-11-27T15:09:08.196 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: expand_channel_meta from 'false' to 'false' 2023-11-27T15:09:08.196 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: expand_channel_meta expanded map: {default=false} 2023-11-27T15:09:08.196 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: expand_channel_meta expand map: {default=info} 2023-11-27T15:09:08.197 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: expand_channel_meta from 'info' to 'info' 2023-11-27T15:09:08.197 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: expand_channel_meta expanded map: {default=info} 2023-11-27T15:09:08.197 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: expand_channel_meta expand map: {default=daemon} 2023-11-27T15:09:08.197 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: expand_channel_meta from 'daemon' to 'daemon' 2023-11-27T15:09:08.197 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: expand_channel_meta expanded map: {default=daemon} 2023-11-27T15:09:08.197 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: expand_channel_meta expand map: {default=debug} 2023-11-27T15:09:08.197 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: expand_channel_meta from 'debug' to 'debug' 2023-11-27T15:09:08.197 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: expand_channel_meta expanded map: {default=debug} 2023-11-27T15:09:08.197 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: mon.a is new leader, mons a in quorum (ranks 0) 2023-11-27T15:09:08.197 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: monmap e1: 1 mons at {a=[v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0]} removed_ranks: {} 2023-11-27T15:09:08.197 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: fsmap 2023-11-27T15:09:08.197 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: osdmap e1: 0 total, 0 up, 0 in 2023-11-27T15:09:08.197 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 ceph-mon[99056]: mgrmap e1: no daemons active 2023-11-27T15:09:08.201 INFO:teuthology.orchestra.run.smithi052.stdout:Setting mon public_network to 172.21.0.0/20 2023-11-27T15:09:08.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:08 smithi052 systemd[1]: Started Ceph mon.a for c6c7df88-8d36-11ee-95a2-87774f69a715. 2023-11-27T15:09:09.753 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.186+0000 7f6e47741700 1 Processor -- start 2023-11-27T15:09:09.753 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.186+0000 7f6e47741700 1 -- start start 2023-11-27T15:09:09.753 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.187+0000 7f6e47741700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e401064d0 0x7f6e401088b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:09.753 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.187+0000 7f6e47741700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e40079e20 con 0x7f6e401064d0 2023-11-27T15:09:09.753 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.187+0000 7f6e454dd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e401064d0 0x7f6e401088b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:09.754 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.187+0000 7f6e454dd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e401064d0 0x7f6e401088b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48700/0 (socket says 172.21.15.52:48700) 2023-11-27T15:09:09.754 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.187+0000 7f6e454dd700 1 -- 172.21.15.52:0/467711045 learned_addr learned my addr 172.21.15.52:0/467711045 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:09.754 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.187+0000 7f6e454dd700 1 -- 172.21.15.52:0/467711045 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e40108df0 con 0x7f6e401064d0 2023-11-27T15:09:09.754 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.188+0000 7f6e454dd700 1 --2- 172.21.15.52:0/467711045 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e401064d0 0x7f6e401088b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6e340098a0 tx=0x7f6e34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3f116d4626802a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:09.754 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.188+0000 7f6e33fff700 1 -- 172.21.15.52:0/467711045 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e34004160 con 0x7f6e401064d0 2023-11-27T15:09:09.754 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.188+0000 7f6e33fff700 1 -- 172.21.15.52:0/467711045 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6e340042c0 con 0x7f6e401064d0 2023-11-27T15:09:09.754 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.188+0000 7f6e33fff700 1 -- 172.21.15.52:0/467711045 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e34011a00 con 0x7f6e401064d0 2023-11-27T15:09:09.754 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.188+0000 7f6e47741700 1 -- 172.21.15.52:0/467711045 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e401064d0 msgr2=0x7f6e401088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:09.754 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.188+0000 7f6e47741700 1 --2- 172.21.15.52:0/467711045 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e401064d0 0x7f6e401088b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6e340098a0 tx=0x7f6e34009bb0 comp rx=0 tx=0).stop 2023-11-27T15:09:09.754 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.188+0000 7f6e47741700 1 -- 172.21.15.52:0/467711045 shutdown_connections 2023-11-27T15:09:09.754 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.188+0000 7f6e47741700 1 --2- 172.21.15.52:0/467711045 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e401064d0 0x7f6e401088b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:09.754 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.188+0000 7f6e47741700 1 -- 172.21.15.52:0/467711045 >> 172.21.15.52:0/467711045 conn(0x7f6e400ffed0 msgr2=0x7f6e40102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:09.755 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.189+0000 7f6e47741700 1 -- 172.21.15.52:0/467711045 shutdown_connections 2023-11-27T15:09:09.755 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.189+0000 7f6e47741700 1 -- 172.21.15.52:0/467711045 wait complete. 2023-11-27T15:09:09.755 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.189+0000 7f6e47741700 1 Processor -- start 2023-11-27T15:09:09.755 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.189+0000 7f6e47741700 1 -- start start 2023-11-27T15:09:09.755 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.189+0000 7f6e47741700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e401064d0 0x7f6e4019b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:09.755 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.189+0000 7f6e47741700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4019bed0 con 0x7f6e401064d0 2023-11-27T15:09:09.755 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.189+0000 7f6e454dd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e401064d0 0x7f6e4019b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:09.755 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.190+0000 7f6e454dd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e401064d0 0x7f6e4019b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48702/0 (socket says 172.21.15.52:48702) 2023-11-27T15:09:09.755 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.190+0000 7f6e454dd700 1 -- 172.21.15.52:0/3306571077 learned_addr learned my addr 172.21.15.52:0/3306571077 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:09.755 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.190+0000 7f6e454dd700 1 -- 172.21.15.52:0/3306571077 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e34009550 con 0x7f6e401064d0 2023-11-27T15:09:09.755 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.190+0000 7f6e454dd700 1 --2- 172.21.15.52:0/3306571077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e401064d0 0x7f6e4019b990 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6e34006af0 tx=0x7f6e34011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:09.755 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.190+0000 7f6e327fc700 1 -- 172.21.15.52:0/3306571077 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e34003a40 con 0x7f6e401064d0 2023-11-27T15:09:09.755 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.190+0000 7f6e327fc700 1 -- 172.21.15.52:0/3306571077 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6e34003ba0 con 0x7f6e401064d0 2023-11-27T15:09:09.756 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.190+0000 7f6e47741700 1 -- 172.21.15.52:0/3306571077 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e40109d20 con 0x7f6e401064d0 2023-11-27T15:09:09.756 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.190+0000 7f6e327fc700 1 -- 172.21.15.52:0/3306571077 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e3401a6e0 con 0x7f6e401064d0 2023-11-27T15:09:09.756 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.190+0000 7f6e47741700 1 -- 172.21.15.52:0/3306571077 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e40109b90 con 0x7f6e401064d0 2023-11-27T15:09:09.756 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.190+0000 7f6e327fc700 1 -- 172.21.15.52:0/3306571077 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6e3401ab60 con 0x7f6e401064d0 2023-11-27T15:09:09.756 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.190+0000 7f6e327fc700 1 -- 172.21.15.52:0/3306571077 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6e34019700 con 0x7f6e401064d0 2023-11-27T15:09:09.756 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.191+0000 7f6e47741700 1 -- 172.21.15.52:0/3306571077 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e40062120 con 0x7f6e401064d0 2023-11-27T15:09:09.756 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.194+0000 7f6e327fc700 1 -- 172.21.15.52:0/3306571077 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f6e3402dde0 con 0x7f6e401064d0 2023-11-27T15:09:09.756 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.319+0000 7f6e47741700 1 -- 172.21.15.52:0/3306571077 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f6e4019c410 con 0x7f6e401064d0 2023-11-27T15:09:09.756 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.321+0000 7f6e327fc700 1 -- 172.21.15.52:0/3306571077 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f6e34024a60 con 0x7f6e401064d0 2023-11-27T15:09:09.756 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.322+0000 7f6e47741700 1 -- 172.21.15.52:0/3306571077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e401064d0 msgr2=0x7f6e4019b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:09.756 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.322+0000 7f6e47741700 1 --2- 172.21.15.52:0/3306571077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e401064d0 0x7f6e4019b990 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6e34006af0 tx=0x7f6e34011ad0 comp rx=0 tx=0).stop 2023-11-27T15:09:09.757 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.322+0000 7f6e47741700 1 -- 172.21.15.52:0/3306571077 shutdown_connections 2023-11-27T15:09:09.757 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.322+0000 7f6e47741700 1 --2- 172.21.15.52:0/3306571077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e401064d0 0x7f6e4019b990 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:09.757 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.322+0000 7f6e47741700 1 -- 172.21.15.52:0/3306571077 >> 172.21.15.52:0/3306571077 conn(0x7f6e400ffed0 msgr2=0x7f6e40101530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:09.757 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.322+0000 7f6e47741700 1 -- 172.21.15.52:0/3306571077 shutdown_connections 2023-11-27T15:09:09.757 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:09.322+0000 7f6e47741700 1 -- 172.21.15.52:0/3306571077 wait complete. 2023-11-27T15:09:09.757 INFO:teuthology.orchestra.run.smithi052.stdout:Wrote config to /etc/ceph/ceph.conf 2023-11-27T15:09:09.757 INFO:teuthology.orchestra.run.smithi052.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-11-27T15:09:09.757 INFO:teuthology.orchestra.run.smithi052.stdout:Creating mgr... 2023-11-27T15:09:09.757 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying port 9283 ... 2023-11-27T15:09:09.757 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying port 8765 ... 2023-11-27T15:09:09.880 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mgr.x 2023-11-27T15:09:09.881 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mgr.x.service: Unit ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mgr.x.service not loaded. 2023-11-27T15:09:09.974 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c6c7df88-8d36-11ee-95a2-87774f69a715.target.wants/ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@.service. 2023-11-27T15:09:09.981 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:09 smithi052 systemd[1]: Starting Ceph mgr.x for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:09:10.542 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:10 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3306571077' entity='client.admin' 2023-11-27T15:09:10.543 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:10 smithi052 podman[99287]: 2023-11-27T15:09:10.582 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-11-27T15:09:10.583 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout disabled 2023-11-27T15:09:10.590 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-11-27T15:09:10.591 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout inactive 2023-11-27T15:09:10.591 INFO:teuthology.orchestra.run.smithi052.stdout:firewalld.service is not enabled 2023-11-27T15:09:10.591 INFO:teuthology.orchestra.run.smithi052.stdout:Not possible to enable service . firewalld.service is not available 2023-11-27T15:09:10.600 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-11-27T15:09:10.600 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout disabled 2023-11-27T15:09:10.609 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-11-27T15:09:10.609 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout inactive 2023-11-27T15:09:10.609 INFO:teuthology.orchestra.run.smithi052.stdout:firewalld.service is not enabled 2023-11-27T15:09:10.609 INFO:teuthology.orchestra.run.smithi052.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-11-27T15:09:10.609 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for mgr to start... 2023-11-27T15:09:10.609 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for mgr... 2023-11-27T15:09:10.852 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:10 smithi052 bash[99287]: 228134862db3c74b4bece70406e632147415dade0974223b1e38403693546cce 2023-11-27T15:09:10.853 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:10 smithi052 systemd[1]: Started Ceph mgr.x for c6c7df88-8d36-11ee-95a2-87774f69a715. 2023-11-27T15:09:10.853 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:10 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:10.689+0000 7f22b6881200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-27T15:09:10.853 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:10 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:10.784+0000 7f22b6881200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-27T15:09:11.117 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:10 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:10.902+0000 7f22b6881200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-27T15:09:11.117 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:11 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:11.014+0000 7f22b6881200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-27T15:09:11.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:11 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3830797902' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-11-27T15:09:11.617 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:11 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:11.226+0000 7f22b6881200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-27T15:09:11.666 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2023-11-27T15:09:11.666 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2023-11-27T15:09:11.666 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsid": "c6c7df88-8d36-11ee-95a2-87774f69a715", 2023-11-27T15:09:11.666 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "health": { 2023-11-27T15:09:11.667 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-11-27T15:09:11.667 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-11-27T15:09:11.667 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-11-27T15:09:11.667 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:09:11.667 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-11-27T15:09:11.667 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-11-27T15:09:11.667 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 0 2023-11-27T15:09:11.667 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:09:11.667 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-11-27T15:09:11.667 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "a" 2023-11-27T15:09:11.667 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:09:11.667 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-11-27T15:09:11.668 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "monmap": { 2023-11-27T15:09:11.668 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:09:11.668 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-11-27T15:09:11.668 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-11-27T15:09:11.668 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:09:11.668 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-11-27T15:09:11.668 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:09:11.668 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-11-27T15:09:11.668 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-11-27T15:09:11.668 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-11-27T15:09:11.668 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-11-27T15:09:11.669 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-11-27T15:09:11.669 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-11-27T15:09:11.669 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:09:11.669 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-11-27T15:09:11.669 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-11-27T15:09:11.669 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-11-27T15:09:11.669 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-11-27T15:09:11.669 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-11-27T15:09:11.669 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-11-27T15:09:11.669 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-11-27T15:09:11.669 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-11-27T15:09:11.669 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-11-27T15:09:11.670 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:09:11.670 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-11-27T15:09:11.670 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:09:11.670 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-11-27T15:09:11.670 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-11-27T15:09:11.670 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:09:11.670 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-11-27T15:09:11.670 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "available": false, 2023-11-27T15:09:11.670 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-11-27T15:09:11.670 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modules": [ 2023-11-27T15:09:11.670 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "iostat", 2023-11-27T15:09:11.670 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "nfs", 2023-11-27T15:09:11.671 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "restful" 2023-11-27T15:09:11.671 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:09:11.671 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-27T15:09:11.671 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:09:11.671 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-11-27T15:09:11.671 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:09:11.671 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modified": "2023-11-27T15:09:03.333886+0000", 2023-11-27T15:09:11.671 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-27T15:09:11.671 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:09:11.671 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-11-27T15:09:11.671 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2023-11-27T15:09:11.671 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.142+0000 7f9dd120a700 1 Processor -- start 2023-11-27T15:09:11.671 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.142+0000 7f9dd120a700 1 -- start start 2023-11-27T15:09:11.672 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.142+0000 7f9dd120a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9dcc105ac0 0x7f9dcc105ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:11.672 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.142+0000 7f9dd120a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc106410 con 0x7f9dcc105ac0 2023-11-27T15:09:11.672 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.142+0000 7f9dcad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9dcc105ac0 0x7f9dcc105ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:11.672 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.142+0000 7f9dcad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9dcc105ac0 0x7f9dcc105ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48708/0 (socket says 172.21.15.52:48708) 2023-11-27T15:09:11.672 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.142+0000 7f9dcad9d700 1 -- 172.21.15.52:0/2864467426 learned_addr learned my addr 172.21.15.52:0/2864467426 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:11.672 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.143+0000 7f9dcad9d700 1 -- 172.21.15.52:0/2864467426 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dcc106550 con 0x7f9dcc105ac0 2023-11-27T15:09:11.672 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.143+0000 7f9dcad9d700 1 --2- 172.21.15.52:0/2864467426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9dcc105ac0 0x7f9dcc105ed0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0098a0 tx=0x7f9dbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61164d627a0efbc server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:11.672 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.143+0000 7f9dc9d9b700 1 -- 172.21.15.52:0/2864467426 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9dbc004160 con 0x7f9dcc105ac0 2023-11-27T15:09:11.672 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.143+0000 7f9dc9d9b700 1 -- 172.21.15.52:0/2864467426 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9dbc0042c0 con 0x7f9dcc105ac0 2023-11-27T15:09:11.672 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.143+0000 7f9dc9d9b700 1 -- 172.21.15.52:0/2864467426 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9dbc011a00 con 0x7f9dcc105ac0 2023-11-27T15:09:11.673 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.144+0000 7f9dd120a700 1 -- 172.21.15.52:0/2864467426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9dcc105ac0 msgr2=0x7f9dcc105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:11.673 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.144+0000 7f9dd120a700 1 --2- 172.21.15.52:0/2864467426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9dcc105ac0 0x7f9dcc105ed0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0098a0 tx=0x7f9dbc009bb0 comp rx=0 tx=0).stop 2023-11-27T15:09:11.673 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.144+0000 7f9dd120a700 1 -- 172.21.15.52:0/2864467426 shutdown_connections 2023-11-27T15:09:11.673 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.144+0000 7f9dd120a700 1 --2- 172.21.15.52:0/2864467426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9dcc105ac0 0x7f9dcc105ed0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:11.673 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.144+0000 7f9dd120a700 1 -- 172.21.15.52:0/2864467426 >> 172.21.15.52:0/2864467426 conn(0x7f9dcc101170 msgr2=0x7f9dcc103580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:11.673 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.144+0000 7f9dd120a700 1 -- 172.21.15.52:0/2864467426 shutdown_connections 2023-11-27T15:09:11.673 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.144+0000 7f9dd120a700 1 -- 172.21.15.52:0/2864467426 wait complete. 2023-11-27T15:09:11.673 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.145+0000 7f9dd120a700 1 Processor -- start 2023-11-27T15:09:11.673 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.145+0000 7f9dd120a700 1 -- start start 2023-11-27T15:09:11.673 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.145+0000 7f9dd120a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9dcc105ac0 0x7f9dcc07bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:11.673 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.145+0000 7f9dd120a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc07c140 con 0x7f9dcc105ac0 2023-11-27T15:09:11.673 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.145+0000 7f9dcad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9dcc105ac0 0x7f9dcc07bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:11.674 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.145+0000 7f9dcad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9dcc105ac0 0x7f9dcc07bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48710/0 (socket says 172.21.15.52:48710) 2023-11-27T15:09:11.674 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.145+0000 7f9dcad9d700 1 -- 172.21.15.52:0/3830797902 learned_addr learned my addr 172.21.15.52:0/3830797902 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:11.674 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.146+0000 7f9dcad9d700 1 -- 172.21.15.52:0/3830797902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dbc009550 con 0x7f9dcc105ac0 2023-11-27T15:09:11.674 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.146+0000 7f9dcad9d700 1 --2- 172.21.15.52:0/3830797902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9dcc105ac0 0x7f9dcc07bc00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc006af0 tx=0x7f9dbc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:11.674 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.146+0000 7f9dc3fff700 1 -- 172.21.15.52:0/3830797902 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9dbc003a40 con 0x7f9dcc105ac0 2023-11-27T15:09:11.674 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.146+0000 7f9dc3fff700 1 -- 172.21.15.52:0/3830797902 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9dbc003ba0 con 0x7f9dcc105ac0 2023-11-27T15:09:11.674 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.146+0000 7f9dd120a700 1 -- 172.21.15.52:0/3830797902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dcc107b10 con 0x7f9dcc105ac0 2023-11-27T15:09:11.674 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.146+0000 7f9dc3fff700 1 -- 172.21.15.52:0/3830797902 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9dbc01a6e0 con 0x7f9dcc105ac0 2023-11-27T15:09:11.674 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.146+0000 7f9dd120a700 1 -- 172.21.15.52:0/3830797902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dcc107980 con 0x7f9dcc105ac0 2023-11-27T15:09:11.674 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.146+0000 7f9dc3fff700 1 -- 172.21.15.52:0/3830797902 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9dbc01ac50 con 0x7f9dcc105ac0 2023-11-27T15:09:11.674 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.146+0000 7f9dc3fff700 1 -- 172.21.15.52:0/3830797902 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9dbc019700 con 0x7f9dcc105ac0 2023-11-27T15:09:11.675 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.147+0000 7f9dd120a700 1 -- 172.21.15.52:0/3830797902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dcc062120 con 0x7f9dcc105ac0 2023-11-27T15:09:11.675 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.151+0000 7f9dc3fff700 1 -- 172.21.15.52:0/3830797902 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f9dbc02de30 con 0x7f9dcc105ac0 2023-11-27T15:09:11.675 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.284+0000 7f9dd120a700 1 -- 172.21.15.52:0/3830797902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f9dcc07ab60 con 0x7f9dcc105ac0 2023-11-27T15:09:11.675 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.285+0000 7f9dc3fff700 1 -- 172.21.15.52:0/3830797902 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f9dbc024a60 con 0x7f9dcc105ac0 2023-11-27T15:09:11.675 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.286+0000 7f9dd120a700 1 -- 172.21.15.52:0/3830797902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9dcc105ac0 msgr2=0x7f9dcc07bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:11.675 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.286+0000 7f9dd120a700 1 --2- 172.21.15.52:0/3830797902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9dcc105ac0 0x7f9dcc07bc00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc006af0 tx=0x7f9dbc011ad0 comp rx=0 tx=0).stop 2023-11-27T15:09:11.675 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.286+0000 7f9dd120a700 1 -- 172.21.15.52:0/3830797902 shutdown_connections 2023-11-27T15:09:11.675 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.286+0000 7f9dd120a700 1 --2- 172.21.15.52:0/3830797902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9dcc105ac0 0x7f9dcc07bc00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:11.675 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.286+0000 7f9dd120a700 1 -- 172.21.15.52:0/3830797902 >> 172.21.15.52:0/3830797902 conn(0x7f9dcc101170 msgr2=0x7f9dcc101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:11.675 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.286+0000 7f9dd120a700 1 -- 172.21.15.52:0/3830797902 shutdown_connections 2023-11-27T15:09:11.675 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:11.286+0000 7f9dd120a700 1 -- 172.21.15.52:0/3830797902 wait complete. 2023-11-27T15:09:11.675 INFO:teuthology.orchestra.run.smithi052.stdout:mgr not available, waiting (1/15)... 2023-11-27T15:09:11.895 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:11 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:11.630+0000 7f22b6881200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-27T15:09:11.895 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:11 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:11.740+0000 7f22b6881200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-27T15:09:12.367 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:11 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:11.890+0000 7f22b6881200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-27T15:09:12.367 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:12 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:12.006+0000 7f22b6881200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-27T15:09:12.867 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:12 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:12.518+0000 7f22b6881200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-27T15:09:13.617 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:13 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:13.223+0000 7f22b6881200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-27T15:09:13.617 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:13 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:13.414+0000 7f22b6881200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-27T15:09:14.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:14 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3686043960' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-11-27T15:09:14.617 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:14 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:14.399+0000 7f22b6881200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:09:14.617 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:14 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:14.516+0000 7f22b6881200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-27T15:09:14.707 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2023-11-27T15:09:14.717 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2023-11-27T15:09:14.717 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsid": "c6c7df88-8d36-11ee-95a2-87774f69a715", 2023-11-27T15:09:14.717 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "health": { 2023-11-27T15:09:14.717 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-11-27T15:09:14.718 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-11-27T15:09:14.718 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-11-27T15:09:14.718 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:09:14.718 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-11-27T15:09:14.718 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-11-27T15:09:14.718 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 0 2023-11-27T15:09:14.718 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:09:14.718 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-11-27T15:09:14.718 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "a" 2023-11-27T15:09:14.718 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:09:14.718 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2023-11-27T15:09:14.719 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "monmap": { 2023-11-27T15:09:14.719 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:09:14.719 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-11-27T15:09:14.719 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-11-27T15:09:14.719 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:09:14.719 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-11-27T15:09:14.719 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:09:14.719 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-11-27T15:09:14.719 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-11-27T15:09:14.719 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-11-27T15:09:14.719 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-11-27T15:09:14.720 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-11-27T15:09:14.720 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-11-27T15:09:14.720 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:09:14.720 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-11-27T15:09:14.720 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-11-27T15:09:14.720 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-11-27T15:09:14.720 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-11-27T15:09:14.720 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-11-27T15:09:14.720 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-11-27T15:09:14.720 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-11-27T15:09:14.720 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-11-27T15:09:14.721 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-11-27T15:09:14.721 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:09:14.721 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-11-27T15:09:14.721 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:09:14.721 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-11-27T15:09:14.721 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-11-27T15:09:14.721 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:09:14.721 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-11-27T15:09:14.721 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "available": false, 2023-11-27T15:09:14.721 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-11-27T15:09:14.721 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modules": [ 2023-11-27T15:09:14.721 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "iostat", 2023-11-27T15:09:14.721 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "nfs", 2023-11-27T15:09:14.722 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "restful" 2023-11-27T15:09:14.722 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:09:14.722 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-27T15:09:14.722 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:09:14.722 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-11-27T15:09:14.722 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:09:14.722 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modified": "2023-11-27T15:09:03.333886+0000", 2023-11-27T15:09:14.722 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-27T15:09:14.722 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:09:14.722 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-11-27T15:09:14.722 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2023-11-27T15:09:14.722 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.168+0000 7fa005c5b700 1 Processor -- start 2023-11-27T15:09:14.723 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.169+0000 7fa005c5b700 1 -- start start 2023-11-27T15:09:14.723 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.169+0000 7fa005c5b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa000106490 0x7fa000108870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:14.723 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.169+0000 7fa005c5b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa000079e60 con 0x7fa000106490 2023-11-27T15:09:14.723 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.169+0000 7f9fff7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa000106490 0x7fa000108870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:14.723 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.169+0000 7f9fff7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa000106490 0x7fa000108870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48712/0 (socket says 172.21.15.52:48712) 2023-11-27T15:09:14.723 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.169+0000 7f9fff7fe700 1 -- 172.21.15.52:0/891960421 learned_addr learned my addr 172.21.15.52:0/891960421 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:14.723 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.169+0000 7f9fff7fe700 1 -- 172.21.15.52:0/891960421 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa000108db0 con 0x7fa000106490 2023-11-27T15:09:14.723 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.170+0000 7f9fff7fe700 1 --2- 172.21.15.52:0/891960421 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa000106490 0x7fa000108870 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00098a0 tx=0x7f9ff0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3435672d6b3cd481 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:14.723 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.170+0000 7f9ffe7fc700 1 -- 172.21.15.52:0/891960421 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ff0004160 con 0x7fa000106490 2023-11-27T15:09:14.723 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.170+0000 7f9ffe7fc700 1 -- 172.21.15.52:0/891960421 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9ff00042c0 con 0x7fa000106490 2023-11-27T15:09:14.723 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.170+0000 7f9ffe7fc700 1 -- 172.21.15.52:0/891960421 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ff0011a00 con 0x7fa000106490 2023-11-27T15:09:14.723 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.171+0000 7fa005c5b700 1 -- 172.21.15.52:0/891960421 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa000106490 msgr2=0x7fa000108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:14.724 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.171+0000 7fa005c5b700 1 --2- 172.21.15.52:0/891960421 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa000106490 0x7fa000108870 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00098a0 tx=0x7f9ff0009bb0 comp rx=0 tx=0).stop 2023-11-27T15:09:14.724 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.171+0000 7fa005c5b700 1 -- 172.21.15.52:0/891960421 shutdown_connections 2023-11-27T15:09:14.724 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.171+0000 7fa005c5b700 1 --2- 172.21.15.52:0/891960421 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa000106490 0x7fa000108870 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:14.724 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.171+0000 7fa005c5b700 1 -- 172.21.15.52:0/891960421 >> 172.21.15.52:0/891960421 conn(0x7fa0000ffed0 msgr2=0x7fa000102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:14.724 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.171+0000 7fa005c5b700 1 -- 172.21.15.52:0/891960421 shutdown_connections 2023-11-27T15:09:14.724 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.171+0000 7fa005c5b700 1 -- 172.21.15.52:0/891960421 wait complete. 2023-11-27T15:09:14.724 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.171+0000 7fa005c5b700 1 Processor -- start 2023-11-27T15:09:14.724 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.172+0000 7fa005c5b700 1 -- start start 2023-11-27T15:09:14.724 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.172+0000 7fa005c5b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa000106490 0x7fa000105df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:14.724 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.172+0000 7fa005c5b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa000106330 con 0x7fa000106490 2023-11-27T15:09:14.724 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.172+0000 7f9fff7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa000106490 0x7fa000105df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:14.724 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.172+0000 7f9fff7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa000106490 0x7fa000105df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48714/0 (socket says 172.21.15.52:48714) 2023-11-27T15:09:14.724 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.172+0000 7f9fff7fe700 1 -- 172.21.15.52:0/3686043960 learned_addr learned my addr 172.21.15.52:0/3686043960 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:14.725 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.172+0000 7f9fff7fe700 1 -- 172.21.15.52:0/3686043960 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff0009550 con 0x7fa000106490 2023-11-27T15:09:14.725 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.172+0000 7f9fff7fe700 1 --2- 172.21.15.52:0/3686043960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa000106490 0x7fa000105df0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0006af0 tx=0x7f9ff0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:14.725 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.173+0000 7f9ffcff9700 1 -- 172.21.15.52:0/3686043960 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ff0003a40 con 0x7fa000106490 2023-11-27T15:09:14.725 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.173+0000 7f9ffcff9700 1 -- 172.21.15.52:0/3686043960 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9ff0003ba0 con 0x7fa000106490 2023-11-27T15:09:14.725 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.173+0000 7fa005c5b700 1 -- 172.21.15.52:0/3686043960 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa000109d40 con 0x7fa000106490 2023-11-27T15:09:14.725 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.173+0000 7f9ffcff9700 1 -- 172.21.15.52:0/3686043960 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ff001a6e0 con 0x7fa000106490 2023-11-27T15:09:14.725 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.173+0000 7fa005c5b700 1 -- 172.21.15.52:0/3686043960 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa000109bb0 con 0x7fa000106490 2023-11-27T15:09:14.725 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.173+0000 7f9ffcff9700 1 -- 172.21.15.52:0/3686043960 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9ff001ab60 con 0x7fa000106490 2023-11-27T15:09:14.725 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.173+0000 7f9ffcff9700 1 -- 172.21.15.52:0/3686043960 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9ff0019700 con 0x7fa000106490 2023-11-27T15:09:14.725 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.174+0000 7fa005c5b700 1 -- 172.21.15.52:0/3686043960 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa000062120 con 0x7fa000106490 2023-11-27T15:09:14.725 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.176+0000 7f9ffcff9700 1 -- 172.21.15.52:0/3686043960 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f9ff002dde0 con 0x7fa000106490 2023-11-27T15:09:14.726 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.314+0000 7fa005c5b700 1 -- 172.21.15.52:0/3686043960 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fa000105260 con 0x7fa000106490 2023-11-27T15:09:14.726 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.315+0000 7f9ffcff9700 1 -- 172.21.15.52:0/3686043960 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f9ff0024a60 con 0x7fa000106490 2023-11-27T15:09:14.726 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.316+0000 7fa005c5b700 1 -- 172.21.15.52:0/3686043960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa000106490 msgr2=0x7fa000105df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:14.726 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.316+0000 7fa005c5b700 1 --2- 172.21.15.52:0/3686043960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa000106490 0x7fa000105df0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0006af0 tx=0x7f9ff0011ad0 comp rx=0 tx=0).stop 2023-11-27T15:09:14.726 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.316+0000 7fa005c5b700 1 -- 172.21.15.52:0/3686043960 shutdown_connections 2023-11-27T15:09:14.726 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.316+0000 7fa005c5b700 1 --2- 172.21.15.52:0/3686043960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa000106490 0x7fa000105df0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:14.726 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.316+0000 7fa005c5b700 1 -- 172.21.15.52:0/3686043960 >> 172.21.15.52:0/3686043960 conn(0x7fa0000ffed0 msgr2=0x7fa000100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:14.726 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.316+0000 7fa005c5b700 1 -- 172.21.15.52:0/3686043960 shutdown_connections 2023-11-27T15:09:14.726 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:14.316+0000 7fa005c5b700 1 -- 172.21.15.52:0/3686043960 wait complete. 2023-11-27T15:09:14.726 INFO:teuthology.orchestra.run.smithi052.stdout:mgr not available, waiting (2/15)... 2023-11-27T15:09:15.087 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:14 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:14.735+0000 7f22b6881200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:09:15.337 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:15 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:15.081+0000 7f22b6881200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-27T15:09:15.617 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:15 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:15.331+0000 7f22b6881200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-27T15:09:15.617 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:15 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:15.512+0000 7f22b6881200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-27T15:09:16.117 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:15 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:15.838+0000 7f22b6881200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-27T15:09:16.957 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:16 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:16.656+0000 7f22b6881200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-27T15:09:16.957 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:16 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:16.815+0000 7f22b6881200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-27T15:09:17.207 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:17 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:17.062+0000 7f22b6881200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-27T15:09:17.485 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:17 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3801781779' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-11-27T15:09:17.485 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:17 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:17.201+0000 7f22b6881200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-27T15:09:17.798 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:17 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:17.479+0000 7f22b6881200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-27T15:09:17.867 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2023-11-27T15:09:17.867 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2023-11-27T15:09:17.867 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsid": "c6c7df88-8d36-11ee-95a2-87774f69a715", 2023-11-27T15:09:17.867 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "health": { 2023-11-27T15:09:17.867 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-11-27T15:09:17.867 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-11-27T15:09:17.867 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-11-27T15:09:17.867 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:09:17.868 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-11-27T15:09:17.868 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-11-27T15:09:17.868 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 0 2023-11-27T15:09:17.868 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:09:17.868 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-11-27T15:09:17.868 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "a" 2023-11-27T15:09:17.868 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:09:17.868 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2023-11-27T15:09:17.868 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "monmap": { 2023-11-27T15:09:17.868 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:09:17.868 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-11-27T15:09:17.868 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-11-27T15:09:17.869 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:09:17.869 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-11-27T15:09:17.869 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:09:17.869 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-11-27T15:09:17.869 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-11-27T15:09:17.869 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-11-27T15:09:17.869 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-11-27T15:09:17.869 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-11-27T15:09:17.869 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-11-27T15:09:17.869 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:09:17.869 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-11-27T15:09:17.869 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-11-27T15:09:17.870 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-11-27T15:09:17.870 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-11-27T15:09:17.870 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-11-27T15:09:17.870 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-11-27T15:09:17.870 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-11-27T15:09:17.870 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-11-27T15:09:17.870 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-11-27T15:09:17.870 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:09:17.870 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-11-27T15:09:17.870 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:09:17.870 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-11-27T15:09:17.870 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-11-27T15:09:17.871 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:09:17.871 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-11-27T15:09:17.871 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "available": false, 2023-11-27T15:09:17.871 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-11-27T15:09:17.871 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modules": [ 2023-11-27T15:09:17.871 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "iostat", 2023-11-27T15:09:17.871 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "nfs", 2023-11-27T15:09:17.871 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "restful" 2023-11-27T15:09:17.871 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:09:17.871 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-27T15:09:17.871 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:09:17.871 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-11-27T15:09:17.871 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:09:17.872 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modified": "2023-11-27T15:09:03.333886+0000", 2023-11-27T15:09:17.872 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-27T15:09:17.872 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:09:17.872 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-11-27T15:09:17.872 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2023-11-27T15:09:17.872 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.240+0000 7fc5735cd700 1 Processor -- start 2023-11-27T15:09:17.872 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.241+0000 7fc5735cd700 1 -- start start 2023-11-27T15:09:17.872 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.241+0000 7fc5735cd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc56c104280 0x7fc56c104690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:17.872 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.241+0000 7fc5735cd700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c079e60 con 0x7fc56c104280 2023-11-27T15:09:17.872 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.241+0000 7fc571369700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc56c104280 0x7fc56c104690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:17.872 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.241+0000 7fc571369700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc56c104280 0x7fc56c104690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48716/0 (socket says 172.21.15.52:48716) 2023-11-27T15:09:17.872 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.241+0000 7fc571369700 1 -- 172.21.15.52:0/1068720248 learned_addr learned my addr 172.21.15.52:0/1068720248 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:17.873 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.241+0000 7fc571369700 1 -- 172.21.15.52:0/1068720248 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc56c104bd0 con 0x7fc56c104280 2023-11-27T15:09:17.873 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.241+0000 7fc571369700 1 --2- 172.21.15.52:0/1068720248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc56c104280 0x7fc56c104690 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc5680098a0 tx=0x7fc568009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8586bade8cfcb0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:17.873 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.242+0000 7fc563fff700 1 -- 172.21.15.52:0/1068720248 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc568004160 con 0x7fc56c104280 2023-11-27T15:09:17.873 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.242+0000 7fc563fff700 1 -- 172.21.15.52:0/1068720248 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc5680042c0 con 0x7fc56c104280 2023-11-27T15:09:17.873 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.242+0000 7fc563fff700 1 -- 172.21.15.52:0/1068720248 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc568011a00 con 0x7fc56c104280 2023-11-27T15:09:17.873 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.242+0000 7fc5735cd700 1 -- 172.21.15.52:0/1068720248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc56c104280 msgr2=0x7fc56c104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:17.873 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.242+0000 7fc5735cd700 1 --2- 172.21.15.52:0/1068720248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc56c104280 0x7fc56c104690 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc5680098a0 tx=0x7fc568009bb0 comp rx=0 tx=0).stop 2023-11-27T15:09:17.873 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.242+0000 7fc5735cd700 1 -- 172.21.15.52:0/1068720248 shutdown_connections 2023-11-27T15:09:17.873 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.242+0000 7fc5735cd700 1 --2- 172.21.15.52:0/1068720248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc56c104280 0x7fc56c104690 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:17.873 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.242+0000 7fc5735cd700 1 -- 172.21.15.52:0/1068720248 >> 172.21.15.52:0/1068720248 conn(0x7fc56c0ffed0 msgr2=0x7fc56c102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:17.873 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.242+0000 7fc5735cd700 1 -- 172.21.15.52:0/1068720248 shutdown_connections 2023-11-27T15:09:17.873 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.242+0000 7fc5735cd700 1 -- 172.21.15.52:0/1068720248 wait complete. 2023-11-27T15:09:17.874 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.243+0000 7fc5735cd700 1 Processor -- start 2023-11-27T15:09:17.874 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.243+0000 7fc5735cd700 1 -- start start 2023-11-27T15:09:17.874 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.243+0000 7fc5735cd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc56c104280 0x7fc56c19b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:17.874 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.243+0000 7fc5735cd700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c19be70 con 0x7fc56c104280 2023-11-27T15:09:17.874 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.243+0000 7fc571369700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc56c104280 0x7fc56c19b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:17.874 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.243+0000 7fc571369700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc56c104280 0x7fc56c19b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48718/0 (socket says 172.21.15.52:48718) 2023-11-27T15:09:17.874 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.243+0000 7fc571369700 1 -- 172.21.15.52:0/3801781779 learned_addr learned my addr 172.21.15.52:0/3801781779 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:17.874 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.244+0000 7fc571369700 1 -- 172.21.15.52:0/3801781779 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc568009550 con 0x7fc56c104280 2023-11-27T15:09:17.874 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.244+0000 7fc571369700 1 --2- 172.21.15.52:0/3801781779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc56c104280 0x7fc56c19b930 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc568009fb0 tx=0x7fc568011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:17.874 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.244+0000 7fc5627fc700 1 -- 172.21.15.52:0/3801781779 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc568003930 con 0x7fc56c104280 2023-11-27T15:09:17.874 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.244+0000 7fc5627fc700 1 -- 172.21.15.52:0/3801781779 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc568003a90 con 0x7fc56c104280 2023-11-27T15:09:17.874 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.244+0000 7fc5627fc700 1 -- 172.21.15.52:0/3801781779 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc568019610 con 0x7fc56c104280 2023-11-27T15:09:17.875 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.244+0000 7fc5735cd700 1 -- 172.21.15.52:0/3801781779 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc56c105bd0 con 0x7fc56c104280 2023-11-27T15:09:17.875 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.244+0000 7fc5735cd700 1 -- 172.21.15.52:0/3801781779 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc56c105a40 con 0x7fc56c104280 2023-11-27T15:09:17.875 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.244+0000 7fc5627fc700 1 -- 172.21.15.52:0/3801781779 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc568019a90 con 0x7fc56c104280 2023-11-27T15:09:17.875 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.244+0000 7fc5627fc700 1 -- 172.21.15.52:0/3801781779 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc5680243f0 con 0x7fc56c104280 2023-11-27T15:09:17.875 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.245+0000 7fc5735cd700 1 -- 172.21.15.52:0/3801781779 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc56c105d60 con 0x7fc56c104280 2023-11-27T15:09:17.875 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.247+0000 7fc5627fc700 1 -- 172.21.15.52:0/3801781779 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7fc568003c00 con 0x7fc56c104280 2023-11-27T15:09:17.875 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.382+0000 7fc5735cd700 1 -- 172.21.15.52:0/3801781779 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc56c195210 con 0x7fc56c104280 2023-11-27T15:09:17.875 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.382+0000 7fc5627fc700 1 -- 172.21.15.52:0/3801781779 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fc56801a9d0 con 0x7fc56c104280 2023-11-27T15:09:17.875 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.383+0000 7fc5735cd700 1 -- 172.21.15.52:0/3801781779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc56c104280 msgr2=0x7fc56c19b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:17.875 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.383+0000 7fc5735cd700 1 --2- 172.21.15.52:0/3801781779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc56c104280 0x7fc56c19b930 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc568009fb0 tx=0x7fc568011ad0 comp rx=0 tx=0).stop 2023-11-27T15:09:17.875 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.383+0000 7fc5735cd700 1 -- 172.21.15.52:0/3801781779 shutdown_connections 2023-11-27T15:09:17.875 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.383+0000 7fc5735cd700 1 --2- 172.21.15.52:0/3801781779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc56c104280 0x7fc56c19b930 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:17.876 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.383+0000 7fc5735cd700 1 -- 172.21.15.52:0/3801781779 >> 172.21.15.52:0/3801781779 conn(0x7fc56c0ffed0 msgr2=0x7fc56c18fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:17.876 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.383+0000 7fc5735cd700 1 -- 172.21.15.52:0/3801781779 shutdown_connections 2023-11-27T15:09:17.876 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:17.383+0000 7fc5735cd700 1 -- 172.21.15.52:0/3801781779 wait complete. 2023-11-27T15:09:17.876 INFO:teuthology.orchestra.run.smithi052.stdout:mgr not available, waiting (3/15)... 2023-11-27T15:09:18.868 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:18 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:18.515+0000 7f22b6881200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-27T15:09:19.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:19 smithi052 ceph-mon[99056]: Activating manager daemon x 2023-11-27T15:09:19.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:19 smithi052 ceph-mon[99056]: mgrmap e2: x(active, starting, since 0.00317988s) 2023-11-27T15:09:19.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:19 smithi052 ceph-mon[99056]: from='mgr.14100 172.21.15.52:0/695799515' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-11-27T15:09:19.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:19 smithi052 ceph-mon[99056]: from='mgr.14100 172.21.15.52:0/695799515' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-11-27T15:09:19.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:19 smithi052 ceph-mon[99056]: from='mgr.14100 172.21.15.52:0/695799515' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-11-27T15:09:19.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:19 smithi052 ceph-mon[99056]: from='mgr.14100 172.21.15.52:0/695799515' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:09:19.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:19 smithi052 ceph-mon[99056]: from='mgr.14100 172.21.15.52:0/695799515' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-11-27T15:09:19.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:19 smithi052 ceph-mon[99056]: Manager daemon x is now available 2023-11-27T15:09:19.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:19 smithi052 ceph-mon[99056]: from='mgr.14100 172.21.15.52:0/695799515' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-27T15:09:19.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:19 smithi052 ceph-mon[99056]: from='mgr.14100 172.21.15.52:0/695799515' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-27T15:09:19.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:19 smithi052 ceph-mon[99056]: from='mgr.14100 172.21.15.52:0/695799515' entity='mgr.x' 2023-11-27T15:09:19.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:19 smithi052 ceph-mon[99056]: from='mgr.14100 172.21.15.52:0/695799515' entity='mgr.x' 2023-11-27T15:09:19.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:19 smithi052 ceph-mon[99056]: from='mgr.14100 172.21.15.52:0/695799515' entity='mgr.x' 2023-11-27T15:09:20.792 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:20 smithi052 ceph-mon[99056]: mgrmap e3: x(active, since 1.00741s) 2023-11-27T15:09:22.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:22 smithi052 ceph-mon[99056]: mgrmap e4: x(active, since 3s) 2023-11-27T15:09:22.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:22 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2112534780' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-11-27T15:09:24.725 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2023-11-27T15:09:24.725 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2023-11-27T15:09:24.725 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsid": "c6c7df88-8d36-11ee-95a2-87774f69a715", 2023-11-27T15:09:24.726 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "health": { 2023-11-27T15:09:24.726 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-11-27T15:09:24.726 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-11-27T15:09:24.726 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-11-27T15:09:24.726 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:09:24.726 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-11-27T15:09:24.726 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-11-27T15:09:24.726 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 0 2023-11-27T15:09:24.726 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:09:24.726 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-11-27T15:09:24.726 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "a" 2023-11-27T15:09:24.726 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:09:24.727 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2023-11-27T15:09:24.727 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "monmap": { 2023-11-27T15:09:24.727 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:09:24.727 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-11-27T15:09:24.727 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-11-27T15:09:24.727 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:09:24.727 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-11-27T15:09:24.727 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:09:24.727 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-11-27T15:09:24.727 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-11-27T15:09:24.727 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-11-27T15:09:24.735 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-11-27T15:09:24.735 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-11-27T15:09:24.735 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-11-27T15:09:24.735 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:09:24.735 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-11-27T15:09:24.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-11-27T15:09:24.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-11-27T15:09:24.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-11-27T15:09:24.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-11-27T15:09:24.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-11-27T15:09:24.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-11-27T15:09:24.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-11-27T15:09:24.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-11-27T15:09:24.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:09:24.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-11-27T15:09:24.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:09:24.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-11-27T15:09:24.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-11-27T15:09:24.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:09:24.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-11-27T15:09:24.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "available": true, 2023-11-27T15:09:24.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-11-27T15:09:24.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modules": [ 2023-11-27T15:09:24.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "iostat", 2023-11-27T15:09:24.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "nfs", 2023-11-27T15:09:24.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "restful" 2023-11-27T15:09:24.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:09:24.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-27T15:09:24.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:09:24.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-11-27T15:09:24.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:09:24.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modified": "2023-11-27T15:09:03.333886+0000", 2023-11-27T15:09:24.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-27T15:09:24.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:09:24.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-11-27T15:09:24.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2023-11-27T15:09:24.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.747+0000 7f3696bbe700 1 Processor -- start 2023-11-27T15:09:24.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.747+0000 7f3696bbe700 1 -- start start 2023-11-27T15:09:24.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.747+0000 7f3696bbe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3690104920 0x7f3690104d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:24.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.747+0000 7f3696bbe700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3690105300 con 0x7f3690104920 2023-11-27T15:09:24.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.747+0000 7f369495a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3690104920 0x7f3690104d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:24.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.747+0000 7f369495a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3690104920 0x7f3690104d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48744/0 (socket says 172.21.15.52:48744) 2023-11-27T15:09:24.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.747+0000 7f369495a700 1 -- 172.21.15.52:0/3951530313 learned_addr learned my addr 172.21.15.52:0/3951530313 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:24.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.747+0000 7f369495a700 1 -- 172.21.15.52:0/3951530313 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3690105b20 con 0x7f3690104920 2023-11-27T15:09:24.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.748+0000 7f369495a700 1 --2- 172.21.15.52:0/3951530313 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3690104920 0x7f3690104d30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f368400bb40 tx=0x7f368400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17168acf358ca7d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:24.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.748+0000 7f368f7fe700 1 -- 172.21.15.52:0/3951530313 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f368400dbe0 con 0x7f3690104920 2023-11-27T15:09:24.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.748+0000 7f368f7fe700 1 -- 172.21.15.52:0/3951530313 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f36840044d0 con 0x7f3690104920 2023-11-27T15:09:24.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.748+0000 7f368f7fe700 1 -- 172.21.15.52:0/3951530313 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3684010470 con 0x7f3690104920 2023-11-27T15:09:24.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.748+0000 7f3696bbe700 1 -- 172.21.15.52:0/3951530313 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3690104920 msgr2=0x7f3690104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:24.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.748+0000 7f3696bbe700 1 --2- 172.21.15.52:0/3951530313 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3690104920 0x7f3690104d30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f368400bb40 tx=0x7f368400be50 comp rx=0 tx=0).stop 2023-11-27T15:09:24.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.748+0000 7f3696bbe700 1 -- 172.21.15.52:0/3951530313 shutdown_connections 2023-11-27T15:09:24.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.748+0000 7f3696bbe700 1 --2- 172.21.15.52:0/3951530313 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3690104920 0x7f3690104d30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:24.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.748+0000 7f3696bbe700 1 -- 172.21.15.52:0/3951530313 >> 172.21.15.52:0/3951530313 conn(0x7f36900ffed0 msgr2=0x7f3690102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:24.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.748+0000 7f3696bbe700 1 -- 172.21.15.52:0/3951530313 shutdown_connections 2023-11-27T15:09:24.740 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.749+0000 7f3696bbe700 1 -- 172.21.15.52:0/3951530313 wait complete. 2023-11-27T15:09:24.740 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.749+0000 7f3696bbe700 1 Processor -- start 2023-11-27T15:09:24.740 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.749+0000 7f3696bbe700 1 -- start start 2023-11-27T15:09:24.740 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.749+0000 7f3696bbe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3690104920 0x7f369019b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:24.740 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.749+0000 7f3696bbe700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369019bee0 con 0x7f3690104920 2023-11-27T15:09:24.740 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.749+0000 7f369495a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3690104920 0x7f369019b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:24.740 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.749+0000 7f369495a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3690104920 0x7f369019b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48746/0 (socket says 172.21.15.52:48746) 2023-11-27T15:09:24.740 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.750+0000 7f369495a700 1 -- 172.21.15.52:0/2112534780 learned_addr learned my addr 172.21.15.52:0/2112534780 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:24.740 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.750+0000 7f369495a700 1 -- 172.21.15.52:0/2112534780 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368400b7f0 con 0x7f3690104920 2023-11-27T15:09:24.740 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.750+0000 7f369495a700 1 --2- 172.21.15.52:0/2112534780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3690104920 0x7f369019b9a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f368400b340 tx=0x7f3684003df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:24.740 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.750+0000 7f368dffb700 1 -- 172.21.15.52:0/2112534780 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3684004310 con 0x7f3690104920 2023-11-27T15:09:24.740 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.750+0000 7f368dffb700 1 -- 172.21.15.52:0/2112534780 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3684010bd0 con 0x7f3690104920 2023-11-27T15:09:24.741 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.750+0000 7f3696bbe700 1 -- 172.21.15.52:0/2112534780 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3690078f80 con 0x7f3690104920 2023-11-27T15:09:24.741 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.750+0000 7f368dffb700 1 -- 172.21.15.52:0/2112534780 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3684021c90 con 0x7f3690104920 2023-11-27T15:09:24.741 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.750+0000 7f3696bbe700 1 -- 172.21.15.52:0/2112534780 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36901068d0 con 0x7f3690104920 2023-11-27T15:09:24.741 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.751+0000 7f368dffb700 1 -- 172.21.15.52:0/2112534780 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f36840193f0 con 0x7f3690104920 2023-11-27T15:09:24.741 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.751+0000 7f368dffb700 1 --2- 172.21.15.52:0/2112534780 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7f3680038670 0x7f368003ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:24.741 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.751+0000 7f368dffb700 1 -- 172.21.15.52:0/2112534780 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f368404c800 con 0x7f3690104920 2023-11-27T15:09:24.741 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.751+0000 7f3696bbe700 1 -- 172.21.15.52:0/2112534780 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36901911b0 con 0x7f3690104920 2023-11-27T15:09:24.741 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.751+0000 7f368ffff700 1 --2- 172.21.15.52:0/2112534780 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7f3680038670 0x7f368003ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:24.741 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.752+0000 7f368ffff700 1 --2- 172.21.15.52:0/2112534780 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7f3680038670 0x7f368003ab20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f367c006fd0 tx=0x7f367c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:24.741 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:21.757+0000 7f368dffb700 1 -- 172.21.15.52:0/2112534780 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f368400e930 con 0x7f3690104920 2023-11-27T15:09:24.741 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:22.276+0000 7f3696bbe700 1 -- 172.21.15.52:0/2112534780 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f369019c610 con 0x7f3690104920 2023-11-27T15:09:24.741 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:22.276+0000 7f368dffb700 1 -- 172.21.15.52:0/2112534780 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f368401d220 con 0x7f3690104920 2023-11-27T15:09:24.742 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:22.279+0000 7f3696bbe700 1 -- 172.21.15.52:0/2112534780 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7f3680038670 msgr2=0x7f368003ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:24.742 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:22.279+0000 7f3696bbe700 1 --2- 172.21.15.52:0/2112534780 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7f3680038670 0x7f368003ab20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f367c006fd0 tx=0x7f367c006e40 comp rx=0 tx=0).stop 2023-11-27T15:09:24.742 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:22.279+0000 7f3696bbe700 1 -- 172.21.15.52:0/2112534780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3690104920 msgr2=0x7f369019b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:24.742 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:22.279+0000 7f3696bbe700 1 --2- 172.21.15.52:0/2112534780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3690104920 0x7f369019b9a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f368400b340 tx=0x7f3684003df0 comp rx=0 tx=0).stop 2023-11-27T15:09:24.742 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:22.279+0000 7f3696bbe700 1 -- 172.21.15.52:0/2112534780 shutdown_connections 2023-11-27T15:09:24.742 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:22.279+0000 7f3696bbe700 1 --2- 172.21.15.52:0/2112534780 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7f3680038670 0x7f368003ab20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:24.742 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:22.279+0000 7f3696bbe700 1 --2- 172.21.15.52:0/2112534780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3690104920 0x7f369019b9a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:24.742 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:22.279+0000 7f3696bbe700 1 -- 172.21.15.52:0/2112534780 >> 172.21.15.52:0/2112534780 conn(0x7f36900ffed0 msgr2=0x7f369018e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:24.742 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:22.279+0000 7f3696bbe700 1 -- 172.21.15.52:0/2112534780 shutdown_connections 2023-11-27T15:09:24.742 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:22.279+0000 7f3696bbe700 1 -- 172.21.15.52:0/2112534780 wait complete. 2023-11-27T15:09:24.742 INFO:teuthology.orchestra.run.smithi052.stdout:mgr is available 2023-11-27T15:09:27.480 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:27 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/163633154' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-11-27T15:09:27.490 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2023-11-27T15:09:27.490 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout [global] 2023-11-27T15:09:27.490 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout fsid = c6c7df88-8d36-11ee-95a2-87774f69a715 2023-11-27T15:09:27.490 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-11-27T15:09:27.491 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-11-27T15:09:27.491 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-11-27T15:09:27.491 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-11-27T15:09:27.491 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2023-11-27T15:09:27.491 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout [mgr] 2023-11-27T15:09:27.491 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-11-27T15:09:27.491 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2023-11-27T15:09:27.491 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout [osd] 2023-11-27T15:09:27.491 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-11-27T15:09:27.491 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-11-27T15:09:27.491 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-11-27T15:09:27.492 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-11-27T15:09:27.492 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-11-27T15:09:27.492 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.719+0000 7fa30b38e700 1 Processor -- start 2023-11-27T15:09:27.492 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.719+0000 7fa30b38e700 1 -- start start 2023-11-27T15:09:27.492 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.720+0000 7fa30b38e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa304104920 0x7fa304104d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:27.492 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.720+0000 7fa30b38e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa304105300 con 0x7fa304104920 2023-11-27T15:09:27.492 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.720+0000 7fa30912a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa304104920 0x7fa304104d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:27.492 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.720+0000 7fa30912a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa304104920 0x7fa304104d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48750/0 (socket says 172.21.15.52:48750) 2023-11-27T15:09:27.492 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.720+0000 7fa30912a700 1 -- 172.21.15.52:0/2353255786 learned_addr learned my addr 172.21.15.52:0/2353255786 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:27.492 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.720+0000 7fa30912a700 1 -- 172.21.15.52:0/2353255786 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa304105b20 con 0x7fa304104920 2023-11-27T15:09:27.492 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.720+0000 7fa30912a700 1 --2- 172.21.15.52:0/2353255786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa304104920 0x7fa304104d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa30000bb40 tx=0x7fa30000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6de5c31458ead5 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:27.492 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.720+0000 7fa2fbfff700 1 -- 172.21.15.52:0/2353255786 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa30000dbe0 con 0x7fa304104920 2023-11-27T15:09:27.492 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.720+0000 7fa2fbfff700 1 -- 172.21.15.52:0/2353255786 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa3000044d0 con 0x7fa304104920 2023-11-27T15:09:27.493 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.720+0000 7fa2fbfff700 1 -- 172.21.15.52:0/2353255786 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa300010470 con 0x7fa304104920 2023-11-27T15:09:27.493 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.721+0000 7fa30b38e700 1 -- 172.21.15.52:0/2353255786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa304104920 msgr2=0x7fa304104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:27.493 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.721+0000 7fa30b38e700 1 --2- 172.21.15.52:0/2353255786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa304104920 0x7fa304104d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa30000bb40 tx=0x7fa30000be50 comp rx=0 tx=0).stop 2023-11-27T15:09:27.493 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.721+0000 7fa30b38e700 1 -- 172.21.15.52:0/2353255786 shutdown_connections 2023-11-27T15:09:27.493 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.721+0000 7fa30b38e700 1 --2- 172.21.15.52:0/2353255786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa304104920 0x7fa304104d30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:27.493 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.721+0000 7fa30b38e700 1 -- 172.21.15.52:0/2353255786 >> 172.21.15.52:0/2353255786 conn(0x7fa3040ffed0 msgr2=0x7fa304102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:27.493 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.721+0000 7fa30b38e700 1 -- 172.21.15.52:0/2353255786 shutdown_connections 2023-11-27T15:09:27.493 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.721+0000 7fa30b38e700 1 -- 172.21.15.52:0/2353255786 wait complete. 2023-11-27T15:09:27.493 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.722+0000 7fa30b38e700 1 Processor -- start 2023-11-27T15:09:27.493 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.722+0000 7fa30b38e700 1 -- start start 2023-11-27T15:09:27.493 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.722+0000 7fa30b38e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa304104920 0x7fa30419b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:27.494 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.722+0000 7fa30b38e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30419bee0 con 0x7fa304104920 2023-11-27T15:09:27.494 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.722+0000 7fa30912a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa304104920 0x7fa30419b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:27.494 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.722+0000 7fa30912a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa304104920 0x7fa30419b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48752/0 (socket says 172.21.15.52:48752) 2023-11-27T15:09:27.494 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.722+0000 7fa30912a700 1 -- 172.21.15.52:0/163633154 learned_addr learned my addr 172.21.15.52:0/163633154 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:27.494 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.722+0000 7fa30912a700 1 -- 172.21.15.52:0/163633154 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa30000b7f0 con 0x7fa304104920 2023-11-27T15:09:27.494 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.722+0000 7fa30912a700 1 --2- 172.21.15.52:0/163633154 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa304104920 0x7fa30419b9a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa300003a80 tx=0x7fa300003df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:27.494 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.723+0000 7fa2fa7fc700 1 -- 172.21.15.52:0/163633154 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa3000106e0 con 0x7fa304104920 2023-11-27T15:09:27.494 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.723+0000 7fa2fa7fc700 1 -- 172.21.15.52:0/163633154 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa300010ce0 con 0x7fa304104920 2023-11-27T15:09:27.494 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.723+0000 7fa30b38e700 1 -- 172.21.15.52:0/163633154 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa304078f80 con 0x7fa304104920 2023-11-27T15:09:27.494 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.723+0000 7fa2fa7fc700 1 -- 172.21.15.52:0/163633154 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa300019c40 con 0x7fa304104920 2023-11-27T15:09:27.494 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.723+0000 7fa30b38e700 1 -- 172.21.15.52:0/163633154 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3041068d0 con 0x7fa304104920 2023-11-27T15:09:27.494 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.724+0000 7fa2fa7fc700 1 -- 172.21.15.52:0/163633154 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7fa30002a450 con 0x7fa304104920 2023-11-27T15:09:27.495 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.724+0000 7fa2fa7fc700 1 --2- 172.21.15.52:0/163633154 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7fa2f0038710 0x7fa2f003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:27.495 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.724+0000 7fa2fa7fc700 1 -- 172.21.15.52:0/163633154 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa30004caa0 con 0x7fa304104920 2023-11-27T15:09:27.495 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.724+0000 7fa30b38e700 1 -- 172.21.15.52:0/163633154 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa304079110 con 0x7fa304104920 2023-11-27T15:09:27.495 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.724+0000 7fa308929700 1 --2- 172.21.15.52:0/163633154 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7fa2f0038710 0x7fa2f003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:27.495 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.725+0000 7fa308929700 1 --2- 172.21.15.52:0/163633154 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7fa2f0038710 0x7fa2f003abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4006fd0 tx=0x7fa2f4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:27.495 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:26.729+0000 7fa2fa7fc700 1 -- 172.21.15.52:0/163633154 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa30002a8e0 con 0x7fa304104920 2023-11-27T15:09:27.495 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.090+0000 7fa30b38e700 1 -- 172.21.15.52:0/163633154 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fa3041911b0 con 0x7fa304104920 2023-11-27T15:09:27.495 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.091+0000 7fa2fa7fc700 1 -- 172.21.15.52:0/163633154 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fa300028350 con 0x7fa304104920 2023-11-27T15:09:27.495 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.093+0000 7fa30b38e700 1 -- 172.21.15.52:0/163633154 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7fa2f0038710 msgr2=0x7fa2f003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:27.495 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.093+0000 7fa30b38e700 1 --2- 172.21.15.52:0/163633154 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7fa2f0038710 0x7fa2f003abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4006fd0 tx=0x7fa2f4006e40 comp rx=0 tx=0).stop 2023-11-27T15:09:27.495 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.093+0000 7fa30b38e700 1 -- 172.21.15.52:0/163633154 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa304104920 msgr2=0x7fa30419b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:27.495 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.093+0000 7fa30b38e700 1 --2- 172.21.15.52:0/163633154 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa304104920 0x7fa30419b9a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa300003a80 tx=0x7fa300003df0 comp rx=0 tx=0).stop 2023-11-27T15:09:27.496 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.093+0000 7fa30b38e700 1 -- 172.21.15.52:0/163633154 shutdown_connections 2023-11-27T15:09:27.496 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.093+0000 7fa30b38e700 1 --2- 172.21.15.52:0/163633154 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7fa2f0038710 0x7fa2f003abc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:27.496 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.094+0000 7fa30b38e700 1 --2- 172.21.15.52:0/163633154 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa304104920 0x7fa30419b9a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:27.496 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.094+0000 7fa30b38e700 1 -- 172.21.15.52:0/163633154 >> 172.21.15.52:0/163633154 conn(0x7fa3040ffed0 msgr2=0x7fa30418e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:27.496 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.094+0000 7fa30b38e700 1 -- 172.21.15.52:0/163633154 shutdown_connections 2023-11-27T15:09:27.496 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.094+0000 7fa30b38e700 1 -- 172.21.15.52:0/163633154 wait complete. 2023-11-27T15:09:27.496 INFO:teuthology.orchestra.run.smithi052.stdout:Enabling cephadm module... 2023-11-27T15:09:29.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:29 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3574669119' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-11-27T15:09:29.367 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:29 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: ignoring --setuser ceph since I am not root 2023-11-27T15:09:29.367 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:29 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: ignoring --setgroup ceph since I am not root 2023-11-27T15:09:29.368 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:29 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:29.309+0000 7fbf2134a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-27T15:09:29.638 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.990+0000 7fd3af168700 1 Processor -- start 2023-11-27T15:09:29.638 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.990+0000 7fd3af168700 1 -- start start 2023-11-27T15:09:29.638 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.990+0000 7fd3af168700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3a8104920 0x7fd3a8104d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:29.638 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.990+0000 7fd3af168700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a8105300 con 0x7fd3a8104920 2023-11-27T15:09:29.638 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.991+0000 7fd3acf04700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3a8104920 0x7fd3a8104d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:29.638 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.991+0000 7fd3acf04700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3a8104920 0x7fd3a8104d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48756/0 (socket says 172.21.15.52:48756) 2023-11-27T15:09:29.639 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.991+0000 7fd3acf04700 1 -- 172.21.15.52:0/1448227882 learned_addr learned my addr 172.21.15.52:0/1448227882 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:29.639 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.991+0000 7fd3acf04700 1 -- 172.21.15.52:0/1448227882 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a8105b20 con 0x7fd3a8104920 2023-11-27T15:09:29.639 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.991+0000 7fd3acf04700 1 --2- 172.21.15.52:0/1448227882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3a8104920 0x7fd3a8104d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0098a0 tx=0x7fd39c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=848d93cec391b8b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:29.639 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.991+0000 7fd3a77fe700 1 -- 172.21.15.52:0/1448227882 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd39c004160 con 0x7fd3a8104920 2023-11-27T15:09:29.639 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.991+0000 7fd3a77fe700 1 -- 172.21.15.52:0/1448227882 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd39c0042c0 con 0x7fd3a8104920 2023-11-27T15:09:29.639 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.991+0000 7fd3a77fe700 1 -- 172.21.15.52:0/1448227882 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd39c011a00 con 0x7fd3a8104920 2023-11-27T15:09:29.639 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.992+0000 7fd3af168700 1 -- 172.21.15.52:0/1448227882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3a8104920 msgr2=0x7fd3a8104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:29.639 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.992+0000 7fd3af168700 1 --2- 172.21.15.52:0/1448227882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3a8104920 0x7fd3a8104d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0098a0 tx=0x7fd39c009bb0 comp rx=0 tx=0).stop 2023-11-27T15:09:29.639 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.992+0000 7fd3af168700 1 -- 172.21.15.52:0/1448227882 shutdown_connections 2023-11-27T15:09:29.639 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.992+0000 7fd3af168700 1 --2- 172.21.15.52:0/1448227882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3a8104920 0x7fd3a8104d30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:29.639 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.992+0000 7fd3af168700 1 -- 172.21.15.52:0/1448227882 >> 172.21.15.52:0/1448227882 conn(0x7fd3a80ffed0 msgr2=0x7fd3a8102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:29.639 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.992+0000 7fd3af168700 1 -- 172.21.15.52:0/1448227882 shutdown_connections 2023-11-27T15:09:29.640 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.992+0000 7fd3af168700 1 -- 172.21.15.52:0/1448227882 wait complete. 2023-11-27T15:09:29.640 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.993+0000 7fd3af168700 1 Processor -- start 2023-11-27T15:09:29.640 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.993+0000 7fd3af168700 1 -- start start 2023-11-27T15:09:29.640 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.993+0000 7fd3af168700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3a8104920 0x7fd3a8199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:29.640 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.993+0000 7fd3af168700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a8199eb0 con 0x7fd3a8104920 2023-11-27T15:09:29.640 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.993+0000 7fd3acf04700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3a8104920 0x7fd3a8199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:29.640 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.993+0000 7fd3acf04700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3a8104920 0x7fd3a8199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48758/0 (socket says 172.21.15.52:48758) 2023-11-27T15:09:29.640 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.993+0000 7fd3acf04700 1 -- 172.21.15.52:0/3574669119 learned_addr learned my addr 172.21.15.52:0/3574669119 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:29.640 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.993+0000 7fd3acf04700 1 -- 172.21.15.52:0/3574669119 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd39c009550 con 0x7fd3a8104920 2023-11-27T15:09:29.640 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.993+0000 7fd3acf04700 1 --2- 172.21.15.52:0/3574669119 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3a8104920 0x7fd3a8199970 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd39c009fb0 tx=0x7fd39c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:29.640 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.994+0000 7fd3a5ffb700 1 -- 172.21.15.52:0/3574669119 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd39c003930 con 0x7fd3a8104920 2023-11-27T15:09:29.641 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.994+0000 7fd3a5ffb700 1 -- 172.21.15.52:0/3574669119 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd39c003a90 con 0x7fd3a8104920 2023-11-27T15:09:29.641 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.994+0000 7fd3af168700 1 -- 172.21.15.52:0/3574669119 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a8108d50 con 0x7fd3a8104920 2023-11-27T15:09:29.641 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.994+0000 7fd3a5ffb700 1 -- 172.21.15.52:0/3574669119 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd39c0195e0 con 0x7fd3a8104920 2023-11-27T15:09:29.641 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.994+0000 7fd3af168700 1 -- 172.21.15.52:0/3574669119 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a8108ee0 con 0x7fd3a8104920 2023-11-27T15:09:29.641 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.995+0000 7fd3a5ffb700 1 -- 172.21.15.52:0/3574669119 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7fd39c0198a0 con 0x7fd3a8104920 2023-11-27T15:09:29.641 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.995+0000 7fd3a5ffb700 1 --2- 172.21.15.52:0/3574669119 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7fd398038670 0x7fd39803ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:29.641 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.995+0000 7fd3a5ffb700 1 -- 172.21.15.52:0/3574669119 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd39c04c600 con 0x7fd3a8104920 2023-11-27T15:09:29.641 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.995+0000 7fd3af168700 1 -- 172.21.15.52:0/3574669119 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3a8106990 con 0x7fd3a8104920 2023-11-27T15:09:29.641 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.995+0000 7fd3a7fff700 1 --2- 172.21.15.52:0/3574669119 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7fd398038670 0x7fd39803ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:29.641 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:27.995+0000 7fd3a7fff700 1 --2- 172.21.15.52:0/3574669119 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7fd398038670 0x7fd39803ab20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd394006fd0 tx=0x7fd394006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:29.641 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:28.000+0000 7fd3a5ffb700 1 -- 172.21.15.52:0/3574669119 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd3a8106990 con 0x7fd3a8104920 2023-11-27T15:09:29.641 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:28.466+0000 7fd3af168700 1 -- 172.21.15.52:0/3574669119 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fd3a8106990 con 0x7fd3a8104920 2023-11-27T15:09:29.642 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:29.115+0000 7fd3a5ffb700 1 -- 172.21.15.52:0/3574669119 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fd39c019d30 con 0x7fd3a8104920 2023-11-27T15:09:29.642 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:29.117+0000 7fd3a5ffb700 1 -- 172.21.15.52:0/3574669119 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fd39c00be10 con 0x7fd3a8104920 2023-11-27T15:09:29.642 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:29.120+0000 7fd3af168700 1 -- 172.21.15.52:0/3574669119 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7fd398038670 msgr2=0x7fd39803ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:29.642 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:29.120+0000 7fd3af168700 1 --2- 172.21.15.52:0/3574669119 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7fd398038670 0x7fd39803ab20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd394006fd0 tx=0x7fd394006e40 comp rx=0 tx=0).stop 2023-11-27T15:09:29.642 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:29.120+0000 7fd3af168700 1 -- 172.21.15.52:0/3574669119 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3a8104920 msgr2=0x7fd3a8199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:29.642 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:29.120+0000 7fd3af168700 1 --2- 172.21.15.52:0/3574669119 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3a8104920 0x7fd3a8199970 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd39c009fb0 tx=0x7fd39c011ad0 comp rx=0 tx=0).stop 2023-11-27T15:09:29.642 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:29.120+0000 7fd3af168700 1 -- 172.21.15.52:0/3574669119 shutdown_connections 2023-11-27T15:09:29.642 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:29.120+0000 7fd3af168700 1 --2- 172.21.15.52:0/3574669119 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7fd398038670 0x7fd39803ab20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:29.642 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:29.120+0000 7fd3af168700 1 --2- 172.21.15.52:0/3574669119 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3a8104920 0x7fd3a8199970 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:29.642 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:29.120+0000 7fd3af168700 1 -- 172.21.15.52:0/3574669119 >> 172.21.15.52:0/3574669119 conn(0x7fd3a80ffed0 msgr2=0x7fd3a8079910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:29.642 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:29.120+0000 7fd3af168700 1 -- 172.21.15.52:0/3574669119 shutdown_connections 2023-11-27T15:09:29.642 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:29.121+0000 7fd3af168700 1 -- 172.21.15.52:0/3574669119 wait complete. 2023-11-27T15:09:29.851 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:29 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:29.405+0000 7fbf2134a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-27T15:09:29.851 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:29 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:29.512+0000 7fbf2134a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-27T15:09:29.852 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:29 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:29.624+0000 7fbf2134a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-27T15:09:30.117 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:29 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:29.845+0000 7fbf2134a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-27T15:09:30.517 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:30 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:30.251+0000 7fbf2134a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-27T15:09:30.517 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:30 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:30.360+0000 7fbf2134a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-27T15:09:30.518 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:30 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3574669119' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-11-27T15:09:30.518 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:30 smithi052 ceph-mon[99056]: mgrmap e5: x(active, since 10s) 2023-11-27T15:09:30.832 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:30 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:30.511+0000 7fbf2134a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-27T15:09:30.833 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:30 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:30.627+0000 7fbf2134a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-27T15:09:31.087 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2023-11-27T15:09:31.087 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-11-27T15:09:31.087 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "available": true, 2023-11-27T15:09:31.088 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-11-27T15:09:31.088 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-11-27T15:09:31.088 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2023-11-27T15:09:31.088 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.188+0000 7ff0099b8700 1 Processor -- start 2023-11-27T15:09:31.088 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.188+0000 7ff0099b8700 1 -- start start 2023-11-27T15:09:31.088 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.188+0000 7ff0099b8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff004106490 0x7ff004108870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:31.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.188+0000 7ff0099b8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff004079e60 con 0x7ff004106490 2023-11-27T15:09:31.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.189+0000 7ff002ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff004106490 0x7ff004108870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:31.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.189+0000 7ff002ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff004106490 0x7ff004108870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48772/0 (socket says 172.21.15.52:48772) 2023-11-27T15:09:31.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.189+0000 7ff002ffd700 1 -- 172.21.15.52:0/2016255585 learned_addr learned my addr 172.21.15.52:0/2016255585 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:31.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.189+0000 7ff002ffd700 1 -- 172.21.15.52:0/2016255585 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff004108db0 con 0x7ff004106490 2023-11-27T15:09:31.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.189+0000 7ff002ffd700 1 --2- 172.21.15.52:0/2016255585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff004106490 0x7ff004108870 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7feff40098a0 tx=0x7feff4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9becd0c6414c585d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:31.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.189+0000 7ff001ffb700 1 -- 172.21.15.52:0/2016255585 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feff4004160 con 0x7ff004106490 2023-11-27T15:09:31.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.189+0000 7ff001ffb700 1 -- 172.21.15.52:0/2016255585 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feff40042c0 con 0x7ff004106490 2023-11-27T15:09:31.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.189+0000 7ff001ffb700 1 -- 172.21.15.52:0/2016255585 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feff4011a00 con 0x7ff004106490 2023-11-27T15:09:31.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.190+0000 7ff0099b8700 1 -- 172.21.15.52:0/2016255585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff004106490 msgr2=0x7ff004108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:31.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.190+0000 7ff0099b8700 1 --2- 172.21.15.52:0/2016255585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff004106490 0x7ff004108870 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7feff40098a0 tx=0x7feff4009bb0 comp rx=0 tx=0).stop 2023-11-27T15:09:31.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.190+0000 7ff0099b8700 1 -- 172.21.15.52:0/2016255585 shutdown_connections 2023-11-27T15:09:31.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.190+0000 7ff0099b8700 1 --2- 172.21.15.52:0/2016255585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff004106490 0x7ff004108870 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:31.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.190+0000 7ff0099b8700 1 -- 172.21.15.52:0/2016255585 >> 172.21.15.52:0/2016255585 conn(0x7ff0040ffed0 msgr2=0x7ff004102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:31.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.190+0000 7ff0099b8700 1 -- 172.21.15.52:0/2016255585 shutdown_connections 2023-11-27T15:09:31.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.190+0000 7ff0099b8700 1 -- 172.21.15.52:0/2016255585 wait complete. 2023-11-27T15:09:31.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.191+0000 7ff0099b8700 1 Processor -- start 2023-11-27T15:09:31.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.191+0000 7ff0099b8700 1 -- start start 2023-11-27T15:09:31.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.191+0000 7ff0099b8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff004106490 0x7ff004105df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:31.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.191+0000 7ff0099b8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff004106330 con 0x7ff004106490 2023-11-27T15:09:31.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.191+0000 7ff002ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff004106490 0x7ff004105df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:31.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.191+0000 7ff002ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff004106490 0x7ff004105df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48774/0 (socket says 172.21.15.52:48774) 2023-11-27T15:09:31.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.191+0000 7ff002ffd700 1 -- 172.21.15.52:0/3016226484 learned_addr learned my addr 172.21.15.52:0/3016226484 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:31.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.192+0000 7ff002ffd700 1 -- 172.21.15.52:0/3016226484 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff4009550 con 0x7ff004106490 2023-11-27T15:09:31.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.192+0000 7ff002ffd700 1 --2- 172.21.15.52:0/3016226484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff004106490 0x7ff004105df0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feff4009fb0 tx=0x7feff4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:31.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.192+0000 7ff0089b6700 1 -- 172.21.15.52:0/3016226484 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feff4003930 con 0x7ff004106490 2023-11-27T15:09:31.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.192+0000 7ff0089b6700 1 -- 172.21.15.52:0/3016226484 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feff4003a90 con 0x7ff004106490 2023-11-27T15:09:31.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.192+0000 7ff0099b8700 1 -- 172.21.15.52:0/3016226484 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff004109d40 con 0x7ff004106490 2023-11-27T15:09:31.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.192+0000 7ff0089b6700 1 -- 172.21.15.52:0/3016226484 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feff4019610 con 0x7ff004106490 2023-11-27T15:09:31.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.192+0000 7ff0099b8700 1 -- 172.21.15.52:0/3016226484 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff004109bb0 con 0x7ff004106490 2023-11-27T15:09:31.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.193+0000 7ff0089b6700 1 -- 172.21.15.52:0/3016226484 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7feff40198d0 con 0x7ff004106490 2023-11-27T15:09:31.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.193+0000 7ff0089b6700 1 --2- 172.21.15.52:0/3016226484 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7fefec0386c0 0x7fefec03ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:31.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.193+0000 7ff0089b6700 1 -- 172.21.15.52:0/3016226484 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7feff404c510 con 0x7ff004106490 2023-11-27T15:09:31.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.193+0000 7ff0027fc700 1 -- 172.21.15.52:0/3016226484 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7fefec0386c0 msgr2=0x7fefec03ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/960704660 2023-11-27T15:09:31.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.193+0000 7ff0027fc700 1 --2- 172.21.15.52:0/3016226484 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7fefec0386c0 0x7fefec03ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-27T15:09:31.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.193+0000 7ff0099b8700 1 -- 172.21.15.52:0/3016226484 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff004062120 con 0x7ff004106490 2023-11-27T15:09:31.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.198+0000 7ff0089b6700 1 -- 172.21.15.52:0/3016226484 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7feff4019bd0 con 0x7ff004106490 2023-11-27T15:09:31.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.394+0000 7ff0027fc700 1 -- 172.21.15.52:0/3016226484 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7fefec0386c0 msgr2=0x7fefec03ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/960704660 2023-11-27T15:09:31.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.394+0000 7ff0027fc700 1 --2- 172.21.15.52:0/3016226484 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7fefec0386c0 0x7fefec03ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-27T15:09:31.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.667+0000 7ff0099b8700 1 -- 172.21.15.52:0/3016226484 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7ff0041052a0 con 0x7ff004106490 2023-11-27T15:09:31.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.668+0000 7ff0089b6700 1 -- 172.21.15.52:0/3016226484 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7feff4018b40 con 0x7ff004106490 2023-11-27T15:09:31.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.670+0000 7ff0099b8700 1 -- 172.21.15.52:0/3016226484 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7fefec0386c0 msgr2=0x7fefec03ab70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-27T15:09:31.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.670+0000 7ff0099b8700 1 --2- 172.21.15.52:0/3016226484 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7fefec0386c0 0x7fefec03ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:31.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.670+0000 7ff0099b8700 1 -- 172.21.15.52:0/3016226484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff004106490 msgr2=0x7ff004105df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:31.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.670+0000 7ff0099b8700 1 --2- 172.21.15.52:0/3016226484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff004106490 0x7ff004105df0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feff4009fb0 tx=0x7feff4011ad0 comp rx=0 tx=0).stop 2023-11-27T15:09:31.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.670+0000 7ff0099b8700 1 -- 172.21.15.52:0/3016226484 shutdown_connections 2023-11-27T15:09:31.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.670+0000 7ff0099b8700 1 --2- 172.21.15.52:0/3016226484 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7fefec0386c0 0x7fefec03ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:31.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.670+0000 7ff0099b8700 1 --2- 172.21.15.52:0/3016226484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff004106490 0x7ff004105df0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:31.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.670+0000 7ff0099b8700 1 -- 172.21.15.52:0/3016226484 >> 172.21.15.52:0/3016226484 conn(0x7ff0040ffed0 msgr2=0x7ff004100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:31.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.671+0000 7ff0099b8700 1 -- 172.21.15.52:0/3016226484 shutdown_connections 2023-11-27T15:09:31.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:30.671+0000 7ff0099b8700 1 -- 172.21.15.52:0/3016226484 wait complete. 2023-11-27T15:09:31.094 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for the mgr to restart... 2023-11-27T15:09:31.094 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for mgr epoch 5... 2023-11-27T15:09:31.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:31 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3016226484' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-11-27T15:09:31.340 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:31 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:31.156+0000 7fbf2134a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-27T15:09:32.117 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:31 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:31.846+0000 7fbf2134a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-27T15:09:32.117 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:32 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:32.037+0000 7fbf2134a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-27T15:09:33.334 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:33 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:32.999+0000 7fbf2134a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:09:33.335 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:33 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:33.109+0000 7fbf2134a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-27T15:09:33.335 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:33 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:33.329+0000 7fbf2134a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:09:33.932 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:33 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:33.675+0000 7fbf2134a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-27T15:09:34.367 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:33 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:33.926+0000 7fbf2134a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-27T15:09:34.367 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:34 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:34.107+0000 7fbf2134a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-27T15:09:34.867 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:34 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:34.434+0000 7fbf2134a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-27T15:09:35.617 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:35 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:35.254+0000 7fbf2134a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-27T15:09:35.617 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:35 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:35.410+0000 7fbf2134a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-27T15:09:36.066 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:35 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:35.652+0000 7fbf2134a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-27T15:09:36.066 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:35 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:35.789+0000 7fbf2134a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-27T15:09:36.367 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:36 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:36.060+0000 7fbf2134a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-27T15:09:37.367 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:37 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:37.104+0000 7fbf2134a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-27T15:09:37.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:37 smithi052 ceph-mon[99056]: Active manager daemon x restarted 2023-11-27T15:09:37.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:37 smithi052 ceph-mon[99056]: Activating manager daemon x 2023-11-27T15:09:37.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:37 smithi052 ceph-mon[99056]: osdmap e2: 0 total, 0 up, 0 in 2023-11-27T15:09:37.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:37 smithi052 ceph-mon[99056]: mgrmap e6: x(active, starting, since 0.00341889s) 2023-11-27T15:09:37.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:37 smithi052 ceph-mon[99056]: from='mgr.14120 172.21.15.52:0/3799281879' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:09:37.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:37 smithi052 ceph-mon[99056]: from='mgr.14120 172.21.15.52:0/3799281879' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-11-27T15:09:37.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:37 smithi052 ceph-mon[99056]: from='mgr.14120 172.21.15.52:0/3799281879' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-11-27T15:09:37.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:37 smithi052 ceph-mon[99056]: from='mgr.14120 172.21.15.52:0/3799281879' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-11-27T15:09:37.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:37 smithi052 ceph-mon[99056]: from='mgr.14120 172.21.15.52:0/3799281879' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-11-27T15:09:37.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:37 smithi052 ceph-mon[99056]: Manager daemon x is now available 2023-11-27T15:09:37.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:37 smithi052 ceph-mon[99056]: from='mgr.14120 172.21.15.52:0/3799281879' entity='mgr.x' 2023-11-27T15:09:37.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:37 smithi052 ceph-mon[99056]: from='mgr.14120 172.21.15.52:0/3799281879' entity='mgr.x' 2023-11-27T15:09:37.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:37 smithi052 ceph-mon[99056]: from='mgr.14120 172.21.15.52:0/3799281879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:09:37.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:37 smithi052 ceph-mon[99056]: from='mgr.14120 172.21.15.52:0/3799281879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:09:38.556 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2023-11-27T15:09:38.556 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-11-27T15:09:38.556 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "initialized": true 2023-11-27T15:09:38.556 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2023-11-27T15:09:38.556 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.571+0000 7f2c25686700 1 Processor -- start 2023-11-27T15:09:38.556 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.571+0000 7f2c25686700 1 -- start start 2023-11-27T15:09:38.556 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.571+0000 7f2c25686700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c20104920 0x7f2c20104d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:38.556 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.571+0000 7f2c25686700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c20105300 con 0x7f2c20104920 2023-11-27T15:09:38.556 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.572+0000 7f2c1effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c20104920 0x7f2c20104d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:38.556 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.572+0000 7f2c1effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c20104920 0x7f2c20104d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48782/0 (socket says 172.21.15.52:48782) 2023-11-27T15:09:38.557 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.572+0000 7f2c1effd700 1 -- 172.21.15.52:0/1328927785 learned_addr learned my addr 172.21.15.52:0/1328927785 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:38.557 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.572+0000 7f2c1effd700 1 -- 172.21.15.52:0/1328927785 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c20105b20 con 0x7f2c20104920 2023-11-27T15:09:38.557 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.572+0000 7f2c1effd700 1 --2- 172.21.15.52:0/1328927785 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c20104920 0x7f2c20104d30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2c100098a0 tx=0x7f2c10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac560759d6a4f2a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:38.557 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.572+0000 7f2c1dffb700 1 -- 172.21.15.52:0/1328927785 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c10004160 con 0x7f2c20104920 2023-11-27T15:09:38.557 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.572+0000 7f2c1dffb700 1 -- 172.21.15.52:0/1328927785 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2c100042c0 con 0x7f2c20104920 2023-11-27T15:09:38.557 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.572+0000 7f2c1dffb700 1 -- 172.21.15.52:0/1328927785 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c10011a00 con 0x7f2c20104920 2023-11-27T15:09:38.557 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.573+0000 7f2c25686700 1 -- 172.21.15.52:0/1328927785 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c20104920 msgr2=0x7f2c20104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:38.557 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.573+0000 7f2c25686700 1 --2- 172.21.15.52:0/1328927785 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c20104920 0x7f2c20104d30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2c100098a0 tx=0x7f2c10009bb0 comp rx=0 tx=0).stop 2023-11-27T15:09:38.557 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.573+0000 7f2c25686700 1 -- 172.21.15.52:0/1328927785 shutdown_connections 2023-11-27T15:09:38.557 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.573+0000 7f2c25686700 1 --2- 172.21.15.52:0/1328927785 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c20104920 0x7f2c20104d30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:38.557 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.573+0000 7f2c25686700 1 -- 172.21.15.52:0/1328927785 >> 172.21.15.52:0/1328927785 conn(0x7f2c200ffed0 msgr2=0x7f2c20102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:38.557 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.573+0000 7f2c25686700 1 -- 172.21.15.52:0/1328927785 shutdown_connections 2023-11-27T15:09:38.557 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.573+0000 7f2c25686700 1 -- 172.21.15.52:0/1328927785 wait complete. 2023-11-27T15:09:38.558 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.574+0000 7f2c25686700 1 Processor -- start 2023-11-27T15:09:38.558 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.574+0000 7f2c25686700 1 -- start start 2023-11-27T15:09:38.558 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.574+0000 7f2c25686700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c20104920 0x7f2c2018fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:38.558 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.574+0000 7f2c25686700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c20190260 con 0x7f2c20104920 2023-11-27T15:09:38.558 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.574+0000 7f2c1effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c20104920 0x7f2c2018fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:38.558 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.574+0000 7f2c1effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c20104920 0x7f2c2018fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48784/0 (socket says 172.21.15.52:48784) 2023-11-27T15:09:38.558 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.574+0000 7f2c1effd700 1 -- 172.21.15.52:0/721528003 learned_addr learned my addr 172.21.15.52:0/721528003 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:38.558 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.575+0000 7f2c1effd700 1 -- 172.21.15.52:0/721528003 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c10009550 con 0x7f2c20104920 2023-11-27T15:09:38.558 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.575+0000 7f2c1effd700 1 --2- 172.21.15.52:0/721528003 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c20104920 0x7f2c2018fd20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009fb0 tx=0x7f2c10011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:38.558 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.575+0000 7f2c07fff700 1 -- 172.21.15.52:0/721528003 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c10003930 con 0x7f2c20104920 2023-11-27T15:09:38.558 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.575+0000 7f2c07fff700 1 -- 172.21.15.52:0/721528003 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2c10003a90 con 0x7f2c20104920 2023-11-27T15:09:38.558 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.575+0000 7f2c07fff700 1 -- 172.21.15.52:0/721528003 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c100195e0 con 0x7f2c20104920 2023-11-27T15:09:38.559 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.575+0000 7f2c25686700 1 -- 172.21.15.52:0/721528003 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c20108d50 con 0x7f2c20104920 2023-11-27T15:09:38.559 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.575+0000 7f2c25686700 1 -- 172.21.15.52:0/721528003 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c20108ee0 con 0x7f2c20104920 2023-11-27T15:09:38.559 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.576+0000 7f2c07fff700 1 -- 172.21.15.52:0/721528003 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f2c100197a0 con 0x7f2c20104920 2023-11-27T15:09:38.559 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.576+0000 7f2c07fff700 1 --2- 172.21.15.52:0/721528003 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7f2c08038760 0x7f2c0803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:38.559 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.576+0000 7f2c07fff700 1 -- 172.21.15.52:0/721528003 --> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c0803b320 con 0x7f2c08038760 2023-11-27T15:09:38.559 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.576+0000 7f2c07fff700 1 -- 172.21.15.52:0/721528003 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2c1004c420 con 0x7f2c20104920 2023-11-27T15:09:38.559 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.576+0000 7f2c1e7fc700 1 -- 172.21.15.52:0/721528003 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7f2c08038760 msgr2=0x7f2c0803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/960704660 2023-11-27T15:09:38.559 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.576+0000 7f2c1e7fc700 1 --2- 172.21.15.52:0/721528003 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7f2c08038760 0x7f2c0803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-27T15:09:38.559 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.777+0000 7f2c1e7fc700 1 -- 172.21.15.52:0/721528003 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7f2c08038760 msgr2=0x7f2c0803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/960704660 2023-11-27T15:09:38.559 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:31.777+0000 7f2c1e7fc700 1 --2- 172.21.15.52:0/721528003 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7f2c08038760 0x7f2c0803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-27T15:09:38.559 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:32.178+0000 7f2c1e7fc700 1 -- 172.21.15.52:0/721528003 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7f2c08038760 msgr2=0x7f2c0803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/960704660 2023-11-27T15:09:38.559 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:32.178+0000 7f2c1e7fc700 1 --2- 172.21.15.52:0/721528003 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7f2c08038760 0x7f2c0803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-27T15:09:38.559 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:32.979+0000 7f2c1e7fc700 1 -- 172.21.15.52:0/721528003 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7f2c08038760 msgr2=0x7f2c0803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/960704660 2023-11-27T15:09:38.560 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:32.979+0000 7f2c1e7fc700 1 --2- 172.21.15.52:0/721528003 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7f2c08038760 0x7f2c0803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-27T15:09:38.560 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:34.580+0000 7f2c1e7fc700 1 -- 172.21.15.52:0/721528003 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7f2c08038760 msgr2=0x7f2c0803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/960704660 2023-11-27T15:09:38.560 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:34.580+0000 7f2c1e7fc700 1 --2- 172.21.15.52:0/721528003 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7f2c08038760 0x7f2c0803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-11-27T15:09:38.560 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:37.119+0000 7f2c07fff700 1 -- 172.21.15.52:0/721528003 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mgrmap(e 6) v1 ==== 45250+0+0 (secure 0 0 0) 0x7f2c10019c80 con 0x7f2c20104920 2023-11-27T15:09:38.560 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:37.119+0000 7f2c07fff700 1 -- 172.21.15.52:0/721528003 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7f2c08038760 msgr2=0x7f2c0803ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-27T15:09:38.560 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:37.119+0000 7f2c07fff700 1 --2- 172.21.15.52:0/721528003 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7f2c08038760 0x7f2c0803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:38.560 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:38.120+0000 7f2c07fff700 1 -- 172.21.15.52:0/721528003 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7f2c10023c90 con 0x7f2c20104920 2023-11-27T15:09:38.560 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:38.120+0000 7f2c07fff700 1 --2- 172.21.15.52:0/721528003 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f2c0803bfe0 0x7f2c0803e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:38.560 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:38.120+0000 7f2c07fff700 1 -- 172.21.15.52:0/721528003 --> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c0803b320 con 0x7f2c0803bfe0 2023-11-27T15:09:38.560 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:38.121+0000 7f2c1e7fc700 1 --2- 172.21.15.52:0/721528003 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f2c0803bfe0 0x7f2c0803e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:38.560 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:38.121+0000 7f2c1e7fc700 1 --2- 172.21.15.52:0/721528003 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f2c0803bfe0 0x7f2c0803e3c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f2c14003a10 tx=0x7f2c140092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:38.560 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:38.122+0000 7f2c07fff700 1 -- 172.21.15.52:0/721528003 <== mgr.14120 v2:172.21.15.52:6800/3470359117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f2c0803b320 con 0x7f2c0803bfe0 2023-11-27T15:09:38.560 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:38.133+0000 7f2c25686700 1 -- 172.21.15.52:0/721528003 --> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f2c2018f8c0 con 0x7f2c0803bfe0 2023-11-27T15:09:38.561 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:38.133+0000 7f2c07fff700 1 -- 172.21.15.52:0/721528003 <== mgr.14120 v2:172.21.15.52:6800/3470359117 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f2c2018f8c0 con 0x7f2c0803bfe0 2023-11-27T15:09:38.561 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:38.133+0000 7f2c25686700 1 -- 172.21.15.52:0/721528003 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f2c0803bfe0 msgr2=0x7f2c0803e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:38.561 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:38.133+0000 7f2c25686700 1 --2- 172.21.15.52:0/721528003 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f2c0803bfe0 0x7f2c0803e3c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f2c14003a10 tx=0x7f2c140092b0 comp rx=0 tx=0).stop 2023-11-27T15:09:38.561 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:38.133+0000 7f2c25686700 1 -- 172.21.15.52:0/721528003 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c20104920 msgr2=0x7f2c2018fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:38.561 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:38.133+0000 7f2c25686700 1 --2- 172.21.15.52:0/721528003 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c20104920 0x7f2c2018fd20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009fb0 tx=0x7f2c10011ad0 comp rx=0 tx=0).stop 2023-11-27T15:09:38.561 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:38.134+0000 7f2c25686700 1 -- 172.21.15.52:0/721528003 shutdown_connections 2023-11-27T15:09:38.561 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:38.134+0000 7f2c25686700 1 --2- 172.21.15.52:0/721528003 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f2c0803bfe0 0x7f2c0803e3c0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:38.561 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:38.134+0000 7f2c25686700 1 --2- 172.21.15.52:0/721528003 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c20104920 0x7f2c2018fd20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:38.561 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:38.134+0000 7f2c25686700 1 --2- 172.21.15.52:0/721528003 >> [v2:172.21.15.52:6800/960704660,v1:172.21.15.52:6801/960704660] conn(0x7f2c08038760 0x7f2c0803ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:38.561 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:38.134+0000 7f2c25686700 1 -- 172.21.15.52:0/721528003 >> 172.21.15.52:0/721528003 conn(0x7f2c200ffed0 msgr2=0x7f2c20079910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:38.561 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:38.134+0000 7f2c25686700 1 -- 172.21.15.52:0/721528003 shutdown_connections 2023-11-27T15:09:38.561 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:38.134+0000 7f2c25686700 1 -- 172.21.15.52:0/721528003 wait complete. 2023-11-27T15:09:38.561 INFO:teuthology.orchestra.run.smithi052.stdout:mgr epoch 5 is available 2023-11-27T15:09:38.562 INFO:teuthology.orchestra.run.smithi052.stdout:Setting orchestrator backend to cephadm... 2023-11-27T15:09:38.581 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:38 smithi052 ceph-mon[99056]: Found migration_current of "None". Setting to last migration. 2023-11-27T15:09:38.581 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:38 smithi052 ceph-mon[99056]: from='mgr.14120 172.21.15.52:0/3799281879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:09:38.581 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:38 smithi052 ceph-mon[99056]: from='mgr.14120 172.21.15.52:0/3799281879' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-27T15:09:38.581 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:38 smithi052 ceph-mon[99056]: from='mgr.14120 172.21.15.52:0/3799281879' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-27T15:09:38.582 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:38 smithi052 ceph-mon[99056]: from='mgr.14120 172.21.15.52:0/3799281879' entity='mgr.x' 2023-11-27T15:09:38.582 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:38 smithi052 ceph-mon[99056]: from='mgr.14120 172.21.15.52:0/3799281879' entity='mgr.x' 2023-11-27T15:09:38.582 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:38 smithi052 ceph-mon[99056]: mgrmap e7: x(active, since 1.00606s) 2023-11-27T15:09:39.461 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:39 smithi052 ceph-mon[99056]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-11-27T15:09:39.461 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:39 smithi052 ceph-mon[99056]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-11-27T15:09:39.909 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.015+0000 7f3a8c708700 1 Processor -- start 2023-11-27T15:09:39.910 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.015+0000 7f3a8c708700 1 -- start start 2023-11-27T15:09:39.910 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.015+0000 7f3a8c708700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a84104300 0x7f3a84104710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:39.910 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.015+0000 7f3a8c708700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a84079de0 con 0x7f3a84104300 2023-11-27T15:09:39.910 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.015+0000 7f3a8a4a4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a84104300 0x7f3a84104710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:39.910 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.015+0000 7f3a8a4a4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a84104300 0x7f3a84104710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48828/0 (socket says 172.21.15.52:48828) 2023-11-27T15:09:39.910 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.015+0000 7f3a8a4a4700 1 -- 172.21.15.52:0/2921634231 learned_addr learned my addr 172.21.15.52:0/2921634231 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:39.910 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.016+0000 7f3a8a4a4700 1 -- 172.21.15.52:0/2921634231 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a84104c50 con 0x7f3a84104300 2023-11-27T15:09:39.910 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.016+0000 7f3a8a4a4700 1 --2- 172.21.15.52:0/2921634231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a84104300 0x7f3a84104710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3a800098a0 tx=0x7f3a80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769bca0c016e4d60 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:39.910 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.016+0000 7f3a894a2700 1 -- 172.21.15.52:0/2921634231 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a80004160 con 0x7f3a84104300 2023-11-27T15:09:39.910 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.016+0000 7f3a894a2700 1 -- 172.21.15.52:0/2921634231 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3a800042c0 con 0x7f3a84104300 2023-11-27T15:09:39.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.016+0000 7f3a894a2700 1 -- 172.21.15.52:0/2921634231 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a80011a00 con 0x7f3a84104300 2023-11-27T15:09:39.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.016+0000 7f3a8c708700 1 -- 172.21.15.52:0/2921634231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a84104300 msgr2=0x7f3a84104710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:39.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.016+0000 7f3a8c708700 1 --2- 172.21.15.52:0/2921634231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a84104300 0x7f3a84104710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3a800098a0 tx=0x7f3a80009bb0 comp rx=0 tx=0).stop 2023-11-27T15:09:39.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.017+0000 7f3a8c708700 1 -- 172.21.15.52:0/2921634231 shutdown_connections 2023-11-27T15:09:39.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.017+0000 7f3a8c708700 1 --2- 172.21.15.52:0/2921634231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a84104300 0x7f3a84104710 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:39.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.017+0000 7f3a8c708700 1 -- 172.21.15.52:0/2921634231 >> 172.21.15.52:0/2921634231 conn(0x7f3a840ffed0 msgr2=0x7f3a84102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:39.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.017+0000 7f3a8c708700 1 -- 172.21.15.52:0/2921634231 shutdown_connections 2023-11-27T15:09:39.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.017+0000 7f3a8c708700 1 -- 172.21.15.52:0/2921634231 wait complete. 2023-11-27T15:09:39.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.017+0000 7f3a8c708700 1 Processor -- start 2023-11-27T15:09:39.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.017+0000 7f3a8c708700 1 -- start start 2023-11-27T15:09:39.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.017+0000 7f3a8c708700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a84104300 0x7f3a8419b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:39.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.018+0000 7f3a8c708700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8419bee0 con 0x7f3a84104300 2023-11-27T15:09:39.912 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.018+0000 7f3a8a4a4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a84104300 0x7f3a8419b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:39.912 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.018+0000 7f3a8a4a4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a84104300 0x7f3a8419b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48830/0 (socket says 172.21.15.52:48830) 2023-11-27T15:09:39.912 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.018+0000 7f3a8a4a4700 1 -- 172.21.15.52:0/535425204 learned_addr learned my addr 172.21.15.52:0/535425204 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:39.912 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.018+0000 7f3a8a4a4700 1 -- 172.21.15.52:0/535425204 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a80009550 con 0x7f3a84104300 2023-11-27T15:09:39.912 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.018+0000 7f3a8a4a4700 1 --2- 172.21.15.52:0/535425204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a84104300 0x7f3a8419b9a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3a80006af0 tx=0x7f3a80011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:39.912 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.018+0000 7f3a7b7fe700 1 -- 172.21.15.52:0/535425204 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a80003a40 con 0x7f3a84104300 2023-11-27T15:09:39.912 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.018+0000 7f3a7b7fe700 1 -- 172.21.15.52:0/535425204 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3a80003ba0 con 0x7f3a84104300 2023-11-27T15:09:39.912 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.018+0000 7f3a7b7fe700 1 -- 172.21.15.52:0/535425204 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a8001a6e0 con 0x7f3a84104300 2023-11-27T15:09:39.912 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.018+0000 7f3a8c708700 1 -- 172.21.15.52:0/535425204 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a84105c50 con 0x7f3a84104300 2023-11-27T15:09:39.912 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.018+0000 7f3a8c708700 1 -- 172.21.15.52:0/535425204 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a84105ac0 con 0x7f3a84104300 2023-11-27T15:09:39.912 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.019+0000 7f3a7b7fe700 1 -- 172.21.15.52:0/535425204 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7f3a8001a860 con 0x7f3a84104300 2023-11-27T15:09:39.912 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.019+0000 7f3a7b7fe700 1 --2- 172.21.15.52:0/535425204 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f3a70038630 0x7f3a7003aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:39.912 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.019+0000 7f3a7b7fe700 1 -- 172.21.15.52:0/535425204 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3a8004c680 con 0x7f3a84104300 2023-11-27T15:09:39.913 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.019+0000 7f3a8c708700 1 -- 172.21.15.52:0/535425204 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a84195250 con 0x7f3a84104300 2023-11-27T15:09:39.913 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.020+0000 7f3a89ca3700 1 --2- 172.21.15.52:0/535425204 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f3a70038630 0x7f3a7003aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:39.913 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.020+0000 7f3a89ca3700 1 --2- 172.21.15.52:0/535425204 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f3a70038630 0x7f3a7003aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3a74006fd0 tx=0x7f3a74006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:39.913 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.025+0000 7f3a7b7fe700 1 -- 172.21.15.52:0/535425204 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3a8001ab10 con 0x7f3a84104300 2023-11-27T15:09:39.913 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.172+0000 7f3a7b7fe700 1 -- 172.21.15.52:0/535425204 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f3a80047070 con 0x7f3a84104300 2023-11-27T15:09:39.913 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.418+0000 7f3a8c708700 1 -- 172.21.15.52:0/535425204 --> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a84060f30 con 0x7f3a70038630 2023-11-27T15:09:39.913 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.429+0000 7f3a7b7fe700 1 -- 172.21.15.52:0/535425204 <== mgr.14120 v2:172.21.15.52:6800/3470359117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3a84060f30 con 0x7f3a70038630 2023-11-27T15:09:39.913 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.432+0000 7f3a8c708700 1 -- 172.21.15.52:0/535425204 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f3a70038630 msgr2=0x7f3a7003aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:39.913 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.432+0000 7f3a8c708700 1 --2- 172.21.15.52:0/535425204 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f3a70038630 0x7f3a7003aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3a74006fd0 tx=0x7f3a74006e40 comp rx=0 tx=0).stop 2023-11-27T15:09:39.913 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.432+0000 7f3a8c708700 1 -- 172.21.15.52:0/535425204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a84104300 msgr2=0x7f3a8419b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:39.913 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.432+0000 7f3a8c708700 1 --2- 172.21.15.52:0/535425204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a84104300 0x7f3a8419b9a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3a80006af0 tx=0x7f3a80011ad0 comp rx=0 tx=0).stop 2023-11-27T15:09:39.913 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.432+0000 7f3a8c708700 1 -- 172.21.15.52:0/535425204 shutdown_connections 2023-11-27T15:09:39.914 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.432+0000 7f3a8c708700 1 --2- 172.21.15.52:0/535425204 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f3a70038630 0x7f3a7003aae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:39.914 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.432+0000 7f3a8c708700 1 --2- 172.21.15.52:0/535425204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a84104300 0x7f3a8419b9a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:39.914 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.432+0000 7f3a8c708700 1 -- 172.21.15.52:0/535425204 >> 172.21.15.52:0/535425204 conn(0x7f3a840ffed0 msgr2=0x7f3a8418fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:39.914 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.432+0000 7f3a8c708700 1 -- 172.21.15.52:0/535425204 shutdown_connections 2023-11-27T15:09:39.914 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:39.432+0000 7f3a8c708700 1 -- 172.21.15.52:0/535425204 wait complete. 2023-11-27T15:09:40.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:40 smithi052 ceph-mon[99056]: mgrmap e8: x(active, since 2s) 2023-11-27T15:09:40.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:40 smithi052 ceph-mon[99056]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:09:40.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:40 smithi052 ceph-mon[99056]: [27/Nov/2023:15:09:39] ENGINE Bus STARTING 2023-11-27T15:09:40.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:40 smithi052 ceph-mon[99056]: from='mgr.14120 172.21.15.52:0/3799281879' entity='mgr.x' 2023-11-27T15:09:40.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:40 smithi052 ceph-mon[99056]: from='mgr.14120 172.21.15.52:0/3799281879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:09:40.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:40 smithi052 ceph-mon[99056]: [27/Nov/2023:15:09:39] ENGINE Serving on https://172.21.15.52:7150 2023-11-27T15:09:40.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:40 smithi052 ceph-mon[99056]: [27/Nov/2023:15:09:39] ENGINE Serving on http://172.21.15.52:8765 2023-11-27T15:09:40.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:40 smithi052 ceph-mon[99056]: [27/Nov/2023:15:09:39] ENGINE Bus STARTED 2023-11-27T15:09:40.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:40 smithi052 ceph-mon[99056]: from='mgr.14120 172.21.15.52:0/3799281879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:09:41.156 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout value unchanged 2023-11-27T15:09:41.156 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.406+0000 7f01d51ad700 1 Processor -- start 2023-11-27T15:09:41.156 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.406+0000 7f01d51ad700 1 -- start start 2023-11-27T15:09:41.156 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.406+0000 7f01d51ad700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01d0105ac0 0x7f01d0105ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:41.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.406+0000 7f01d51ad700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d0106410 con 0x7f01d0105ac0 2023-11-27T15:09:41.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.406+0000 7f01ced9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01d0105ac0 0x7f01d0105ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:41.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.406+0000 7f01ced9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01d0105ac0 0x7f01d0105ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48842/0 (socket says 172.21.15.52:48842) 2023-11-27T15:09:41.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.406+0000 7f01ced9d700 1 -- 172.21.15.52:0/3937846332 learned_addr learned my addr 172.21.15.52:0/3937846332 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:41.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.407+0000 7f01ced9d700 1 -- 172.21.15.52:0/3937846332 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d0106550 con 0x7f01d0105ac0 2023-11-27T15:09:41.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.407+0000 7f01ced9d700 1 --2- 172.21.15.52:0/3937846332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01d0105ac0 0x7f01d0105ed0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f01c00098a0 tx=0x7f01c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4491f4c50272dce1 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:41.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.407+0000 7f01cdd9b700 1 -- 172.21.15.52:0/3937846332 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01c0004160 con 0x7f01d0105ac0 2023-11-27T15:09:41.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.407+0000 7f01cdd9b700 1 -- 172.21.15.52:0/3937846332 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f01c00042c0 con 0x7f01d0105ac0 2023-11-27T15:09:41.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.407+0000 7f01cdd9b700 1 -- 172.21.15.52:0/3937846332 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01c0011a00 con 0x7f01d0105ac0 2023-11-27T15:09:41.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.408+0000 7f01d51ad700 1 -- 172.21.15.52:0/3937846332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01d0105ac0 msgr2=0x7f01d0105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:41.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.408+0000 7f01d51ad700 1 --2- 172.21.15.52:0/3937846332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01d0105ac0 0x7f01d0105ed0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f01c00098a0 tx=0x7f01c0009bb0 comp rx=0 tx=0).stop 2023-11-27T15:09:41.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.408+0000 7f01d51ad700 1 -- 172.21.15.52:0/3937846332 shutdown_connections 2023-11-27T15:09:41.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.408+0000 7f01d51ad700 1 --2- 172.21.15.52:0/3937846332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01d0105ac0 0x7f01d0105ed0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:41.158 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.408+0000 7f01d51ad700 1 -- 172.21.15.52:0/3937846332 >> 172.21.15.52:0/3937846332 conn(0x7f01d0101170 msgr2=0x7f01d0103580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:41.158 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.408+0000 7f01d51ad700 1 -- 172.21.15.52:0/3937846332 shutdown_connections 2023-11-27T15:09:41.158 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.408+0000 7f01d51ad700 1 -- 172.21.15.52:0/3937846332 wait complete. 2023-11-27T15:09:41.158 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.409+0000 7f01d51ad700 1 Processor -- start 2023-11-27T15:09:41.158 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.409+0000 7f01d51ad700 1 -- start start 2023-11-27T15:09:41.158 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.409+0000 7f01d51ad700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01d0105ac0 0x7f01d007bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:41.158 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.409+0000 7f01d51ad700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d007c140 con 0x7f01d0105ac0 2023-11-27T15:09:41.158 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.409+0000 7f01ced9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01d0105ac0 0x7f01d007bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:41.158 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.409+0000 7f01ced9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01d0105ac0 0x7f01d007bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48844/0 (socket says 172.21.15.52:48844) 2023-11-27T15:09:41.159 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.409+0000 7f01ced9d700 1 -- 172.21.15.52:0/335335003 learned_addr learned my addr 172.21.15.52:0/335335003 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:41.159 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.410+0000 7f01ced9d700 1 -- 172.21.15.52:0/335335003 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c0009550 con 0x7f01d0105ac0 2023-11-27T15:09:41.159 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.410+0000 7f01ced9d700 1 --2- 172.21.15.52:0/335335003 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01d0105ac0 0x7f01d007bc00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f01c0009fb0 tx=0x7f01c0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:41.159 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.410+0000 7f01b7fff700 1 -- 172.21.15.52:0/335335003 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01c0003930 con 0x7f01d0105ac0 2023-11-27T15:09:41.159 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.410+0000 7f01d51ad700 1 -- 172.21.15.52:0/335335003 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d0107b10 con 0x7f01d0105ac0 2023-11-27T15:09:41.159 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.410+0000 7f01b7fff700 1 -- 172.21.15.52:0/335335003 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f01c0003a90 con 0x7f01d0105ac0 2023-11-27T15:09:41.159 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.410+0000 7f01b7fff700 1 -- 172.21.15.52:0/335335003 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01c0019610 con 0x7f01d0105ac0 2023-11-27T15:09:41.159 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.410+0000 7f01d51ad700 1 -- 172.21.15.52:0/335335003 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d0107980 con 0x7f01d0105ac0 2023-11-27T15:09:41.159 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.411+0000 7f01b7fff700 1 -- 172.21.15.52:0/335335003 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f01c0019910 con 0x7f01d0105ac0 2023-11-27T15:09:41.159 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.411+0000 7f01b7fff700 1 --2- 172.21.15.52:0/335335003 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f01b80386c0 0x7f01b803ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:41.159 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.411+0000 7f01b7fff700 1 -- 172.21.15.52:0/335335003 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f01c004c980 con 0x7f01d0105ac0 2023-11-27T15:09:41.159 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.411+0000 7f01b5ffb700 1 -- 172.21.15.52:0/335335003 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b00052f0 con 0x7f01d0105ac0 2023-11-27T15:09:41.160 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.411+0000 7f01ce59c700 1 --2- 172.21.15.52:0/335335003 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f01b80386c0 0x7f01b803ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:41.160 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.412+0000 7f01ce59c700 1 --2- 172.21.15.52:0/335335003 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f01b80386c0 0x7f01b803ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f01c4006fd0 tx=0x7f01c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:41.160 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.416+0000 7f01b7fff700 1 -- 172.21.15.52:0/335335003 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f01c0019c10 con 0x7f01d0105ac0 2023-11-27T15:09:41.160 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.780+0000 7f01b5ffb700 1 -- 172.21.15.52:0/335335003 --> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f01b0000bc0 con 0x7f01b80386c0 2023-11-27T15:09:41.160 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.781+0000 7f01b7fff700 1 -- 172.21.15.52:0/335335003 <== mgr.14120 v2:172.21.15.52:6800/3470359117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f01b0000bc0 con 0x7f01b80386c0 2023-11-27T15:09:41.160 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.783+0000 7f01b5ffb700 1 -- 172.21.15.52:0/335335003 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f01b80386c0 msgr2=0x7f01b803ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:41.160 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.783+0000 7f01b5ffb700 1 --2- 172.21.15.52:0/335335003 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f01b80386c0 0x7f01b803ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f01c4006fd0 tx=0x7f01c4006e40 comp rx=0 tx=0).stop 2023-11-27T15:09:41.160 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.783+0000 7f01b5ffb700 1 -- 172.21.15.52:0/335335003 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01d0105ac0 msgr2=0x7f01d007bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:41.160 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.783+0000 7f01b5ffb700 1 --2- 172.21.15.52:0/335335003 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01d0105ac0 0x7f01d007bc00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f01c0009fb0 tx=0x7f01c0011ad0 comp rx=0 tx=0).stop 2023-11-27T15:09:41.160 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.783+0000 7f01b5ffb700 1 -- 172.21.15.52:0/335335003 shutdown_connections 2023-11-27T15:09:41.160 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.783+0000 7f01b5ffb700 1 --2- 172.21.15.52:0/335335003 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f01b80386c0 0x7f01b803ab70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:41.160 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.783+0000 7f01b5ffb700 1 --2- 172.21.15.52:0/335335003 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01d0105ac0 0x7f01d007bc00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:41.161 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.783+0000 7f01b5ffb700 1 -- 172.21.15.52:0/335335003 >> 172.21.15.52:0/335335003 conn(0x7f01d0101170 msgr2=0x7f01d0101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:41.161 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.783+0000 7f01b5ffb700 1 -- 172.21.15.52:0/335335003 shutdown_connections 2023-11-27T15:09:41.161 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:40.783+0000 7f01b5ffb700 1 -- 172.21.15.52:0/335335003 wait complete. 2023-11-27T15:09:41.161 INFO:teuthology.orchestra.run.smithi052.stdout:Generating ssh key... 2023-11-27T15:09:41.420 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:41 smithi052 ceph-mon[99056]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:09:42.436 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:42 smithi052 ceph-mon[99056]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:09:42.436 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:42 smithi052 ceph-mon[99056]: Generating ssh key... 2023-11-27T15:09:42.436 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:42 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: Generating public/private rsa key pair. 2023-11-27T15:09:42.436 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:42 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: Your identification has been saved in /tmp/tmpogcz0707/key. 2023-11-27T15:09:42.436 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:42 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: Your public key has been saved in /tmp/tmpogcz0707/key.pub. 2023-11-27T15:09:42.437 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:42 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: The key fingerprint is: 2023-11-27T15:09:42.437 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:42 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: SHA256:ZVH+xufeFc+iIUk7P8wpyTEBbmCuFxN2570Gizfw2ls ceph-c6c7df88-8d36-11ee-95a2-87774f69a715 2023-11-27T15:09:42.437 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:42 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: The key's randomart image is: 2023-11-27T15:09:42.437 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:42 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: +---[RSA 3072]----+ 2023-11-27T15:09:42.437 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:42 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: | ... | 2023-11-27T15:09:42.437 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:42 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: | = o .o | 2023-11-27T15:09:42.437 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:42 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: | + = +o.. | 2023-11-27T15:09:42.437 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:42 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: | + +o+ .o | 2023-11-27T15:09:42.437 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:42 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: | . +S+.+ .+..| 2023-11-27T15:09:42.437 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:42 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: | . . ..Boo. +o| 2023-11-27T15:09:42.437 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:42 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: | . ==OE.. =| 2023-11-27T15:09:42.437 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:42 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: | . ===o oo| 2023-11-27T15:09:42.437 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:42 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: | .oo. o| 2023-11-27T15:09:42.438 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:42 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: +----[SHA256]-----+ 2023-11-27T15:09:43.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.662+0000 7fbf35fda700 1 Processor -- start 2023-11-27T15:09:43.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.662+0000 7fbf35fda700 1 -- start start 2023-11-27T15:09:43.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.662+0000 7fbf35fda700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf30104920 0x7fbf30104d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:43.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.662+0000 7fbf35fda700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf30105300 con 0x7fbf30104920 2023-11-27T15:09:43.004 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.662+0000 7fbf2f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf30104920 0x7fbf30104d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:43.004 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.662+0000 7fbf2f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf30104920 0x7fbf30104d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48848/0 (socket says 172.21.15.52:48848) 2023-11-27T15:09:43.004 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.662+0000 7fbf2f7fe700 1 -- 172.21.15.52:0/4225615741 learned_addr learned my addr 172.21.15.52:0/4225615741 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:43.004 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.662+0000 7fbf2f7fe700 1 -- 172.21.15.52:0/4225615741 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf30105b20 con 0x7fbf30104920 2023-11-27T15:09:43.005 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.663+0000 7fbf2f7fe700 1 --2- 172.21.15.52:0/4225615741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf30104920 0x7fbf30104d30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbf200098a0 tx=0x7fbf20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d39c66f63759ac server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:43.005 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.663+0000 7fbf2e7fc700 1 -- 172.21.15.52:0/4225615741 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf20004160 con 0x7fbf30104920 2023-11-27T15:09:43.005 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.663+0000 7fbf2e7fc700 1 -- 172.21.15.52:0/4225615741 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbf200042c0 con 0x7fbf30104920 2023-11-27T15:09:43.005 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.663+0000 7fbf2e7fc700 1 -- 172.21.15.52:0/4225615741 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf20011a00 con 0x7fbf30104920 2023-11-27T15:09:43.006 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.663+0000 7fbf35fda700 1 -- 172.21.15.52:0/4225615741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf30104920 msgr2=0x7fbf30104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:43.006 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.663+0000 7fbf35fda700 1 --2- 172.21.15.52:0/4225615741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf30104920 0x7fbf30104d30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbf200098a0 tx=0x7fbf20009bb0 comp rx=0 tx=0).stop 2023-11-27T15:09:43.006 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.663+0000 7fbf35fda700 1 -- 172.21.15.52:0/4225615741 shutdown_connections 2023-11-27T15:09:43.006 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.663+0000 7fbf35fda700 1 --2- 172.21.15.52:0/4225615741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf30104920 0x7fbf30104d30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:43.007 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.663+0000 7fbf35fda700 1 -- 172.21.15.52:0/4225615741 >> 172.21.15.52:0/4225615741 conn(0x7fbf300ffed0 msgr2=0x7fbf30102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:43.007 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.664+0000 7fbf35fda700 1 -- 172.21.15.52:0/4225615741 shutdown_connections 2023-11-27T15:09:43.007 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.664+0000 7fbf35fda700 1 -- 172.21.15.52:0/4225615741 wait complete. 2023-11-27T15:09:43.008 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.664+0000 7fbf35fda700 1 Processor -- start 2023-11-27T15:09:43.008 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.664+0000 7fbf35fda700 1 -- start start 2023-11-27T15:09:43.008 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.665+0000 7fbf35fda700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf30104920 0x7fbf30079ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:43.008 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.665+0000 7fbf35fda700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3019c6b0 con 0x7fbf30104920 2023-11-27T15:09:43.009 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.665+0000 7fbf2f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf30104920 0x7fbf30079ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:43.009 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.665+0000 7fbf2f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf30104920 0x7fbf30079ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48850/0 (socket says 172.21.15.52:48850) 2023-11-27T15:09:43.009 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.665+0000 7fbf2f7fe700 1 -- 172.21.15.52:0/250459403 learned_addr learned my addr 172.21.15.52:0/250459403 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:43.009 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.665+0000 7fbf2f7fe700 1 -- 172.21.15.52:0/250459403 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf20009550 con 0x7fbf30104920 2023-11-27T15:09:43.009 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.665+0000 7fbf2f7fe700 1 --2- 172.21.15.52:0/250459403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf30104920 0x7fbf30079ae0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbf20006af0 tx=0x7fbf20011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:43.010 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.665+0000 7fbf2cff9700 1 -- 172.21.15.52:0/250459403 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf20003a40 con 0x7fbf30104920 2023-11-27T15:09:43.010 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.665+0000 7fbf2cff9700 1 -- 172.21.15.52:0/250459403 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbf20003ba0 con 0x7fbf30104920 2023-11-27T15:09:43.010 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.665+0000 7fbf2cff9700 1 -- 172.21.15.52:0/250459403 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf2001a700 con 0x7fbf30104920 2023-11-27T15:09:43.010 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.665+0000 7fbf35fda700 1 -- 172.21.15.52:0/250459403 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf30108d50 con 0x7fbf30104920 2023-11-27T15:09:43.010 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.665+0000 7fbf35fda700 1 -- 172.21.15.52:0/250459403 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf30108ee0 con 0x7fbf30104920 2023-11-27T15:09:43.010 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.666+0000 7fbf2cff9700 1 -- 172.21.15.52:0/250459403 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fbf2001a930 con 0x7fbf30104920 2023-11-27T15:09:43.010 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.666+0000 7fbf2cff9700 1 --2- 172.21.15.52:0/250459403 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fbf18038760 0x7fbf1803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:43.010 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.667+0000 7fbf2cff9700 1 -- 172.21.15.52:0/250459403 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbf2004c7b0 con 0x7fbf30104920 2023-11-27T15:09:43.010 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.667+0000 7fbf35fda700 1 -- 172.21.15.52:0/250459403 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf30106990 con 0x7fbf30104920 2023-11-27T15:09:43.010 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.667+0000 7fbf2effd700 1 --2- 172.21.15.52:0/250459403 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fbf18038760 0x7fbf1803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:43.010 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.667+0000 7fbf2effd700 1 --2- 172.21.15.52:0/250459403 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fbf18038760 0x7fbf1803ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbf24006fd0 tx=0x7fbf24006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:43.010 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:41.672+0000 7fbf2cff9700 1 -- 172.21.15.52:0/250459403 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbf30106990 con 0x7fbf30104920 2023-11-27T15:09:43.011 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:42.032+0000 7fbf35fda700 1 -- 172.21.15.52:0/250459403 --> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf30106990 con 0x7fbf18038760 2023-11-27T15:09:43.011 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:42.404+0000 7fbf2cff9700 1 -- 172.21.15.52:0/250459403 <== mgr.14120 v2:172.21.15.52:6800/3470359117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbf30106990 con 0x7fbf18038760 2023-11-27T15:09:43.011 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:42.406+0000 7fbf35fda700 1 -- 172.21.15.52:0/250459403 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fbf18038760 msgr2=0x7fbf1803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:43.011 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:42.406+0000 7fbf35fda700 1 --2- 172.21.15.52:0/250459403 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fbf18038760 0x7fbf1803ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbf24006fd0 tx=0x7fbf24006e40 comp rx=0 tx=0).stop 2023-11-27T15:09:43.011 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:42.406+0000 7fbf35fda700 1 -- 172.21.15.52:0/250459403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf30104920 msgr2=0x7fbf30079ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:43.011 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:42.406+0000 7fbf35fda700 1 --2- 172.21.15.52:0/250459403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf30104920 0x7fbf30079ae0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbf20006af0 tx=0x7fbf20011ad0 comp rx=0 tx=0).stop 2023-11-27T15:09:43.011 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:42.406+0000 7fbf35fda700 1 -- 172.21.15.52:0/250459403 shutdown_connections 2023-11-27T15:09:43.011 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:42.406+0000 7fbf35fda700 1 --2- 172.21.15.52:0/250459403 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fbf18038760 0x7fbf1803ac10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:43.011 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:42.406+0000 7fbf35fda700 1 --2- 172.21.15.52:0/250459403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf30104920 0x7fbf30079ae0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:43.011 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:42.406+0000 7fbf35fda700 1 -- 172.21.15.52:0/250459403 >> 172.21.15.52:0/250459403 conn(0x7fbf300ffed0 msgr2=0x7fbf30101570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:43.011 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:42.406+0000 7fbf35fda700 1 -- 172.21.15.52:0/250459403 shutdown_connections 2023-11-27T15:09:43.011 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:42.406+0000 7fbf35fda700 1 -- 172.21.15.52:0/250459403 wait complete. 2023-11-27T15:09:43.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:43 smithi052 ceph-mon[99056]: from='mgr.14120 172.21.15.52:0/3799281879' entity='mgr.x' 2023-11-27T15:09:43.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:43 smithi052 ceph-mon[99056]: from='mgr.14120 172.21.15.52:0/3799281879' entity='mgr.x' 2023-11-27T15:09:44.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLCE2NNTGbbJIJFAABAKWK9Tyg1T6556o23zQIHlyRx96xVQ7mYDOweDwTqeanDrGpJK/B63C3eiBwvXsymgJOflZIMepVY8GNJjbE6C1xp2hiYWzsqDOsNdVQmW9dzwdVzlzg9uBFtVPVmpQLXROYBaOVDK97vIPq36ng4+cENb0mM66xNrBx8KRrF8stCB4LZn/UA+0pUAp0kN8rjLTRGiSyXIDiqKquzgZl2f7M7C38YTKqOPb2FcqMlW7enwLO2QHNEbKAwEnIsRvUns9Oddj+lh8C4Xi89Ag5sqMRc4+NdElMF/M+xFAvkHsCn5ELf96IzyFNE7tQ1JpIKY3xwsFOLz2ShN7z3BAuhReJObR0dnSRetD+5e7I/OT9QxfjYj53zTNXlChj8FohCaYp5vmpO/cPF7LSghFWzCtWgV5UipC5aONBFnvpqVeBhIiGk44qvwnlNZmifTLIaZd4PCtwPajQGsJpRL3enbJCLltUOitSsw2dB7bNRZRYock= ceph-c6c7df88-8d36-11ee-95a2-87774f69a715 2023-11-27T15:09:44.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.497+0000 7f91007e3700 1 Processor -- start 2023-11-27T15:09:44.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.497+0000 7f91007e3700 1 -- start start 2023-11-27T15:09:44.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.498+0000 7f91007e3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90f8103d10 0x7f90f8104120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:44.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.498+0000 7f91007e3700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f81046f0 con 0x7f90f8103d10 2023-11-27T15:09:44.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.498+0000 7f90fe57f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90f8103d10 0x7f90f8104120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:44.347 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.498+0000 7f90fe57f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90f8103d10 0x7f90f8104120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48854/0 (socket says 172.21.15.52:48854) 2023-11-27T15:09:44.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.498+0000 7f90fe57f700 1 -- 172.21.15.52:0/973604455 learned_addr learned my addr 172.21.15.52:0/973604455 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:44.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.498+0000 7f90fe57f700 1 -- 172.21.15.52:0/973604455 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f8104830 con 0x7f90f8103d10 2023-11-27T15:09:44.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.498+0000 7f90fe57f700 1 --2- 172.21.15.52:0/973604455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90f8103d10 0x7f90f8104120 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f90f40098a0 tx=0x7f90f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c17bb2bd3a3bb896 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:44.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.498+0000 7f90fd57d700 1 -- 172.21.15.52:0/973604455 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f90f4004160 con 0x7f90f8103d10 2023-11-27T15:09:44.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.498+0000 7f90fd57d700 1 -- 172.21.15.52:0/973604455 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f90f40042c0 con 0x7f90f8103d10 2023-11-27T15:09:44.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.498+0000 7f90fd57d700 1 -- 172.21.15.52:0/973604455 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f90f4011a00 con 0x7f90f8103d10 2023-11-27T15:09:44.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.499+0000 7f91007e3700 1 -- 172.21.15.52:0/973604455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90f8103d10 msgr2=0x7f90f8104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:44.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.499+0000 7f91007e3700 1 --2- 172.21.15.52:0/973604455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90f8103d10 0x7f90f8104120 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f90f40098a0 tx=0x7f90f4009bb0 comp rx=0 tx=0).stop 2023-11-27T15:09:44.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.499+0000 7f91007e3700 1 -- 172.21.15.52:0/973604455 shutdown_connections 2023-11-27T15:09:44.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.499+0000 7f91007e3700 1 --2- 172.21.15.52:0/973604455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90f8103d10 0x7f90f8104120 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:44.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.499+0000 7f91007e3700 1 -- 172.21.15.52:0/973604455 >> 172.21.15.52:0/973604455 conn(0x7f90f80ff650 msgr2=0x7f90f8101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:44.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.499+0000 7f91007e3700 1 -- 172.21.15.52:0/973604455 shutdown_connections 2023-11-27T15:09:44.348 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.499+0000 7f91007e3700 1 -- 172.21.15.52:0/973604455 wait complete. 2023-11-27T15:09:44.349 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.500+0000 7f91007e3700 1 Processor -- start 2023-11-27T15:09:44.349 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.500+0000 7f91007e3700 1 -- start start 2023-11-27T15:09:44.349 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.500+0000 7f91007e3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90f8103d10 0x7f90f8199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:44.349 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.500+0000 7f91007e3700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f8199c50 con 0x7f90f8103d10 2023-11-27T15:09:44.349 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.500+0000 7f90fe57f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90f8103d10 0x7f90f8199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:44.349 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.501+0000 7f90fe57f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90f8103d10 0x7f90f8199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48856/0 (socket says 172.21.15.52:48856) 2023-11-27T15:09:44.349 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.501+0000 7f90fe57f700 1 -- 172.21.15.52:0/3385969462 learned_addr learned my addr 172.21.15.52:0/3385969462 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:44.349 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.501+0000 7f90fe57f700 1 -- 172.21.15.52:0/3385969462 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f4009550 con 0x7f90f8103d10 2023-11-27T15:09:44.349 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.501+0000 7f90fe57f700 1 --2- 172.21.15.52:0/3385969462 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90f8103d10 0x7f90f8199710 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f90f4006af0 tx=0x7f90f4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:44.349 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.501+0000 7f90ef7fe700 1 -- 172.21.15.52:0/3385969462 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f90f4003a40 con 0x7f90f8103d10 2023-11-27T15:09:44.349 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.501+0000 7f90ef7fe700 1 -- 172.21.15.52:0/3385969462 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f90f4003ba0 con 0x7f90f8103d10 2023-11-27T15:09:44.349 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.501+0000 7f90ef7fe700 1 -- 172.21.15.52:0/3385969462 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f90f401a700 con 0x7f90f8103d10 2023-11-27T15:09:44.350 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.501+0000 7f91007e3700 1 -- 172.21.15.52:0/3385969462 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f8107ae0 con 0x7f90f8103d10 2023-11-27T15:09:44.350 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.501+0000 7f91007e3700 1 -- 172.21.15.52:0/3385969462 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f8107950 con 0x7f90f8103d10 2023-11-27T15:09:44.350 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.502+0000 7f90ef7fe700 1 -- 172.21.15.52:0/3385969462 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f90f4024ad0 con 0x7f90f8103d10 2023-11-27T15:09:44.350 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.502+0000 7f90ef7fe700 1 --2- 172.21.15.52:0/3385969462 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f90e4038760 0x7f90e403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:44.350 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.502+0000 7f91007e3700 1 -- 172.21.15.52:0/3385969462 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f8062120 con 0x7f90f8103d10 2023-11-27T15:09:44.350 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.502+0000 7f90ef7fe700 1 -- 172.21.15.52:0/3385969462 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f90f404c6a0 con 0x7f90f8103d10 2023-11-27T15:09:44.350 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.502+0000 7f90fdd7e700 1 --2- 172.21.15.52:0/3385969462 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f90e4038760 0x7f90e403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:44.350 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.503+0000 7f90fdd7e700 1 --2- 172.21.15.52:0/3385969462 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f90e4038760 0x7f90e403ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f90e8006fd0 tx=0x7f90e8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:44.350 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.507+0000 7f90ef7fe700 1 -- 172.21.15.52:0/3385969462 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f90f4018db0 con 0x7f90f8103d10 2023-11-27T15:09:44.350 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.866+0000 7f91007e3700 1 -- 172.21.15.52:0/3385969462 --> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f90f8101910 con 0x7f90e4038760 2023-11-27T15:09:44.350 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.867+0000 7f90ef7fe700 1 -- 172.21.15.52:0/3385969462 <== mgr.14120 v2:172.21.15.52:6800/3470359117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f90f8101910 con 0x7f90e4038760 2023-11-27T15:09:44.351 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.869+0000 7f91007e3700 1 -- 172.21.15.52:0/3385969462 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f90e4038760 msgr2=0x7f90e403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:44.351 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.869+0000 7f91007e3700 1 --2- 172.21.15.52:0/3385969462 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f90e4038760 0x7f90e403ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f90e8006fd0 tx=0x7f90e8006e40 comp rx=0 tx=0).stop 2023-11-27T15:09:44.351 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.869+0000 7f91007e3700 1 -- 172.21.15.52:0/3385969462 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90f8103d10 msgr2=0x7f90f8199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:44.351 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.869+0000 7f91007e3700 1 --2- 172.21.15.52:0/3385969462 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90f8103d10 0x7f90f8199710 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f90f4006af0 tx=0x7f90f4011ad0 comp rx=0 tx=0).stop 2023-11-27T15:09:44.351 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.869+0000 7f91007e3700 1 -- 172.21.15.52:0/3385969462 shutdown_connections 2023-11-27T15:09:44.351 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.869+0000 7f91007e3700 1 --2- 172.21.15.52:0/3385969462 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f90e4038760 0x7f90e403ac10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:44.351 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.869+0000 7f91007e3700 1 --2- 172.21.15.52:0/3385969462 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90f8103d10 0x7f90f8199710 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:44.351 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.869+0000 7f91007e3700 1 -- 172.21.15.52:0/3385969462 >> 172.21.15.52:0/3385969462 conn(0x7f90f80ff650 msgr2=0x7f90f8100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:44.351 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.869+0000 7f91007e3700 1 -- 172.21.15.52:0/3385969462 shutdown_connections 2023-11-27T15:09:44.351 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:43.869+0000 7f91007e3700 1 -- 172.21.15.52:0/3385969462 wait complete. 2023-11-27T15:09:44.351 INFO:teuthology.orchestra.run.smithi052.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-11-27T15:09:44.351 INFO:teuthology.orchestra.run.smithi052.stdout:Adding key to root@localhost authorized_keys... 2023-11-27T15:09:44.352 INFO:teuthology.orchestra.run.smithi052.stdout:Adding host smithi052... 2023-11-27T15:09:44.577 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:44 smithi052 ceph-mon[99056]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:09:46.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:46 smithi052 ceph-mon[99056]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi052", "addr": "172.21.15.52", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:09:46.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:46 smithi052 ceph-mon[99056]: Deploying cephadm binary to smithi052 2023-11-27T15:09:48.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout Added host 'smithi052' with addr '172.21.15.52' 2023-11-27T15:09:48.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.841+0000 7fde38de6700 1 Processor -- start 2023-11-27T15:09:48.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.842+0000 7fde38de6700 1 -- start start 2023-11-27T15:09:48.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.842+0000 7fde38de6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde34106490 0x7fde34108870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:48.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.842+0000 7fde38de6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde34079e60 con 0x7fde34106490 2023-11-27T15:09:48.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.842+0000 7fde3259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde34106490 0x7fde34108870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:48.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.842+0000 7fde3259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde34106490 0x7fde34108870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48860/0 (socket says 172.21.15.52:48860) 2023-11-27T15:09:48.001 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.842+0000 7fde3259c700 1 -- 172.21.15.52:0/2033240547 learned_addr learned my addr 172.21.15.52:0/2033240547 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:48.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.843+0000 7fde3259c700 1 -- 172.21.15.52:0/2033240547 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde34108db0 con 0x7fde34106490 2023-11-27T15:09:48.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.843+0000 7fde3259c700 1 --2- 172.21.15.52:0/2033240547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde34106490 0x7fde34108870 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fde240098a0 tx=0x7fde24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61718905443d01 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:48.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.843+0000 7fde3159a700 1 -- 172.21.15.52:0/2033240547 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde24004160 con 0x7fde34106490 2023-11-27T15:09:48.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.843+0000 7fde3159a700 1 -- 172.21.15.52:0/2033240547 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fde240042c0 con 0x7fde34106490 2023-11-27T15:09:48.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.843+0000 7fde3159a700 1 -- 172.21.15.52:0/2033240547 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde24011a00 con 0x7fde34106490 2023-11-27T15:09:48.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.844+0000 7fde38de6700 1 -- 172.21.15.52:0/2033240547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde34106490 msgr2=0x7fde34108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:48.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.844+0000 7fde38de6700 1 --2- 172.21.15.52:0/2033240547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde34106490 0x7fde34108870 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fde240098a0 tx=0x7fde24009bb0 comp rx=0 tx=0).stop 2023-11-27T15:09:48.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.844+0000 7fde38de6700 1 -- 172.21.15.52:0/2033240547 shutdown_connections 2023-11-27T15:09:48.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.844+0000 7fde38de6700 1 --2- 172.21.15.52:0/2033240547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde34106490 0x7fde34108870 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:48.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.844+0000 7fde38de6700 1 -- 172.21.15.52:0/2033240547 >> 172.21.15.52:0/2033240547 conn(0x7fde340ffed0 msgr2=0x7fde34102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:48.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.844+0000 7fde38de6700 1 -- 172.21.15.52:0/2033240547 shutdown_connections 2023-11-27T15:09:48.002 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.844+0000 7fde38de6700 1 -- 172.21.15.52:0/2033240547 wait complete. 2023-11-27T15:09:48.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.845+0000 7fde38de6700 1 Processor -- start 2023-11-27T15:09:48.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.845+0000 7fde38de6700 1 -- start start 2023-11-27T15:09:48.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.845+0000 7fde38de6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde34106490 0x7fde34105df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:48.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.845+0000 7fde38de6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde34106330 con 0x7fde34106490 2023-11-27T15:09:48.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.845+0000 7fde3259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde34106490 0x7fde34105df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:48.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.845+0000 7fde3259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde34106490 0x7fde34105df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48862/0 (socket says 172.21.15.52:48862) 2023-11-27T15:09:48.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.845+0000 7fde3259c700 1 -- 172.21.15.52:0/2800924392 learned_addr learned my addr 172.21.15.52:0/2800924392 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:48.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.845+0000 7fde3259c700 1 -- 172.21.15.52:0/2800924392 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde24009550 con 0x7fde34106490 2023-11-27T15:09:48.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.845+0000 7fde3259c700 1 --2- 172.21.15.52:0/2800924392 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde34106490 0x7fde34105df0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fde24009fb0 tx=0x7fde24011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:48.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.845+0000 7fde237fe700 1 -- 172.21.15.52:0/2800924392 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde24003930 con 0x7fde34106490 2023-11-27T15:09:48.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.846+0000 7fde237fe700 1 -- 172.21.15.52:0/2800924392 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fde24003a90 con 0x7fde34106490 2023-11-27T15:09:48.003 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.846+0000 7fde38de6700 1 -- 172.21.15.52:0/2800924392 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde34109d40 con 0x7fde34106490 2023-11-27T15:09:48.004 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.846+0000 7fde237fe700 1 -- 172.21.15.52:0/2800924392 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde24019610 con 0x7fde34106490 2023-11-27T15:09:48.004 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.846+0000 7fde38de6700 1 -- 172.21.15.52:0/2800924392 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde34109bb0 con 0x7fde34106490 2023-11-27T15:09:48.004 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.846+0000 7fde237fe700 1 -- 172.21.15.52:0/2800924392 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fde240198d0 con 0x7fde34106490 2023-11-27T15:09:48.004 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.847+0000 7fde237fe700 1 --2- 172.21.15.52:0/2800924392 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fde1c0386c0 0x7fde1c03ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:48.004 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.847+0000 7fde237fe700 1 -- 172.21.15.52:0/2800924392 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fde2404c8b0 con 0x7fde34106490 2023-11-27T15:09:48.004 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.847+0000 7fde31d9b700 1 --2- 172.21.15.52:0/2800924392 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fde1c0386c0 0x7fde1c03ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:48.004 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.847+0000 7fde38de6700 1 -- 172.21.15.52:0/2800924392 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde34062120 con 0x7fde34106490 2023-11-27T15:09:48.004 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.847+0000 7fde31d9b700 1 --2- 172.21.15.52:0/2800924392 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fde1c0386c0 0x7fde1c03ab70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fde28006fd0 tx=0x7fde28006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:48.004 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:44.852+0000 7fde237fe700 1 -- 172.21.15.52:0/2800924392 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fde24019bd0 con 0x7fde34106490 2023-11-27T15:09:48.004 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:45.218+0000 7fde38de6700 1 -- 172.21.15.52:0/2800924392 --> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi052", "addr": "172.21.15.52", "target": ["mon-mgr", ""]}) v1 -- 0x7fde34105310 con 0x7fde1c0386c0 2023-11-27T15:09:48.004 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:47.635+0000 7fde237fe700 1 -- 172.21.15.52:0/2800924392 <== mgr.14120 v2:172.21.15.52:6800/3470359117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fde34105310 con 0x7fde1c0386c0 2023-11-27T15:09:48.004 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:47.637+0000 7fde38de6700 1 -- 172.21.15.52:0/2800924392 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fde1c0386c0 msgr2=0x7fde1c03ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:48.005 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:47.637+0000 7fde38de6700 1 --2- 172.21.15.52:0/2800924392 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fde1c0386c0 0x7fde1c03ab70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fde28006fd0 tx=0x7fde28006e40 comp rx=0 tx=0).stop 2023-11-27T15:09:48.005 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:47.637+0000 7fde38de6700 1 -- 172.21.15.52:0/2800924392 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde34106490 msgr2=0x7fde34105df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:48.005 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:47.637+0000 7fde38de6700 1 --2- 172.21.15.52:0/2800924392 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde34106490 0x7fde34105df0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fde24009fb0 tx=0x7fde24011ad0 comp rx=0 tx=0).stop 2023-11-27T15:09:48.005 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:47.638+0000 7fde38de6700 1 -- 172.21.15.52:0/2800924392 shutdown_connections 2023-11-27T15:09:48.005 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:47.638+0000 7fde38de6700 1 --2- 172.21.15.52:0/2800924392 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fde1c0386c0 0x7fde1c03ab70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:48.005 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:47.638+0000 7fde38de6700 1 --2- 172.21.15.52:0/2800924392 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde34106490 0x7fde34105df0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:48.005 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:47.638+0000 7fde38de6700 1 -- 172.21.15.52:0/2800924392 >> 172.21.15.52:0/2800924392 conn(0x7fde340ffed0 msgr2=0x7fde34100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:48.005 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:47.638+0000 7fde38de6700 1 -- 172.21.15.52:0/2800924392 shutdown_connections 2023-11-27T15:09:48.005 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:47.638+0000 7fde38de6700 1 -- 172.21.15.52:0/2800924392 wait complete. 2023-11-27T15:09:48.005 INFO:teuthology.orchestra.run.smithi052.stdout:Deploying unmanaged mon service... 2023-11-27T15:09:48.960 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:48 smithi052 ceph-mon[99056]: from='mgr.14120 172.21.15.52:0/3799281879' entity='mgr.x' 2023-11-27T15:09:48.961 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:48 smithi052 ceph-mon[99056]: Added host smithi052 2023-11-27T15:09:48.961 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:48 smithi052 ceph-mon[99056]: from='mgr.14120 172.21.15.52:0/3799281879' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:09:49.547 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-11-27T15:09:49.547 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.514+0000 7f9071b32700 1 Processor -- start 2023-11-27T15:09:49.547 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.514+0000 7f9071b32700 1 -- start start 2023-11-27T15:09:49.547 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.514+0000 7f9071b32700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f906c104e20 0x7f906c105230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:49.547 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.514+0000 7f9071b32700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c105800 con 0x7f906c104e20 2023-11-27T15:09:49.547 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.515+0000 7f906b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f906c104e20 0x7f906c105230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:49.547 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.515+0000 7f906b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f906c104e20 0x7f906c105230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48896/0 (socket says 172.21.15.52:48896) 2023-11-27T15:09:49.547 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.515+0000 7f906b7fe700 1 -- 172.21.15.52:0/371113391 learned_addr learned my addr 172.21.15.52:0/371113391 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:49.547 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.515+0000 7f906b7fe700 1 -- 172.21.15.52:0/371113391 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f906c105b40 con 0x7f906c104e20 2023-11-27T15:09:49.548 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.515+0000 7f906b7fe700 1 --2- 172.21.15.52:0/371113391 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f906c104e20 0x7f906c105230 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f905400bb40 tx=0x7f905400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a8b1c321bde706 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:49.548 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.515+0000 7f906a7fc700 1 -- 172.21.15.52:0/371113391 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f905400dbe0 con 0x7f906c104e20 2023-11-27T15:09:49.548 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.515+0000 7f906a7fc700 1 -- 172.21.15.52:0/371113391 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f90540044d0 con 0x7f906c104e20 2023-11-27T15:09:49.548 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.515+0000 7f906a7fc700 1 -- 172.21.15.52:0/371113391 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9054010470 con 0x7f906c104e20 2023-11-27T15:09:49.548 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.516+0000 7f9071b32700 1 -- 172.21.15.52:0/371113391 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f906c104e20 msgr2=0x7f906c105230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:49.548 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.516+0000 7f9071b32700 1 --2- 172.21.15.52:0/371113391 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f906c104e20 0x7f906c105230 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f905400bb40 tx=0x7f905400be50 comp rx=0 tx=0).stop 2023-11-27T15:09:49.548 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.516+0000 7f9071b32700 1 -- 172.21.15.52:0/371113391 shutdown_connections 2023-11-27T15:09:49.548 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.516+0000 7f9071b32700 1 --2- 172.21.15.52:0/371113391 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f906c104e20 0x7f906c105230 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:49.548 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.516+0000 7f9071b32700 1 -- 172.21.15.52:0/371113391 >> 172.21.15.52:0/371113391 conn(0x7f906c0ff650 msgr2=0x7f906c101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:49.548 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.516+0000 7f9071b32700 1 -- 172.21.15.52:0/371113391 shutdown_connections 2023-11-27T15:09:49.548 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.516+0000 7f9071b32700 1 -- 172.21.15.52:0/371113391 wait complete. 2023-11-27T15:09:49.548 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.517+0000 7f9071b32700 1 Processor -- start 2023-11-27T15:09:49.548 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.517+0000 7f9071b32700 1 -- start start 2023-11-27T15:09:49.549 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.517+0000 7f9071b32700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f906c104e20 0x7f906c197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:49.549 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.517+0000 7f9071b32700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c197a70 con 0x7f906c104e20 2023-11-27T15:09:49.549 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.517+0000 7f906b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f906c104e20 0x7f906c197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:49.549 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.517+0000 7f906b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f906c104e20 0x7f906c197530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48898/0 (socket says 172.21.15.52:48898) 2023-11-27T15:09:49.549 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.517+0000 7f906b7fe700 1 -- 172.21.15.52:0/1812077663 learned_addr learned my addr 172.21.15.52:0/1812077663 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:49.549 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.517+0000 7f906b7fe700 1 -- 172.21.15.52:0/1812077663 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905400b7f0 con 0x7f906c104e20 2023-11-27T15:09:49.549 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.517+0000 7f906b7fe700 1 --2- 172.21.15.52:0/1812077663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f906c104e20 0x7f906c197530 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f905400b340 tx=0x7f9054003df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:49.549 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.518+0000 7f9068ff9700 1 -- 172.21.15.52:0/1812077663 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f905401e070 con 0x7f906c104e20 2023-11-27T15:09:49.549 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.518+0000 7f9068ff9700 1 -- 172.21.15.52:0/1812077663 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9054004300 con 0x7f906c104e20 2023-11-27T15:09:49.549 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.518+0000 7f9071b32700 1 -- 172.21.15.52:0/1812077663 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f906c103980 con 0x7f906c104e20 2023-11-27T15:09:49.549 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.518+0000 7f9068ff9700 1 -- 172.21.15.52:0/1812077663 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9054022ac0 con 0x7f906c104e20 2023-11-27T15:09:49.549 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.518+0000 7f9071b32700 1 -- 172.21.15.52:0/1812077663 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f906c106880 con 0x7f906c104e20 2023-11-27T15:09:49.550 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.519+0000 7f9068ff9700 1 -- 172.21.15.52:0/1812077663 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f905402c440 con 0x7f906c104e20 2023-11-27T15:09:49.550 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.519+0000 7f9068ff9700 1 --2- 172.21.15.52:0/1812077663 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f9058038710 0x7f905803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:49.550 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.519+0000 7f9068ff9700 1 -- 172.21.15.52:0/1812077663 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f905404eea0 con 0x7f906c104e20 2023-11-27T15:09:49.550 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.519+0000 7f906affd700 1 --2- 172.21.15.52:0/1812077663 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f9058038710 0x7f905803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:49.550 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.519+0000 7f9071b32700 1 -- 172.21.15.52:0/1812077663 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f906c103b10 con 0x7f906c104e20 2023-11-27T15:09:49.550 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.519+0000 7f906affd700 1 --2- 172.21.15.52:0/1812077663 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f9058038710 0x7f905803abc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f905c006fd0 tx=0x7f905c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:49.550 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.524+0000 7f9068ff9700 1 -- 172.21.15.52:0/1812077663 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9054027030 con 0x7f906c104e20 2023-11-27T15:09:49.550 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.919+0000 7f9071b32700 1 -- 172.21.15.52:0/1812077663 --> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f906c1981b0 con 0x7f9058038710 2023-11-27T15:09:49.550 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.924+0000 7f9068ff9700 1 -- 172.21.15.52:0/1812077663 <== mgr.14120 v2:172.21.15.52:6800/3470359117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f906c1981b0 con 0x7f9058038710 2023-11-27T15:09:49.550 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.926+0000 7f9071b32700 1 -- 172.21.15.52:0/1812077663 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f9058038710 msgr2=0x7f905803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:49.550 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.926+0000 7f9071b32700 1 --2- 172.21.15.52:0/1812077663 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f9058038710 0x7f905803abc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f905c006fd0 tx=0x7f905c006e40 comp rx=0 tx=0).stop 2023-11-27T15:09:49.551 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.926+0000 7f9071b32700 1 -- 172.21.15.52:0/1812077663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f906c104e20 msgr2=0x7f906c197530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:49.551 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.926+0000 7f9071b32700 1 --2- 172.21.15.52:0/1812077663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f906c104e20 0x7f906c197530 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f905400b340 tx=0x7f9054003df0 comp rx=0 tx=0).stop 2023-11-27T15:09:49.551 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.927+0000 7f9071b32700 1 -- 172.21.15.52:0/1812077663 shutdown_connections 2023-11-27T15:09:49.551 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.927+0000 7f9071b32700 1 --2- 172.21.15.52:0/1812077663 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f9058038710 0x7f905803abc0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:49.551 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.927+0000 7f9071b32700 1 --2- 172.21.15.52:0/1812077663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f906c104e20 0x7f906c197530 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:49.551 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.927+0000 7f9071b32700 1 -- 172.21.15.52:0/1812077663 >> 172.21.15.52:0/1812077663 conn(0x7f906c0ff650 msgr2=0x7f906c18e310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:49.551 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.927+0000 7f9071b32700 1 -- 172.21.15.52:0/1812077663 shutdown_connections 2023-11-27T15:09:49.551 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:48.927+0000 7f9071b32700 1 -- 172.21.15.52:0/1812077663 wait complete. 2023-11-27T15:09:49.551 INFO:teuthology.orchestra.run.smithi052.stdout:Deploying unmanaged mgr service... 2023-11-27T15:09:50.080 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:49 smithi052 ceph-mon[99056]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:09:50.081 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:49 smithi052 ceph-mon[99056]: Saving service mon spec with placement count:5 2023-11-27T15:09:50.081 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:49 smithi052 ceph-mon[99056]: from='mgr.14120 172.21.15.52:0/3799281879' entity='mgr.x' 2023-11-27T15:09:51.085 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-11-27T15:09:51.086 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.309+0000 7f942a393700 1 Processor -- start 2023-11-27T15:09:51.086 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.309+0000 7f942a393700 1 -- start start 2023-11-27T15:09:51.086 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.309+0000 7f942a393700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9424105a20 0x7f9424105e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:51.086 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.309+0000 7f942a393700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9424106400 con 0x7f9424105a20 2023-11-27T15:09:51.086 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.310+0000 7f9423fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9424105a20 0x7f9424105e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:51.086 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.310+0000 7f9423fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9424105a20 0x7f9424105e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48902/0 (socket says 172.21.15.52:48902) 2023-11-27T15:09:51.086 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.310+0000 7f9423fff700 1 -- 172.21.15.52:0/3616021594 learned_addr learned my addr 172.21.15.52:0/3616021594 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:51.086 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.310+0000 7f9423fff700 1 -- 172.21.15.52:0/3616021594 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9424106c20 con 0x7f9424105a20 2023-11-27T15:09:51.086 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.310+0000 7f9423fff700 1 --2- 172.21.15.52:0/3616021594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9424105a20 0x7f9424105e30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f94140098a0 tx=0x7f9414009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5383c0040f58e96b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:51.086 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.310+0000 7f9422ffd700 1 -- 172.21.15.52:0/3616021594 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9414004160 con 0x7f9424105a20 2023-11-27T15:09:51.087 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.310+0000 7f9422ffd700 1 -- 172.21.15.52:0/3616021594 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94140042c0 con 0x7f9424105a20 2023-11-27T15:09:51.087 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.310+0000 7f9422ffd700 1 -- 172.21.15.52:0/3616021594 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94140119b0 con 0x7f9424105a20 2023-11-27T15:09:51.087 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.311+0000 7f942a393700 1 -- 172.21.15.52:0/3616021594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9424105a20 msgr2=0x7f9424105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:51.087 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.311+0000 7f942a393700 1 --2- 172.21.15.52:0/3616021594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9424105a20 0x7f9424105e30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f94140098a0 tx=0x7f9414009bb0 comp rx=0 tx=0).stop 2023-11-27T15:09:51.087 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.311+0000 7f942a393700 1 -- 172.21.15.52:0/3616021594 shutdown_connections 2023-11-27T15:09:51.087 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.311+0000 7f942a393700 1 --2- 172.21.15.52:0/3616021594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9424105a20 0x7f9424105e30 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:51.087 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.311+0000 7f942a393700 1 -- 172.21.15.52:0/3616021594 >> 172.21.15.52:0/3616021594 conn(0x7f9424100fd0 msgr2=0x7f9424103400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:51.087 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.311+0000 7f942a393700 1 -- 172.21.15.52:0/3616021594 shutdown_connections 2023-11-27T15:09:51.087 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.311+0000 7f942a393700 1 -- 172.21.15.52:0/3616021594 wait complete. 2023-11-27T15:09:51.087 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.311+0000 7f942a393700 1 Processor -- start 2023-11-27T15:09:51.087 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.312+0000 7f942a393700 1 -- start start 2023-11-27T15:09:51.087 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.312+0000 7f942a393700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9424105a20 0x7f94241997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:51.088 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.312+0000 7f942a393700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9424199cf0 con 0x7f9424105a20 2023-11-27T15:09:51.088 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.312+0000 7f9423fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9424105a20 0x7f94241997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:51.088 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.312+0000 7f9423fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9424105a20 0x7f94241997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48904/0 (socket says 172.21.15.52:48904) 2023-11-27T15:09:51.088 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.312+0000 7f9423fff700 1 -- 172.21.15.52:0/2859914927 learned_addr learned my addr 172.21.15.52:0/2859914927 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:51.088 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.312+0000 7f9423fff700 1 -- 172.21.15.52:0/2859914927 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9414009550 con 0x7f9424105a20 2023-11-27T15:09:51.088 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.312+0000 7f9423fff700 1 --2- 172.21.15.52:0/2859914927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9424105a20 0x7f94241997b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9414003e60 tx=0x7f9414011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:51.088 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.313+0000 7f94217fa700 1 -- 172.21.15.52:0/2859914927 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9414003970 con 0x7f9424105a20 2023-11-27T15:09:51.088 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.313+0000 7f94217fa700 1 -- 172.21.15.52:0/2859914927 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9414003ad0 con 0x7f9424105a20 2023-11-27T15:09:51.088 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.313+0000 7f942a393700 1 -- 172.21.15.52:0/2859914927 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9424109fe0 con 0x7f9424105a20 2023-11-27T15:09:51.088 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.313+0000 7f94217fa700 1 -- 172.21.15.52:0/2859914927 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94140195e0 con 0x7f9424105a20 2023-11-27T15:09:51.088 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.313+0000 7f942a393700 1 -- 172.21.15.52:0/2859914927 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9424109e50 con 0x7f9424105a20 2023-11-27T15:09:51.088 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.314+0000 7f94217fa700 1 -- 172.21.15.52:0/2859914927 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f94140198a0 con 0x7f9424105a20 2023-11-27T15:09:51.089 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.314+0000 7f942a393700 1 -- 172.21.15.52:0/2859914927 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9424062190 con 0x7f9424105a20 2023-11-27T15:09:51.089 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.314+0000 7f94217fa700 1 --2- 172.21.15.52:0/2859914927 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f940c038710 0x7f940c03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:51.089 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.314+0000 7f94217fa700 1 -- 172.21.15.52:0/2859914927 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f941404c8e0 con 0x7f9424105a20 2023-11-27T15:09:51.089 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.314+0000 7f94237fe700 1 --2- 172.21.15.52:0/2859914927 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f940c038710 0x7f940c03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:51.089 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.315+0000 7f94237fe700 1 --2- 172.21.15.52:0/2859914927 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f940c038710 0x7f940c03abc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9418006fd0 tx=0x7f9418006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:51.089 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.319+0000 7f94217fa700 1 -- 172.21.15.52:0/2859914927 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9424062190 con 0x7f9424105a20 2023-11-27T15:09:51.089 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.711+0000 7f942a393700 1 -- 172.21.15.52:0/2859914927 --> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f9424062190 con 0x7f940c038710 2023-11-27T15:09:51.089 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.716+0000 7f94217fa700 1 -- 172.21.15.52:0/2859914927 <== mgr.14120 v2:172.21.15.52:6800/3470359117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f9424062190 con 0x7f940c038710 2023-11-27T15:09:51.089 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.718+0000 7f942a393700 1 -- 172.21.15.52:0/2859914927 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f940c038710 msgr2=0x7f940c03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:51.089 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.718+0000 7f942a393700 1 --2- 172.21.15.52:0/2859914927 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f940c038710 0x7f940c03abc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9418006fd0 tx=0x7f9418006e40 comp rx=0 tx=0).stop 2023-11-27T15:09:51.089 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.718+0000 7f942a393700 1 -- 172.21.15.52:0/2859914927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9424105a20 msgr2=0x7f94241997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:51.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.718+0000 7f942a393700 1 --2- 172.21.15.52:0/2859914927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9424105a20 0x7f94241997b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9414003e60 tx=0x7f9414011aa0 comp rx=0 tx=0).stop 2023-11-27T15:09:51.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.718+0000 7f942a393700 1 -- 172.21.15.52:0/2859914927 shutdown_connections 2023-11-27T15:09:51.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.718+0000 7f942a393700 1 --2- 172.21.15.52:0/2859914927 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f940c038710 0x7f940c03abc0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:51.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.718+0000 7f942a393700 1 --2- 172.21.15.52:0/2859914927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9424105a20 0x7f94241997b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:51.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.718+0000 7f942a393700 1 -- 172.21.15.52:0/2859914927 >> 172.21.15.52:0/2859914927 conn(0x7f9424100fd0 msgr2=0x7f9424102680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:51.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.718+0000 7f942a393700 1 -- 172.21.15.52:0/2859914927 shutdown_connections 2023-11-27T15:09:51.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:50.718+0000 7f942a393700 1 -- 172.21.15.52:0/2859914927 wait complete. 2023-11-27T15:09:51.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:50 smithi052 ceph-mon[99056]: from='mgr.14120 172.21.15.52:0/3799281879' entity='mgr.x' 2023-11-27T15:09:51.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:50 smithi052 ceph-mon[99056]: from='mgr.14120 172.21.15.52:0/3799281879' entity='mgr.x' 2023-11-27T15:09:51.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:50 smithi052 ceph-mon[99056]: from='mgr.14120 172.21.15.52:0/3799281879' entity='mgr.x' 2023-11-27T15:09:52.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:51 smithi052 ceph-mon[99056]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:09:52.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:51 smithi052 ceph-mon[99056]: Saving service mgr spec with placement count:2 2023-11-27T15:09:52.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:51 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/695337874' entity='client.admin' 2023-11-27T15:09:52.430 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.564+0000 7fa27c6fe700 1 Processor -- start 2023-11-27T15:09:52.430 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.564+0000 7fa27c6fe700 1 -- start start 2023-11-27T15:09:52.430 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.564+0000 7fa27c6fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2741064d0 0x7fa2741088b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:52.430 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.564+0000 7fa27c6fe700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa274079e20 con 0x7fa2741064d0 2023-11-27T15:09:52.430 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.564+0000 7fa27a49a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2741064d0 0x7fa2741088b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:52.430 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.564+0000 7fa27a49a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2741064d0 0x7fa2741088b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48908/0 (socket says 172.21.15.52:48908) 2023-11-27T15:09:52.430 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.564+0000 7fa27a49a700 1 -- 172.21.15.52:0/588250642 learned_addr learned my addr 172.21.15.52:0/588250642 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:52.431 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.565+0000 7fa27a49a700 1 -- 172.21.15.52:0/588250642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa274108df0 con 0x7fa2741064d0 2023-11-27T15:09:52.431 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.565+0000 7fa27a49a700 1 --2- 172.21.15.52:0/588250642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2741064d0 0x7fa2741088b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa2700098a0 tx=0x7fa270009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5758cae02f7ce697 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:52.431 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.565+0000 7fa279498700 1 -- 172.21.15.52:0/588250642 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa270004160 con 0x7fa2741064d0 2023-11-27T15:09:52.431 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.565+0000 7fa279498700 1 -- 172.21.15.52:0/588250642 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2700042c0 con 0x7fa2741064d0 2023-11-27T15:09:52.431 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.565+0000 7fa279498700 1 -- 172.21.15.52:0/588250642 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa2700119b0 con 0x7fa2741064d0 2023-11-27T15:09:52.431 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.566+0000 7fa27c6fe700 1 -- 172.21.15.52:0/588250642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2741064d0 msgr2=0x7fa2741088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:52.431 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.566+0000 7fa27c6fe700 1 --2- 172.21.15.52:0/588250642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2741064d0 0x7fa2741088b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa2700098a0 tx=0x7fa270009bb0 comp rx=0 tx=0).stop 2023-11-27T15:09:52.431 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.566+0000 7fa27c6fe700 1 -- 172.21.15.52:0/588250642 shutdown_connections 2023-11-27T15:09:52.431 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.566+0000 7fa27c6fe700 1 --2- 172.21.15.52:0/588250642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2741064d0 0x7fa2741088b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:52.431 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.566+0000 7fa27c6fe700 1 -- 172.21.15.52:0/588250642 >> 172.21.15.52:0/588250642 conn(0x7fa2740ffed0 msgr2=0x7fa274102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:52.431 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.566+0000 7fa27c6fe700 1 -- 172.21.15.52:0/588250642 shutdown_connections 2023-11-27T15:09:52.431 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.566+0000 7fa27c6fe700 1 -- 172.21.15.52:0/588250642 wait complete. 2023-11-27T15:09:52.432 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.566+0000 7fa27c6fe700 1 Processor -- start 2023-11-27T15:09:52.432 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.566+0000 7fa27c6fe700 1 -- start start 2023-11-27T15:09:52.432 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.567+0000 7fa27c6fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2741064d0 0x7fa274105e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:52.432 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.567+0000 7fa27c6fe700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa274106370 con 0x7fa2741064d0 2023-11-27T15:09:52.432 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.567+0000 7fa27a49a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2741064d0 0x7fa274105e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:52.432 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.567+0000 7fa27a49a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2741064d0 0x7fa274105e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48910/0 (socket says 172.21.15.52:48910) 2023-11-27T15:09:52.432 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.567+0000 7fa27a49a700 1 -- 172.21.15.52:0/695337874 learned_addr learned my addr 172.21.15.52:0/695337874 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:52.432 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.567+0000 7fa27a49a700 1 -- 172.21.15.52:0/695337874 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa270009550 con 0x7fa2741064d0 2023-11-27T15:09:52.432 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.567+0000 7fa27a49a700 1 --2- 172.21.15.52:0/695337874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2741064d0 0x7fa274105e30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa270011980 tx=0x7fa270011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:52.432 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.567+0000 7fa26b7fe700 1 -- 172.21.15.52:0/695337874 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa270003a60 con 0x7fa2741064d0 2023-11-27T15:09:52.432 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.567+0000 7fa26b7fe700 1 -- 172.21.15.52:0/695337874 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa270003bc0 con 0x7fa2741064d0 2023-11-27T15:09:52.433 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.567+0000 7fa27c6fe700 1 -- 172.21.15.52:0/695337874 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa274109d60 con 0x7fa2741064d0 2023-11-27T15:09:52.433 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.567+0000 7fa26b7fe700 1 -- 172.21.15.52:0/695337874 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa27001a670 con 0x7fa2741064d0 2023-11-27T15:09:52.433 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.567+0000 7fa27c6fe700 1 -- 172.21.15.52:0/695337874 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa274109bd0 con 0x7fa2741064d0 2023-11-27T15:09:52.433 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.568+0000 7fa26b7fe700 1 -- 172.21.15.52:0/695337874 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fa270024a00 con 0x7fa2741064d0 2023-11-27T15:09:52.433 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.568+0000 7fa26b7fe700 1 --2- 172.21.15.52:0/695337874 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fa2600387b0 0x7fa26003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:52.433 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.569+0000 7fa26b7fe700 1 -- 172.21.15.52:0/695337874 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa27004c8b0 con 0x7fa2741064d0 2023-11-27T15:09:52.433 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.569+0000 7fa27c6fe700 1 -- 172.21.15.52:0/695337874 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa274062120 con 0x7fa2741064d0 2023-11-27T15:09:52.433 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.569+0000 7fa279c99700 1 --2- 172.21.15.52:0/695337874 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fa2600387b0 0x7fa26003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:52.433 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.569+0000 7fa279c99700 1 --2- 172.21.15.52:0/695337874 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fa2600387b0 0x7fa26003ac60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa264006fd0 tx=0x7fa264006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:52.433 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.573+0000 7fa26b7fe700 1 -- 172.21.15.52:0/695337874 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa270024440 con 0x7fa2741064d0 2023-11-27T15:09:52.433 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.945+0000 7fa27c6fe700 1 -- 172.21.15.52:0/695337874 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fa274105230 con 0x7fa2741064d0 2023-11-27T15:09:52.433 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.948+0000 7fa26b7fe700 1 -- 172.21.15.52:0/695337874 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fa270018350 con 0x7fa2741064d0 2023-11-27T15:09:52.434 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.950+0000 7fa27c6fe700 1 -- 172.21.15.52:0/695337874 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fa2600387b0 msgr2=0x7fa26003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:52.434 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.950+0000 7fa27c6fe700 1 --2- 172.21.15.52:0/695337874 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fa2600387b0 0x7fa26003ac60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa264006fd0 tx=0x7fa264006e40 comp rx=0 tx=0).stop 2023-11-27T15:09:52.434 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.950+0000 7fa27c6fe700 1 -- 172.21.15.52:0/695337874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2741064d0 msgr2=0x7fa274105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:52.434 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.950+0000 7fa27c6fe700 1 --2- 172.21.15.52:0/695337874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2741064d0 0x7fa274105e30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa270011980 tx=0x7fa270011a60 comp rx=0 tx=0).stop 2023-11-27T15:09:52.434 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.950+0000 7fa27c6fe700 1 -- 172.21.15.52:0/695337874 shutdown_connections 2023-11-27T15:09:52.434 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.950+0000 7fa27c6fe700 1 --2- 172.21.15.52:0/695337874 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fa2600387b0 0x7fa26003ac60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:52.434 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.950+0000 7fa27c6fe700 1 --2- 172.21.15.52:0/695337874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa2741064d0 0x7fa274105e30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:52.434 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.950+0000 7fa27c6fe700 1 -- 172.21.15.52:0/695337874 >> 172.21.15.52:0/695337874 conn(0x7fa2740ffed0 msgr2=0x7fa2741015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:52.434 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.950+0000 7fa27c6fe700 1 -- 172.21.15.52:0/695337874 shutdown_connections 2023-11-27T15:09:52.434 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:51.950+0000 7fa27c6fe700 1 -- 172.21.15.52:0/695337874 wait complete. 2023-11-27T15:09:53.780 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:53 smithi052 ceph-mon[99056]: from='mgr.14120 172.21.15.52:0/3799281879' entity='mgr.x' 2023-11-27T15:09:54.077 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.231+0000 7f2251de7700 1 Processor -- start 2023-11-27T15:09:54.078 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.231+0000 7f2251de7700 1 -- start start 2023-11-27T15:09:54.078 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.231+0000 7f2251de7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f224c105f00 0x7f224c106310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:54.078 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.231+0000 7f2251de7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224c1068e0 con 0x7f224c105f00 2023-11-27T15:09:54.078 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.232+0000 7f224b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f224c105f00 0x7f224c106310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:54.078 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.232+0000 7f224b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f224c105f00 0x7f224c106310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48914/0 (socket says 172.21.15.52:48914) 2023-11-27T15:09:54.078 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.232+0000 7f224b7fe700 1 -- 172.21.15.52:0/1622525681 learned_addr learned my addr 172.21.15.52:0/1622525681 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:54.078 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.232+0000 7f224b7fe700 1 -- 172.21.15.52:0/1622525681 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f224c104b10 con 0x7f224c105f00 2023-11-27T15:09:54.078 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.232+0000 7f224b7fe700 1 --2- 172.21.15.52:0/1622525681 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f224c105f00 0x7f224c106310 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f223c00bb40 tx=0x7f223c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1428f3bac8c92e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:54.078 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.232+0000 7f224a7fc700 1 -- 172.21.15.52:0/1622525681 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f223c00dbe0 con 0x7f224c105f00 2023-11-27T15:09:54.079 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.232+0000 7f224a7fc700 1 -- 172.21.15.52:0/1622525681 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f223c0044f0 con 0x7f224c105f00 2023-11-27T15:09:54.079 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.232+0000 7f224a7fc700 1 -- 172.21.15.52:0/1622525681 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f223c003e60 con 0x7f224c105f00 2023-11-27T15:09:54.079 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.233+0000 7f2251de7700 1 -- 172.21.15.52:0/1622525681 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f224c105f00 msgr2=0x7f224c106310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:54.079 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.233+0000 7f2251de7700 1 --2- 172.21.15.52:0/1622525681 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f224c105f00 0x7f224c106310 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f223c00bb40 tx=0x7f223c00be50 comp rx=0 tx=0).stop 2023-11-27T15:09:54.079 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.233+0000 7f2251de7700 1 -- 172.21.15.52:0/1622525681 shutdown_connections 2023-11-27T15:09:54.079 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.233+0000 7f2251de7700 1 --2- 172.21.15.52:0/1622525681 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f224c105f00 0x7f224c106310 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:54.079 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.233+0000 7f2251de7700 1 -- 172.21.15.52:0/1622525681 >> 172.21.15.52:0/1622525681 conn(0x7f224c0ff650 msgr2=0x7f224c101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:54.079 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.233+0000 7f2251de7700 1 -- 172.21.15.52:0/1622525681 shutdown_connections 2023-11-27T15:09:54.079 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.233+0000 7f2251de7700 1 -- 172.21.15.52:0/1622525681 wait complete. 2023-11-27T15:09:54.079 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.234+0000 7f2251de7700 1 Processor -- start 2023-11-27T15:09:54.079 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.234+0000 7f2251de7700 1 -- start start 2023-11-27T15:09:54.079 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.234+0000 7f2251de7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f224c105f00 0x7f224c197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:54.080 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.234+0000 7f2251de7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224c197b60 con 0x7f224c105f00 2023-11-27T15:09:54.080 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.234+0000 7f224b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f224c105f00 0x7f224c197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:54.080 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.234+0000 7f224b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f224c105f00 0x7f224c197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48916/0 (socket says 172.21.15.52:48916) 2023-11-27T15:09:54.080 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.234+0000 7f224b7fe700 1 -- 172.21.15.52:0/3495332152 learned_addr learned my addr 172.21.15.52:0/3495332152 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:54.080 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.235+0000 7f224b7fe700 1 -- 172.21.15.52:0/3495332152 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223c00b7f0 con 0x7f224c105f00 2023-11-27T15:09:54.080 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.235+0000 7f224b7fe700 1 --2- 172.21.15.52:0/3495332152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f224c105f00 0x7f224c197620 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f223c006fa0 tx=0x7f223c004020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:54.080 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.235+0000 7f2248ff9700 1 -- 172.21.15.52:0/3495332152 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f223c01e070 con 0x7f224c105f00 2023-11-27T15:09:54.080 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.235+0000 7f2248ff9700 1 -- 172.21.15.52:0/3495332152 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f223c010af0 con 0x7f224c105f00 2023-11-27T15:09:54.080 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.235+0000 7f2251de7700 1 -- 172.21.15.52:0/3495332152 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f224c103a00 con 0x7f224c105f00 2023-11-27T15:09:54.080 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.235+0000 7f2248ff9700 1 -- 172.21.15.52:0/3495332152 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f223c022a30 con 0x7f224c105f00 2023-11-27T15:09:54.080 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.235+0000 7f2251de7700 1 -- 172.21.15.52:0/3495332152 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f224c105900 con 0x7f224c105f00 2023-11-27T15:09:54.080 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.236+0000 7f2248ff9700 1 -- 172.21.15.52:0/3495332152 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f223c022b90 con 0x7f224c105f00 2023-11-27T15:09:54.081 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.236+0000 7f2251de7700 1 -- 172.21.15.52:0/3495332152 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f224c103b90 con 0x7f224c105f00 2023-11-27T15:09:54.081 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.236+0000 7f2248ff9700 1 --2- 172.21.15.52:0/3495332152 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f2234038760 0x7f223403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:54.081 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.237+0000 7f2248ff9700 1 -- 172.21.15.52:0/3495332152 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f223c04ecd0 con 0x7f224c105f00 2023-11-27T15:09:54.081 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.237+0000 7f224affd700 1 --2- 172.21.15.52:0/3495332152 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f2234038760 0x7f223403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:54.081 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.237+0000 7f224affd700 1 --2- 172.21.15.52:0/3495332152 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f2234038760 0x7f223403ac10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2240006fd0 tx=0x7f2240006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:54.081 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.241+0000 7f2248ff9700 1 -- 172.21.15.52:0/3495332152 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f223c00f3d0 con 0x7f224c105f00 2023-11-27T15:09:54.081 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.602+0000 7f2251de7700 1 -- 172.21.15.52:0/3495332152 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f224c1911b0 con 0x7f224c105f00 2023-11-27T15:09:54.081 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.605+0000 7f2248ff9700 1 -- 172.21.15.52:0/3495332152 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f223c027030 con 0x7f224c105f00 2023-11-27T15:09:54.081 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.608+0000 7f2251de7700 1 -- 172.21.15.52:0/3495332152 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f2234038760 msgr2=0x7f223403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:54.081 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.608+0000 7f2251de7700 1 --2- 172.21.15.52:0/3495332152 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f2234038760 0x7f223403ac10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2240006fd0 tx=0x7f2240006e40 comp rx=0 tx=0).stop 2023-11-27T15:09:54.081 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.608+0000 7f2251de7700 1 -- 172.21.15.52:0/3495332152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f224c105f00 msgr2=0x7f224c197620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:54.082 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.608+0000 7f2251de7700 1 --2- 172.21.15.52:0/3495332152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f224c105f00 0x7f224c197620 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f223c006fa0 tx=0x7f223c004020 comp rx=0 tx=0).stop 2023-11-27T15:09:54.082 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.608+0000 7f2251de7700 1 -- 172.21.15.52:0/3495332152 shutdown_connections 2023-11-27T15:09:54.082 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.608+0000 7f2251de7700 1 --2- 172.21.15.52:0/3495332152 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f2234038760 0x7f223403ac10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:54.082 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.608+0000 7f2251de7700 1 --2- 172.21.15.52:0/3495332152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f224c105f00 0x7f224c197620 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:54.082 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.608+0000 7f2251de7700 1 -- 172.21.15.52:0/3495332152 >> 172.21.15.52:0/3495332152 conn(0x7f224c0ff650 msgr2=0x7f224c18e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:54.082 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.608+0000 7f2251de7700 1 -- 172.21.15.52:0/3495332152 shutdown_connections 2023-11-27T15:09:54.082 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:53.608+0000 7f2251de7700 1 -- 172.21.15.52:0/3495332152 wait complete. 2023-11-27T15:09:54.082 INFO:teuthology.orchestra.run.smithi052.stdout:Enabling the dashboard module... 2023-11-27T15:09:54.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:54 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3495332152' entity='client.admin' 2023-11-27T15:09:54.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:54 smithi052 ceph-mon[99056]: from='mgr.14120 172.21.15.52:0/3799281879' entity='mgr.x' 2023-11-27T15:09:55.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:55 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2685776879' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-11-27T15:09:55.910 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:55 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: ignoring --setuser ceph since I am not root 2023-11-27T15:09:55.910 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:55 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: ignoring --setgroup ceph since I am not root 2023-11-27T15:09:55.910 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:55 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:55.808+0000 7fea84779200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-27T15:09:55.910 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:55 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:55.905+0000 7fea84779200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-27T15:09:56.129 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.586+0000 7f7fd2ba5700 1 Processor -- start 2023-11-27T15:09:56.129 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.586+0000 7f7fd2ba5700 1 -- start start 2023-11-27T15:09:56.129 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.587+0000 7f7fd2ba5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fcc104920 0x7f7fcc104d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:56.130 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.587+0000 7f7fd2ba5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc105300 con 0x7f7fcc104920 2023-11-27T15:09:56.130 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.587+0000 7f7fd0941700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fcc104920 0x7f7fcc104d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:56.130 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.587+0000 7f7fd0941700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fcc104920 0x7f7fcc104d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48920/0 (socket says 172.21.15.52:48920) 2023-11-27T15:09:56.130 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.587+0000 7f7fd0941700 1 -- 172.21.15.52:0/1773779622 learned_addr learned my addr 172.21.15.52:0/1773779622 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:56.130 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.587+0000 7f7fd0941700 1 -- 172.21.15.52:0/1773779622 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fcc105b20 con 0x7f7fcc104920 2023-11-27T15:09:56.130 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.588+0000 7f7fd0941700 1 --2- 172.21.15.52:0/1773779622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fcc104920 0x7f7fcc104d30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00098a0 tx=0x7f7fc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d912ed07eec70b77 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:56.130 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.588+0000 7f7fcb7fe700 1 -- 172.21.15.52:0/1773779622 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7fc0004160 con 0x7f7fcc104920 2023-11-27T15:09:56.130 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.588+0000 7f7fcb7fe700 1 -- 172.21.15.52:0/1773779622 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fc00042c0 con 0x7f7fcc104920 2023-11-27T15:09:56.130 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.588+0000 7f7fcb7fe700 1 -- 172.21.15.52:0/1773779622 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7fc00119b0 con 0x7f7fcc104920 2023-11-27T15:09:56.130 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.588+0000 7f7fd2ba5700 1 -- 172.21.15.52:0/1773779622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fcc104920 msgr2=0x7f7fcc104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:56.130 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.588+0000 7f7fd2ba5700 1 --2- 172.21.15.52:0/1773779622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fcc104920 0x7f7fcc104d30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00098a0 tx=0x7f7fc0009bb0 comp rx=0 tx=0).stop 2023-11-27T15:09:56.130 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.588+0000 7f7fd2ba5700 1 -- 172.21.15.52:0/1773779622 shutdown_connections 2023-11-27T15:09:56.131 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.588+0000 7f7fd2ba5700 1 --2- 172.21.15.52:0/1773779622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fcc104920 0x7f7fcc104d30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:56.131 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.588+0000 7f7fd2ba5700 1 -- 172.21.15.52:0/1773779622 >> 172.21.15.52:0/1773779622 conn(0x7f7fcc0ffed0 msgr2=0x7f7fcc102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:56.131 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.589+0000 7f7fd2ba5700 1 -- 172.21.15.52:0/1773779622 shutdown_connections 2023-11-27T15:09:56.131 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.589+0000 7f7fd2ba5700 1 -- 172.21.15.52:0/1773779622 wait complete. 2023-11-27T15:09:56.131 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.589+0000 7f7fd2ba5700 1 Processor -- start 2023-11-27T15:09:56.131 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.589+0000 7f7fd2ba5700 1 -- start start 2023-11-27T15:09:56.131 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.589+0000 7f7fd2ba5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fcc104920 0x7f7fcc18fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:56.131 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.589+0000 7f7fd2ba5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc1900d0 con 0x7f7fcc104920 2023-11-27T15:09:56.131 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.589+0000 7f7fd0941700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fcc104920 0x7f7fcc18fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:56.131 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.589+0000 7f7fd0941700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fcc104920 0x7f7fcc18fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48922/0 (socket says 172.21.15.52:48922) 2023-11-27T15:09:56.131 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.589+0000 7f7fd0941700 1 -- 172.21.15.52:0/2685776879 learned_addr learned my addr 172.21.15.52:0/2685776879 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:56.131 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.590+0000 7f7fd0941700 1 -- 172.21.15.52:0/2685776879 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc0009550 con 0x7f7fcc104920 2023-11-27T15:09:56.131 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.590+0000 7f7fd0941700 1 --2- 172.21.15.52:0/2685776879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fcc104920 0x7f7fcc18fb90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0000c00 tx=0x7f7fc0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:56.132 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.590+0000 7f7fc9ffb700 1 -- 172.21.15.52:0/2685776879 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7fc00039f0 con 0x7f7fcc104920 2023-11-27T15:09:56.132 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.590+0000 7f7fd2ba5700 1 -- 172.21.15.52:0/2685776879 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fcc108ee0 con 0x7f7fcc104920 2023-11-27T15:09:56.132 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.590+0000 7f7fc9ffb700 1 -- 172.21.15.52:0/2685776879 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fc0003b50 con 0x7f7fcc104920 2023-11-27T15:09:56.132 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.590+0000 7f7fc9ffb700 1 -- 172.21.15.52:0/2685776879 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7fc001a780 con 0x7f7fcc104920 2023-11-27T15:09:56.132 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.590+0000 7f7fd2ba5700 1 -- 172.21.15.52:0/2685776879 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fcc108d50 con 0x7f7fcc104920 2023-11-27T15:09:56.132 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.591+0000 7f7fc9ffb700 1 -- 172.21.15.52:0/2685776879 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f7fc001aa40 con 0x7f7fcc104920 2023-11-27T15:09:56.132 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.591+0000 7f7fc9ffb700 1 --2- 172.21.15.52:0/2685776879 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f7fbc038710 0x7f7fbc03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:56.132 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.591+0000 7f7fc9ffb700 1 -- 172.21.15.52:0/2685776879 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7fc004c7e0 con 0x7f7fcc104920 2023-11-27T15:09:56.132 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.591+0000 7f7fd2ba5700 1 -- 172.21.15.52:0/2685776879 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fcc109070 con 0x7f7fcc104920 2023-11-27T15:09:56.132 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.591+0000 7f7fcbfff700 1 --2- 172.21.15.52:0/2685776879 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f7fbc038710 0x7f7fbc03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:56.132 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.592+0000 7f7fcbfff700 1 --2- 172.21.15.52:0/2685776879 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f7fbc038710 0x7f7fbc03abc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8006fd0 tx=0x7f7fb8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:56.133 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:54.596+0000 7f7fc9ffb700 1 -- 172.21.15.52:0/2685776879 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7fc001acf0 con 0x7f7fcc104920 2023-11-27T15:09:56.133 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:55.080+0000 7f7fd2ba5700 1 -- 172.21.15.52:0/2685776879 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f7fcc18eff0 con 0x7f7fcc104920 2023-11-27T15:09:56.133 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:55.608+0000 7f7fc9ffb700 1 -- 172.21.15.52:0/2685776879 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f7fc0047080 con 0x7f7fcc104920 2023-11-27T15:09:56.133 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:55.610+0000 7f7fc9ffb700 1 -- 172.21.15.52:0/2685776879 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f7fc000be30 con 0x7f7fcc104920 2023-11-27T15:09:56.133 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:55.612+0000 7f7fd2ba5700 1 -- 172.21.15.52:0/2685776879 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f7fbc038710 msgr2=0x7f7fbc03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:56.133 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:55.612+0000 7f7fd2ba5700 1 --2- 172.21.15.52:0/2685776879 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f7fbc038710 0x7f7fbc03abc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8006fd0 tx=0x7f7fb8006e40 comp rx=0 tx=0).stop 2023-11-27T15:09:56.133 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:55.612+0000 7f7fd2ba5700 1 -- 172.21.15.52:0/2685776879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fcc104920 msgr2=0x7f7fcc18fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:56.133 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:55.612+0000 7f7fd2ba5700 1 --2- 172.21.15.52:0/2685776879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fcc104920 0x7f7fcc18fb90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0000c00 tx=0x7f7fc0011a30 comp rx=0 tx=0).stop 2023-11-27T15:09:56.133 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:55.612+0000 7f7fd2ba5700 1 -- 172.21.15.52:0/2685776879 shutdown_connections 2023-11-27T15:09:56.133 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:55.612+0000 7f7fd2ba5700 1 --2- 172.21.15.52:0/2685776879 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7f7fbc038710 0x7f7fbc03abc0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:56.133 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:55.612+0000 7f7fd2ba5700 1 --2- 172.21.15.52:0/2685776879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fcc104920 0x7f7fcc18fb90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:56.133 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:55.612+0000 7f7fd2ba5700 1 -- 172.21.15.52:0/2685776879 >> 172.21.15.52:0/2685776879 conn(0x7f7fcc0ffed0 msgr2=0x7f7fcc101580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:56.133 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:55.612+0000 7f7fd2ba5700 1 -- 172.21.15.52:0/2685776879 shutdown_connections 2023-11-27T15:09:56.134 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:55.613+0000 7f7fd2ba5700 1 -- 172.21.15.52:0/2685776879 wait complete. 2023-11-27T15:09:56.342 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:56 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:56.012+0000 7fea84779200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-27T15:09:56.342 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:56 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:56.124+0000 7fea84779200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-27T15:09:56.342 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:56 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:56.336+0000 7fea84779200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-27T15:09:56.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:56 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2685776879' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-11-27T15:09:56.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:56 smithi052 ceph-mon[99056]: mgrmap e9: x(active, since 18s) 2023-11-27T15:09:56.867 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:56 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:56.743+0000 7fea84779200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-27T15:09:56.868 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:56 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:56.852+0000 7fea84779200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-27T15:09:57.313 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:57 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:57.004+0000 7fea84779200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-27T15:09:57.313 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:57 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:57.119+0000 7fea84779200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-27T15:09:57.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2023-11-27T15:09:57.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-11-27T15:09:57.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "available": true, 2023-11-27T15:09:57.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-11-27T15:09:57.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-11-27T15:09:57.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2023-11-27T15:09:57.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.687+0000 7feb3d621700 1 Processor -- start 2023-11-27T15:09:57.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.687+0000 7feb3d621700 1 -- start start 2023-11-27T15:09:57.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.687+0000 7feb3d621700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb381042e0 0x7feb381046f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:57.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.687+0000 7feb3d621700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb38079e00 con 0x7feb381042e0 2023-11-27T15:09:57.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.687+0000 7feb36ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb381042e0 0x7feb381046f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:57.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.688+0000 7feb36ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb381042e0 0x7feb381046f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48936/0 (socket says 172.21.15.52:48936) 2023-11-27T15:09:57.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.688+0000 7feb36ffd700 1 -- 172.21.15.52:0/3393877674 learned_addr learned my addr 172.21.15.52:0/3393877674 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:57.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.688+0000 7feb36ffd700 1 -- 172.21.15.52:0/3393877674 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb38104c30 con 0x7feb381042e0 2023-11-27T15:09:57.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.688+0000 7feb36ffd700 1 --2- 172.21.15.52:0/3393877674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb381042e0 0x7feb381046f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7feb280098a0 tx=0x7feb28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30cc765e3bad0686 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:57.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.688+0000 7feb35ffb700 1 -- 172.21.15.52:0/3393877674 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb28004160 con 0x7feb381042e0 2023-11-27T15:09:57.578 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.688+0000 7feb35ffb700 1 -- 172.21.15.52:0/3393877674 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb280042c0 con 0x7feb381042e0 2023-11-27T15:09:57.578 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.688+0000 7feb35ffb700 1 -- 172.21.15.52:0/3393877674 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb280119b0 con 0x7feb381042e0 2023-11-27T15:09:57.578 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.689+0000 7feb3d621700 1 -- 172.21.15.52:0/3393877674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb381042e0 msgr2=0x7feb381046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:57.578 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.689+0000 7feb3d621700 1 --2- 172.21.15.52:0/3393877674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb381042e0 0x7feb381046f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7feb280098a0 tx=0x7feb28009bb0 comp rx=0 tx=0).stop 2023-11-27T15:09:57.578 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.689+0000 7feb3d621700 1 -- 172.21.15.52:0/3393877674 shutdown_connections 2023-11-27T15:09:57.578 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.689+0000 7feb3d621700 1 --2- 172.21.15.52:0/3393877674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb381042e0 0x7feb381046f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:57.578 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.689+0000 7feb3d621700 1 -- 172.21.15.52:0/3393877674 >> 172.21.15.52:0/3393877674 conn(0x7feb380ffed0 msgr2=0x7feb38102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:57.578 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.689+0000 7feb3d621700 1 -- 172.21.15.52:0/3393877674 shutdown_connections 2023-11-27T15:09:57.578 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.689+0000 7feb3d621700 1 -- 172.21.15.52:0/3393877674 wait complete. 2023-11-27T15:09:57.578 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.690+0000 7feb3d621700 1 Processor -- start 2023-11-27T15:09:57.578 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.690+0000 7feb3d621700 1 -- start start 2023-11-27T15:09:57.578 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.690+0000 7feb3d621700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb381042e0 0x7feb3819b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:57.579 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.690+0000 7feb3d621700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3819bed0 con 0x7feb381042e0 2023-11-27T15:09:57.579 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.690+0000 7feb36ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb381042e0 0x7feb3819b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:09:57.579 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.690+0000 7feb36ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb381042e0 0x7feb3819b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48938/0 (socket says 172.21.15.52:48938) 2023-11-27T15:09:57.579 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.690+0000 7feb36ffd700 1 -- 172.21.15.52:0/2236907873 learned_addr learned my addr 172.21.15.52:0/2236907873 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:09:57.579 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.690+0000 7feb36ffd700 1 -- 172.21.15.52:0/2236907873 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb28009550 con 0x7feb381042e0 2023-11-27T15:09:57.579 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.691+0000 7feb36ffd700 1 --2- 172.21.15.52:0/2236907873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb381042e0 0x7feb3819b990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7feb28003e60 tx=0x7feb28011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:09:57.579 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.691+0000 7feb1ffff700 1 -- 172.21.15.52:0/2236907873 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb28003970 con 0x7feb381042e0 2023-11-27T15:09:57.579 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.691+0000 7feb3d621700 1 -- 172.21.15.52:0/2236907873 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb38105c70 con 0x7feb381042e0 2023-11-27T15:09:57.579 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.691+0000 7feb1ffff700 1 -- 172.21.15.52:0/2236907873 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb28003ad0 con 0x7feb381042e0 2023-11-27T15:09:57.579 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.691+0000 7feb1ffff700 1 -- 172.21.15.52:0/2236907873 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb28019590 con 0x7feb381042e0 2023-11-27T15:09:57.579 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.691+0000 7feb3d621700 1 -- 172.21.15.52:0/2236907873 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb38105ae0 con 0x7feb381042e0 2023-11-27T15:09:57.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.692+0000 7feb1ffff700 1 -- 172.21.15.52:0/2236907873 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7feb28019800 con 0x7feb381042e0 2023-11-27T15:09:57.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.692+0000 7feb1ffff700 1 --2- 172.21.15.52:0/2236907873 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7feb200387b0 0x7feb2003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:09:57.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.692+0000 7feb1ffff700 1 -- 172.21.15.52:0/2236907873 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7feb2804c870 con 0x7feb381042e0 2023-11-27T15:09:57.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.692+0000 7feb367fc700 1 -- 172.21.15.52:0/2236907873 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7feb200387b0 msgr2=0x7feb2003ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3470359117 2023-11-27T15:09:57.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.692+0000 7feb367fc700 1 --2- 172.21.15.52:0/2236907873 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7feb200387b0 0x7feb2003ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-27T15:09:57.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.692+0000 7feb3d621700 1 -- 172.21.15.52:0/2236907873 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb38105e00 con 0x7feb381042e0 2023-11-27T15:09:57.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.697+0000 7feb1ffff700 1 -- 172.21.15.52:0/2236907873 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7feb28019b50 con 0x7feb381042e0 2023-11-27T15:09:57.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.893+0000 7feb367fc700 1 -- 172.21.15.52:0/2236907873 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7feb200387b0 msgr2=0x7feb2003ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3470359117 2023-11-27T15:09:57.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:56.893+0000 7feb367fc700 1 --2- 172.21.15.52:0/2236907873 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7feb200387b0 0x7feb2003ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-27T15:09:57.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:57.163+0000 7feb3d621700 1 -- 172.21.15.52:0/2236907873 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7feb38195280 con 0x7feb381042e0 2023-11-27T15:09:57.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:57.163+0000 7feb1ffff700 1 -- 172.21.15.52:0/2236907873 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7feb2804b070 con 0x7feb381042e0 2023-11-27T15:09:57.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:57.166+0000 7feb3d621700 1 -- 172.21.15.52:0/2236907873 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7feb200387b0 msgr2=0x7feb2003ac60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-27T15:09:57.581 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:57.166+0000 7feb3d621700 1 --2- 172.21.15.52:0/2236907873 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7feb200387b0 0x7feb2003ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:57.581 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:57.166+0000 7feb3d621700 1 -- 172.21.15.52:0/2236907873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb381042e0 msgr2=0x7feb3819b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:09:57.581 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:57.166+0000 7feb3d621700 1 --2- 172.21.15.52:0/2236907873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb381042e0 0x7feb3819b990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7feb28003e60 tx=0x7feb28011aa0 comp rx=0 tx=0).stop 2023-11-27T15:09:57.581 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:57.166+0000 7feb3d621700 1 -- 172.21.15.52:0/2236907873 shutdown_connections 2023-11-27T15:09:57.581 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:57.166+0000 7feb3d621700 1 --2- 172.21.15.52:0/2236907873 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7feb200387b0 0x7feb2003ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:57.581 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:57.166+0000 7feb3d621700 1 --2- 172.21.15.52:0/2236907873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb381042e0 0x7feb3819b990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:09:57.581 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:57.166+0000 7feb3d621700 1 -- 172.21.15.52:0/2236907873 >> 172.21.15.52:0/2236907873 conn(0x7feb380ffed0 msgr2=0x7feb3818fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:09:57.581 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:57.166+0000 7feb3d621700 1 -- 172.21.15.52:0/2236907873 shutdown_connections 2023-11-27T15:09:57.581 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:57.166+0000 7feb3d621700 1 -- 172.21.15.52:0/2236907873 wait complete. 2023-11-27T15:09:57.581 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for the mgr to restart... 2023-11-27T15:09:57.581 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for mgr epoch 9... 2023-11-27T15:09:57.832 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:09:57 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2236907873' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-11-27T15:09:57.833 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:57 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:57.648+0000 7fea84779200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-27T15:09:58.617 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:58 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:58.347+0000 7fea84779200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-27T15:09:58.617 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:58 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:58.539+0000 7fea84779200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-27T15:09:59.838 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:59 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:59.503+0000 7fea84779200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:09:59.839 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:59 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:59.614+0000 7fea84779200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-27T15:09:59.839 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:09:59 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:09:59.834+0000 7fea84779200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:10:00.438 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:10:00 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:10:00.183+0000 7fea84779200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-27T15:10:00.867 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:10:00 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:10:00.433+0000 7fea84779200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-27T15:10:00.867 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:10:00 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:10:00.616+0000 7fea84779200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-27T15:10:01.367 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:10:00 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:10:00.946+0000 7fea84779200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-27T15:10:02.117 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:10:01 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:10:01.772+0000 7fea84779200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-27T15:10:02.117 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:10:01 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:10:01.929+0000 7fea84779200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-27T15:10:02.587 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:10:02 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:10:02.172+0000 7fea84779200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-27T15:10:02.587 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:10:02 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:10:02.309+0000 7fea84779200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-27T15:10:02.587 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:10:02 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:10:02.581+0000 7fea84779200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-27T15:10:04.117 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:10:03 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:10:03.631+0000 7fea84779200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-27T15:10:04.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:04 smithi052 ceph-mon[99056]: Active manager daemon x restarted 2023-11-27T15:10:04.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:04 smithi052 ceph-mon[99056]: Activating manager daemon x 2023-11-27T15:10:04.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:04 smithi052 ceph-mon[99056]: osdmap e3: 0 total, 0 up, 0 in 2023-11-27T15:10:04.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:04 smithi052 ceph-mon[99056]: mgrmap e10: x(active, starting, since 0.00310817s) 2023-11-27T15:10:04.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:04 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:10:04.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:04 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-11-27T15:10:04.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:04 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-11-27T15:10:04.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:04 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-11-27T15:10:04.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:04 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-11-27T15:10:04.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:04 smithi052 ceph-mon[99056]: Manager daemon x is now available 2023-11-27T15:10:04.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:04 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:10:04.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:04 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:10:04.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:04 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-27T15:10:04.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:04 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-27T15:10:05.241 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2023-11-27T15:10:05.241 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-11-27T15:10:05.241 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "initialized": true 2023-11-27T15:10:05.241 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2023-11-27T15:10:05.241 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.055+0000 7fbc77cb1700 1 Processor -- start 2023-11-27T15:10:05.241 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.055+0000 7fbc77cb1700 1 -- start start 2023-11-27T15:10:05.241 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.056+0000 7fbc77cb1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc70104920 0x7fbc70104d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:05.241 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.056+0000 7fbc77cb1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc70105300 con 0x7fbc70104920 2023-11-27T15:10:05.241 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.056+0000 7fbc75a4d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc70104920 0x7fbc70104d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:05.242 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.056+0000 7fbc75a4d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc70104920 0x7fbc70104d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48946/0 (socket says 172.21.15.52:48946) 2023-11-27T15:10:05.242 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.056+0000 7fbc75a4d700 1 -- 172.21.15.52:0/3765754876 learned_addr learned my addr 172.21.15.52:0/3765754876 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:10:05.242 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.056+0000 7fbc75a4d700 1 -- 172.21.15.52:0/3765754876 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc70105b20 con 0x7fbc70104920 2023-11-27T15:10:05.242 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.056+0000 7fbc75a4d700 1 --2- 172.21.15.52:0/3765754876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc70104920 0x7fbc70104d30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0098a0 tx=0x7fbc6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6493e106bf1ce90 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:05.242 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.056+0000 7fbc74a4b700 1 -- 172.21.15.52:0/3765754876 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc6c004160 con 0x7fbc70104920 2023-11-27T15:10:05.242 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.057+0000 7fbc74a4b700 1 -- 172.21.15.52:0/3765754876 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc6c0042c0 con 0x7fbc70104920 2023-11-27T15:10:05.243 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.057+0000 7fbc74a4b700 1 -- 172.21.15.52:0/3765754876 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc6c0119b0 con 0x7fbc70104920 2023-11-27T15:10:05.243 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.057+0000 7fbc77cb1700 1 -- 172.21.15.52:0/3765754876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc70104920 msgr2=0x7fbc70104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:05.243 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.057+0000 7fbc77cb1700 1 --2- 172.21.15.52:0/3765754876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc70104920 0x7fbc70104d30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0098a0 tx=0x7fbc6c009bb0 comp rx=0 tx=0).stop 2023-11-27T15:10:05.243 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.057+0000 7fbc77cb1700 1 -- 172.21.15.52:0/3765754876 shutdown_connections 2023-11-27T15:10:05.243 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.057+0000 7fbc77cb1700 1 --2- 172.21.15.52:0/3765754876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc70104920 0x7fbc70104d30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:05.243 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.057+0000 7fbc77cb1700 1 -- 172.21.15.52:0/3765754876 >> 172.21.15.52:0/3765754876 conn(0x7fbc700ffed0 msgr2=0x7fbc70102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:05.244 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.057+0000 7fbc77cb1700 1 -- 172.21.15.52:0/3765754876 shutdown_connections 2023-11-27T15:10:05.244 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.057+0000 7fbc77cb1700 1 -- 172.21.15.52:0/3765754876 wait complete. 2023-11-27T15:10:05.244 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.058+0000 7fbc77cb1700 1 Processor -- start 2023-11-27T15:10:05.244 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.058+0000 7fbc77cb1700 1 -- start start 2023-11-27T15:10:05.244 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.058+0000 7fbc77cb1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc70104920 0x7fbc701997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:05.244 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.058+0000 7fbc77cb1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc70199cf0 con 0x7fbc70104920 2023-11-27T15:10:05.245 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.058+0000 7fbc75a4d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc70104920 0x7fbc701997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:05.245 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.058+0000 7fbc75a4d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc70104920 0x7fbc701997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48948/0 (socket says 172.21.15.52:48948) 2023-11-27T15:10:05.245 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.058+0000 7fbc75a4d700 1 -- 172.21.15.52:0/2262196997 learned_addr learned my addr 172.21.15.52:0/2262196997 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:10:05.245 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.059+0000 7fbc75a4d700 1 -- 172.21.15.52:0/2262196997 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc6c009550 con 0x7fbc70104920 2023-11-27T15:10:05.245 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.059+0000 7fbc75a4d700 1 --2- 172.21.15.52:0/2262196997 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc70104920 0x7fbc701997b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c000c00 tx=0x7fbc6c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:05.245 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.059+0000 7fbc66ffd700 1 -- 172.21.15.52:0/2262196997 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc6c0039f0 con 0x7fbc70104920 2023-11-27T15:10:05.246 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.059+0000 7fbc66ffd700 1 -- 172.21.15.52:0/2262196997 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc6c003b50 con 0x7fbc70104920 2023-11-27T15:10:05.246 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.059+0000 7fbc77cb1700 1 -- 172.21.15.52:0/2262196997 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc70108ee0 con 0x7fbc70104920 2023-11-27T15:10:05.246 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.059+0000 7fbc66ffd700 1 -- 172.21.15.52:0/2262196997 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc6c01a780 con 0x7fbc70104920 2023-11-27T15:10:05.246 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.059+0000 7fbc77cb1700 1 -- 172.21.15.52:0/2262196997 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc70108d50 con 0x7fbc70104920 2023-11-27T15:10:05.246 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.061+0000 7fbc66ffd700 1 -- 172.21.15.52:0/2262196997 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fbc6c01a9f0 con 0x7fbc70104920 2023-11-27T15:10:05.246 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.061+0000 7fbc66ffd700 1 --2- 172.21.15.52:0/2262196997 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fbc5c0387b0 0x7fbc5c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:05.247 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.061+0000 7fbc66ffd700 1 -- 172.21.15.52:0/2262196997 --> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbc5c03b370 con 0x7fbc5c0387b0 2023-11-27T15:10:05.247 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.061+0000 7fbc66ffd700 1 -- 172.21.15.52:0/2262196997 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbc6c04c8b0 con 0x7fbc70104920 2023-11-27T15:10:05.247 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.061+0000 7fbc7524c700 1 -- 172.21.15.52:0/2262196997 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fbc5c0387b0 msgr2=0x7fbc5c03ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3470359117 2023-11-27T15:10:05.247 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.061+0000 7fbc7524c700 1 --2- 172.21.15.52:0/2262196997 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fbc5c0387b0 0x7fbc5c03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-27T15:10:05.247 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.262+0000 7fbc7524c700 1 -- 172.21.15.52:0/2262196997 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fbc5c0387b0 msgr2=0x7fbc5c03ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3470359117 2023-11-27T15:10:05.247 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.262+0000 7fbc7524c700 1 --2- 172.21.15.52:0/2262196997 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fbc5c0387b0 0x7fbc5c03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-27T15:10:05.248 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.663+0000 7fbc7524c700 1 -- 172.21.15.52:0/2262196997 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fbc5c0387b0 msgr2=0x7fbc5c03ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3470359117 2023-11-27T15:10:05.248 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:58.663+0000 7fbc7524c700 1 --2- 172.21.15.52:0/2262196997 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fbc5c0387b0 0x7fbc5c03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-27T15:10:05.248 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:59.464+0000 7fbc7524c700 1 -- 172.21.15.52:0/2262196997 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fbc5c0387b0 msgr2=0x7fbc5c03ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3470359117 2023-11-27T15:10:05.248 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:09:59.464+0000 7fbc7524c700 1 --2- 172.21.15.52:0/2262196997 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fbc5c0387b0 0x7fbc5c03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-27T15:10:05.248 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:01.065+0000 7fbc7524c700 1 -- 172.21.15.52:0/2262196997 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fbc5c0387b0 msgr2=0x7fbc5c03ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3470359117 2023-11-27T15:10:05.249 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:01.065+0000 7fbc7524c700 1 --2- 172.21.15.52:0/2262196997 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fbc5c0387b0 0x7fbc5c03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-11-27T15:10:05.249 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:03.646+0000 7fbc66ffd700 1 -- 172.21.15.52:0/2262196997 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mgrmap(e 10) v1 ==== 45263+0+0 (secure 0 0 0) 0x7fbc6c024960 con 0x7fbc70104920 2023-11-27T15:10:05.249 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:03.646+0000 7fbc66ffd700 1 -- 172.21.15.52:0/2262196997 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fbc5c0387b0 msgr2=0x7fbc5c03ac60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-27T15:10:05.249 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:03.646+0000 7fbc66ffd700 1 --2- 172.21.15.52:0/2262196997 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fbc5c0387b0 0x7fbc5c03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:05.249 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:04.648+0000 7fbc66ffd700 1 -- 172.21.15.52:0/2262196997 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7fbc6c01aca0 con 0x7fbc70104920 2023-11-27T15:10:05.249 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:04.648+0000 7fbc66ffd700 1 --2- 172.21.15.52:0/2262196997 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fbc5c03bff0 0x7fbc5c03e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:05.250 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:04.648+0000 7fbc66ffd700 1 -- 172.21.15.52:0/2262196997 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbc5c03b370 con 0x7fbc5c03bff0 2023-11-27T15:10:05.250 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:04.648+0000 7fbc7524c700 1 --2- 172.21.15.52:0/2262196997 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fbc5c03bff0 0x7fbc5c03e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:05.250 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:04.649+0000 7fbc7524c700 1 --2- 172.21.15.52:0/2262196997 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fbc5c03bff0 0x7fbc5c03e3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbc60003a10 tx=0x7fbc60009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:05.250 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:04.649+0000 7fbc66ffd700 1 -- 172.21.15.52:0/2262196997 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fbc5c03b370 con 0x7fbc5c03bff0 2023-11-27T15:10:05.250 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:04.660+0000 7fbc77cb1700 1 -- 172.21.15.52:0/2262196997 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fbc7019aaf0 con 0x7fbc5c03bff0 2023-11-27T15:10:05.251 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:04.661+0000 7fbc66ffd700 1 -- 172.21.15.52:0/2262196997 <== mgr.14152 v2:172.21.15.52:6800/2640036585 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fbc7019aaf0 con 0x7fbc5c03bff0 2023-11-27T15:10:05.251 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:04.661+0000 7fbc77cb1700 1 -- 172.21.15.52:0/2262196997 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fbc5c03bff0 msgr2=0x7fbc5c03e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:05.251 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:04.661+0000 7fbc77cb1700 1 --2- 172.21.15.52:0/2262196997 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fbc5c03bff0 0x7fbc5c03e3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbc60003a10 tx=0x7fbc60009270 comp rx=0 tx=0).stop 2023-11-27T15:10:05.251 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:04.661+0000 7fbc77cb1700 1 -- 172.21.15.52:0/2262196997 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc70104920 msgr2=0x7fbc701997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:05.251 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:04.661+0000 7fbc77cb1700 1 --2- 172.21.15.52:0/2262196997 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc70104920 0x7fbc701997b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c000c00 tx=0x7fbc6c011a30 comp rx=0 tx=0).stop 2023-11-27T15:10:05.251 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:04.661+0000 7fbc77cb1700 1 -- 172.21.15.52:0/2262196997 shutdown_connections 2023-11-27T15:10:05.252 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:04.661+0000 7fbc77cb1700 1 --2- 172.21.15.52:0/2262196997 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fbc5c03bff0 0x7fbc5c03e3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:05.252 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:04.661+0000 7fbc77cb1700 1 --2- 172.21.15.52:0/2262196997 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc70104920 0x7fbc701997b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:05.252 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:04.661+0000 7fbc77cb1700 1 --2- 172.21.15.52:0/2262196997 >> [v2:172.21.15.52:6800/3470359117,v1:172.21.15.52:6801/3470359117] conn(0x7fbc5c0387b0 0x7fbc5c03ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:05.252 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:04.661+0000 7fbc77cb1700 1 -- 172.21.15.52:0/2262196997 >> 172.21.15.52:0/2262196997 conn(0x7fbc700ffed0 msgr2=0x7fbc70079870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:05.252 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:04.661+0000 7fbc77cb1700 1 -- 172.21.15.52:0/2262196997 shutdown_connections 2023-11-27T15:10:05.252 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:04.661+0000 7fbc77cb1700 1 -- 172.21.15.52:0/2262196997 wait complete. 2023-11-27T15:10:05.253 INFO:teuthology.orchestra.run.smithi052.stdout:mgr epoch 9 is available 2023-11-27T15:10:05.253 INFO:teuthology.orchestra.run.smithi052.stdout:Generating a dashboard self-signed certificate... 2023-11-27T15:10:05.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:05 smithi052 ceph-mon[99056]: [27/Nov/2023:15:10:04] ENGINE Bus STARTING 2023-11-27T15:10:05.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:05 smithi052 ceph-mon[99056]: [27/Nov/2023:15:10:04] ENGINE Serving on https://172.21.15.52:7150 2023-11-27T15:10:05.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:05 smithi052 ceph-mon[99056]: [27/Nov/2023:15:10:04] ENGINE Serving on http://172.21.15.52:8765 2023-11-27T15:10:05.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:05 smithi052 ceph-mon[99056]: [27/Nov/2023:15:10:04] ENGINE Bus STARTED 2023-11-27T15:10:05.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:05 smithi052 ceph-mon[99056]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-11-27T15:10:05.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:05 smithi052 ceph-mon[99056]: mgrmap e11: x(active, since 1.00595s) 2023-11-27T15:10:05.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:05 smithi052 ceph-mon[99056]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-11-27T15:10:06.684 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-11-27T15:10:06.684 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.838+0000 7f3a297bf700 1 Processor -- start 2023-11-27T15:10:06.685 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.838+0000 7f3a297bf700 1 -- start start 2023-11-27T15:10:06.685 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.838+0000 7f3a297bf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a241064d0 0x7f3a241088b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:06.685 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.838+0000 7f3a297bf700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a24079e20 con 0x7f3a241064d0 2023-11-27T15:10:06.685 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.839+0000 7f3a22ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a241064d0 0x7f3a241088b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:06.685 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.839+0000 7f3a22ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a241064d0 0x7f3a241088b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49002/0 (socket says 172.21.15.52:49002) 2023-11-27T15:10:06.685 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.839+0000 7f3a22ffd700 1 -- 172.21.15.52:0/3161194839 learned_addr learned my addr 172.21.15.52:0/3161194839 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:10:06.685 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.839+0000 7f3a22ffd700 1 -- 172.21.15.52:0/3161194839 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a24108df0 con 0x7f3a241064d0 2023-11-27T15:10:06.685 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.839+0000 7f3a22ffd700 1 --2- 172.21.15.52:0/3161194839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a241064d0 0x7f3a241088b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3a140098a0 tx=0x7f3a14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267ea2a9edd8aded server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:06.685 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.839+0000 7f3a21ffb700 1 -- 172.21.15.52:0/3161194839 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a14004160 con 0x7f3a241064d0 2023-11-27T15:10:06.685 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.839+0000 7f3a21ffb700 1 -- 172.21.15.52:0/3161194839 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a140042c0 con 0x7f3a241064d0 2023-11-27T15:10:06.685 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.839+0000 7f3a21ffb700 1 -- 172.21.15.52:0/3161194839 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a140119b0 con 0x7f3a241064d0 2023-11-27T15:10:06.685 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.840+0000 7f3a297bf700 1 -- 172.21.15.52:0/3161194839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a241064d0 msgr2=0x7f3a241088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:06.686 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.840+0000 7f3a297bf700 1 --2- 172.21.15.52:0/3161194839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a241064d0 0x7f3a241088b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3a140098a0 tx=0x7f3a14009bb0 comp rx=0 tx=0).stop 2023-11-27T15:10:06.686 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.840+0000 7f3a297bf700 1 -- 172.21.15.52:0/3161194839 shutdown_connections 2023-11-27T15:10:06.686 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.840+0000 7f3a297bf700 1 --2- 172.21.15.52:0/3161194839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a241064d0 0x7f3a241088b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:06.686 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.840+0000 7f3a297bf700 1 -- 172.21.15.52:0/3161194839 >> 172.21.15.52:0/3161194839 conn(0x7f3a240ffed0 msgr2=0x7f3a24102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:06.686 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.840+0000 7f3a297bf700 1 -- 172.21.15.52:0/3161194839 shutdown_connections 2023-11-27T15:10:06.686 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.840+0000 7f3a297bf700 1 -- 172.21.15.52:0/3161194839 wait complete. 2023-11-27T15:10:06.686 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.841+0000 7f3a297bf700 1 Processor -- start 2023-11-27T15:10:06.686 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.841+0000 7f3a297bf700 1 -- start start 2023-11-27T15:10:06.686 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.841+0000 7f3a297bf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a241064d0 0x7f3a24105e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:06.686 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.841+0000 7f3a297bf700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a24106370 con 0x7f3a241064d0 2023-11-27T15:10:06.686 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.841+0000 7f3a22ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a241064d0 0x7f3a24105e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:06.686 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.842+0000 7f3a22ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a241064d0 0x7f3a24105e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49004/0 (socket says 172.21.15.52:49004) 2023-11-27T15:10:06.687 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.842+0000 7f3a22ffd700 1 -- 172.21.15.52:0/1619602181 learned_addr learned my addr 172.21.15.52:0/1619602181 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:10:06.687 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.842+0000 7f3a22ffd700 1 -- 172.21.15.52:0/1619602181 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a14009550 con 0x7f3a241064d0 2023-11-27T15:10:06.687 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.842+0000 7f3a22ffd700 1 --2- 172.21.15.52:0/1619602181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a241064d0 0x7f3a24105e30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3a14003e60 tx=0x7f3a14011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:06.687 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.842+0000 7f3a0bfff700 1 -- 172.21.15.52:0/1619602181 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a14003970 con 0x7f3a241064d0 2023-11-27T15:10:06.687 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.842+0000 7f3a0bfff700 1 -- 172.21.15.52:0/1619602181 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a14003ad0 con 0x7f3a241064d0 2023-11-27T15:10:06.687 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.842+0000 7f3a297bf700 1 -- 172.21.15.52:0/1619602181 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a24109d60 con 0x7f3a241064d0 2023-11-27T15:10:06.687 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.842+0000 7f3a0bfff700 1 -- 172.21.15.52:0/1619602181 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a140195e0 con 0x7f3a241064d0 2023-11-27T15:10:06.687 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.842+0000 7f3a297bf700 1 -- 172.21.15.52:0/1619602181 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a24109bd0 con 0x7f3a241064d0 2023-11-27T15:10:06.687 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.843+0000 7f3a0bfff700 1 -- 172.21.15.52:0/1619602181 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7f3a140198e0 con 0x7f3a241064d0 2023-11-27T15:10:06.687 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.843+0000 7f3a297bf700 1 -- 172.21.15.52:0/1619602181 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a24062120 con 0x7f3a241064d0 2023-11-27T15:10:06.687 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.844+0000 7f3a0bfff700 1 --2- 172.21.15.52:0/1619602181 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f3a0c0385f0 0x7f3a0c03aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:06.687 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.844+0000 7f3a0bfff700 1 -- 172.21.15.52:0/1619602181 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3a1404c7d0 con 0x7f3a241064d0 2023-11-27T15:10:06.687 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.844+0000 7f3a227fc700 1 --2- 172.21.15.52:0/1619602181 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f3a0c0385f0 0x7f3a0c03aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:06.688 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.844+0000 7f3a227fc700 1 --2- 172.21.15.52:0/1619602181 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f3a0c0385f0 0x7f3a0c03aaa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3a18006fd0 tx=0x7f3a18006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:06.688 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:05.848+0000 7f3a0bfff700 1 -- 172.21.15.52:0/1619602181 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3a1402b340 con 0x7f3a241064d0 2023-11-27T15:10:06.688 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:06.227+0000 7f3a297bf700 1 -- 172.21.15.52:0/1619602181 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a24105320 con 0x7f3a0c0385f0 2023-11-27T15:10:06.688 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:06.325+0000 7f3a0bfff700 1 -- 172.21.15.52:0/1619602181 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f3a24105320 con 0x7f3a0c0385f0 2023-11-27T15:10:06.688 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:06.327+0000 7f3a297bf700 1 -- 172.21.15.52:0/1619602181 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f3a0c0385f0 msgr2=0x7f3a0c03aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:06.688 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:06.327+0000 7f3a297bf700 1 --2- 172.21.15.52:0/1619602181 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f3a0c0385f0 0x7f3a0c03aaa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3a18006fd0 tx=0x7f3a18006e40 comp rx=0 tx=0).stop 2023-11-27T15:10:06.688 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:06.327+0000 7f3a297bf700 1 -- 172.21.15.52:0/1619602181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a241064d0 msgr2=0x7f3a24105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:06.688 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:06.327+0000 7f3a297bf700 1 --2- 172.21.15.52:0/1619602181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a241064d0 0x7f3a24105e30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3a14003e60 tx=0x7f3a14011aa0 comp rx=0 tx=0).stop 2023-11-27T15:10:06.688 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:06.327+0000 7f3a297bf700 1 -- 172.21.15.52:0/1619602181 shutdown_connections 2023-11-27T15:10:06.688 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:06.327+0000 7f3a297bf700 1 --2- 172.21.15.52:0/1619602181 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f3a0c0385f0 0x7f3a0c03aaa0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:06.688 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:06.327+0000 7f3a297bf700 1 --2- 172.21.15.52:0/1619602181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a241064d0 0x7f3a24105e30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:06.688 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:06.327+0000 7f3a297bf700 1 -- 172.21.15.52:0/1619602181 >> 172.21.15.52:0/1619602181 conn(0x7f3a240ffed0 msgr2=0x7f3a24101570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:06.688 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:06.328+0000 7f3a297bf700 1 -- 172.21.15.52:0/1619602181 shutdown_connections 2023-11-27T15:10:06.689 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:06.328+0000 7f3a297bf700 1 -- 172.21.15.52:0/1619602181 wait complete. 2023-11-27T15:10:06.689 INFO:teuthology.orchestra.run.smithi052.stdout:Creating initial admin user... 2023-11-27T15:10:06.923 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:06 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:06.924 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:06 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:06.924 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:06 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:06.924 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:06 smithi052 ceph-mon[99056]: mgrmap e12: x(active, since 3s) 2023-11-27T15:10:07.953 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:07 smithi052 ceph-mon[99056]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:10:07.953 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:07 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:08.230 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$98eClHNqj70Fv6bMnHJDdusX3zyJgXgiM0r95iJNlJdmSorKr8pEa", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1701097807, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-11-27T15:10:08.231 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.131+0000 7f6232eeb700 1 Processor -- start 2023-11-27T15:10:08.231 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.131+0000 7f6232eeb700 1 -- start start 2023-11-27T15:10:08.231 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.131+0000 7f6232eeb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f622c105a20 0x7f622c105e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:08.231 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.131+0000 7f6232eeb700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c106400 con 0x7f622c105a20 2023-11-27T15:10:08.231 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.131+0000 7f6230c87700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f622c105a20 0x7f622c105e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:08.231 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.131+0000 7f6230c87700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f622c105a20 0x7f622c105e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49008/0 (socket says 172.21.15.52:49008) 2023-11-27T15:10:08.231 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.131+0000 7f6230c87700 1 -- 172.21.15.52:0/65211190 learned_addr learned my addr 172.21.15.52:0/65211190 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:10:08.231 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.132+0000 7f6230c87700 1 -- 172.21.15.52:0/65211190 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622c106c20 con 0x7f622c105a20 2023-11-27T15:10:08.231 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.132+0000 7f6230c87700 1 --2- 172.21.15.52:0/65211190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f622c105a20 0x7f622c105e30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f62200098a0 tx=0x7f6220009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee720dc94a3b9f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:08.231 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.132+0000 7f622b7fe700 1 -- 172.21.15.52:0/65211190 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6220004160 con 0x7f622c105a20 2023-11-27T15:10:08.231 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.132+0000 7f622b7fe700 1 -- 172.21.15.52:0/65211190 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62200042c0 con 0x7f622c105a20 2023-11-27T15:10:08.232 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.132+0000 7f622b7fe700 1 -- 172.21.15.52:0/65211190 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62200119b0 con 0x7f622c105a20 2023-11-27T15:10:08.232 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.133+0000 7f6232eeb700 1 -- 172.21.15.52:0/65211190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f622c105a20 msgr2=0x7f622c105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:08.232 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.133+0000 7f6232eeb700 1 --2- 172.21.15.52:0/65211190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f622c105a20 0x7f622c105e30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f62200098a0 tx=0x7f6220009bb0 comp rx=0 tx=0).stop 2023-11-27T15:10:08.232 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.133+0000 7f6232eeb700 1 -- 172.21.15.52:0/65211190 shutdown_connections 2023-11-27T15:10:08.232 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.133+0000 7f6232eeb700 1 --2- 172.21.15.52:0/65211190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f622c105a20 0x7f622c105e30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:08.232 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.133+0000 7f6232eeb700 1 -- 172.21.15.52:0/65211190 >> 172.21.15.52:0/65211190 conn(0x7f622c100fd0 msgr2=0x7f622c103400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:08.232 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.133+0000 7f6232eeb700 1 -- 172.21.15.52:0/65211190 shutdown_connections 2023-11-27T15:10:08.232 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.133+0000 7f6232eeb700 1 -- 172.21.15.52:0/65211190 wait complete. 2023-11-27T15:10:08.232 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.134+0000 7f6232eeb700 1 Processor -- start 2023-11-27T15:10:08.232 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.134+0000 7f6232eeb700 1 -- start start 2023-11-27T15:10:08.232 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.134+0000 7f6232eeb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f622c105a20 0x7f622c1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:08.233 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.134+0000 7f6232eeb700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c199ce0 con 0x7f622c105a20 2023-11-27T15:10:08.233 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.134+0000 7f6230c87700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f622c105a20 0x7f622c1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:08.233 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.134+0000 7f6230c87700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f622c105a20 0x7f622c1997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49010/0 (socket says 172.21.15.52:49010) 2023-11-27T15:10:08.233 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.134+0000 7f6230c87700 1 -- 172.21.15.52:0/2515169057 learned_addr learned my addr 172.21.15.52:0/2515169057 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:10:08.233 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.135+0000 7f6230c87700 1 -- 172.21.15.52:0/2515169057 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6220009550 con 0x7f622c105a20 2023-11-27T15:10:08.233 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.135+0000 7f6230c87700 1 --2- 172.21.15.52:0/2515169057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f622c105a20 0x7f622c1997a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6220003e60 tx=0x7f6220011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:08.233 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.135+0000 7f6229ffb700 1 -- 172.21.15.52:0/2515169057 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6220003970 con 0x7f622c105a20 2023-11-27T15:10:08.233 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.135+0000 7f6229ffb700 1 -- 172.21.15.52:0/2515169057 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6220003ad0 con 0x7f622c105a20 2023-11-27T15:10:08.234 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.135+0000 7f6232eeb700 1 -- 172.21.15.52:0/2515169057 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f622c109fe0 con 0x7f622c105a20 2023-11-27T15:10:08.234 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.135+0000 7f6229ffb700 1 -- 172.21.15.52:0/2515169057 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62200195e0 con 0x7f622c105a20 2023-11-27T15:10:08.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.135+0000 7f6232eeb700 1 -- 172.21.15.52:0/2515169057 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f622c109e50 con 0x7f622c105a20 2023-11-27T15:10:08.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.136+0000 7f6229ffb700 1 -- 172.21.15.52:0/2515169057 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f62200198a0 con 0x7f622c105a20 2023-11-27T15:10:08.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.136+0000 7f6229ffb700 1 --2- 172.21.15.52:0/2515169057 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f621c038710 0x7f621c03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:08.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.136+0000 7f6229ffb700 1 -- 172.21.15.52:0/2515169057 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f622004d980 con 0x7f622c105a20 2023-11-27T15:10:08.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.136+0000 7f622bfff700 1 --2- 172.21.15.52:0/2515169057 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f621c038710 0x7f621c03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:08.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.137+0000 7f6232eeb700 1 -- 172.21.15.52:0/2515169057 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f622c062120 con 0x7f622c105a20 2023-11-27T15:10:08.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.137+0000 7f622bfff700 1 --2- 172.21.15.52:0/2515169057 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f621c038710 0x7f621c03abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6218006fd0 tx=0x7f6218006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:08.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.142+0000 7f6229ffb700 1 -- 172.21.15.52:0/2515169057 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6220019bf0 con 0x7f622c105a20 2023-11-27T15:10:08.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.543+0000 7f6232eeb700 1 -- 172.21.15.52:0/2515169057 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f622c061f40 con 0x7f621c038710 2023-11-27T15:10:08.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.791+0000 7f6229ffb700 1 -- 172.21.15.52:0/2515169057 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f622c061f40 con 0x7f621c038710 2023-11-27T15:10:08.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.794+0000 7f6232eeb700 1 -- 172.21.15.52:0/2515169057 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f621c038710 msgr2=0x7f621c03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:08.236 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.794+0000 7f6232eeb700 1 --2- 172.21.15.52:0/2515169057 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f621c038710 0x7f621c03abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6218006fd0 tx=0x7f6218006e40 comp rx=0 tx=0).stop 2023-11-27T15:10:08.236 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.794+0000 7f6232eeb700 1 -- 172.21.15.52:0/2515169057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f622c105a20 msgr2=0x7f622c1997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:08.236 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.794+0000 7f6232eeb700 1 --2- 172.21.15.52:0/2515169057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f622c105a20 0x7f622c1997a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6220003e60 tx=0x7f6220011aa0 comp rx=0 tx=0).stop 2023-11-27T15:10:08.236 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.794+0000 7f6232eeb700 1 -- 172.21.15.52:0/2515169057 shutdown_connections 2023-11-27T15:10:08.236 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.794+0000 7f6232eeb700 1 --2- 172.21.15.52:0/2515169057 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f621c038710 0x7f621c03abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:08.236 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.794+0000 7f6232eeb700 1 --2- 172.21.15.52:0/2515169057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f622c105a20 0x7f622c1997a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:08.236 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.794+0000 7f6232eeb700 1 -- 172.21.15.52:0/2515169057 >> 172.21.15.52:0/2515169057 conn(0x7f622c100fd0 msgr2=0x7f622c102630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:08.236 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.794+0000 7f6232eeb700 1 -- 172.21.15.52:0/2515169057 shutdown_connections 2023-11-27T15:10:08.236 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:07.794+0000 7f6232eeb700 1 -- 172.21.15.52:0/2515169057 wait complete. 2023-11-27T15:10:08.236 INFO:teuthology.orchestra.run.smithi052.stdout:Fetching dashboard port number... 2023-11-27T15:10:08.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:08 smithi052 ceph-mon[99056]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:10:08.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:08 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:09.625 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 8443 2023-11-27T15:10:09.625 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.885+0000 7f464cdb6700 1 Processor -- start 2023-11-27T15:10:09.625 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.885+0000 7f464cdb6700 1 -- start start 2023-11-27T15:10:09.625 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.885+0000 7f464cdb6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4648104920 0x7f4648104d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:09.625 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.885+0000 7f464cdb6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4648105300 con 0x7f4648104920 2023-11-27T15:10:09.625 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.886+0000 7f464659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4648104920 0x7f4648104d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:09.625 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.886+0000 7f464659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4648104920 0x7f4648104d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49014/0 (socket says 172.21.15.52:49014) 2023-11-27T15:10:09.625 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.886+0000 7f464659c700 1 -- 172.21.15.52:0/1897859154 learned_addr learned my addr 172.21.15.52:0/1897859154 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:10:09.625 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.886+0000 7f464659c700 1 -- 172.21.15.52:0/1897859154 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4648105b20 con 0x7f4648104920 2023-11-27T15:10:09.625 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.886+0000 7f464659c700 1 --2- 172.21.15.52:0/1897859154 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4648104920 0x7f4648104d30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f46380098a0 tx=0x7f4638009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1649b5fc2c3db4 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:09.626 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.886+0000 7f464559a700 1 -- 172.21.15.52:0/1897859154 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4638004160 con 0x7f4648104920 2023-11-27T15:10:09.626 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.886+0000 7f464559a700 1 -- 172.21.15.52:0/1897859154 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46380042c0 con 0x7f4648104920 2023-11-27T15:10:09.626 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.886+0000 7f464559a700 1 -- 172.21.15.52:0/1897859154 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46380119b0 con 0x7f4648104920 2023-11-27T15:10:09.626 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.887+0000 7f464cdb6700 1 -- 172.21.15.52:0/1897859154 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4648104920 msgr2=0x7f4648104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:09.626 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.887+0000 7f464cdb6700 1 --2- 172.21.15.52:0/1897859154 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4648104920 0x7f4648104d30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f46380098a0 tx=0x7f4638009bb0 comp rx=0 tx=0).stop 2023-11-27T15:10:09.626 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.887+0000 7f464cdb6700 1 -- 172.21.15.52:0/1897859154 shutdown_connections 2023-11-27T15:10:09.626 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.887+0000 7f464cdb6700 1 --2- 172.21.15.52:0/1897859154 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4648104920 0x7f4648104d30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:09.626 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.887+0000 7f464cdb6700 1 -- 172.21.15.52:0/1897859154 >> 172.21.15.52:0/1897859154 conn(0x7f46480ffed0 msgr2=0x7f4648102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:09.626 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.887+0000 7f464cdb6700 1 -- 172.21.15.52:0/1897859154 shutdown_connections 2023-11-27T15:10:09.626 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.887+0000 7f464cdb6700 1 -- 172.21.15.52:0/1897859154 wait complete. 2023-11-27T15:10:09.626 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.888+0000 7f464cdb6700 1 Processor -- start 2023-11-27T15:10:09.626 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.888+0000 7f464cdb6700 1 -- start start 2023-11-27T15:10:09.627 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.888+0000 7f464cdb6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4648104920 0x7f46481997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:09.627 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.888+0000 7f464cdb6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4648199ce0 con 0x7f4648104920 2023-11-27T15:10:09.627 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.888+0000 7f464659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4648104920 0x7f46481997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:09.627 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.888+0000 7f464659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4648104920 0x7f46481997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49016/0 (socket says 172.21.15.52:49016) 2023-11-27T15:10:09.627 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.888+0000 7f464659c700 1 -- 172.21.15.52:0/2587467300 learned_addr learned my addr 172.21.15.52:0/2587467300 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:10:09.627 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.889+0000 7f464659c700 1 -- 172.21.15.52:0/2587467300 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4638009550 con 0x7f4648104920 2023-11-27T15:10:09.627 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.889+0000 7f464659c700 1 --2- 172.21.15.52:0/2587467300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4648104920 0x7f46481997a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4638000c00 tx=0x7f4638011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:09.627 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.889+0000 7f46377fe700 1 -- 172.21.15.52:0/2587467300 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46380039f0 con 0x7f4648104920 2023-11-27T15:10:09.627 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.889+0000 7f46377fe700 1 -- 172.21.15.52:0/2587467300 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4638003b50 con 0x7f4648104920 2023-11-27T15:10:09.627 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.889+0000 7f46377fe700 1 -- 172.21.15.52:0/2587467300 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f463801a780 con 0x7f4648104920 2023-11-27T15:10:09.627 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.889+0000 7f464cdb6700 1 -- 172.21.15.52:0/2587467300 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4648108ee0 con 0x7f4648104920 2023-11-27T15:10:09.627 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.889+0000 7f464cdb6700 1 -- 172.21.15.52:0/2587467300 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4648108d50 con 0x7f4648104920 2023-11-27T15:10:09.627 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.890+0000 7f46377fe700 1 -- 172.21.15.52:0/2587467300 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f463801a9f0 con 0x7f4648104920 2023-11-27T15:10:09.628 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.890+0000 7f46377fe700 1 --2- 172.21.15.52:0/2587467300 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f46300387b0 0x7f463003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:09.628 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.890+0000 7f46377fe700 1 -- 172.21.15.52:0/2587467300 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f463804c7e0 con 0x7f4648104920 2023-11-27T15:10:09.628 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.890+0000 7f4645d9b700 1 --2- 172.21.15.52:0/2587467300 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f46300387b0 0x7f463003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:09.628 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.891+0000 7f464cdb6700 1 -- 172.21.15.52:0/2587467300 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4648062120 con 0x7f4648104920 2023-11-27T15:10:09.628 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.891+0000 7f4645d9b700 1 --2- 172.21.15.52:0/2587467300 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f46300387b0 0x7f463003ac60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f463c006fd0 tx=0x7f463c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:09.628 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:08.896+0000 7f46377fe700 1 -- 172.21.15.52:0/2587467300 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f463801acf0 con 0x7f4648104920 2023-11-27T15:10:09.628 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:09.253+0000 7f464cdb6700 1 -- 172.21.15.52:0/2587467300 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f4648109070 con 0x7f4648104920 2023-11-27T15:10:09.628 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:09.254+0000 7f46377fe700 1 -- 172.21.15.52:0/2587467300 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f463802c350 con 0x7f4648104920 2023-11-27T15:10:09.628 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:09.256+0000 7f464cdb6700 1 -- 172.21.15.52:0/2587467300 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f46300387b0 msgr2=0x7f463003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:09.628 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:09.256+0000 7f464cdb6700 1 --2- 172.21.15.52:0/2587467300 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f46300387b0 0x7f463003ac60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f463c006fd0 tx=0x7f463c006e40 comp rx=0 tx=0).stop 2023-11-27T15:10:09.628 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:09.256+0000 7f464cdb6700 1 -- 172.21.15.52:0/2587467300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4648104920 msgr2=0x7f46481997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:09.628 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:09.256+0000 7f464cdb6700 1 --2- 172.21.15.52:0/2587467300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4648104920 0x7f46481997a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4638000c00 tx=0x7f4638011a30 comp rx=0 tx=0).stop 2023-11-27T15:10:09.629 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:09.256+0000 7f464cdb6700 1 -- 172.21.15.52:0/2587467300 shutdown_connections 2023-11-27T15:10:09.629 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:09.256+0000 7f464cdb6700 1 --2- 172.21.15.52:0/2587467300 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f46300387b0 0x7f463003ac60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:09.629 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:09.256+0000 7f464cdb6700 1 --2- 172.21.15.52:0/2587467300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4648104920 0x7f46481997a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:09.629 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:09.256+0000 7f464cdb6700 1 -- 172.21.15.52:0/2587467300 >> 172.21.15.52:0/2587467300 conn(0x7f46480ffed0 msgr2=0x7f4648079890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:09.629 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:09.256+0000 7f464cdb6700 1 -- 172.21.15.52:0/2587467300 shutdown_connections 2023-11-27T15:10:09.629 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:09.256+0000 7f464cdb6700 1 -- 172.21.15.52:0/2587467300 wait complete. 2023-11-27T15:10:09.632 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-11-27T15:10:09.632 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout disabled 2023-11-27T15:10:09.640 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-11-27T15:10:09.640 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout inactive 2023-11-27T15:10:09.640 INFO:teuthology.orchestra.run.smithi052.stdout:firewalld.service is not enabled 2023-11-27T15:10:09.640 INFO:teuthology.orchestra.run.smithi052.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-11-27T15:10:09.642 INFO:teuthology.orchestra.run.smithi052.stdout:Ceph Dashboard is now available at: 2023-11-27T15:10:09.642 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:10:09.642 INFO:teuthology.orchestra.run.smithi052.stdout: URL: https://smithi052:8443/ 2023-11-27T15:10:09.642 INFO:teuthology.orchestra.run.smithi052.stdout: User: admin 2023-11-27T15:10:09.642 INFO:teuthology.orchestra.run.smithi052.stdout: Password: 3u3umfvhif 2023-11-27T15:10:09.642 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:10:09.642 INFO:teuthology.orchestra.run.smithi052.stdout:Saving cluster configuration to /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config directory 2023-11-27T15:10:09.643 INFO:teuthology.orchestra.run.smithi052.stdout:Enabling autotune for osd_memory_target 2023-11-27T15:10:09.906 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:09 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2587467300' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-11-27T15:10:10.888 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.123+0000 7f91d2cbd700 1 Processor -- start 2023-11-27T15:10:10.888 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.123+0000 7f91d2cbd700 1 -- start start 2023-11-27T15:10:10.888 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.123+0000 7f91d2cbd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91cc106490 0x7f91cc108870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:10.888 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.123+0000 7f91d2cbd700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc079e60 con 0x7f91cc106490 2023-11-27T15:10:10.888 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.123+0000 7f91d0a59700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91cc106490 0x7f91cc108870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:10.888 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.123+0000 7f91d0a59700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91cc106490 0x7f91cc108870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49024/0 (socket says 172.21.15.52:49024) 2023-11-27T15:10:10.888 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.123+0000 7f91d0a59700 1 -- 172.21.15.52:0/1468474840 learned_addr learned my addr 172.21.15.52:0/1468474840 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:10:10.888 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.123+0000 7f91d0a59700 1 -- 172.21.15.52:0/1468474840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc108db0 con 0x7f91cc106490 2023-11-27T15:10:10.888 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.124+0000 7f91d0a59700 1 --2- 172.21.15.52:0/1468474840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91cc106490 0x7f91cc108870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f91c00098a0 tx=0x7f91c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1b7168e0faf497 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:10.888 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.124+0000 7f91cb7fe700 1 -- 172.21.15.52:0/1468474840 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91c0004160 con 0x7f91cc106490 2023-11-27T15:10:10.889 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.124+0000 7f91cb7fe700 1 -- 172.21.15.52:0/1468474840 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91c00042c0 con 0x7f91cc106490 2023-11-27T15:10:10.889 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.124+0000 7f91cb7fe700 1 -- 172.21.15.52:0/1468474840 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91c00119b0 con 0x7f91cc106490 2023-11-27T15:10:10.889 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.124+0000 7f91d2cbd700 1 -- 172.21.15.52:0/1468474840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91cc106490 msgr2=0x7f91cc108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:10.889 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.124+0000 7f91d2cbd700 1 --2- 172.21.15.52:0/1468474840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91cc106490 0x7f91cc108870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f91c00098a0 tx=0x7f91c0009bb0 comp rx=0 tx=0).stop 2023-11-27T15:10:10.889 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.124+0000 7f91d2cbd700 1 -- 172.21.15.52:0/1468474840 shutdown_connections 2023-11-27T15:10:10.889 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.124+0000 7f91d2cbd700 1 --2- 172.21.15.52:0/1468474840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91cc106490 0x7f91cc108870 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:10.889 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.124+0000 7f91d2cbd700 1 -- 172.21.15.52:0/1468474840 >> 172.21.15.52:0/1468474840 conn(0x7f91cc0ffed0 msgr2=0x7f91cc102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:10.889 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.124+0000 7f91d2cbd700 1 -- 172.21.15.52:0/1468474840 shutdown_connections 2023-11-27T15:10:10.889 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.125+0000 7f91d2cbd700 1 -- 172.21.15.52:0/1468474840 wait complete. 2023-11-27T15:10:10.889 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.125+0000 7f91d2cbd700 1 Processor -- start 2023-11-27T15:10:10.889 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.125+0000 7f91d2cbd700 1 -- start start 2023-11-27T15:10:10.890 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.125+0000 7f91d2cbd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91cc106490 0x7f91cc105df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:10.890 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.125+0000 7f91d2cbd700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc106330 con 0x7f91cc106490 2023-11-27T15:10:10.890 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.125+0000 7f91d0a59700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91cc106490 0x7f91cc105df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:10.890 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.125+0000 7f91d0a59700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91cc106490 0x7f91cc105df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49026/0 (socket says 172.21.15.52:49026) 2023-11-27T15:10:10.890 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.125+0000 7f91d0a59700 1 -- 172.21.15.52:0/3845350116 learned_addr learned my addr 172.21.15.52:0/3845350116 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:10:10.890 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.126+0000 7f91d0a59700 1 -- 172.21.15.52:0/3845350116 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c0009550 con 0x7f91cc106490 2023-11-27T15:10:10.890 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.126+0000 7f91d0a59700 1 --2- 172.21.15.52:0/3845350116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91cc106490 0x7f91cc105df0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f91c0003e60 tx=0x7f91c0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:10.890 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.126+0000 7f91c9ffb700 1 -- 172.21.15.52:0/3845350116 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91c0003970 con 0x7f91cc106490 2023-11-27T15:10:10.890 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.126+0000 7f91c9ffb700 1 -- 172.21.15.52:0/3845350116 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91c0003ad0 con 0x7f91cc106490 2023-11-27T15:10:10.890 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.126+0000 7f91d2cbd700 1 -- 172.21.15.52:0/3845350116 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91cc109d80 con 0x7f91cc106490 2023-11-27T15:10:10.890 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.126+0000 7f91c9ffb700 1 -- 172.21.15.52:0/3845350116 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91c00195e0 con 0x7f91cc106490 2023-11-27T15:10:10.891 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.126+0000 7f91d2cbd700 1 -- 172.21.15.52:0/3845350116 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91cc109bf0 con 0x7f91cc106490 2023-11-27T15:10:10.891 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.127+0000 7f91c9ffb700 1 -- 172.21.15.52:0/3845350116 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f91c0019850 con 0x7f91cc106490 2023-11-27T15:10:10.891 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.127+0000 7f91d2cbd700 1 -- 172.21.15.52:0/3845350116 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91cc109f10 con 0x7f91cc106490 2023-11-27T15:10:10.891 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.127+0000 7f91c9ffb700 1 --2- 172.21.15.52:0/3845350116 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f91bc038760 0x7f91bc03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:10.891 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.128+0000 7f91c9ffb700 1 -- 172.21.15.52:0/3845350116 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f91c004c9b0 con 0x7f91cc106490 2023-11-27T15:10:10.891 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.128+0000 7f91cbfff700 1 --2- 172.21.15.52:0/3845350116 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f91bc038760 0x7f91bc03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:10.891 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.128+0000 7f91cbfff700 1 --2- 172.21.15.52:0/3845350116 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f91bc038760 0x7f91bc03ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f91b8006fd0 tx=0x7f91b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:10.891 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.132+0000 7f91c9ffb700 1 -- 172.21.15.52:0/3845350116 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f91c002b340 con 0x7f91cc106490 2023-11-27T15:10:10.891 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.487+0000 7f91d2cbd700 1 -- 172.21.15.52:0/3845350116 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f91cc104fe0 con 0x7f91cc106490 2023-11-27T15:10:10.891 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.487+0000 7f91c9ffb700 1 -- 172.21.15.52:0/3845350116 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f91c0018b40 con 0x7f91cc106490 2023-11-27T15:10:10.891 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.490+0000 7f91d2cbd700 1 -- 172.21.15.52:0/3845350116 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f91bc038760 msgr2=0x7f91bc03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:10.891 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.490+0000 7f91d2cbd700 1 --2- 172.21.15.52:0/3845350116 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f91bc038760 0x7f91bc03ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f91b8006fd0 tx=0x7f91b8006e40 comp rx=0 tx=0).stop 2023-11-27T15:10:10.892 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.490+0000 7f91d2cbd700 1 -- 172.21.15.52:0/3845350116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91cc106490 msgr2=0x7f91cc105df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:10.892 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.490+0000 7f91d2cbd700 1 --2- 172.21.15.52:0/3845350116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91cc106490 0x7f91cc105df0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f91c0003e60 tx=0x7f91c0011aa0 comp rx=0 tx=0).stop 2023-11-27T15:10:10.892 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.490+0000 7f91d2cbd700 1 -- 172.21.15.52:0/3845350116 shutdown_connections 2023-11-27T15:10:10.892 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.490+0000 7f91d2cbd700 1 --2- 172.21.15.52:0/3845350116 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f91bc038760 0x7f91bc03ac10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:10.892 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.490+0000 7f91d2cbd700 1 --2- 172.21.15.52:0/3845350116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91cc106490 0x7f91cc105df0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:10.892 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.490+0000 7f91d2cbd700 1 -- 172.21.15.52:0/3845350116 >> 172.21.15.52:0/3845350116 conn(0x7f91cc0ffed0 msgr2=0x7f91cc101530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:10.892 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.490+0000 7f91d2cbd700 1 -- 172.21.15.52:0/3845350116 shutdown_connections 2023-11-27T15:10:10.892 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:10.490+0000 7f91d2cbd700 1 -- 172.21.15.52:0/3845350116 wait complete. 2023-11-27T15:10:11.940 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:11 smithi052 ceph-mon[99056]: mgrmap e13: x(active, since 7s) 2023-11-27T15:10:12.234 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.320+0000 7fd2a1d54700 1 Processor -- start 2023-11-27T15:10:12.234 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.320+0000 7fd2a1d54700 1 -- start start 2023-11-27T15:10:12.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.320+0000 7fd2a1d54700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd29c104300 0x7fd29c104710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:12.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.320+0000 7fd2a1d54700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c079de0 con 0x7fd29c104300 2023-11-27T15:10:12.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.321+0000 7fd29b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd29c104300 0x7fd29c104710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:12.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.321+0000 7fd29b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd29c104300 0x7fd29c104710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49030/0 (socket says 172.21.15.52:49030) 2023-11-27T15:10:12.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.321+0000 7fd29b7fe700 1 -- 172.21.15.52:0/2727688398 learned_addr learned my addr 172.21.15.52:0/2727688398 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:10:12.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.321+0000 7fd29b7fe700 1 -- 172.21.15.52:0/2727688398 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29c104c50 con 0x7fd29c104300 2023-11-27T15:10:12.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.321+0000 7fd29b7fe700 1 --2- 172.21.15.52:0/2727688398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd29c104300 0x7fd29c104710 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd28c0098a0 tx=0x7fd28c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bebf5d7d902b5146 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:12.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.321+0000 7fd29a7fc700 1 -- 172.21.15.52:0/2727688398 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd28c004160 con 0x7fd29c104300 2023-11-27T15:10:12.235 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.321+0000 7fd29a7fc700 1 -- 172.21.15.52:0/2727688398 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd28c0042c0 con 0x7fd29c104300 2023-11-27T15:10:12.236 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.321+0000 7fd29a7fc700 1 -- 172.21.15.52:0/2727688398 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd28c0119b0 con 0x7fd29c104300 2023-11-27T15:10:12.236 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.322+0000 7fd2a1d54700 1 -- 172.21.15.52:0/2727688398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd29c104300 msgr2=0x7fd29c104710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:12.236 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.322+0000 7fd2a1d54700 1 --2- 172.21.15.52:0/2727688398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd29c104300 0x7fd29c104710 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd28c0098a0 tx=0x7fd28c009bb0 comp rx=0 tx=0).stop 2023-11-27T15:10:12.236 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.322+0000 7fd2a1d54700 1 -- 172.21.15.52:0/2727688398 shutdown_connections 2023-11-27T15:10:12.236 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.322+0000 7fd2a1d54700 1 --2- 172.21.15.52:0/2727688398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd29c104300 0x7fd29c104710 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:12.236 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.322+0000 7fd2a1d54700 1 -- 172.21.15.52:0/2727688398 >> 172.21.15.52:0/2727688398 conn(0x7fd29c0ffed0 msgr2=0x7fd29c102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:12.236 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.322+0000 7fd2a1d54700 1 -- 172.21.15.52:0/2727688398 shutdown_connections 2023-11-27T15:10:12.236 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.322+0000 7fd2a1d54700 1 -- 172.21.15.52:0/2727688398 wait complete. 2023-11-27T15:10:12.237 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.323+0000 7fd2a1d54700 1 Processor -- start 2023-11-27T15:10:12.237 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.323+0000 7fd2a1d54700 1 -- start start 2023-11-27T15:10:12.237 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.323+0000 7fd2a1d54700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd29c104300 0x7fd29c19b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:12.237 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.323+0000 7fd2a1d54700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c19bf00 con 0x7fd29c104300 2023-11-27T15:10:12.237 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.323+0000 7fd29b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd29c104300 0x7fd29c19b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:12.237 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.323+0000 7fd29b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd29c104300 0x7fd29c19b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49032/0 (socket says 172.21.15.52:49032) 2023-11-27T15:10:12.237 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.323+0000 7fd29b7fe700 1 -- 172.21.15.52:0/1184612253 learned_addr learned my addr 172.21.15.52:0/1184612253 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:10:12.237 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.324+0000 7fd29b7fe700 1 -- 172.21.15.52:0/1184612253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28c009550 con 0x7fd29c104300 2023-11-27T15:10:12.237 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.324+0000 7fd29b7fe700 1 --2- 172.21.15.52:0/1184612253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd29c104300 0x7fd29c19b9c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd28c011980 tx=0x7fd28c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:12.238 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.324+0000 7fd298ff9700 1 -- 172.21.15.52:0/1184612253 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd28c003a60 con 0x7fd29c104300 2023-11-27T15:10:12.238 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.324+0000 7fd2a1d54700 1 -- 172.21.15.52:0/1184612253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd29c105c90 con 0x7fd29c104300 2023-11-27T15:10:12.238 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.324+0000 7fd2a1d54700 1 -- 172.21.15.52:0/1184612253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd29c105b00 con 0x7fd29c104300 2023-11-27T15:10:12.238 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.324+0000 7fd298ff9700 1 -- 172.21.15.52:0/1184612253 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd28c003bc0 con 0x7fd29c104300 2023-11-27T15:10:12.238 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.324+0000 7fd298ff9700 1 -- 172.21.15.52:0/1184612253 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd28c01a670 con 0x7fd29c104300 2023-11-27T15:10:12.238 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.325+0000 7fd298ff9700 1 -- 172.21.15.52:0/1184612253 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fd28c01a900 con 0x7fd29c104300 2023-11-27T15:10:12.238 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.325+0000 7fd298ff9700 1 --2- 172.21.15.52:0/1184612253 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd284038850 0x7fd28403ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:12.238 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.325+0000 7fd2a1d54700 1 -- 172.21.15.52:0/1184612253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd288005320 con 0x7fd29c104300 2023-11-27T15:10:12.238 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.325+0000 7fd298ff9700 1 -- 172.21.15.52:0/1184612253 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd28c04c920 con 0x7fd29c104300 2023-11-27T15:10:12.238 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.325+0000 7fd29affd700 1 --2- 172.21.15.52:0/1184612253 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd284038850 0x7fd28403ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:12.239 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.326+0000 7fd29affd700 1 --2- 172.21.15.52:0/1184612253 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd284038850 0x7fd28403ad00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd290006fd0 tx=0x7fd290006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:12.239 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.330+0000 7fd298ff9700 1 -- 172.21.15.52:0/1184612253 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd28c01ac00 con 0x7fd29c104300 2023-11-27T15:10:12.239 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.903+0000 7fd2a1d54700 1 -- 172.21.15.52:0/1184612253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fd288005f70 con 0x7fd29c104300 2023-11-27T15:10:12.239 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.906+0000 7fd298ff9700 1 -- 172.21.15.52:0/1184612253 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7fd28c018350 con 0x7fd29c104300 2023-11-27T15:10:12.239 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.908+0000 7fd2a1d54700 1 -- 172.21.15.52:0/1184612253 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd284038850 msgr2=0x7fd28403ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:12.239 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.908+0000 7fd2a1d54700 1 --2- 172.21.15.52:0/1184612253 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd284038850 0x7fd28403ad00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd290006fd0 tx=0x7fd290006e40 comp rx=0 tx=0).stop 2023-11-27T15:10:12.239 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.908+0000 7fd2a1d54700 1 -- 172.21.15.52:0/1184612253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd29c104300 msgr2=0x7fd29c19b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:12.239 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.908+0000 7fd2a1d54700 1 --2- 172.21.15.52:0/1184612253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd29c104300 0x7fd29c19b9c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd28c011980 tx=0x7fd28c011a60 comp rx=0 tx=0).stop 2023-11-27T15:10:12.239 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.908+0000 7fd2a1d54700 1 -- 172.21.15.52:0/1184612253 shutdown_connections 2023-11-27T15:10:12.240 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.908+0000 7fd2a1d54700 1 --2- 172.21.15.52:0/1184612253 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd284038850 0x7fd28403ad00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:12.240 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.908+0000 7fd2a1d54700 1 --2- 172.21.15.52:0/1184612253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd29c104300 0x7fd29c19b9c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:12.240 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.908+0000 7fd2a1d54700 1 -- 172.21.15.52:0/1184612253 >> 172.21.15.52:0/1184612253 conn(0x7fd29c0ffed0 msgr2=0x7fd29c18fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:12.240 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.908+0000 7fd2a1d54700 1 -- 172.21.15.52:0/1184612253 shutdown_connections 2023-11-27T15:10:12.240 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2023-11-27T15:10:11.908+0000 7fd2a1d54700 1 -- 172.21.15.52:0/1184612253 wait complete. 2023-11-27T15:10:12.240 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-11-27T15:10:12.240 INFO:teuthology.orchestra.run.smithi052.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-11-27T15:10:12.240 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:10:12.241 INFO:teuthology.orchestra.run.smithi052.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-11-27T15:10:12.241 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:10:12.241 INFO:teuthology.orchestra.run.smithi052.stdout:Or, if you are only running a single cluster on this host: 2023-11-27T15:10:12.241 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:10:12.241 INFO:teuthology.orchestra.run.smithi052.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-11-27T15:10:12.241 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:10:12.241 INFO:teuthology.orchestra.run.smithi052.stdout:Please consider enabling telemetry to help improve Ceph: 2023-11-27T15:10:12.241 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:10:12.241 INFO:teuthology.orchestra.run.smithi052.stdout: ceph telemetry on 2023-11-27T15:10:12.242 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:10:12.242 INFO:teuthology.orchestra.run.smithi052.stdout:For more information see: 2023-11-27T15:10:12.242 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:10:12.242 INFO:teuthology.orchestra.run.smithi052.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-11-27T15:10:12.242 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:10:12.242 INFO:teuthology.orchestra.run.smithi052.stdout:Bootstrap complete. 2023-11-27T15:10:12.268 INFO:tasks.cephadm:Fetching config... 2023-11-27T15:10:12.268 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2023-11-27T15:10:12.268 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-11-27T15:10:12.285 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-11-27T15:10:12.285 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2023-11-27T15:10:12.285 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-11-27T15:10:12.345 INFO:tasks.cephadm:Fetching mon keyring... 2023-11-27T15:10:12.345 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2023-11-27T15:10:12.345 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/keyring of=/dev/stdout 2023-11-27T15:10:12.415 INFO:tasks.cephadm:Fetching pub ssh key... 2023-11-27T15:10:12.415 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2023-11-27T15:10:12.415 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-11-27T15:10:12.473 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-11-27T15:10:12.473 DEBUG:teuthology.orchestra.run.smithi052:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLCE2NNTGbbJIJFAABAKWK9Tyg1T6556o23zQIHlyRx96xVQ7mYDOweDwTqeanDrGpJK/B63C3eiBwvXsymgJOflZIMepVY8GNJjbE6C1xp2hiYWzsqDOsNdVQmW9dzwdVzlzg9uBFtVPVmpQLXROYBaOVDK97vIPq36ng4+cENb0mM66xNrBx8KRrF8stCB4LZn/UA+0pUAp0kN8rjLTRGiSyXIDiqKquzgZl2f7M7C38YTKqOPb2FcqMlW7enwLO2QHNEbKAwEnIsRvUns9Oddj+lh8C4Xi89Ag5sqMRc4+NdElMF/M+xFAvkHsCn5ELf96IzyFNE7tQ1JpIKY3xwsFOLz2ShN7z3BAuhReJObR0dnSRetD+5e7I/OT9QxfjYj53zTNXlChj8FohCaYp5vmpO/cPF7LSghFWzCtWgV5UipC5aONBFnvpqVeBhIiGk44qvwnlNZmifTLIaZd4PCtwPajQGsJpRL3enbJCLltUOitSsw2dB7bNRZRYock= ceph-c6c7df88-8d36-11ee-95a2-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-11-27T15:10:12.550 INFO:teuthology.orchestra.run.smithi052.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLCE2NNTGbbJIJFAABAKWK9Tyg1T6556o23zQIHlyRx96xVQ7mYDOweDwTqeanDrGpJK/B63C3eiBwvXsymgJOflZIMepVY8GNJjbE6C1xp2hiYWzsqDOsNdVQmW9dzwdVzlzg9uBFtVPVmpQLXROYBaOVDK97vIPq36ng4+cENb0mM66xNrBx8KRrF8stCB4LZn/UA+0pUAp0kN8rjLTRGiSyXIDiqKquzgZl2f7M7C38YTKqOPb2FcqMlW7enwLO2QHNEbKAwEnIsRvUns9Oddj+lh8C4Xi89Ag5sqMRc4+NdElMF/M+xFAvkHsCn5ELf96IzyFNE7tQ1JpIKY3xwsFOLz2ShN7z3BAuhReJObR0dnSRetD+5e7I/OT9QxfjYj53zTNXlChj8FohCaYp5vmpO/cPF7LSghFWzCtWgV5UipC5aONBFnvpqVeBhIiGk44qvwnlNZmifTLIaZd4PCtwPajQGsJpRL3enbJCLltUOitSsw2dB7bNRZRYock= ceph-c6c7df88-8d36-11ee-95a2-87774f69a715 2023-11-27T15:10:12.565 DEBUG:teuthology.orchestra.run.smithi102:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLCE2NNTGbbJIJFAABAKWK9Tyg1T6556o23zQIHlyRx96xVQ7mYDOweDwTqeanDrGpJK/B63C3eiBwvXsymgJOflZIMepVY8GNJjbE6C1xp2hiYWzsqDOsNdVQmW9dzwdVzlzg9uBFtVPVmpQLXROYBaOVDK97vIPq36ng4+cENb0mM66xNrBx8KRrF8stCB4LZn/UA+0pUAp0kN8rjLTRGiSyXIDiqKquzgZl2f7M7C38YTKqOPb2FcqMlW7enwLO2QHNEbKAwEnIsRvUns9Oddj+lh8C4Xi89Ag5sqMRc4+NdElMF/M+xFAvkHsCn5ELf96IzyFNE7tQ1JpIKY3xwsFOLz2ShN7z3BAuhReJObR0dnSRetD+5e7I/OT9QxfjYj53zTNXlChj8FohCaYp5vmpO/cPF7LSghFWzCtWgV5UipC5aONBFnvpqVeBhIiGk44qvwnlNZmifTLIaZd4PCtwPajQGsJpRL3enbJCLltUOitSsw2dB7bNRZRYock= ceph-c6c7df88-8d36-11ee-95a2-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-11-27T15:10:12.612 INFO:teuthology.orchestra.run.smithi102.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLCE2NNTGbbJIJFAABAKWK9Tyg1T6556o23zQIHlyRx96xVQ7mYDOweDwTqeanDrGpJK/B63C3eiBwvXsymgJOflZIMepVY8GNJjbE6C1xp2hiYWzsqDOsNdVQmW9dzwdVzlzg9uBFtVPVmpQLXROYBaOVDK97vIPq36ng4+cENb0mM66xNrBx8KRrF8stCB4LZn/UA+0pUAp0kN8rjLTRGiSyXIDiqKquzgZl2f7M7C38YTKqOPb2FcqMlW7enwLO2QHNEbKAwEnIsRvUns9Oddj+lh8C4Xi89Ag5sqMRc4+NdElMF/M+xFAvkHsCn5ELf96IzyFNE7tQ1JpIKY3xwsFOLz2ShN7z3BAuhReJObR0dnSRetD+5e7I/OT9QxfjYj53zTNXlChj8FohCaYp5vmpO/cPF7LSghFWzCtWgV5UipC5aONBFnvpqVeBhIiGk44qvwnlNZmifTLIaZd4PCtwPajQGsJpRL3enbJCLltUOitSsw2dB7bNRZRYock= ceph-c6c7df88-8d36-11ee-95a2-87774f69a715 2023-11-27T15:10:12.624 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLCE2NNTGbbJIJFAABAKWK9Tyg1T6556o23zQIHlyRx96xVQ7mYDOweDwTqeanDrGpJK/B63C3eiBwvXsymgJOflZIMepVY8GNJjbE6C1xp2hiYWzsqDOsNdVQmW9dzwdVzlzg9uBFtVPVmpQLXROYBaOVDK97vIPq36ng4+cENb0mM66xNrBx8KRrF8stCB4LZn/UA+0pUAp0kN8rjLTRGiSyXIDiqKquzgZl2f7M7C38YTKqOPb2FcqMlW7enwLO2QHNEbKAwEnIsRvUns9Oddj+lh8C4Xi89Ag5sqMRc4+NdElMF/M+xFAvkHsCn5ELf96IzyFNE7tQ1JpIKY3xwsFOLz2ShN7z3BAuhReJObR0dnSRetD+5e7I/OT9QxfjYj53zTNXlChj8FohCaYp5vmpO/cPF7LSghFWzCtWgV5UipC5aONBFnvpqVeBhIiGk44qvwnlNZmifTLIaZd4PCtwPajQGsJpRL3enbJCLltUOitSsw2dB7bNRZRYock= ceph-c6c7df88-8d36-11ee-95a2-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-11-27T15:10:12.672 INFO:teuthology.orchestra.run.smithi150.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLCE2NNTGbbJIJFAABAKWK9Tyg1T6556o23zQIHlyRx96xVQ7mYDOweDwTqeanDrGpJK/B63C3eiBwvXsymgJOflZIMepVY8GNJjbE6C1xp2hiYWzsqDOsNdVQmW9dzwdVzlzg9uBFtVPVmpQLXROYBaOVDK97vIPq36ng4+cENb0mM66xNrBx8KRrF8stCB4LZn/UA+0pUAp0kN8rjLTRGiSyXIDiqKquzgZl2f7M7C38YTKqOPb2FcqMlW7enwLO2QHNEbKAwEnIsRvUns9Oddj+lh8C4Xi89Ag5sqMRc4+NdElMF/M+xFAvkHsCn5ELf96IzyFNE7tQ1JpIKY3xwsFOLz2ShN7z3BAuhReJObR0dnSRetD+5e7I/OT9QxfjYj53zTNXlChj8FohCaYp5vmpO/cPF7LSghFWzCtWgV5UipC5aONBFnvpqVeBhIiGk44qvwnlNZmifTLIaZd4PCtwPajQGsJpRL3enbJCLltUOitSsw2dB7bNRZRYock= ceph-c6c7df88-8d36-11ee-95a2-87774f69a715 2023-11-27T15:10:12.685 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-11-27T15:10:13.186 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:12 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1184612253' entity='client.admin' 2023-11-27T15:10:13.567 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:10:15.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.215+0000 7f0f9fc06700 1 -- 172.21.15.52:0/1078921852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f98074b20 msgr2=0x7f0f98072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:15.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.215+0000 7f0f9fc06700 1 --2- 172.21.15.52:0/1078921852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f98074b20 0x7f0f98072f80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0f94009910 tx=0x7f0f94009c20 comp rx=0 tx=0).stop 2023-11-27T15:10:15.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.215+0000 7f0f9fc06700 1 -- 172.21.15.52:0/1078921852 shutdown_connections 2023-11-27T15:10:15.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.215+0000 7f0f9fc06700 1 --2- 172.21.15.52:0/1078921852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f98074b20 0x7f0f98072f80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:15.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.215+0000 7f0f9fc06700 1 -- 172.21.15.52:0/1078921852 >> 172.21.15.52:0/1078921852 conn(0x7f0f980fb2b0 msgr2=0x7f0f980fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:15.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.215+0000 7f0f9fc06700 1 -- 172.21.15.52:0/1078921852 shutdown_connections 2023-11-27T15:10:15.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.215+0000 7f0f9fc06700 1 -- 172.21.15.52:0/1078921852 wait complete. 2023-11-27T15:10:15.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.216+0000 7f0f9fc06700 1 Processor -- start 2023-11-27T15:10:15.222 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.216+0000 7f0f9fc06700 1 -- start start 2023-11-27T15:10:15.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.216+0000 7f0f9fc06700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f98074b20 0x7f0f98197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:15.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.216+0000 7f0f9fc06700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f98197540 con 0x7f0f98074b20 2023-11-27T15:10:15.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.216+0000 7f0f9d9a2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f98074b20 0x7f0f98197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:15.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.216+0000 7f0f9d9a2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f98074b20 0x7f0f98197000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49038/0 (socket says 172.21.15.52:49038) 2023-11-27T15:10:15.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.217+0000 7f0f9d9a2700 1 -- 172.21.15.52:0/3953909513 learned_addr learned my addr 172.21.15.52:0/3953909513 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:10:15.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.217+0000 7f0f9d9a2700 1 -- 172.21.15.52:0/3953909513 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f940095f0 con 0x7f0f98074b20 2023-11-27T15:10:15.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.217+0000 7f0f9d9a2700 1 --2- 172.21.15.52:0/3953909513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f98074b20 0x7f0f98197000 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0f94005f50 tx=0x7f0f940057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:15.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.217+0000 7f0f8effd700 1 -- 172.21.15.52:0/3953909513 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f9401d070 con 0x7f0f98074b20 2023-11-27T15:10:15.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.217+0000 7f0f8effd700 1 -- 172.21.15.52:0/3953909513 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f94022470 con 0x7f0f98074b20 2023-11-27T15:10:15.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.217+0000 7f0f8effd700 1 -- 172.21.15.52:0/3953909513 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f9400f5f0 con 0x7f0f98074b20 2023-11-27T15:10:15.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.217+0000 7f0f9fc06700 1 -- 172.21.15.52:0/3953909513 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f98107dd0 con 0x7f0f98074b20 2023-11-27T15:10:15.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.217+0000 7f0f9fc06700 1 -- 172.21.15.52:0/3953909513 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f98107c40 con 0x7f0f98074b20 2023-11-27T15:10:15.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.218+0000 7f0f8effd700 1 -- 172.21.15.52:0/3953909513 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f0f9400f8b0 con 0x7f0f98074b20 2023-11-27T15:10:15.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.218+0000 7f0f9fc06700 1 -- 172.21.15.52:0/3953909513 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f98107f60 con 0x7f0f98074b20 2023-11-27T15:10:15.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.218+0000 7f0f8effd700 1 --2- 172.21.15.52:0/3953909513 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0f84038870 0x7f0f8403ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:15.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.218+0000 7f0f8effd700 1 -- 172.21.15.52:0/3953909513 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0f9404cc10 con 0x7f0f98074b20 2023-11-27T15:10:15.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.218+0000 7f0f9d1a1700 1 --2- 172.21.15.52:0/3953909513 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0f84038870 0x7f0f8403ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:15.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.219+0000 7f0f9d1a1700 1 --2- 172.21.15.52:0/3953909513 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0f84038870 0x7f0f8403ad20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0f88006fd0 tx=0x7f0f88006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:15.228 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.223+0000 7f0f8effd700 1 -- 172.21.15.52:0/3953909513 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0f94020d40 con 0x7f0f98074b20 2023-11-27T15:10:15.433 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.428+0000 7f0f9fc06700 1 -- 172.21.15.52:0/3953909513 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f0f98197b30 con 0x7f0f98074b20 2023-11-27T15:10:15.436 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.431+0000 7f0f8effd700 1 -- 172.21.15.52:0/3953909513 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f0f94027030 con 0x7f0f98074b20 2023-11-27T15:10:15.439 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.434+0000 7f0f9fc06700 1 -- 172.21.15.52:0/3953909513 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0f84038870 msgr2=0x7f0f8403ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:15.439 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.434+0000 7f0f9fc06700 1 --2- 172.21.15.52:0/3953909513 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0f84038870 0x7f0f8403ad20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0f88006fd0 tx=0x7f0f88006e40 comp rx=0 tx=0).stop 2023-11-27T15:10:15.439 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.434+0000 7f0f9fc06700 1 -- 172.21.15.52:0/3953909513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f98074b20 msgr2=0x7f0f98197000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:15.439 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.434+0000 7f0f9fc06700 1 --2- 172.21.15.52:0/3953909513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f98074b20 0x7f0f98197000 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0f94005f50 tx=0x7f0f940057c0 comp rx=0 tx=0).stop 2023-11-27T15:10:15.439 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.434+0000 7f0f9fc06700 1 -- 172.21.15.52:0/3953909513 shutdown_connections 2023-11-27T15:10:15.440 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.434+0000 7f0f9fc06700 1 --2- 172.21.15.52:0/3953909513 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0f84038870 0x7f0f8403ad20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:15.440 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.434+0000 7f0f9fc06700 1 --2- 172.21.15.52:0/3953909513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f98074b20 0x7f0f98197000 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:15.440 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.434+0000 7f0f9fc06700 1 -- 172.21.15.52:0/3953909513 >> 172.21.15.52:0/3953909513 conn(0x7f0f980fb2b0 msgr2=0x7f0f980fbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:15.440 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.434+0000 7f0f9fc06700 1 -- 172.21.15.52:0/3953909513 shutdown_connections 2023-11-27T15:10:15.440 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:15.434+0000 7f0f9fc06700 1 -- 172.21.15.52:0/3953909513 wait complete. 2023-11-27T15:10:15.843 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-11-27T15:10:15.843 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-11-27T15:10:16.494 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:10:16.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:16 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:16.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:16 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:16.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:16 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi052", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:10:16.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:16 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:16.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:16 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:10:16.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:16 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:16.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:16 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3953909513' entity='client.admin' 2023-11-27T15:10:16.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:16 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:10:16.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:16 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:17.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.728+0000 7f8bcd54c700 1 -- 172.21.15.52:0/2582136181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8bc8100820 msgr2=0x7f8bc8102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:17.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.728+0000 7f8bcd54c700 1 --2- 172.21.15.52:0/2582136181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8bc8100820 0x7f8bc8102c00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009910 tx=0x7f8bb8009c20 comp rx=0 tx=0).stop 2023-11-27T15:10:17.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.728+0000 7f8bcd54c700 1 -- 172.21.15.52:0/2582136181 shutdown_connections 2023-11-27T15:10:17.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.728+0000 7f8bcd54c700 1 --2- 172.21.15.52:0/2582136181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8bc8100820 0x7f8bc8102c00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:17.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.728+0000 7f8bcd54c700 1 -- 172.21.15.52:0/2582136181 >> 172.21.15.52:0/2582136181 conn(0x7f8bc80fa240 msgr2=0x7f8bc80fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:17.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.728+0000 7f8bcd54c700 1 -- 172.21.15.52:0/2582136181 shutdown_connections 2023-11-27T15:10:17.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.728+0000 7f8bcd54c700 1 -- 172.21.15.52:0/2582136181 wait complete. 2023-11-27T15:10:17.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.729+0000 7f8bcd54c700 1 Processor -- start 2023-11-27T15:10:17.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.729+0000 7f8bcd54c700 1 -- start start 2023-11-27T15:10:17.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.729+0000 7f8bcd54c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8bc8100820 0x7f8bc8194de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:17.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.729+0000 7f8bcd54c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc8195320 con 0x7f8bc8100820 2023-11-27T15:10:17.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.729+0000 7f8bc6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8bc8100820 0x7f8bc8194de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:17.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.729+0000 7f8bc6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8bc8100820 0x7f8bc8194de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49044/0 (socket says 172.21.15.52:49044) 2023-11-27T15:10:17.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.729+0000 7f8bc6ffd700 1 -- 172.21.15.52:0/1238616028 learned_addr learned my addr 172.21.15.52:0/1238616028 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:10:17.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.730+0000 7f8bc6ffd700 1 -- 172.21.15.52:0/1238616028 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb80095f0 con 0x7f8bc8100820 2023-11-27T15:10:17.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.730+0000 7f8bc6ffd700 1 --2- 172.21.15.52:0/1238616028 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8bc8100820 0x7f8bc8194de0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8006010 tx=0x7f8bb80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:17.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.730+0000 7f8baffff700 1 -- 172.21.15.52:0/1238616028 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8bb801d070 con 0x7f8bc8100820 2023-11-27T15:10:17.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.730+0000 7f8baffff700 1 -- 172.21.15.52:0/1238616028 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bb80044e0 con 0x7f8bc8100820 2023-11-27T15:10:17.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.730+0000 7f8bcd54c700 1 -- 172.21.15.52:0/1238616028 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc8105b30 con 0x7f8bc8100820 2023-11-27T15:10:17.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.730+0000 7f8baffff700 1 -- 172.21.15.52:0/1238616028 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8bb80039b0 con 0x7f8bc8100820 2023-11-27T15:10:17.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.730+0000 7f8bcd54c700 1 -- 172.21.15.52:0/1238616028 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc81059a0 con 0x7f8bc8100820 2023-11-27T15:10:17.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.731+0000 7f8baffff700 1 -- 172.21.15.52:0/1238616028 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f8bb8003c70 con 0x7f8bc8100820 2023-11-27T15:10:17.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.731+0000 7f8bcd54c700 1 -- 172.21.15.52:0/1238616028 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc8105cc0 con 0x7f8bc8100820 2023-11-27T15:10:17.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.731+0000 7f8baffff700 1 --2- 172.21.15.52:0/1238616028 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f8bb0038870 0x7f8bb003ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:17.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.731+0000 7f8baffff700 1 -- 172.21.15.52:0/1238616028 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8bb804c810 con 0x7f8bc8100820 2023-11-27T15:10:17.738 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.731+0000 7f8bc67fc700 1 --2- 172.21.15.52:0/1238616028 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f8bb0038870 0x7f8bb003ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:17.738 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.731+0000 7f8bc67fc700 1 --2- 172.21.15.52:0/1238616028 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f8bb0038870 0x7f8bb003ad20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc006fd0 tx=0x7f8bbc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:17.741 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.736+0000 7f8baffff700 1 -- 172.21.15.52:0/1238616028 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8bb802a930 con 0x7f8bc8100820 2023-11-27T15:10:17.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.937+0000 7f8bcd54c700 1 -- 172.21.15.52:0/1238616028 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f8bc8060f30 con 0x7f8bb0038870 2023-11-27T15:10:17.945 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.940+0000 7f8baffff700 1 -- 172.21.15.52:0/1238616028 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8bc8060f30 con 0x7f8bb0038870 2023-11-27T15:10:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.942+0000 7f8bcd54c700 1 -- 172.21.15.52:0/1238616028 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f8bb0038870 msgr2=0x7f8bb003ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.942+0000 7f8bcd54c700 1 --2- 172.21.15.52:0/1238616028 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f8bb0038870 0x7f8bb003ad20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc006fd0 tx=0x7f8bbc006e40 comp rx=0 tx=0).stop 2023-11-27T15:10:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.942+0000 7f8bcd54c700 1 -- 172.21.15.52:0/1238616028 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8bc8100820 msgr2=0x7f8bc8194de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.942+0000 7f8bcd54c700 1 --2- 172.21.15.52:0/1238616028 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8bc8100820 0x7f8bc8194de0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8006010 tx=0x7f8bb80057c0 comp rx=0 tx=0).stop 2023-11-27T15:10:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.942+0000 7f8bcd54c700 1 -- 172.21.15.52:0/1238616028 shutdown_connections 2023-11-27T15:10:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.942+0000 7f8bcd54c700 1 --2- 172.21.15.52:0/1238616028 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f8bb0038870 0x7f8bb003ad20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.942+0000 7f8bcd54c700 1 --2- 172.21.15.52:0/1238616028 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8bc8100820 0x7f8bc8194de0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.942+0000 7f8bcd54c700 1 -- 172.21.15.52:0/1238616028 >> 172.21.15.52:0/1238616028 conn(0x7f8bc80fa240 msgr2=0x7f8bc80faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.942+0000 7f8bcd54c700 1 -- 172.21.15.52:0/1238616028 shutdown_connections 2023-11-27T15:10:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:17.942+0000 7f8bcd54c700 1 -- 172.21.15.52:0/1238616028 wait complete. 2023-11-27T15:10:18.358 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi102 2023-11-27T15:10:18.358 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2023-11-27T15:10:18.358 DEBUG:teuthology.orchestra.run.smithi102:> dd of=/etc/ceph/ceph.conf 2023-11-27T15:10:18.379 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2023-11-27T15:10:18.379 DEBUG:teuthology.orchestra.run.smithi102:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:10:18.439 INFO:tasks.cephadm:Adding host smithi102 to orchestrator... 2023-11-27T15:10:18.439 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph orch host add smithi102 2023-11-27T15:10:19.116 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:10:19.349 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:18 smithi052 ceph-mon[99056]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:10:19.349 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:18 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:19.349 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:18 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:10:19.350 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:18 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:10:19.350 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:18 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:10:19.350 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:18 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:10:19.350 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:18 smithi052 ceph-mon[99056]: Updating smithi052:/etc/ceph/ceph.conf 2023-11-27T15:10:19.350 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:18 smithi052 ceph-mon[99056]: Updating smithi052:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:10:19.350 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:18 smithi052 ceph-mon[99056]: Updating smithi052:/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:10:20.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:20.286+0000 7fa65d672700 1 -- 172.21.15.52:0/1785179005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6580fee80 msgr2=0x7fa6580ff250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:20.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:20.286+0000 7fa65d672700 1 --2- 172.21.15.52:0/1785179005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6580fee80 0x7fa6580ff250 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa648009910 tx=0x7fa648009c20 comp rx=0 tx=0).stop 2023-11-27T15:10:20.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:20.286+0000 7fa65d672700 1 -- 172.21.15.52:0/1785179005 shutdown_connections 2023-11-27T15:10:20.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:20.286+0000 7fa65d672700 1 --2- 172.21.15.52:0/1785179005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6580fee80 0x7fa6580ff250 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:20.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:20.286+0000 7fa65d672700 1 -- 172.21.15.52:0/1785179005 >> 172.21.15.52:0/1785179005 conn(0x7fa6580fa760 msgr2=0x7fa6580fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:20.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:20.286+0000 7fa65d672700 1 -- 172.21.15.52:0/1785179005 shutdown_connections 2023-11-27T15:10:20.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:20.286+0000 7fa65d672700 1 -- 172.21.15.52:0/1785179005 wait complete. 2023-11-27T15:10:20.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:20.287+0000 7fa65d672700 1 Processor -- start 2023-11-27T15:10:20.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:20.287+0000 7fa65d672700 1 -- start start 2023-11-27T15:10:20.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:20.287+0000 7fa65d672700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6580fee80 0x7fa6581951a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:20.287+0000 7fa65d672700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6581956e0 con 0x7fa6580fee80 2023-11-27T15:10:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:20.287+0000 7fa656ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6580fee80 0x7fa6581951a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:20.287+0000 7fa656ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6580fee80 0x7fa6581951a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49050/0 (socket says 172.21.15.52:49050) 2023-11-27T15:10:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:20.287+0000 7fa656ffd700 1 -- 172.21.15.52:0/1584033745 learned_addr learned my addr 172.21.15.52:0/1584033745 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:10:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:20.288+0000 7fa656ffd700 1 -- 172.21.15.52:0/1584033745 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6480095f0 con 0x7fa6580fee80 2023-11-27T15:10:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:20.288+0000 7fa656ffd700 1 --2- 172.21.15.52:0/1584033745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6580fee80 0x7fa6581951a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa648004f40 tx=0x7fa648005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:20.288+0000 7fa654ff9700 1 -- 172.21.15.52:0/1584033745 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa64801d070 con 0x7fa6580fee80 2023-11-27T15:10:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:20.288+0000 7fa654ff9700 1 -- 172.21.15.52:0/1584033745 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa648004370 con 0x7fa6580fee80 2023-11-27T15:10:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:20.288+0000 7fa65d672700 1 -- 172.21.15.52:0/1584033745 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa658109060 con 0x7fa6580fee80 2023-11-27T15:10:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:20.288+0000 7fa654ff9700 1 -- 172.21.15.52:0/1584033745 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6480039b0 con 0x7fa6580fee80 2023-11-27T15:10:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:20.288+0000 7fa65d672700 1 -- 172.21.15.52:0/1584033745 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa658108ed0 con 0x7fa6580fee80 2023-11-27T15:10:20.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:20.289+0000 7fa654ff9700 1 -- 172.21.15.52:0/1584033745 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fa648003c20 con 0x7fa6580fee80 2023-11-27T15:10:20.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:20.289+0000 7fa65d672700 1 -- 172.21.15.52:0/1584033745 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6581091f0 con 0x7fa6580fee80 2023-11-27T15:10:20.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:20.289+0000 7fa654ff9700 1 --2- 172.21.15.52:0/1584033745 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fa6440388c0 0x7fa64403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:20.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:20.289+0000 7fa654ff9700 1 -- 172.21.15.52:0/1584033745 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa64804c550 con 0x7fa6580fee80 2023-11-27T15:10:20.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:20.289+0000 7fa64ffff700 1 --2- 172.21.15.52:0/1584033745 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fa6440388c0 0x7fa64403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:20.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:20.290+0000 7fa64ffff700 1 --2- 172.21.15.52:0/1584033745 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fa6440388c0 0x7fa64403ad70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa640006fd0 tx=0x7fa640006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:20.299 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:20.294+0000 7fa654ff9700 1 -- 172.21.15.52:0/1584033745 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa648029950 con 0x7fa6580fee80 2023-11-27T15:10:20.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:20 smithi052 ceph-mon[99056]: Updating smithi052:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-11-27T15:10:20.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:20 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:20.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:20 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:20.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:20 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:20.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:20.495+0000 7fa65d672700 1 -- 172.21.15.52:0/1584033745 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi102", "target": ["mon-mgr", ""]}) v1 -- 0x7fa658195ce0 con 0x7fa6440388c0 2023-11-27T15:10:21.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:21 smithi052 ceph-mon[99056]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi102", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:10:22.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:22 smithi052 ceph-mon[99056]: Deploying cephadm binary to smithi102 2023-11-27T15:10:22.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:22.919+0000 7fa654ff9700 1 -- 172.21.15.52:0/1584033745 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fa658195ce0 con 0x7fa6440388c0 2023-11-27T15:10:22.924 INFO:teuthology.orchestra.run.smithi052.stdout:Added host 'smithi102' with addr '172.21.15.102' 2023-11-27T15:10:22.926 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:22.922+0000 7fa65d672700 1 -- 172.21.15.52:0/1584033745 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fa6440388c0 msgr2=0x7fa64403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:22.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:22.922+0000 7fa65d672700 1 --2- 172.21.15.52:0/1584033745 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fa6440388c0 0x7fa64403ad70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa640006fd0 tx=0x7fa640006e40 comp rx=0 tx=0).stop 2023-11-27T15:10:22.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:22.922+0000 7fa65d672700 1 -- 172.21.15.52:0/1584033745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6580fee80 msgr2=0x7fa6581951a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:22.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:22.922+0000 7fa65d672700 1 --2- 172.21.15.52:0/1584033745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6580fee80 0x7fa6581951a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa648004f40 tx=0x7fa648005020 comp rx=0 tx=0).stop 2023-11-27T15:10:22.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:22.922+0000 7fa65d672700 1 -- 172.21.15.52:0/1584033745 shutdown_connections 2023-11-27T15:10:22.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:22.922+0000 7fa65d672700 1 --2- 172.21.15.52:0/1584033745 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fa6440388c0 0x7fa64403ad70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:22.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:22.922+0000 7fa65d672700 1 --2- 172.21.15.52:0/1584033745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6580fee80 0x7fa6581951a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:22.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:22.922+0000 7fa65d672700 1 -- 172.21.15.52:0/1584033745 >> 172.21.15.52:0/1584033745 conn(0x7fa6580fa760 msgr2=0x7fa658068720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:22.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:22.922+0000 7fa65d672700 1 -- 172.21.15.52:0/1584033745 shutdown_connections 2023-11-27T15:10:22.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:22.922+0000 7fa65d672700 1 -- 172.21.15.52:0/1584033745 wait complete. 2023-11-27T15:10:23.286 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph orch host ls --format=json 2023-11-27T15:10:23.947 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:10:24.175 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:23 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:24.175 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:23 smithi052 ceph-mon[99056]: Added host smithi102 2023-11-27T15:10:24.175 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:23 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:10:24.175 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:23 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:24.175 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:23 smithi052 ceph-mon[99056]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:25.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.155+0000 7fd5318cf700 1 -- 172.21.15.52:0/4214677384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd52c100ef0 msgr2=0x7fd52c1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:25.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.155+0000 7fd5318cf700 1 --2- 172.21.15.52:0/4214677384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd52c100ef0 0x7fd52c1012c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd51c009910 tx=0x7fd51c009c20 comp rx=0 tx=0).stop 2023-11-27T15:10:25.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.155+0000 7fd5318cf700 1 -- 172.21.15.52:0/4214677384 shutdown_connections 2023-11-27T15:10:25.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.155+0000 7fd5318cf700 1 --2- 172.21.15.52:0/4214677384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd52c100ef0 0x7fd52c1012c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:25.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.155+0000 7fd5318cf700 1 -- 172.21.15.52:0/4214677384 >> 172.21.15.52:0/4214677384 conn(0x7fd52c0fa730 msgr2=0x7fd52c0fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:25.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.155+0000 7fd5318cf700 1 -- 172.21.15.52:0/4214677384 shutdown_connections 2023-11-27T15:10:25.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.155+0000 7fd5318cf700 1 -- 172.21.15.52:0/4214677384 wait complete. 2023-11-27T15:10:25.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.156+0000 7fd5318cf700 1 Processor -- start 2023-11-27T15:10:25.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.156+0000 7fd5318cf700 1 -- start start 2023-11-27T15:10:25.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.157+0000 7fd5318cf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd52c100ef0 0x7fd52c0feb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:25.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.157+0000 7fd5318cf700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c100b20 con 0x7fd52c100ef0 2023-11-27T15:10:25.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.157+0000 7fd52affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd52c100ef0 0x7fd52c0feb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:25.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.157+0000 7fd52affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd52c100ef0 0x7fd52c0feb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49076/0 (socket says 172.21.15.52:49076) 2023-11-27T15:10:25.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.157+0000 7fd52affd700 1 -- 172.21.15.52:0/3571305029 learned_addr learned my addr 172.21.15.52:0/3571305029 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:10:25.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.157+0000 7fd52affd700 1 -- 172.21.15.52:0/3571305029 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51c0095f0 con 0x7fd52c100ef0 2023-11-27T15:10:25.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.157+0000 7fd52affd700 1 --2- 172.21.15.52:0/3571305029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd52c100ef0 0x7fd52c0feb50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd51c004f40 tx=0x7fd51c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:25.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.157+0000 7fd5308cd700 1 -- 172.21.15.52:0/3571305029 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd51c01d070 con 0x7fd52c100ef0 2023-11-27T15:10:25.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.157+0000 7fd5308cd700 1 -- 172.21.15.52:0/3571305029 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd51c004370 con 0x7fd52c100ef0 2023-11-27T15:10:25.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.157+0000 7fd5318cf700 1 -- 172.21.15.52:0/3571305029 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd52c10a180 con 0x7fd52c100ef0 2023-11-27T15:10:25.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.157+0000 7fd5308cd700 1 -- 172.21.15.52:0/3571305029 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd51c0039b0 con 0x7fd52c100ef0 2023-11-27T15:10:25.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.157+0000 7fd5318cf700 1 -- 172.21.15.52:0/3571305029 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd52c109ff0 con 0x7fd52c100ef0 2023-11-27T15:10:25.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.158+0000 7fd5318cf700 1 -- 172.21.15.52:0/3571305029 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd52c10a310 con 0x7fd52c100ef0 2023-11-27T15:10:25.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.159+0000 7fd5308cd700 1 -- 172.21.15.52:0/3571305029 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fd51c003c70 con 0x7fd52c100ef0 2023-11-27T15:10:25.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.159+0000 7fd5308cd700 1 --2- 172.21.15.52:0/3571305029 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd514038870 0x7fd51403ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:25.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.159+0000 7fd5308cd700 1 -- 172.21.15.52:0/3571305029 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd51c04c930 con 0x7fd52c100ef0 2023-11-27T15:10:25.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.159+0000 7fd52a7fc700 1 --2- 172.21.15.52:0/3571305029 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd514038870 0x7fd51403ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:25.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.159+0000 7fd52a7fc700 1 --2- 172.21.15.52:0/3571305029 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd514038870 0x7fd51403ad20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd520006fd0 tx=0x7fd520006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:25.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.163+0000 7fd5308cd700 1 -- 172.21.15.52:0/3571305029 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd51c02a980 con 0x7fd52c100ef0 2023-11-27T15:10:25.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:25 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:25.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.366+0000 7fd5318cf700 1 -- 172.21.15.52:0/3571305029 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd52c100080 con 0x7fd514038870 2023-11-27T15:10:25.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.367+0000 7fd5308cd700 1 -- 172.21.15.52:0/3571305029 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7fd52c100080 con 0x7fd514038870 2023-11-27T15:10:25.372 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:10:25.372 INFO:teuthology.orchestra.run.smithi052.stdout:[{"addr": "172.21.15.52", "hostname": "smithi052", "labels": [], "status": ""}, {"addr": "172.21.15.102", "hostname": "smithi102", "labels": [], "status": ""}] 2023-11-27T15:10:25.374 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.369+0000 7fd5318cf700 1 -- 172.21.15.52:0/3571305029 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd514038870 msgr2=0x7fd51403ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:25.374 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.369+0000 7fd5318cf700 1 --2- 172.21.15.52:0/3571305029 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd514038870 0x7fd51403ad20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd520006fd0 tx=0x7fd520006e40 comp rx=0 tx=0).stop 2023-11-27T15:10:25.374 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.369+0000 7fd5318cf700 1 -- 172.21.15.52:0/3571305029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd52c100ef0 msgr2=0x7fd52c0feb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:25.374 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.369+0000 7fd5318cf700 1 --2- 172.21.15.52:0/3571305029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd52c100ef0 0x7fd52c0feb50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd51c004f40 tx=0x7fd51c005020 comp rx=0 tx=0).stop 2023-11-27T15:10:25.374 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.369+0000 7fd5318cf700 1 -- 172.21.15.52:0/3571305029 shutdown_connections 2023-11-27T15:10:25.374 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.369+0000 7fd5318cf700 1 --2- 172.21.15.52:0/3571305029 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd514038870 0x7fd51403ad20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:25.374 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.369+0000 7fd5318cf700 1 --2- 172.21.15.52:0/3571305029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd52c100ef0 0x7fd52c0feb50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:25.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.369+0000 7fd5318cf700 1 -- 172.21.15.52:0/3571305029 >> 172.21.15.52:0/3571305029 conn(0x7fd52c0fa730 msgr2=0x7fd52c104540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:25.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.369+0000 7fd5318cf700 1 -- 172.21.15.52:0/3571305029 shutdown_connections 2023-11-27T15:10:25.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:25.369+0000 7fd5318cf700 1 -- 172.21.15.52:0/3571305029 wait complete. 2023-11-27T15:10:25.807 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi150 2023-11-27T15:10:25.808 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-27T15:10:25.808 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/etc/ceph/ceph.conf 2023-11-27T15:10:25.829 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-27T15:10:25.829 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:10:25.888 INFO:tasks.cephadm:Adding host smithi150 to orchestrator... 2023-11-27T15:10:25.888 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph orch host add smithi150 2023-11-27T15:10:26.545 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:10:26.741 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:26 smithi052 ceph-mon[99056]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:10:26.741 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:26 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:26.741 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:26 smithi052 ceph-mon[99056]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:27.708 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:27.703+0000 7fc556488700 1 -- 172.21.15.52:0/296922406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5500ffe80 msgr2=0x7fc550100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:27.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:27.703+0000 7fc556488700 1 --2- 172.21.15.52:0/296922406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5500ffe80 0x7fc550100250 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc540009910 tx=0x7fc540009c20 comp rx=0 tx=0).stop 2023-11-27T15:10:27.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:27.703+0000 7fc556488700 1 -- 172.21.15.52:0/296922406 shutdown_connections 2023-11-27T15:10:27.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:27.703+0000 7fc556488700 1 --2- 172.21.15.52:0/296922406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5500ffe80 0x7fc550100250 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:27.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:27.703+0000 7fc556488700 1 -- 172.21.15.52:0/296922406 >> 172.21.15.52:0/296922406 conn(0x7fc550074350 msgr2=0x7fc550074750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:27.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:27.703+0000 7fc556488700 1 -- 172.21.15.52:0/296922406 shutdown_connections 2023-11-27T15:10:27.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:27.703+0000 7fc556488700 1 -- 172.21.15.52:0/296922406 wait complete. 2023-11-27T15:10:27.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:27.704+0000 7fc556488700 1 Processor -- start 2023-11-27T15:10:27.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:27.704+0000 7fc556488700 1 -- start start 2023-11-27T15:10:27.711 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:27.704+0000 7fc556488700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5500ffe80 0x7fc5501960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:27.711 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:27.704+0000 7fc556488700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc550196620 con 0x7fc5500ffe80 2023-11-27T15:10:27.711 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:27.704+0000 7fc54ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5500ffe80 0x7fc5501960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:27.711 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:27.704+0000 7fc54ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5500ffe80 0x7fc5501960e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49082/0 (socket says 172.21.15.52:49082) 2023-11-27T15:10:27.711 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:27.704+0000 7fc54ffff700 1 -- 172.21.15.52:0/3691358044 learned_addr learned my addr 172.21.15.52:0/3691358044 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:10:27.711 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:27.705+0000 7fc54ffff700 1 -- 172.21.15.52:0/3691358044 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5400095f0 con 0x7fc5500ffe80 2023-11-27T15:10:27.712 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:27.705+0000 7fc54ffff700 1 --2- 172.21.15.52:0/3691358044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5500ffe80 0x7fc5501960e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc540005850 tx=0x7fc540004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:27.712 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:27.705+0000 7fc54d7fa700 1 -- 172.21.15.52:0/3691358044 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc54001d070 con 0x7fc5500ffe80 2023-11-27T15:10:27.712 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:27.705+0000 7fc54d7fa700 1 -- 172.21.15.52:0/3691358044 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc540004370 con 0x7fc5500ffe80 2023-11-27T15:10:27.712 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:27.705+0000 7fc556488700 1 -- 172.21.15.52:0/3691358044 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5501091a0 con 0x7fc5500ffe80 2023-11-27T15:10:27.712 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:27.705+0000 7fc54d7fa700 1 -- 172.21.15.52:0/3691358044 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc540003ab0 con 0x7fc5500ffe80 2023-11-27T15:10:27.712 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:27.705+0000 7fc556488700 1 -- 172.21.15.52:0/3691358044 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc550109010 con 0x7fc5500ffe80 2023-11-27T15:10:27.712 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:27.706+0000 7fc54d7fa700 1 -- 172.21.15.52:0/3691358044 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fc540003d20 con 0x7fc5500ffe80 2023-11-27T15:10:27.712 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:27.706+0000 7fc54d7fa700 1 --2- 172.21.15.52:0/3691358044 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fc5380388c0 0x7fc53803ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:27.712 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:27.706+0000 7fc54d7fa700 1 -- 172.21.15.52:0/3691358044 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc54004d430 con 0x7fc5500ffe80 2023-11-27T15:10:27.712 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:27.706+0000 7fc54f7fe700 1 --2- 172.21.15.52:0/3691358044 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fc5380388c0 0x7fc53803ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:27.712 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:27.706+0000 7fc556488700 1 -- 172.21.15.52:0/3691358044 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc550109330 con 0x7fc5500ffe80 2023-11-27T15:10:27.713 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:27.707+0000 7fc54f7fe700 1 --2- 172.21.15.52:0/3691358044 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fc5380388c0 0x7fc53803ad70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc544006fd0 tx=0x7fc544006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:27.716 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:27.711+0000 7fc54d7fa700 1 -- 172.21.15.52:0/3691358044 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc540026070 con 0x7fc5500ffe80 2023-11-27T15:10:27.918 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:27.913+0000 7fc556488700 1 -- 172.21.15.52:0/3691358044 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi150", "target": ["mon-mgr", ""]}) v1 -- 0x7fc550062190 con 0x7fc5380388c0 2023-11-27T15:10:28.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:27 smithi052 ceph-mon[99056]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:29.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:28 smithi052 ceph-mon[99056]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi150", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:10:29.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:28 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:29.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:28 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:29.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:28 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:29.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:28 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:29.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:28 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi102", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:10:29.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:28 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:10:29.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:28 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:10:29.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:28 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:10:29.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:28 smithi052 ceph-mon[99056]: Updating smithi102:/etc/ceph/ceph.conf 2023-11-27T15:10:29.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:28 smithi052 ceph-mon[99056]: Updating smithi102:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:10:29.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:28 smithi052 ceph-mon[99056]: Deploying cephadm binary to smithi150 2023-11-27T15:10:29.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:28 smithi052 ceph-mon[99056]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:10:30.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:30 smithi052 ceph-mon[99056]: Updating smithi102:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-11-27T15:10:30.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:30 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:30.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:30 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:30.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:30 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:30.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:30 smithi052 ceph-mon[99056]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:30.389 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:30.383+0000 7fc54d7fa700 1 -- 172.21.15.52:0/3691358044 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fc550062190 con 0x7fc5380388c0 2023-11-27T15:10:30.389 INFO:teuthology.orchestra.run.smithi052.stdout:Added host 'smithi150' with addr '172.21.15.150' 2023-11-27T15:10:30.391 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:30.386+0000 7fc556488700 1 -- 172.21.15.52:0/3691358044 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fc5380388c0 msgr2=0x7fc53803ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:30.392 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:30.386+0000 7fc556488700 1 --2- 172.21.15.52:0/3691358044 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fc5380388c0 0x7fc53803ad70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc544006fd0 tx=0x7fc544006e40 comp rx=0 tx=0).stop 2023-11-27T15:10:30.392 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:30.386+0000 7fc556488700 1 -- 172.21.15.52:0/3691358044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5500ffe80 msgr2=0x7fc5501960e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:30.392 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:30.386+0000 7fc556488700 1 --2- 172.21.15.52:0/3691358044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5500ffe80 0x7fc5501960e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc540005850 tx=0x7fc540004ff0 comp rx=0 tx=0).stop 2023-11-27T15:10:30.392 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:30.386+0000 7fc556488700 1 -- 172.21.15.52:0/3691358044 shutdown_connections 2023-11-27T15:10:30.392 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:30.386+0000 7fc556488700 1 --2- 172.21.15.52:0/3691358044 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fc5380388c0 0x7fc53803ad70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:30.392 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:30.386+0000 7fc556488700 1 --2- 172.21.15.52:0/3691358044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5500ffe80 0x7fc5501960e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:30.392 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:30.386+0000 7fc556488700 1 -- 172.21.15.52:0/3691358044 >> 172.21.15.52:0/3691358044 conn(0x7fc550074350 msgr2=0x7fc550103730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:30.392 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:30.386+0000 7fc556488700 1 -- 172.21.15.52:0/3691358044 shutdown_connections 2023-11-27T15:10:30.392 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:30.386+0000 7fc556488700 1 -- 172.21.15.52:0/3691358044 wait complete. 2023-11-27T15:10:30.702 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph orch host ls --format=json 2023-11-27T15:10:31.370 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:10:31.729 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:31 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:31.729 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:31 smithi052 ceph-mon[99056]: Added host smithi150 2023-11-27T15:10:31.729 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:31 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:10:31.730 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:31 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:32.534 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.527+0000 7f79b9e2d700 1 -- 172.21.15.52:0/1280446459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79b4100df0 msgr2=0x7f79b41011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:32.534 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.527+0000 7f79b9e2d700 1 --2- 172.21.15.52:0/1280446459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79b4100df0 0x7f79b41011c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f79a4009910 tx=0x7f79a4009c20 comp rx=0 tx=0).stop 2023-11-27T15:10:32.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.527+0000 7f79b9e2d700 1 -- 172.21.15.52:0/1280446459 shutdown_connections 2023-11-27T15:10:32.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.527+0000 7f79b9e2d700 1 --2- 172.21.15.52:0/1280446459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79b4100df0 0x7f79b41011c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:32.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.527+0000 7f79b9e2d700 1 -- 172.21.15.52:0/1280446459 >> 172.21.15.52:0/1280446459 conn(0x7f79b4074930 msgr2=0x7f79b4074d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:32.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.527+0000 7f79b9e2d700 1 -- 172.21.15.52:0/1280446459 shutdown_connections 2023-11-27T15:10:32.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.527+0000 7f79b9e2d700 1 -- 172.21.15.52:0/1280446459 wait complete. 2023-11-27T15:10:32.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.527+0000 7f79b9e2d700 1 Processor -- start 2023-11-27T15:10:32.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.528+0000 7f79b9e2d700 1 -- start start 2023-11-27T15:10:32.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.528+0000 7f79b9e2d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79b4100df0 0x7f79b40fec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:32.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.528+0000 7f79b9e2d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b4100ba0 con 0x7f79b4100df0 2023-11-27T15:10:32.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.528+0000 7f79b37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79b4100df0 0x7f79b40fec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:32.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.528+0000 7f79b37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79b4100df0 0x7f79b40fec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49108/0 (socket says 172.21.15.52:49108) 2023-11-27T15:10:32.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.528+0000 7f79b37fe700 1 -- 172.21.15.52:0/1444295441 learned_addr learned my addr 172.21.15.52:0/1444295441 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:10:32.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.528+0000 7f79b37fe700 1 -- 172.21.15.52:0/1444295441 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a40095f0 con 0x7f79b4100df0 2023-11-27T15:10:32.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.528+0000 7f79b37fe700 1 --2- 172.21.15.52:0/1444295441 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79b4100df0 0x7f79b40fec20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f79a4006010 tx=0x7f79a4004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:32.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.528+0000 7f79b0ff9700 1 -- 172.21.15.52:0/1444295441 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79a401d070 con 0x7f79b4100df0 2023-11-27T15:10:32.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.528+0000 7f79b0ff9700 1 -- 172.21.15.52:0/1444295441 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79a40044a0 con 0x7f79b4100df0 2023-11-27T15:10:32.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.528+0000 7f79b9e2d700 1 -- 172.21.15.52:0/1444295441 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b410a160 con 0x7f79b4100df0 2023-11-27T15:10:32.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.528+0000 7f79b0ff9700 1 -- 172.21.15.52:0/1444295441 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79a4003a70 con 0x7f79b4100df0 2023-11-27T15:10:32.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.529+0000 7f79b9e2d700 1 -- 172.21.15.52:0/1444295441 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b4109fd0 con 0x7f79b4100df0 2023-11-27T15:10:32.537 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.529+0000 7f79b0ff9700 1 -- 172.21.15.52:0/1444295441 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f79a4003cb0 con 0x7f79b4100df0 2023-11-27T15:10:32.537 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.529+0000 7f79b9e2d700 1 -- 172.21.15.52:0/1444295441 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b410a2f0 con 0x7f79b4100df0 2023-11-27T15:10:32.537 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.530+0000 7f79b0ff9700 1 --2- 172.21.15.52:0/1444295441 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f799c038870 0x7f799c03ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:32.537 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.530+0000 7f79b0ff9700 1 -- 172.21.15.52:0/1444295441 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f79a404c590 con 0x7f79b4100df0 2023-11-27T15:10:32.537 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.530+0000 7f79b2ffd700 1 --2- 172.21.15.52:0/1444295441 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f799c038870 0x7f799c03ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:32.538 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.530+0000 7f79b2ffd700 1 --2- 172.21.15.52:0/1444295441 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f799c038870 0x7f799c03ad20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f79a8006fd0 tx=0x7f79a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:32.540 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.534+0000 7f79b0ff9700 1 -- 172.21.15.52:0/1444295441 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f79a4029950 con 0x7f79b4100df0 2023-11-27T15:10:32.746 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.740+0000 7f79b9e2d700 1 -- 172.21.15.52:0/1444295441 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f79b4100150 con 0x7f799c038870 2023-11-27T15:10:32.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.741+0000 7f79b0ff9700 1 -- 172.21.15.52:0/1444295441 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7f79b4100150 con 0x7f799c038870 2023-11-27T15:10:32.747 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:10:32.747 INFO:teuthology.orchestra.run.smithi052.stdout:[{"addr": "172.21.15.52", "hostname": "smithi052", "labels": [], "status": ""}, {"addr": "172.21.15.102", "hostname": "smithi102", "labels": [], "status": ""}, {"addr": "172.21.15.150", "hostname": "smithi150", "labels": [], "status": ""}] 2023-11-27T15:10:32.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.743+0000 7f79b9e2d700 1 -- 172.21.15.52:0/1444295441 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f799c038870 msgr2=0x7f799c03ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:32.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.743+0000 7f79b9e2d700 1 --2- 172.21.15.52:0/1444295441 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f799c038870 0x7f799c03ad20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f79a8006fd0 tx=0x7f79a8006e40 comp rx=0 tx=0).stop 2023-11-27T15:10:32.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.743+0000 7f79b9e2d700 1 -- 172.21.15.52:0/1444295441 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79b4100df0 msgr2=0x7f79b40fec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:32.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.743+0000 7f79b9e2d700 1 --2- 172.21.15.52:0/1444295441 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79b4100df0 0x7f79b40fec20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f79a4006010 tx=0x7f79a4004ff0 comp rx=0 tx=0).stop 2023-11-27T15:10:32.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.743+0000 7f79b9e2d700 1 -- 172.21.15.52:0/1444295441 shutdown_connections 2023-11-27T15:10:32.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.743+0000 7f79b9e2d700 1 --2- 172.21.15.52:0/1444295441 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f799c038870 0x7f799c03ad20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:32.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.743+0000 7f79b9e2d700 1 --2- 172.21.15.52:0/1444295441 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79b4100df0 0x7f79b40fec20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:32.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.743+0000 7f79b9e2d700 1 -- 172.21.15.52:0/1444295441 >> 172.21.15.52:0/1444295441 conn(0x7f79b4074930 msgr2=0x7f79b4104610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:32.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.744+0000 7f79b9e2d700 1 -- 172.21.15.52:0/1444295441 shutdown_connections 2023-11-27T15:10:32.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:32.744+0000 7f79b9e2d700 1 -- 172.21.15.52:0/1444295441 wait complete. 2023-11-27T15:10:33.078 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:32 smithi052 ceph-mon[99056]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:33.079 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:32 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:33.119 INFO:tasks.cephadm:Setting crush tunables to default 2023-11-27T15:10:33.119 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph osd crush tunables default 2023-11-27T15:10:33.773 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:10:34.216 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:33 smithi052 ceph-mon[99056]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:10:34.216 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:33 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:34.216 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:33 smithi052 ceph-mon[99056]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:34.983 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:34.977+0000 7f3e6ab00700 1 -- 172.21.15.52:0/156118294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e640fef00 msgr2=0x7f3e640ff2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:34.983 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:34.977+0000 7f3e6ab00700 1 --2- 172.21.15.52:0/156118294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e640fef00 0x7f3e640ff2d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3e58009910 tx=0x7f3e58009c20 comp rx=0 tx=0).stop 2023-11-27T15:10:34.983 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:34.977+0000 7f3e6ab00700 1 -- 172.21.15.52:0/156118294 shutdown_connections 2023-11-27T15:10:34.983 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:34.977+0000 7f3e6ab00700 1 --2- 172.21.15.52:0/156118294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e640fef00 0x7f3e640ff2d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:34.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:34.977+0000 7f3e6ab00700 1 -- 172.21.15.52:0/156118294 >> 172.21.15.52:0/156118294 conn(0x7f3e64074930 msgr2=0x7f3e64074d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:34.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:34.977+0000 7f3e6ab00700 1 -- 172.21.15.52:0/156118294 shutdown_connections 2023-11-27T15:10:34.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:34.977+0000 7f3e6ab00700 1 -- 172.21.15.52:0/156118294 wait complete. 2023-11-27T15:10:34.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:34.977+0000 7f3e6ab00700 1 Processor -- start 2023-11-27T15:10:34.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:34.977+0000 7f3e6ab00700 1 -- start start 2023-11-27T15:10:34.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:34.978+0000 7f3e6ab00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e640fef00 0x7f3e64196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:34.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:34.978+0000 7f3e6ab00700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e64196550 con 0x7f3e640fef00 2023-11-27T15:10:34.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:34.978+0000 7f3e6889c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e640fef00 0x7f3e64196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:34.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:34.978+0000 7f3e6889c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e640fef00 0x7f3e64196010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49114/0 (socket says 172.21.15.52:49114) 2023-11-27T15:10:34.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:34.978+0000 7f3e6889c700 1 -- 172.21.15.52:0/2229004337 learned_addr learned my addr 172.21.15.52:0/2229004337 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:10:34.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:34.978+0000 7f3e6889c700 1 -- 172.21.15.52:0/2229004337 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e580095f0 con 0x7f3e640fef00 2023-11-27T15:10:34.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:34.978+0000 7f3e6889c700 1 --2- 172.21.15.52:0/2229004337 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e640fef00 0x7f3e64196010 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3e58004f40 tx=0x7f3e58005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:34.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:34.978+0000 7f3e61ffb700 1 -- 172.21.15.52:0/2229004337 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e5801d070 con 0x7f3e640fef00 2023-11-27T15:10:34.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:34.978+0000 7f3e61ffb700 1 -- 172.21.15.52:0/2229004337 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e58004370 con 0x7f3e640fef00 2023-11-27T15:10:34.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:34.978+0000 7f3e61ffb700 1 -- 172.21.15.52:0/2229004337 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e580039b0 con 0x7f3e640fef00 2023-11-27T15:10:34.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:34.978+0000 7f3e6ab00700 1 -- 172.21.15.52:0/2229004337 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e64109090 con 0x7f3e640fef00 2023-11-27T15:10:34.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:34.979+0000 7f3e6ab00700 1 -- 172.21.15.52:0/2229004337 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e64108f00 con 0x7f3e640fef00 2023-11-27T15:10:34.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:34.980+0000 7f3e6ab00700 1 -- 172.21.15.52:0/2229004337 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e64109220 con 0x7f3e640fef00 2023-11-27T15:10:34.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:34.980+0000 7f3e61ffb700 1 -- 172.21.15.52:0/2229004337 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f3e58003c20 con 0x7f3e640fef00 2023-11-27T15:10:34.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:34.980+0000 7f3e61ffb700 1 --2- 172.21.15.52:0/2229004337 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f3e540388c0 0x7f3e5403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:34.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:34.980+0000 7f3e61ffb700 1 -- 172.21.15.52:0/2229004337 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3e5804c8f0 con 0x7f3e640fef00 2023-11-27T15:10:34.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:34.980+0000 7f3e63fff700 1 --2- 172.21.15.52:0/2229004337 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f3e540388c0 0x7f3e5403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:34.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:34.981+0000 7f3e63fff700 1 --2- 172.21.15.52:0/2229004337 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f3e540388c0 0x7f3e5403ad70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3e50006fd0 tx=0x7f3e50006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:34.990 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:34.985+0000 7f3e61ffb700 1 -- 172.21.15.52:0/2229004337 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3e5802a980 con 0x7f3e640fef00 2023-11-27T15:10:35.196 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:35.190+0000 7f3e6ab00700 1 -- 172.21.15.52:0/2229004337 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f3e64068300 con 0x7f3e640fef00 2023-11-27T15:10:35.249 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:35.243+0000 7f3e61ffb700 1 -- 172.21.15.52:0/2229004337 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3e58026020 con 0x7f3e640fef00 2023-11-27T15:10:35.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:35.245+0000 7f3e6ab00700 1 -- 172.21.15.52:0/2229004337 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f3e540388c0 msgr2=0x7f3e5403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:35.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:35.246+0000 7f3e6ab00700 1 --2- 172.21.15.52:0/2229004337 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f3e540388c0 0x7f3e5403ad70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3e50006fd0 tx=0x7f3e50006e40 comp rx=0 tx=0).stop 2023-11-27T15:10:35.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:35.246+0000 7f3e6ab00700 1 -- 172.21.15.52:0/2229004337 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e640fef00 msgr2=0x7f3e64196010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:35.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:35.246+0000 7f3e6ab00700 1 --2- 172.21.15.52:0/2229004337 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e640fef00 0x7f3e64196010 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3e58004f40 tx=0x7f3e58005020 comp rx=0 tx=0).stop 2023-11-27T15:10:35.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:35.246+0000 7f3e6ab00700 1 -- 172.21.15.52:0/2229004337 shutdown_connections 2023-11-27T15:10:35.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:35.246+0000 7f3e6ab00700 1 --2- 172.21.15.52:0/2229004337 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f3e540388c0 0x7f3e5403ad70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:35.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:35.246+0000 7f3e6ab00700 1 --2- 172.21.15.52:0/2229004337 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e640fef00 0x7f3e64196010 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:35.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:35.246+0000 7f3e6ab00700 1 -- 172.21.15.52:0/2229004337 >> 172.21.15.52:0/2229004337 conn(0x7f3e64074930 msgr2=0x7f3e64103090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:35.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:35.246+0000 7f3e6ab00700 1 -- 172.21.15.52:0/2229004337 shutdown_connections 2023-11-27T15:10:35.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:35.246+0000 7f3e6ab00700 1 -- 172.21.15.52:0/2229004337 wait complete. 2023-11-27T15:10:35.253 INFO:teuthology.orchestra.run.smithi052.stderr:adjusted tunables profile to default 2023-11-27T15:10:35.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:35 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2229004337' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-11-27T15:10:35.623 INFO:tasks.cephadm:Adding mon.a on smithi052 2023-11-27T15:10:35.623 INFO:tasks.cephadm:Adding mon.b on smithi102 2023-11-27T15:10:35.623 INFO:tasks.cephadm:Adding mon.c on smithi150 2023-11-27T15:10:35.623 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph orch apply mon '3;smithi052:172.21.15.52=a;smithi102:172.21.15.102=b;smithi150:172.21.15.150=c' 2023-11-27T15:10:36.315 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:10:36.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:36 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2229004337' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-11-27T15:10:36.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:36 smithi052 ceph-mon[99056]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:10:36.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:36 smithi052 ceph-mon[99056]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:36.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:36 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:36.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:36 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:36.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:36 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:36.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:36 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:36.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:36 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:10:36.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:36 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:10:36.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:36 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:10:36.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:36 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:10:36.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:36 smithi052 ceph-mon[99056]: Updating smithi150:/etc/ceph/ceph.conf 2023-11-27T15:10:37.548 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.542+0000 7efffd755700 1 -- 172.21.15.150:0/3975848426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efff80fee90 msgr2=0x7efff80ff260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:37.548 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.542+0000 7efffd755700 1 --2- 172.21.15.150:0/3975848426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efff80fee90 0x7efff80ff260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7effe8009910 tx=0x7effe8009c20 comp rx=0 tx=0).stop 2023-11-27T15:10:37.548 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.542+0000 7efffd755700 1 -- 172.21.15.150:0/3975848426 shutdown_connections 2023-11-27T15:10:37.548 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.542+0000 7efffd755700 1 --2- 172.21.15.150:0/3975848426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efff80fee90 0x7efff80ff260 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:37.548 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.542+0000 7efffd755700 1 -- 172.21.15.150:0/3975848426 >> 172.21.15.150:0/3975848426 conn(0x7efff80fa750 msgr2=0x7efff80fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:37.548 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.542+0000 7efffd755700 1 -- 172.21.15.150:0/3975848426 shutdown_connections 2023-11-27T15:10:37.549 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.542+0000 7efffd755700 1 -- 172.21.15.150:0/3975848426 wait complete. 2023-11-27T15:10:37.551 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.543+0000 7efffd755700 1 Processor -- start 2023-11-27T15:10:37.551 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.543+0000 7efffd755700 1 -- start start 2023-11-27T15:10:37.551 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.543+0000 7efffd755700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efff80fee90 0x7efff8195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:37.551 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.544+0000 7efffd755700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff8196530 con 0x7efff80fee90 2023-11-27T15:10:37.551 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.544+0000 7efff6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efff80fee90 0x7efff8195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:37.552 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.544+0000 7efff6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efff80fee90 0x7efff8195ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.150:58216/0 (socket says 172.21.15.150:58216) 2023-11-27T15:10:37.552 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.544+0000 7efff6ffd700 1 -- 172.21.15.150:0/3944198533 learned_addr learned my addr 172.21.15.150:0/3944198533 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-27T15:10:37.552 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.544+0000 7efff6ffd700 1 -- 172.21.15.150:0/3944198533 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe80095f0 con 0x7efff80fee90 2023-11-27T15:10:37.552 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.544+0000 7efff6ffd700 1 --2- 172.21.15.150:0/3944198533 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efff80fee90 0x7efff8195ff0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7effe8004f40 tx=0x7effe8005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:37.552 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.544+0000 7effdffff700 1 -- 172.21.15.150:0/3944198533 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effe801d070 con 0x7efff80fee90 2023-11-27T15:10:37.552 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.544+0000 7efffd755700 1 -- 172.21.15.150:0/3944198533 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efff8109070 con 0x7efff80fee90 2023-11-27T15:10:37.552 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.545+0000 7effdffff700 1 -- 172.21.15.150:0/3944198533 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effe8004370 con 0x7efff80fee90 2023-11-27T15:10:37.552 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.545+0000 7effdffff700 1 -- 172.21.15.150:0/3944198533 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effe80039b0 con 0x7efff80fee90 2023-11-27T15:10:37.552 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.545+0000 7efffd755700 1 -- 172.21.15.150:0/3944198533 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff8108ee0 con 0x7efff80fee90 2023-11-27T15:10:37.552 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.545+0000 7efffd755700 1 -- 172.21.15.150:0/3944198533 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efff8109200 con 0x7efff80fee90 2023-11-27T15:10:37.552 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.546+0000 7effdffff700 1 -- 172.21.15.150:0/3944198533 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7effe8003c70 con 0x7efff80fee90 2023-11-27T15:10:37.552 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.546+0000 7effdffff700 1 --2- 172.21.15.150:0/3944198533 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7effe0038870 0x7effe003ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:37.553 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.546+0000 7effdffff700 1 -- 172.21.15.150:0/3944198533 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7effe804c8a0 con 0x7efff80fee90 2023-11-27T15:10:37.553 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.546+0000 7efff67fc700 1 --2- 172.21.15.150:0/3944198533 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7effe0038870 0x7effe003ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:37.553 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.547+0000 7efff67fc700 1 --2- 172.21.15.150:0/3944198533 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7effe0038870 0x7effe003ad20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7effec006fd0 tx=0x7effec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:37.556 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.551+0000 7effdffff700 1 -- 172.21.15.150:0/3944198533 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7effe8029360 con 0x7efff80fee90 2023-11-27T15:10:37.750 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.745+0000 7efffd755700 1 -- 172.21.15.150:0/3944198533 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi052:172.21.15.52=a;smithi102:172.21.15.102=b;smithi150:172.21.15.150=c", "target": ["mon-mgr", ""]}) v1 -- 0x7efff8068e70 con 0x7effe0038870 2023-11-27T15:10:37.757 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.752+0000 7effdffff700 1 -- 172.21.15.150:0/3944198533 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7efff8068e70 con 0x7effe0038870 2023-11-27T15:10:37.757 INFO:teuthology.orchestra.run.smithi150.stdout:Scheduled mon update... 2023-11-27T15:10:37.759 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.753+0000 7efffd755700 1 -- 172.21.15.150:0/3944198533 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7effe0038870 msgr2=0x7effe003ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:37.759 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.753+0000 7efffd755700 1 --2- 172.21.15.150:0/3944198533 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7effe0038870 0x7effe003ad20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7effec006fd0 tx=0x7effec006e40 comp rx=0 tx=0).stop 2023-11-27T15:10:37.759 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.753+0000 7efffd755700 1 -- 172.21.15.150:0/3944198533 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efff80fee90 msgr2=0x7efff8195ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:37.759 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.753+0000 7efffd755700 1 --2- 172.21.15.150:0/3944198533 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efff80fee90 0x7efff8195ff0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7effe8004f40 tx=0x7effe8005020 comp rx=0 tx=0).stop 2023-11-27T15:10:37.759 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.753+0000 7efffd755700 1 -- 172.21.15.150:0/3944198533 shutdown_connections 2023-11-27T15:10:37.760 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.753+0000 7efffd755700 1 --2- 172.21.15.150:0/3944198533 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7effe0038870 0x7effe003ad20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:37.760 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.753+0000 7efffd755700 1 --2- 172.21.15.150:0/3944198533 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efff80fee90 0x7efff8195ff0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:37.760 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.753+0000 7efffd755700 1 -- 172.21.15.150:0/3944198533 >> 172.21.15.150:0/3944198533 conn(0x7efff80fa750 msgr2=0x7efff8103070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:37.760 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.754+0000 7efffd755700 1 -- 172.21.15.150:0/3944198533 shutdown_connections 2023-11-27T15:10:37.760 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:37.754+0000 7efffd755700 1 -- 172.21.15.150:0/3944198533 wait complete. 2023-11-27T15:10:37.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:37 smithi052 ceph-mon[99056]: Updating smithi150:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:10:37.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:37 smithi052 ceph-mon[99056]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:10:37.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:37 smithi052 ceph-mon[99056]: Updating smithi150:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-11-27T15:10:37.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:37 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:37.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:37 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:37.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:37 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:38.178 DEBUG:teuthology.orchestra.run.smithi102:mon.b> sudo journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mon.b.service 2023-11-27T15:10:38.181 DEBUG:teuthology.orchestra.run.smithi150:mon.c> sudo journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mon.c.service 2023-11-27T15:10:38.183 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-11-27T15:10:38.183 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-11-27T15:10:38.211 INFO:journalctl@ceph.mon.c.smithi150.stdout:-- Logs begin at Mon 2023-11-27 14:53:12 UTC. -- 2023-11-27T15:10:38.216 INFO:journalctl@ceph.mon.b.smithi102.stdout:-- Logs begin at Mon 2023-11-27 14:52:53 UTC. -- 2023-11-27T15:10:38.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:38 smithi052 ceph-mon[99056]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:38.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:38 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:38.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:38 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:10:38.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:38 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:10:38.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:38 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:10:38.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:38 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:38.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:38 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:10:38.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:38 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:10:39.130 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:10:39.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:39 smithi052 ceph-mon[99056]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi052:172.21.15.52=a;smithi102:172.21.15.102=b;smithi150:172.21.15.150=c", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:10:39.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:39 smithi052 ceph-mon[99056]: Saving service mon spec with placement smithi052:172.21.15.52=a;smithi102:172.21.15.102=b;smithi150:172.21.15.150=c;count:3 2023-11-27T15:10:39.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:39 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:10:39.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:39 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:10:39.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:39 smithi052 ceph-mon[99056]: Deploying daemon mon.c on smithi150 2023-11-27T15:10:40.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:40 smithi052 ceph-mon[99056]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:40.959 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:40.952+0000 7fba8dfee700 1 -- 172.21.15.150:0/526924841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba880fef00 msgr2=0x7fba880ff2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:40.959 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:40.952+0000 7fba8dfee700 1 --2- 172.21.15.150:0/526924841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba880fef00 0x7fba880ff2d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fba78009910 tx=0x7fba78009c20 comp rx=0 tx=0).stop 2023-11-27T15:10:40.959 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:40.952+0000 7fba8dfee700 1 -- 172.21.15.150:0/526924841 shutdown_connections 2023-11-27T15:10:40.959 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:40.952+0000 7fba8dfee700 1 --2- 172.21.15.150:0/526924841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba880fef00 0x7fba880ff2d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:40.959 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:40.952+0000 7fba8dfee700 1 -- 172.21.15.150:0/526924841 >> 172.21.15.150:0/526924841 conn(0x7fba88074930 msgr2=0x7fba88074d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:40.959 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:40.952+0000 7fba8dfee700 1 -- 172.21.15.150:0/526924841 shutdown_connections 2023-11-27T15:10:40.959 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:40.952+0000 7fba8dfee700 1 -- 172.21.15.150:0/526924841 wait complete. 2023-11-27T15:10:40.959 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:40.952+0000 7fba8dfee700 1 Processor -- start 2023-11-27T15:10:40.959 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:40.953+0000 7fba8dfee700 1 -- start start 2023-11-27T15:10:40.960 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:40.953+0000 7fba8dfee700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba880fef00 0x7fba881940c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:40.960 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:40.953+0000 7fba8dfee700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba88194600 con 0x7fba880fef00 2023-11-27T15:10:40.960 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:40.953+0000 7fba877fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba880fef00 0x7fba881940c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:40.960 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:40.953+0000 7fba877fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba880fef00 0x7fba881940c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.150:58222/0 (socket says 172.21.15.150:58222) 2023-11-27T15:10:40.960 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:40.953+0000 7fba877fe700 1 -- 172.21.15.150:0/1600221094 learned_addr learned my addr 172.21.15.150:0/1600221094 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-27T15:10:40.960 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:40.954+0000 7fba877fe700 1 -- 172.21.15.150:0/1600221094 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba780095f0 con 0x7fba880fef00 2023-11-27T15:10:40.960 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:40.954+0000 7fba877fe700 1 --2- 172.21.15.150:0/1600221094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba880fef00 0x7fba881940c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fba78004f40 tx=0x7fba78005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:40.961 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:40.954+0000 7fba84ff9700 1 -- 172.21.15.150:0/1600221094 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba7801f070 con 0x7fba880fef00 2023-11-27T15:10:40.961 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:40.954+0000 7fba84ff9700 1 -- 172.21.15.150:0/1600221094 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba78004370 con 0x7fba880fef00 2023-11-27T15:10:40.961 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:40.954+0000 7fba8dfee700 1 -- 172.21.15.150:0/1600221094 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba88109090 con 0x7fba880fef00 2023-11-27T15:10:40.961 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:40.954+0000 7fba84ff9700 1 -- 172.21.15.150:0/1600221094 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba780039b0 con 0x7fba880fef00 2023-11-27T15:10:40.961 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:40.954+0000 7fba8dfee700 1 -- 172.21.15.150:0/1600221094 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba88108f00 con 0x7fba880fef00 2023-11-27T15:10:40.962 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:40.955+0000 7fba84ff9700 1 -- 172.21.15.150:0/1600221094 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fba78003c20 con 0x7fba880fef00 2023-11-27T15:10:40.962 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:40.955+0000 7fba8dfee700 1 -- 172.21.15.150:0/1600221094 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba88109220 con 0x7fba880fef00 2023-11-27T15:10:40.962 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:40.955+0000 7fba84ff9700 1 --2- 172.21.15.150:0/1600221094 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fba700388c0 0x7fba7003ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:40.962 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:40.956+0000 7fba84ff9700 1 -- 172.21.15.150:0/1600221094 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fba7804e9f0 con 0x7fba880fef00 2023-11-27T15:10:40.962 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:40.956+0000 7fba86ffd700 1 --2- 172.21.15.150:0/1600221094 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fba700388c0 0x7fba7003ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:40.962 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:40.956+0000 7fba86ffd700 1 --2- 172.21.15.150:0/1600221094 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fba700388c0 0x7fba7003ad70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fba7c006fd0 tx=0x7fba7c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:40.966 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:40.961+0000 7fba84ff9700 1 -- 172.21.15.150:0/1600221094 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fba7802b950 con 0x7fba880fef00 2023-11-27T15:10:41.204 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:40 smithi150 systemd[1]: Starting Ceph mon.c for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:10:41.251 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:41.245+0000 7fba8dfee700 1 -- 172.21.15.150:0/1600221094 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fba88106bb0 con 0x7fba880fef00 2023-11-27T15:10:41.251 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:41.246+0000 7fba84ff9700 1 -- 172.21.15.150:0/1600221094 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7fba78028030 con 0x7fba880fef00 2023-11-27T15:10:41.252 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-27T15:10:41.252 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":1,"fsid":"c6c7df88-8d36-11ee-95a2-87774f69a715","modified":"2023-11-27T15:08:59.495725Z","created":"2023-11-27T15:08:59.495725Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:3300","nonce":0},{"type":"v1","addr":"172.21.15.52:6789","nonce":0}]},"addr":"172.21.15.52:6789/0","public_addr":"172.21.15.52:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-11-27T15:10:41.253 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:41.248+0000 7fba8dfee700 1 -- 172.21.15.150:0/1600221094 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fba700388c0 msgr2=0x7fba7003ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:41.253 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:41.248+0000 7fba8dfee700 1 --2- 172.21.15.150:0/1600221094 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fba700388c0 0x7fba7003ad70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fba7c006fd0 tx=0x7fba7c006e40 comp rx=0 tx=0).stop 2023-11-27T15:10:41.254 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:41.248+0000 7fba8dfee700 1 -- 172.21.15.150:0/1600221094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba880fef00 msgr2=0x7fba881940c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:41.254 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:41.248+0000 7fba8dfee700 1 --2- 172.21.15.150:0/1600221094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba880fef00 0x7fba881940c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fba78004f40 tx=0x7fba78005020 comp rx=0 tx=0).stop 2023-11-27T15:10:41.254 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:41.248+0000 7fba8dfee700 1 -- 172.21.15.150:0/1600221094 shutdown_connections 2023-11-27T15:10:41.254 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:41.248+0000 7fba8dfee700 1 --2- 172.21.15.150:0/1600221094 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fba700388c0 0x7fba7003ad70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:41.254 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:41.248+0000 7fba8dfee700 1 --2- 172.21.15.150:0/1600221094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba880fef00 0x7fba881940c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:41.254 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:41.248+0000 7fba8dfee700 1 -- 172.21.15.150:0/1600221094 >> 172.21.15.150:0/1600221094 conn(0x7fba88074930 msgr2=0x7fba88103090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:41.254 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:41.248+0000 7fba8dfee700 1 -- 172.21.15.150:0/1600221094 shutdown_connections 2023-11-27T15:10:41.254 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:41.248+0000 7fba8dfee700 1 -- 172.21.15.150:0/1600221094 wait complete. 2023-11-27T15:10:41.255 INFO:teuthology.orchestra.run.smithi150.stderr:dumped monmap epoch 1 2023-11-27T15:10:41.459 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 podman[100495]: 2023-11-27T15:10:41.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: set uid:gid to 167:167 (ceph:ceph) 2023-11-27T15:10:41.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev), process ceph-mon, pid 7 2023-11-27T15:10:41.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: pidfile_write: ignore empty --pid-file 2023-11-27T15:10:41.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 bash[100495]: 4f1f45ffa809c7b7a435eb69d65e1a5c1f7c0f3ee9a227fd91f9c60dd061cad4 2023-11-27T15:10:41.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: load: jerasure load: lrc 2023-11-27T15:10:41.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: RocksDB version: 7.9.2 2023-11-27T15:10:41.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Git sha 0 2023-11-27T15:10:41.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Compile date 2023-11-16 19:40:53 2023-11-27T15:10:41.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: DB SUMMARY 2023-11-27T15:10:41.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: DB Session ID: H47531BRYZIJ2N8CEY0I 2023-11-27T15:10:41.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: CURRENT file: CURRENT 2023-11-27T15:10:41.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: IDENTITY file: IDENTITY 2023-11-27T15:10:41.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-11-27T15:10:41.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-11-27T15:10:41.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-11-27T15:10:41.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.error_if_exists: 0 2023-11-27T15:10:41.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.create_if_missing: 0 2023-11-27T15:10:41.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.paranoid_checks: 1 2023-11-27T15:10:41.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.flush_verify_memtable_count: 1 2023-11-27T15:10:41.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-11-27T15:10:41.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-11-27T15:10:41.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.env: 0x55b5092a9720 2023-11-27T15:10:41.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.fs: PosixFileSystem 2023-11-27T15:10:41.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.info_log: 0x55b50bd113c0 2023-11-27T15:10:41.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.max_file_opening_threads: 16 2023-11-27T15:10:41.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.statistics: (nil) 2023-11-27T15:10:41.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.use_fsync: 0 2023-11-27T15:10:41.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.max_log_file_size: 0 2023-11-27T15:10:41.782 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-11-27T15:10:41.782 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.log_file_time_to_roll: 0 2023-11-27T15:10:41.782 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.keep_log_file_num: 1000 2023-11-27T15:10:41.782 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.recycle_log_file_num: 0 2023-11-27T15:10:41.782 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.allow_fallocate: 1 2023-11-27T15:10:41.782 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.allow_mmap_reads: 0 2023-11-27T15:10:41.782 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.allow_mmap_writes: 0 2023-11-27T15:10:41.782 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.use_direct_reads: 0 2023-11-27T15:10:41.782 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-11-27T15:10:41.782 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.create_missing_column_families: 0 2023-11-27T15:10:41.782 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.db_log_dir: 2023-11-27T15:10:41.782 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.wal_dir: 2023-11-27T15:10:41.784 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.table_cache_numshardbits: 6 2023-11-27T15:10:41.784 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.WAL_ttl_seconds: 0 2023-11-27T15:10:41.784 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.WAL_size_limit_MB: 0 2023-11-27T15:10:41.784 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-11-27T15:10:41.784 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-11-27T15:10:41.784 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.is_fd_close_on_exec: 1 2023-11-27T15:10:41.785 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.advise_random_on_open: 1 2023-11-27T15:10:41.785 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.db_write_buffer_size: 0 2023-11-27T15:10:41.785 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.write_buffer_manager: 0x55b50af823c0 2023-11-27T15:10:41.785 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-11-27T15:10:41.785 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-11-27T15:10:41.785 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.use_adaptive_mutex: 0 2023-11-27T15:10:41.785 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.rate_limiter: (nil) 2023-11-27T15:10:41.785 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-11-27T15:10:41.785 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.wal_recovery_mode: 2 2023-11-27T15:10:41.785 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.enable_thread_tracking: 0 2023-11-27T15:10:41.785 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.enable_pipelined_write: 0 2023-11-27T15:10:41.785 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.unordered_write: 0 2023-11-27T15:10:41.785 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-11-27T15:10:41.786 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-11-27T15:10:41.786 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-11-27T15:10:41.786 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-11-27T15:10:41.786 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.row_cache: None 2023-11-27T15:10:41.786 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.wal_filter: None 2023-11-27T15:10:41.786 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-11-27T15:10:41.786 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.allow_ingest_behind: 0 2023-11-27T15:10:41.786 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.two_write_queues: 0 2023-11-27T15:10:41.786 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.manual_wal_flush: 0 2023-11-27T15:10:41.786 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.wal_compression: 0 2023-11-27T15:10:41.786 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.atomic_flush: 0 2023-11-27T15:10:41.787 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-11-27T15:10:41.787 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.persist_stats_to_disk: 0 2023-11-27T15:10:41.787 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.write_dbid_to_manifest: 0 2023-11-27T15:10:41.787 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.log_readahead_size: 0 2023-11-27T15:10:41.787 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-11-27T15:10:41.787 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.best_efforts_recovery: 0 2023-11-27T15:10:41.787 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-11-27T15:10:41.787 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-11-27T15:10:41.787 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.allow_data_in_errors: 0 2023-11-27T15:10:41.787 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.db_host_id: __hostname__ 2023-11-27T15:10:41.787 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.enforce_single_del_contracts: true 2023-11-27T15:10:41.787 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.max_background_jobs: 2 2023-11-27T15:10:41.788 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.max_background_compactions: -1 2023-11-27T15:10:41.788 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.max_subcompactions: 1 2023-11-27T15:10:41.788 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-11-27T15:10:41.788 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-11-27T15:10:41.788 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.delayed_write_rate : 16777216 2023-11-27T15:10:41.788 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.max_total_wal_size: 0 2023-11-27T15:10:41.788 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-11-27T15:10:41.788 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.stats_dump_period_sec: 600 2023-11-27T15:10:41.788 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.stats_persist_period_sec: 600 2023-11-27T15:10:41.788 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-11-27T15:10:41.788 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.max_open_files: -1 2023-11-27T15:10:41.788 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.bytes_per_sync: 0 2023-11-27T15:10:41.789 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.wal_bytes_per_sync: 0 2023-11-27T15:10:41.789 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.strict_bytes_per_sync: 0 2023-11-27T15:10:41.789 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.compaction_readahead_size: 0 2023-11-27T15:10:41.789 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.max_background_flushes: -1 2023-11-27T15:10:41.789 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Compression algorithms supported: 2023-11-27T15:10:41.789 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-11-27T15:10:41.789 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: kZSTD supported: 0 2023-11-27T15:10:41.789 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: kXpressCompression supported: 0 2023-11-27T15:10:41.789 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: kLZ4HCCompression supported: 1 2023-11-27T15:10:41.789 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: kZlibCompression supported: 1 2023-11-27T15:10:41.789 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: kSnappyCompression supported: 1 2023-11-27T15:10:41.789 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: kLZ4Compression supported: 1 2023-11-27T15:10:41.790 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: kBZip2Compression supported: 0 2023-11-27T15:10:41.790 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Fast CRC32 supported: Supported on x86 2023-11-27T15:10:41.790 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: DMutex implementation: pthread_mutex_t 2023-11-27T15:10:41.790 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-11-27T15:10:41.790 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-11-27T15:10:41.790 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-11-27T15:10:41.790 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.merge_operator: 2023-11-27T15:10:41.790 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.compaction_filter: None 2023-11-27T15:10:41.790 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.compaction_filter_factory: None 2023-11-27T15:10:41.790 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.sst_partitioner_factory: None 2023-11-27T15:10:41.790 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.memtable_factory: SkipListFactory 2023-11-27T15:10:41.790 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.table_factory: BlockBasedTable 2023-11-27T15:10:41.791 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b50bd114e0) 2023-11-27T15:10:41.791 INFO:journalctl@ceph.mon.c.smithi150.stdout: cache_index_and_filter_blocks: 1 2023-11-27T15:10:41.791 INFO:journalctl@ceph.mon.c.smithi150.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-11-27T15:10:41.791 INFO:journalctl@ceph.mon.c.smithi150.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-11-27T15:10:41.791 INFO:journalctl@ceph.mon.c.smithi150.stdout: pin_top_level_index_and_filter: 1 2023-11-27T15:10:41.791 INFO:journalctl@ceph.mon.c.smithi150.stdout: index_type: 0 2023-11-27T15:10:41.791 INFO:journalctl@ceph.mon.c.smithi150.stdout: data_block_index_type: 0 2023-11-27T15:10:41.791 INFO:journalctl@ceph.mon.c.smithi150.stdout: index_shortening: 1 2023-11-27T15:10:41.791 INFO:journalctl@ceph.mon.c.smithi150.stdout: data_block_hash_table_util_ratio: 0.750000 2023-11-27T15:10:41.791 INFO:journalctl@ceph.mon.c.smithi150.stdout: checksum: 4 2023-11-27T15:10:41.791 INFO:journalctl@ceph.mon.c.smithi150.stdout: no_block_cache: 0 2023-11-27T15:10:41.791 INFO:journalctl@ceph.mon.c.smithi150.stdout: block_cache: 0x55b50b0231f0 2023-11-27T15:10:41.791 INFO:journalctl@ceph.mon.c.smithi150.stdout: block_cache_name: BinnedLRUCache 2023-11-27T15:10:41.792 INFO:journalctl@ceph.mon.c.smithi150.stdout: block_cache_options: 2023-11-27T15:10:41.792 INFO:journalctl@ceph.mon.c.smithi150.stdout: capacity : 536870912 2023-11-27T15:10:41.792 INFO:journalctl@ceph.mon.c.smithi150.stdout: num_shard_bits : 4 2023-11-27T15:10:41.792 INFO:journalctl@ceph.mon.c.smithi150.stdout: strict_capacity_limit : 0 2023-11-27T15:10:41.792 INFO:journalctl@ceph.mon.c.smithi150.stdout: high_pri_pool_ratio: 0.000 2023-11-27T15:10:41.792 INFO:journalctl@ceph.mon.c.smithi150.stdout: block_cache_compressed: (nil) 2023-11-27T15:10:41.792 INFO:journalctl@ceph.mon.c.smithi150.stdout: persistent_cache: (nil) 2023-11-27T15:10:41.792 INFO:journalctl@ceph.mon.c.smithi150.stdout: block_size: 4096 2023-11-27T15:10:41.792 INFO:journalctl@ceph.mon.c.smithi150.stdout: block_size_deviation: 10 2023-11-27T15:10:41.792 INFO:journalctl@ceph.mon.c.smithi150.stdout: block_restart_interval: 16 2023-11-27T15:10:41.792 INFO:journalctl@ceph.mon.c.smithi150.stdout: index_block_restart_interval: 1 2023-11-27T15:10:41.792 INFO:journalctl@ceph.mon.c.smithi150.stdout: metadata_block_size: 4096 2023-11-27T15:10:41.792 INFO:journalctl@ceph.mon.c.smithi150.stdout: partition_filters: 0 2023-11-27T15:10:41.793 INFO:journalctl@ceph.mon.c.smithi150.stdout: use_delta_encoding: 1 2023-11-27T15:10:41.793 INFO:journalctl@ceph.mon.c.smithi150.stdout: filter_policy: bloomfilter 2023-11-27T15:10:41.793 INFO:journalctl@ceph.mon.c.smithi150.stdout: whole_key_filtering: 1 2023-11-27T15:10:41.793 INFO:journalctl@ceph.mon.c.smithi150.stdout: verify_compression: 0 2023-11-27T15:10:41.793 INFO:journalctl@ceph.mon.c.smithi150.stdout: read_amp_bytes_per_bit: 0 2023-11-27T15:10:41.793 INFO:journalctl@ceph.mon.c.smithi150.stdout: format_version: 5 2023-11-27T15:10:41.793 INFO:journalctl@ceph.mon.c.smithi150.stdout: enable_index_compression: 1 2023-11-27T15:10:41.793 INFO:journalctl@ceph.mon.c.smithi150.stdout: block_align: 0 2023-11-27T15:10:41.793 INFO:journalctl@ceph.mon.c.smithi150.stdout: max_auto_readahead_size: 262144 2023-11-27T15:10:41.793 INFO:journalctl@ceph.mon.c.smithi150.stdout: prepopulate_block_cache: 0 2023-11-27T15:10:41.793 INFO:journalctl@ceph.mon.c.smithi150.stdout: initial_auto_readahead_size: 8192 2023-11-27T15:10:41.793 INFO:journalctl@ceph.mon.c.smithi150.stdout: num_file_reads_for_auto_readahead: 2 2023-11-27T15:10:41.794 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.write_buffer_size: 33554432 2023-11-27T15:10:41.794 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.max_write_buffer_number: 2 2023-11-27T15:10:41.794 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.compression: NoCompression 2023-11-27T15:10:41.794 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.bottommost_compression: Disabled 2023-11-27T15:10:41.794 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.prefix_extractor: nullptr 2023-11-27T15:10:41.794 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-11-27T15:10:41.794 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.num_levels: 7 2023-11-27T15:10:41.794 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-11-27T15:10:41.794 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-11-27T15:10:41.794 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-11-27T15:10:41.794 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-11-27T15:10:41.794 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-11-27T15:10:41.794 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-11-27T15:10:41.795 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-11-27T15:10:41.795 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-11-27T15:10:41.795 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-11-27T15:10:41.795 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-11-27T15:10:41.795 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-11-27T15:10:41.795 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-11-27T15:10:41.795 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.compression_opts.window_bits: -14 2023-11-27T15:10:41.795 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.compression_opts.level: 32767 2023-11-27T15:10:41.795 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.compression_opts.strategy: 0 2023-11-27T15:10:41.795 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-11-27T15:10:41.795 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-11-27T15:10:41.795 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-11-27T15:10:41.796 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-11-27T15:10:41.796 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.compression_opts.enabled: false 2023-11-27T15:10:41.796 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-11-27T15:10:41.796 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-11-27T15:10:41.796 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-11-27T15:10:41.796 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-11-27T15:10:41.796 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.target_file_size_base: 67108864 2023-11-27T15:10:41.796 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.target_file_size_multiplier: 1 2023-11-27T15:10:41.796 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-11-27T15:10:41.796 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-11-27T15:10:41.796 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-11-27T15:10:41.796 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-11-27T15:10:41.797 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-11-27T15:10:41.797 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-11-27T15:10:41.797 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-11-27T15:10:41.797 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-11-27T15:10:41.797 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-11-27T15:10:41.797 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-11-27T15:10:41.797 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-11-27T15:10:41.797 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-11-27T15:10:41.797 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-11-27T15:10:41.797 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.arena_block_size: 1048576 2023-11-27T15:10:41.797 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-11-27T15:10:41.797 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-11-27T15:10:41.797 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.disable_auto_compactions: 0 2023-11-27T15:10:41.798 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-11-27T15:10:41.798 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-11-27T15:10:41.798 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-11-27T15:10:41.798 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-11-27T15:10:41.798 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-11-27T15:10:41.798 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-11-27T15:10:41.798 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-11-27T15:10:41.798 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-11-27T15:10:41.798 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-11-27T15:10:41.798 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-11-27T15:10:41.798 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.table_properties_collectors: 2023-11-27T15:10:41.799 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.inplace_update_support: 0 2023-11-27T15:10:41.799 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.inplace_update_num_locks: 10000 2023-11-27T15:10:41.799 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-11-27T15:10:41.799 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-11-27T15:10:41.799 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.memtable_huge_page_size: 0 2023-11-27T15:10:41.799 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.bloom_locality: 0 2023-11-27T15:10:41.799 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.max_successive_merges: 0 2023-11-27T15:10:41.799 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.optimize_filters_for_hits: 0 2023-11-27T15:10:41.799 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.paranoid_file_checks: 0 2023-11-27T15:10:41.799 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.force_consistency_checks: 1 2023-11-27T15:10:41.799 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.report_bg_io_stats: 0 2023-11-27T15:10:41.799 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.ttl: 2592000 2023-11-27T15:10:41.799 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.periodic_compaction_seconds: 0 2023-11-27T15:10:41.800 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-11-27T15:10:41.800 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-11-27T15:10:41.800 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.enable_blob_files: false 2023-11-27T15:10:41.800 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.min_blob_size: 0 2023-11-27T15:10:41.800 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.blob_file_size: 268435456 2023-11-27T15:10:41.800 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.blob_compression_type: NoCompression 2023-11-27T15:10:41.800 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.enable_blob_garbage_collection: false 2023-11-27T15:10:41.800 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-11-27T15:10:41.800 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-11-27T15:10:41.800 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-11-27T15:10:41.800 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.blob_file_starting_level: 0 2023-11-27T15:10:41.800 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-11-27T15:10:41.801 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-11-27T15:10:41.801 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-11-27T15:10:41.801 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d1a412d2-0ca7-4cb3-9b33-f1a6ecf1abdb 2023-11-27T15:10:41.801 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701097841461763, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-11-27T15:10:41.801 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-11-27T15:10:41.801 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701097841462183, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1701097841, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d1a412d2-0ca7-4cb3-9b33-f1a6ecf1abdb", "db_session_id": "H47531BRYZIJ2N8CEY0I", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-11-27T15:10:41.801 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701097841462250, "job": 1, "event": "recovery_finished"} 2023-11-27T15:10:41.801 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-11-27T15:10:41.801 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-11-27T15:10:41.801 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b50b0c0000 2023-11-27T15:10:41.801 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: DB pointer 0x55b50b0ac000 2023-11-27T15:10:41.801 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-11-27T15:10:41.802 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-11-27T15:10:41.802 INFO:journalctl@ceph.mon.c.smithi150.stdout: ** DB Stats ** 2023-11-27T15:10:41.802 INFO:journalctl@ceph.mon.c.smithi150.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-27T15:10:41.802 INFO:journalctl@ceph.mon.c.smithi150.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-11-27T15:10:41.802 INFO:journalctl@ceph.mon.c.smithi150.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-27T15:10:41.802 INFO:journalctl@ceph.mon.c.smithi150.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-27T15:10:41.802 INFO:journalctl@ceph.mon.c.smithi150.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-11-27T15:10:41.802 INFO:journalctl@ceph.mon.c.smithi150.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-27T15:10:41.802 INFO:journalctl@ceph.mon.c.smithi150.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-27T15:10:41.802 INFO:journalctl@ceph.mon.c.smithi150.stdout: 2023-11-27T15:10:41.802 INFO:journalctl@ceph.mon.c.smithi150.stdout: ** Compaction Stats [default] ** 2023-11-27T15:10:41.802 INFO:journalctl@ceph.mon.c.smithi150.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-27T15:10:41.803 INFO:journalctl@ceph.mon.c.smithi150.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-11-27T15:10:41.803 INFO:journalctl@ceph.mon.c.smithi150.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-27T15:10:41.803 INFO:journalctl@ceph.mon.c.smithi150.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-27T15:10:41.803 INFO:journalctl@ceph.mon.c.smithi150.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-27T15:10:41.803 INFO:journalctl@ceph.mon.c.smithi150.stdout: 2023-11-27T15:10:41.803 INFO:journalctl@ceph.mon.c.smithi150.stdout: ** Compaction Stats [default] ** 2023-11-27T15:10:41.803 INFO:journalctl@ceph.mon.c.smithi150.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-27T15:10:41.803 INFO:journalctl@ceph.mon.c.smithi150.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-11-27T15:10:41.803 INFO:journalctl@ceph.mon.c.smithi150.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-27T15:10:41.803 INFO:journalctl@ceph.mon.c.smithi150.stdout: 2023-11-27T15:10:41.803 INFO:journalctl@ceph.mon.c.smithi150.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-11-27T15:10:41.803 INFO:journalctl@ceph.mon.c.smithi150.stdout: 2023-11-27T15:10:41.803 INFO:journalctl@ceph.mon.c.smithi150.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-27T15:10:41.804 INFO:journalctl@ceph.mon.c.smithi150.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-11-27T15:10:41.804 INFO:journalctl@ceph.mon.c.smithi150.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-11-27T15:10:41.804 INFO:journalctl@ceph.mon.c.smithi150.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-11-27T15:10:41.804 INFO:journalctl@ceph.mon.c.smithi150.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-11-27T15:10:41.804 INFO:journalctl@ceph.mon.c.smithi150.stdout: AddFile(Keys): cumulative 0, interval 0 2023-11-27T15:10:41.804 INFO:journalctl@ceph.mon.c.smithi150.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-27T15:10:41.804 INFO:journalctl@ceph.mon.c.smithi150.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-27T15:10:41.804 INFO:journalctl@ceph.mon.c.smithi150.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-11-27T15:10:41.804 INFO:journalctl@ceph.mon.c.smithi150.stdout: Block cache BinnedLRUCache@0x55b50b0231f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2023-11-27T15:10:41.804 INFO:journalctl@ceph.mon.c.smithi150.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-11-27T15:10:41.804 INFO:journalctl@ceph.mon.c.smithi150.stdout: 2023-11-27T15:10:41.804 INFO:journalctl@ceph.mon.c.smithi150.stdout: ** File Read Latency Histogram By Level [default] ** 2023-11-27T15:10:41.805 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-11-27T15:10:41.805 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: using public_addr v2:172.21.15.150:0/0 -> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] 2023-11-27T15:10:41.805 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: starting mon.c rank -1 at public addrs [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] at bind addrs [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid c6c7df88-8d36-11ee-95a2-87774f69a715 2023-11-27T15:10:41.805 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: mon.c@-1(???) e0 preinit fsid c6c7df88-8d36-11ee-95a2-87774f69a715 2023-11-27T15:10:41.805 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: mon.c@-1(synchronizing).mds e1 new map 2023-11-27T15:10:41.805 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: mon.c@-1(synchronizing).mds e1 print_map 2023-11-27T15:10:41.805 INFO:journalctl@ceph.mon.c.smithi150.stdout: e1 2023-11-27T15:10:41.805 INFO:journalctl@ceph.mon.c.smithi150.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-11-27T15:10:41.805 INFO:journalctl@ceph.mon.c.smithi150.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-27T15:10:41.805 INFO:journalctl@ceph.mon.c.smithi150.stdout: legacy client fscid: -1 2023-11-27T15:10:41.805 INFO:journalctl@ceph.mon.c.smithi150.stdout: 2023-11-27T15:10:41.805 INFO:journalctl@ceph.mon.c.smithi150.stdout: No filesystems configured 2023-11-27T15:10:41.806 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-11-27T15:10:41.806 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-11-27T15:10:41.806 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-11-27T15:10:41.806 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-11-27T15:10:41.806 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-11-27T15:10:41.806 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-11-27T15:10:41.806 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-11-27T15:10:41.806 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-27T15:10:41.806 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-27T15:10:41.806 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-27T15:10:41.806 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:41.806 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi150", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:10:41.807 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:41.807 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:41.807 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:41.807 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:41.807 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi102", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:10:41.807 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:10:41.807 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:10:41.807 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:10:41.807 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: Updating smithi102:/etc/ceph/ceph.conf 2023-11-27T15:10:41.807 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: Updating smithi102:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:10:41.807 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: Deploying cephadm binary to smithi150 2023-11-27T15:10:41.807 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:10:41.808 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: Updating smithi102:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-11-27T15:10:41.808 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:41.808 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:41.808 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:41.808 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:41.808 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:41.808 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: Added host smithi150 2023-11-27T15:10:41.808 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:10:41.808 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:41.808 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:41.808 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:41.808 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:10:41.808 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:41.809 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:41.809 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2229004337' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-11-27T15:10:41.809 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2229004337' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-11-27T15:10:41.809 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:10:41.809 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:41.809 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:41.809 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:41.809 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:41.809 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:41.809 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:10:41.809 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:10:41.809 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:10:41.810 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:10:41.810 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: Updating smithi150:/etc/ceph/ceph.conf 2023-11-27T15:10:41.810 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: Updating smithi150:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:10:41.810 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:10:41.810 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: Updating smithi150:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-11-27T15:10:41.810 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:41.810 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:41.810 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:41.810 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:41.810 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:41.810 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:10:41.810 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:10:41.811 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:10:41.811 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:41.811 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:10:41.811 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:10:41.811 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi052:172.21.15.52=a;smithi102:172.21.15.102=b;smithi150:172.21.15.150=c", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:10:41.811 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: Saving service mon spec with placement smithi052:172.21.15.52=a;smithi102:172.21.15.102=b;smithi150:172.21.15.150=c;count:3 2023-11-27T15:10:41.811 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:10:41.811 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:10:41.811 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: Deploying daemon mon.c on smithi150 2023-11-27T15:10:41.811 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:41.811 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-11-27T15:10:41.811 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-11-27T15:10:41.811 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: expand_channel_meta expand map: {default=false} 2023-11-27T15:10:41.812 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: expand_channel_meta from 'false' to 'false' 2023-11-27T15:10:41.812 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: expand_channel_meta expanded map: {default=false} 2023-11-27T15:10:41.812 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: expand_channel_meta expand map: {default=info} 2023-11-27T15:10:41.812 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: expand_channel_meta from 'info' to 'info' 2023-11-27T15:10:41.812 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: expand_channel_meta expanded map: {default=info} 2023-11-27T15:10:41.812 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: expand_channel_meta expand map: {default=daemon} 2023-11-27T15:10:41.812 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: expand_channel_meta from 'daemon' to 'daemon' 2023-11-27T15:10:41.812 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: expand_channel_meta expanded map: {default=daemon} 2023-11-27T15:10:41.812 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: expand_channel_meta expand map: {default=debug} 2023-11-27T15:10:41.812 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: expand_channel_meta from 'debug' to 'debug' 2023-11-27T15:10:41.812 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 ceph-mon[100559]: expand_channel_meta expanded map: {default=debug} 2023-11-27T15:10:41.812 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:41 smithi150 systemd[1]: Started Ceph mon.c for c6c7df88-8d36-11ee-95a2-87774f69a715. 2023-11-27T15:10:42.699 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-11-27T15:10:42.700 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-11-27T15:10:43.352 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:10:46.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:46 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:10:46.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:46 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:10:46.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:46 smithi052 ceph-mon[99056]: mon.a calling monitor election 2023-11-27T15:10:46.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:46 smithi052 ceph-mon[99056]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:46.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:46 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:10:46.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:46 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:10:46.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:46 smithi052 ceph-mon[99056]: mon.c calling monitor election 2023-11-27T15:10:46.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:46 smithi052 ceph-mon[99056]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:46.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:46 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:10:46.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:46 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:10:46.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:46 smithi052 ceph-mon[99056]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:46.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:46 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:10:46.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:46 smithi052 ceph-mon[99056]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-27T15:10:46.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:46 smithi052 ceph-mon[99056]: monmap e2: 2 mons at {a=[v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0],c=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0]} removed_ranks: {} 2023-11-27T15:10:46.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:46 smithi052 ceph-mon[99056]: fsmap 2023-11-27T15:10:46.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:46 smithi052 ceph-mon[99056]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:10:46.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:46 smithi052 ceph-mon[99056]: mgrmap e13: x(active, since 42s) 2023-11-27T15:10:46.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:46 smithi052 ceph-mon[99056]: overall HEALTH_OK 2023-11-27T15:10:46.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:46 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:46.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:46 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:47.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:46 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:10:47.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:46 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:10:47.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:46 smithi150 ceph-mon[100559]: mon.a calling monitor election 2023-11-27T15:10:47.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:46 smithi150 ceph-mon[100559]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:47.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:46 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:10:47.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:46 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:10:47.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:46 smithi150 ceph-mon[100559]: mon.c calling monitor election 2023-11-27T15:10:47.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:46 smithi150 ceph-mon[100559]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:47.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:46 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:10:47.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:46 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:10:47.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:46 smithi150 ceph-mon[100559]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:47.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:46 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:10:47.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:46 smithi150 ceph-mon[100559]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-27T15:10:47.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:46 smithi150 ceph-mon[100559]: monmap e2: 2 mons at {a=[v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0],c=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0]} removed_ranks: {} 2023-11-27T15:10:47.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:46 smithi150 ceph-mon[100559]: fsmap 2023-11-27T15:10:47.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:46 smithi150 ceph-mon[100559]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:10:47.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:46 smithi150 ceph-mon[100559]: mgrmap e13: x(active, since 42s) 2023-11-27T15:10:47.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:46 smithi150 ceph-mon[100559]: overall HEALTH_OK 2023-11-27T15:10:47.031 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:46 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:47.031 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:46 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:47.534 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.529+0000 7f4c13f70700 1 -- 172.21.15.150:0/2498299650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bf80035c0 msgr2=0x7f4bf8005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:47.535 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.529+0000 7f4c13f70700 1 --2- 172.21.15.150:0/2498299650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bf80035c0 0x7f4bf8005a40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c076150 tx=0x7f4bfc00fe60 comp rx=0 tx=0).stop 2023-11-27T15:10:47.535 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.529+0000 7f4c13f70700 1 -- 172.21.15.150:0/2498299650 shutdown_connections 2023-11-27T15:10:47.535 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.529+0000 7f4c13f70700 1 --2- 172.21.15.150:0/2498299650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bf80035c0 0x7f4bf8005a40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:47.535 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.529+0000 7f4c13f70700 1 -- 172.21.15.150:0/2498299650 >> 172.21.15.150:0/2498299650 conn(0x7f4c0c074c40 msgr2=0x7f4c0c075040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:47.535 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.529+0000 7f4c13f70700 1 -- 172.21.15.150:0/2498299650 shutdown_connections 2023-11-27T15:10:47.535 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.529+0000 7f4c13f70700 1 -- 172.21.15.150:0/2498299650 wait complete. 2023-11-27T15:10:47.536 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.530+0000 7f4c13f70700 1 Processor -- start 2023-11-27T15:10:47.536 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.530+0000 7f4c13f70700 1 -- start start 2023-11-27T15:10:47.537 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.530+0000 7f4c13f70700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bf80035c0 0x7f4c0c1941f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:47.537 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.531+0000 7f4c13f70700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c0c194730 0x7f4c0c198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:47.537 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.531+0000 7f4c13f70700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c194cb0 con 0x7f4c0c194730 2023-11-27T15:10:47.537 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.531+0000 7f4c13f70700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c194e20 con 0x7f4bf80035c0 2023-11-27T15:10:47.537 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.531+0000 7f4c11d0c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bf80035c0 0x7f4c0c1941f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:47.537 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.531+0000 7f4c11d0c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bf80035c0 0x7f4c0c1941f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49312/0 (socket says 172.21.15.150:49312) 2023-11-27T15:10:47.537 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.531+0000 7f4c11d0c700 1 -- 172.21.15.150:0/4075601112 learned_addr learned my addr 172.21.15.150:0/4075601112 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-27T15:10:47.537 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.531+0000 7f4c11d0c700 1 -- 172.21.15.150:0/4075601112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bf80035c0 msgr2=0x7f4c0c1941f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-11-27T15:10:47.537 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.531+0000 7f4c11d0c700 1 -- 172.21.15.150:0/4075601112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bf80035c0 msgr2=0x7f4c0c1941f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-27T15:10:47.537 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.531+0000 7f4c11d0c700 1 --2- 172.21.15.150:0/4075601112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bf80035c0 0x7f4c0c1941f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-27T15:10:47.537 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.531+0000 7f4c11d0c700 1 --2- 172.21.15.150:0/4075601112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bf80035c0 0x7f4c0c1941f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-27T15:10:47.537 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.531+0000 7f4c1150b700 1 --2- 172.21.15.150:0/4075601112 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c0c194730 0x7f4c0c198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:47.538 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.531+0000 7f4c1150b700 1 -- 172.21.15.150:0/4075601112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bf80035c0 msgr2=0x7f4c0c1941f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-27T15:10:47.538 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.531+0000 7f4c1150b700 1 --2- 172.21.15.150:0/4075601112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bf80035c0 0x7f4c0c1941f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:47.538 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.531+0000 7f4c1150b700 1 -- 172.21.15.150:0/4075601112 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bfc00f8b0 con 0x7f4c0c194730 2023-11-27T15:10:47.538 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.532+0000 7f4c1150b700 1 --2- 172.21.15.150:0/4075601112 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c0c194730 0x7f4c0c198ba0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800b4d0 tx=0x7f4c0800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:47.539 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.532+0000 7f4c02ffd700 1 -- 172.21.15.150:0/4075601112 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f4c08011630 con 0x7f4c0c194730 2023-11-27T15:10:47.539 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.532+0000 7f4c02ffd700 1 -- 172.21.15.150:0/4075601112 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c08011c50 con 0x7f4c0c194730 2023-11-27T15:10:47.539 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.532+0000 7f4c13f70700 1 -- 172.21.15.150:0/4075601112 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c0c106c50 con 0x7f4c0c194730 2023-11-27T15:10:47.539 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.532+0000 7f4c02ffd700 1 -- 172.21.15.150:0/4075601112 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f4c08010d90 con 0x7f4c0c194730 2023-11-27T15:10:47.539 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.532+0000 7f4c13f70700 1 -- 172.21.15.150:0/4075601112 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0c106ac0 con 0x7f4c0c194730 2023-11-27T15:10:47.540 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.533+0000 7f4c02ffd700 1 -- 172.21.15.150:0/4075601112 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f4c0801a400 con 0x7f4c0c194730 2023-11-27T15:10:47.540 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.533+0000 7f4c13f70700 1 -- 172.21.15.150:0/4075601112 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c0c106de0 con 0x7f4c0c194730 2023-11-27T15:10:47.540 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.533+0000 7f4c02ffd700 1 --2- 172.21.15.150:0/4075601112 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4bf00386b0 0x7f4bf003ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:47.540 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.533+0000 7f4c02ffd700 1 -- 172.21.15.150:0/4075601112 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4c0804aef0 con 0x7f4c0c194730 2023-11-27T15:10:47.540 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.533+0000 7f4c11d0c700 1 --2- 172.21.15.150:0/4075601112 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4bf00386b0 0x7f4bf003ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:47.540 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.534+0000 7f4c11d0c700 1 --2- 172.21.15.150:0/4075601112 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4bf00386b0 0x7f4bf003ab60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc005ed0 tx=0x7f4bfc001e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:47.543 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.538+0000 7f4c02ffd700 1 -- 172.21.15.150:0/4075601112 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4c080176d0 con 0x7f4c0c194730 2023-11-27T15:10:47.808 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.803+0000 7f4c13f70700 1 -- 172.21.15.150:0/4075601112 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4c0c199a20 con 0x7f4c0c194730 2023-11-27T15:10:47.809 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.804+0000 7f4c02ffd700 1 -- 172.21.15.150:0/4075601112 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7f4c080172c0 con 0x7f4c0c194730 2023-11-27T15:10:47.809 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-27T15:10:47.809 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":2,"fsid":"c6c7df88-8d36-11ee-95a2-87774f69a715","modified":"2023-11-27T15:10:41.484807Z","created":"2023-11-27T15:08:59.495725Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:3300","nonce":0},{"type":"v1","addr":"172.21.15.52:6789","nonce":0}]},"addr":"172.21.15.52:6789/0","public_addr":"172.21.15.52:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:3300","nonce":0},{"type":"v1","addr":"172.21.15.150:6789","nonce":0}]},"addr":"172.21.15.150:6789/0","public_addr":"172.21.15.150:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-11-27T15:10:47.811 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.806+0000 7f4c13f70700 1 -- 172.21.15.150:0/4075601112 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4bf00386b0 msgr2=0x7f4bf003ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:47.811 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.806+0000 7f4c13f70700 1 --2- 172.21.15.150:0/4075601112 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4bf00386b0 0x7f4bf003ab60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc005ed0 tx=0x7f4bfc001e70 comp rx=0 tx=0).stop 2023-11-27T15:10:47.811 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.806+0000 7f4c13f70700 1 -- 172.21.15.150:0/4075601112 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c0c194730 msgr2=0x7f4c0c198ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:47.811 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.806+0000 7f4c13f70700 1 --2- 172.21.15.150:0/4075601112 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c0c194730 0x7f4c0c198ba0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800b4d0 tx=0x7f4c0800b890 comp rx=0 tx=0).stop 2023-11-27T15:10:47.812 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.806+0000 7f4c13f70700 1 -- 172.21.15.150:0/4075601112 shutdown_connections 2023-11-27T15:10:47.812 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.806+0000 7f4c13f70700 1 --2- 172.21.15.150:0/4075601112 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4bf00386b0 0x7f4bf003ab60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:47.812 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.806+0000 7f4c13f70700 1 --2- 172.21.15.150:0/4075601112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bf80035c0 0x7f4c0c1941f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:47.812 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.806+0000 7f4c13f70700 1 --2- 172.21.15.150:0/4075601112 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c0c194730 0x7f4c0c198ba0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:47.812 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.806+0000 7f4c13f70700 1 -- 172.21.15.150:0/4075601112 >> 172.21.15.150:0/4075601112 conn(0x7f4c0c074c40 msgr2=0x7f4c0c18cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:47.812 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.806+0000 7f4c13f70700 1 -- 172.21.15.150:0/4075601112 shutdown_connections 2023-11-27T15:10:47.812 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:47.806+0000 7f4c13f70700 1 -- 172.21.15.150:0/4075601112 wait complete. 2023-11-27T15:10:47.812 INFO:teuthology.orchestra.run.smithi150.stderr:dumped monmap epoch 2 2023-11-27T15:10:47.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:47 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:47.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:47 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:10:47.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:47 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:10:47.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:47 smithi052 ceph-mon[99056]: Deploying daemon mon.b on smithi102 2023-11-27T15:10:47.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:47 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:10:47.972 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:47 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:47.972 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:47 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:10:47.972 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:47 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:10:47.972 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:47 smithi150 ceph-mon[100559]: Deploying daemon mon.b on smithi102 2023-11-27T15:10:47.972 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:47 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:10:48.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:48 smithi052 ceph-mon[99056]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:48.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:48 smithi052 ceph-mon[99056]: from='client.? 172.21.15.150:0/4075601112' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-27T15:10:49.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:48 smithi150 ceph-mon[100559]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:49.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:48 smithi150 ceph-mon[100559]: from='client.? 172.21.15.150:0/4075601112' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-27T15:10:49.200 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-11-27T15:10:49.201 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-11-27T15:10:49.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 systemd[1]: Starting Ceph mon.b for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:10:49.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 podman[100256]: 2023-11-27T15:10:49.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 bash[100256]: 9e5952c696502340d22cd9dc68478802729d44f457ad1161bd7c43637508749b 2023-11-27T15:10:49.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: set uid:gid to 167:167 (ceph:ceph) 2023-11-27T15:10:49.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev), process ceph-mon, pid 7 2023-11-27T15:10:49.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: pidfile_write: ignore empty --pid-file 2023-11-27T15:10:49.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: load: jerasure load: lrc 2023-11-27T15:10:49.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: RocksDB version: 7.9.2 2023-11-27T15:10:49.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Git sha 0 2023-11-27T15:10:49.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Compile date 2023-11-16 19:40:53 2023-11-27T15:10:49.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: DB SUMMARY 2023-11-27T15:10:49.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: DB Session ID: W8G7F9DBMD4IIPT41VRR 2023-11-27T15:10:49.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: CURRENT file: CURRENT 2023-11-27T15:10:49.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: IDENTITY file: IDENTITY 2023-11-27T15:10:49.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-11-27T15:10:49.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-11-27T15:10:49.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-11-27T15:10:49.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.error_if_exists: 0 2023-11-27T15:10:49.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.create_if_missing: 0 2023-11-27T15:10:49.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.paranoid_checks: 1 2023-11-27T15:10:49.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.flush_verify_memtable_count: 1 2023-11-27T15:10:49.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-11-27T15:10:49.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-11-27T15:10:49.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.env: 0x56351ff63720 2023-11-27T15:10:49.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.fs: PosixFileSystem 2023-11-27T15:10:49.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.info_log: 0x56352296f3c0 2023-11-27T15:10:49.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.max_file_opening_threads: 16 2023-11-27T15:10:49.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.statistics: (nil) 2023-11-27T15:10:49.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.use_fsync: 0 2023-11-27T15:10:49.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.max_log_file_size: 0 2023-11-27T15:10:49.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-11-27T15:10:49.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.log_file_time_to_roll: 0 2023-11-27T15:10:49.832 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.keep_log_file_num: 1000 2023-11-27T15:10:49.832 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.recycle_log_file_num: 0 2023-11-27T15:10:49.832 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.allow_fallocate: 1 2023-11-27T15:10:49.832 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.allow_mmap_reads: 0 2023-11-27T15:10:49.832 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.allow_mmap_writes: 0 2023-11-27T15:10:49.832 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.use_direct_reads: 0 2023-11-27T15:10:49.832 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-11-27T15:10:49.832 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.create_missing_column_families: 0 2023-11-27T15:10:49.832 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.db_log_dir: 2023-11-27T15:10:49.834 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.wal_dir: 2023-11-27T15:10:49.834 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.table_cache_numshardbits: 6 2023-11-27T15:10:49.834 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.WAL_ttl_seconds: 0 2023-11-27T15:10:49.834 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.WAL_size_limit_MB: 0 2023-11-27T15:10:49.834 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-11-27T15:10:49.834 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-11-27T15:10:49.834 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.is_fd_close_on_exec: 1 2023-11-27T15:10:49.834 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.advise_random_on_open: 1 2023-11-27T15:10:49.834 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.db_write_buffer_size: 0 2023-11-27T15:10:49.835 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.write_buffer_manager: 0x563521be03c0 2023-11-27T15:10:49.835 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-11-27T15:10:49.835 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-11-27T15:10:49.835 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.use_adaptive_mutex: 0 2023-11-27T15:10:49.835 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.rate_limiter: (nil) 2023-11-27T15:10:49.835 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-11-27T15:10:49.835 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.wal_recovery_mode: 2 2023-11-27T15:10:49.835 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.enable_thread_tracking: 0 2023-11-27T15:10:49.835 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.enable_pipelined_write: 0 2023-11-27T15:10:49.835 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.unordered_write: 0 2023-11-27T15:10:49.835 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-11-27T15:10:49.835 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-11-27T15:10:49.836 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-11-27T15:10:49.836 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-11-27T15:10:49.836 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.row_cache: None 2023-11-27T15:10:49.836 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.wal_filter: None 2023-11-27T15:10:49.836 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-11-27T15:10:49.836 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.allow_ingest_behind: 0 2023-11-27T15:10:49.836 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.two_write_queues: 0 2023-11-27T15:10:49.836 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.manual_wal_flush: 0 2023-11-27T15:10:49.836 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.wal_compression: 0 2023-11-27T15:10:49.836 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.atomic_flush: 0 2023-11-27T15:10:49.837 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-11-27T15:10:49.837 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.persist_stats_to_disk: 0 2023-11-27T15:10:49.837 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.write_dbid_to_manifest: 0 2023-11-27T15:10:49.837 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.log_readahead_size: 0 2023-11-27T15:10:49.837 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-11-27T15:10:49.837 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.best_efforts_recovery: 0 2023-11-27T15:10:49.837 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-11-27T15:10:49.837 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-11-27T15:10:49.837 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.allow_data_in_errors: 0 2023-11-27T15:10:49.838 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.db_host_id: __hostname__ 2023-11-27T15:10:49.838 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.enforce_single_del_contracts: true 2023-11-27T15:10:49.838 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.max_background_jobs: 2 2023-11-27T15:10:49.838 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.max_background_compactions: -1 2023-11-27T15:10:49.839 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.max_subcompactions: 1 2023-11-27T15:10:49.839 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-11-27T15:10:49.839 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-11-27T15:10:49.839 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.delayed_write_rate : 16777216 2023-11-27T15:10:49.839 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.max_total_wal_size: 0 2023-11-27T15:10:49.839 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-11-27T15:10:49.839 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.stats_dump_period_sec: 600 2023-11-27T15:10:49.839 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.stats_persist_period_sec: 600 2023-11-27T15:10:49.839 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-11-27T15:10:49.839 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.max_open_files: -1 2023-11-27T15:10:49.839 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.bytes_per_sync: 0 2023-11-27T15:10:49.839 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.wal_bytes_per_sync: 0 2023-11-27T15:10:49.840 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.strict_bytes_per_sync: 0 2023-11-27T15:10:49.840 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.compaction_readahead_size: 0 2023-11-27T15:10:49.840 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.max_background_flushes: -1 2023-11-27T15:10:49.840 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Compression algorithms supported: 2023-11-27T15:10:49.840 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-11-27T15:10:49.840 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: kZSTD supported: 0 2023-11-27T15:10:49.840 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: kXpressCompression supported: 0 2023-11-27T15:10:49.840 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: kLZ4HCCompression supported: 1 2023-11-27T15:10:49.840 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: kZlibCompression supported: 1 2023-11-27T15:10:49.840 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: kSnappyCompression supported: 1 2023-11-27T15:10:49.840 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: kLZ4Compression supported: 1 2023-11-27T15:10:49.841 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: kBZip2Compression supported: 0 2023-11-27T15:10:49.841 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Fast CRC32 supported: Supported on x86 2023-11-27T15:10:49.841 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: DMutex implementation: pthread_mutex_t 2023-11-27T15:10:49.841 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-11-27T15:10:49.841 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-11-27T15:10:49.841 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-11-27T15:10:49.841 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.merge_operator: 2023-11-27T15:10:49.841 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.compaction_filter: None 2023-11-27T15:10:49.841 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.compaction_filter_factory: None 2023-11-27T15:10:49.841 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.sst_partitioner_factory: None 2023-11-27T15:10:49.841 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.memtable_factory: SkipListFactory 2023-11-27T15:10:49.841 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.table_factory: BlockBasedTable 2023-11-27T15:10:49.842 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56352296f4e0) 2023-11-27T15:10:49.842 INFO:journalctl@ceph.mon.b.smithi102.stdout: cache_index_and_filter_blocks: 1 2023-11-27T15:10:49.842 INFO:journalctl@ceph.mon.b.smithi102.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-11-27T15:10:49.842 INFO:journalctl@ceph.mon.b.smithi102.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-11-27T15:10:49.842 INFO:journalctl@ceph.mon.b.smithi102.stdout: pin_top_level_index_and_filter: 1 2023-11-27T15:10:49.842 INFO:journalctl@ceph.mon.b.smithi102.stdout: index_type: 0 2023-11-27T15:10:49.842 INFO:journalctl@ceph.mon.b.smithi102.stdout: data_block_index_type: 0 2023-11-27T15:10:49.842 INFO:journalctl@ceph.mon.b.smithi102.stdout: index_shortening: 1 2023-11-27T15:10:49.842 INFO:journalctl@ceph.mon.b.smithi102.stdout: data_block_hash_table_util_ratio: 0.750000 2023-11-27T15:10:49.842 INFO:journalctl@ceph.mon.b.smithi102.stdout: checksum: 4 2023-11-27T15:10:49.842 INFO:journalctl@ceph.mon.b.smithi102.stdout: no_block_cache: 0 2023-11-27T15:10:49.843 INFO:journalctl@ceph.mon.b.smithi102.stdout: block_cache: 0x563521c811f0 2023-11-27T15:10:49.843 INFO:journalctl@ceph.mon.b.smithi102.stdout: block_cache_name: BinnedLRUCache 2023-11-27T15:10:49.843 INFO:journalctl@ceph.mon.b.smithi102.stdout: block_cache_options: 2023-11-27T15:10:49.843 INFO:journalctl@ceph.mon.b.smithi102.stdout: capacity : 536870912 2023-11-27T15:10:49.843 INFO:journalctl@ceph.mon.b.smithi102.stdout: num_shard_bits : 4 2023-11-27T15:10:49.843 INFO:journalctl@ceph.mon.b.smithi102.stdout: strict_capacity_limit : 0 2023-11-27T15:10:49.843 INFO:journalctl@ceph.mon.b.smithi102.stdout: high_pri_pool_ratio: 0.000 2023-11-27T15:10:49.843 INFO:journalctl@ceph.mon.b.smithi102.stdout: block_cache_compressed: (nil) 2023-11-27T15:10:49.843 INFO:journalctl@ceph.mon.b.smithi102.stdout: persistent_cache: (nil) 2023-11-27T15:10:49.843 INFO:journalctl@ceph.mon.b.smithi102.stdout: block_size: 4096 2023-11-27T15:10:49.843 INFO:journalctl@ceph.mon.b.smithi102.stdout: block_size_deviation: 10 2023-11-27T15:10:49.843 INFO:journalctl@ceph.mon.b.smithi102.stdout: block_restart_interval: 16 2023-11-27T15:10:49.844 INFO:journalctl@ceph.mon.b.smithi102.stdout: index_block_restart_interval: 1 2023-11-27T15:10:49.844 INFO:journalctl@ceph.mon.b.smithi102.stdout: metadata_block_size: 4096 2023-11-27T15:10:49.844 INFO:journalctl@ceph.mon.b.smithi102.stdout: partition_filters: 0 2023-11-27T15:10:49.844 INFO:journalctl@ceph.mon.b.smithi102.stdout: use_delta_encoding: 1 2023-11-27T15:10:49.844 INFO:journalctl@ceph.mon.b.smithi102.stdout: filter_policy: bloomfilter 2023-11-27T15:10:49.844 INFO:journalctl@ceph.mon.b.smithi102.stdout: whole_key_filtering: 1 2023-11-27T15:10:49.844 INFO:journalctl@ceph.mon.b.smithi102.stdout: verify_compression: 0 2023-11-27T15:10:49.844 INFO:journalctl@ceph.mon.b.smithi102.stdout: read_amp_bytes_per_bit: 0 2023-11-27T15:10:49.844 INFO:journalctl@ceph.mon.b.smithi102.stdout: format_version: 5 2023-11-27T15:10:49.844 INFO:journalctl@ceph.mon.b.smithi102.stdout: enable_index_compression: 1 2023-11-27T15:10:49.844 INFO:journalctl@ceph.mon.b.smithi102.stdout: block_align: 0 2023-11-27T15:10:49.844 INFO:journalctl@ceph.mon.b.smithi102.stdout: max_auto_readahead_size: 262144 2023-11-27T15:10:49.844 INFO:journalctl@ceph.mon.b.smithi102.stdout: prepopulate_block_cache: 0 2023-11-27T15:10:49.844 INFO:journalctl@ceph.mon.b.smithi102.stdout: initial_auto_readahead_size: 8192 2023-11-27T15:10:49.845 INFO:journalctl@ceph.mon.b.smithi102.stdout: num_file_reads_for_auto_readahead: 2 2023-11-27T15:10:49.845 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.write_buffer_size: 33554432 2023-11-27T15:10:49.845 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.max_write_buffer_number: 2 2023-11-27T15:10:49.845 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.compression: NoCompression 2023-11-27T15:10:49.845 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.bottommost_compression: Disabled 2023-11-27T15:10:49.845 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.prefix_extractor: nullptr 2023-11-27T15:10:49.845 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-11-27T15:10:49.845 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.num_levels: 7 2023-11-27T15:10:49.845 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-11-27T15:10:49.845 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-11-27T15:10:49.845 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-11-27T15:10:49.845 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-11-27T15:10:49.845 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-11-27T15:10:49.846 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-11-27T15:10:49.846 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-11-27T15:10:49.846 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-11-27T15:10:49.846 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-11-27T15:10:49.846 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-11-27T15:10:49.846 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-11-27T15:10:49.846 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-11-27T15:10:49.846 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.compression_opts.window_bits: -14 2023-11-27T15:10:49.846 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.compression_opts.level: 32767 2023-11-27T15:10:49.846 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.compression_opts.strategy: 0 2023-11-27T15:10:49.846 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-11-27T15:10:49.847 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-11-27T15:10:49.847 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-11-27T15:10:49.847 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-11-27T15:10:49.847 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.compression_opts.enabled: false 2023-11-27T15:10:49.847 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-11-27T15:10:49.847 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-11-27T15:10:49.847 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-11-27T15:10:49.847 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-11-27T15:10:49.848 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.target_file_size_base: 67108864 2023-11-27T15:10:49.848 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.target_file_size_multiplier: 1 2023-11-27T15:10:49.848 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-11-27T15:10:49.848 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-11-27T15:10:49.848 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-11-27T15:10:49.848 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-11-27T15:10:49.848 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-11-27T15:10:49.848 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-11-27T15:10:49.848 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-11-27T15:10:49.850 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-11-27T15:10:49.850 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-11-27T15:10:49.850 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-11-27T15:10:49.850 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-11-27T15:10:49.850 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-11-27T15:10:49.850 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-11-27T15:10:49.850 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.arena_block_size: 1048576 2023-11-27T15:10:49.850 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-11-27T15:10:49.850 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-11-27T15:10:49.850 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.disable_auto_compactions: 0 2023-11-27T15:10:49.851 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-11-27T15:10:49.851 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-11-27T15:10:49.851 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-11-27T15:10:49.851 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-11-27T15:10:49.851 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-11-27T15:10:49.851 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-11-27T15:10:49.851 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-11-27T15:10:49.851 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-11-27T15:10:49.851 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-11-27T15:10:49.851 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-11-27T15:10:49.851 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.table_properties_collectors: 2023-11-27T15:10:49.852 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.inplace_update_support: 0 2023-11-27T15:10:49.852 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.inplace_update_num_locks: 10000 2023-11-27T15:10:49.852 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-11-27T15:10:49.852 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-11-27T15:10:49.852 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.memtable_huge_page_size: 0 2023-11-27T15:10:49.852 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.bloom_locality: 0 2023-11-27T15:10:49.852 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.max_successive_merges: 0 2023-11-27T15:10:49.852 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.optimize_filters_for_hits: 0 2023-11-27T15:10:49.852 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.paranoid_file_checks: 0 2023-11-27T15:10:49.852 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.force_consistency_checks: 1 2023-11-27T15:10:49.852 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.report_bg_io_stats: 0 2023-11-27T15:10:49.852 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.ttl: 2592000 2023-11-27T15:10:49.853 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.periodic_compaction_seconds: 0 2023-11-27T15:10:49.853 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-11-27T15:10:49.853 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-11-27T15:10:49.853 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.enable_blob_files: false 2023-11-27T15:10:49.853 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.min_blob_size: 0 2023-11-27T15:10:49.853 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.blob_file_size: 268435456 2023-11-27T15:10:49.853 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.blob_compression_type: NoCompression 2023-11-27T15:10:49.853 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.enable_blob_garbage_collection: false 2023-11-27T15:10:49.853 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-11-27T15:10:49.853 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-11-27T15:10:49.853 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-11-27T15:10:49.853 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.blob_file_starting_level: 0 2023-11-27T15:10:49.854 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-11-27T15:10:49.854 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-11-27T15:10:49.854 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-11-27T15:10:49.854 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: aef852c7-4a3d-4bbd-b204-47327d95012d 2023-11-27T15:10:49.854 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701097849573327, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-11-27T15:10:49.854 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-11-27T15:10:49.854 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701097849573754, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1701097849, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "aef852c7-4a3d-4bbd-b204-47327d95012d", "db_session_id": "W8G7F9DBMD4IIPT41VRR", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-11-27T15:10:49.854 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701097849573834, "job": 1, "event": "recovery_finished"} 2023-11-27T15:10:49.854 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-11-27T15:10:49.854 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-11-27T15:10:49.854 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563521d1e000 2023-11-27T15:10:49.855 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: DB pointer 0x563521d0a000 2023-11-27T15:10:49.855 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-11-27T15:10:49.855 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-11-27T15:10:49.855 INFO:journalctl@ceph.mon.b.smithi102.stdout: ** DB Stats ** 2023-11-27T15:10:49.855 INFO:journalctl@ceph.mon.b.smithi102.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-27T15:10:49.855 INFO:journalctl@ceph.mon.b.smithi102.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-11-27T15:10:49.855 INFO:journalctl@ceph.mon.b.smithi102.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-27T15:10:49.855 INFO:journalctl@ceph.mon.b.smithi102.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-27T15:10:49.855 INFO:journalctl@ceph.mon.b.smithi102.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-11-27T15:10:49.855 INFO:journalctl@ceph.mon.b.smithi102.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-27T15:10:49.855 INFO:journalctl@ceph.mon.b.smithi102.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-27T15:10:49.856 INFO:journalctl@ceph.mon.b.smithi102.stdout: 2023-11-27T15:10:49.856 INFO:journalctl@ceph.mon.b.smithi102.stdout: ** Compaction Stats [default] ** 2023-11-27T15:10:49.856 INFO:journalctl@ceph.mon.b.smithi102.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-27T15:10:49.856 INFO:journalctl@ceph.mon.b.smithi102.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-11-27T15:10:49.856 INFO:journalctl@ceph.mon.b.smithi102.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-27T15:10:49.856 INFO:journalctl@ceph.mon.b.smithi102.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-27T15:10:49.856 INFO:journalctl@ceph.mon.b.smithi102.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-27T15:10:49.856 INFO:journalctl@ceph.mon.b.smithi102.stdout: 2023-11-27T15:10:49.856 INFO:journalctl@ceph.mon.b.smithi102.stdout: ** Compaction Stats [default] ** 2023-11-27T15:10:49.857 INFO:journalctl@ceph.mon.b.smithi102.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-27T15:10:49.857 INFO:journalctl@ceph.mon.b.smithi102.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-11-27T15:10:49.857 INFO:journalctl@ceph.mon.b.smithi102.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-27T15:10:49.857 INFO:journalctl@ceph.mon.b.smithi102.stdout: 2023-11-27T15:10:49.857 INFO:journalctl@ceph.mon.b.smithi102.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-11-27T15:10:49.857 INFO:journalctl@ceph.mon.b.smithi102.stdout: 2023-11-27T15:10:49.857 INFO:journalctl@ceph.mon.b.smithi102.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-27T15:10:49.857 INFO:journalctl@ceph.mon.b.smithi102.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-11-27T15:10:49.857 INFO:journalctl@ceph.mon.b.smithi102.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-11-27T15:10:49.857 INFO:journalctl@ceph.mon.b.smithi102.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-11-27T15:10:49.857 INFO:journalctl@ceph.mon.b.smithi102.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-11-27T15:10:49.857 INFO:journalctl@ceph.mon.b.smithi102.stdout: AddFile(Keys): cumulative 0, interval 0 2023-11-27T15:10:49.858 INFO:journalctl@ceph.mon.b.smithi102.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-27T15:10:49.858 INFO:journalctl@ceph.mon.b.smithi102.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-27T15:10:49.858 INFO:journalctl@ceph.mon.b.smithi102.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-11-27T15:10:49.858 INFO:journalctl@ceph.mon.b.smithi102.stdout: Block cache BinnedLRUCache@0x563521c811f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2023-11-27T15:10:49.858 INFO:journalctl@ceph.mon.b.smithi102.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-11-27T15:10:49.858 INFO:journalctl@ceph.mon.b.smithi102.stdout: 2023-11-27T15:10:49.858 INFO:journalctl@ceph.mon.b.smithi102.stdout: ** File Read Latency Histogram By Level [default] ** 2023-11-27T15:10:49.858 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-11-27T15:10:49.858 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: using public_addr v2:172.21.15.102:0/0 -> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] 2023-11-27T15:10:49.858 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: starting mon.b rank -1 at public addrs [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] at bind addrs [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid c6c7df88-8d36-11ee-95a2-87774f69a715 2023-11-27T15:10:49.859 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: mon.b@-1(???) e0 preinit fsid c6c7df88-8d36-11ee-95a2-87774f69a715 2023-11-27T15:10:49.859 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: mon.b@-1(synchronizing).mds e1 new map 2023-11-27T15:10:49.859 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: mon.b@-1(synchronizing).mds e1 print_map 2023-11-27T15:10:49.859 INFO:journalctl@ceph.mon.b.smithi102.stdout: e1 2023-11-27T15:10:49.859 INFO:journalctl@ceph.mon.b.smithi102.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-11-27T15:10:49.859 INFO:journalctl@ceph.mon.b.smithi102.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-27T15:10:49.859 INFO:journalctl@ceph.mon.b.smithi102.stdout: legacy client fscid: -1 2023-11-27T15:10:49.859 INFO:journalctl@ceph.mon.b.smithi102.stdout: 2023-11-27T15:10:49.859 INFO:journalctl@ceph.mon.b.smithi102.stdout: No filesystems configured 2023-11-27T15:10:49.859 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-11-27T15:10:49.859 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-11-27T15:10:49.859 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-11-27T15:10:49.859 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-11-27T15:10:49.859 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-11-27T15:10:49.860 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-11-27T15:10:49.860 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-11-27T15:10:49.860 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-27T15:10:49.860 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-27T15:10:49.860 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-27T15:10:49.860 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:49.860 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi150", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:10:49.860 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:49.860 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:49.860 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:49.860 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:49.860 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi102", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:10:49.861 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:10:49.861 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:10:49.861 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:10:49.861 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: Updating smithi102:/etc/ceph/ceph.conf 2023-11-27T15:10:49.861 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: Updating smithi102:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:10:49.861 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: Deploying cephadm binary to smithi150 2023-11-27T15:10:49.861 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:10:49.861 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: Updating smithi102:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-11-27T15:10:49.861 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:49.861 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:49.861 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:49.861 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:49.862 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:49.862 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: Added host smithi150 2023-11-27T15:10:49.862 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:10:49.862 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:49.862 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:49.862 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:49.862 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:10:49.862 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:49.862 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:49.862 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2229004337' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-11-27T15:10:49.862 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2229004337' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-11-27T15:10:49.862 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:10:49.862 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:49.863 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:49.863 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:49.863 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:49.863 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:49.863 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:10:49.863 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:10:49.863 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:10:49.863 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:10:49.863 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: Updating smithi150:/etc/ceph/ceph.conf 2023-11-27T15:10:49.863 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: Updating smithi150:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:10:49.863 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:10:49.863 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: Updating smithi150:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-11-27T15:10:49.864 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:49.864 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:49.864 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:49.864 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:49.864 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:49.864 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:10:49.864 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:10:49.864 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:10:49.864 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:49.864 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:10:49.864 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:10:49.865 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi052:172.21.15.52=a;smithi102:172.21.15.102=b;smithi150:172.21.15.150=c", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:10:49.865 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: Saving service mon spec with placement smithi052:172.21.15.52=a;smithi102:172.21.15.102=b;smithi150:172.21.15.150=c;count:3 2023-11-27T15:10:49.865 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:10:49.865 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:10:49.865 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: Deploying daemon mon.c on smithi150 2023-11-27T15:10:49.865 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:49.865 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:10:49.865 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:10:49.865 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: mon.a calling monitor election 2023-11-27T15:10:49.865 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:49.865 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:10:49.865 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:10:49.866 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: mon.c calling monitor election 2023-11-27T15:10:49.866 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:49.866 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:10:49.866 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:10:49.866 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:49.866 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:10:49.866 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-27T15:10:49.866 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: monmap e2: 2 mons at {a=[v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0],c=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0]} removed_ranks: {} 2023-11-27T15:10:49.866 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: fsmap 2023-11-27T15:10:49.866 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:10:49.866 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: mgrmap e13: x(active, since 42s) 2023-11-27T15:10:49.867 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: overall HEALTH_OK 2023-11-27T15:10:49.867 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:49.867 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:49.867 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:49.867 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:10:49.867 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:10:49.867 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: Deploying daemon mon.b on smithi102 2023-11-27T15:10:49.867 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:10:49.867 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:49.868 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: from='client.? 172.21.15.150:0/4075601112' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-27T15:10:49.868 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-11-27T15:10:49.868 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: expand_channel_meta expand map: {default=false} 2023-11-27T15:10:49.868 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: expand_channel_meta from 'false' to 'false' 2023-11-27T15:10:49.868 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: expand_channel_meta expanded map: {default=false} 2023-11-27T15:10:49.868 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: expand_channel_meta expand map: {default=info} 2023-11-27T15:10:49.868 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: expand_channel_meta from 'info' to 'info' 2023-11-27T15:10:49.868 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: expand_channel_meta expanded map: {default=info} 2023-11-27T15:10:49.868 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: expand_channel_meta expand map: {default=daemon} 2023-11-27T15:10:49.868 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: expand_channel_meta from 'daemon' to 'daemon' 2023-11-27T15:10:49.869 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: expand_channel_meta expanded map: {default=daemon} 2023-11-27T15:10:49.869 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: expand_channel_meta expand map: {default=debug} 2023-11-27T15:10:49.869 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: expand_channel_meta from 'debug' to 'debug' 2023-11-27T15:10:49.869 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: expand_channel_meta expanded map: {default=debug} 2023-11-27T15:10:49.869 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 ceph-mon[100296]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-11-27T15:10:49.869 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:49 smithi102 systemd[1]: Started Ceph mon.b for c6c7df88-8d36-11ee-95a2-87774f69a715. 2023-11-27T15:10:49.869 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:10:54.672 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.666+0000 7f82b6173700 1 -- 172.21.15.150:0/1517755773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82980035c0 msgr2=0x7f8298005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:54.672 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.666+0000 7f82b6173700 1 --2- 172.21.15.150:0/1517755773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82980035c0 0x7f8298005a40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f82b00fec90 tx=0x7f82a4014ec0 comp rx=0 tx=0).stop 2023-11-27T15:10:54.672 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.666+0000 7f82b6173700 1 -- 172.21.15.150:0/1517755773 shutdown_connections 2023-11-27T15:10:54.672 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.666+0000 7f82b6173700 1 --2- 172.21.15.150:0/1517755773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82980035c0 0x7f8298005a40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:54.673 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.666+0000 7f82b6173700 1 -- 172.21.15.150:0/1517755773 >> 172.21.15.150:0/1517755773 conn(0x7f82b00fa8a0 msgr2=0x7f82b00fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:54.673 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.666+0000 7f82b6173700 1 -- 172.21.15.150:0/1517755773 shutdown_connections 2023-11-27T15:10:54.673 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.666+0000 7f82b6173700 1 -- 172.21.15.150:0/1517755773 wait complete. 2023-11-27T15:10:54.676 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.667+0000 7f82b6173700 1 Processor -- start 2023-11-27T15:10:54.676 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.667+0000 7f82b6173700 1 -- start start 2023-11-27T15:10:54.676 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.668+0000 7f82b6173700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82980035c0 0x7f82b01009d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:54.676 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.668+0000 7f82b6173700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82b0100f10 0x7f82b0101380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:54.676 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.668+0000 7f82b6173700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82b01a1580 0x7f82b01a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:54.677 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.668+0000 7f82b6173700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b01a3fb0 con 0x7f82b01a1580 2023-11-27T15:10:54.677 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.668+0000 7f82b6173700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b01a4120 con 0x7f82980035c0 2023-11-27T15:10:54.677 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.668+0000 7f82b6173700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b01a42e0 con 0x7f82b0100f10 2023-11-27T15:10:54.677 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.668+0000 7f82aeffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82b0100f10 0x7f82b0101380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:54.677 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.668+0000 7f82aeffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82b0100f10 0x7f82b0101380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49336/0 (socket says 172.21.15.150:49336) 2023-11-27T15:10:54.677 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.668+0000 7f82aeffd700 1 -- 172.21.15.150:0/2872054896 learned_addr learned my addr 172.21.15.150:0/2872054896 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-27T15:10:54.677 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.668+0000 7f82aeffd700 1 -- 172.21.15.150:0/2872054896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82b0100f10 msgr2=0x7f82b0101380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-11-27T15:10:54.678 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.668+0000 7f82aeffd700 1 -- 172.21.15.150:0/2872054896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82b0100f10 msgr2=0x7f82b0101380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-27T15:10:54.678 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.668+0000 7f82aeffd700 1 --2- 172.21.15.150:0/2872054896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82b0100f10 0x7f82b0101380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-27T15:10:54.678 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.668+0000 7f82af7fe700 1 --2- 172.21.15.150:0/2872054896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82980035c0 0x7f82b01009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:54.678 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.668+0000 7f82aeffd700 1 --2- 172.21.15.150:0/2872054896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82b0100f10 0x7f82b0101380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-27T15:10:54.678 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.668+0000 7f82affff700 1 --2- 172.21.15.150:0/2872054896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82b01a1580 0x7f82b01a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:54.678 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.668+0000 7f82af7fe700 1 -- 172.21.15.150:0/2872054896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82980035c0 msgr2=0x7f82b01009d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-11-27T15:10:54.679 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.668+0000 7f82af7fe700 1 -- 172.21.15.150:0/2872054896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82980035c0 msgr2=0x7f82b01009d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-27T15:10:54.679 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.668+0000 7f82af7fe700 1 --2- 172.21.15.150:0/2872054896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82980035c0 0x7f82b01009d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-27T15:10:54.679 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.668+0000 7f82af7fe700 1 --2- 172.21.15.150:0/2872054896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82980035c0 0x7f82b01009d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-27T15:10:54.679 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.668+0000 7f82affff700 1 -- 172.21.15.150:0/2872054896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82b0100f10 msgr2=0x7f82b0101380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-27T15:10:54.679 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.668+0000 7f82affff700 1 --2- 172.21.15.150:0/2872054896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82b0100f10 0x7f82b0101380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:54.679 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.668+0000 7f82affff700 1 -- 172.21.15.150:0/2872054896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82980035c0 msgr2=0x7f82b01009d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-27T15:10:54.679 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.668+0000 7f82affff700 1 --2- 172.21.15.150:0/2872054896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82980035c0 0x7f82b01009d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:54.680 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.668+0000 7f82affff700 1 -- 172.21.15.150:0/2872054896 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a4014910 con 0x7f82b01a1580 2023-11-27T15:10:54.680 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.669+0000 7f82affff700 1 --2- 172.21.15.150:0/2872054896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82b01a1580 0x7f82b01a3960 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f82a800bd50 tx=0x7f82a800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:54.680 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.669+0000 7f82acff9700 1 -- 172.21.15.150:0/2872054896 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a800cd10 con 0x7f82b01a1580 2023-11-27T15:10:54.680 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.669+0000 7f82acff9700 1 -- 172.21.15.150:0/2872054896 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82a800ce70 con 0x7f82b01a1580 2023-11-27T15:10:54.680 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.669+0000 7f82b6173700 1 -- 172.21.15.150:0/2872054896 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b0106b60 con 0x7f82b01a1580 2023-11-27T15:10:54.680 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.669+0000 7f82acff9700 1 -- 172.21.15.150:0/2872054896 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a8012840 con 0x7f82b01a1580 2023-11-27T15:10:54.680 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.669+0000 7f82b6173700 1 -- 172.21.15.150:0/2872054896 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b01069d0 con 0x7f82b01a1580 2023-11-27T15:10:54.681 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.670+0000 7f82acff9700 1 -- 172.21.15.150:0/2872054896 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f82a80129c0 con 0x7f82b01a1580 2023-11-27T15:10:54.681 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.670+0000 7f82acff9700 1 --2- 172.21.15.150:0/2872054896 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f829c038d40 0x7f829c03b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:54.681 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.670+0000 7f82acff9700 1 -- 172.21.15.150:0/2872054896 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f82a804ae10 con 0x7f82b01a1580 2023-11-27T15:10:54.681 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.670+0000 7f82b6173700 1 -- 172.21.15.150:0/2872054896 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b0106cf0 con 0x7f82b01a1580 2023-11-27T15:10:54.681 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.671+0000 7f82af7fe700 1 --2- 172.21.15.150:0/2872054896 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f829c038d40 0x7f829c03b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:54.681 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.671+0000 7f82af7fe700 1 --2- 172.21.15.150:0/2872054896 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f829c038d40 0x7f829c03b1f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f82a4010010 tx=0x7f82a4001e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:54.682 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.676+0000 7f82acff9700 1 -- 172.21.15.150:0/2872054896 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f82a8019070 con 0x7f82b01a1580 2023-11-27T15:10:54.959 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.954+0000 7f82b6173700 1 -- 172.21.15.150:0/2872054896 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f82b0101fa0 con 0x7f82b01a1580 2023-11-27T15:10:54.960 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.955+0000 7f82acff9700 1 -- 172.21.15.150:0/2872054896 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1274 (secure 0 0 0) 0x7f82a801d3b0 con 0x7f82b01a1580 2023-11-27T15:10:54.960 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-27T15:10:54.960 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":3,"fsid":"c6c7df88-8d36-11ee-95a2-87774f69a715","modified":"2023-11-27T15:10:49.596916Z","created":"2023-11-27T15:08:59.495725Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:3300","nonce":0},{"type":"v1","addr":"172.21.15.52:6789","nonce":0}]},"addr":"172.21.15.52:6789/0","public_addr":"172.21.15.52:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:3300","nonce":0},{"type":"v1","addr":"172.21.15.150:6789","nonce":0}]},"addr":"172.21.15.150:6789/0","public_addr":"172.21.15.150:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-11-27T15:10:54.962 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.957+0000 7f82b6173700 1 -- 172.21.15.150:0/2872054896 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f829c038d40 msgr2=0x7f829c03b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:54.962 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.957+0000 7f82b6173700 1 --2- 172.21.15.150:0/2872054896 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f829c038d40 0x7f829c03b1f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f82a4010010 tx=0x7f82a4001e50 comp rx=0 tx=0).stop 2023-11-27T15:10:54.962 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.957+0000 7f82b6173700 1 -- 172.21.15.150:0/2872054896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82b01a1580 msgr2=0x7f82b01a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:54.962 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.957+0000 7f82b6173700 1 --2- 172.21.15.150:0/2872054896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82b01a1580 0x7f82b01a3960 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f82a800bd50 tx=0x7f82a800be30 comp rx=0 tx=0).stop 2023-11-27T15:10:54.963 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.957+0000 7f82b6173700 1 -- 172.21.15.150:0/2872054896 shutdown_connections 2023-11-27T15:10:54.963 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.957+0000 7f82b6173700 1 --2- 172.21.15.150:0/2872054896 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f829c038d40 0x7f829c03b1f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:54.963 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.957+0000 7f82b6173700 1 --2- 172.21.15.150:0/2872054896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82980035c0 0x7f82b01009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:54.963 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.957+0000 7f82b6173700 1 --2- 172.21.15.150:0/2872054896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82b01a1580 0x7f82b01a3960 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:54.963 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.957+0000 7f82b6173700 1 --2- 172.21.15.150:0/2872054896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82b0100f10 0x7f82b0101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:54.963 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.957+0000 7f82b6173700 1 -- 172.21.15.150:0/2872054896 >> 172.21.15.150:0/2872054896 conn(0x7f82b00fa8a0 msgr2=0x7f82b00fbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:54.963 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.957+0000 7f82b6173700 1 -- 172.21.15.150:0/2872054896 shutdown_connections 2023-11-27T15:10:54.963 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:54.957+0000 7f82b6173700 1 -- 172.21.15.150:0/2872054896 wait complete. 2023-11-27T15:10:54.963 INFO:teuthology.orchestra.run.smithi150.stderr:dumped monmap epoch 3 2023-11-27T15:10:54.994 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:54 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:10:54.994 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:54 smithi150 ceph-mon[100559]: mon.a calling monitor election 2023-11-27T15:10:54.994 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:54 smithi150 ceph-mon[100559]: mon.c calling monitor election 2023-11-27T15:10:54.994 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:54 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:10:54.994 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:54 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:10:54.994 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:54 smithi150 ceph-mon[100559]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:54.995 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:54 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:10:54.995 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:54 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:10:54.995 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:54 smithi150 ceph-mon[100559]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:54.995 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:54 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:10:54.995 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:54 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:10:54.995 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:54 smithi150 ceph-mon[100559]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:54.995 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:54 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:10:54.995 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:54 smithi150 ceph-mon[100559]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-27T15:10:54.995 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:54 smithi150 ceph-mon[100559]: monmap e3: 3 mons at {a=[v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0],b=[v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0],c=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0]} removed_ranks: {} 2023-11-27T15:10:54.995 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:54 smithi150 ceph-mon[100559]: fsmap 2023-11-27T15:10:54.995 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:54 smithi150 ceph-mon[100559]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:10:54.995 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:54 smithi150 ceph-mon[100559]: mgrmap e13: x(active, since 50s) 2023-11-27T15:10:54.996 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:54 smithi150 ceph-mon[100559]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-11-27T15:10:54.996 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:54 smithi150 ceph-mon[100559]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-11-27T15:10:54.996 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:54 smithi150 ceph-mon[100559]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-11-27T15:10:54.996 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:54 smithi150 ceph-mon[100559]: mon.b (rank 2) addr [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] is down (out of quorum) 2023-11-27T15:10:54.996 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:54 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:54.996 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:54 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:55.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:54 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:10:55.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:54 smithi052 ceph-mon[99056]: mon.a calling monitor election 2023-11-27T15:10:55.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:54 smithi052 ceph-mon[99056]: mon.c calling monitor election 2023-11-27T15:10:55.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:54 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:10:55.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:54 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:10:55.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:54 smithi052 ceph-mon[99056]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:55.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:54 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:10:55.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:54 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:10:55.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:54 smithi052 ceph-mon[99056]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:55.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:54 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:10:55.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:54 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:10:55.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:54 smithi052 ceph-mon[99056]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:55.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:54 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:10:55.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:54 smithi052 ceph-mon[99056]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-27T15:10:55.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:54 smithi052 ceph-mon[99056]: monmap e3: 3 mons at {a=[v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0],b=[v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0],c=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0]} removed_ranks: {} 2023-11-27T15:10:55.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:54 smithi052 ceph-mon[99056]: fsmap 2023-11-27T15:10:55.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:54 smithi052 ceph-mon[99056]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:10:55.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:54 smithi052 ceph-mon[99056]: mgrmap e13: x(active, since 50s) 2023-11-27T15:10:55.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:54 smithi052 ceph-mon[99056]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-11-27T15:10:55.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:54 smithi052 ceph-mon[99056]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-11-27T15:10:55.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:54 smithi052 ceph-mon[99056]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-11-27T15:10:55.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:54 smithi052 ceph-mon[99056]: mon.b (rank 2) addr [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] is down (out of quorum) 2023-11-27T15:10:55.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:54 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:55.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:54 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:55.301 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-11-27T15:10:55.302 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph config generate-minimal-conf 2023-11-27T15:10:55.971 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:10:56.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:55 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:56.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:55 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:56.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:55 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:10:56.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:55 smithi150 ceph-mon[100559]: from='client.? 172.21.15.150:0/2872054896' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-27T15:10:56.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:55 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:10:56.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:55 smithi150 ceph-mon[100559]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:56.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:55 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:56.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:55 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:56.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:55 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:10:56.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:55 smithi052 ceph-mon[99056]: from='client.? 172.21.15.150:0/2872054896' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-27T15:10:56.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:55 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:10:56.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:55 smithi052 ceph-mon[99056]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:57.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:56 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:10:57.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:56 smithi102 ceph-mon[100296]: mon.a calling monitor election 2023-11-27T15:10:57.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:56 smithi102 ceph-mon[100296]: mon.c calling monitor election 2023-11-27T15:10:57.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:56 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:10:57.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:56 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:10:57.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:56 smithi102 ceph-mon[100296]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:57.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:56 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:10:57.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:56 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:10:57.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:56 smithi102 ceph-mon[100296]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:57.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:56 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:10:57.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:56 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:10:57.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:56 smithi102 ceph-mon[100296]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:57.080 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:56 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:10:57.080 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:56 smithi102 ceph-mon[100296]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-27T15:10:57.080 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:56 smithi102 ceph-mon[100296]: monmap e3: 3 mons at {a=[v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0],b=[v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0],c=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0]} removed_ranks: {} 2023-11-27T15:10:57.080 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:56 smithi102 ceph-mon[100296]: fsmap 2023-11-27T15:10:57.080 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:56 smithi102 ceph-mon[100296]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:10:57.080 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:56 smithi102 ceph-mon[100296]: mgrmap e13: x(active, since 50s) 2023-11-27T15:10:57.080 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:56 smithi102 ceph-mon[100296]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-11-27T15:10:57.080 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:56 smithi102 ceph-mon[100296]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-11-27T15:10:57.080 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:56 smithi102 ceph-mon[100296]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-11-27T15:10:57.080 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:56 smithi102 ceph-mon[100296]: mon.b (rank 2) addr [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] is down (out of quorum) 2023-11-27T15:10:57.080 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:56 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:57.080 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:56 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:57.081 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:56 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:57.081 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:56 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:57.081 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:56 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:10:57.081 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:56 smithi102 ceph-mon[100296]: from='client.? 172.21.15.150:0/2872054896' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-27T15:10:57.081 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:56 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:10:57.081 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:56 smithi102 ceph-mon[100296]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:57.196 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.190+0000 7f2cacbfc700 1 -- 172.21.15.52:0/4230260666 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ca8100e10 msgr2=0x7f2ca81011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:57.196 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.190+0000 7f2cacbfc700 1 --2- 172.21.15.52:0/4230260666 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ca8100e10 0x7f2ca81011e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2c98009910 tx=0x7f2c98009c20 comp rx=0 tx=0).stop 2023-11-27T15:10:57.196 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.190+0000 7f2cacbfc700 1 -- 172.21.15.52:0/4230260666 shutdown_connections 2023-11-27T15:10:57.197 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.190+0000 7f2cacbfc700 1 --2- 172.21.15.52:0/4230260666 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ca8100e10 0x7f2ca81011e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:57.197 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.190+0000 7f2cacbfc700 1 -- 172.21.15.52:0/4230260666 >> 172.21.15.52:0/4230260666 conn(0x7f2ca8074930 msgr2=0x7f2ca8074d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:57.197 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.190+0000 7f2cacbfc700 1 -- 172.21.15.52:0/4230260666 shutdown_connections 2023-11-27T15:10:57.197 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.190+0000 7f2cacbfc700 1 -- 172.21.15.52:0/4230260666 wait complete. 2023-11-27T15:10:57.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.191+0000 7f2cacbfc700 1 Processor -- start 2023-11-27T15:10:57.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.191+0000 7f2cacbfc700 1 -- start start 2023-11-27T15:10:57.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.191+0000 7f2cacbfc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ca8100e10 0x7f2ca8195380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:57.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.191+0000 7f2cacbfc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ca81958c0 0x7f2ca8199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:57.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.191+0000 7f2cacbfc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ca819a270 0x7f2ca819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:57.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.191+0000 7f2cacbfc700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca8195f90 con 0x7f2ca8100e10 2023-11-27T15:10:57.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.191+0000 7f2cacbfc700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca81a1fe0 con 0x7f2ca819a270 2023-11-27T15:10:57.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.191+0000 7f2cacbfc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca81a2120 con 0x7f2ca81958c0 2023-11-27T15:10:57.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.191+0000 7f2ca659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ca8100e10 0x7f2ca8195380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:57.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.192+0000 7f2ca659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ca8100e10 0x7f2ca8195380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49120/0 (socket says 172.21.15.52:49120) 2023-11-27T15:10:57.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.192+0000 7f2ca659c700 1 -- 172.21.15.52:0/2556036621 learned_addr learned my addr 172.21.15.52:0/2556036621 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:10:57.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.192+0000 7f2ca6d9d700 1 --2- 172.21.15.52:0/2556036621 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ca819a270 0x7f2ca819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:57.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.192+0000 7f2ca5d9b700 1 --2- 172.21.15.52:0/2556036621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ca81958c0 0x7f2ca8199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:57.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.192+0000 7f2ca659c700 1 -- 172.21.15.52:0/2556036621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ca81958c0 msgr2=0x7f2ca8199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:57.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.192+0000 7f2ca659c700 1 --2- 172.21.15.52:0/2556036621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ca81958c0 0x7f2ca8199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:57.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.192+0000 7f2ca659c700 1 -- 172.21.15.52:0/2556036621 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ca819a270 msgr2=0x7f2ca819a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:57.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.192+0000 7f2ca659c700 1 --2- 172.21.15.52:0/2556036621 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ca819a270 0x7f2ca819a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:57.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.192+0000 7f2ca659c700 1 -- 172.21.15.52:0/2556036621 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c980095f0 con 0x7f2ca8100e10 2023-11-27T15:10:57.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.192+0000 7f2ca5d9b700 1 --2- 172.21.15.52:0/2556036621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ca81958c0 0x7f2ca8199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:10:57.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.192+0000 7f2ca659c700 1 --2- 172.21.15.52:0/2556036621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ca8100e10 0x7f2ca8195380 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2c98004cc0 tx=0x7f2c98004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:57.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.192+0000 7f2c977fe700 1 -- 172.21.15.52:0/2556036621 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9801d070 con 0x7f2ca8100e10 2023-11-27T15:10:57.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.192+0000 7f2cacbfc700 1 -- 172.21.15.52:0/2556036621 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ca810a150 con 0x7f2ca8100e10 2023-11-27T15:10:57.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.192+0000 7f2c977fe700 1 -- 172.21.15.52:0/2556036621 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c9800be20 con 0x7f2ca8100e10 2023-11-27T15:10:57.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.192+0000 7f2c977fe700 1 -- 172.21.15.52:0/2556036621 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9800f460 con 0x7f2ca8100e10 2023-11-27T15:10:57.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.192+0000 7f2cacbfc700 1 -- 172.21.15.52:0/2556036621 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ca8109fc0 con 0x7f2ca8100e10 2023-11-27T15:10:57.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.193+0000 7f2c977fe700 1 -- 172.21.15.52:0/2556036621 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f2c9800f5c0 con 0x7f2ca8100e10 2023-11-27T15:10:57.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.193+0000 7f2c977fe700 1 --2- 172.21.15.52:0/2556036621 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f2c90038cf0 0x7f2c9003b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:57.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.193+0000 7f2cacbfc700 1 -- 172.21.15.52:0/2556036621 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ca810a2e0 con 0x7f2ca8100e10 2023-11-27T15:10:57.202 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.193+0000 7f2c977fe700 1 -- 172.21.15.52:0/2556036621 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2c9804c7d0 con 0x7f2ca8100e10 2023-11-27T15:10:57.202 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.193+0000 7f2ca5d9b700 1 --2- 172.21.15.52:0/2556036621 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f2c90038cf0 0x7f2c9003b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:57.202 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.194+0000 7f2ca5d9b700 1 --2- 172.21.15.52:0/2556036621 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f2c90038cf0 0x7f2c9003b1a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009910 tx=0x7f2c9c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:57.204 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.198+0000 7f2c977fe700 1 -- 172.21.15.52:0/2556036621 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2c98020730 con 0x7f2ca8100e10 2023-11-27T15:10:57.405 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.399+0000 7f2cacbfc700 1 -- 172.21.15.52:0/2556036621 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f2ca8107730 con 0x7f2ca8100e10 2023-11-27T15:10:57.405 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.399+0000 7f2c977fe700 1 -- 172.21.15.52:0/2556036621 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7f2c98027070 con 0x7f2ca8100e10 2023-11-27T15:10:57.405 INFO:teuthology.orchestra.run.smithi052.stdout:# minimal ceph.conf for c6c7df88-8d36-11ee-95a2-87774f69a715 2023-11-27T15:10:57.405 INFO:teuthology.orchestra.run.smithi052.stdout:[global] 2023-11-27T15:10:57.406 INFO:teuthology.orchestra.run.smithi052.stdout: fsid = c6c7df88-8d36-11ee-95a2-87774f69a715 2023-11-27T15:10:57.406 INFO:teuthology.orchestra.run.smithi052.stdout: mon_host = [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] 2023-11-27T15:10:57.407 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.401+0000 7f2cacbfc700 1 -- 172.21.15.52:0/2556036621 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f2c90038cf0 msgr2=0x7f2c9003b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:57.408 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.401+0000 7f2cacbfc700 1 --2- 172.21.15.52:0/2556036621 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f2c90038cf0 0x7f2c9003b1a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009910 tx=0x7f2c9c008040 comp rx=0 tx=0).stop 2023-11-27T15:10:57.408 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.401+0000 7f2cacbfc700 1 -- 172.21.15.52:0/2556036621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ca8100e10 msgr2=0x7f2ca8195380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:57.408 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.401+0000 7f2cacbfc700 1 --2- 172.21.15.52:0/2556036621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ca8100e10 0x7f2ca8195380 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2c98004cc0 tx=0x7f2c98004cf0 comp rx=0 tx=0).stop 2023-11-27T15:10:57.408 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.401+0000 7f2cacbfc700 1 -- 172.21.15.52:0/2556036621 shutdown_connections 2023-11-27T15:10:57.408 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.401+0000 7f2cacbfc700 1 --2- 172.21.15.52:0/2556036621 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f2c90038cf0 0x7f2c9003b1a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:57.408 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.401+0000 7f2cacbfc700 1 --2- 172.21.15.52:0/2556036621 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ca819a270 0x7f2ca819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:57.408 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.401+0000 7f2cacbfc700 1 --2- 172.21.15.52:0/2556036621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ca8100e10 0x7f2ca8195380 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:57.408 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.401+0000 7f2cacbfc700 1 --2- 172.21.15.52:0/2556036621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ca81958c0 0x7f2ca8199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:57.408 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.401+0000 7f2cacbfc700 1 -- 172.21.15.52:0/2556036621 >> 172.21.15.52:0/2556036621 conn(0x7f2ca8074930 msgr2=0x7f2ca8104870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:57.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.402+0000 7f2cacbfc700 1 -- 172.21.15.52:0/2556036621 shutdown_connections 2023-11-27T15:10:57.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:10:57.402+0000 7f2cacbfc700 1 -- 172.21.15.52:0/2556036621 wait complete. 2023-11-27T15:10:57.744 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:57 smithi052 ceph-mon[99056]: mon.b calling monitor election 2023-11-27T15:10:57.744 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:57 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:10:57.744 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:57 smithi052 ceph-mon[99056]: Updating smithi052:/etc/ceph/ceph.conf 2023-11-27T15:10:57.745 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:57 smithi052 ceph-mon[99056]: Updating smithi102:/etc/ceph/ceph.conf 2023-11-27T15:10:57.745 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:57 smithi052 ceph-mon[99056]: Updating smithi150:/etc/ceph/ceph.conf 2023-11-27T15:10:57.745 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:57 smithi052 ceph-mon[99056]: mon.b calling monitor election 2023-11-27T15:10:57.745 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:57 smithi052 ceph-mon[99056]: mon.a calling monitor election 2023-11-27T15:10:57.745 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:57 smithi052 ceph-mon[99056]: mon.c calling monitor election 2023-11-27T15:10:57.745 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:57 smithi052 ceph-mon[99056]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-11-27T15:10:57.745 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:57 smithi052 ceph-mon[99056]: monmap e3: 3 mons at {a=[v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0],b=[v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0],c=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0]} removed_ranks: {} 2023-11-27T15:10:57.745 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:57 smithi052 ceph-mon[99056]: fsmap 2023-11-27T15:10:57.745 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:57 smithi052 ceph-mon[99056]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:10:57.745 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:57 smithi052 ceph-mon[99056]: mgrmap e13: x(active, since 53s) 2023-11-27T15:10:57.745 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:57 smithi052 ceph-mon[99056]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-11-27T15:10:57.746 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:57 smithi052 ceph-mon[99056]: Cluster is now healthy 2023-11-27T15:10:57.746 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:57 smithi052 ceph-mon[99056]: overall HEALTH_OK 2023-11-27T15:10:57.746 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:57 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:57.746 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:57 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:57.746 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:57 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:57.746 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:57 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:57.746 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:57 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:57.746 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:57 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:57.746 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:57 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:57.746 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:57 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:10:57.746 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:57 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-27T15:10:57.747 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:57 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:10:57.747 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:57 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2556036621' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:10:57.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:57 smithi150 ceph-mon[100559]: mon.b calling monitor election 2023-11-27T15:10:57.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:57 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:10:57.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:57 smithi150 ceph-mon[100559]: Updating smithi052:/etc/ceph/ceph.conf 2023-11-27T15:10:57.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:57 smithi150 ceph-mon[100559]: Updating smithi102:/etc/ceph/ceph.conf 2023-11-27T15:10:57.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:57 smithi150 ceph-mon[100559]: Updating smithi150:/etc/ceph/ceph.conf 2023-11-27T15:10:57.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:57 smithi150 ceph-mon[100559]: mon.b calling monitor election 2023-11-27T15:10:57.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:57 smithi150 ceph-mon[100559]: mon.a calling monitor election 2023-11-27T15:10:57.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:57 smithi150 ceph-mon[100559]: mon.c calling monitor election 2023-11-27T15:10:57.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:57 smithi150 ceph-mon[100559]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-11-27T15:10:57.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:57 smithi150 ceph-mon[100559]: monmap e3: 3 mons at {a=[v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0],b=[v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0],c=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0]} removed_ranks: {} 2023-11-27T15:10:57.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:57 smithi150 ceph-mon[100559]: fsmap 2023-11-27T15:10:57.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:57 smithi150 ceph-mon[100559]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:10:57.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:57 smithi150 ceph-mon[100559]: mgrmap e13: x(active, since 53s) 2023-11-27T15:10:57.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:57 smithi150 ceph-mon[100559]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-11-27T15:10:57.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:57 smithi150 ceph-mon[100559]: Cluster is now healthy 2023-11-27T15:10:57.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:57 smithi150 ceph-mon[100559]: overall HEALTH_OK 2023-11-27T15:10:57.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:57 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:57.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:57 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:57.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:57 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:57.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:57 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:57.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:57 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:57.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:57 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:57.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:57 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:57.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:57 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:10:57.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:57 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-27T15:10:57.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:57 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:10:57.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:57 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2556036621' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:10:57.825 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-11-27T15:10:57.825 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2023-11-27T15:10:57.825 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/ceph/ceph.conf 2023-11-27T15:10:57.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:57 smithi102 ceph-mon[100296]: mon.b calling monitor election 2023-11-27T15:10:57.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:57 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:10:57.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:57 smithi102 ceph-mon[100296]: Updating smithi052:/etc/ceph/ceph.conf 2023-11-27T15:10:57.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:57 smithi102 ceph-mon[100296]: Updating smithi102:/etc/ceph/ceph.conf 2023-11-27T15:10:57.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:57 smithi102 ceph-mon[100296]: Updating smithi150:/etc/ceph/ceph.conf 2023-11-27T15:10:57.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:57 smithi102 ceph-mon[100296]: mon.b calling monitor election 2023-11-27T15:10:57.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:57 smithi102 ceph-mon[100296]: mon.a calling monitor election 2023-11-27T15:10:57.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:57 smithi102 ceph-mon[100296]: mon.c calling monitor election 2023-11-27T15:10:57.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:57 smithi102 ceph-mon[100296]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-11-27T15:10:57.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:57 smithi102 ceph-mon[100296]: monmap e3: 3 mons at {a=[v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0],b=[v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0],c=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0]} removed_ranks: {} 2023-11-27T15:10:57.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:57 smithi102 ceph-mon[100296]: fsmap 2023-11-27T15:10:57.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:57 smithi102 ceph-mon[100296]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:10:57.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:57 smithi102 ceph-mon[100296]: mgrmap e13: x(active, since 53s) 2023-11-27T15:10:57.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:57 smithi102 ceph-mon[100296]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-11-27T15:10:57.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:57 smithi102 ceph-mon[100296]: Cluster is now healthy 2023-11-27T15:10:57.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:57 smithi102 ceph-mon[100296]: overall HEALTH_OK 2023-11-27T15:10:57.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:57 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:57.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:57 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:57.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:57 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:57.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:57 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:57.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:57 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:57.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:57 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:57.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:57 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:57.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:57 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:10:57.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:57 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-27T15:10:57.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:57 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:10:57.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:57 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2556036621' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:10:57.856 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2023-11-27T15:10:57.856 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:10:57.924 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2023-11-27T15:10:57.925 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/ceph.conf 2023-11-27T15:10:57.958 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2023-11-27T15:10:57.958 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:10:58.026 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-27T15:10:58.027 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.conf 2023-11-27T15:10:58.061 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-27T15:10:58.061 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:10:58.129 INFO:tasks.cephadm:Adding mgr.x on smithi052 2023-11-27T15:10:58.130 INFO:tasks.cephadm:Adding mgr.y on smithi102 2023-11-27T15:10:58.130 INFO:tasks.cephadm:Adding mgr.z on smithi150 2023-11-27T15:10:58.130 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph orch apply mgr '3;smithi052=x;smithi102=y;smithi150=z' 2023-11-27T15:10:58.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:58 smithi052 ceph-mon[99056]: Updating smithi150:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:10:58.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:58 smithi052 ceph-mon[99056]: Updating smithi102:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:10:58.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:58 smithi052 ceph-mon[99056]: Updating smithi052:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:10:58.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:58 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:10:58.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:58 smithi052 ceph-mon[99056]: Reconfiguring mon.a (unknown last config time)... 2023-11-27T15:10:58.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:58 smithi052 ceph-mon[99056]: Reconfiguring daemon mon.a on smithi052 2023-11-27T15:10:58.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:58 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:10:58.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:58 smithi052 ceph-mon[99056]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:58.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:58 smithi150 ceph-mon[100559]: Updating smithi150:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:10:58.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:58 smithi150 ceph-mon[100559]: Updating smithi102:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:10:58.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:58 smithi150 ceph-mon[100559]: Updating smithi052:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:10:58.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:58 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:10:58.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:58 smithi150 ceph-mon[100559]: Reconfiguring mon.a (unknown last config time)... 2023-11-27T15:10:58.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:58 smithi150 ceph-mon[100559]: Reconfiguring daemon mon.a on smithi052 2023-11-27T15:10:58.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:58 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:10:58.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:58 smithi150 ceph-mon[100559]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:58.803 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:10:58.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:58 smithi102 ceph-mon[100296]: Updating smithi150:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:10:58.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:58 smithi102 ceph-mon[100296]: Updating smithi102:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:10:58.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:58 smithi102 ceph-mon[100296]: Updating smithi052:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:10:58.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:58 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:10:58.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:58 smithi102 ceph-mon[100296]: Reconfiguring mon.a (unknown last config time)... 2023-11-27T15:10:58.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:58 smithi102 ceph-mon[100296]: Reconfiguring daemon mon.a on smithi052 2023-11-27T15:10:58.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:58 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:10:58.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:58 smithi102 ceph-mon[100296]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:10:59.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:59 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:59.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:59 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:59.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:59 smithi102 ceph-mon[100296]: Reconfiguring mon.b (monmap changed)... 2023-11-27T15:10:59.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:59 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:10:59.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:59 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-27T15:10:59.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:59 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:10:59.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:59 smithi102 ceph-mon[100296]: Reconfiguring daemon mon.b on smithi102 2023-11-27T15:10:59.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:10:59 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:59.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:59 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:59.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:59 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:59.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:59 smithi052 ceph-mon[99056]: Reconfiguring mon.b (monmap changed)... 2023-11-27T15:10:59.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:59 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:10:59.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:59 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-27T15:10:59.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:59 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:10:59.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:59 smithi052 ceph-mon[99056]: Reconfiguring daemon mon.b on smithi102 2023-11-27T15:10:59.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:10:59 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:10:59.991 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.986+0000 7f55d81ef700 1 -- 172.21.15.150:0/4088854355 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55d00ffe40 msgr2=0x7f55d0100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:59.991 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.986+0000 7f55d81ef700 1 --2- 172.21.15.150:0/4088854355 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55d00ffe40 0x7f55d0100250 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f55cc009910 tx=0x7f55cc009c20 comp rx=0 tx=0).stop 2023-11-27T15:10:59.991 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.986+0000 7f55d81ef700 1 -- 172.21.15.150:0/4088854355 shutdown_connections 2023-11-27T15:10:59.991 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.986+0000 7f55d81ef700 1 --2- 172.21.15.150:0/4088854355 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55d00ffe40 0x7f55d0100250 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:59.992 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.986+0000 7f55d81ef700 1 -- 172.21.15.150:0/4088854355 >> 172.21.15.150:0/4088854355 conn(0x7f55d00fb3f0 msgr2=0x7f55d00fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:10:59.992 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.986+0000 7f55d81ef700 1 -- 172.21.15.150:0/4088854355 shutdown_connections 2023-11-27T15:10:59.992 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.986+0000 7f55d81ef700 1 -- 172.21.15.150:0/4088854355 wait complete. 2023-11-27T15:10:59.994 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.987+0000 7f55d81ef700 1 Processor -- start 2023-11-27T15:10:59.994 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.987+0000 7f55d81ef700 1 -- start start 2023-11-27T15:10:59.994 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.987+0000 7f55d81ef700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55d00ffe40 0x7f55d0074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:59.994 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.987+0000 7f55d81ef700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55d0072ee0 0x7f55d0073350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:59.994 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.987+0000 7f55d81ef700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55d0073890 0x7f55d019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:59.994 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.987+0000 7f55d81ef700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d019f620 con 0x7f55d00ffe40 2023-11-27T15:10:59.995 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.987+0000 7f55d81ef700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d019f760 con 0x7f55d0072ee0 2023-11-27T15:10:59.995 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.987+0000 7f55d81ef700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d019f8a0 con 0x7f55d0073890 2023-11-27T15:10:59.995 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.987+0000 7f55d5f8b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55d00ffe40 0x7f55d0074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:59.995 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.987+0000 7f55d678c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55d0073890 0x7f55d019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:59.995 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.987+0000 7f55d578a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55d0072ee0 0x7f55d0073350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:59.995 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.987+0000 7f55d678c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55d0073890 0x7f55d019f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49348/0 (socket says 172.21.15.150:49348) 2023-11-27T15:10:59.995 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.987+0000 7f55d678c700 1 -- 172.21.15.150:0/2941604319 learned_addr learned my addr 172.21.15.150:0/2941604319 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-27T15:10:59.996 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.987+0000 7f55d5f8b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55d00ffe40 0x7f55d0074890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.150:58286/0 (socket says 172.21.15.150:58286) 2023-11-27T15:10:59.996 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.988+0000 7f55d678c700 1 -- 172.21.15.150:0/2941604319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55d0072ee0 msgr2=0x7f55d0073350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:59.996 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.988+0000 7f55d678c700 1 --2- 172.21.15.150:0/2941604319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55d0072ee0 0x7f55d0073350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:59.996 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.988+0000 7f55d678c700 1 -- 172.21.15.150:0/2941604319 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55d00ffe40 msgr2=0x7f55d0074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:10:59.996 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.988+0000 7f55d678c700 1 --2- 172.21.15.150:0/2941604319 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55d00ffe40 0x7f55d0074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:10:59.996 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.988+0000 7f55d678c700 1 -- 172.21.15.150:0/2941604319 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55cc0095f0 con 0x7f55d0073890 2023-11-27T15:10:59.996 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.988+0000 7f55d578a700 1 --2- 172.21.15.150:0/2941604319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55d0072ee0 0x7f55d0073350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:10:59.997 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.988+0000 7f55d678c700 1 --2- 172.21.15.150:0/2941604319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55d0073890 0x7f55d019f0e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f55c800ba50 tx=0x7f55c800bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:59.997 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.988+0000 7f55c6ffd700 1 -- 172.21.15.150:0/2941604319 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c800c930 con 0x7f55d0073890 2023-11-27T15:10:59.997 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.988+0000 7f55c6ffd700 1 -- 172.21.15.150:0/2941604319 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55c8014440 con 0x7f55d0073890 2023-11-27T15:10:59.997 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.988+0000 7f55d81ef700 1 -- 172.21.15.150:0/2941604319 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55d0105c40 con 0x7f55d0073890 2023-11-27T15:10:59.997 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.988+0000 7f55c6ffd700 1 -- 172.21.15.150:0/2941604319 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c80075b0 con 0x7f55d0073890 2023-11-27T15:10:59.997 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.988+0000 7f55d81ef700 1 -- 172.21.15.150:0/2941604319 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55d0105ab0 con 0x7f55d0073890 2023-11-27T15:10:59.998 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.989+0000 7f55c6ffd700 1 -- 172.21.15.150:0/2941604319 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f55c8007710 con 0x7f55d0073890 2023-11-27T15:10:59.998 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.989+0000 7f55d81ef700 1 -- 172.21.15.150:0/2941604319 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55d018e970 con 0x7f55d0073890 2023-11-27T15:10:59.998 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.989+0000 7f55c6ffd700 1 --2- 172.21.15.150:0/2941604319 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f55bc038cf0 0x7f55bc03b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:10:59.998 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.989+0000 7f55c6ffd700 1 -- 172.21.15.150:0/2941604319 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f55c804a940 con 0x7f55d0073890 2023-11-27T15:10:59.998 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.990+0000 7f55d5f8b700 1 --2- 172.21.15.150:0/2941604319 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f55bc038cf0 0x7f55bc03b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:10:59.998 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.990+0000 7f55d5f8b700 1 --2- 172.21.15.150:0/2941604319 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f55bc038cf0 0x7f55bc03b1a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f55cc000c00 tx=0x7f55cc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:10:59.999 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:10:59.994+0000 7f55c6ffd700 1 -- 172.21.15.150:0/2941604319 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f55c801b8b0 con 0x7f55d0073890 2023-11-27T15:11:00.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:59 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:00.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:59 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:00.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:59 smithi150 ceph-mon[100559]: Reconfiguring mon.b (monmap changed)... 2023-11-27T15:11:00.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:59 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:11:00.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:59 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-27T15:11:00.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:59 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:00.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:59 smithi150 ceph-mon[100559]: Reconfiguring daemon mon.b on smithi102 2023-11-27T15:11:00.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:10:59 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:00.206 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:00.201+0000 7f55d81ef700 1 -- 172.21.15.150:0/2941604319 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi052=x;smithi102=y;smithi150=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f55d0061bf0 con 0x7f55bc038cf0 2023-11-27T15:11:00.214 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:00.208+0000 7f55c6ffd700 1 -- 172.21.15.150:0/2941604319 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f55d0061bf0 con 0x7f55bc038cf0 2023-11-27T15:11:00.215 INFO:teuthology.orchestra.run.smithi150.stdout:Scheduled mgr update... 2023-11-27T15:11:00.216 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:00.210+0000 7f55d81ef700 1 -- 172.21.15.150:0/2941604319 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f55bc038cf0 msgr2=0x7f55bc03b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:11:00.216 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:00.210+0000 7f55d81ef700 1 --2- 172.21.15.150:0/2941604319 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f55bc038cf0 0x7f55bc03b1a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f55cc000c00 tx=0x7f55cc005fd0 comp rx=0 tx=0).stop 2023-11-27T15:11:00.216 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:00.210+0000 7f55d81ef700 1 -- 172.21.15.150:0/2941604319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55d0073890 msgr2=0x7f55d019f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:11:00.217 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:00.210+0000 7f55d81ef700 1 --2- 172.21.15.150:0/2941604319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55d0073890 0x7f55d019f0e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f55c800ba50 tx=0x7f55c800bd60 comp rx=0 tx=0).stop 2023-11-27T15:11:00.217 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:00.210+0000 7f55d81ef700 1 -- 172.21.15.150:0/2941604319 shutdown_connections 2023-11-27T15:11:00.217 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:00.210+0000 7f55d81ef700 1 --2- 172.21.15.150:0/2941604319 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f55bc038cf0 0x7f55bc03b1a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:11:00.217 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:00.210+0000 7f55d81ef700 1 --2- 172.21.15.150:0/2941604319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55d0073890 0x7f55d019f0e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:11:00.217 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:00.210+0000 7f55d81ef700 1 --2- 172.21.15.150:0/2941604319 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55d00ffe40 0x7f55d0074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:11:00.217 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:00.210+0000 7f55d81ef700 1 --2- 172.21.15.150:0/2941604319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55d0072ee0 0x7f55d0073350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:11:00.217 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:00.210+0000 7f55d81ef700 1 -- 172.21.15.150:0/2941604319 >> 172.21.15.150:0/2941604319 conn(0x7f55d00fb3f0 msgr2=0x7f55d00fc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:11:00.217 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:00.211+0000 7f55d81ef700 1 -- 172.21.15.150:0/2941604319 shutdown_connections 2023-11-27T15:11:00.218 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:00.211+0000 7f55d81ef700 1 -- 172.21.15.150:0/2941604319 wait complete. 2023-11-27T15:11:00.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:00 smithi150 ceph-mon[100559]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:00.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:00 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:00.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:00 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:00.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:00 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:11:00.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:00 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-27T15:11:00.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:00 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:00.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:00 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:00.847 DEBUG:teuthology.orchestra.run.smithi102:mgr.y> sudo journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mgr.y.service 2023-11-27T15:11:00.849 DEBUG:teuthology.orchestra.run.smithi150:mgr.z> sudo journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mgr.z.service 2023-11-27T15:11:00.852 INFO:tasks.cephadm:Deploying OSDs... 2023-11-27T15:11:00.852 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2023-11-27T15:11:00.852 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/scratch_devs of=/dev/stdout 2023-11-27T15:11:00.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:00 smithi052 ceph-mon[99056]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:00.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:00 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:00.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:00 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:00.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:00 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:11:00.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:00 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-27T15:11:00.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:00 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:00.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:00 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:00.874 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-27T15:11:00.874 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_1 2023-11-27T15:11:00.879 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:00 smithi102 ceph-mon[100296]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:00.879 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:00 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:00.879 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:00 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:00.879 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:00 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:11:00.879 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:00 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-27T15:11:00.879 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:00 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:00.879 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:00 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:00.880 INFO:journalctl@ceph.mgr.z.smithi150.stdout:-- Logs begin at Mon 2023-11-27 14:53:12 UTC. -- 2023-11-27T15:11:00.885 INFO:journalctl@ceph.mgr.y.smithi102.stdout:-- Logs begin at Mon 2023-11-27 14:52:53 UTC. -- 2023-11-27T15:11:00.936 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-27T15:11:00.936 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:11:00.936 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 6h/6d Inode: 394276 Links: 1 2023-11-27T15:11:00.936 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:11:00.936 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:11:00.936 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2023-11-27 15:10:14.353187718 +0000 2023-11-27T15:11:00.937 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2023-11-27 15:04:33.479900594 +0000 2023-11-27T15:11:00.937 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2023-11-27 15:04:33.479900594 +0000 2023-11-27T15:11:00.937 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: - 2023-11-27T15:11:00.937 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-27T15:11:01.008 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2023-11-27T15:11:01.008 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2023-11-27T15:11:01.008 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000119069 s, 4.3 MB/s 2023-11-27T15:11:01.010 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-27T15:11:01.071 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_2 2023-11-27T15:11:01.132 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-27T15:11:01.132 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:11:01.132 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 6h/6d Inode: 395284 Links: 1 2023-11-27T15:11:01.132 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:11:01.132 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:11:01.132 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2023-11-27 15:10:14.353187718 +0000 2023-11-27T15:11:01.132 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2023-11-27 15:04:34.001893125 +0000 2023-11-27T15:11:01.133 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2023-11-27 15:04:34.001893125 +0000 2023-11-27T15:11:01.133 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: - 2023-11-27T15:11:01.133 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-27T15:11:01.203 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2023-11-27T15:11:01.204 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2023-11-27T15:11:01.204 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000134014 s, 3.8 MB/s 2023-11-27T15:11:01.205 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-27T15:11:01.267 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_3 2023-11-27T15:11:01.327 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-27T15:11:01.327 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:11:01.327 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 6h/6d Inode: 394477 Links: 1 2023-11-27T15:11:01.327 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:11:01.327 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:11:01.327 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2023-11-27 15:10:14.353187718 +0000 2023-11-27T15:11:01.327 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2023-11-27 15:04:34.498886013 +0000 2023-11-27T15:11:01.328 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2023-11-27 15:04:34.498886013 +0000 2023-11-27T15:11:01.328 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: - 2023-11-27T15:11:01.328 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-27T15:11:01.396 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2023-11-27T15:11:01.396 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2023-11-27T15:11:01.396 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000122456 s, 4.2 MB/s 2023-11-27T15:11:01.397 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-27T15:11:01.457 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_4 2023-11-27T15:11:01.516 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-27T15:11:01.563 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:11:01.563 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 6h/6d Inode: 395418 Links: 1 2023-11-27T15:11:01.563 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:11:01.563 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:11:01.563 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2023-11-27 15:10:14.353187718 +0000 2023-11-27T15:11:01.563 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2023-11-27 15:04:35.017878587 +0000 2023-11-27T15:11:01.564 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2023-11-27 15:04:35.017878587 +0000 2023-11-27T15:11:01.564 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: - 2023-11-27T15:11:01.564 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-27T15:11:01.592 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2023-11-27T15:11:01.592 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2023-11-27T15:11:01.592 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000100596 s, 5.1 MB/s 2023-11-27T15:11:01.593 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-27T15:11:01.652 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2023-11-27T15:11:01.652 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/scratch_devs of=/dev/stdout 2023-11-27T15:11:01.673 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-27T15:11:01.673 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_1 2023-11-27T15:11:01.734 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-27T15:11:01.734 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:11:01.734 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 6h/6d Inode: 386928 Links: 1 2023-11-27T15:11:01.734 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:11:01.734 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:11:01.734 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2023-11-27 15:10:49.068652807 +0000 2023-11-27T15:11:01.734 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2023-11-27 15:04:19.183909877 +0000 2023-11-27T15:11:01.734 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2023-11-27 15:04:19.183909877 +0000 2023-11-27T15:11:01.735 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: - 2023-11-27T15:11:01.735 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-27T15:11:01.805 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2023-11-27T15:11:01.806 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2023-11-27T15:11:01.806 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000254171 s, 2.0 MB/s 2023-11-27T15:11:01.807 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-27T15:11:01.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:01 smithi052 ceph-mon[99056]: Reconfiguring mon.c (monmap changed)... 2023-11-27T15:11:01.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:01 smithi052 ceph-mon[99056]: Reconfiguring daemon mon.c on smithi150 2023-11-27T15:11:01.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:01 smithi052 ceph-mon[99056]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi052=x;smithi102=y;smithi150=z", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:11:01.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:01 smithi052 ceph-mon[99056]: Saving service mgr spec with placement smithi052=x;smithi102=y;smithi150=z;count:3 2023-11-27T15:11:01.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:01 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:01.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:01 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:01.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:01 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:11:01.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:01 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:01.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:01 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:11:01.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:01 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:01.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:01 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-27T15:11:01.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:01 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-27T15:11:01.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:01 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-27T15:11:01.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:01 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:01.870 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_2 2023-11-27T15:11:01.929 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-27T15:11:01.930 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:11:01.930 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 6h/6d Inode: 391705 Links: 1 2023-11-27T15:11:01.930 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:11:01.930 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:11:01.931 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2023-11-27 15:10:49.068652807 +0000 2023-11-27T15:11:01.931 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2023-11-27 15:04:19.694898402 +0000 2023-11-27T15:11:01.931 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2023-11-27 15:04:19.694898402 +0000 2023-11-27T15:11:01.931 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: - 2023-11-27T15:11:01.931 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-27T15:11:01.934 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:01 smithi150 ceph-mon[100559]: Reconfiguring mon.c (monmap changed)... 2023-11-27T15:11:01.934 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:01 smithi150 ceph-mon[100559]: Reconfiguring daemon mon.c on smithi150 2023-11-27T15:11:01.934 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:01 smithi150 ceph-mon[100559]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi052=x;smithi102=y;smithi150=z", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:11:01.934 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:01 smithi150 ceph-mon[100559]: Saving service mgr spec with placement smithi052=x;smithi102=y;smithi150=z;count:3 2023-11-27T15:11:01.934 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:01 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:01.935 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:01 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:01.935 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:01 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:11:01.935 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:01 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:01.935 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:01 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:11:01.935 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:01 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:01.935 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:01 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-27T15:11:01.935 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:01 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-27T15:11:01.936 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:01 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-27T15:11:01.936 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:01 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:01.994 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:01 smithi102 ceph-mon[100296]: Reconfiguring mon.c (monmap changed)... 2023-11-27T15:11:01.995 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:01 smithi102 ceph-mon[100296]: Reconfiguring daemon mon.c on smithi150 2023-11-27T15:11:01.995 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:01 smithi102 ceph-mon[100296]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi052=x;smithi102=y;smithi150=z", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:11:01.995 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:01 smithi102 ceph-mon[100296]: Saving service mgr spec with placement smithi052=x;smithi102=y;smithi150=z;count:3 2023-11-27T15:11:01.995 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:01 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:01.996 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:01 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:01.996 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:01 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:11:01.996 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:01 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:01.996 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:01 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:11:01.996 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:01 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:01.996 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:01 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-27T15:11:01.996 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:01 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-27T15:11:01.997 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:01 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-27T15:11:01.997 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:01 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:01.997 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2023-11-27T15:11:01.997 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2023-11-27T15:11:01.997 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000224002 s, 2.3 MB/s 2023-11-27T15:11:01.998 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-27T15:11:02.057 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_3 2023-11-27T15:11:02.115 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-27T15:11:02.115 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:11:02.115 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 6h/6d Inode: 384720 Links: 1 2023-11-27T15:11:02.115 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:11:02.115 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:11:02.115 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2023-11-27 15:10:49.068652807 +0000 2023-11-27T15:11:02.115 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2023-11-27 15:04:20.207886881 +0000 2023-11-27T15:11:02.115 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2023-11-27 15:04:20.207886881 +0000 2023-11-27T15:11:02.116 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: - 2023-11-27T15:11:02.116 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-27T15:11:02.181 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2023-11-27T15:11:02.181 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2023-11-27T15:11:02.181 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000219555 s, 2.3 MB/s 2023-11-27T15:11:02.182 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-27T15:11:02.240 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_4 2023-11-27T15:11:02.298 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-27T15:11:02.298 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:11:02.298 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 6h/6d Inode: 393899 Links: 1 2023-11-27T15:11:02.298 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:11:02.298 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:11:02.298 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2023-11-27 15:10:49.068652807 +0000 2023-11-27T15:11:02.298 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2023-11-27 15:04:20.744874822 +0000 2023-11-27T15:11:02.298 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2023-11-27 15:04:20.744874822 +0000 2023-11-27T15:11:02.298 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: - 2023-11-27T15:11:02.299 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-27T15:11:02.364 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2023-11-27T15:11:02.364 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2023-11-27T15:11:02.364 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000174827 s, 2.9 MB/s 2023-11-27T15:11:02.365 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-27T15:11:02.423 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-27T15:11:02.423 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/scratch_devs of=/dev/stdout 2023-11-27T15:11:02.441 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-27T15:11:02.441 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_1 2023-11-27T15:11:02.498 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-27T15:11:02.498 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:11:02.498 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 391477 Links: 1 2023-11-27T15:11:02.498 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:11:02.498 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:11:02.499 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2023-11-27 15:10:40.856658951 +0000 2023-11-27T15:11:02.499 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2023-11-27 15:04:19.184718666 +0000 2023-11-27T15:11:02.499 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2023-11-27 15:04:19.184718666 +0000 2023-11-27T15:11:02.499 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2023-11-27T15:11:02.499 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-27T15:11:02.605 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2023-11-27T15:11:02.605 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2023-11-27T15:11:02.605 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.00023469 s, 2.2 MB/s 2023-11-27T15:11:02.607 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-27T15:11:02.624 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_2 2023-11-27T15:11:02.682 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-27T15:11:02.682 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:11:02.683 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 387876 Links: 1 2023-11-27T15:11:02.683 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:11:02.683 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:11:02.683 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2023-11-27 15:11:02.548242090 +0000 2023-11-27T15:11:02.683 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2023-11-27 15:04:19.715707725 +0000 2023-11-27T15:11:02.683 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2023-11-27 15:04:19.715707725 +0000 2023-11-27T15:11:02.684 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2023-11-27T15:11:02.684 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-27T15:11:02.751 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2023-11-27T15:11:02.751 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2023-11-27T15:11:02.751 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000249247 s, 2.1 MB/s 2023-11-27T15:11:02.752 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-27T15:11:02.813 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_3 2023-11-27T15:11:02.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:02 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:11:02.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:02 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:11:02.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:02 smithi102 ceph-mon[100296]: Deploying daemon mgr.z on smithi150 2023-11-27T15:11:02.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:02 smithi102 ceph-mon[100296]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:02.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:02 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:11:02.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:02 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:11:02.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:02 smithi052 ceph-mon[99056]: Deploying daemon mgr.z on smithi150 2023-11-27T15:11:02.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:02 smithi052 ceph-mon[99056]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:02.872 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-27T15:11:02.872 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:11:02.872 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 392581 Links: 1 2023-11-27T15:11:02.872 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:11:02.872 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:11:02.872 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2023-11-27 15:11:02.681239534 +0000 2023-11-27T15:11:02.872 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2023-11-27 15:04:20.215697423 +0000 2023-11-27T15:11:02.872 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2023-11-27 15:04:20.215697423 +0000 2023-11-27T15:11:02.872 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2023-11-27T15:11:02.872 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-27T15:11:02.937 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:02 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:11:02.937 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:02 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:11:02.937 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:02 smithi150 ceph-mon[100559]: Deploying daemon mgr.z on smithi150 2023-11-27T15:11:02.937 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:02 smithi150 ceph-mon[100559]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:02.937 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:11:02 smithi150 systemd[1]: Starting Ceph mgr.z for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:11:02.940 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2023-11-27T15:11:02.940 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2023-11-27T15:11:02.940 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.00021873 s, 2.3 MB/s 2023-11-27T15:11:02.941 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-27T15:11:02.998 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_4 2023-11-27T15:11:03.056 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-27T15:11:03.056 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:11:03.056 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 395359 Links: 1 2023-11-27T15:11:03.056 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:11:03.056 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:11:03.056 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2023-11-27 15:11:02.682239515 +0000 2023-11-27T15:11:03.056 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2023-11-27 15:04:20.892683471 +0000 2023-11-27T15:11:03.056 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2023-11-27 15:04:20.892683471 +0000 2023-11-27T15:11:03.056 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2023-11-27T15:11:03.057 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-27T15:11:03.121 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2023-11-27T15:11:03.122 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2023-11-27T15:11:03.122 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000168826 s, 3.0 MB/s 2023-11-27T15:11:03.123 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-27T15:11:03.180 INFO:tasks.cephadm:Deploying osd.0 on smithi052 with /dev/vg_nvme/lv_4... 2023-11-27T15:11:03.181 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-11-27T15:11:03.198 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:11:03 smithi150 podman[102564]: 2023-11-27T15:11:03.617 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:11:03 smithi150 ceph-mgr[102709]: -- 172.21.15.150:0/353856619 <== mon.1 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56385741b520 con 0x5638573f1000 2023-11-27T15:11:03.617 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:11:03 smithi150 bash[102564]: 41bf55970cbc14301dee052faf401f9f84803cd1e818f437987826546eacb231 2023-11-27T15:11:03.617 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:11:03 smithi150 systemd[1]: Started Ceph mgr.z for c6c7df88-8d36-11ee-95a2-87774f69a715. 2023-11-27T15:11:03.617 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:11:03 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:11:03.436+0000 7fa2c070d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-27T15:11:03.617 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:11:03 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:11:03.522+0000 7fa2c070d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-27T15:11:03.617 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:11:03 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:11:03.611+0000 7fa2c070d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-27T15:11:03.842 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:11:03.876 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:11:03 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:11:03.703+0000 7fa2c070d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-27T15:11:04.180 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:11:03 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:11:03.871+0000 7fa2c070d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-27T15:11:04.482 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:11:04 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:11:04.175+0000 7fa2c070d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-27T15:11:04.482 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:11:04 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:11:04.263+0000 7fa2c070d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-27T15:11:04.483 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:11:04 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:11:04.391+0000 7fa2c070d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-27T15:11:04.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:04 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:04.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:04 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:04.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:04 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:04.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:04 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-27T15:11:04.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:04 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-27T15:11:04.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:04 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-27T15:11:04.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:04 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:04.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:04 smithi052 ceph-mon[99056]: Deploying daemon mgr.y on smithi102 2023-11-27T15:11:04.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:04 smithi052 ceph-mon[99056]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:04.750 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:04 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:04.750 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:04 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:04.750 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:04 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:04.750 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:04 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-27T15:11:04.750 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:04 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-27T15:11:04.750 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:04 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-27T15:11:04.751 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:04 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:04.751 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:04 smithi102 ceph-mon[100296]: Deploying daemon mgr.y on smithi102 2023-11-27T15:11:04.751 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:04 smithi102 ceph-mon[100296]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:04.778 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:11:04 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:11:04.486+0000 7fa2c070d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-27T15:11:04.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:04 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:04.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:04 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:04.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:04 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:04.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:04 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-27T15:11:04.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:04 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-27T15:11:04.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:04 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-27T15:11:04.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:04 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:04.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:04 smithi150 ceph-mon[100559]: Deploying daemon mgr.y on smithi102 2023-11-27T15:11:04.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:04 smithi150 ceph-mon[100559]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:05.278 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:11:05 smithi102 systemd[1]: Starting Ceph mgr.y for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:11:05.279 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:11:04 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:11:04.904+0000 7fa2c070d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-27T15:11:05.324 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-11-27T15:11:05.324 INFO:teuthology.orchestra.run.smithi052.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-11-27T15:11:05.324 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10+0 records in 2023-11-27T15:11:05.324 INFO:teuthology.orchestra.run.smithi052.stderr:10+0 records out 2023-11-27T15:11:05.324 INFO:teuthology.orchestra.run.smithi052.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00810901 s, 1.3 GB/s 2023-11-27T15:11:05.324 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping successful for: 2023-11-27T15:11:05.531 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:11:05 smithi102 podman[101740]: 2023-11-27T15:11:05.779 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:11:05 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:11:05.473+0000 7fa2c070d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-27T15:11:05.779 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:11:05 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:11:05.643+0000 7fa2c070d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-27T15:11:05.787 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi052:vg_nvme/lv_4 2023-11-27T15:11:05.828 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:11:05 smithi102 bash[101740]: 9c5d94e7dac39148e5a6b99e5de8db7fdeb94f675a3ee577239c266664aadbc1 2023-11-27T15:11:05.828 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:11:05 smithi102 ceph-mgr[101780]: -- 172.21.15.102:0/1123241790 <== mon.2 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e16a50b520 con 0x55e16a4e0800 2023-11-27T15:11:05.829 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:11:05 smithi102 systemd[1]: Started Ceph mgr.y for c6c7df88-8d36-11ee-95a2-87774f69a715. 2023-11-27T15:11:05.829 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:11:05 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:11:05.655+0000 7fc7b36ea200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-27T15:11:05.829 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:11:05 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:11:05.734+0000 7fc7b36ea200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-27T15:11:06.099 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:11:05 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:11:05.825+0000 7fc7b36ea200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-27T15:11:06.099 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:11:05 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:11:05.917+0000 7fc7b36ea200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-27T15:11:06.427 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:11:06 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:11:06.094+0000 7fc7b36ea200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-27T15:11:06.442 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:11:06.715 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:11:06 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:11:06.435+0000 7fa2c070d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:11:06.715 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:11:06 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:11:06.531+0000 7fa2c070d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-27T15:11:06.715 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:06 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:06.715 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:06 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:06.715 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:06 smithi150 ceph-mon[100559]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:06.715 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:06 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:06.715 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:06 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:06.715 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:06 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:11:06.746 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:11:06 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:11:06.422+0000 7fc7b36ea200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-27T15:11:06.746 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:11:06 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:11:06.515+0000 7fc7b36ea200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-27T15:11:06.747 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:11:06 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:11:06.647+0000 7fc7b36ea200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-27T15:11:06.747 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:06 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:06.747 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:06 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:06.747 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:06 smithi102 ceph-mon[100296]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:06.747 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:06 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:06.747 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:06 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:06.747 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:06 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:11:06.962 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:06 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:06.962 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:06 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:06.962 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:06 smithi052 ceph-mon[99056]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:06.962 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:06 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:06.962 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:06 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:06.962 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:06 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:11:07.001 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:11:06 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:11:06.709+0000 7fa2c070d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:11:07.078 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:11:06 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:11:06.742+0000 7fc7b36ea200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-27T15:11:07.279 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:11:06 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:11:06.996+0000 7fa2c070d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-27T15:11:07.279 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:11:07 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:11:07.235+0000 7fa2c070d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-27T15:11:07.578 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:11:07 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:11:07.192+0000 7fc7b36ea200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-27T15:11:07.662 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:11:07 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:11:07.392+0000 7fa2c070d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-27T15:11:07.668 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.664+0000 7f0a58af3700 1 -- 172.21.15.52:0/1245821952 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a54101690 msgr2=0x7f0a54101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:11:07.669 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.664+0000 7f0a58af3700 1 --2- 172.21.15.52:0/1245821952 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a54101690 0x7f0a54101ae0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0a480099c0 tx=0x7f0a48009cd0 comp rx=0 tx=0).stop 2023-11-27T15:11:07.669 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.664+0000 7f0a58af3700 1 -- 172.21.15.52:0/1245821952 shutdown_connections 2023-11-27T15:11:07.669 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.664+0000 7f0a58af3700 1 --2- 172.21.15.52:0/1245821952 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a540fe610 0x7f0a540fea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:11:07.669 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.664+0000 7f0a58af3700 1 --2- 172.21.15.52:0/1245821952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a54102020 0x7f0a541088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:11:07.669 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.664+0000 7f0a58af3700 1 --2- 172.21.15.52:0/1245821952 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a54101690 0x7f0a54101ae0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:11:07.669 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.664+0000 7f0a58af3700 1 -- 172.21.15.52:0/1245821952 >> 172.21.15.52:0/1245821952 conn(0x7f0a540fa000 msgr2=0x7f0a540fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:11:07.669 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.664+0000 7f0a58af3700 1 -- 172.21.15.52:0/1245821952 shutdown_connections 2023-11-27T15:11:07.669 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.664+0000 7f0a58af3700 1 -- 172.21.15.52:0/1245821952 wait complete. 2023-11-27T15:11:07.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.664+0000 7f0a58af3700 1 Processor -- start 2023-11-27T15:11:07.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.665+0000 7f0a58af3700 1 -- start start 2023-11-27T15:11:07.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.665+0000 7f0a58af3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a540fe610 0x7f0a54195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:11:07.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.665+0000 7f0a58af3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a54101690 0x7f0a54196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:11:07.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.665+0000 7f0a58af3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a54102020 0x7f0a5419d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:11:07.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.665+0000 7f0a58af3700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5419db70 con 0x7f0a54102020 2023-11-27T15:11:07.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.665+0000 7f0a58af3700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5419dcb0 con 0x7f0a540fe610 2023-11-27T15:11:07.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.665+0000 7f0a58af3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5419de40 con 0x7f0a54101690 2023-11-27T15:11:07.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.665+0000 7f0a52d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a54102020 0x7f0a5419d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:11:07.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.665+0000 7f0a52d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a54102020 0x7f0a5419d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49134/0 (socket says 172.21.15.52:49134) 2023-11-27T15:11:07.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.665+0000 7f0a52d9d700 1 -- 172.21.15.52:0/3566065411 learned_addr learned my addr 172.21.15.52:0/3566065411 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:11:07.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.665+0000 7f0a51d9b700 1 --2- 172.21.15.52:0/3566065411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a54101690 0x7f0a54196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:11:07.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.665+0000 7f0a52d9d700 1 -- 172.21.15.52:0/3566065411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a54101690 msgr2=0x7f0a54196520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:11:07.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.665+0000 7f0a52d9d700 1 --2- 172.21.15.52:0/3566065411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a54101690 0x7f0a54196520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:11:07.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.665+0000 7f0a52d9d700 1 -- 172.21.15.52:0/3566065411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a540fe610 msgr2=0x7f0a54195fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:11:07.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.665+0000 7f0a52d9d700 1 --2- 172.21.15.52:0/3566065411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a540fe610 0x7f0a54195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:11:07.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.665+0000 7f0a52d9d700 1 -- 172.21.15.52:0/3566065411 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a480095f0 con 0x7f0a54102020 2023-11-27T15:11:07.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.665+0000 7f0a52d9d700 1 --2- 172.21.15.52:0/3566065411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a54102020 0x7f0a5419d5a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c00ef60 tx=0x7f0a4c00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:11:07.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.666+0000 7f0a437fe700 1 -- 172.21.15.52:0/3566065411 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c018070 con 0x7f0a54102020 2023-11-27T15:11:07.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.666+0000 7f0a437fe700 1 -- 172.21.15.52:0/3566065411 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a4c00f040 con 0x7f0a54102020 2023-11-27T15:11:07.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.666+0000 7f0a437fe700 1 -- 172.21.15.52:0/3566065411 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c00cd70 con 0x7f0a54102020 2023-11-27T15:11:07.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.666+0000 7f0a58af3700 1 -- 172.21.15.52:0/3566065411 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a5410bca0 con 0x7f0a54102020 2023-11-27T15:11:07.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.666+0000 7f0a58af3700 1 -- 172.21.15.52:0/3566065411 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a5410bb10 con 0x7f0a54102020 2023-11-27T15:11:07.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.667+0000 7f0a437fe700 1 -- 172.21.15.52:0/3566065411 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f0a4c004000 con 0x7f0a54102020 2023-11-27T15:11:07.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.667+0000 7f0a58af3700 1 -- 172.21.15.52:0/3566065411 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a54107990 con 0x7f0a54102020 2023-11-27T15:11:07.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.667+0000 7f0a437fe700 1 --2- 172.21.15.52:0/3566065411 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0a3c0389e0 0x7f0a3c03ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:11:07.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.667+0000 7f0a437fe700 1 -- 172.21.15.52:0/3566065411 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0a4c04ae30 con 0x7f0a54102020 2023-11-27T15:11:07.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.667+0000 7f0a5259c700 1 --2- 172.21.15.52:0/3566065411 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0a3c0389e0 0x7f0a3c03ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:11:07.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.667+0000 7f0a5259c700 1 --2- 172.21.15.52:0/3566065411 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0a3c0389e0 0x7f0a3c03ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0a54068e40 tx=0x7f0a44009470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:11:07.676 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.672+0000 7f0a437fe700 1 -- 172.21.15.52:0/3566065411 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0a4c005020 con 0x7f0a54102020 2023-11-27T15:11:07.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:07 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:11:07.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:07 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:07.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:07 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:07.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:07 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:07.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:07 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:07.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:07 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:07.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:07 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:11:07.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:07 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:07.884 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:07.879+0000 7f0a58af3700 1 -- 172.21.15.52:0/3566065411 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a54061c90 con 0x7f0a3c0389e0 2023-11-27T15:11:07.948 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:11:07 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:11:07.777+0000 7fc7b36ea200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-27T15:11:07.949 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:07 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:11:07.949 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:07 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:07.949 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:07 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:07.949 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:07 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:07.949 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:07 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:07.949 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:07 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:07.949 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:07 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:11:07.949 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:07 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:08.029 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:11:07 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:11:07.662+0000 7fa2c070d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-27T15:11:08.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:07 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:11:08.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:07 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:08.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:07 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:08.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:07 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:08.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:07 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:08.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:07 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:08.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:07 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:11:08.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:07 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:08.328 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:11:07 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:11:07.944+0000 7fc7b36ea200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-27T15:11:08.671 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:11:08 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:11:08.363+0000 7fa2c070d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-27T15:11:08.672 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:11:08 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:11:08.494+0000 7fa2c070d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-27T15:11:08.937 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:08 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:11:08.937 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:08 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:11:08.937 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:08 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:11:08.937 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:08 smithi052 ceph-mon[99056]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:08.937 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:08 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-27T15:11:08.937 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:08 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-27T15:11:08.937 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:08 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:08.937 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:08 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:11:08.938 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:08 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:11:08.938 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:08 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:08.961 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:11:08 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:11:08.696+0000 7fc7b36ea200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:11:08.962 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:11:08 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:11:08.787+0000 7fc7b36ea200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-27T15:11:08.962 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:08 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:11:08.962 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:08 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:11:08.962 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:08 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:11:08.962 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:08 smithi102 ceph-mon[100296]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:08.962 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:08 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-27T15:11:08.962 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:08 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-27T15:11:08.963 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:08 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:08.963 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:08 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:11:08.963 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:08 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:11:08.963 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:08 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:09.016 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:11:08 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:11:08.683+0000 7fa2c070d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-27T15:11:09.016 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:11:08 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:11:08.797+0000 7fa2c070d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-27T15:11:09.016 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:11:09 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:11:09.011+0000 7fa2c070d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-27T15:11:09.017 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:08 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:11:09.017 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:08 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:11:09.017 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:08 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:11:09.017 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:08 smithi150 ceph-mon[100559]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:09.017 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:08 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-27T15:11:09.017 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:08 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-27T15:11:09.017 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:08 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:09.017 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:08 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:11:09.017 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:08 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:11:09.017 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:08 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:09.236 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:11:08 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:11:08.957+0000 7fc7b36ea200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:11:09.578 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:11:09 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:11:09.231+0000 7fc7b36ea200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-27T15:11:09.578 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:11:09 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:11:09.454+0000 7fc7b36ea200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-27T15:11:09.864 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:11:09 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:11:09.600+0000 7fc7b36ea200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-27T15:11:09.864 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:09 smithi102 ceph-mon[100296]: Reconfiguring mgr.x (unknown last config time)... 2023-11-27T15:11:09.864 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:09 smithi102 ceph-mon[100296]: Reconfiguring daemon mgr.x on smithi052 2023-11-27T15:11:09.864 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:09 smithi102 ceph-mon[100296]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:11:09.864 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:09 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:09.864 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:09 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:09.865 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:09 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:09.865 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:09 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:11:09.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:09 smithi052 ceph-mon[99056]: Reconfiguring mgr.x (unknown last config time)... 2023-11-27T15:11:09.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:09 smithi052 ceph-mon[99056]: Reconfiguring daemon mgr.x on smithi052 2023-11-27T15:11:09.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:09 smithi052 ceph-mon[99056]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:11:09.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:09 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:09.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:09 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:09.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:09 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:09.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:09 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:11:10.029 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:11:09 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:11:09.868+0000 7fa2c070d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-27T15:11:10.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:09 smithi150 ceph-mon[100559]: Reconfiguring mgr.x (unknown last config time)... 2023-11-27T15:11:10.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:09 smithi150 ceph-mon[100559]: Reconfiguring daemon mgr.x on smithi052 2023-11-27T15:11:10.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:09 smithi150 ceph-mon[100559]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:11:10.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:09 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:10.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:09 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:10.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:09 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:10.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:09 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:11:10.328 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:11:09 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:11:09.859+0000 7fc7b36ea200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-27T15:11:10.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:10.775+0000 7f0a437fe700 1 -- 172.21.15.52:0/3566065411 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 14) v1 ==== 90599+0+0 (secure 0 0 0) 0x7f0a4c007880 con 0x7f0a54102020 2023-11-27T15:11:10.903 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:11:10 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:11:10.575+0000 7fc7b36ea200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-27T15:11:10.904 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:11:10 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:11:10.706+0000 7fc7b36ea200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-27T15:11:10.904 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:10 smithi102 ceph-mon[100296]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:10.904 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:10 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:10.904 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:10 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:11:10.904 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:10 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:10.904 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:10 smithi102 ceph-mon[100296]: Standby manager daemon z started 2023-11-27T15:11:10.905 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:10 smithi102 ceph-mon[100296]: from='mgr.? 172.21.15.150:0/2360362573' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-11-27T15:11:10.905 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:10 smithi102 ceph-mon[100296]: from='mgr.? 172.21.15.150:0/2360362573' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-27T15:11:10.905 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:10 smithi102 ceph-mon[100296]: from='mgr.? 172.21.15.150:0/2360362573' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-11-27T15:11:10.905 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:10 smithi102 ceph-mon[100296]: from='mgr.? 172.21.15.150:0/2360362573' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-27T15:11:11.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:10 smithi150 ceph-mon[100559]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:11.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:10 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:11.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:10 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:11:11.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:10 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:11.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:10 smithi150 ceph-mon[100559]: Standby manager daemon z started 2023-11-27T15:11:11.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:10 smithi150 ceph-mon[100559]: from='mgr.? 172.21.15.150:0/2360362573' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-11-27T15:11:11.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:10 smithi150 ceph-mon[100559]: from='mgr.? 172.21.15.150:0/2360362573' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-27T15:11:11.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:10 smithi150 ceph-mon[100559]: from='mgr.? 172.21.15.150:0/2360362573' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-11-27T15:11:11.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:10 smithi150 ceph-mon[100559]: from='mgr.? 172.21.15.150:0/2360362573' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-27T15:11:11.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:10 smithi052 ceph-mon[99056]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:11.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:10 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:11.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:10 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:11:11.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:10 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:11.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:10 smithi052 ceph-mon[99056]: Standby manager daemon z started 2023-11-27T15:11:11.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:10 smithi052 ceph-mon[99056]: from='mgr.? 172.21.15.150:0/2360362573' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-11-27T15:11:11.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:10 smithi052 ceph-mon[99056]: from='mgr.? 172.21.15.150:0/2360362573' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-27T15:11:11.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:10 smithi052 ceph-mon[99056]: from='mgr.? 172.21.15.150:0/2360362573' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-11-27T15:11:11.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:10 smithi052 ceph-mon[99056]: from='mgr.? 172.21.15.150:0/2360362573' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-27T15:11:11.232 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:11:10 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:11:10.898+0000 7fc7b36ea200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-27T15:11:11.232 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:11:11 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:11:11.010+0000 7fc7b36ea200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-27T15:11:11.579 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:11:11 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:11:11.227+0000 7fc7b36ea200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-27T15:11:12.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:11 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:11:12.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:11 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:11:12.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:11 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:11:12.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:11 smithi150 ceph-mon[100559]: mgrmap e14: x(active, since 67s), standbys: z 2023-11-27T15:11:12.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:11 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-11-27T15:11:12.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:11 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3760076002' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58d69ede-c293-49f5-abbe-8187793f2b8e"}]: dispatch 2023-11-27T15:11:12.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:11 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3760076002' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58d69ede-c293-49f5-abbe-8187793f2b8e"}]': finished 2023-11-27T15:11:12.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:11 smithi150 ceph-mon[100559]: osdmap e5: 1 total, 0 up, 1 in 2023-11-27T15:11:12.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:11 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:11:12.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:11 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3487517109' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:11:12.072 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:11 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:11:12.072 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:11 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:11:12.072 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:11 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:11:12.072 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:11 smithi102 ceph-mon[100296]: mgrmap e14: x(active, since 67s), standbys: z 2023-11-27T15:11:12.072 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:11 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-11-27T15:11:12.072 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:11 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3760076002' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58d69ede-c293-49f5-abbe-8187793f2b8e"}]: dispatch 2023-11-27T15:11:12.072 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:11 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3760076002' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58d69ede-c293-49f5-abbe-8187793f2b8e"}]': finished 2023-11-27T15:11:12.072 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:11 smithi102 ceph-mon[100296]: osdmap e5: 1 total, 0 up, 1 in 2023-11-27T15:11:12.073 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:11 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:11:12.073 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:11 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3487517109' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:11:12.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:11 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:11:12.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:11 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:11:12.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:11 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:11:12.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:11 smithi052 ceph-mon[99056]: mgrmap e14: x(active, since 67s), standbys: z 2023-11-27T15:11:12.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:11 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-11-27T15:11:12.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:11 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3760076002' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58d69ede-c293-49f5-abbe-8187793f2b8e"}]: dispatch 2023-11-27T15:11:12.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:11 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3760076002' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58d69ede-c293-49f5-abbe-8187793f2b8e"}]': finished 2023-11-27T15:11:12.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:11 smithi052 ceph-mon[99056]: osdmap e5: 1 total, 0 up, 1 in 2023-11-27T15:11:12.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:11 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:11:12.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:11 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3487517109' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:11:12.328 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:11:12 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:11:12.067+0000 7fc7b36ea200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-27T15:11:12.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:12.865+0000 7f0a437fe700 1 -- 172.21.15.52:0/3566065411 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f0a4c01c3c0 con 0x7f0a54102020 2023-11-27T15:11:13.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:12 smithi150 ceph-mon[100559]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:13.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:12 smithi150 ceph-mon[100559]: Standby manager daemon y started 2023-11-27T15:11:13.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:12 smithi150 ceph-mon[100559]: from='mgr.? 172.21.15.102:0/2318901441' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-11-27T15:11:13.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:12 smithi150 ceph-mon[100559]: from='mgr.? 172.21.15.102:0/2318901441' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-27T15:11:13.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:12 smithi150 ceph-mon[100559]: from='mgr.? 172.21.15.102:0/2318901441' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-11-27T15:11:13.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:12 smithi150 ceph-mon[100559]: from='mgr.? 172.21.15.102:0/2318901441' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-27T15:11:13.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:12 smithi102 ceph-mon[100296]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:13.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:12 smithi102 ceph-mon[100296]: Standby manager daemon y started 2023-11-27T15:11:13.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:12 smithi102 ceph-mon[100296]: from='mgr.? 172.21.15.102:0/2318901441' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-11-27T15:11:13.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:12 smithi102 ceph-mon[100296]: from='mgr.? 172.21.15.102:0/2318901441' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-27T15:11:13.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:12 smithi102 ceph-mon[100296]: from='mgr.? 172.21.15.102:0/2318901441' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-11-27T15:11:13.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:12 smithi102 ceph-mon[100296]: from='mgr.? 172.21.15.102:0/2318901441' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-27T15:11:13.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:12 smithi052 ceph-mon[99056]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:13.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:12 smithi052 ceph-mon[99056]: Standby manager daemon y started 2023-11-27T15:11:13.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:12 smithi052 ceph-mon[99056]: from='mgr.? 172.21.15.102:0/2318901441' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-11-27T15:11:13.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:12 smithi052 ceph-mon[99056]: from='mgr.? 172.21.15.102:0/2318901441' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-27T15:11:13.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:12 smithi052 ceph-mon[99056]: from='mgr.? 172.21.15.102:0/2318901441' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-11-27T15:11:13.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:12 smithi052 ceph-mon[99056]: from='mgr.? 172.21.15.102:0/2318901441' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-27T15:11:14.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:13 smithi052 ceph-mon[99056]: mgrmap e15: x(active, since 69s), standbys: z, y 2023-11-27T15:11:14.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:13 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-11-27T15:11:14.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:13 smithi052 ceph-mon[99056]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:14.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:13 smithi150 ceph-mon[100559]: mgrmap e15: x(active, since 69s), standbys: z, y 2023-11-27T15:11:14.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:13 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-11-27T15:11:14.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:13 smithi150 ceph-mon[100559]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:14.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:13 smithi102 ceph-mon[100296]: mgrmap e15: x(active, since 69s), standbys: z, y 2023-11-27T15:11:14.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:13 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-11-27T15:11:14.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:13 smithi102 ceph-mon[100296]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:16.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:15 smithi150 ceph-mon[100559]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:16.049 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:15 smithi052 ceph-mon[99056]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:16.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:15 smithi102 ceph-mon[100296]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:17.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:17 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-11-27T15:11:17.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:17 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:17.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:17 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-11-27T15:11:17.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:17 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:18.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:17 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-11-27T15:11:18.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:17 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:18.791 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:18 smithi052 ceph-mon[99056]: Deploying daemon osd.0 on smithi052 2023-11-27T15:11:18.791 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:18 smithi052 ceph-mon[99056]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:18.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:18 smithi102 ceph-mon[100296]: Deploying daemon osd.0 on smithi052 2023-11-27T15:11:18.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:18 smithi102 ceph-mon[100296]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:19.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:18 smithi150 ceph-mon[100559]: Deploying daemon osd.0 on smithi052 2023-11-27T15:11:19.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:18 smithi150 ceph-mon[100559]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:20.007 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:19 smithi052 ceph-mon[99056]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:20.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:19 smithi150 ceph-mon[100559]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:20.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:19 smithi102 ceph-mon[100296]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:21.877 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:21 smithi052 ceph-mon[99056]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:22.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:21 smithi150 ceph-mon[100559]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:22.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:21 smithi102 ceph-mon[100296]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:23.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:22 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:23.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:22 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:23.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:22 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:11:23.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:22 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:23.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:22 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:23.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:22 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:11:23.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:22 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:23.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:22 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:23.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:22 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:11:24.517 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:24 smithi052 ceph-mon[99056]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:24.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:24 smithi150 ceph-mon[100559]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:24.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:24 smithi102 ceph-mon[100296]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:24.837 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:24.832+0000 7f0a437fe700 1 -- 172.21.15.52:0/3566065411 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0a54061c90 con 0x7f0a3c0389e0 2023-11-27T15:11:24.838 INFO:teuthology.orchestra.run.smithi052.stdout:Created osd(s) 0 on host 'smithi052' 2023-11-27T15:11:24.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:24.835+0000 7f0a58af3700 1 -- 172.21.15.52:0/3566065411 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0a3c0389e0 msgr2=0x7f0a3c03ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:11:24.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:24.835+0000 7f0a58af3700 1 --2- 172.21.15.52:0/3566065411 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0a3c0389e0 0x7f0a3c03ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0a54068e40 tx=0x7f0a44009470 comp rx=0 tx=0).stop 2023-11-27T15:11:24.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:24.835+0000 7f0a58af3700 1 -- 172.21.15.52:0/3566065411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a54102020 msgr2=0x7f0a5419d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:11:24.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:24.835+0000 7f0a58af3700 1 --2- 172.21.15.52:0/3566065411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a54102020 0x7f0a5419d5a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c00ef60 tx=0x7f0a4c00ef90 comp rx=0 tx=0).stop 2023-11-27T15:11:24.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:24.835+0000 7f0a58af3700 1 -- 172.21.15.52:0/3566065411 shutdown_connections 2023-11-27T15:11:24.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:24.835+0000 7f0a58af3700 1 --2- 172.21.15.52:0/3566065411 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0a3c0389e0 0x7f0a3c03ae90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:11:24.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:24.835+0000 7f0a58af3700 1 --2- 172.21.15.52:0/3566065411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a540fe610 0x7f0a54195fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:11:24.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:24.835+0000 7f0a58af3700 1 --2- 172.21.15.52:0/3566065411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a54102020 0x7f0a5419d5a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:11:24.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:24.835+0000 7f0a58af3700 1 --2- 172.21.15.52:0/3566065411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a54101690 0x7f0a54196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:11:24.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:24.835+0000 7f0a58af3700 1 -- 172.21.15.52:0/3566065411 >> 172.21.15.52:0/3566065411 conn(0x7f0a540fa000 msgr2=0x7f0a54104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:11:24.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:24.835+0000 7f0a58af3700 1 -- 172.21.15.52:0/3566065411 shutdown_connections 2023-11-27T15:11:24.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:11:24.835+0000 7f0a58af3700 1 -- 172.21.15.52:0/3566065411 wait complete. 2023-11-27T15:11:25.347 DEBUG:teuthology.orchestra.run.smithi052:osd.0> sudo journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.0.service 2023-11-27T15:11:25.349 INFO:tasks.cephadm:Deploying osd.1 on smithi102 with /dev/vg_nvme/lv_4... 2023-11-27T15:11:25.349 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-11-27T15:11:25.379 INFO:journalctl@ceph.osd.0.smithi052.stdout:-- Logs begin at Mon 2023-11-27 14:52:43 UTC. -- 2023-11-27T15:11:25.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:25 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:25.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:25 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:25.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:25 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:25.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:25 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:11:25.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:25 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:11:25.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:25 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:25.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:25 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:11:25.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:25 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:11:25.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:25 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:25.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:25 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:25.530 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:25 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:11:25.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:25 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:25.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:25 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:25.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:25 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:25.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:25 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:11:25.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:25 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:11:25.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:25 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:25.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:25 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:11:25.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:25 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:11:25.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:25 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:25.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:25 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:25.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:25 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:11:25.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:25 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:25.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:25 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:25.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:25 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:25.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:25 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:11:25.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:25 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:11:25.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:25 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:25.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:25 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:11:25.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:25 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:11:25.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:25 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:25.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:25 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:25.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:25 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:11:26.003 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:11:26.059 INFO:journalctl@ceph.osd.0.smithi052.stdout:Nov 27 15:11:25 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-0[111956]: 2023-11-27T15:11:25.820+0000 7fb7d92d5640 -1 osd.0 0 log_to_monitors true 2023-11-27T15:11:26.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:26 smithi052 ceph-mon[99056]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:26.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:26 smithi052 ceph-mon[99056]: from='osd.0 [v2:172.21.15.52:6802/608216415,v1:172.21.15.52:6803/608216415]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-11-27T15:11:26.441 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:26 smithi102 ceph-mon[100296]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:26.441 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:26 smithi102 ceph-mon[100296]: from='osd.0 [v2:172.21.15.52:6802/608216415,v1:172.21.15.52:6803/608216415]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-11-27T15:11:26.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:26 smithi150 ceph-mon[100559]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:26.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:26 smithi150 ceph-mon[100559]: from='osd.0 [v2:172.21.15.52:6802/608216415,v1:172.21.15.52:6803/608216415]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-11-27T15:11:27.330 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:27 smithi102 ceph-mon[100296]: from='osd.0 [v2:172.21.15.52:6802/608216415,v1:172.21.15.52:6803/608216415]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-11-27T15:11:27.330 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:27 smithi102 ceph-mon[100296]: osdmap e6: 1 total, 0 up, 1 in 2023-11-27T15:11:27.330 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:27 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:11:27.330 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:27 smithi102 ceph-mon[100296]: from='osd.0 [v2:172.21.15.52:6802/608216415,v1:172.21.15.52:6803/608216415]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2023-11-27T15:11:27.462 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-11-27T15:11:27.462 INFO:teuthology.orchestra.run.smithi102.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-11-27T15:11:27.462 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10+0 records in 2023-11-27T15:11:27.462 INFO:teuthology.orchestra.run.smithi102.stderr:10+0 records out 2023-11-27T15:11:27.462 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.010996 s, 954 MB/s 2023-11-27T15:11:27.462 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping successful for: 2023-11-27T15:11:27.501 INFO:journalctl@ceph.osd.0.smithi052.stdout:Nov 27 15:11:27 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-0[111956]: 2023-11-27T15:11:27.173+0000 7fb7ce94c700 -1 osd.0 0 waiting for initial osdmap 2023-11-27T15:11:27.502 INFO:journalctl@ceph.osd.0.smithi052.stdout:Nov 27 15:11:27 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-0[111956]: 2023-11-27T15:11:27.180+0000 7fb7c873d700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:11:27.502 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:27 smithi052 ceph-mon[99056]: from='osd.0 [v2:172.21.15.52:6802/608216415,v1:172.21.15.52:6803/608216415]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-11-27T15:11:27.502 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:27 smithi052 ceph-mon[99056]: osdmap e6: 1 total, 0 up, 1 in 2023-11-27T15:11:27.502 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:27 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:11:27.502 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:27 smithi052 ceph-mon[99056]: from='osd.0 [v2:172.21.15.52:6802/608216415,v1:172.21.15.52:6803/608216415]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2023-11-27T15:11:27.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:27 smithi150 ceph-mon[100559]: from='osd.0 [v2:172.21.15.52:6802/608216415,v1:172.21.15.52:6803/608216415]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-11-27T15:11:27.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:27 smithi150 ceph-mon[100559]: osdmap e6: 1 total, 0 up, 1 in 2023-11-27T15:11:27.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:27 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:11:27.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:27 smithi150 ceph-mon[100559]: from='osd.0 [v2:172.21.15.52:6802/608216415,v1:172.21.15.52:6803/608216415]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2023-11-27T15:11:27.842 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi102:vg_nvme/lv_4 2023-11-27T15:11:28.479 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:11:28.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:28 smithi150 ceph-mon[100559]: from='osd.0 [v2:172.21.15.52:6802/608216415,v1:172.21.15.52:6803/608216415]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2023-11-27T15:11:28.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:28 smithi150 ceph-mon[100559]: osdmap e7: 1 total, 0 up, 1 in 2023-11-27T15:11:28.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:28 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:11:28.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:28 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:11:28.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:28 smithi150 ceph-mon[100559]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:28.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:28 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:28.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:28 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:28.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:28 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:11:28.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:28 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:28.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:28 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:28.530 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:28 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:11:28.530 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:28 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:28.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:28 smithi102 ceph-mon[100296]: from='osd.0 [v2:172.21.15.52:6802/608216415,v1:172.21.15.52:6803/608216415]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2023-11-27T15:11:28.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:28 smithi102 ceph-mon[100296]: osdmap e7: 1 total, 0 up, 1 in 2023-11-27T15:11:28.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:28 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:11:28.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:28 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:11:28.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:28 smithi102 ceph-mon[100296]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:28.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:28 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:28.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:28 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:28.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:28 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:11:28.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:28 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:28.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:28 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:28.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:28 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:11:28.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:28 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:28.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:28 smithi052 ceph-mon[99056]: from='osd.0 [v2:172.21.15.52:6802/608216415,v1:172.21.15.52:6803/608216415]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2023-11-27T15:11:28.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:28 smithi052 ceph-mon[99056]: osdmap e7: 1 total, 0 up, 1 in 2023-11-27T15:11:28.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:28 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:11:28.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:28 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:11:28.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:28 smithi052 ceph-mon[99056]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:11:28.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:28 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:28.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:28 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:28.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:28 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:11:28.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:28 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:28.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:28 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:28.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:28 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:11:28.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:28 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:29.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:29 smithi102 ceph-mon[100296]: purged_snaps scrub starts 2023-11-27T15:11:29.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:29 smithi102 ceph-mon[100296]: purged_snaps scrub ok 2023-11-27T15:11:29.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:29 smithi102 ceph-mon[100296]: Detected new or changed devices on smithi052 2023-11-27T15:11:29.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:29 smithi102 ceph-mon[100296]: Adjusting osd_memory_target on smithi052 to 17173M 2023-11-27T15:11:29.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:29 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:11:29.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:29 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:11:29.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:29 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:11:29.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:29 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:11:29.330 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:29 smithi102 ceph-mon[100296]: osd.0 [v2:172.21.15.52:6802/608216415,v1:172.21.15.52:6803/608216415] boot 2023-11-27T15:11:29.330 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:29 smithi102 ceph-mon[100296]: osdmap e8: 1 total, 1 up, 1 in 2023-11-27T15:11:29.330 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:29 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:11:29.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:29 smithi150 ceph-mon[100559]: purged_snaps scrub starts 2023-11-27T15:11:29.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:29 smithi150 ceph-mon[100559]: purged_snaps scrub ok 2023-11-27T15:11:29.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:29 smithi150 ceph-mon[100559]: Detected new or changed devices on smithi052 2023-11-27T15:11:29.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:29 smithi150 ceph-mon[100559]: Adjusting osd_memory_target on smithi052 to 17173M 2023-11-27T15:11:29.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:29 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:11:29.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:29 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:11:29.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:29 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:11:29.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:29 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:11:29.530 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:29 smithi150 ceph-mon[100559]: osd.0 [v2:172.21.15.52:6802/608216415,v1:172.21.15.52:6803/608216415] boot 2023-11-27T15:11:29.530 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:29 smithi150 ceph-mon[100559]: osdmap e8: 1 total, 1 up, 1 in 2023-11-27T15:11:29.530 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:29 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:11:29.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:29 smithi052 ceph-mon[99056]: purged_snaps scrub starts 2023-11-27T15:11:29.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:29 smithi052 ceph-mon[99056]: purged_snaps scrub ok 2023-11-27T15:11:29.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:29 smithi052 ceph-mon[99056]: Detected new or changed devices on smithi052 2023-11-27T15:11:29.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:29 smithi052 ceph-mon[99056]: Adjusting osd_memory_target on smithi052 to 17173M 2023-11-27T15:11:29.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:29 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:11:29.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:29 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:11:29.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:29 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:11:29.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:29 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:11:29.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:29 smithi052 ceph-mon[99056]: osd.0 [v2:172.21.15.52:6802/608216415,v1:172.21.15.52:6803/608216415] boot 2023-11-27T15:11:29.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:29 smithi052 ceph-mon[99056]: osdmap e8: 1 total, 1 up, 1 in 2023-11-27T15:11:29.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:29 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:11:29.660 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.652+0000 7fa138a3b700 1 -- 172.21.15.102:0/2329821900 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa1341002f0 msgr2=0x7fa134100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:11:29.660 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.652+0000 7fa138a3b700 1 --2- 172.21.15.102:0/2329821900 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa1341002f0 0x7fa134100700 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa128009990 tx=0x7fa128009ca0 comp rx=0 tx=0).stop 2023-11-27T15:11:29.660 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.653+0000 7fa138a3b700 1 -- 172.21.15.102:0/2329821900 shutdown_connections 2023-11-27T15:11:29.660 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.653+0000 7fa138a3b700 1 --2- 172.21.15.102:0/2329821900 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa1341002f0 0x7fa134100700 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:11:29.660 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.653+0000 7fa138a3b700 1 --2- 172.21.15.102:0/2329821900 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa134101e80 0x7fa134108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:11:29.661 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.653+0000 7fa138a3b700 1 --2- 172.21.15.102:0/2329821900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1341014f0 0x7fa134101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:11:29.661 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.653+0000 7fa138a3b700 1 -- 172.21.15.102:0/2329821900 >> 172.21.15.102:0/2329821900 conn(0x7fa1340fb900 msgr2=0x7fa1340fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:11:29.661 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.653+0000 7fa138a3b700 1 -- 172.21.15.102:0/2329821900 shutdown_connections 2023-11-27T15:11:29.661 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.653+0000 7fa138a3b700 1 -- 172.21.15.102:0/2329821900 wait complete. 2023-11-27T15:11:29.661 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.653+0000 7fa138a3b700 1 Processor -- start 2023-11-27T15:11:29.661 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.653+0000 7fa138a3b700 1 -- start start 2023-11-27T15:11:29.661 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.653+0000 7fa138a3b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa1341002f0 0x7fa134196120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:11:29.661 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.653+0000 7fa138a3b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1341014f0 0x7fa134196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:11:29.661 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.654+0000 7fa138a3b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa134101e80 0x7fa13419d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:11:29.661 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.654+0000 7fa138a3b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13419dcb0 con 0x7fa1341014f0 2023-11-27T15:11:29.662 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.654+0000 7fa138a3b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13419ddf0 con 0x7fa1341002f0 2023-11-27T15:11:29.662 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.654+0000 7fa138a3b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13419df80 con 0x7fa134101e80 2023-11-27T15:11:29.662 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.654+0000 7fa13259c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa1341002f0 0x7fa134196120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:11:29.662 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.654+0000 7fa13259c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa1341002f0 0x7fa134196120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41702/0 (socket says 172.21.15.102:41702) 2023-11-27T15:11:29.662 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.654+0000 7fa13259c700 1 -- 172.21.15.102:0/2232342850 learned_addr learned my addr 172.21.15.102:0/2232342850 (peer_addr_for_me v2:172.21.15.102:0/0) 2023-11-27T15:11:29.662 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.654+0000 7fa131d9b700 1 --2- 172.21.15.102:0/2232342850 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1341014f0 0x7fa134196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:11:29.662 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.654+0000 7fa132d9d700 1 --2- 172.21.15.102:0/2232342850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa134101e80 0x7fa13419d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:11:29.662 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.654+0000 7fa13259c700 1 -- 172.21.15.102:0/2232342850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa134101e80 msgr2=0x7fa13419d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:11:29.662 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.654+0000 7fa13259c700 1 --2- 172.21.15.102:0/2232342850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa134101e80 0x7fa13419d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:11:29.662 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.654+0000 7fa13259c700 1 -- 172.21.15.102:0/2232342850 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1341014f0 msgr2=0x7fa134196660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:11:29.663 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.654+0000 7fa13259c700 1 --2- 172.21.15.102:0/2232342850 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1341014f0 0x7fa134196660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:11:29.663 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.654+0000 7fa13259c700 1 -- 172.21.15.102:0/2232342850 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1280095f0 con 0x7fa1341002f0 2023-11-27T15:11:29.663 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.654+0000 7fa132d9d700 1 --2- 172.21.15.102:0/2232342850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa134101e80 0x7fa13419d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:11:29.663 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.654+0000 7fa131d9b700 1 --2- 172.21.15.102:0/2232342850 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1341014f0 0x7fa134196660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:11:29.663 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.654+0000 7fa13259c700 1 --2- 172.21.15.102:0/2232342850 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa1341002f0 0x7fa134196120 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa128000c00 tx=0x7fa12800f7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:11:29.663 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.654+0000 7fa1237fe700 1 -- 172.21.15.102:0/2232342850 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12801d070 con 0x7fa1341002f0 2023-11-27T15:11:29.663 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.654+0000 7fa1237fe700 1 -- 172.21.15.102:0/2232342850 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa12800fca0 con 0x7fa1341002f0 2023-11-27T15:11:29.664 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.654+0000 7fa1237fe700 1 -- 172.21.15.102:0/2232342850 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa128017720 con 0x7fa1341002f0 2023-11-27T15:11:29.664 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.654+0000 7fa138a3b700 1 -- 172.21.15.102:0/2232342850 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa13410bd40 con 0x7fa1341002f0 2023-11-27T15:11:29.664 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.654+0000 7fa138a3b700 1 -- 172.21.15.102:0/2232342850 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa13410bbb0 con 0x7fa1341002f0 2023-11-27T15:11:29.664 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.655+0000 7fa138a3b700 1 -- 172.21.15.102:0/2232342850 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa13419e540 con 0x7fa1341002f0 2023-11-27T15:11:29.665 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.656+0000 7fa1237fe700 1 -- 172.21.15.102:0/2232342850 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7fa128017880 con 0x7fa1341002f0 2023-11-27T15:11:29.665 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.657+0000 7fa1237fe700 1 --2- 172.21.15.102:0/2232342850 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fa11c0a10c0 0x7fa11c0a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:11:29.665 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.657+0000 7fa1237fe700 1 -- 172.21.15.102:0/2232342850 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7fa1280cc840 con 0x7fa1341002f0 2023-11-27T15:11:29.665 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.657+0000 7fa131d9b700 1 --2- 172.21.15.102:0/2232342850 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fa11c0a10c0 0x7fa11c0a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:11:29.665 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.658+0000 7fa131d9b700 1 --2- 172.21.15.102:0/2232342850 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fa11c0a10c0 0x7fa11c0a3570 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa1240099d0 tx=0x7fa124008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:11:29.666 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.660+0000 7fa1237fe700 1 -- 172.21.15.102:0/2232342850 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa128097a90 con 0x7fa1341002f0 2023-11-27T15:11:29.860 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:29.854+0000 7fa138a3b700 1 -- 172.21.15.102:0/2232342850 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa134060f30 con 0x7fa11c0a10c0 2023-11-27T15:11:30.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:30 smithi150 ceph-mon[100559]: osdmap e9: 1 total, 1 up, 1 in 2023-11-27T15:11:30.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:30 smithi150 ceph-mon[100559]: pgmap v42: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:30.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:30 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:11:30.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:30 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:11:30.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:30 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:30.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:30 smithi102 ceph-mon[100296]: osdmap e9: 1 total, 1 up, 1 in 2023-11-27T15:11:30.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:30 smithi102 ceph-mon[100296]: pgmap v42: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:30.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:30 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:11:30.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:30 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:11:30.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:30 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:30.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:30 smithi052 ceph-mon[99056]: osdmap e9: 1 total, 1 up, 1 in 2023-11-27T15:11:30.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:30 smithi052 ceph-mon[99056]: pgmap v42: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:30.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:30 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:11:30.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:30 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:11:30.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:30 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:31.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:31 smithi150 ceph-mon[100559]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:11:31.562 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:31 smithi102 ceph-mon[100296]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:11:31.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:31 smithi052 ceph-mon[99056]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:11:32.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:32 smithi150 ceph-mon[100559]: pgmap v43: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:32.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:32 smithi102 ceph-mon[100296]: pgmap v43: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:32.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:32 smithi052 ceph-mon[99056]: pgmap v43: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:33.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:33 smithi150 ceph-mon[100559]: from='client.? 172.21.15.102:0/1281311751' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6a46b91-adc8-45ef-80f5-566ae400911f"}]: dispatch 2023-11-27T15:11:33.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:33 smithi150 ceph-mon[100559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6a46b91-adc8-45ef-80f5-566ae400911f"}]: dispatch 2023-11-27T15:11:33.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:33 smithi150 ceph-mon[100559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6a46b91-adc8-45ef-80f5-566ae400911f"}]': finished 2023-11-27T15:11:33.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:33 smithi150 ceph-mon[100559]: osdmap e10: 2 total, 1 up, 2 in 2023-11-27T15:11:33.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:33 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:11:33.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:33 smithi150 ceph-mon[100559]: from='client.? 172.21.15.102:0/4191010572' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:11:33.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:33 smithi102 ceph-mon[100296]: from='client.? 172.21.15.102:0/1281311751' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6a46b91-adc8-45ef-80f5-566ae400911f"}]: dispatch 2023-11-27T15:11:33.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:33 smithi102 ceph-mon[100296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6a46b91-adc8-45ef-80f5-566ae400911f"}]: dispatch 2023-11-27T15:11:33.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:33 smithi102 ceph-mon[100296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6a46b91-adc8-45ef-80f5-566ae400911f"}]': finished 2023-11-27T15:11:33.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:33 smithi102 ceph-mon[100296]: osdmap e10: 2 total, 1 up, 2 in 2023-11-27T15:11:33.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:33 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:11:33.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:33 smithi102 ceph-mon[100296]: from='client.? 172.21.15.102:0/4191010572' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:11:33.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:33 smithi052 ceph-mon[99056]: from='client.? 172.21.15.102:0/1281311751' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6a46b91-adc8-45ef-80f5-566ae400911f"}]: dispatch 2023-11-27T15:11:33.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:33 smithi052 ceph-mon[99056]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6a46b91-adc8-45ef-80f5-566ae400911f"}]: dispatch 2023-11-27T15:11:33.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:33 smithi052 ceph-mon[99056]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6a46b91-adc8-45ef-80f5-566ae400911f"}]': finished 2023-11-27T15:11:33.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:33 smithi052 ceph-mon[99056]: osdmap e10: 2 total, 1 up, 2 in 2023-11-27T15:11:33.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:33 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:11:33.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:33 smithi052 ceph-mon[99056]: from='client.? 172.21.15.102:0/4191010572' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:11:34.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:34 smithi150 ceph-mon[100559]: pgmap v45: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:34.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:34 smithi102 ceph-mon[100296]: pgmap v45: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:34.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:34 smithi052 ceph-mon[99056]: pgmap v45: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:36.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:35 smithi150 ceph-mon[100559]: pgmap v46: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:36.070 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:35 smithi102 ceph-mon[100296]: pgmap v46: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:36.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:35 smithi052 ceph-mon[99056]: pgmap v46: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:38.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:37 smithi150 ceph-mon[100559]: pgmap v47: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:38.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:37 smithi102 ceph-mon[100296]: pgmap v47: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:38.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:37 smithi052 ceph-mon[99056]: pgmap v47: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:39.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:39 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-11-27T15:11:39.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:39 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:39.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:39 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-11-27T15:11:39.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:39 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:39.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:39 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-11-27T15:11:39.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:39 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:40.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:40 smithi102 ceph-mon[100296]: Deploying daemon osd.1 on smithi102 2023-11-27T15:11:40.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:40 smithi102 ceph-mon[100296]: pgmap v48: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:40.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:40 smithi150 ceph-mon[100559]: Deploying daemon osd.1 on smithi102 2023-11-27T15:11:40.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:40 smithi150 ceph-mon[100559]: pgmap v48: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:40.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:40 smithi052 ceph-mon[99056]: Deploying daemon osd.1 on smithi102 2023-11-27T15:11:40.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:40 smithi052 ceph-mon[99056]: pgmap v48: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:42.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:41 smithi150 ceph-mon[100559]: pgmap v49: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:42.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:41 smithi102 ceph-mon[100296]: pgmap v49: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:42.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:41 smithi052 ceph-mon[99056]: pgmap v49: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:44.644 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:44 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:44.644 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:44 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:11:44.644 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:44 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:44.644 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:44 smithi102 ceph-mon[100296]: pgmap v50: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:44.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:44 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:44.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:44 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:11:44.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:44 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:44.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:44 smithi150 ceph-mon[100559]: pgmap v50: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:44.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:44 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:44.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:44 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:11:44.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:44 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:44.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:44 smithi052 ceph-mon[99056]: pgmap v50: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:46.429 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:46.423+0000 7fa1237fe700 1 -- 172.21.15.102:0/2232342850 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa134060f30 con 0x7fa11c0a10c0 2023-11-27T15:11:46.429 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 1 on host 'smithi102' 2023-11-27T15:11:46.431 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:46.426+0000 7fa138a3b700 1 -- 172.21.15.102:0/2232342850 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fa11c0a10c0 msgr2=0x7fa11c0a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:11:46.431 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:46.426+0000 7fa138a3b700 1 --2- 172.21.15.102:0/2232342850 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fa11c0a10c0 0x7fa11c0a3570 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa1240099d0 tx=0x7fa124008040 comp rx=0 tx=0).stop 2023-11-27T15:11:46.431 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:46.426+0000 7fa138a3b700 1 -- 172.21.15.102:0/2232342850 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa1341002f0 msgr2=0x7fa134196120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:11:46.431 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:46.426+0000 7fa138a3b700 1 --2- 172.21.15.102:0/2232342850 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa1341002f0 0x7fa134196120 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa128000c00 tx=0x7fa12800f7b0 comp rx=0 tx=0).stop 2023-11-27T15:11:46.432 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:46.426+0000 7fa138a3b700 1 -- 172.21.15.102:0/2232342850 shutdown_connections 2023-11-27T15:11:46.432 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:46.426+0000 7fa138a3b700 1 --2- 172.21.15.102:0/2232342850 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fa11c0a10c0 0x7fa11c0a3570 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:11:46.432 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:46.426+0000 7fa138a3b700 1 --2- 172.21.15.102:0/2232342850 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa1341002f0 0x7fa134196120 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:11:46.432 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:46.426+0000 7fa138a3b700 1 --2- 172.21.15.102:0/2232342850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa134101e80 0x7fa13419d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:11:46.432 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:46.426+0000 7fa138a3b700 1 --2- 172.21.15.102:0/2232342850 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1341014f0 0x7fa134196660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:11:46.432 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:46.426+0000 7fa138a3b700 1 -- 172.21.15.102:0/2232342850 >> 172.21.15.102:0/2232342850 conn(0x7fa1340fb900 msgr2=0x7fa1340fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:11:46.432 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:46.426+0000 7fa138a3b700 1 -- 172.21.15.102:0/2232342850 shutdown_connections 2023-11-27T15:11:46.432 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:11:46.426+0000 7fa138a3b700 1 -- 172.21.15.102:0/2232342850 wait complete. 2023-11-27T15:11:46.861 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:46 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:46.862 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:46 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:46.862 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:46 smithi102 ceph-mon[100296]: pgmap v51: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:46.862 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:46 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:46.862 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:46 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:11:46.862 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:46 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:46.862 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:46 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:46.862 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:46 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:46.862 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:46 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:11:46.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:46 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:46.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:46 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:46.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:46 smithi052 ceph-mon[99056]: pgmap v51: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:46.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:46 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:46.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:46 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:11:46.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:46 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:46.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:46 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:46.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:46 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:46.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:46 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:11:46.875 DEBUG:teuthology.orchestra.run.smithi102:osd.1> sudo journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.1.service 2023-11-27T15:11:46.877 INFO:tasks.cephadm:Deploying osd.2 on smithi150 with /dev/vg_nvme/lv_4... 2023-11-27T15:11:46.877 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-11-27T15:11:46.906 INFO:journalctl@ceph.osd.1.smithi102.stdout:-- Logs begin at Mon 2023-11-27 14:52:53 UTC. -- 2023-11-27T15:11:46.909 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:46 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:46.909 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:46 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:46.909 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:46 smithi150 ceph-mon[100559]: pgmap v51: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:46.909 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:46 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:46.909 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:46 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:11:46.909 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:46 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:46.909 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:46 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:46.909 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:46 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:46.909 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:46 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:11:47.526 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:11:47.685 INFO:journalctl@ceph.osd.1.smithi102.stdout:Nov 27 15:11:47 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-1[105297]: 2023-11-27T15:11:47.375+0000 7fd74c9f0640 -1 osd.1 0 log_to_monitors true 2023-11-27T15:11:47.686 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:47 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:11:47.686 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:47 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:11:47.686 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:47 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:11:47.686 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:47 smithi102 ceph-mon[100296]: from='osd.1 [v2:172.21.15.102:6800/2538384087,v1:172.21.15.102:6801/2538384087]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-27T15:11:47.686 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:47 smithi102 ceph-mon[100296]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-27T15:11:47.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:47 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:11:47.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:47 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:11:47.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:47 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:11:47.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:47 smithi052 ceph-mon[99056]: from='osd.1 [v2:172.21.15.102:6800/2538384087,v1:172.21.15.102:6801/2538384087]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-27T15:11:47.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:47 smithi052 ceph-mon[99056]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-27T15:11:47.937 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:47 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:11:47.938 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:47 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:11:47.938 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:47 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:11:47.938 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:47 smithi150 ceph-mon[100559]: from='osd.1 [v2:172.21.15.102:6800/2538384087,v1:172.21.15.102:6801/2538384087]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-27T15:11:47.938 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:47 smithi150 ceph-mon[100559]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-27T15:11:48.981 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-11-27T15:11:48.981 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-11-27T15:11:48.982 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2023-11-27T15:11:48.982 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2023-11-27T15:11:48.982 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109997 s, 953 MB/s 2023-11-27T15:11:48.982 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2023-11-27T15:11:48.985 INFO:journalctl@ceph.osd.1.smithi102.stdout:Nov 27 15:11:48 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-1[105297]: 2023-11-27T15:11:48.619+0000 7fd742067700 -1 osd.1 0 waiting for initial osdmap 2023-11-27T15:11:48.985 INFO:journalctl@ceph.osd.1.smithi102.stdout:Nov 27 15:11:48 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-1[105297]: 2023-11-27T15:11:48.623+0000 7fd73b657700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:11:48.985 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:48 smithi102 ceph-mon[100296]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-11-27T15:11:48.986 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:48 smithi102 ceph-mon[100296]: osdmap e11: 2 total, 1 up, 2 in 2023-11-27T15:11:48.986 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:48 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:11:48.986 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:48 smithi102 ceph-mon[100296]: from='osd.1 [v2:172.21.15.102:6800/2538384087,v1:172.21.15.102:6801/2538384087]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2023-11-27T15:11:48.986 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:48 smithi102 ceph-mon[100296]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2023-11-27T15:11:48.986 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:48 smithi102 ceph-mon[100296]: pgmap v53: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:49.021 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:48 smithi150 ceph-mon[100559]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-11-27T15:11:49.021 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:48 smithi150 ceph-mon[100559]: osdmap e11: 2 total, 1 up, 2 in 2023-11-27T15:11:49.022 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:48 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:11:49.022 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:48 smithi150 ceph-mon[100559]: from='osd.1 [v2:172.21.15.102:6800/2538384087,v1:172.21.15.102:6801/2538384087]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2023-11-27T15:11:49.022 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:48 smithi150 ceph-mon[100559]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2023-11-27T15:11:49.022 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:48 smithi150 ceph-mon[100559]: pgmap v53: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:49.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:48 smithi052 ceph-mon[99056]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-11-27T15:11:49.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:48 smithi052 ceph-mon[99056]: osdmap e11: 2 total, 1 up, 2 in 2023-11-27T15:11:49.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:48 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:11:49.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:48 smithi052 ceph-mon[99056]: from='osd.1 [v2:172.21.15.102:6800/2538384087,v1:172.21.15.102:6801/2538384087]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2023-11-27T15:11:49.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:48 smithi052 ceph-mon[99056]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2023-11-27T15:11:49.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:48 smithi052 ceph-mon[99056]: pgmap v53: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:49.382 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi150:vg_nvme/lv_4 2023-11-27T15:11:50.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:49 smithi150 ceph-mon[100559]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2023-11-27T15:11:50.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:49 smithi150 ceph-mon[100559]: osdmap e12: 2 total, 1 up, 2 in 2023-11-27T15:11:50.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:49 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:11:50.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:49 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:11:50.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:49 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:50.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:49 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:50.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:49 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:11:50.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:49 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:50.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:49 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:50.031 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:49 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:11:50.031 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:49 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:50.066 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:11:50.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:49 smithi102 ceph-mon[100296]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2023-11-27T15:11:50.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:49 smithi102 ceph-mon[100296]: osdmap e12: 2 total, 1 up, 2 in 2023-11-27T15:11:50.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:11:50.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:11:50.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:50.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:50.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:11:50.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:50.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:50.080 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:11:50.080 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:50.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:49 smithi052 ceph-mon[99056]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2023-11-27T15:11:50.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:49 smithi052 ceph-mon[99056]: osdmap e12: 2 total, 1 up, 2 in 2023-11-27T15:11:50.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:49 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:11:50.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:49 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:11:50.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:49 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:50.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:49 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:50.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:49 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:11:50.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:49 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:50.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:49 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:50.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:49 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:11:50.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:49 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:11:51.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:50 smithi150 ceph-mon[100559]: purged_snaps scrub starts 2023-11-27T15:11:51.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:50 smithi150 ceph-mon[100559]: purged_snaps scrub ok 2023-11-27T15:11:51.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:50 smithi150 ceph-mon[100559]: Detected new or changed devices on smithi102 2023-11-27T15:11:51.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:50 smithi150 ceph-mon[100559]: Adjusting osd_memory_target on smithi102 to 17168M 2023-11-27T15:11:51.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:50 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:11:51.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:50 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:11:51.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:50 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:11:51.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:50 smithi150 ceph-mon[100559]: osd.1 [v2:172.21.15.102:6800/2538384087,v1:172.21.15.102:6801/2538384087] boot 2023-11-27T15:11:51.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:50 smithi150 ceph-mon[100559]: osdmap e13: 2 total, 2 up, 2 in 2023-11-27T15:11:51.031 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:50 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:11:51.031 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:50 smithi150 ceph-mon[100559]: pgmap v56: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:51.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:50 smithi102 ceph-mon[100296]: purged_snaps scrub starts 2023-11-27T15:11:51.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:50 smithi102 ceph-mon[100296]: purged_snaps scrub ok 2023-11-27T15:11:51.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:50 smithi102 ceph-mon[100296]: Detected new or changed devices on smithi102 2023-11-27T15:11:51.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:50 smithi102 ceph-mon[100296]: Adjusting osd_memory_target on smithi102 to 17168M 2023-11-27T15:11:51.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:50 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:11:51.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:50 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:11:51.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:50 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:11:51.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:50 smithi102 ceph-mon[100296]: osd.1 [v2:172.21.15.102:6800/2538384087,v1:172.21.15.102:6801/2538384087] boot 2023-11-27T15:11:51.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:50 smithi102 ceph-mon[100296]: osdmap e13: 2 total, 2 up, 2 in 2023-11-27T15:11:51.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:50 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:11:51.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:50 smithi102 ceph-mon[100296]: pgmap v56: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:51.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:50 smithi052 ceph-mon[99056]: purged_snaps scrub starts 2023-11-27T15:11:51.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:50 smithi052 ceph-mon[99056]: purged_snaps scrub ok 2023-11-27T15:11:51.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:50 smithi052 ceph-mon[99056]: Detected new or changed devices on smithi102 2023-11-27T15:11:51.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:50 smithi052 ceph-mon[99056]: Adjusting osd_memory_target on smithi102 to 17168M 2023-11-27T15:11:51.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:50 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:11:51.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:50 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:11:51.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:50 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:11:51.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:50 smithi052 ceph-mon[99056]: osd.1 [v2:172.21.15.102:6800/2538384087,v1:172.21.15.102:6801/2538384087] boot 2023-11-27T15:11:51.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:50 smithi052 ceph-mon[99056]: osdmap e13: 2 total, 2 up, 2 in 2023-11-27T15:11:51.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:50 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:11:51.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:50 smithi052 ceph-mon[99056]: pgmap v56: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:11:51.226 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.221+0000 7f5006fde700 1 -- 172.21.15.150:0/1182988122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5000101510 msgr2=0x7f5000101960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:11:51.226 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.221+0000 7f5006fde700 1 --2- 172.21.15.150:0/1182988122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5000101510 0x7f5000101960 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4fec009990 tx=0x7f4fec009ca0 comp rx=0 tx=0).stop 2023-11-27T15:11:51.226 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.221+0000 7f5006fde700 1 -- 172.21.15.150:0/1182988122 shutdown_connections 2023-11-27T15:11:51.226 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.221+0000 7f5006fde700 1 --2- 172.21.15.150:0/1182988122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5000101ea0 0x7f5000108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:11:51.227 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.221+0000 7f5006fde700 1 --2- 172.21.15.150:0/1182988122 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5000100310 0x7f5000100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:11:51.227 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.221+0000 7f5006fde700 1 --2- 172.21.15.150:0/1182988122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5000101510 0x7f5000101960 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:11:51.227 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.221+0000 7f5006fde700 1 -- 172.21.15.150:0/1182988122 >> 172.21.15.150:0/1182988122 conn(0x7f50000fb960 msgr2=0x7f50000fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:11:51.227 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.221+0000 7f5006fde700 1 -- 172.21.15.150:0/1182988122 shutdown_connections 2023-11-27T15:11:51.227 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.221+0000 7f5006fde700 1 -- 172.21.15.150:0/1182988122 wait complete. 2023-11-27T15:11:51.229 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.222+0000 7f5006fde700 1 Processor -- start 2023-11-27T15:11:51.229 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.222+0000 7f5006fde700 1 -- start start 2023-11-27T15:11:51.229 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.222+0000 7f5006fde700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5000100310 0x7f5000196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:11:51.229 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.222+0000 7f5006fde700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5000101510 0x7f5000196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:11:51.229 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.222+0000 7f5006fde700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5000101ea0 0x7f500019d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:11:51.229 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.222+0000 7f5006fde700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500019dbb0 con 0x7f5000100310 2023-11-27T15:11:51.229 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.222+0000 7f5006fde700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500019dcf0 con 0x7f5000101510 2023-11-27T15:11:51.229 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.222+0000 7f5006fde700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500019de80 con 0x7f5000101ea0 2023-11-27T15:11:51.229 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.222+0000 7f500557b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5000101ea0 0x7f500019d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:11:51.229 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.222+0000 7f500557b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5000101ea0 0x7f500019d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49378/0 (socket says 172.21.15.150:49378) 2023-11-27T15:11:51.230 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.222+0000 7f500557b700 1 -- 172.21.15.150:0/1739228354 learned_addr learned my addr 172.21.15.150:0/1739228354 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-27T15:11:51.230 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.222+0000 7f5004d7a700 1 --2- 172.21.15.150:0/1739228354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5000100310 0x7f5000196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:11:51.230 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.222+0000 7f4ffffff700 1 --2- 172.21.15.150:0/1739228354 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5000101510 0x7f5000196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:11:51.230 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.222+0000 7f500557b700 1 -- 172.21.15.150:0/1739228354 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5000101510 msgr2=0x7f5000196560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:11:51.230 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.222+0000 7f500557b700 1 --2- 172.21.15.150:0/1739228354 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5000101510 0x7f5000196560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:11:51.230 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.222+0000 7f500557b700 1 -- 172.21.15.150:0/1739228354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5000100310 msgr2=0x7f5000196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:11:51.230 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.222+0000 7f500557b700 1 --2- 172.21.15.150:0/1739228354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5000100310 0x7f5000196020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:11:51.230 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.222+0000 7f500557b700 1 -- 172.21.15.150:0/1739228354 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fec0095f0 con 0x7f5000101ea0 2023-11-27T15:11:51.230 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.223+0000 7f500557b700 1 --2- 172.21.15.150:0/1739228354 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5000101ea0 0x7f500019d5e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4ff800add0 tx=0x7f4ff800c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:11:51.230 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.223+0000 7f4ffdffb700 1 -- 172.21.15.150:0/1739228354 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff800f070 con 0x7f5000101ea0 2023-11-27T15:11:51.230 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.223+0000 7f4ffdffb700 1 -- 172.21.15.150:0/1739228354 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ff8007b30 con 0x7f5000101ea0 2023-11-27T15:11:51.230 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.223+0000 7f5006fde700 1 -- 172.21.15.150:0/1739228354 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f500010bc90 con 0x7f5000101ea0 2023-11-27T15:11:51.231 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.223+0000 7f4ffdffb700 1 -- 172.21.15.150:0/1739228354 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8005310 con 0x7f5000101ea0 2023-11-27T15:11:51.231 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.223+0000 7f5006fde700 1 -- 172.21.15.150:0/1739228354 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f500010bb00 con 0x7f5000101ea0 2023-11-27T15:11:51.232 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.225+0000 7f5006fde700 1 -- 172.21.15.150:0/1739228354 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5000066ea0 con 0x7f5000101ea0 2023-11-27T15:11:51.232 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.225+0000 7f4ffdffb700 1 -- 172.21.15.150:0/1739228354 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f4ff800c940 con 0x7f5000101ea0 2023-11-27T15:11:51.232 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.226+0000 7f4ffdffb700 1 --2- 172.21.15.150:0/1739228354 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4ff00a11c0 0x7f4ff00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:11:51.232 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.226+0000 7f4ffdffb700 1 -- 172.21.15.150:0/1739228354 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f4ff80ca690 con 0x7f5000101ea0 2023-11-27T15:11:51.232 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.226+0000 7f5004d7a700 1 --2- 172.21.15.150:0/1739228354 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4ff00a11c0 0x7f4ff00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:11:51.232 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.226+0000 7f5004d7a700 1 --2- 172.21.15.150:0/1739228354 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4ff00a11c0 0x7f4ff00a3670 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5000101370 tx=0x7f4ff400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:11:51.235 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.230+0000 7f4ffdffb700 1 -- 172.21.15.150:0/1739228354 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4ff8095620 con 0x7f5000101ea0 2023-11-27T15:11:51.429 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:11:51.424+0000 7f5006fde700 1 -- 172.21.15.150:0/1739228354 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f5000060f30 con 0x7f4ff00a11c0 2023-11-27T15:11:52.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:51 smithi150 ceph-mon[100559]: osdmap e14: 2 total, 2 up, 2 in 2023-11-27T15:11:52.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:51 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:11:52.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:51 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:11:52.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:51 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:52.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:51 smithi102 ceph-mon[100296]: osdmap e14: 2 total, 2 up, 2 in 2023-11-27T15:11:52.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:51 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:11:52.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:51 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:11:52.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:51 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:52.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:51 smithi052 ceph-mon[99056]: osdmap e14: 2 total, 2 up, 2 in 2023-11-27T15:11:52.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:51 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:11:52.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:51 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:11:52.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:51 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:11:52.903 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:52 smithi150 ceph-mon[100559]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:11:52.904 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:52 smithi150 ceph-mon[100559]: pgmap v58: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:11:53.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:52 smithi102 ceph-mon[100296]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:11:53.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:52 smithi102 ceph-mon[100296]: pgmap v58: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:11:53.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:52 smithi052 ceph-mon[99056]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:11:53.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:52 smithi052 ceph-mon[99056]: pgmap v58: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:11:54.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:54 smithi150 ceph-mon[100559]: pgmap v59: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:11:54.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:54 smithi150 ceph-mon[100559]: from='client.? 172.21.15.150:0/2697789870' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "846ba125-11f8-46b2-8941-bd5e1947cc8a"}]: dispatch 2023-11-27T15:11:54.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:54 smithi150 ceph-mon[100559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "846ba125-11f8-46b2-8941-bd5e1947cc8a"}]: dispatch 2023-11-27T15:11:54.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:54 smithi150 ceph-mon[100559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "846ba125-11f8-46b2-8941-bd5e1947cc8a"}]': finished 2023-11-27T15:11:54.530 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:54 smithi150 ceph-mon[100559]: osdmap e15: 3 total, 2 up, 3 in 2023-11-27T15:11:54.530 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:54 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:11:54.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:54 smithi102 ceph-mon[100296]: pgmap v59: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:11:54.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:54 smithi102 ceph-mon[100296]: from='client.? 172.21.15.150:0/2697789870' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "846ba125-11f8-46b2-8941-bd5e1947cc8a"}]: dispatch 2023-11-27T15:11:54.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:54 smithi102 ceph-mon[100296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "846ba125-11f8-46b2-8941-bd5e1947cc8a"}]: dispatch 2023-11-27T15:11:54.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:54 smithi102 ceph-mon[100296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "846ba125-11f8-46b2-8941-bd5e1947cc8a"}]': finished 2023-11-27T15:11:54.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:54 smithi102 ceph-mon[100296]: osdmap e15: 3 total, 2 up, 3 in 2023-11-27T15:11:54.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:54 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:11:54.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:54 smithi052 ceph-mon[99056]: pgmap v59: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:11:54.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:54 smithi052 ceph-mon[99056]: from='client.? 172.21.15.150:0/2697789870' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "846ba125-11f8-46b2-8941-bd5e1947cc8a"}]: dispatch 2023-11-27T15:11:54.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:54 smithi052 ceph-mon[99056]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "846ba125-11f8-46b2-8941-bd5e1947cc8a"}]: dispatch 2023-11-27T15:11:54.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:54 smithi052 ceph-mon[99056]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "846ba125-11f8-46b2-8941-bd5e1947cc8a"}]': finished 2023-11-27T15:11:54.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:54 smithi052 ceph-mon[99056]: osdmap e15: 3 total, 2 up, 3 in 2023-11-27T15:11:54.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:54 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:11:55.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:55 smithi150 ceph-mon[100559]: from='client.? 172.21.15.150:0/2027456970' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:11:55.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:55 smithi102 ceph-mon[100296]: from='client.? 172.21.15.150:0/2027456970' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:11:55.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:55 smithi052 ceph-mon[99056]: from='client.? 172.21.15.150:0/2027456970' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:11:56.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:56 smithi150 ceph-mon[100559]: pgmap v61: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:11:56.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:56 smithi102 ceph-mon[100296]: pgmap v61: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:11:56.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:56 smithi052 ceph-mon[99056]: pgmap v61: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:11:58.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:57 smithi150 ceph-mon[100559]: pgmap v62: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:11:58.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:57 smithi102 ceph-mon[100296]: pgmap v62: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:11:58.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:57 smithi052 ceph-mon[99056]: pgmap v62: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:12:00.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:11:59 smithi150 ceph-mon[100559]: pgmap v63: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:12:00.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:11:59 smithi102 ceph-mon[100296]: pgmap v63: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:12:00.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:11:59 smithi052 ceph-mon[99056]: pgmap v63: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:12:01.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:00 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-11-27T15:12:01.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:00 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:01.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:00 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-11-27T15:12:01.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:00 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:01.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:00 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-11-27T15:12:01.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:00 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:02.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:01 smithi150 ceph-mon[100559]: Deploying daemon osd.2 on smithi150 2023-11-27T15:12:02.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:01 smithi150 ceph-mon[100559]: pgmap v64: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:12:02.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:01 smithi102 ceph-mon[100296]: Deploying daemon osd.2 on smithi150 2023-11-27T15:12:02.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:01 smithi102 ceph-mon[100296]: pgmap v64: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:12:02.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:01 smithi052 ceph-mon[99056]: Deploying daemon osd.2 on smithi150 2023-11-27T15:12:02.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:01 smithi052 ceph-mon[99056]: pgmap v64: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:12:04.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:04 smithi150 ceph-mon[100559]: pgmap v65: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:12:04.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:04 smithi102 ceph-mon[100296]: pgmap v65: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:12:04.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:04 smithi052 ceph-mon[99056]: pgmap v65: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:12:06.712 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:06 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:06.712 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:06 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:06.712 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:06 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:12:06.712 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:06 smithi150 ceph-mon[100559]: pgmap v66: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:12:06.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:06 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:06.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:06 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:06.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:06 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:12:06.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:06 smithi102 ceph-mon[100296]: pgmap v66: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:12:06.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:06 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:06.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:06 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:06.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:06 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:12:06.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:06 smithi052 ceph-mon[99056]: pgmap v66: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:12:08.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:07 smithi150 ceph-mon[100559]: pgmap v67: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:12:08.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:07 smithi102 ceph-mon[100296]: pgmap v67: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:12:08.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:07 smithi052 ceph-mon[99056]: pgmap v67: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:12:08.345 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:08.340+0000 7f4ffdffb700 1 -- 172.21.15.150:0/1739228354 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5000060f30 con 0x7f4ff00a11c0 2023-11-27T15:12:08.345 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 2 on host 'smithi150' 2023-11-27T15:12:08.348 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:08.342+0000 7f5006fde700 1 -- 172.21.15.150:0/1739228354 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4ff00a11c0 msgr2=0x7f4ff00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:12:08.348 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:08.342+0000 7f5006fde700 1 --2- 172.21.15.150:0/1739228354 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4ff00a11c0 0x7f4ff00a3670 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5000101370 tx=0x7f4ff400a380 comp rx=0 tx=0).stop 2023-11-27T15:12:08.348 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:08.342+0000 7f5006fde700 1 -- 172.21.15.150:0/1739228354 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5000101ea0 msgr2=0x7f500019d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:12:08.348 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:08.342+0000 7f5006fde700 1 --2- 172.21.15.150:0/1739228354 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5000101ea0 0x7f500019d5e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4ff800add0 tx=0x7f4ff800c330 comp rx=0 tx=0).stop 2023-11-27T15:12:08.348 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:08.343+0000 7f5006fde700 1 -- 172.21.15.150:0/1739228354 shutdown_connections 2023-11-27T15:12:08.348 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:08.343+0000 7f5006fde700 1 --2- 172.21.15.150:0/1739228354 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4ff00a11c0 0x7f4ff00a3670 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:12:08.348 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:08.343+0000 7f5006fde700 1 --2- 172.21.15.150:0/1739228354 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5000101ea0 0x7f500019d5e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:12:08.348 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:08.343+0000 7f5006fde700 1 --2- 172.21.15.150:0/1739228354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5000100310 0x7f5000196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:12:08.349 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:08.343+0000 7f5006fde700 1 --2- 172.21.15.150:0/1739228354 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5000101510 0x7f5000196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:12:08.349 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:08.343+0000 7f5006fde700 1 -- 172.21.15.150:0/1739228354 >> 172.21.15.150:0/1739228354 conn(0x7f50000fb960 msgr2=0x7f50000fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:12:08.349 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:08.343+0000 7f5006fde700 1 -- 172.21.15.150:0/1739228354 shutdown_connections 2023-11-27T15:12:08.349 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:08.343+0000 7f5006fde700 1 -- 172.21.15.150:0/1739228354 wait complete. 2023-11-27T15:12:08.755 DEBUG:teuthology.orchestra.run.smithi150:osd.2> sudo journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.2.service 2023-11-27T15:12:08.757 INFO:tasks.cephadm:Deploying osd.3 on smithi052 with /dev/vg_nvme/lv_3... 2023-11-27T15:12:08.757 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-11-27T15:12:08.786 INFO:journalctl@ceph.osd.2.smithi150.stdout:-- Logs begin at Mon 2023-11-27 14:53:12 UTC. -- 2023-11-27T15:12:09.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:08 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:09.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:08 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:09.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:08 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:09.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:08 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:12:09.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:08 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:12:09.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:08 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:09.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:08 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:12:09.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:08 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:12:09.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:08 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:09.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:08 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:09.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:08 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:12:09.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:08 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:09.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:08 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:09.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:08 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:09.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:08 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:12:09.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:08 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:12:09.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:08 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:09.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:08 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:12:09.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:08 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:12:09.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:08 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:09.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:08 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:09.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:08 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:12:09.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:08 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:09.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:08 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:09.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:08 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:09.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:08 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:12:09.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:08 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:12:09.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:08 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:09.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:08 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:12:09.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:08 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:12:09.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:08 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:09.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:08 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:09.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:08 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:12:09.429 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:12:09.529 INFO:journalctl@ceph.osd.2.smithi150.stdout:Nov 27 15:12:09 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-2[106369]: 2023-11-27T15:12:09.427+0000 7fa30179a640 -1 osd.2 0 log_to_monitors true 2023-11-27T15:12:09.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:09 smithi052 ceph-mon[99056]: from='osd.2 [v2:172.21.15.150:6800/4258038221,v1:172.21.15.150:6801/4258038221]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-27T15:12:09.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:09 smithi052 ceph-mon[99056]: pgmap v68: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:12:09.879 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:09 smithi150 ceph-mon[100559]: from='osd.2 [v2:172.21.15.150:6800/4258038221,v1:172.21.15.150:6801/4258038221]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-27T15:12:09.880 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:09 smithi150 ceph-mon[100559]: pgmap v68: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:12:10.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:09 smithi102 ceph-mon[100296]: from='osd.2 [v2:172.21.15.150:6800/4258038221,v1:172.21.15.150:6801/4258038221]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-27T15:12:10.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:09 smithi102 ceph-mon[100296]: pgmap v68: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:12:10.896 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-11-27T15:12:10.896 INFO:teuthology.orchestra.run.smithi052.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-11-27T15:12:10.896 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10+0 records in 2023-11-27T15:12:10.897 INFO:teuthology.orchestra.run.smithi052.stderr:10+0 records out 2023-11-27T15:12:10.897 INFO:teuthology.orchestra.run.smithi052.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00817554 s, 1.3 GB/s 2023-11-27T15:12:10.897 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping successful for: 2023-11-27T15:12:11.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:10 smithi150 ceph-mon[100559]: from='osd.2 [v2:172.21.15.150:6800/4258038221,v1:172.21.15.150:6801/4258038221]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-11-27T15:12:11.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:10 smithi150 ceph-mon[100559]: osdmap e16: 3 total, 2 up, 3 in 2023-11-27T15:12:11.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:10 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:12:11.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:10 smithi150 ceph-mon[100559]: from='osd.2 [v2:172.21.15.150:6800/4258038221,v1:172.21.15.150:6801/4258038221]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2023-11-27T15:12:11.030 INFO:journalctl@ceph.osd.2.smithi150.stdout:Nov 27 15:12:10 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-2[106369]: 2023-11-27T15:12:10.775+0000 7fa2f6e11700 -1 osd.2 0 waiting for initial osdmap 2023-11-27T15:12:11.030 INFO:journalctl@ceph.osd.2.smithi150.stdout:Nov 27 15:12:10 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-2[106369]: 2023-11-27T15:12:10.780+0000 7fa2f2c06700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:12:11.070 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:10 smithi052 ceph-mon[99056]: from='osd.2 [v2:172.21.15.150:6800/4258038221,v1:172.21.15.150:6801/4258038221]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-11-27T15:12:11.070 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:10 smithi052 ceph-mon[99056]: osdmap e16: 3 total, 2 up, 3 in 2023-11-27T15:12:11.070 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:10 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:12:11.071 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:10 smithi052 ceph-mon[99056]: from='osd.2 [v2:172.21.15.150:6800/4258038221,v1:172.21.15.150:6801/4258038221]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2023-11-27T15:12:11.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:10 smithi102 ceph-mon[100296]: from='osd.2 [v2:172.21.15.150:6800/4258038221,v1:172.21.15.150:6801/4258038221]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-11-27T15:12:11.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:10 smithi102 ceph-mon[100296]: osdmap e16: 3 total, 2 up, 3 in 2023-11-27T15:12:11.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:10 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:12:11.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:10 smithi102 ceph-mon[100296]: from='osd.2 [v2:172.21.15.150:6800/4258038221,v1:172.21.15.150:6801/4258038221]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2023-11-27T15:12:11.374 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi052:vg_nvme/lv_3 2023-11-27T15:12:12.027 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:12:12.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:11 smithi102 ceph-mon[100296]: from='osd.2 [v2:172.21.15.150:6800/4258038221,v1:172.21.15.150:6801/4258038221]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2023-11-27T15:12:12.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:11 smithi102 ceph-mon[100296]: osdmap e17: 3 total, 2 up, 3 in 2023-11-27T15:12:12.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:11 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:12:12.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:11 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:12:12.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:11 smithi102 ceph-mon[100296]: Detected new or changed devices on smithi150 2023-11-27T15:12:12.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:11 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:12.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:11 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:12.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:11 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:12:12.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:11 smithi102 ceph-mon[100296]: Adjusting osd_memory_target on smithi150 to 17168M 2023-11-27T15:12:12.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:11 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:12.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:11 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:12.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:11 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:12:12.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:11 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:12:12.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:11 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:12.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:11 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:12:12.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:11 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:12:12.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:11 smithi102 ceph-mon[100296]: pgmap v71: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:12:12.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:11 smithi052 ceph-mon[99056]: from='osd.2 [v2:172.21.15.150:6800/4258038221,v1:172.21.15.150:6801/4258038221]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2023-11-27T15:12:12.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:11 smithi052 ceph-mon[99056]: osdmap e17: 3 total, 2 up, 3 in 2023-11-27T15:12:12.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:11 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:12:12.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:11 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:12:12.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:11 smithi052 ceph-mon[99056]: Detected new or changed devices on smithi150 2023-11-27T15:12:12.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:11 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:12.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:11 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:12.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:11 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:12:12.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:11 smithi052 ceph-mon[99056]: Adjusting osd_memory_target on smithi150 to 17168M 2023-11-27T15:12:12.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:11 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:12.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:11 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:12.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:11 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:12:12.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:11 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:12:12.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:11 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:12.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:11 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:12:12.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:11 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:12:12.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:11 smithi052 ceph-mon[99056]: pgmap v71: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:12:12.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:11 smithi150 ceph-mon[100559]: from='osd.2 [v2:172.21.15.150:6800/4258038221,v1:172.21.15.150:6801/4258038221]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2023-11-27T15:12:12.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:11 smithi150 ceph-mon[100559]: osdmap e17: 3 total, 2 up, 3 in 2023-11-27T15:12:12.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:11 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:12:12.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:11 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:12:12.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:11 smithi150 ceph-mon[100559]: Detected new or changed devices on smithi150 2023-11-27T15:12:12.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:11 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:12.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:11 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:12.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:11 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:12:12.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:11 smithi150 ceph-mon[100559]: Adjusting osd_memory_target on smithi150 to 17168M 2023-11-27T15:12:12.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:11 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:12.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:11 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:12.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:11 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:12:12.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:11 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:12:12.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:11 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:12.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:11 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:12:12.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:11 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:12:12.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:11 smithi150 ceph-mon[100559]: pgmap v71: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:12:12.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:12 smithi052 ceph-mon[99056]: purged_snaps scrub starts 2023-11-27T15:12:12.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:12 smithi052 ceph-mon[99056]: purged_snaps scrub ok 2023-11-27T15:12:12.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:12 smithi052 ceph-mon[99056]: osd.2 [v2:172.21.15.150:6800/4258038221,v1:172.21.15.150:6801/4258038221] boot 2023-11-27T15:12:12.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:12 smithi052 ceph-mon[99056]: osdmap e18: 3 total, 3 up, 3 in 2023-11-27T15:12:12.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:12 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:12:13.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:12 smithi102 ceph-mon[100296]: purged_snaps scrub starts 2023-11-27T15:12:13.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:12 smithi102 ceph-mon[100296]: purged_snaps scrub ok 2023-11-27T15:12:13.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:12 smithi102 ceph-mon[100296]: osd.2 [v2:172.21.15.150:6800/4258038221,v1:172.21.15.150:6801/4258038221] boot 2023-11-27T15:12:13.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:12 smithi102 ceph-mon[100296]: osdmap e18: 3 total, 3 up, 3 in 2023-11-27T15:12:13.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:12 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:12:13.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:12 smithi150 ceph-mon[100559]: purged_snaps scrub starts 2023-11-27T15:12:13.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:12 smithi150 ceph-mon[100559]: purged_snaps scrub ok 2023-11-27T15:12:13.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:12 smithi150 ceph-mon[100559]: osd.2 [v2:172.21.15.150:6800/4258038221,v1:172.21.15.150:6801/4258038221] boot 2023-11-27T15:12:13.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:12 smithi150 ceph-mon[100559]: osdmap e18: 3 total, 3 up, 3 in 2023-11-27T15:12:13.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:12 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:12:13.371 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.365+0000 7ff8958a0700 1 -- 172.21.15.52:0/1521295434 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8901036f0 msgr2=0x7ff890105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:12:13.371 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.365+0000 7ff8958a0700 1 --2- 172.21.15.52:0/1521295434 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8901036f0 0x7ff890105ad0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff880009990 tx=0x7ff880009ca0 comp rx=0 tx=0).stop 2023-11-27T15:12:13.371 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.365+0000 7ff8958a0700 1 -- 172.21.15.52:0/1521295434 shutdown_connections 2023-11-27T15:12:13.371 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.365+0000 7ff8958a0700 1 --2- 172.21.15.52:0/1521295434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff890100dd0 0x7ff8901031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:12:13.371 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.365+0000 7ff8958a0700 1 --2- 172.21.15.52:0/1521295434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff890106010 0x7ff890108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:12:13.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.365+0000 7ff8958a0700 1 --2- 172.21.15.52:0/1521295434 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8901036f0 0x7ff890105ad0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:12:13.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.365+0000 7ff8958a0700 1 -- 172.21.15.52:0/1521295434 >> 172.21.15.52:0/1521295434 conn(0x7ff8900fa7f0 msgr2=0x7ff8900fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:12:13.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.365+0000 7ff8958a0700 1 -- 172.21.15.52:0/1521295434 shutdown_connections 2023-11-27T15:12:13.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.365+0000 7ff8958a0700 1 -- 172.21.15.52:0/1521295434 wait complete. 2023-11-27T15:12:13.374 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.366+0000 7ff8958a0700 1 Processor -- start 2023-11-27T15:12:13.374 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.366+0000 7ff8958a0700 1 -- start start 2023-11-27T15:12:13.374 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.366+0000 7ff8958a0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff890100dd0 0x7ff8901960d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:12:13.374 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.366+0000 7ff8958a0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8901036f0 0x7ff890196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:12:13.374 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.366+0000 7ff8958a0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff890106010 0x7ff89019d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:12:13.374 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.366+0000 7ff8958a0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89019dc60 con 0x7ff890106010 2023-11-27T15:12:13.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.367+0000 7ff8958a0700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89019dda0 con 0x7ff8901036f0 2023-11-27T15:12:13.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.367+0000 7ff8958a0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89019df30 con 0x7ff890100dd0 2023-11-27T15:12:13.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.367+0000 7ff88f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff890106010 0x7ff89019d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:12:13.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.367+0000 7ff88f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff890106010 0x7ff89019d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49206/0 (socket says 172.21.15.52:49206) 2023-11-27T15:12:13.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.367+0000 7ff88f7fe700 1 -- 172.21.15.52:0/4258306309 learned_addr learned my addr 172.21.15.52:0/4258306309 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:12:13.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.367+0000 7ff88effd700 1 --2- 172.21.15.52:0/4258306309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff890100dd0 0x7ff8901960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:12:13.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.367+0000 7ff88e7fc700 1 --2- 172.21.15.52:0/4258306309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8901036f0 0x7ff890196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:12:13.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.367+0000 7ff88f7fe700 1 -- 172.21.15.52:0/4258306309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff890100dd0 msgr2=0x7ff8901960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:12:13.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.367+0000 7ff88f7fe700 1 --2- 172.21.15.52:0/4258306309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff890100dd0 0x7ff8901960d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:12:13.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.367+0000 7ff88f7fe700 1 -- 172.21.15.52:0/4258306309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8901036f0 msgr2=0x7ff890196610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:12:13.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.367+0000 7ff88f7fe700 1 --2- 172.21.15.52:0/4258306309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8901036f0 0x7ff890196610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:12:13.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.367+0000 7ff88f7fe700 1 -- 172.21.15.52:0/4258306309 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8800095f0 con 0x7ff890106010 2023-11-27T15:12:13.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.367+0000 7ff88e7fc700 1 --2- 172.21.15.52:0/4258306309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8901036f0 0x7ff890196610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:12:13.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.367+0000 7ff88f7fe700 1 --2- 172.21.15.52:0/4258306309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff890106010 0x7ff89019d690 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff888007ee0 tx=0x7ff88800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:12:13.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.367+0000 7ff89489e700 1 -- 172.21.15.52:0/4258306309 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff888004db0 con 0x7ff890106010 2023-11-27T15:12:13.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.367+0000 7ff89489e700 1 -- 172.21.15.52:0/4258306309 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff88800f040 con 0x7ff890106010 2023-11-27T15:12:13.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.367+0000 7ff8958a0700 1 -- 172.21.15.52:0/4258306309 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff89010bcd0 con 0x7ff890106010 2023-11-27T15:12:13.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.367+0000 7ff89489e700 1 -- 172.21.15.52:0/4258306309 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff888005910 con 0x7ff890106010 2023-11-27T15:12:13.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.367+0000 7ff8958a0700 1 -- 172.21.15.52:0/4258306309 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff89010bb40 con 0x7ff890106010 2023-11-27T15:12:13.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.368+0000 7ff8958a0700 1 -- 172.21.15.52:0/4258306309 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8900fc360 con 0x7ff890106010 2023-11-27T15:12:13.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.369+0000 7ff89489e700 1 -- 172.21.15.52:0/4258306309 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7ff888005020 con 0x7ff890106010 2023-11-27T15:12:13.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.370+0000 7ff89489e700 1 --2- 172.21.15.52:0/4258306309 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff8780a10c0 0x7ff8780a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:12:13.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.370+0000 7ff88effd700 1 --2- 172.21.15.52:0/4258306309 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff8780a10c0 0x7ff8780a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:12:13.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.370+0000 7ff89489e700 1 -- 172.21.15.52:0/4258306309 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 2851+0+0 (secure 0 0 0) 0x7ff8880cb460 con 0x7ff890106010 2023-11-27T15:12:13.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.371+0000 7ff88effd700 1 --2- 172.21.15.52:0/4258306309 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff8780a10c0 0x7ff8780a3570 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff884005950 tx=0x7ff88400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:12:13.379 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.374+0000 7ff89489e700 1 -- 172.21.15.52:0/4258306309 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff8880961e0 con 0x7ff890106010 2023-11-27T15:12:13.585 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:13.580+0000 7ff8958a0700 1 -- 172.21.15.52:0/4258306309 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff890061c20 con 0x7ff8780a10c0 2023-11-27T15:12:14.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:14 smithi052 ceph-mon[99056]: osdmap e19: 3 total, 3 up, 3 in 2023-11-27T15:12:14.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:14 smithi052 ceph-mon[99056]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:12:14.475 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:14 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:12:14.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:14 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:12:14.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:14 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:14.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:14 smithi052 ceph-mon[99056]: pgmap v74: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:14.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:14 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-11-27T15:12:14.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:14 smithi150 ceph-mon[100559]: osdmap e19: 3 total, 3 up, 3 in 2023-11-27T15:12:14.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:14 smithi150 ceph-mon[100559]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:12:14.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:14 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:12:14.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:14 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:12:14.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:14 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:14.530 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:14 smithi150 ceph-mon[100559]: pgmap v74: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:14.530 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:14 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-11-27T15:12:14.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:14 smithi102 ceph-mon[100296]: osdmap e19: 3 total, 3 up, 3 in 2023-11-27T15:12:14.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:14 smithi102 ceph-mon[100296]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:12:14.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:14 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:12:14.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:14 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:12:14.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:14 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:14.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:14 smithi102 ceph-mon[100296]: pgmap v74: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:14.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:14 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-11-27T15:12:15.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:15 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-11-27T15:12:15.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:15 smithi150 ceph-mon[100559]: osdmap e20: 3 total, 3 up, 3 in 2023-11-27T15:12:15.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:15 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-11-27T15:12:15.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:15 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-11-27T15:12:15.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:15 smithi102 ceph-mon[100296]: osdmap e20: 3 total, 3 up, 3 in 2023-11-27T15:12:15.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:15 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-11-27T15:12:15.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:15 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-11-27T15:12:15.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:15 smithi052 ceph-mon[99056]: osdmap e20: 3 total, 3 up, 3 in 2023-11-27T15:12:15.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:15 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-11-27T15:12:16.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:16 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-11-27T15:12:16.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:16 smithi150 ceph-mon[100559]: osdmap e21: 3 total, 3 up, 3 in 2023-11-27T15:12:16.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:16 smithi150 ceph-mon[100559]: pgmap v77: 1 pgs: 1 creating+activating; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:16.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:16 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2522553534' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "950fcd67-1057-4288-a389-714f1ec94261"}]: dispatch 2023-11-27T15:12:16.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:16 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2522553534' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "950fcd67-1057-4288-a389-714f1ec94261"}]': finished 2023-11-27T15:12:16.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:16 smithi150 ceph-mon[100559]: osdmap e22: 4 total, 3 up, 4 in 2023-11-27T15:12:16.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:16 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:12:16.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:16 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-11-27T15:12:16.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:16 smithi102 ceph-mon[100296]: osdmap e21: 3 total, 3 up, 3 in 2023-11-27T15:12:16.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:16 smithi102 ceph-mon[100296]: pgmap v77: 1 pgs: 1 creating+activating; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:16.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:16 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2522553534' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "950fcd67-1057-4288-a389-714f1ec94261"}]: dispatch 2023-11-27T15:12:16.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:16 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2522553534' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "950fcd67-1057-4288-a389-714f1ec94261"}]': finished 2023-11-27T15:12:16.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:16 smithi102 ceph-mon[100296]: osdmap e22: 4 total, 3 up, 4 in 2023-11-27T15:12:16.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:16 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:12:16.617 INFO:journalctl@ceph.osd.0.smithi052.stdout:Nov 27 15:12:16 smithi052 sudo[115021]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-27T15:12:16.617 INFO:journalctl@ceph.osd.0.smithi052.stdout:Nov 27 15:12:16 smithi052 sudo[115021]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:12:16.618 INFO:journalctl@ceph.osd.0.smithi052.stdout:Nov 27 15:12:16 smithi052 sudo[115021]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:12:16.618 INFO:journalctl@ceph.osd.0.smithi052.stdout:Nov 27 15:12:16 smithi052 sudo[115021]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:12:16.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:16 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-11-27T15:12:16.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:16 smithi052 ceph-mon[99056]: osdmap e21: 3 total, 3 up, 3 in 2023-11-27T15:12:16.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:16 smithi052 ceph-mon[99056]: pgmap v77: 1 pgs: 1 creating+activating; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:16.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:16 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2522553534' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "950fcd67-1057-4288-a389-714f1ec94261"}]: dispatch 2023-11-27T15:12:16.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:16 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2522553534' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "950fcd67-1057-4288-a389-714f1ec94261"}]': finished 2023-11-27T15:12:16.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:16 smithi052 ceph-mon[99056]: osdmap e22: 4 total, 3 up, 4 in 2023-11-27T15:12:16.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:16 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:12:17.117 INFO:journalctl@ceph.osd.0.smithi052.stdout:Nov 27 15:12:16 smithi052 sudo[115053]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-27T15:12:17.117 INFO:journalctl@ceph.osd.0.smithi052.stdout:Nov 27 15:12:16 smithi052 sudo[115053]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:12:17.117 INFO:journalctl@ceph.osd.0.smithi052.stdout:Nov 27 15:12:16 smithi052 sudo[115053]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:12:17.117 INFO:journalctl@ceph.osd.0.smithi052.stdout:Nov 27 15:12:16 smithi052 sudo[115053]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:12:17.152 INFO:journalctl@ceph.osd.1.smithi102.stdout:Nov 27 15:12:16 smithi102 sudo[107705]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-27T15:12:17.152 INFO:journalctl@ceph.osd.1.smithi102.stdout:Nov 27 15:12:16 smithi102 sudo[107705]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:12:17.152 INFO:journalctl@ceph.osd.1.smithi102.stdout:Nov 27 15:12:16 smithi102 sudo[107705]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:12:17.152 INFO:journalctl@ceph.osd.1.smithi102.stdout:Nov 27 15:12:16 smithi102 sudo[107705]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:12:17.208 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:17.204+0000 7ff89489e700 1 -- 172.21.15.52:0/4258306309 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff8880965b0 con 0x7ff890106010 2023-11-27T15:12:17.578 INFO:journalctl@ceph.osd.1.smithi102.stdout:Nov 27 15:12:17 smithi102 sudo[107708]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-27T15:12:17.578 INFO:journalctl@ceph.osd.1.smithi102.stdout:Nov 27 15:12:17 smithi102 sudo[107708]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:12:17.578 INFO:journalctl@ceph.osd.1.smithi102.stdout:Nov 27 15:12:17 smithi102 sudo[107708]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:12:17.578 INFO:journalctl@ceph.osd.1.smithi102.stdout:Nov 27 15:12:17 smithi102 sudo[107708]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:12:17.672 INFO:journalctl@ceph.osd.2.smithi150.stdout:Nov 27 15:12:17 smithi150 sudo[108613]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-27T15:12:17.672 INFO:journalctl@ceph.osd.2.smithi150.stdout:Nov 27 15:12:17 smithi150 sudo[108613]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:12:17.672 INFO:journalctl@ceph.osd.2.smithi150.stdout:Nov 27 15:12:17 smithi150 sudo[108613]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:12:17.672 INFO:journalctl@ceph.osd.2.smithi150.stdout:Nov 27 15:12:17 smithi150 sudo[108613]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:12:17.672 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:17 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:17.672 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:17 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3585944997' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:12:17.672 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:17 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:17.672 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:17 smithi150 ceph-mon[100559]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-11-27T15:12:18.029 INFO:journalctl@ceph.osd.2.smithi150.stdout:Nov 27 15:12:17 smithi150 sudo[108616]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-27T15:12:18.029 INFO:journalctl@ceph.osd.2.smithi150.stdout:Nov 27 15:12:17 smithi150 sudo[108616]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:12:18.029 INFO:journalctl@ceph.osd.2.smithi150.stdout:Nov 27 15:12:17 smithi150 sudo[108616]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:12:18.029 INFO:journalctl@ceph.osd.2.smithi150.stdout:Nov 27 15:12:17 smithi150 sudo[108616]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:12:18.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:17 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:18.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:17 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3585944997' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:12:18.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:17 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:18.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:17 smithi102 ceph-mon[100296]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-11-27T15:12:18.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:17 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:18.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:17 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3585944997' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:12:18.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:17 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:18.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:17 smithi052 ceph-mon[99056]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-11-27T15:12:18.593 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:18 smithi052 sudo[115662]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-27T15:12:18.593 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:18 smithi052 sudo[115662]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:12:18.593 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:18 smithi052 sudo[115662]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:12:18.593 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:18 smithi052 sudo[115662]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:12:18.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:18 smithi052 sudo[115849]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-27T15:12:18.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:18 smithi052 sudo[115849]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:12:18.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:18 smithi052 sudo[115849]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:12:18.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:18 smithi052 sudo[115849]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:12:19.092 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:18 smithi150 sudo[108619]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-27T15:12:19.092 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:18 smithi150 sudo[108619]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:12:19.092 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:18 smithi150 sudo[108619]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:12:19.092 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:18 smithi150 sudo[108619]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:12:19.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:19 smithi150 sudo[108622]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-27T15:12:19.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:19 smithi150 sudo[108622]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:12:19.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:19 smithi150 sudo[108622]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:12:19.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:19 smithi150 sudo[108622]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:12:19.586 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:19 smithi102 sudo[107711]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-27T15:12:19.586 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:19 smithi102 sudo[107711]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:12:19.586 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:19 smithi102 sudo[107711]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:12:19.586 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:19 smithi102 sudo[107711]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:12:19.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:19 smithi052 ceph-mon[99056]: pgmap v79: 1 pgs: 1 creating+activating; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:19.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:19 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:19.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:19 smithi052 ceph-mon[99056]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-27T15:12:19.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:19 smithi052 ceph-mon[99056]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-27T15:12:19.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:19 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:12:19.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:19 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:12:19.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:19 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:12:19.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:19 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:12:19.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:19 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:12:19.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:19 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:12:20.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:19 smithi150 ceph-mon[100559]: pgmap v79: 1 pgs: 1 creating+activating; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:20.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:19 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:20.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:19 smithi150 ceph-mon[100559]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-27T15:12:20.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:19 smithi150 ceph-mon[100559]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-27T15:12:20.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:19 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:12:20.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:19 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:12:20.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:19 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:12:20.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:19 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:12:20.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:19 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:12:20.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:19 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:12:20.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:19 smithi102 sudo[107714]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-27T15:12:20.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:19 smithi102 sudo[107714]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:12:20.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:19 smithi102 sudo[107714]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:12:20.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:19 smithi102 sudo[107714]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:12:20.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:19 smithi102 ceph-mon[100296]: pgmap v79: 1 pgs: 1 creating+activating; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:20.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:19 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:20.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:19 smithi102 ceph-mon[100296]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-27T15:12:20.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:19 smithi102 ceph-mon[100296]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-27T15:12:20.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:19 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:12:20.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:19 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:12:20.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:19 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:12:20.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:19 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:12:20.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:19 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:12:20.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:19 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:12:20.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:20 smithi052 ceph-mon[99056]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-27T15:12:20.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:20 smithi052 ceph-mon[99056]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-27T15:12:20.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:20 smithi052 ceph-mon[99056]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-27T15:12:20.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:20 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:12:20.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:20 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:12:20.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:20 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:12:20.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:20 smithi052 ceph-mon[99056]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-27T15:12:20.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:20 smithi052 ceph-mon[99056]: pgmap v80: 1 pgs: 1 creating+activating; 0 B data, 16 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:21.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:20 smithi150 ceph-mon[100559]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-27T15:12:21.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:20 smithi150 ceph-mon[100559]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-27T15:12:21.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:20 smithi150 ceph-mon[100559]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-27T15:12:21.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:20 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:12:21.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:20 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:12:21.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:20 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:12:21.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:20 smithi150 ceph-mon[100559]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-27T15:12:21.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:20 smithi150 ceph-mon[100559]: pgmap v80: 1 pgs: 1 creating+activating; 0 B data, 16 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:21.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:20 smithi102 ceph-mon[100296]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-27T15:12:21.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:20 smithi102 ceph-mon[100296]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-27T15:12:21.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:20 smithi102 ceph-mon[100296]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-27T15:12:21.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:20 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:12:21.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:20 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:12:21.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:20 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:12:21.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:20 smithi102 ceph-mon[100296]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-27T15:12:21.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:20 smithi102 ceph-mon[100296]: pgmap v80: 1 pgs: 1 creating+activating; 0 B data, 16 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:22.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:21 smithi150 ceph-mon[100559]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:22.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:21 smithi102 ceph-mon[100296]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:22.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:21 smithi052 ceph-mon[99056]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:23.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:23 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-11-27T15:12:23.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:23 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:23.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:23 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-11-27T15:12:23.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:23 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:23.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:23 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-11-27T15:12:23.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:23 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:24.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:24 smithi052 ceph-mon[99056]: Deploying daemon osd.3 on smithi052 2023-11-27T15:12:24.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:24 smithi052 ceph-mon[99056]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:25.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:24 smithi150 ceph-mon[100559]: Deploying daemon osd.3 on smithi052 2023-11-27T15:12:25.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:24 smithi150 ceph-mon[100559]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:25.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:24 smithi102 ceph-mon[100296]: Deploying daemon osd.3 on smithi052 2023-11-27T15:12:25.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:24 smithi102 ceph-mon[100296]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:25.854 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:25 smithi052 ceph-mon[99056]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:26.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:25 smithi150 ceph-mon[100559]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:26.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:25 smithi102 ceph-mon[100296]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:28.560 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:28 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:28.560 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:28 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:28.560 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:28 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:12:28.560 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:28 smithi052 ceph-mon[99056]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:28.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:28 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:28.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:28 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:28.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:28 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:12:28.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:28 smithi102 ceph-mon[100296]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:28.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:28 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:28.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:28 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:28.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:28 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:12:28.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:28 smithi150 ceph-mon[100559]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:30.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:29 smithi150 ceph-mon[100559]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:30.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:29 smithi102 ceph-mon[100296]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:30.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:29 smithi052 ceph-mon[99056]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:30.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:30.133+0000 7ff89489e700 1 -- 172.21.15.52:0/4258306309 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff890061c20 con 0x7ff8780a10c0 2023-11-27T15:12:30.138 INFO:teuthology.orchestra.run.smithi052.stdout:Created osd(s) 3 on host 'smithi052' 2023-11-27T15:12:30.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:30.136+0000 7ff8958a0700 1 -- 172.21.15.52:0/4258306309 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff8780a10c0 msgr2=0x7ff8780a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:12:30.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:30.136+0000 7ff8958a0700 1 --2- 172.21.15.52:0/4258306309 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff8780a10c0 0x7ff8780a3570 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff884005950 tx=0x7ff88400a300 comp rx=0 tx=0).stop 2023-11-27T15:12:30.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:30.136+0000 7ff8958a0700 1 -- 172.21.15.52:0/4258306309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff890106010 msgr2=0x7ff89019d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:12:30.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:30.136+0000 7ff8958a0700 1 --2- 172.21.15.52:0/4258306309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff890106010 0x7ff89019d690 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff888007ee0 tx=0x7ff88800d3b0 comp rx=0 tx=0).stop 2023-11-27T15:12:30.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:30.136+0000 7ff8958a0700 1 -- 172.21.15.52:0/4258306309 shutdown_connections 2023-11-27T15:12:30.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:30.136+0000 7ff8958a0700 1 --2- 172.21.15.52:0/4258306309 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff8780a10c0 0x7ff8780a3570 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:12:30.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:30.136+0000 7ff8958a0700 1 --2- 172.21.15.52:0/4258306309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff890106010 0x7ff89019d690 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:12:30.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:30.136+0000 7ff8958a0700 1 --2- 172.21.15.52:0/4258306309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff890100dd0 0x7ff8901960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:12:30.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:30.136+0000 7ff8958a0700 1 --2- 172.21.15.52:0/4258306309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8901036f0 0x7ff890196610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:12:30.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:30.136+0000 7ff8958a0700 1 -- 172.21.15.52:0/4258306309 >> 172.21.15.52:0/4258306309 conn(0x7ff8900fa7f0 msgr2=0x7ff890101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:12:30.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:30.136+0000 7ff8958a0700 1 -- 172.21.15.52:0/4258306309 shutdown_connections 2023-11-27T15:12:30.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:12:30.136+0000 7ff8958a0700 1 -- 172.21.15.52:0/4258306309 wait complete. 2023-11-27T15:12:30.536 DEBUG:teuthology.orchestra.run.smithi052:osd.3> sudo journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.3.service 2023-11-27T15:12:30.538 INFO:tasks.cephadm:Deploying osd.4 on smithi102 with /dev/vg_nvme/lv_3... 2023-11-27T15:12:30.539 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-11-27T15:12:30.572 INFO:journalctl@ceph.osd.3.smithi052.stdout:-- Logs begin at Mon 2023-11-27 14:52:43 UTC. -- 2023-11-27T15:12:31.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:30 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:31.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:30 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:31.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:30 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:31.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:30 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:31.195 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:12:31.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:30 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:31.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:30 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:31.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:30 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:31.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:30 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:31.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:30 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:31.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:30 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:31.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:30 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:31.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:30 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:31.617 INFO:journalctl@ceph.osd.3.smithi052.stdout:Nov 27 15:12:31 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-3[117304]: 2023-11-27T15:12:31.314+0000 7f8f08c36640 -1 osd.3 0 log_to_monitors true 2023-11-27T15:12:31.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:31 smithi052 ceph-mon[99056]: from='osd.3 [v2:172.21.15.52:6810/2622100752,v1:172.21.15.52:6811/2622100752]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-11-27T15:12:31.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:31 smithi052 ceph-mon[99056]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:31.947 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:31 smithi102 ceph-mon[100296]: from='osd.3 [v2:172.21.15.52:6810/2622100752,v1:172.21.15.52:6811/2622100752]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-11-27T15:12:31.948 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:31 smithi102 ceph-mon[100296]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:32.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:31 smithi150 ceph-mon[100559]: from='osd.3 [v2:172.21.15.52:6810/2622100752,v1:172.21.15.52:6811/2622100752]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-11-27T15:12:32.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:31 smithi150 ceph-mon[100559]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:32.752 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-11-27T15:12:32.752 INFO:teuthology.orchestra.run.smithi102.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-11-27T15:12:32.752 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10+0 records in 2023-11-27T15:12:32.752 INFO:teuthology.orchestra.run.smithi102.stderr:10+0 records out 2023-11-27T15:12:32.752 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110481 s, 949 MB/s 2023-11-27T15:12:32.753 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping successful for: 2023-11-27T15:12:33.071 INFO:journalctl@ceph.osd.3.smithi052.stdout:Nov 27 15:12:32 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-3[117304]: 2023-11-27T15:12:32.854+0000 7f8efe2ad700 -1 osd.3 0 waiting for initial osdmap 2023-11-27T15:12:33.071 INFO:journalctl@ceph.osd.3.smithi052.stdout:Nov 27 15:12:32 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-3[117304]: 2023-11-27T15:12:32.860+0000 7f8ef809e700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:12:33.071 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:32 smithi052 ceph-mon[99056]: from='osd.3 [v2:172.21.15.52:6810/2622100752,v1:172.21.15.52:6811/2622100752]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-11-27T15:12:33.071 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:32 smithi052 ceph-mon[99056]: osdmap e23: 4 total, 3 up, 4 in 2023-11-27T15:12:33.071 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:32 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:12:33.071 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:32 smithi052 ceph-mon[99056]: from='osd.3 [v2:172.21.15.52:6810/2622100752,v1:172.21.15.52:6811/2622100752]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2023-11-27T15:12:33.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:32 smithi102 ceph-mon[100296]: from='osd.3 [v2:172.21.15.52:6810/2622100752,v1:172.21.15.52:6811/2622100752]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-11-27T15:12:33.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:32 smithi102 ceph-mon[100296]: osdmap e23: 4 total, 3 up, 4 in 2023-11-27T15:12:33.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:32 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:12:33.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:32 smithi102 ceph-mon[100296]: from='osd.3 [v2:172.21.15.52:6810/2622100752,v1:172.21.15.52:6811/2622100752]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2023-11-27T15:12:33.236 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi102:vg_nvme/lv_3 2023-11-27T15:12:33.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:32 smithi150 ceph-mon[100559]: from='osd.3 [v2:172.21.15.52:6810/2622100752,v1:172.21.15.52:6811/2622100752]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-11-27T15:12:33.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:32 smithi150 ceph-mon[100559]: osdmap e23: 4 total, 3 up, 4 in 2023-11-27T15:12:33.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:32 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:12:33.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:32 smithi150 ceph-mon[100559]: from='osd.3 [v2:172.21.15.52:6810/2622100752,v1:172.21.15.52:6811/2622100752]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2023-11-27T15:12:33.874 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:12:34.109 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:33 smithi102 ceph-mon[100296]: from='osd.3 [v2:172.21.15.52:6810/2622100752,v1:172.21.15.52:6811/2622100752]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2023-11-27T15:12:34.109 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:33 smithi102 ceph-mon[100296]: osdmap e24: 4 total, 3 up, 4 in 2023-11-27T15:12:34.109 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:33 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:12:34.109 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:33 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:12:34.110 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:33 smithi102 ceph-mon[100296]: Detected new or changed devices on smithi052 2023-11-27T15:12:34.110 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:33 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:34.110 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:33 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:34.110 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:33 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:12:34.110 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:33 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:12:34.110 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:33 smithi102 ceph-mon[100296]: Adjusting osd_memory_target on smithi052 to 8586M 2023-11-27T15:12:34.110 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:33 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:34.110 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:33 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:34.110 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:33 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:12:34.110 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:33 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:12:34.110 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:33 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:34.110 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:33 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:12:34.110 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:33 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:12:34.111 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:33 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:12:34.111 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:33 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:34.111 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:33 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:12:34.111 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:33 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:12:34.111 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:33 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:34.111 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:33 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:12:34.111 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:33 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:12:34.111 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:33 smithi102 ceph-mon[100296]: osd.3 [v2:172.21.15.52:6810/2622100752,v1:172.21.15.52:6811/2622100752] boot 2023-11-27T15:12:34.111 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:33 smithi102 ceph-mon[100296]: osdmap e25: 4 total, 4 up, 4 in 2023-11-27T15:12:34.111 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:33 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:12:34.111 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:33 smithi102 ceph-mon[100296]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:34.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:33 smithi052 ceph-mon[99056]: from='osd.3 [v2:172.21.15.52:6810/2622100752,v1:172.21.15.52:6811/2622100752]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2023-11-27T15:12:34.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:33 smithi052 ceph-mon[99056]: osdmap e24: 4 total, 3 up, 4 in 2023-11-27T15:12:34.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:33 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:12:34.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:33 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:12:34.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:33 smithi052 ceph-mon[99056]: Detected new or changed devices on smithi052 2023-11-27T15:12:34.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:33 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:34.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:33 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:34.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:33 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:12:34.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:33 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:12:34.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:33 smithi052 ceph-mon[99056]: Adjusting osd_memory_target on smithi052 to 8586M 2023-11-27T15:12:34.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:33 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:34.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:33 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:34.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:33 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:12:34.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:33 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:12:34.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:33 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:34.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:33 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:12:34.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:33 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:12:34.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:33 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:12:34.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:33 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:34.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:33 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:12:34.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:33 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:12:34.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:33 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:34.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:33 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:12:34.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:33 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:12:34.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:33 smithi052 ceph-mon[99056]: osd.3 [v2:172.21.15.52:6810/2622100752,v1:172.21.15.52:6811/2622100752] boot 2023-11-27T15:12:34.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:33 smithi052 ceph-mon[99056]: osdmap e25: 4 total, 4 up, 4 in 2023-11-27T15:12:34.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:33 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:12:34.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:33 smithi052 ceph-mon[99056]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:34.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:33 smithi150 ceph-mon[100559]: from='osd.3 [v2:172.21.15.52:6810/2622100752,v1:172.21.15.52:6811/2622100752]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2023-11-27T15:12:34.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:33 smithi150 ceph-mon[100559]: osdmap e24: 4 total, 3 up, 4 in 2023-11-27T15:12:34.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:33 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:12:34.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:33 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:12:34.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:33 smithi150 ceph-mon[100559]: Detected new or changed devices on smithi052 2023-11-27T15:12:34.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:33 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:34.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:33 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:34.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:33 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:12:34.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:33 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:12:34.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:33 smithi150 ceph-mon[100559]: Adjusting osd_memory_target on smithi052 to 8586M 2023-11-27T15:12:34.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:33 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:34.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:33 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:34.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:33 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:12:34.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:33 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:12:34.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:33 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:34.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:33 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:12:34.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:33 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:12:34.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:33 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:12:34.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:33 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:34.281 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:33 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:12:34.281 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:33 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:12:34.281 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:33 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:34.281 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:33 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:12:34.281 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:33 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:12:34.281 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:33 smithi150 ceph-mon[100559]: osd.3 [v2:172.21.15.52:6810/2622100752,v1:172.21.15.52:6811/2622100752] boot 2023-11-27T15:12:34.281 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:33 smithi150 ceph-mon[100559]: osdmap e25: 4 total, 4 up, 4 in 2023-11-27T15:12:34.281 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:33 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:12:34.281 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:33 smithi150 ceph-mon[100559]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:12:35.051 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.044+0000 7ff877287700 1 -- 172.21.15.102:0/812157816 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff870100d70 msgr2=0x7ff870107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:12:35.051 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.044+0000 7ff877287700 1 --2- 172.21.15.102:0/812157816 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff870100d70 0x7ff870107630 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff86800b0a0 tx=0x7ff86800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:12:35.051 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.044+0000 7ff877287700 1 -- 172.21.15.102:0/812157816 shutdown_connections 2023-11-27T15:12:35.051 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.044+0000 7ff877287700 1 --2- 172.21.15.102:0/812157816 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff870100d70 0x7ff870107630 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:12:35.052 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.044+0000 7ff877287700 1 --2- 172.21.15.102:0/812157816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8700ff1e0 0x7ff8700ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:12:35.052 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.044+0000 7ff877287700 1 --2- 172.21.15.102:0/812157816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8701003e0 0x7ff870100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:12:35.052 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.044+0000 7ff877287700 1 -- 172.21.15.102:0/812157816 >> 172.21.15.102:0/812157816 conn(0x7ff8700fa7f0 msgr2=0x7ff8700fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:12:35.052 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.044+0000 7ff877287700 1 -- 172.21.15.102:0/812157816 shutdown_connections 2023-11-27T15:12:35.052 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.044+0000 7ff877287700 1 -- 172.21.15.102:0/812157816 wait complete. 2023-11-27T15:12:35.052 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.045+0000 7ff877287700 1 Processor -- start 2023-11-27T15:12:35.052 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.045+0000 7ff877287700 1 -- start start 2023-11-27T15:12:35.052 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.045+0000 7ff877287700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8700ff1e0 0x7ff870194fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:12:35.052 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.045+0000 7ff877287700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8701003e0 0x7ff870195500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:12:35.052 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.045+0000 7ff877287700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff870100d70 0x7ff87019c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:12:35.052 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.045+0000 7ff877287700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87019cb50 con 0x7ff870100d70 2023-11-27T15:12:35.053 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.045+0000 7ff877287700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87019cc90 con 0x7ff8701003e0 2023-11-27T15:12:35.053 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.045+0000 7ff877287700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87019ce20 con 0x7ff8700ff1e0 2023-11-27T15:12:35.053 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.045+0000 7ff874822700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8701003e0 0x7ff870195500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:12:35.053 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.045+0000 7ff874822700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8701003e0 0x7ff870195500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41786/0 (socket says 172.21.15.102:41786) 2023-11-27T15:12:35.053 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.045+0000 7ff874822700 1 -- 172.21.15.102:0/519314082 learned_addr learned my addr 172.21.15.102:0/519314082 (peer_addr_for_me v2:172.21.15.102:0/0) 2023-11-27T15:12:35.053 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.045+0000 7ff875023700 1 --2- 172.21.15.102:0/519314082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8700ff1e0 0x7ff870194fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:12:35.053 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.045+0000 7ff874822700 1 -- 172.21.15.102:0/519314082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8700ff1e0 msgr2=0x7ff870194fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:12:35.053 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.045+0000 7ff875824700 1 --2- 172.21.15.102:0/519314082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff870100d70 0x7ff87019c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:12:35.053 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.045+0000 7ff874822700 1 --2- 172.21.15.102:0/519314082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8700ff1e0 0x7ff870194fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:12:35.053 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.045+0000 7ff874822700 1 -- 172.21.15.102:0/519314082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff870100d70 msgr2=0x7ff87019c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:12:35.053 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.045+0000 7ff874822700 1 --2- 172.21.15.102:0/519314082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff870100d70 0x7ff87019c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:12:35.053 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.045+0000 7ff874822700 1 -- 172.21.15.102:0/519314082 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff868009d00 con 0x7ff8701003e0 2023-11-27T15:12:35.054 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.045+0000 7ff875824700 1 --2- 172.21.15.102:0/519314082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff870100d70 0x7ff87019c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:12:35.054 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.045+0000 7ff874822700 1 --2- 172.21.15.102:0/519314082 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8701003e0 0x7ff870195500 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00d950 tx=0x7ff86c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:12:35.054 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.045+0000 7ff8667fc700 1 -- 172.21.15.102:0/519314082 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c009940 con 0x7ff8701003e0 2023-11-27T15:12:35.054 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.046+0000 7ff8667fc700 1 -- 172.21.15.102:0/519314082 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff86c010460 con 0x7ff8701003e0 2023-11-27T15:12:35.054 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.046+0000 7ff877287700 1 -- 172.21.15.102:0/519314082 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff87010ab30 con 0x7ff8701003e0 2023-11-27T15:12:35.054 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.046+0000 7ff8667fc700 1 -- 172.21.15.102:0/519314082 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c00f600 con 0x7ff8701003e0 2023-11-27T15:12:35.054 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.046+0000 7ff877287700 1 -- 172.21.15.102:0/519314082 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff87010a9a0 con 0x7ff8701003e0 2023-11-27T15:12:35.055 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.047+0000 7ff877287700 1 -- 172.21.15.102:0/519314082 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff87019d550 con 0x7ff8701003e0 2023-11-27T15:12:35.055 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.048+0000 7ff8667fc700 1 -- 172.21.15.102:0/519314082 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff86c0105d0 con 0x7ff8701003e0 2023-11-27T15:12:35.055 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.048+0000 7ff8667fc700 1 --2- 172.21.15.102:0/519314082 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff85c0a11d0 0x7ff85c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:12:35.055 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.048+0000 7ff8667fc700 1 -- 172.21.15.102:0/519314082 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3698+0+0 (secure 0 0 0) 0x7ff86c0cb510 con 0x7ff8701003e0 2023-11-27T15:12:35.055 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.048+0000 7ff875023700 1 --2- 172.21.15.102:0/519314082 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff85c0a11d0 0x7ff85c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:12:35.055 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.049+0000 7ff875023700 1 --2- 172.21.15.102:0/519314082 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff85c0a11d0 0x7ff85c0a3680 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff870100240 tx=0x7ff86000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:12:35.057 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.053+0000 7ff8667fc700 1 -- 172.21.15.102:0/519314082 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff86c095e60 con 0x7ff8701003e0 2023-11-27T15:12:35.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:34 smithi052 ceph-mon[99056]: purged_snaps scrub starts 2023-11-27T15:12:35.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:34 smithi052 ceph-mon[99056]: purged_snaps scrub ok 2023-11-27T15:12:35.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:34 smithi052 ceph-mon[99056]: osdmap e26: 4 total, 4 up, 4 in 2023-11-27T15:12:35.249 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:35.244+0000 7ff877287700 1 -- 172.21.15.102:0/519314082 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff870060f30 con 0x7ff85c0a11d0 2023-11-27T15:12:35.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:34 smithi150 ceph-mon[100559]: purged_snaps scrub starts 2023-11-27T15:12:35.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:34 smithi150 ceph-mon[100559]: purged_snaps scrub ok 2023-11-27T15:12:35.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:34 smithi150 ceph-mon[100559]: osdmap e26: 4 total, 4 up, 4 in 2023-11-27T15:12:35.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:34 smithi102 ceph-mon[100296]: purged_snaps scrub starts 2023-11-27T15:12:35.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:34 smithi102 ceph-mon[100296]: purged_snaps scrub ok 2023-11-27T15:12:35.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:34 smithi102 ceph-mon[100296]: osdmap e26: 4 total, 4 up, 4 in 2023-11-27T15:12:36.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:35 smithi052 ceph-mon[99056]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:12:36.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:35 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:12:36.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:35 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:12:36.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:35 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:36.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:35 smithi052 ceph-mon[99056]: osdmap e27: 4 total, 4 up, 4 in 2023-11-27T15:12:36.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:35 smithi052 ceph-mon[99056]: pgmap v93: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2023-11-27T15:12:36.159 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:35 smithi102 ceph-mon[100296]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:12:36.159 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:35 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:12:36.159 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:35 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:12:36.159 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:35 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:36.160 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:35 smithi102 ceph-mon[100296]: osdmap e27: 4 total, 4 up, 4 in 2023-11-27T15:12:36.160 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:35 smithi102 ceph-mon[100296]: pgmap v93: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2023-11-27T15:12:36.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:35 smithi150 ceph-mon[100559]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:12:36.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:35 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:12:36.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:35 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:12:36.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:35 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:36.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:35 smithi150 ceph-mon[100559]: osdmap e27: 4 total, 4 up, 4 in 2023-11-27T15:12:36.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:35 smithi150 ceph-mon[100559]: pgmap v93: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2023-11-27T15:12:37.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:36 smithi052 ceph-mon[99056]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2023-11-27T15:12:37.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:36 smithi052 ceph-mon[99056]: osdmap e28: 4 total, 4 up, 4 in 2023-11-27T15:12:37.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:36 smithi150 ceph-mon[100559]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2023-11-27T15:12:37.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:36 smithi150 ceph-mon[100559]: osdmap e28: 4 total, 4 up, 4 in 2023-11-27T15:12:37.293 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:36 smithi102 ceph-mon[100296]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2023-11-27T15:12:37.293 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:36 smithi102 ceph-mon[100296]: osdmap e28: 4 total, 4 up, 4 in 2023-11-27T15:12:39.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:38 smithi150 ceph-mon[100559]: osdmap e29: 4 total, 4 up, 4 in 2023-11-27T15:12:39.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:38 smithi150 ceph-mon[100559]: pgmap v96: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2023-11-27T15:12:39.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:38 smithi150 ceph-mon[100559]: from='client.? 172.21.15.102:0/2491312584' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c96a676-cea9-4deb-bad1-f7fd240c3d95"}]: dispatch 2023-11-27T15:12:39.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:38 smithi150 ceph-mon[100559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c96a676-cea9-4deb-bad1-f7fd240c3d95"}]: dispatch 2023-11-27T15:12:39.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:38 smithi150 ceph-mon[100559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c96a676-cea9-4deb-bad1-f7fd240c3d95"}]': finished 2023-11-27T15:12:39.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:38 smithi150 ceph-mon[100559]: osdmap e30: 5 total, 4 up, 5 in 2023-11-27T15:12:39.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:38 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:12:39.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:38 smithi102 ceph-mon[100296]: osdmap e29: 4 total, 4 up, 4 in 2023-11-27T15:12:39.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:38 smithi102 ceph-mon[100296]: pgmap v96: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2023-11-27T15:12:39.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:38 smithi102 ceph-mon[100296]: from='client.? 172.21.15.102:0/2491312584' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c96a676-cea9-4deb-bad1-f7fd240c3d95"}]: dispatch 2023-11-27T15:12:39.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:38 smithi102 ceph-mon[100296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c96a676-cea9-4deb-bad1-f7fd240c3d95"}]: dispatch 2023-11-27T15:12:39.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:38 smithi102 ceph-mon[100296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c96a676-cea9-4deb-bad1-f7fd240c3d95"}]': finished 2023-11-27T15:12:39.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:38 smithi102 ceph-mon[100296]: osdmap e30: 5 total, 4 up, 5 in 2023-11-27T15:12:39.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:38 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:12:39.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:38 smithi052 ceph-mon[99056]: osdmap e29: 4 total, 4 up, 4 in 2023-11-27T15:12:39.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:38 smithi052 ceph-mon[99056]: pgmap v96: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2023-11-27T15:12:39.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:38 smithi052 ceph-mon[99056]: from='client.? 172.21.15.102:0/2491312584' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c96a676-cea9-4deb-bad1-f7fd240c3d95"}]: dispatch 2023-11-27T15:12:39.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:38 smithi052 ceph-mon[99056]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c96a676-cea9-4deb-bad1-f7fd240c3d95"}]: dispatch 2023-11-27T15:12:39.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:38 smithi052 ceph-mon[99056]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c96a676-cea9-4deb-bad1-f7fd240c3d95"}]': finished 2023-11-27T15:12:39.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:38 smithi052 ceph-mon[99056]: osdmap e30: 5 total, 4 up, 5 in 2023-11-27T15:12:39.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:38 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:12:40.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:39 smithi150 ceph-mon[100559]: from='client.? 172.21.15.102:0/2483280508' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:12:40.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:39 smithi102 ceph-mon[100296]: from='client.? 172.21.15.102:0/2483280508' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:12:40.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:39 smithi052 ceph-mon[99056]: from='client.? 172.21.15.102:0/2483280508' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:12:41.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:40 smithi150 ceph-mon[100559]: pgmap v98: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2023-11-27T15:12:41.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:40 smithi102 ceph-mon[100296]: pgmap v98: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2023-11-27T15:12:41.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:40 smithi052 ceph-mon[99056]: pgmap v98: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2023-11-27T15:12:42.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:41 smithi150 ceph-mon[100559]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-11-27T15:12:42.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:41 smithi150 ceph-mon[100559]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2023-11-27T15:12:42.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:41 smithi150 ceph-mon[100559]: Cluster is now healthy 2023-11-27T15:12:42.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:41 smithi102 ceph-mon[100296]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-11-27T15:12:42.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:41 smithi102 ceph-mon[100296]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2023-11-27T15:12:42.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:41 smithi102 ceph-mon[100296]: Cluster is now healthy 2023-11-27T15:12:42.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:41 smithi052 ceph-mon[99056]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-11-27T15:12:42.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:41 smithi052 ceph-mon[99056]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2023-11-27T15:12:42.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:41 smithi052 ceph-mon[99056]: Cluster is now healthy 2023-11-27T15:12:44.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:44 smithi052 ceph-mon[99056]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-11-27T15:12:45.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:44 smithi150 ceph-mon[100559]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-11-27T15:12:45.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:44 smithi102 ceph-mon[100296]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-11-27T15:12:45.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:45 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-11-27T15:12:45.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:45 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:45.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:45 smithi052 ceph-mon[99056]: Deploying daemon osd.4 on smithi102 2023-11-27T15:12:45.891 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:45 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-11-27T15:12:45.892 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:45 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:45.892 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:45 smithi102 ceph-mon[100296]: Deploying daemon osd.4 on smithi102 2023-11-27T15:12:46.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:45 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-11-27T15:12:46.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:45 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:46.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:45 smithi150 ceph-mon[100559]: Deploying daemon osd.4 on smithi102 2023-11-27T15:12:46.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:46 smithi102 ceph-mon[100296]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2023-11-27T15:12:46.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:46 smithi052 ceph-mon[99056]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2023-11-27T15:12:47.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:46 smithi150 ceph-mon[100559]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2023-11-27T15:12:48.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:47 smithi150 ceph-mon[100559]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-11-27T15:12:48.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:47 smithi102 ceph-mon[100296]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-11-27T15:12:48.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:47 smithi052 ceph-mon[99056]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-11-27T15:12:50.460 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:50 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:50.460 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:50 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:12:50.461 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:50 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:50.461 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:50 smithi102 ceph-mon[100296]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-11-27T15:12:50.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:50 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:50.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:50 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:12:50.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:50 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:50.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:50 smithi150 ceph-mon[100559]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-11-27T15:12:50.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:50 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:50.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:50 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:12:50.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:50 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:50.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:50 smithi052 ceph-mon[99056]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-11-27T15:12:52.171 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:52.167+0000 7ff8667fc700 1 -- 172.21.15.102:0/519314082 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff870060f30 con 0x7ff85c0a11d0 2023-11-27T15:12:52.171 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 4 on host 'smithi102' 2023-11-27T15:12:52.173 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:52.169+0000 7ff877287700 1 -- 172.21.15.102:0/519314082 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff85c0a11d0 msgr2=0x7ff85c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:12:52.173 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:52.169+0000 7ff877287700 1 --2- 172.21.15.102:0/519314082 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff85c0a11d0 0x7ff85c0a3680 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff870100240 tx=0x7ff86000b410 comp rx=0 tx=0).stop 2023-11-27T15:12:52.173 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:52.169+0000 7ff877287700 1 -- 172.21.15.102:0/519314082 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8701003e0 msgr2=0x7ff870195500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:12:52.174 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:52.169+0000 7ff877287700 1 --2- 172.21.15.102:0/519314082 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8701003e0 0x7ff870195500 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00d950 tx=0x7ff86c00dc60 comp rx=0 tx=0).stop 2023-11-27T15:12:52.174 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:52.169+0000 7ff877287700 1 -- 172.21.15.102:0/519314082 shutdown_connections 2023-11-27T15:12:52.174 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:52.169+0000 7ff877287700 1 --2- 172.21.15.102:0/519314082 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff85c0a11d0 0x7ff85c0a3680 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:12:52.174 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:52.169+0000 7ff877287700 1 --2- 172.21.15.102:0/519314082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff870100d70 0x7ff87019c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:12:52.174 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:52.169+0000 7ff877287700 1 --2- 172.21.15.102:0/519314082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8700ff1e0 0x7ff870194fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:12:52.174 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:52.169+0000 7ff877287700 1 --2- 172.21.15.102:0/519314082 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8701003e0 0x7ff870195500 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:12:52.174 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:52.169+0000 7ff877287700 1 -- 172.21.15.102:0/519314082 >> 172.21.15.102:0/519314082 conn(0x7ff8700fa7f0 msgr2=0x7ff8700fca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:12:52.174 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:52.169+0000 7ff877287700 1 -- 172.21.15.102:0/519314082 shutdown_connections 2023-11-27T15:12:52.174 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:12:52.169+0000 7ff877287700 1 -- 172.21.15.102:0/519314082 wait complete. 2023-11-27T15:12:52.565 DEBUG:teuthology.orchestra.run.smithi102:osd.4> sudo journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.4.service 2023-11-27T15:12:52.567 INFO:tasks.cephadm:Deploying osd.5 on smithi150 with /dev/vg_nvme/lv_3... 2023-11-27T15:12:52.567 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-11-27T15:12:52.595 INFO:journalctl@ceph.osd.4.smithi102.stdout:-- Logs begin at Mon 2023-11-27 14:52:53 UTC. -- 2023-11-27T15:12:52.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:52 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:52.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:52 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:52.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:52 smithi150 ceph-mon[100559]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-11-27T15:12:52.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:52 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:52.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:52 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:12:52.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:52 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:52.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:52 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:52.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:52 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:52.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:52 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:12:52.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:52 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:52.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:52 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:52.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:52 smithi102 ceph-mon[100296]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-11-27T15:12:52.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:52 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:52.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:52 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:12:52.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:52 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:52.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:52 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:52.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:52 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:52.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:52 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:12:52.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:52 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:52.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:52 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:52.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:52 smithi052 ceph-mon[99056]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-11-27T15:12:52.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:52 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:52.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:52 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:12:52.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:52 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:52.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:52 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:52.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:52 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:52.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:52 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:12:53.217 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:12:53.328 INFO:journalctl@ceph.osd.4.smithi102.stdout:Nov 27 15:12:53 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-4[110815]: 2023-11-27T15:12:53.061+0000 7f958b767640 -1 osd.4 0 log_to_monitors true 2023-11-27T15:12:53.705 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:53 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:12:53.706 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:53 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:12:53.706 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:53 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:12:53.706 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:53 smithi150 ceph-mon[100559]: from='osd.4 [v2:172.21.15.102:6808/1546794763,v1:172.21.15.102:6809/1546794763]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-27T15:12:53.706 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:53 smithi150 ceph-mon[100559]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-27T15:12:53.826 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:53 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:12:53.826 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:53 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:12:53.826 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:53 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:12:53.826 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:53 smithi102 ceph-mon[100296]: from='osd.4 [v2:172.21.15.102:6808/1546794763,v1:172.21.15.102:6809/1546794763]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-27T15:12:53.827 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:53 smithi102 ceph-mon[100296]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-27T15:12:53.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:53 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:12:53.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:53 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:12:53.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:53 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:12:53.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:53 smithi052 ceph-mon[99056]: from='osd.4 [v2:172.21.15.102:6808/1546794763,v1:172.21.15.102:6809/1546794763]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-27T15:12:53.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:53 smithi052 ceph-mon[99056]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-27T15:12:54.670 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-11-27T15:12:54.671 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-11-27T15:12:54.671 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2023-11-27T15:12:54.671 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2023-11-27T15:12:54.671 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0158319 s, 662 MB/s 2023-11-27T15:12:54.671 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2023-11-27T15:12:54.704 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:54 smithi150 ceph-mon[100559]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-11-27T15:12:54.704 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:54 smithi150 ceph-mon[100559]: osdmap e31: 5 total, 4 up, 5 in 2023-11-27T15:12:54.704 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:54 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:12:54.704 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:54 smithi150 ceph-mon[100559]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2023-11-27T15:12:54.704 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:54 smithi150 ceph-mon[100559]: from='osd.4 [v2:172.21.15.102:6808/1546794763,v1:172.21.15.102:6809/1546794763]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2023-11-27T15:12:54.705 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:54 smithi150 ceph-mon[100559]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:12:54.739 INFO:journalctl@ceph.osd.4.smithi102.stdout:Nov 27 15:12:54 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-4[110815]: 2023-11-27T15:12:54.428+0000 7f9580dde700 -1 osd.4 0 waiting for initial osdmap 2023-11-27T15:12:54.739 INFO:journalctl@ceph.osd.4.smithi102.stdout:Nov 27 15:12:54 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-4[110815]: 2023-11-27T15:12:54.434+0000 7f957abcf700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:12:54.739 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:54 smithi102 ceph-mon[100296]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-11-27T15:12:54.740 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:54 smithi102 ceph-mon[100296]: osdmap e31: 5 total, 4 up, 5 in 2023-11-27T15:12:54.740 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:54 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:12:54.740 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:54 smithi102 ceph-mon[100296]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2023-11-27T15:12:54.740 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:54 smithi102 ceph-mon[100296]: from='osd.4 [v2:172.21.15.102:6808/1546794763,v1:172.21.15.102:6809/1546794763]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2023-11-27T15:12:54.740 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:54 smithi102 ceph-mon[100296]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:12:54.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:54 smithi052 ceph-mon[99056]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-11-27T15:12:54.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:54 smithi052 ceph-mon[99056]: osdmap e31: 5 total, 4 up, 5 in 2023-11-27T15:12:54.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:54 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:12:54.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:54 smithi052 ceph-mon[99056]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2023-11-27T15:12:54.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:54 smithi052 ceph-mon[99056]: from='osd.4 [v2:172.21.15.102:6808/1546794763,v1:172.21.15.102:6809/1546794763]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2023-11-27T15:12:54.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:54 smithi052 ceph-mon[99056]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:12:55.058 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi150:vg_nvme/lv_3 2023-11-27T15:12:55.700 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:12:55.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:55 smithi150 ceph-mon[100559]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2023-11-27T15:12:55.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:55 smithi150 ceph-mon[100559]: osdmap e32: 5 total, 4 up, 5 in 2023-11-27T15:12:55.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:55 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:12:55.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:55 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:12:55.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:55 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:55.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:55 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:55.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:55 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:12:55.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:55 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:12:55.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:55 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:55.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:55 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:55.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:55 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:12:55.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:55 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:55.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:55 smithi102 ceph-mon[100296]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2023-11-27T15:12:55.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:55 smithi102 ceph-mon[100296]: osdmap e32: 5 total, 4 up, 5 in 2023-11-27T15:12:55.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:55 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:12:55.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:55 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:12:55.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:55 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:55.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:55 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:55.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:55 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:12:55.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:55 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:12:55.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:55 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:55.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:55 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:55.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:55 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:12:55.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:55 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:55.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:55 smithi052 ceph-mon[99056]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2023-11-27T15:12:55.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:55 smithi052 ceph-mon[99056]: osdmap e32: 5 total, 4 up, 5 in 2023-11-27T15:12:55.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:55 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:12:55.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:55 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:12:55.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:55 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:55.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:55 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:55.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:55 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:12:55.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:55 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:12:55.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:55 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:55.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:55 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:55.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:55 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:12:55.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:55 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:12:56.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:56 smithi150 ceph-mon[100559]: purged_snaps scrub starts 2023-11-27T15:12:56.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:56 smithi150 ceph-mon[100559]: purged_snaps scrub ok 2023-11-27T15:12:56.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:56 smithi150 ceph-mon[100559]: Detected new or changed devices on smithi102 2023-11-27T15:12:56.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:56 smithi150 ceph-mon[100559]: Adjusting osd_memory_target on smithi102 to 8584M 2023-11-27T15:12:56.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:56 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:12:56.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:56 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:12:56.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:56 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:12:56.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:56 smithi150 ceph-mon[100559]: osd.4 [v2:172.21.15.102:6808/1546794763,v1:172.21.15.102:6809/1546794763] boot 2023-11-27T15:12:56.530 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:56 smithi150 ceph-mon[100559]: osdmap e33: 5 total, 5 up, 5 in 2023-11-27T15:12:56.530 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:56 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:12:56.530 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:56 smithi150 ceph-mon[100559]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:12:56.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:56 smithi102 ceph-mon[100296]: purged_snaps scrub starts 2023-11-27T15:12:56.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:56 smithi102 ceph-mon[100296]: purged_snaps scrub ok 2023-11-27T15:12:56.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:56 smithi102 ceph-mon[100296]: Detected new or changed devices on smithi102 2023-11-27T15:12:56.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:56 smithi102 ceph-mon[100296]: Adjusting osd_memory_target on smithi102 to 8584M 2023-11-27T15:12:56.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:56 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:12:56.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:56 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:12:56.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:56 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:12:56.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:56 smithi102 ceph-mon[100296]: osd.4 [v2:172.21.15.102:6808/1546794763,v1:172.21.15.102:6809/1546794763] boot 2023-11-27T15:12:56.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:56 smithi102 ceph-mon[100296]: osdmap e33: 5 total, 5 up, 5 in 2023-11-27T15:12:56.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:56 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:12:56.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:56 smithi102 ceph-mon[100296]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:12:56.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:56 smithi052 ceph-mon[99056]: purged_snaps scrub starts 2023-11-27T15:12:56.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:56 smithi052 ceph-mon[99056]: purged_snaps scrub ok 2023-11-27T15:12:56.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:56 smithi052 ceph-mon[99056]: Detected new or changed devices on smithi102 2023-11-27T15:12:56.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:56 smithi052 ceph-mon[99056]: Adjusting osd_memory_target on smithi102 to 8584M 2023-11-27T15:12:56.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:56 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:12:56.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:56 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:12:56.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:56 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:12:56.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:56 smithi052 ceph-mon[99056]: osd.4 [v2:172.21.15.102:6808/1546794763,v1:172.21.15.102:6809/1546794763] boot 2023-11-27T15:12:56.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:56 smithi052 ceph-mon[99056]: osdmap e33: 5 total, 5 up, 5 in 2023-11-27T15:12:56.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:56 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:12:56.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:56 smithi052 ceph-mon[99056]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:12:56.884 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.879+0000 7f5061cd8700 1 -- 172.21.15.150:0/3408367856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f505c1003e0 msgr2=0x7f505c100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:12:56.884 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.879+0000 7f5061cd8700 1 --2- 172.21.15.150:0/3408367856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f505c1003e0 0x7f505c100830 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5050009990 tx=0x7f5050009ca0 comp rx=0 tx=0).stop 2023-11-27T15:12:56.884 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.879+0000 7f5061cd8700 1 -- 172.21.15.150:0/3408367856 shutdown_connections 2023-11-27T15:12:56.885 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.879+0000 7f5061cd8700 1 --2- 172.21.15.150:0/3408367856 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f505c0ff1e0 0x7f505c0ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:12:56.885 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.879+0000 7f5061cd8700 1 --2- 172.21.15.150:0/3408367856 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f505c100d70 0x7f505c107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:12:56.885 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.879+0000 7f5061cd8700 1 --2- 172.21.15.150:0/3408367856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f505c1003e0 0x7f505c100830 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:12:56.885 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.879+0000 7f5061cd8700 1 -- 172.21.15.150:0/3408367856 >> 172.21.15.150:0/3408367856 conn(0x7f505c0fa7f0 msgr2=0x7f505c0fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:12:56.885 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.879+0000 7f5061cd8700 1 -- 172.21.15.150:0/3408367856 shutdown_connections 2023-11-27T15:12:56.885 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.879+0000 7f5061cd8700 1 -- 172.21.15.150:0/3408367856 wait complete. 2023-11-27T15:12:56.887 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.880+0000 7f5061cd8700 1 Processor -- start 2023-11-27T15:12:56.887 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.880+0000 7f5061cd8700 1 -- start start 2023-11-27T15:12:56.887 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.880+0000 7f5061cd8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f505c0ff1e0 0x7f505c0699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:12:56.888 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.880+0000 7f5061cd8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f505c1003e0 0x7f505c19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:12:56.888 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.880+0000 7f5061cd8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f505c100d70 0x7f505c19d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:12:56.888 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.880+0000 7f5061cd8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c19dbe0 con 0x7f505c100d70 2023-11-27T15:12:56.888 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.880+0000 7f5061cd8700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c19dd50 con 0x7f505c0ff1e0 2023-11-27T15:12:56.888 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.880+0000 7f5061cd8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c19df10 con 0x7f505c1003e0 2023-11-27T15:12:56.888 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.880+0000 7f505affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f505c1003e0 0x7f505c19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:12:56.888 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.880+0000 7f505affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f505c1003e0 0x7f505c19adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49458/0 (socket says 172.21.15.150:49458) 2023-11-27T15:12:56.889 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.880+0000 7f505affd700 1 -- 172.21.15.150:0/3035683116 learned_addr learned my addr 172.21.15.150:0/3035683116 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-27T15:12:56.889 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.880+0000 7f505b7fe700 1 --2- 172.21.15.150:0/3035683116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f505c0ff1e0 0x7f505c0699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:12:56.889 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.880+0000 7f505bfff700 1 --2- 172.21.15.150:0/3035683116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f505c100d70 0x7f505c19d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:12:56.889 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.880+0000 7f505affd700 1 -- 172.21.15.150:0/3035683116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f505c0ff1e0 msgr2=0x7f505c0699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:12:56.889 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.880+0000 7f505affd700 1 --2- 172.21.15.150:0/3035683116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f505c0ff1e0 0x7f505c0699c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:12:56.889 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.880+0000 7f505affd700 1 -- 172.21.15.150:0/3035683116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f505c100d70 msgr2=0x7f505c19d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:12:56.889 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.880+0000 7f505affd700 1 --2- 172.21.15.150:0/3035683116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f505c100d70 0x7f505c19d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:12:56.889 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.880+0000 7f505affd700 1 -- 172.21.15.150:0/3035683116 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50500095f0 con 0x7f505c1003e0 2023-11-27T15:12:56.889 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.880+0000 7f505affd700 1 --2- 172.21.15.150:0/3035683116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f505c1003e0 0x7f505c19adf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5050000c00 tx=0x7f505000f830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:12:56.889 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.881+0000 7f5058ff9700 1 -- 172.21.15.150:0/3035683116 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505001d070 con 0x7f505c1003e0 2023-11-27T15:12:56.889 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.881+0000 7f5061cd8700 1 -- 172.21.15.150:0/3035683116 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f505c10ac50 con 0x7f505c1003e0 2023-11-27T15:12:56.889 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.881+0000 7f5058ff9700 1 -- 172.21.15.150:0/3035683116 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f505000fce0 con 0x7f505c1003e0 2023-11-27T15:12:56.889 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.881+0000 7f5058ff9700 1 -- 172.21.15.150:0/3035683116 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50500176e0 con 0x7f505c1003e0 2023-11-27T15:12:56.890 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.881+0000 7f5061cd8700 1 -- 172.21.15.150:0/3035683116 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f505c10aac0 con 0x7f505c1003e0 2023-11-27T15:12:56.890 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.882+0000 7f5061cd8700 1 -- 172.21.15.150:0/3035683116 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f505c106790 con 0x7f505c1003e0 2023-11-27T15:12:56.890 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.883+0000 7f5058ff9700 1 -- 172.21.15.150:0/3035683116 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5050022470 con 0x7f505c1003e0 2023-11-27T15:12:56.891 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.883+0000 7f5058ff9700 1 --2- 172.21.15.150:0/3035683116 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f50440a1150 0x7f50440a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:12:56.891 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.883+0000 7f5058ff9700 1 -- 172.21.15.150:0/3035683116 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f50500cd1a0 con 0x7f505c1003e0 2023-11-27T15:12:56.891 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.884+0000 7f505b7fe700 1 --2- 172.21.15.150:0/3035683116 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f50440a1150 0x7f50440a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:12:56.891 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.884+0000 7f505b7fe700 1 --2- 172.21.15.150:0/3035683116 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f50440a1150 0x7f50440a3600 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f505c100240 tx=0x7f50540094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:12:56.892 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:56.888+0000 7f5058ff9700 1 -- 172.21.15.150:0/3035683116 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f50500979f0 con 0x7f505c1003e0 2023-11-27T15:12:57.085 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:12:57.080+0000 7f5061cd8700 1 -- 172.21.15.150:0/3035683116 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f505c061b90 con 0x7f50440a1150 2023-11-27T15:12:57.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:57 smithi150 ceph-mon[100559]: osdmap e34: 5 total, 5 up, 5 in 2023-11-27T15:12:57.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:57 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:12:57.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:57 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:12:57.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:57 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:57.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:57 smithi102 ceph-mon[100296]: osdmap e34: 5 total, 5 up, 5 in 2023-11-27T15:12:57.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:57 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:12:57.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:57 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:12:57.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:57 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:57.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:57 smithi052 ceph-mon[99056]: osdmap e34: 5 total, 5 up, 5 in 2023-11-27T15:12:57.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:57 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:12:57.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:57 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:12:57.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:57 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:12:58.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:58 smithi150 ceph-mon[100559]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:12:58.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:58 smithi150 ceph-mon[100559]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:12:58.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:58 smithi102 ceph-mon[100296]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:12:58.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:58 smithi102 ceph-mon[100296]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:12:58.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:58 smithi052 ceph-mon[99056]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:12:58.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:58 smithi052 ceph-mon[99056]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:12:59.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:59 smithi052 ceph-mon[99056]: from='client.? 172.21.15.150:0/3671380466' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38a75af1-fd4d-433e-b579-0609109c851c"}]: dispatch 2023-11-27T15:12:59.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:12:59 smithi052 ceph-mon[99056]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38a75af1-fd4d-433e-b579-0609109c851c"}]: dispatch 2023-11-27T15:13:00.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:59 smithi150 ceph-mon[100559]: from='client.? 172.21.15.150:0/3671380466' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38a75af1-fd4d-433e-b579-0609109c851c"}]: dispatch 2023-11-27T15:13:00.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:12:59 smithi150 ceph-mon[100559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38a75af1-fd4d-433e-b579-0609109c851c"}]: dispatch 2023-11-27T15:13:00.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:59 smithi102 ceph-mon[100296]: from='client.? 172.21.15.150:0/3671380466' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38a75af1-fd4d-433e-b579-0609109c851c"}]: dispatch 2023-11-27T15:13:00.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:12:59 smithi102 ceph-mon[100296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "38a75af1-fd4d-433e-b579-0609109c851c"}]: dispatch 2023-11-27T15:13:01.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:00 smithi150 ceph-mon[100559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "38a75af1-fd4d-433e-b579-0609109c851c"}]': finished 2023-11-27T15:13:01.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:00 smithi150 ceph-mon[100559]: osdmap e35: 6 total, 5 up, 6 in 2023-11-27T15:13:01.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:00 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:13:01.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:00 smithi150 ceph-mon[100559]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:13:01.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:00 smithi150 ceph-mon[100559]: from='client.? 172.21.15.150:0/1617410654' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:13:01.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:00 smithi102 ceph-mon[100296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "38a75af1-fd4d-433e-b579-0609109c851c"}]': finished 2023-11-27T15:13:01.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:00 smithi102 ceph-mon[100296]: osdmap e35: 6 total, 5 up, 6 in 2023-11-27T15:13:01.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:00 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:13:01.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:00 smithi102 ceph-mon[100296]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:13:01.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:00 smithi102 ceph-mon[100296]: from='client.? 172.21.15.150:0/1617410654' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:13:01.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:00 smithi052 ceph-mon[99056]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "38a75af1-fd4d-433e-b579-0609109c851c"}]': finished 2023-11-27T15:13:01.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:00 smithi052 ceph-mon[99056]: osdmap e35: 6 total, 5 up, 6 in 2023-11-27T15:13:01.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:00 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:13:01.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:00 smithi052 ceph-mon[99056]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:13:01.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:00 smithi052 ceph-mon[99056]: from='client.? 172.21.15.150:0/1617410654' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:13:02.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:01 smithi150 ceph-mon[100559]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:13:02.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:01 smithi102 ceph-mon[100296]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:13:02.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:01 smithi052 ceph-mon[99056]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:13:04.862 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:04 smithi150 ceph-mon[100559]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:13:04.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:04 smithi052 ceph-mon[99056]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:13:05.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:04 smithi102 ceph-mon[100296]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:13:06.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:05 smithi150 ceph-mon[100559]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:13:06.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:05 smithi102 ceph-mon[100296]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:13:06.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:05 smithi052 ceph-mon[99056]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:13:07.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:06 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-11-27T15:13:07.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:06 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:13:07.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:06 smithi150 ceph-mon[100559]: Deploying daemon osd.5 on smithi150 2023-11-27T15:13:07.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:06 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-11-27T15:13:07.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:06 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:13:07.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:06 smithi102 ceph-mon[100296]: Deploying daemon osd.5 on smithi150 2023-11-27T15:13:07.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:06 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-11-27T15:13:07.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:06 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:13:07.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:06 smithi052 ceph-mon[99056]: Deploying daemon osd.5 on smithi150 2023-11-27T15:13:08.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:07 smithi150 ceph-mon[100559]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:13:08.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:07 smithi102 ceph-mon[100296]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:13:08.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:07 smithi052 ceph-mon[99056]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:13:10.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:09 smithi150 ceph-mon[100559]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:13:10.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:09 smithi102 ceph-mon[100296]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:13:10.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:09 smithi052 ceph-mon[99056]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:13:11.862 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:11 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:11.862 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:11 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:11.862 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:11 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:13:12.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:11 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:12.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:11 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:12.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:11 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:13:12.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:11 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:12.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:11 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:12.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:11 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:13:12.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:12 smithi150 ceph-mon[100559]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:13:13.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:12 smithi102 ceph-mon[100296]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:13:13.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:12 smithi052 ceph-mon[99056]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:13:13.450 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 5 on host 'smithi150' 2023-11-27T15:13:13.450 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:13:13.444+0000 7f5058ff9700 1 -- 172.21.15.150:0/3035683116 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f505c061b90 con 0x7f50440a1150 2023-11-27T15:13:13.452 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:13:13.447+0000 7f5061cd8700 1 -- 172.21.15.150:0/3035683116 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f50440a1150 msgr2=0x7f50440a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:13.452 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:13:13.447+0000 7f5061cd8700 1 --2- 172.21.15.150:0/3035683116 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f50440a1150 0x7f50440a3600 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f505c100240 tx=0x7f50540094f0 comp rx=0 tx=0).stop 2023-11-27T15:13:13.452 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:13:13.447+0000 7f5061cd8700 1 -- 172.21.15.150:0/3035683116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f505c1003e0 msgr2=0x7f505c19adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:13.452 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:13:13.447+0000 7f5061cd8700 1 --2- 172.21.15.150:0/3035683116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f505c1003e0 0x7f505c19adf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5050000c00 tx=0x7f505000f830 comp rx=0 tx=0).stop 2023-11-27T15:13:13.452 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:13:13.447+0000 7f5061cd8700 1 -- 172.21.15.150:0/3035683116 shutdown_connections 2023-11-27T15:13:13.452 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:13:13.447+0000 7f5061cd8700 1 --2- 172.21.15.150:0/3035683116 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f50440a1150 0x7f50440a3600 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:13.452 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:13:13.447+0000 7f5061cd8700 1 --2- 172.21.15.150:0/3035683116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f505c0ff1e0 0x7f505c0699c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:13.452 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:13:13.447+0000 7f5061cd8700 1 --2- 172.21.15.150:0/3035683116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f505c100d70 0x7f505c19d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:13.453 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:13:13.447+0000 7f5061cd8700 1 --2- 172.21.15.150:0/3035683116 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f505c1003e0 0x7f505c19adf0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:13.453 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:13:13.447+0000 7f5061cd8700 1 -- 172.21.15.150:0/3035683116 >> 172.21.15.150:0/3035683116 conn(0x7f505c0fa7f0 msgr2=0x7f505c1038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:13.453 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:13:13.447+0000 7f5061cd8700 1 -- 172.21.15.150:0/3035683116 shutdown_connections 2023-11-27T15:13:13.453 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:13:13.447+0000 7f5061cd8700 1 -- 172.21.15.150:0/3035683116 wait complete. 2023-11-27T15:13:13.898 DEBUG:teuthology.orchestra.run.smithi150:osd.5> sudo journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.5.service 2023-11-27T15:13:13.900 INFO:tasks.cephadm:Deploying osd.6 on smithi052 with /dev/vg_nvme/lv_2... 2023-11-27T15:13:13.900 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-11-27T15:13:13.949 INFO:journalctl@ceph.osd.5.smithi150.stdout:-- Logs begin at Mon 2023-11-27 14:53:12 UTC. -- 2023-11-27T15:13:14.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:14 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:14.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:14 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:14.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:14 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:13:14.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:14 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:13:14.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:14 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:13:14.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:14 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:14.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:14 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:13:14.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:14 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:13:14.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:14 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:14.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:14 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:14.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:14 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:13:14.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:14 smithi150 ceph-mon[100559]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:13:14.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:14 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:14.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:14 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:14.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:14 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:13:14.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:14 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:13:14.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:14 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:13:14.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:14 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:14.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:14 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:13:14.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:14 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:13:14.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:14 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:14.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:14 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:14.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:14 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:13:14.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:14 smithi102 ceph-mon[100296]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:13:14.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:14 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:14.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:14 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:14.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:14 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:13:14.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:14 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:13:14.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:14 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:13:14.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:14 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:14.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:14 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:13:14.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:14 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:13:14.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:14 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:14.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:14 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:14.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:14 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:13:14.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:14 smithi052 ceph-mon[99056]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:13:14.581 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:13:14.770 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:13:14 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:13:14.643+0000 7f0cf8400640 -1 osd.5 0 log_to_monitors true 2023-11-27T15:13:15.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:15 smithi052 ceph-mon[99056]: from='osd.5 [v2:172.21.15.150:6808/461511375,v1:172.21.15.150:6809/461511375]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-27T15:13:15.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:15 smithi052 ceph-mon[99056]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-27T15:13:16.019 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:15 smithi150 ceph-mon[100559]: from='osd.5 [v2:172.21.15.150:6808/461511375,v1:172.21.15.150:6809/461511375]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-27T15:13:16.019 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:15 smithi150 ceph-mon[100559]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-27T15:13:16.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:15 smithi102 ceph-mon[100296]: from='osd.5 [v2:172.21.15.150:6808/461511375,v1:172.21.15.150:6809/461511375]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-27T15:13:16.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:15 smithi102 ceph-mon[100296]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-27T15:13:16.175 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-11-27T15:13:16.175 INFO:teuthology.orchestra.run.smithi052.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-11-27T15:13:16.175 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10+0 records in 2023-11-27T15:13:16.176 INFO:teuthology.orchestra.run.smithi052.stderr:10+0 records out 2023-11-27T15:13:16.176 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0084332 s, 1.2 GB/s 2023-11-27T15:13:16.176 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping successful for: 2023-11-27T15:13:16.538 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi052:vg_nvme/lv_2 2023-11-27T15:13:16.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:16 smithi052 ceph-mon[99056]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-11-27T15:13:16.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:16 smithi052 ceph-mon[99056]: osdmap e36: 6 total, 5 up, 6 in 2023-11-27T15:13:16.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:16 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:13:16.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:16 smithi052 ceph-mon[99056]: from='osd.5 [v2:172.21.15.150:6808/461511375,v1:172.21.15.150:6809/461511375]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2023-11-27T15:13:16.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:16 smithi052 ceph-mon[99056]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2023-11-27T15:13:16.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:16 smithi052 ceph-mon[99056]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:13:16.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:16 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:16.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:16 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:16.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:16 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:13:16.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:16 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:13:16.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:16 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:16.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:16 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:13:16.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:16 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:13:16.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:16 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:17.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:16 smithi150 ceph-mon[100559]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-11-27T15:13:17.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:16 smithi150 ceph-mon[100559]: osdmap e36: 6 total, 5 up, 6 in 2023-11-27T15:13:17.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:16 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:13:17.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:16 smithi150 ceph-mon[100559]: from='osd.5 [v2:172.21.15.150:6808/461511375,v1:172.21.15.150:6809/461511375]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2023-11-27T15:13:17.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:16 smithi150 ceph-mon[100559]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2023-11-27T15:13:17.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:16 smithi150 ceph-mon[100559]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:13:17.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:16 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:17.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:16 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:17.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:16 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:13:17.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:16 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:13:17.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:16 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:17.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:16 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:13:17.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:16 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:13:17.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:16 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:17.030 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:13:16 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:13:16.628+0000 7f0ceda77700 -1 osd.5 0 waiting for initial osdmap 2023-11-27T15:13:17.030 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:13:16 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:13:16.635+0000 7f0ce986c700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:13:17.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:16 smithi102 ceph-mon[100296]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-11-27T15:13:17.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:16 smithi102 ceph-mon[100296]: osdmap e36: 6 total, 5 up, 6 in 2023-11-27T15:13:17.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:16 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:13:17.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:16 smithi102 ceph-mon[100296]: from='osd.5 [v2:172.21.15.150:6808/461511375,v1:172.21.15.150:6809/461511375]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2023-11-27T15:13:17.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:16 smithi102 ceph-mon[100296]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2023-11-27T15:13:17.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:16 smithi102 ceph-mon[100296]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:13:17.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:16 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:17.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:16 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:17.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:16 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:13:17.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:16 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:13:17.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:16 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:17.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:16 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:13:17.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:16 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:13:17.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:16 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:17.196 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:13:17.745 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:17 smithi052 ceph-mon[99056]: Detected new or changed devices on smithi150 2023-11-27T15:13:17.745 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:17 smithi052 ceph-mon[99056]: Adjusting osd_memory_target on smithi150 to 8584M 2023-11-27T15:13:17.745 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:17 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:13:17.745 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:17 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:13:17.745 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:17 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:13:17.745 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:17 smithi052 ceph-mon[99056]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2023-11-27T15:13:17.745 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:17 smithi052 ceph-mon[99056]: osdmap e37: 6 total, 5 up, 6 in 2023-11-27T15:13:17.745 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:17 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:13:17.745 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:17 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:13:17.746 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:17 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:13:18.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:17 smithi150 ceph-mon[100559]: Detected new or changed devices on smithi150 2023-11-27T15:13:18.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:17 smithi150 ceph-mon[100559]: Adjusting osd_memory_target on smithi150 to 8584M 2023-11-27T15:13:18.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:17 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:13:18.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:17 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:13:18.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:17 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:13:18.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:17 smithi150 ceph-mon[100559]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2023-11-27T15:13:18.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:17 smithi150 ceph-mon[100559]: osdmap e37: 6 total, 5 up, 6 in 2023-11-27T15:13:18.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:17 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:13:18.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:17 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:13:18.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:17 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:13:18.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:17 smithi102 ceph-mon[100296]: Detected new or changed devices on smithi150 2023-11-27T15:13:18.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:17 smithi102 ceph-mon[100296]: Adjusting osd_memory_target on smithi150 to 8584M 2023-11-27T15:13:18.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:17 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:13:18.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:17 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:13:18.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:17 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:13:18.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:17 smithi102 ceph-mon[100296]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2023-11-27T15:13:18.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:17 smithi102 ceph-mon[100296]: osdmap e37: 6 total, 5 up, 6 in 2023-11-27T15:13:18.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:17 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:13:18.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:17 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:13:18.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:17 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:13:18.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.406+0000 7f08bc65d700 1 -- 172.21.15.52:0/448256842 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08b41002f0 msgr2=0x7f08b4100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:18.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.406+0000 7f08bc65d700 1 --2- 172.21.15.52:0/448256842 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08b41002f0 0x7f08b4100700 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f08b0009990 tx=0x7f08b0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:13:18.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.406+0000 7f08bc65d700 1 -- 172.21.15.52:0/448256842 shutdown_connections 2023-11-27T15:13:18.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.406+0000 7f08bc65d700 1 --2- 172.21.15.52:0/448256842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08b4101e80 0x7f08b4108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:18.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.406+0000 7f08bc65d700 1 --2- 172.21.15.52:0/448256842 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08b41002f0 0x7f08b4100700 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:18.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.406+0000 7f08bc65d700 1 --2- 172.21.15.52:0/448256842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08b41014f0 0x7f08b4101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:18.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.406+0000 7f08bc65d700 1 -- 172.21.15.52:0/448256842 >> 172.21.15.52:0/448256842 conn(0x7f08b40fb900 msgr2=0x7f08b40fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:18.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.406+0000 7f08bc65d700 1 -- 172.21.15.52:0/448256842 shutdown_connections 2023-11-27T15:13:18.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.406+0000 7f08bc65d700 1 -- 172.21.15.52:0/448256842 wait complete. 2023-11-27T15:13:18.414 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.406+0000 7f08bc65d700 1 Processor -- start 2023-11-27T15:13:18.414 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.407+0000 7f08bc65d700 1 -- start start 2023-11-27T15:13:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.407+0000 7f08bc65d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08b41002f0 0x7f08b4074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.407+0000 7f08bc65d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08b41014f0 0x7f08b4072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.407+0000 7f08bc65d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08b4101e80 0x7f08b4073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.407+0000 7f08bc65d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b4073ba0 con 0x7f08b41014f0 2023-11-27T15:13:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.407+0000 7f08bc65d700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b4073d10 con 0x7f08b4101e80 2023-11-27T15:13:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.407+0000 7f08bc65d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b41a0130 con 0x7f08b41002f0 2023-11-27T15:13:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.407+0000 7f08b9bf8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08b41014f0 0x7f08b4072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.407+0000 7f08b9bf8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08b41014f0 0x7f08b4072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49330/0 (socket says 172.21.15.52:49330) 2023-11-27T15:13:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.407+0000 7f08b9bf8700 1 -- 172.21.15.52:0/4181287235 learned_addr learned my addr 172.21.15.52:0/4181287235 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:13:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.407+0000 7f08ba3f9700 1 --2- 172.21.15.52:0/4181287235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08b41002f0 0x7f08b4074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.407+0000 7f08babfa700 1 --2- 172.21.15.52:0/4181287235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08b4101e80 0x7f08b4073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.407+0000 7f08b9bf8700 1 -- 172.21.15.52:0/4181287235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08b41002f0 msgr2=0x7f08b4074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.407+0000 7f08b9bf8700 1 --2- 172.21.15.52:0/4181287235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08b41002f0 0x7f08b4074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.407+0000 7f08b9bf8700 1 -- 172.21.15.52:0/4181287235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08b4101e80 msgr2=0x7f08b4073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.407+0000 7f08b9bf8700 1 --2- 172.21.15.52:0/4181287235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08b4101e80 0x7f08b4073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.407+0000 7f08b9bf8700 1 -- 172.21.15.52:0/4181287235 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b00095f0 con 0x7f08b41014f0 2023-11-27T15:13:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.407+0000 7f08b9bf8700 1 --2- 172.21.15.52:0/4181287235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08b41014f0 0x7f08b4072ee0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f08a800d980 tx=0x7f08a800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.408+0000 7f08a77fe700 1 -- 172.21.15.52:0/4181287235 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a8009b40 con 0x7f08b41014f0 2023-11-27T15:13:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.408+0000 7f08a77fe700 1 -- 172.21.15.52:0/4181287235 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08a800de60 con 0x7f08b41014f0 2023-11-27T15:13:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.408+0000 7f08bc65d700 1 -- 172.21.15.52:0/4181287235 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08b410bcd0 con 0x7f08b41014f0 2023-11-27T15:13:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.408+0000 7f08a77fe700 1 -- 172.21.15.52:0/4181287235 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a8005580 con 0x7f08b41014f0 2023-11-27T15:13:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.408+0000 7f08bc65d700 1 -- 172.21.15.52:0/4181287235 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b410bb40 con 0x7f08b41014f0 2023-11-27T15:13:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.409+0000 7f08bc65d700 1 -- 172.21.15.52:0/4181287235 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08b4107940 con 0x7f08b41014f0 2023-11-27T15:13:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.410+0000 7f08a77fe700 1 -- 172.21.15.52:0/4181287235 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f08a8004750 con 0x7f08b41014f0 2023-11-27T15:13:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.410+0000 7f08a77fe700 1 --2- 172.21.15.52:0/4181287235 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f08a00a1150 0x7f08a00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.410+0000 7f08ba3f9700 1 --2- 172.21.15.52:0/4181287235 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f08a00a1150 0x7f08a00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.410+0000 7f08a77fe700 1 -- 172.21.15.52:0/4181287235 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4566+0+0 (secure 0 0 0) 0x7f08a80cbe60 con 0x7f08b41014f0 2023-11-27T15:13:18.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.411+0000 7f08ba3f9700 1 --2- 172.21.15.52:0/4181287235 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f08a00a1150 0x7f08a00a3600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f08b000b5c0 tx=0x7f08b0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:18.419 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.414+0000 7f08a77fe700 1 -- 172.21.15.52:0/4181287235 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f08a8096450 con 0x7f08b41014f0 2023-11-27T15:13:18.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:18.622+0000 7f08bc65d700 1 -- 172.21.15.52:0/4181287235 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f08b4061c70 con 0x7f08a00a1150 2023-11-27T15:13:19.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:18 smithi150 ceph-mon[100559]: purged_snaps scrub starts 2023-11-27T15:13:19.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:18 smithi150 ceph-mon[100559]: purged_snaps scrub ok 2023-11-27T15:13:19.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:18 smithi150 ceph-mon[100559]: osd.5 [v2:172.21.15.150:6808/461511375,v1:172.21.15.150:6809/461511375] boot 2023-11-27T15:13:19.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:18 smithi150 ceph-mon[100559]: osdmap e38: 6 total, 6 up, 6 in 2023-11-27T15:13:19.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:18 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:13:19.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:18 smithi150 ceph-mon[100559]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:13:19.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:18 smithi150 ceph-mon[100559]: osdmap e39: 6 total, 6 up, 6 in 2023-11-27T15:13:19.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:18 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:13:19.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:18 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:13:19.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:18 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:13:19.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:18 smithi102 ceph-mon[100296]: purged_snaps scrub starts 2023-11-27T15:13:19.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:18 smithi102 ceph-mon[100296]: purged_snaps scrub ok 2023-11-27T15:13:19.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:18 smithi102 ceph-mon[100296]: osd.5 [v2:172.21.15.150:6808/461511375,v1:172.21.15.150:6809/461511375] boot 2023-11-27T15:13:19.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:18 smithi102 ceph-mon[100296]: osdmap e38: 6 total, 6 up, 6 in 2023-11-27T15:13:19.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:18 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:13:19.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:18 smithi102 ceph-mon[100296]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:13:19.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:18 smithi102 ceph-mon[100296]: osdmap e39: 6 total, 6 up, 6 in 2023-11-27T15:13:19.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:18 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:13:19.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:18 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:13:19.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:18 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:13:19.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:18 smithi052 ceph-mon[99056]: purged_snaps scrub starts 2023-11-27T15:13:19.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:18 smithi052 ceph-mon[99056]: purged_snaps scrub ok 2023-11-27T15:13:19.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:18 smithi052 ceph-mon[99056]: osd.5 [v2:172.21.15.150:6808/461511375,v1:172.21.15.150:6809/461511375] boot 2023-11-27T15:13:19.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:18 smithi052 ceph-mon[99056]: osdmap e38: 6 total, 6 up, 6 in 2023-11-27T15:13:19.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:18 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:13:19.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:18 smithi052 ceph-mon[99056]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:13:19.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:18 smithi052 ceph-mon[99056]: osdmap e39: 6 total, 6 up, 6 in 2023-11-27T15:13:19.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:18 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:13:19.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:18 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:13:19.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:18 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:13:19.861 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:19 smithi052 ceph-mon[99056]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:13:19.861 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:19 smithi052 ceph-mon[99056]: osdmap e40: 6 total, 6 up, 6 in 2023-11-27T15:13:20.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:19 smithi150 ceph-mon[100559]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:13:20.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:19 smithi150 ceph-mon[100559]: osdmap e40: 6 total, 6 up, 6 in 2023-11-27T15:13:20.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:19 smithi102 ceph-mon[100296]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:13:20.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:19 smithi102 ceph-mon[100296]: osdmap e40: 6 total, 6 up, 6 in 2023-11-27T15:13:21.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:20 smithi150 ceph-mon[100559]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:13:21.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:20 smithi150 ceph-mon[100559]: osdmap e41: 6 total, 6 up, 6 in 2023-11-27T15:13:21.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:20 smithi102 ceph-mon[100296]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:13:21.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:20 smithi102 ceph-mon[100296]: osdmap e41: 6 total, 6 up, 6 in 2023-11-27T15:13:21.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:20 smithi052 ceph-mon[99056]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:13:21.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:20 smithi052 ceph-mon[99056]: osdmap e41: 6 total, 6 up, 6 in 2023-11-27T15:13:22.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:21 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2734042692' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f0ede4a-d175-4b4a-b876-c0f442048901"}]: dispatch 2023-11-27T15:13:22.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:21 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2734042692' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f0ede4a-d175-4b4a-b876-c0f442048901"}]': finished 2023-11-27T15:13:22.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:21 smithi150 ceph-mon[100559]: osdmap e42: 7 total, 6 up, 7 in 2023-11-27T15:13:22.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:21 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:13:22.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:21 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2734042692' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f0ede4a-d175-4b4a-b876-c0f442048901"}]: dispatch 2023-11-27T15:13:22.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:21 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2734042692' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f0ede4a-d175-4b4a-b876-c0f442048901"}]': finished 2023-11-27T15:13:22.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:21 smithi102 ceph-mon[100296]: osdmap e42: 7 total, 6 up, 7 in 2023-11-27T15:13:22.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:21 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:13:22.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:21 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2734042692' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f0ede4a-d175-4b4a-b876-c0f442048901"}]: dispatch 2023-11-27T15:13:22.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:21 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2734042692' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f0ede4a-d175-4b4a-b876-c0f442048901"}]': finished 2023-11-27T15:13:22.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:21 smithi052 ceph-mon[99056]: osdmap e42: 7 total, 6 up, 7 in 2023-11-27T15:13:22.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:21 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:13:23.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:22 smithi150 ceph-mon[100559]: pgmap v131: 1 pgs: 1 peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-27T15:13:23.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:22 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1055632969' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:13:23.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:22 smithi102 ceph-mon[100296]: pgmap v131: 1 pgs: 1 peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-27T15:13:23.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:22 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1055632969' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:13:23.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:22 smithi052 ceph-mon[99056]: pgmap v131: 1 pgs: 1 peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-27T15:13:23.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:22 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1055632969' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:13:24.944 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:24 smithi052 ceph-mon[99056]: pgmap v132: 1 pgs: 1 peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2023-11-27T15:13:25.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:24 smithi150 ceph-mon[100559]: pgmap v132: 1 pgs: 1 peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2023-11-27T15:13:25.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:24 smithi102 ceph-mon[100296]: pgmap v132: 1 pgs: 1 peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2023-11-27T15:13:26.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:25 smithi150 ceph-mon[100559]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-11-27T15:13:26.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:25 smithi102 ceph-mon[100296]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-11-27T15:13:26.082 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:25 smithi052 ceph-mon[99056]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-11-27T15:13:28.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:27 smithi150 ceph-mon[100559]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-27T15:13:28.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:27 smithi102 ceph-mon[100296]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-27T15:13:28.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:27 smithi052 ceph-mon[99056]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-27T15:13:28.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:28 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-11-27T15:13:28.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:28 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:13:28.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:28 smithi052 ceph-mon[99056]: Deploying daemon osd.6 on smithi052 2023-11-27T15:13:29.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:28 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-11-27T15:13:29.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:28 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:13:29.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:28 smithi150 ceph-mon[100559]: Deploying daemon osd.6 on smithi052 2023-11-27T15:13:29.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:28 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-11-27T15:13:29.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:28 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:13:29.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:28 smithi102 ceph-mon[100296]: Deploying daemon osd.6 on smithi052 2023-11-27T15:13:29.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:29 smithi052 ceph-mon[99056]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-11-27T15:13:30.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:29 smithi150 ceph-mon[100559]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-11-27T15:13:30.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:29 smithi102 ceph-mon[100296]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-11-27T15:13:31.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:31 smithi052 ceph-mon[99056]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:13:32.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:31 smithi150 ceph-mon[100559]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:13:32.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:31 smithi102 ceph-mon[100296]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:13:33.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:33 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:33.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:33 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:13:33.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:33 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:33.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:33 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:33.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:33 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:13:33.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:33 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:34.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:33 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:34.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:33 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:13:34.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:33 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:34.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:34 smithi052 ceph-mon[99056]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:13:35.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:34 smithi150 ceph-mon[100559]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:13:35.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:34 smithi102 ceph-mon[100296]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:13:35.581 INFO:teuthology.orchestra.run.smithi052.stdout:Created osd(s) 6 on host 'smithi052' 2023-11-27T15:13:35.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:35.576+0000 7f08a77fe700 1 -- 172.21.15.52:0/4181287235 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f08b4061c70 con 0x7f08a00a1150 2023-11-27T15:13:35.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:35.578+0000 7f08bc65d700 1 -- 172.21.15.52:0/4181287235 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f08a00a1150 msgr2=0x7f08a00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:35.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:35.578+0000 7f08bc65d700 1 --2- 172.21.15.52:0/4181287235 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f08a00a1150 0x7f08a00a3600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f08b000b5c0 tx=0x7f08b0005fb0 comp rx=0 tx=0).stop 2023-11-27T15:13:35.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:35.578+0000 7f08bc65d700 1 -- 172.21.15.52:0/4181287235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08b41014f0 msgr2=0x7f08b4072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:35.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:35.578+0000 7f08bc65d700 1 --2- 172.21.15.52:0/4181287235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08b41014f0 0x7f08b4072ee0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f08a800d980 tx=0x7f08a800dd40 comp rx=0 tx=0).stop 2023-11-27T15:13:35.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:35.578+0000 7f08bc65d700 1 -- 172.21.15.52:0/4181287235 shutdown_connections 2023-11-27T15:13:35.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:35.578+0000 7f08bc65d700 1 --2- 172.21.15.52:0/4181287235 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f08a00a1150 0x7f08a00a3600 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:35.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:35.578+0000 7f08bc65d700 1 --2- 172.21.15.52:0/4181287235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08b4101e80 0x7f08b4073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:35.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:35.578+0000 7f08bc65d700 1 --2- 172.21.15.52:0/4181287235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08b41002f0 0x7f08b4074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:35.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:35.578+0000 7f08bc65d700 1 --2- 172.21.15.52:0/4181287235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08b41014f0 0x7f08b4072ee0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:35.584 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:35.578+0000 7f08bc65d700 1 -- 172.21.15.52:0/4181287235 >> 172.21.15.52:0/4181287235 conn(0x7f08b40fb900 msgr2=0x7f08b4104a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:35.584 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:35.578+0000 7f08bc65d700 1 -- 172.21.15.52:0/4181287235 shutdown_connections 2023-11-27T15:13:35.584 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:13:35.578+0000 7f08bc65d700 1 -- 172.21.15.52:0/4181287235 wait complete. 2023-11-27T15:13:35.996 DEBUG:teuthology.orchestra.run.smithi052:osd.6> sudo journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.6.service 2023-11-27T15:13:35.999 INFO:tasks.cephadm:Deploying osd.7 on smithi102 with /dev/vg_nvme/lv_2... 2023-11-27T15:13:35.999 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-11-27T15:13:36.031 INFO:journalctl@ceph.osd.6.smithi052.stdout:-- Logs begin at Mon 2023-11-27 14:52:43 UTC. -- 2023-11-27T15:13:36.235 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:13:36 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:13:36.203+0000 7fa2c9df7640 -1 osd.6 0 log_to_monitors true 2023-11-27T15:13:36.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:36 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:36.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:36 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:36.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:36 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:36.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:36 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:36.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:36 smithi102 ceph-mon[100296]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:13:36.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:36 smithi102 ceph-mon[100296]: from='osd.6 [v2:172.21.15.52:6818/1002457464,v1:172.21.15.52:6819/1002457464]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-11-27T15:13:36.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:36 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:36.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:36 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:36.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:36 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:36.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:36 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:36.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:36 smithi150 ceph-mon[100559]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:13:36.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:36 smithi150 ceph-mon[100559]: from='osd.6 [v2:172.21.15.52:6818/1002457464,v1:172.21.15.52:6819/1002457464]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-11-27T15:13:36.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:36 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:36.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:36 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:36.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:36 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:36.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:36 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:36.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:36 smithi052 ceph-mon[99056]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:13:36.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:36 smithi052 ceph-mon[99056]: from='osd.6 [v2:172.21.15.52:6818/1002457464,v1:172.21.15.52:6819/1002457464]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-11-27T15:13:36.662 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:13:37.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:37 smithi102 ceph-mon[100296]: from='osd.6 [v2:172.21.15.52:6818/1002457464,v1:172.21.15.52:6819/1002457464]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-11-27T15:13:39.211 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:37 smithi102 ceph-mon[100296]: osdmap e43: 7 total, 6 up, 7 in 2023-11-27T15:13:39.211 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:37 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:13:39.211 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:37 smithi102 ceph-mon[100296]: from='osd.6 [v2:172.21.15.52:6818/1002457464,v1:172.21.15.52:6819/1002457464]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2023-11-27T15:13:39.212 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:13:37 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:13:37.583+0000 7fa2bf46e700 -1 osd.6 0 waiting for initial osdmap 2023-11-27T15:13:39.212 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:13:37 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:13:37.597+0000 7fa2bb263700 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:13:39.213 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:37 smithi150 ceph-mon[100559]: from='osd.6 [v2:172.21.15.52:6818/1002457464,v1:172.21.15.52:6819/1002457464]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-11-27T15:13:39.213 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:37 smithi150 ceph-mon[100559]: osdmap e43: 7 total, 6 up, 7 in 2023-11-27T15:13:39.213 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:37 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:13:39.213 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:37 smithi150 ceph-mon[100559]: from='osd.6 [v2:172.21.15.52:6818/1002457464,v1:172.21.15.52:6819/1002457464]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2023-11-27T15:13:39.213 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:38 smithi150 ceph-mon[100559]: from='osd.6 [v2:172.21.15.52:6818/1002457464,v1:172.21.15.52:6819/1002457464]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2023-11-27T15:13:39.213 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:38 smithi150 ceph-mon[100559]: osdmap e44: 7 total, 6 up, 7 in 2023-11-27T15:13:39.213 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:38 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:13:39.213 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:38 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:13:39.213 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:38 smithi150 ceph-mon[100559]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:13:39.213 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:38 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:39.214 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:38 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:39.214 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:38 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:13:39.214 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:38 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:13:39.214 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:38 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:13:39.214 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:38 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:39.214 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:38 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:13:39.214 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:38 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:13:39.214 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:38 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:39.214 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:37 smithi052 ceph-mon[99056]: from='osd.6 [v2:172.21.15.52:6818/1002457464,v1:172.21.15.52:6819/1002457464]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-11-27T15:13:39.215 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:37 smithi052 ceph-mon[99056]: osdmap e43: 7 total, 6 up, 7 in 2023-11-27T15:13:39.215 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:37 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:13:39.215 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:37 smithi052 ceph-mon[99056]: from='osd.6 [v2:172.21.15.52:6818/1002457464,v1:172.21.15.52:6819/1002457464]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2023-11-27T15:13:39.215 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:38 smithi052 ceph-mon[99056]: from='osd.6 [v2:172.21.15.52:6818/1002457464,v1:172.21.15.52:6819/1002457464]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2023-11-27T15:13:39.215 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:38 smithi052 ceph-mon[99056]: osdmap e44: 7 total, 6 up, 7 in 2023-11-27T15:13:39.215 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:38 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:13:39.215 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:38 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:13:39.215 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:38 smithi052 ceph-mon[99056]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:13:39.215 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:38 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:39.215 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:38 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:39.215 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:38 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:13:39.215 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:38 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:13:39.215 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:38 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:13:39.216 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:38 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:39.216 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:38 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:13:39.216 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:38 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:13:39.216 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:38 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:39.216 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-11-27T15:13:39.230 INFO:teuthology.orchestra.run.smithi102.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-11-27T15:13:39.231 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10+0 records in 2023-11-27T15:13:39.231 INFO:teuthology.orchestra.run.smithi102.stderr:10+0 records out 2023-11-27T15:13:39.231 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0158669 s, 661 MB/s 2023-11-27T15:13:39.231 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping successful for: 2023-11-27T15:13:39.231 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi102:vg_nvme/lv_2 2023-11-27T15:13:39.232 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:38 smithi102 ceph-mon[100296]: from='osd.6 [v2:172.21.15.52:6818/1002457464,v1:172.21.15.52:6819/1002457464]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2023-11-27T15:13:39.232 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:38 smithi102 ceph-mon[100296]: osdmap e44: 7 total, 6 up, 7 in 2023-11-27T15:13:39.232 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:38 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:13:39.232 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:38 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:13:39.232 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:38 smithi102 ceph-mon[100296]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:13:39.232 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:38 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:39.232 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:38 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:39.232 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:38 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:13:39.232 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:38 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:13:39.232 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:38 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:13:39.232 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:38 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:39.232 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:38 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:13:39.232 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:38 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:13:39.233 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:38 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:39.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:39 smithi052 ceph-mon[99056]: purged_snaps scrub starts 2023-11-27T15:13:39.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:39 smithi052 ceph-mon[99056]: purged_snaps scrub ok 2023-11-27T15:13:39.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:39 smithi052 ceph-mon[99056]: Detected new or changed devices on smithi052 2023-11-27T15:13:39.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:39 smithi052 ceph-mon[99056]: Adjusting osd_memory_target on smithi052 to 5724M 2023-11-27T15:13:39.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:39 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:13:39.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:39 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:13:39.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:39 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:13:39.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:39 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:13:39.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:39 smithi052 ceph-mon[99056]: osd.6 [v2:172.21.15.52:6818/1002457464,v1:172.21.15.52:6819/1002457464] boot 2023-11-27T15:13:39.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:39 smithi052 ceph-mon[99056]: osdmap e45: 7 total, 7 up, 7 in 2023-11-27T15:13:39.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:39 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:13:39.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:39 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:13:39.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:39 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:13:39.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:39 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:13:39.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:39 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:39.871 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:13:40.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:39 smithi150 ceph-mon[100559]: purged_snaps scrub starts 2023-11-27T15:13:40.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:39 smithi150 ceph-mon[100559]: purged_snaps scrub ok 2023-11-27T15:13:40.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:39 smithi150 ceph-mon[100559]: Detected new or changed devices on smithi052 2023-11-27T15:13:40.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:39 smithi150 ceph-mon[100559]: Adjusting osd_memory_target on smithi052 to 5724M 2023-11-27T15:13:40.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:39 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:13:40.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:39 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:13:40.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:39 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:13:40.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:39 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:13:40.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:39 smithi150 ceph-mon[100559]: osd.6 [v2:172.21.15.52:6818/1002457464,v1:172.21.15.52:6819/1002457464] boot 2023-11-27T15:13:40.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:39 smithi150 ceph-mon[100559]: osdmap e45: 7 total, 7 up, 7 in 2023-11-27T15:13:40.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:39 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:13:40.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:39 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:13:40.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:39 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:13:40.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:39 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:13:40.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:39 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:40.059 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:39 smithi102 ceph-mon[100296]: purged_snaps scrub starts 2023-11-27T15:13:40.059 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:39 smithi102 ceph-mon[100296]: purged_snaps scrub ok 2023-11-27T15:13:40.059 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:39 smithi102 ceph-mon[100296]: Detected new or changed devices on smithi052 2023-11-27T15:13:40.059 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:39 smithi102 ceph-mon[100296]: Adjusting osd_memory_target on smithi052 to 5724M 2023-11-27T15:13:40.059 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:39 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:13:40.060 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:39 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:13:40.060 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:39 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:13:40.060 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:39 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:13:40.060 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:39 smithi102 ceph-mon[100296]: osd.6 [v2:172.21.15.52:6818/1002457464,v1:172.21.15.52:6819/1002457464] boot 2023-11-27T15:13:40.060 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:39 smithi102 ceph-mon[100296]: osdmap e45: 7 total, 7 up, 7 in 2023-11-27T15:13:40.060 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:39 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:13:40.060 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:39 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:13:40.060 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:39 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:13:40.060 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:39 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:13:40.060 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:39 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:40.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:40 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:13:40.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:40 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:13:40.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:40 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:13:40.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:40 smithi052 ceph-mon[99056]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 39 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:13:40.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:40 smithi052 ceph-mon[99056]: osdmap e46: 7 total, 7 up, 7 in 2023-11-27T15:13:40.935 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.929+0000 7f6cc3027700 1 -- 172.21.15.102:0/1745147985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cbc1003e0 msgr2=0x7f6cbc100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:40.935 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.929+0000 7f6cc3027700 1 --2- 172.21.15.102:0/1745147985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cbc1003e0 0x7f6cbc100830 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009990 tx=0x7f6ca8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:13:40.936 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.929+0000 7f6cc3027700 1 -- 172.21.15.102:0/1745147985 shutdown_connections 2023-11-27T15:13:40.936 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.929+0000 7f6cc3027700 1 --2- 172.21.15.102:0/1745147985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6cbc100d70 0x7f6cbc107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:40.936 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.929+0000 7f6cc3027700 1 --2- 172.21.15.102:0/1745147985 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cbc0ff1e0 0x7f6cbc0ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:40.936 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.929+0000 7f6cc3027700 1 --2- 172.21.15.102:0/1745147985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cbc1003e0 0x7f6cbc100830 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:40.936 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.929+0000 7f6cc3027700 1 -- 172.21.15.102:0/1745147985 >> 172.21.15.102:0/1745147985 conn(0x7f6cbc0fa7f0 msgr2=0x7f6cbc0fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:40.936 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.929+0000 7f6cc3027700 1 -- 172.21.15.102:0/1745147985 shutdown_connections 2023-11-27T15:13:40.936 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.929+0000 7f6cc3027700 1 -- 172.21.15.102:0/1745147985 wait complete. 2023-11-27T15:13:40.937 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.930+0000 7f6cc3027700 1 Processor -- start 2023-11-27T15:13:40.937 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.930+0000 7f6cc3027700 1 -- start start 2023-11-27T15:13:40.937 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.930+0000 7f6cc3027700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6cbc0ff1e0 0x7f6cbc0699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:40.937 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.930+0000 7f6cc3027700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cbc1003e0 0x7f6cbc19aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:40.937 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.930+0000 7f6cc3027700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cbc100d70 0x7f6cbc19d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:40.937 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.930+0000 7f6cc3027700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc19da90 con 0x7f6cbc1003e0 2023-11-27T15:13:40.937 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.930+0000 7f6cc3027700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc19dc00 con 0x7f6cbc100d70 2023-11-27T15:13:40.937 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.930+0000 7f6cc3027700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc19ddc0 con 0x7f6cbc0ff1e0 2023-11-27T15:13:40.938 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.930+0000 7f6cc15c4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cbc100d70 0x7f6cbc19d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:40.938 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.930+0000 7f6cc15c4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cbc100d70 0x7f6cbc19d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:41902/0 (socket says 172.21.15.102:41902) 2023-11-27T15:13:40.938 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.930+0000 7f6cc15c4700 1 -- 172.21.15.102:0/1830418797 learned_addr learned my addr 172.21.15.102:0/1830418797 (peer_addr_for_me v2:172.21.15.102:0/0) 2023-11-27T15:13:40.938 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.930+0000 7f6cc0dc3700 1 --2- 172.21.15.102:0/1830418797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6cbc0ff1e0 0x7f6cbc0699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:40.938 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.930+0000 7f6cbbfff700 1 --2- 172.21.15.102:0/1830418797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cbc1003e0 0x7f6cbc19aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:40.938 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.931+0000 7f6cc15c4700 1 -- 172.21.15.102:0/1830418797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6cbc0ff1e0 msgr2=0x7f6cbc0699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:40.938 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.931+0000 7f6cc15c4700 1 --2- 172.21.15.102:0/1830418797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6cbc0ff1e0 0x7f6cbc0699c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:40.939 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.931+0000 7f6cc15c4700 1 -- 172.21.15.102:0/1830418797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cbc1003e0 msgr2=0x7f6cbc19aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:40.939 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.931+0000 7f6cc15c4700 1 --2- 172.21.15.102:0/1830418797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cbc1003e0 0x7f6cbc19aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:40.939 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.931+0000 7f6cc15c4700 1 -- 172.21.15.102:0/1830418797 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca80095f0 con 0x7f6cbc100d70 2023-11-27T15:13:40.939 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.931+0000 7f6cc15c4700 1 --2- 172.21.15.102:0/1830418797 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cbc100d70 0x7f6cbc19d2f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400ada0 tx=0x7f6cb400c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:40.939 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.931+0000 7f6cb9ffb700 1 -- 172.21.15.102:0/1830418797 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb400f070 con 0x7f6cbc100d70 2023-11-27T15:13:40.939 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.931+0000 7f6cb9ffb700 1 -- 172.21.15.102:0/1830418797 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cb4007b30 con 0x7f6cbc100d70 2023-11-27T15:13:40.939 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.931+0000 7f6cc3027700 1 -- 172.21.15.102:0/1830418797 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cbc10ab40 con 0x7f6cbc100d70 2023-11-27T15:13:40.939 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.931+0000 7f6cb9ffb700 1 -- 172.21.15.102:0/1830418797 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4005280 con 0x7f6cbc100d70 2023-11-27T15:13:40.939 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.931+0000 7f6cc3027700 1 -- 172.21.15.102:0/1830418797 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cbc10a9b0 con 0x7f6cbc100d70 2023-11-27T15:13:40.940 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.932+0000 7f6cc3027700 1 -- 172.21.15.102:0/1830418797 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cbc106740 con 0x7f6cbc100d70 2023-11-27T15:13:40.940 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.933+0000 7f6cb9ffb700 1 -- 172.21.15.102:0/1830418797 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6cb400a090 con 0x7f6cbc100d70 2023-11-27T15:13:40.940 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.934+0000 7f6cb9ffb700 1 --2- 172.21.15.102:0/1830418797 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6cac0a11d0 0x7f6cac0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:40.941 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.934+0000 7f6cb9ffb700 1 -- 172.21.15.102:0/1830418797 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f6cb40cb6f0 con 0x7f6cbc100d70 2023-11-27T15:13:40.941 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.934+0000 7f6cc0dc3700 1 --2- 172.21.15.102:0/1830418797 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6cac0a11d0 0x7f6cac0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:40.941 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.935+0000 7f6cc0dc3700 1 --2- 172.21.15.102:0/1830418797 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6cac0a11d0 0x7f6cac0a3680 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc100240 tx=0x7f6cb000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:40.941 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:40.937+0000 7f6cb9ffb700 1 -- 172.21.15.102:0/1830418797 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6cb40995d0 con 0x7f6cbc100d70 2023-11-27T15:13:41.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:40 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:13:41.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:40 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:13:41.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:40 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:13:41.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:40 smithi150 ceph-mon[100559]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 39 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:13:41.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:40 smithi150 ceph-mon[100559]: osdmap e46: 7 total, 7 up, 7 in 2023-11-27T15:13:41.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:40 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:13:41.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:40 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:13:41.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:40 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:13:41.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:40 smithi102 ceph-mon[100296]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 39 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:13:41.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:40 smithi102 ceph-mon[100296]: osdmap e46: 7 total, 7 up, 7 in 2023-11-27T15:13:41.135 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:41.131+0000 7f6cc3027700 1 -- 172.21.15.102:0/1830418797 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f6cbc061bc0 con 0x7f6cac0a11d0 2023-11-27T15:13:41.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:41 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:13:41.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:41 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:13:41.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:41 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:13:42.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:13:42.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:13:42.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:41 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:13:42.034 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:41 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:13:42.034 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:41 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:13:42.034 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:41 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:13:42.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:42 smithi052 ceph-mon[99056]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:13:42.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:42 smithi052 ceph-mon[99056]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:13:43.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:42 smithi150 ceph-mon[100559]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:13:43.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:42 smithi150 ceph-mon[100559]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:13:43.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:42 smithi102 ceph-mon[100296]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:13:43.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:42 smithi102 ceph-mon[100296]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:13:43.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:43 smithi052 ceph-mon[99056]: from='client.? 172.21.15.102:0/131714267' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b635192-abcf-4bbf-bf78-a019b351bca5"}]: dispatch 2023-11-27T15:13:43.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:43 smithi052 ceph-mon[99056]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b635192-abcf-4bbf-bf78-a019b351bca5"}]: dispatch 2023-11-27T15:13:43.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:43 smithi052 ceph-mon[99056]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b635192-abcf-4bbf-bf78-a019b351bca5"}]': finished 2023-11-27T15:13:43.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:43 smithi052 ceph-mon[99056]: osdmap e47: 8 total, 7 up, 8 in 2023-11-27T15:13:43.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:43 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:13:44.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:43 smithi150 ceph-mon[100559]: from='client.? 172.21.15.102:0/131714267' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b635192-abcf-4bbf-bf78-a019b351bca5"}]: dispatch 2023-11-27T15:13:44.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:43 smithi150 ceph-mon[100559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b635192-abcf-4bbf-bf78-a019b351bca5"}]: dispatch 2023-11-27T15:13:44.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:43 smithi150 ceph-mon[100559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b635192-abcf-4bbf-bf78-a019b351bca5"}]': finished 2023-11-27T15:13:44.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:43 smithi150 ceph-mon[100559]: osdmap e47: 8 total, 7 up, 8 in 2023-11-27T15:13:44.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:43 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:13:44.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:43 smithi102 ceph-mon[100296]: from='client.? 172.21.15.102:0/131714267' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b635192-abcf-4bbf-bf78-a019b351bca5"}]: dispatch 2023-11-27T15:13:44.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:43 smithi102 ceph-mon[100296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b635192-abcf-4bbf-bf78-a019b351bca5"}]: dispatch 2023-11-27T15:13:44.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:43 smithi102 ceph-mon[100296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b635192-abcf-4bbf-bf78-a019b351bca5"}]': finished 2023-11-27T15:13:44.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:43 smithi102 ceph-mon[100296]: osdmap e47: 8 total, 7 up, 8 in 2023-11-27T15:13:44.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:43 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:13:45.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:44 smithi150 ceph-mon[100559]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:13:45.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:44 smithi150 ceph-mon[100559]: from='client.? 172.21.15.102:0/3944512443' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:13:45.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:44 smithi102 ceph-mon[100296]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:13:45.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:44 smithi102 ceph-mon[100296]: from='client.? 172.21.15.102:0/3944512443' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:13:45.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:44 smithi052 ceph-mon[99056]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:13:45.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:44 smithi052 ceph-mon[99056]: from='client.? 172.21.15.102:0/3944512443' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:13:46.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:45 smithi150 ceph-mon[100559]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:13:46.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:45 smithi102 ceph-mon[100296]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:13:46.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:45 smithi052 ceph-mon[99056]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:13:48.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:47 smithi150 ceph-mon[100559]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:13:48.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:47 smithi102 ceph-mon[100296]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:13:48.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:47 smithi052 ceph-mon[99056]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:13:49.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:49 smithi102 ceph-mon[100296]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:13:50.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:49 smithi150 ceph-mon[100559]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:13:50.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:49 smithi052 ceph-mon[99056]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:13:50.910 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:50 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-11-27T15:13:50.910 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:50 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:13:50.911 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:50 smithi102 ceph-mon[100296]: Deploying daemon osd.7 on smithi102 2023-11-27T15:13:51.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:50 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-11-27T15:13:51.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:50 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:13:51.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:50 smithi150 ceph-mon[100559]: Deploying daemon osd.7 on smithi102 2023-11-27T15:13:51.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:50 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-11-27T15:13:51.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:50 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:13:51.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:50 smithi052 ceph-mon[99056]: Deploying daemon osd.7 on smithi102 2023-11-27T15:13:52.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:51 smithi150 ceph-mon[100559]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:13:52.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:51 smithi102 ceph-mon[100296]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:13:52.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:51 smithi052 ceph-mon[99056]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:13:55.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:54 smithi150 ceph-mon[100559]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:13:55.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:54 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:55.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:54 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:55.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:54 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:13:55.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:54 smithi102 ceph-mon[100296]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:13:55.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:54 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:55.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:54 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:55.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:54 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:13:55.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:54 smithi052 ceph-mon[99056]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:13:55.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:54 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:55.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:54 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:55.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:54 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:13:56.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:55 smithi150 ceph-mon[100559]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:13:56.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:55 smithi102 ceph-mon[100296]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:13:56.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:55 smithi052 ceph-mon[99056]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:13:57.199 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:57.194+0000 7f6cb9ffb700 1 -- 172.21.15.102:0/1830418797 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6cbc061bc0 con 0x7f6cac0a11d0 2023-11-27T15:13:57.199 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 7 on host 'smithi102' 2023-11-27T15:13:57.201 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:57.197+0000 7f6cc3027700 1 -- 172.21.15.102:0/1830418797 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6cac0a11d0 msgr2=0x7f6cac0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:57.201 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:57.197+0000 7f6cc3027700 1 --2- 172.21.15.102:0/1830418797 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6cac0a11d0 0x7f6cac0a3680 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc100240 tx=0x7f6cb000a3b0 comp rx=0 tx=0).stop 2023-11-27T15:13:57.201 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:57.197+0000 7f6cc3027700 1 -- 172.21.15.102:0/1830418797 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cbc100d70 msgr2=0x7f6cbc19d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:57.201 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:57.197+0000 7f6cc3027700 1 --2- 172.21.15.102:0/1830418797 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cbc100d70 0x7f6cbc19d2f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400ada0 tx=0x7f6cb400c330 comp rx=0 tx=0).stop 2023-11-27T15:13:57.201 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:57.197+0000 7f6cc3027700 1 -- 172.21.15.102:0/1830418797 shutdown_connections 2023-11-27T15:13:57.201 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:57.197+0000 7f6cc3027700 1 --2- 172.21.15.102:0/1830418797 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6cac0a11d0 0x7f6cac0a3680 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:57.202 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:57.197+0000 7f6cc3027700 1 --2- 172.21.15.102:0/1830418797 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6cbc100d70 0x7f6cbc19d2f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:57.202 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:57.197+0000 7f6cc3027700 1 --2- 172.21.15.102:0/1830418797 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6cbc0ff1e0 0x7f6cbc0699c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:57.202 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:57.197+0000 7f6cc3027700 1 --2- 172.21.15.102:0/1830418797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cbc1003e0 0x7f6cbc19aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:57.202 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:57.197+0000 7f6cc3027700 1 -- 172.21.15.102:0/1830418797 >> 172.21.15.102:0/1830418797 conn(0x7f6cbc0fa7f0 msgr2=0x7f6cbc103880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:57.202 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:57.197+0000 7f6cc3027700 1 -- 172.21.15.102:0/1830418797 shutdown_connections 2023-11-27T15:13:57.202 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:13:57.197+0000 7f6cc3027700 1 -- 172.21.15.102:0/1830418797 wait complete. 2023-11-27T15:13:57.663 DEBUG:teuthology.orchestra.run.smithi102:osd.7> sudo journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.7.service 2023-11-27T15:13:57.665 INFO:tasks.cephadm:Deploying osd.8 on smithi150 with /dev/vg_nvme/lv_2... 2023-11-27T15:13:57.665 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-11-27T15:13:57.693 INFO:journalctl@ceph.osd.7.smithi102.stdout:-- Logs begin at Mon 2023-11-27 14:52:53 UTC. -- 2023-11-27T15:13:58.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:57 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:58.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:57 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:58.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:57 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:58.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:57 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:58.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:57 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:58.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:57 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:58.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:57 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:58.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:57 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:58.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:57 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:58.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:57 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:58.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:57 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:58.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:57 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:13:58.318 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:13:58.459 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:13:58 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:13:58.194+0000 7f773fcb4640 -1 osd.7 0 log_to_monitors true 2023-11-27T15:13:58.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:58 smithi150 ceph-mon[100559]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:13:58.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:58 smithi150 ceph-mon[100559]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-27T15:13:58.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:13:58 smithi150 ceph-mon[100559]: from='osd.7 [v2:172.21.15.102:6816/3249840064,v1:172.21.15.102:6817/3249840064]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-27T15:13:58.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:58 smithi102 ceph-mon[100296]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:13:58.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:58 smithi102 ceph-mon[100296]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-27T15:13:58.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:13:58 smithi102 ceph-mon[100296]: from='osd.7 [v2:172.21.15.102:6816/3249840064,v1:172.21.15.102:6817/3249840064]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-27T15:13:59.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:58 smithi052 ceph-mon[99056]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:13:59.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:58 smithi052 ceph-mon[99056]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-27T15:13:59.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:13:58 smithi052 ceph-mon[99056]: from='osd.7 [v2:172.21.15.102:6816/3249840064,v1:172.21.15.102:6817/3249840064]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-27T15:13:59.814 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-11-27T15:13:59.814 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-11-27T15:13:59.814 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2023-11-27T15:13:59.814 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2023-11-27T15:13:59.815 INFO:teuthology.orchestra.run.smithi150.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111004 s, 945 MB/s 2023-11-27T15:13:59.815 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 2023-11-27T15:13:59.815 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2023-11-27T15:14:00.259 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi150:vg_nvme/lv_2 2023-11-27T15:14:00.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:00 smithi150 ceph-mon[100559]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-11-27T15:14:00.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:00 smithi150 ceph-mon[100559]: osdmap e48: 8 total, 7 up, 8 in 2023-11-27T15:14:00.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:00 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:14:00.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:00 smithi150 ceph-mon[100559]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2023-11-27T15:14:00.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:00 smithi150 ceph-mon[100559]: from='osd.7 [v2:172.21.15.102:6816/3249840064,v1:172.21.15.102:6817/3249840064]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2023-11-27T15:14:00.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:00 smithi150 ceph-mon[100559]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:14:00.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:00 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:00.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:00 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:00.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:00 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:00.530 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:00 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:00.530 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:00 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:00.530 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:00 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:00.530 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:00 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:00.530 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:00 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:00.530 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:00 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:00.530 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:00 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:14:00.530 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:00 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:00.530 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:00 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:00.530 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:00 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:00.578 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:14:00 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:14:00.155+0000 7f773532b700 -1 osd.7 0 waiting for initial osdmap 2023-11-27T15:14:00.578 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:14:00 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:14:00.164+0000 7f772e91b700 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:14:00.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:00 smithi102 ceph-mon[100296]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-11-27T15:14:00.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:00 smithi102 ceph-mon[100296]: osdmap e48: 8 total, 7 up, 8 in 2023-11-27T15:14:00.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:00 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:14:00.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:00 smithi102 ceph-mon[100296]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2023-11-27T15:14:00.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:00 smithi102 ceph-mon[100296]: from='osd.7 [v2:172.21.15.102:6816/3249840064,v1:172.21.15.102:6817/3249840064]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2023-11-27T15:14:00.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:00 smithi102 ceph-mon[100296]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:14:00.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:00 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:00.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:00 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:00.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:00 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:00.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:00 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:00.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:00 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:00.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:00 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:00.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:00 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:00.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:00 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:00.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:00 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:00.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:00 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:14:00.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:00 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:00.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:00 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:00.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:00 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:00.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:00 smithi052 ceph-mon[99056]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-11-27T15:14:00.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:00 smithi052 ceph-mon[99056]: osdmap e48: 8 total, 7 up, 8 in 2023-11-27T15:14:00.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:00 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:14:00.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:00 smithi052 ceph-mon[99056]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2023-11-27T15:14:00.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:00 smithi052 ceph-mon[99056]: from='osd.7 [v2:172.21.15.102:6816/3249840064,v1:172.21.15.102:6817/3249840064]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2023-11-27T15:14:00.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:00 smithi052 ceph-mon[99056]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:14:00.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:00 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:00.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:00 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:00.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:00 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:00.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:00 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:00.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:00 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:00.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:00 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:00.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:00 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:00.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:00 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:00.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:00 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:00.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:00 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:14:00.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:00 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:00.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:00 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:00.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:00 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:00.900 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:14:01.380 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:01 smithi150 ceph-mon[100559]: Detected new or changed devices on smithi102 2023-11-27T15:14:01.380 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:01 smithi150 ceph-mon[100559]: Adjusting osd_memory_target on smithi102 to 5722M 2023-11-27T15:14:01.380 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:01 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:14:01.380 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:01 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:14:01.380 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:01 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:14:01.380 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:01 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:14:01.380 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:01 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:14:01.380 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:01 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:14:01.380 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:01 smithi150 ceph-mon[100559]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2023-11-27T15:14:01.380 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:01 smithi150 ceph-mon[100559]: osdmap e49: 8 total, 7 up, 8 in 2023-11-27T15:14:01.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:01 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:14:01.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:01 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:14:01.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:01 smithi102 ceph-mon[100296]: Detected new or changed devices on smithi102 2023-11-27T15:14:01.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:01 smithi102 ceph-mon[100296]: Adjusting osd_memory_target on smithi102 to 5722M 2023-11-27T15:14:01.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:01 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:14:01.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:01 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:14:01.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:01 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:14:01.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:01 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:14:01.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:01 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:14:01.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:01 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:14:01.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:01 smithi102 ceph-mon[100296]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2023-11-27T15:14:01.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:01 smithi102 ceph-mon[100296]: osdmap e49: 8 total, 7 up, 8 in 2023-11-27T15:14:01.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:01 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:14:01.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:01 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:14:01.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:01 smithi052 ceph-mon[99056]: Detected new or changed devices on smithi102 2023-11-27T15:14:01.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:01 smithi052 ceph-mon[99056]: Adjusting osd_memory_target on smithi102 to 5722M 2023-11-27T15:14:01.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:01 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:14:01.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:01 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:14:01.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:01 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:14:01.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:01 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:14:01.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:01 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:14:01.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:01 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:14:01.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:01 smithi052 ceph-mon[99056]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2023-11-27T15:14:01.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:01 smithi052 ceph-mon[99056]: osdmap e49: 8 total, 7 up, 8 in 2023-11-27T15:14:01.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:01 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:14:01.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:01 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:14:02.096 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.091+0000 7fec6f5f0700 1 -- 172.21.15.150:0/2270590883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec680ff460 msgr2=0x7fec680ff870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:02.096 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.091+0000 7fec6f5f0700 1 --2- 172.21.15.150:0/2270590883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec680ff460 0x7fec680ff870 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fec64009990 tx=0x7fec64009ca0 comp rx=0 tx=0).stop 2023-11-27T15:14:02.096 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.091+0000 7fec6f5f0700 1 -- 172.21.15.150:0/2270590883 shutdown_connections 2023-11-27T15:14:02.096 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.091+0000 7fec6f5f0700 1 --2- 172.21.15.150:0/2270590883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec68100ff0 0x7fec681078b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:02.096 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.091+0000 7fec6f5f0700 1 --2- 172.21.15.150:0/2270590883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec680ff460 0x7fec680ff870 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:02.096 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.091+0000 7fec6f5f0700 1 --2- 172.21.15.150:0/2270590883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec68100660 0x7fec68100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:02.096 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.091+0000 7fec6f5f0700 1 -- 172.21.15.150:0/2270590883 >> 172.21.15.150:0/2270590883 conn(0x7fec680faad0 msgr2=0x7fec680fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:02.096 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.091+0000 7fec6f5f0700 1 -- 172.21.15.150:0/2270590883 shutdown_connections 2023-11-27T15:14:02.097 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.091+0000 7fec6f5f0700 1 -- 172.21.15.150:0/2270590883 wait complete. 2023-11-27T15:14:02.098 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.091+0000 7fec6f5f0700 1 Processor -- start 2023-11-27T15:14:02.098 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.091+0000 7fec6f5f0700 1 -- start start 2023-11-27T15:14:02.099 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.092+0000 7fec6f5f0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec680ff460 0x7fec68195020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:02.099 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.092+0000 7fec6f5f0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec68100660 0x7fec68195560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:02.099 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.092+0000 7fec6f5f0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec68100ff0 0x7fec6819c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:02.099 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.092+0000 7fec6f5f0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6819cbb0 con 0x7fec680ff460 2023-11-27T15:14:02.099 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.092+0000 7fec6f5f0700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6819ccf0 con 0x7fec68100660 2023-11-27T15:14:02.099 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.092+0000 7fec6f5f0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6819ce80 con 0x7fec68100ff0 2023-11-27T15:14:02.099 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.092+0000 7fec6db8d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec68100ff0 0x7fec6819c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:02.099 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.092+0000 7fec6db8d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec68100ff0 0x7fec6819c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49576/0 (socket says 172.21.15.150:49576) 2023-11-27T15:14:02.099 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.092+0000 7fec6db8d700 1 -- 172.21.15.150:0/2552688185 learned_addr learned my addr 172.21.15.150:0/2552688185 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-27T15:14:02.099 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.092+0000 7fec6cb8b700 1 --2- 172.21.15.150:0/2552688185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec68100660 0x7fec68195560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:02.100 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.092+0000 7fec6d38c700 1 --2- 172.21.15.150:0/2552688185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec680ff460 0x7fec68195020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:02.100 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.092+0000 7fec6db8d700 1 -- 172.21.15.150:0/2552688185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec68100660 msgr2=0x7fec68195560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:02.100 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.092+0000 7fec6db8d700 1 --2- 172.21.15.150:0/2552688185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec68100660 0x7fec68195560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:02.100 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.092+0000 7fec6db8d700 1 -- 172.21.15.150:0/2552688185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec680ff460 msgr2=0x7fec68195020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:02.100 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.092+0000 7fec6db8d700 1 --2- 172.21.15.150:0/2552688185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec680ff460 0x7fec68195020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:02.100 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.092+0000 7fec6db8d700 1 -- 172.21.15.150:0/2552688185 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec640095f0 con 0x7fec68100ff0 2023-11-27T15:14:02.100 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.092+0000 7fec6db8d700 1 --2- 172.21.15.150:0/2552688185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec68100ff0 0x7fec6819c5e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fec6000c270 tx=0x7fec6000c580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:02.100 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.092+0000 7fec5e7fc700 1 -- 172.21.15.150:0/2552688185 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec60011670 con 0x7fec68100ff0 2023-11-27T15:14:02.100 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.092+0000 7fec6f5f0700 1 -- 172.21.15.150:0/2552688185 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec6810adb0 con 0x7fec68100ff0 2023-11-27T15:14:02.100 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.092+0000 7fec5e7fc700 1 -- 172.21.15.150:0/2552688185 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec60011c90 con 0x7fec68100ff0 2023-11-27T15:14:02.101 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.092+0000 7fec5e7fc700 1 -- 172.21.15.150:0/2552688185 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6001ac70 con 0x7fec68100ff0 2023-11-27T15:14:02.101 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.092+0000 7fec6f5f0700 1 -- 172.21.15.150:0/2552688185 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec6810ac20 con 0x7fec68100ff0 2023-11-27T15:14:02.101 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.093+0000 7fec6f5f0700 1 -- 172.21.15.150:0/2552688185 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec68066ea0 con 0x7fec68100ff0 2023-11-27T15:14:02.101 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.094+0000 7fec5e7fc700 1 -- 172.21.15.150:0/2552688185 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fec600117d0 con 0x7fec68100ff0 2023-11-27T15:14:02.102 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.095+0000 7fec5e7fc700 1 --2- 172.21.15.150:0/2552688185 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fec540a1150 0x7fec540a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:02.102 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.095+0000 7fec5e7fc700 1 -- 172.21.15.150:0/2552688185 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5430+0+0 (secure 0 0 0) 0x7fec600ce820 con 0x7fec68100ff0 2023-11-27T15:14:02.102 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.095+0000 7fec6d38c700 1 --2- 172.21.15.150:0/2552688185 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fec540a1150 0x7fec540a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:02.102 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.096+0000 7fec6d38c700 1 --2- 172.21.15.150:0/2552688185 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fec540a1150 0x7fec540a3600 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fec64006010 tx=0x7fec6400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:02.103 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.098+0000 7fec5e7fc700 1 -- 172.21.15.150:0/2552688185 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fec60098b40 con 0x7fec68100ff0 2023-11-27T15:14:02.302 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:02.298+0000 7fec6f5f0700 1 -- 172.21.15.150:0/2552688185 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fec68069e90 con 0x7fec540a1150 2023-11-27T15:14:02.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:02 smithi150 ceph-mon[100559]: purged_snaps scrub starts 2023-11-27T15:14:02.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:02 smithi150 ceph-mon[100559]: purged_snaps scrub ok 2023-11-27T15:14:02.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:02 smithi150 ceph-mon[100559]: osd.7 [v2:172.21.15.102:6816/3249840064,v1:172.21.15.102:6817/3249840064] boot 2023-11-27T15:14:02.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:02 smithi150 ceph-mon[100559]: osdmap e50: 8 total, 8 up, 8 in 2023-11-27T15:14:02.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:02 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:14:02.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:02 smithi150 ceph-mon[100559]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 45 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:02.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:02 smithi150 ceph-mon[100559]: osdmap e51: 8 total, 8 up, 8 in 2023-11-27T15:14:02.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:02 smithi102 ceph-mon[100296]: purged_snaps scrub starts 2023-11-27T15:14:02.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:02 smithi102 ceph-mon[100296]: purged_snaps scrub ok 2023-11-27T15:14:02.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:02 smithi102 ceph-mon[100296]: osd.7 [v2:172.21.15.102:6816/3249840064,v1:172.21.15.102:6817/3249840064] boot 2023-11-27T15:14:02.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:02 smithi102 ceph-mon[100296]: osdmap e50: 8 total, 8 up, 8 in 2023-11-27T15:14:02.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:02 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:14:02.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:02 smithi102 ceph-mon[100296]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 45 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:02.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:02 smithi102 ceph-mon[100296]: osdmap e51: 8 total, 8 up, 8 in 2023-11-27T15:14:02.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:02 smithi052 ceph-mon[99056]: purged_snaps scrub starts 2023-11-27T15:14:02.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:02 smithi052 ceph-mon[99056]: purged_snaps scrub ok 2023-11-27T15:14:02.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:02 smithi052 ceph-mon[99056]: osd.7 [v2:172.21.15.102:6816/3249840064,v1:172.21.15.102:6817/3249840064] boot 2023-11-27T15:14:02.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:02 smithi052 ceph-mon[99056]: osdmap e50: 8 total, 8 up, 8 in 2023-11-27T15:14:02.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:02 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:14:02.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:02 smithi052 ceph-mon[99056]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 45 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:02.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:02 smithi052 ceph-mon[99056]: osdmap e51: 8 total, 8 up, 8 in 2023-11-27T15:14:03.488 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:03 smithi150 ceph-mon[100559]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:14:03.488 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:03 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:14:03.489 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:03 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:14:03.489 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:03 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:03.489 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:03 smithi150 ceph-mon[100559]: osdmap e52: 8 total, 8 up, 8 in 2023-11-27T15:14:03.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:03 smithi102 ceph-mon[100296]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:14:03.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:03 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:14:03.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:03 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:14:03.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:03 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:03.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:03 smithi102 ceph-mon[100296]: osdmap e52: 8 total, 8 up, 8 in 2023-11-27T15:14:03.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:03 smithi052 ceph-mon[99056]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:14:03.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:03 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:14:03.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:03 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:14:03.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:03 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:03.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:03 smithi052 ceph-mon[99056]: osdmap e52: 8 total, 8 up, 8 in 2023-11-27T15:14:05.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:04 smithi150 ceph-mon[100559]: osdmap e53: 8 total, 8 up, 8 in 2023-11-27T15:14:05.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:04 smithi150 ceph-mon[100559]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 45 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:05.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:04 smithi102 ceph-mon[100296]: osdmap e53: 8 total, 8 up, 8 in 2023-11-27T15:14:05.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:04 smithi102 ceph-mon[100296]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 45 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:05.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:04 smithi052 ceph-mon[99056]: osdmap e53: 8 total, 8 up, 8 in 2023-11-27T15:14:05.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:04 smithi052 ceph-mon[99056]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 45 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:06.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:05 smithi150 ceph-mon[100559]: osdmap e54: 8 total, 8 up, 8 in 2023-11-27T15:14:06.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:05 smithi150 ceph-mon[100559]: from='client.? 172.21.15.150:0/2719513878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7ca8007-7170-47bf-956c-3f00edf27e74"}]: dispatch 2023-11-27T15:14:06.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:05 smithi150 ceph-mon[100559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7ca8007-7170-47bf-956c-3f00edf27e74"}]: dispatch 2023-11-27T15:14:06.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:05 smithi150 ceph-mon[100559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c7ca8007-7170-47bf-956c-3f00edf27e74"}]': finished 2023-11-27T15:14:06.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:05 smithi150 ceph-mon[100559]: osdmap e55: 9 total, 8 up, 9 in 2023-11-27T15:14:06.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:05 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:14:06.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:05 smithi150 ceph-mon[100559]: from='client.? 172.21.15.150:0/1840596581' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:14:06.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:05 smithi102 ceph-mon[100296]: osdmap e54: 8 total, 8 up, 8 in 2023-11-27T15:14:06.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:05 smithi102 ceph-mon[100296]: from='client.? 172.21.15.150:0/2719513878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7ca8007-7170-47bf-956c-3f00edf27e74"}]: dispatch 2023-11-27T15:14:06.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:05 smithi102 ceph-mon[100296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7ca8007-7170-47bf-956c-3f00edf27e74"}]: dispatch 2023-11-27T15:14:06.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:05 smithi102 ceph-mon[100296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c7ca8007-7170-47bf-956c-3f00edf27e74"}]': finished 2023-11-27T15:14:06.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:05 smithi102 ceph-mon[100296]: osdmap e55: 9 total, 8 up, 9 in 2023-11-27T15:14:06.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:05 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:14:06.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:05 smithi102 ceph-mon[100296]: from='client.? 172.21.15.150:0/1840596581' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:14:06.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:05 smithi052 ceph-mon[99056]: osdmap e54: 8 total, 8 up, 8 in 2023-11-27T15:14:06.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:05 smithi052 ceph-mon[99056]: from='client.? 172.21.15.150:0/2719513878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7ca8007-7170-47bf-956c-3f00edf27e74"}]: dispatch 2023-11-27T15:14:06.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:05 smithi052 ceph-mon[99056]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7ca8007-7170-47bf-956c-3f00edf27e74"}]: dispatch 2023-11-27T15:14:06.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:05 smithi052 ceph-mon[99056]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c7ca8007-7170-47bf-956c-3f00edf27e74"}]': finished 2023-11-27T15:14:06.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:05 smithi052 ceph-mon[99056]: osdmap e55: 9 total, 8 up, 9 in 2023-11-27T15:14:06.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:05 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:14:06.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:05 smithi052 ceph-mon[99056]: from='client.? 172.21.15.150:0/1840596581' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:14:07.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:06 smithi150 ceph-mon[100559]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:07.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:06 smithi102 ceph-mon[100296]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:07.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:06 smithi052 ceph-mon[99056]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:08.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:07 smithi150 ceph-mon[100559]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:08.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:07 smithi102 ceph-mon[100296]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:08.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:07 smithi052 ceph-mon[99056]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:10.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:09 smithi150 ceph-mon[100559]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:10.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:09 smithi102 ceph-mon[100296]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:10.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:09 smithi052 ceph-mon[99056]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:11.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:11 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-11-27T15:14:11.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:11 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:11.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:11 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-11-27T15:14:11.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:11 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:11.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:11 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-11-27T15:14:11.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:11 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:12.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:12 smithi150 ceph-mon[100559]: Deploying daemon osd.8 on smithi150 2023-11-27T15:14:12.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:12 smithi150 ceph-mon[100559]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:12.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:12 smithi102 ceph-mon[100296]: Deploying daemon osd.8 on smithi150 2023-11-27T15:14:12.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:12 smithi102 ceph-mon[100296]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:12.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:12 smithi052 ceph-mon[99056]: Deploying daemon osd.8 on smithi150 2023-11-27T15:14:12.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:12 smithi052 ceph-mon[99056]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:14.921 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:14 smithi150 ceph-mon[100559]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:15.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:14 smithi102 ceph-mon[100296]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:15.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:14 smithi052 ceph-mon[99056]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:16.957 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:16 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:16.958 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:16 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:16.958 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:16 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:14:16.958 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:16 smithi150 ceph-mon[100559]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:17.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:16 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:17.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:16 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:17.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:16 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:14:17.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:16 smithi102 ceph-mon[100296]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:17.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:16 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:17.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:16 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:17.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:16 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:14:17.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:16 smithi052 ceph-mon[99056]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:17.858 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:17 smithi150 ceph-mon[100559]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:18.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:17 smithi102 ceph-mon[100296]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:18.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:17 smithi052 ceph-mon[99056]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:18.716 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:18.710+0000 7fec5e7fc700 1 -- 172.21.15.150:0/2552688185 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fec68069e90 con 0x7fec540a1150 2023-11-27T15:14:18.717 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 8 on host 'smithi150' 2023-11-27T15:14:18.718 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:18.713+0000 7fec6f5f0700 1 -- 172.21.15.150:0/2552688185 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fec540a1150 msgr2=0x7fec540a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:18.718 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:18.713+0000 7fec6f5f0700 1 --2- 172.21.15.150:0/2552688185 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fec540a1150 0x7fec540a3600 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fec64006010 tx=0x7fec6400b540 comp rx=0 tx=0).stop 2023-11-27T15:14:18.718 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:18.713+0000 7fec6f5f0700 1 -- 172.21.15.150:0/2552688185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec68100ff0 msgr2=0x7fec6819c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:18.718 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:18.713+0000 7fec6f5f0700 1 --2- 172.21.15.150:0/2552688185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec68100ff0 0x7fec6819c5e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fec6000c270 tx=0x7fec6000c580 comp rx=0 tx=0).stop 2023-11-27T15:14:18.719 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:18.713+0000 7fec6f5f0700 1 -- 172.21.15.150:0/2552688185 shutdown_connections 2023-11-27T15:14:18.719 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:18.713+0000 7fec6f5f0700 1 --2- 172.21.15.150:0/2552688185 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fec540a1150 0x7fec540a3600 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:18.719 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:18.713+0000 7fec6f5f0700 1 --2- 172.21.15.150:0/2552688185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec68100ff0 0x7fec6819c5e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:18.719 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:18.713+0000 7fec6f5f0700 1 --2- 172.21.15.150:0/2552688185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec680ff460 0x7fec68195020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:18.719 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:18.713+0000 7fec6f5f0700 1 --2- 172.21.15.150:0/2552688185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec68100660 0x7fec68195560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:18.719 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:18.713+0000 7fec6f5f0700 1 -- 172.21.15.150:0/2552688185 >> 172.21.15.150:0/2552688185 conn(0x7fec680faad0 msgr2=0x7fec680fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:18.719 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:18.713+0000 7fec6f5f0700 1 -- 172.21.15.150:0/2552688185 shutdown_connections 2023-11-27T15:14:18.720 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:14:18.713+0000 7fec6f5f0700 1 -- 172.21.15.150:0/2552688185 wait complete. 2023-11-27T15:14:19.179 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:18 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:19.179 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:18 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:19.179 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:18 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:19.179 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:18 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:19.180 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:18 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:14:19.180 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:18 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:19.180 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:18 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:14:19.180 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:18 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:14:19.180 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:18 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:19.180 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:18 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:19.180 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:18 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:14:19.182 DEBUG:teuthology.orchestra.run.smithi150:osd.8> sudo journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.8.service 2023-11-27T15:14:19.184 INFO:tasks.cephadm:Deploying osd.9 on smithi052 with /dev/vg_nvme/lv_1... 2023-11-27T15:14:19.184 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-11-27T15:14:19.216 INFO:journalctl@ceph.osd.8.smithi150.stdout:-- Logs begin at Mon 2023-11-27 14:53:12 UTC. -- 2023-11-27T15:14:19.218 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:18 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:19.218 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:18 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:19.218 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:18 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:19.218 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:18 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:19.218 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:18 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:14:19.218 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:18 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:19.218 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:18 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:14:19.219 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:18 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:14:19.219 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:18 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:19.219 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:18 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:19.219 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:18 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:14:19.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:18 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:19.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:18 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:19.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:18 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:19.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:18 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:19.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:18 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:14:19.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:18 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:19.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:18 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:14:19.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:18 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:14:19.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:18 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:19.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:18 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:19.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:18 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:14:19.855 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:14:19.919 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:14:19 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:14:19.700+0000 7faf5cb17640 -1 osd.8 0 log_to_monitors true 2023-11-27T15:14:20.237 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:19 smithi052 ceph-mon[99056]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:20.237 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:19 smithi052 ceph-mon[99056]: from='osd.8 [v2:172.21.15.150:6816/3672583574,v1:172.21.15.150:6817/3672583574]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-27T15:14:20.238 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:19 smithi052 ceph-mon[99056]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-27T15:14:20.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:19 smithi150 ceph-mon[100559]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:20.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:19 smithi150 ceph-mon[100559]: from='osd.8 [v2:172.21.15.150:6816/3672583574,v1:172.21.15.150:6817/3672583574]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-27T15:14:20.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:19 smithi150 ceph-mon[100559]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-27T15:14:20.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:19 smithi102 ceph-mon[100296]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:20.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:19 smithi102 ceph-mon[100296]: from='osd.8 [v2:172.21.15.150:6816/3672583574,v1:172.21.15.150:6817/3672583574]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-27T15:14:20.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:19 smithi102 ceph-mon[100296]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-27T15:14:21.201 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:20 smithi150 ceph-mon[100559]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-11-27T15:14:21.202 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:20 smithi150 ceph-mon[100559]: osdmap e56: 9 total, 8 up, 9 in 2023-11-27T15:14:21.202 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:20 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:14:21.202 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:20 smithi150 ceph-mon[100559]: from='osd.8 [v2:172.21.15.150:6816/3672583574,v1:172.21.15.150:6817/3672583574]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2023-11-27T15:14:21.202 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:20 smithi150 ceph-mon[100559]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2023-11-27T15:14:21.203 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:14:20 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:14:20.973+0000 7faf5218e700 -1 osd.8 0 waiting for initial osdmap 2023-11-27T15:14:21.203 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:14:20 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:14:20.987+0000 7faf49f7b700 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:14:21.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:20 smithi102 ceph-mon[100296]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-11-27T15:14:21.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:20 smithi102 ceph-mon[100296]: osdmap e56: 9 total, 8 up, 9 in 2023-11-27T15:14:21.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:20 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:14:21.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:20 smithi102 ceph-mon[100296]: from='osd.8 [v2:172.21.15.150:6816/3672583574,v1:172.21.15.150:6817/3672583574]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2023-11-27T15:14:21.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:20 smithi102 ceph-mon[100296]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2023-11-27T15:14:21.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:20 smithi052 ceph-mon[99056]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-11-27T15:14:21.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:20 smithi052 ceph-mon[99056]: osdmap e56: 9 total, 8 up, 9 in 2023-11-27T15:14:21.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:20 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:14:21.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:20 smithi052 ceph-mon[99056]: from='osd.8 [v2:172.21.15.150:6816/3672583574,v1:172.21.15.150:6817/3672583574]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2023-11-27T15:14:21.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:20 smithi052 ceph-mon[99056]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2023-11-27T15:14:21.376 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-11-27T15:14:21.376 INFO:teuthology.orchestra.run.smithi052.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-11-27T15:14:21.376 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10+0 records in 2023-11-27T15:14:21.376 INFO:teuthology.orchestra.run.smithi052.stderr:10+0 records out 2023-11-27T15:14:21.377 INFO:teuthology.orchestra.run.smithi052.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00843801 s, 1.2 GB/s 2023-11-27T15:14:21.377 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping successful for: 2023-11-27T15:14:21.817 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi052:vg_nvme/lv_1 2023-11-27T15:14:22.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:21 smithi052 ceph-mon[99056]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2023-11-27T15:14:22.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:21 smithi052 ceph-mon[99056]: osdmap e57: 9 total, 8 up, 9 in 2023-11-27T15:14:22.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:21 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:14:22.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:21 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:14:22.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:21 smithi052 ceph-mon[99056]: Detected new or changed devices on smithi150 2023-11-27T15:14:22.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:21 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:22.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:21 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:22.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:21 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:22.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:21 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:22.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:21 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:22.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:21 smithi052 ceph-mon[99056]: Adjusting osd_memory_target on smithi150 to 5722M 2023-11-27T15:14:22.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:21 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:22.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:21 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:22.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:21 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:22.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:21 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:14:22.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:21 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:22.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:21 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:14:22.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:21 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:14:22.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:21 smithi052 ceph-mon[99056]: pgmap v176: 1 pgs: 1 peering; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:22.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:21 smithi150 ceph-mon[100559]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2023-11-27T15:14:22.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:21 smithi150 ceph-mon[100559]: osdmap e57: 9 total, 8 up, 9 in 2023-11-27T15:14:22.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:21 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:14:22.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:21 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:14:22.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:21 smithi150 ceph-mon[100559]: Detected new or changed devices on smithi150 2023-11-27T15:14:22.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:21 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:22.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:21 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:22.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:21 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:22.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:21 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:22.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:21 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:22.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:21 smithi150 ceph-mon[100559]: Adjusting osd_memory_target on smithi150 to 5722M 2023-11-27T15:14:22.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:21 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:22.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:21 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:22.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:21 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:22.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:21 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:14:22.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:21 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:22.281 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:21 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:14:22.281 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:21 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:14:22.281 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:21 smithi150 ceph-mon[100559]: pgmap v176: 1 pgs: 1 peering; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:22.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:21 smithi102 ceph-mon[100296]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2023-11-27T15:14:22.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:21 smithi102 ceph-mon[100296]: osdmap e57: 9 total, 8 up, 9 in 2023-11-27T15:14:22.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:21 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:14:22.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:21 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:14:22.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:21 smithi102 ceph-mon[100296]: Detected new or changed devices on smithi150 2023-11-27T15:14:22.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:21 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:22.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:21 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:22.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:21 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:22.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:21 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:22.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:21 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:22.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:21 smithi102 ceph-mon[100296]: Adjusting osd_memory_target on smithi150 to 5722M 2023-11-27T15:14:22.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:21 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:22.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:21 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:22.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:21 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:22.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:21 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:14:22.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:21 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:22.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:21 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:14:22.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:21 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:14:22.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:21 smithi102 ceph-mon[100296]: pgmap v176: 1 pgs: 1 peering; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:14:22.515 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:14:23.261 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:22 smithi052 ceph-mon[99056]: purged_snaps scrub starts 2023-11-27T15:14:23.261 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:22 smithi052 ceph-mon[99056]: purged_snaps scrub ok 2023-11-27T15:14:23.261 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:22 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:14:23.261 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:22 smithi052 ceph-mon[99056]: osd.8 [v2:172.21.15.150:6816/3672583574,v1:172.21.15.150:6817/3672583574] boot 2023-11-27T15:14:23.261 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:22 smithi052 ceph-mon[99056]: osdmap e58: 9 total, 9 up, 9 in 2023-11-27T15:14:23.261 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:22 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:14:23.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:22 smithi150 ceph-mon[100559]: purged_snaps scrub starts 2023-11-27T15:14:23.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:22 smithi150 ceph-mon[100559]: purged_snaps scrub ok 2023-11-27T15:14:23.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:22 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:14:23.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:22 smithi150 ceph-mon[100559]: osd.8 [v2:172.21.15.150:6816/3672583574,v1:172.21.15.150:6817/3672583574] boot 2023-11-27T15:14:23.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:22 smithi150 ceph-mon[100559]: osdmap e58: 9 total, 9 up, 9 in 2023-11-27T15:14:23.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:22 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:14:23.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:22 smithi102 ceph-mon[100296]: purged_snaps scrub starts 2023-11-27T15:14:23.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:22 smithi102 ceph-mon[100296]: purged_snaps scrub ok 2023-11-27T15:14:23.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:22 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:14:23.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:22 smithi102 ceph-mon[100296]: osd.8 [v2:172.21.15.150:6816/3672583574,v1:172.21.15.150:6817/3672583574] boot 2023-11-27T15:14:23.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:22 smithi102 ceph-mon[100296]: osdmap e58: 9 total, 9 up, 9 in 2023-11-27T15:14:23.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:22 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:14:23.732 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.724+0000 7fe68cf28700 1 -- 172.21.15.52:0/2440402654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe688100d70 msgr2=0x7fe688107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:23.732 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.724+0000 7fe68cf28700 1 --2- 172.21.15.52:0/2440402654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe688100d70 0x7fe688107630 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe68000b0a0 tx=0x7fe68000b3b0 comp rx=0 tx=0).stop 2023-11-27T15:14:23.732 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.724+0000 7fe68cf28700 1 -- 172.21.15.52:0/2440402654 shutdown_connections 2023-11-27T15:14:23.732 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.724+0000 7fe68cf28700 1 --2- 172.21.15.52:0/2440402654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6880ff1e0 0x7fe6880ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:23.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.724+0000 7fe68cf28700 1 --2- 172.21.15.52:0/2440402654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe688100d70 0x7fe688107630 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:23.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.724+0000 7fe68cf28700 1 --2- 172.21.15.52:0/2440402654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6881003e0 0x7fe688100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:23.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.724+0000 7fe68cf28700 1 -- 172.21.15.52:0/2440402654 >> 172.21.15.52:0/2440402654 conn(0x7fe6880fa7f0 msgr2=0x7fe6880fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:23.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.724+0000 7fe68cf28700 1 -- 172.21.15.52:0/2440402654 shutdown_connections 2023-11-27T15:14:23.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.724+0000 7fe68cf28700 1 -- 172.21.15.52:0/2440402654 wait complete. 2023-11-27T15:14:23.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.725+0000 7fe68cf28700 1 Processor -- start 2023-11-27T15:14:23.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.725+0000 7fe68cf28700 1 -- start start 2023-11-27T15:14:23.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.725+0000 7fe68cf28700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6880ff1e0 0x7fe6880699a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:23.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.725+0000 7fe68cf28700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6881003e0 0x7fe68819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:23.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.725+0000 7fe68cf28700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe688100d70 0x7fe68819d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:23.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.725+0000 7fe68cf28700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68819db80 con 0x7fe6881003e0 2023-11-27T15:14:23.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.725+0000 7fe68cf28700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68819dcf0 con 0x7fe688100d70 2023-11-27T15:14:23.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.725+0000 7fe68cf28700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68819deb0 con 0x7fe6880ff1e0 2023-11-27T15:14:23.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.725+0000 7fe685d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6881003e0 0x7fe68819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:23.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.725+0000 7fe685d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6881003e0 0x7fe68819ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49476/0 (socket says 172.21.15.52:49476) 2023-11-27T15:14:23.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.725+0000 7fe685d9b700 1 -- 172.21.15.52:0/274131146 learned_addr learned my addr 172.21.15.52:0/274131146 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:14:23.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.725+0000 7fe686d9d700 1 --2- 172.21.15.52:0/274131146 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe688100d70 0x7fe68819d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:23.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.725+0000 7fe68659c700 1 --2- 172.21.15.52:0/274131146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6880ff1e0 0x7fe6880699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:23.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.725+0000 7fe685d9b700 1 -- 172.21.15.52:0/274131146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6880ff1e0 msgr2=0x7fe6880699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:23.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.725+0000 7fe685d9b700 1 --2- 172.21.15.52:0/274131146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6880ff1e0 0x7fe6880699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:23.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.725+0000 7fe685d9b700 1 -- 172.21.15.52:0/274131146 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe688100d70 msgr2=0x7fe68819d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:23.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.725+0000 7fe685d9b700 1 --2- 172.21.15.52:0/274131146 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe688100d70 0x7fe68819d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:23.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.725+0000 7fe685d9b700 1 -- 172.21.15.52:0/274131146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe680009d00 con 0x7fe6881003e0 2023-11-27T15:14:23.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.726+0000 7fe685d9b700 1 --2- 172.21.15.52:0/274131146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6881003e0 0x7fe68819ada0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe67c00d950 tx=0x7fe67c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:23.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.726+0000 7fe6777fe700 1 -- 172.21.15.52:0/274131146 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c00f890 con 0x7fe6881003e0 2023-11-27T15:14:23.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.726+0000 7fe6777fe700 1 -- 172.21.15.52:0/274131146 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe67c010460 con 0x7fe6881003e0 2023-11-27T15:14:23.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.726+0000 7fe68cf28700 1 -- 172.21.15.52:0/274131146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe68810ab30 con 0x7fe6881003e0 2023-11-27T15:14:23.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.726+0000 7fe6777fe700 1 -- 172.21.15.52:0/274131146 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c00e610 con 0x7fe6881003e0 2023-11-27T15:14:23.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.726+0000 7fe68cf28700 1 -- 172.21.15.52:0/274131146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe68810a9a0 con 0x7fe6881003e0 2023-11-27T15:14:23.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.727+0000 7fe68cf28700 1 -- 172.21.15.52:0/274131146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6881066c0 con 0x7fe6881003e0 2023-11-27T15:14:23.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.728+0000 7fe6777fe700 1 -- 172.21.15.52:0/274131146 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe67c0108d0 con 0x7fe6881003e0 2023-11-27T15:14:23.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.728+0000 7fe6777fe700 1 --2- 172.21.15.52:0/274131146 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fe6700a11d0 0x7fe6700a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:23.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.728+0000 7fe68659c700 1 --2- 172.21.15.52:0/274131146 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fe6700a11d0 0x7fe6700a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:23.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.729+0000 7fe6777fe700 1 -- 172.21.15.52:0/274131146 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5858+0+0 (secure 0 0 0) 0x7fe67c0cbd60 con 0x7fe6881003e0 2023-11-27T15:14:23.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.729+0000 7fe68659c700 1 --2- 172.21.15.52:0/274131146 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fe6700a11d0 0x7fe6700a3680 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe688100240 tx=0x7fe67800a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:23.738 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.733+0000 7fe6777fe700 1 -- 172.21.15.52:0/274131146 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe67c095da0 con 0x7fe6881003e0 2023-11-27T15:14:23.945 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:23.939+0000 7fe68cf28700 1 -- 172.21.15.52:0/274131146 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe688061cb0 con 0x7fe6700a11d0 2023-11-27T15:14:24.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:23 smithi150 ceph-mon[100559]: osdmap e59: 9 total, 9 up, 9 in 2023-11-27T15:14:24.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:23 smithi150 ceph-mon[100559]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:14:24.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:23 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:14:24.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:23 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:14:24.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:23 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:24.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:23 smithi102 ceph-mon[100296]: osdmap e59: 9 total, 9 up, 9 in 2023-11-27T15:14:24.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:23 smithi102 ceph-mon[100296]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:14:24.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:23 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:14:24.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:23 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:14:24.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:23 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:24.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:23 smithi052 ceph-mon[99056]: osdmap e59: 9 total, 9 up, 9 in 2023-11-27T15:14:24.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:23 smithi052 ceph-mon[99056]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:14:24.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:23 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:14:24.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:23 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:14:24.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:23 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:25.195 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:24 smithi052 ceph-mon[99056]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:14:25.195 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:24 smithi052 ceph-mon[99056]: osdmap e60: 9 total, 9 up, 9 in 2023-11-27T15:14:25.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:24 smithi150 ceph-mon[100559]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:14:25.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:24 smithi150 ceph-mon[100559]: osdmap e60: 9 total, 9 up, 9 in 2023-11-27T15:14:25.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:24 smithi102 ceph-mon[100296]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:14:25.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:24 smithi102 ceph-mon[100296]: osdmap e60: 9 total, 9 up, 9 in 2023-11-27T15:14:26.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:25 smithi052 ceph-mon[99056]: osdmap e61: 9 total, 9 up, 9 in 2023-11-27T15:14:26.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:25 smithi052 ceph-mon[99056]: pgmap v182: 1 pgs: 1 peering; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:14:26.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:25 smithi150 ceph-mon[100559]: osdmap e61: 9 total, 9 up, 9 in 2023-11-27T15:14:26.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:25 smithi150 ceph-mon[100559]: pgmap v182: 1 pgs: 1 peering; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:14:26.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:25 smithi102 ceph-mon[100296]: osdmap e61: 9 total, 9 up, 9 in 2023-11-27T15:14:26.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:25 smithi102 ceph-mon[100296]: pgmap v182: 1 pgs: 1 peering; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:14:27.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:26 smithi150 ceph-mon[100559]: osdmap e62: 9 total, 9 up, 9 in 2023-11-27T15:14:27.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:26 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2838956041' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e252c2c-01a4-48ba-832d-067ed940fefb"}]: dispatch 2023-11-27T15:14:27.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:26 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2838956041' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e252c2c-01a4-48ba-832d-067ed940fefb"}]': finished 2023-11-27T15:14:27.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:26 smithi150 ceph-mon[100559]: osdmap e63: 10 total, 9 up, 10 in 2023-11-27T15:14:27.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:26 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:14:27.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:26 smithi102 ceph-mon[100296]: osdmap e62: 9 total, 9 up, 9 in 2023-11-27T15:14:27.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:26 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2838956041' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e252c2c-01a4-48ba-832d-067ed940fefb"}]: dispatch 2023-11-27T15:14:27.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:26 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2838956041' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e252c2c-01a4-48ba-832d-067ed940fefb"}]': finished 2023-11-27T15:14:27.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:26 smithi102 ceph-mon[100296]: osdmap e63: 10 total, 9 up, 10 in 2023-11-27T15:14:27.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:26 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:14:27.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:26 smithi052 ceph-mon[99056]: osdmap e62: 9 total, 9 up, 9 in 2023-11-27T15:14:27.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:26 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2838956041' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e252c2c-01a4-48ba-832d-067ed940fefb"}]: dispatch 2023-11-27T15:14:27.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:26 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2838956041' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e252c2c-01a4-48ba-832d-067ed940fefb"}]': finished 2023-11-27T15:14:27.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:26 smithi052 ceph-mon[99056]: osdmap e63: 10 total, 9 up, 10 in 2023-11-27T15:14:27.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:26 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:14:28.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:27 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1460408260' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:14:28.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:27 smithi150 ceph-mon[100559]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-27T15:14:28.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:27 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1460408260' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:14:28.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:27 smithi102 ceph-mon[100296]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-27T15:14:28.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:27 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1460408260' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:14:28.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:27 smithi052 ceph-mon[99056]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-27T15:14:30.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:29 smithi150 ceph-mon[100559]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 101 KiB/s, 0 objects/s recovering 2023-11-27T15:14:30.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:29 smithi102 ceph-mon[100296]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 101 KiB/s, 0 objects/s recovering 2023-11-27T15:14:30.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:29 smithi052 ceph-mon[99056]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 101 KiB/s, 0 objects/s recovering 2023-11-27T15:14:32.018 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:31 smithi052 ceph-mon[99056]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2023-11-27T15:14:32.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:31 smithi150 ceph-mon[100559]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2023-11-27T15:14:32.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:31 smithi102 ceph-mon[100296]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2023-11-27T15:14:33.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:33 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-11-27T15:14:33.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:33 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:33.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:33 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-11-27T15:14:33.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:33 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:33.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:33 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-11-27T15:14:33.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:33 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:34.742 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:34 smithi052 ceph-mon[99056]: Deploying daemon osd.9 on smithi052 2023-11-27T15:14:34.742 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:34 smithi052 ceph-mon[99056]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-27T15:14:35.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:34 smithi150 ceph-mon[100559]: Deploying daemon osd.9 on smithi052 2023-11-27T15:14:35.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:34 smithi150 ceph-mon[100559]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-27T15:14:35.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:34 smithi102 ceph-mon[100296]: Deploying daemon osd.9 on smithi052 2023-11-27T15:14:35.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:34 smithi102 ceph-mon[100296]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-27T15:14:36.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:35 smithi150 ceph-mon[100559]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2023-11-27T15:14:36.056 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:35 smithi052 ceph-mon[99056]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2023-11-27T15:14:36.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:35 smithi102 ceph-mon[100296]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2023-11-27T15:14:38.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:37 smithi150 ceph-mon[100559]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:14:38.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:37 smithi102 ceph-mon[100296]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:14:38.082 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:37 smithi052 ceph-mon[99056]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:14:39.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:38 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:14:39.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:38 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:39.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:38 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:39.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:38 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:14:39.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:38 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:39.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:38 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:39.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:38 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:14:39.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:38 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:39.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:38 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:39.932 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:39 smithi052 ceph-mon[99056]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:14:40.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:39 smithi150 ceph-mon[100559]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:14:40.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:39 smithi102 ceph-mon[100296]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:14:41.551 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:41.544+0000 7fe6777fe700 1 -- 172.21.15.52:0/274131146 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe688061cb0 con 0x7fe6700a11d0 2023-11-27T15:14:41.552 INFO:teuthology.orchestra.run.smithi052.stdout:Created osd(s) 9 on host 'smithi052' 2023-11-27T15:14:41.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:41.547+0000 7fe68cf28700 1 -- 172.21.15.52:0/274131146 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fe6700a11d0 msgr2=0x7fe6700a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:41.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:41.547+0000 7fe68cf28700 1 --2- 172.21.15.52:0/274131146 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fe6700a11d0 0x7fe6700a3680 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe688100240 tx=0x7fe67800a3b0 comp rx=0 tx=0).stop 2023-11-27T15:14:41.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:41.547+0000 7fe68cf28700 1 -- 172.21.15.52:0/274131146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6881003e0 msgr2=0x7fe68819ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:41.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:41.547+0000 7fe68cf28700 1 --2- 172.21.15.52:0/274131146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6881003e0 0x7fe68819ada0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe67c00d950 tx=0x7fe67c00dc60 comp rx=0 tx=0).stop 2023-11-27T15:14:41.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:41.547+0000 7fe68cf28700 1 -- 172.21.15.52:0/274131146 shutdown_connections 2023-11-27T15:14:41.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:41.547+0000 7fe68cf28700 1 --2- 172.21.15.52:0/274131146 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fe6700a11d0 0x7fe6700a3680 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:41.554 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:41.547+0000 7fe68cf28700 1 --2- 172.21.15.52:0/274131146 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe688100d70 0x7fe68819d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:41.554 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:41.547+0000 7fe68cf28700 1 --2- 172.21.15.52:0/274131146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6880ff1e0 0x7fe6880699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:41.554 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:41.547+0000 7fe68cf28700 1 --2- 172.21.15.52:0/274131146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6881003e0 0x7fe68819ada0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:41.554 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:41.547+0000 7fe68cf28700 1 -- 172.21.15.52:0/274131146 >> 172.21.15.52:0/274131146 conn(0x7fe6880fa7f0 msgr2=0x7fe688103800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:41.554 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:41.547+0000 7fe68cf28700 1 -- 172.21.15.52:0/274131146 shutdown_connections 2023-11-27T15:14:41.554 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:14:41.547+0000 7fe68cf28700 1 -- 172.21.15.52:0/274131146 wait complete. 2023-11-27T15:14:42.105 DEBUG:teuthology.orchestra.run.smithi052:osd.9> sudo journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.9.service 2023-11-27T15:14:42.107 INFO:tasks.cephadm:Deploying osd.10 on smithi102 with /dev/vg_nvme/lv_1... 2023-11-27T15:14:42.107 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-11-27T15:14:42.138 INFO:journalctl@ceph.osd.9.smithi052.stdout:-- Logs begin at Mon 2023-11-27 14:52:43 UTC. -- 2023-11-27T15:14:42.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:42 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:42.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:42 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:42.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:42 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:42.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:42 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:42.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:42 smithi102 ceph-mon[100296]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:14:42.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:42 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:42.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:42 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:42.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:42 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:42.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:42 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:42.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:42 smithi052 ceph-mon[99056]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:14:42.617 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:14:42 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:14:42.366+0000 7fb68f821640 -1 osd.9 0 log_to_monitors true 2023-11-27T15:14:42.767 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:14:42.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:42 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:42.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:42 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:42.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:42 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:42.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:42 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:42.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:42 smithi150 ceph-mon[100559]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:14:43.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:43 smithi102 ceph-mon[100296]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-11-27T15:14:43.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:43 smithi102 ceph-mon[100296]: from='osd.9 [v2:172.21.15.52:6826/4287934320,v1:172.21.15.52:6827/4287934320]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-11-27T15:14:43.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:43 smithi052 ceph-mon[99056]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-11-27T15:14:43.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:43 smithi052 ceph-mon[99056]: from='osd.9 [v2:172.21.15.52:6826/4287934320,v1:172.21.15.52:6827/4287934320]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-11-27T15:14:43.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:43 smithi150 ceph-mon[100559]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-11-27T15:14:43.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:43 smithi150 ceph-mon[100559]: from='osd.9 [v2:172.21.15.52:6826/4287934320,v1:172.21.15.52:6827/4287934320]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-11-27T15:14:44.152 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-11-27T15:14:44.152 INFO:teuthology.orchestra.run.smithi102.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-11-27T15:14:44.152 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10+0 records in 2023-11-27T15:14:44.152 INFO:teuthology.orchestra.run.smithi102.stderr:10+0 records out 2023-11-27T15:14:44.152 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110819 s, 946 MB/s 2023-11-27T15:14:44.152 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping successful for: 2023-11-27T15:14:44.509 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:44 smithi102 ceph-mon[100296]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-11-27T15:14:44.509 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:44 smithi102 ceph-mon[100296]: osdmap e64: 10 total, 9 up, 10 in 2023-11-27T15:14:44.509 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:44 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:14:44.509 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:44 smithi102 ceph-mon[100296]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2023-11-27T15:14:44.509 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:44 smithi102 ceph-mon[100296]: from='osd.9 [v2:172.21.15.52:6826/4287934320,v1:172.21.15.52:6827/4287934320]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2023-11-27T15:14:44.509 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:44 smithi102 ceph-mon[100296]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:14:44.524 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi102:vg_nvme/lv_1 2023-11-27T15:14:44.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:44 smithi052 ceph-mon[99056]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-11-27T15:14:44.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:44 smithi052 ceph-mon[99056]: osdmap e64: 10 total, 9 up, 10 in 2023-11-27T15:14:44.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:44 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:14:44.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:44 smithi052 ceph-mon[99056]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2023-11-27T15:14:44.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:44 smithi052 ceph-mon[99056]: from='osd.9 [v2:172.21.15.52:6826/4287934320,v1:172.21.15.52:6827/4287934320]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2023-11-27T15:14:44.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:44 smithi052 ceph-mon[99056]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:14:44.618 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:14:44 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:14:44.357+0000 7fb684e98700 -1 osd.9 0 waiting for initial osdmap 2023-11-27T15:14:44.618 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:14:44 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:14:44.370+0000 7fb67e488700 -1 osd.9 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:14:44.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:44 smithi150 ceph-mon[100559]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-11-27T15:14:44.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:44 smithi150 ceph-mon[100559]: osdmap e64: 10 total, 9 up, 10 in 2023-11-27T15:14:44.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:44 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:14:44.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:44 smithi150 ceph-mon[100559]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2023-11-27T15:14:44.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:44 smithi150 ceph-mon[100559]: from='osd.9 [v2:172.21.15.52:6826/4287934320,v1:172.21.15.52:6827/4287934320]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2023-11-27T15:14:44.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:44 smithi150 ceph-mon[100559]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:14:45.167 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:14:45.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:45 smithi052 ceph-mon[99056]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2023-11-27T15:14:45.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:45 smithi052 ceph-mon[99056]: osdmap e65: 10 total, 9 up, 10 in 2023-11-27T15:14:45.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:45 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:14:45.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:45 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:14:45.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:45 smithi052 ceph-mon[99056]: Detected new or changed devices on smithi052 2023-11-27T15:14:45.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:45 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:45.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:45 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:45.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:45 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:45.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:45 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:45.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:45 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:45.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:45 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:45.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:45 smithi052 ceph-mon[99056]: Adjusting osd_memory_target on smithi052 to 4293M 2023-11-27T15:14:45.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:45 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:45.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:45 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:45.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:45 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:45.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:45 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:14:45.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:45 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:45.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:45 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:14:45.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:45 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:14:45.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:45 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:14:45.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:45 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:45.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:45 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:45.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:45 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:45.761 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:45 smithi102 ceph-mon[100296]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2023-11-27T15:14:45.761 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:45 smithi102 ceph-mon[100296]: osdmap e65: 10 total, 9 up, 10 in 2023-11-27T15:14:45.761 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:45 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:14:45.761 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:45 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:14:45.762 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:45 smithi102 ceph-mon[100296]: Detected new or changed devices on smithi052 2023-11-27T15:14:45.762 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:45 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:45.762 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:45 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:45.762 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:45 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:45.762 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:45 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:45.762 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:45 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:45.762 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:45 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:45.762 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:45 smithi102 ceph-mon[100296]: Adjusting osd_memory_target on smithi052 to 4293M 2023-11-27T15:14:45.762 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:45 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:45.762 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:45 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:45.762 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:45 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:45.762 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:45 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:14:45.762 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:45 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:45.763 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:45 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:14:45.763 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:45 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:14:45.763 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:45 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:14:45.763 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:45 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:45.763 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:45 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:45.763 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:45 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:45.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:45 smithi150 ceph-mon[100559]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2023-11-27T15:14:45.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:45 smithi150 ceph-mon[100559]: osdmap e65: 10 total, 9 up, 10 in 2023-11-27T15:14:45.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:45 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:14:45.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:45 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:14:45.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:45 smithi150 ceph-mon[100559]: Detected new or changed devices on smithi052 2023-11-27T15:14:45.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:45 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:45.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:45 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:45.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:45 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:45.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:45 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:45.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:45 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:45.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:45 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:45.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:45 smithi150 ceph-mon[100559]: Adjusting osd_memory_target on smithi052 to 4293M 2023-11-27T15:14:45.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:45 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:45.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:45 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:45.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:45 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:45.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:45 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:14:45.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:45 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:45.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:45 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:14:45.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:45 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:14:45.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:45 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:14:45.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:45 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:45.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:45 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:45.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:45 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:14:46.486 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.479+0000 7f7647ff0700 1 -- 172.21.15.102:0/3764463526 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76400ff1e0 msgr2=0x7f76400ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:46.486 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.479+0000 7f7647ff0700 1 --2- 172.21.15.102:0/3764463526 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76400ff1e0 0x7f76400ff5f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7630009990 tx=0x7f7630009ca0 comp rx=0 tx=0).stop 2023-11-27T15:14:46.486 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.479+0000 7f7647ff0700 1 -- 172.21.15.102:0/3764463526 shutdown_connections 2023-11-27T15:14:46.486 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.479+0000 7f7647ff0700 1 --2- 172.21.15.102:0/3764463526 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76400ff1e0 0x7f76400ff5f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:46.486 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.479+0000 7f7647ff0700 1 --2- 172.21.15.102:0/3764463526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7640100d70 0x7f7640107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:46.486 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.479+0000 7f7647ff0700 1 --2- 172.21.15.102:0/3764463526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76401003e0 0x7f7640100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:46.486 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.479+0000 7f7647ff0700 1 -- 172.21.15.102:0/3764463526 >> 172.21.15.102:0/3764463526 conn(0x7f76400fa7f0 msgr2=0x7f76400fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:46.487 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.480+0000 7f7647ff0700 1 -- 172.21.15.102:0/3764463526 shutdown_connections 2023-11-27T15:14:46.487 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.480+0000 7f7647ff0700 1 -- 172.21.15.102:0/3764463526 wait complete. 2023-11-27T15:14:46.487 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.480+0000 7f7647ff0700 1 Processor -- start 2023-11-27T15:14:46.487 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.480+0000 7f7647ff0700 1 -- start start 2023-11-27T15:14:46.487 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.480+0000 7f7647ff0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76400ff1e0 0x7f76401950c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:46.487 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.480+0000 7f7647ff0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76401003e0 0x7f7640195600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:46.487 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.480+0000 7f7647ff0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7640100d70 0x7f764019c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:46.487 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.480+0000 7f7647ff0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764019cc50 con 0x7f7640100d70 2023-11-27T15:14:46.487 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.480+0000 7f7647ff0700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764019cd90 con 0x7f76400ff1e0 2023-11-27T15:14:46.487 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.480+0000 7f7647ff0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764019cf20 con 0x7f76401003e0 2023-11-27T15:14:46.488 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.480+0000 7f7645d8c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76400ff1e0 0x7f76401950c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:46.488 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.480+0000 7f7645d8c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76400ff1e0 0x7f76401950c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:42044/0 (socket says 172.21.15.102:42044) 2023-11-27T15:14:46.488 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.480+0000 7f7645d8c700 1 -- 172.21.15.102:0/928270329 learned_addr learned my addr 172.21.15.102:0/928270329 (peer_addr_for_me v2:172.21.15.102:0/0) 2023-11-27T15:14:46.488 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.481+0000 7f764558b700 1 --2- 172.21.15.102:0/928270329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76401003e0 0x7f7640195600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:46.488 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.481+0000 7f7645d8c700 1 -- 172.21.15.102:0/928270329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76401003e0 msgr2=0x7f7640195600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:46.488 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.481+0000 7f7645d8c700 1 --2- 172.21.15.102:0/928270329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76401003e0 0x7f7640195600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:46.488 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.481+0000 7f7645d8c700 1 -- 172.21.15.102:0/928270329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7640100d70 msgr2=0x7f764019c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:46.488 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.481+0000 7f7645d8c700 1 --2- 172.21.15.102:0/928270329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7640100d70 0x7f764019c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:46.488 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.481+0000 7f7645d8c700 1 -- 172.21.15.102:0/928270329 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76300095f0 con 0x7f76400ff1e0 2023-11-27T15:14:46.488 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.481+0000 7f7645d8c700 1 --2- 172.21.15.102:0/928270329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76400ff1e0 0x7f76401950c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f763001a870 tx=0x7f763001a8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:46.489 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.481+0000 7f7636ffd700 1 -- 172.21.15.102:0/928270329 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763001f070 con 0x7f76400ff1e0 2023-11-27T15:14:46.489 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.481+0000 7f7636ffd700 1 -- 172.21.15.102:0/928270329 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7630004cf0 con 0x7f76400ff1e0 2023-11-27T15:14:46.489 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.481+0000 7f7647ff0700 1 -- 172.21.15.102:0/928270329 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764010ac80 con 0x7f76400ff1e0 2023-11-27T15:14:46.489 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.481+0000 7f7636ffd700 1 -- 172.21.15.102:0/928270329 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7630005520 con 0x7f76400ff1e0 2023-11-27T15:14:46.489 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.481+0000 7f7647ff0700 1 -- 172.21.15.102:0/928270329 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764010aaf0 con 0x7f76400ff1e0 2023-11-27T15:14:46.489 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.482+0000 7f7647ff0700 1 -- 172.21.15.102:0/928270329 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7640069170 con 0x7f76400ff1e0 2023-11-27T15:14:46.490 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.483+0000 7f7636ffd700 1 -- 172.21.15.102:0/928270329 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7630023070 con 0x7f76400ff1e0 2023-11-27T15:14:46.490 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.483+0000 7f7636ffd700 1 --2- 172.21.15.102:0/928270329 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f762c0a11d0 0x7f762c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:46.490 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.484+0000 7f7636ffd700 1 -- 172.21.15.102:0/928270329 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f76300cf2b0 con 0x7f76400ff1e0 2023-11-27T15:14:46.490 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.484+0000 7f764558b700 1 --2- 172.21.15.102:0/928270329 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f762c0a11d0 0x7f762c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:46.490 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.484+0000 7f764558b700 1 --2- 172.21.15.102:0/928270329 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f762c0a11d0 0x7f762c0a3680 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f763c009a10 tx=0x7f763c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:46.492 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.488+0000 7f7636ffd700 1 -- 172.21.15.102:0/928270329 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f763005e730 con 0x7f76400ff1e0 2023-11-27T15:14:46.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:46 smithi102 ceph-mon[100296]: purged_snaps scrub starts 2023-11-27T15:14:46.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:46 smithi102 ceph-mon[100296]: purged_snaps scrub ok 2023-11-27T15:14:46.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:46 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:14:46.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:46 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:14:46.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:46 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:14:46.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:46 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:14:46.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:46 smithi102 ceph-mon[100296]: osd.9 [v2:172.21.15.52:6826/4287934320,v1:172.21.15.52:6827/4287934320] boot 2023-11-27T15:14:46.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:46 smithi102 ceph-mon[100296]: osdmap e66: 10 total, 10 up, 10 in 2023-11-27T15:14:46.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:46 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:14:46.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:46 smithi102 ceph-mon[100296]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:14:46.685 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:14:46.681+0000 7f7647ff0700 1 -- 172.21.15.102:0/928270329 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f76400689a0 con 0x7f762c0a11d0 2023-11-27T15:14:46.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:46 smithi150 ceph-mon[100559]: purged_snaps scrub starts 2023-11-27T15:14:46.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:46 smithi150 ceph-mon[100559]: purged_snaps scrub ok 2023-11-27T15:14:46.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:46 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:14:46.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:46 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:14:46.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:46 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:14:46.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:46 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:14:46.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:46 smithi150 ceph-mon[100559]: osd.9 [v2:172.21.15.52:6826/4287934320,v1:172.21.15.52:6827/4287934320] boot 2023-11-27T15:14:46.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:46 smithi150 ceph-mon[100559]: osdmap e66: 10 total, 10 up, 10 in 2023-11-27T15:14:46.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:46 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:14:46.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:46 smithi150 ceph-mon[100559]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:14:46.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:46 smithi052 ceph-mon[99056]: purged_snaps scrub starts 2023-11-27T15:14:46.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:46 smithi052 ceph-mon[99056]: purged_snaps scrub ok 2023-11-27T15:14:46.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:46 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:14:46.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:46 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:14:46.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:46 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:14:46.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:46 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:14:46.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:46 smithi052 ceph-mon[99056]: osd.9 [v2:172.21.15.52:6826/4287934320,v1:172.21.15.52:6827/4287934320] boot 2023-11-27T15:14:46.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:46 smithi052 ceph-mon[99056]: osdmap e66: 10 total, 10 up, 10 in 2023-11-27T15:14:46.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:46 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:14:46.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:46 smithi052 ceph-mon[99056]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:14:47.639 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:47 smithi102 ceph-mon[100296]: osdmap e67: 10 total, 10 up, 10 in 2023-11-27T15:14:47.639 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:47 smithi102 ceph-mon[100296]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:14:47.639 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:47 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:14:47.639 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:47 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:14:47.639 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:47 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:47.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:47 smithi150 ceph-mon[100559]: osdmap e67: 10 total, 10 up, 10 in 2023-11-27T15:14:47.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:47 smithi150 ceph-mon[100559]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:14:47.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:47 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:14:47.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:47 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:14:47.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:47 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:47.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:47 smithi052 ceph-mon[99056]: osdmap e67: 10 total, 10 up, 10 in 2023-11-27T15:14:47.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:47 smithi052 ceph-mon[99056]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:14:47.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:47 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:14:47.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:47 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:14:47.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:47 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:48.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:48 smithi150 ceph-mon[100559]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:14:48.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:48 smithi102 ceph-mon[100296]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:14:48.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:48 smithi052 ceph-mon[99056]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:14:50.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:49 smithi150 ceph-mon[100559]: from='client.? 172.21.15.102:0/2504759341' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb540142-7563-44a4-b7a3-cb662714c128"}]: dispatch 2023-11-27T15:14:50.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:49 smithi150 ceph-mon[100559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb540142-7563-44a4-b7a3-cb662714c128"}]: dispatch 2023-11-27T15:14:50.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:49 smithi150 ceph-mon[100559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb540142-7563-44a4-b7a3-cb662714c128"}]': finished 2023-11-27T15:14:50.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:49 smithi150 ceph-mon[100559]: osdmap e68: 11 total, 10 up, 11 in 2023-11-27T15:14:50.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:49 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:14:50.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:49 smithi102 ceph-mon[100296]: from='client.? 172.21.15.102:0/2504759341' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb540142-7563-44a4-b7a3-cb662714c128"}]: dispatch 2023-11-27T15:14:50.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:49 smithi102 ceph-mon[100296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb540142-7563-44a4-b7a3-cb662714c128"}]: dispatch 2023-11-27T15:14:50.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:49 smithi102 ceph-mon[100296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb540142-7563-44a4-b7a3-cb662714c128"}]': finished 2023-11-27T15:14:50.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:49 smithi102 ceph-mon[100296]: osdmap e68: 11 total, 10 up, 11 in 2023-11-27T15:14:50.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:14:50.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:49 smithi052 ceph-mon[99056]: from='client.? 172.21.15.102:0/2504759341' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb540142-7563-44a4-b7a3-cb662714c128"}]: dispatch 2023-11-27T15:14:50.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:49 smithi052 ceph-mon[99056]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb540142-7563-44a4-b7a3-cb662714c128"}]: dispatch 2023-11-27T15:14:50.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:49 smithi052 ceph-mon[99056]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb540142-7563-44a4-b7a3-cb662714c128"}]': finished 2023-11-27T15:14:50.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:49 smithi052 ceph-mon[99056]: osdmap e68: 11 total, 10 up, 11 in 2023-11-27T15:14:50.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:49 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:14:51.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:50 smithi150 ceph-mon[100559]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:14:51.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:50 smithi150 ceph-mon[100559]: from='client.? 172.21.15.102:0/2743209822' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:14:51.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:50 smithi102 ceph-mon[100296]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:14:51.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:50 smithi102 ceph-mon[100296]: from='client.? 172.21.15.102:0/2743209822' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:14:51.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:50 smithi052 ceph-mon[99056]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:14:51.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:50 smithi052 ceph-mon[99056]: from='client.? 172.21.15.102:0/2743209822' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:14:52.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:51 smithi150 ceph-mon[100559]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:14:52.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:51 smithi102 ceph-mon[100296]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:14:52.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:51 smithi052 ceph-mon[99056]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:14:54.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:54 smithi102 ceph-mon[100296]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:14:55.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:54 smithi150 ceph-mon[100559]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:14:55.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:54 smithi052 ceph-mon[99056]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:14:56.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:55 smithi150 ceph-mon[100559]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:14:56.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:55 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-11-27T15:14:56.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:55 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:56.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:55 smithi102 ceph-mon[100296]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:14:56.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:55 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-11-27T15:14:56.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:55 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:56.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:55 smithi052 ceph-mon[99056]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:14:56.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:55 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-11-27T15:14:56.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:55 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:57.009 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:56 smithi102 ceph-mon[100296]: Deploying daemon osd.10 on smithi102 2023-11-27T15:14:57.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:56 smithi150 ceph-mon[100559]: Deploying daemon osd.10 on smithi102 2023-11-27T15:14:57.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:56 smithi052 ceph-mon[99056]: Deploying daemon osd.10 on smithi102 2023-11-27T15:14:58.047 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:57 smithi102 ceph-mon[100296]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:14:58.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:57 smithi052 ceph-mon[99056]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:14:58.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:57 smithi150 ceph-mon[100559]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:15:00.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:14:59 smithi150 ceph-mon[100559]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:15:00.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:14:59 smithi102 ceph-mon[100296]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:15:00.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:14:59 smithi052 ceph-mon[99056]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:15:01.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:01 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:01.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:01 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:01.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:01 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:01.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:01 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:01.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:01 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:01.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:01 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:01.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:01 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:01.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:01 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:01.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:01 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:02.492 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:02 smithi102 ceph-mon[100296]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:15:02.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:02 smithi052 ceph-mon[99056]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:15:02.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:02 smithi150 ceph-mon[100559]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:15:03.457 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:15:03.452+0000 7f7636ffd700 1 -- 172.21.15.102:0/928270329 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f76400689a0 con 0x7f762c0a11d0 2023-11-27T15:15:03.457 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 10 on host 'smithi102' 2023-11-27T15:15:03.459 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:15:03.454+0000 7f7647ff0700 1 -- 172.21.15.102:0/928270329 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f762c0a11d0 msgr2=0x7f762c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:03.459 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:15:03.454+0000 7f7647ff0700 1 --2- 172.21.15.102:0/928270329 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f762c0a11d0 0x7f762c0a3680 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f763c009a10 tx=0x7f763c009190 comp rx=0 tx=0).stop 2023-11-27T15:15:03.459 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:15:03.454+0000 7f7647ff0700 1 -- 172.21.15.102:0/928270329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76400ff1e0 msgr2=0x7f76401950c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:03.460 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:15:03.454+0000 7f7647ff0700 1 --2- 172.21.15.102:0/928270329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76400ff1e0 0x7f76401950c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f763001a870 tx=0x7f763001a8a0 comp rx=0 tx=0).stop 2023-11-27T15:15:03.460 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:15:03.454+0000 7f7647ff0700 1 -- 172.21.15.102:0/928270329 shutdown_connections 2023-11-27T15:15:03.460 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:15:03.454+0000 7f7647ff0700 1 --2- 172.21.15.102:0/928270329 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f762c0a11d0 0x7f762c0a3680 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:03.460 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:15:03.454+0000 7f7647ff0700 1 --2- 172.21.15.102:0/928270329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76400ff1e0 0x7f76401950c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:03.460 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:15:03.454+0000 7f7647ff0700 1 --2- 172.21.15.102:0/928270329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7640100d70 0x7f764019c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:03.460 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:15:03.454+0000 7f7647ff0700 1 --2- 172.21.15.102:0/928270329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76401003e0 0x7f7640195600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:03.460 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:15:03.454+0000 7f7647ff0700 1 -- 172.21.15.102:0/928270329 >> 172.21.15.102:0/928270329 conn(0x7f76400fa7f0 msgr2=0x7f76401039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:03.460 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:15:03.454+0000 7f7647ff0700 1 -- 172.21.15.102:0/928270329 shutdown_connections 2023-11-27T15:15:03.460 INFO:teuthology.orchestra.run.smithi102.stderr:2023-11-27T15:15:03.455+0000 7f7647ff0700 1 -- 172.21.15.102:0/928270329 wait complete. 2023-11-27T15:15:03.867 DEBUG:teuthology.orchestra.run.smithi102:osd.10> sudo journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.10.service 2023-11-27T15:15:03.869 INFO:tasks.cephadm:Deploying osd.11 on smithi150 with /dev/vg_nvme/lv_1... 2023-11-27T15:15:03.869 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-11-27T15:15:03.898 INFO:journalctl@ceph.osd.10.smithi102.stdout:-- Logs begin at Mon 2023-11-27 14:52:53 UTC. -- 2023-11-27T15:15:04.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:04 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:04.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:04 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:04.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:04 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:04.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:04 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:04.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:04 smithi150 ceph-mon[100559]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:15:04.518 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:15:04.578 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:15:04 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:15:04.265+0000 7f92b7aa5640 -1 osd.10 0 log_to_monitors true 2023-11-27T15:15:04.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:04 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:04.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:04 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:04.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:04 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:04.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:04 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:04.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:04 smithi102 ceph-mon[100296]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:15:04.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:04 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:04.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:04 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:04.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:04 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:04.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:04 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:04.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:04 smithi052 ceph-mon[99056]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:15:05.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:05 smithi150 ceph-mon[100559]: from='osd.10 [v2:172.21.15.102:6824/3056034201,v1:172.21.15.102:6825/3056034201]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-27T15:15:05.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:05 smithi150 ceph-mon[100559]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-27T15:15:05.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:05 smithi102 ceph-mon[100296]: from='osd.10 [v2:172.21.15.102:6824/3056034201,v1:172.21.15.102:6825/3056034201]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-27T15:15:05.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:05 smithi102 ceph-mon[100296]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-27T15:15:05.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:05 smithi052 ceph-mon[99056]: from='osd.10 [v2:172.21.15.102:6824/3056034201,v1:172.21.15.102:6825/3056034201]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-27T15:15:05.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:05 smithi052 ceph-mon[99056]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-27T15:15:05.876 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-11-27T15:15:05.877 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-11-27T15:15:05.877 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2023-11-27T15:15:05.877 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2023-11-27T15:15:05.877 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111149 s, 943 MB/s 2023-11-27T15:15:05.877 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2023-11-27T15:15:06.206 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:06 smithi150 ceph-mon[100559]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-11-27T15:15:06.206 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:06 smithi150 ceph-mon[100559]: osdmap e69: 11 total, 10 up, 11 in 2023-11-27T15:15:06.206 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:06 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:15:06.206 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:06 smithi150 ceph-mon[100559]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2023-11-27T15:15:06.206 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:06 smithi150 ceph-mon[100559]: from='osd.10 [v2:172.21.15.102:6824/3056034201,v1:172.21.15.102:6825/3056034201]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2023-11-27T15:15:06.207 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:06 smithi150 ceph-mon[100559]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:15:06.260 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi150:vg_nvme/lv_1 2023-11-27T15:15:06.515 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:06 smithi102 ceph-mon[100296]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-11-27T15:15:06.515 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:06 smithi102 ceph-mon[100296]: osdmap e69: 11 total, 10 up, 11 in 2023-11-27T15:15:06.515 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:06 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:15:06.516 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:06 smithi102 ceph-mon[100296]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2023-11-27T15:15:06.516 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:06 smithi102 ceph-mon[100296]: from='osd.10 [v2:172.21.15.102:6824/3056034201,v1:172.21.15.102:6825/3056034201]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2023-11-27T15:15:06.516 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:06 smithi102 ceph-mon[100296]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:15:06.516 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:15:06 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:15:06.144+0000 7f92ad11c700 -1 osd.10 0 waiting for initial osdmap 2023-11-27T15:15:06.516 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:15:06 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:15:06.159+0000 7f92a4f09700 -1 osd.10 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:15:06.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:06 smithi052 ceph-mon[99056]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-11-27T15:15:06.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:06 smithi052 ceph-mon[99056]: osdmap e69: 11 total, 10 up, 11 in 2023-11-27T15:15:06.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:06 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:15:06.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:06 smithi052 ceph-mon[99056]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2023-11-27T15:15:06.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:06 smithi052 ceph-mon[99056]: from='osd.10 [v2:172.21.15.102:6824/3056034201,v1:172.21.15.102:6825/3056034201]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2023-11-27T15:15:06.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:06 smithi052 ceph-mon[99056]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:15:06.900 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:15:07.380 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:07 smithi150 ceph-mon[100559]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2023-11-27T15:15:07.380 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:07 smithi150 ceph-mon[100559]: osdmap e70: 11 total, 10 up, 11 in 2023-11-27T15:15:07.380 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:07 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:15:07.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:07 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:15:07.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:07 smithi150 ceph-mon[100559]: Detected new or changed devices on smithi102 2023-11-27T15:15:07.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:07 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:07.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:07 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:07.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:07 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:15:07.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:07 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:15:07.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:07 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:15:07.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:07 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:15:07.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:07 smithi150 ceph-mon[100559]: Adjusting osd_memory_target on smithi102 to 4292M 2023-11-27T15:15:07.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:07 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:07.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:07 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:07.381 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:07 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:07.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:07 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:15:07.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:07 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:07.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:07 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:15:07.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:07 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:15:07.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:07 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:07.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:07 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:07.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:07 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:07.382 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:07 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:07.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:07 smithi102 ceph-mon[100296]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2023-11-27T15:15:07.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:07 smithi102 ceph-mon[100296]: osdmap e70: 11 total, 10 up, 11 in 2023-11-27T15:15:07.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:07 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:15:07.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:07 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:15:07.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:07 smithi102 ceph-mon[100296]: Detected new or changed devices on smithi102 2023-11-27T15:15:07.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:07 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:07.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:07 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:07.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:07 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:15:07.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:07 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:15:07.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:07 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:15:07.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:07 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:15:07.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:07 smithi102 ceph-mon[100296]: Adjusting osd_memory_target on smithi102 to 4292M 2023-11-27T15:15:07.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:07 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:07.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:07 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:07.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:07 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:07.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:07 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:15:07.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:07 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:07.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:07 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:15:07.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:07 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:15:07.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:07 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:07.580 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:07 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:07.580 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:07 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:07.580 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:07 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:07.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:07 smithi052 ceph-mon[99056]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2023-11-27T15:15:07.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:07 smithi052 ceph-mon[99056]: osdmap e70: 11 total, 10 up, 11 in 2023-11-27T15:15:07.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:07 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:15:07.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:07 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:15:07.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:07 smithi052 ceph-mon[99056]: Detected new or changed devices on smithi102 2023-11-27T15:15:07.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:07 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:07.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:07 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:07.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:07 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:15:07.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:07 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:15:07.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:07 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:15:07.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:07 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:15:07.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:07 smithi052 ceph-mon[99056]: Adjusting osd_memory_target on smithi102 to 4292M 2023-11-27T15:15:07.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:07 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:07.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:07 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:07.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:07 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:07.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:07 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:15:07.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:07 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:07.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:07 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:15:07.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:07 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:15:07.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:07 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:07.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:07 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:07.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:07 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:07.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:07 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:08.067 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.062+0000 7f6ed2337700 1 -- 172.21.15.150:0/2326259896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ecc1003e0 msgr2=0x7f6ecc100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:08.068 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.062+0000 7f6ed2337700 1 --2- 172.21.15.150:0/2326259896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ecc1003e0 0x7f6ecc100830 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009990 tx=0x7f6eb4009ca0 comp rx=0 tx=0).stop 2023-11-27T15:15:08.068 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.062+0000 7f6ed2337700 1 -- 172.21.15.150:0/2326259896 shutdown_connections 2023-11-27T15:15:08.068 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.062+0000 7f6ed2337700 1 --2- 172.21.15.150:0/2326259896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ecc0ff1e0 0x7f6ecc0ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:08.068 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.062+0000 7f6ed2337700 1 --2- 172.21.15.150:0/2326259896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ecc100d70 0x7f6ecc107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:08.068 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.062+0000 7f6ed2337700 1 --2- 172.21.15.150:0/2326259896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ecc1003e0 0x7f6ecc100830 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:08.068 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.062+0000 7f6ed2337700 1 -- 172.21.15.150:0/2326259896 >> 172.21.15.150:0/2326259896 conn(0x7f6ecc0fa7f0 msgr2=0x7f6ecc0fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:08.068 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.062+0000 7f6ed2337700 1 -- 172.21.15.150:0/2326259896 shutdown_connections 2023-11-27T15:15:08.068 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.062+0000 7f6ed2337700 1 -- 172.21.15.150:0/2326259896 wait complete. 2023-11-27T15:15:08.068 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.063+0000 7f6ed2337700 1 Processor -- start 2023-11-27T15:15:08.068 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.063+0000 7f6ed2337700 1 -- start start 2023-11-27T15:15:08.070 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.063+0000 7f6ed2337700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ecc0ff1e0 0x7f6ecc194ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:08.070 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.063+0000 7f6ed2337700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ecc1003e0 0x7f6ecc195430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:08.070 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.063+0000 7f6ed2337700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ecc100d70 0x7f6ecc19c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:08.070 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.063+0000 7f6ed2337700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc19ca80 con 0x7f6ecc0ff1e0 2023-11-27T15:15:08.070 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.063+0000 7f6ed2337700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc19cbc0 con 0x7f6ecc100d70 2023-11-27T15:15:08.070 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.063+0000 7f6ed2337700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc19cd50 con 0x7f6ecc1003e0 2023-11-27T15:15:08.070 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.064+0000 7f6ecb7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ecc1003e0 0x7f6ecc195430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:08.070 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.064+0000 7f6ecb7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ecc1003e0 0x7f6ecc195430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49728/0 (socket says 172.21.15.150:49728) 2023-11-27T15:15:08.071 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.064+0000 7f6ecb7fe700 1 -- 172.21.15.150:0/3737056873 learned_addr learned my addr 172.21.15.150:0/3737056873 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-27T15:15:08.071 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.064+0000 7f6ed08d4700 1 --2- 172.21.15.150:0/3737056873 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ecc100d70 0x7f6ecc19c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:08.071 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.064+0000 7f6ecbfff700 1 --2- 172.21.15.150:0/3737056873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ecc0ff1e0 0x7f6ecc194ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:08.071 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.064+0000 7f6ecb7fe700 1 -- 172.21.15.150:0/3737056873 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ecc100d70 msgr2=0x7f6ecc19c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:08.071 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.064+0000 7f6ecb7fe700 1 --2- 172.21.15.150:0/3737056873 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ecc100d70 0x7f6ecc19c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:08.071 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.064+0000 7f6ecb7fe700 1 -- 172.21.15.150:0/3737056873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ecc0ff1e0 msgr2=0x7f6ecc194ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:08.071 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.064+0000 7f6ecb7fe700 1 --2- 172.21.15.150:0/3737056873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ecc0ff1e0 0x7f6ecc194ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:08.071 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.064+0000 7f6ecb7fe700 1 -- 172.21.15.150:0/3737056873 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb40095f0 con 0x7f6ecc1003e0 2023-11-27T15:15:08.071 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.064+0000 7f6ecb7fe700 1 --2- 172.21.15.150:0/3737056873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ecc1003e0 0x7f6ecc195430 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4005910 tx=0x7f6eb400f780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:08.071 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.064+0000 7f6ec97fa700 1 -- 172.21.15.150:0/3737056873 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb401d070 con 0x7f6ecc1003e0 2023-11-27T15:15:08.071 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.064+0000 7f6ec97fa700 1 -- 172.21.15.150:0/3737056873 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6eb400fbb0 con 0x7f6ecc1003e0 2023-11-27T15:15:08.072 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.064+0000 7f6ec97fa700 1 -- 172.21.15.150:0/3737056873 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4017650 con 0x7f6ecc1003e0 2023-11-27T15:15:08.072 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.064+0000 7f6ed2337700 1 -- 172.21.15.150:0/3737056873 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ecc10ab50 con 0x7f6ecc1003e0 2023-11-27T15:15:08.072 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.065+0000 7f6ed2337700 1 -- 172.21.15.150:0/3737056873 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ecc10a9c0 con 0x7f6ecc1003e0 2023-11-27T15:15:08.072 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.065+0000 7f6ed2337700 1 -- 172.21.15.150:0/3737056873 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ecc19d260 con 0x7f6ecc1003e0 2023-11-27T15:15:08.073 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.066+0000 7f6ec97fa700 1 -- 172.21.15.150:0/3737056873 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6eb400fd20 con 0x7f6ecc1003e0 2023-11-27T15:15:08.073 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.067+0000 7f6ec97fa700 1 --2- 172.21.15.150:0/3737056873 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6eb80a1150 0x7f6eb80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:08.073 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.067+0000 7f6ec97fa700 1 -- 172.21.15.150:0/3737056873 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6727+0+0 (secure 0 0 0) 0x7f6eb40cd600 con 0x7f6ecc1003e0 2023-11-27T15:15:08.073 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.067+0000 7f6ecbfff700 1 --2- 172.21.15.150:0/3737056873 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6eb80a1150 0x7f6eb80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:08.073 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.068+0000 7f6ecbfff700 1 --2- 172.21.15.150:0/3737056873 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6eb80a1150 0x7f6eb80a3600 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc100240 tx=0x7f6ebc00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:08.076 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.071+0000 7f6ec97fa700 1 -- 172.21.15.150:0/3737056873 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6eb4097410 con 0x7f6ecc1003e0 2023-11-27T15:15:08.273 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:08.268+0000 7f6ed2337700 1 -- 172.21.15.150:0/3737056873 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ecc060f30 con 0x7f6eb80a1150 2023-11-27T15:15:08.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:08 smithi150 ceph-mon[100559]: purged_snaps scrub starts 2023-11-27T15:15:08.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:08 smithi150 ceph-mon[100559]: purged_snaps scrub ok 2023-11-27T15:15:08.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:08 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:15:08.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:08 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:15:08.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:08 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:15:08.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:08 smithi150 ceph-mon[100559]: osd.10 [v2:172.21.15.102:6824/3056034201,v1:172.21.15.102:6825/3056034201] boot 2023-11-27T15:15:08.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:08 smithi150 ceph-mon[100559]: osdmap e71: 11 total, 11 up, 11 in 2023-11-27T15:15:08.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:08 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:15:08.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:08 smithi150 ceph-mon[100559]: pgmap v213: 1 pgs: 1 peering; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:15:08.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:08 smithi102 ceph-mon[100296]: purged_snaps scrub starts 2023-11-27T15:15:08.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:08 smithi102 ceph-mon[100296]: purged_snaps scrub ok 2023-11-27T15:15:08.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:08 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:15:08.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:08 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:15:08.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:08 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:15:08.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:08 smithi102 ceph-mon[100296]: osd.10 [v2:172.21.15.102:6824/3056034201,v1:172.21.15.102:6825/3056034201] boot 2023-11-27T15:15:08.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:08 smithi102 ceph-mon[100296]: osdmap e71: 11 total, 11 up, 11 in 2023-11-27T15:15:08.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:08 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:15:08.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:08 smithi102 ceph-mon[100296]: pgmap v213: 1 pgs: 1 peering; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:15:08.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:08 smithi052 ceph-mon[99056]: purged_snaps scrub starts 2023-11-27T15:15:08.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:08 smithi052 ceph-mon[99056]: purged_snaps scrub ok 2023-11-27T15:15:08.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:08 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:15:08.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:08 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:15:08.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:08 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:15:08.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:08 smithi052 ceph-mon[99056]: osd.10 [v2:172.21.15.102:6824/3056034201,v1:172.21.15.102:6825/3056034201] boot 2023-11-27T15:15:08.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:08 smithi052 ceph-mon[99056]: osdmap e71: 11 total, 11 up, 11 in 2023-11-27T15:15:08.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:08 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:15:08.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:08 smithi052 ceph-mon[99056]: pgmap v213: 1 pgs: 1 peering; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:15:09.421 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:09 smithi150 ceph-mon[100559]: osdmap e72: 11 total, 11 up, 11 in 2023-11-27T15:15:09.421 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:09 smithi150 ceph-mon[100559]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:15:09.421 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:09 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:15:09.421 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:09 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:15:09.422 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:09 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:09.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:09 smithi102 ceph-mon[100296]: osdmap e72: 11 total, 11 up, 11 in 2023-11-27T15:15:09.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:09 smithi102 ceph-mon[100296]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:15:09.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:09 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:15:09.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:09 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:15:09.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:09 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:09.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:09 smithi052 ceph-mon[99056]: osdmap e72: 11 total, 11 up, 11 in 2023-11-27T15:15:09.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:09 smithi052 ceph-mon[99056]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:15:09.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:09 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:15:09.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:09 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:15:09.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:09 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:10.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:10 smithi150 ceph-mon[100559]: osdmap e73: 11 total, 11 up, 11 in 2023-11-27T15:15:10.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:10 smithi150 ceph-mon[100559]: pgmap v216: 1 pgs: 1 peering; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:15:10.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:10 smithi102 ceph-mon[100296]: osdmap e73: 11 total, 11 up, 11 in 2023-11-27T15:15:10.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:10 smithi102 ceph-mon[100296]: pgmap v216: 1 pgs: 1 peering; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:15:10.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:10 smithi052 ceph-mon[99056]: osdmap e73: 11 total, 11 up, 11 in 2023-11-27T15:15:10.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:10 smithi052 ceph-mon[99056]: pgmap v216: 1 pgs: 1 peering; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:15:11.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:11 smithi150 ceph-mon[100559]: osdmap e74: 11 total, 11 up, 11 in 2023-11-27T15:15:11.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:11 smithi150 ceph-mon[100559]: from='client.? 172.21.15.150:0/3645679541' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3156528e-e0f8-4d44-9946-c8b6339c2bd4"}]: dispatch 2023-11-27T15:15:11.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:11 smithi150 ceph-mon[100559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3156528e-e0f8-4d44-9946-c8b6339c2bd4"}]: dispatch 2023-11-27T15:15:11.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:11 smithi150 ceph-mon[100559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3156528e-e0f8-4d44-9946-c8b6339c2bd4"}]': finished 2023-11-27T15:15:11.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:11 smithi150 ceph-mon[100559]: osdmap e75: 12 total, 11 up, 12 in 2023-11-27T15:15:11.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:11 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:15:11.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:11 smithi102 ceph-mon[100296]: osdmap e74: 11 total, 11 up, 11 in 2023-11-27T15:15:11.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:11 smithi102 ceph-mon[100296]: from='client.? 172.21.15.150:0/3645679541' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3156528e-e0f8-4d44-9946-c8b6339c2bd4"}]: dispatch 2023-11-27T15:15:11.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:11 smithi102 ceph-mon[100296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3156528e-e0f8-4d44-9946-c8b6339c2bd4"}]: dispatch 2023-11-27T15:15:11.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:11 smithi102 ceph-mon[100296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3156528e-e0f8-4d44-9946-c8b6339c2bd4"}]': finished 2023-11-27T15:15:11.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:11 smithi102 ceph-mon[100296]: osdmap e75: 12 total, 11 up, 12 in 2023-11-27T15:15:11.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:11 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:15:11.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:11 smithi052 ceph-mon[99056]: osdmap e74: 11 total, 11 up, 11 in 2023-11-27T15:15:11.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:11 smithi052 ceph-mon[99056]: from='client.? 172.21.15.150:0/3645679541' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3156528e-e0f8-4d44-9946-c8b6339c2bd4"}]: dispatch 2023-11-27T15:15:11.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:11 smithi052 ceph-mon[99056]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3156528e-e0f8-4d44-9946-c8b6339c2bd4"}]: dispatch 2023-11-27T15:15:11.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:11 smithi052 ceph-mon[99056]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3156528e-e0f8-4d44-9946-c8b6339c2bd4"}]': finished 2023-11-27T15:15:11.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:11 smithi052 ceph-mon[99056]: osdmap e75: 12 total, 11 up, 12 in 2023-11-27T15:15:11.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:11 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:15:12.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:12 smithi150 ceph-mon[100559]: from='client.? 172.21.15.150:0/2774925826' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:15:12.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:12 smithi150 ceph-mon[100559]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-27T15:15:12.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:12 smithi102 ceph-mon[100296]: from='client.? 172.21.15.150:0/2774925826' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:15:12.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:12 smithi102 ceph-mon[100296]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-27T15:15:12.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:12 smithi052 ceph-mon[99056]: from='client.? 172.21.15.150:0/2774925826' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:15:12.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:12 smithi052 ceph-mon[99056]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-27T15:15:15.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:14 smithi150 ceph-mon[100559]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail; 104 KiB/s, 0 objects/s recovering 2023-11-27T15:15:15.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:14 smithi102 ceph-mon[100296]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail; 104 KiB/s, 0 objects/s recovering 2023-11-27T15:15:15.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:14 smithi052 ceph-mon[99056]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail; 104 KiB/s, 0 objects/s recovering 2023-11-27T15:15:16.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:15 smithi150 ceph-mon[100559]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 88 KiB/s, 0 objects/s recovering 2023-11-27T15:15:16.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:15 smithi102 ceph-mon[100296]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 88 KiB/s, 0 objects/s recovering 2023-11-27T15:15:16.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:15 smithi052 ceph-mon[99056]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 88 KiB/s, 0 objects/s recovering 2023-11-27T15:15:17.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:17 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-11-27T15:15:17.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:17 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:17.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:17 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-11-27T15:15:17.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:17 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:17.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:17 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-11-27T15:15:17.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:17 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:18.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:18 smithi150 ceph-mon[100559]: Deploying daemon osd.11 on smithi150 2023-11-27T15:15:18.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:18 smithi150 ceph-mon[100559]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-27T15:15:18.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:18 smithi102 ceph-mon[100296]: Deploying daemon osd.11 on smithi150 2023-11-27T15:15:18.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:18 smithi102 ceph-mon[100296]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-27T15:15:18.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:18 smithi052 ceph-mon[99056]: Deploying daemon osd.11 on smithi150 2023-11-27T15:15:18.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:18 smithi052 ceph-mon[99056]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-27T15:15:20.024 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:19 smithi150 ceph-mon[100559]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2023-11-27T15:15:20.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:19 smithi102 ceph-mon[100296]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2023-11-27T15:15:20.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:19 smithi052 ceph-mon[99056]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2023-11-27T15:15:21.777 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:21 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:21.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:21 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:21.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:21 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:21.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:21 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:21.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:21 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:21.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:21 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:21.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:21 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:21.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:21 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:21.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:21 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:22.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:22 smithi150 ceph-mon[100559]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-11-27T15:15:22.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:22 smithi102 ceph-mon[100296]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-11-27T15:15:22.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:22 smithi052 ceph-mon[99056]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-11-27T15:15:24.238 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:24.233+0000 7f6ec97fa700 1 -- 172.21.15.150:0/3737056873 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f6ecc060f30 con 0x7f6eb80a1150 2023-11-27T15:15:24.238 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 11 on host 'smithi150' 2023-11-27T15:15:24.240 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:24.235+0000 7f6ed2337700 1 -- 172.21.15.150:0/3737056873 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6eb80a1150 msgr2=0x7f6eb80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:24.240 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:24.235+0000 7f6ed2337700 1 --2- 172.21.15.150:0/3737056873 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6eb80a1150 0x7f6eb80a3600 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc100240 tx=0x7f6ebc00a380 comp rx=0 tx=0).stop 2023-11-27T15:15:24.240 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:24.235+0000 7f6ed2337700 1 -- 172.21.15.150:0/3737056873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ecc1003e0 msgr2=0x7f6ecc195430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:24.240 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:24.235+0000 7f6ed2337700 1 --2- 172.21.15.150:0/3737056873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ecc1003e0 0x7f6ecc195430 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4005910 tx=0x7f6eb400f780 comp rx=0 tx=0).stop 2023-11-27T15:15:24.241 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:24.235+0000 7f6ed2337700 1 -- 172.21.15.150:0/3737056873 shutdown_connections 2023-11-27T15:15:24.241 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:24.235+0000 7f6ed2337700 1 --2- 172.21.15.150:0/3737056873 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6eb80a1150 0x7f6eb80a3600 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:24.241 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:24.235+0000 7f6ed2337700 1 --2- 172.21.15.150:0/3737056873 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ecc100d70 0x7f6ecc19c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:24.241 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:24.235+0000 7f6ed2337700 1 --2- 172.21.15.150:0/3737056873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ecc0ff1e0 0x7f6ecc194ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:24.241 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:24.235+0000 7f6ed2337700 1 --2- 172.21.15.150:0/3737056873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ecc1003e0 0x7f6ecc195430 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:24.241 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:24.235+0000 7f6ed2337700 1 -- 172.21.15.150:0/3737056873 >> 172.21.15.150:0/3737056873 conn(0x7f6ecc0fa7f0 msgr2=0x7f6ecc0fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:24.241 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:24.235+0000 7f6ed2337700 1 -- 172.21.15.150:0/3737056873 shutdown_connections 2023-11-27T15:15:24.242 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:24.235+0000 7f6ed2337700 1 -- 172.21.15.150:0/3737056873 wait complete. 2023-11-27T15:15:24.598 DEBUG:teuthology.orchestra.run.smithi150:osd.11> sudo journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.11.service 2023-11-27T15:15:24.601 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-11-27T15:15:24.601 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph osd stat -f json 2023-11-27T15:15:24.639 INFO:journalctl@ceph.osd.11.smithi150.stdout:-- Logs begin at Mon 2023-11-27 14:53:12 UTC. -- 2023-11-27T15:15:25.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:24 smithi150 ceph-mon[100559]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:15:25.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:24 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:25.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:24 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:25.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:24 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:25.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:24 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:25.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:24 smithi102 ceph-mon[100296]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:15:25.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:24 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:25.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:24 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:25.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:24 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:25.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:24 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:25.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:24 smithi052 ceph-mon[99056]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:15:25.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:24 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:25.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:24 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:25.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:24 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:25.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:24 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:25.274 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:15:25.478 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:15:25 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:15:25.336+0000 7fa0c1999640 -1 osd.11 0 log_to_monitors true 2023-11-27T15:15:25.736 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:25 smithi052 ceph-mon[99056]: from='osd.11 [v2:172.21.15.150:6824/3207559739,v1:172.21.15.150:6825/3207559739]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-27T15:15:25.736 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:25 smithi052 ceph-mon[99056]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-27T15:15:25.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:25 smithi150 ceph-mon[100559]: from='osd.11 [v2:172.21.15.150:6824/3207559739,v1:172.21.15.150:6825/3207559739]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-27T15:15:25.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:25 smithi150 ceph-mon[100559]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-27T15:15:26.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:25 smithi102 ceph-mon[100296]: from='osd.11 [v2:172.21.15.150:6824/3207559739,v1:172.21.15.150:6825/3207559739]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-27T15:15:26.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:25 smithi102 ceph-mon[100296]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-27T15:15:26.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.488+0000 7efdf708d700 1 -- 172.21.15.52:0/608901567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdf00ff160 msgr2=0x7efdf00ff570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:26.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.488+0000 7efdf708d700 1 --2- 172.21.15.52:0/608901567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdf00ff160 0x7efdf00ff570 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efde4009990 tx=0x7efde4009ca0 comp rx=0 tx=0).stop 2023-11-27T15:15:26.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.488+0000 7efdf708d700 1 -- 172.21.15.52:0/608901567 shutdown_connections 2023-11-27T15:15:26.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.488+0000 7efdf708d700 1 --2- 172.21.15.52:0/608901567 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efdf0100cf0 0x7efdf01075b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:26.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.488+0000 7efdf708d700 1 --2- 172.21.15.52:0/608901567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdf00ff160 0x7efdf00ff570 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:26.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.488+0000 7efdf708d700 1 --2- 172.21.15.52:0/608901567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdf0100360 0x7efdf01007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:26.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.488+0000 7efdf708d700 1 -- 172.21.15.52:0/608901567 >> 172.21.15.52:0/608901567 conn(0x7efdf00fa790 msgr2=0x7efdf00fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:26.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.488+0000 7efdf708d700 1 -- 172.21.15.52:0/608901567 shutdown_connections 2023-11-27T15:15:26.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.488+0000 7efdf708d700 1 -- 172.21.15.52:0/608901567 wait complete. 2023-11-27T15:15:26.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.489+0000 7efdf708d700 1 Processor -- start 2023-11-27T15:15:26.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.489+0000 7efdf708d700 1 -- start start 2023-11-27T15:15:26.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.489+0000 7efdf708d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdf00ff160 0x7efdf01981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:26.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.489+0000 7efdf708d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efdf0100360 0x7efdf01986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:26.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.489+0000 7efdf708d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdf0100cf0 0x7efdf019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:26.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.489+0000 7efdf708d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf019fd40 con 0x7efdf0100cf0 2023-11-27T15:15:26.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.489+0000 7efdf708d700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf019fe80 con 0x7efdf00ff160 2023-11-27T15:15:26.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.489+0000 7efdf708d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf01a0010 con 0x7efdf0100360 2023-11-27T15:15:26.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.489+0000 7efdf562a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdf0100cf0 0x7efdf019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:26.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.489+0000 7efdf562a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdf0100cf0 0x7efdf019f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49638/0 (socket says 172.21.15.52:49638) 2023-11-27T15:15:26.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.489+0000 7efdf562a700 1 -- 172.21.15.52:0/396132740 learned_addr learned my addr 172.21.15.52:0/396132740 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:15:26.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.489+0000 7efdeffff700 1 --2- 172.21.15.52:0/396132740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efdf0100360 0x7efdf01986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:26.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.489+0000 7efdf4e29700 1 --2- 172.21.15.52:0/396132740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdf00ff160 0x7efdf01981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:26.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.489+0000 7efdf562a700 1 -- 172.21.15.52:0/396132740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efdf0100360 msgr2=0x7efdf01986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:26.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.489+0000 7efdf562a700 1 --2- 172.21.15.52:0/396132740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efdf0100360 0x7efdf01986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:26.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.489+0000 7efdf562a700 1 -- 172.21.15.52:0/396132740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdf00ff160 msgr2=0x7efdf01981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:26.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.489+0000 7efdf562a700 1 --2- 172.21.15.52:0/396132740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdf00ff160 0x7efdf01981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:26.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.489+0000 7efdf562a700 1 -- 172.21.15.52:0/396132740 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde40095f0 con 0x7efdf0100cf0 2023-11-27T15:15:26.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.489+0000 7efdf4e29700 1 --2- 172.21.15.52:0/396132740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdf00ff160 0x7efdf01981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:15:26.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.489+0000 7efdf562a700 1 --2- 172.21.15.52:0/396132740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdf0100cf0 0x7efdf019f770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efde800ef60 tx=0x7efde800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:26.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.489+0000 7efdedffb700 1 -- 172.21.15.52:0/396132740 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde8018070 con 0x7efdf0100cf0 2023-11-27T15:15:26.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.490+0000 7efdedffb700 1 -- 172.21.15.52:0/396132740 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efde800f040 con 0x7efdf0100cf0 2023-11-27T15:15:26.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.490+0000 7efdf708d700 1 -- 172.21.15.52:0/396132740 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf010aac0 con 0x7efdf0100cf0 2023-11-27T15:15:26.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.490+0000 7efdedffb700 1 -- 172.21.15.52:0/396132740 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde8003e30 con 0x7efdf0100cf0 2023-11-27T15:15:26.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.490+0000 7efdf708d700 1 -- 172.21.15.52:0/396132740 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf010a930 con 0x7efdf0100cf0 2023-11-27T15:15:26.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.491+0000 7efdf708d700 1 -- 172.21.15.52:0/396132740 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdf004e930 con 0x7efdf0100cf0 2023-11-27T15:15:26.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.492+0000 7efdedffb700 1 -- 172.21.15.52:0/396132740 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7efde8004ad0 con 0x7efdf0100cf0 2023-11-27T15:15:26.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.492+0000 7efdedffb700 1 --2- 172.21.15.52:0/396132740 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7efde00a1150 0x7efde00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:26.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.492+0000 7efdf4e29700 1 --2- 172.21.15.52:0/396132740 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7efde00a1150 0x7efde00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:26.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.492+0000 7efdedffb700 1 -- 172.21.15.52:0/396132740 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6831+0+0 (secure 0 0 0) 0x7efde80cbf80 con 0x7efdf0100cf0 2023-11-27T15:15:26.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.493+0000 7efdf4e29700 1 --2- 172.21.15.52:0/396132740 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7efde00a1150 0x7efde00a3600 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efde400b2e0 tx=0x7efde4005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:26.501 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.497+0000 7efdedffb700 1 -- 172.21.15.52:0/396132740 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7efde8095d30 con 0x7efdf0100cf0 2023-11-27T15:15:26.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.696+0000 7efdf708d700 1 -- 172.21.15.52:0/396132740 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7efdf01a0b50 con 0x7efdf0100cf0 2023-11-27T15:15:26.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.696+0000 7efdedffb700 1 -- 172.21.15.52:0/396132740 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) v1 ==== 74+0+133 (secure 0 0 0) 0x7efde8099350 con 0x7efdf0100cf0 2023-11-27T15:15:26.701 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:15:26.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.698+0000 7efdf708d700 1 -- 172.21.15.52:0/396132740 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7efde00a1150 msgr2=0x7efde00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:26.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.698+0000 7efdf708d700 1 --2- 172.21.15.52:0/396132740 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7efde00a1150 0x7efde00a3600 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efde400b2e0 tx=0x7efde4005e80 comp rx=0 tx=0).stop 2023-11-27T15:15:26.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.698+0000 7efdf708d700 1 -- 172.21.15.52:0/396132740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdf0100cf0 msgr2=0x7efdf019f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:26.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.698+0000 7efdf708d700 1 --2- 172.21.15.52:0/396132740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdf0100cf0 0x7efdf019f770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efde800ef60 tx=0x7efde800ef90 comp rx=0 tx=0).stop 2023-11-27T15:15:26.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.699+0000 7efdf708d700 1 -- 172.21.15.52:0/396132740 shutdown_connections 2023-11-27T15:15:26.704 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.699+0000 7efdf708d700 1 --2- 172.21.15.52:0/396132740 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7efde00a1150 0x7efde00a3600 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:26.704 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.699+0000 7efdf708d700 1 --2- 172.21.15.52:0/396132740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efdf00ff160 0x7efdf01981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:26.704 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.699+0000 7efdf708d700 1 --2- 172.21.15.52:0/396132740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdf0100cf0 0x7efdf019f770 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:26.704 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.699+0000 7efdf708d700 1 --2- 172.21.15.52:0/396132740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efdf0100360 0x7efdf01986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:26.704 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.699+0000 7efdf708d700 1 -- 172.21.15.52:0/396132740 >> 172.21.15.52:0/396132740 conn(0x7efdf00fa790 msgr2=0x7efdf0068ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:26.704 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.699+0000 7efdf708d700 1 -- 172.21.15.52:0/396132740 shutdown_connections 2023-11-27T15:15:26.704 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:26.699+0000 7efdf708d700 1 -- 172.21.15.52:0/396132740 wait complete. 2023-11-27T15:15:26.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:26 smithi150 ceph-mon[100559]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-11-27T15:15:26.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:26 smithi150 ceph-mon[100559]: osdmap e76: 12 total, 11 up, 12 in 2023-11-27T15:15:26.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:26 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:15:26.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:26 smithi150 ceph-mon[100559]: from='osd.11 [v2:172.21.15.150:6824/3207559739,v1:172.21.15.150:6825/3207559739]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2023-11-27T15:15:26.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:26 smithi150 ceph-mon[100559]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2023-11-27T15:15:26.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:26 smithi150 ceph-mon[100559]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:15:26.780 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:15:26 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:15:26.659+0000 7fa0b7010700 -1 osd.11 0 waiting for initial osdmap 2023-11-27T15:15:26.780 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:15:26 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:15:26.673+0000 7fa0b2e05700 -1 osd.11 77 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:15:27.076 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:26 smithi052 ceph-mon[99056]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-11-27T15:15:27.076 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:26 smithi052 ceph-mon[99056]: osdmap e76: 12 total, 11 up, 12 in 2023-11-27T15:15:27.076 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:26 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:15:27.076 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:26 smithi052 ceph-mon[99056]: from='osd.11 [v2:172.21.15.150:6824/3207559739,v1:172.21.15.150:6825/3207559739]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2023-11-27T15:15:27.077 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:26 smithi052 ceph-mon[99056]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2023-11-27T15:15:27.077 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:26 smithi052 ceph-mon[99056]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:15:27.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:26 smithi102 ceph-mon[100296]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-11-27T15:15:27.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:26 smithi102 ceph-mon[100296]: osdmap e76: 12 total, 11 up, 12 in 2023-11-27T15:15:27.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:26 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:15:27.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:26 smithi102 ceph-mon[100296]: from='osd.11 [v2:172.21.15.150:6824/3207559739,v1:172.21.15.150:6825/3207559739]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2023-11-27T15:15:27.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:26 smithi102 ceph-mon[100296]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2023-11-27T15:15:27.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:26 smithi102 ceph-mon[100296]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:15:27.124 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":77,"num_osds":12,"num_up_osds":11,"osd_up_since":1701098107,"num_in_osds":12,"osd_in_since":1701098110,"num_remapped_pgs":0} 2023-11-27T15:15:28.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:27 smithi150 ceph-mon[100559]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2023-11-27T15:15:28.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:27 smithi150 ceph-mon[100559]: osdmap e77: 12 total, 11 up, 12 in 2023-11-27T15:15:28.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:27 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:15:28.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:27 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:15:28.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:27 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/396132740' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-27T15:15:28.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:27 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:28.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:27 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:28.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:27 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:15:28.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:27 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:15:28.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:27 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:15:28.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:27 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:15:28.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:27 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:28.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:27 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:28.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:27 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:28.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:27 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:28.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:27 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:28.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:27 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:28.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:27 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:28.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:27 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:28.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:27 smithi102 ceph-mon[100296]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2023-11-27T15:15:28.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:27 smithi102 ceph-mon[100296]: osdmap e77: 12 total, 11 up, 12 in 2023-11-27T15:15:28.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:27 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:15:28.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:27 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:15:28.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:27 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/396132740' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-27T15:15:28.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:27 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:28.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:27 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:28.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:27 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:15:28.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:27 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:15:28.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:27 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:15:28.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:27 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:15:28.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:27 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:28.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:27 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:28.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:27 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:28.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:27 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:28.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:27 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:28.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:27 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:28.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:27 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:28.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:27 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:28.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:27 smithi052 ceph-mon[99056]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2023-11-27T15:15:28.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:27 smithi052 ceph-mon[99056]: osdmap e77: 12 total, 11 up, 12 in 2023-11-27T15:15:28.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:27 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:15:28.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:27 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:15:28.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:27 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/396132740' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-27T15:15:28.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:27 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:28.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:27 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:28.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:27 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:15:28.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:27 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:15:28.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:27 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:15:28.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:27 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:15:28.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:27 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:28.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:27 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:28.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:27 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:28.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:27 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:28.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:27 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:28.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:27 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:28.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:27 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:28.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:27 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:28.125 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph osd stat -f json 2023-11-27T15:15:28.793 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:15:29.026 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:28 smithi052 ceph-mon[99056]: purged_snaps scrub starts 2023-11-27T15:15:29.026 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:28 smithi052 ceph-mon[99056]: purged_snaps scrub ok 2023-11-27T15:15:29.026 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:28 smithi052 ceph-mon[99056]: Detected new or changed devices on smithi150 2023-11-27T15:15:29.026 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:28 smithi052 ceph-mon[99056]: Adjusting osd_memory_target on smithi150 to 4292M 2023-11-27T15:15:29.026 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:28 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:15:29.026 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:28 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:15:29.026 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:28 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:15:29.026 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:28 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:15:29.026 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:28 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:15:29.026 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:28 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:15:29.027 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:28 smithi052 ceph-mon[99056]: osd.11 [v2:172.21.15.150:6824/3207559739,v1:172.21.15.150:6825/3207559739] boot 2023-11-27T15:15:29.027 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:28 smithi052 ceph-mon[99056]: osdmap e78: 12 total, 12 up, 12 in 2023-11-27T15:15:29.027 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:28 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:15:29.027 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:28 smithi052 ceph-mon[99056]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:15:29.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:28 smithi150 ceph-mon[100559]: purged_snaps scrub starts 2023-11-27T15:15:29.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:28 smithi150 ceph-mon[100559]: purged_snaps scrub ok 2023-11-27T15:15:29.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:28 smithi150 ceph-mon[100559]: Detected new or changed devices on smithi150 2023-11-27T15:15:29.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:28 smithi150 ceph-mon[100559]: Adjusting osd_memory_target on smithi150 to 4292M 2023-11-27T15:15:29.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:28 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:15:29.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:28 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:15:29.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:28 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:15:29.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:28 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:15:29.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:28 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:15:29.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:28 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:15:29.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:28 smithi150 ceph-mon[100559]: osd.11 [v2:172.21.15.150:6824/3207559739,v1:172.21.15.150:6825/3207559739] boot 2023-11-27T15:15:29.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:28 smithi150 ceph-mon[100559]: osdmap e78: 12 total, 12 up, 12 in 2023-11-27T15:15:29.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:28 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:15:29.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:28 smithi150 ceph-mon[100559]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:15:29.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:28 smithi102 ceph-mon[100296]: purged_snaps scrub starts 2023-11-27T15:15:29.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:28 smithi102 ceph-mon[100296]: purged_snaps scrub ok 2023-11-27T15:15:29.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:28 smithi102 ceph-mon[100296]: Detected new or changed devices on smithi150 2023-11-27T15:15:29.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:28 smithi102 ceph-mon[100296]: Adjusting osd_memory_target on smithi150 to 4292M 2023-11-27T15:15:29.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:28 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:15:29.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:28 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:15:29.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:28 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:15:29.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:28 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:15:29.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:28 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:15:29.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:28 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:15:29.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:28 smithi102 ceph-mon[100296]: osd.11 [v2:172.21.15.150:6824/3207559739,v1:172.21.15.150:6825/3207559739] boot 2023-11-27T15:15:29.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:28 smithi102 ceph-mon[100296]: osdmap e78: 12 total, 12 up, 12 in 2023-11-27T15:15:29.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:28 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:15:29.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:28 smithi102 ceph-mon[100296]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:15:29.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.986+0000 7fd9e8d4c700 1 -- 172.21.15.52:0/2700428821 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9e40ff1e0 msgr2=0x7fd9e40ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:29.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.987+0000 7fd9e8d4c700 1 --2- 172.21.15.52:0/2700428821 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9e40ff1e0 0x7fd9e40ff5f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4009990 tx=0x7fd9d4009ca0 comp rx=0 tx=0).stop 2023-11-27T15:15:29.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.987+0000 7fd9e8d4c700 1 -- 172.21.15.52:0/2700428821 shutdown_connections 2023-11-27T15:15:29.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.987+0000 7fd9e8d4c700 1 --2- 172.21.15.52:0/2700428821 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9e4100d70 0x7fd9e4107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:29.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.987+0000 7fd9e8d4c700 1 --2- 172.21.15.52:0/2700428821 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9e40ff1e0 0x7fd9e40ff5f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:29.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.987+0000 7fd9e8d4c700 1 --2- 172.21.15.52:0/2700428821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9e41003e0 0x7fd9e4100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:29.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.987+0000 7fd9e8d4c700 1 -- 172.21.15.52:0/2700428821 >> 172.21.15.52:0/2700428821 conn(0x7fd9e40fa7f0 msgr2=0x7fd9e40fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:29.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.987+0000 7fd9e8d4c700 1 -- 172.21.15.52:0/2700428821 shutdown_connections 2023-11-27T15:15:29.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.987+0000 7fd9e8d4c700 1 -- 172.21.15.52:0/2700428821 wait complete. 2023-11-27T15:15:29.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.987+0000 7fd9e8d4c700 1 Processor -- start 2023-11-27T15:15:29.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.987+0000 7fd9e8d4c700 1 -- start start 2023-11-27T15:15:29.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.988+0000 7fd9e8d4c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9e40ff1e0 0x7fd9e4069130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:29.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.988+0000 7fd9e8d4c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9e41003e0 0x7fd9e419d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:29.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.988+0000 7fd9e8d4c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9e4100d70 0x7fd9e419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:29.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.988+0000 7fd9e8d4c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e419fe10 con 0x7fd9e41003e0 2023-11-27T15:15:29.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.988+0000 7fd9e8d4c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e419ff80 con 0x7fd9e40ff1e0 2023-11-27T15:15:29.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.988+0000 7fd9e8d4c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e41a0140 con 0x7fd9e4100d70 2023-11-27T15:15:29.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.988+0000 7fd9e1d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9e41003e0 0x7fd9e419d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:29.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.988+0000 7fd9e259c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9e40ff1e0 0x7fd9e4069130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:29.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.988+0000 7fd9e1d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9e41003e0 0x7fd9e419d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49652/0 (socket says 172.21.15.52:49652) 2023-11-27T15:15:29.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.988+0000 7fd9e1d9b700 1 -- 172.21.15.52:0/1601525434 learned_addr learned my addr 172.21.15.52:0/1601525434 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:15:29.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.988+0000 7fd9e2d9d700 1 --2- 172.21.15.52:0/1601525434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9e4100d70 0x7fd9e419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:29.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.988+0000 7fd9e1d9b700 1 -- 172.21.15.52:0/1601525434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9e4100d70 msgr2=0x7fd9e419f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:29.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.988+0000 7fd9e1d9b700 1 --2- 172.21.15.52:0/1601525434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9e4100d70 0x7fd9e419f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:29.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.988+0000 7fd9e1d9b700 1 -- 172.21.15.52:0/1601525434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9e40ff1e0 msgr2=0x7fd9e4069130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:29.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.988+0000 7fd9e1d9b700 1 --2- 172.21.15.52:0/1601525434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9e40ff1e0 0x7fd9e4069130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:29.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.988+0000 7fd9e1d9b700 1 -- 172.21.15.52:0/1601525434 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d40095f0 con 0x7fd9e41003e0 2023-11-27T15:15:29.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.988+0000 7fd9e1d9b700 1 --2- 172.21.15.52:0/1601525434 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9e41003e0 0x7fd9e419d210 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00e9a0 tx=0x7fd9cc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:29.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.989+0000 7fd9db7fe700 1 -- 172.21.15.52:0/1601525434 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc009790 con 0x7fd9e41003e0 2023-11-27T15:15:29.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.989+0000 7fd9e8d4c700 1 -- 172.21.15.52:0/1601525434 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e410ab30 con 0x7fd9e41003e0 2023-11-27T15:15:29.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.989+0000 7fd9db7fe700 1 -- 172.21.15.52:0/1601525434 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9cc00cbd0 con 0x7fd9e41003e0 2023-11-27T15:15:29.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.989+0000 7fd9db7fe700 1 -- 172.21.15.52:0/1601525434 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc018680 con 0x7fd9e41003e0 2023-11-27T15:15:29.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.989+0000 7fd9e8d4c700 1 -- 172.21.15.52:0/1601525434 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e410a9a0 con 0x7fd9e41003e0 2023-11-27T15:15:29.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.990+0000 7fd9e8d4c700 1 -- 172.21.15.52:0/1601525434 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e4106880 con 0x7fd9e41003e0 2023-11-27T15:15:29.997 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.991+0000 7fd9db7fe700 1 -- 172.21.15.52:0/1601525434 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd9cc00cd40 con 0x7fd9e41003e0 2023-11-27T15:15:29.997 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.991+0000 7fd9db7fe700 1 --2- 172.21.15.52:0/1601525434 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd9d00a1150 0x7fd9d00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:29.997 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.992+0000 7fd9e259c700 1 --2- 172.21.15.52:0/1601525434 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd9d00a1150 0x7fd9d00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:29.997 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.992+0000 7fd9db7fe700 1 -- 172.21.15.52:0/1601525434 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd9cc014070 con 0x7fd9e41003e0 2023-11-27T15:15:29.997 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.992+0000 7fd9e259c700 1 --2- 172.21.15.52:0/1601525434 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd9d00a1150 0x7fd9d00a3600 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400b2e0 tx=0x7fd9d4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:29.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:29.995+0000 7fd9db7fe700 1 -- 172.21.15.52:0/1601525434 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd9cc096590 con 0x7fd9e41003e0 2023-11-27T15:15:30.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:29 smithi150 ceph-mon[100559]: osdmap e79: 12 total, 12 up, 12 in 2023-11-27T15:15:30.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:29 smithi102 ceph-mon[100296]: osdmap e79: 12 total, 12 up, 12 in 2023-11-27T15:15:30.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:29 smithi052 ceph-mon[99056]: osdmap e79: 12 total, 12 up, 12 in 2023-11-27T15:15:30.196 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:30.192+0000 7fd9e8d4c700 1 -- 172.21.15.52:0/1601525434 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fd9e4061ad0 con 0x7fd9e41003e0 2023-11-27T15:15:30.196 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:30.192+0000 7fd9db7fe700 1 -- 172.21.15.52:0/1601525434 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) v1 ==== 74+0+133 (secure 0 0 0) 0x7fd9cc094e60 con 0x7fd9e41003e0 2023-11-27T15:15:30.197 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:15:30.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:30.194+0000 7fd9e8d4c700 1 -- 172.21.15.52:0/1601525434 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd9d00a1150 msgr2=0x7fd9d00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:30.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:30.194+0000 7fd9e8d4c700 1 --2- 172.21.15.52:0/1601525434 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd9d00a1150 0x7fd9d00a3600 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400b2e0 tx=0x7fd9d4005fd0 comp rx=0 tx=0).stop 2023-11-27T15:15:30.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:30.194+0000 7fd9e8d4c700 1 -- 172.21.15.52:0/1601525434 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9e41003e0 msgr2=0x7fd9e419d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:30.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:30.194+0000 7fd9e8d4c700 1 --2- 172.21.15.52:0/1601525434 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9e41003e0 0x7fd9e419d210 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00e9a0 tx=0x7fd9cc00ecb0 comp rx=0 tx=0).stop 2023-11-27T15:15:30.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:30.194+0000 7fd9e8d4c700 1 -- 172.21.15.52:0/1601525434 shutdown_connections 2023-11-27T15:15:30.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:30.194+0000 7fd9e8d4c700 1 --2- 172.21.15.52:0/1601525434 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd9d00a1150 0x7fd9d00a3600 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:30.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:30.194+0000 7fd9e8d4c700 1 --2- 172.21.15.52:0/1601525434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9e40ff1e0 0x7fd9e4069130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:30.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:30.194+0000 7fd9e8d4c700 1 --2- 172.21.15.52:0/1601525434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9e4100d70 0x7fd9e419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:30.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:30.194+0000 7fd9e8d4c700 1 --2- 172.21.15.52:0/1601525434 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9e41003e0 0x7fd9e419d210 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:30.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:30.194+0000 7fd9e8d4c700 1 -- 172.21.15.52:0/1601525434 >> 172.21.15.52:0/1601525434 conn(0x7fd9e40fa7f0 msgr2=0x7fd9e41039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:30.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:30.194+0000 7fd9e8d4c700 1 -- 172.21.15.52:0/1601525434 shutdown_connections 2023-11-27T15:15:30.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:30.195+0000 7fd9e8d4c700 1 -- 172.21.15.52:0/1601525434 wait complete. 2023-11-27T15:15:30.554 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":79,"num_osds":12,"num_up_osds":12,"osd_up_since":1701098127,"num_in_osds":12,"osd_in_since":1701098110,"num_remapped_pgs":0} 2023-11-27T15:15:30.554 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph osd dump --format=json 2023-11-27T15:15:30.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:30 smithi052 ceph-mon[99056]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:30.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:30 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1601525434' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-27T15:15:31.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:30 smithi150 ceph-mon[100559]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:31.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:30 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1601525434' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-27T15:15:31.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:30 smithi102 ceph-mon[100296]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:31.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:30 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1601525434' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-27T15:15:31.209 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:15:32.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:31 smithi102 ceph-mon[100296]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:32.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:31 smithi052 ceph-mon[99056]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:32.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:31 smithi150 ceph-mon[100559]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:32.400 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.394+0000 7f07b10e9700 1 -- 172.21.15.52:0/4256555859 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07ac0ffe00 msgr2=0x7f07ac103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:32.401 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.394+0000 7f07b10e9700 1 --2- 172.21.15.52:0/4256555859 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07ac0ffe00 0x7f07ac103dd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f07a00099c0 tx=0x7f07a0009cd0 comp rx=0 tx=0).stop 2023-11-27T15:15:32.401 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.394+0000 7f07b10e9700 1 -- 172.21.15.52:0/4256555859 shutdown_connections 2023-11-27T15:15:32.401 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.394+0000 7f07b10e9700 1 --2- 172.21.15.52:0/4256555859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07ac0ff460 0x7f07ac0ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:32.401 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.394+0000 7f07b10e9700 1 --2- 172.21.15.52:0/4256555859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07ac104500 0x7f07ac10c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:32.401 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.394+0000 7f07b10e9700 1 --2- 172.21.15.52:0/4256555859 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07ac0ffe00 0x7f07ac103dd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:32.401 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.395+0000 7f07b10e9700 1 -- 172.21.15.52:0/4256555859 >> 172.21.15.52:0/4256555859 conn(0x7f07ac0770a0 msgr2=0x7f07ac0774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:32.401 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.395+0000 7f07b10e9700 1 -- 172.21.15.52:0/4256555859 shutdown_connections 2023-11-27T15:15:32.401 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.395+0000 7f07b10e9700 1 -- 172.21.15.52:0/4256555859 wait complete. 2023-11-27T15:15:32.401 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.395+0000 7f07b10e9700 1 Processor -- start 2023-11-27T15:15:32.403 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.396+0000 7f07b10e9700 1 -- start start 2023-11-27T15:15:32.403 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.396+0000 7f07b10e9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07ac0ff460 0x7f07ac1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:32.403 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.396+0000 7f07b10e9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07ac0ffe00 0x7f07ac198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:32.403 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.396+0000 7f07b10e9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07ac104500 0x7f07ac19cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:32.403 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.396+0000 7f07b10e9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac19d5a0 con 0x7f07ac0ffe00 2023-11-27T15:15:32.404 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.396+0000 7f07b10e9700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac19d710 con 0x7f07ac0ff460 2023-11-27T15:15:32.404 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.396+0000 7f07b10e9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac19d8d0 con 0x7f07ac104500 2023-11-27T15:15:32.404 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.396+0000 7f07aa59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07ac0ffe00 0x7f07ac198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:32.404 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.396+0000 7f07aa59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07ac0ffe00 0x7f07ac198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49676/0 (socket says 172.21.15.52:49676) 2023-11-27T15:15:32.404 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.396+0000 7f07aa59c700 1 -- 172.21.15.52:0/410526449 learned_addr learned my addr 172.21.15.52:0/410526449 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:15:32.404 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.396+0000 7f07aad9d700 1 --2- 172.21.15.52:0/410526449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07ac0ff460 0x7f07ac1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:32.404 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.396+0000 7f07ab59e700 1 --2- 172.21.15.52:0/410526449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07ac104500 0x7f07ac19cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:32.404 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.396+0000 7f07aa59c700 1 -- 172.21.15.52:0/410526449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07ac104500 msgr2=0x7f07ac19cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:32.404 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.396+0000 7f07aa59c700 1 --2- 172.21.15.52:0/410526449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07ac104500 0x7f07ac19cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:32.404 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.396+0000 7f07aa59c700 1 -- 172.21.15.52:0/410526449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07ac0ff460 msgr2=0x7f07ac1985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:32.404 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.396+0000 7f07aa59c700 1 --2- 172.21.15.52:0/410526449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07ac0ff460 0x7f07ac1985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:32.404 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.396+0000 7f07aa59c700 1 -- 172.21.15.52:0/410526449 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a00095f0 con 0x7f07ac0ffe00 2023-11-27T15:15:32.404 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.397+0000 7f07aa59c700 1 --2- 172.21.15.52:0/410526449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07ac0ffe00 0x7f07ac198b20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009fd0 tx=0x7f07a0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:32.405 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.397+0000 7f0793fff700 1 -- 172.21.15.52:0/410526449 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a001d070 con 0x7f07ac0ffe00 2023-11-27T15:15:32.405 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.397+0000 7f0793fff700 1 -- 172.21.15.52:0/410526449 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07a0005030 con 0x7f07ac0ffe00 2023-11-27T15:15:32.405 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.397+0000 7f07b10e9700 1 -- 172.21.15.52:0/410526449 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07ac10f230 con 0x7f07ac0ffe00 2023-11-27T15:15:32.405 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.397+0000 7f0793fff700 1 -- 172.21.15.52:0/410526449 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a0004b10 con 0x7f07ac0ffe00 2023-11-27T15:15:32.405 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.397+0000 7f07b10e9700 1 -- 172.21.15.52:0/410526449 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07ac10f0a0 con 0x7f07ac0ffe00 2023-11-27T15:15:32.405 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.398+0000 7f07b10e9700 1 -- 172.21.15.52:0/410526449 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07ac10f3c0 con 0x7f07ac0ffe00 2023-11-27T15:15:32.406 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.399+0000 7f0793fff700 1 -- 172.21.15.52:0/410526449 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f07a00054c0 con 0x7f07ac0ffe00 2023-11-27T15:15:32.406 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.400+0000 7f0793fff700 1 --2- 172.21.15.52:0/410526449 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f07940a1150 0x7f07940a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:32.406 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.400+0000 7f0793fff700 1 -- 172.21.15.52:0/410526449 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f07a00cd620 con 0x7f07ac0ffe00 2023-11-27T15:15:32.406 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.400+0000 7f07aad9d700 1 --2- 172.21.15.52:0/410526449 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f07940a1150 0x7f07940a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:32.406 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.401+0000 7f07aad9d700 1 --2- 172.21.15.52:0/410526449 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f07940a1150 0x7f07940a3600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f079c009c60 tx=0x7f079c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:32.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.403+0000 7f0793fff700 1 -- 172.21.15.52:0/410526449 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f07a00972a0 con 0x7f07ac0ffe00 2023-11-27T15:15:32.605 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.599+0000 7f07b10e9700 1 -- 172.21.15.52:0/410526449 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07ac1a5660 con 0x7f07ac0ffe00 2023-11-27T15:15:32.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.600+0000 7f0793fff700 1 -- 172.21.15.52:0/410526449 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18433 (secure 0 0 0) 0x7f07a009a8c0 con 0x7f07ac0ffe00 2023-11-27T15:15:32.606 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:15:32.606 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":79,"fsid":"c6c7df88-8d36-11ee-95a2-87774f69a715","created":"2023-11-27T15:09:03.333345+0000","modified":"2023-11-27T15:15:28.662216+0000","last_up_change":"2023-11-27T15:15:27.656134+0000","last_in_change":"2023-11-27T15:15:10.671153+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:12:14.177523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58d69ede-c293-49f5-abbe-8187793f2b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6803","nonce":608216415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6805","nonce":608216415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6809","nonce":608216415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6807","nonce":608216415}]},"public_addr":"172.21.15.52:6803/608216415","cluster_addr":"172.21.15.52:6805/608216415","heartbeat_back_addr":"172.21.15.52:6809/608216415","heartbeat_front_addr":"172.21.15.52:6807/608216415","state":["exists","up"]},{"osd":1,"uuid":"d6a46b91-adc8-45ef-80f5-566ae400911f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6801","nonce":2538384087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6803","nonce":2538384087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6807","nonce":2538384087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6805","nonce":2538384087}]},"public_addr":"172.21.15.102:6801/2538384087","cluster_addr":"172.21.15.102:6803/2538384087","heartbeat_back_addr":"172.21.15.102:6807/2538384087","heartbeat_front_addr":"172.21.15.102:6805/2538384087","state":["exists","up"]},{"osd":2,"uuid":"846ba125-11f8-46b2-8941-bd5e1947cc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6801","nonce":4258038221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6803","nonce":4258038221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6807","nonce":4258038221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6805","nonce":4258038221}]},"public_addr":"172.21.15.150:6801/4258038221","cluster_addr":"172.21.15.150:6803/4258038221","heartbeat_back_addr":"172.21.15.150:6807/4258038221","heartbeat_front_addr":"172.21.15.150:6805/4258038221","state":["exists","up"]},{"osd":3,"uuid":"950fcd67-1057-4288-a389-714f1ec94261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6811","nonce":2622100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6813","nonce":2622100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6817","nonce":2622100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6815","nonce":2622100752}]},"public_addr":"172.21.15.52:6811/2622100752","cluster_addr":"172.21.15.52:6813/2622100752","heartbeat_back_addr":"172.21.15.52:6817/2622100752","heartbeat_front_addr":"172.21.15.52:6815/2622100752","state":["exists","up"]},{"osd":4,"uuid":"1c96a676-cea9-4deb-bad1-f7fd240c3d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6809","nonce":1546794763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6811","nonce":1546794763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6815","nonce":1546794763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6813","nonce":1546794763}]},"public_addr":"172.21.15.102:6809/1546794763","cluster_addr":"172.21.15.102:6811/1546794763","heartbeat_back_addr":"172.21.15.102:6815/1546794763","heartbeat_front_addr":"172.21.15.102:6813/1546794763","state":["exists","up"]},{"osd":5,"uuid":"38a75af1-fd4d-433e-b579-0609109c851c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6809","nonce":461511375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6811","nonce":461511375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6815","nonce":461511375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6813","nonce":461511375}]},"public_addr":"172.21.15.150:6809/461511375","cluster_addr":"172.21.15.150:6811/461511375","heartbeat_back_addr":"172.21.15.150:6815/461511375","heartbeat_front_addr":"172.21.15.150:6813/461511375","state":["exists","up"]},{"osd":6,"uuid":"2f0ede4a-d175-4b4a-b876-c0f442048901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6819","nonce":1002457464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6821","nonce":1002457464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6825","nonce":1002457464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6823","nonce":1002457464}]},"public_addr":"172.21.15.52:6819/1002457464","cluster_addr":"172.21.15.52:6821/1002457464","heartbeat_back_addr":"172.21.15.52:6825/1002457464","heartbeat_front_addr":"172.21.15.52:6823/1002457464","state":["exists","up"]},{"osd":7,"uuid":"6b635192-abcf-4bbf-bf78-a019b351bca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6817","nonce":3249840064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6819","nonce":3249840064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6823","nonce":3249840064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6821","nonce":3249840064}]},"public_addr":"172.21.15.102:6817/3249840064","cluster_addr":"172.21.15.102:6819/3249840064","heartbeat_back_addr":"172.21.15.102:6823/3249840064","heartbeat_front_addr":"172.21.15.102:6821/3249840064","state":["exists","up"]},{"osd":8,"uuid":"c7ca8007-7170-47bf-956c-3f00edf27e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6817","nonce":3672583574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6819","nonce":3672583574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6823","nonce":3672583574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6821","nonce":3672583574}]},"public_addr":"172.21.15.150:6817/3672583574","cluster_addr":"172.21.15.150:6819/3672583574","heartbeat_back_addr":"172.21.15.150:6823/3672583574","heartbeat_front_addr":"172.21.15.150:6821/3672583574","state":["exists","up"]},{"osd":9,"uuid":"0e252c2c-01a4-48ba-832d-067ed940fefb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6827","nonce":4287934320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6829","nonce":4287934320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6833","nonce":4287934320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6831","nonce":4287934320}]},"public_addr":"172.21.15.52:6827/4287934320","cluster_addr":"172.21.15.52:6829/4287934320","heartbeat_back_addr":"172.21.15.52:6833/4287934320","heartbeat_front_addr":"172.21.15.52:6831/4287934320","state":["exists","up"]},{"osd":10,"uuid":"bb540142-7563-44a4-b7a3-cb662714c128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6825","nonce":3056034201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6827","nonce":3056034201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6831","nonce":3056034201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6829","nonce":3056034201}]},"public_addr":"172.21.15.102:6825/3056034201","cluster_addr":"172.21.15.102:6827/3056034201","heartbeat_back_addr":"172.21.15.102:6831/3056034201","heartbeat_front_addr":"172.21.15.102:6829/3056034201","state":["exists","up"]},{"osd":11,"uuid":"3156528e-e0f8-4d44-9946-c8b6339c2bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6825","nonce":3207559739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6827","nonce":3207559739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6831","nonce":3207559739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6829","nonce":3207559739}]},"public_addr":"172.21.15.150:6825/3207559739","cluster_addr":"172.21.15.150:6827/3207559739","heartbeat_back_addr":"172.21.15.150:6831/3207559739","heartbeat_front_addr":"172.21.15.150:6829/3207559739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:26.826099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:48.331371+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:10.459738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:32.356472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:54.043170+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:15.648582+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:37.166463+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:59.240819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:20.710097+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:43.364563+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:05.279433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:26.344573+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/1392282110":"2023-11-28T15:10:03.644870+0000","172.21.15.52:0/1479993479":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6800/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/961562652":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/1290665390":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2410529742":"2023-11-28T15:09:37.118690+0000","172.21.15.52:6800/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2130037044":"2023-11-28T15:10:03.644870+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:15:32.608 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.602+0000 7f07b10e9700 1 -- 172.21.15.52:0/410526449 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f07940a1150 msgr2=0x7f07940a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:32.608 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.602+0000 7f07b10e9700 1 --2- 172.21.15.52:0/410526449 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f07940a1150 0x7f07940a3600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f079c009c60 tx=0x7f079c009260 comp rx=0 tx=0).stop 2023-11-27T15:15:32.608 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.603+0000 7f07b10e9700 1 -- 172.21.15.52:0/410526449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07ac0ffe00 msgr2=0x7f07ac198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:32.608 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.603+0000 7f07b10e9700 1 --2- 172.21.15.52:0/410526449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07ac0ffe00 0x7f07ac198b20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009fd0 tx=0x7f07a0009580 comp rx=0 tx=0).stop 2023-11-27T15:15:32.608 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.603+0000 7f07b10e9700 1 -- 172.21.15.52:0/410526449 shutdown_connections 2023-11-27T15:15:32.608 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.603+0000 7f07b10e9700 1 --2- 172.21.15.52:0/410526449 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f07940a1150 0x7f07940a3600 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:32.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.603+0000 7f07b10e9700 1 --2- 172.21.15.52:0/410526449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07ac0ff460 0x7f07ac1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:32.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.603+0000 7f07b10e9700 1 --2- 172.21.15.52:0/410526449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07ac104500 0x7f07ac19cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:32.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.603+0000 7f07b10e9700 1 --2- 172.21.15.52:0/410526449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07ac0ffe00 0x7f07ac198b20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:32.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.603+0000 7f07b10e9700 1 -- 172.21.15.52:0/410526449 >> 172.21.15.52:0/410526449 conn(0x7f07ac0770a0 msgr2=0x7f07ac068f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:32.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.603+0000 7f07b10e9700 1 -- 172.21.15.52:0/410526449 shutdown_connections 2023-11-27T15:15:32.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:32.603+0000 7f07b10e9700 1 -- 172.21.15.52:0/410526449 wait complete. 2023-11-27T15:15:32.950 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:32 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/410526449' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:15:32.961 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:12:14.177523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-27T15:15:32.961 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph osd pool get .mgr pg_num 2023-11-27T15:15:33.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:32 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/410526449' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:15:33.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:32 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/410526449' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:15:33.613 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:15:35.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:34 smithi150 ceph-mon[100559]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:35.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.063+0000 7f0fe1fd8700 1 -- 172.21.15.52:0/4176552353 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0fdc0ffe00 msgr2=0x7f0fdc103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:35.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.063+0000 7f0fe1fd8700 1 --2- 172.21.15.52:0/4176552353 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0fdc0ffe00 0x7f0fdc103dd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0fd00099c0 tx=0x7f0fd0009cd0 comp rx=0 tx=0).stop 2023-11-27T15:15:35.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.063+0000 7f0fe1fd8700 1 -- 172.21.15.52:0/4176552353 shutdown_connections 2023-11-27T15:15:35.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.063+0000 7f0fe1fd8700 1 --2- 172.21.15.52:0/4176552353 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fdc104500 0x7f0fdc10c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:35.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.063+0000 7f0fe1fd8700 1 --2- 172.21.15.52:0/4176552353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fdc0ff460 0x7f0fdc0ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:35.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.063+0000 7f0fe1fd8700 1 --2- 172.21.15.52:0/4176552353 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0fdc0ffe00 0x7f0fdc103dd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:35.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.063+0000 7f0fe1fd8700 1 -- 172.21.15.52:0/4176552353 >> 172.21.15.52:0/4176552353 conn(0x7f0fdc0770a0 msgr2=0x7f0fdc0774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:35.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.063+0000 7f0fe1fd8700 1 -- 172.21.15.52:0/4176552353 shutdown_connections 2023-11-27T15:15:35.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.064+0000 7f0fe1fd8700 1 -- 172.21.15.52:0/4176552353 wait complete. 2023-11-27T15:15:35.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.064+0000 7f0fe1fd8700 1 Processor -- start 2023-11-27T15:15:35.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.064+0000 7f0fe1fd8700 1 -- start start 2023-11-27T15:15:35.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.064+0000 7f0fe1fd8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fdc0ff460 0x7f0fdc196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:35.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.064+0000 7f0fe1fd8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fdc0ffe00 0x7f0fdc196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:35.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.064+0000 7f0fe1fd8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0fdc104500 0x7f0fdc19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:35.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.064+0000 7f0fe1fd8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc19b3c0 con 0x7f0fdc104500 2023-11-27T15:15:35.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.064+0000 7f0fe1fd8700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc19b530 con 0x7f0fdc0ff460 2023-11-27T15:15:35.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.064+0000 7f0fe1fd8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc19b6f0 con 0x7f0fdc0ffe00 2023-11-27T15:15:35.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.065+0000 7f0fdbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0fdc104500 0x7f0fdc19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:35.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.065+0000 7f0fdbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0fdc104500 0x7f0fdc19adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49690/0 (socket says 172.21.15.52:49690) 2023-11-27T15:15:35.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.065+0000 7f0fdbfff700 1 -- 172.21.15.52:0/2513064943 learned_addr learned my addr 172.21.15.52:0/2513064943 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:15:35.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.065+0000 7f0fdaffd700 1 --2- 172.21.15.52:0/2513064943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fdc0ffe00 0x7f0fdc196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:35.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.065+0000 7f0fdb7fe700 1 --2- 172.21.15.52:0/2513064943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fdc0ff460 0x7f0fdc196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:35.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.065+0000 7f0fdbfff700 1 -- 172.21.15.52:0/2513064943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fdc0ffe00 msgr2=0x7f0fdc196940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:35.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.065+0000 7f0fdbfff700 1 --2- 172.21.15.52:0/2513064943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fdc0ffe00 0x7f0fdc196940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:35.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.065+0000 7f0fdbfff700 1 -- 172.21.15.52:0/2513064943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fdc0ff460 msgr2=0x7f0fdc196400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:35.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.065+0000 7f0fdbfff700 1 --2- 172.21.15.52:0/2513064943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fdc0ff460 0x7f0fdc196400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:35.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.065+0000 7f0fdbfff700 1 -- 172.21.15.52:0/2513064943 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd00095f0 con 0x7f0fdc104500 2023-11-27T15:15:35.074 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.065+0000 7f0fdbfff700 1 --2- 172.21.15.52:0/2513064943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0fdc104500 0x7f0fdc19adb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0fd400ada0 tx=0x7f0fd400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:35.074 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.065+0000 7f0fd8ff9700 1 -- 172.21.15.52:0/2513064943 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd400f070 con 0x7f0fdc104500 2023-11-27T15:15:35.074 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.065+0000 7f0fd8ff9700 1 -- 172.21.15.52:0/2513064943 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fd4007b30 con 0x7f0fdc104500 2023-11-27T15:15:35.074 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.065+0000 7f0fe1fd8700 1 -- 172.21.15.52:0/2513064943 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fdc10f230 con 0x7f0fdc104500 2023-11-27T15:15:35.074 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.065+0000 7f0fd8ff9700 1 -- 172.21.15.52:0/2513064943 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4005440 con 0x7f0fdc104500 2023-11-27T15:15:35.074 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.065+0000 7f0fe1fd8700 1 -- 172.21.15.52:0/2513064943 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fdc10f0a0 con 0x7f0fdc104500 2023-11-27T15:15:35.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.066+0000 7f0fe1fd8700 1 -- 172.21.15.52:0/2513064943 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fdc10f3c0 con 0x7f0fdc104500 2023-11-27T15:15:35.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.067+0000 7f0fd8ff9700 1 -- 172.21.15.52:0/2513064943 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0fd4004ad0 con 0x7f0fdc104500 2023-11-27T15:15:35.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.068+0000 7f0fd8ff9700 1 --2- 172.21.15.52:0/2513064943 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0fc40a11d0 0x7f0fc40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:35.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.068+0000 7f0fdb7fe700 1 --2- 172.21.15.52:0/2513064943 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0fc40a11d0 0x7f0fc40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:35.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.068+0000 7f0fd8ff9700 1 -- 172.21.15.52:0/2513064943 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0fd40cc080 con 0x7f0fdc104500 2023-11-27T15:15:35.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.069+0000 7f0fdb7fe700 1 --2- 172.21.15.52:0/2513064943 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0fc40a11d0 0x7f0fc40a3680 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc005950 tx=0x7f0fcc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:35.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.072+0000 7f0fd8ff9700 1 -- 172.21.15.52:0/2513064943 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0fd4095c60 con 0x7f0fdc104500 2023-11-27T15:15:35.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:34 smithi102 ceph-mon[100296]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:35.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:34 smithi052 ceph-mon[99056]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:35.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.266+0000 7f0fe1fd8700 1 -- 172.21.15.52:0/2513064943 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0fdc1a34e0 con 0x7f0fdc104500 2023-11-27T15:15:35.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.266+0000 7f0fd8ff9700 1 -- 172.21.15.52:0/2513064943 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0fd4099280 con 0x7f0fdc104500 2023-11-27T15:15:35.272 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2023-11-27T15:15:35.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.268+0000 7f0fe1fd8700 1 -- 172.21.15.52:0/2513064943 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0fc40a11d0 msgr2=0x7f0fc40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:35.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.268+0000 7f0fe1fd8700 1 --2- 172.21.15.52:0/2513064943 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0fc40a11d0 0x7f0fc40a3680 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc005950 tx=0x7f0fcc0058e0 comp rx=0 tx=0).stop 2023-11-27T15:15:35.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.268+0000 7f0fe1fd8700 1 -- 172.21.15.52:0/2513064943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0fdc104500 msgr2=0x7f0fdc19adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:35.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.268+0000 7f0fe1fd8700 1 --2- 172.21.15.52:0/2513064943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0fdc104500 0x7f0fdc19adb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0fd400ada0 tx=0x7f0fd400c330 comp rx=0 tx=0).stop 2023-11-27T15:15:35.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.268+0000 7f0fe1fd8700 1 -- 172.21.15.52:0/2513064943 shutdown_connections 2023-11-27T15:15:35.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.268+0000 7f0fe1fd8700 1 --2- 172.21.15.52:0/2513064943 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0fc40a11d0 0x7f0fc40a3680 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:35.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.268+0000 7f0fe1fd8700 1 --2- 172.21.15.52:0/2513064943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fdc0ff460 0x7f0fdc196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:35.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.268+0000 7f0fe1fd8700 1 --2- 172.21.15.52:0/2513064943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0fdc104500 0x7f0fdc19adb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:35.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.268+0000 7f0fe1fd8700 1 --2- 172.21.15.52:0/2513064943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fdc0ffe00 0x7f0fdc196940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:35.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.268+0000 7f0fe1fd8700 1 -- 172.21.15.52:0/2513064943 >> 172.21.15.52:0/2513064943 conn(0x7f0fdc0770a0 msgr2=0x7f0fdc0685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:35.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.268+0000 7f0fe1fd8700 1 -- 172.21.15.52:0/2513064943 shutdown_connections 2023-11-27T15:15:35.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:35.268+0000 7f0fe1fd8700 1 -- 172.21.15.52:0/2513064943 wait complete. 2023-11-27T15:15:35.675 INFO:tasks.cephadm:Adding mds.a on smithi052 2023-11-27T15:15:35.676 INFO:tasks.cephadm:Adding mds.d on smithi052 2023-11-27T15:15:35.676 INFO:tasks.cephadm:Adding mds.g on smithi052 2023-11-27T15:15:35.676 INFO:tasks.cephadm:Adding mds.j on smithi052 2023-11-27T15:15:35.676 INFO:tasks.cephadm:Adding mds.b on smithi102 2023-11-27T15:15:35.676 INFO:tasks.cephadm:Adding mds.e on smithi102 2023-11-27T15:15:35.676 INFO:tasks.cephadm:Adding mds.h on smithi102 2023-11-27T15:15:35.676 INFO:tasks.cephadm:Adding mds.k on smithi102 2023-11-27T15:15:35.676 INFO:tasks.cephadm:Adding mds.c on smithi150 2023-11-27T15:15:35.676 INFO:tasks.cephadm:Adding mds.f on smithi150 2023-11-27T15:15:35.676 INFO:tasks.cephadm:Adding mds.i on smithi150 2023-11-27T15:15:35.676 INFO:tasks.cephadm:Adding mds.l on smithi150 2023-11-27T15:15:35.677 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph orch apply mds all '12;smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi102=b;smithi102=e;smithi102=h;smithi102=k;smithi150=c;smithi150=f;smithi150=i;smithi150=l' 2023-11-27T15:15:35.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:35 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2513064943' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:15:36.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:35 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2513064943' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:15:36.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:35 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2513064943' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:15:36.329 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:15:36.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:36 smithi150 ceph-mon[100559]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:37.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:36 smithi102 ceph-mon[100296]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:37.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:36 smithi052 ceph-mon[99056]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:37.374 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.369+0000 7f70a106e700 1 -- 172.21.15.150:0/4227592056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f709c1003e0 msgr2=0x7f709c100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:37.375 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.369+0000 7f70a106e700 1 --2- 172.21.15.150:0/4227592056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f709c1003e0 0x7f709c100830 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7084009990 tx=0x7f7084009ca0 comp rx=0 tx=0).stop 2023-11-27T15:15:37.375 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.369+0000 7f70a106e700 1 -- 172.21.15.150:0/4227592056 shutdown_connections 2023-11-27T15:15:37.375 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.369+0000 7f70a106e700 1 --2- 172.21.15.150:0/4227592056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f709c0ff1e0 0x7f709c0ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:37.375 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.369+0000 7f70a106e700 1 --2- 172.21.15.150:0/4227592056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f709c100d70 0x7f709c107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:37.375 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.369+0000 7f70a106e700 1 --2- 172.21.15.150:0/4227592056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f709c1003e0 0x7f709c100830 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:37.375 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.369+0000 7f70a106e700 1 -- 172.21.15.150:0/4227592056 >> 172.21.15.150:0/4227592056 conn(0x7f709c0fa7f0 msgr2=0x7f709c0fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:37.375 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.369+0000 7f70a106e700 1 -- 172.21.15.150:0/4227592056 shutdown_connections 2023-11-27T15:15:37.375 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.369+0000 7f70a106e700 1 -- 172.21.15.150:0/4227592056 wait complete. 2023-11-27T15:15:37.377 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.370+0000 7f70a106e700 1 Processor -- start 2023-11-27T15:15:37.377 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.371+0000 7f70a106e700 1 -- start start 2023-11-27T15:15:37.377 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.371+0000 7f70a106e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f709c0ff1e0 0x7f709c194f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:37.377 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.371+0000 7f70a106e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f709c1003e0 0x7f709c1954b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:37.377 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.371+0000 7f70a106e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f709c100d70 0x7f709c19c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:37.377 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.371+0000 7f70a106e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c19cb00 con 0x7f709c100d70 2023-11-27T15:15:37.378 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.371+0000 7f70a106e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c19cc40 con 0x7f709c0ff1e0 2023-11-27T15:15:37.378 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.371+0000 7f70a106e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c19cdd0 con 0x7f709c1003e0 2023-11-27T15:15:37.378 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.371+0000 7f709a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f709c1003e0 0x7f709c1954b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:37.378 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.371+0000 7f709a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f709c1003e0 0x7f709c1954b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49848/0 (socket says 172.21.15.150:49848) 2023-11-27T15:15:37.378 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.371+0000 7f709a59c700 1 -- 172.21.15.150:0/965688289 learned_addr learned my addr 172.21.15.150:0/965688289 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-27T15:15:37.378 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.371+0000 7f709ad9d700 1 --2- 172.21.15.150:0/965688289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f709c0ff1e0 0x7f709c194f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:37.378 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.371+0000 7f709b59e700 1 --2- 172.21.15.150:0/965688289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f709c100d70 0x7f709c19c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:37.378 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.371+0000 7f709a59c700 1 -- 172.21.15.150:0/965688289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f709c0ff1e0 msgr2=0x7f709c194f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:37.378 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.371+0000 7f709a59c700 1 --2- 172.21.15.150:0/965688289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f709c0ff1e0 0x7f709c194f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:37.378 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.371+0000 7f709a59c700 1 -- 172.21.15.150:0/965688289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f709c100d70 msgr2=0x7f709c19c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:37.378 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.371+0000 7f709a59c700 1 --2- 172.21.15.150:0/965688289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f709c100d70 0x7f709c19c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:37.378 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.371+0000 7f709a59c700 1 -- 172.21.15.150:0/965688289 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70840095f0 con 0x7f709c1003e0 2023-11-27T15:15:37.379 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.371+0000 7f709b59e700 1 --2- 172.21.15.150:0/965688289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f709c100d70 0x7f709c19c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:15:37.379 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.371+0000 7f709a59c700 1 --2- 172.21.15.150:0/965688289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f709c1003e0 0x7f709c1954b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7084000c00 tx=0x7f7084005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:37.379 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.372+0000 7f7093fff700 1 -- 172.21.15.150:0/965688289 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708401d070 con 0x7f709c1003e0 2023-11-27T15:15:37.379 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.372+0000 7f7093fff700 1 -- 172.21.15.150:0/965688289 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7084004ca0 con 0x7f709c1003e0 2023-11-27T15:15:37.379 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.372+0000 7f70a106e700 1 -- 172.21.15.150:0/965688289 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f709c10ab40 con 0x7f709c1003e0 2023-11-27T15:15:37.379 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.372+0000 7f7093fff700 1 -- 172.21.15.150:0/965688289 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7084005590 con 0x7f709c1003e0 2023-11-27T15:15:37.379 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.372+0000 7f70a106e700 1 -- 172.21.15.150:0/965688289 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f709c10a9b0 con 0x7f709c1003e0 2023-11-27T15:15:37.380 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.373+0000 7f70a106e700 1 -- 172.21.15.150:0/965688289 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f709c1067c0 con 0x7f709c1003e0 2023-11-27T15:15:37.380 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.374+0000 7f7093fff700 1 -- 172.21.15.150:0/965688289 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f708400fc80 con 0x7f709c1003e0 2023-11-27T15:15:37.380 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.374+0000 7f7093fff700 1 --2- 172.21.15.150:0/965688289 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f70880a1150 0x7f70880a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:37.380 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.374+0000 7f7093fff700 1 -- 172.21.15.150:0/965688289 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f70840cd5c0 con 0x7f709c1003e0 2023-11-27T15:15:37.380 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.374+0000 7f709ad9d700 1 --2- 172.21.15.150:0/965688289 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f70880a1150 0x7f70880a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:37.380 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.375+0000 7f709ad9d700 1 --2- 172.21.15.150:0/965688289 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f70880a1150 0x7f70880a3600 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f708c005950 tx=0x7f708c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:37.383 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.378+0000 7f7093fff700 1 -- 172.21.15.150:0/965688289 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7084097240 con 0x7f709c1003e0 2023-11-27T15:15:37.578 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.574+0000 7f70a106e700 1 -- 172.21.15.150:0/965688289 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi102=b;smithi102=e;smithi102=h;smithi102=k;smithi150=c;smithi150=f;smithi150=i;smithi150=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f709c061dd0 con 0x7f70880a1150 2023-11-27T15:15:37.588 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.583+0000 7f7093fff700 1 -- 172.21.15.150:0/965688289 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f709c061dd0 con 0x7f70880a1150 2023-11-27T15:15:37.588 INFO:teuthology.orchestra.run.smithi150.stdout:Scheduled mds.all update... 2023-11-27T15:15:37.590 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.585+0000 7f70a106e700 1 -- 172.21.15.150:0/965688289 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f70880a1150 msgr2=0x7f70880a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:37.590 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.585+0000 7f70a106e700 1 --2- 172.21.15.150:0/965688289 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f70880a1150 0x7f70880a3600 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f708c005950 tx=0x7f708c0058e0 comp rx=0 tx=0).stop 2023-11-27T15:15:37.591 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.585+0000 7f70a106e700 1 -- 172.21.15.150:0/965688289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f709c1003e0 msgr2=0x7f709c1954b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:37.591 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.586+0000 7f70a106e700 1 --2- 172.21.15.150:0/965688289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f709c1003e0 0x7f709c1954b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7084000c00 tx=0x7f7084005ef0 comp rx=0 tx=0).stop 2023-11-27T15:15:37.591 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.586+0000 7f70a106e700 1 -- 172.21.15.150:0/965688289 shutdown_connections 2023-11-27T15:15:37.591 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.586+0000 7f70a106e700 1 --2- 172.21.15.150:0/965688289 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f70880a1150 0x7f70880a3600 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:37.591 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.586+0000 7f70a106e700 1 --2- 172.21.15.150:0/965688289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f709c0ff1e0 0x7f709c194f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:37.591 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.586+0000 7f70a106e700 1 --2- 172.21.15.150:0/965688289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f709c100d70 0x7f709c19c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:37.591 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.586+0000 7f70a106e700 1 --2- 172.21.15.150:0/965688289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f709c1003e0 0x7f709c1954b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:37.591 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.586+0000 7f70a106e700 1 -- 172.21.15.150:0/965688289 >> 172.21.15.150:0/965688289 conn(0x7f709c0fa7f0 msgr2=0x7f709c103900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:37.591 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.586+0000 7f70a106e700 1 -- 172.21.15.150:0/965688289 shutdown_connections 2023-11-27T15:15:37.591 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-27T15:15:37.586+0000 7f70a106e700 1 -- 172.21.15.150:0/965688289 wait complete. 2023-11-27T15:15:37.972 DEBUG:teuthology.orchestra.run.smithi052:mds.a> sudo journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.a.service 2023-11-27T15:15:37.975 DEBUG:teuthology.orchestra.run.smithi052:mds.d> sudo journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.d.service 2023-11-27T15:15:38.012 INFO:journalctl@ceph.mds.a.smithi052.stdout:-- Logs begin at Mon 2023-11-27 14:52:43 UTC. -- 2023-11-27T15:15:38.013 DEBUG:teuthology.orchestra.run.smithi052:mds.g> sudo journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.g.service 2023-11-27T15:15:38.044 INFO:journalctl@ceph.mds.d.smithi052.stdout:-- Logs begin at Mon 2023-11-27 14:52:43 UTC. -- 2023-11-27T15:15:38.045 DEBUG:teuthology.orchestra.run.smithi052:mds.j> sudo journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.j.service 2023-11-27T15:15:38.073 INFO:journalctl@ceph.mds.g.smithi052.stdout:-- Logs begin at Mon 2023-11-27 14:52:43 UTC. -- 2023-11-27T15:15:38.074 DEBUG:teuthology.orchestra.run.smithi102:mds.b> sudo journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.b.service 2023-11-27T15:15:38.077 DEBUG:teuthology.orchestra.run.smithi102:mds.e> sudo journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.e.service 2023-11-27T15:15:38.102 INFO:journalctl@ceph.mds.j.smithi052.stdout:-- Logs begin at Mon 2023-11-27 14:52:43 UTC. -- 2023-11-27T15:15:38.114 INFO:journalctl@ceph.mds.b.smithi102.stdout:-- Logs begin at Mon 2023-11-27 14:52:53 UTC. -- 2023-11-27T15:15:38.115 DEBUG:teuthology.orchestra.run.smithi102:mds.h> sudo journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.h.service 2023-11-27T15:15:38.147 INFO:journalctl@ceph.mds.e.smithi102.stdout:-- Logs begin at Mon 2023-11-27 14:52:53 UTC. -- 2023-11-27T15:15:38.148 DEBUG:teuthology.orchestra.run.smithi102:mds.k> sudo journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.k.service 2023-11-27T15:15:38.180 INFO:journalctl@ceph.mds.h.smithi102.stdout:-- Logs begin at Mon 2023-11-27 14:52:53 UTC. -- 2023-11-27T15:15:38.181 DEBUG:teuthology.orchestra.run.smithi150:mds.c> sudo journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.c.service 2023-11-27T15:15:38.183 DEBUG:teuthology.orchestra.run.smithi150:mds.f> sudo journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.f.service 2023-11-27T15:15:38.211 INFO:journalctl@ceph.mds.c.smithi150.stdout:-- Logs begin at Mon 2023-11-27 14:53:12 UTC. -- 2023-11-27T15:15:38.212 DEBUG:teuthology.orchestra.run.smithi150:mds.i> sudo journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.i.service 2023-11-27T15:15:38.213 INFO:journalctl@ceph.mds.k.smithi102.stdout:-- Logs begin at Mon 2023-11-27 14:52:53 UTC. -- 2023-11-27T15:15:38.238 INFO:journalctl@ceph.mds.f.smithi150.stdout:-- Logs begin at Mon 2023-11-27 14:53:12 UTC. -- 2023-11-27T15:15:38.239 DEBUG:teuthology.orchestra.run.smithi150:mds.l> sudo journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.l.service 2023-11-27T15:15:38.264 INFO:journalctl@ceph.mds.i.smithi150.stdout:-- Logs begin at Mon 2023-11-27 14:53:12 UTC. -- 2023-11-27T15:15:38.265 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-11-27T15:15:38.265 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:15:38.290 INFO:journalctl@ceph.mds.l.smithi150.stdout:-- Logs begin at Mon 2023-11-27 14:53:12 UTC. -- 2023-11-27T15:15:38.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.422+0000 7f8d72f5a700 1 -- 172.21.15.52:0/1771086612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d6c0fe8b0 msgr2=0x7f8d6c0fed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:38.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.422+0000 7f8d72f5a700 1 --2- 172.21.15.52:0/1771086612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d6c0fe8b0 0x7f8d6c0fed00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400b0a0 tx=0x7f8d6400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:15:38.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.422+0000 7f8d72f5a700 1 -- 172.21.15.52:0/1771086612 shutdown_connections 2023-11-27T15:15:38.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.422+0000 7f8d72f5a700 1 --2- 172.21.15.52:0/1771086612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d6c101820 0x7f8d6c101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:38.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.422+0000 7f8d72f5a700 1 --2- 172.21.15.52:0/1771086612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d6c0fe8b0 0x7f8d6c0fed00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:38.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.422+0000 7f8d72f5a700 1 --2- 172.21.15.52:0/1771086612 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d6c102170 0x7f8d6c0fe290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:38.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.422+0000 7f8d72f5a700 1 -- 172.21.15.52:0/1771086612 >> 172.21.15.52:0/1771086612 conn(0x7f8d6c0f9e40 msgr2=0x7f8d6c0fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:38.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.422+0000 7f8d72f5a700 1 -- 172.21.15.52:0/1771086612 shutdown_connections 2023-11-27T15:15:38.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.422+0000 7f8d72f5a700 1 -- 172.21.15.52:0/1771086612 wait complete. 2023-11-27T15:15:38.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.423+0000 7f8d72f5a700 1 Processor -- start 2023-11-27T15:15:38.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.423+0000 7f8d72f5a700 1 -- start start 2023-11-27T15:15:38.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.423+0000 7f8d72f5a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d6c0fe8b0 0x7f8d6c196040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:38.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.423+0000 7f8d72f5a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d6c101820 0x7f8d6c196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:38.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.423+0000 7f8d72f5a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d6c102170 0x7f8d6c19d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:38.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.423+0000 7f8d72f5a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6c19dbc0 con 0x7f8d6c101820 2023-11-27T15:15:38.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.423+0000 7f8d72f5a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6c19dd00 con 0x7f8d6c102170 2023-11-27T15:15:38.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.423+0000 7f8d72f5a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6c19de90 con 0x7f8d6c0fe8b0 2023-11-27T15:15:38.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.423+0000 7f8d6bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d6c101820 0x7f8d6c196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:38.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.423+0000 7f8d70cf6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d6c0fe8b0 0x7f8d6c196040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:38.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.423+0000 7f8d6bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d6c101820 0x7f8d6c196580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49700/0 (socket says 172.21.15.52:49700) 2023-11-27T15:15:38.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.423+0000 7f8d6bfff700 1 -- 172.21.15.52:0/2220878435 learned_addr learned my addr 172.21.15.52:0/2220878435 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:15:38.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.423+0000 7f8d714f7700 1 --2- 172.21.15.52:0/2220878435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d6c102170 0x7f8d6c19d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:38.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.423+0000 7f8d70cf6700 1 -- 172.21.15.52:0/2220878435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d6c102170 msgr2=0x7f8d6c19d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:38.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.423+0000 7f8d70cf6700 1 --2- 172.21.15.52:0/2220878435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d6c102170 0x7f8d6c19d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:38.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.423+0000 7f8d70cf6700 1 -- 172.21.15.52:0/2220878435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d6c101820 msgr2=0x7f8d6c196580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:38.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.423+0000 7f8d70cf6700 1 --2- 172.21.15.52:0/2220878435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d6c101820 0x7f8d6c196580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:38.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.423+0000 7f8d70cf6700 1 -- 172.21.15.52:0/2220878435 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d64009d00 con 0x7f8d6c0fe8b0 2023-11-27T15:15:38.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.423+0000 7f8d6bfff700 1 --2- 172.21.15.52:0/2220878435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d6c101820 0x7f8d6c196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:15:38.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.423+0000 7f8d70cf6700 1 --2- 172.21.15.52:0/2220878435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d6c0fe8b0 0x7f8d6c196040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8d6000d950 tx=0x7f8d6000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:38.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.424+0000 7f8d69ffb700 1 -- 172.21.15.52:0/2220878435 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6000f890 con 0x7f8d6c0fe8b0 2023-11-27T15:15:38.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.424+0000 7f8d69ffb700 1 -- 172.21.15.52:0/2220878435 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d60010460 con 0x7f8d6c0fe8b0 2023-11-27T15:15:38.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.424+0000 7f8d72f5a700 1 -- 172.21.15.52:0/2220878435 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d6c10bba0 con 0x7f8d6c0fe8b0 2023-11-27T15:15:38.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.424+0000 7f8d69ffb700 1 -- 172.21.15.52:0/2220878435 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6000e610 con 0x7f8d6c0fe8b0 2023-11-27T15:15:38.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.424+0000 7f8d72f5a700 1 -- 172.21.15.52:0/2220878435 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d6c10ba10 con 0x7f8d6c0fe8b0 2023-11-27T15:15:38.433 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.425+0000 7f8d72f5a700 1 -- 172.21.15.52:0/2220878435 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d6c106f90 con 0x7f8d6c0fe8b0 2023-11-27T15:15:38.433 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.426+0000 7f8d69ffb700 1 -- 172.21.15.52:0/2220878435 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8d600108d0 con 0x7f8d6c0fe8b0 2023-11-27T15:15:38.433 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.426+0000 7f8d69ffb700 1 --2- 172.21.15.52:0/2220878435 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f8d5c0a1150 0x7f8d5c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:38.433 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.427+0000 7f8d6bfff700 1 --2- 172.21.15.52:0/2220878435 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f8d5c0a1150 0x7f8d5c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:38.434 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.427+0000 7f8d69ffb700 1 -- 172.21.15.52:0/2220878435 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8d600cc1f0 con 0x7f8d6c0fe8b0 2023-11-27T15:15:38.434 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.427+0000 7f8d6bfff700 1 --2- 172.21.15.52:0/2220878435 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f8d5c0a1150 0x7f8d5c0a3600 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8d58005950 tx=0x7f8d5800b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:38.435 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.430+0000 7f8d69ffb700 1 -- 172.21.15.52:0/2220878435 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8d60095de0 con 0x7f8d6c0fe8b0 2023-11-27T15:15:38.638 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.633+0000 7f8d72f5a700 1 -- 172.21.15.52:0/2220878435 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d6c02cfe0 con 0x7f8d6c0fe8b0 2023-11-27T15:15:38.640 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.635+0000 7f8d69ffb700 1 -- 172.21.15.52:0/2220878435 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18433 (secure 0 0 0) 0x7f8d60099400 con 0x7f8d6c0fe8b0 2023-11-27T15:15:38.640 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:15:38.640 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":79,"fsid":"c6c7df88-8d36-11ee-95a2-87774f69a715","created":"2023-11-27T15:09:03.333345+0000","modified":"2023-11-27T15:15:28.662216+0000","last_up_change":"2023-11-27T15:15:27.656134+0000","last_in_change":"2023-11-27T15:15:10.671153+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:12:14.177523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58d69ede-c293-49f5-abbe-8187793f2b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6803","nonce":608216415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6805","nonce":608216415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6809","nonce":608216415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6807","nonce":608216415}]},"public_addr":"172.21.15.52:6803/608216415","cluster_addr":"172.21.15.52:6805/608216415","heartbeat_back_addr":"172.21.15.52:6809/608216415","heartbeat_front_addr":"172.21.15.52:6807/608216415","state":["exists","up"]},{"osd":1,"uuid":"d6a46b91-adc8-45ef-80f5-566ae400911f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6801","nonce":2538384087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6803","nonce":2538384087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6807","nonce":2538384087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6805","nonce":2538384087}]},"public_addr":"172.21.15.102:6801/2538384087","cluster_addr":"172.21.15.102:6803/2538384087","heartbeat_back_addr":"172.21.15.102:6807/2538384087","heartbeat_front_addr":"172.21.15.102:6805/2538384087","state":["exists","up"]},{"osd":2,"uuid":"846ba125-11f8-46b2-8941-bd5e1947cc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6801","nonce":4258038221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6803","nonce":4258038221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6807","nonce":4258038221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6805","nonce":4258038221}]},"public_addr":"172.21.15.150:6801/4258038221","cluster_addr":"172.21.15.150:6803/4258038221","heartbeat_back_addr":"172.21.15.150:6807/4258038221","heartbeat_front_addr":"172.21.15.150:6805/4258038221","state":["exists","up"]},{"osd":3,"uuid":"950fcd67-1057-4288-a389-714f1ec94261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6811","nonce":2622100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6813","nonce":2622100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6817","nonce":2622100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6815","nonce":2622100752}]},"public_addr":"172.21.15.52:6811/2622100752","cluster_addr":"172.21.15.52:6813/2622100752","heartbeat_back_addr":"172.21.15.52:6817/2622100752","heartbeat_front_addr":"172.21.15.52:6815/2622100752","state":["exists","up"]},{"osd":4,"uuid":"1c96a676-cea9-4deb-bad1-f7fd240c3d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6809","nonce":1546794763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6811","nonce":1546794763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6815","nonce":1546794763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6813","nonce":1546794763}]},"public_addr":"172.21.15.102:6809/1546794763","cluster_addr":"172.21.15.102:6811/1546794763","heartbeat_back_addr":"172.21.15.102:6815/1546794763","heartbeat_front_addr":"172.21.15.102:6813/1546794763","state":["exists","up"]},{"osd":5,"uuid":"38a75af1-fd4d-433e-b579-0609109c851c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6809","nonce":461511375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6811","nonce":461511375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6815","nonce":461511375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6813","nonce":461511375}]},"public_addr":"172.21.15.150:6809/461511375","cluster_addr":"172.21.15.150:6811/461511375","heartbeat_back_addr":"172.21.15.150:6815/461511375","heartbeat_front_addr":"172.21.15.150:6813/461511375","state":["exists","up"]},{"osd":6,"uuid":"2f0ede4a-d175-4b4a-b876-c0f442048901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6819","nonce":1002457464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6821","nonce":1002457464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6825","nonce":1002457464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6823","nonce":1002457464}]},"public_addr":"172.21.15.52:6819/1002457464","cluster_addr":"172.21.15.52:6821/1002457464","heartbeat_back_addr":"172.21.15.52:6825/1002457464","heartbeat_front_addr":"172.21.15.52:6823/1002457464","state":["exists","up"]},{"osd":7,"uuid":"6b635192-abcf-4bbf-bf78-a019b351bca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6817","nonce":3249840064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6819","nonce":3249840064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6823","nonce":3249840064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6821","nonce":3249840064}]},"public_addr":"172.21.15.102:6817/3249840064","cluster_addr":"172.21.15.102:6819/3249840064","heartbeat_back_addr":"172.21.15.102:6823/3249840064","heartbeat_front_addr":"172.21.15.102:6821/3249840064","state":["exists","up"]},{"osd":8,"uuid":"c7ca8007-7170-47bf-956c-3f00edf27e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6817","nonce":3672583574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6819","nonce":3672583574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6823","nonce":3672583574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6821","nonce":3672583574}]},"public_addr":"172.21.15.150:6817/3672583574","cluster_addr":"172.21.15.150:6819/3672583574","heartbeat_back_addr":"172.21.15.150:6823/3672583574","heartbeat_front_addr":"172.21.15.150:6821/3672583574","state":["exists","up"]},{"osd":9,"uuid":"0e252c2c-01a4-48ba-832d-067ed940fefb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6827","nonce":4287934320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6829","nonce":4287934320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6833","nonce":4287934320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6831","nonce":4287934320}]},"public_addr":"172.21.15.52:6827/4287934320","cluster_addr":"172.21.15.52:6829/4287934320","heartbeat_back_addr":"172.21.15.52:6833/4287934320","heartbeat_front_addr":"172.21.15.52:6831/4287934320","state":["exists","up"]},{"osd":10,"uuid":"bb540142-7563-44a4-b7a3-cb662714c128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6825","nonce":3056034201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6827","nonce":3056034201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6831","nonce":3056034201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6829","nonce":3056034201}]},"public_addr":"172.21.15.102:6825/3056034201","cluster_addr":"172.21.15.102:6827/3056034201","heartbeat_back_addr":"172.21.15.102:6831/3056034201","heartbeat_front_addr":"172.21.15.102:6829/3056034201","state":["exists","up"]},{"osd":11,"uuid":"3156528e-e0f8-4d44-9946-c8b6339c2bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6825","nonce":3207559739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6827","nonce":3207559739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6831","nonce":3207559739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6829","nonce":3207559739}]},"public_addr":"172.21.15.150:6825/3207559739","cluster_addr":"172.21.15.150:6827/3207559739","heartbeat_back_addr":"172.21.15.150:6831/3207559739","heartbeat_front_addr":"172.21.15.150:6829/3207559739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:26.826099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:48.331371+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:10.459738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:32.356472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:54.043170+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:15.648582+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:37.166463+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:59.240819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:20.710097+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:43.364563+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:05.279433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:26.344573+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/1392282110":"2023-11-28T15:10:03.644870+0000","172.21.15.52:0/1479993479":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6800/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/961562652":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/1290665390":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2410529742":"2023-11-28T15:09:37.118690+0000","172.21.15.52:6800/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2130037044":"2023-11-28T15:10:03.644870+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:15:38.642 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.636+0000 7f8d72f5a700 1 -- 172.21.15.52:0/2220878435 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f8d5c0a1150 msgr2=0x7f8d5c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:38.642 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.636+0000 7f8d72f5a700 1 --2- 172.21.15.52:0/2220878435 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f8d5c0a1150 0x7f8d5c0a3600 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8d58005950 tx=0x7f8d5800b500 comp rx=0 tx=0).stop 2023-11-27T15:15:38.642 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.636+0000 7f8d72f5a700 1 -- 172.21.15.52:0/2220878435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d6c0fe8b0 msgr2=0x7f8d6c196040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:38.642 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.636+0000 7f8d72f5a700 1 --2- 172.21.15.52:0/2220878435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d6c0fe8b0 0x7f8d6c196040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8d6000d950 tx=0x7f8d6000dc60 comp rx=0 tx=0).stop 2023-11-27T15:15:38.642 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.636+0000 7f8d72f5a700 1 -- 172.21.15.52:0/2220878435 shutdown_connections 2023-11-27T15:15:38.642 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.636+0000 7f8d72f5a700 1 --2- 172.21.15.52:0/2220878435 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f8d5c0a1150 0x7f8d5c0a3600 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:38.642 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.636+0000 7f8d72f5a700 1 --2- 172.21.15.52:0/2220878435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d6c102170 0x7f8d6c19d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:38.642 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.636+0000 7f8d72f5a700 1 --2- 172.21.15.52:0/2220878435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d6c0fe8b0 0x7f8d6c196040 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:38.643 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.636+0000 7f8d72f5a700 1 --2- 172.21.15.52:0/2220878435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d6c101820 0x7f8d6c196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:38.643 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.636+0000 7f8d72f5a700 1 -- 172.21.15.52:0/2220878435 >> 172.21.15.52:0/2220878435 conn(0x7f8d6c0f9e40 msgr2=0x7f8d6c0fbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:38.643 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.636+0000 7f8d72f5a700 1 -- 172.21.15.52:0/2220878435 shutdown_connections 2023-11-27T15:15:38.643 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.636+0000 7f8d72f5a700 1 -- 172.21.15.52:0/2220878435 wait complete. 2023-11-27T15:15:38.655 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:12:14.177523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-27T15:15:38.656 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:15:38.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:38 smithi150 ceph-mon[100559]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi102=b;smithi102=e;smithi102=h;smithi102=k;smithi150=c;smithi150=f;smithi150=i;smithi150=l", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:15:38.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:38 smithi150 ceph-mon[100559]: Saving service mds.all spec with placement smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi102=b;smithi102=e;smithi102=h;smithi102=k;smithi150=c;smithi150=f;smithi150=i;smithi150=l;count:12 2023-11-27T15:15:38.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:38 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:38.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:38 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:38.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:38 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:38.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:38 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:38.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:38 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:15:38.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:38 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:38.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:38 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:15:38.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:38 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:15:38.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:38 smithi150 ceph-mon[100559]: Service mds.all has no related daemon already placed. 2023-11-27T15:15:38.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:38 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:15:38.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:38 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:15:38.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:38 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:38.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:38 smithi150 ceph-mon[100559]: Deploying daemon mds.l on smithi150 2023-11-27T15:15:38.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:38 smithi150 ceph-mon[100559]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:38.784 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.779+0000 7f084767c700 1 -- 172.21.15.52:0/1689629919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08401003d0 msgr2=0x7f0840100820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:38.784 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.779+0000 7f084767c700 1 --2- 172.21.15.52:0/1689629919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08401003d0 0x7f0840100820 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0830009990 tx=0x7f0830009ca0 comp rx=0 tx=0).stop 2023-11-27T15:15:38.784 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.779+0000 7f084767c700 1 -- 172.21.15.52:0/1689629919 shutdown_connections 2023-11-27T15:15:38.784 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.779+0000 7f084767c700 1 --2- 172.21.15.52:0/1689629919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0840100d60 0x7f0840107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:38.784 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.779+0000 7f084767c700 1 --2- 172.21.15.52:0/1689629919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08400ff1e0 0x7f08400ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:38.785 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.779+0000 7f084767c700 1 --2- 172.21.15.52:0/1689629919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08401003d0 0x7f0840100820 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:38.785 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.779+0000 7f084767c700 1 -- 172.21.15.52:0/1689629919 >> 172.21.15.52:0/1689629919 conn(0x7f08400fa800 msgr2=0x7f08400fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:38.785 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.779+0000 7f084767c700 1 -- 172.21.15.52:0/1689629919 shutdown_connections 2023-11-27T15:15:38.785 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.779+0000 7f084767c700 1 -- 172.21.15.52:0/1689629919 wait complete. 2023-11-27T15:15:38.787 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.779+0000 7f084767c700 1 Processor -- start 2023-11-27T15:15:38.787 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.779+0000 7f084767c700 1 -- start start 2023-11-27T15:15:38.787 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.780+0000 7f084767c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08400ff1e0 0x7f0840196110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:38.787 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.780+0000 7f084767c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08401003d0 0x7f0840196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:38.787 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.780+0000 7f084767c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0840100d60 0x7f084019d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:38.787 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.780+0000 7f084767c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084019dc90 con 0x7f08401003d0 2023-11-27T15:15:38.787 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.780+0000 7f084767c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084019ddd0 con 0x7f0840100d60 2023-11-27T15:15:38.787 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.780+0000 7f084767c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084019df60 con 0x7f08400ff1e0 2023-11-27T15:15:38.787 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.780+0000 7f0844c17700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08401003d0 0x7f0840196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:38.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.780+0000 7f0844c17700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08401003d0 0x7f0840196650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49714/0 (socket says 172.21.15.52:49714) 2023-11-27T15:15:38.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.780+0000 7f0844c17700 1 -- 172.21.15.52:0/1723851616 learned_addr learned my addr 172.21.15.52:0/1723851616 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:15:38.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.780+0000 7f0845418700 1 --2- 172.21.15.52:0/1723851616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08400ff1e0 0x7f0840196110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:38.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.780+0000 7f0845c19700 1 --2- 172.21.15.52:0/1723851616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0840100d60 0x7f084019d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:38.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.780+0000 7f0844c17700 1 -- 172.21.15.52:0/1723851616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08400ff1e0 msgr2=0x7f0840196110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:38.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.780+0000 7f0844c17700 1 --2- 172.21.15.52:0/1723851616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08400ff1e0 0x7f0840196110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:38.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.780+0000 7f0844c17700 1 -- 172.21.15.52:0/1723851616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0840100d60 msgr2=0x7f084019d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:38.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.780+0000 7f0844c17700 1 --2- 172.21.15.52:0/1723851616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0840100d60 0x7f084019d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:38.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.780+0000 7f0844c17700 1 -- 172.21.15.52:0/1723851616 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08300095f0 con 0x7f08401003d0 2023-11-27T15:15:38.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.780+0000 7f0845c19700 1 --2- 172.21.15.52:0/1723851616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0840100d60 0x7f084019d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:15:38.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.780+0000 7f0844c17700 1 --2- 172.21.15.52:0/1723851616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08401003d0 0x7f0840196650 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0830000c00 tx=0x7f08300047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:38.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.780+0000 7f08367fc700 1 -- 172.21.15.52:0/1723851616 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083001d070 con 0x7f08401003d0 2023-11-27T15:15:38.789 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.780+0000 7f084767c700 1 -- 172.21.15.52:0/1723851616 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f084010bd10 con 0x7f08401003d0 2023-11-27T15:15:38.789 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.780+0000 7f08367fc700 1 -- 172.21.15.52:0/1723851616 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0830022470 con 0x7f08401003d0 2023-11-27T15:15:38.789 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.780+0000 7f08367fc700 1 -- 172.21.15.52:0/1723851616 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0830017650 con 0x7f08401003d0 2023-11-27T15:15:38.789 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.780+0000 7f084767c700 1 -- 172.21.15.52:0/1723851616 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f084010bb80 con 0x7f08401003d0 2023-11-27T15:15:38.789 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.781+0000 7f084767c700 1 -- 172.21.15.52:0/1723851616 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08401067d0 con 0x7f08401003d0 2023-11-27T15:15:38.789 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.782+0000 7f08367fc700 1 -- 172.21.15.52:0/1723851616 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f08300225e0 con 0x7f08401003d0 2023-11-27T15:15:38.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.783+0000 7f08367fc700 1 --2- 172.21.15.52:0/1723851616 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f082c0a1150 0x7f082c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:38.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.783+0000 7f08367fc700 1 -- 172.21.15.52:0/1723851616 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f08300cda90 con 0x7f08401003d0 2023-11-27T15:15:38.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.783+0000 7f0845418700 1 --2- 172.21.15.52:0/1723851616 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f082c0a1150 0x7f082c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:38.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.783+0000 7f0845418700 1 --2- 172.21.15.52:0/1723851616 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f082c0a1150 0x7f082c0a3600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0840100230 tx=0x7f083c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:38.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.787+0000 7f08367fc700 1 -- 172.21.15.52:0/1723851616 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0830097710 con 0x7f08401003d0 2023-11-27T15:15:38.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:38 smithi052 ceph-mon[99056]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi102=b;smithi102=e;smithi102=h;smithi102=k;smithi150=c;smithi150=f;smithi150=i;smithi150=l", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:15:38.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:38 smithi052 ceph-mon[99056]: Saving service mds.all spec with placement smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi102=b;smithi102=e;smithi102=h;smithi102=k;smithi150=c;smithi150=f;smithi150=i;smithi150=l;count:12 2023-11-27T15:15:38.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:38 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:38.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:38 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:38.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:38 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:38.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:38 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:38.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:38 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:15:38.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:38 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:38.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:38 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:15:38.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:38 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:15:38.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:38 smithi052 ceph-mon[99056]: Service mds.all has no related daemon already placed. 2023-11-27T15:15:38.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:38 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:15:38.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:38 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:15:38.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:38 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:38.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:38 smithi052 ceph-mon[99056]: Deploying daemon mds.l on smithi150 2023-11-27T15:15:38.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:38 smithi052 ceph-mon[99056]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:38.991 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.986+0000 7f084767c700 1 -- 172.21.15.52:0/1723851616 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0840067f50 con 0x7f08401003d0 2023-11-27T15:15:38.991 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.986+0000 7f08367fc700 1 -- 172.21.15.52:0/1723851616 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f083009ad30 con 0x7f08401003d0 2023-11-27T15:15:38.992 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2023-11-27T15:15:38.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.988+0000 7f084767c700 1 -- 172.21.15.52:0/1723851616 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f082c0a1150 msgr2=0x7f082c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:38.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.988+0000 7f084767c700 1 --2- 172.21.15.52:0/1723851616 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f082c0a1150 0x7f082c0a3600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0840100230 tx=0x7f083c009260 comp rx=0 tx=0).stop 2023-11-27T15:15:38.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.988+0000 7f084767c700 1 -- 172.21.15.52:0/1723851616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08401003d0 msgr2=0x7f0840196650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:38.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.988+0000 7f084767c700 1 --2- 172.21.15.52:0/1723851616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08401003d0 0x7f0840196650 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0830000c00 tx=0x7f08300047a0 comp rx=0 tx=0).stop 2023-11-27T15:15:38.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.988+0000 7f084767c700 1 -- 172.21.15.52:0/1723851616 shutdown_connections 2023-11-27T15:15:38.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.988+0000 7f084767c700 1 --2- 172.21.15.52:0/1723851616 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f082c0a1150 0x7f082c0a3600 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:38.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.988+0000 7f084767c700 1 --2- 172.21.15.52:0/1723851616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0840100d60 0x7f084019d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:38.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.988+0000 7f084767c700 1 --2- 172.21.15.52:0/1723851616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08400ff1e0 0x7f0840196110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:38.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.988+0000 7f084767c700 1 --2- 172.21.15.52:0/1723851616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08401003d0 0x7f0840196650 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:38.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.988+0000 7f084767c700 1 -- 172.21.15.52:0/1723851616 >> 172.21.15.52:0/1723851616 conn(0x7f08400fa800 msgr2=0x7f0840103910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:38.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.988+0000 7f084767c700 1 -- 172.21.15.52:0/1723851616 shutdown_connections 2023-11-27T15:15:38.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:38.988+0000 7f084767c700 1 -- 172.21.15.52:0/1723851616 wait complete. 2023-11-27T15:15:39.006 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:15:39.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:38 smithi102 ceph-mon[100296]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi102=b;smithi102=e;smithi102=h;smithi102=k;smithi150=c;smithi150=f;smithi150=i;smithi150=l", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:15:39.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:38 smithi102 ceph-mon[100296]: Saving service mds.all spec with placement smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi102=b;smithi102=e;smithi102=h;smithi102=k;smithi150=c;smithi150=f;smithi150=i;smithi150=l;count:12 2023-11-27T15:15:39.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:38 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:39.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:38 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:39.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:38 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:39.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:38 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:39.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:38 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:15:39.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:38 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:39.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:38 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:15:39.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:38 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:15:39.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:38 smithi102 ceph-mon[100296]: Service mds.all has no related daemon already placed. 2023-11-27T15:15:39.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:38 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:15:39.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:38 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:15:39.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:38 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:39.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:38 smithi102 ceph-mon[100296]: Deploying daemon mds.l on smithi150 2023-11-27T15:15:39.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:38 smithi102 ceph-mon[100296]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:39.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.126+0000 7f15104a8700 1 -- 172.21.15.52:0/568038425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15080ff170 msgr2=0x7f15080ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:39.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.126+0000 7f15104a8700 1 --2- 172.21.15.52:0/568038425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15080ff170 0x7f15080ff580 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1504009990 tx=0x7f1504009ca0 comp rx=0 tx=0).stop 2023-11-27T15:15:39.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.126+0000 7f15104a8700 1 -- 172.21.15.52:0/568038425 shutdown_connections 2023-11-27T15:15:39.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.126+0000 7f15104a8700 1 --2- 172.21.15.52:0/568038425 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1508100db0 0x7f1508107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:39.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.126+0000 7f15104a8700 1 --2- 172.21.15.52:0/568038425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15080ff170 0x7f15080ff580 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:39.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.126+0000 7f15104a8700 1 --2- 172.21.15.52:0/568038425 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1508100400 0x7f1508100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:39.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.126+0000 7f15104a8700 1 -- 172.21.15.52:0/568038425 >> 172.21.15.52:0/568038425 conn(0x7f15080fa750 msgr2=0x7f15080fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:39.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.126+0000 7f15104a8700 1 -- 172.21.15.52:0/568038425 shutdown_connections 2023-11-27T15:15:39.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.126+0000 7f15104a8700 1 -- 172.21.15.52:0/568038425 wait complete. 2023-11-27T15:15:39.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.127+0000 7f15104a8700 1 Processor -- start 2023-11-27T15:15:39.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.127+0000 7f15104a8700 1 -- start start 2023-11-27T15:15:39.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.127+0000 7f15104a8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15080ff170 0x7f1508068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:39.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.127+0000 7f15104a8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1508100400 0x7f150819ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:39.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.127+0000 7f15104a8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1508100db0 0x7f150819d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:39.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.127+0000 7f15104a8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150819daa0 con 0x7f15080ff170 2023-11-27T15:15:39.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.127+0000 7f15104a8700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150819dc10 con 0x7f1508100400 2023-11-27T15:15:39.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.127+0000 7f15104a8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150819ddd0 con 0x7f1508100db0 2023-11-27T15:15:39.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.127+0000 7f150e244700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15080ff170 0x7f1508068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:39.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.127+0000 7f150e244700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15080ff170 0x7f1508068960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49726/0 (socket says 172.21.15.52:49726) 2023-11-27T15:15:39.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.127+0000 7f150e244700 1 -- 172.21.15.52:0/2879424076 learned_addr learned my addr 172.21.15.52:0/2879424076 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:15:39.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.127+0000 7f150ea45700 1 --2- 172.21.15.52:0/2879424076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1508100db0 0x7f150819d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:39.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.127+0000 7f150da43700 1 --2- 172.21.15.52:0/2879424076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1508100400 0x7f150819ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:39.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.127+0000 7f150e244700 1 -- 172.21.15.52:0/2879424076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1508100db0 msgr2=0x7f150819d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:39.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.127+0000 7f150e244700 1 --2- 172.21.15.52:0/2879424076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1508100db0 0x7f150819d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:39.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.127+0000 7f150e244700 1 -- 172.21.15.52:0/2879424076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1508100400 msgr2=0x7f150819ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:39.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.127+0000 7f150e244700 1 --2- 172.21.15.52:0/2879424076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1508100400 0x7f150819ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:39.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.127+0000 7f150e244700 1 -- 172.21.15.52:0/2879424076 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15040095f0 con 0x7f15080ff170 2023-11-27T15:15:39.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.127+0000 7f150e244700 1 --2- 172.21.15.52:0/2879424076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15080ff170 0x7f1508068960 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f15040059d0 tx=0x7f15040047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:39.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.128+0000 7f14fb7fe700 1 -- 172.21.15.52:0/2879424076 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150401d070 con 0x7f15080ff170 2023-11-27T15:15:39.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.128+0000 7f14fb7fe700 1 -- 172.21.15.52:0/2879424076 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f150400fb90 con 0x7f15080ff170 2023-11-27T15:15:39.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.128+0000 7f15104a8700 1 -- 172.21.15.52:0/2879424076 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f150810ab10 con 0x7f15080ff170 2023-11-27T15:15:39.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.128+0000 7f14fb7fe700 1 -- 172.21.15.52:0/2879424076 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1504017650 con 0x7f15080ff170 2023-11-27T15:15:39.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.128+0000 7f15104a8700 1 -- 172.21.15.52:0/2879424076 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f150810a980 con 0x7f15080ff170 2023-11-27T15:15:39.137 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.129+0000 7f15104a8700 1 -- 172.21.15.52:0/2879424076 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f150819e3d0 con 0x7f15080ff170 2023-11-27T15:15:39.137 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.130+0000 7f14fb7fe700 1 -- 172.21.15.52:0/2879424076 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f150400fd00 con 0x7f15080ff170 2023-11-27T15:15:39.137 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.130+0000 7f14fb7fe700 1 --2- 172.21.15.52:0/2879424076 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f14f40a11d0 0x7f14f40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:39.137 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.130+0000 7f14fb7fe700 1 -- 172.21.15.52:0/2879424076 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f15040cdf70 con 0x7f15080ff170 2023-11-27T15:15:39.137 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.130+0000 7f150da43700 1 --2- 172.21.15.52:0/2879424076 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f14f40a11d0 0x7f14f40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:39.137 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.131+0000 7f150da43700 1 --2- 172.21.15.52:0/2879424076 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f14f40a11d0 0x7f14f40a3680 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f15080686d0 tx=0x7f14fc0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:39.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.134+0000 7f14fb7fe700 1 -- 172.21.15.52:0/2879424076 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1504097bf0 con 0x7f15080ff170 2023-11-27T15:15:39.144 INFO:journalctl@ceph.mds.l.smithi150.stdout:Nov 27 15:15:39 smithi150 systemd[1]: Starting Ceph mds.l for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:15:39.384 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.378+0000 7f15104a8700 1 -- 172.21.15.52:0/2879424076 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1508106140 con 0x7f15080ff170 2023-11-27T15:15:39.384 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.378+0000 7f14fb7fe700 1 -- 172.21.15.52:0/2879424076 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7f150409b210 con 0x7f15080ff170 2023-11-27T15:15:39.385 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:15:39.385 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-11-27T15:15:39.387 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.381+0000 7f15104a8700 1 -- 172.21.15.52:0/2879424076 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f14f40a11d0 msgr2=0x7f14f40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:39.387 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.381+0000 7f15104a8700 1 --2- 172.21.15.52:0/2879424076 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f14f40a11d0 0x7f14f40a3680 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f15080686d0 tx=0x7f14fc0073d0 comp rx=0 tx=0).stop 2023-11-27T15:15:39.387 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.381+0000 7f15104a8700 1 -- 172.21.15.52:0/2879424076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15080ff170 msgr2=0x7f1508068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:39.388 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.381+0000 7f15104a8700 1 --2- 172.21.15.52:0/2879424076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15080ff170 0x7f1508068960 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f15040059d0 tx=0x7f15040047a0 comp rx=0 tx=0).stop 2023-11-27T15:15:39.388 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.381+0000 7f15104a8700 1 -- 172.21.15.52:0/2879424076 shutdown_connections 2023-11-27T15:15:39.388 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.381+0000 7f15104a8700 1 --2- 172.21.15.52:0/2879424076 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f14f40a11d0 0x7f14f40a3680 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:39.388 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.381+0000 7f15104a8700 1 --2- 172.21.15.52:0/2879424076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1508100db0 0x7f150819d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:39.388 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.381+0000 7f15104a8700 1 --2- 172.21.15.52:0/2879424076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15080ff170 0x7f1508068960 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:39.388 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.381+0000 7f15104a8700 1 --2- 172.21.15.52:0/2879424076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1508100400 0x7f150819ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:39.388 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.381+0000 7f15104a8700 1 -- 172.21.15.52:0/2879424076 >> 172.21.15.52:0/2879424076 conn(0x7f15080fa750 msgr2=0x7f15080fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:39.389 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.381+0000 7f15104a8700 1 -- 172.21.15.52:0/2879424076 shutdown_connections 2023-11-27T15:15:39.389 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:39.381+0000 7f15104a8700 1 -- 172.21.15.52:0/2879424076 wait complete. 2023-11-27T15:15:39.389 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 1 2023-11-27T15:15:39.728 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:39 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2220878435' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:15:39.728 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:39 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1723851616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:15:39.728 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:39 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2879424076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:15:39.728 INFO:journalctl@ceph.mds.l.smithi150.stdout:Nov 27 15:15:39 smithi150 podman[125553]: 2023-11-27T15:15:39.728 INFO:journalctl@ceph.mds.l.smithi150.stdout:Nov 27 15:15:39 smithi150 ceph-mds[125593]: -- 172.21.15.150:0/1851794903 <== mon.1 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ac3f921520 con 0x55ac3f8f6800 2023-11-27T15:15:39.728 INFO:journalctl@ceph.mds.l.smithi150.stdout:Nov 27 15:15:39 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-l[125569]: starting mds.l at 2023-11-27T15:15:39.729 INFO:journalctl@ceph.mds.l.smithi150.stdout:Nov 27 15:15:39 smithi150 bash[125553]: 10c71557d7b7329556eb73fdb91b1cebb72ac0b03997a5ab579c311671df0c06 2023-11-27T15:15:39.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:39 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2220878435' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:15:39.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:39 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1723851616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:15:39.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:39 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2879424076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:15:40.028 INFO:journalctl@ceph.mds.l.smithi150.stdout:Nov 27 15:15:39 smithi150 systemd[1]: Started Ceph mds.l for c6c7df88-8d36-11ee-95a2-87774f69a715. 2023-11-27T15:15:40.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:39 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2220878435' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:15:40.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:39 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1723851616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:15:40.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:39 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2879424076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:15:40.804 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:40 smithi102 ceph-mon[100296]: mds.? [v2:172.21.15.150:6832/1857157307,v1:172.21.15.150:6833/1857157307] up:boot 2023-11-27T15:15:40.804 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:40 smithi102 ceph-mon[100296]: fsmap 1 up:standby 2023-11-27T15:15:40.804 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:40 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-27T15:15:40.804 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:40 smithi102 ceph-mon[100296]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:40.804 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:40 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:40.804 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:40 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:40.804 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:40 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:40.804 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:40 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:15:40.804 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:40 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:15:40.805 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:40 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:41.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:40 smithi150 ceph-mon[100559]: mds.? [v2:172.21.15.150:6832/1857157307,v1:172.21.15.150:6833/1857157307] up:boot 2023-11-27T15:15:41.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:40 smithi150 ceph-mon[100559]: fsmap 1 up:standby 2023-11-27T15:15:41.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:40 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-27T15:15:41.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:40 smithi150 ceph-mon[100559]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:41.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:40 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:41.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:40 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:41.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:40 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:41.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:40 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:15:41.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:40 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:15:41.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:40 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:41.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:40 smithi052 ceph-mon[99056]: mds.? [v2:172.21.15.150:6832/1857157307,v1:172.21.15.150:6833/1857157307] up:boot 2023-11-27T15:15:41.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:40 smithi052 ceph-mon[99056]: fsmap 1 up:standby 2023-11-27T15:15:41.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:40 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-27T15:15:41.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:40 smithi052 ceph-mon[99056]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:41.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:40 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:41.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:40 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:41.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:40 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:41.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:40 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:15:41.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:40 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:15:41.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:40 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:41.399 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:15:41.541 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.536+0000 7f7c7a635700 1 -- 172.21.15.52:0/1513023259 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c741003b0 msgr2=0x7f7c74100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:41.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.536+0000 7f7c7a635700 1 --2- 172.21.15.52:0/1513023259 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c741003b0 0x7f7c74100800 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7c68009990 tx=0x7f7c68009ca0 comp rx=0 tx=0).stop 2023-11-27T15:15:41.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.536+0000 7f7c7a635700 1 -- 172.21.15.52:0/1513023259 shutdown_connections 2023-11-27T15:15:41.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.536+0000 7f7c7a635700 1 --2- 172.21.15.52:0/1513023259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c740ff1c0 0x7f7c740ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:41.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.536+0000 7f7c7a635700 1 --2- 172.21.15.52:0/1513023259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c74100d40 0x7f7c74107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:41.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.536+0000 7f7c7a635700 1 --2- 172.21.15.52:0/1513023259 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c741003b0 0x7f7c74100800 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:41.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.536+0000 7f7c7a635700 1 -- 172.21.15.52:0/1513023259 >> 172.21.15.52:0/1513023259 conn(0x7f7c740fa7a0 msgr2=0x7f7c740fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:41.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.536+0000 7f7c7a635700 1 -- 172.21.15.52:0/1513023259 shutdown_connections 2023-11-27T15:15:41.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.536+0000 7f7c7a635700 1 -- 172.21.15.52:0/1513023259 wait complete. 2023-11-27T15:15:41.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.537+0000 7f7c7a635700 1 Processor -- start 2023-11-27T15:15:41.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.537+0000 7f7c7a635700 1 -- start start 2023-11-27T15:15:41.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.537+0000 7f7c7a635700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c740ff1c0 0x7f7c74069990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:41.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.537+0000 7f7c7a635700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c741003b0 0x7f7c7419acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:41.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.537+0000 7f7c7a635700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c74100d40 0x7f7c7419d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:41.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.537+0000 7f7c7a635700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7419daf0 con 0x7f7c741003b0 2023-11-27T15:15:41.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.537+0000 7f7c7a635700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7419dc60 con 0x7f7c74100d40 2023-11-27T15:15:41.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.537+0000 7f7c7a635700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7419de20 con 0x7f7c740ff1c0 2023-11-27T15:15:41.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.537+0000 7f7c737fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c741003b0 0x7f7c7419acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:41.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.537+0000 7f7c737fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c741003b0 0x7f7c7419acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49742/0 (socket says 172.21.15.52:49742) 2023-11-27T15:15:41.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.537+0000 7f7c737fe700 1 -- 172.21.15.52:0/3375076085 learned_addr learned my addr 172.21.15.52:0/3375076085 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:15:41.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.537+0000 7f7c73fff700 1 --2- 172.21.15.52:0/3375076085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c740ff1c0 0x7f7c74069990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:41.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.537+0000 7f7c78bd2700 1 --2- 172.21.15.52:0/3375076085 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c74100d40 0x7f7c7419d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:41.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.538+0000 7f7c737fe700 1 -- 172.21.15.52:0/3375076085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c740ff1c0 msgr2=0x7f7c74069990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:41.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.538+0000 7f7c737fe700 1 --2- 172.21.15.52:0/3375076085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c740ff1c0 0x7f7c74069990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:41.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.538+0000 7f7c737fe700 1 -- 172.21.15.52:0/3375076085 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c74100d40 msgr2=0x7f7c7419d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:41.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.538+0000 7f7c737fe700 1 --2- 172.21.15.52:0/3375076085 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c74100d40 0x7f7c7419d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:41.546 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.538+0000 7f7c737fe700 1 -- 172.21.15.52:0/3375076085 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c680095f0 con 0x7f7c741003b0 2023-11-27T15:15:41.546 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.538+0000 7f7c78bd2700 1 --2- 172.21.15.52:0/3375076085 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c74100d40 0x7f7c7419d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:15:41.546 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.538+0000 7f7c737fe700 1 --2- 172.21.15.52:0/3375076085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c741003b0 0x7f7c7419acc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7c68000c00 tx=0x7f7c680047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:41.546 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.538+0000 7f7c717fa700 1 -- 172.21.15.52:0/3375076085 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6801d070 con 0x7f7c741003b0 2023-11-27T15:15:41.546 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.538+0000 7f7c717fa700 1 -- 172.21.15.52:0/3375076085 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c68022470 con 0x7f7c741003b0 2023-11-27T15:15:41.546 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.538+0000 7f7c7a635700 1 -- 172.21.15.52:0/3375076085 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c7410ab30 con 0x7f7c741003b0 2023-11-27T15:15:41.546 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.538+0000 7f7c717fa700 1 -- 172.21.15.52:0/3375076085 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c68017650 con 0x7f7c741003b0 2023-11-27T15:15:41.546 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.538+0000 7f7c7a635700 1 -- 172.21.15.52:0/3375076085 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c7410a9a0 con 0x7f7c741003b0 2023-11-27T15:15:41.546 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.539+0000 7f7c7a635700 1 -- 172.21.15.52:0/3375076085 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c74106750 con 0x7f7c741003b0 2023-11-27T15:15:41.547 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.540+0000 7f7c717fa700 1 -- 172.21.15.52:0/3375076085 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7c680225e0 con 0x7f7c741003b0 2023-11-27T15:15:41.547 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.541+0000 7f7c717fa700 1 --2- 172.21.15.52:0/3375076085 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7c5c0a1250 0x7f7c5c0a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:41.547 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.541+0000 7f7c717fa700 1 -- 172.21.15.52:0/3375076085 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7c680cdeb0 con 0x7f7c741003b0 2023-11-27T15:15:41.547 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.541+0000 7f7c73fff700 1 --2- 172.21.15.52:0/3375076085 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7c5c0a1250 0x7f7c5c0a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:41.547 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.541+0000 7f7c73fff700 1 --2- 172.21.15.52:0/3375076085 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7c5c0a1250 0x7f7c5c0a3700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7c74100210 tx=0x7f7c64009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:41.549 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.544+0000 7f7c717fa700 1 -- 172.21.15.52:0/3375076085 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7c68097a90 con 0x7f7c741003b0 2023-11-27T15:15:41.578 INFO:journalctl@ceph.mds.k.smithi102.stdout:Nov 27 15:15:41 smithi102 systemd[1]: Starting Ceph mds.k for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:15:41.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.791+0000 7f7c7a635700 1 -- 172.21.15.52:0/3375076085 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c74061700 con 0x7f7c741003b0 2023-11-27T15:15:41.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.791+0000 7f7c717fa700 1 -- 172.21.15.52:0/3375076085 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1260 (secure 0 0 0) 0x7f7c6809b0b0 con 0x7f7c741003b0 2023-11-27T15:15:41.796 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:15:41.797 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-11-27T15:15:41.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.793+0000 7f7c7a635700 1 -- 172.21.15.52:0/3375076085 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7c5c0a1250 msgr2=0x7f7c5c0a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:41.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.793+0000 7f7c7a635700 1 --2- 172.21.15.52:0/3375076085 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7c5c0a1250 0x7f7c5c0a3700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7c74100210 tx=0x7f7c64009260 comp rx=0 tx=0).stop 2023-11-27T15:15:41.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.793+0000 7f7c7a635700 1 -- 172.21.15.52:0/3375076085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c741003b0 msgr2=0x7f7c7419acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:41.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.793+0000 7f7c7a635700 1 --2- 172.21.15.52:0/3375076085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c741003b0 0x7f7c7419acc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7c68000c00 tx=0x7f7c680047a0 comp rx=0 tx=0).stop 2023-11-27T15:15:41.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.793+0000 7f7c7a635700 1 -- 172.21.15.52:0/3375076085 shutdown_connections 2023-11-27T15:15:41.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.793+0000 7f7c7a635700 1 --2- 172.21.15.52:0/3375076085 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7c5c0a1250 0x7f7c5c0a3700 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:41.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.793+0000 7f7c7a635700 1 --2- 172.21.15.52:0/3375076085 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c74100d40 0x7f7c7419d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:41.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.793+0000 7f7c7a635700 1 --2- 172.21.15.52:0/3375076085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c740ff1c0 0x7f7c74069990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:41.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.793+0000 7f7c7a635700 1 --2- 172.21.15.52:0/3375076085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c741003b0 0x7f7c7419acc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:41.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.793+0000 7f7c7a635700 1 -- 172.21.15.52:0/3375076085 >> 172.21.15.52:0/3375076085 conn(0x7f7c740fa7a0 msgr2=0x7f7c74103890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:41.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.793+0000 7f7c7a635700 1 -- 172.21.15.52:0/3375076085 shutdown_connections 2023-11-27T15:15:41.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:41.793+0000 7f7c7a635700 1 -- 172.21.15.52:0/3375076085 wait complete. 2023-11-27T15:15:41.800 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 2 2023-11-27T15:15:41.877 INFO:journalctl@ceph.mds.k.smithi102.stdout:Nov 27 15:15:41 smithi102 podman[124456]: 2023-11-27T15:15:41.878 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:41 smithi102 ceph-mon[100296]: Deploying daemon mds.k on smithi102 2023-11-27T15:15:42.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:41 smithi150 ceph-mon[100559]: Deploying daemon mds.k on smithi102 2023-11-27T15:15:42.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:41 smithi052 ceph-mon[99056]: Deploying daemon mds.k on smithi102 2023-11-27T15:15:42.328 INFO:journalctl@ceph.mds.k.smithi102.stdout:Nov 27 15:15:41 smithi102 bash[124456]: 9454bf594bec523616cf8c15fe96e70851f68bcaab23c6bc2b7b3ab64c2ecc3c 2023-11-27T15:15:42.328 INFO:journalctl@ceph.mds.k.smithi102.stdout:Nov 27 15:15:41 smithi102 ceph-mds[124496]: -- 172.21.15.102:0/2838010715 <== mon.2 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c4a6615520 con 0x55c4a65eac00 2023-11-27T15:15:42.328 INFO:journalctl@ceph.mds.k.smithi102.stdout:Nov 27 15:15:41 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-k[124472]: starting mds.k at 2023-11-27T15:15:42.328 INFO:journalctl@ceph.mds.k.smithi102.stdout:Nov 27 15:15:41 smithi102 systemd[1]: Started Ceph mds.k for c6c7df88-8d36-11ee-95a2-87774f69a715. 2023-11-27T15:15:43.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:42 smithi150 ceph-mon[100559]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:43.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:42 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3375076085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:15:43.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:42 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:43.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:42 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:43.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:42 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:43.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:42 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:15:43.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:42 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:15:43.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:42 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:43.042 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:42 smithi052 ceph-mon[99056]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:43.042 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:42 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3375076085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:15:43.042 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:42 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:43.043 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:42 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:43.043 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:42 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:43.043 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:42 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:15:43.043 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:42 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:15:43.043 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:42 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:43.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:42 smithi102 ceph-mon[100296]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:43.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:42 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3375076085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:15:43.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:42 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:43.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:42 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:43.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:42 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:43.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:42 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:15:43.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:42 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:15:43.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:42 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:43.722 INFO:journalctl@ceph.mds.j.smithi052.stdout:Nov 27 15:15:43 smithi052 systemd[1]: Starting Ceph mds.j for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:15:43.811 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:15:43.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.942+0000 7faa476e5700 1 -- 172.21.15.52:0/668956940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa40105e50 msgr2=0x7faa40108300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:43.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.942+0000 7faa476e5700 1 --2- 172.21.15.52:0/668956940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa40105e50 0x7faa40108300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faa3800b0a0 tx=0x7faa3800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:15:43.948 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.942+0000 7faa476e5700 1 -- 172.21.15.52:0/668956940 shutdown_connections 2023-11-27T15:15:43.948 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.942+0000 7faa476e5700 1 --2- 172.21.15.52:0/668956940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa40105e50 0x7faa40108300 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:43.948 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.942+0000 7faa476e5700 1 --2- 172.21.15.52:0/668956940 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa40068ab0 0x7faa40102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:43.948 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.942+0000 7faa476e5700 1 --2- 172.21.15.52:0/668956940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa401033a0 0x7faa401057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:43.948 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.942+0000 7faa476e5700 1 -- 172.21.15.52:0/668956940 >> 172.21.15.52:0/668956940 conn(0x7faa400fa750 msgr2=0x7faa400fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:43.948 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.942+0000 7faa476e5700 1 -- 172.21.15.52:0/668956940 shutdown_connections 2023-11-27T15:15:43.948 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.942+0000 7faa476e5700 1 -- 172.21.15.52:0/668956940 wait complete. 2023-11-27T15:15:43.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.943+0000 7faa476e5700 1 Processor -- start 2023-11-27T15:15:43.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.943+0000 7faa476e5700 1 -- start start 2023-11-27T15:15:43.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.943+0000 7faa476e5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa40068ab0 0x7faa401007c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:43.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.943+0000 7faa476e5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa401033a0 0x7faa400fee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:43.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.943+0000 7faa476e5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa40105e50 0x7faa400ff340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:43.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.943+0000 7faa476e5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa400ffae0 con 0x7faa40105e50 2023-11-27T15:15:43.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.943+0000 7faa476e5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa400ffc50 con 0x7faa401033a0 2023-11-27T15:15:43.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.943+0000 7faa476e5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa401a00b0 con 0x7faa40068ab0 2023-11-27T15:15:43.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.943+0000 7faa45c82700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa40105e50 0x7faa400ff340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:43.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.943+0000 7faa45c82700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa40105e50 0x7faa400ff340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49754/0 (socket says 172.21.15.52:49754) 2023-11-27T15:15:43.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.943+0000 7faa45c82700 1 -- 172.21.15.52:0/822176170 learned_addr learned my addr 172.21.15.52:0/822176170 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:15:43.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.943+0000 7faa44c80700 1 --2- 172.21.15.52:0/822176170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa401033a0 0x7faa400fee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:43.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.943+0000 7faa45481700 1 --2- 172.21.15.52:0/822176170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa40068ab0 0x7faa401007c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:43.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.943+0000 7faa45c82700 1 -- 172.21.15.52:0/822176170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa40068ab0 msgr2=0x7faa401007c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:43.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.943+0000 7faa45c82700 1 --2- 172.21.15.52:0/822176170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa40068ab0 0x7faa401007c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:43.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.943+0000 7faa45c82700 1 -- 172.21.15.52:0/822176170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa401033a0 msgr2=0x7faa400fee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:43.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.943+0000 7faa45c82700 1 --2- 172.21.15.52:0/822176170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa401033a0 0x7faa400fee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:43.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.943+0000 7faa45c82700 1 -- 172.21.15.52:0/822176170 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa38009d00 con 0x7faa40105e50 2023-11-27T15:15:43.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.943+0000 7faa45c82700 1 --2- 172.21.15.52:0/822176170 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa40105e50 0x7faa400ff340 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faa380062a0 tx=0x7faa38004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:43.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.943+0000 7faa367fc700 1 -- 172.21.15.52:0/822176170 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3800e040 con 0x7faa40105e50 2023-11-27T15:15:43.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.944+0000 7faa476e5700 1 -- 172.21.15.52:0/822176170 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa4010bbf0 con 0x7faa40105e50 2023-11-27T15:15:43.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.944+0000 7faa367fc700 1 -- 172.21.15.52:0/822176170 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa3801db50 con 0x7faa40105e50 2023-11-27T15:15:43.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.944+0000 7faa367fc700 1 -- 172.21.15.52:0/822176170 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3801bc40 con 0x7faa40105e50 2023-11-27T15:15:43.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.944+0000 7faa476e5700 1 -- 172.21.15.52:0/822176170 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa4010ba60 con 0x7faa40105e50 2023-11-27T15:15:43.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.944+0000 7faa476e5700 1 -- 172.21.15.52:0/822176170 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa400fc2c0 con 0x7faa40105e50 2023-11-27T15:15:43.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.946+0000 7faa367fc700 1 -- 172.21.15.52:0/822176170 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7faa3801d690 con 0x7faa40105e50 2023-11-27T15:15:43.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.946+0000 7faa367fc700 1 --2- 172.21.15.52:0/822176170 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7faa2c0a1150 0x7faa2c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:43.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.946+0000 7faa45481700 1 --2- 172.21.15.52:0/822176170 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7faa2c0a1150 0x7faa2c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:43.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.946+0000 7faa367fc700 1 -- 172.21.15.52:0/822176170 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7faa380ce7b0 con 0x7faa40105e50 2023-11-27T15:15:43.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.947+0000 7faa45481700 1 --2- 172.21.15.52:0/822176170 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7faa2c0a1150 0x7faa2c0a3600 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7faa3c005950 tx=0x7faa3c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:43.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:43.950+0000 7faa367fc700 1 -- 172.21.15.52:0/822176170 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7faa38098430 con 0x7faa40105e50 2023-11-27T15:15:44.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:43 smithi150 ceph-mon[100559]: Deploying daemon mds.j on smithi052 2023-11-27T15:15:44.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:43 smithi150 ceph-mon[100559]: mds.? [v2:172.21.15.102:6832/883247933,v1:172.21.15.102:6833/883247933] up:boot 2023-11-27T15:15:44.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:43 smithi150 ceph-mon[100559]: fsmap 2 up:standby 2023-11-27T15:15:44.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:43 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-27T15:15:44.049 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:43 smithi052 ceph-mon[99056]: Deploying daemon mds.j on smithi052 2023-11-27T15:15:44.049 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:43 smithi052 ceph-mon[99056]: mds.? [v2:172.21.15.102:6832/883247933,v1:172.21.15.102:6833/883247933] up:boot 2023-11-27T15:15:44.049 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:43 smithi052 ceph-mon[99056]: fsmap 2 up:standby 2023-11-27T15:15:44.049 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:43 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-27T15:15:44.050 INFO:journalctl@ceph.mds.j.smithi052.stdout:Nov 27 15:15:43 smithi052 podman[132238]: 2023-11-27T15:15:44.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:43 smithi102 ceph-mon[100296]: Deploying daemon mds.j on smithi052 2023-11-27T15:15:44.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:43 smithi102 ceph-mon[100296]: mds.? [v2:172.21.15.102:6832/883247933,v1:172.21.15.102:6833/883247933] up:boot 2023-11-27T15:15:44.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:43 smithi102 ceph-mon[100296]: fsmap 2 up:standby 2023-11-27T15:15:44.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:43 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-27T15:15:44.216 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:44.211+0000 7faa476e5700 1 -- 172.21.15.52:0/822176170 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa40061700 con 0x7faa40105e50 2023-11-27T15:15:44.217 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:44.212+0000 7faa367fc700 1 -- 172.21.15.52:0/822176170 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2017 (secure 0 0 0) 0x7faa3809ba50 con 0x7faa40105e50 2023-11-27T15:15:44.217 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:15:44.218 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-11-27T15:15:44.220 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:44.215+0000 7faa476e5700 1 -- 172.21.15.52:0/822176170 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7faa2c0a1150 msgr2=0x7faa2c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:44.220 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:44.215+0000 7faa476e5700 1 --2- 172.21.15.52:0/822176170 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7faa2c0a1150 0x7faa2c0a3600 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7faa3c005950 tx=0x7faa3c0058e0 comp rx=0 tx=0).stop 2023-11-27T15:15:44.220 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:44.215+0000 7faa476e5700 1 -- 172.21.15.52:0/822176170 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa40105e50 msgr2=0x7faa400ff340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:44.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:44.215+0000 7faa476e5700 1 --2- 172.21.15.52:0/822176170 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa40105e50 0x7faa400ff340 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faa380062a0 tx=0x7faa38004db0 comp rx=0 tx=0).stop 2023-11-27T15:15:44.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:44.215+0000 7faa476e5700 1 -- 172.21.15.52:0/822176170 shutdown_connections 2023-11-27T15:15:44.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:44.215+0000 7faa476e5700 1 --2- 172.21.15.52:0/822176170 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7faa2c0a1150 0x7faa2c0a3600 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:44.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:44.215+0000 7faa476e5700 1 --2- 172.21.15.52:0/822176170 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa40105e50 0x7faa400ff340 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:44.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:44.215+0000 7faa476e5700 1 --2- 172.21.15.52:0/822176170 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa40068ab0 0x7faa401007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:44.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:44.215+0000 7faa476e5700 1 --2- 172.21.15.52:0/822176170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa401033a0 0x7faa400fee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:44.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:44.215+0000 7faa476e5700 1 -- 172.21.15.52:0/822176170 >> 172.21.15.52:0/822176170 conn(0x7faa400fa750 msgr2=0x7faa401016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:44.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:44.215+0000 7faa476e5700 1 -- 172.21.15.52:0/822176170 shutdown_connections 2023-11-27T15:15:44.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:44.215+0000 7faa476e5700 1 -- 172.21.15.52:0/822176170 wait complete. 2023-11-27T15:15:44.222 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 3 2023-11-27T15:15:44.367 INFO:journalctl@ceph.mds.j.smithi052.stdout:Nov 27 15:15:44 smithi052 bash[132238]: dc2294d91b6cbf167e440dd739c4399b64ff1443025653e9eea6fc743fcd16b4 2023-11-27T15:15:44.367 INFO:journalctl@ceph.mds.j.smithi052.stdout:Nov 27 15:15:44 smithi052 ceph-mds[132321]: -- 172.21.15.52:0/1351864386 <== mon.0 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c7c05e3520 con 0x55c7c05b8c00 2023-11-27T15:15:44.367 INFO:journalctl@ceph.mds.j.smithi052.stdout:Nov 27 15:15:44 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-j[132277]: starting mds.j at 2023-11-27T15:15:44.367 INFO:journalctl@ceph.mds.j.smithi052.stdout:Nov 27 15:15:44 smithi052 systemd[1]: Started Ceph mds.j for c6c7df88-8d36-11ee-95a2-87774f69a715. 2023-11-27T15:15:45.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:44 smithi150 ceph-mon[100559]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:45.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:44 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:45.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:44 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:45.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:44 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:45.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:44 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:15:45.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:44 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:15:45.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:44 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:45.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:44 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/822176170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:15:45.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:44 smithi102 ceph-mon[100296]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:45.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:44 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:45.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:44 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:45.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:44 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:45.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:44 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:15:45.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:44 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:15:45.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:44 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:45.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:44 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/822176170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:15:45.112 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:44 smithi052 ceph-mon[99056]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:45.112 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:44 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:45.112 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:44 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:45.112 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:44 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:45.112 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:44 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:15:45.113 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:44 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:15:45.113 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:44 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:45.113 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:44 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/822176170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:15:46.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:45 smithi150 ceph-mon[100559]: Deploying daemon mds.d on smithi052 2023-11-27T15:15:46.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:45 smithi150 ceph-mon[100559]: mds.? [v2:172.21.15.52:6834/3561136724,v1:172.21.15.52:6835/3561136724] up:boot 2023-11-27T15:15:46.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:45 smithi150 ceph-mon[100559]: fsmap 3 up:standby 2023-11-27T15:15:46.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:45 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-11-27T15:15:46.064 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:45 smithi052 ceph-mon[99056]: Deploying daemon mds.d on smithi052 2023-11-27T15:15:46.064 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:45 smithi052 ceph-mon[99056]: mds.? [v2:172.21.15.52:6834/3561136724,v1:172.21.15.52:6835/3561136724] up:boot 2023-11-27T15:15:46.064 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:45 smithi052 ceph-mon[99056]: fsmap 3 up:standby 2023-11-27T15:15:46.064 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:45 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-11-27T15:15:46.065 INFO:journalctl@ceph.mds.d.smithi052.stdout:Nov 27 15:15:45 smithi052 systemd[1]: Starting Ceph mds.d for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:15:46.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:45 smithi102 ceph-mon[100296]: Deploying daemon mds.d on smithi052 2023-11-27T15:15:46.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:45 smithi102 ceph-mon[100296]: mds.? [v2:172.21.15.52:6834/3561136724,v1:172.21.15.52:6835/3561136724] up:boot 2023-11-27T15:15:46.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:45 smithi102 ceph-mon[100296]: fsmap 3 up:standby 2023-11-27T15:15:46.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:45 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-11-27T15:15:46.234 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:15:46.324 INFO:journalctl@ceph.mds.d.smithi052.stdout:Nov 27 15:15:46 smithi052 podman[132624]: 2023-11-27T15:15:46.324 INFO:journalctl@ceph.mds.d.smithi052.stdout:Nov 27 15:15:46 smithi052 ceph-mds[132687]: -- 172.21.15.52:0/3543736304 <== mon.0 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5605bed85520 con 0x5605bed5a800 2023-11-27T15:15:46.324 INFO:journalctl@ceph.mds.d.smithi052.stdout:Nov 27 15:15:46 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-d[132640]: starting mds.d at 2023-11-27T15:15:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.366+0000 7ff378f54700 1 -- 172.21.15.52:0/2146234586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3741003b0 msgr2=0x7ff374100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.366+0000 7ff378f54700 1 --2- 172.21.15.52:0/2146234586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3741003b0 0x7ff374100800 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff368009990 tx=0x7ff368009ca0 comp rx=0 tx=0).stop 2023-11-27T15:15:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.367+0000 7ff378f54700 1 -- 172.21.15.52:0/2146234586 shutdown_connections 2023-11-27T15:15:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.367+0000 7ff378f54700 1 --2- 172.21.15.52:0/2146234586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff374100d40 0x7ff374107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.367+0000 7ff378f54700 1 --2- 172.21.15.52:0/2146234586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3740ff1c0 0x7ff3740ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:46.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.367+0000 7ff378f54700 1 --2- 172.21.15.52:0/2146234586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff3741003b0 0x7ff374100800 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:46.373 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.367+0000 7ff378f54700 1 -- 172.21.15.52:0/2146234586 >> 172.21.15.52:0/2146234586 conn(0x7ff3740fa7a0 msgr2=0x7ff3740fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:46.373 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.367+0000 7ff378f54700 1 -- 172.21.15.52:0/2146234586 shutdown_connections 2023-11-27T15:15:46.373 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.367+0000 7ff378f54700 1 -- 172.21.15.52:0/2146234586 wait complete. 2023-11-27T15:15:46.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.367+0000 7ff378f54700 1 Processor -- start 2023-11-27T15:15:46.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.367+0000 7ff378f54700 1 -- start start 2023-11-27T15:15:46.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.368+0000 7ff378f54700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3740ff1c0 0x7ff374195f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:46.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.368+0000 7ff378f54700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3741003b0 0x7ff374196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:46.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.368+0000 7ff378f54700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff374100d40 0x7ff37419d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:46.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.368+0000 7ff378f54700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37419daa0 con 0x7ff374100d40 2023-11-27T15:15:46.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.368+0000 7ff378f54700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37419dbe0 con 0x7ff3740ff1c0 2023-11-27T15:15:46.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.368+0000 7ff378f54700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37419dd70 con 0x7ff3741003b0 2023-11-27T15:15:46.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.368+0000 7ff372d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff374100d40 0x7ff37419d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:46.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.368+0000 7ff372d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff374100d40 0x7ff37419d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49792/0 (socket says 172.21.15.52:49792) 2023-11-27T15:15:46.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.368+0000 7ff372d9d700 1 -- 172.21.15.52:0/654124666 learned_addr learned my addr 172.21.15.52:0/654124666 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:15:46.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.368+0000 7ff37259c700 1 --2- 172.21.15.52:0/654124666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3740ff1c0 0x7ff374195f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:46.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.368+0000 7ff371d9b700 1 --2- 172.21.15.52:0/654124666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3741003b0 0x7ff374196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:46.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.368+0000 7ff372d9d700 1 -- 172.21.15.52:0/654124666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3741003b0 msgr2=0x7ff374196460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:46.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.368+0000 7ff372d9d700 1 --2- 172.21.15.52:0/654124666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3741003b0 0x7ff374196460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:46.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.368+0000 7ff372d9d700 1 -- 172.21.15.52:0/654124666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3740ff1c0 msgr2=0x7ff374195f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:46.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.368+0000 7ff372d9d700 1 --2- 172.21.15.52:0/654124666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3740ff1c0 0x7ff374195f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:46.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.368+0000 7ff372d9d700 1 -- 172.21.15.52:0/654124666 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3680095f0 con 0x7ff374100d40 2023-11-27T15:15:46.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.368+0000 7ff371d9b700 1 --2- 172.21.15.52:0/654124666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3741003b0 0x7ff374196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:15:46.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.368+0000 7ff372d9d700 1 --2- 172.21.15.52:0/654124666 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff374100d40 0x7ff37419d4d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff36c007d70 tx=0x7ff36c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:46.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.368+0000 7ff3637fe700 1 -- 172.21.15.52:0/654124666 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c00ad40 con 0x7ff374100d40 2023-11-27T15:15:46.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.368+0000 7ff378f54700 1 -- 172.21.15.52:0/654124666 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff37410ab70 con 0x7ff374100d40 2023-11-27T15:15:46.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.368+0000 7ff3637fe700 1 -- 172.21.15.52:0/654124666 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff36c0128e0 con 0x7ff374100d40 2023-11-27T15:15:46.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.368+0000 7ff3637fe700 1 -- 172.21.15.52:0/654124666 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c0116b0 con 0x7ff374100d40 2023-11-27T15:15:46.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.368+0000 7ff378f54700 1 -- 172.21.15.52:0/654124666 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff37410a9e0 con 0x7ff374100d40 2023-11-27T15:15:46.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.369+0000 7ff378f54700 1 -- 172.21.15.52:0/654124666 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff374107200 con 0x7ff374100d40 2023-11-27T15:15:46.378 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.370+0000 7ff3637fe700 1 -- 172.21.15.52:0/654124666 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff36c012420 con 0x7ff374100d40 2023-11-27T15:15:46.378 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.371+0000 7ff3637fe700 1 --2- 172.21.15.52:0/654124666 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff35c0a1150 0x7ff35c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:46.378 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.371+0000 7ff37259c700 1 --2- 172.21.15.52:0/654124666 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff35c0a1150 0x7ff35c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:46.378 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.371+0000 7ff3637fe700 1 -- 172.21.15.52:0/654124666 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff36c0cc780 con 0x7ff374100d40 2023-11-27T15:15:46.379 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.371+0000 7ff37259c700 1 --2- 172.21.15.52:0/654124666 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff35c0a1150 0x7ff35c0a3600 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff374100210 tx=0x7ff36400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:46.380 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.375+0000 7ff3637fe700 1 -- 172.21.15.52:0/654124666 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff36c0962d0 con 0x7ff374100d40 2023-11-27T15:15:46.617 INFO:journalctl@ceph.mds.d.smithi052.stdout:Nov 27 15:15:46 smithi052 bash[132624]: 6ccfc6682deec44bd976bcea746f5fada38852b79668b58faa1daf9ee121eb21 2023-11-27T15:15:46.617 INFO:journalctl@ceph.mds.d.smithi052.stdout:Nov 27 15:15:46 smithi052 systemd[1]: Started Ceph mds.d for c6c7df88-8d36-11ee-95a2-87774f69a715. 2023-11-27T15:15:46.655 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.650+0000 7ff378f54700 1 -- 172.21.15.52:0/654124666 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff37402cfe0 con 0x7ff374100d40 2023-11-27T15:15:46.655 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.650+0000 7ff3637fe700 1 -- 172.21.15.52:0/654124666 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2774 (secure 0 0 0) 0x7ff36c0998f0 con 0x7ff374100d40 2023-11-27T15:15:46.656 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:15:46.656 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-11-27T15:15:46.658 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.653+0000 7ff378f54700 1 -- 172.21.15.52:0/654124666 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff35c0a1150 msgr2=0x7ff35c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:46.658 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.653+0000 7ff378f54700 1 --2- 172.21.15.52:0/654124666 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff35c0a1150 0x7ff35c0a3600 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff374100210 tx=0x7ff36400b410 comp rx=0 tx=0).stop 2023-11-27T15:15:46.658 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.653+0000 7ff378f54700 1 -- 172.21.15.52:0/654124666 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff374100d40 msgr2=0x7ff37419d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:46.658 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.653+0000 7ff378f54700 1 --2- 172.21.15.52:0/654124666 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff374100d40 0x7ff37419d4d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff36c007d70 tx=0x7ff36c00a340 comp rx=0 tx=0).stop 2023-11-27T15:15:46.659 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.653+0000 7ff378f54700 1 -- 172.21.15.52:0/654124666 shutdown_connections 2023-11-27T15:15:46.659 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.653+0000 7ff378f54700 1 --2- 172.21.15.52:0/654124666 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff35c0a1150 0x7ff35c0a3600 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:46.659 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.653+0000 7ff378f54700 1 --2- 172.21.15.52:0/654124666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3740ff1c0 0x7ff374195f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:46.659 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.653+0000 7ff378f54700 1 --2- 172.21.15.52:0/654124666 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff374100d40 0x7ff37419d4d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:46.659 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.653+0000 7ff378f54700 1 --2- 172.21.15.52:0/654124666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3741003b0 0x7ff374196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:46.659 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.653+0000 7ff378f54700 1 -- 172.21.15.52:0/654124666 >> 172.21.15.52:0/654124666 conn(0x7ff3740fa7a0 msgr2=0x7ff3740fc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:46.659 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.653+0000 7ff378f54700 1 -- 172.21.15.52:0/654124666 shutdown_connections 2023-11-27T15:15:46.659 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:46.653+0000 7ff378f54700 1 -- 172.21.15.52:0/654124666 wait complete. 2023-11-27T15:15:46.659 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 4 2023-11-27T15:15:47.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:46 smithi150 ceph-mon[100559]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:47.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:46 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:47.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:46 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:47.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:46 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:47.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:46 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:15:47.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:46 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:15:47.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:46 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:47.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:46 smithi150 ceph-mon[100559]: Deploying daemon mds.e on smithi102 2023-11-27T15:15:47.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:46 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/654124666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:15:47.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:46 smithi102 ceph-mon[100296]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:47.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:46 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:47.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:46 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:47.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:46 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:47.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:46 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:15:47.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:46 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:15:47.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:46 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:47.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:46 smithi102 ceph-mon[100296]: Deploying daemon mds.e on smithi102 2023-11-27T15:15:47.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:46 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/654124666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:15:47.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:46 smithi052 ceph-mon[99056]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:47.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:46 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:47.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:46 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:47.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:46 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:47.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:46 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:15:47.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:46 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:15:47.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:46 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:47.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:46 smithi052 ceph-mon[99056]: Deploying daemon mds.e on smithi102 2023-11-27T15:15:47.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:46 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/654124666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:15:48.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:47 smithi150 ceph-mon[100559]: mds.? [v2:172.21.15.52:6836/3583014325,v1:172.21.15.52:6837/3583014325] up:boot 2023-11-27T15:15:48.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:47 smithi150 ceph-mon[100559]: fsmap 4 up:standby 2023-11-27T15:15:48.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:47 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-11-27T15:15:48.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:47 smithi150 ceph-mon[100559]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:48.078 INFO:journalctl@ceph.mds.e.smithi102.stdout:Nov 27 15:15:47 smithi102 systemd[1]: Starting Ceph mds.e for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:15:48.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:47 smithi102 ceph-mon[100296]: mds.? [v2:172.21.15.52:6836/3583014325,v1:172.21.15.52:6837/3583014325] up:boot 2023-11-27T15:15:48.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:47 smithi102 ceph-mon[100296]: fsmap 4 up:standby 2023-11-27T15:15:48.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:47 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-11-27T15:15:48.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:47 smithi102 ceph-mon[100296]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:48.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:47 smithi052 ceph-mon[99056]: mds.? [v2:172.21.15.52:6836/3583014325,v1:172.21.15.52:6837/3583014325] up:boot 2023-11-27T15:15:48.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:47 smithi052 ceph-mon[99056]: fsmap 4 up:standby 2023-11-27T15:15:48.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:47 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-11-27T15:15:48.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:47 smithi052 ceph-mon[99056]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:48.535 INFO:journalctl@ceph.mds.e.smithi102.stdout:Nov 27 15:15:48 smithi102 podman[124800]: 2023-11-27T15:15:48.535 INFO:journalctl@ceph.mds.e.smithi102.stdout:Nov 27 15:15:48 smithi102 ceph-mds[124840]: -- 172.21.15.102:0/561232244 <== mon.2 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a06e26f520 con 0x55a06e244800 2023-11-27T15:15:48.535 INFO:journalctl@ceph.mds.e.smithi102.stdout:Nov 27 15:15:48 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-e[124816]: starting mds.e at 2023-11-27T15:15:48.671 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:15:48.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.805+0000 7f5485b68700 1 -- 172.21.15.52:0/1633873828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54801036b0 msgr2=0x7f5480105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:48.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.805+0000 7f5485b68700 1 --2- 172.21.15.52:0/1633873828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54801036b0 0x7f5480105a90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5474009990 tx=0x7f5474009ca0 comp rx=0 tx=0).stop 2023-11-27T15:15:48.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.806+0000 7f5485b68700 1 -- 172.21.15.52:0/1633873828 shutdown_connections 2023-11-27T15:15:48.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.806+0000 7f5485b68700 1 --2- 172.21.15.52:0/1633873828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5480100d90 0x7f5480103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:48.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.806+0000 7f5485b68700 1 --2- 172.21.15.52:0/1633873828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5480105fd0 0x7f54801083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:48.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.806+0000 7f5485b68700 1 --2- 172.21.15.52:0/1633873828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54801036b0 0x7f5480105a90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:48.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.806+0000 7f5485b68700 1 -- 172.21.15.52:0/1633873828 >> 172.21.15.52:0/1633873828 conn(0x7f54800fa750 msgr2=0x7f54800fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:48.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.806+0000 7f5485b68700 1 -- 172.21.15.52:0/1633873828 shutdown_connections 2023-11-27T15:15:48.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.806+0000 7f5485b68700 1 -- 172.21.15.52:0/1633873828 wait complete. 2023-11-27T15:15:48.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.806+0000 7f5485b68700 1 Processor -- start 2023-11-27T15:15:48.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.806+0000 7f5485b68700 1 -- start start 2023-11-27T15:15:48.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.807+0000 7f5485b68700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5480100d90 0x7f5480100700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:48.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.807+0000 7f5485b68700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54801036b0 0x7f54800fed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:48.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.807+0000 7f5485b68700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5480105fd0 0x7f54800ff280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:48.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.807+0000 7f5485b68700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54800ffa20 con 0x7f5480100d90 2023-11-27T15:15:48.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.807+0000 7f5485b68700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54800ffb90 con 0x7f54801036b0 2023-11-27T15:15:48.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.807+0000 7f5485b68700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54801a0020 con 0x7f5480105fd0 2023-11-27T15:15:48.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.807+0000 7f547f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5480100d90 0x7f5480100700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:48.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.807+0000 7f547f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5480100d90 0x7f5480100700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49804/0 (socket says 172.21.15.52:49804) 2023-11-27T15:15:48.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.807+0000 7f547f7fe700 1 -- 172.21.15.52:0/1976358754 learned_addr learned my addr 172.21.15.52:0/1976358754 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:15:48.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.807+0000 7f547ffff700 1 --2- 172.21.15.52:0/1976358754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5480105fd0 0x7f54800ff280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:48.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.807+0000 7f547effd700 1 --2- 172.21.15.52:0/1976358754 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54801036b0 0x7f54800fed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:48.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.807+0000 7f547f7fe700 1 -- 172.21.15.52:0/1976358754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5480105fd0 msgr2=0x7f54800ff280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:48.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.807+0000 7f547f7fe700 1 --2- 172.21.15.52:0/1976358754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5480105fd0 0x7f54800ff280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:48.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.807+0000 7f547f7fe700 1 -- 172.21.15.52:0/1976358754 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54801036b0 msgr2=0x7f54800fed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:48.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.807+0000 7f547f7fe700 1 --2- 172.21.15.52:0/1976358754 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54801036b0 0x7f54800fed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:48.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.807+0000 7f547f7fe700 1 -- 172.21.15.52:0/1976358754 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54740095f0 con 0x7f5480100d90 2023-11-27T15:15:48.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.807+0000 7f547effd700 1 --2- 172.21.15.52:0/1976358754 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54801036b0 0x7f54800fed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:15:48.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.807+0000 7f547f7fe700 1 --2- 172.21.15.52:0/1976358754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5480100d90 0x7f5480100700 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f547000e970 tx=0x7f547000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:48.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.807+0000 7f547cff9700 1 -- 172.21.15.52:0/1976358754 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5470009790 con 0x7f5480100d90 2023-11-27T15:15:48.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.807+0000 7f547cff9700 1 -- 172.21.15.52:0/1976358754 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5470004cf0 con 0x7f5480100d90 2023-11-27T15:15:48.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.807+0000 7f5485b68700 1 -- 172.21.15.52:0/1976358754 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f548010bc60 con 0x7f5480100d90 2023-11-27T15:15:48.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.807+0000 7f547cff9700 1 -- 172.21.15.52:0/1976358754 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5470005680 con 0x7f5480100d90 2023-11-27T15:15:48.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.807+0000 7f5485b68700 1 -- 172.21.15.52:0/1976358754 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548010bad0 con 0x7f5480100d90 2023-11-27T15:15:48.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.808+0000 7f5485b68700 1 -- 172.21.15.52:0/1976358754 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54800fc2c0 con 0x7f5480100d90 2023-11-27T15:15:48.817 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.809+0000 7f547cff9700 1 -- 172.21.15.52:0/1976358754 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5470004750 con 0x7f5480100d90 2023-11-27T15:15:48.817 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.810+0000 7f547cff9700 1 --2- 172.21.15.52:0/1976358754 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f54680a1150 0x7f54680a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:48.817 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.810+0000 7f547effd700 1 --2- 172.21.15.52:0/1976358754 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f54680a1150 0x7f54680a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:48.817 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.810+0000 7f547cff9700 1 -- 172.21.15.52:0/1976358754 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5470014070 con 0x7f5480100d90 2023-11-27T15:15:48.817 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.810+0000 7f547effd700 1 --2- 172.21.15.52:0/1976358754 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f54680a1150 0x7f54680a3600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f547400b2e0 tx=0x7f5474005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:48.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:48.814+0000 7f547cff9700 1 -- 172.21.15.52:0/1976358754 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5470096350 con 0x7f5480100d90 2023-11-27T15:15:48.827 INFO:journalctl@ceph.mds.e.smithi102.stdout:Nov 27 15:15:48 smithi102 bash[124800]: 2d123c3448f501ec870e3f6e072f2614ca5bb804a377046da64520008ce8c0ab 2023-11-27T15:15:48.828 INFO:journalctl@ceph.mds.e.smithi102.stdout:Nov 27 15:15:48 smithi102 systemd[1]: Started Ceph mds.e for c6c7df88-8d36-11ee-95a2-87774f69a715. 2023-11-27T15:15:49.063 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:49.057+0000 7f5485b68700 1 -- 172.21.15.52:0/1976358754 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f548002cfe0 con 0x7f5480100d90 2023-11-27T15:15:49.063 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:49.058+0000 7f547cff9700 1 -- 172.21.15.52:0/1976358754 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4291 (secure 0 0 0) 0x7f5470099970 con 0x7f5480100d90 2023-11-27T15:15:49.063 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:15:49.063 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-11-27T15:15:49.065 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:49.060+0000 7f5485b68700 1 -- 172.21.15.52:0/1976358754 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f54680a1150 msgr2=0x7f54680a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:49.066 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:49.060+0000 7f5485b68700 1 --2- 172.21.15.52:0/1976358754 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f54680a1150 0x7f54680a3600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f547400b2e0 tx=0x7f5474005fb0 comp rx=0 tx=0).stop 2023-11-27T15:15:49.066 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:49.060+0000 7f5485b68700 1 -- 172.21.15.52:0/1976358754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5480100d90 msgr2=0x7f5480100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:49.066 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:49.060+0000 7f5485b68700 1 --2- 172.21.15.52:0/1976358754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5480100d90 0x7f5480100700 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f547000e970 tx=0x7f547000ed30 comp rx=0 tx=0).stop 2023-11-27T15:15:49.066 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:49.060+0000 7f5485b68700 1 -- 172.21.15.52:0/1976358754 shutdown_connections 2023-11-27T15:15:49.066 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:49.060+0000 7f5485b68700 1 --2- 172.21.15.52:0/1976358754 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f54680a1150 0x7f54680a3600 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:49.066 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:49.060+0000 7f5485b68700 1 --2- 172.21.15.52:0/1976358754 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5480105fd0 0x7f54800ff280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:49.066 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:49.060+0000 7f5485b68700 1 --2- 172.21.15.52:0/1976358754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5480100d90 0x7f5480100700 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:49.066 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:49.060+0000 7f5485b68700 1 --2- 172.21.15.52:0/1976358754 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54801036b0 0x7f54800fed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:49.066 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:49.060+0000 7f5485b68700 1 -- 172.21.15.52:0/1976358754 >> 172.21.15.52:0/1976358754 conn(0x7f54800fa750 msgr2=0x7f54801019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:49.066 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:49.060+0000 7f5485b68700 1 -- 172.21.15.52:0/1976358754 shutdown_connections 2023-11-27T15:15:49.067 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:49.060+0000 7f5485b68700 1 -- 172.21.15.52:0/1976358754 wait complete. 2023-11-27T15:15:49.067 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 6 2023-11-27T15:15:49.657 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:49 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:49.658 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:49 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:49.658 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:49 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:49.658 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:49 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:15:49.658 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:49 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:15:49.658 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:49 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:49.658 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:49 smithi150 ceph-mon[100559]: Deploying daemon mds.f on smithi150 2023-11-27T15:15:49.658 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:49 smithi150 ceph-mon[100559]: mds.? [v2:172.21.15.102:6834/2398692013,v1:172.21.15.102:6835/2398692013] up:boot 2023-11-27T15:15:49.658 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:49 smithi150 ceph-mon[100559]: fsmap 5 up:standby 2023-11-27T15:15:49.658 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:49 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-27T15:15:49.658 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:49 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1976358754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:15:49.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:49 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:49.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:49 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:49.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:49 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:49.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:49 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:15:49.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:49 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:15:49.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:49 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:49.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:49 smithi052 ceph-mon[99056]: Deploying daemon mds.f on smithi150 2023-11-27T15:15:49.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:49 smithi052 ceph-mon[99056]: mds.? [v2:172.21.15.102:6834/2398692013,v1:172.21.15.102:6835/2398692013] up:boot 2023-11-27T15:15:49.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:49 smithi052 ceph-mon[99056]: fsmap 5 up:standby 2023-11-27T15:15:49.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:49 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-27T15:15:49.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:49 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1976358754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:15:50.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:50.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:50.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:50.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:15:50.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:15:50.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:50.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:49 smithi102 ceph-mon[100296]: Deploying daemon mds.f on smithi150 2023-11-27T15:15:50.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:49 smithi102 ceph-mon[100296]: mds.? [v2:172.21.15.102:6834/2398692013,v1:172.21.15.102:6835/2398692013] up:boot 2023-11-27T15:15:50.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:49 smithi102 ceph-mon[100296]: fsmap 5 up:standby 2023-11-27T15:15:50.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:49 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-27T15:15:50.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:49 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1976358754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:15:50.430 INFO:journalctl@ceph.mds.f.smithi150.stdout:Nov 27 15:15:50 smithi150 systemd[1]: Starting Ceph mds.f for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:15:50.735 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:50 smithi150 ceph-mon[100559]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:50.735 INFO:journalctl@ceph.mds.f.smithi150.stdout:Nov 27 15:15:50 smithi150 podman[125892]: 2023-11-27T15:15:50.735 INFO:journalctl@ceph.mds.f.smithi150.stdout:Nov 27 15:15:50 smithi150 ceph-mds[125932]: -- 172.21.15.150:0/3362637231 <== mon.1 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562bff759520 con 0x562bff72f000 2023-11-27T15:15:50.736 INFO:journalctl@ceph.mds.f.smithi150.stdout:Nov 27 15:15:50 smithi150 bash[125892]: f6f622f9962d66f61f832231fc31250ac1f3f88cc92d0b8fe63db44b96f0460d 2023-11-27T15:15:50.736 INFO:journalctl@ceph.mds.f.smithi150.stdout:Nov 27 15:15:50 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-f[125908]: starting mds.f at 2023-11-27T15:15:50.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:50 smithi052 ceph-mon[99056]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:51.028 INFO:journalctl@ceph.mds.f.smithi150.stdout:Nov 27 15:15:50 smithi150 systemd[1]: Started Ceph mds.f for c6c7df88-8d36-11ee-95a2-87774f69a715. 2023-11-27T15:15:51.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:50 smithi102 ceph-mon[100296]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:51.078 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:15:51.220 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.214+0000 7f27bcc4d700 1 -- 172.21.15.52:0/821345360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27b80ff170 msgr2=0x7f27b80ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:51.220 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.214+0000 7f27bcc4d700 1 --2- 172.21.15.52:0/821345360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27b80ff170 0x7f27b80ff580 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f27ac009990 tx=0x7f27ac009ca0 comp rx=0 tx=0).stop 2023-11-27T15:15:51.220 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.214+0000 7f27bcc4d700 1 -- 172.21.15.52:0/821345360 shutdown_connections 2023-11-27T15:15:51.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.214+0000 7f27bcc4d700 1 --2- 172.21.15.52:0/821345360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27b8100db0 0x7f27b8107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:51.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.214+0000 7f27bcc4d700 1 --2- 172.21.15.52:0/821345360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27b80ff170 0x7f27b80ff580 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:51.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.214+0000 7f27bcc4d700 1 --2- 172.21.15.52:0/821345360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27b8100400 0x7f27b8100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:51.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.214+0000 7f27bcc4d700 1 -- 172.21.15.52:0/821345360 >> 172.21.15.52:0/821345360 conn(0x7f27b80fa750 msgr2=0x7f27b80fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:51.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.215+0000 7f27bcc4d700 1 -- 172.21.15.52:0/821345360 shutdown_connections 2023-11-27T15:15:51.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.215+0000 7f27bcc4d700 1 -- 172.21.15.52:0/821345360 wait complete. 2023-11-27T15:15:51.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.215+0000 7f27bcc4d700 1 Processor -- start 2023-11-27T15:15:51.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.215+0000 7f27bcc4d700 1 -- start start 2023-11-27T15:15:51.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.216+0000 7f27bcc4d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27b80ff170 0x7f27b8194ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:51.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.216+0000 7f27bcc4d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27b8100400 0x7f27b81953e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:51.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.216+0000 7f27bcc4d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27b8100db0 0x7f27b819c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:51.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.216+0000 7f27bcc4d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b819ca20 con 0x7f27b8100db0 2023-11-27T15:15:51.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.216+0000 7f27bcc4d700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b819cb60 con 0x7f27b80ff170 2023-11-27T15:15:51.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.216+0000 7f27bcc4d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b819ccf0 con 0x7f27b8100400 2023-11-27T15:15:51.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.216+0000 7f27b6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27b8100db0 0x7f27b819c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:51.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.216+0000 7f27b6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27b8100db0 0x7f27b819c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49820/0 (socket says 172.21.15.52:49820) 2023-11-27T15:15:51.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.216+0000 7f27b6d9d700 1 -- 172.21.15.52:0/234890885 learned_addr learned my addr 172.21.15.52:0/234890885 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:15:51.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.216+0000 7f27b659c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27b80ff170 0x7f27b8194ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:51.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.216+0000 7f27b5d9b700 1 --2- 172.21.15.52:0/234890885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27b8100400 0x7f27b81953e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:51.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.216+0000 7f27b6d9d700 1 -- 172.21.15.52:0/234890885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27b8100400 msgr2=0x7f27b81953e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:51.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.216+0000 7f27b6d9d700 1 --2- 172.21.15.52:0/234890885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27b8100400 0x7f27b81953e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:51.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.216+0000 7f27b6d9d700 1 -- 172.21.15.52:0/234890885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27b80ff170 msgr2=0x7f27b8194ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:51.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.216+0000 7f27b6d9d700 1 --2- 172.21.15.52:0/234890885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27b80ff170 0x7f27b8194ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:51.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.216+0000 7f27b6d9d700 1 -- 172.21.15.52:0/234890885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ac0095f0 con 0x7f27b8100db0 2023-11-27T15:15:51.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.216+0000 7f27b6d9d700 1 --2- 172.21.15.52:0/234890885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27b8100db0 0x7f27b819c450 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f27b0007ee0 tx=0x7f27b000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:51.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.216+0000 7f27a77fe700 1 -- 172.21.15.52:0/234890885 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b0017070 con 0x7f27b8100db0 2023-11-27T15:15:51.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.216+0000 7f27a77fe700 1 -- 172.21.15.52:0/234890885 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27b000f040 con 0x7f27b8100db0 2023-11-27T15:15:51.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.216+0000 7f27bcc4d700 1 -- 172.21.15.52:0/234890885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b810ab60 con 0x7f27b8100db0 2023-11-27T15:15:51.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.216+0000 7f27a77fe700 1 -- 172.21.15.52:0/234890885 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b0005580 con 0x7f27b8100db0 2023-11-27T15:15:51.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.216+0000 7f27bcc4d700 1 -- 172.21.15.52:0/234890885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b810a9d0 con 0x7f27b8100db0 2023-11-27T15:15:51.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.217+0000 7f27bcc4d700 1 -- 172.21.15.52:0/234890885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b8067a60 con 0x7f27b8100db0 2023-11-27T15:15:51.226 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.219+0000 7f27a77fe700 1 -- 172.21.15.52:0/234890885 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f27b000db20 con 0x7f27b8100db0 2023-11-27T15:15:51.226 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.219+0000 7f27a77fe700 1 --2- 172.21.15.52:0/234890885 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f27a00a1150 0x7f27a00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:51.226 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.219+0000 7f27b659c700 1 --2- 172.21.15.52:0/234890885 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f27a00a1150 0x7f27a00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:51.226 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.219+0000 7f27a77fe700 1 -- 172.21.15.52:0/234890885 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f27b00cc1c0 con 0x7f27b8100db0 2023-11-27T15:15:51.226 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.220+0000 7f27b659c700 1 --2- 172.21.15.52:0/234890885 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f27a00a1150 0x7f27a00a3600 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f27ac00b2e0 tx=0x7f27ac005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:51.228 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.223+0000 7f27a77fe700 1 -- 172.21.15.52:0/234890885 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f27b0095db0 con 0x7f27b8100db0 2023-11-27T15:15:51.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.469+0000 7f27bcc4d700 1 -- 172.21.15.52:0/234890885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27b8106140 con 0x7f27b8100db0 2023-11-27T15:15:51.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.469+0000 7f27a77fe700 1 -- 172.21.15.52:0/234890885 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4291 (secure 0 0 0) 0x7f27b00993d0 con 0x7f27b8100db0 2023-11-27T15:15:51.475 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:15:51.475 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-11-27T15:15:51.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.471+0000 7f27bcc4d700 1 -- 172.21.15.52:0/234890885 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f27a00a1150 msgr2=0x7f27a00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:51.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.471+0000 7f27bcc4d700 1 --2- 172.21.15.52:0/234890885 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f27a00a1150 0x7f27a00a3600 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f27ac00b2e0 tx=0x7f27ac005fb0 comp rx=0 tx=0).stop 2023-11-27T15:15:51.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.471+0000 7f27bcc4d700 1 -- 172.21.15.52:0/234890885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27b8100db0 msgr2=0x7f27b819c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:51.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.471+0000 7f27bcc4d700 1 --2- 172.21.15.52:0/234890885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27b8100db0 0x7f27b819c450 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f27b0007ee0 tx=0x7f27b000d3b0 comp rx=0 tx=0).stop 2023-11-27T15:15:51.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.471+0000 7f27bcc4d700 1 -- 172.21.15.52:0/234890885 shutdown_connections 2023-11-27T15:15:51.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.471+0000 7f27bcc4d700 1 --2- 172.21.15.52:0/234890885 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f27a00a1150 0x7f27a00a3600 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:51.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.471+0000 7f27bcc4d700 1 --2- 172.21.15.52:0/234890885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27b80ff170 0x7f27b8194ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:51.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.471+0000 7f27bcc4d700 1 --2- 172.21.15.52:0/234890885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27b8100db0 0x7f27b819c450 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:51.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.471+0000 7f27bcc4d700 1 --2- 172.21.15.52:0/234890885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27b8100400 0x7f27b81953e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:51.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.471+0000 7f27bcc4d700 1 -- 172.21.15.52:0/234890885 >> 172.21.15.52:0/234890885 conn(0x7f27b80fa750 msgr2=0x7f27b80fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.471+0000 7f27bcc4d700 1 -- 172.21.15.52:0/234890885 shutdown_connections 2023-11-27T15:15:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:51.471+0000 7f27bcc4d700 1 -- 172.21.15.52:0/234890885 wait complete. 2023-11-27T15:15:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 6 2023-11-27T15:15:52.065 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:51 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:52.065 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:51 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:52.065 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:51 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:52.065 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:51 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:15:52.065 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:51 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:15:52.066 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:51 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:52.066 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:51 smithi150 ceph-mon[100559]: Deploying daemon mds.c on smithi150 2023-11-27T15:15:52.066 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:51 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/234890885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:15:52.066 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:51 smithi150 ceph-mon[100559]: mds.? [v2:172.21.15.150:6834/617413375,v1:172.21.15.150:6835/617413375] up:boot 2023-11-27T15:15:52.066 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:51 smithi150 ceph-mon[100559]: fsmap 6 up:standby 2023-11-27T15:15:52.066 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:51 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-27T15:15:52.066 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:51 smithi150 ceph-mon[100559]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:52.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:51 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:52.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:51 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:52.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:51 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:52.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:51 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:15:52.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:51 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:15:52.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:51 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:52.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:51 smithi102 ceph-mon[100296]: Deploying daemon mds.c on smithi150 2023-11-27T15:15:52.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:51 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/234890885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:15:52.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:51 smithi102 ceph-mon[100296]: mds.? [v2:172.21.15.150:6834/617413375,v1:172.21.15.150:6835/617413375] up:boot 2023-11-27T15:15:52.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:51 smithi102 ceph-mon[100296]: fsmap 6 up:standby 2023-11-27T15:15:52.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:51 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-27T15:15:52.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:51 smithi102 ceph-mon[100296]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:52.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:51 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:52.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:51 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:52.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:51 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:52.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:51 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:15:52.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:51 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:15:52.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:51 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:52.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:51 smithi052 ceph-mon[99056]: Deploying daemon mds.c on smithi150 2023-11-27T15:15:52.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:51 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/234890885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:15:52.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:51 smithi052 ceph-mon[99056]: mds.? [v2:172.21.15.150:6834/617413375,v1:172.21.15.150:6835/617413375] up:boot 2023-11-27T15:15:52.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:51 smithi052 ceph-mon[99056]: fsmap 6 up:standby 2023-11-27T15:15:52.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:51 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-27T15:15:52.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:51 smithi052 ceph-mon[99056]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:52.529 INFO:journalctl@ceph.mds.c.smithi150.stdout:Nov 27 15:15:52 smithi150 systemd[1]: Starting Ceph mds.c for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:15:52.864 INFO:journalctl@ceph.mds.c.smithi150.stdout:Nov 27 15:15:52 smithi150 podman[126235]: 2023-11-27T15:15:53.278 INFO:journalctl@ceph.mds.c.smithi150.stdout:Nov 27 15:15:52 smithi150 ceph-mds[126275]: -- 172.21.15.150:0/2573909197 <== mon.1 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560598f9b520 con 0x560598f70800 2023-11-27T15:15:53.278 INFO:journalctl@ceph.mds.c.smithi150.stdout:Nov 27 15:15:52 smithi150 bash[126235]: e7d32af36f97fcbe70537dd4b52c91dad650d416898e1f3a98e7103917c2861d 2023-11-27T15:15:53.279 INFO:journalctl@ceph.mds.c.smithi150.stdout:Nov 27 15:15:52 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-c[126251]: starting mds.c at 2023-11-27T15:15:53.279 INFO:journalctl@ceph.mds.c.smithi150.stdout:Nov 27 15:15:52 smithi150 systemd[1]: Started Ceph mds.c for c6c7df88-8d36-11ee-95a2-87774f69a715. 2023-11-27T15:15:53.490 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:15:53.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.626+0000 7ff60d879700 1 -- 172.21.15.52:0/2061909588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff608100400 msgr2=0x7ff608100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:53.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.626+0000 7ff60d879700 1 --2- 172.21.15.52:0/2061909588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff608100400 0x7ff608100870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc009990 tx=0x7ff5fc009ca0 comp rx=0 tx=0).stop 2023-11-27T15:15:53.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.626+0000 7ff60d879700 1 -- 172.21.15.52:0/2061909588 shutdown_connections 2023-11-27T15:15:53.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.626+0000 7ff60d879700 1 --2- 172.21.15.52:0/2061909588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6080ff170 0x7ff6080ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:53.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.626+0000 7ff60d879700 1 --2- 172.21.15.52:0/2061909588 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff608100db0 0x7ff608107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:53.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.626+0000 7ff60d879700 1 --2- 172.21.15.52:0/2061909588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff608100400 0x7ff608100870 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:53.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.626+0000 7ff60d879700 1 -- 172.21.15.52:0/2061909588 >> 172.21.15.52:0/2061909588 conn(0x7ff6080fa750 msgr2=0x7ff6080fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:53.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.626+0000 7ff60d879700 1 -- 172.21.15.52:0/2061909588 shutdown_connections 2023-11-27T15:15:53.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.626+0000 7ff60d879700 1 -- 172.21.15.52:0/2061909588 wait complete. 2023-11-27T15:15:53.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.627+0000 7ff60d879700 1 Processor -- start 2023-11-27T15:15:53.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.627+0000 7ff60d879700 1 -- start start 2023-11-27T15:15:53.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.627+0000 7ff60d879700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6080ff170 0x7ff608195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:53.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.627+0000 7ff60d879700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff608100400 0x7ff6081964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:53.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.627+0000 7ff60d879700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff608100db0 0x7ff60819d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:53.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.627+0000 7ff60d879700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60819daf0 con 0x7ff608100db0 2023-11-27T15:15:53.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.627+0000 7ff60d879700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60819dc30 con 0x7ff608100400 2023-11-27T15:15:53.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.627+0000 7ff60d879700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60819ddc0 con 0x7ff6080ff170 2023-11-27T15:15:53.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.627+0000 7ff6077fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff608100db0 0x7ff60819d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:53.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.627+0000 7ff6067fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff608100400 0x7ff6081964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:53.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.627+0000 7ff6077fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff608100db0 0x7ff60819d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49834/0 (socket says 172.21.15.52:49834) 2023-11-27T15:15:53.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.627+0000 7ff6077fe700 1 -- 172.21.15.52:0/2815248523 learned_addr learned my addr 172.21.15.52:0/2815248523 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:15:53.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.627+0000 7ff606ffd700 1 --2- 172.21.15.52:0/2815248523 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6080ff170 0x7ff608195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:53.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.628+0000 7ff6077fe700 1 -- 172.21.15.52:0/2815248523 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6080ff170 msgr2=0x7ff608195f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:53.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.628+0000 7ff6077fe700 1 --2- 172.21.15.52:0/2815248523 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6080ff170 0x7ff608195f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:53.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.628+0000 7ff6077fe700 1 -- 172.21.15.52:0/2815248523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff608100400 msgr2=0x7ff6081964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:53.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.628+0000 7ff6077fe700 1 --2- 172.21.15.52:0/2815248523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff608100400 0x7ff6081964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:53.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.628+0000 7ff6077fe700 1 -- 172.21.15.52:0/2815248523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5fc0095f0 con 0x7ff608100db0 2023-11-27T15:15:53.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.628+0000 7ff6067fc700 1 --2- 172.21.15.52:0/2815248523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff608100400 0x7ff6081964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:15:53.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.628+0000 7ff6077fe700 1 --2- 172.21.15.52:0/2815248523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff608100db0 0x7ff60819d520 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff60000ada0 tx=0x7ff60000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:53.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.628+0000 7ff60c877700 1 -- 172.21.15.52:0/2815248523 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60000f070 con 0x7ff608100db0 2023-11-27T15:15:53.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.628+0000 7ff60c877700 1 -- 172.21.15.52:0/2815248523 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff600007b30 con 0x7ff608100db0 2023-11-27T15:15:53.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.628+0000 7ff60c877700 1 -- 172.21.15.52:0/2815248523 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff600005280 con 0x7ff608100db0 2023-11-27T15:15:53.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.628+0000 7ff60d879700 1 -- 172.21.15.52:0/2815248523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff60810ab10 con 0x7ff608100db0 2023-11-27T15:15:53.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.628+0000 7ff60d879700 1 -- 172.21.15.52:0/2815248523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff60810a980 con 0x7ff608100db0 2023-11-27T15:15:53.638 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.629+0000 7ff60d879700 1 -- 172.21.15.52:0/2815248523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff608107360 con 0x7ff608100db0 2023-11-27T15:15:53.638 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.630+0000 7ff60c877700 1 -- 172.21.15.52:0/2815248523 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff60000a090 con 0x7ff608100db0 2023-11-27T15:15:53.638 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.631+0000 7ff60c877700 1 --2- 172.21.15.52:0/2815248523 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff5f00a11d0 0x7ff5f00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:53.638 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.631+0000 7ff606ffd700 1 --2- 172.21.15.52:0/2815248523 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff5f00a11d0 0x7ff5f00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:53.638 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.631+0000 7ff60c877700 1 -- 172.21.15.52:0/2815248523 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff6000cbf00 con 0x7ff608100db0 2023-11-27T15:15:53.638 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.631+0000 7ff606ffd700 1 --2- 172.21.15.52:0/2815248523 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff5f00a11d0 0x7ff5f00a3680 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009c60 tx=0x7ff5f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:53.640 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.634+0000 7ff60c877700 1 -- 172.21.15.52:0/2815248523 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff600099610 con 0x7ff608100db0 2023-11-27T15:15:53.885 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.880+0000 7ff60d879700 1 -- 172.21.15.52:0/2815248523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff60802cfe0 con 0x7ff608100db0 2023-11-27T15:15:53.886 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.880+0000 7ff60c877700 1 -- 172.21.15.52:0/2815248523 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5805 (secure 0 0 0) 0x7ff6000991a0 con 0x7ff608100db0 2023-11-27T15:15:53.886 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:15:53.886 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-11-27T15:15:53.888 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.882+0000 7ff60d879700 1 -- 172.21.15.52:0/2815248523 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff5f00a11d0 msgr2=0x7ff5f00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:53.888 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.883+0000 7ff60d879700 1 --2- 172.21.15.52:0/2815248523 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff5f00a11d0 0x7ff5f00a3680 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009c60 tx=0x7ff5f8009260 comp rx=0 tx=0).stop 2023-11-27T15:15:53.888 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.883+0000 7ff60d879700 1 -- 172.21.15.52:0/2815248523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff608100db0 msgr2=0x7ff60819d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:53.888 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.883+0000 7ff60d879700 1 --2- 172.21.15.52:0/2815248523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff608100db0 0x7ff60819d520 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff60000ada0 tx=0x7ff60000c330 comp rx=0 tx=0).stop 2023-11-27T15:15:53.888 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.883+0000 7ff60d879700 1 -- 172.21.15.52:0/2815248523 shutdown_connections 2023-11-27T15:15:53.889 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.883+0000 7ff60d879700 1 --2- 172.21.15.52:0/2815248523 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff5f00a11d0 0x7ff5f00a3680 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:53.889 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.883+0000 7ff60d879700 1 --2- 172.21.15.52:0/2815248523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff608100db0 0x7ff60819d520 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:53.889 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.883+0000 7ff60d879700 1 --2- 172.21.15.52:0/2815248523 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6080ff170 0x7ff608195f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:53.889 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.883+0000 7ff60d879700 1 --2- 172.21.15.52:0/2815248523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff608100400 0x7ff6081964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:53.889 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.883+0000 7ff60d879700 1 -- 172.21.15.52:0/2815248523 >> 172.21.15.52:0/2815248523 conn(0x7ff6080fa750 msgr2=0x7ff6080fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:53.889 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.883+0000 7ff60d879700 1 -- 172.21.15.52:0/2815248523 shutdown_connections 2023-11-27T15:15:53.889 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:53.883+0000 7ff60d879700 1 -- 172.21.15.52:0/2815248523 wait complete. 2023-11-27T15:15:53.889 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 8 2023-11-27T15:15:54.053 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:53 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:54.053 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:53 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:54.053 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:53 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:54.054 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:53 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:15:54.054 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:53 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:15:54.054 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:53 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:54.054 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:53 smithi102 ceph-mon[100296]: Deploying daemon mds.h on smithi102 2023-11-27T15:15:54.054 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:53 smithi102 ceph-mon[100296]: mds.? [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] up:boot 2023-11-27T15:15:54.055 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:53 smithi102 ceph-mon[100296]: fsmap 7 up:standby 2023-11-27T15:15:54.055 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:53 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-27T15:15:54.055 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:53 smithi102 ceph-mon[100296]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:54.055 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:53 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2815248523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:15:54.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:53 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:54.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:53 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:54.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:53 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:54.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:53 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:15:54.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:53 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:15:54.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:53 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:54.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:53 smithi150 ceph-mon[100559]: Deploying daemon mds.h on smithi102 2023-11-27T15:15:54.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:53 smithi150 ceph-mon[100559]: mds.? [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] up:boot 2023-11-27T15:15:54.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:53 smithi150 ceph-mon[100559]: fsmap 7 up:standby 2023-11-27T15:15:54.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:53 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-27T15:15:54.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:53 smithi150 ceph-mon[100559]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:54.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:53 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2815248523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:15:54.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:53 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:54.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:53 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:54.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:53 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:54.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:53 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:15:54.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:53 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:15:54.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:53 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:54.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:53 smithi052 ceph-mon[99056]: Deploying daemon mds.h on smithi102 2023-11-27T15:15:54.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:53 smithi052 ceph-mon[99056]: mds.? [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] up:boot 2023-11-27T15:15:54.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:53 smithi052 ceph-mon[99056]: fsmap 7 up:standby 2023-11-27T15:15:54.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:53 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-27T15:15:54.369 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:53 smithi052 ceph-mon[99056]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:54.369 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:53 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2815248523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:15:54.785 INFO:journalctl@ceph.mds.h.smithi102.stdout:Nov 27 15:15:54 smithi102 systemd[1]: Starting Ceph mds.h for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:15:55.035 INFO:journalctl@ceph.mds.h.smithi102.stdout:Nov 27 15:15:54 smithi102 podman[125141]: 2023-11-27T15:15:55.035 INFO:journalctl@ceph.mds.h.smithi102.stdout:Nov 27 15:15:55 smithi102 ceph-mds[125181]: -- 172.21.15.102:0/3112966598 <== mon.2 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562c78751520 con 0x562c78726800 2023-11-27T15:15:55.328 INFO:journalctl@ceph.mds.h.smithi102.stdout:Nov 27 15:15:55 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-h[125157]: starting mds.h at 2023-11-27T15:15:55.328 INFO:journalctl@ceph.mds.h.smithi102.stdout:Nov 27 15:15:55 smithi102 bash[125141]: b36eecfb36982b5b70fede11d7660159c06e91e87642502702a8581734b057d6 2023-11-27T15:15:55.328 INFO:journalctl@ceph.mds.h.smithi102.stdout:Nov 27 15:15:55 smithi102 systemd[1]: Started Ceph mds.h for c6c7df88-8d36-11ee-95a2-87774f69a715. 2023-11-27T15:15:55.901 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:15:56.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.033+0000 7f18ec5fc700 1 -- 172.21.15.52:0/3632741324 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18e4100d40 msgr2=0x7f18e4107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:56.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.033+0000 7f18ec5fc700 1 --2- 172.21.15.52:0/3632741324 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18e4100d40 0x7f18e4107600 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00b0a0 tx=0x7f18dc00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:15:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.033+0000 7f18ec5fc700 1 -- 172.21.15.52:0/3632741324 shutdown_connections 2023-11-27T15:15:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.033+0000 7f18ec5fc700 1 --2- 172.21.15.52:0/3632741324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18e40ff1c0 0x7f18e40ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.033+0000 7f18ec5fc700 1 --2- 172.21.15.52:0/3632741324 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18e4100d40 0x7f18e4107600 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.033+0000 7f18ec5fc700 1 --2- 172.21.15.52:0/3632741324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18e41003b0 0x7f18e4100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.033+0000 7f18ec5fc700 1 -- 172.21.15.52:0/3632741324 >> 172.21.15.52:0/3632741324 conn(0x7f18e40fa7c0 msgr2=0x7f18e40fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.033+0000 7f18ec5fc700 1 -- 172.21.15.52:0/3632741324 shutdown_connections 2023-11-27T15:15:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.033+0000 7f18ec5fc700 1 -- 172.21.15.52:0/3632741324 wait complete. 2023-11-27T15:15:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.033+0000 7f18ec5fc700 1 Processor -- start 2023-11-27T15:15:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.034+0000 7f18ec5fc700 1 -- start start 2023-11-27T15:15:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.034+0000 7f18ec5fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18e40ff1c0 0x7f18e4196060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.034+0000 7f18ec5fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18e41003b0 0x7f18e41965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.034+0000 7f18ec5fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18e4100d40 0x7f18e419d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:56.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.034+0000 7f18ec5fc700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e419dbe0 con 0x7f18e4100d40 2023-11-27T15:15:56.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.034+0000 7f18ec5fc700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e419dd20 con 0x7f18e40ff1c0 2023-11-27T15:15:56.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.034+0000 7f18ec5fc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e419deb0 con 0x7f18e41003b0 2023-11-27T15:15:56.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.034+0000 7f18eab99700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18e4100d40 0x7f18e419d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:56.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.034+0000 7f18eab99700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18e4100d40 0x7f18e419d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49848/0 (socket says 172.21.15.52:49848) 2023-11-27T15:15:56.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.034+0000 7f18eab99700 1 -- 172.21.15.52:0/2182226435 learned_addr learned my addr 172.21.15.52:0/2182226435 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:15:56.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.034+0000 7f18e9b97700 1 --2- 172.21.15.52:0/2182226435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18e41003b0 0x7f18e41965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:56.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.034+0000 7f18ea398700 1 --2- 172.21.15.52:0/2182226435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18e40ff1c0 0x7f18e4196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:56.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.034+0000 7f18eab99700 1 -- 172.21.15.52:0/2182226435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18e41003b0 msgr2=0x7f18e41965a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:56.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.034+0000 7f18eab99700 1 --2- 172.21.15.52:0/2182226435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18e41003b0 0x7f18e41965a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:56.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.034+0000 7f18eab99700 1 -- 172.21.15.52:0/2182226435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18e40ff1c0 msgr2=0x7f18e4196060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:56.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.034+0000 7f18eab99700 1 --2- 172.21.15.52:0/2182226435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18e40ff1c0 0x7f18e4196060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:56.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.034+0000 7f18eab99700 1 -- 172.21.15.52:0/2182226435 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18dc009d00 con 0x7f18e4100d40 2023-11-27T15:15:56.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.034+0000 7f18eab99700 1 --2- 172.21.15.52:0/2182226435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18e4100d40 0x7f18e419d610 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00d010 tx=0x7f18dc00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:56.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.034+0000 7f18db7fe700 1 -- 172.21.15.52:0/2182226435 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc00e040 con 0x7f18e4100d40 2023-11-27T15:15:56.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.035+0000 7f18db7fe700 1 -- 172.21.15.52:0/2182226435 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18dc00bdb0 con 0x7f18e4100d40 2023-11-27T15:15:56.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.035+0000 7f18ec5fc700 1 -- 172.21.15.52:0/2182226435 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e410ab80 con 0x7f18e4100d40 2023-11-27T15:15:56.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.035+0000 7f18db7fe700 1 -- 172.21.15.52:0/2182226435 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc005260 con 0x7f18e4100d40 2023-11-27T15:15:56.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.035+0000 7f18ec5fc700 1 -- 172.21.15.52:0/2182226435 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e410a9f0 con 0x7f18e4100d40 2023-11-27T15:15:56.044 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.036+0000 7f18ec5fc700 1 -- 172.21.15.52:0/2182226435 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18e419e4b0 con 0x7f18e4100d40 2023-11-27T15:15:56.044 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.037+0000 7f18db7fe700 1 -- 172.21.15.52:0/2182226435 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f18dc01d480 con 0x7f18e4100d40 2023-11-27T15:15:56.044 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.037+0000 7f18db7fe700 1 --2- 172.21.15.52:0/2182226435 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f18d00a11d0 0x7f18d00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:56.044 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.037+0000 7f18ea398700 1 --2- 172.21.15.52:0/2182226435 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f18d00a11d0 0x7f18d00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:56.044 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.037+0000 7f18db7fe700 1 -- 172.21.15.52:0/2182226435 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f18dc0cd7f0 con 0x7f18e4100d40 2023-11-27T15:15:56.045 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.038+0000 7f18ea398700 1 --2- 172.21.15.52:0/2182226435 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f18d00a11d0 0x7f18d00a3680 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f18e4100210 tx=0x7f18d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:56.046 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.041+0000 7f18db7fe700 1 -- 172.21.15.52:0/2182226435 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f18dc097520 con 0x7f18e4100d40 2023-11-27T15:15:56.216 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:56 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:56.216 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:56 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:56.216 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:56 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:56.216 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:56 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:15:56.216 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:56 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:15:56.216 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:56 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:56.216 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:56 smithi150 ceph-mon[100559]: Deploying daemon mds.i on smithi150 2023-11-27T15:15:56.216 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:56 smithi150 ceph-mon[100559]: mds.? [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] up:boot 2023-11-27T15:15:56.216 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:56 smithi150 ceph-mon[100559]: fsmap 8 up:standby 2023-11-27T15:15:56.216 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:56 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-27T15:15:56.217 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:56 smithi150 ceph-mon[100559]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:56.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.286+0000 7f18ec5fc700 1 -- 172.21.15.52:0/2182226435 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18e419ea60 con 0x7f18e4100d40 2023-11-27T15:15:56.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.287+0000 7f18db7fe700 1 -- 172.21.15.52:0/2182226435 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6565 (secure 0 0 0) 0x7f18dc09ab40 con 0x7f18e4100d40 2023-11-27T15:15:56.292 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:15:56.292 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-11-27T15:15:56.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.289+0000 7f18ec5fc700 1 -- 172.21.15.52:0/2182226435 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f18d00a11d0 msgr2=0x7f18d00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:56.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.289+0000 7f18ec5fc700 1 --2- 172.21.15.52:0/2182226435 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f18d00a11d0 0x7f18d00a3680 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f18e4100210 tx=0x7f18d4009260 comp rx=0 tx=0).stop 2023-11-27T15:15:56.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.289+0000 7f18ec5fc700 1 -- 172.21.15.52:0/2182226435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18e4100d40 msgr2=0x7f18e419d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:56.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.289+0000 7f18ec5fc700 1 --2- 172.21.15.52:0/2182226435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18e4100d40 0x7f18e419d610 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00d010 tx=0x7f18dc00bc80 comp rx=0 tx=0).stop 2023-11-27T15:15:56.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.289+0000 7f18ec5fc700 1 -- 172.21.15.52:0/2182226435 shutdown_connections 2023-11-27T15:15:56.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.289+0000 7f18ec5fc700 1 --2- 172.21.15.52:0/2182226435 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f18d00a11d0 0x7f18d00a3680 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:56.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.289+0000 7f18ec5fc700 1 --2- 172.21.15.52:0/2182226435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18e40ff1c0 0x7f18e4196060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:56.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.289+0000 7f18ec5fc700 1 --2- 172.21.15.52:0/2182226435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18e4100d40 0x7f18e419d610 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:56.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.289+0000 7f18ec5fc700 1 --2- 172.21.15.52:0/2182226435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18e41003b0 0x7f18e41965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:56.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.289+0000 7f18ec5fc700 1 -- 172.21.15.52:0/2182226435 >> 172.21.15.52:0/2182226435 conn(0x7f18e40fa7c0 msgr2=0x7f18e4069010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:56.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.289+0000 7f18ec5fc700 1 -- 172.21.15.52:0/2182226435 shutdown_connections 2023-11-27T15:15:56.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:56.289+0000 7f18ec5fc700 1 -- 172.21.15.52:0/2182226435 wait complete. 2023-11-27T15:15:56.296 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 9 2023-11-27T15:15:56.305 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:56 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:56.305 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:56 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:56.306 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:56 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:56.306 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:56 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:15:56.306 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:56 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:15:56.306 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:56 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:56.306 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:56 smithi052 ceph-mon[99056]: Deploying daemon mds.i on smithi150 2023-11-27T15:15:56.306 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:56 smithi052 ceph-mon[99056]: mds.? [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] up:boot 2023-11-27T15:15:56.306 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:56 smithi052 ceph-mon[99056]: fsmap 8 up:standby 2023-11-27T15:15:56.306 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:56 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-27T15:15:56.306 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:56 smithi052 ceph-mon[99056]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:56.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:56 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:56.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:56 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:56.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:56 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:56.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:56 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:15:56.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:56 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:15:56.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:56 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:56.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:56 smithi102 ceph-mon[100296]: Deploying daemon mds.i on smithi150 2023-11-27T15:15:56.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:56 smithi102 ceph-mon[100296]: mds.? [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] up:boot 2023-11-27T15:15:56.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:56 smithi102 ceph-mon[100296]: fsmap 8 up:standby 2023-11-27T15:15:56.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:56 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-27T15:15:56.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:56 smithi102 ceph-mon[100296]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:56.984 INFO:journalctl@ceph.mds.i.smithi150.stdout:Nov 27 15:15:56 smithi150 systemd[1]: Starting Ceph mds.i for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:15:57.263 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:57 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2182226435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:15:57.264 INFO:journalctl@ceph.mds.i.smithi150.stdout:Nov 27 15:15:56 smithi150 podman[126580]: 2023-11-27T15:15:57.528 INFO:journalctl@ceph.mds.i.smithi150.stdout:Nov 27 15:15:57 smithi150 ceph-mds[126620]: -- 172.21.15.150:0/2237905901 <== mon.1 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561f619bb520 con 0x561f61991000 2023-11-27T15:15:57.529 INFO:journalctl@ceph.mds.i.smithi150.stdout:Nov 27 15:15:57 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-i[126596]: starting mds.i at 2023-11-27T15:15:57.529 INFO:journalctl@ceph.mds.i.smithi150.stdout:Nov 27 15:15:57 smithi150 bash[126580]: 9d006c406e6d1aafc6167cb896fc083f390f2c627244d0a28a8e47b5c327f7e3 2023-11-27T15:15:57.529 INFO:journalctl@ceph.mds.i.smithi150.stdout:Nov 27 15:15:57 smithi150 systemd[1]: Started Ceph mds.i for c6c7df88-8d36-11ee-95a2-87774f69a715. 2023-11-27T15:15:57.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:57 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2182226435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:15:57.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:57 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2182226435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:15:58.308 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:15:58.439 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.433+0000 7f1bd0fb5700 1 -- 172.21.15.52:0/1232019498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bcc100400 msgr2=0x7f1bcc100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:58.439 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.433+0000 7f1bd0fb5700 1 --2- 172.21.15.52:0/1232019498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bcc100400 0x7f1bcc100870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4009990 tx=0x7f1bb4009ca0 comp rx=0 tx=0).stop 2023-11-27T15:15:58.439 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.433+0000 7f1bd0fb5700 1 -- 172.21.15.52:0/1232019498 shutdown_connections 2023-11-27T15:15:58.439 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.433+0000 7f1bd0fb5700 1 --2- 172.21.15.52:0/1232019498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bcc100db0 0x7f1bcc107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:58.439 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.433+0000 7f1bd0fb5700 1 --2- 172.21.15.52:0/1232019498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bcc0ff170 0x7f1bcc0ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:58.439 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.433+0000 7f1bd0fb5700 1 --2- 172.21.15.52:0/1232019498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bcc100400 0x7f1bcc100870 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:58.439 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.433+0000 7f1bd0fb5700 1 -- 172.21.15.52:0/1232019498 >> 172.21.15.52:0/1232019498 conn(0x7f1bcc0fa750 msgr2=0x7f1bcc0fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:58.439 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.433+0000 7f1bd0fb5700 1 -- 172.21.15.52:0/1232019498 shutdown_connections 2023-11-27T15:15:58.440 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.433+0000 7f1bd0fb5700 1 -- 172.21.15.52:0/1232019498 wait complete. 2023-11-27T15:15:58.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.434+0000 7f1bd0fb5700 1 Processor -- start 2023-11-27T15:15:58.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.434+0000 7f1bd0fb5700 1 -- start start 2023-11-27T15:15:58.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.435+0000 7f1bd0fb5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bcc0ff170 0x7f1bcc1031a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:58.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.435+0000 7f1bd0fb5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bcc100400 0x7f1bcc1017e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:58.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.435+0000 7f1bd0fb5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bcc100db0 0x7f1bcc101d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:58.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.435+0000 7f1bd0fb5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc1024c0 con 0x7f1bcc0ff170 2023-11-27T15:15:58.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.435+0000 7f1bd0fb5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc102630 con 0x7f1bcc100400 2023-11-27T15:15:58.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.435+0000 7f1bd0fb5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc1a0000 con 0x7f1bcc100db0 2023-11-27T15:15:58.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.435+0000 7f1bca59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bcc0ff170 0x7f1bcc1031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:58.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.435+0000 7f1bca59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bcc0ff170 0x7f1bcc1031a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49858/0 (socket says 172.21.15.52:49858) 2023-11-27T15:15:58.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.435+0000 7f1bca59c700 1 -- 172.21.15.52:0/4141736218 learned_addr learned my addr 172.21.15.52:0/4141736218 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:15:58.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.435+0000 7f1bc9d9b700 1 --2- 172.21.15.52:0/4141736218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bcc100400 0x7f1bcc1017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:58.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.435+0000 7f1bcad9d700 1 --2- 172.21.15.52:0/4141736218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bcc100db0 0x7f1bcc101d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:58.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.435+0000 7f1bca59c700 1 -- 172.21.15.52:0/4141736218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bcc100db0 msgr2=0x7f1bcc101d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:58.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.435+0000 7f1bca59c700 1 --2- 172.21.15.52:0/4141736218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bcc100db0 0x7f1bcc101d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:58.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.435+0000 7f1bca59c700 1 -- 172.21.15.52:0/4141736218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bcc100400 msgr2=0x7f1bcc1017e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:58.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.435+0000 7f1bca59c700 1 --2- 172.21.15.52:0/4141736218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bcc100400 0x7f1bcc1017e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:58.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.435+0000 7f1bca59c700 1 -- 172.21.15.52:0/4141736218 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb40095f0 con 0x7f1bcc0ff170 2023-11-27T15:15:58.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.435+0000 7f1bca59c700 1 --2- 172.21.15.52:0/4141736218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bcc0ff170 0x7f1bcc1031a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00d950 tx=0x7f1bbc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:58.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.436+0000 7f1bc37fe700 1 -- 172.21.15.52:0/4141736218 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc00f890 con 0x7f1bcc0ff170 2023-11-27T15:15:58.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.436+0000 7f1bd0fb5700 1 -- 172.21.15.52:0/4141736218 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bcc10abc0 con 0x7f1bcc0ff170 2023-11-27T15:15:58.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.436+0000 7f1bc37fe700 1 -- 172.21.15.52:0/4141736218 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bbc010460 con 0x7f1bcc0ff170 2023-11-27T15:15:58.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.436+0000 7f1bc37fe700 1 -- 172.21.15.52:0/4141736218 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc00e610 con 0x7f1bcc0ff170 2023-11-27T15:15:58.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.436+0000 7f1bd0fb5700 1 -- 172.21.15.52:0/4141736218 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bcc10aa30 con 0x7f1bcc0ff170 2023-11-27T15:15:58.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.437+0000 7f1bd0fb5700 1 -- 172.21.15.52:0/4141736218 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bcc1a0740 con 0x7f1bcc0ff170 2023-11-27T15:15:58.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.438+0000 7f1bc37fe700 1 -- 172.21.15.52:0/4141736218 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1bbc0108d0 con 0x7f1bcc0ff170 2023-11-27T15:15:58.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.438+0000 7f1bc37fe700 1 --2- 172.21.15.52:0/4141736218 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f1bb80a1150 0x7f1bb80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:58.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.438+0000 7f1bc37fe700 1 -- 172.21.15.52:0/4141736218 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1bbc0cc250 con 0x7f1bcc0ff170 2023-11-27T15:15:58.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.438+0000 7f1bc9d9b700 1 --2- 172.21.15.52:0/4141736218 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f1bb80a1150 0x7f1bb80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:58.445 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:58 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:58.445 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:58 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:58.445 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:58 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:58.445 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:58 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:15:58.446 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:58 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:15:58.446 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:58 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:58.446 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:58 smithi052 ceph-mon[99056]: Deploying daemon mds.g on smithi052 2023-11-27T15:15:58.446 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:58 smithi052 ceph-mon[99056]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:58.446 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:58 smithi052 ceph-mon[99056]: mds.? [v2:172.21.15.150:6838/215725838,v1:172.21.15.150:6839/215725838] up:boot 2023-11-27T15:15:58.446 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:58 smithi052 ceph-mon[99056]: fsmap 9 up:standby 2023-11-27T15:15:58.446 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:58 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-27T15:15:58.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.439+0000 7f1bc9d9b700 1 --2- 172.21.15.52:0/4141736218 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f1bb80a1150 0x7f1bb80a3600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1bb400b5c0 tx=0x7f1bb4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:58.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.443+0000 7f1bc37fe700 1 -- 172.21.15.52:0/4141736218 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1bbc095e40 con 0x7f1bcc0ff170 2023-11-27T15:15:58.708 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.703+0000 7f1bd0fb5700 1 -- 172.21.15.52:0/4141736218 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1bcc1a0de0 con 0x7f1bcc0ff170 2023-11-27T15:15:58.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.703+0000 7f1bc37fe700 1 -- 172.21.15.52:0/4141736218 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7324 (secure 0 0 0) 0x7f1bbc099460 con 0x7f1bcc0ff170 2023-11-27T15:15:58.709 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:15:58.709 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/215725838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":215725838},{"type":"v1","addr":"172.21.15.150:6839","nonce":215725838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-11-27T15:15:58.711 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.705+0000 7f1bd0fb5700 1 -- 172.21.15.52:0/4141736218 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f1bb80a1150 msgr2=0x7f1bb80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:58.711 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.705+0000 7f1bd0fb5700 1 --2- 172.21.15.52:0/4141736218 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f1bb80a1150 0x7f1bb80a3600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1bb400b5c0 tx=0x7f1bb4005fb0 comp rx=0 tx=0).stop 2023-11-27T15:15:58.711 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.705+0000 7f1bd0fb5700 1 -- 172.21.15.52:0/4141736218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bcc0ff170 msgr2=0x7f1bcc1031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:58.711 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.705+0000 7f1bd0fb5700 1 --2- 172.21.15.52:0/4141736218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bcc0ff170 0x7f1bcc1031a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00d950 tx=0x7f1bbc00dc60 comp rx=0 tx=0).stop 2023-11-27T15:15:58.712 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.705+0000 7f1bd0fb5700 1 -- 172.21.15.52:0/4141736218 shutdown_connections 2023-11-27T15:15:58.712 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.705+0000 7f1bd0fb5700 1 --2- 172.21.15.52:0/4141736218 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f1bb80a1150 0x7f1bb80a3600 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:58.712 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.705+0000 7f1bd0fb5700 1 --2- 172.21.15.52:0/4141736218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bcc100db0 0x7f1bcc101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:58.712 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.705+0000 7f1bd0fb5700 1 --2- 172.21.15.52:0/4141736218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bcc0ff170 0x7f1bcc1031a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:58.712 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.705+0000 7f1bd0fb5700 1 --2- 172.21.15.52:0/4141736218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bcc100400 0x7f1bcc1017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:58.712 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.705+0000 7f1bd0fb5700 1 -- 172.21.15.52:0/4141736218 >> 172.21.15.52:0/4141736218 conn(0x7f1bcc0fa750 msgr2=0x7f1bcc067dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:58.712 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.706+0000 7f1bd0fb5700 1 -- 172.21.15.52:0/4141736218 shutdown_connections 2023-11-27T15:15:58.712 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:15:58.706+0000 7f1bd0fb5700 1 -- 172.21.15.52:0/4141736218 wait complete. 2023-11-27T15:15:58.712 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 10 2023-11-27T15:15:58.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:58 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:58.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:58 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:58.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:58 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:58.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:58 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:15:58.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:58 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:15:58.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:58 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:58.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:58 smithi150 ceph-mon[100559]: Deploying daemon mds.g on smithi052 2023-11-27T15:15:58.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:58 smithi150 ceph-mon[100559]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:58.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:58 smithi150 ceph-mon[100559]: mds.? [v2:172.21.15.150:6838/215725838,v1:172.21.15.150:6839/215725838] up:boot 2023-11-27T15:15:58.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:58 smithi150 ceph-mon[100559]: fsmap 9 up:standby 2023-11-27T15:15:58.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:58 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-27T15:15:58.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:58 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:58.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:58 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:58.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:58 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:15:58.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:58 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:15:58.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:58 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:15:58.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:58 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:58.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:58 smithi102 ceph-mon[100296]: Deploying daemon mds.g on smithi052 2023-11-27T15:15:58.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:58 smithi102 ceph-mon[100296]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:15:58.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:58 smithi102 ceph-mon[100296]: mds.? [v2:172.21.15.150:6838/215725838,v1:172.21.15.150:6839/215725838] up:boot 2023-11-27T15:15:58.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:58 smithi102 ceph-mon[100296]: fsmap 9 up:standby 2023-11-27T15:15:58.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:58 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-27T15:15:59.367 INFO:journalctl@ceph.mds.g.smithi052.stdout:Nov 27 15:15:59 smithi052 systemd[1]: Starting Ceph mds.g for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:15:59.774 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:15:59 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/4141736218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:15:59.775 INFO:journalctl@ceph.mds.g.smithi052.stdout:Nov 27 15:15:59 smithi052 podman[133243]: 2023-11-27T15:16:00.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:15:59 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/4141736218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:16:00.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:15:59 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/4141736218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:16:00.117 INFO:journalctl@ceph.mds.g.smithi052.stdout:Nov 27 15:15:59 smithi052 ceph-mds[133283]: -- 172.21.15.52:0/3653445544 <== mon.0 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c65662b520 con 0x55c656600800 2023-11-27T15:16:00.117 INFO:journalctl@ceph.mds.g.smithi052.stdout:Nov 27 15:15:59 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-g[133259]: starting mds.g at 2023-11-27T15:16:00.117 INFO:journalctl@ceph.mds.g.smithi052.stdout:Nov 27 15:15:59 smithi052 bash[133243]: a85c57f9d522ae3e0c74d39246abe16e364f5299cd7ba415a9df162afbb85277 2023-11-27T15:16:00.117 INFO:journalctl@ceph.mds.g.smithi052.stdout:Nov 27 15:15:59 smithi052 systemd[1]: Started Ceph mds.g for c6c7df88-8d36-11ee-95a2-87774f69a715. 2023-11-27T15:16:00.724 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:16:00.861 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.853+0000 7fb39ba4b700 1 -- 172.21.15.52:0/2415577625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb394100db0 msgr2=0x7fb394107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:00.861 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.853+0000 7fb39ba4b700 1 --2- 172.21.15.52:0/2415577625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb394100db0 0x7fb394107650 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb38c00b0a0 tx=0x7fb38c00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:16:00.861 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.854+0000 7fb39ba4b700 1 -- 172.21.15.52:0/2415577625 shutdown_connections 2023-11-27T15:16:00.861 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.854+0000 7fb39ba4b700 1 --2- 172.21.15.52:0/2415577625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb394100db0 0x7fb394107650 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:00.861 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.854+0000 7fb39ba4b700 1 --2- 172.21.15.52:0/2415577625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3940ff170 0x7fb3940ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:00.861 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.854+0000 7fb39ba4b700 1 --2- 172.21.15.52:0/2415577625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb394100400 0x7fb394100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:00.861 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.854+0000 7fb39ba4b700 1 -- 172.21.15.52:0/2415577625 >> 172.21.15.52:0/2415577625 conn(0x7fb3940fa750 msgr2=0x7fb3940fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:00.861 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.854+0000 7fb39ba4b700 1 -- 172.21.15.52:0/2415577625 shutdown_connections 2023-11-27T15:16:00.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.854+0000 7fb39ba4b700 1 -- 172.21.15.52:0/2415577625 wait complete. 2023-11-27T15:16:00.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.854+0000 7fb39ba4b700 1 Processor -- start 2023-11-27T15:16:00.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.854+0000 7fb39ba4b700 1 -- start start 2023-11-27T15:16:00.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.854+0000 7fb39ba4b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3940ff170 0x7fb394193e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:00.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.854+0000 7fb39ba4b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb394100400 0x7fb394194380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:00.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.854+0000 7fb39ba4b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb394100db0 0x7fb39419b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:00.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.854+0000 7fb39ba4b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39419b9c0 con 0x7fb394100400 2023-11-27T15:16:00.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.854+0000 7fb39ba4b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39419bb00 con 0x7fb3940ff170 2023-11-27T15:16:00.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.854+0000 7fb39ba4b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39419bc90 con 0x7fb394100db0 2023-11-27T15:16:00.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.855+0000 7fb398fe6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb394100400 0x7fb394194380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:00.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.855+0000 7fb398fe6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb394100400 0x7fb394194380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49888/0 (socket says 172.21.15.52:49888) 2023-11-27T15:16:00.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.855+0000 7fb398fe6700 1 -- 172.21.15.52:0/625257737 learned_addr learned my addr 172.21.15.52:0/625257737 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:00.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.855+0000 7fb3997e7700 1 --2- 172.21.15.52:0/625257737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3940ff170 0x7fb394193e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:00.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.855+0000 7fb399fe8700 1 --2- 172.21.15.52:0/625257737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb394100db0 0x7fb39419b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:00.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.855+0000 7fb398fe6700 1 -- 172.21.15.52:0/625257737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb394100db0 msgr2=0x7fb39419b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:00.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.855+0000 7fb398fe6700 1 --2- 172.21.15.52:0/625257737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb394100db0 0x7fb39419b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:00.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.855+0000 7fb398fe6700 1 -- 172.21.15.52:0/625257737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3940ff170 msgr2=0x7fb394193e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:00.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.855+0000 7fb398fe6700 1 --2- 172.21.15.52:0/625257737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3940ff170 0x7fb394193e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:00.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.855+0000 7fb398fe6700 1 -- 172.21.15.52:0/625257737 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb38c009d00 con 0x7fb394100400 2023-11-27T15:16:00.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.855+0000 7fb398fe6700 1 --2- 172.21.15.52:0/625257737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb394100400 0x7fb394194380 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb390009c70 tx=0x7fb390009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:00.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.855+0000 7fb38a7fc700 1 -- 172.21.15.52:0/625257737 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb390007b20 con 0x7fb394100400 2023-11-27T15:16:00.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.855+0000 7fb38a7fc700 1 -- 172.21.15.52:0/625257737 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb390015040 con 0x7fb394100400 2023-11-27T15:16:00.864 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.855+0000 7fb39ba4b700 1 -- 172.21.15.52:0/625257737 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb394109a20 con 0x7fb394100400 2023-11-27T15:16:00.864 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.855+0000 7fb38a7fc700 1 -- 172.21.15.52:0/625257737 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3900185b0 con 0x7fb394100400 2023-11-27T15:16:00.864 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.855+0000 7fb39ba4b700 1 -- 172.21.15.52:0/625257737 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb394109890 con 0x7fb394100400 2023-11-27T15:16:00.864 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.856+0000 7fb39ba4b700 1 -- 172.21.15.52:0/625257737 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb394106140 con 0x7fb394100400 2023-11-27T15:16:00.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.858+0000 7fb38a7fc700 1 -- 172.21.15.52:0/625257737 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb3900193f0 con 0x7fb394100400 2023-11-27T15:16:00.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.859+0000 7fb38a7fc700 1 --2- 172.21.15.52:0/625257737 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb3800a11d0 0x7fb3800a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:00.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.859+0000 7fb3997e7700 1 --2- 172.21.15.52:0/625257737 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb3800a11d0 0x7fb3800a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:00.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.859+0000 7fb38a7fc700 1 -- 172.21.15.52:0/625257737 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb390019e40 con 0x7fb394100400 2023-11-27T15:16:00.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.859+0000 7fb3997e7700 1 --2- 172.21.15.52:0/625257737 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb3800a11d0 0x7fb3800a3680 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb384005950 tx=0x7fb38400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:00.867 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:00.862+0000 7fb38a7fc700 1 -- 172.21.15.52:0/625257737 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb39009a2f0 con 0x7fb394100400 2023-11-27T15:16:01.008 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:00 smithi052 ceph-mon[99056]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:16:01.008 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:00 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:01.008 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:00 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:01.009 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:00 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:01.009 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:00 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:16:01.009 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:00 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:16:01.009 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:00 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:01.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:00 smithi150 ceph-mon[100559]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:16:01.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:00 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:01.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:00 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:01.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:00 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:01.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:00 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:16:01.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:00 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:16:01.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:00 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:01.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:00 smithi102 ceph-mon[100296]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:16:01.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:00 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:01.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:00 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:01.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:00 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:01.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:00 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:16:01.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:00 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:16:01.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:00 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:01.124 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:01.119+0000 7fb39ba4b700 1 -- 172.21.15.52:0/625257737 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb394061700 con 0x7fb394100400 2023-11-27T15:16:01.125 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:01.120+0000 7fb38a7fc700 1 -- 172.21.15.52:0/625257737 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8082 (secure 0 0 0) 0x7fb39009d910 con 0x7fb394100400 2023-11-27T15:16:01.125 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:16:01.125 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2328881376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2328881376},{"type":"v1","addr":"172.21.15.52:6839","nonce":2328881376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/215725838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":215725838},{"type":"v1","addr":"172.21.15.150:6839","nonce":215725838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-11-27T15:16:01.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:01.122+0000 7fb39ba4b700 1 -- 172.21.15.52:0/625257737 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb3800a11d0 msgr2=0x7fb3800a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:01.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:01.122+0000 7fb39ba4b700 1 --2- 172.21.15.52:0/625257737 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb3800a11d0 0x7fb3800a3680 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb384005950 tx=0x7fb38400d040 comp rx=0 tx=0).stop 2023-11-27T15:16:01.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:01.122+0000 7fb39ba4b700 1 -- 172.21.15.52:0/625257737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb394100400 msgr2=0x7fb394194380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:01.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:01.122+0000 7fb39ba4b700 1 --2- 172.21.15.52:0/625257737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb394100400 0x7fb394194380 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb390009c70 tx=0x7fb390009f80 comp rx=0 tx=0).stop 2023-11-27T15:16:01.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:01.122+0000 7fb39ba4b700 1 -- 172.21.15.52:0/625257737 shutdown_connections 2023-11-27T15:16:01.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:01.122+0000 7fb39ba4b700 1 --2- 172.21.15.52:0/625257737 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb3800a11d0 0x7fb3800a3680 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:01.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:01.122+0000 7fb39ba4b700 1 --2- 172.21.15.52:0/625257737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3940ff170 0x7fb394193e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:01.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:01.122+0000 7fb39ba4b700 1 --2- 172.21.15.52:0/625257737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb394100db0 0x7fb39419b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:01.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:01.122+0000 7fb39ba4b700 1 --2- 172.21.15.52:0/625257737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb394100400 0x7fb394194380 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:01.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:01.122+0000 7fb39ba4b700 1 -- 172.21.15.52:0/625257737 >> 172.21.15.52:0/625257737 conn(0x7fb3940fa750 msgr2=0x7fb3940fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:01.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:01.122+0000 7fb39ba4b700 1 -- 172.21.15.52:0/625257737 shutdown_connections 2023-11-27T15:16:01.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:01.122+0000 7fb39ba4b700 1 -- 172.21.15.52:0/625257737 wait complete. 2023-11-27T15:16:01.128 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 11 2023-11-27T15:16:01.694 INFO:journalctl@ceph.mds.a.smithi052.stdout:Nov 27 15:16:01 smithi052 systemd[1]: Starting Ceph mds.a for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:16:01.695 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:01 smithi052 ceph-mon[99056]: Deploying daemon mds.a on smithi052 2023-11-27T15:16:01.695 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:01 smithi052 ceph-mon[99056]: mds.? [v2:172.21.15.52:6838/2328881376,v1:172.21.15.52:6839/2328881376] up:boot 2023-11-27T15:16:01.695 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:01 smithi052 ceph-mon[99056]: fsmap 10 up:standby 2023-11-27T15:16:01.695 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:01 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-27T15:16:01.695 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:01 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/625257737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:16:02.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:01 smithi150 ceph-mon[100559]: Deploying daemon mds.a on smithi052 2023-11-27T15:16:02.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:01 smithi150 ceph-mon[100559]: mds.? [v2:172.21.15.52:6838/2328881376,v1:172.21.15.52:6839/2328881376] up:boot 2023-11-27T15:16:02.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:01 smithi150 ceph-mon[100559]: fsmap 10 up:standby 2023-11-27T15:16:02.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:01 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-27T15:16:02.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:01 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/625257737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:16:02.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:01 smithi102 ceph-mon[100296]: Deploying daemon mds.a on smithi052 2023-11-27T15:16:02.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:01 smithi102 ceph-mon[100296]: mds.? [v2:172.21.15.52:6838/2328881376,v1:172.21.15.52:6839/2328881376] up:boot 2023-11-27T15:16:02.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:01 smithi102 ceph-mon[100296]: fsmap 10 up:standby 2023-11-27T15:16:02.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:01 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-27T15:16:02.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:01 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/625257737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:16:02.253 INFO:journalctl@ceph.mds.a.smithi052.stdout:Nov 27 15:16:01 smithi052 podman[133630]: 2023-11-27T15:16:02.617 INFO:journalctl@ceph.mds.a.smithi052.stdout:Nov 27 15:16:02 smithi052 ceph-mds[133670]: -- 172.21.15.52:0/827290331 <== mon.0 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5582286a3520 con 0x558228678c00 2023-11-27T15:16:02.617 INFO:journalctl@ceph.mds.a.smithi052.stdout:Nov 27 15:16:02 smithi052 bash[133630]: 3bfad5603ceb1821f0fe54f5943553f6b75c02fe67c86c2ac92e45fff4510069 2023-11-27T15:16:02.617 INFO:journalctl@ceph.mds.a.smithi052.stdout:Nov 27 15:16:02 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-a[133646]: starting mds.a at 2023-11-27T15:16:02.617 INFO:journalctl@ceph.mds.a.smithi052.stdout:Nov 27 15:16:02 smithi052 systemd[1]: Started Ceph mds.a for c6c7df88-8d36-11ee-95a2-87774f69a715. 2023-11-27T15:16:03.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:02 smithi150 ceph-mon[100559]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:16:03.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:02 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:03.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:02 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:03.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:02 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:03.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:02 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:16:03.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:02 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:16:03.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:02 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:03.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:02 smithi102 ceph-mon[100296]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:16:03.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:02 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:03.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:02 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:03.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:02 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:03.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:02 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:16:03.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:02 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:16:03.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:02 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:03.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:02 smithi052 ceph-mon[99056]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:16:03.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:02 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:03.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:02 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:03.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:02 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:03.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:02 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:16:03.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:02 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:16:03.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:02 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:03.140 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:16:03.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.281+0000 7faf1fdb4700 1 -- 172.21.15.52:0/782513258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf18100db0 msgr2=0x7faf18107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:03.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.281+0000 7faf1fdb4700 1 --2- 172.21.15.52:0/782513258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf18100db0 0x7faf18107650 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7faf1000b0a0 tx=0x7faf1000b3b0 comp rx=0 tx=0).stop 2023-11-27T15:16:03.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.281+0000 7faf1fdb4700 1 -- 172.21.15.52:0/782513258 shutdown_connections 2023-11-27T15:16:03.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.281+0000 7faf1fdb4700 1 --2- 172.21.15.52:0/782513258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf18100db0 0x7faf18107650 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:03.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.281+0000 7faf1fdb4700 1 --2- 172.21.15.52:0/782513258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf180ff170 0x7faf180ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:03.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.281+0000 7faf1fdb4700 1 --2- 172.21.15.52:0/782513258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf18100400 0x7faf18100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:03.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.281+0000 7faf1fdb4700 1 -- 172.21.15.52:0/782513258 >> 172.21.15.52:0/782513258 conn(0x7faf180fa750 msgr2=0x7faf180fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:03.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.281+0000 7faf1fdb4700 1 -- 172.21.15.52:0/782513258 shutdown_connections 2023-11-27T15:16:03.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.281+0000 7faf1fdb4700 1 -- 172.21.15.52:0/782513258 wait complete. 2023-11-27T15:16:03.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.282+0000 7faf1fdb4700 1 Processor -- start 2023-11-27T15:16:03.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.282+0000 7faf1fdb4700 1 -- start start 2023-11-27T15:16:03.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.282+0000 7faf1fdb4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf180ff170 0x7faf18068940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:03.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.282+0000 7faf1fdb4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf18100400 0x7faf1819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:03.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.282+0000 7faf1fdb4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf18100db0 0x7faf1819d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:03.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.282+0000 7faf1fdb4700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1819daf0 con 0x7faf18100db0 2023-11-27T15:16:03.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.282+0000 7faf1fdb4700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1819dc60 con 0x7faf180ff170 2023-11-27T15:16:03.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.282+0000 7faf1fdb4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1819de20 con 0x7faf18100400 2023-11-27T15:16:03.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.282+0000 7faf1e351700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf18100db0 0x7faf1819d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:03.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.282+0000 7faf1e351700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf18100db0 0x7faf1819d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49916/0 (socket says 172.21.15.52:49916) 2023-11-27T15:16:03.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.282+0000 7faf1e351700 1 -- 172.21.15.52:0/1358341953 learned_addr learned my addr 172.21.15.52:0/1358341953 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:03.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.282+0000 7faf1db50700 1 --2- 172.21.15.52:0/1358341953 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf180ff170 0x7faf18068940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:03.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.282+0000 7faf1d34f700 1 --2- 172.21.15.52:0/1358341953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf18100400 0x7faf1819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:03.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.282+0000 7faf1e351700 1 -- 172.21.15.52:0/1358341953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf18100400 msgr2=0x7faf1819ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:03.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.282+0000 7faf1e351700 1 --2- 172.21.15.52:0/1358341953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf18100400 0x7faf1819ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:03.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.282+0000 7faf1e351700 1 -- 172.21.15.52:0/1358341953 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf180ff170 msgr2=0x7faf18068940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:03.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.282+0000 7faf1e351700 1 --2- 172.21.15.52:0/1358341953 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf180ff170 0x7faf18068940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:03.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.282+0000 7faf1e351700 1 -- 172.21.15.52:0/1358341953 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf10009d00 con 0x7faf18100db0 2023-11-27T15:16:03.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.282+0000 7faf1d34f700 1 --2- 172.21.15.52:0/1358341953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf18100400 0x7faf1819ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:16:03.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.283+0000 7faf1e351700 1 --2- 172.21.15.52:0/1358341953 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf18100db0 0x7faf1819d350 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faf1000b780 tx=0x7faf1000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:03.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.283+0000 7faf0effd700 1 -- 172.21.15.52:0/1358341953 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1000e040 con 0x7faf18100db0 2023-11-27T15:16:03.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.283+0000 7faf1fdb4700 1 -- 172.21.15.52:0/1358341953 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf1810ab70 con 0x7faf18100db0 2023-11-27T15:16:03.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.283+0000 7faf0effd700 1 -- 172.21.15.52:0/1358341953 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf10004e60 con 0x7faf18100db0 2023-11-27T15:16:03.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.283+0000 7faf0effd700 1 -- 172.21.15.52:0/1358341953 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf100053b0 con 0x7faf18100db0 2023-11-27T15:16:03.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.283+0000 7faf1fdb4700 1 -- 172.21.15.52:0/1358341953 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf1810a9e0 con 0x7faf18100db0 2023-11-27T15:16:03.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.284+0000 7faf1fdb4700 1 -- 172.21.15.52:0/1358341953 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf18106050 con 0x7faf18100db0 2023-11-27T15:16:03.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.285+0000 7faf0effd700 1 -- 172.21.15.52:0/1358341953 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7faf10005510 con 0x7faf18100db0 2023-11-27T15:16:03.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.285+0000 7faf0effd700 1 --2- 172.21.15.52:0/1358341953 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7faf040a1150 0x7faf040a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:03.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.286+0000 7faf0effd700 1 -- 172.21.15.52:0/1358341953 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7faf100cdf60 con 0x7faf18100db0 2023-11-27T15:16:03.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.286+0000 7faf1db50700 1 --2- 172.21.15.52:0/1358341953 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7faf040a1150 0x7faf040a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:03.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.286+0000 7faf1db50700 1 --2- 172.21.15.52:0/1358341953 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7faf040a1150 0x7faf040a3600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faf14009bf0 tx=0x7faf14009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:03.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.289+0000 7faf0effd700 1 -- 172.21.15.52:0/1358341953 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7faf10097be0 con 0x7faf18100db0 2023-11-27T15:16:03.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.532+0000 7faf1fdb4700 1 -- 172.21.15.52:0/1358341953 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf1802cfe0 con 0x7faf18100db0 2023-11-27T15:16:03.537 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.533+0000 7faf0effd700 1 -- 172.21.15.52:0/1358341953 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8840 (secure 0 0 0) 0x7faf1009b200 con 0x7faf18100db0 2023-11-27T15:16:03.537 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:16:03.537 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2328881376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2328881376},{"type":"v1","addr":"172.21.15.52:6839","nonce":2328881376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2799551570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2799551570},{"type":"v1","addr":"172.21.15.52:6841","nonce":2799551570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/215725838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":215725838},{"type":"v1","addr":"172.21.15.150:6839","nonce":215725838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-11-27T15:16:03.539 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.534+0000 7faf1fdb4700 1 -- 172.21.15.52:0/1358341953 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7faf040a1150 msgr2=0x7faf040a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:03.539 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.534+0000 7faf1fdb4700 1 --2- 172.21.15.52:0/1358341953 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7faf040a1150 0x7faf040a3600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faf14009bf0 tx=0x7faf14009260 comp rx=0 tx=0).stop 2023-11-27T15:16:03.539 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.535+0000 7faf1fdb4700 1 -- 172.21.15.52:0/1358341953 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf18100db0 msgr2=0x7faf1819d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:03.539 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.535+0000 7faf1fdb4700 1 --2- 172.21.15.52:0/1358341953 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf18100db0 0x7faf1819d350 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faf1000b780 tx=0x7faf1000be10 comp rx=0 tx=0).stop 2023-11-27T15:16:03.539 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.535+0000 7faf1fdb4700 1 -- 172.21.15.52:0/1358341953 shutdown_connections 2023-11-27T15:16:03.539 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.535+0000 7faf1fdb4700 1 --2- 172.21.15.52:0/1358341953 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7faf040a1150 0x7faf040a3600 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:03.540 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.535+0000 7faf1fdb4700 1 --2- 172.21.15.52:0/1358341953 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faf180ff170 0x7faf18068940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:03.540 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.535+0000 7faf1fdb4700 1 --2- 172.21.15.52:0/1358341953 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf18100db0 0x7faf1819d350 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:03.540 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.535+0000 7faf1fdb4700 1 --2- 172.21.15.52:0/1358341953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf18100400 0x7faf1819ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:03.540 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.535+0000 7faf1fdb4700 1 -- 172.21.15.52:0/1358341953 >> 172.21.15.52:0/1358341953 conn(0x7faf180fa750 msgr2=0x7faf180fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:03.540 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.535+0000 7faf1fdb4700 1 -- 172.21.15.52:0/1358341953 shutdown_connections 2023-11-27T15:16:03.540 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:03.535+0000 7faf1fdb4700 1 -- 172.21.15.52:0/1358341953 wait complete. 2023-11-27T15:16:03.540 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 12 2023-11-27T15:16:03.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:03 smithi052 ceph-mon[99056]: Deploying daemon mds.b on smithi102 2023-11-27T15:16:03.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:03 smithi052 ceph-mon[99056]: mds.? [v2:172.21.15.52:6840/2799551570,v1:172.21.15.52:6841/2799551570] up:boot 2023-11-27T15:16:03.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:03 smithi052 ceph-mon[99056]: fsmap 11 up:standby 2023-11-27T15:16:03.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:03 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-27T15:16:03.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:03 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1358341953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:16:03.967 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:03 smithi102 ceph-mon[100296]: Deploying daemon mds.b on smithi102 2023-11-27T15:16:03.967 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:03 smithi102 ceph-mon[100296]: mds.? [v2:172.21.15.52:6840/2799551570,v1:172.21.15.52:6841/2799551570] up:boot 2023-11-27T15:16:03.967 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:03 smithi102 ceph-mon[100296]: fsmap 11 up:standby 2023-11-27T15:16:03.967 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:03 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-27T15:16:03.967 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:03 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1358341953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:16:04.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:03 smithi150 ceph-mon[100559]: Deploying daemon mds.b on smithi102 2023-11-27T15:16:04.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:03 smithi150 ceph-mon[100559]: mds.? [v2:172.21.15.52:6840/2799551570,v1:172.21.15.52:6841/2799551570] up:boot 2023-11-27T15:16:04.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:03 smithi150 ceph-mon[100559]: fsmap 11 up:standby 2023-11-27T15:16:04.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:03 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-27T15:16:04.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:03 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1358341953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:16:04.328 INFO:journalctl@ceph.mds.b.smithi102.stdout:Nov 27 15:16:04 smithi102 systemd[1]: Starting Ceph mds.b for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:16:04.641 INFO:journalctl@ceph.mds.b.smithi102.stdout:Nov 27 15:16:04 smithi102 podman[125488]: 2023-11-27T15:16:05.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:04 smithi150 ceph-mon[100559]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:16:05.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:04 smithi102 ceph-mon[100296]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:16:05.078 INFO:journalctl@ceph.mds.b.smithi102.stdout:Nov 27 15:16:04 smithi102 ceph-mds[125528]: -- 172.21.15.102:0/1012439432 <== mon.2 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560209c81520 con 0x560209c56800 2023-11-27T15:16:05.078 INFO:journalctl@ceph.mds.b.smithi102.stdout:Nov 27 15:16:04 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-b[125504]: starting mds.b at 2023-11-27T15:16:05.078 INFO:journalctl@ceph.mds.b.smithi102.stdout:Nov 27 15:16:04 smithi102 bash[125488]: ef6fb055b075704d3596e8dad1dfdc570a23066071c1230845c9fee4612cd76a 2023-11-27T15:16:05.078 INFO:journalctl@ceph.mds.b.smithi102.stdout:Nov 27 15:16:04 smithi102 systemd[1]: Started Ceph mds.b for c6c7df88-8d36-11ee-95a2-87774f69a715. 2023-11-27T15:16:05.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:04 smithi052 ceph-mon[99056]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:16:05.552 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:16:05.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.680+0000 7f185b414700 1 -- 172.21.15.52:0/2409366596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1854105e50 msgr2=0x7f1854108300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:05.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.680+0000 7f185b414700 1 --2- 172.21.15.52:0/2409366596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1854105e50 0x7f1854108300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f184c00b0a0 tx=0x7f184c00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:16:05.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.681+0000 7f185b414700 1 -- 172.21.15.52:0/2409366596 shutdown_connections 2023-11-27T15:16:05.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.681+0000 7f185b414700 1 --2- 172.21.15.52:0/2409366596 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1854068ad0 0x7f1854102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:05.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.681+0000 7f185b414700 1 --2- 172.21.15.52:0/2409366596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1854105e50 0x7f1854108300 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:05.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.681+0000 7f185b414700 1 --2- 172.21.15.52:0/2409366596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1854103380 0x7f18541057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:05.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.681+0000 7f185b414700 1 -- 172.21.15.52:0/2409366596 >> 172.21.15.52:0/2409366596 conn(0x7f18540fa750 msgr2=0x7f18540fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:05.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.681+0000 7f185b414700 1 -- 172.21.15.52:0/2409366596 shutdown_connections 2023-11-27T15:16:05.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.681+0000 7f185b414700 1 -- 172.21.15.52:0/2409366596 wait complete. 2023-11-27T15:16:05.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.681+0000 7f185b414700 1 Processor -- start 2023-11-27T15:16:05.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.681+0000 7f185b414700 1 -- start start 2023-11-27T15:16:05.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.681+0000 7f185b414700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1854068ad0 0x7f1854196070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:05.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.681+0000 7f185b414700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1854103380 0x7f18541965b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:05.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.681+0000 7f185b414700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1854105e50 0x7f185419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:05.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.681+0000 7f185b414700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185419dbf0 con 0x7f1854068ad0 2023-11-27T15:16:05.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.681+0000 7f185b414700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185419dd30 con 0x7f1854103380 2023-11-27T15:16:05.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.681+0000 7f185b414700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185419dec0 con 0x7f1854105e50 2023-11-27T15:16:05.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.682+0000 7f18591b0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1854068ad0 0x7f1854196070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:05.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.682+0000 7f18591b0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1854068ad0 0x7f1854196070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49926/0 (socket says 172.21.15.52:49926) 2023-11-27T15:16:05.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.682+0000 7f18591b0700 1 -- 172.21.15.52:0/1551450902 learned_addr learned my addr 172.21.15.52:0/1551450902 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:05.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.682+0000 7f18589af700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1854103380 0x7f18541965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:05.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.682+0000 7f18599b1700 1 --2- 172.21.15.52:0/1551450902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1854105e50 0x7f185419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:05.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.682+0000 7f18589af700 1 -- 172.21.15.52:0/1551450902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1854105e50 msgr2=0x7f185419d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:05.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.682+0000 7f18589af700 1 --2- 172.21.15.52:0/1551450902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1854105e50 0x7f185419d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:05.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.682+0000 7f18589af700 1 -- 172.21.15.52:0/1551450902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1854068ad0 msgr2=0x7f1854196070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:05.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.682+0000 7f18589af700 1 --2- 172.21.15.52:0/1551450902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1854068ad0 0x7f1854196070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:05.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.682+0000 7f18589af700 1 -- 172.21.15.52:0/1551450902 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f184c009d00 con 0x7f1854103380 2023-11-27T15:16:05.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.682+0000 7f18591b0700 1 --2- 172.21.15.52:0/1551450902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1854068ad0 0x7f1854196070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:16:05.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.682+0000 7f18589af700 1 --2- 172.21.15.52:0/1551450902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1854103380 0x7f18541965b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f185000e9f0 tx=0x7f185000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:05.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.682+0000 7f18467fc700 1 -- 172.21.15.52:0/1551450902 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185000eee0 con 0x7f1854103380 2023-11-27T15:16:05.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.682+0000 7f18467fc700 1 -- 172.21.15.52:0/1551450902 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f185000cbd0 con 0x7f1854103380 2023-11-27T15:16:05.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.682+0000 7f185b414700 1 -- 172.21.15.52:0/1551450902 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f185410bbf0 con 0x7f1854103380 2023-11-27T15:16:05.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.682+0000 7f18467fc700 1 -- 172.21.15.52:0/1551450902 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1850018680 con 0x7f1854103380 2023-11-27T15:16:05.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.682+0000 7f185b414700 1 -- 172.21.15.52:0/1551450902 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f185410ba60 con 0x7f1854103380 2023-11-27T15:16:05.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.683+0000 7f185b414700 1 -- 172.21.15.52:0/1551450902 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1854067a60 con 0x7f1854103380 2023-11-27T15:16:05.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.684+0000 7f18467fc700 1 -- 172.21.15.52:0/1551450902 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f185000cd40 con 0x7f1854103380 2023-11-27T15:16:05.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.685+0000 7f18467fc700 1 --2- 172.21.15.52:0/1551450902 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f18400a11d0 0x7f18400a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:05.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.685+0000 7f18591b0700 1 --2- 172.21.15.52:0/1551450902 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f18400a11d0 0x7f18400a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:05.693 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.685+0000 7f18467fc700 1 -- 172.21.15.52:0/1551450902 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1850014070 con 0x7f1854103380 2023-11-27T15:16:05.693 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.685+0000 7f18591b0700 1 --2- 172.21.15.52:0/1551450902 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f18400a11d0 0x7f18400a3680 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1848005950 tx=0x7f1848009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:05.693 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.688+0000 7f18467fc700 1 -- 172.21.15.52:0/1551450902 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1850096450 con 0x7f1854103380 2023-11-27T15:16:05.866 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:05 smithi052 ceph-mon[99056]: mds.? [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] up:boot 2023-11-27T15:16:05.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:05 smithi052 ceph-mon[99056]: fsmap 12 up:standby 2023-11-27T15:16:05.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:05 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-27T15:16:05.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:05 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:05.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:05 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:05.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:05 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:05.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:05 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:05.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:05 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:05.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:05 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:16:05.943 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.938+0000 7f185b414700 1 -- 172.21.15.52:0/1551450902 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f185419eae0 con 0x7f1854103380 2023-11-27T15:16:05.943 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.939+0000 7f18467fc700 1 -- 172.21.15.52:0/1551450902 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9601 (secure 0 0 0) 0x7f1850099a70 con 0x7f1854103380 2023-11-27T15:16:05.944 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:16:05.944 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2328881376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2328881376},{"type":"v1","addr":"172.21.15.52:6839","nonce":2328881376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2799551570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2799551570},{"type":"v1","addr":"172.21.15.52:6841","nonce":2799551570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/215725838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":215725838},{"type":"v1","addr":"172.21.15.150:6839","nonce":215725838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24536,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/3571830559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3571830559},{"type":"v1","addr":"172.21.15.102:6839","nonce":3571830559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2023-11-27T15:16:05.946 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.941+0000 7f185b414700 1 -- 172.21.15.52:0/1551450902 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f18400a11d0 msgr2=0x7f18400a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:05.946 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.941+0000 7f185b414700 1 --2- 172.21.15.52:0/1551450902 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f18400a11d0 0x7f18400a3680 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1848005950 tx=0x7f1848009260 comp rx=0 tx=0).stop 2023-11-27T15:16:05.946 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.941+0000 7f185b414700 1 -- 172.21.15.52:0/1551450902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1854103380 msgr2=0x7f18541965b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:05.946 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.941+0000 7f185b414700 1 --2- 172.21.15.52:0/1551450902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1854103380 0x7f18541965b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f185000e9f0 tx=0x7f185000ed00 comp rx=0 tx=0).stop 2023-11-27T15:16:05.946 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.941+0000 7f185b414700 1 -- 172.21.15.52:0/1551450902 shutdown_connections 2023-11-27T15:16:05.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.941+0000 7f185b414700 1 --2- 172.21.15.52:0/1551450902 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f18400a11d0 0x7f18400a3680 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:05.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.941+0000 7f185b414700 1 --2- 172.21.15.52:0/1551450902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1854105e50 0x7f185419d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:05.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.941+0000 7f185b414700 1 --2- 172.21.15.52:0/1551450902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1854068ad0 0x7f1854196070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:05.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.941+0000 7f185b414700 1 --2- 172.21.15.52:0/1551450902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1854103380 0x7f18541965b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:05.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.941+0000 7f185b414700 1 -- 172.21.15.52:0/1551450902 >> 172.21.15.52:0/1551450902 conn(0x7f18540fa750 msgr2=0x7f1854101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:05.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.942+0000 7f185b414700 1 -- 172.21.15.52:0/1551450902 shutdown_connections 2023-11-27T15:16:05.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:05.942+0000 7f185b414700 1 -- 172.21.15.52:0/1551450902 wait complete. 2023-11-27T15:16:05.947 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 13 2023-11-27T15:16:05.959 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:16:06.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:05 smithi150 ceph-mon[100559]: mds.? [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] up:boot 2023-11-27T15:16:06.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:05 smithi150 ceph-mon[100559]: fsmap 12 up:standby 2023-11-27T15:16:06.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:05 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-27T15:16:06.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:05 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:06.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:05 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:06.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:05 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:06.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:05 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:06.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:05 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:06.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:05 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:16:06.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:05 smithi102 ceph-mon[100296]: mds.? [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] up:boot 2023-11-27T15:16:06.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:05 smithi102 ceph-mon[100296]: fsmap 12 up:standby 2023-11-27T15:16:06.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:05 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-27T15:16:06.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:05 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:06.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:05 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:06.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:05 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:06.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:05 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:06.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:05 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:06.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:05 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:16:06.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.084+0000 7f32f6f06700 1 -- 172.21.15.52:0/3188938651 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32f01002b0 msgr2=0x7f32f01006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:06.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.084+0000 7f32f6f06700 1 --2- 172.21.15.52:0/3188938651 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32f01002b0 0x7f32f01006c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f32e4009990 tx=0x7f32e4009ca0 comp rx=0 tx=0).stop 2023-11-27T15:16:06.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.084+0000 7f32f6f06700 1 -- 172.21.15.52:0/3188938651 shutdown_connections 2023-11-27T15:16:06.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.084+0000 7f32f6f06700 1 --2- 172.21.15.52:0/3188938651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32f0101e30 0x7f32f01086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:06.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.084+0000 7f32f6f06700 1 --2- 172.21.15.52:0/3188938651 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32f01002b0 0x7f32f01006c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:06.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.084+0000 7f32f6f06700 1 --2- 172.21.15.52:0/3188938651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32f01014a0 0x7f32f01018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:06.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.084+0000 7f32f6f06700 1 -- 172.21.15.52:0/3188938651 >> 172.21.15.52:0/3188938651 conn(0x7f32f00fb890 msgr2=0x7f32f00fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:06.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.084+0000 7f32f6f06700 1 -- 172.21.15.52:0/3188938651 shutdown_connections 2023-11-27T15:16:06.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.084+0000 7f32f6f06700 1 -- 172.21.15.52:0/3188938651 wait complete. 2023-11-27T15:16:06.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.085+0000 7f32f6f06700 1 Processor -- start 2023-11-27T15:16:06.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.085+0000 7f32f6f06700 1 -- start start 2023-11-27T15:16:06.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.085+0000 7f32f6f06700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32f01002b0 0x7f32f0195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:06.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.085+0000 7f32f6f06700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32f01014a0 0x7f32f01964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:06.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.085+0000 7f32f6f06700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32f0101e30 0x7f32f019d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:06.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.085+0000 7f32f6f06700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f019db00 con 0x7f32f01002b0 2023-11-27T15:16:06.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.085+0000 7f32f6f06700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f019dc40 con 0x7f32f0101e30 2023-11-27T15:16:06.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.085+0000 7f32f6f06700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f019ddd0 con 0x7f32f01014a0 2023-11-27T15:16:06.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.085+0000 7f32f4ca2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32f01002b0 0x7f32f0195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:06.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.085+0000 7f32f4ca2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32f01002b0 0x7f32f0195f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49940/0 (socket says 172.21.15.52:49940) 2023-11-27T15:16:06.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.085+0000 7f32f4ca2700 1 -- 172.21.15.52:0/3349671554 learned_addr learned my addr 172.21.15.52:0/3349671554 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:06.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.085+0000 7f32f54a3700 1 --2- 172.21.15.52:0/3349671554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32f0101e30 0x7f32f019d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:06.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.085+0000 7f32effff700 1 --2- 172.21.15.52:0/3349671554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32f01014a0 0x7f32f01964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:06.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.086+0000 7f32f4ca2700 1 -- 172.21.15.52:0/3349671554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32f01014a0 msgr2=0x7f32f01964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:06.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.086+0000 7f32f4ca2700 1 --2- 172.21.15.52:0/3349671554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32f01014a0 0x7f32f01964c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:06.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.086+0000 7f32f4ca2700 1 -- 172.21.15.52:0/3349671554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32f0101e30 msgr2=0x7f32f019d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:06.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.086+0000 7f32f4ca2700 1 --2- 172.21.15.52:0/3349671554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32f0101e30 0x7f32f019d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:06.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.086+0000 7f32f4ca2700 1 -- 172.21.15.52:0/3349671554 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e40095f0 con 0x7f32f01002b0 2023-11-27T15:16:06.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.086+0000 7f32f4ca2700 1 --2- 172.21.15.52:0/3349671554 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32f01002b0 0x7f32f0195f80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f32e4000c00 tx=0x7f32e4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:06.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.086+0000 7f32edffb700 1 -- 172.21.15.52:0/3349671554 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e401d070 con 0x7f32f01002b0 2023-11-27T15:16:06.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.086+0000 7f32edffb700 1 -- 172.21.15.52:0/3349671554 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32e4003f60 con 0x7f32f01002b0 2023-11-27T15:16:06.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.086+0000 7f32f6f06700 1 -- 172.21.15.52:0/3349671554 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f010bc80 con 0x7f32f01002b0 2023-11-27T15:16:06.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.086+0000 7f32edffb700 1 -- 172.21.15.52:0/3349671554 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e4022b50 con 0x7f32f01002b0 2023-11-27T15:16:06.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.086+0000 7f32f6f06700 1 -- 172.21.15.52:0/3349671554 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f010baf0 con 0x7f32f01002b0 2023-11-27T15:16:06.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.087+0000 7f32f6f06700 1 -- 172.21.15.52:0/3349671554 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f019e370 con 0x7f32f01002b0 2023-11-27T15:16:06.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.088+0000 7f32edffb700 1 -- 172.21.15.52:0/3349671554 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f32e40040d0 con 0x7f32f01002b0 2023-11-27T15:16:06.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.089+0000 7f32edffb700 1 --2- 172.21.15.52:0/3349671554 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f32e00a1150 0x7f32e00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:06.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.089+0000 7f32edffb700 1 -- 172.21.15.52:0/3349671554 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f32e40ce1e0 con 0x7f32f01002b0 2023-11-27T15:16:06.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.089+0000 7f32effff700 1 --2- 172.21.15.52:0/3349671554 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f32e00a1150 0x7f32e00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:06.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.089+0000 7f32effff700 1 --2- 172.21.15.52:0/3349671554 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f32e00a1150 0x7f32e00a3600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f32dc005950 tx=0x7f32dc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:06.097 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.092+0000 7f32edffb700 1 -- 172.21.15.52:0/3349671554 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f32e4097e60 con 0x7f32f01002b0 2023-11-27T15:16:06.303 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.298+0000 7f32f6f06700 1 -- 172.21.15.52:0/3349671554 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32f019ea10 con 0x7f32f01002b0 2023-11-27T15:16:06.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.299+0000 7f32edffb700 1 -- 172.21.15.52:0/3349671554 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18433 (secure 0 0 0) 0x7f32e409b480 con 0x7f32f01002b0 2023-11-27T15:16:06.304 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:16:06.304 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":79,"fsid":"c6c7df88-8d36-11ee-95a2-87774f69a715","created":"2023-11-27T15:09:03.333345+0000","modified":"2023-11-27T15:15:28.662216+0000","last_up_change":"2023-11-27T15:15:27.656134+0000","last_in_change":"2023-11-27T15:15:10.671153+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:12:14.177523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58d69ede-c293-49f5-abbe-8187793f2b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6803","nonce":608216415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6805","nonce":608216415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6809","nonce":608216415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6807","nonce":608216415}]},"public_addr":"172.21.15.52:6803/608216415","cluster_addr":"172.21.15.52:6805/608216415","heartbeat_back_addr":"172.21.15.52:6809/608216415","heartbeat_front_addr":"172.21.15.52:6807/608216415","state":["exists","up"]},{"osd":1,"uuid":"d6a46b91-adc8-45ef-80f5-566ae400911f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6801","nonce":2538384087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6803","nonce":2538384087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6807","nonce":2538384087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6805","nonce":2538384087}]},"public_addr":"172.21.15.102:6801/2538384087","cluster_addr":"172.21.15.102:6803/2538384087","heartbeat_back_addr":"172.21.15.102:6807/2538384087","heartbeat_front_addr":"172.21.15.102:6805/2538384087","state":["exists","up"]},{"osd":2,"uuid":"846ba125-11f8-46b2-8941-bd5e1947cc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6801","nonce":4258038221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6803","nonce":4258038221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6807","nonce":4258038221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6805","nonce":4258038221}]},"public_addr":"172.21.15.150:6801/4258038221","cluster_addr":"172.21.15.150:6803/4258038221","heartbeat_back_addr":"172.21.15.150:6807/4258038221","heartbeat_front_addr":"172.21.15.150:6805/4258038221","state":["exists","up"]},{"osd":3,"uuid":"950fcd67-1057-4288-a389-714f1ec94261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6811","nonce":2622100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6813","nonce":2622100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6817","nonce":2622100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6815","nonce":2622100752}]},"public_addr":"172.21.15.52:6811/2622100752","cluster_addr":"172.21.15.52:6813/2622100752","heartbeat_back_addr":"172.21.15.52:6817/2622100752","heartbeat_front_addr":"172.21.15.52:6815/2622100752","state":["exists","up"]},{"osd":4,"uuid":"1c96a676-cea9-4deb-bad1-f7fd240c3d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6809","nonce":1546794763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6811","nonce":1546794763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6815","nonce":1546794763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6813","nonce":1546794763}]},"public_addr":"172.21.15.102:6809/1546794763","cluster_addr":"172.21.15.102:6811/1546794763","heartbeat_back_addr":"172.21.15.102:6815/1546794763","heartbeat_front_addr":"172.21.15.102:6813/1546794763","state":["exists","up"]},{"osd":5,"uuid":"38a75af1-fd4d-433e-b579-0609109c851c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6809","nonce":461511375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6811","nonce":461511375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6815","nonce":461511375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6813","nonce":461511375}]},"public_addr":"172.21.15.150:6809/461511375","cluster_addr":"172.21.15.150:6811/461511375","heartbeat_back_addr":"172.21.15.150:6815/461511375","heartbeat_front_addr":"172.21.15.150:6813/461511375","state":["exists","up"]},{"osd":6,"uuid":"2f0ede4a-d175-4b4a-b876-c0f442048901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6819","nonce":1002457464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6821","nonce":1002457464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6825","nonce":1002457464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6823","nonce":1002457464}]},"public_addr":"172.21.15.52:6819/1002457464","cluster_addr":"172.21.15.52:6821/1002457464","heartbeat_back_addr":"172.21.15.52:6825/1002457464","heartbeat_front_addr":"172.21.15.52:6823/1002457464","state":["exists","up"]},{"osd":7,"uuid":"6b635192-abcf-4bbf-bf78-a019b351bca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6817","nonce":3249840064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6819","nonce":3249840064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6823","nonce":3249840064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6821","nonce":3249840064}]},"public_addr":"172.21.15.102:6817/3249840064","cluster_addr":"172.21.15.102:6819/3249840064","heartbeat_back_addr":"172.21.15.102:6823/3249840064","heartbeat_front_addr":"172.21.15.102:6821/3249840064","state":["exists","up"]},{"osd":8,"uuid":"c7ca8007-7170-47bf-956c-3f00edf27e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6817","nonce":3672583574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6819","nonce":3672583574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6823","nonce":3672583574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6821","nonce":3672583574}]},"public_addr":"172.21.15.150:6817/3672583574","cluster_addr":"172.21.15.150:6819/3672583574","heartbeat_back_addr":"172.21.15.150:6823/3672583574","heartbeat_front_addr":"172.21.15.150:6821/3672583574","state":["exists","up"]},{"osd":9,"uuid":"0e252c2c-01a4-48ba-832d-067ed940fefb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6827","nonce":4287934320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6829","nonce":4287934320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6833","nonce":4287934320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6831","nonce":4287934320}]},"public_addr":"172.21.15.52:6827/4287934320","cluster_addr":"172.21.15.52:6829/4287934320","heartbeat_back_addr":"172.21.15.52:6833/4287934320","heartbeat_front_addr":"172.21.15.52:6831/4287934320","state":["exists","up"]},{"osd":10,"uuid":"bb540142-7563-44a4-b7a3-cb662714c128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6825","nonce":3056034201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6827","nonce":3056034201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6831","nonce":3056034201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6829","nonce":3056034201}]},"public_addr":"172.21.15.102:6825/3056034201","cluster_addr":"172.21.15.102:6827/3056034201","heartbeat_back_addr":"172.21.15.102:6831/3056034201","heartbeat_front_addr":"172.21.15.102:6829/3056034201","state":["exists","up"]},{"osd":11,"uuid":"3156528e-e0f8-4d44-9946-c8b6339c2bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6825","nonce":3207559739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6827","nonce":3207559739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6831","nonce":3207559739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6829","nonce":3207559739}]},"public_addr":"172.21.15.150:6825/3207559739","cluster_addr":"172.21.15.150:6827/3207559739","heartbeat_back_addr":"172.21.15.150:6831/3207559739","heartbeat_front_addr":"172.21.15.150:6829/3207559739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:26.826099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:48.331371+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:10.459738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:32.356472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:54.043170+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:15.648582+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:37.166463+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:59.240819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:20.710097+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:43.364563+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:05.279433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:26.344573+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/1392282110":"2023-11-28T15:10:03.644870+0000","172.21.15.52:0/1479993479":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6800/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/961562652":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/1290665390":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2410529742":"2023-11-28T15:09:37.118690+0000","172.21.15.52:6800/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2130037044":"2023-11-28T15:10:03.644870+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:16:06.306 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.301+0000 7f32f6f06700 1 -- 172.21.15.52:0/3349671554 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f32e00a1150 msgr2=0x7f32e00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:06.306 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.301+0000 7f32f6f06700 1 --2- 172.21.15.52:0/3349671554 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f32e00a1150 0x7f32e00a3600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f32dc005950 tx=0x7f32dc00b410 comp rx=0 tx=0).stop 2023-11-27T15:16:06.306 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.301+0000 7f32f6f06700 1 -- 172.21.15.52:0/3349671554 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32f01002b0 msgr2=0x7f32f0195f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:06.306 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.301+0000 7f32f6f06700 1 --2- 172.21.15.52:0/3349671554 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32f01002b0 0x7f32f0195f80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f32e4000c00 tx=0x7f32e4005740 comp rx=0 tx=0).stop 2023-11-27T15:16:06.306 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.302+0000 7f32f6f06700 1 -- 172.21.15.52:0/3349671554 shutdown_connections 2023-11-27T15:16:06.306 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.302+0000 7f32f6f06700 1 --2- 172.21.15.52:0/3349671554 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f32e00a1150 0x7f32e00a3600 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:06.306 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.302+0000 7f32f6f06700 1 --2- 172.21.15.52:0/3349671554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32f0101e30 0x7f32f019d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:06.307 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.302+0000 7f32f6f06700 1 --2- 172.21.15.52:0/3349671554 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32f01002b0 0x7f32f0195f80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:06.307 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.302+0000 7f32f6f06700 1 --2- 172.21.15.52:0/3349671554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32f01014a0 0x7f32f01964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:06.307 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.302+0000 7f32f6f06700 1 -- 172.21.15.52:0/3349671554 >> 172.21.15.52:0/3349671554 conn(0x7f32f00fb890 msgr2=0x7f32f00fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:06.307 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.302+0000 7f32f6f06700 1 -- 172.21.15.52:0/3349671554 shutdown_connections 2023-11-27T15:16:06.307 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.302+0000 7f32f6f06700 1 -- 172.21.15.52:0/3349671554 wait complete. 2023-11-27T15:16:06.319 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:12:14.177523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-27T15:16:06.320 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:16:06.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.448+0000 7fd76cab1700 1 -- 172.21.15.52:0/1837740037 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd7680ff1c0 msgr2=0x7fd7680ff5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:06.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.448+0000 7fd76cab1700 1 --2- 172.21.15.52:0/1837740037 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd7680ff1c0 0x7fd7680ff5d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd758009990 tx=0x7fd758009ca0 comp rx=0 tx=0).stop 2023-11-27T15:16:06.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.448+0000 7fd76cab1700 1 -- 172.21.15.52:0/1837740037 shutdown_connections 2023-11-27T15:16:06.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.448+0000 7fd76cab1700 1 --2- 172.21.15.52:0/1837740037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd768100d40 0x7fd768107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:06.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.448+0000 7fd76cab1700 1 --2- 172.21.15.52:0/1837740037 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd7680ff1c0 0x7fd7680ff5d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:06.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.448+0000 7fd76cab1700 1 --2- 172.21.15.52:0/1837740037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7681003b0 0x7fd768100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:06.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.448+0000 7fd76cab1700 1 -- 172.21.15.52:0/1837740037 >> 172.21.15.52:0/1837740037 conn(0x7fd7680fa7c0 msgr2=0x7fd7680fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:06.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.448+0000 7fd76cab1700 1 -- 172.21.15.52:0/1837740037 shutdown_connections 2023-11-27T15:16:06.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.448+0000 7fd76cab1700 1 -- 172.21.15.52:0/1837740037 wait complete. 2023-11-27T15:16:06.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.449+0000 7fd76cab1700 1 Processor -- start 2023-11-27T15:16:06.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.449+0000 7fd76cab1700 1 -- start start 2023-11-27T15:16:06.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.449+0000 7fd76cab1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7680ff1c0 0x7fd7680689a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:06.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.449+0000 7fd76cab1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7681003b0 0x7fd76819ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:06.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.449+0000 7fd76cab1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd768100d40 0x7fd76819d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:06.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.449+0000 7fd76cab1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76819dbc0 con 0x7fd768100d40 2023-11-27T15:16:06.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.449+0000 7fd76cab1700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76819dd30 con 0x7fd7680ff1c0 2023-11-27T15:16:06.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.449+0000 7fd76cab1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76819def0 con 0x7fd7681003b0 2023-11-27T15:16:06.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.449+0000 7fd76659c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7680ff1c0 0x7fd7680689a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:06.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.449+0000 7fd766d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd768100d40 0x7fd76819d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:06.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.449+0000 7fd76659c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7680ff1c0 0x7fd7680689a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:39612/0 (socket says 172.21.15.52:39612) 2023-11-27T15:16:06.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.449+0000 7fd76659c700 1 -- 172.21.15.52:0/2650994169 learned_addr learned my addr 172.21.15.52:0/2650994169 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:06.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.449+0000 7fd765d9b700 1 --2- 172.21.15.52:0/2650994169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7681003b0 0x7fd76819ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:06.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.450+0000 7fd766d9d700 1 -- 172.21.15.52:0/2650994169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7681003b0 msgr2=0x7fd76819ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:06.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.450+0000 7fd766d9d700 1 --2- 172.21.15.52:0/2650994169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7681003b0 0x7fd76819ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:06.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.450+0000 7fd766d9d700 1 -- 172.21.15.52:0/2650994169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7680ff1c0 msgr2=0x7fd7680689a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:06.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.450+0000 7fd766d9d700 1 --2- 172.21.15.52:0/2650994169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7680ff1c0 0x7fd7680689a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:06.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.450+0000 7fd766d9d700 1 -- 172.21.15.52:0/2650994169 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7580095f0 con 0x7fd768100d40 2023-11-27T15:16:06.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.450+0000 7fd765d9b700 1 --2- 172.21.15.52:0/2650994169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7681003b0 0x7fd76819ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:16:06.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.450+0000 7fd76659c700 1 --2- 172.21.15.52:0/2650994169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7680ff1c0 0x7fd7680689a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:16:06.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.450+0000 7fd766d9d700 1 --2- 172.21.15.52:0/2650994169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd768100d40 0x7fd76819d420 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd760007f10 tx=0x7fd76000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:06.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.450+0000 7fd7577fe700 1 -- 172.21.15.52:0/2650994169 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd760004db0 con 0x7fd768100d40 2023-11-27T15:16:06.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.450+0000 7fd7577fe700 1 -- 172.21.15.52:0/2650994169 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd76000f040 con 0x7fd768100d40 2023-11-27T15:16:06.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.450+0000 7fd7577fe700 1 -- 172.21.15.52:0/2650994169 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd760005990 con 0x7fd768100d40 2023-11-27T15:16:06.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.450+0000 7fd76cab1700 1 -- 172.21.15.52:0/2650994169 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd76810ab50 con 0x7fd768100d40 2023-11-27T15:16:06.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.450+0000 7fd76cab1700 1 -- 172.21.15.52:0/2650994169 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd76810a9c0 con 0x7fd768100d40 2023-11-27T15:16:06.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.451+0000 7fd76cab1700 1 -- 172.21.15.52:0/2650994169 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd768106140 con 0x7fd768100d40 2023-11-27T15:16:06.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.452+0000 7fd7577fe700 1 -- 172.21.15.52:0/2650994169 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd76000db20 con 0x7fd768100d40 2023-11-27T15:16:06.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.453+0000 7fd7577fe700 1 --2- 172.21.15.52:0/2650994169 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd7500a11d0 0x7fd7500a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:06.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.453+0000 7fd76659c700 1 --2- 172.21.15.52:0/2650994169 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd7500a11d0 0x7fd7500a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:06.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.453+0000 7fd7577fe700 1 -- 172.21.15.52:0/2650994169 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd7600cc4e0 con 0x7fd768100d40 2023-11-27T15:16:06.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.453+0000 7fd76659c700 1 --2- 172.21.15.52:0/2650994169 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd7500a11d0 0x7fd7500a3680 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd75800b2e0 tx=0x7fd75801e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:06.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.456+0000 7fd7577fe700 1 -- 172.21.15.52:0/2650994169 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd760096030 con 0x7fd768100d40 2023-11-27T15:16:06.669 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.665+0000 7fd76cab1700 1 -- 172.21.15.52:0/2650994169 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd768061700 con 0x7fd768100d40 2023-11-27T15:16:06.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.665+0000 7fd7577fe700 1 -- 172.21.15.52:0/2650994169 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd760023020 con 0x7fd768100d40 2023-11-27T15:16:06.670 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2023-11-27T15:16:06.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.667+0000 7fd76cab1700 1 -- 172.21.15.52:0/2650994169 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd7500a11d0 msgr2=0x7fd7500a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:06.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.667+0000 7fd76cab1700 1 --2- 172.21.15.52:0/2650994169 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd7500a11d0 0x7fd7500a3680 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd75800b2e0 tx=0x7fd75801e040 comp rx=0 tx=0).stop 2023-11-27T15:16:06.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.667+0000 7fd76cab1700 1 -- 172.21.15.52:0/2650994169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd768100d40 msgr2=0x7fd76819d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:06.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.667+0000 7fd76cab1700 1 --2- 172.21.15.52:0/2650994169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd768100d40 0x7fd76819d420 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd760007f10 tx=0x7fd76000d3b0 comp rx=0 tx=0).stop 2023-11-27T15:16:06.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.668+0000 7fd76cab1700 1 -- 172.21.15.52:0/2650994169 shutdown_connections 2023-11-27T15:16:06.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.668+0000 7fd76cab1700 1 --2- 172.21.15.52:0/2650994169 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd7500a11d0 0x7fd7500a3680 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:06.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.668+0000 7fd76cab1700 1 --2- 172.21.15.52:0/2650994169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7680ff1c0 0x7fd7680689a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:06.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.668+0000 7fd76cab1700 1 --2- 172.21.15.52:0/2650994169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd768100d40 0x7fd76819d420 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:06.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.668+0000 7fd76cab1700 1 --2- 172.21.15.52:0/2650994169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7681003b0 0x7fd76819ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:06.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.668+0000 7fd76cab1700 1 -- 172.21.15.52:0/2650994169 >> 172.21.15.52:0/2650994169 conn(0x7fd7680fa7c0 msgr2=0x7fd7680fcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:06.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.668+0000 7fd76cab1700 1 -- 172.21.15.52:0/2650994169 shutdown_connections 2023-11-27T15:16:06.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.668+0000 7fd76cab1700 1 -- 172.21.15.52:0/2650994169 wait complete. 2023-11-27T15:16:06.688 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-27T15:16:06.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.813+0000 7f26d67bb700 1 -- 172.21.15.52:0/2151027689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26d0100d20 msgr2=0x7f26d01075e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:06.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.813+0000 7f26d67bb700 1 --2- 172.21.15.52:0/2151027689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26d0100d20 0x7f26d01075e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f26c800b0a0 tx=0x7f26c800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:16:06.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.813+0000 7f26d67bb700 1 -- 172.21.15.52:0/2151027689 shutdown_connections 2023-11-27T15:16:06.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.813+0000 7f26d67bb700 1 --2- 172.21.15.52:0/2151027689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26d00ff1a0 0x7f26d00ff5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:06.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.813+0000 7f26d67bb700 1 --2- 172.21.15.52:0/2151027689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26d0100d20 0x7f26d01075e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:06.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.813+0000 7f26d67bb700 1 --2- 172.21.15.52:0/2151027689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26d0100390 0x7f26d01007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:06.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.813+0000 7f26d67bb700 1 -- 172.21.15.52:0/2151027689 >> 172.21.15.52:0/2151027689 conn(0x7f26d00fa780 msgr2=0x7f26d00fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:06.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.813+0000 7f26d67bb700 1 -- 172.21.15.52:0/2151027689 shutdown_connections 2023-11-27T15:16:06.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.813+0000 7f26d67bb700 1 -- 172.21.15.52:0/2151027689 wait complete. 2023-11-27T15:16:06.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.813+0000 7f26d67bb700 1 Processor -- start 2023-11-27T15:16:06.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.813+0000 7f26d67bb700 1 -- start start 2023-11-27T15:16:06.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.814+0000 7f26d67bb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26d00ff1a0 0x7f26d0109a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:06.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.814+0000 7f26d67bb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26d0100390 0x7f26d01080d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:06.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.814+0000 7f26d67bb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26d0100d20 0x7f26d0108610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:06.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.814+0000 7f26d67bb700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d0108db0 con 0x7f26d0100d20 2023-11-27T15:16:06.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.814+0000 7f26d67bb700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d0108f20 con 0x7f26d0100390 2023-11-27T15:16:06.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.814+0000 7f26d67bb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d01a00f0 con 0x7f26d00ff1a0 2023-11-27T15:16:06.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.814+0000 7f26d4d58700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26d0100d20 0x7f26d0108610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:06.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.814+0000 7f26cffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26d00ff1a0 0x7f26d0109a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:06.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.814+0000 7f26cffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26d00ff1a0 0x7f26d0109a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:36612/0 (socket says 172.21.15.52:36612) 2023-11-27T15:16:06.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.814+0000 7f26d4d58700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26d0100d20 0x7f26d0108610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49972/0 (socket says 172.21.15.52:49972) 2023-11-27T15:16:06.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.814+0000 7f26cffff700 1 -- 172.21.15.52:0/3447082967 learned_addr learned my addr 172.21.15.52:0/3447082967 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:06.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.814+0000 7f26c7fff700 1 --2- 172.21.15.52:0/3447082967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26d0100390 0x7f26d01080d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:06.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.814+0000 7f26d4d58700 1 -- 172.21.15.52:0/3447082967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26d00ff1a0 msgr2=0x7f26d0109a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:06.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.814+0000 7f26d4d58700 1 --2- 172.21.15.52:0/3447082967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26d00ff1a0 0x7f26d0109a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:06.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.814+0000 7f26d4d58700 1 -- 172.21.15.52:0/3447082967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26d0100390 msgr2=0x7f26d01080d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:06.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.814+0000 7f26d4d58700 1 --2- 172.21.15.52:0/3447082967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26d0100390 0x7f26d01080d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:06.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.814+0000 7f26d4d58700 1 -- 172.21.15.52:0/3447082967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c8009d00 con 0x7f26d0100d20 2023-11-27T15:16:06.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.814+0000 7f26cffff700 1 --2- 172.21.15.52:0/3447082967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26d00ff1a0 0x7f26d0109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:16:06.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.814+0000 7f26d4d58700 1 --2- 172.21.15.52:0/3447082967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26d0100d20 0x7f26d0108610 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f26c800b070 tx=0x7f26c800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:06.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.814+0000 7f26cdffb700 1 -- 172.21.15.52:0/3447082967 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c800e040 con 0x7f26d0100d20 2023-11-27T15:16:06.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.814+0000 7f26cdffb700 1 -- 172.21.15.52:0/3447082967 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26c8003e10 con 0x7f26d0100d20 2023-11-27T15:16:06.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.814+0000 7f26cdffb700 1 -- 172.21.15.52:0/3447082967 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c80052a0 con 0x7f26d0100d20 2023-11-27T15:16:06.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.814+0000 7f26d67bb700 1 -- 172.21.15.52:0/3447082967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d010bd30 con 0x7f26d0100d20 2023-11-27T15:16:06.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.815+0000 7f26d67bb700 1 -- 172.21.15.52:0/3447082967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d010bba0 con 0x7f26d0100d20 2023-11-27T15:16:06.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.816+0000 7f26d67bb700 1 -- 172.21.15.52:0/3447082967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26d0106030 con 0x7f26d0100d20 2023-11-27T15:16:06.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.817+0000 7f26cdffb700 1 -- 172.21.15.52:0/3447082967 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f26c801d480 con 0x7f26d0100d20 2023-11-27T15:16:06.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.817+0000 7f26cdffb700 1 --2- 172.21.15.52:0/3447082967 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f26bc0a11d0 0x7f26bc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:06.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.818+0000 7f26cdffb700 1 -- 172.21.15.52:0/3447082967 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f26c80cd560 con 0x7f26d0100d20 2023-11-27T15:16:06.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.818+0000 7f26cffff700 1 --2- 172.21.15.52:0/3447082967 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f26bc0a11d0 0x7f26bc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:06.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.818+0000 7f26cffff700 1 --2- 172.21.15.52:0/3447082967 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f26bc0a11d0 0x7f26bc0a3680 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f26c0007fd0 tx=0x7f26c0007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:06.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:06.827+0000 7f26cdffb700 1 -- 172.21.15.52:0/3447082967 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f26c80971e0 con 0x7f26d0100d20 2023-11-27T15:16:07.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:06 smithi150 ceph-mon[100559]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:16:07.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:06 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1551450902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:16:07.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:06 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3349671554' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:16:07.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:06 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2650994169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:16:07.039 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.035+0000 7f26d67bb700 1 -- 172.21.15.52:0/3447082967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f26d0061700 con 0x7f26d0100d20 2023-11-27T15:16:07.039 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.035+0000 7f26cdffb700 1 -- 172.21.15.52:0/3447082967 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) v1 ==== 84+0+65 (secure 0 0 0) 0x7f26c809a800 con 0x7f26d0100d20 2023-11-27T15:16:07.040 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:16:07.040 INFO:teuthology.orchestra.run.smithi052.stdout:[ 2023-11-27T15:16:07.040 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:16:07.040 INFO:teuthology.orchestra.run.smithi052.stdout: "poolnum": 1, 2023-11-27T15:16:07.040 INFO:teuthology.orchestra.run.smithi052.stdout: "poolname": ".mgr" 2023-11-27T15:16:07.040 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:16:07.040 INFO:teuthology.orchestra.run.smithi052.stdout:] 2023-11-27T15:16:07.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.037+0000 7f26d67bb700 1 -- 172.21.15.52:0/3447082967 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f26bc0a11d0 msgr2=0x7f26bc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:07.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.037+0000 7f26d67bb700 1 --2- 172.21.15.52:0/3447082967 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f26bc0a11d0 0x7f26bc0a3680 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f26c0007fd0 tx=0x7f26c0007480 comp rx=0 tx=0).stop 2023-11-27T15:16:07.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.037+0000 7f26d67bb700 1 -- 172.21.15.52:0/3447082967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26d0100d20 msgr2=0x7f26d0108610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:07.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.037+0000 7f26d67bb700 1 --2- 172.21.15.52:0/3447082967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26d0100d20 0x7f26d0108610 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f26c800b070 tx=0x7f26c800bc80 comp rx=0 tx=0).stop 2023-11-27T15:16:07.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.037+0000 7f26d67bb700 1 -- 172.21.15.52:0/3447082967 shutdown_connections 2023-11-27T15:16:07.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.037+0000 7f26d67bb700 1 --2- 172.21.15.52:0/3447082967 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f26bc0a11d0 0x7f26bc0a3680 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:07.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.037+0000 7f26d67bb700 1 --2- 172.21.15.52:0/3447082967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26d0100d20 0x7f26d0108610 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:07.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.037+0000 7f26d67bb700 1 --2- 172.21.15.52:0/3447082967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26d00ff1a0 0x7f26d0109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:07.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.037+0000 7f26d67bb700 1 --2- 172.21.15.52:0/3447082967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26d0100390 0x7f26d01080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:07.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.038+0000 7f26d67bb700 1 -- 172.21.15.52:0/3447082967 >> 172.21.15.52:0/3447082967 conn(0x7f26d00fa780 msgr2=0x7f26d00fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:07.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.038+0000 7f26d67bb700 1 -- 172.21.15.52:0/3447082967 shutdown_connections 2023-11-27T15:16:07.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.038+0000 7f26d67bb700 1 -- 172.21.15.52:0/3447082967 wait complete. 2023-11-27T15:16:07.053 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:06 smithi052 ceph-mon[99056]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:16:07.053 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:06 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1551450902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:16:07.053 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:06 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3349671554' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:16:07.053 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:06 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2650994169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:16:07.054 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-27T15:16:07.054 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-27T15:16:07.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:06 smithi102 ceph-mon[100296]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:16:07.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:06 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1551450902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:16:07.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:06 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3349671554' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:16:07.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:06 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2650994169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:16:07.193 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.187+0000 7f52019df700 1 -- 172.21.15.52:0/3541677062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51fc0ff730 msgr2=0x7f51fc0ffb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:07.193 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.187+0000 7f52019df700 1 --2- 172.21.15.52:0/3541677062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51fc0ff730 0x7f51fc0ffb40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f51ec009990 tx=0x7f51ec009ca0 comp rx=0 tx=0).stop 2023-11-27T15:16:07.193 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.187+0000 7f52019df700 1 -- 172.21.15.52:0/3541677062 shutdown_connections 2023-11-27T15:16:07.193 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.187+0000 7f52019df700 1 --2- 172.21.15.52:0/3541677062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51fc0ff730 0x7f51fc0ffb40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:07.193 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.187+0000 7f52019df700 1 --2- 172.21.15.52:0/3541677062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51fc100df0 0x7f51fc107690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:07.194 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.187+0000 7f52019df700 1 --2- 172.21.15.52:0/3541677062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51fc100440 0x7f51fc1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:07.194 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.187+0000 7f52019df700 1 -- 172.21.15.52:0/3541677062 >> 172.21.15.52:0/3541677062 conn(0x7f51fc0f9fb0 msgr2=0x7f51fc0fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:07.194 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.187+0000 7f52019df700 1 -- 172.21.15.52:0/3541677062 shutdown_connections 2023-11-27T15:16:07.194 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.187+0000 7f52019df700 1 -- 172.21.15.52:0/3541677062 wait complete. 2023-11-27T15:16:07.194 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.187+0000 7f52019df700 1 Processor -- start 2023-11-27T15:16:07.194 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.188+0000 7f52019df700 1 -- start start 2023-11-27T15:16:07.194 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.188+0000 7f52019df700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51fc0ff730 0x7f51fc196060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:07.194 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.188+0000 7f52019df700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51fc100440 0x7f51fc1965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:07.194 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.188+0000 7f52019df700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51fc100df0 0x7f51fc19d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:07.194 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.188+0000 7f52019df700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc19dbe0 con 0x7f51fc100440 2023-11-27T15:16:07.194 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.188+0000 7f52019df700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc19dd20 con 0x7f51fc0ff730 2023-11-27T15:16:07.194 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.188+0000 7f52019df700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc19deb0 con 0x7f51fc100df0 2023-11-27T15:16:07.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.188+0000 7f51fa7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51fc100440 0x7f51fc1965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:07.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.188+0000 7f51faffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51fc0ff730 0x7f51fc196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:07.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.188+0000 7f51faffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51fc0ff730 0x7f51fc196060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:39640/0 (socket says 172.21.15.52:39640) 2023-11-27T15:16:07.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.188+0000 7f51fa7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51fc100440 0x7f51fc1965a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49984/0 (socket says 172.21.15.52:49984) 2023-11-27T15:16:07.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.188+0000 7f51faffd700 1 -- 172.21.15.52:0/1070361894 learned_addr learned my addr 172.21.15.52:0/1070361894 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:07.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.188+0000 7f51fb7fe700 1 --2- 172.21.15.52:0/1070361894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51fc100df0 0x7f51fc19d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:07.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.188+0000 7f51fa7fc700 1 -- 172.21.15.52:0/1070361894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51fc100df0 msgr2=0x7f51fc19d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:07.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.188+0000 7f51fa7fc700 1 --2- 172.21.15.52:0/1070361894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51fc100df0 0x7f51fc19d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:07.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.188+0000 7f51fa7fc700 1 -- 172.21.15.52:0/1070361894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51fc0ff730 msgr2=0x7f51fc196060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:07.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.188+0000 7f51fa7fc700 1 --2- 172.21.15.52:0/1070361894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51fc0ff730 0x7f51fc196060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:07.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.188+0000 7f51fa7fc700 1 -- 172.21.15.52:0/1070361894 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ec0095f0 con 0x7f51fc100440 2023-11-27T15:16:07.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.188+0000 7f51faffd700 1 --2- 172.21.15.52:0/1070361894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51fc0ff730 0x7f51fc196060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:16:07.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.188+0000 7f51fa7fc700 1 --2- 172.21.15.52:0/1070361894 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51fc100440 0x7f51fc1965a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f51f000e9c0 tx=0x7f51f000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:07.196 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.189+0000 7f52009dd700 1 -- 172.21.15.52:0/1070361894 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f0004220 con 0x7f51fc100440 2023-11-27T15:16:07.196 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.189+0000 7f52019df700 1 -- 172.21.15.52:0/1070361894 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51fc10ab80 con 0x7f51fc100440 2023-11-27T15:16:07.196 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.189+0000 7f52009dd700 1 -- 172.21.15.52:0/1070361894 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51f0004380 con 0x7f51fc100440 2023-11-27T15:16:07.196 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.189+0000 7f52009dd700 1 -- 172.21.15.52:0/1070361894 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f0003be0 con 0x7f51fc100440 2023-11-27T15:16:07.196 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.189+0000 7f52019df700 1 -- 172.21.15.52:0/1070361894 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51fc10a9f0 con 0x7f51fc100440 2023-11-27T15:16:07.197 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.190+0000 7f52019df700 1 -- 172.21.15.52:0/1070361894 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51fc067f50 con 0x7f51fc100440 2023-11-27T15:16:07.197 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.191+0000 7f52009dd700 1 -- 172.21.15.52:0/1070361894 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f51f0003d40 con 0x7f51fc100440 2023-11-27T15:16:07.197 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.191+0000 7f52009dd700 1 --2- 172.21.15.52:0/1070361894 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f51e40a1150 0x7f51e40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:07.197 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.192+0000 7f52009dd700 1 -- 172.21.15.52:0/1070361894 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f51f0014070 con 0x7f51fc100440 2023-11-27T15:16:07.197 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.192+0000 7f51faffd700 1 --2- 172.21.15.52:0/1070361894 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f51e40a1150 0x7f51e40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:07.197 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.192+0000 7f51faffd700 1 --2- 172.21.15.52:0/1070361894 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f51e40a1150 0x7f51e40a3600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00b2e0 tx=0x7f51ec005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:07.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.195+0000 7f52009dd700 1 -- 172.21.15.52:0/1070361894 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f51f0096490 con 0x7f51fc100440 2023-11-27T15:16:07.402 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.398+0000 7f52019df700 1 -- 172.21.15.52:0/1070361894 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f51fc19ea70 con 0x7f51fc100440 2023-11-27T15:16:07.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.895+0000 7f52009dd700 1 -- 172.21.15.52:0/1070361894 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) v1 ==== 138+0+0 (secure 0 0 0) 0x7f51f001f340 con 0x7f51fc100440 2023-11-27T15:16:07.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.899+0000 7f52019df700 1 -- 172.21.15.52:0/1070361894 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f51e40a1150 msgr2=0x7f51e40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:07.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.899+0000 7f52019df700 1 --2- 172.21.15.52:0/1070361894 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f51e40a1150 0x7f51e40a3600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00b2e0 tx=0x7f51ec005fb0 comp rx=0 tx=0).stop 2023-11-27T15:16:07.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.899+0000 7f52019df700 1 -- 172.21.15.52:0/1070361894 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51fc100440 msgr2=0x7f51fc1965a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:07.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.899+0000 7f52019df700 1 --2- 172.21.15.52:0/1070361894 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51fc100440 0x7f51fc1965a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f51f000e9c0 tx=0x7f51f000ecd0 comp rx=0 tx=0).stop 2023-11-27T15:16:07.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.900+0000 7f52019df700 1 -- 172.21.15.52:0/1070361894 shutdown_connections 2023-11-27T15:16:07.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.900+0000 7f52019df700 1 --2- 172.21.15.52:0/1070361894 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f51e40a1150 0x7f51e40a3600 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:07.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.900+0000 7f52019df700 1 --2- 172.21.15.52:0/1070361894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51fc0ff730 0x7f51fc196060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:07.905 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.900+0000 7f52019df700 1 --2- 172.21.15.52:0/1070361894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51fc100df0 0x7f51fc19d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:07.905 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.900+0000 7f52019df700 1 --2- 172.21.15.52:0/1070361894 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51fc100440 0x7f51fc1965a0 secure :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f51f000e9c0 tx=0x7f51f000ecd0 comp rx=0 tx=0).stop 2023-11-27T15:16:07.905 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.900+0000 7f52019df700 1 -- 172.21.15.52:0/1070361894 >> 172.21.15.52:0/1070361894 conn(0x7f51fc0f9fb0 msgr2=0x7f51fc0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:07.905 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.900+0000 7f52019df700 1 -- 172.21.15.52:0/1070361894 shutdown_connections 2023-11-27T15:16:07.905 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:07.901+0000 7f52019df700 1 -- 172.21.15.52:0/1070361894 wait complete. 2023-11-27T15:16:07.906 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_metadata' created 2023-11-27T15:16:07.921 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-27T15:16:08.066 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.062+0000 7fb4d3ec2700 1 -- 172.21.15.52:0/2226241538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4cc101f90 msgr2=0x7fb4cc108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:08.067 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.062+0000 7fb4d3ec2700 1 --2- 172.21.15.52:0/2226241538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4cc101f90 0x7fb4cc108850 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb4c400b0a0 tx=0x7fb4c400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:16:08.067 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.062+0000 7fb4d3ec2700 1 -- 172.21.15.52:0/2226241538 shutdown_connections 2023-11-27T15:16:08.067 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.062+0000 7fb4d3ec2700 1 --2- 172.21.15.52:0/2226241538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4cc101f90 0x7fb4cc108850 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:08.067 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.062+0000 7fb4d3ec2700 1 --2- 172.21.15.52:0/2226241538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4cc0fe590 0x7fb4cc0fe9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:08.067 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.062+0000 7fb4d3ec2700 1 --2- 172.21.15.52:0/2226241538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4cc101600 0x7fb4cc101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:08.067 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.062+0000 7fb4d3ec2700 1 -- 172.21.15.52:0/2226241538 >> 172.21.15.52:0/2226241538 conn(0x7fb4cc0f9f30 msgr2=0x7fb4cc0fc380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:08.067 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.062+0000 7fb4d3ec2700 1 -- 172.21.15.52:0/2226241538 shutdown_connections 2023-11-27T15:16:08.067 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.062+0000 7fb4d3ec2700 1 -- 172.21.15.52:0/2226241538 wait complete. 2023-11-27T15:16:08.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.063+0000 7fb4d3ec2700 1 Processor -- start 2023-11-27T15:16:08.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.063+0000 7fb4d3ec2700 1 -- start start 2023-11-27T15:16:08.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.063+0000 7fb4d3ec2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4cc0fe590 0x7fb4cc196100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:08.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.063+0000 7fb4d3ec2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4cc101600 0x7fb4cc196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:08.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.063+0000 7fb4d3ec2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4cc101f90 0x7fb4cc19d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:08.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.063+0000 7fb4d3ec2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc19dc80 con 0x7fb4cc0fe590 2023-11-27T15:16:08.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.063+0000 7fb4d3ec2700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc19ddc0 con 0x7fb4cc101f90 2023-11-27T15:16:08.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.063+0000 7fb4d3ec2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc19df50 con 0x7fb4cc101600 2023-11-27T15:16:08.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.063+0000 7fb4d1c5e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4cc0fe590 0x7fb4cc196100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:08.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.063+0000 7fb4d1c5e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4cc0fe590 0x7fb4cc196100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49994/0 (socket says 172.21.15.52:49994) 2023-11-27T15:16:08.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.063+0000 7fb4d1c5e700 1 -- 172.21.15.52:0/3349244992 learned_addr learned my addr 172.21.15.52:0/3349244992 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:08.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.063+0000 7fb4d145d700 1 --2- 172.21.15.52:0/3349244992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4cc101600 0x7fb4cc196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:08.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.063+0000 7fb4d245f700 1 --2- 172.21.15.52:0/3349244992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4cc101f90 0x7fb4cc19d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:08.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.063+0000 7fb4d1c5e700 1 -- 172.21.15.52:0/3349244992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4cc101600 msgr2=0x7fb4cc196640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:08.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.063+0000 7fb4d1c5e700 1 --2- 172.21.15.52:0/3349244992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4cc101600 0x7fb4cc196640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:08.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.063+0000 7fb4d1c5e700 1 -- 172.21.15.52:0/3349244992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4cc101f90 msgr2=0x7fb4cc19d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:08.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.063+0000 7fb4d1c5e700 1 --2- 172.21.15.52:0/3349244992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4cc101f90 0x7fb4cc19d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:08.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.063+0000 7fb4d1c5e700 1 -- 172.21.15.52:0/3349244992 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4bc009520 con 0x7fb4cc0fe590 2023-11-27T15:16:08.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.063+0000 7fb4d1c5e700 1 --2- 172.21.15.52:0/3349244992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4cc0fe590 0x7fb4cc196100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb4c800e970 tx=0x7fb4c800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:08.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.064+0000 7fb4c2ffd700 1 -- 172.21.15.52:0/3349244992 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8009790 con 0x7fb4cc0fe590 2023-11-27T15:16:08.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.064+0000 7fb4d3ec2700 1 -- 172.21.15.52:0/3349244992 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4c4009d00 con 0x7fb4cc0fe590 2023-11-27T15:16:08.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.064+0000 7fb4c2ffd700 1 -- 172.21.15.52:0/3349244992 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4c800c7e0 con 0x7fb4cc0fe590 2023-11-27T15:16:08.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.064+0000 7fb4c2ffd700 1 -- 172.21.15.52:0/3349244992 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8003c70 con 0x7fb4cc0fe590 2023-11-27T15:16:08.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.064+0000 7fb4d3ec2700 1 -- 172.21.15.52:0/3349244992 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4cc10bc30 con 0x7fb4cc0fe590 2023-11-27T15:16:08.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.065+0000 7fb4d3ec2700 1 -- 172.21.15.52:0/3349244992 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4cc067270 con 0x7fb4cc0fe590 2023-11-27T15:16:08.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.066+0000 7fb4c2ffd700 1 -- 172.21.15.52:0/3349244992 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb4c800cc50 con 0x7fb4cc0fe590 2023-11-27T15:16:08.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.067+0000 7fb4c2ffd700 1 --2- 172.21.15.52:0/3349244992 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb4b80a1150 0x7fb4b80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:08.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.067+0000 7fb4d145d700 1 --2- 172.21.15.52:0/3349244992 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb4b80a1150 0x7fb4b80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:08.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.067+0000 7fb4c2ffd700 1 -- 172.21.15.52:0/3349244992 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fb4c8014070 con 0x7fb4cc0fe590 2023-11-27T15:16:08.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.068+0000 7fb4d145d700 1 --2- 172.21.15.52:0/3349244992 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb4b80a1150 0x7fb4b80a3600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009df0 tx=0x7fb4bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:08.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.071+0000 7fb4c2ffd700 1 -- 172.21.15.52:0/3349244992 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb4c8096350 con 0x7fb4cc0fe590 2023-11-27T15:16:08.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:07 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3447082967' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-11-27T15:16:08.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:07 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3447082967' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-11-27T15:16:08.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.287+0000 7fb4d3ec2700 1 -- 172.21.15.52:0/3349244992 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb4cc107270 con 0x7fb4cc0fe590 2023-11-27T15:16:08.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:07 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3447082967' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-11-27T15:16:08.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.899+0000 7fb4c2ffd700 1 -- 172.21.15.52:0/3349244992 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb4c8003dd0 con 0x7fb4cc0fe590 2023-11-27T15:16:08.910 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.906+0000 7fb4d3ec2700 1 -- 172.21.15.52:0/3349244992 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb4b80a1150 msgr2=0x7fb4b80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:08.910 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.906+0000 7fb4d3ec2700 1 --2- 172.21.15.52:0/3349244992 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb4b80a1150 0x7fb4b80a3600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009df0 tx=0x7fb4bc009260 comp rx=0 tx=0).stop 2023-11-27T15:16:08.910 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.906+0000 7fb4d3ec2700 1 -- 172.21.15.52:0/3349244992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4cc0fe590 msgr2=0x7fb4cc196100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:08.910 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.906+0000 7fb4d3ec2700 1 --2- 172.21.15.52:0/3349244992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4cc0fe590 0x7fb4cc196100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb4c800e970 tx=0x7fb4c800ec80 comp rx=0 tx=0).stop 2023-11-27T15:16:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.908+0000 7fb4d3ec2700 1 -- 172.21.15.52:0/3349244992 shutdown_connections 2023-11-27T15:16:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.908+0000 7fb4d3ec2700 1 --2- 172.21.15.52:0/3349244992 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb4b80a1150 0x7fb4b80a3600 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.908+0000 7fb4d3ec2700 1 --2- 172.21.15.52:0/3349244992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4cc101f90 0x7fb4cc19d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.908+0000 7fb4d3ec2700 1 --2- 172.21.15.52:0/3349244992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4cc0fe590 0x7fb4cc196100 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.908+0000 7fb4d3ec2700 1 --2- 172.21.15.52:0/3349244992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4cc101600 0x7fb4cc196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:08.913 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.908+0000 7fb4d3ec2700 1 -- 172.21.15.52:0/3349244992 >> 172.21.15.52:0/3349244992 conn(0x7fb4cc0f9f30 msgr2=0x7fb4cc0fc100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:08.914 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.909+0000 7fb4d3ec2700 1 -- 172.21.15.52:0/3349244992 shutdown_connections 2023-11-27T15:16:08.914 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:08.909+0000 7fb4d3ec2700 1 -- 172.21.15.52:0/3349244992 wait complete. 2023-11-27T15:16:08.922 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data' created 2023-11-27T15:16:08.932 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-27T15:16:09.065 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.060+0000 7fa96d4df700 1 -- 172.21.15.52:0/337422492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9680ffee0 msgr2=0x7fa968100350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:09.065 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.060+0000 7fa96d4df700 1 --2- 172.21.15.52:0/337422492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9680ffee0 0x7fa968100350 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa96000b0a0 tx=0x7fa96000b3b0 comp rx=0 tx=0).stop 2023-11-27T15:16:09.065 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.061+0000 7fa96d4df700 1 -- 172.21.15.52:0/337422492 shutdown_connections 2023-11-27T15:16:09.066 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.061+0000 7fa96d4df700 1 --2- 172.21.15.52:0/337422492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9680febe0 0x7fa9680feff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:09.066 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.061+0000 7fa96d4df700 1 --2- 172.21.15.52:0/337422492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9680ffee0 0x7fa968100350 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:09.066 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.061+0000 7fa96d4df700 1 --2- 172.21.15.52:0/337422492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9680ff530 0x7fa9680ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:09.066 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.061+0000 7fa96d4df700 1 -- 172.21.15.52:0/337422492 >> 172.21.15.52:0/337422492 conn(0x7fa9680fa810 msgr2=0x7fa9680fcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:09.066 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.061+0000 7fa96d4df700 1 -- 172.21.15.52:0/337422492 shutdown_connections 2023-11-27T15:16:09.066 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.061+0000 7fa96d4df700 1 -- 172.21.15.52:0/337422492 wait complete. 2023-11-27T15:16:09.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.061+0000 7fa96d4df700 1 Processor -- start 2023-11-27T15:16:09.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.062+0000 7fa96d4df700 1 -- start start 2023-11-27T15:16:09.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.062+0000 7fa96d4df700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9680febe0 0x7fa968198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:09.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.062+0000 7fa96d4df700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9680ff530 0x7fa968198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:09.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.062+0000 7fa96d4df700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9680ffee0 0x7fa96819f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:09.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.062+0000 7fa96d4df700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96819fe90 con 0x7fa9680febe0 2023-11-27T15:16:09.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.062+0000 7fa96d4df700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96819ffd0 con 0x7fa9680ff530 2023-11-27T15:16:09.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.062+0000 7fa96d4df700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9681a0160 con 0x7fa9680ffee0 2023-11-27T15:16:09.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.062+0000 7fa966ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9680febe0 0x7fa968198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:09.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.062+0000 7fa966ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9680febe0 0x7fa968198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50012/0 (socket says 172.21.15.52:50012) 2023-11-27T15:16:09.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.062+0000 7fa966ffd700 1 -- 172.21.15.52:0/3112629778 learned_addr learned my addr 172.21.15.52:0/3112629778 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:09.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.062+0000 7fa9677fe700 1 --2- 172.21.15.52:0/3112629778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9680ffee0 0x7fa96819f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:09.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.062+0000 7fa9667fc700 1 --2- 172.21.15.52:0/3112629778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9680ff530 0x7fa968198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:09.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.062+0000 7fa9677fe700 1 -- 172.21.15.52:0/3112629778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9680ff530 msgr2=0x7fa968198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:09.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.062+0000 7fa9677fe700 1 --2- 172.21.15.52:0/3112629778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9680ff530 0x7fa968198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:09.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.062+0000 7fa9677fe700 1 -- 172.21.15.52:0/3112629778 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9680febe0 msgr2=0x7fa968198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:09.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.062+0000 7fa9677fe700 1 --2- 172.21.15.52:0/3112629778 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9680febe0 0x7fa968198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:09.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.062+0000 7fa9677fe700 1 -- 172.21.15.52:0/3112629778 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa958009520 con 0x7fa9680ffee0 2023-11-27T15:16:09.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.062+0000 7fa966ffd700 1 --2- 172.21.15.52:0/3112629778 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9680febe0 0x7fa968198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:16:09.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.063+0000 7fa9677fe700 1 --2- 172.21.15.52:0/3112629778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9680ffee0 0x7fa96819f8c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa96000b0a0 tx=0x7fa960007f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:09.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.063+0000 7fa94ffff700 1 -- 172.21.15.52:0/3112629778 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96000e040 con 0x7fa9680ffee0 2023-11-27T15:16:09.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.063+0000 7fa96d4df700 1 -- 172.21.15.52:0/3112629778 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa960009d00 con 0x7fa9680ffee0 2023-11-27T15:16:09.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.063+0000 7fa94ffff700 1 -- 172.21.15.52:0/3112629778 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa96000bde0 con 0x7fa9680ffee0 2023-11-27T15:16:09.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.063+0000 7fa94ffff700 1 -- 172.21.15.52:0/3112629778 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96001b6b0 con 0x7fa9680ffee0 2023-11-27T15:16:09.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.063+0000 7fa96d4df700 1 -- 172.21.15.52:0/3112629778 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa96810de40 con 0x7fa9680ffee0 2023-11-27T15:16:09.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.064+0000 7fa96d4df700 1 -- 172.21.15.52:0/3112629778 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa96810dfd0 con 0x7fa9680ffee0 2023-11-27T15:16:09.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.065+0000 7fa94ffff700 1 -- 172.21.15.52:0/3112629778 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa96001d480 con 0x7fa9680ffee0 2023-11-27T15:16:09.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.066+0000 7fa94ffff700 1 --2- 172.21.15.52:0/3112629778 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fa9500a1150 0x7fa9500a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:09.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.066+0000 7fa966ffd700 1 --2- 172.21.15.52:0/3112629778 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fa9500a1150 0x7fa9500a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:09.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.066+0000 7fa94ffff700 1 -- 172.21.15.52:0/3112629778 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7879+0+0 (secure 0 0 0) 0x7fa9600cdde0 con 0x7fa9680ffee0 2023-11-27T15:16:09.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.066+0000 7fa966ffd700 1 --2- 172.21.15.52:0/3112629778 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fa9500a1150 0x7fa9500a3600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa958009fd0 tx=0x7fa958009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:09.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.069+0000 7fa94ffff700 1 -- 172.21.15.52:0/3112629778 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa9600977f0 con 0x7fa9680ffee0 2023-11-27T15:16:09.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:08 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1070361894' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-11-27T15:16:09.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:08 smithi052 ceph-mon[99056]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:16:09.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:08 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:09.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:08 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:09.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:08 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1070361894' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-11-27T15:16:09.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:08 smithi052 ceph-mon[99056]: osdmap e80: 12 total, 12 up, 12 in 2023-11-27T15:16:09.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:08 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:09.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:08 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:09.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:08 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:09.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:08 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:09.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:08 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3349244992' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-11-27T15:16:09.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:08 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:09.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:08 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:09.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:08 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:16:09.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:08 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:09.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:08 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1070361894' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-11-27T15:16:09.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:08 smithi150 ceph-mon[100559]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:16:09.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:08 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:09.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:08 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:09.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:08 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1070361894' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-11-27T15:16:09.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:08 smithi150 ceph-mon[100559]: osdmap e80: 12 total, 12 up, 12 in 2023-11-27T15:16:09.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:08 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:09.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:08 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:09.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:08 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:09.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:08 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:09.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:08 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3349244992' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-11-27T15:16:09.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:08 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:09.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:08 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:09.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:08 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:16:09.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:08 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:09.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.317+0000 7fa96d4df700 1 -- 172.21.15.52:0/3112629778 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa96802cfe0 con 0x7fa9680ffee0 2023-11-27T15:16:09.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:08 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1070361894' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-11-27T15:16:09.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:08 smithi102 ceph-mon[100296]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:16:09.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:08 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:09.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:08 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:09.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:08 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1070361894' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-11-27T15:16:09.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:08 smithi102 ceph-mon[100296]: osdmap e80: 12 total, 12 up, 12 in 2023-11-27T15:16:09.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:08 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:09.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:08 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:09.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:08 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:09.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:08 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:09.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:08 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3349244992' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-11-27T15:16:09.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:08 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:09.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:08 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:09.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:08 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:16:09.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:08 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:09.329 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.324+0000 7fa94ffff700 1 -- 172.21.15.52:0/3112629778 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fa9600210c0 con 0x7fa9680ffee0 2023-11-27T15:16:09.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.332+0000 7fa96d4df700 1 -- 172.21.15.52:0/3112629778 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fa9500a1150 msgr2=0x7fa9500a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:09.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.332+0000 7fa96d4df700 1 --2- 172.21.15.52:0/3112629778 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fa9500a1150 0x7fa9500a3600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa958009fd0 tx=0x7fa958009260 comp rx=0 tx=0).stop 2023-11-27T15:16:09.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.332+0000 7fa96d4df700 1 -- 172.21.15.52:0/3112629778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9680ffee0 msgr2=0x7fa96819f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:09.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.332+0000 7fa96d4df700 1 --2- 172.21.15.52:0/3112629778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9680ffee0 0x7fa96819f8c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa96000b0a0 tx=0x7fa960007f30 comp rx=0 tx=0).stop 2023-11-27T15:16:09.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.332+0000 7fa96d4df700 1 -- 172.21.15.52:0/3112629778 shutdown_connections 2023-11-27T15:16:09.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.332+0000 7fa96d4df700 1 --2- 172.21.15.52:0/3112629778 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fa9500a1150 0x7fa9500a3600 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa958009fd0 tx=0x7fa958009260 comp rx=0 tx=0).stop 2023-11-27T15:16:09.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.332+0000 7fa96d4df700 1 --2- 172.21.15.52:0/3112629778 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9680ffee0 0x7fa96819f8c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:09.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.332+0000 7fa96d4df700 1 --2- 172.21.15.52:0/3112629778 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9680febe0 0x7fa968198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:09.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.332+0000 7fa96d4df700 1 --2- 172.21.15.52:0/3112629778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9680ff530 0x7fa968198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:09.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.332+0000 7fa96d4df700 1 -- 172.21.15.52:0/3112629778 >> 172.21.15.52:0/3112629778 conn(0x7fa9680fa810 msgr2=0x7fa9680fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:09.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.332+0000 7fa96d4df700 1 -- 172.21.15.52:0/3112629778 shutdown_connections 2023-11-27T15:16:09.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.332+0000 7fa96d4df700 1 -- 172.21.15.52:0/3112629778 wait complete. 2023-11-27T15:16:09.347 INFO:teuthology.orchestra.run.smithi052.stderr:new fs with metadata pool 2 and data pool 3 2023-11-27T15:16:09.360 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:16:09.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.491+0000 7f5e217a7700 1 -- 172.21.15.52:0/828138514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e1c1080b0 msgr2=0x7f5e1c10a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:09.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.491+0000 7f5e217a7700 1 --2- 172.21.15.52:0/828138514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e1c1080b0 0x7f5e1c10a560 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400b0a0 tx=0x7f5e1400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:16:09.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.491+0000 7f5e217a7700 1 -- 172.21.15.52:0/828138514 shutdown_connections 2023-11-27T15:16:09.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.491+0000 7f5e217a7700 1 --2- 172.21.15.52:0/828138514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e1c069d90 0x7f5e1c105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:09.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.491+0000 7f5e217a7700 1 --2- 172.21.15.52:0/828138514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e1c1080b0 0x7f5e1c10a560 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:09.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.491+0000 7f5e217a7700 1 --2- 172.21.15.52:0/828138514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e1c105790 0x7f5e1c107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:09.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.491+0000 7f5e217a7700 1 -- 172.21.15.52:0/828138514 >> 172.21.15.52:0/828138514 conn(0x7f5e1c0fa7c0 msgr2=0x7f5e1c0fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:09.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.491+0000 7f5e217a7700 1 -- 172.21.15.52:0/828138514 shutdown_connections 2023-11-27T15:16:09.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.491+0000 7f5e217a7700 1 -- 172.21.15.52:0/828138514 wait complete. 2023-11-27T15:16:09.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.492+0000 7f5e217a7700 1 Processor -- start 2023-11-27T15:16:09.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.492+0000 7f5e217a7700 1 -- start start 2023-11-27T15:16:09.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.492+0000 7f5e217a7700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e1c069d90 0x7f5e1c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:09.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.492+0000 7f5e217a7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e1c105790 0x7f5e1c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:09.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.492+0000 7f5e217a7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e1c1080b0 0x7f5e1c19f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:09.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.492+0000 7f5e217a7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1c19fee0 con 0x7f5e1c105790 2023-11-27T15:16:09.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.492+0000 7f5e217a7700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1c1a0020 con 0x7f5e1c069d90 2023-11-27T15:16:09.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.492+0000 7f5e217a7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1c1a01b0 con 0x7f5e1c1080b0 2023-11-27T15:16:09.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.493+0000 7f5e1a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e1c105790 0x7f5e1c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:09.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.493+0000 7f5e1a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e1c105790 0x7f5e1c1988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50032/0 (socket says 172.21.15.52:50032) 2023-11-27T15:16:09.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.493+0000 7f5e1affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e1c069d90 0x7f5e1c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:09.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.493+0000 7f5e1a7fc700 1 -- 172.21.15.52:0/659042253 learned_addr learned my addr 172.21.15.52:0/659042253 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:09.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.493+0000 7f5e1b7fe700 1 --2- 172.21.15.52:0/659042253 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e1c1080b0 0x7f5e1c19f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:09.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.493+0000 7f5e1a7fc700 1 -- 172.21.15.52:0/659042253 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e1c1080b0 msgr2=0x7f5e1c19f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:09.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.493+0000 7f5e1a7fc700 1 --2- 172.21.15.52:0/659042253 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e1c1080b0 0x7f5e1c19f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:09.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.493+0000 7f5e1a7fc700 1 -- 172.21.15.52:0/659042253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e1c069d90 msgr2=0x7f5e1c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:09.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.493+0000 7f5e1a7fc700 1 --2- 172.21.15.52:0/659042253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e1c069d90 0x7f5e1c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:09.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.493+0000 7f5e1a7fc700 1 -- 172.21.15.52:0/659042253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e14009d00 con 0x7f5e1c105790 2023-11-27T15:16:09.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.493+0000 7f5e1a7fc700 1 --2- 172.21.15.52:0/659042253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e1c105790 0x7f5e1c1988a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000b530 tx=0x7f5e1000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:09.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.493+0000 7f5e03fff700 1 -- 172.21.15.52:0/659042253 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10011830 con 0x7f5e1c105790 2023-11-27T15:16:09.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.493+0000 7f5e03fff700 1 -- 172.21.15.52:0/659042253 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e10011e50 con 0x7f5e1c105790 2023-11-27T15:16:09.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.493+0000 7f5e03fff700 1 -- 172.21.15.52:0/659042253 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1000f2e0 con 0x7f5e1c105790 2023-11-27T15:16:09.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.493+0000 7f5e217a7700 1 -- 172.21.15.52:0/659042253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e1c10df10 con 0x7f5e1c105790 2023-11-27T15:16:09.501 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.493+0000 7f5e217a7700 1 -- 172.21.15.52:0/659042253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e1c10dd80 con 0x7f5e1c105790 2023-11-27T15:16:09.501 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.494+0000 7f5e217a7700 1 -- 172.21.15.52:0/659042253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e1c0fc330 con 0x7f5e1c105790 2023-11-27T15:16:09.501 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.495+0000 7f5e03fff700 1 -- 172.21.15.52:0/659042253 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5e10010380 con 0x7f5e1c105790 2023-11-27T15:16:09.501 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.496+0000 7f5e03fff700 1 --2- 172.21.15.52:0/659042253 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f5e040a1150 0x7f5e040a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:09.501 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.496+0000 7f5e03fff700 1 -- 172.21.15.52:0/659042253 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5e100cc8c0 con 0x7f5e1c105790 2023-11-27T15:16:09.501 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.496+0000 7f5e1affd700 1 --2- 172.21.15.52:0/659042253 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f5e040a1150 0x7f5e040a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:09.501 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.496+0000 7f5e1affd700 1 --2- 172.21.15.52:0/659042253 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f5e040a1150 0x7f5e040a3600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c005950 tx=0x7f5e0c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:09.504 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.500+0000 7f5e03fff700 1 -- 172.21.15.52:0/659042253 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5e10096160 con 0x7f5e1c105790 2023-11-27T15:16:09.712 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.708+0000 7f5e217a7700 1 -- 172.21.15.52:0/659042253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e1c02cfe0 con 0x7f5e1c105790 2023-11-27T15:16:09.714 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.710+0000 7f5e03fff700 1 -- 172.21.15.52:0/659042253 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22143 (secure 0 0 0) 0x7f5e10099780 con 0x7f5e1c105790 2023-11-27T15:16:09.715 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:16:09.715 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":82,"fsid":"c6c7df88-8d36-11ee-95a2-87774f69a715","created":"2023-11-27T15:09:03.333345+0000","modified":"2023-11-27T15:16:09.319166+0000","last_up_change":"2023-11-27T15:15:27.656134+0000","last_in_change":"2023-11-27T15:15:10.671153+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:12:14.177523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:16:07.842251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:16:08.728520+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58d69ede-c293-49f5-abbe-8187793f2b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6803","nonce":608216415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6805","nonce":608216415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6809","nonce":608216415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6807","nonce":608216415}]},"public_addr":"172.21.15.52:6803/608216415","cluster_addr":"172.21.15.52:6805/608216415","heartbeat_back_addr":"172.21.15.52:6809/608216415","heartbeat_front_addr":"172.21.15.52:6807/608216415","state":["exists","up"]},{"osd":1,"uuid":"d6a46b91-adc8-45ef-80f5-566ae400911f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6801","nonce":2538384087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6803","nonce":2538384087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6807","nonce":2538384087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6805","nonce":2538384087}]},"public_addr":"172.21.15.102:6801/2538384087","cluster_addr":"172.21.15.102:6803/2538384087","heartbeat_back_addr":"172.21.15.102:6807/2538384087","heartbeat_front_addr":"172.21.15.102:6805/2538384087","state":["exists","up"]},{"osd":2,"uuid":"846ba125-11f8-46b2-8941-bd5e1947cc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6801","nonce":4258038221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6803","nonce":4258038221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6807","nonce":4258038221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6805","nonce":4258038221}]},"public_addr":"172.21.15.150:6801/4258038221","cluster_addr":"172.21.15.150:6803/4258038221","heartbeat_back_addr":"172.21.15.150:6807/4258038221","heartbeat_front_addr":"172.21.15.150:6805/4258038221","state":["exists","up"]},{"osd":3,"uuid":"950fcd67-1057-4288-a389-714f1ec94261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6811","nonce":2622100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6813","nonce":2622100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6817","nonce":2622100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6815","nonce":2622100752}]},"public_addr":"172.21.15.52:6811/2622100752","cluster_addr":"172.21.15.52:6813/2622100752","heartbeat_back_addr":"172.21.15.52:6817/2622100752","heartbeat_front_addr":"172.21.15.52:6815/2622100752","state":["exists","up"]},{"osd":4,"uuid":"1c96a676-cea9-4deb-bad1-f7fd240c3d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6809","nonce":1546794763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6811","nonce":1546794763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6815","nonce":1546794763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6813","nonce":1546794763}]},"public_addr":"172.21.15.102:6809/1546794763","cluster_addr":"172.21.15.102:6811/1546794763","heartbeat_back_addr":"172.21.15.102:6815/1546794763","heartbeat_front_addr":"172.21.15.102:6813/1546794763","state":["exists","up"]},{"osd":5,"uuid":"38a75af1-fd4d-433e-b579-0609109c851c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6809","nonce":461511375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6811","nonce":461511375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6815","nonce":461511375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6813","nonce":461511375}]},"public_addr":"172.21.15.150:6809/461511375","cluster_addr":"172.21.15.150:6811/461511375","heartbeat_back_addr":"172.21.15.150:6815/461511375","heartbeat_front_addr":"172.21.15.150:6813/461511375","state":["exists","up"]},{"osd":6,"uuid":"2f0ede4a-d175-4b4a-b876-c0f442048901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6819","nonce":1002457464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6821","nonce":1002457464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6825","nonce":1002457464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6823","nonce":1002457464}]},"public_addr":"172.21.15.52:6819/1002457464","cluster_addr":"172.21.15.52:6821/1002457464","heartbeat_back_addr":"172.21.15.52:6825/1002457464","heartbeat_front_addr":"172.21.15.52:6823/1002457464","state":["exists","up"]},{"osd":7,"uuid":"6b635192-abcf-4bbf-bf78-a019b351bca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6817","nonce":3249840064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6819","nonce":3249840064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6823","nonce":3249840064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6821","nonce":3249840064}]},"public_addr":"172.21.15.102:6817/3249840064","cluster_addr":"172.21.15.102:6819/3249840064","heartbeat_back_addr":"172.21.15.102:6823/3249840064","heartbeat_front_addr":"172.21.15.102:6821/3249840064","state":["exists","up"]},{"osd":8,"uuid":"c7ca8007-7170-47bf-956c-3f00edf27e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6817","nonce":3672583574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6819","nonce":3672583574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6823","nonce":3672583574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6821","nonce":3672583574}]},"public_addr":"172.21.15.150:6817/3672583574","cluster_addr":"172.21.15.150:6819/3672583574","heartbeat_back_addr":"172.21.15.150:6823/3672583574","heartbeat_front_addr":"172.21.15.150:6821/3672583574","state":["exists","up"]},{"osd":9,"uuid":"0e252c2c-01a4-48ba-832d-067ed940fefb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6827","nonce":4287934320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6829","nonce":4287934320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6833","nonce":4287934320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6831","nonce":4287934320}]},"public_addr":"172.21.15.52:6827/4287934320","cluster_addr":"172.21.15.52:6829/4287934320","heartbeat_back_addr":"172.21.15.52:6833/4287934320","heartbeat_front_addr":"172.21.15.52:6831/4287934320","state":["exists","up"]},{"osd":10,"uuid":"bb540142-7563-44a4-b7a3-cb662714c128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6825","nonce":3056034201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6827","nonce":3056034201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6831","nonce":3056034201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6829","nonce":3056034201}]},"public_addr":"172.21.15.102:6825/3056034201","cluster_addr":"172.21.15.102:6827/3056034201","heartbeat_back_addr":"172.21.15.102:6831/3056034201","heartbeat_front_addr":"172.21.15.102:6829/3056034201","state":["exists","up"]},{"osd":11,"uuid":"3156528e-e0f8-4d44-9946-c8b6339c2bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6825","nonce":3207559739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6827","nonce":3207559739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6831","nonce":3207559739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6829","nonce":3207559739}]},"public_addr":"172.21.15.150:6825/3207559739","cluster_addr":"172.21.15.150:6827/3207559739","heartbeat_back_addr":"172.21.15.150:6831/3207559739","heartbeat_front_addr":"172.21.15.150:6829/3207559739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:26.826099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:48.331371+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:10.459738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:32.356472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:54.043170+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:15.648582+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:37.166463+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:59.240819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:20.710097+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:43.364563+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:05.279433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:26.344573+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/1392282110":"2023-11-28T15:10:03.644870+0000","172.21.15.52:0/1479993479":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6800/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/961562652":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/1290665390":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2410529742":"2023-11-28T15:09:37.118690+0000","172.21.15.52:6800/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2130037044":"2023-11-28T15:10:03.644870+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:16:09.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.712+0000 7f5e217a7700 1 -- 172.21.15.52:0/659042253 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f5e040a1150 msgr2=0x7f5e040a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:09.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.712+0000 7f5e217a7700 1 --2- 172.21.15.52:0/659042253 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f5e040a1150 0x7f5e040a3600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c005950 tx=0x7f5e0c00b410 comp rx=0 tx=0).stop 2023-11-27T15:16:09.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.712+0000 7f5e217a7700 1 -- 172.21.15.52:0/659042253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e1c105790 msgr2=0x7f5e1c1988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:09.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.712+0000 7f5e217a7700 1 --2- 172.21.15.52:0/659042253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e1c105790 0x7f5e1c1988a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000b530 tx=0x7f5e1000b840 comp rx=0 tx=0).stop 2023-11-27T15:16:09.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.712+0000 7f5e217a7700 1 -- 172.21.15.52:0/659042253 shutdown_connections 2023-11-27T15:16:09.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.712+0000 7f5e217a7700 1 --2- 172.21.15.52:0/659042253 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f5e040a1150 0x7f5e040a3600 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:09.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.712+0000 7f5e217a7700 1 --2- 172.21.15.52:0/659042253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e1c069d90 0x7f5e1c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:09.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.712+0000 7f5e217a7700 1 --2- 172.21.15.52:0/659042253 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e1c1080b0 0x7f5e1c19f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:09.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.712+0000 7f5e217a7700 1 --2- 172.21.15.52:0/659042253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e1c105790 0x7f5e1c1988a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:09.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.712+0000 7f5e217a7700 1 -- 172.21.15.52:0/659042253 >> 172.21.15.52:0/659042253 conn(0x7f5e1c0fa7c0 msgr2=0x7f5e1c1063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:09.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.713+0000 7f5e217a7700 1 -- 172.21.15.52:0/659042253 shutdown_connections 2023-11-27T15:16:09.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.713+0000 7f5e217a7700 1 -- 172.21.15.52:0/659042253 wait complete. 2023-11-27T15:16:09.729 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:16:09.860 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.854+0000 7ff2f86a4700 1 -- 172.21.15.52:0/1192298432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2f0100290 msgr2=0x7ff2f01006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:09.861 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.854+0000 7ff2f86a4700 1 --2- 172.21.15.52:0/1192298432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2f0100290 0x7ff2f01006a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec009990 tx=0x7ff2ec009ca0 comp rx=0 tx=0).stop 2023-11-27T15:16:09.861 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.855+0000 7ff2f86a4700 1 -- 172.21.15.52:0/1192298432 shutdown_connections 2023-11-27T15:16:09.861 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.855+0000 7ff2f86a4700 1 --2- 172.21.15.52:0/1192298432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2f0101e10 0x7ff2f01086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:09.861 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.855+0000 7ff2f86a4700 1 --2- 172.21.15.52:0/1192298432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2f0100290 0x7ff2f01006a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:09.861 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.855+0000 7ff2f86a4700 1 --2- 172.21.15.52:0/1192298432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2f0101480 0x7ff2f01018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:09.861 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.855+0000 7ff2f86a4700 1 -- 172.21.15.52:0/1192298432 >> 172.21.15.52:0/1192298432 conn(0x7ff2f00fb870 msgr2=0x7ff2f00fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:09.861 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.855+0000 7ff2f86a4700 1 -- 172.21.15.52:0/1192298432 shutdown_connections 2023-11-27T15:16:09.861 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.855+0000 7ff2f86a4700 1 -- 172.21.15.52:0/1192298432 wait complete. 2023-11-27T15:16:09.861 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.855+0000 7ff2f86a4700 1 Processor -- start 2023-11-27T15:16:09.861 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.855+0000 7ff2f86a4700 1 -- start start 2023-11-27T15:16:09.861 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.855+0000 7ff2f86a4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2f0100290 0x7ff2f00748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.855+0000 7ff2f86a4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2f0101480 0x7ff2f0072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.855+0000 7ff2f86a4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2f0101e10 0x7ff2f0073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.855+0000 7ff2f86a4700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f0073bc0 con 0x7ff2f0100290 2023-11-27T15:16:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.855+0000 7ff2f86a4700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f0073d30 con 0x7ff2f0101480 2023-11-27T15:16:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.855+0000 7ff2f86a4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f019fff0 con 0x7ff2f0101e10 2023-11-27T15:16:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.855+0000 7ff2f6440700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2f0100290 0x7ff2f00748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.856+0000 7ff2f6440700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2f0100290 0x7ff2f00748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50044/0 (socket says 172.21.15.52:50044) 2023-11-27T15:16:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.856+0000 7ff2f6440700 1 -- 172.21.15.52:0/661295469 learned_addr learned my addr 172.21.15.52:0/661295469 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.856+0000 7ff2f5c3f700 1 --2- 172.21.15.52:0/661295469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2f0101480 0x7ff2f0072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.856+0000 7ff2f6c41700 1 --2- 172.21.15.52:0/661295469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2f0101e10 0x7ff2f0073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.856+0000 7ff2f6440700 1 -- 172.21.15.52:0/661295469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2f0101e10 msgr2=0x7ff2f0073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:09.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.856+0000 7ff2f6440700 1 --2- 172.21.15.52:0/661295469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2f0101e10 0x7ff2f0073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:09.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.856+0000 7ff2f6440700 1 -- 172.21.15.52:0/661295469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2f0101480 msgr2=0x7ff2f0072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:09.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.856+0000 7ff2f6440700 1 --2- 172.21.15.52:0/661295469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2f0101480 0x7ff2f0072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:09.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.856+0000 7ff2f6440700 1 -- 172.21.15.52:0/661295469 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ec0095f0 con 0x7ff2f0100290 2023-11-27T15:16:09.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.856+0000 7ff2f6440700 1 --2- 172.21.15.52:0/661295469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2f0100290 0x7ff2f00748a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec0098e0 tx=0x7ff2ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:09.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.856+0000 7ff2e77fe700 1 -- 172.21.15.52:0/661295469 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec01d070 con 0x7ff2f0100290 2023-11-27T15:16:09.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.856+0000 7ff2e77fe700 1 -- 172.21.15.52:0/661295469 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2ec0054f0 con 0x7ff2f0100290 2023-11-27T15:16:09.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.856+0000 7ff2f86a4700 1 -- 172.21.15.52:0/661295469 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f010bc40 con 0x7ff2f0100290 2023-11-27T15:16:09.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.856+0000 7ff2e77fe700 1 -- 172.21.15.52:0/661295469 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec004ca0 con 0x7ff2f0100290 2023-11-27T15:16:09.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.856+0000 7ff2f86a4700 1 -- 172.21.15.52:0/661295469 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f010bab0 con 0x7ff2f0100290 2023-11-27T15:16:09.864 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.857+0000 7ff2f86a4700 1 -- 172.21.15.52:0/661295469 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2f01071c0 con 0x7ff2f0100290 2023-11-27T15:16:09.864 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.858+0000 7ff2e77fe700 1 -- 172.21.15.52:0/661295469 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff2ec00fb90 con 0x7ff2f0100290 2023-11-27T15:16:09.864 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.859+0000 7ff2e77fe700 1 --2- 172.21.15.52:0/661295469 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff2dc0a11d0 0x7ff2dc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:09.864 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.859+0000 7ff2f5c3f700 1 --2- 172.21.15.52:0/661295469 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff2dc0a11d0 0x7ff2dc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:09.864 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.859+0000 7ff2e77fe700 1 -- 172.21.15.52:0/661295469 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff2ec0cdae0 con 0x7ff2f0100290 2023-11-27T15:16:09.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.859+0000 7ff2f5c3f700 1 --2- 172.21.15.52:0/661295469 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff2dc0a11d0 0x7ff2dc0a3680 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0009c60 tx=0x7ff2e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:09.867 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:09.863+0000 7ff2e77fe700 1 -- 172.21.15.52:0/661295469 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff2ec05a730 con 0x7ff2f0100290 2023-11-27T15:16:10.088 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.084+0000 7ff2f86a4700 1 -- 172.21.15.52:0/661295469 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff2f002cfe0 con 0x7ff2f0100290 2023-11-27T15:16:10.089 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.085+0000 7ff2e77fe700 1 -- 172.21.15.52:0/661295469 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22143 (secure 0 0 0) 0x7ff2ec097370 con 0x7ff2f0100290 2023-11-27T15:16:10.090 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:16:10.090 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":82,"fsid":"c6c7df88-8d36-11ee-95a2-87774f69a715","created":"2023-11-27T15:09:03.333345+0000","modified":"2023-11-27T15:16:09.319166+0000","last_up_change":"2023-11-27T15:15:27.656134+0000","last_in_change":"2023-11-27T15:15:10.671153+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:12:14.177523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:16:07.842251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:16:08.728520+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58d69ede-c293-49f5-abbe-8187793f2b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6803","nonce":608216415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6805","nonce":608216415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6809","nonce":608216415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6807","nonce":608216415}]},"public_addr":"172.21.15.52:6803/608216415","cluster_addr":"172.21.15.52:6805/608216415","heartbeat_back_addr":"172.21.15.52:6809/608216415","heartbeat_front_addr":"172.21.15.52:6807/608216415","state":["exists","up"]},{"osd":1,"uuid":"d6a46b91-adc8-45ef-80f5-566ae400911f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6801","nonce":2538384087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6803","nonce":2538384087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6807","nonce":2538384087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6805","nonce":2538384087}]},"public_addr":"172.21.15.102:6801/2538384087","cluster_addr":"172.21.15.102:6803/2538384087","heartbeat_back_addr":"172.21.15.102:6807/2538384087","heartbeat_front_addr":"172.21.15.102:6805/2538384087","state":["exists","up"]},{"osd":2,"uuid":"846ba125-11f8-46b2-8941-bd5e1947cc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6801","nonce":4258038221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6803","nonce":4258038221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6807","nonce":4258038221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6805","nonce":4258038221}]},"public_addr":"172.21.15.150:6801/4258038221","cluster_addr":"172.21.15.150:6803/4258038221","heartbeat_back_addr":"172.21.15.150:6807/4258038221","heartbeat_front_addr":"172.21.15.150:6805/4258038221","state":["exists","up"]},{"osd":3,"uuid":"950fcd67-1057-4288-a389-714f1ec94261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6811","nonce":2622100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6813","nonce":2622100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6817","nonce":2622100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6815","nonce":2622100752}]},"public_addr":"172.21.15.52:6811/2622100752","cluster_addr":"172.21.15.52:6813/2622100752","heartbeat_back_addr":"172.21.15.52:6817/2622100752","heartbeat_front_addr":"172.21.15.52:6815/2622100752","state":["exists","up"]},{"osd":4,"uuid":"1c96a676-cea9-4deb-bad1-f7fd240c3d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6809","nonce":1546794763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6811","nonce":1546794763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6815","nonce":1546794763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6813","nonce":1546794763}]},"public_addr":"172.21.15.102:6809/1546794763","cluster_addr":"172.21.15.102:6811/1546794763","heartbeat_back_addr":"172.21.15.102:6815/1546794763","heartbeat_front_addr":"172.21.15.102:6813/1546794763","state":["exists","up"]},{"osd":5,"uuid":"38a75af1-fd4d-433e-b579-0609109c851c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6809","nonce":461511375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6811","nonce":461511375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6815","nonce":461511375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6813","nonce":461511375}]},"public_addr":"172.21.15.150:6809/461511375","cluster_addr":"172.21.15.150:6811/461511375","heartbeat_back_addr":"172.21.15.150:6815/461511375","heartbeat_front_addr":"172.21.15.150:6813/461511375","state":["exists","up"]},{"osd":6,"uuid":"2f0ede4a-d175-4b4a-b876-c0f442048901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6819","nonce":1002457464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6821","nonce":1002457464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6825","nonce":1002457464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6823","nonce":1002457464}]},"public_addr":"172.21.15.52:6819/1002457464","cluster_addr":"172.21.15.52:6821/1002457464","heartbeat_back_addr":"172.21.15.52:6825/1002457464","heartbeat_front_addr":"172.21.15.52:6823/1002457464","state":["exists","up"]},{"osd":7,"uuid":"6b635192-abcf-4bbf-bf78-a019b351bca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6817","nonce":3249840064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6819","nonce":3249840064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6823","nonce":3249840064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6821","nonce":3249840064}]},"public_addr":"172.21.15.102:6817/3249840064","cluster_addr":"172.21.15.102:6819/3249840064","heartbeat_back_addr":"172.21.15.102:6823/3249840064","heartbeat_front_addr":"172.21.15.102:6821/3249840064","state":["exists","up"]},{"osd":8,"uuid":"c7ca8007-7170-47bf-956c-3f00edf27e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6817","nonce":3672583574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6819","nonce":3672583574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6823","nonce":3672583574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6821","nonce":3672583574}]},"public_addr":"172.21.15.150:6817/3672583574","cluster_addr":"172.21.15.150:6819/3672583574","heartbeat_back_addr":"172.21.15.150:6823/3672583574","heartbeat_front_addr":"172.21.15.150:6821/3672583574","state":["exists","up"]},{"osd":9,"uuid":"0e252c2c-01a4-48ba-832d-067ed940fefb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6827","nonce":4287934320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6829","nonce":4287934320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6833","nonce":4287934320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6831","nonce":4287934320}]},"public_addr":"172.21.15.52:6827/4287934320","cluster_addr":"172.21.15.52:6829/4287934320","heartbeat_back_addr":"172.21.15.52:6833/4287934320","heartbeat_front_addr":"172.21.15.52:6831/4287934320","state":["exists","up"]},{"osd":10,"uuid":"bb540142-7563-44a4-b7a3-cb662714c128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6825","nonce":3056034201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6827","nonce":3056034201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6831","nonce":3056034201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6829","nonce":3056034201}]},"public_addr":"172.21.15.102:6825/3056034201","cluster_addr":"172.21.15.102:6827/3056034201","heartbeat_back_addr":"172.21.15.102:6831/3056034201","heartbeat_front_addr":"172.21.15.102:6829/3056034201","state":["exists","up"]},{"osd":11,"uuid":"3156528e-e0f8-4d44-9946-c8b6339c2bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6825","nonce":3207559739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6827","nonce":3207559739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6831","nonce":3207559739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6829","nonce":3207559739}]},"public_addr":"172.21.15.150:6825/3207559739","cluster_addr":"172.21.15.150:6827/3207559739","heartbeat_back_addr":"172.21.15.150:6831/3207559739","heartbeat_front_addr":"172.21.15.150:6829/3207559739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:26.826099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:48.331371+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:10.459738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:32.356472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:54.043170+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:15.648582+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:37.166463+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:59.240819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:20.710097+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:43.364563+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:05.279433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:26.344573+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/1392282110":"2023-11-28T15:10:03.644870+0000","172.21.15.52:0/1479993479":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6800/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/961562652":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/1290665390":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2410529742":"2023-11-28T15:09:37.118690+0000","172.21.15.52:6800/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2130037044":"2023-11-28T15:10:03.644870+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:16:10.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.087+0000 7ff2f86a4700 1 -- 172.21.15.52:0/661295469 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff2dc0a11d0 msgr2=0x7ff2dc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:10.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.087+0000 7ff2f86a4700 1 --2- 172.21.15.52:0/661295469 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff2dc0a11d0 0x7ff2dc0a3680 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0009c60 tx=0x7ff2e0009260 comp rx=0 tx=0).stop 2023-11-27T15:16:10.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.087+0000 7ff2f86a4700 1 -- 172.21.15.52:0/661295469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2f0100290 msgr2=0x7ff2f00748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:10.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.087+0000 7ff2f86a4700 1 --2- 172.21.15.52:0/661295469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2f0100290 0x7ff2f00748a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec0098e0 tx=0x7ff2ec004730 comp rx=0 tx=0).stop 2023-11-27T15:16:10.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.087+0000 7ff2f86a4700 1 -- 172.21.15.52:0/661295469 shutdown_connections 2023-11-27T15:16:10.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.087+0000 7ff2f86a4700 1 --2- 172.21.15.52:0/661295469 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff2dc0a11d0 0x7ff2dc0a3680 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:10.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.087+0000 7ff2f86a4700 1 --2- 172.21.15.52:0/661295469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2f0101e10 0x7ff2f0073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:10.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.087+0000 7ff2f86a4700 1 --2- 172.21.15.52:0/661295469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2f0100290 0x7ff2f00748a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:10.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.087+0000 7ff2f86a4700 1 --2- 172.21.15.52:0/661295469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2f0101480 0x7ff2f0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:10.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.087+0000 7ff2f86a4700 1 -- 172.21.15.52:0/661295469 >> 172.21.15.52:0/661295469 conn(0x7ff2f00fb870 msgr2=0x7ff2f00fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:10.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.088+0000 7ff2f86a4700 1 -- 172.21.15.52:0/661295469 shutdown_connections 2023-11-27T15:16:10.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.088+0000 7ff2f86a4700 1 -- 172.21.15.52:0/661295469 wait complete. 2023-11-27T15:16:10.103 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:09 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:16:10.103 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:09 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:16:10.103 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:09 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:16:10.103 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:09 smithi052 ceph-mon[99056]: Service mds.all has no related daemon already placed. 2023-11-27T15:16:10.104 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:09 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3349244992' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-11-27T15:16:10.104 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:09 smithi052 ceph-mon[99056]: osdmap e81: 12 total, 12 up, 12 in 2023-11-27T15:16:10.104 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:09 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:16:10.104 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:09 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3112629778' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-27T15:16:10.104 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:09 smithi052 ceph-mon[99056]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-27T15:16:10.104 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:09 smithi052 ceph-mon[99056]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-11-27T15:16:10.104 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:09 smithi052 ceph-mon[99056]: osdmap e82: 12 total, 12 up, 12 in 2023-11-27T15:16:10.104 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:09 smithi052 ceph-mon[99056]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-11-27T15:16:10.104 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:09 smithi052 ceph-mon[99056]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-11-27T15:16:10.104 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:09 smithi052 ceph-mon[99056]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-11-27T15:16:10.104 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:09 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/659042253' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:16:10.104 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:09 smithi052 ceph-mon[99056]: pgmap v255: 97 pgs: 24 creating+peering, 10 active+clean, 63 unknown; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-11-27T15:16:10.106 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-27T15:16:10.249 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.245+0000 7f1c0fe86700 1 -- 172.21.15.52:0/3543209457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c08103700 msgr2=0x7f1c08103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:10.249 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.245+0000 7f1c0fe86700 1 --2- 172.21.15.52:0/3543209457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c08103700 0x7f1c08103b50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8009990 tx=0x7f1bf8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:16:10.249 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.245+0000 7f1c0fe86700 1 -- 172.21.15.52:0/3543209457 shutdown_connections 2023-11-27T15:16:10.250 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.245+0000 7f1c0fe86700 1 --2- 172.21.15.52:0/3543209457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c08102510 0x7f1c08102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:10.250 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.245+0000 7f1c0fe86700 1 --2- 172.21.15.52:0/3543209457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c08104090 0x7f1c0810a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:10.250 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.245+0000 7f1c0fe86700 1 --2- 172.21.15.52:0/3543209457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c08103700 0x7f1c08103b50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:10.250 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.245+0000 7f1c0fe86700 1 -- 172.21.15.52:0/3543209457 >> 172.21.15.52:0/3543209457 conn(0x7f1c080fdb30 msgr2=0x7f1c080fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:10.250 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.245+0000 7f1c0fe86700 1 -- 172.21.15.52:0/3543209457 shutdown_connections 2023-11-27T15:16:10.250 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.245+0000 7f1c0fe86700 1 -- 172.21.15.52:0/3543209457 wait complete. 2023-11-27T15:16:10.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.245+0000 7f1c0fe86700 1 Processor -- start 2023-11-27T15:16:10.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.246+0000 7f1c0fe86700 1 -- start start 2023-11-27T15:16:10.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.246+0000 7f1c0fe86700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c08102510 0x7f1c08198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:10.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.246+0000 7f1c0fe86700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c08103700 0x7f1c08198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:10.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.246+0000 7f1c0fe86700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c08104090 0x7f1c0819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:10.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.246+0000 7f1c0fe86700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0819fdd0 con 0x7f1c08103700 2023-11-27T15:16:10.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.246+0000 7f1c0fe86700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0819ff10 con 0x7f1c08104090 2023-11-27T15:16:10.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.246+0000 7f1c0fe86700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c081a00a0 con 0x7f1c08102510 2023-11-27T15:16:10.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.246+0000 7f1c0d421700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c08103700 0x7f1c08198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:10.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.246+0000 7f1c0d421700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c08103700 0x7f1c08198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50060/0 (socket says 172.21.15.52:50060) 2023-11-27T15:16:10.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.246+0000 7f1c0d421700 1 -- 172.21.15.52:0/2205237859 learned_addr learned my addr 172.21.15.52:0/2205237859 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:10.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.246+0000 7f1c0dc22700 1 --2- 172.21.15.52:0/2205237859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c08102510 0x7f1c08198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:10.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.246+0000 7f1c0e423700 1 --2- 172.21.15.52:0/2205237859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c08104090 0x7f1c0819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:10.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.246+0000 7f1c0d421700 1 -- 172.21.15.52:0/2205237859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c08102510 msgr2=0x7f1c08198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:10.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.246+0000 7f1c0d421700 1 --2- 172.21.15.52:0/2205237859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c08102510 0x7f1c08198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:10.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.246+0000 7f1c0d421700 1 -- 172.21.15.52:0/2205237859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c08104090 msgr2=0x7f1c0819f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:10.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.246+0000 7f1c0d421700 1 --2- 172.21.15.52:0/2205237859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c08104090 0x7f1c0819f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:10.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.246+0000 7f1c0d421700 1 -- 172.21.15.52:0/2205237859 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bf80095f0 con 0x7f1c08103700 2023-11-27T15:16:10.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.247+0000 7f1c0d421700 1 --2- 172.21.15.52:0/2205237859 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c08103700 0x7f1c08198790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8005770 tx=0x7f1bf80057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:10.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.247+0000 7f1bfeffd700 1 -- 172.21.15.52:0/2205237859 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf801d070 con 0x7f1c08103700 2023-11-27T15:16:10.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.247+0000 7f1bfeffd700 1 -- 172.21.15.52:0/2205237859 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bf8004060 con 0x7f1c08103700 2023-11-27T15:16:10.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.247+0000 7f1c0fe86700 1 -- 172.21.15.52:0/2205237859 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c0810de80 con 0x7f1c08103700 2023-11-27T15:16:10.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.247+0000 7f1bfeffd700 1 -- 172.21.15.52:0/2205237859 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8022d70 con 0x7f1c08103700 2023-11-27T15:16:10.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.247+0000 7f1c0fe86700 1 -- 172.21.15.52:0/2205237859 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c0810dcf0 con 0x7f1c08103700 2023-11-27T15:16:10.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.248+0000 7f1c0fe86700 1 -- 172.21.15.52:0/2205237859 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c081a0640 con 0x7f1c08103700 2023-11-27T15:16:10.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.250+0000 7f1bfeffd700 1 -- 172.21.15.52:0/2205237859 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1bf8022470 con 0x7f1c08103700 2023-11-27T15:16:10.255 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.250+0000 7f1bfeffd700 1 --2- 172.21.15.52:0/2205237859 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f1bf40a11d0 0x7f1bf40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:10.255 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.251+0000 7f1bfeffd700 1 -- 172.21.15.52:0/2205237859 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1bf80ce450 con 0x7f1c08103700 2023-11-27T15:16:10.255 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.251+0000 7f1c0dc22700 1 --2- 172.21.15.52:0/2205237859 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f1bf40a11d0 0x7f1bf40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:10.255 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.251+0000 7f1c0dc22700 1 --2- 172.21.15.52:0/2205237859 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f1bf40a11d0 0x7f1bf40a3680 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1c08103560 tx=0x7f1c04007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:10.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.254+0000 7f1bfeffd700 1 -- 172.21.15.52:0/2205237859 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1bf8097ce0 con 0x7f1c08103700 2023-11-27T15:16:10.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:09 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:16:10.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:09 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:16:10.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:09 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:16:10.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:09 smithi150 ceph-mon[100559]: Service mds.all has no related daemon already placed. 2023-11-27T15:16:10.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:09 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3349244992' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-11-27T15:16:10.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:09 smithi150 ceph-mon[100559]: osdmap e81: 12 total, 12 up, 12 in 2023-11-27T15:16:10.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:09 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:16:10.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:09 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3112629778' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-27T15:16:10.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:09 smithi150 ceph-mon[100559]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-27T15:16:10.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:09 smithi150 ceph-mon[100559]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-11-27T15:16:10.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:09 smithi150 ceph-mon[100559]: osdmap e82: 12 total, 12 up, 12 in 2023-11-27T15:16:10.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:09 smithi150 ceph-mon[100559]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-11-27T15:16:10.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:09 smithi150 ceph-mon[100559]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-11-27T15:16:10.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:09 smithi150 ceph-mon[100559]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-11-27T15:16:10.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:09 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/659042253' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:16:10.280 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:09 smithi150 ceph-mon[100559]: pgmap v255: 97 pgs: 24 creating+peering, 10 active+clean, 63 unknown; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-11-27T15:16:10.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:09 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:16:10.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:09 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:16:10.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:09 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:16:10.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:09 smithi102 ceph-mon[100296]: Service mds.all has no related daemon already placed. 2023-11-27T15:16:10.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:09 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3349244992' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-11-27T15:16:10.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:09 smithi102 ceph-mon[100296]: osdmap e81: 12 total, 12 up, 12 in 2023-11-27T15:16:10.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:09 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:16:10.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:09 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3112629778' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-27T15:16:10.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:09 smithi102 ceph-mon[100296]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-27T15:16:10.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:09 smithi102 ceph-mon[100296]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-11-27T15:16:10.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:09 smithi102 ceph-mon[100296]: osdmap e82: 12 total, 12 up, 12 in 2023-11-27T15:16:10.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:09 smithi102 ceph-mon[100296]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-11-27T15:16:10.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:09 smithi102 ceph-mon[100296]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-11-27T15:16:10.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:09 smithi102 ceph-mon[100296]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-11-27T15:16:10.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:09 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/659042253' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:16:10.329 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:09 smithi102 ceph-mon[100296]: pgmap v255: 97 pgs: 24 creating+peering, 10 active+clean, 63 unknown; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-11-27T15:16:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:10.510+0000 7f1c0fe86700 1 -- 172.21.15.52:0/2205237859 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1c081093c0 con 0x7f1c08103700 2023-11-27T15:16:11.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:10 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/661295469' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:16:11.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:10 smithi150 ceph-mon[100559]: mds.? [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] up:active 2023-11-27T15:16:11.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:10 smithi150 ceph-mon[100559]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-27T15:16:11.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:10 smithi150 ceph-mon[100559]: osdmap e83: 12 total, 12 up, 12 in 2023-11-27T15:16:11.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:10 smithi150 ceph-mon[100559]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-27T15:16:11.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:10 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2205237859' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-11-27T15:16:11.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:10 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/661295469' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:16:11.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:10 smithi102 ceph-mon[100296]: mds.? [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] up:active 2023-11-27T15:16:11.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:10 smithi102 ceph-mon[100296]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-27T15:16:11.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:10 smithi102 ceph-mon[100296]: osdmap e83: 12 total, 12 up, 12 in 2023-11-27T15:16:11.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:10 smithi102 ceph-mon[100296]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-27T15:16:11.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:10 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2205237859' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-11-27T15:16:11.341 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.337+0000 7f1bfeffd700 1 -- 172.21.15.52:0/2205237859 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1bf809b300 con 0x7f1c08103700 2023-11-27T15:16:11.343 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.339+0000 7f1c0fe86700 1 -- 172.21.15.52:0/2205237859 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f1bf40a11d0 msgr2=0x7f1bf40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:11.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.339+0000 7f1c0fe86700 1 --2- 172.21.15.52:0/2205237859 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f1bf40a11d0 0x7f1bf40a3680 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1c08103560 tx=0x7f1c04007400 comp rx=0 tx=0).stop 2023-11-27T15:16:11.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.339+0000 7f1c0fe86700 1 -- 172.21.15.52:0/2205237859 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c08103700 msgr2=0x7f1c08198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:11.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.339+0000 7f1c0fe86700 1 --2- 172.21.15.52:0/2205237859 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c08103700 0x7f1c08198790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8005770 tx=0x7f1bf80057a0 comp rx=0 tx=0).stop 2023-11-27T15:16:11.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.339+0000 7f1c0fe86700 1 -- 172.21.15.52:0/2205237859 shutdown_connections 2023-11-27T15:16:11.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.339+0000 7f1c0fe86700 1 --2- 172.21.15.52:0/2205237859 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f1bf40a11d0 0x7f1bf40a3680 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:11.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.339+0000 7f1c0fe86700 1 --2- 172.21.15.52:0/2205237859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1c08104090 0x7f1c0819f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:11.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.339+0000 7f1c0fe86700 1 --2- 172.21.15.52:0/2205237859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c08102510 0x7f1c08198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:11.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.339+0000 7f1c0fe86700 1 --2- 172.21.15.52:0/2205237859 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c08103700 0x7f1c08198790 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:11.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.339+0000 7f1c0fe86700 1 -- 172.21.15.52:0/2205237859 >> 172.21.15.52:0/2205237859 conn(0x7f1c080fdb30 msgr2=0x7f1c080ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:11.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.339+0000 7f1c0fe86700 1 -- 172.21.15.52:0/2205237859 shutdown_connections 2023-11-27T15:16:11.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.339+0000 7f1c0fe86700 1 -- 172.21.15.52:0/2205237859 wait complete. 2023-11-27T15:16:11.355 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:10 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/661295469' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:16:11.355 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:10 smithi052 ceph-mon[99056]: mds.? [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] up:active 2023-11-27T15:16:11.355 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:10 smithi052 ceph-mon[99056]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-27T15:16:11.355 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:10 smithi052 ceph-mon[99056]: osdmap e83: 12 total, 12 up, 12 in 2023-11-27T15:16:11.355 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:10 smithi052 ceph-mon[99056]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-27T15:16:11.355 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:10 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2205237859' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-11-27T15:16:11.356 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2023-11-27T15:16:11.356 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-11-27T15:16:11.491 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.487+0000 7efd16a5a700 1 -- 172.21.15.52:0/1367912906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd100ff1c0 msgr2=0x7efd100ff5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:11.491 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.487+0000 7efd16a5a700 1 --2- 172.21.15.52:0/1367912906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd100ff1c0 0x7efd100ff5d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efd00009990 tx=0x7efd00009ca0 comp rx=0 tx=0).stop 2023-11-27T15:16:11.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.487+0000 7efd16a5a700 1 -- 172.21.15.52:0/1367912906 shutdown_connections 2023-11-27T15:16:11.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.487+0000 7efd16a5a700 1 --2- 172.21.15.52:0/1367912906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd10100d40 0x7efd10107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:11.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.487+0000 7efd16a5a700 1 --2- 172.21.15.52:0/1367912906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd100ff1c0 0x7efd100ff5d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:11.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.487+0000 7efd16a5a700 1 --2- 172.21.15.52:0/1367912906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd101003b0 0x7efd10100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:11.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.487+0000 7efd16a5a700 1 -- 172.21.15.52:0/1367912906 >> 172.21.15.52:0/1367912906 conn(0x7efd100fa7c0 msgr2=0x7efd100fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:11.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.487+0000 7efd16a5a700 1 -- 172.21.15.52:0/1367912906 shutdown_connections 2023-11-27T15:16:11.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.487+0000 7efd16a5a700 1 -- 172.21.15.52:0/1367912906 wait complete. 2023-11-27T15:16:11.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.488+0000 7efd16a5a700 1 Processor -- start 2023-11-27T15:16:11.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.489+0000 7efd16a5a700 1 -- start start 2023-11-27T15:16:11.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.489+0000 7efd16a5a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd100ff1c0 0x7efd10193ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:11.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.489+0000 7efd16a5a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd101003b0 0x7efd10194420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:11.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.489+0000 7efd16a5a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd10100d40 0x7efd1019b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:11.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.489+0000 7efd16a5a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1019ba60 con 0x7efd10100d40 2023-11-27T15:16:11.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.489+0000 7efd16a5a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1019bbd0 con 0x7efd101003b0 2023-11-27T15:16:11.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.489+0000 7efd16a5a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1019bd90 con 0x7efd100ff1c0 2023-11-27T15:16:11.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.489+0000 7efd14ff7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd10100d40 0x7efd1019b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:11.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.489+0000 7efd0ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd100ff1c0 0x7efd10193ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:11.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.489+0000 7efd14ff7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd10100d40 0x7efd1019b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50072/0 (socket says 172.21.15.52:50072) 2023-11-27T15:16:11.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.489+0000 7efd14ff7700 1 -- 172.21.15.52:0/1158013965 learned_addr learned my addr 172.21.15.52:0/1158013965 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:11.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.490+0000 7efd0f7fe700 1 --2- 172.21.15.52:0/1158013965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd101003b0 0x7efd10194420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:11.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.490+0000 7efd0ffff700 1 -- 172.21.15.52:0/1158013965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd101003b0 msgr2=0x7efd10194420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:11.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.490+0000 7efd0ffff700 1 --2- 172.21.15.52:0/1158013965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd101003b0 0x7efd10194420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:11.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.490+0000 7efd0ffff700 1 -- 172.21.15.52:0/1158013965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd10100d40 msgr2=0x7efd1019b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:11.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.490+0000 7efd0ffff700 1 --2- 172.21.15.52:0/1158013965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd10100d40 0x7efd1019b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:11.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.490+0000 7efd0ffff700 1 -- 172.21.15.52:0/1158013965 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd000095f0 con 0x7efd100ff1c0 2023-11-27T15:16:11.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.490+0000 7efd14ff7700 1 --2- 172.21.15.52:0/1158013965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd10100d40 0x7efd1019b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:16:11.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.490+0000 7efd0ffff700 1 --2- 172.21.15.52:0/1158013965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd100ff1c0 0x7efd10193ee0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efd0000b2e0 tx=0x7efd00004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:11.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.490+0000 7efd0d7fa700 1 -- 172.21.15.52:0/1158013965 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0001d070 con 0x7efd100ff1c0 2023-11-27T15:16:11.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.490+0000 7efd16a5a700 1 -- 172.21.15.52:0/1158013965 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd1010ab80 con 0x7efd100ff1c0 2023-11-27T15:16:11.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.490+0000 7efd0d7fa700 1 -- 172.21.15.52:0/1158013965 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd00005030 con 0x7efd100ff1c0 2023-11-27T15:16:11.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.490+0000 7efd0d7fa700 1 -- 172.21.15.52:0/1158013965 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd00004ad0 con 0x7efd100ff1c0 2023-11-27T15:16:11.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.490+0000 7efd16a5a700 1 -- 172.21.15.52:0/1158013965 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd1010a9f0 con 0x7efd100ff1c0 2023-11-27T15:16:11.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.491+0000 7efd16a5a700 1 -- 172.21.15.52:0/1158013965 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd10067b00 con 0x7efd100ff1c0 2023-11-27T15:16:11.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.492+0000 7efd0d7fa700 1 -- 172.21.15.52:0/1158013965 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7efd000051a0 con 0x7efd100ff1c0 2023-11-27T15:16:11.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.493+0000 7efd0d7fa700 1 --2- 172.21.15.52:0/1158013965 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7efcf80a1150 0x7efcf80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:11.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.493+0000 7efd0d7fa700 1 -- 172.21.15.52:0/1158013965 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7efd000cded0 con 0x7efd100ff1c0 2023-11-27T15:16:11.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.493+0000 7efd0f7fe700 1 --2- 172.21.15.52:0/1158013965 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7efcf80a1150 0x7efcf80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:11.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.494+0000 7efd0f7fe700 1 --2- 172.21.15.52:0/1158013965 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7efcf80a1150 0x7efcf80a3600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efd04005950 tx=0x7efd0400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:11.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.496+0000 7efd0d7fa700 1 -- 172.21.15.52:0/1158013965 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7efd00097800 con 0x7efd100ff1c0 2023-11-27T15:16:11.754 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:11.750+0000 7efd16a5a700 1 -- 172.21.15.52:0/1158013965 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7efd1019c8e0 con 0x7efd100ff1c0 2023-11-27T15:16:12.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:12 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2205237859' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-11-27T15:16:12.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:12 smithi052 ceph-mon[99056]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-27T15:16:12.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:12 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:12.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:12 smithi052 ceph-mon[99056]: pgmap v257: 97 pgs: 24 creating+peering, 38 active+clean, 35 unknown; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 2 op/s 2023-11-27T15:16:12.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:12 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:12.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:12 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1158013965' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-11-27T15:16:12.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:12 smithi052 ceph-mon[99056]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-11-27T15:16:12.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:12 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:12.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:12 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:12.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:12 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:12.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:12 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:12.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:12 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:12.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:12 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:16:12.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:12 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:12.732 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.728+0000 7efd0d7fa700 1 -- 172.21.15.52:0/1158013965 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7efd0009ae20 con 0x7efd100ff1c0 2023-11-27T15:16:12.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.730+0000 7efd16a5a700 1 -- 172.21.15.52:0/1158013965 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7efcf80a1150 msgr2=0x7efcf80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:12.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.730+0000 7efd16a5a700 1 --2- 172.21.15.52:0/1158013965 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7efcf80a1150 0x7efcf80a3600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efd04005950 tx=0x7efd0400a300 comp rx=0 tx=0).stop 2023-11-27T15:16:12.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.730+0000 7efd16a5a700 1 -- 172.21.15.52:0/1158013965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd100ff1c0 msgr2=0x7efd10193ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:12.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.730+0000 7efd16a5a700 1 --2- 172.21.15.52:0/1158013965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd100ff1c0 0x7efd10193ee0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efd0000b2e0 tx=0x7efd00004730 comp rx=0 tx=0).stop 2023-11-27T15:16:12.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.731+0000 7efd16a5a700 1 -- 172.21.15.52:0/1158013965 shutdown_connections 2023-11-27T15:16:12.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.731+0000 7efd16a5a700 1 --2- 172.21.15.52:0/1158013965 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7efcf80a1150 0x7efcf80a3600 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:12.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.731+0000 7efd16a5a700 1 --2- 172.21.15.52:0/1158013965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd10100d40 0x7efd1019b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:12.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.731+0000 7efd16a5a700 1 --2- 172.21.15.52:0/1158013965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd100ff1c0 0x7efd10193ee0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:12.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.731+0000 7efd16a5a700 1 --2- 172.21.15.52:0/1158013965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd101003b0 0x7efd10194420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:12.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.731+0000 7efd16a5a700 1 -- 172.21.15.52:0/1158013965 >> 172.21.15.52:0/1158013965 conn(0x7efd100fa7c0 msgr2=0x7efd10067e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:12.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.731+0000 7efd16a5a700 1 -- 172.21.15.52:0/1158013965 shutdown_connections 2023-11-27T15:16:12.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.731+0000 7efd16a5a700 1 -- 172.21.15.52:0/1158013965 wait complete. 2023-11-27T15:16:12.754 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-11-27T15:16:12.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:12 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2205237859' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-11-27T15:16:12.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:12 smithi150 ceph-mon[100559]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-27T15:16:12.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:12 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:12.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:12 smithi150 ceph-mon[100559]: pgmap v257: 97 pgs: 24 creating+peering, 38 active+clean, 35 unknown; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 2 op/s 2023-11-27T15:16:12.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:12 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:12.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:12 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1158013965' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-11-27T15:16:12.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:12 smithi150 ceph-mon[100559]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-11-27T15:16:12.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:12 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:12.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:12 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:12.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:12 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:12.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:12 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:12.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:12 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:12.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:12 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:16:12.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:12 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:12.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:12 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2205237859' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-11-27T15:16:12.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:12 smithi102 ceph-mon[100296]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-27T15:16:12.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:12 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:12.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:12 smithi102 ceph-mon[100296]: pgmap v257: 97 pgs: 24 creating+peering, 38 active+clean, 35 unknown; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 2 op/s 2023-11-27T15:16:12.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:12 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:12.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:12 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1158013965' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-11-27T15:16:12.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:12 smithi102 ceph-mon[100296]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-11-27T15:16:12.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:12 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:12.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:12 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:12.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:12 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:12.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:12 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:12.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:12 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:12.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:12 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:16:12.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:12 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:16:12.889 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.885+0000 7fcb6622e700 1 -- 172.21.15.52:0/1955740726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb601003b0 msgr2=0x7fcb60100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:12.889 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.885+0000 7fcb6622e700 1 --2- 172.21.15.52:0/1955740726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb601003b0 0x7fcb60100800 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcb50009990 tx=0x7fcb50009ca0 comp rx=0 tx=0).stop 2023-11-27T15:16:12.889 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.885+0000 7fcb6622e700 1 -- 172.21.15.52:0/1955740726 shutdown_connections 2023-11-27T15:16:12.889 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.885+0000 7fcb6622e700 1 --2- 172.21.15.52:0/1955740726 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb60100d40 0x7fcb60107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:12.889 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.885+0000 7fcb6622e700 1 --2- 172.21.15.52:0/1955740726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb600ff1c0 0x7fcb600ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:12.890 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.885+0000 7fcb6622e700 1 --2- 172.21.15.52:0/1955740726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb601003b0 0x7fcb60100800 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:12.890 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.885+0000 7fcb6622e700 1 -- 172.21.15.52:0/1955740726 >> 172.21.15.52:0/1955740726 conn(0x7fcb600fa7c0 msgr2=0x7fcb600fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:12.890 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.885+0000 7fcb6622e700 1 -- 172.21.15.52:0/1955740726 shutdown_connections 2023-11-27T15:16:12.890 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.885+0000 7fcb6622e700 1 -- 172.21.15.52:0/1955740726 wait complete. 2023-11-27T15:16:12.892 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.885+0000 7fcb6622e700 1 Processor -- start 2023-11-27T15:16:12.892 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.885+0000 7fcb6622e700 1 -- start start 2023-11-27T15:16:12.892 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.886+0000 7fcb6622e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb600ff1c0 0x7fcb60194f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:12.892 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.886+0000 7fcb6622e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb601003b0 0x7fcb601954b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:12.893 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.886+0000 7fcb6622e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb60100d40 0x7fcb6019c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:12.893 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.886+0000 7fcb6622e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6019caf0 con 0x7fcb601003b0 2023-11-27T15:16:12.893 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.886+0000 7fcb6622e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6019cc30 con 0x7fcb60100d40 2023-11-27T15:16:12.893 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.886+0000 7fcb6622e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6019cdc0 con 0x7fcb600ff1c0 2023-11-27T15:16:12.893 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.886+0000 7fcb5effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb601003b0 0x7fcb601954b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:12.893 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.886+0000 7fcb5effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb601003b0 0x7fcb601954b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50084/0 (socket says 172.21.15.52:50084) 2023-11-27T15:16:12.893 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.886+0000 7fcb5effd700 1 -- 172.21.15.52:0/2634423145 learned_addr learned my addr 172.21.15.52:0/2634423145 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:12.893 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.886+0000 7fcb5f7fe700 1 --2- 172.21.15.52:0/2634423145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb600ff1c0 0x7fcb60194f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:12.893 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.886+0000 7fcb5ffff700 1 --2- 172.21.15.52:0/2634423145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb60100d40 0x7fcb6019c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:12.893 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.886+0000 7fcb5effd700 1 -- 172.21.15.52:0/2634423145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb600ff1c0 msgr2=0x7fcb60194f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:12.893 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.886+0000 7fcb5effd700 1 --2- 172.21.15.52:0/2634423145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb600ff1c0 0x7fcb60194f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:12.893 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.886+0000 7fcb5effd700 1 -- 172.21.15.52:0/2634423145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb60100d40 msgr2=0x7fcb6019c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:12.894 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.886+0000 7fcb5effd700 1 --2- 172.21.15.52:0/2634423145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb60100d40 0x7fcb6019c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:12.894 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.886+0000 7fcb5effd700 1 -- 172.21.15.52:0/2634423145 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb500095f0 con 0x7fcb601003b0 2023-11-27T15:16:12.894 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.886+0000 7fcb5ffff700 1 --2- 172.21.15.52:0/2634423145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb60100d40 0x7fcb6019c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:16:12.894 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.886+0000 7fcb5effd700 1 --2- 172.21.15.52:0/2634423145 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb601003b0 0x7fcb601954b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcb500095c0 tx=0x7fcb50005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:12.894 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.886+0000 7fcb5cff9700 1 -- 172.21.15.52:0/2634423145 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5001d070 con 0x7fcb601003b0 2023-11-27T15:16:12.894 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.886+0000 7fcb5cff9700 1 -- 172.21.15.52:0/2634423145 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb50004ca0 con 0x7fcb601003b0 2023-11-27T15:16:12.894 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.886+0000 7fcb6622e700 1 -- 172.21.15.52:0/2634423145 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb6010ab30 con 0x7fcb601003b0 2023-11-27T15:16:12.894 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.886+0000 7fcb5cff9700 1 -- 172.21.15.52:0/2634423145 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50005590 con 0x7fcb601003b0 2023-11-27T15:16:12.894 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.887+0000 7fcb6622e700 1 -- 172.21.15.52:0/2634423145 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb6010a9a0 con 0x7fcb601003b0 2023-11-27T15:16:12.894 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.887+0000 7fcb6622e700 1 -- 172.21.15.52:0/2634423145 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb60106170 con 0x7fcb601003b0 2023-11-27T15:16:12.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.888+0000 7fcb5cff9700 1 -- 172.21.15.52:0/2634423145 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fcb5000fbf0 con 0x7fcb601003b0 2023-11-27T15:16:12.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.889+0000 7fcb5cff9700 1 --2- 172.21.15.52:0/2634423145 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fcb480a1150 0x7fcb480a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:12.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.889+0000 7fcb5f7fe700 1 --2- 172.21.15.52:0/2634423145 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fcb480a1150 0x7fcb480a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:12.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.889+0000 7fcb5cff9700 1 -- 172.21.15.52:0/2634423145 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcb500cdb00 con 0x7fcb601003b0 2023-11-27T15:16:12.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.889+0000 7fcb5f7fe700 1 --2- 172.21.15.52:0/2634423145 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fcb480a1150 0x7fcb480a3600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcb54005950 tx=0x7fcb540058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:12.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:12.893+0000 7fcb5cff9700 1 -- 172.21.15.52:0/2634423145 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcb5005a730 con 0x7fcb601003b0 2023-11-27T15:16:13.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.139+0000 7fcb6622e700 1 -- 172.21.15.52:0/2634423145 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fcb60061700 con 0x7fcb601003b0 2023-11-27T15:16:13.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.733+0000 7fcb5cff9700 1 -- 172.21.15.52:0/2634423145 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7fcb50097430 con 0x7fcb601003b0 2023-11-27T15:16:13.740 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.736+0000 7fcb6622e700 1 -- 172.21.15.52:0/2634423145 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fcb480a1150 msgr2=0x7fcb480a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:13.741 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.736+0000 7fcb6622e700 1 --2- 172.21.15.52:0/2634423145 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fcb480a1150 0x7fcb480a3600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcb54005950 tx=0x7fcb540058e0 comp rx=0 tx=0).stop 2023-11-27T15:16:13.741 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.736+0000 7fcb6622e700 1 -- 172.21.15.52:0/2634423145 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb601003b0 msgr2=0x7fcb601954b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:13.741 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.736+0000 7fcb6622e700 1 --2- 172.21.15.52:0/2634423145 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb601003b0 0x7fcb601954b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcb500095c0 tx=0x7fcb50005ef0 comp rx=0 tx=0).stop 2023-11-27T15:16:13.741 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.736+0000 7fcb6622e700 1 -- 172.21.15.52:0/2634423145 shutdown_connections 2023-11-27T15:16:13.741 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.736+0000 7fcb6622e700 1 --2- 172.21.15.52:0/2634423145 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fcb480a1150 0x7fcb480a3600 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:13.741 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.736+0000 7fcb6622e700 1 --2- 172.21.15.52:0/2634423145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb60100d40 0x7fcb6019c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:13.741 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.736+0000 7fcb6622e700 1 --2- 172.21.15.52:0/2634423145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb600ff1c0 0x7fcb60194f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:13.741 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.736+0000 7fcb6622e700 1 --2- 172.21.15.52:0/2634423145 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb601003b0 0x7fcb601954b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:13.741 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.736+0000 7fcb6622e700 1 -- 172.21.15.52:0/2634423145 >> 172.21.15.52:0/2634423145 conn(0x7fcb600fa7c0 msgr2=0x7fcb600fcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:13.741 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.736+0000 7fcb6622e700 1 -- 172.21.15.52:0/2634423145 shutdown_connections 2023-11-27T15:16:13.741 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.736+0000 7fcb6622e700 1 -- 172.21.15.52:0/2634423145 wait complete. 2023-11-27T15:16:13.760 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-11-27T15:16:13.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.889+0000 7f6e72b13700 1 -- 172.21.15.52:0/1930458564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e6c100780 msgr2=0x7f6e6c100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:13.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.889+0000 7f6e72b13700 1 --2- 172.21.15.52:0/1930458564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e6c100780 0x7f6e6c100b90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009990 tx=0x7f6e60009ca0 comp rx=0 tx=0).stop 2023-11-27T15:16:13.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.889+0000 7f6e72b13700 1 -- 172.21.15.52:0/1930458564 shutdown_connections 2023-11-27T15:16:13.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.889+0000 7f6e72b13700 1 --2- 172.21.15.52:0/1930458564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e6c0ffd80 0x7f6e6c107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:13.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.889+0000 7f6e72b13700 1 --2- 172.21.15.52:0/1930458564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e6c100780 0x7f6e6c100b90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:13.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.889+0000 7f6e72b13700 1 --2- 172.21.15.52:0/1930458564 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e6c0ff3d0 0x7f6e6c0ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:13.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.889+0000 7f6e72b13700 1 -- 172.21.15.52:0/1930458564 >> 172.21.15.52:0/1930458564 conn(0x7f6e6c0f9f30 msgr2=0x7f6e6c0fc380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:13.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.889+0000 7f6e72b13700 1 -- 172.21.15.52:0/1930458564 shutdown_connections 2023-11-27T15:16:13.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.889+0000 7f6e72b13700 1 -- 172.21.15.52:0/1930458564 wait complete. 2023-11-27T15:16:13.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.890+0000 7f6e72b13700 1 Processor -- start 2023-11-27T15:16:13.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.890+0000 7f6e72b13700 1 -- start start 2023-11-27T15:16:13.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.890+0000 7f6e72b13700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e6c0ff3d0 0x7f6e6c0fee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:13.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.890+0000 7f6e72b13700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e6c0ffd80 0x7f6e6c19acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:13.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.890+0000 7f6e72b13700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e6c100780 0x7f6e6c19d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:13.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.890+0000 7f6e72b13700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c19db10 con 0x7f6e6c100780 2023-11-27T15:16:13.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.890+0000 7f6e72b13700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c19dc80 con 0x7f6e6c0ffd80 2023-11-27T15:16:13.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.890+0000 7f6e72b13700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c19de40 con 0x7f6e6c0ff3d0 2023-11-27T15:16:13.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.890+0000 7f6e710b0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e6c100780 0x7f6e6c19d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:13.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.890+0000 7f6e710b0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e6c100780 0x7f6e6c19d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50100/0 (socket says 172.21.15.52:50100) 2023-11-27T15:16:13.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.890+0000 7f6e6bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e6c0ffd80 0x7f6e6c19acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:13.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.890+0000 7f6e710b0700 1 -- 172.21.15.52:0/3177794449 learned_addr learned my addr 172.21.15.52:0/3177794449 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:13.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.890+0000 7f6e708af700 1 --2- 172.21.15.52:0/3177794449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e6c0ff3d0 0x7f6e6c0fee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:13.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.890+0000 7f6e710b0700 1 -- 172.21.15.52:0/3177794449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e6c0ff3d0 msgr2=0x7f6e6c0fee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:13.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.890+0000 7f6e710b0700 1 --2- 172.21.15.52:0/3177794449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e6c0ff3d0 0x7f6e6c0fee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:13.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.890+0000 7f6e710b0700 1 -- 172.21.15.52:0/3177794449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e6c0ffd80 msgr2=0x7f6e6c19acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:13.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.890+0000 7f6e710b0700 1 --2- 172.21.15.52:0/3177794449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e6c0ffd80 0x7f6e6c19acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:13.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.890+0000 7f6e710b0700 1 -- 172.21.15.52:0/3177794449 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e600095f0 con 0x7f6e6c100780 2023-11-27T15:16:13.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.890+0000 7f6e710b0700 1 --2- 172.21.15.52:0/3177794449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e6c100780 0x7f6e6c19d340 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6e64007ee0 tx=0x7f6e6400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:13.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.891+0000 7f6e69ffb700 1 -- 172.21.15.52:0/3177794449 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6400dd20 con 0x7f6e6c100780 2023-11-27T15:16:13.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.891+0000 7f6e69ffb700 1 -- 172.21.15.52:0/3177794449 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e6400f040 con 0x7f6e6c100780 2023-11-27T15:16:13.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.891+0000 7f6e72b13700 1 -- 172.21.15.52:0/3177794449 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e6c10ab20 con 0x7f6e6c100780 2023-11-27T15:16:13.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.891+0000 7f6e69ffb700 1 -- 172.21.15.52:0/3177794449 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e640048a0 con 0x7f6e6c100780 2023-11-27T15:16:13.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.891+0000 7f6e72b13700 1 -- 172.21.15.52:0/3177794449 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e6c10a990 con 0x7f6e6c100780 2023-11-27T15:16:13.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.892+0000 7f6e72b13700 1 -- 172.21.15.52:0/3177794449 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e6c19e4e0 con 0x7f6e6c100780 2023-11-27T15:16:13.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.893+0000 7f6e69ffb700 1 -- 172.21.15.52:0/3177794449 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6e64003a00 con 0x7f6e6c100780 2023-11-27T15:16:13.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.893+0000 7f6e69ffb700 1 --2- 172.21.15.52:0/3177794449 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6e5c0a1150 0x7f6e5c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:13.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.893+0000 7f6e69ffb700 1 -- 172.21.15.52:0/3177794449 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6e640cc020 con 0x7f6e6c100780 2023-11-27T15:16:13.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.894+0000 7f6e708af700 1 --2- 172.21.15.52:0/3177794449 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6e5c0a1150 0x7f6e5c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:13.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.894+0000 7f6e708af700 1 --2- 172.21.15.52:0/3177794449 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6e5c0a1150 0x7f6e5c0a3600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6e60006010 tx=0x7f6e60005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:13.902 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:13.898+0000 7f6e69ffb700 1 -- 172.21.15.52:0/3177794449 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6e6404e730 con 0x7f6e6c100780 2023-11-27T15:16:14.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:13 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:16:14.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:13 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:16:14.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:13 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:16:14.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:13 smithi150 ceph-mon[100559]: Service mds.all has no related daemon already placed. 2023-11-27T15:16:14.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:13 smithi150 ceph-mon[100559]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-11-27T15:16:14.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:13 smithi150 ceph-mon[100559]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-11-27T15:16:14.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:13 smithi150 ceph-mon[100559]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-11-27T15:16:14.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:13 smithi150 ceph-mon[100559]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-27T15:16:14.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:13 smithi150 ceph-mon[100559]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2023-11-27T15:16:14.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:13 smithi150 ceph-mon[100559]: daemon mds.h is now active in filesystem cephfs as rank 1 2023-11-27T15:16:14.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:13 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2634423145' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-11-27T15:16:14.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:13 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:16:14.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:13 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:16:14.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:13 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:16:14.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:13 smithi102 ceph-mon[100296]: Service mds.all has no related daemon already placed. 2023-11-27T15:16:14.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:13 smithi102 ceph-mon[100296]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-11-27T15:16:14.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:13 smithi102 ceph-mon[100296]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-11-27T15:16:14.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:13 smithi102 ceph-mon[100296]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-11-27T15:16:14.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:13 smithi102 ceph-mon[100296]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-27T15:16:14.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:13 smithi102 ceph-mon[100296]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2023-11-27T15:16:14.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:13 smithi102 ceph-mon[100296]: daemon mds.h is now active in filesystem cephfs as rank 1 2023-11-27T15:16:14.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:13 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2634423145' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-11-27T15:16:14.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:13 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:16:14.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:13 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:16:14.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:13 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:16:14.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:13 smithi052 ceph-mon[99056]: Service mds.all has no related daemon already placed. 2023-11-27T15:16:14.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:13 smithi052 ceph-mon[99056]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-11-27T15:16:14.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:13 smithi052 ceph-mon[99056]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-11-27T15:16:14.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:13 smithi052 ceph-mon[99056]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-11-27T15:16:14.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:13 smithi052 ceph-mon[99056]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-27T15:16:14.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:13 smithi052 ceph-mon[99056]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2023-11-27T15:16:14.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:13 smithi052 ceph-mon[99056]: daemon mds.h is now active in filesystem cephfs as rank 1 2023-11-27T15:16:14.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:13 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2634423145' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-11-27T15:16:14.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.146+0000 7f6e72b13700 1 -- 172.21.15.52:0/3177794449 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f6e6c1060c0 con 0x7f6e6c100780 2023-11-27T15:16:14.746 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.741+0000 7f6e69ffb700 1 -- 172.21.15.52:0/3177794449 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7f6e640958c0 con 0x7f6e6c100780 2023-11-27T15:16:14.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.744+0000 7f6e72b13700 1 -- 172.21.15.52:0/3177794449 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6e5c0a1150 msgr2=0x7f6e5c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:14.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.744+0000 7f6e72b13700 1 --2- 172.21.15.52:0/3177794449 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6e5c0a1150 0x7f6e5c0a3600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6e60006010 tx=0x7f6e60005960 comp rx=0 tx=0).stop 2023-11-27T15:16:14.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.744+0000 7f6e72b13700 1 -- 172.21.15.52:0/3177794449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e6c100780 msgr2=0x7f6e6c19d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:14.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.744+0000 7f6e72b13700 1 --2- 172.21.15.52:0/3177794449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e6c100780 0x7f6e6c19d340 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6e64007ee0 tx=0x7f6e6400d3b0 comp rx=0 tx=0).stop 2023-11-27T15:16:14.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.744+0000 7f6e72b13700 1 -- 172.21.15.52:0/3177794449 shutdown_connections 2023-11-27T15:16:14.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.744+0000 7f6e72b13700 1 --2- 172.21.15.52:0/3177794449 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6e5c0a1150 0x7f6e5c0a3600 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:14.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.744+0000 7f6e72b13700 1 --2- 172.21.15.52:0/3177794449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e6c100780 0x7f6e6c19d340 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:14.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.744+0000 7f6e72b13700 1 --2- 172.21.15.52:0/3177794449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e6c0ff3d0 0x7f6e6c0fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:14.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.744+0000 7f6e72b13700 1 --2- 172.21.15.52:0/3177794449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e6c0ffd80 0x7f6e6c19acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:14.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.744+0000 7f6e72b13700 1 -- 172.21.15.52:0/3177794449 >> 172.21.15.52:0/3177794449 conn(0x7f6e6c0f9f30 msgr2=0x7f6e6c0fc150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:14.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.745+0000 7f6e72b13700 1 -- 172.21.15.52:0/3177794449 shutdown_connections 2023-11-27T15:16:14.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.745+0000 7f6e72b13700 1 -- 172.21.15.52:0/3177794449 wait complete. 2023-11-27T15:16:14.764 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2023-11-27T15:16:14.764 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-11-27T15:16:14.903 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.896+0000 7f711640e700 1 -- 172.21.15.52:0/2934535041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71101036c0 msgr2=0x7f7110105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.896+0000 7f711640e700 1 --2- 172.21.15.52:0/2934535041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71101036c0 0x7f7110105aa0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7104009990 tx=0x7f7104009ca0 comp rx=0 tx=0).stop 2023-11-27T15:16:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.897+0000 7f711640e700 1 -- 172.21.15.52:0/2934535041 shutdown_connections 2023-11-27T15:16:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.897+0000 7f711640e700 1 --2- 172.21.15.52:0/2934535041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7110105fe0 0x7f71101083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.897+0000 7f711640e700 1 --2- 172.21.15.52:0/2934535041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7110100da0 0x7f7110103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.897+0000 7f711640e700 1 --2- 172.21.15.52:0/2934535041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71101036c0 0x7f7110105aa0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.897+0000 7f711640e700 1 -- 172.21.15.52:0/2934535041 >> 172.21.15.52:0/2934535041 conn(0x7f71100fa7c0 msgr2=0x7f71100fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.897+0000 7f711640e700 1 -- 172.21.15.52:0/2934535041 shutdown_connections 2023-11-27T15:16:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.897+0000 7f711640e700 1 -- 172.21.15.52:0/2934535041 wait complete. 2023-11-27T15:16:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.897+0000 7f711640e700 1 Processor -- start 2023-11-27T15:16:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.897+0000 7f711640e700 1 -- start start 2023-11-27T15:16:14.905 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.898+0000 7f711640e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7110100da0 0x7f7110196070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:14.905 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.898+0000 7f711640e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71101036c0 0x7f71101965b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:14.905 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.898+0000 7f711640e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7110105fe0 0x7f711019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:14.905 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.898+0000 7f711640e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711019dbf0 con 0x7f7110105fe0 2023-11-27T15:16:14.905 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.898+0000 7f711640e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711019dd30 con 0x7f71101036c0 2023-11-27T15:16:14.905 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.898+0000 7f711640e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711019dec0 con 0x7f7110100da0 2023-11-27T15:16:14.905 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.898+0000 7f71149ab700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7110105fe0 0x7f711019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:14.905 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.898+0000 7f71149ab700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7110105fe0 0x7f711019d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50114/0 (socket says 172.21.15.52:50114) 2023-11-27T15:16:14.905 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.898+0000 7f71149ab700 1 -- 172.21.15.52:0/3990417161 learned_addr learned my addr 172.21.15.52:0/3990417161 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:14.905 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.898+0000 7f710ffff700 1 --2- 172.21.15.52:0/3990417161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7110100da0 0x7f7110196070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:14.905 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.898+0000 7f710f7fe700 1 --2- 172.21.15.52:0/3990417161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71101036c0 0x7f71101965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:14.905 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.898+0000 7f71149ab700 1 -- 172.21.15.52:0/3990417161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7110100da0 msgr2=0x7f7110196070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.898+0000 7f71149ab700 1 --2- 172.21.15.52:0/3990417161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7110100da0 0x7f7110196070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.898+0000 7f71149ab700 1 -- 172.21.15.52:0/3990417161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71101036c0 msgr2=0x7f71101965b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.898+0000 7f71149ab700 1 --2- 172.21.15.52:0/3990417161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71101036c0 0x7f71101965b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.898+0000 7f71149ab700 1 -- 172.21.15.52:0/3990417161 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71040095f0 con 0x7f7110105fe0 2023-11-27T15:16:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.898+0000 7f710ffff700 1 --2- 172.21.15.52:0/3990417161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7110100da0 0x7f7110196070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:16:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.898+0000 7f710f7fe700 1 --2- 172.21.15.52:0/3990417161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71101036c0 0x7f71101965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:16:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.898+0000 7f71149ab700 1 --2- 172.21.15.52:0/3990417161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7110105fe0 0x7f711019d620 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7108007ee0 tx=0x7f710800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.898+0000 7f710d7fa700 1 -- 172.21.15.52:0/3990417161 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108017070 con 0x7f7110105fe0 2023-11-27T15:16:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.898+0000 7f710d7fa700 1 -- 172.21.15.52:0/3990417161 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f710800f040 con 0x7f7110105fe0 2023-11-27T15:16:14.907 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.898+0000 7f710d7fa700 1 -- 172.21.15.52:0/3990417161 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108005740 con 0x7f7110105fe0 2023-11-27T15:16:14.907 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.898+0000 7f711640e700 1 -- 172.21.15.52:0/3990417161 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f711010bcd0 con 0x7f7110105fe0 2023-11-27T15:16:14.907 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.898+0000 7f711640e700 1 -- 172.21.15.52:0/3990417161 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f711010bb40 con 0x7f7110105fe0 2023-11-27T15:16:14.907 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.899+0000 7f711640e700 1 -- 172.21.15.52:0/3990417161 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71100fc330 con 0x7f7110105fe0 2023-11-27T15:16:14.907 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.901+0000 7f710d7fa700 1 -- 172.21.15.52:0/3990417161 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f71080058a0 con 0x7f7110105fe0 2023-11-27T15:16:14.908 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.901+0000 7f710d7fa700 1 --2- 172.21.15.52:0/3990417161 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f70f80a1150 0x7f70f80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:14.908 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.901+0000 7f710ffff700 1 --2- 172.21.15.52:0/3990417161 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f70f80a1150 0x7f70f80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:14.908 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.901+0000 7f710d7fa700 1 -- 172.21.15.52:0/3990417161 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f71080cc640 con 0x7f7110105fe0 2023-11-27T15:16:14.908 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.902+0000 7f710ffff700 1 --2- 172.21.15.52:0/3990417161 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f70f80a1150 0x7f70f80a3600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7100005fd0 tx=0x7f7100005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:14.908 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:14.904+0000 7f710d7fa700 1 -- 172.21.15.52:0/3990417161 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7108095ee0 con 0x7f7110105fe0 2023-11-27T15:16:15.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:14 smithi150 ceph-mon[100559]: pgmap v258: 97 pgs: 10 creating+peering, 79 active+clean, 8 unknown; 580 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s wr, 11 op/s 2023-11-27T15:16:15.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:14 smithi150 ceph-mon[100559]: mds.? [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] up:active 2023-11-27T15:16:15.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:14 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2634423145' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-11-27T15:16:15.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:14 smithi150 ceph-mon[100559]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-11-27T15:16:15.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:14 smithi150 ceph-mon[100559]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-11-27T15:16:15.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:14 smithi150 ceph-mon[100559]: Cluster is now healthy 2023-11-27T15:16:15.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:14 smithi150 ceph-mon[100559]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 2 up:standby-replay 8 up:standby 2023-11-27T15:16:15.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:14 smithi150 ceph-mon[100559]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:creating} 2 up:standby-replay 7 up:standby 2023-11-27T15:16:15.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:14 smithi150 ceph-mon[100559]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-11-27T15:16:15.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:14 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3177794449' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-11-27T15:16:15.078 INFO:journalctl@ceph.mds.b.smithi102.stdout:Nov 27 15:16:14 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-b[125504]: 2023-11-27T15:16:14.679+0000 7f44293b9700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-27T15:16:15.078 INFO:journalctl@ceph.mds.b.smithi102.stdout:Nov 27 15:16:14 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-b[125504]: 2023-11-27T15:16:14.679+0000 7f44293b9700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-27T15:16:15.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:14 smithi102 ceph-mon[100296]: pgmap v258: 97 pgs: 10 creating+peering, 79 active+clean, 8 unknown; 580 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s wr, 11 op/s 2023-11-27T15:16:15.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:14 smithi102 ceph-mon[100296]: mds.? [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] up:active 2023-11-27T15:16:15.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:14 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2634423145' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-11-27T15:16:15.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:14 smithi102 ceph-mon[100296]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-11-27T15:16:15.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:14 smithi102 ceph-mon[100296]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-11-27T15:16:15.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:14 smithi102 ceph-mon[100296]: Cluster is now healthy 2023-11-27T15:16:15.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:14 smithi102 ceph-mon[100296]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 2 up:standby-replay 8 up:standby 2023-11-27T15:16:15.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:14 smithi102 ceph-mon[100296]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:creating} 2 up:standby-replay 7 up:standby 2023-11-27T15:16:15.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:14 smithi102 ceph-mon[100296]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-11-27T15:16:15.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:14 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3177794449' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-11-27T15:16:15.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:14 smithi052 ceph-mon[99056]: pgmap v258: 97 pgs: 10 creating+peering, 79 active+clean, 8 unknown; 580 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s wr, 11 op/s 2023-11-27T15:16:15.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:14 smithi052 ceph-mon[99056]: mds.? [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] up:active 2023-11-27T15:16:15.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:14 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2634423145' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-11-27T15:16:15.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:14 smithi052 ceph-mon[99056]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-11-27T15:16:15.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:14 smithi052 ceph-mon[99056]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-11-27T15:16:15.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:14 smithi052 ceph-mon[99056]: Cluster is now healthy 2023-11-27T15:16:15.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:14 smithi052 ceph-mon[99056]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 2 up:standby-replay 8 up:standby 2023-11-27T15:16:15.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:14 smithi052 ceph-mon[99056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:creating} 2 up:standby-replay 7 up:standby 2023-11-27T15:16:15.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:14 smithi052 ceph-mon[99056]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-11-27T15:16:15.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:14 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3177794449' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-11-27T15:16:15.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.135+0000 7f711640e700 1 -- 172.21.15.52:0/3990417161 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f7110061c10 con 0x7f70f80a1150 2023-11-27T15:16:15.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.180+0000 7f710d7fa700 1 -- 172.21.15.52:0/3990417161 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7110061c10 con 0x7f70f80a1150 2023-11-27T15:16:15.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.183+0000 7f711640e700 1 -- 172.21.15.52:0/3990417161 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f70f80a1150 msgr2=0x7f70f80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:15.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.183+0000 7f711640e700 1 --2- 172.21.15.52:0/3990417161 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f70f80a1150 0x7f70f80a3600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7100005fd0 tx=0x7f7100005f60 comp rx=0 tx=0).stop 2023-11-27T15:16:15.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.183+0000 7f711640e700 1 -- 172.21.15.52:0/3990417161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7110105fe0 msgr2=0x7f711019d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:15.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.183+0000 7f711640e700 1 --2- 172.21.15.52:0/3990417161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7110105fe0 0x7f711019d620 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7108007ee0 tx=0x7f710800d3b0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.183+0000 7f711640e700 1 -- 172.21.15.52:0/3990417161 shutdown_connections 2023-11-27T15:16:15.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.183+0000 7f711640e700 1 --2- 172.21.15.52:0/3990417161 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f70f80a1150 0x7f70f80a3600 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.183+0000 7f711640e700 1 --2- 172.21.15.52:0/3990417161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7110105fe0 0x7f711019d620 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.183+0000 7f711640e700 1 --2- 172.21.15.52:0/3990417161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7110100da0 0x7f7110196070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.183+0000 7f711640e700 1 --2- 172.21.15.52:0/3990417161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71101036c0 0x7f71101965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.183+0000 7f711640e700 1 -- 172.21.15.52:0/3990417161 >> 172.21.15.52:0/3990417161 conn(0x7f71100fa7c0 msgr2=0x7f7110101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:15.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.183+0000 7f711640e700 1 -- 172.21.15.52:0/3990417161 shutdown_connections 2023-11-27T15:16:15.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.183+0000 7f711640e700 1 -- 172.21.15.52:0/3990417161 wait complete. 2023-11-27T15:16:15.199 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-11-27T15:16:15.330 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.324+0000 7f7a36bf0700 1 -- 172.21.15.52:0/4015982203 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a300fe300 msgr2=0x7f7a300fe750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:15.330 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.324+0000 7f7a36bf0700 1 --2- 172.21.15.52:0/4015982203 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a300fe300 0x7f7a300fe750 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c009990 tx=0x7f7a1c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.330 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.324+0000 7f7a36bf0700 1 -- 172.21.15.52:0/4015982203 shutdown_connections 2023-11-27T15:16:15.330 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.324+0000 7f7a36bf0700 1 --2- 172.21.15.52:0/4015982203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a300fec90 0x7f7a30108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.330 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.324+0000 7f7a36bf0700 1 --2- 172.21.15.52:0/4015982203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a30101890 0x7f7a30101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.324+0000 7f7a36bf0700 1 --2- 172.21.15.52:0/4015982203 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a300fe300 0x7f7a300fe750 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.324+0000 7f7a36bf0700 1 -- 172.21.15.52:0/4015982203 >> 172.21.15.52:0/4015982203 conn(0x7f7a300f9f30 msgr2=0x7f7a300fc380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:15.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.324+0000 7f7a36bf0700 1 -- 172.21.15.52:0/4015982203 shutdown_connections 2023-11-27T15:16:15.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.324+0000 7f7a36bf0700 1 -- 172.21.15.52:0/4015982203 wait complete. 2023-11-27T15:16:15.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.324+0000 7f7a36bf0700 1 Processor -- start 2023-11-27T15:16:15.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.324+0000 7f7a36bf0700 1 -- start start 2023-11-27T15:16:15.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.324+0000 7f7a36bf0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a300fe300 0x7f7a30195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:15.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.324+0000 7f7a36bf0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a300fec90 0x7f7a301964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:15.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.324+0000 7f7a36bf0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a30101890 0x7f7a3019d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:15.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.325+0000 7f7a36bf0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3019daf0 con 0x7f7a30101890 2023-11-27T15:16:15.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.325+0000 7f7a36bf0700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3019dc30 con 0x7f7a300fec90 2023-11-27T15:16:15.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.325+0000 7f7a36bf0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3019ddc0 con 0x7f7a300fe300 2023-11-27T15:16:15.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.325+0000 7f7a3518d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a30101890 0x7f7a3019d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:15.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.325+0000 7f7a3518d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a30101890 0x7f7a3019d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50146/0 (socket says 172.21.15.52:50146) 2023-11-27T15:16:15.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.325+0000 7f7a3518d700 1 -- 172.21.15.52:0/725096100 learned_addr learned my addr 172.21.15.52:0/725096100 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:15.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.325+0000 7f7a3498c700 1 --2- 172.21.15.52:0/725096100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a300fe300 0x7f7a30195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:15.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.325+0000 7f7a2ffff700 1 --2- 172.21.15.52:0/725096100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a300fec90 0x7f7a301964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:15.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.325+0000 7f7a3518d700 1 -- 172.21.15.52:0/725096100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a300fe300 msgr2=0x7f7a30195f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:15.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.325+0000 7f7a3518d700 1 --2- 172.21.15.52:0/725096100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a300fe300 0x7f7a30195f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.325+0000 7f7a3518d700 1 -- 172.21.15.52:0/725096100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a300fec90 msgr2=0x7f7a301964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:15.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.325+0000 7f7a3518d700 1 --2- 172.21.15.52:0/725096100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a300fec90 0x7f7a301964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.325+0000 7f7a3518d700 1 -- 172.21.15.52:0/725096100 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a1c0095f0 con 0x7f7a30101890 2023-11-27T15:16:15.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.325+0000 7f7a2ffff700 1 --2- 172.21.15.52:0/725096100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a300fec90 0x7f7a301964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:16:15.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.325+0000 7f7a3518d700 1 --2- 172.21.15.52:0/725096100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a30101890 0x7f7a3019d520 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800ef30 tx=0x7f7a2800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:15.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.325+0000 7f7a2dffb700 1 -- 172.21.15.52:0/725096100 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a28018070 con 0x7f7a30101890 2023-11-27T15:16:15.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.325+0000 7f7a2dffb700 1 -- 172.21.15.52:0/725096100 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a2800f040 con 0x7f7a30101890 2023-11-27T15:16:15.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.325+0000 7f7a36bf0700 1 -- 172.21.15.52:0/725096100 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a3010bc10 con 0x7f7a30101890 2023-11-27T15:16:15.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.325+0000 7f7a2dffb700 1 -- 172.21.15.52:0/725096100 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2800cdb0 con 0x7f7a30101890 2023-11-27T15:16:15.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.326+0000 7f7a36bf0700 1 -- 172.21.15.52:0/725096100 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a3010ba80 con 0x7f7a30101890 2023-11-27T15:16:15.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.326+0000 7f7a36bf0700 1 -- 172.21.15.52:0/725096100 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a30107950 con 0x7f7a30101890 2023-11-27T15:16:15.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.327+0000 7f7a2dffb700 1 -- 172.21.15.52:0/725096100 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7a28003a00 con 0x7f7a30101890 2023-11-27T15:16:15.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.328+0000 7f7a2dffb700 1 --2- 172.21.15.52:0/725096100 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7a200a11d0 0x7f7a200a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:15.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.328+0000 7f7a2dffb700 1 -- 172.21.15.52:0/725096100 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7a280cc2f0 con 0x7f7a30101890 2023-11-27T15:16:15.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.328+0000 7f7a3498c700 1 --2- 172.21.15.52:0/725096100 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7a200a11d0 0x7f7a200a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:15.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.328+0000 7f7a3498c700 1 --2- 172.21.15.52:0/725096100 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7a200a11d0 0x7f7a200a3680 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7a30067af0 tx=0x7f7a2400a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:15.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.332+0000 7f7a2dffb700 1 -- 172.21.15.52:0/725096100 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7a28095c20 con 0x7f7a30101890 2023-11-27T15:16:15.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.567+0000 7f7a36bf0700 1 -- 172.21.15.52:0/725096100 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a30061c70 con 0x7f7a200a11d0 2023-11-27T15:16:15.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.572+0000 7f7a2dffb700 1 -- 172.21.15.52:0/725096100 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f7a30061c70 con 0x7f7a200a11d0 2023-11-27T15:16:15.576 INFO:teuthology.orchestra.run.smithi052.stdout:/volumes/_nogroup/sv_0/ad0bae76-7c55-44f4-bb6d-804c35f8bcda 2023-11-27T15:16:15.578 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.574+0000 7f7a36bf0700 1 -- 172.21.15.52:0/725096100 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7a200a11d0 msgr2=0x7f7a200a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:15.578 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.574+0000 7f7a36bf0700 1 --2- 172.21.15.52:0/725096100 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7a200a11d0 0x7f7a200a3680 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7a30067af0 tx=0x7f7a2400a3b0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.578 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.574+0000 7f7a36bf0700 1 -- 172.21.15.52:0/725096100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a30101890 msgr2=0x7f7a3019d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:15.579 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.574+0000 7f7a36bf0700 1 --2- 172.21.15.52:0/725096100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a30101890 0x7f7a3019d520 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800ef30 tx=0x7f7a2800c3a0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.579 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.574+0000 7f7a36bf0700 1 -- 172.21.15.52:0/725096100 shutdown_connections 2023-11-27T15:16:15.579 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.574+0000 7f7a36bf0700 1 --2- 172.21.15.52:0/725096100 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7a200a11d0 0x7f7a200a3680 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.579 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.574+0000 7f7a36bf0700 1 --2- 172.21.15.52:0/725096100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a30101890 0x7f7a3019d520 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.579 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.574+0000 7f7a36bf0700 1 --2- 172.21.15.52:0/725096100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a300fe300 0x7f7a30195f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.579 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.574+0000 7f7a36bf0700 1 --2- 172.21.15.52:0/725096100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a300fec90 0x7f7a301964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.579 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.574+0000 7f7a36bf0700 1 -- 172.21.15.52:0/725096100 >> 172.21.15.52:0/725096100 conn(0x7f7a300f9f30 msgr2=0x7f7a30104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:15.579 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.574+0000 7f7a36bf0700 1 -- 172.21.15.52:0/725096100 shutdown_connections 2023-11-27T15:16:15.579 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.574+0000 7f7a36bf0700 1 -- 172.21.15.52:0/725096100 wait complete. 2023-11-27T15:16:15.589 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:16:15 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:16:15.140+0000 7fe9e76de700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-27T15:16:15.589 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:16:15 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:16:15.140+0000 7fe9e76de700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-27T15:16:15.590 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:16:15 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:16:15.140+0000 7fe9e76de700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-27T15:16:15.590 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:16:15 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:16:15.140+0000 7fe9e76de700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-27T15:16:15.590 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:16:15 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:16:15.140+0000 7fe9e76de700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-27T15:16:15.591 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-11-27T15:16:15.723 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.716+0000 7f2fa79d6700 1 -- 172.21.15.52:0/765180879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fa01036a0 msgr2=0x7f2fa0105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:15.723 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.716+0000 7f2fa79d6700 1 --2- 172.21.15.52:0/765180879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fa01036a0 0x7f2fa0105a80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2f90009990 tx=0x7f2f90009ca0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.723 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.716+0000 7f2fa79d6700 1 -- 172.21.15.52:0/765180879 shutdown_connections 2023-11-27T15:16:15.723 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.716+0000 7f2fa79d6700 1 --2- 172.21.15.52:0/765180879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fa0100d80 0x7f2fa0103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.723 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.716+0000 7f2fa79d6700 1 --2- 172.21.15.52:0/765180879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fa0105fc0 0x7f2fa01083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.723 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.716+0000 7f2fa79d6700 1 --2- 172.21.15.52:0/765180879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fa01036a0 0x7f2fa0105a80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.723 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.716+0000 7f2fa79d6700 1 -- 172.21.15.52:0/765180879 >> 172.21.15.52:0/765180879 conn(0x7f2fa00fa7c0 msgr2=0x7f2fa00fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:15.724 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.716+0000 7f2fa79d6700 1 -- 172.21.15.52:0/765180879 shutdown_connections 2023-11-27T15:16:15.724 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.716+0000 7f2fa79d6700 1 -- 172.21.15.52:0/765180879 wait complete. 2023-11-27T15:16:15.724 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.717+0000 7f2fa79d6700 1 Processor -- start 2023-11-27T15:16:15.724 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.717+0000 7f2fa79d6700 1 -- start start 2023-11-27T15:16:15.724 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.717+0000 7f2fa79d6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fa0100d80 0x7f2fa0195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:15.724 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.717+0000 7f2fa79d6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fa01036a0 0x7f2fa0196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:15.724 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.717+0000 7f2fa79d6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fa0105fc0 0x7f2fa019d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:15.724 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.717+0000 7f2fa79d6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa019db50 con 0x7f2fa01036a0 2023-11-27T15:16:15.724 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.717+0000 7f2fa79d6700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa019dc90 con 0x7f2fa0100d80 2023-11-27T15:16:15.724 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.717+0000 7f2fa79d6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa019de20 con 0x7f2fa0105fc0 2023-11-27T15:16:15.724 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.717+0000 7f2fa4f71700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fa01036a0 0x7f2fa0196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:15.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.717+0000 7f2fa5772700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fa0100d80 0x7f2fa0195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:15.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.717+0000 7f2fa4f71700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fa01036a0 0x7f2fa0196510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50158/0 (socket says 172.21.15.52:50158) 2023-11-27T15:16:15.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.717+0000 7f2fa4f71700 1 -- 172.21.15.52:0/2368809957 learned_addr learned my addr 172.21.15.52:0/2368809957 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:15.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.717+0000 7f2fa5f73700 1 --2- 172.21.15.52:0/2368809957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fa0105fc0 0x7f2fa019d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:15.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.717+0000 7f2fa4f71700 1 -- 172.21.15.52:0/2368809957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fa0105fc0 msgr2=0x7f2fa019d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:15.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.717+0000 7f2fa4f71700 1 --2- 172.21.15.52:0/2368809957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fa0105fc0 0x7f2fa019d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.717+0000 7f2fa4f71700 1 -- 172.21.15.52:0/2368809957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fa0100d80 msgr2=0x7f2fa0195fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:15.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.717+0000 7f2fa4f71700 1 --2- 172.21.15.52:0/2368809957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fa0100d80 0x7f2fa0195fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.718+0000 7f2fa4f71700 1 -- 172.21.15.52:0/2368809957 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f900095f0 con 0x7f2fa01036a0 2023-11-27T15:16:15.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.718+0000 7f2fa5772700 1 --2- 172.21.15.52:0/2368809957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fa0100d80 0x7f2fa0195fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:16:15.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.718+0000 7f2fa4f71700 1 --2- 172.21.15.52:0/2368809957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fa01036a0 0x7f2fa0196510 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2f90000c00 tx=0x7f2f9000f700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:15.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.718+0000 7f2f967fc700 1 -- 172.21.15.52:0/2368809957 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9001d070 con 0x7f2fa01036a0 2023-11-27T15:16:15.726 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.718+0000 7f2f967fc700 1 -- 172.21.15.52:0/2368809957 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f9000fbe0 con 0x7f2fa01036a0 2023-11-27T15:16:15.726 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.718+0000 7f2fa79d6700 1 -- 172.21.15.52:0/2368809957 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa010bcd0 con 0x7f2fa01036a0 2023-11-27T15:16:15.726 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.718+0000 7f2f967fc700 1 -- 172.21.15.52:0/2368809957 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f90017650 con 0x7f2fa01036a0 2023-11-27T15:16:15.726 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.718+0000 7f2fa79d6700 1 -- 172.21.15.52:0/2368809957 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa010bb40 con 0x7f2fa01036a0 2023-11-27T15:16:15.726 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.719+0000 7f2fa79d6700 1 -- 172.21.15.52:0/2368809957 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa00fc330 con 0x7f2fa01036a0 2023-11-27T15:16:15.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.720+0000 7f2f967fc700 1 -- 172.21.15.52:0/2368809957 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2f9000fd50 con 0x7f2fa01036a0 2023-11-27T15:16:15.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.721+0000 7f2f967fc700 1 --2- 172.21.15.52:0/2368809957 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f2f8c0a1150 0x7f2f8c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:15.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.721+0000 7f2fa5772700 1 --2- 172.21.15.52:0/2368809957 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f2f8c0a1150 0x7f2f8c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:15.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.721+0000 7f2f967fc700 1 -- 172.21.15.52:0/2368809957 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2f900cde40 con 0x7f2fa01036a0 2023-11-27T15:16:15.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.721+0000 7f2fa5772700 1 --2- 172.21.15.52:0/2368809957 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f2f8c0a1150 0x7f2f8c0a3600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c00afd0 tx=0x7f2f9c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:15.728 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.724+0000 7f2f967fc700 1 -- 172.21.15.52:0/2368809957 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2f90097770 con 0x7f2fa01036a0 2023-11-27T15:16:15.760 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.756+0000 7f2f967fc700 1 -- 172.21.15.52:0/2368809957 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f2f90097b40 con 0x7f2fa01036a0 2023-11-27T15:16:15.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:15 smithi052 ceph-mon[99056]: mds.? [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] up:active 2023-11-27T15:16:15.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:15 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3177794449' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-11-27T15:16:15.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:15 smithi052 ceph-mon[99056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2023-11-27T15:16:15.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:15 smithi052 ceph-mon[99056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:16:15.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:15 smithi052 ceph-mon[99056]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:15.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:15 smithi052 ceph-mon[99056]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:15.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:15 smithi052 ceph-mon[99056]: pgmap v259: 97 pgs: 97 active+clean; 580 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s wr, 11 op/s 2023-11-27T15:16:15.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.963+0000 7f2fa79d6700 1 -- 172.21.15.52:0/2368809957 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fa0061bc0 con 0x7f2f8c0a1150 2023-11-27T15:16:15.991 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.987+0000 7f2f967fc700 1 -- 172.21.15.52:0/2368809957 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2fa0061bc0 con 0x7f2f8c0a1150 2023-11-27T15:16:15.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.988+0000 7f2fa79d6700 1 -- 172.21.15.52:0/2368809957 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f2f8c0a1150 msgr2=0x7f2f8c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:15.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.988+0000 7f2fa79d6700 1 --2- 172.21.15.52:0/2368809957 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f2f8c0a1150 0x7f2f8c0a3600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c00afd0 tx=0x7f2f9c00a380 comp rx=0 tx=0).stop 2023-11-27T15:16:15.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.988+0000 7f2fa79d6700 1 -- 172.21.15.52:0/2368809957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fa01036a0 msgr2=0x7f2fa0196510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:15.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.988+0000 7f2fa79d6700 1 --2- 172.21.15.52:0/2368809957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fa01036a0 0x7f2fa0196510 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2f90000c00 tx=0x7f2f9000f700 comp rx=0 tx=0).stop 2023-11-27T15:16:15.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.989+0000 7f2fa79d6700 1 -- 172.21.15.52:0/2368809957 shutdown_connections 2023-11-27T15:16:15.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.989+0000 7f2fa79d6700 1 --2- 172.21.15.52:0/2368809957 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f2f8c0a1150 0x7f2f8c0a3600 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.989+0000 7f2fa79d6700 1 --2- 172.21.15.52:0/2368809957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fa0100d80 0x7f2fa0195fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.989+0000 7f2fa79d6700 1 --2- 172.21.15.52:0/2368809957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2fa0105fc0 0x7f2fa019d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.989+0000 7f2fa79d6700 1 --2- 172.21.15.52:0/2368809957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fa01036a0 0x7f2fa0196510 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.989+0000 7f2fa79d6700 1 -- 172.21.15.52:0/2368809957 >> 172.21.15.52:0/2368809957 conn(0x7f2fa00fa7c0 msgr2=0x7f2fa01019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:15.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.989+0000 7f2fa79d6700 1 -- 172.21.15.52:0/2368809957 shutdown_connections 2023-11-27T15:16:15.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:15.989+0000 7f2fa79d6700 1 -- 172.21.15.52:0/2368809957 wait complete. 2023-11-27T15:16:16.005 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-11-27T15:16:16.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:15 smithi150 ceph-mon[100559]: mds.? [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] up:active 2023-11-27T15:16:16.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:15 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3177794449' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-11-27T15:16:16.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:15 smithi150 ceph-mon[100559]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2023-11-27T15:16:16.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:15 smithi150 ceph-mon[100559]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:16:16.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:15 smithi150 ceph-mon[100559]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:16.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:15 smithi150 ceph-mon[100559]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:16.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:15 smithi150 ceph-mon[100559]: pgmap v259: 97 pgs: 97 active+clean; 580 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s wr, 11 op/s 2023-11-27T15:16:16.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:15 smithi102 ceph-mon[100296]: mds.? [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] up:active 2023-11-27T15:16:16.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:15 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3177794449' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-11-27T15:16:16.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:15 smithi102 ceph-mon[100296]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2023-11-27T15:16:16.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:15 smithi102 ceph-mon[100296]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:16:16.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:15 smithi102 ceph-mon[100296]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:16.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:15 smithi102 ceph-mon[100296]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:16.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:15 smithi102 ceph-mon[100296]: pgmap v259: 97 pgs: 97 active+clean; 580 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s wr, 11 op/s 2023-11-27T15:16:16.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.130+0000 7f640e827700 1 -- 172.21.15.52:0/439761262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6408100d40 msgr2=0x7f6408103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:16.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.130+0000 7f640e827700 1 --2- 172.21.15.52:0/439761262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6408100d40 0x7f6408103120 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f63f8009990 tx=0x7f63f8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:16:16.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.130+0000 7f640e827700 1 -- 172.21.15.52:0/439761262 shutdown_connections 2023-11-27T15:16:16.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.130+0000 7f640e827700 1 --2- 172.21.15.52:0/439761262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6408105f80 0x7f64081083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:16.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.130+0000 7f640e827700 1 --2- 172.21.15.52:0/439761262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6408100d40 0x7f6408103120 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:16.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.130+0000 7f640e827700 1 --2- 172.21.15.52:0/439761262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6408103660 0x7f6408105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:16.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.130+0000 7f640e827700 1 -- 172.21.15.52:0/439761262 >> 172.21.15.52:0/439761262 conn(0x7f64080fa7c0 msgr2=0x7f64080fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:16.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.130+0000 7f640e827700 1 -- 172.21.15.52:0/439761262 shutdown_connections 2023-11-27T15:16:16.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.130+0000 7f640e827700 1 -- 172.21.15.52:0/439761262 wait complete. 2023-11-27T15:16:16.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.130+0000 7f640e827700 1 Processor -- start 2023-11-27T15:16:16.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.130+0000 7f640e827700 1 -- start start 2023-11-27T15:16:16.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.131+0000 7f640e827700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6408100d40 0x7f64081960c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:16.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.131+0000 7f640e827700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6408103660 0x7f6408196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:16.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.131+0000 7f640e827700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6408105f80 0x7f640819d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:16.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.131+0000 7f640e827700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640819dc40 con 0x7f6408100d40 2023-11-27T15:16:16.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.131+0000 7f640e827700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640819dd80 con 0x7f6408103660 2023-11-27T15:16:16.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.131+0000 7f640e827700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640819df10 con 0x7f6408105f80 2023-11-27T15:16:16.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.131+0000 7f6407fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6408100d40 0x7f64081960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:16.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.131+0000 7f6407fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6408100d40 0x7f64081960c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50172/0 (socket says 172.21.15.52:50172) 2023-11-27T15:16:16.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.131+0000 7f6407fff700 1 -- 172.21.15.52:0/2848297597 learned_addr learned my addr 172.21.15.52:0/2848297597 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:16.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.131+0000 7f640cdc4700 1 --2- 172.21.15.52:0/2848297597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6408105f80 0x7f640819d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:16.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.131+0000 7f64077fe700 1 --2- 172.21.15.52:0/2848297597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6408103660 0x7f6408196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:16.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.131+0000 7f6407fff700 1 -- 172.21.15.52:0/2848297597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6408105f80 msgr2=0x7f640819d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:16.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.131+0000 7f6407fff700 1 --2- 172.21.15.52:0/2848297597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6408105f80 0x7f640819d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:16.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.131+0000 7f6407fff700 1 -- 172.21.15.52:0/2848297597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6408103660 msgr2=0x7f6408196600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:16.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.131+0000 7f6407fff700 1 --2- 172.21.15.52:0/2848297597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6408103660 0x7f6408196600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:16.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.131+0000 7f6407fff700 1 -- 172.21.15.52:0/2848297597 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f80095f0 con 0x7f6408100d40 2023-11-27T15:16:16.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.131+0000 7f640cdc4700 1 --2- 172.21.15.52:0/2848297597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6408105f80 0x7f640819d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:16:16.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.131+0000 7f6407fff700 1 --2- 172.21.15.52:0/2848297597 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6408100d40 0x7f64081960c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f63f800b680 tx=0x7f63f8005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:16.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.132+0000 7f64057fa700 1 -- 172.21.15.52:0/2848297597 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f801d070 con 0x7f6408100d40 2023-11-27T15:16:16.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.132+0000 7f640e827700 1 -- 172.21.15.52:0/2848297597 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f640810bca0 con 0x7f6408100d40 2023-11-27T15:16:16.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.132+0000 7f64057fa700 1 -- 172.21.15.52:0/2848297597 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63f800fbc0 con 0x7f6408100d40 2023-11-27T15:16:16.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.132+0000 7f64057fa700 1 -- 172.21.15.52:0/2848297597 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f8017650 con 0x7f6408100d40 2023-11-27T15:16:16.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.132+0000 7f640e827700 1 -- 172.21.15.52:0/2848297597 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f640810bb10 con 0x7f6408100d40 2023-11-27T15:16:16.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.133+0000 7f640e827700 1 -- 172.21.15.52:0/2848297597 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64080fc330 con 0x7f6408100d40 2023-11-27T15:16:16.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.134+0000 7f64057fa700 1 -- 172.21.15.52:0/2848297597 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f63f8021410 con 0x7f6408100d40 2023-11-27T15:16:16.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.134+0000 7f64057fa700 1 --2- 172.21.15.52:0/2848297597 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f63f00a11e0 0x7f63f00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:16.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.134+0000 7f64077fe700 1 --2- 172.21.15.52:0/2848297597 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f63f00a11e0 0x7f63f00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:16.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.134+0000 7f64057fa700 1 -- 172.21.15.52:0/2848297597 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f63f80cdeb0 con 0x7f6408100d40 2023-11-27T15:16:16.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.135+0000 7f64077fe700 1 --2- 172.21.15.52:0/2848297597 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f63f00a11e0 0x7f63f00a3690 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00bbd0 tx=0x7f63fc00b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:16.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.139+0000 7f64057fa700 1 -- 172.21.15.52:0/2848297597 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f63f80976e0 con 0x7f6408100d40 2023-11-27T15:16:16.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.372+0000 7f640e827700 1 -- 172.21.15.52:0/2848297597 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f6408061bf0 con 0x7f63f00a11e0 2023-11-27T15:16:16.380 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.376+0000 7f64057fa700 1 -- 172.21.15.52:0/2848297597 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f6408061bf0 con 0x7f63f00a11e0 2023-11-27T15:16:16.381 INFO:teuthology.orchestra.run.smithi052.stdout:/volumes/_nogroup/sv_1/fdbc3811-1284-497a-bc65-bbb78ba5ec15 2023-11-27T15:16:16.383 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.378+0000 7f640e827700 1 -- 172.21.15.52:0/2848297597 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f63f00a11e0 msgr2=0x7f63f00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:16.383 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.378+0000 7f640e827700 1 --2- 172.21.15.52:0/2848297597 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f63f00a11e0 0x7f63f00a3690 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00bbd0 tx=0x7f63fc00b5b0 comp rx=0 tx=0).stop 2023-11-27T15:16:16.383 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.378+0000 7f640e827700 1 -- 172.21.15.52:0/2848297597 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6408100d40 msgr2=0x7f64081960c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:16.383 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.378+0000 7f640e827700 1 --2- 172.21.15.52:0/2848297597 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6408100d40 0x7f64081960c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f63f800b680 tx=0x7f63f8005ef0 comp rx=0 tx=0).stop 2023-11-27T15:16:16.383 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.378+0000 7f640e827700 1 -- 172.21.15.52:0/2848297597 shutdown_connections 2023-11-27T15:16:16.383 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.378+0000 7f640e827700 1 --2- 172.21.15.52:0/2848297597 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f63f00a11e0 0x7f63f00a3690 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:16.383 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.378+0000 7f640e827700 1 --2- 172.21.15.52:0/2848297597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6408105f80 0x7f640819d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:16.383 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.378+0000 7f640e827700 1 --2- 172.21.15.52:0/2848297597 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6408100d40 0x7f64081960c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:16.383 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.378+0000 7f640e827700 1 --2- 172.21.15.52:0/2848297597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6408103660 0x7f6408196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:16.383 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.378+0000 7f640e827700 1 -- 172.21.15.52:0/2848297597 >> 172.21.15.52:0/2848297597 conn(0x7f64080fa7c0 msgr2=0x7f64081019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:16.384 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.378+0000 7f640e827700 1 -- 172.21.15.52:0/2848297597 shutdown_connections 2023-11-27T15:16:16.384 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.378+0000 7f640e827700 1 -- 172.21.15.52:0/2848297597 wait complete. 2023-11-27T15:16:16.395 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:16:16.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.516+0000 7fed41832700 1 -- 172.21.15.52:0/4075829340 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed3c0ff170 msgr2=0x7fed3c0ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:16.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.516+0000 7fed41832700 1 --2- 172.21.15.52:0/4075829340 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed3c0ff170 0x7fed3c0ff580 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fed2c009990 tx=0x7fed2c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:16:16.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.516+0000 7fed41832700 1 -- 172.21.15.52:0/4075829340 shutdown_connections 2023-11-27T15:16:16.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.516+0000 7fed41832700 1 --2- 172.21.15.52:0/4075829340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed3c100db0 0x7fed3c107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:16.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.516+0000 7fed41832700 1 --2- 172.21.15.52:0/4075829340 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed3c0ff170 0x7fed3c0ff580 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:16.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.516+0000 7fed41832700 1 --2- 172.21.15.52:0/4075829340 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed3c100400 0x7fed3c100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:16.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.516+0000 7fed41832700 1 -- 172.21.15.52:0/4075829340 >> 172.21.15.52:0/4075829340 conn(0x7fed3c0fa770 msgr2=0x7fed3c0fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:16.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.517+0000 7fed41832700 1 -- 172.21.15.52:0/4075829340 shutdown_connections 2023-11-27T15:16:16.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.517+0000 7fed41832700 1 -- 172.21.15.52:0/4075829340 wait complete. 2023-11-27T15:16:16.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.517+0000 7fed41832700 1 Processor -- start 2023-11-27T15:16:16.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.517+0000 7fed41832700 1 -- start start 2023-11-27T15:16:16.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.517+0000 7fed41832700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed3c0ff170 0x7fed3c069c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:16.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.517+0000 7fed41832700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed3c100400 0x7fed3c19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:16.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.517+0000 7fed41832700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed3c100db0 0x7fed3c19d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:16.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.517+0000 7fed41832700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c19da60 con 0x7fed3c0ff170 2023-11-27T15:16:16.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.517+0000 7fed41832700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c19dbd0 con 0x7fed3c100400 2023-11-27T15:16:16.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.517+0000 7fed41832700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c19dd90 con 0x7fed3c100db0 2023-11-27T15:16:16.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.518+0000 7fed3affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed3c0ff170 0x7fed3c069c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:16.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.518+0000 7fed3affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed3c0ff170 0x7fed3c069c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50186/0 (socket says 172.21.15.52:50186) 2023-11-27T15:16:16.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.518+0000 7fed3affd700 1 -- 172.21.15.52:0/801939919 learned_addr learned my addr 172.21.15.52:0/801939919 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:16.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.518+0000 7fed3a7fc700 1 --2- 172.21.15.52:0/801939919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed3c100400 0x7fed3c19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:16.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.518+0000 7fed3b7fe700 1 --2- 172.21.15.52:0/801939919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed3c100db0 0x7fed3c19d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:16.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.518+0000 7fed3affd700 1 -- 172.21.15.52:0/801939919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed3c100db0 msgr2=0x7fed3c19d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:16.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.518+0000 7fed3affd700 1 --2- 172.21.15.52:0/801939919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed3c100db0 0x7fed3c19d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:16.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.518+0000 7fed3affd700 1 -- 172.21.15.52:0/801939919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed3c100400 msgr2=0x7fed3c19ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:16.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.518+0000 7fed3affd700 1 --2- 172.21.15.52:0/801939919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed3c100400 0x7fed3c19ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:16.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.518+0000 7fed3affd700 1 -- 172.21.15.52:0/801939919 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2c0095f0 con 0x7fed3c0ff170 2023-11-27T15:16:16.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.518+0000 7fed3affd700 1 --2- 172.21.15.52:0/801939919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed3c0ff170 0x7fed3c069c60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fed2c004740 tx=0x7fed2c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:16.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.518+0000 7fed40830700 1 -- 172.21.15.52:0/801939919 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2c01d070 con 0x7fed3c0ff170 2023-11-27T15:16:16.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.518+0000 7fed41832700 1 -- 172.21.15.52:0/801939919 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed3c10ab60 con 0x7fed3c0ff170 2023-11-27T15:16:16.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.518+0000 7fed40830700 1 -- 172.21.15.52:0/801939919 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed2c004830 con 0x7fed3c0ff170 2023-11-27T15:16:16.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.518+0000 7fed40830700 1 -- 172.21.15.52:0/801939919 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2c005450 con 0x7fed3c0ff170 2023-11-27T15:16:16.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.518+0000 7fed41832700 1 -- 172.21.15.52:0/801939919 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3c10a9d0 con 0x7fed3c0ff170 2023-11-27T15:16:16.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.519+0000 7fed41832700 1 -- 172.21.15.52:0/801939919 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed3c106190 con 0x7fed3c0ff170 2023-11-27T15:16:16.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.520+0000 7fed40830700 1 -- 172.21.15.52:0/801939919 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fed2c0055d0 con 0x7fed3c0ff170 2023-11-27T15:16:16.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.521+0000 7fed40830700 1 --2- 172.21.15.52:0/801939919 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fed240a11e0 0x7fed240a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:16.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.521+0000 7fed40830700 1 -- 172.21.15.52:0/801939919 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fed2c0ce0c0 con 0x7fed3c0ff170 2023-11-27T15:16:16.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.521+0000 7fed3a7fc700 1 --2- 172.21.15.52:0/801939919 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fed240a11e0 0x7fed240a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:16.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.521+0000 7fed3a7fc700 1 --2- 172.21.15.52:0/801939919 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fed240a11e0 0x7fed240a3690 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fed3c0699d0 tx=0x7fed30009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:16.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.525+0000 7fed40830700 1 -- 172.21.15.52:0/801939919 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fed2c0977f0 con 0x7fed3c0ff170 2023-11-27T15:16:16.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.770+0000 7fed41832700 1 -- 172.21.15.52:0/801939919 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed3c02cfe0 con 0x7fed3c0ff170 2023-11-27T15:16:16.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.770+0000 7fed40830700 1 -- 172.21.15.52:0/801939919 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10804 (secure 0 0 0) 0x7fed2c09ae10 con 0x7fed3c0ff170 2023-11-27T15:16:16.775 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:16:16.775 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2328881376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2328881376},{"type":"v1","addr":"172.21.15.52:6839","nonce":2328881376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2799551570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2799551570},{"type":"v1","addr":"172.21.15.52:6841","nonce":2799551570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:16:09.319630+0000","modified":"2023-11-27T15:16:14.742405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24494,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/215725838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":215725838},{"type":"v1","addr":"172.21.15.150:6839","nonce":215725838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3571830559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3571830559},{"type":"v1","addr":"172.21.15.102:6839","nonce":3571830559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:16:16.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.772+0000 7fed41832700 1 -- 172.21.15.52:0/801939919 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fed240a11e0 msgr2=0x7fed240a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:16.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.772+0000 7fed41832700 1 --2- 172.21.15.52:0/801939919 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fed240a11e0 0x7fed240a3690 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fed3c0699d0 tx=0x7fed30009260 comp rx=0 tx=0).stop 2023-11-27T15:16:16.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.772+0000 7fed41832700 1 -- 172.21.15.52:0/801939919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed3c0ff170 msgr2=0x7fed3c069c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:16.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.772+0000 7fed41832700 1 --2- 172.21.15.52:0/801939919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed3c0ff170 0x7fed3c069c60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fed2c004740 tx=0x7fed2c009580 comp rx=0 tx=0).stop 2023-11-27T15:16:16.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.772+0000 7fed41832700 1 -- 172.21.15.52:0/801939919 shutdown_connections 2023-11-27T15:16:16.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.772+0000 7fed41832700 1 --2- 172.21.15.52:0/801939919 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fed240a11e0 0x7fed240a3690 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:16.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.772+0000 7fed41832700 1 --2- 172.21.15.52:0/801939919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fed3c100db0 0x7fed3c19d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:16.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.772+0000 7fed41832700 1 --2- 172.21.15.52:0/801939919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed3c0ff170 0x7fed3c069c60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:16.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.772+0000 7fed41832700 1 --2- 172.21.15.52:0/801939919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed3c100400 0x7fed3c19ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:16.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.772+0000 7fed41832700 1 -- 172.21.15.52:0/801939919 >> 172.21.15.52:0/801939919 conn(0x7fed3c0fa770 msgr2=0x7fed3c0fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:16.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.773+0000 7fed41832700 1 -- 172.21.15.52:0/801939919 shutdown_connections 2023-11-27T15:16:16.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.773+0000 7fed41832700 1 -- 172.21.15.52:0/801939919 wait complete. 2023-11-27T15:16:16.778 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 23 2023-11-27T15:16:16.789 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:16:16.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.912+0000 7fb05fefc700 1 -- 172.21.15.52:0/3083004876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb058103660 msgr2=0x7fb058105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:16.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.912+0000 7fb05fefc700 1 --2- 172.21.15.52:0/3083004876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb058103660 0x7fb058105a40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb0480098f0 tx=0x7fb048009c00 comp rx=0 tx=0).stop 2023-11-27T15:16:16.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.912+0000 7fb05fefc700 1 -- 172.21.15.52:0/3083004876 shutdown_connections 2023-11-27T15:16:16.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.912+0000 7fb05fefc700 1 --2- 172.21.15.52:0/3083004876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb058100d40 0x7fb058103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:16.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.912+0000 7fb05fefc700 1 --2- 172.21.15.52:0/3083004876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb058105f80 0x7fb0581083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:16.917 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.912+0000 7fb05fefc700 1 --2- 172.21.15.52:0/3083004876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb058103660 0x7fb058105a40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:16.917 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.912+0000 7fb05fefc700 1 -- 172.21.15.52:0/3083004876 >> 172.21.15.52:0/3083004876 conn(0x7fb0580fa7c0 msgr2=0x7fb0580fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:16.917 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.912+0000 7fb05fefc700 1 -- 172.21.15.52:0/3083004876 shutdown_connections 2023-11-27T15:16:16.917 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.912+0000 7fb05fefc700 1 -- 172.21.15.52:0/3083004876 wait complete. 2023-11-27T15:16:16.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.912+0000 7fb05fefc700 1 Processor -- start 2023-11-27T15:16:16.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.912+0000 7fb05fefc700 1 -- start start 2023-11-27T15:16:16.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.913+0000 7fb05fefc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb058100d40 0x7fb0581006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:16.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.913+0000 7fb05fefc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb058103660 0x7fb0580fecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:16.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.913+0000 7fb05fefc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb058105f80 0x7fb0580ff230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:16.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.913+0000 7fb05fefc700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0580ff9d0 con 0x7fb058105f80 2023-11-27T15:16:16.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.913+0000 7fb05fefc700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0580ffb40 con 0x7fb058103660 2023-11-27T15:16:16.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.913+0000 7fb05fefc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0581a0030 con 0x7fb058100d40 2023-11-27T15:16:16.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.913+0000 7fb05e499700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb058105f80 0x7fb0580ff230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:16.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.913+0000 7fb05dc98700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb058100d40 0x7fb0581006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:16.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.913+0000 7fb05d497700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb058103660 0x7fb0580fecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:16.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.913+0000 7fb05e499700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb058105f80 0x7fb0580ff230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50204/0 (socket says 172.21.15.52:50204) 2023-11-27T15:16:16.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.913+0000 7fb05e499700 1 -- 172.21.15.52:0/745563203 learned_addr learned my addr 172.21.15.52:0/745563203 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:16.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.913+0000 7fb05e499700 1 -- 172.21.15.52:0/745563203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb058100d40 msgr2=0x7fb0581006b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:16.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.913+0000 7fb05e499700 1 --2- 172.21.15.52:0/745563203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb058100d40 0x7fb0581006b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:16.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.913+0000 7fb05e499700 1 -- 172.21.15.52:0/745563203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb058103660 msgr2=0x7fb0580fecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:16.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.913+0000 7fb05e499700 1 --2- 172.21.15.52:0/745563203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb058103660 0x7fb0580fecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:16.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.913+0000 7fb05e499700 1 -- 172.21.15.52:0/745563203 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0540095f0 con 0x7fb058105f80 2023-11-27T15:16:16.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.913+0000 7fb05e499700 1 --2- 172.21.15.52:0/745563203 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb058105f80 0x7fb0580ff230 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb05000ef30 tx=0x7fb05000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:16.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.913+0000 7fb04effd700 1 -- 172.21.15.52:0/745563203 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb050018070 con 0x7fb058105f80 2023-11-27T15:16:16.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.913+0000 7fb04effd700 1 -- 172.21.15.52:0/745563203 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb05000f040 con 0x7fb058105f80 2023-11-27T15:16:16.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.914+0000 7fb05fefc700 1 -- 172.21.15.52:0/745563203 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb048009520 con 0x7fb058105f80 2023-11-27T15:16:16.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.914+0000 7fb04effd700 1 -- 172.21.15.52:0/745563203 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05000cd00 con 0x7fb058105f80 2023-11-27T15:16:16.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.914+0000 7fb05fefc700 1 -- 172.21.15.52:0/745563203 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb05810bca0 con 0x7fb058105f80 2023-11-27T15:16:16.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.915+0000 7fb05fefc700 1 -- 172.21.15.52:0/745563203 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0580fc330 con 0x7fb058105f80 2023-11-27T15:16:16.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.916+0000 7fb04effd700 1 -- 172.21.15.52:0/745563203 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb050003a00 con 0x7fb058105f80 2023-11-27T15:16:16.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.916+0000 7fb04effd700 1 --2- 172.21.15.52:0/745563203 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb0440a1260 0x7fb0440a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:16.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.916+0000 7fb04effd700 1 -- 172.21.15.52:0/745563203 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb0500cc3b0 con 0x7fb058105f80 2023-11-27T15:16:16.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.916+0000 7fb05dc98700 1 --2- 172.21.15.52:0/745563203 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb0440a1260 0x7fb0440a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:16.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.917+0000 7fb05dc98700 1 --2- 172.21.15.52:0/745563203 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb0440a1260 0x7fb0440a3710 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb054005fd0 tx=0x7fb054009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:16.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:16.921+0000 7fb04effd700 1 -- 172.21.15.52:0/745563203 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb050095be0 con 0x7fb058105f80 2023-11-27T15:16:17.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:16 smithi150 ceph-mon[100559]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-11-27T15:16:17.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:16 smithi150 ceph-mon[100559]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:17.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:16 smithi150 ceph-mon[100559]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:17.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:16 smithi102 ceph-mon[100296]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-11-27T15:16:17.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:16 smithi102 ceph-mon[100296]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:17.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:16 smithi102 ceph-mon[100296]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:17.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:16 smithi052 ceph-mon[99056]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-11-27T15:16:17.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:16 smithi052 ceph-mon[99056]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:17.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:16 smithi052 ceph-mon[99056]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:17.123 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.119+0000 7fb05fefc700 1 -- 172.21.15.52:0/745563203 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb058067eb0 con 0x7fb058105f80 2023-11-27T15:16:17.125 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.121+0000 7fb04effd700 1 -- 172.21.15.52:0/745563203 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22130 (secure 0 0 0) 0x7fb050099200 con 0x7fb058105f80 2023-11-27T15:16:17.125 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:16:17.126 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":83,"fsid":"c6c7df88-8d36-11ee-95a2-87774f69a715","created":"2023-11-27T15:09:03.333345+0000","modified":"2023-11-27T15:16:10.321506+0000","last_up_change":"2023-11-27T15:15:27.656134+0000","last_in_change":"2023-11-27T15:15:10.671153+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:12:14.177523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:16:07.842251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:16:08.728520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58d69ede-c293-49f5-abbe-8187793f2b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6803","nonce":608216415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6805","nonce":608216415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6809","nonce":608216415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6807","nonce":608216415}]},"public_addr":"172.21.15.52:6803/608216415","cluster_addr":"172.21.15.52:6805/608216415","heartbeat_back_addr":"172.21.15.52:6809/608216415","heartbeat_front_addr":"172.21.15.52:6807/608216415","state":["exists","up"]},{"osd":1,"uuid":"d6a46b91-adc8-45ef-80f5-566ae400911f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6801","nonce":2538384087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6803","nonce":2538384087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6807","nonce":2538384087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6805","nonce":2538384087}]},"public_addr":"172.21.15.102:6801/2538384087","cluster_addr":"172.21.15.102:6803/2538384087","heartbeat_back_addr":"172.21.15.102:6807/2538384087","heartbeat_front_addr":"172.21.15.102:6805/2538384087","state":["exists","up"]},{"osd":2,"uuid":"846ba125-11f8-46b2-8941-bd5e1947cc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6801","nonce":4258038221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6803","nonce":4258038221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6807","nonce":4258038221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6805","nonce":4258038221}]},"public_addr":"172.21.15.150:6801/4258038221","cluster_addr":"172.21.15.150:6803/4258038221","heartbeat_back_addr":"172.21.15.150:6807/4258038221","heartbeat_front_addr":"172.21.15.150:6805/4258038221","state":["exists","up"]},{"osd":3,"uuid":"950fcd67-1057-4288-a389-714f1ec94261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6811","nonce":2622100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6813","nonce":2622100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6817","nonce":2622100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6815","nonce":2622100752}]},"public_addr":"172.21.15.52:6811/2622100752","cluster_addr":"172.21.15.52:6813/2622100752","heartbeat_back_addr":"172.21.15.52:6817/2622100752","heartbeat_front_addr":"172.21.15.52:6815/2622100752","state":["exists","up"]},{"osd":4,"uuid":"1c96a676-cea9-4deb-bad1-f7fd240c3d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6809","nonce":1546794763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6811","nonce":1546794763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6815","nonce":1546794763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6813","nonce":1546794763}]},"public_addr":"172.21.15.102:6809/1546794763","cluster_addr":"172.21.15.102:6811/1546794763","heartbeat_back_addr":"172.21.15.102:6815/1546794763","heartbeat_front_addr":"172.21.15.102:6813/1546794763","state":["exists","up"]},{"osd":5,"uuid":"38a75af1-fd4d-433e-b579-0609109c851c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6809","nonce":461511375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6811","nonce":461511375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6815","nonce":461511375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6813","nonce":461511375}]},"public_addr":"172.21.15.150:6809/461511375","cluster_addr":"172.21.15.150:6811/461511375","heartbeat_back_addr":"172.21.15.150:6815/461511375","heartbeat_front_addr":"172.21.15.150:6813/461511375","state":["exists","up"]},{"osd":6,"uuid":"2f0ede4a-d175-4b4a-b876-c0f442048901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6819","nonce":1002457464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6821","nonce":1002457464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6825","nonce":1002457464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6823","nonce":1002457464}]},"public_addr":"172.21.15.52:6819/1002457464","cluster_addr":"172.21.15.52:6821/1002457464","heartbeat_back_addr":"172.21.15.52:6825/1002457464","heartbeat_front_addr":"172.21.15.52:6823/1002457464","state":["exists","up"]},{"osd":7,"uuid":"6b635192-abcf-4bbf-bf78-a019b351bca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6817","nonce":3249840064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6819","nonce":3249840064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6823","nonce":3249840064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6821","nonce":3249840064}]},"public_addr":"172.21.15.102:6817/3249840064","cluster_addr":"172.21.15.102:6819/3249840064","heartbeat_back_addr":"172.21.15.102:6823/3249840064","heartbeat_front_addr":"172.21.15.102:6821/3249840064","state":["exists","up"]},{"osd":8,"uuid":"c7ca8007-7170-47bf-956c-3f00edf27e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6817","nonce":3672583574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6819","nonce":3672583574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6823","nonce":3672583574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6821","nonce":3672583574}]},"public_addr":"172.21.15.150:6817/3672583574","cluster_addr":"172.21.15.150:6819/3672583574","heartbeat_back_addr":"172.21.15.150:6823/3672583574","heartbeat_front_addr":"172.21.15.150:6821/3672583574","state":["exists","up"]},{"osd":9,"uuid":"0e252c2c-01a4-48ba-832d-067ed940fefb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6827","nonce":4287934320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6829","nonce":4287934320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6833","nonce":4287934320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6831","nonce":4287934320}]},"public_addr":"172.21.15.52:6827/4287934320","cluster_addr":"172.21.15.52:6829/4287934320","heartbeat_back_addr":"172.21.15.52:6833/4287934320","heartbeat_front_addr":"172.21.15.52:6831/4287934320","state":["exists","up"]},{"osd":10,"uuid":"bb540142-7563-44a4-b7a3-cb662714c128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6825","nonce":3056034201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6827","nonce":3056034201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6831","nonce":3056034201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6829","nonce":3056034201}]},"public_addr":"172.21.15.102:6825/3056034201","cluster_addr":"172.21.15.102:6827/3056034201","heartbeat_back_addr":"172.21.15.102:6831/3056034201","heartbeat_front_addr":"172.21.15.102:6829/3056034201","state":["exists","up"]},{"osd":11,"uuid":"3156528e-e0f8-4d44-9946-c8b6339c2bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6825","nonce":3207559739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6827","nonce":3207559739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6831","nonce":3207559739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6829","nonce":3207559739}]},"public_addr":"172.21.15.150:6825/3207559739","cluster_addr":"172.21.15.150:6827/3207559739","heartbeat_back_addr":"172.21.15.150:6831/3207559739","heartbeat_front_addr":"172.21.15.150:6829/3207559739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:26.826099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:48.331371+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:10.459738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:32.356472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:54.043170+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:15.648582+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:37.166463+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:59.240819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:20.710097+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:43.364563+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:05.279433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:26.344573+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/1392282110":"2023-11-28T15:10:03.644870+0000","172.21.15.52:0/1479993479":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6800/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/961562652":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/1290665390":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2410529742":"2023-11-28T15:09:37.118690+0000","172.21.15.52:6800/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2130037044":"2023-11-28T15:10:03.644870+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:16:17.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.123+0000 7fb05fefc700 1 -- 172.21.15.52:0/745563203 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb0440a1260 msgr2=0x7fb0440a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:17.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.123+0000 7fb05fefc700 1 --2- 172.21.15.52:0/745563203 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb0440a1260 0x7fb0440a3710 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb054005fd0 tx=0x7fb054009310 comp rx=0 tx=0).stop 2023-11-27T15:16:17.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.123+0000 7fb05fefc700 1 -- 172.21.15.52:0/745563203 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb058105f80 msgr2=0x7fb0580ff230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:17.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.123+0000 7fb05fefc700 1 --2- 172.21.15.52:0/745563203 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb058105f80 0x7fb0580ff230 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb05000ef30 tx=0x7fb05000c3a0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.123+0000 7fb05fefc700 1 -- 172.21.15.52:0/745563203 shutdown_connections 2023-11-27T15:16:17.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.123+0000 7fb05fefc700 1 --2- 172.21.15.52:0/745563203 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb0440a1260 0x7fb0440a3710 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.123+0000 7fb05fefc700 1 --2- 172.21.15.52:0/745563203 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb058105f80 0x7fb0580ff230 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.123+0000 7fb05fefc700 1 --2- 172.21.15.52:0/745563203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb058100d40 0x7fb0581006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.123+0000 7fb05fefc700 1 --2- 172.21.15.52:0/745563203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb058103660 0x7fb0580fecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.123+0000 7fb05fefc700 1 -- 172.21.15.52:0/745563203 >> 172.21.15.52:0/745563203 conn(0x7fb0580fa7c0 msgr2=0x7fb0581019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:17.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.123+0000 7fb05fefc700 1 -- 172.21.15.52:0/745563203 shutdown_connections 2023-11-27T15:16:17.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.123+0000 7fb05fefc700 1 -- 172.21.15.52:0/745563203 wait complete. 2023-11-27T15:16:17.140 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-27T15:16:17.140 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-27T15:16:17.268 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.262+0000 7f178ed69700 1 -- 172.21.15.52:0/795254602 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17880ff170 msgr2=0x7f17880ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:17.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.262+0000 7f178ed69700 1 --2- 172.21.15.52:0/795254602 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17880ff170 0x7f17880ff580 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f177c009990 tx=0x7f177c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.262+0000 7f178ed69700 1 -- 172.21.15.52:0/795254602 shutdown_connections 2023-11-27T15:16:17.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.262+0000 7f178ed69700 1 --2- 172.21.15.52:0/795254602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1788100db0 0x7f1788107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.262+0000 7f178ed69700 1 --2- 172.21.15.52:0/795254602 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17880ff170 0x7f17880ff580 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.262+0000 7f178ed69700 1 --2- 172.21.15.52:0/795254602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1788100400 0x7f1788100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.262+0000 7f178ed69700 1 -- 172.21.15.52:0/795254602 >> 172.21.15.52:0/795254602 conn(0x7f17880fa750 msgr2=0x7f17880fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:17.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.262+0000 7f178ed69700 1 -- 172.21.15.52:0/795254602 shutdown_connections 2023-11-27T15:16:17.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.262+0000 7f178ed69700 1 -- 172.21.15.52:0/795254602 wait complete. 2023-11-27T15:16:17.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.263+0000 7f178ed69700 1 Processor -- start 2023-11-27T15:16:17.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.263+0000 7f178ed69700 1 -- start start 2023-11-27T15:16:17.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.263+0000 7f178ed69700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17880ff170 0x7f1788195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:17.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.263+0000 7f178ed69700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1788100400 0x7f1788196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:17.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.263+0000 7f178ed69700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1788100db0 0x7f178819d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:17.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.263+0000 7f178ed69700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178819dac0 con 0x7f1788100400 2023-11-27T15:16:17.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.263+0000 7f178ed69700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178819dc00 con 0x7f17880ff170 2023-11-27T15:16:17.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.263+0000 7f178ed69700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178819dd90 con 0x7f1788100db0 2023-11-27T15:16:17.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.263+0000 7f1787fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1788100400 0x7f1788196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:17.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.263+0000 7f1787fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1788100400 0x7f1788196480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50216/0 (socket says 172.21.15.52:50216) 2023-11-27T15:16:17.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.263+0000 7f1787fff700 1 -- 172.21.15.52:0/3230839657 learned_addr learned my addr 172.21.15.52:0/3230839657 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:17.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.263+0000 7f178d306700 1 --2- 172.21.15.52:0/3230839657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1788100db0 0x7f178819d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:17.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.263+0000 7f178cb05700 1 --2- 172.21.15.52:0/3230839657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17880ff170 0x7f1788195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:17.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.263+0000 7f1787fff700 1 -- 172.21.15.52:0/3230839657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1788100db0 msgr2=0x7f178819d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:17.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.263+0000 7f1787fff700 1 --2- 172.21.15.52:0/3230839657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1788100db0 0x7f178819d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.263+0000 7f1787fff700 1 -- 172.21.15.52:0/3230839657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17880ff170 msgr2=0x7f1788195f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:17.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.263+0000 7f1787fff700 1 --2- 172.21.15.52:0/3230839657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17880ff170 0x7f1788195f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.263+0000 7f1787fff700 1 -- 172.21.15.52:0/3230839657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f177c0095f0 con 0x7f1788100400 2023-11-27T15:16:17.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.263+0000 7f178d306700 1 --2- 172.21.15.52:0/3230839657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1788100db0 0x7f178819d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:16:17.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.263+0000 7f178cb05700 1 --2- 172.21.15.52:0/3230839657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17880ff170 0x7f1788195f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:16:17.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.263+0000 7f1787fff700 1 --2- 172.21.15.52:0/3230839657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1788100400 0x7f1788196480 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f177400d980 tx=0x7f177400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:17.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.263+0000 7f1785ffb700 1 -- 172.21.15.52:0/3230839657 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177400bb60 con 0x7f1788100400 2023-11-27T15:16:17.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.264+0000 7f1785ffb700 1 -- 172.21.15.52:0/3230839657 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1774007640 con 0x7f1788100400 2023-11-27T15:16:17.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.264+0000 7f178ed69700 1 -- 172.21.15.52:0/3230839657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f178810ab10 con 0x7f1788100400 2023-11-27T15:16:17.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.264+0000 7f1785ffb700 1 -- 172.21.15.52:0/3230839657 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1774004e90 con 0x7f1788100400 2023-11-27T15:16:17.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.264+0000 7f178ed69700 1 -- 172.21.15.52:0/3230839657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f178810a980 con 0x7f1788100400 2023-11-27T15:16:17.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.265+0000 7f178ed69700 1 -- 172.21.15.52:0/3230839657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17881073c0 con 0x7f1788100400 2023-11-27T15:16:17.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.266+0000 7f1785ffb700 1 -- 172.21.15.52:0/3230839657 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f177400bcc0 con 0x7f1788100400 2023-11-27T15:16:17.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.266+0000 7f1785ffb700 1 --2- 172.21.15.52:0/3230839657 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f17780a11e0 0x7f17780a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:17.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.266+0000 7f178cb05700 1 --2- 172.21.15.52:0/3230839657 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f17780a11e0 0x7f17780a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:17.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.266+0000 7f1785ffb700 1 -- 172.21.15.52:0/3230839657 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f17740cc7f0 con 0x7f1788100400 2023-11-27T15:16:17.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.267+0000 7f178cb05700 1 --2- 172.21.15.52:0/3230839657 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f17780a11e0 0x7f17780a3690 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f177c00b5c0 tx=0x7f177c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:17.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.270+0000 7f1785ffb700 1 -- 172.21.15.52:0/3230839657 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1774099a20 con 0x7f1788100400 2023-11-27T15:16:17.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.470+0000 7f178ed69700 1 -- 172.21.15.52:0/3230839657 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1788061a30 con 0x7f17780a11e0 2023-11-27T15:16:17.487 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.482+0000 7f1785ffb700 1 -- 172.21.15.52:0/3230839657 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+334931 (secure 0 0 0) 0x7f1788061a30 con 0x7f17780a11e0 2023-11-27T15:16:17.491 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:16:17.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.486+0000 7f178ed69700 1 -- 172.21.15.52:0/3230839657 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f17780a11e0 msgr2=0x7f17780a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:17.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.486+0000 7f178ed69700 1 --2- 172.21.15.52:0/3230839657 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f17780a11e0 0x7f17780a3690 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f177c00b5c0 tx=0x7f177c005fb0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.486+0000 7f178ed69700 1 -- 172.21.15.52:0/3230839657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1788100400 msgr2=0x7f1788196480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:17.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.486+0000 7f178ed69700 1 --2- 172.21.15.52:0/3230839657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1788100400 0x7f1788196480 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f177400d980 tx=0x7f177400dd40 comp rx=0 tx=0).stop 2023-11-27T15:16:17.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.486+0000 7f178ed69700 1 -- 172.21.15.52:0/3230839657 shutdown_connections 2023-11-27T15:16:17.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.486+0000 7f178ed69700 1 --2- 172.21.15.52:0/3230839657 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f17780a11e0 0x7f17780a3690 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.486+0000 7f178ed69700 1 --2- 172.21.15.52:0/3230839657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17880ff170 0x7f1788195f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.486+0000 7f178ed69700 1 --2- 172.21.15.52:0/3230839657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1788100db0 0x7f178819d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.486+0000 7f178ed69700 1 --2- 172.21.15.52:0/3230839657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1788100400 0x7f1788196480 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.486+0000 7f178ed69700 1 -- 172.21.15.52:0/3230839657 >> 172.21.15.52:0/3230839657 conn(0x7f17880fa750 msgr2=0x7f17880fcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:17.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.487+0000 7f178ed69700 1 -- 172.21.15.52:0/3230839657 shutdown_connections 2023-11-27T15:16:17.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.487+0000 7f178ed69700 1 -- 172.21.15.52:0/3230839657 wait complete. 2023-11-27T15:16:17.495 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2023-11-27T15:16:17.503 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2023-11-27T15:16:15.722641+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594178,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":262,"num_write_kb":2959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":230,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":77860,"kb_used_data":10692,"kb_used_omap":0,"kb_used_meta":67072,"kb_avail":1124863964,"statfs":{"total":1151940427776,"available":1151860699136,"internally_reserved":0,"allocated":10948608,"data_stored":6904567,"data_compressed":108282,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":68681728},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3810,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.825918"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329249+0000","last_change":"2023-11-27T15:16:09.343007+0000","last_active":"2023-11-27T15:16:10.329249+0000","last_peered":"2023-11-27T15:16:10.329249+0000","last_clean":"2023-11-27T15:16:10.329249+0000","last_became_active":"2023-11-27T15:16:09.342125+0000","last_became_peered":"2023-11-27T15:16:09.342125+0000","last_unstale":"2023-11-27T15:16:10.329249+0000","last_undegraded":"2023-11-27T15:16:10.329249+0000","last_fullsized":"2023-11-27T15:16:10.329249+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:10:56.620347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.865187+0000","last_change":"2023-11-27T15:16:09.338942+0000","last_active":"2023-11-27T15:16:10.865187+0000","last_peered":"2023-11-27T15:16:10.865187+0000","last_clean":"2023-11-27T15:16:10.865187+0000","last_became_active":"2023-11-27T15:16:09.338636+0000","last_became_peered":"2023-11-27T15:16:09.338636+0000","last_unstale":"2023-11-27T15:16:10.865187+0000","last_undegraded":"2023-11-27T15:16:10.865187+0000","last_fullsized":"2023-11-27T15:16:10.865187+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:30:06.448581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.865094+0000","last_change":"2023-11-27T15:16:09.347845+0000","last_active":"2023-11-27T15:16:10.865094+0000","last_peered":"2023-11-27T15:16:10.865094+0000","last_clean":"2023-11-27T15:16:10.865094+0000","last_became_active":"2023-11-27T15:16:09.347533+0000","last_became_peered":"2023-11-27T15:16:09.347533+0000","last_unstale":"2023-11-27T15:16:10.865094+0000","last_undegraded":"2023-11-27T15:16:10.865094+0000","last_fullsized":"2023-11-27T15:16:10.865094+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:01:40.820155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.019299+0000","last_change":"2023-11-27T15:16:09.333251+0000","last_active":"2023-11-27T15:16:11.019299+0000","last_peered":"2023-11-27T15:16:11.019299+0000","last_clean":"2023-11-27T15:16:11.019299+0000","last_became_active":"2023-11-27T15:16:09.331510+0000","last_became_peered":"2023-11-27T15:16:09.331510+0000","last_unstale":"2023-11-27T15:16:11.019299+0000","last_undegraded":"2023-11-27T15:16:11.019299+0000","last_fullsized":"2023-11-27T15:16:11.019299+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:34:53.253074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018096+0000","last_change":"2023-11-27T15:16:09.349097+0000","last_active":"2023-11-27T15:16:11.018096+0000","last_peered":"2023-11-27T15:16:11.018096+0000","last_clean":"2023-11-27T15:16:11.018096+0000","last_became_active":"2023-11-27T15:16:09.348793+0000","last_became_peered":"2023-11-27T15:16:09.348793+0000","last_unstale":"2023-11-27T15:16:11.018096+0000","last_undegraded":"2023-11-27T15:16:11.018096+0000","last_fullsized":"2023-11-27T15:16:11.018096+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:55:31.214202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.019174+0000","last_change":"2023-11-27T15:16:09.338246+0000","last_active":"2023-11-27T15:16:11.019174+0000","last_peered":"2023-11-27T15:16:11.019174+0000","last_clean":"2023-11-27T15:16:11.019174+0000","last_became_active":"2023-11-27T15:16:09.338003+0000","last_became_peered":"2023-11-27T15:16:09.338003+0000","last_unstale":"2023-11-27T15:16:11.019174+0000","last_undegraded":"2023-11-27T15:16:11.019174+0000","last_fullsized":"2023-11-27T15:16:11.019174+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:25:15.821446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017705+0000","last_change":"2023-11-27T15:16:09.342131+0000","last_active":"2023-11-27T15:16:11.017705+0000","last_peered":"2023-11-27T15:16:11.017705+0000","last_clean":"2023-11-27T15:16:11.017705+0000","last_became_active":"2023-11-27T15:16:09.341834+0000","last_became_peered":"2023-11-27T15:16:09.341834+0000","last_unstale":"2023-11-27T15:16:11.017705+0000","last_undegraded":"2023-11-27T15:16:11.017705+0000","last_fullsized":"2023-11-27T15:16:11.017705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:06:07.633604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018355+0000","last_change":"2023-11-27T15:16:09.337308+0000","last_active":"2023-11-27T15:16:11.018355+0000","last_peered":"2023-11-27T15:16:11.018355+0000","last_clean":"2023-11-27T15:16:11.018355+0000","last_became_active":"2023-11-27T15:16:09.336958+0000","last_became_peered":"2023-11-27T15:16:09.336958+0000","last_unstale":"2023-11-27T15:16:11.018355+0000","last_undegraded":"2023-11-27T15:16:11.018355+0000","last_fullsized":"2023-11-27T15:16:11.018355+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:00:54.104008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.332789+0000","last_change":"2023-11-27T15:16:09.340646+0000","last_active":"2023-11-27T15:16:10.332789+0000","last_peered":"2023-11-27T15:16:10.332789+0000","last_clean":"2023-11-27T15:16:10.332789+0000","last_became_active":"2023-11-27T15:16:09.340084+0000","last_became_peered":"2023-11-27T15:16:09.340084+0000","last_unstale":"2023-11-27T15:16:10.332789+0000","last_undegraded":"2023-11-27T15:16:10.332789+0000","last_fullsized":"2023-11-27T15:16:10.332789+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:33:29.581983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.019348+0000","last_change":"2023-11-27T15:16:09.336871+0000","last_active":"2023-11-27T15:16:11.019348+0000","last_peered":"2023-11-27T15:16:11.019348+0000","last_clean":"2023-11-27T15:16:11.019348+0000","last_became_active":"2023-11-27T15:16:09.336496+0000","last_became_peered":"2023-11-27T15:16:09.336496+0000","last_unstale":"2023-11-27T15:16:11.019348+0000","last_undegraded":"2023-11-27T15:16:11.019348+0000","last_fullsized":"2023-11-27T15:16:11.019348+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:37:00.015095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018026+0000","last_change":"2023-11-27T15:16:09.343120+0000","last_active":"2023-11-27T15:16:11.018026+0000","last_peered":"2023-11-27T15:16:11.018026+0000","last_clean":"2023-11-27T15:16:11.018026+0000","last_became_active":"2023-11-27T15:16:09.342797+0000","last_became_peered":"2023-11-27T15:16:09.342797+0000","last_unstale":"2023-11-27T15:16:11.018026+0000","last_undegraded":"2023-11-27T15:16:11.018026+0000","last_fullsized":"2023-11-27T15:16:11.018026+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:39:21.071837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.019248+0000","last_change":"2023-11-27T15:16:09.336521+0000","last_active":"2023-11-27T15:16:11.019248+0000","last_peered":"2023-11-27T15:16:11.019248+0000","last_clean":"2023-11-27T15:16:11.019248+0000","last_became_active":"2023-11-27T15:16:09.336183+0000","last_became_peered":"2023-11-27T15:16:09.336183+0000","last_unstale":"2023-11-27T15:16:11.019248+0000","last_undegraded":"2023-11-27T15:16:11.019248+0000","last_fullsized":"2023-11-27T15:16:11.019248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:25:55.091755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018282+0000","last_change":"2023-11-27T15:16:09.346839+0000","last_active":"2023-11-27T15:16:11.018282+0000","last_peered":"2023-11-27T15:16:11.018282+0000","last_clean":"2023-11-27T15:16:11.018282+0000","last_became_active":"2023-11-27T15:16:09.345230+0000","last_became_peered":"2023-11-27T15:16:09.345230+0000","last_unstale":"2023-11-27T15:16:11.018282+0000","last_undegraded":"2023-11-27T15:16:11.018282+0000","last_fullsized":"2023-11-27T15:16:11.018282+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:22:56.772626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.864881+0000","last_change":"2023-11-27T15:16:09.342380+0000","last_active":"2023-11-27T15:16:10.864881+0000","last_peered":"2023-11-27T15:16:10.864881+0000","last_clean":"2023-11-27T15:16:10.864881+0000","last_became_active":"2023-11-27T15:16:09.341967+0000","last_became_peered":"2023-11-27T15:16:09.341967+0000","last_unstale":"2023-11-27T15:16:10.864881+0000","last_undegraded":"2023-11-27T15:16:10.864881+0000","last_fullsized":"2023-11-27T15:16:10.864881+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:35:13.999971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.864779+0000","last_change":"2023-11-27T15:16:09.349738+0000","last_active":"2023-11-27T15:16:10.864779+0000","last_peered":"2023-11-27T15:16:10.864779+0000","last_clean":"2023-11-27T15:16:10.864779+0000","last_became_active":"2023-11-27T15:16:09.349349+0000","last_became_peered":"2023-11-27T15:16:09.349349+0000","last_unstale":"2023-11-27T15:16:10.864779+0000","last_undegraded":"2023-11-27T15:16:10.864779+0000","last_fullsized":"2023-11-27T15:16:10.864779+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:13:03.443885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.332227+0000","last_change":"2023-11-27T15:16:09.339171+0000","last_active":"2023-11-27T15:16:10.332227+0000","last_peered":"2023-11-27T15:16:10.332227+0000","last_clean":"2023-11-27T15:16:10.332227+0000","last_became_active":"2023-11-27T15:16:09.338887+0000","last_became_peered":"2023-11-27T15:16:09.338887+0000","last_unstale":"2023-11-27T15:16:10.332227+0000","last_undegraded":"2023-11-27T15:16:10.332227+0000","last_fullsized":"2023-11-27T15:16:10.332227+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:42:45.286967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329413+0000","last_change":"2023-11-27T15:16:09.343202+0000","last_active":"2023-11-27T15:16:10.329413+0000","last_peered":"2023-11-27T15:16:10.329413+0000","last_clean":"2023-11-27T15:16:10.329413+0000","last_became_active":"2023-11-27T15:16:09.342539+0000","last_became_peered":"2023-11-27T15:16:09.342539+0000","last_unstale":"2023-11-27T15:16:10.329413+0000","last_undegraded":"2023-11-27T15:16:10.329413+0000","last_fullsized":"2023-11-27T15:16:10.329413+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:31:31.313017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018587+0000","last_change":"2023-11-27T15:16:09.339915+0000","last_active":"2023-11-27T15:16:11.018587+0000","last_peered":"2023-11-27T15:16:11.018587+0000","last_clean":"2023-11-27T15:16:11.018587+0000","last_became_active":"2023-11-27T15:16:09.339637+0000","last_became_peered":"2023-11-27T15:16:09.339637+0000","last_unstale":"2023-11-27T15:16:11.018587+0000","last_undegraded":"2023-11-27T15:16:11.018587+0000","last_fullsized":"2023-11-27T15:16:11.018587+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:37:13.698277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329316+0000","last_change":"2023-11-27T15:16:09.347852+0000","last_active":"2023-11-27T15:16:10.329316+0000","last_peered":"2023-11-27T15:16:10.329316+0000","last_clean":"2023-11-27T15:16:10.329316+0000","last_became_active":"2023-11-27T15:16:09.347281+0000","last_became_peered":"2023-11-27T15:16:09.347281+0000","last_unstale":"2023-11-27T15:16:10.329316+0000","last_undegraded":"2023-11-27T15:16:10.329316+0000","last_fullsized":"2023-11-27T15:16:10.329316+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:54:00.346911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018790+0000","last_change":"2023-11-27T15:16:08.912206+0000","last_active":"2023-11-27T15:16:11.018790+0000","last_peered":"2023-11-27T15:16:11.018790+0000","last_clean":"2023-11-27T15:16:11.018790+0000","last_became_active":"2023-11-27T15:16:08.911846+0000","last_became_peered":"2023-11-27T15:16:08.911846+0000","last_unstale":"2023-11-27T15:16:11.018790+0000","last_undegraded":"2023-11-27T15:16:11.018790+0000","last_fullsized":"2023-11-27T15:16:11.018790+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:21:37.095544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018680+0000","last_change":"2023-11-27T15:16:09.336485+0000","last_active":"2023-11-27T15:16:11.018680+0000","last_peered":"2023-11-27T15:16:11.018680+0000","last_clean":"2023-11-27T15:16:11.018680+0000","last_became_active":"2023-11-27T15:16:09.336236+0000","last_became_peered":"2023-11-27T15:16:09.336236+0000","last_unstale":"2023-11-27T15:16:11.018680+0000","last_undegraded":"2023-11-27T15:16:11.018680+0000","last_fullsized":"2023-11-27T15:16:11.018680+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:00:31.457583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018872+0000","last_change":"2023-11-27T15:16:08.921198+0000","last_active":"2023-11-27T15:16:11.018872+0000","last_peered":"2023-11-27T15:16:11.018872+0000","last_clean":"2023-11-27T15:16:11.018872+0000","last_became_active":"2023-11-27T15:16:08.920804+0000","last_became_peered":"2023-11-27T15:16:08.920804+0000","last_unstale":"2023-11-27T15:16:11.018872+0000","last_undegraded":"2023-11-27T15:16:11.018872+0000","last_fullsized":"2023-11-27T15:16:11.018872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:25:26.133784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.865444+0000","last_change":"2023-11-27T15:16:09.340642+0000","last_active":"2023-11-27T15:16:10.865444+0000","last_peered":"2023-11-27T15:16:10.865444+0000","last_clean":"2023-11-27T15:16:10.865444+0000","last_became_active":"2023-11-27T15:16:09.340328+0000","last_became_peered":"2023-11-27T15:16:09.340328+0000","last_unstale":"2023-11-27T15:16:10.865444+0000","last_undegraded":"2023-11-27T15:16:10.865444+0000","last_fullsized":"2023-11-27T15:16:10.865444+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:48:12.279600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018832+0000","last_change":"2023-11-27T15:16:08.915317+0000","last_active":"2023-11-27T15:16:11.018832+0000","last_peered":"2023-11-27T15:16:11.018832+0000","last_clean":"2023-11-27T15:16:11.018832+0000","last_became_active":"2023-11-27T15:16:08.914949+0000","last_became_peered":"2023-11-27T15:16:08.914949+0000","last_unstale":"2023-11-27T15:16:11.018832+0000","last_undegraded":"2023-11-27T15:16:11.018832+0000","last_fullsized":"2023-11-27T15:16:11.018832+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:35:47.443716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.332402+0000","last_change":"2023-11-27T15:16:09.348483+0000","last_active":"2023-11-27T15:16:10.332402+0000","last_peered":"2023-11-27T15:16:10.332402+0000","last_clean":"2023-11-27T15:16:10.332402+0000","last_became_active":"2023-11-27T15:16:09.348185+0000","last_became_peered":"2023-11-27T15:16:09.348185+0000","last_unstale":"2023-11-27T15:16:10.332402+0000","last_undegraded":"2023-11-27T15:16:10.332402+0000","last_fullsized":"2023-11-27T15:16:10.332402+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:23:59.373819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018251+0000","last_change":"2023-11-27T15:16:08.911018+0000","last_active":"2023-11-27T15:16:11.018251+0000","last_peered":"2023-11-27T15:16:11.018251+0000","last_clean":"2023-11-27T15:16:11.018251+0000","last_became_active":"2023-11-27T15:16:08.910684+0000","last_became_peered":"2023-11-27T15:16:08.910684+0000","last_unstale":"2023-11-27T15:16:11.018251+0000","last_undegraded":"2023-11-27T15:16:11.018251+0000","last_fullsized":"2023-11-27T15:16:11.018251+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:17:37.062173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018419+0000","last_change":"2023-11-27T15:16:09.341414+0000","last_active":"2023-11-27T15:16:11.018419+0000","last_peered":"2023-11-27T15:16:11.018419+0000","last_clean":"2023-11-27T15:16:11.018419+0000","last_became_active":"2023-11-27T15:16:09.341068+0000","last_became_peered":"2023-11-27T15:16:09.341068+0000","last_unstale":"2023-11-27T15:16:11.018419+0000","last_undegraded":"2023-11-27T15:16:11.018419+0000","last_fullsized":"2023-11-27T15:16:11.018419+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:08:37.252539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.743053+0000","last_change":"2023-11-27T15:16:08.915093+0000","last_active":"2023-11-27T15:16:12.743053+0000","last_peered":"2023-11-27T15:16:12.743053+0000","last_clean":"2023-11-27T15:16:12.743053+0000","last_became_active":"2023-11-27T15:16:08.914761+0000","last_became_peered":"2023-11-27T15:16:08.914761+0000","last_unstale":"2023-11-27T15:16:12.743053+0000","last_undegraded":"2023-11-27T15:16:12.743053+0000","last_fullsized":"2023-11-27T15:16:12.743053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:36:34.653519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018143+0000","last_change":"2023-11-27T15:16:09.337330+0000","last_active":"2023-11-27T15:16:11.018143+0000","last_peered":"2023-11-27T15:16:11.018143+0000","last_clean":"2023-11-27T15:16:11.018143+0000","last_became_active":"2023-11-27T15:16:09.336991+0000","last_became_peered":"2023-11-27T15:16:09.336991+0000","last_unstale":"2023-11-27T15:16:11.018143+0000","last_undegraded":"2023-11-27T15:16:11.018143+0000","last_fullsized":"2023-11-27T15:16:11.018143+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:12:42.839632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018993+0000","last_change":"2023-11-27T15:16:08.912208+0000","last_active":"2023-11-27T15:16:11.018993+0000","last_peered":"2023-11-27T15:16:11.018993+0000","last_clean":"2023-11-27T15:16:11.018993+0000","last_became_active":"2023-11-27T15:16:08.911879+0000","last_became_peered":"2023-11-27T15:16:08.911879+0000","last_unstale":"2023-11-27T15:16:11.018993+0000","last_undegraded":"2023-11-27T15:16:11.018993+0000","last_fullsized":"2023-11-27T15:16:11.018993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:05:04.291584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018417+0000","last_change":"2023-11-27T15:16:09.339660+0000","last_active":"2023-11-27T15:16:11.018417+0000","last_peered":"2023-11-27T15:16:11.018417+0000","last_clean":"2023-11-27T15:16:11.018417+0000","last_became_active":"2023-11-27T15:16:09.339323+0000","last_became_peered":"2023-11-27T15:16:09.339323+0000","last_unstale":"2023-11-27T15:16:11.018417+0000","last_undegraded":"2023-11-27T15:16:11.018417+0000","last_fullsized":"2023-11-27T15:16:11.018417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:54:38.570422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:13.750343+0000","last_change":"2023-11-27T15:16:08.911103+0000","last_active":"2023-11-27T15:16:13.750343+0000","last_peered":"2023-11-27T15:16:13.750343+0000","last_clean":"2023-11-27T15:16:13.750343+0000","last_became_active":"2023-11-27T15:16:08.910765+0000","last_became_peered":"2023-11-27T15:16:08.910765+0000","last_unstale":"2023-11-27T15:16:13.750343+0000","last_undegraded":"2023-11-27T15:16:13.750343+0000","last_fullsized":"2023-11-27T15:16:13.750343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:35:58.046428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018483+0000","last_change":"2023-11-27T15:16:09.351741+0000","last_active":"2023-11-27T15:16:11.018483+0000","last_peered":"2023-11-27T15:16:11.018483+0000","last_clean":"2023-11-27T15:16:11.018483+0000","last_became_active":"2023-11-27T15:16:09.351280+0000","last_became_peered":"2023-11-27T15:16:09.351280+0000","last_unstale":"2023-11-27T15:16:11.018483+0000","last_undegraded":"2023-11-27T15:16:11.018483+0000","last_fullsized":"2023-11-27T15:16:11.018483+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:23:19.955510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018060+0000","last_change":"2023-11-27T15:16:08.915630+0000","last_active":"2023-11-27T15:16:11.018060+0000","last_peered":"2023-11-27T15:16:11.018060+0000","last_clean":"2023-11-27T15:16:11.018060+0000","last_became_active":"2023-11-27T15:16:08.915294+0000","last_became_peered":"2023-11-27T15:16:08.915294+0000","last_unstale":"2023-11-27T15:16:11.018060+0000","last_undegraded":"2023-11-27T15:16:11.018060+0000","last_fullsized":"2023-11-27T15:16:11.018060+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:48:25.055189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018460+0000","last_change":"2023-11-27T15:16:09.338436+0000","last_active":"2023-11-27T15:16:11.018460+0000","last_peered":"2023-11-27T15:16:11.018460+0000","last_clean":"2023-11-27T15:16:11.018460+0000","last_became_active":"2023-11-27T15:16:09.338148+0000","last_became_peered":"2023-11-27T15:16:09.338148+0000","last_unstale":"2023-11-27T15:16:11.018460+0000","last_undegraded":"2023-11-27T15:16:11.018460+0000","last_fullsized":"2023-11-27T15:16:11.018460+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:11:53.119360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.743319+0000","last_change":"2023-11-27T15:16:08.913072+0000","last_active":"2023-11-27T15:16:12.743319+0000","last_peered":"2023-11-27T15:16:12.743319+0000","last_clean":"2023-11-27T15:16:12.743319+0000","last_became_active":"2023-11-27T15:16:08.912757+0000","last_became_peered":"2023-11-27T15:16:08.912757+0000","last_unstale":"2023-11-27T15:16:12.743319+0000","last_undegraded":"2023-11-27T15:16:12.743319+0000","last_fullsized":"2023-11-27T15:16:12.743319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:41:31.325128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.332666+0000","last_change":"2023-11-27T15:16:09.340380+0000","last_active":"2023-11-27T15:16:10.332666+0000","last_peered":"2023-11-27T15:16:10.332666+0000","last_clean":"2023-11-27T15:16:10.332666+0000","last_became_active":"2023-11-27T15:16:09.339581+0000","last_became_peered":"2023-11-27T15:16:09.339581+0000","last_unstale":"2023-11-27T15:16:10.332666+0000","last_undegraded":"2023-11-27T15:16:10.332666+0000","last_fullsized":"2023-11-27T15:16:10.332666+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:00:41.810407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017986+0000","last_change":"2023-11-27T15:16:08.917924+0000","last_active":"2023-11-27T15:16:11.017986+0000","last_peered":"2023-11-27T15:16:11.017986+0000","last_clean":"2023-11-27T15:16:11.017986+0000","last_became_active":"2023-11-27T15:16:08.917618+0000","last_became_peered":"2023-11-27T15:16:08.917618+0000","last_unstale":"2023-11-27T15:16:11.017986+0000","last_undegraded":"2023-11-27T15:16:11.017986+0000","last_fullsized":"2023-11-27T15:16:11.017986+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:05:48.700431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.865352+0000","last_change":"2023-11-27T15:16:09.346020+0000","last_active":"2023-11-27T15:16:10.865352+0000","last_peered":"2023-11-27T15:16:10.865352+0000","last_clean":"2023-11-27T15:16:10.865352+0000","last_became_active":"2023-11-27T15:16:09.345822+0000","last_became_peered":"2023-11-27T15:16:09.345822+0000","last_unstale":"2023-11-27T15:16:10.865352+0000","last_undegraded":"2023-11-27T15:16:10.865352+0000","last_fullsized":"2023-11-27T15:16:10.865352+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:20:28.114199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018365+0000","last_change":"2023-11-27T15:16:08.921810+0000","last_active":"2023-11-27T15:16:11.018365+0000","last_peered":"2023-11-27T15:16:11.018365+0000","last_clean":"2023-11-27T15:16:11.018365+0000","last_became_active":"2023-11-27T15:16:08.921518+0000","last_became_peered":"2023-11-27T15:16:08.921518+0000","last_unstale":"2023-11-27T15:16:11.018365+0000","last_undegraded":"2023-11-27T15:16:11.018365+0000","last_fullsized":"2023-11-27T15:16:11.018365+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:14:14.359239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.332512+0000","last_change":"2023-11-27T15:16:09.351083+0000","last_active":"2023-11-27T15:16:10.332512+0000","last_peered":"2023-11-27T15:16:10.332512+0000","last_clean":"2023-11-27T15:16:10.332512+0000","last_became_active":"2023-11-27T15:16:09.350540+0000","last_became_peered":"2023-11-27T15:16:09.350540+0000","last_unstale":"2023-11-27T15:16:10.332512+0000","last_undegraded":"2023-11-27T15:16:10.332512+0000","last_fullsized":"2023-11-27T15:16:10.332512+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:16:25.645689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.743377+0000","last_change":"2023-11-27T15:16:08.913023+0000","last_active":"2023-11-27T15:16:12.743377+0000","last_peered":"2023-11-27T15:16:12.743377+0000","last_clean":"2023-11-27T15:16:12.743377+0000","last_became_active":"2023-11-27T15:16:08.912693+0000","last_became_peered":"2023-11-27T15:16:08.912693+0000","last_unstale":"2023-11-27T15:16:12.743377+0000","last_undegraded":"2023-11-27T15:16:12.743377+0000","last_fullsized":"2023-11-27T15:16:12.743377+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:02:06.453103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018683+0000","last_change":"2023-11-27T15:16:09.343446+0000","last_active":"2023-11-27T15:16:11.018683+0000","last_peered":"2023-11-27T15:16:11.018683+0000","last_clean":"2023-11-27T15:16:11.018683+0000","last_became_active":"2023-11-27T15:16:09.341980+0000","last_became_peered":"2023-11-27T15:16:09.341980+0000","last_unstale":"2023-11-27T15:16:11.018683+0000","last_undegraded":"2023-11-27T15:16:11.018683+0000","last_fullsized":"2023-11-27T15:16:11.018683+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:03:09.462489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.745246+0000","last_change":"2023-11-27T15:16:08.914181+0000","last_active":"2023-11-27T15:16:12.745246+0000","last_peered":"2023-11-27T15:16:12.745246+0000","last_clean":"2023-11-27T15:16:12.745246+0000","last_became_active":"2023-11-27T15:16:08.913836+0000","last_became_peered":"2023-11-27T15:16:08.913836+0000","last_unstale":"2023-11-27T15:16:12.745246+0000","last_undegraded":"2023-11-27T15:16:12.745246+0000","last_fullsized":"2023-11-27T15:16:12.745246+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:28:48.961843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018010+0000","last_change":"2023-11-27T15:16:09.344446+0000","last_active":"2023-11-27T15:16:11.018010+0000","last_peered":"2023-11-27T15:16:11.018010+0000","last_clean":"2023-11-27T15:16:11.018010+0000","last_became_active":"2023-11-27T15:16:09.344104+0000","last_became_peered":"2023-11-27T15:16:09.344104+0000","last_unstale":"2023-11-27T15:16:11.018010+0000","last_undegraded":"2023-11-27T15:16:11.018010+0000","last_fullsized":"2023-11-27T15:16:11.018010+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:09:38.847885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.744575+0000","last_change":"2023-11-27T15:16:08.915597+0000","last_active":"2023-11-27T15:16:12.744575+0000","last_peered":"2023-11-27T15:16:12.744575+0000","last_clean":"2023-11-27T15:16:12.744575+0000","last_became_active":"2023-11-27T15:16:08.914727+0000","last_became_peered":"2023-11-27T15:16:08.914727+0000","last_unstale":"2023-11-27T15:16:12.744575+0000","last_undegraded":"2023-11-27T15:16:12.744575+0000","last_fullsized":"2023-11-27T15:16:12.744575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:32:36.396885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017899+0000","last_change":"2023-11-27T15:16:09.352643+0000","last_active":"2023-11-27T15:16:11.017899+0000","last_peered":"2023-11-27T15:16:11.017899+0000","last_clean":"2023-11-27T15:16:11.017899+0000","last_became_active":"2023-11-27T15:16:09.352336+0000","last_became_peered":"2023-11-27T15:16:09.352336+0000","last_unstale":"2023-11-27T15:16:11.017899+0000","last_undegraded":"2023-11-27T15:16:11.017899+0000","last_fullsized":"2023-11-27T15:16:11.017899+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:27:00.130120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.019339+0000","last_change":"2023-11-27T15:16:08.914303+0000","last_active":"2023-11-27T15:16:11.019339+0000","last_peered":"2023-11-27T15:16:11.019339+0000","last_clean":"2023-11-27T15:16:11.019339+0000","last_became_active":"2023-11-27T15:16:08.913601+0000","last_became_peered":"2023-11-27T15:16:08.913601+0000","last_unstale":"2023-11-27T15:16:11.019339+0000","last_undegraded":"2023-11-27T15:16:11.019339+0000","last_fullsized":"2023-11-27T15:16:11.019339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:35:53.186395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017672+0000","last_change":"2023-11-27T15:16:09.341342+0000","last_active":"2023-11-27T15:16:11.017672+0000","last_peered":"2023-11-27T15:16:11.017672+0000","last_clean":"2023-11-27T15:16:11.017672+0000","last_became_active":"2023-11-27T15:16:09.341100+0000","last_became_peered":"2023-11-27T15:16:09.341100+0000","last_unstale":"2023-11-27T15:16:11.017672+0000","last_undegraded":"2023-11-27T15:16:11.017672+0000","last_fullsized":"2023-11-27T15:16:11.017672+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:52:19.668174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'10","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.741758+0000","last_change":"2023-11-27T15:16:08.921771+0000","last_active":"2023-11-27T15:16:12.741758+0000","last_peered":"2023-11-27T15:16:12.741758+0000","last_clean":"2023-11-27T15:16:12.741758+0000","last_became_active":"2023-11-27T15:16:08.921396+0000","last_became_peered":"2023-11-27T15:16:08.921396+0000","last_unstale":"2023-11-27T15:16:12.741758+0000","last_undegraded":"2023-11-27T15:16:12.741758+0000","last_fullsized":"2023-11-27T15:16:12.741758+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:32:00.774950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018456+0000","last_change":"2023-11-27T15:16:09.341258+0000","last_active":"2023-11-27T15:16:11.018456+0000","last_peered":"2023-11-27T15:16:11.018456+0000","last_clean":"2023-11-27T15:16:11.018456+0000","last_became_active":"2023-11-27T15:16:09.340921+0000","last_became_peered":"2023-11-27T15:16:09.340921+0000","last_unstale":"2023-11-27T15:16:11.018456+0000","last_undegraded":"2023-11-27T15:16:11.018456+0000","last_fullsized":"2023-11-27T15:16:11.018456+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:59:25.144109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018494+0000","last_change":"2023-11-27T15:16:08.911219+0000","last_active":"2023-11-27T15:16:11.018494+0000","last_peered":"2023-11-27T15:16:11.018494+0000","last_clean":"2023-11-27T15:16:11.018494+0000","last_became_active":"2023-11-27T15:16:08.910788+0000","last_became_peered":"2023-11-27T15:16:08.910788+0000","last_unstale":"2023-11-27T15:16:11.018494+0000","last_undegraded":"2023-11-27T15:16:11.018494+0000","last_fullsized":"2023-11-27T15:16:11.018494+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:02:11.224624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329287+0000","last_change":"2023-11-27T15:16:09.343793+0000","last_active":"2023-11-27T15:16:10.329287+0000","last_peered":"2023-11-27T15:16:10.329287+0000","last_clean":"2023-11-27T15:16:10.329287+0000","last_became_active":"2023-11-27T15:16:09.343480+0000","last_became_peered":"2023-11-27T15:16:09.343480+0000","last_unstale":"2023-11-27T15:16:10.329287+0000","last_undegraded":"2023-11-27T15:16:10.329287+0000","last_fullsized":"2023-11-27T15:16:10.329287+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:23:15.936537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018282+0000","last_change":"2023-11-27T15:16:08.920651+0000","last_active":"2023-11-27T15:16:11.018282+0000","last_peered":"2023-11-27T15:16:11.018282+0000","last_clean":"2023-11-27T15:16:11.018282+0000","last_became_active":"2023-11-27T15:16:08.920294+0000","last_became_peered":"2023-11-27T15:16:08.920294+0000","last_unstale":"2023-11-27T15:16:11.018282+0000","last_undegraded":"2023-11-27T15:16:11.018282+0000","last_fullsized":"2023-11-27T15:16:11.018282+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:57:51.796562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.019096+0000","last_change":"2023-11-27T15:16:09.348279+0000","last_active":"2023-11-27T15:16:11.019096+0000","last_peered":"2023-11-27T15:16:11.019096+0000","last_clean":"2023-11-27T15:16:11.019096+0000","last_became_active":"2023-11-27T15:16:09.347760+0000","last_became_peered":"2023-11-27T15:16:09.347760+0000","last_unstale":"2023-11-27T15:16:11.019096+0000","last_undegraded":"2023-11-27T15:16:11.019096+0000","last_fullsized":"2023-11-27T15:16:11.019096+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:35:32.477154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018062+0000","last_change":"2023-11-27T15:16:08.915622+0000","last_active":"2023-11-27T15:16:11.018062+0000","last_peered":"2023-11-27T15:16:11.018062+0000","last_clean":"2023-11-27T15:16:11.018062+0000","last_became_active":"2023-11-27T15:16:08.915293+0000","last_became_peered":"2023-11-27T15:16:08.915293+0000","last_unstale":"2023-11-27T15:16:11.018062+0000","last_undegraded":"2023-11-27T15:16:11.018062+0000","last_fullsized":"2023-11-27T15:16:11.018062+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:42:31.199595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.019331+0000","last_change":"2023-11-27T15:16:09.337096+0000","last_active":"2023-11-27T15:16:11.019331+0000","last_peered":"2023-11-27T15:16:11.019331+0000","last_clean":"2023-11-27T15:16:11.019331+0000","last_became_active":"2023-11-27T15:16:09.336359+0000","last_became_peered":"2023-11-27T15:16:09.336359+0000","last_unstale":"2023-11-27T15:16:11.019331+0000","last_undegraded":"2023-11-27T15:16:11.019331+0000","last_fullsized":"2023-11-27T15:16:11.019331+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:01:29.480942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018063+0000","last_change":"2023-11-27T15:16:09.340478+0000","last_active":"2023-11-27T15:16:11.018063+0000","last_peered":"2023-11-27T15:16:11.018063+0000","last_clean":"2023-11-27T15:16:11.018063+0000","last_became_active":"2023-11-27T15:16:09.340214+0000","last_became_peered":"2023-11-27T15:16:09.340214+0000","last_unstale":"2023-11-27T15:16:11.018063+0000","last_undegraded":"2023-11-27T15:16:11.018063+0000","last_fullsized":"2023-11-27T15:16:11.018063+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:09:06.499851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:13.751044+0000","last_change":"2023-11-27T15:16:08.920387+0000","last_active":"2023-11-27T15:16:13.751044+0000","last_peered":"2023-11-27T15:16:13.751044+0000","last_clean":"2023-11-27T15:16:13.751044+0000","last_became_active":"2023-11-27T15:16:08.920004+0000","last_became_peered":"2023-11-27T15:16:08.920004+0000","last_unstale":"2023-11-27T15:16:13.751044+0000","last_undegraded":"2023-11-27T15:16:13.751044+0000","last_fullsized":"2023-11-27T15:16:13.751044+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:02:23.895980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018879+0000","last_change":"2023-11-27T15:16:09.352291+0000","last_active":"2023-11-27T15:16:11.018879+0000","last_peered":"2023-11-27T15:16:11.018879+0000","last_clean":"2023-11-27T15:16:11.018879+0000","last_became_active":"2023-11-27T15:16:09.351944+0000","last_became_peered":"2023-11-27T15:16:09.351944+0000","last_unstale":"2023-11-27T15:16:11.018879+0000","last_undegraded":"2023-11-27T15:16:11.018879+0000","last_fullsized":"2023-11-27T15:16:11.018879+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:59:40.185533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018889+0000","last_change":"2023-11-27T15:16:08.909694+0000","last_active":"2023-11-27T15:16:11.018889+0000","last_peered":"2023-11-27T15:16:11.018889+0000","last_clean":"2023-11-27T15:16:11.018889+0000","last_became_active":"2023-11-27T15:16:08.909324+0000","last_became_peered":"2023-11-27T15:16:08.909324+0000","last_unstale":"2023-11-27T15:16:11.018889+0000","last_undegraded":"2023-11-27T15:16:11.018889+0000","last_fullsized":"2023-11-27T15:16:11.018889+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:37:52.187833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":340,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017780+0000","last_change":"2023-11-27T15:15:10.678805+0000","last_active":"2023-11-27T15:16:11.017780+0000","last_peered":"2023-11-27T15:16:11.017780+0000","last_clean":"2023-11-27T15:16:11.017780+0000","last_became_active":"2023-11-27T15:15:10.678267+0000","last_became_peered":"2023-11-27T15:15:10.678267+0000","last_unstale":"2023-11-27T15:16:11.017780+0000","last_undegraded":"2023-11-27T15:16:11.017780+0000","last_fullsized":"2023-11-27T15:16:11.017780+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:12:14.179315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:12:14.179315+0000","last_clean_scrub_stamp":"2023-11-27T15:12:14.179315+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:53:29.305985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017431+0000","last_change":"2023-11-27T15:16:09.346622+0000","last_active":"2023-11-27T15:16:11.017431+0000","last_peered":"2023-11-27T15:16:11.017431+0000","last_clean":"2023-11-27T15:16:11.017431+0000","last_became_active":"2023-11-27T15:16:09.346287+0000","last_became_peered":"2023-11-27T15:16:09.346287+0000","last_unstale":"2023-11-27T15:16:11.017431+0000","last_undegraded":"2023-11-27T15:16:11.017431+0000","last_fullsized":"2023-11-27T15:16:11.017431+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:06:51.424504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017928+0000","last_change":"2023-11-27T15:16:08.915598+0000","last_active":"2023-11-27T15:16:11.017928+0000","last_peered":"2023-11-27T15:16:11.017928+0000","last_clean":"2023-11-27T15:16:11.017928+0000","last_became_active":"2023-11-27T15:16:08.915255+0000","last_became_peered":"2023-11-27T15:16:08.915255+0000","last_unstale":"2023-11-27T15:16:11.017928+0000","last_undegraded":"2023-11-27T15:16:11.017928+0000","last_fullsized":"2023-11-27T15:16:11.017928+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:59:36.190517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018749+0000","last_change":"2023-11-27T15:16:08.921855+0000","last_active":"2023-11-27T15:16:11.018749+0000","last_peered":"2023-11-27T15:16:11.018749+0000","last_clean":"2023-11-27T15:16:11.018749+0000","last_became_active":"2023-11-27T15:16:08.921525+0000","last_became_peered":"2023-11-27T15:16:08.921525+0000","last_unstale":"2023-11-27T15:16:11.018749+0000","last_undegraded":"2023-11-27T15:16:11.018749+0000","last_fullsized":"2023-11-27T15:16:11.018749+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:18:46.190023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017617+0000","last_change":"2023-11-27T15:16:09.345119+0000","last_active":"2023-11-27T15:16:11.017617+0000","last_peered":"2023-11-27T15:16:11.017617+0000","last_clean":"2023-11-27T15:16:11.017617+0000","last_became_active":"2023-11-27T15:16:09.344807+0000","last_became_peered":"2023-11-27T15:16:09.344807+0000","last_unstale":"2023-11-27T15:16:11.017617+0000","last_undegraded":"2023-11-27T15:16:11.017617+0000","last_fullsized":"2023-11-27T15:16:11.017617+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:35:43.938240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018498+0000","last_change":"2023-11-27T15:16:08.911564+0000","last_active":"2023-11-27T15:16:11.018498+0000","last_peered":"2023-11-27T15:16:11.018498+0000","last_clean":"2023-11-27T15:16:11.018498+0000","last_became_active":"2023-11-27T15:16:08.911057+0000","last_became_peered":"2023-11-27T15:16:08.911057+0000","last_unstale":"2023-11-27T15:16:11.018498+0000","last_undegraded":"2023-11-27T15:16:11.018498+0000","last_fullsized":"2023-11-27T15:16:11.018498+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:12:28.660065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018236+0000","last_change":"2023-11-27T15:16:09.339085+0000","last_active":"2023-11-27T15:16:11.018236+0000","last_peered":"2023-11-27T15:16:11.018236+0000","last_clean":"2023-11-27T15:16:11.018236+0000","last_became_active":"2023-11-27T15:16:09.338777+0000","last_became_peered":"2023-11-27T15:16:09.338777+0000","last_unstale":"2023-11-27T15:16:11.018236+0000","last_undegraded":"2023-11-27T15:16:11.018236+0000","last_fullsized":"2023-11-27T15:16:11.018236+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:37:58.241374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.019216+0000","last_change":"2023-11-27T15:16:08.913939+0000","last_active":"2023-11-27T15:16:11.019216+0000","last_peered":"2023-11-27T15:16:11.019216+0000","last_clean":"2023-11-27T15:16:11.019216+0000","last_became_active":"2023-11-27T15:16:08.913629+0000","last_became_peered":"2023-11-27T15:16:08.913629+0000","last_unstale":"2023-11-27T15:16:11.019216+0000","last_undegraded":"2023-11-27T15:16:11.019216+0000","last_fullsized":"2023-11-27T15:16:11.019216+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:46:10.524140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.865269+0000","last_change":"2023-11-27T15:16:09.350712+0000","last_active":"2023-11-27T15:16:10.865269+0000","last_peered":"2023-11-27T15:16:10.865269+0000","last_clean":"2023-11-27T15:16:10.865269+0000","last_became_active":"2023-11-27T15:16:09.350359+0000","last_became_peered":"2023-11-27T15:16:09.350359+0000","last_unstale":"2023-11-27T15:16:10.865269+0000","last_undegraded":"2023-11-27T15:16:10.865269+0000","last_fullsized":"2023-11-27T15:16:10.865269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:30:08.385906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.744120+0000","last_change":"2023-11-27T15:16:08.919501+0000","last_active":"2023-11-27T15:16:12.744120+0000","last_peered":"2023-11-27T15:16:12.744120+0000","last_clean":"2023-11-27T15:16:12.744120+0000","last_became_active":"2023-11-27T15:16:08.919258+0000","last_became_peered":"2023-11-27T15:16:08.919258+0000","last_unstale":"2023-11-27T15:16:12.744120+0000","last_undegraded":"2023-11-27T15:16:12.744120+0000","last_fullsized":"2023-11-27T15:16:12.744120+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:12:35.863121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017921+0000","last_change":"2023-11-27T15:16:09.344066+0000","last_active":"2023-11-27T15:16:11.017921+0000","last_peered":"2023-11-27T15:16:11.017921+0000","last_clean":"2023-11-27T15:16:11.017921+0000","last_became_active":"2023-11-27T15:16:09.343757+0000","last_became_peered":"2023-11-27T15:16:09.343757+0000","last_unstale":"2023-11-27T15:16:11.017921+0000","last_undegraded":"2023-11-27T15:16:11.017921+0000","last_fullsized":"2023-11-27T15:16:11.017921+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:45:10.744579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.744748+0000","last_change":"2023-11-27T15:16:08.906205+0000","last_active":"2023-11-27T15:16:12.744748+0000","last_peered":"2023-11-27T15:16:12.744748+0000","last_clean":"2023-11-27T15:16:12.744748+0000","last_became_active":"2023-11-27T15:16:08.905885+0000","last_became_peered":"2023-11-27T15:16:08.905885+0000","last_unstale":"2023-11-27T15:16:12.744748+0000","last_undegraded":"2023-11-27T15:16:12.744748+0000","last_fullsized":"2023-11-27T15:16:12.744748+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:15:37.488708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.328941+0000","last_change":"2023-11-27T15:16:09.349642+0000","last_active":"2023-11-27T15:16:10.328941+0000","last_peered":"2023-11-27T15:16:10.328941+0000","last_clean":"2023-11-27T15:16:10.328941+0000","last_became_active":"2023-11-27T15:16:09.349086+0000","last_became_peered":"2023-11-27T15:16:09.349086+0000","last_unstale":"2023-11-27T15:16:10.328941+0000","last_undegraded":"2023-11-27T15:16:10.328941+0000","last_fullsized":"2023-11-27T15:16:10.328941+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:54:34.111407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329196+0000","last_change":"2023-11-27T15:16:08.917835+0000","last_active":"2023-11-27T15:16:10.329196+0000","last_peered":"2023-11-27T15:16:10.329196+0000","last_clean":"2023-11-27T15:16:10.329196+0000","last_became_active":"2023-11-27T15:16:08.917532+0000","last_became_peered":"2023-11-27T15:16:08.917532+0000","last_unstale":"2023-11-27T15:16:10.329196+0000","last_undegraded":"2023-11-27T15:16:10.329196+0000","last_fullsized":"2023-11-27T15:16:10.329196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:35:27.771748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.019054+0000","last_change":"2023-11-27T15:16:09.338459+0000","last_active":"2023-11-27T15:16:11.019054+0000","last_peered":"2023-11-27T15:16:11.019054+0000","last_clean":"2023-11-27T15:16:11.019054+0000","last_became_active":"2023-11-27T15:16:09.338148+0000","last_became_peered":"2023-11-27T15:16:09.338148+0000","last_unstale":"2023-11-27T15:16:11.019054+0000","last_undegraded":"2023-11-27T15:16:11.019054+0000","last_fullsized":"2023-11-27T15:16:11.019054+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:50:17.721161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.742934+0000","last_change":"2023-11-27T15:16:08.915401+0000","last_active":"2023-11-27T15:16:12.742934+0000","last_peered":"2023-11-27T15:16:12.742934+0000","last_clean":"2023-11-27T15:16:12.742934+0000","last_became_active":"2023-11-27T15:16:08.915012+0000","last_became_peered":"2023-11-27T15:16:08.915012+0000","last_unstale":"2023-11-27T15:16:12.742934+0000","last_undegraded":"2023-11-27T15:16:12.742934+0000","last_fullsized":"2023-11-27T15:16:12.742934+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:25:39.081595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018186+0000","last_change":"2023-11-27T15:16:09.352309+0000","last_active":"2023-11-27T15:16:11.018186+0000","last_peered":"2023-11-27T15:16:11.018186+0000","last_clean":"2023-11-27T15:16:11.018186+0000","last_became_active":"2023-11-27T15:16:09.352039+0000","last_became_peered":"2023-11-27T15:16:09.352039+0000","last_unstale":"2023-11-27T15:16:11.018186+0000","last_undegraded":"2023-11-27T15:16:11.018186+0000","last_fullsized":"2023-11-27T15:16:11.018186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:02:54.926789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329777+0000","last_change":"2023-11-27T15:16:08.920446+0000","last_active":"2023-11-27T15:16:10.329777+0000","last_peered":"2023-11-27T15:16:10.329777+0000","last_clean":"2023-11-27T15:16:10.329777+0000","last_became_active":"2023-11-27T15:16:08.920232+0000","last_became_peered":"2023-11-27T15:16:08.920232+0000","last_unstale":"2023-11-27T15:16:10.329777+0000","last_undegraded":"2023-11-27T15:16:10.329777+0000","last_fullsized":"2023-11-27T15:16:10.329777+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:28:22.523270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018226+0000","last_change":"2023-11-27T15:16:09.344328+0000","last_active":"2023-11-27T15:16:11.018226+0000","last_peered":"2023-11-27T15:16:11.018226+0000","last_clean":"2023-11-27T15:16:11.018226+0000","last_became_active":"2023-11-27T15:16:09.344047+0000","last_became_peered":"2023-11-27T15:16:09.344047+0000","last_unstale":"2023-11-27T15:16:11.018226+0000","last_undegraded":"2023-11-27T15:16:11.018226+0000","last_fullsized":"2023-11-27T15:16:11.018226+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:10:10.564306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018175+0000","last_change":"2023-11-27T15:16:08.909865+0000","last_active":"2023-11-27T15:16:11.018175+0000","last_peered":"2023-11-27T15:16:11.018175+0000","last_clean":"2023-11-27T15:16:11.018175+0000","last_became_active":"2023-11-27T15:16:08.909559+0000","last_became_peered":"2023-11-27T15:16:08.909559+0000","last_unstale":"2023-11-27T15:16:11.018175+0000","last_undegraded":"2023-11-27T15:16:11.018175+0000","last_fullsized":"2023-11-27T15:16:11.018175+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:21:20.674879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018009+0000","last_change":"2023-11-27T15:16:09.342525+0000","last_active":"2023-11-27T15:16:11.018009+0000","last_peered":"2023-11-27T15:16:11.018009+0000","last_clean":"2023-11-27T15:16:11.018009+0000","last_became_active":"2023-11-27T15:16:09.340987+0000","last_became_peered":"2023-11-27T15:16:09.340987+0000","last_unstale":"2023-11-27T15:16:11.018009+0000","last_undegraded":"2023-11-27T15:16:11.018009+0000","last_fullsized":"2023-11-27T15:16:11.018009+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:57:33.064763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017934+0000","last_change":"2023-11-27T15:16:08.914088+0000","last_active":"2023-11-27T15:16:11.017934+0000","last_peered":"2023-11-27T15:16:11.017934+0000","last_clean":"2023-11-27T15:16:11.017934+0000","last_became_active":"2023-11-27T15:16:08.913750+0000","last_became_peered":"2023-11-27T15:16:08.913750+0000","last_unstale":"2023-11-27T15:16:11.017934+0000","last_undegraded":"2023-11-27T15:16:11.017934+0000","last_fullsized":"2023-11-27T15:16:11.017934+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:52:50.990435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018281+0000","last_change":"2023-11-27T15:16:09.344403+0000","last_active":"2023-11-27T15:16:11.018281+0000","last_peered":"2023-11-27T15:16:11.018281+0000","last_clean":"2023-11-27T15:16:11.018281+0000","last_became_active":"2023-11-27T15:16:09.344120+0000","last_became_peered":"2023-11-27T15:16:09.344120+0000","last_unstale":"2023-11-27T15:16:11.018281+0000","last_undegraded":"2023-11-27T15:16:11.018281+0000","last_fullsized":"2023-11-27T15:16:11.018281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:47:52.787834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.328999+0000","last_change":"2023-11-27T15:16:09.349780+0000","last_active":"2023-11-27T15:16:10.328999+0000","last_peered":"2023-11-27T15:16:10.328999+0000","last_clean":"2023-11-27T15:16:10.328999+0000","last_became_active":"2023-11-27T15:16:09.349493+0000","last_became_peered":"2023-11-27T15:16:09.349493+0000","last_unstale":"2023-11-27T15:16:10.328999+0000","last_undegraded":"2023-11-27T15:16:10.328999+0000","last_fullsized":"2023-11-27T15:16:10.328999+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:11:07.759158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329634+0000","last_change":"2023-11-27T15:16:09.340528+0000","last_active":"2023-11-27T15:16:10.329634+0000","last_peered":"2023-11-27T15:16:10.329634+0000","last_clean":"2023-11-27T15:16:10.329634+0000","last_became_active":"2023-11-27T15:16:09.339435+0000","last_became_peered":"2023-11-27T15:16:09.339435+0000","last_unstale":"2023-11-27T15:16:10.329634+0000","last_undegraded":"2023-11-27T15:16:10.329634+0000","last_fullsized":"2023-11-27T15:16:10.329634+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:08:23.254070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329091+0000","last_change":"2023-11-27T15:16:09.337989+0000","last_active":"2023-11-27T15:16:10.329091+0000","last_peered":"2023-11-27T15:16:10.329091+0000","last_clean":"2023-11-27T15:16:10.329091+0000","last_became_active":"2023-11-27T15:16:09.336712+0000","last_became_peered":"2023-11-27T15:16:09.336712+0000","last_unstale":"2023-11-27T15:16:10.329091+0000","last_undegraded":"2023-11-27T15:16:10.329091+0000","last_fullsized":"2023-11-27T15:16:10.329091+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:53:19.057499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.865031+0000","last_change":"2023-11-27T15:16:09.347077+0000","last_active":"2023-11-27T15:16:10.865031+0000","last_peered":"2023-11-27T15:16:10.865031+0000","last_clean":"2023-11-27T15:16:10.865031+0000","last_became_active":"2023-11-27T15:16:09.346722+0000","last_became_peered":"2023-11-27T15:16:09.346722+0000","last_unstale":"2023-11-27T15:16:10.865031+0000","last_undegraded":"2023-11-27T15:16:10.865031+0000","last_fullsized":"2023-11-27T15:16:10.865031+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:09:22.987678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017743+0000","last_change":"2023-11-27T15:16:09.344317+0000","last_active":"2023-11-27T15:16:11.017743+0000","last_peered":"2023-11-27T15:16:11.017743+0000","last_clean":"2023-11-27T15:16:11.017743+0000","last_became_active":"2023-11-27T15:16:09.343962+0000","last_became_peered":"2023-11-27T15:16:09.343962+0000","last_unstale":"2023-11-27T15:16:11.017743+0000","last_undegraded":"2023-11-27T15:16:11.017743+0000","last_fullsized":"2023-11-27T15:16:11.017743+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:49:36.883005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329146+0000","last_change":"2023-11-27T15:16:09.344690+0000","last_active":"2023-11-27T15:16:10.329146+0000","last_peered":"2023-11-27T15:16:10.329146+0000","last_clean":"2023-11-27T15:16:10.329146+0000","last_became_active":"2023-11-27T15:16:09.344259+0000","last_became_peered":"2023-11-27T15:16:09.344259+0000","last_unstale":"2023-11-27T15:16:10.329146+0000","last_undegraded":"2023-11-27T15:16:10.329146+0000","last_fullsized":"2023-11-27T15:16:10.329146+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:47:31.777045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329769+0000","last_change":"2023-11-27T15:16:09.340459+0000","last_active":"2023-11-27T15:16:10.329769+0000","last_peered":"2023-11-27T15:16:10.329769+0000","last_clean":"2023-11-27T15:16:10.329769+0000","last_became_active":"2023-11-27T15:16:09.340140+0000","last_became_peered":"2023-11-27T15:16:09.340140+0000","last_unstale":"2023-11-27T15:16:10.329769+0000","last_undegraded":"2023-11-27T15:16:10.329769+0000","last_fullsized":"2023-11-27T15:16:10.329769+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:51:52.323186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018191+0000","last_change":"2023-11-27T15:16:09.336625+0000","last_active":"2023-11-27T15:16:11.018191+0000","last_peered":"2023-11-27T15:16:11.018191+0000","last_clean":"2023-11-27T15:16:11.018191+0000","last_became_active":"2023-11-27T15:16:09.336287+0000","last_became_peered":"2023-11-27T15:16:09.336287+0000","last_unstale":"2023-11-27T15:16:11.018191+0000","last_undegraded":"2023-11-27T15:16:11.018191+0000","last_fullsized":"2023-11-27T15:16:11.018191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:19:26.038950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017922+0000","last_change":"2023-11-27T15:16:09.335480+0000","last_active":"2023-11-27T15:16:11.017922+0000","last_peered":"2023-11-27T15:16:11.017922+0000","last_clean":"2023-11-27T15:16:11.017922+0000","last_became_active":"2023-11-27T15:16:09.335163+0000","last_became_peered":"2023-11-27T15:16:09.335163+0000","last_unstale":"2023-11-27T15:16:11.017922+0000","last_undegraded":"2023-11-27T15:16:11.017922+0000","last_fullsized":"2023-11-27T15:16:11.017922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:27:19.391538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017703+0000","last_change":"2023-11-27T15:16:09.336879+0000","last_active":"2023-11-27T15:16:11.017703+0000","last_peered":"2023-11-27T15:16:11.017703+0000","last_clean":"2023-11-27T15:16:11.017703+0000","last_became_active":"2023-11-27T15:16:09.336580+0000","last_became_peered":"2023-11-27T15:16:09.336580+0000","last_unstale":"2023-11-27T15:16:11.017703+0000","last_undegraded":"2023-11-27T15:16:11.017703+0000","last_fullsized":"2023-11-27T15:16:11.017703+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:45:37.465813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.332430+0000","last_change":"2023-11-27T15:16:09.341400+0000","last_active":"2023-11-27T15:16:10.332430+0000","last_peered":"2023-11-27T15:16:10.332430+0000","last_clean":"2023-11-27T15:16:10.332430+0000","last_became_active":"2023-11-27T15:16:09.341103+0000","last_became_peered":"2023-11-27T15:16:09.341103+0000","last_unstale":"2023-11-27T15:16:10.332430+0000","last_undegraded":"2023-11-27T15:16:10.332430+0000","last_fullsized":"2023-11-27T15:16:10.332430+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:59:30.634393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018153+0000","last_change":"2023-11-27T15:16:09.345938+0000","last_active":"2023-11-27T15:16:11.018153+0000","last_peered":"2023-11-27T15:16:11.018153+0000","last_clean":"2023-11-27T15:16:11.018153+0000","last_became_active":"2023-11-27T15:16:09.345624+0000","last_became_peered":"2023-11-27T15:16:09.345624+0000","last_unstale":"2023-11-27T15:16:11.018153+0000","last_undegraded":"2023-11-27T15:16:11.018153+0000","last_fullsized":"2023-11-27T15:16:11.018153+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:03:33.299715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329231+0000","last_change":"2023-11-27T15:16:09.343896+0000","last_active":"2023-11-27T15:16:10.329231+0000","last_peered":"2023-11-27T15:16:10.329231+0000","last_clean":"2023-11-27T15:16:10.329231+0000","last_became_active":"2023-11-27T15:16:09.343603+0000","last_became_peered":"2023-11-27T15:16:09.343603+0000","last_unstale":"2023-11-27T15:16:10.329231+0000","last_undegraded":"2023-11-27T15:16:10.329231+0000","last_fullsized":"2023-11-27T15:16:10.329231+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:58:06.401944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":249380,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":3810,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":192512,"data_stored":28713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71,"ondisk_log_size":71,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449100,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5932,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":93739220,"statfs":{"total":95995035648,"available":95988961280,"internally_reserved":0,"allocated":888832,"data_stored":426296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]}]},{"osd":10,"up_from":71,"seq":304942678032,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6196,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":93738956,"statfs":{"total":95995035648,"available":95988690944,"internally_reserved":0,"allocated":962560,"data_stored":1018422,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":1,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.248},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.27100000000000002}]},{"osd":2,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.39000000000000001}]},{"osd":3,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.441},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.307}]},{"osd":4,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.374},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.35599999999999998}]},{"osd":5,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.46800000000000003}]},{"osd":6,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.45900000000000002}]},{"osd":7,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.53300000000000003}]},{"osd":8,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.504}]},{"osd":9,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.53000000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]},{"osd":9,"up_from":66,"seq":283467841556,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6064,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739088,"statfs":{"total":95995035648,"available":95988826112,"internally_reserved":0,"allocated":892928,"data_stored":426878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.378},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.438}]},{"osd":1,"last update":"Mon Nov 27 15:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.42499999999999999}]},{"osd":2,"last update":"Mon Nov 27 15:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.52900000000000003}]},{"osd":3,"last update":"Mon Nov 27 15:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.56799999999999995}]},{"osd":4,"last update":"Mon Nov 27 15:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.497},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.40699999999999997}]},{"osd":5,"last update":"Mon Nov 27 15:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.53800000000000003}]},{"osd":6,"last update":"Mon Nov 27 15:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.55600000000000005}]},{"osd":7,"last update":"Mon Nov 27 15:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.48199999999999998}]},{"osd":8,"last update":"Mon Nov 27 15:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.58499999999999996}]},{"osd":10,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.61599999999999999}]}]},{"osd":8,"up_from":58,"seq":249108103193,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6260,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738892,"statfs":{"total":95995035648,"available":95988625408,"internally_reserved":0,"allocated":897024,"data_stored":426900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.42299999999999999}]},{"osd":1,"last update":"Mon Nov 27 15:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.54500000000000004}]},{"osd":2,"last update":"Mon Nov 27 15:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.63200000000000001}]},{"osd":3,"last update":"Mon Nov 27 15:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.57799999999999996}]},{"osd":4,"last update":"Mon Nov 27 15:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.495},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.64800000000000002}]},{"osd":5,"last update":"Mon Nov 27 15:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.55600000000000005}]},{"osd":6,"last update":"Mon Nov 27 15:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.70699999999999996}]},{"osd":7,"last update":"Mon Nov 27 15:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.68300000000000005}]},{"osd":9,"last update":"Mon Nov 27 15:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.66200000000000003}]},{"osd":10,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.754}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]}]},{"osd":7,"up_from":50,"seq":214748364828,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6304,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738848,"statfs":{"total":95995035648,"available":95988580352,"internally_reserved":0,"allocated":942080,"data_stored":1017246,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.35599999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.76300000000000001,"5min":1.052,"15min":1.052},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.51800000000000002,"15min":0.503},"min":{"1min":0.315,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.38700000000000001}]},{"osd":1,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.39400000000000002,"15min":0.38200000000000001},"min":{"1min":0.33400000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.60499999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.40999999999999998,"15min":0.39900000000000002},"min":{"1min":0.28199999999999997,"5min":0.254,"15min":0.254},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.34599999999999997}]},{"osd":2,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.39500000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.624},{"interface":"front","average":{"1min":0.629,"5min":0.55300000000000005,"15min":0.54100000000000004},"min":{"1min":0.41499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.59499999999999997}]},{"osd":3,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.54000000000000004,"15min":0.52400000000000002},"min":{"1min":0.38800000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.59299999999999997,"15min":0.57999999999999996},"min":{"1min":0.45700000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88300000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.48499999999999999}]},{"osd":4,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.49299999999999999,"15min":0.47499999999999998},"min":{"1min":0.43099999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.48999999999999999,"15min":0.47399999999999998},"min":{"1min":0.41399999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.33000000000000002}]},{"osd":5,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.61799999999999999,"15min":0.59499999999999997},"min":{"1min":0.47099999999999997,"5min":0.375,"15min":0.375},"max":{"1min":2.3719999999999999,"5min":2.3719999999999999,"15min":2.3719999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.64100000000000001,"15min":0.61199999999999999},"min":{"1min":0.44700000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":3.1059999999999999,"5min":3.1059999999999999,"15min":3.1059999999999999},"last":0.61499999999999999}]},{"osd":6,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.66600000000000004,"15min":0.65400000000000003},"min":{"1min":0.51200000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.95299999999999996,"5min":1.008,"15min":1.008},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.625,"15min":0.61099999999999999},"min":{"1min":0.52900000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.47299999999999998}]},{"osd":8,"last update":"Mon Nov 27 15:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.64200000000000002}]},{"osd":9,"last update":"Mon Nov 27 15:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.54400000000000004}]},{"osd":10,"last update":"Mon Nov 27 15:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.50900000000000001}]}]},{"osd":6,"up_from":45,"seq":193273528354,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6256,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738896,"statfs":{"total":95995035648,"available":95988629504,"internally_reserved":0,"allocated":892928,"data_stored":426386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.44600000000000001,"15min":0.44800000000000001},"min":{"1min":0.309,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.65500000000000003,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.52900000000000003,"15min":0.54100000000000004},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.70599999999999996,"5min":2.508,"15min":2.508},"last":0.39600000000000002}]},{"osd":1,"last update":"Mon Nov 27 15:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.59499999999999997,"15min":0.58699999999999997},"min":{"1min":0.44600000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.88400000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.63700000000000001,"15min":0.64500000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85899999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":0.55700000000000005}]},{"osd":2,"last update":"Mon Nov 27 15:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.67400000000000004,"15min":0.68200000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.81499999999999995,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65500000000000003,"15min":0.65300000000000002},"min":{"1min":0.48499999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.84499999999999997,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.502}]},{"osd":3,"last update":"Mon Nov 27 15:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.53700000000000003,"15min":0.53300000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79000000000000004,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.57999999999999996,"15min":0.59799999999999998},"min":{"1min":0.30599999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.63300000000000001,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":0.433}]},{"osd":4,"last update":"Mon Nov 27 15:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.42099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0529999999999999,"5min":1.51,"15min":1.51},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.65100000000000002,"15min":0.64100000000000001},"min":{"1min":0.48999999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.748}]},{"osd":5,"last update":"Mon Nov 27 15:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.72499999999999998,"15min":0.74099999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92200000000000004,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.71999999999999997,"15min":0.72499999999999998},"min":{"1min":0.52200000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.93600000000000005,"5min":1.599,"15min":1.599},"last":0.63600000000000001}]},{"osd":7,"last update":"Mon Nov 27 15:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72599999999999998,"15min":0.72499999999999998},"min":{"1min":0.46000000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.995,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.73999999999999999,"15min":0.73599999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.79700000000000004}]},{"osd":8,"last update":"Mon Nov 27 15:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.766,"5min":2.766,"15min":2.766},"last":0.61599999999999999}]},{"osd":9,"last update":"Mon Nov 27 15:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.90000000000000002}]},{"osd":10,"last update":"Mon Nov 27 15:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.86499999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7176,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":93737976,"statfs":{"total":95995035648,"available":95987687424,"internally_reserved":0,"allocated":917504,"data_stored":445225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.502,"15min":0.46999999999999997},"min":{"1min":0.34000000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77000000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.373},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.505,"15min":0.46500000000000002},"min":{"1min":0.35499999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.42999999999999999}]},{"osd":2,"last update":"Mon Nov 27 15:15:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.53500000000000003,"15min":0.53700000000000003},"min":{"1min":0.36199999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.67300000000000004,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52800000000000002,"15min":0.52200000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67500000000000004,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.72999999999999998}]},{"osd":3,"last update":"Mon Nov 27 15:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.55000000000000004,"15min":0.52800000000000002},"min":{"1min":0.35999999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.86799999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.51400000000000001}]},{"osd":4,"last update":"Mon Nov 27 15:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.44,"15min":0.42799999999999999},"min":{"1min":0.314,"5min":0.251,"15min":0.251},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.307},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.45500000000000002,"15min":0.441},"min":{"1min":0.26400000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.45500000000000002}]},{"osd":5,"last update":"Mon Nov 27 15:15:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58299999999999996,"15min":0.58399999999999996},"min":{"1min":0.44,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.71899999999999997,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.745},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61599999999999999,"15min":0.61799999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.69599999999999995,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.79500000000000004}]},{"osd":6,"last update":"Mon Nov 27 15:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58199999999999996,"15min":0.57499999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.625,"15min":0.623},"min":{"1min":0.39800000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.59999999999999998}]},{"osd":7,"last update":"Mon Nov 27 15:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.53600000000000003,"15min":0.52600000000000002},"min":{"1min":0.38,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.58199999999999996,"15min":0.57299999999999995},"min":{"1min":0.432,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.33900000000000002}]},{"osd":8,"last update":"Mon Nov 27 15:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.82199999999999995}]},{"osd":9,"last update":"Mon Nov 27 15:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.66900000000000004}]},{"osd":10,"last update":"Mon Nov 27 15:16:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.53600000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738427,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7160,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":93737992,"statfs":{"total":95995035648,"available":95987703808,"internally_reserved":0,"allocated":901120,"data_stored":426990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 27 15:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.58199999999999996,"15min":0.54600000000000004},"min":{"1min":0.45900000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.97099999999999997,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.622},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.60499999999999998,"15min":0.55900000000000005},"min":{"1min":0.51000000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.018,"5min":1.55,"15min":1.55},"last":0.53600000000000003}]},{"osd":2,"last update":"Mon Nov 27 15:15:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.55700000000000005,"15min":0.53700000000000003},"min":{"1min":0.40100000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.53000000000000003,"15min":0.51500000000000001},"min":{"1min":0.44400000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.435}]},{"osd":3,"last update":"Mon Nov 27 15:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.38900000000000001,"15min":0.371},"min":{"1min":0.26000000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.441,"15min":0.42899999999999999},"min":{"1min":0.30399999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.754}]},{"osd":4,"last update":"Mon Nov 27 15:16:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.57699999999999996,"15min":0.55000000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.96099999999999997,"5min":0.999,"15min":0.999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.58899999999999997,"15min":0.55900000000000005},"min":{"1min":0.38300000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.56599999999999995}]},{"osd":5,"last update":"Mon Nov 27 15:15:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.625,"15min":0.61499999999999999},"min":{"1min":0.45900000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.626,"15min":0.61099999999999999},"min":{"1min":0.53800000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.60699999999999998}]},{"osd":6,"last update":"Mon Nov 27 15:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.56000000000000005,"15min":0.54800000000000004},"min":{"1min":0.29099999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.60299999999999998,"15min":0.59099999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.73499999999999999}]},{"osd":7,"last update":"Mon Nov 27 15:16:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67800000000000005,"15min":0.67200000000000004},"min":{"1min":0.434,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.46100000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.31}]},{"osd":8,"last update":"Mon Nov 27 15:15:26 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.77600000000000002}]},{"osd":9,"last update":"Mon Nov 27 15:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.71899999999999997}]},{"osd":10,"last update":"Mon Nov 27 15:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.69299999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411378,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7164,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":93737988,"statfs":{"total":95995035648,"available":95987699712,"internally_reserved":0,"allocated":905216,"data_stored":427894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.49099999999999999,"15min":0.47099999999999997},"min":{"1min":0.38500000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.79000000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.48899999999999999,"15min":0.47699999999999998},"min":{"1min":0.35499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.45800000000000002}]},{"osd":1,"last update":"Mon Nov 27 15:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52000000000000002,"15min":0.50600000000000001},"min":{"1min":0.34399999999999997,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.503,"15min":0.48599999999999999},"min":{"1min":0.36399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.60399999999999998}]},{"osd":3,"last update":"Mon Nov 27 15:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.55600000000000005,"15min":0.53200000000000003},"min":{"1min":0.56000000000000005,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.55700000000000005,"15min":0.53300000000000003},"min":{"1min":0.46200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.55000000000000004}]},{"osd":4,"last update":"Mon Nov 27 15:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.38400000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.76200000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.35799999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79200000000000004,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.51000000000000001}]},{"osd":5,"last update":"Mon Nov 27 15:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.41699999999999998,"15min":0.39500000000000002},"min":{"1min":0.32200000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.41899999999999998,"15min":0.40699999999999997},"min":{"1min":0.35999999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.36199999999999999}]},{"osd":6,"last update":"Mon Nov 27 15:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.435,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.629,"15min":0.61799999999999999},"min":{"1min":0.495,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.441}]},{"osd":7,"last update":"Mon Nov 27 15:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.39800000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.44},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69199999999999995,"15min":0.69599999999999995},"min":{"1min":0.46300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93300000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.51100000000000001}]},{"osd":8,"last update":"Mon Nov 27 15:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.379},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.42999999999999999}]},{"osd":9,"last update":"Mon Nov 27 15:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.57299999999999995}]},{"osd":10,"last update":"Mon Nov 27 15:16:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.623},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.67000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6520,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738632,"statfs":{"total":95995035648,"available":95988359168,"internally_reserved":0,"allocated":901120,"data_stored":426490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.34999999999999998,"15min":0.33000000000000002},"min":{"1min":0.28299999999999997,"5min":0.22,"15min":0.22},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.36799999999999999,"15min":0.35199999999999998},"min":{"1min":0.33700000000000002,"5min":0.214,"15min":0.214},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.38800000000000001}]},{"osd":1,"last update":"Mon Nov 27 15:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.52500000000000002,"15min":0.501},"min":{"1min":0.432,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81999999999999995,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.53200000000000003,"15min":0.52200000000000002},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.44600000000000001}]},{"osd":2,"last update":"Mon Nov 27 15:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.57299999999999995,"15min":0.54100000000000004},"min":{"1min":0.48399999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.56399999999999995,"15min":0.53800000000000003},"min":{"1min":0.47099999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.63300000000000001}]},{"osd":4,"last update":"Mon Nov 27 15:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.54300000000000004,"15min":0.52400000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.56000000000000005,"15min":0.52200000000000002},"min":{"1min":0.495,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.52800000000000002}]},{"osd":5,"last update":"Mon Nov 27 15:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.47099999999999997,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.72099999999999997}]},{"osd":6,"last update":"Mon Nov 27 15:15:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.56899999999999995,"15min":0.55000000000000004},"min":{"1min":0.46899999999999997,"5min":0.37,"15min":0.37},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.56000000000000005,"15min":0.54400000000000004},"min":{"1min":0.42299999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.51300000000000001}]},{"osd":7,"last update":"Mon Nov 27 15:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.496},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.72499999999999998,"15min":0.70899999999999996},"min":{"1min":0.59599999999999997,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.70099999999999996}]},{"osd":8,"last update":"Mon Nov 27 15:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.628}]},{"osd":9,"last update":"Mon Nov 27 15:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68799999999999994}]},{"osd":10,"last update":"Mon Nov 27 15:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.67600000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920810,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6308,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738844,"statfs":{"total":95995035648,"available":95988576256,"internally_reserved":0,"allocated":880640,"data_stored":417748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.48099999999999998,"15min":0.46800000000000003},"min":{"1min":0.35299999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.52200000000000002,"15min":0.50700000000000001},"min":{"1min":0.40300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.432}]},{"osd":1,"last update":"Mon Nov 27 15:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.36899999999999999,"15min":0.34799999999999998},"min":{"1min":0.30299999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.36899999999999999,"15min":0.34699999999999998},"min":{"1min":0.28399999999999997,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.50600000000000001}]},{"osd":2,"last update":"Mon Nov 27 15:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.50700000000000001,"15min":0.48899999999999999},"min":{"1min":0.36699999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.54500000000000004,"15min":0.53500000000000003},"min":{"1min":0.40200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.54500000000000004}]},{"osd":3,"last update":"Mon Nov 27 15:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.53900000000000003,"15min":0.51400000000000001},"min":{"1min":0.45100000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.40999999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80700000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.56399999999999995}]},{"osd":5,"last update":"Mon Nov 27 15:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.378,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.375,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.61799999999999999}]},{"osd":6,"last update":"Mon Nov 27 15:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59199999999999997,"15min":0.58599999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.628,"15min":0.622},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.629}]},{"osd":7,"last update":"Mon Nov 27 15:16:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.34000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.34899999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.65200000000000002}]},{"osd":8,"last update":"Mon Nov 27 15:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.70299999999999996}]},{"osd":9,"last update":"Mon Nov 27 15:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.755},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.71899999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757286,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6520,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738632,"statfs":{"total":95995035648,"available":95988359168,"internally_reserved":0,"allocated":966656,"data_stored":1018092,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.501,"15min":0.49399999999999999},"min":{"1min":0.36099999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.47799999999999998,"15min":0.46999999999999997},"min":{"1min":0.33900000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.36699999999999999}]},{"osd":1,"last update":"Mon Nov 27 15:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.53400000000000003,"15min":0.52400000000000002},"min":{"1min":0.433,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.52200000000000002,"15min":0.50900000000000001},"min":{"1min":0.41099999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.54700000000000004}]},{"osd":2,"last update":"Mon Nov 27 15:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.39100000000000001,"15min":0.38700000000000001},"min":{"1min":0.30199999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.442,"5min":0.40600000000000003,"15min":0.40000000000000002},"min":{"1min":0.28000000000000003,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.42299999999999999}]},{"osd":3,"last update":"Mon Nov 27 15:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55000000000000004,"15min":0.54700000000000004},"min":{"1min":0.374,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.38600000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.443}]},{"osd":4,"last update":"Mon Nov 27 15:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.55400000000000005,"15min":0.54800000000000004},"min":{"1min":0.434,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.44600000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.40799999999999997}]},{"osd":6,"last update":"Mon Nov 27 15:15:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58899999999999997,"15min":0.58699999999999997},"min":{"1min":0.45300000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.86399999999999999,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.85599999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.64000000000000001}]},{"osd":7,"last update":"Mon Nov 27 15:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.50600000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.48099999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.56100000000000005}]},{"osd":8,"last update":"Mon Nov 27 15:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.54000000000000004}]},{"osd":9,"last update":"Mon Nov 27 15:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.499},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.67100000000000004}]},{"osd":10,"last update":"Mon Nov 27 15:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.59899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34455,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34455,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":18929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-27T15:16:17.510 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-27T15:16:17.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.665+0000 7f79a3d4b700 1 -- 172.21.15.52:0/2896316226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f799c0fe540 msgr2=0x7f799c0fe950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:17.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.665+0000 7f79a3d4b700 1 --2- 172.21.15.52:0/2896316226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f799c0fe540 0x7f799c0fe950 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7998009990 tx=0x7f7998009ca0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.665+0000 7f79a3d4b700 1 -- 172.21.15.52:0/2896316226 shutdown_connections 2023-11-27T15:16:17.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.665+0000 7f79a3d4b700 1 --2- 172.21.15.52:0/2896316226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f799c0ffa00 0x7f799c100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.665+0000 7f79a3d4b700 1 --2- 172.21.15.52:0/2896316226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f799c0fe540 0x7f799c0fe950 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.665+0000 7f79a3d4b700 1 --2- 172.21.15.52:0/2896316226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f799c0fee90 0x7f799c0ff340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.665+0000 7f79a3d4b700 1 -- 172.21.15.52:0/2896316226 >> 172.21.15.52:0/2896316226 conn(0x7f799c0f9ee0 msgr2=0x7f799c0fc350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:17.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.665+0000 7f79a3d4b700 1 -- 172.21.15.52:0/2896316226 shutdown_connections 2023-11-27T15:16:17.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.665+0000 7f79a3d4b700 1 -- 172.21.15.52:0/2896316226 wait complete. 2023-11-27T15:16:17.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.665+0000 7f79a3d4b700 1 Processor -- start 2023-11-27T15:16:17.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.665+0000 7f79a3d4b700 1 -- start start 2023-11-27T15:16:17.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.665+0000 7f79a3d4b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f799c0fe540 0x7f799c198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:17.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.665+0000 7f79a3d4b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f799c0fee90 0x7f799c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:17.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.665+0000 7f79a3d4b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f799c0ffa00 0x7f799c19f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:17.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.665+0000 7f79a3d4b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c19fd80 con 0x7f799c0fe540 2023-11-27T15:16:17.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.665+0000 7f79a3d4b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c19fec0 con 0x7f799c0fee90 2023-11-27T15:16:17.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.666+0000 7f79a3d4b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c1a0050 con 0x7f799c0ffa00 2023-11-27T15:16:17.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.666+0000 7f79a1ae7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f799c0fe540 0x7f799c198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:17.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.666+0000 7f79a1ae7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f799c0fe540 0x7f799c198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50228/0 (socket says 172.21.15.52:50228) 2023-11-27T15:16:17.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.666+0000 7f79a1ae7700 1 -- 172.21.15.52:0/1174583723 learned_addr learned my addr 172.21.15.52:0/1174583723 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:17.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.666+0000 7f79a22e8700 1 --2- 172.21.15.52:0/1174583723 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f799c0ffa00 0x7f799c19f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:17.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.666+0000 7f79a12e6700 1 --2- 172.21.15.52:0/1174583723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f799c0fee90 0x7f799c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:17.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.666+0000 7f79a1ae7700 1 -- 172.21.15.52:0/1174583723 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f799c0ffa00 msgr2=0x7f799c19f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:17.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.666+0000 7f79a1ae7700 1 --2- 172.21.15.52:0/1174583723 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f799c0ffa00 0x7f799c19f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.666+0000 7f79a1ae7700 1 -- 172.21.15.52:0/1174583723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f799c0fee90 msgr2=0x7f799c198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:17.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.666+0000 7f79a1ae7700 1 --2- 172.21.15.52:0/1174583723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f799c0fee90 0x7f799c198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.666+0000 7f79a1ae7700 1 -- 172.21.15.52:0/1174583723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79980095f0 con 0x7f799c0fe540 2023-11-27T15:16:17.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.666+0000 7f79a1ae7700 1 --2- 172.21.15.52:0/1174583723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f799c0fe540 0x7f799c198200 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7998005910 tx=0x7f799800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:17.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.666+0000 7f7992ffd700 1 -- 172.21.15.52:0/1174583723 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799801d070 con 0x7f799c0fe540 2023-11-27T15:16:17.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.666+0000 7f7992ffd700 1 -- 172.21.15.52:0/1174583723 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f799800fc30 con 0x7f799c0fe540 2023-11-27T15:16:17.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.666+0000 7f79a3d4b700 1 -- 172.21.15.52:0/1174583723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f799c10de90 con 0x7f799c0fe540 2023-11-27T15:16:17.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.666+0000 7f7992ffd700 1 -- 172.21.15.52:0/1174583723 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7998017650 con 0x7f799c0fe540 2023-11-27T15:16:17.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.666+0000 7f79a3d4b700 1 -- 172.21.15.52:0/1174583723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f799c10dd00 con 0x7f799c0fe540 2023-11-27T15:16:17.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.667+0000 7f79a3d4b700 1 -- 172.21.15.52:0/1174583723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f799c10a180 con 0x7f799c0fe540 2023-11-27T15:16:17.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.669+0000 7f7992ffd700 1 -- 172.21.15.52:0/1174583723 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f7998022470 con 0x7f799c0fe540 2023-11-27T15:16:17.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.669+0000 7f7992ffd700 1 --2- 172.21.15.52:0/1174583723 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f79880a11e0 0x7f79880a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:17.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.669+0000 7f79a12e6700 1 --2- 172.21.15.52:0/1174583723 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f79880a11e0 0x7f79880a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:17.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.669+0000 7f7992ffd700 1 -- 172.21.15.52:0/1174583723 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f79980ce180 con 0x7f799c0fe540 2023-11-27T15:16:17.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.670+0000 7f79a12e6700 1 --2- 172.21.15.52:0/1174583723 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f79880a11e0 0x7f79880a3690 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f798c005950 tx=0x7f798c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:17.677 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.673+0000 7f7992ffd700 1 -- 172.21.15.52:0/1174583723 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7998097a40 con 0x7f799c0fe540 2023-11-27T15:16:17.872 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.868+0000 7f79a3d4b700 1 -- 172.21.15.52:0/1174583723 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f799c061c00 con 0x7f79880a11e0 2023-11-27T15:16:17.884 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.880+0000 7f7992ffd700 1 -- 172.21.15.52:0/1174583723 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335846 (secure 0 0 0) 0x7f799c061c00 con 0x7f79880a11e0 2023-11-27T15:16:17.888 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:16:17.891 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.884+0000 7f79a3d4b700 1 -- 172.21.15.52:0/1174583723 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f79880a11e0 msgr2=0x7f79880a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:17.891 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.884+0000 7f79a3d4b700 1 --2- 172.21.15.52:0/1174583723 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f79880a11e0 0x7f79880a3690 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f798c005950 tx=0x7f798c0058e0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.891 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.884+0000 7f79a3d4b700 1 -- 172.21.15.52:0/1174583723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f799c0fe540 msgr2=0x7f799c198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:17.891 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.884+0000 7f79a3d4b700 1 --2- 172.21.15.52:0/1174583723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f799c0fe540 0x7f799c198200 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7998005910 tx=0x7f799800f740 comp rx=0 tx=0).stop 2023-11-27T15:16:17.891 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.884+0000 7f79a3d4b700 1 -- 172.21.15.52:0/1174583723 shutdown_connections 2023-11-27T15:16:17.891 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.884+0000 7f79a3d4b700 1 --2- 172.21.15.52:0/1174583723 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f79880a11e0 0x7f79880a3690 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.891 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.884+0000 7f79a3d4b700 1 --2- 172.21.15.52:0/1174583723 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f799c0ffa00 0x7f799c19f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.891 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.884+0000 7f79a3d4b700 1 --2- 172.21.15.52:0/1174583723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f799c0fe540 0x7f799c198200 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.891 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.884+0000 7f79a3d4b700 1 --2- 172.21.15.52:0/1174583723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f799c0fee90 0x7f799c198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.892 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.884+0000 7f79a3d4b700 1 -- 172.21.15.52:0/1174583723 >> 172.21.15.52:0/1174583723 conn(0x7f799c0f9ee0 msgr2=0x7f799c0fbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:17.893 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.884+0000 7f79a3d4b700 1 -- 172.21.15.52:0/1174583723 shutdown_connections 2023-11-27T15:16:17.893 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:17.884+0000 7f79a3d4b700 1 -- 172.21.15.52:0/1174583723 wait complete. 2023-11-27T15:16:17.893 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2023-11-27T15:16:17.900 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":260,"stamp":"2023-11-27T15:16:17.723090+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595265,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":289,"num_write_kb":2969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":78040,"kb_used_data":10744,"kb_used_omap":0,"kb_used_meta":67200,"kb_avail":1124863784,"statfs":{"total":1151940427776,"available":1151860514816,"internally_reserved":0,"allocated":11001856,"data_stored":6916354,"data_compressed":108282,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":68812800},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4897,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":108,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.401389"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329249+0000","last_change":"2023-11-27T15:16:09.343007+0000","last_active":"2023-11-27T15:16:10.329249+0000","last_peered":"2023-11-27T15:16:10.329249+0000","last_clean":"2023-11-27T15:16:10.329249+0000","last_became_active":"2023-11-27T15:16:09.342125+0000","last_became_peered":"2023-11-27T15:16:09.342125+0000","last_unstale":"2023-11-27T15:16:10.329249+0000","last_undegraded":"2023-11-27T15:16:10.329249+0000","last_fullsized":"2023-11-27T15:16:10.329249+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:10:56.620347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.865187+0000","last_change":"2023-11-27T15:16:09.338942+0000","last_active":"2023-11-27T15:16:10.865187+0000","last_peered":"2023-11-27T15:16:10.865187+0000","last_clean":"2023-11-27T15:16:10.865187+0000","last_became_active":"2023-11-27T15:16:09.338636+0000","last_became_peered":"2023-11-27T15:16:09.338636+0000","last_unstale":"2023-11-27T15:16:10.865187+0000","last_undegraded":"2023-11-27T15:16:10.865187+0000","last_fullsized":"2023-11-27T15:16:10.865187+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:30:06.448581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.865094+0000","last_change":"2023-11-27T15:16:09.347845+0000","last_active":"2023-11-27T15:16:10.865094+0000","last_peered":"2023-11-27T15:16:10.865094+0000","last_clean":"2023-11-27T15:16:10.865094+0000","last_became_active":"2023-11-27T15:16:09.347533+0000","last_became_peered":"2023-11-27T15:16:09.347533+0000","last_unstale":"2023-11-27T15:16:10.865094+0000","last_undegraded":"2023-11-27T15:16:10.865094+0000","last_fullsized":"2023-11-27T15:16:10.865094+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:01:40.820155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.019299+0000","last_change":"2023-11-27T15:16:09.333251+0000","last_active":"2023-11-27T15:16:11.019299+0000","last_peered":"2023-11-27T15:16:11.019299+0000","last_clean":"2023-11-27T15:16:11.019299+0000","last_became_active":"2023-11-27T15:16:09.331510+0000","last_became_peered":"2023-11-27T15:16:09.331510+0000","last_unstale":"2023-11-27T15:16:11.019299+0000","last_undegraded":"2023-11-27T15:16:11.019299+0000","last_fullsized":"2023-11-27T15:16:11.019299+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:34:53.253074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018096+0000","last_change":"2023-11-27T15:16:09.349097+0000","last_active":"2023-11-27T15:16:11.018096+0000","last_peered":"2023-11-27T15:16:11.018096+0000","last_clean":"2023-11-27T15:16:11.018096+0000","last_became_active":"2023-11-27T15:16:09.348793+0000","last_became_peered":"2023-11-27T15:16:09.348793+0000","last_unstale":"2023-11-27T15:16:11.018096+0000","last_undegraded":"2023-11-27T15:16:11.018096+0000","last_fullsized":"2023-11-27T15:16:11.018096+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:55:31.214202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.019174+0000","last_change":"2023-11-27T15:16:09.338246+0000","last_active":"2023-11-27T15:16:11.019174+0000","last_peered":"2023-11-27T15:16:11.019174+0000","last_clean":"2023-11-27T15:16:11.019174+0000","last_became_active":"2023-11-27T15:16:09.338003+0000","last_became_peered":"2023-11-27T15:16:09.338003+0000","last_unstale":"2023-11-27T15:16:11.019174+0000","last_undegraded":"2023-11-27T15:16:11.019174+0000","last_fullsized":"2023-11-27T15:16:11.019174+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:25:15.821446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017705+0000","last_change":"2023-11-27T15:16:09.342131+0000","last_active":"2023-11-27T15:16:11.017705+0000","last_peered":"2023-11-27T15:16:11.017705+0000","last_clean":"2023-11-27T15:16:11.017705+0000","last_became_active":"2023-11-27T15:16:09.341834+0000","last_became_peered":"2023-11-27T15:16:09.341834+0000","last_unstale":"2023-11-27T15:16:11.017705+0000","last_undegraded":"2023-11-27T15:16:11.017705+0000","last_fullsized":"2023-11-27T15:16:11.017705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:06:07.633604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018355+0000","last_change":"2023-11-27T15:16:09.337308+0000","last_active":"2023-11-27T15:16:11.018355+0000","last_peered":"2023-11-27T15:16:11.018355+0000","last_clean":"2023-11-27T15:16:11.018355+0000","last_became_active":"2023-11-27T15:16:09.336958+0000","last_became_peered":"2023-11-27T15:16:09.336958+0000","last_unstale":"2023-11-27T15:16:11.018355+0000","last_undegraded":"2023-11-27T15:16:11.018355+0000","last_fullsized":"2023-11-27T15:16:11.018355+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:00:54.104008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.332789+0000","last_change":"2023-11-27T15:16:09.340646+0000","last_active":"2023-11-27T15:16:10.332789+0000","last_peered":"2023-11-27T15:16:10.332789+0000","last_clean":"2023-11-27T15:16:10.332789+0000","last_became_active":"2023-11-27T15:16:09.340084+0000","last_became_peered":"2023-11-27T15:16:09.340084+0000","last_unstale":"2023-11-27T15:16:10.332789+0000","last_undegraded":"2023-11-27T15:16:10.332789+0000","last_fullsized":"2023-11-27T15:16:10.332789+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:33:29.581983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.019348+0000","last_change":"2023-11-27T15:16:09.336871+0000","last_active":"2023-11-27T15:16:11.019348+0000","last_peered":"2023-11-27T15:16:11.019348+0000","last_clean":"2023-11-27T15:16:11.019348+0000","last_became_active":"2023-11-27T15:16:09.336496+0000","last_became_peered":"2023-11-27T15:16:09.336496+0000","last_unstale":"2023-11-27T15:16:11.019348+0000","last_undegraded":"2023-11-27T15:16:11.019348+0000","last_fullsized":"2023-11-27T15:16:11.019348+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:37:00.015095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018026+0000","last_change":"2023-11-27T15:16:09.343120+0000","last_active":"2023-11-27T15:16:11.018026+0000","last_peered":"2023-11-27T15:16:11.018026+0000","last_clean":"2023-11-27T15:16:11.018026+0000","last_became_active":"2023-11-27T15:16:09.342797+0000","last_became_peered":"2023-11-27T15:16:09.342797+0000","last_unstale":"2023-11-27T15:16:11.018026+0000","last_undegraded":"2023-11-27T15:16:11.018026+0000","last_fullsized":"2023-11-27T15:16:11.018026+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:39:21.071837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.019248+0000","last_change":"2023-11-27T15:16:09.336521+0000","last_active":"2023-11-27T15:16:11.019248+0000","last_peered":"2023-11-27T15:16:11.019248+0000","last_clean":"2023-11-27T15:16:11.019248+0000","last_became_active":"2023-11-27T15:16:09.336183+0000","last_became_peered":"2023-11-27T15:16:09.336183+0000","last_unstale":"2023-11-27T15:16:11.019248+0000","last_undegraded":"2023-11-27T15:16:11.019248+0000","last_fullsized":"2023-11-27T15:16:11.019248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:25:55.091755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018282+0000","last_change":"2023-11-27T15:16:09.346839+0000","last_active":"2023-11-27T15:16:11.018282+0000","last_peered":"2023-11-27T15:16:11.018282+0000","last_clean":"2023-11-27T15:16:11.018282+0000","last_became_active":"2023-11-27T15:16:09.345230+0000","last_became_peered":"2023-11-27T15:16:09.345230+0000","last_unstale":"2023-11-27T15:16:11.018282+0000","last_undegraded":"2023-11-27T15:16:11.018282+0000","last_fullsized":"2023-11-27T15:16:11.018282+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:22:56.772626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.864881+0000","last_change":"2023-11-27T15:16:09.342380+0000","last_active":"2023-11-27T15:16:10.864881+0000","last_peered":"2023-11-27T15:16:10.864881+0000","last_clean":"2023-11-27T15:16:10.864881+0000","last_became_active":"2023-11-27T15:16:09.341967+0000","last_became_peered":"2023-11-27T15:16:09.341967+0000","last_unstale":"2023-11-27T15:16:10.864881+0000","last_undegraded":"2023-11-27T15:16:10.864881+0000","last_fullsized":"2023-11-27T15:16:10.864881+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:35:13.999971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.864779+0000","last_change":"2023-11-27T15:16:09.349738+0000","last_active":"2023-11-27T15:16:10.864779+0000","last_peered":"2023-11-27T15:16:10.864779+0000","last_clean":"2023-11-27T15:16:10.864779+0000","last_became_active":"2023-11-27T15:16:09.349349+0000","last_became_peered":"2023-11-27T15:16:09.349349+0000","last_unstale":"2023-11-27T15:16:10.864779+0000","last_undegraded":"2023-11-27T15:16:10.864779+0000","last_fullsized":"2023-11-27T15:16:10.864779+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:13:03.443885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.332227+0000","last_change":"2023-11-27T15:16:09.339171+0000","last_active":"2023-11-27T15:16:10.332227+0000","last_peered":"2023-11-27T15:16:10.332227+0000","last_clean":"2023-11-27T15:16:10.332227+0000","last_became_active":"2023-11-27T15:16:09.338887+0000","last_became_peered":"2023-11-27T15:16:09.338887+0000","last_unstale":"2023-11-27T15:16:10.332227+0000","last_undegraded":"2023-11-27T15:16:10.332227+0000","last_fullsized":"2023-11-27T15:16:10.332227+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:42:45.286967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329413+0000","last_change":"2023-11-27T15:16:09.343202+0000","last_active":"2023-11-27T15:16:10.329413+0000","last_peered":"2023-11-27T15:16:10.329413+0000","last_clean":"2023-11-27T15:16:10.329413+0000","last_became_active":"2023-11-27T15:16:09.342539+0000","last_became_peered":"2023-11-27T15:16:09.342539+0000","last_unstale":"2023-11-27T15:16:10.329413+0000","last_undegraded":"2023-11-27T15:16:10.329413+0000","last_fullsized":"2023-11-27T15:16:10.329413+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:31:31.313017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018587+0000","last_change":"2023-11-27T15:16:09.339915+0000","last_active":"2023-11-27T15:16:11.018587+0000","last_peered":"2023-11-27T15:16:11.018587+0000","last_clean":"2023-11-27T15:16:11.018587+0000","last_became_active":"2023-11-27T15:16:09.339637+0000","last_became_peered":"2023-11-27T15:16:09.339637+0000","last_unstale":"2023-11-27T15:16:11.018587+0000","last_undegraded":"2023-11-27T15:16:11.018587+0000","last_fullsized":"2023-11-27T15:16:11.018587+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:37:13.698277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329316+0000","last_change":"2023-11-27T15:16:09.347852+0000","last_active":"2023-11-27T15:16:10.329316+0000","last_peered":"2023-11-27T15:16:10.329316+0000","last_clean":"2023-11-27T15:16:10.329316+0000","last_became_active":"2023-11-27T15:16:09.347281+0000","last_became_peered":"2023-11-27T15:16:09.347281+0000","last_unstale":"2023-11-27T15:16:10.329316+0000","last_undegraded":"2023-11-27T15:16:10.329316+0000","last_fullsized":"2023-11-27T15:16:10.329316+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:54:00.346911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:13.751316+0000","last_change":"2023-11-27T15:16:08.912206+0000","last_active":"2023-11-27T15:16:13.751316+0000","last_peered":"2023-11-27T15:16:13.751316+0000","last_clean":"2023-11-27T15:16:13.751316+0000","last_became_active":"2023-11-27T15:16:08.911846+0000","last_became_peered":"2023-11-27T15:16:08.911846+0000","last_unstale":"2023-11-27T15:16:13.751316+0000","last_undegraded":"2023-11-27T15:16:13.751316+0000","last_fullsized":"2023-11-27T15:16:13.751316+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:21:37.095544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018680+0000","last_change":"2023-11-27T15:16:09.336485+0000","last_active":"2023-11-27T15:16:11.018680+0000","last_peered":"2023-11-27T15:16:11.018680+0000","last_clean":"2023-11-27T15:16:11.018680+0000","last_became_active":"2023-11-27T15:16:09.336236+0000","last_became_peered":"2023-11-27T15:16:09.336236+0000","last_unstale":"2023-11-27T15:16:11.018680+0000","last_undegraded":"2023-11-27T15:16:11.018680+0000","last_fullsized":"2023-11-27T15:16:11.018680+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:00:31.457583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018872+0000","last_change":"2023-11-27T15:16:08.921198+0000","last_active":"2023-11-27T15:16:11.018872+0000","last_peered":"2023-11-27T15:16:11.018872+0000","last_clean":"2023-11-27T15:16:11.018872+0000","last_became_active":"2023-11-27T15:16:08.920804+0000","last_became_peered":"2023-11-27T15:16:08.920804+0000","last_unstale":"2023-11-27T15:16:11.018872+0000","last_undegraded":"2023-11-27T15:16:11.018872+0000","last_fullsized":"2023-11-27T15:16:11.018872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:25:26.133784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.865444+0000","last_change":"2023-11-27T15:16:09.340642+0000","last_active":"2023-11-27T15:16:10.865444+0000","last_peered":"2023-11-27T15:16:10.865444+0000","last_clean":"2023-11-27T15:16:10.865444+0000","last_became_active":"2023-11-27T15:16:09.340328+0000","last_became_peered":"2023-11-27T15:16:09.340328+0000","last_unstale":"2023-11-27T15:16:10.865444+0000","last_undegraded":"2023-11-27T15:16:10.865444+0000","last_fullsized":"2023-11-27T15:16:10.865444+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:48:12.279600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018832+0000","last_change":"2023-11-27T15:16:08.915317+0000","last_active":"2023-11-27T15:16:11.018832+0000","last_peered":"2023-11-27T15:16:11.018832+0000","last_clean":"2023-11-27T15:16:11.018832+0000","last_became_active":"2023-11-27T15:16:08.914949+0000","last_became_peered":"2023-11-27T15:16:08.914949+0000","last_unstale":"2023-11-27T15:16:11.018832+0000","last_undegraded":"2023-11-27T15:16:11.018832+0000","last_fullsized":"2023-11-27T15:16:11.018832+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:35:47.443716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.332402+0000","last_change":"2023-11-27T15:16:09.348483+0000","last_active":"2023-11-27T15:16:10.332402+0000","last_peered":"2023-11-27T15:16:10.332402+0000","last_clean":"2023-11-27T15:16:10.332402+0000","last_became_active":"2023-11-27T15:16:09.348185+0000","last_became_peered":"2023-11-27T15:16:09.348185+0000","last_unstale":"2023-11-27T15:16:10.332402+0000","last_undegraded":"2023-11-27T15:16:10.332402+0000","last_fullsized":"2023-11-27T15:16:10.332402+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:23:59.373819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018251+0000","last_change":"2023-11-27T15:16:08.911018+0000","last_active":"2023-11-27T15:16:11.018251+0000","last_peered":"2023-11-27T15:16:11.018251+0000","last_clean":"2023-11-27T15:16:11.018251+0000","last_became_active":"2023-11-27T15:16:08.910684+0000","last_became_peered":"2023-11-27T15:16:08.910684+0000","last_unstale":"2023-11-27T15:16:11.018251+0000","last_undegraded":"2023-11-27T15:16:11.018251+0000","last_fullsized":"2023-11-27T15:16:11.018251+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:17:37.062173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018419+0000","last_change":"2023-11-27T15:16:09.341414+0000","last_active":"2023-11-27T15:16:11.018419+0000","last_peered":"2023-11-27T15:16:11.018419+0000","last_clean":"2023-11-27T15:16:11.018419+0000","last_became_active":"2023-11-27T15:16:09.341068+0000","last_became_peered":"2023-11-27T15:16:09.341068+0000","last_unstale":"2023-11-27T15:16:11.018419+0000","last_undegraded":"2023-11-27T15:16:11.018419+0000","last_fullsized":"2023-11-27T15:16:11.018419+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:08:37.252539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.743053+0000","last_change":"2023-11-27T15:16:08.915093+0000","last_active":"2023-11-27T15:16:12.743053+0000","last_peered":"2023-11-27T15:16:12.743053+0000","last_clean":"2023-11-27T15:16:12.743053+0000","last_became_active":"2023-11-27T15:16:08.914761+0000","last_became_peered":"2023-11-27T15:16:08.914761+0000","last_unstale":"2023-11-27T15:16:12.743053+0000","last_undegraded":"2023-11-27T15:16:12.743053+0000","last_fullsized":"2023-11-27T15:16:12.743053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:36:34.653519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018143+0000","last_change":"2023-11-27T15:16:09.337330+0000","last_active":"2023-11-27T15:16:11.018143+0000","last_peered":"2023-11-27T15:16:11.018143+0000","last_clean":"2023-11-27T15:16:11.018143+0000","last_became_active":"2023-11-27T15:16:09.336991+0000","last_became_peered":"2023-11-27T15:16:09.336991+0000","last_unstale":"2023-11-27T15:16:11.018143+0000","last_undegraded":"2023-11-27T15:16:11.018143+0000","last_fullsized":"2023-11-27T15:16:11.018143+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:12:42.839632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018993+0000","last_change":"2023-11-27T15:16:08.912208+0000","last_active":"2023-11-27T15:16:11.018993+0000","last_peered":"2023-11-27T15:16:11.018993+0000","last_clean":"2023-11-27T15:16:11.018993+0000","last_became_active":"2023-11-27T15:16:08.911879+0000","last_became_peered":"2023-11-27T15:16:08.911879+0000","last_unstale":"2023-11-27T15:16:11.018993+0000","last_undegraded":"2023-11-27T15:16:11.018993+0000","last_fullsized":"2023-11-27T15:16:11.018993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:05:04.291584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018417+0000","last_change":"2023-11-27T15:16:09.339660+0000","last_active":"2023-11-27T15:16:11.018417+0000","last_peered":"2023-11-27T15:16:11.018417+0000","last_clean":"2023-11-27T15:16:11.018417+0000","last_became_active":"2023-11-27T15:16:09.339323+0000","last_became_peered":"2023-11-27T15:16:09.339323+0000","last_unstale":"2023-11-27T15:16:11.018417+0000","last_undegraded":"2023-11-27T15:16:11.018417+0000","last_fullsized":"2023-11-27T15:16:11.018417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:54:38.570422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:13.750343+0000","last_change":"2023-11-27T15:16:08.911103+0000","last_active":"2023-11-27T15:16:13.750343+0000","last_peered":"2023-11-27T15:16:13.750343+0000","last_clean":"2023-11-27T15:16:13.750343+0000","last_became_active":"2023-11-27T15:16:08.910765+0000","last_became_peered":"2023-11-27T15:16:08.910765+0000","last_unstale":"2023-11-27T15:16:13.750343+0000","last_undegraded":"2023-11-27T15:16:13.750343+0000","last_fullsized":"2023-11-27T15:16:13.750343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:35:58.046428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018483+0000","last_change":"2023-11-27T15:16:09.351741+0000","last_active":"2023-11-27T15:16:11.018483+0000","last_peered":"2023-11-27T15:16:11.018483+0000","last_clean":"2023-11-27T15:16:11.018483+0000","last_became_active":"2023-11-27T15:16:09.351280+0000","last_became_peered":"2023-11-27T15:16:09.351280+0000","last_unstale":"2023-11-27T15:16:11.018483+0000","last_undegraded":"2023-11-27T15:16:11.018483+0000","last_fullsized":"2023-11-27T15:16:11.018483+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:23:19.955510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018060+0000","last_change":"2023-11-27T15:16:08.915630+0000","last_active":"2023-11-27T15:16:11.018060+0000","last_peered":"2023-11-27T15:16:11.018060+0000","last_clean":"2023-11-27T15:16:11.018060+0000","last_became_active":"2023-11-27T15:16:08.915294+0000","last_became_peered":"2023-11-27T15:16:08.915294+0000","last_unstale":"2023-11-27T15:16:11.018060+0000","last_undegraded":"2023-11-27T15:16:11.018060+0000","last_fullsized":"2023-11-27T15:16:11.018060+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:48:25.055189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018460+0000","last_change":"2023-11-27T15:16:09.338436+0000","last_active":"2023-11-27T15:16:11.018460+0000","last_peered":"2023-11-27T15:16:11.018460+0000","last_clean":"2023-11-27T15:16:11.018460+0000","last_became_active":"2023-11-27T15:16:09.338148+0000","last_became_peered":"2023-11-27T15:16:09.338148+0000","last_unstale":"2023-11-27T15:16:11.018460+0000","last_undegraded":"2023-11-27T15:16:11.018460+0000","last_fullsized":"2023-11-27T15:16:11.018460+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:11:53.119360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.743319+0000","last_change":"2023-11-27T15:16:08.913072+0000","last_active":"2023-11-27T15:16:12.743319+0000","last_peered":"2023-11-27T15:16:12.743319+0000","last_clean":"2023-11-27T15:16:12.743319+0000","last_became_active":"2023-11-27T15:16:08.912757+0000","last_became_peered":"2023-11-27T15:16:08.912757+0000","last_unstale":"2023-11-27T15:16:12.743319+0000","last_undegraded":"2023-11-27T15:16:12.743319+0000","last_fullsized":"2023-11-27T15:16:12.743319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:41:31.325128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.332666+0000","last_change":"2023-11-27T15:16:09.340380+0000","last_active":"2023-11-27T15:16:10.332666+0000","last_peered":"2023-11-27T15:16:10.332666+0000","last_clean":"2023-11-27T15:16:10.332666+0000","last_became_active":"2023-11-27T15:16:09.339581+0000","last_became_peered":"2023-11-27T15:16:09.339581+0000","last_unstale":"2023-11-27T15:16:10.332666+0000","last_undegraded":"2023-11-27T15:16:10.332666+0000","last_fullsized":"2023-11-27T15:16:10.332666+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:00:41.810407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.743621+0000","last_change":"2023-11-27T15:16:08.917924+0000","last_active":"2023-11-27T15:16:12.743621+0000","last_peered":"2023-11-27T15:16:12.743621+0000","last_clean":"2023-11-27T15:16:12.743621+0000","last_became_active":"2023-11-27T15:16:08.917618+0000","last_became_peered":"2023-11-27T15:16:08.917618+0000","last_unstale":"2023-11-27T15:16:12.743621+0000","last_undegraded":"2023-11-27T15:16:12.743621+0000","last_fullsized":"2023-11-27T15:16:12.743621+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:05:48.700431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.865352+0000","last_change":"2023-11-27T15:16:09.346020+0000","last_active":"2023-11-27T15:16:10.865352+0000","last_peered":"2023-11-27T15:16:10.865352+0000","last_clean":"2023-11-27T15:16:10.865352+0000","last_became_active":"2023-11-27T15:16:09.345822+0000","last_became_peered":"2023-11-27T15:16:09.345822+0000","last_unstale":"2023-11-27T15:16:10.865352+0000","last_undegraded":"2023-11-27T15:16:10.865352+0000","last_fullsized":"2023-11-27T15:16:10.865352+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:20:28.114199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018365+0000","last_change":"2023-11-27T15:16:08.921810+0000","last_active":"2023-11-27T15:16:11.018365+0000","last_peered":"2023-11-27T15:16:11.018365+0000","last_clean":"2023-11-27T15:16:11.018365+0000","last_became_active":"2023-11-27T15:16:08.921518+0000","last_became_peered":"2023-11-27T15:16:08.921518+0000","last_unstale":"2023-11-27T15:16:11.018365+0000","last_undegraded":"2023-11-27T15:16:11.018365+0000","last_fullsized":"2023-11-27T15:16:11.018365+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:14:14.359239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.332512+0000","last_change":"2023-11-27T15:16:09.351083+0000","last_active":"2023-11-27T15:16:10.332512+0000","last_peered":"2023-11-27T15:16:10.332512+0000","last_clean":"2023-11-27T15:16:10.332512+0000","last_became_active":"2023-11-27T15:16:09.350540+0000","last_became_peered":"2023-11-27T15:16:09.350540+0000","last_unstale":"2023-11-27T15:16:10.332512+0000","last_undegraded":"2023-11-27T15:16:10.332512+0000","last_fullsized":"2023-11-27T15:16:10.332512+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:16:25.645689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.743377+0000","last_change":"2023-11-27T15:16:08.913023+0000","last_active":"2023-11-27T15:16:12.743377+0000","last_peered":"2023-11-27T15:16:12.743377+0000","last_clean":"2023-11-27T15:16:12.743377+0000","last_became_active":"2023-11-27T15:16:08.912693+0000","last_became_peered":"2023-11-27T15:16:08.912693+0000","last_unstale":"2023-11-27T15:16:12.743377+0000","last_undegraded":"2023-11-27T15:16:12.743377+0000","last_fullsized":"2023-11-27T15:16:12.743377+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:02:06.453103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018683+0000","last_change":"2023-11-27T15:16:09.343446+0000","last_active":"2023-11-27T15:16:11.018683+0000","last_peered":"2023-11-27T15:16:11.018683+0000","last_clean":"2023-11-27T15:16:11.018683+0000","last_became_active":"2023-11-27T15:16:09.341980+0000","last_became_peered":"2023-11-27T15:16:09.341980+0000","last_unstale":"2023-11-27T15:16:11.018683+0000","last_undegraded":"2023-11-27T15:16:11.018683+0000","last_fullsized":"2023-11-27T15:16:11.018683+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:03:09.462489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.745246+0000","last_change":"2023-11-27T15:16:08.914181+0000","last_active":"2023-11-27T15:16:12.745246+0000","last_peered":"2023-11-27T15:16:12.745246+0000","last_clean":"2023-11-27T15:16:12.745246+0000","last_became_active":"2023-11-27T15:16:08.913836+0000","last_became_peered":"2023-11-27T15:16:08.913836+0000","last_unstale":"2023-11-27T15:16:12.745246+0000","last_undegraded":"2023-11-27T15:16:12.745246+0000","last_fullsized":"2023-11-27T15:16:12.745246+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:28:48.961843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018010+0000","last_change":"2023-11-27T15:16:09.344446+0000","last_active":"2023-11-27T15:16:11.018010+0000","last_peered":"2023-11-27T15:16:11.018010+0000","last_clean":"2023-11-27T15:16:11.018010+0000","last_became_active":"2023-11-27T15:16:09.344104+0000","last_became_peered":"2023-11-27T15:16:09.344104+0000","last_unstale":"2023-11-27T15:16:11.018010+0000","last_undegraded":"2023-11-27T15:16:11.018010+0000","last_fullsized":"2023-11-27T15:16:11.018010+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:09:38.847885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.744575+0000","last_change":"2023-11-27T15:16:08.915597+0000","last_active":"2023-11-27T15:16:12.744575+0000","last_peered":"2023-11-27T15:16:12.744575+0000","last_clean":"2023-11-27T15:16:12.744575+0000","last_became_active":"2023-11-27T15:16:08.914727+0000","last_became_peered":"2023-11-27T15:16:08.914727+0000","last_unstale":"2023-11-27T15:16:12.744575+0000","last_undegraded":"2023-11-27T15:16:12.744575+0000","last_fullsized":"2023-11-27T15:16:12.744575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:32:36.396885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017899+0000","last_change":"2023-11-27T15:16:09.352643+0000","last_active":"2023-11-27T15:16:11.017899+0000","last_peered":"2023-11-27T15:16:11.017899+0000","last_clean":"2023-11-27T15:16:11.017899+0000","last_became_active":"2023-11-27T15:16:09.352336+0000","last_became_peered":"2023-11-27T15:16:09.352336+0000","last_unstale":"2023-11-27T15:16:11.017899+0000","last_undegraded":"2023-11-27T15:16:11.017899+0000","last_fullsized":"2023-11-27T15:16:11.017899+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:27:00.130120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:15.756324+0000","last_change":"2023-11-27T15:16:08.914303+0000","last_active":"2023-11-27T15:16:15.756324+0000","last_peered":"2023-11-27T15:16:15.756324+0000","last_clean":"2023-11-27T15:16:15.756324+0000","last_became_active":"2023-11-27T15:16:08.913601+0000","last_became_peered":"2023-11-27T15:16:08.913601+0000","last_unstale":"2023-11-27T15:16:15.756324+0000","last_undegraded":"2023-11-27T15:16:15.756324+0000","last_fullsized":"2023-11-27T15:16:15.756324+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:35:53.186395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017672+0000","last_change":"2023-11-27T15:16:09.341342+0000","last_active":"2023-11-27T15:16:11.017672+0000","last_peered":"2023-11-27T15:16:11.017672+0000","last_clean":"2023-11-27T15:16:11.017672+0000","last_became_active":"2023-11-27T15:16:09.341100+0000","last_became_peered":"2023-11-27T15:16:09.341100+0000","last_unstale":"2023-11-27T15:16:11.017672+0000","last_undegraded":"2023-11-27T15:16:11.017672+0000","last_fullsized":"2023-11-27T15:16:11.017672+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:52:19.668174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'10","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.741758+0000","last_change":"2023-11-27T15:16:08.921771+0000","last_active":"2023-11-27T15:16:12.741758+0000","last_peered":"2023-11-27T15:16:12.741758+0000","last_clean":"2023-11-27T15:16:12.741758+0000","last_became_active":"2023-11-27T15:16:08.921396+0000","last_became_peered":"2023-11-27T15:16:08.921396+0000","last_unstale":"2023-11-27T15:16:12.741758+0000","last_undegraded":"2023-11-27T15:16:12.741758+0000","last_fullsized":"2023-11-27T15:16:12.741758+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:32:00.774950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018456+0000","last_change":"2023-11-27T15:16:09.341258+0000","last_active":"2023-11-27T15:16:11.018456+0000","last_peered":"2023-11-27T15:16:11.018456+0000","last_clean":"2023-11-27T15:16:11.018456+0000","last_became_active":"2023-11-27T15:16:09.340921+0000","last_became_peered":"2023-11-27T15:16:09.340921+0000","last_unstale":"2023-11-27T15:16:11.018456+0000","last_undegraded":"2023-11-27T15:16:11.018456+0000","last_fullsized":"2023-11-27T15:16:11.018456+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:59:25.144109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018494+0000","last_change":"2023-11-27T15:16:08.911219+0000","last_active":"2023-11-27T15:16:11.018494+0000","last_peered":"2023-11-27T15:16:11.018494+0000","last_clean":"2023-11-27T15:16:11.018494+0000","last_became_active":"2023-11-27T15:16:08.910788+0000","last_became_peered":"2023-11-27T15:16:08.910788+0000","last_unstale":"2023-11-27T15:16:11.018494+0000","last_undegraded":"2023-11-27T15:16:11.018494+0000","last_fullsized":"2023-11-27T15:16:11.018494+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:02:11.224624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329287+0000","last_change":"2023-11-27T15:16:09.343793+0000","last_active":"2023-11-27T15:16:10.329287+0000","last_peered":"2023-11-27T15:16:10.329287+0000","last_clean":"2023-11-27T15:16:10.329287+0000","last_became_active":"2023-11-27T15:16:09.343480+0000","last_became_peered":"2023-11-27T15:16:09.343480+0000","last_unstale":"2023-11-27T15:16:10.329287+0000","last_undegraded":"2023-11-27T15:16:10.329287+0000","last_fullsized":"2023-11-27T15:16:10.329287+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:23:15.936537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018282+0000","last_change":"2023-11-27T15:16:08.920651+0000","last_active":"2023-11-27T15:16:11.018282+0000","last_peered":"2023-11-27T15:16:11.018282+0000","last_clean":"2023-11-27T15:16:11.018282+0000","last_became_active":"2023-11-27T15:16:08.920294+0000","last_became_peered":"2023-11-27T15:16:08.920294+0000","last_unstale":"2023-11-27T15:16:11.018282+0000","last_undegraded":"2023-11-27T15:16:11.018282+0000","last_fullsized":"2023-11-27T15:16:11.018282+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:57:51.796562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.019096+0000","last_change":"2023-11-27T15:16:09.348279+0000","last_active":"2023-11-27T15:16:11.019096+0000","last_peered":"2023-11-27T15:16:11.019096+0000","last_clean":"2023-11-27T15:16:11.019096+0000","last_became_active":"2023-11-27T15:16:09.347760+0000","last_became_peered":"2023-11-27T15:16:09.347760+0000","last_unstale":"2023-11-27T15:16:11.019096+0000","last_undegraded":"2023-11-27T15:16:11.019096+0000","last_fullsized":"2023-11-27T15:16:11.019096+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:35:32.477154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.170186+0000","last_change":"2023-11-27T15:16:08.915622+0000","last_active":"2023-11-27T15:16:11.170186+0000","last_peered":"2023-11-27T15:16:11.170186+0000","last_clean":"2023-11-27T15:16:11.170186+0000","last_became_active":"2023-11-27T15:16:08.915293+0000","last_became_peered":"2023-11-27T15:16:08.915293+0000","last_unstale":"2023-11-27T15:16:11.170186+0000","last_undegraded":"2023-11-27T15:16:11.170186+0000","last_fullsized":"2023-11-27T15:16:11.170186+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:42:31.199595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.019331+0000","last_change":"2023-11-27T15:16:09.337096+0000","last_active":"2023-11-27T15:16:11.019331+0000","last_peered":"2023-11-27T15:16:11.019331+0000","last_clean":"2023-11-27T15:16:11.019331+0000","last_became_active":"2023-11-27T15:16:09.336359+0000","last_became_peered":"2023-11-27T15:16:09.336359+0000","last_unstale":"2023-11-27T15:16:11.019331+0000","last_undegraded":"2023-11-27T15:16:11.019331+0000","last_fullsized":"2023-11-27T15:16:11.019331+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:01:29.480942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018063+0000","last_change":"2023-11-27T15:16:09.340478+0000","last_active":"2023-11-27T15:16:11.018063+0000","last_peered":"2023-11-27T15:16:11.018063+0000","last_clean":"2023-11-27T15:16:11.018063+0000","last_became_active":"2023-11-27T15:16:09.340214+0000","last_became_peered":"2023-11-27T15:16:09.340214+0000","last_unstale":"2023-11-27T15:16:11.018063+0000","last_undegraded":"2023-11-27T15:16:11.018063+0000","last_fullsized":"2023-11-27T15:16:11.018063+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:09:06.499851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:13.751044+0000","last_change":"2023-11-27T15:16:08.920387+0000","last_active":"2023-11-27T15:16:13.751044+0000","last_peered":"2023-11-27T15:16:13.751044+0000","last_clean":"2023-11-27T15:16:13.751044+0000","last_became_active":"2023-11-27T15:16:08.920004+0000","last_became_peered":"2023-11-27T15:16:08.920004+0000","last_unstale":"2023-11-27T15:16:13.751044+0000","last_undegraded":"2023-11-27T15:16:13.751044+0000","last_fullsized":"2023-11-27T15:16:13.751044+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:02:23.895980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018879+0000","last_change":"2023-11-27T15:16:09.352291+0000","last_active":"2023-11-27T15:16:11.018879+0000","last_peered":"2023-11-27T15:16:11.018879+0000","last_clean":"2023-11-27T15:16:11.018879+0000","last_became_active":"2023-11-27T15:16:09.351944+0000","last_became_peered":"2023-11-27T15:16:09.351944+0000","last_unstale":"2023-11-27T15:16:11.018879+0000","last_undegraded":"2023-11-27T15:16:11.018879+0000","last_fullsized":"2023-11-27T15:16:11.018879+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:59:40.185533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:13.748748+0000","last_change":"2023-11-27T15:16:08.909694+0000","last_active":"2023-11-27T15:16:13.748748+0000","last_peered":"2023-11-27T15:16:13.748748+0000","last_clean":"2023-11-27T15:16:13.748748+0000","last_became_active":"2023-11-27T15:16:08.909324+0000","last_became_peered":"2023-11-27T15:16:08.909324+0000","last_unstale":"2023-11-27T15:16:13.748748+0000","last_undegraded":"2023-11-27T15:16:13.748748+0000","last_fullsized":"2023-11-27T15:16:13.748748+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:37:52.187833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":340,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017780+0000","last_change":"2023-11-27T15:15:10.678805+0000","last_active":"2023-11-27T15:16:11.017780+0000","last_peered":"2023-11-27T15:16:11.017780+0000","last_clean":"2023-11-27T15:16:11.017780+0000","last_became_active":"2023-11-27T15:15:10.678267+0000","last_became_peered":"2023-11-27T15:15:10.678267+0000","last_unstale":"2023-11-27T15:16:11.017780+0000","last_undegraded":"2023-11-27T15:16:11.017780+0000","last_fullsized":"2023-11-27T15:16:11.017780+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:12:14.179315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:12:14.179315+0000","last_clean_scrub_stamp":"2023-11-27T15:12:14.179315+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:53:29.305985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017431+0000","last_change":"2023-11-27T15:16:09.346622+0000","last_active":"2023-11-27T15:16:11.017431+0000","last_peered":"2023-11-27T15:16:11.017431+0000","last_clean":"2023-11-27T15:16:11.017431+0000","last_became_active":"2023-11-27T15:16:09.346287+0000","last_became_peered":"2023-11-27T15:16:09.346287+0000","last_unstale":"2023-11-27T15:16:11.017431+0000","last_undegraded":"2023-11-27T15:16:11.017431+0000","last_fullsized":"2023-11-27T15:16:11.017431+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:06:51.424504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:13.749235+0000","last_change":"2023-11-27T15:16:08.915598+0000","last_active":"2023-11-27T15:16:13.749235+0000","last_peered":"2023-11-27T15:16:13.749235+0000","last_clean":"2023-11-27T15:16:13.749235+0000","last_became_active":"2023-11-27T15:16:08.915255+0000","last_became_peered":"2023-11-27T15:16:08.915255+0000","last_unstale":"2023-11-27T15:16:13.749235+0000","last_undegraded":"2023-11-27T15:16:13.749235+0000","last_fullsized":"2023-11-27T15:16:13.749235+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:59:36.190517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018749+0000","last_change":"2023-11-27T15:16:08.921855+0000","last_active":"2023-11-27T15:16:11.018749+0000","last_peered":"2023-11-27T15:16:11.018749+0000","last_clean":"2023-11-27T15:16:11.018749+0000","last_became_active":"2023-11-27T15:16:08.921525+0000","last_became_peered":"2023-11-27T15:16:08.921525+0000","last_unstale":"2023-11-27T15:16:11.018749+0000","last_undegraded":"2023-11-27T15:16:11.018749+0000","last_fullsized":"2023-11-27T15:16:11.018749+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:18:46.190023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017617+0000","last_change":"2023-11-27T15:16:09.345119+0000","last_active":"2023-11-27T15:16:11.017617+0000","last_peered":"2023-11-27T15:16:11.017617+0000","last_clean":"2023-11-27T15:16:11.017617+0000","last_became_active":"2023-11-27T15:16:09.344807+0000","last_became_peered":"2023-11-27T15:16:09.344807+0000","last_unstale":"2023-11-27T15:16:11.017617+0000","last_undegraded":"2023-11-27T15:16:11.017617+0000","last_fullsized":"2023-11-27T15:16:11.017617+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:35:43.938240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:13.750300+0000","last_change":"2023-11-27T15:16:08.911564+0000","last_active":"2023-11-27T15:16:13.750300+0000","last_peered":"2023-11-27T15:16:13.750300+0000","last_clean":"2023-11-27T15:16:13.750300+0000","last_became_active":"2023-11-27T15:16:08.911057+0000","last_became_peered":"2023-11-27T15:16:08.911057+0000","last_unstale":"2023-11-27T15:16:13.750300+0000","last_undegraded":"2023-11-27T15:16:13.750300+0000","last_fullsized":"2023-11-27T15:16:13.750300+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:12:28.660065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018236+0000","last_change":"2023-11-27T15:16:09.339085+0000","last_active":"2023-11-27T15:16:11.018236+0000","last_peered":"2023-11-27T15:16:11.018236+0000","last_clean":"2023-11-27T15:16:11.018236+0000","last_became_active":"2023-11-27T15:16:09.338777+0000","last_became_peered":"2023-11-27T15:16:09.338777+0000","last_unstale":"2023-11-27T15:16:11.018236+0000","last_undegraded":"2023-11-27T15:16:11.018236+0000","last_fullsized":"2023-11-27T15:16:11.018236+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:37:58.241374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.744087+0000","last_change":"2023-11-27T15:16:08.913939+0000","last_active":"2023-11-27T15:16:12.744087+0000","last_peered":"2023-11-27T15:16:12.744087+0000","last_clean":"2023-11-27T15:16:12.744087+0000","last_became_active":"2023-11-27T15:16:08.913629+0000","last_became_peered":"2023-11-27T15:16:08.913629+0000","last_unstale":"2023-11-27T15:16:12.744087+0000","last_undegraded":"2023-11-27T15:16:12.744087+0000","last_fullsized":"2023-11-27T15:16:12.744087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:46:10.524140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.865269+0000","last_change":"2023-11-27T15:16:09.350712+0000","last_active":"2023-11-27T15:16:10.865269+0000","last_peered":"2023-11-27T15:16:10.865269+0000","last_clean":"2023-11-27T15:16:10.865269+0000","last_became_active":"2023-11-27T15:16:09.350359+0000","last_became_peered":"2023-11-27T15:16:09.350359+0000","last_unstale":"2023-11-27T15:16:10.865269+0000","last_undegraded":"2023-11-27T15:16:10.865269+0000","last_fullsized":"2023-11-27T15:16:10.865269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:30:08.385906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.744120+0000","last_change":"2023-11-27T15:16:08.919501+0000","last_active":"2023-11-27T15:16:12.744120+0000","last_peered":"2023-11-27T15:16:12.744120+0000","last_clean":"2023-11-27T15:16:12.744120+0000","last_became_active":"2023-11-27T15:16:08.919258+0000","last_became_peered":"2023-11-27T15:16:08.919258+0000","last_unstale":"2023-11-27T15:16:12.744120+0000","last_undegraded":"2023-11-27T15:16:12.744120+0000","last_fullsized":"2023-11-27T15:16:12.744120+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:12:35.863121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017921+0000","last_change":"2023-11-27T15:16:09.344066+0000","last_active":"2023-11-27T15:16:11.017921+0000","last_peered":"2023-11-27T15:16:11.017921+0000","last_clean":"2023-11-27T15:16:11.017921+0000","last_became_active":"2023-11-27T15:16:09.343757+0000","last_became_peered":"2023-11-27T15:16:09.343757+0000","last_unstale":"2023-11-27T15:16:11.017921+0000","last_undegraded":"2023-11-27T15:16:11.017921+0000","last_fullsized":"2023-11-27T15:16:11.017921+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:45:10.744579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.744748+0000","last_change":"2023-11-27T15:16:08.906205+0000","last_active":"2023-11-27T15:16:12.744748+0000","last_peered":"2023-11-27T15:16:12.744748+0000","last_clean":"2023-11-27T15:16:12.744748+0000","last_became_active":"2023-11-27T15:16:08.905885+0000","last_became_peered":"2023-11-27T15:16:08.905885+0000","last_unstale":"2023-11-27T15:16:12.744748+0000","last_undegraded":"2023-11-27T15:16:12.744748+0000","last_fullsized":"2023-11-27T15:16:12.744748+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:15:37.488708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.328941+0000","last_change":"2023-11-27T15:16:09.349642+0000","last_active":"2023-11-27T15:16:10.328941+0000","last_peered":"2023-11-27T15:16:10.328941+0000","last_clean":"2023-11-27T15:16:10.328941+0000","last_became_active":"2023-11-27T15:16:09.349086+0000","last_became_peered":"2023-11-27T15:16:09.349086+0000","last_unstale":"2023-11-27T15:16:10.328941+0000","last_undegraded":"2023-11-27T15:16:10.328941+0000","last_fullsized":"2023-11-27T15:16:10.328941+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:54:34.111407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329196+0000","last_change":"2023-11-27T15:16:08.917835+0000","last_active":"2023-11-27T15:16:10.329196+0000","last_peered":"2023-11-27T15:16:10.329196+0000","last_clean":"2023-11-27T15:16:10.329196+0000","last_became_active":"2023-11-27T15:16:08.917532+0000","last_became_peered":"2023-11-27T15:16:08.917532+0000","last_unstale":"2023-11-27T15:16:10.329196+0000","last_undegraded":"2023-11-27T15:16:10.329196+0000","last_fullsized":"2023-11-27T15:16:10.329196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:35:27.771748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.019054+0000","last_change":"2023-11-27T15:16:09.338459+0000","last_active":"2023-11-27T15:16:11.019054+0000","last_peered":"2023-11-27T15:16:11.019054+0000","last_clean":"2023-11-27T15:16:11.019054+0000","last_became_active":"2023-11-27T15:16:09.338148+0000","last_became_peered":"2023-11-27T15:16:09.338148+0000","last_unstale":"2023-11-27T15:16:11.019054+0000","last_undegraded":"2023-11-27T15:16:11.019054+0000","last_fullsized":"2023-11-27T15:16:11.019054+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:50:17.721161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.742934+0000","last_change":"2023-11-27T15:16:08.915401+0000","last_active":"2023-11-27T15:16:12.742934+0000","last_peered":"2023-11-27T15:16:12.742934+0000","last_clean":"2023-11-27T15:16:12.742934+0000","last_became_active":"2023-11-27T15:16:08.915012+0000","last_became_peered":"2023-11-27T15:16:08.915012+0000","last_unstale":"2023-11-27T15:16:12.742934+0000","last_undegraded":"2023-11-27T15:16:12.742934+0000","last_fullsized":"2023-11-27T15:16:12.742934+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:25:39.081595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018186+0000","last_change":"2023-11-27T15:16:09.352309+0000","last_active":"2023-11-27T15:16:11.018186+0000","last_peered":"2023-11-27T15:16:11.018186+0000","last_clean":"2023-11-27T15:16:11.018186+0000","last_became_active":"2023-11-27T15:16:09.352039+0000","last_became_peered":"2023-11-27T15:16:09.352039+0000","last_unstale":"2023-11-27T15:16:11.018186+0000","last_undegraded":"2023-11-27T15:16:11.018186+0000","last_fullsized":"2023-11-27T15:16:11.018186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:02:54.926789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329777+0000","last_change":"2023-11-27T15:16:08.920446+0000","last_active":"2023-11-27T15:16:10.329777+0000","last_peered":"2023-11-27T15:16:10.329777+0000","last_clean":"2023-11-27T15:16:10.329777+0000","last_became_active":"2023-11-27T15:16:08.920232+0000","last_became_peered":"2023-11-27T15:16:08.920232+0000","last_unstale":"2023-11-27T15:16:10.329777+0000","last_undegraded":"2023-11-27T15:16:10.329777+0000","last_fullsized":"2023-11-27T15:16:10.329777+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:28:22.523270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018226+0000","last_change":"2023-11-27T15:16:09.344328+0000","last_active":"2023-11-27T15:16:11.018226+0000","last_peered":"2023-11-27T15:16:11.018226+0000","last_clean":"2023-11-27T15:16:11.018226+0000","last_became_active":"2023-11-27T15:16:09.344047+0000","last_became_peered":"2023-11-27T15:16:09.344047+0000","last_unstale":"2023-11-27T15:16:11.018226+0000","last_undegraded":"2023-11-27T15:16:11.018226+0000","last_fullsized":"2023-11-27T15:16:11.018226+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:10:10.564306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018175+0000","last_change":"2023-11-27T15:16:08.909865+0000","last_active":"2023-11-27T15:16:11.018175+0000","last_peered":"2023-11-27T15:16:11.018175+0000","last_clean":"2023-11-27T15:16:11.018175+0000","last_became_active":"2023-11-27T15:16:08.909559+0000","last_became_peered":"2023-11-27T15:16:08.909559+0000","last_unstale":"2023-11-27T15:16:11.018175+0000","last_undegraded":"2023-11-27T15:16:11.018175+0000","last_fullsized":"2023-11-27T15:16:11.018175+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:21:20.674879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.170220+0000","last_change":"2023-11-27T15:16:09.342525+0000","last_active":"2023-11-27T15:16:11.170220+0000","last_peered":"2023-11-27T15:16:11.170220+0000","last_clean":"2023-11-27T15:16:11.170220+0000","last_became_active":"2023-11-27T15:16:09.340987+0000","last_became_peered":"2023-11-27T15:16:09.340987+0000","last_unstale":"2023-11-27T15:16:11.170220+0000","last_undegraded":"2023-11-27T15:16:11.170220+0000","last_fullsized":"2023-11-27T15:16:11.170220+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:57:33.064763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017934+0000","last_change":"2023-11-27T15:16:08.914088+0000","last_active":"2023-11-27T15:16:11.017934+0000","last_peered":"2023-11-27T15:16:11.017934+0000","last_clean":"2023-11-27T15:16:11.017934+0000","last_became_active":"2023-11-27T15:16:08.913750+0000","last_became_peered":"2023-11-27T15:16:08.913750+0000","last_unstale":"2023-11-27T15:16:11.017934+0000","last_undegraded":"2023-11-27T15:16:11.017934+0000","last_fullsized":"2023-11-27T15:16:11.017934+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:52:50.990435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018281+0000","last_change":"2023-11-27T15:16:09.344403+0000","last_active":"2023-11-27T15:16:11.018281+0000","last_peered":"2023-11-27T15:16:11.018281+0000","last_clean":"2023-11-27T15:16:11.018281+0000","last_became_active":"2023-11-27T15:16:09.344120+0000","last_became_peered":"2023-11-27T15:16:09.344120+0000","last_unstale":"2023-11-27T15:16:11.018281+0000","last_undegraded":"2023-11-27T15:16:11.018281+0000","last_fullsized":"2023-11-27T15:16:11.018281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:47:52.787834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.328999+0000","last_change":"2023-11-27T15:16:09.349780+0000","last_active":"2023-11-27T15:16:10.328999+0000","last_peered":"2023-11-27T15:16:10.328999+0000","last_clean":"2023-11-27T15:16:10.328999+0000","last_became_active":"2023-11-27T15:16:09.349493+0000","last_became_peered":"2023-11-27T15:16:09.349493+0000","last_unstale":"2023-11-27T15:16:10.328999+0000","last_undegraded":"2023-11-27T15:16:10.328999+0000","last_fullsized":"2023-11-27T15:16:10.328999+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:11:07.759158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329634+0000","last_change":"2023-11-27T15:16:09.340528+0000","last_active":"2023-11-27T15:16:10.329634+0000","last_peered":"2023-11-27T15:16:10.329634+0000","last_clean":"2023-11-27T15:16:10.329634+0000","last_became_active":"2023-11-27T15:16:09.339435+0000","last_became_peered":"2023-11-27T15:16:09.339435+0000","last_unstale":"2023-11-27T15:16:10.329634+0000","last_undegraded":"2023-11-27T15:16:10.329634+0000","last_fullsized":"2023-11-27T15:16:10.329634+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:08:23.254070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329091+0000","last_change":"2023-11-27T15:16:09.337989+0000","last_active":"2023-11-27T15:16:10.329091+0000","last_peered":"2023-11-27T15:16:10.329091+0000","last_clean":"2023-11-27T15:16:10.329091+0000","last_became_active":"2023-11-27T15:16:09.336712+0000","last_became_peered":"2023-11-27T15:16:09.336712+0000","last_unstale":"2023-11-27T15:16:10.329091+0000","last_undegraded":"2023-11-27T15:16:10.329091+0000","last_fullsized":"2023-11-27T15:16:10.329091+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:53:19.057499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.865031+0000","last_change":"2023-11-27T15:16:09.347077+0000","last_active":"2023-11-27T15:16:10.865031+0000","last_peered":"2023-11-27T15:16:10.865031+0000","last_clean":"2023-11-27T15:16:10.865031+0000","last_became_active":"2023-11-27T15:16:09.346722+0000","last_became_peered":"2023-11-27T15:16:09.346722+0000","last_unstale":"2023-11-27T15:16:10.865031+0000","last_undegraded":"2023-11-27T15:16:10.865031+0000","last_fullsized":"2023-11-27T15:16:10.865031+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:09:22.987678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017743+0000","last_change":"2023-11-27T15:16:09.344317+0000","last_active":"2023-11-27T15:16:11.017743+0000","last_peered":"2023-11-27T15:16:11.017743+0000","last_clean":"2023-11-27T15:16:11.017743+0000","last_became_active":"2023-11-27T15:16:09.343962+0000","last_became_peered":"2023-11-27T15:16:09.343962+0000","last_unstale":"2023-11-27T15:16:11.017743+0000","last_undegraded":"2023-11-27T15:16:11.017743+0000","last_fullsized":"2023-11-27T15:16:11.017743+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:49:36.883005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329146+0000","last_change":"2023-11-27T15:16:09.344690+0000","last_active":"2023-11-27T15:16:10.329146+0000","last_peered":"2023-11-27T15:16:10.329146+0000","last_clean":"2023-11-27T15:16:10.329146+0000","last_became_active":"2023-11-27T15:16:09.344259+0000","last_became_peered":"2023-11-27T15:16:09.344259+0000","last_unstale":"2023-11-27T15:16:10.329146+0000","last_undegraded":"2023-11-27T15:16:10.329146+0000","last_fullsized":"2023-11-27T15:16:10.329146+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:47:31.777045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329769+0000","last_change":"2023-11-27T15:16:09.340459+0000","last_active":"2023-11-27T15:16:10.329769+0000","last_peered":"2023-11-27T15:16:10.329769+0000","last_clean":"2023-11-27T15:16:10.329769+0000","last_became_active":"2023-11-27T15:16:09.340140+0000","last_became_peered":"2023-11-27T15:16:09.340140+0000","last_unstale":"2023-11-27T15:16:10.329769+0000","last_undegraded":"2023-11-27T15:16:10.329769+0000","last_fullsized":"2023-11-27T15:16:10.329769+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:51:52.323186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018191+0000","last_change":"2023-11-27T15:16:09.336625+0000","last_active":"2023-11-27T15:16:11.018191+0000","last_peered":"2023-11-27T15:16:11.018191+0000","last_clean":"2023-11-27T15:16:11.018191+0000","last_became_active":"2023-11-27T15:16:09.336287+0000","last_became_peered":"2023-11-27T15:16:09.336287+0000","last_unstale":"2023-11-27T15:16:11.018191+0000","last_undegraded":"2023-11-27T15:16:11.018191+0000","last_fullsized":"2023-11-27T15:16:11.018191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:19:26.038950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017922+0000","last_change":"2023-11-27T15:16:09.335480+0000","last_active":"2023-11-27T15:16:11.017922+0000","last_peered":"2023-11-27T15:16:11.017922+0000","last_clean":"2023-11-27T15:16:11.017922+0000","last_became_active":"2023-11-27T15:16:09.335163+0000","last_became_peered":"2023-11-27T15:16:09.335163+0000","last_unstale":"2023-11-27T15:16:11.017922+0000","last_undegraded":"2023-11-27T15:16:11.017922+0000","last_fullsized":"2023-11-27T15:16:11.017922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:27:19.391538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017703+0000","last_change":"2023-11-27T15:16:09.336879+0000","last_active":"2023-11-27T15:16:11.017703+0000","last_peered":"2023-11-27T15:16:11.017703+0000","last_clean":"2023-11-27T15:16:11.017703+0000","last_became_active":"2023-11-27T15:16:09.336580+0000","last_became_peered":"2023-11-27T15:16:09.336580+0000","last_unstale":"2023-11-27T15:16:11.017703+0000","last_undegraded":"2023-11-27T15:16:11.017703+0000","last_fullsized":"2023-11-27T15:16:11.017703+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:45:37.465813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.332430+0000","last_change":"2023-11-27T15:16:09.341400+0000","last_active":"2023-11-27T15:16:10.332430+0000","last_peered":"2023-11-27T15:16:10.332430+0000","last_clean":"2023-11-27T15:16:10.332430+0000","last_became_active":"2023-11-27T15:16:09.341103+0000","last_became_peered":"2023-11-27T15:16:09.341103+0000","last_unstale":"2023-11-27T15:16:10.332430+0000","last_undegraded":"2023-11-27T15:16:10.332430+0000","last_fullsized":"2023-11-27T15:16:10.332430+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:59:30.634393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018153+0000","last_change":"2023-11-27T15:16:09.345938+0000","last_active":"2023-11-27T15:16:11.018153+0000","last_peered":"2023-11-27T15:16:11.018153+0000","last_clean":"2023-11-27T15:16:11.018153+0000","last_became_active":"2023-11-27T15:16:09.345624+0000","last_became_peered":"2023-11-27T15:16:09.345624+0000","last_unstale":"2023-11-27T15:16:11.018153+0000","last_undegraded":"2023-11-27T15:16:11.018153+0000","last_fullsized":"2023-11-27T15:16:11.018153+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:03:33.299715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329231+0000","last_change":"2023-11-27T15:16:09.343896+0000","last_active":"2023-11-27T15:16:10.329231+0000","last_peered":"2023-11-27T15:16:10.329231+0000","last_clean":"2023-11-27T15:16:10.329231+0000","last_became_active":"2023-11-27T15:16:09.343603+0000","last_became_peered":"2023-11-27T15:16:09.343603+0000","last_unstale":"2023-11-27T15:16:10.329231+0000","last_undegraded":"2023-11-27T15:16:10.329231+0000","last_fullsized":"2023-11-27T15:16:10.329231+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:58:06.401944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":249380,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4774,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":107,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":229376,"data_stored":31807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94,"ondisk_log_size":94,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449100,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5932,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":93739220,"statfs":{"total":95995035648,"available":95988961280,"internally_reserved":0,"allocated":888832,"data_stored":426296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]}]},{"osd":10,"up_from":71,"seq":304942678032,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6196,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":93738956,"statfs":{"total":95995035648,"available":95988690944,"internally_reserved":0,"allocated":962560,"data_stored":1018422,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":1,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.248},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.27100000000000002}]},{"osd":2,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.39000000000000001}]},{"osd":3,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.441},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.307}]},{"osd":4,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.374},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.35599999999999998}]},{"osd":5,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.46800000000000003}]},{"osd":6,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.45900000000000002}]},{"osd":7,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.53300000000000003}]},{"osd":8,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.504}]},{"osd":9,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.53000000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]},{"osd":9,"up_from":66,"seq":283467841557,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6080,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739072,"statfs":{"total":95995035648,"available":95988809728,"internally_reserved":0,"allocated":909312,"data_stored":427114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.502},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.52100000000000002}]},{"osd":1,"last update":"Mon Nov 27 15:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.69699999999999995}]},{"osd":2,"last update":"Mon Nov 27 15:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.629},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.59899999999999998}]},{"osd":3,"last update":"Mon Nov 27 15:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.65500000000000003}]},{"osd":4,"last update":"Mon Nov 27 15:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.75800000000000001}]},{"osd":5,"last update":"Mon Nov 27 15:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.70899999999999996}]},{"osd":6,"last update":"Mon Nov 27 15:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.64500000000000002}]},{"osd":7,"last update":"Mon Nov 27 15:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.85299999999999998}]},{"osd":8,"last update":"Mon Nov 27 15:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.72099999999999997}]},{"osd":10,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.84099999999999997}]}]},{"osd":8,"up_from":58,"seq":249108103193,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6260,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738892,"statfs":{"total":95995035648,"available":95988625408,"internally_reserved":0,"allocated":897024,"data_stored":426900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.42299999999999999}]},{"osd":1,"last update":"Mon Nov 27 15:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.54500000000000004}]},{"osd":2,"last update":"Mon Nov 27 15:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.63200000000000001}]},{"osd":3,"last update":"Mon Nov 27 15:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.57799999999999996}]},{"osd":4,"last update":"Mon Nov 27 15:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.495},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.64800000000000002}]},{"osd":5,"last update":"Mon Nov 27 15:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.55600000000000005}]},{"osd":6,"last update":"Mon Nov 27 15:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.70699999999999996}]},{"osd":7,"last update":"Mon Nov 27 15:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.68300000000000005}]},{"osd":9,"last update":"Mon Nov 27 15:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.66200000000000003}]},{"osd":10,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.754}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]}]},{"osd":7,"up_from":50,"seq":214748364829,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6384,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738768,"statfs":{"total":95995035648,"available":95988498432,"internally_reserved":0,"allocated":958464,"data_stored":1019263,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.35599999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.76300000000000001,"5min":1.052,"15min":1.052},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.51800000000000002,"15min":0.503},"min":{"1min":0.315,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.38700000000000001}]},{"osd":1,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.39400000000000002,"15min":0.38200000000000001},"min":{"1min":0.33400000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.60499999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.40999999999999998,"15min":0.39900000000000002},"min":{"1min":0.28199999999999997,"5min":0.254,"15min":0.254},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.54600000000000004}]},{"osd":2,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.39500000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.629,"5min":0.55300000000000005,"15min":0.54100000000000004},"min":{"1min":0.41499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.51100000000000001}]},{"osd":3,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.54000000000000004,"15min":0.52400000000000002},"min":{"1min":0.38800000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.59299999999999997,"15min":0.57999999999999996},"min":{"1min":0.45700000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88300000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.66900000000000004}]},{"osd":4,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.49299999999999999,"15min":0.47499999999999998},"min":{"1min":0.43099999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.624},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.48999999999999999,"15min":0.47399999999999998},"min":{"1min":0.41399999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.73299999999999998}]},{"osd":5,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.61799999999999999,"15min":0.59499999999999997},"min":{"1min":0.47099999999999997,"5min":0.375,"15min":0.375},"max":{"1min":2.3719999999999999,"5min":2.3719999999999999,"15min":2.3719999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.64100000000000001,"15min":0.61199999999999999},"min":{"1min":0.44700000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":3.1059999999999999,"5min":3.1059999999999999,"15min":3.1059999999999999},"last":0.70299999999999996}]},{"osd":6,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.66600000000000004,"15min":0.65400000000000003},"min":{"1min":0.51200000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.95299999999999996,"5min":1.008,"15min":1.008},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.625,"15min":0.61099999999999999},"min":{"1min":0.52900000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.81499999999999995}]},{"osd":8,"last update":"Mon Nov 27 15:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.82599999999999996}]},{"osd":9,"last update":"Mon Nov 27 15:15:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.83799999999999997}]},{"osd":10,"last update":"Mon Nov 27 15:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.86499999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]}]},{"osd":6,"up_from":45,"seq":193273528354,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6256,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738896,"statfs":{"total":95995035648,"available":95988629504,"internally_reserved":0,"allocated":892928,"data_stored":426386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.44600000000000001,"15min":0.44800000000000001},"min":{"1min":0.309,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.65500000000000003,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.52900000000000003,"15min":0.54100000000000004},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.70599999999999996,"5min":2.508,"15min":2.508},"last":0.39600000000000002}]},{"osd":1,"last update":"Mon Nov 27 15:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.59499999999999997,"15min":0.58699999999999997},"min":{"1min":0.44600000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.88400000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.63700000000000001,"15min":0.64500000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85899999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":0.55700000000000005}]},{"osd":2,"last update":"Mon Nov 27 15:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.67400000000000004,"15min":0.68200000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.81499999999999995,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65500000000000003,"15min":0.65300000000000002},"min":{"1min":0.48499999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.84499999999999997,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.502}]},{"osd":3,"last update":"Mon Nov 27 15:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.53700000000000003,"15min":0.53300000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79000000000000004,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.57999999999999996,"15min":0.59799999999999998},"min":{"1min":0.30599999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.63300000000000001,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":0.433}]},{"osd":4,"last update":"Mon Nov 27 15:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.42099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0529999999999999,"5min":1.51,"15min":1.51},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.65100000000000002,"15min":0.64100000000000001},"min":{"1min":0.48999999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.748}]},{"osd":5,"last update":"Mon Nov 27 15:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.72499999999999998,"15min":0.74099999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92200000000000004,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.71999999999999997,"15min":0.72499999999999998},"min":{"1min":0.52200000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.93600000000000005,"5min":1.599,"15min":1.599},"last":0.63600000000000001}]},{"osd":7,"last update":"Mon Nov 27 15:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72599999999999998,"15min":0.72499999999999998},"min":{"1min":0.46000000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.995,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.73999999999999999,"15min":0.73599999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.79700000000000004}]},{"osd":8,"last update":"Mon Nov 27 15:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.766,"5min":2.766,"15min":2.766},"last":0.61599999999999999}]},{"osd":9,"last update":"Mon Nov 27 15:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.90000000000000002}]},{"osd":10,"last update":"Mon Nov 27 15:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.86499999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7176,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":93737976,"statfs":{"total":95995035648,"available":95987687424,"internally_reserved":0,"allocated":917504,"data_stored":445225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.502,"15min":0.46999999999999997},"min":{"1min":0.34000000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77000000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.373},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.505,"15min":0.46500000000000002},"min":{"1min":0.35499999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.42999999999999999}]},{"osd":2,"last update":"Mon Nov 27 15:15:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.53500000000000003,"15min":0.53700000000000003},"min":{"1min":0.36199999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.67300000000000004,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52800000000000002,"15min":0.52200000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67500000000000004,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.72999999999999998}]},{"osd":3,"last update":"Mon Nov 27 15:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.55000000000000004,"15min":0.52800000000000002},"min":{"1min":0.35999999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.86799999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.51400000000000001}]},{"osd":4,"last update":"Mon Nov 27 15:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.44,"15min":0.42799999999999999},"min":{"1min":0.314,"5min":0.251,"15min":0.251},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.307},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.45500000000000002,"15min":0.441},"min":{"1min":0.26400000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.45500000000000002}]},{"osd":5,"last update":"Mon Nov 27 15:15:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58299999999999996,"15min":0.58399999999999996},"min":{"1min":0.44,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.71899999999999997,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.745},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61599999999999999,"15min":0.61799999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.69599999999999995,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.79500000000000004}]},{"osd":6,"last update":"Mon Nov 27 15:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58199999999999996,"15min":0.57499999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.625,"15min":0.623},"min":{"1min":0.39800000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.59999999999999998}]},{"osd":7,"last update":"Mon Nov 27 15:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.53600000000000003,"15min":0.52600000000000002},"min":{"1min":0.38,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.58199999999999996,"15min":0.57299999999999995},"min":{"1min":0.432,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.33900000000000002}]},{"osd":8,"last update":"Mon Nov 27 15:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.82199999999999995}]},{"osd":9,"last update":"Mon Nov 27 15:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.66900000000000004}]},{"osd":10,"last update":"Mon Nov 27 15:16:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.53600000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738427,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7160,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":93737992,"statfs":{"total":95995035648,"available":95987703808,"internally_reserved":0,"allocated":901120,"data_stored":426990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 27 15:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.58199999999999996,"15min":0.54600000000000004},"min":{"1min":0.45900000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.97099999999999997,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.622},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.60499999999999998,"15min":0.55900000000000005},"min":{"1min":0.51000000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.018,"5min":1.55,"15min":1.55},"last":0.53600000000000003}]},{"osd":2,"last update":"Mon Nov 27 15:15:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.55700000000000005,"15min":0.53700000000000003},"min":{"1min":0.40100000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.53000000000000003,"15min":0.51500000000000001},"min":{"1min":0.44400000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.435}]},{"osd":3,"last update":"Mon Nov 27 15:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.38900000000000001,"15min":0.371},"min":{"1min":0.26000000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.441,"15min":0.42899999999999999},"min":{"1min":0.30399999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.754}]},{"osd":4,"last update":"Mon Nov 27 15:16:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.57699999999999996,"15min":0.55000000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.96099999999999997,"5min":0.999,"15min":0.999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.58899999999999997,"15min":0.55900000000000005},"min":{"1min":0.38300000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.56599999999999995}]},{"osd":5,"last update":"Mon Nov 27 15:15:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.625,"15min":0.61499999999999999},"min":{"1min":0.45900000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.626,"15min":0.61099999999999999},"min":{"1min":0.53800000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.60699999999999998}]},{"osd":6,"last update":"Mon Nov 27 15:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.56000000000000005,"15min":0.54800000000000004},"min":{"1min":0.29099999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.60299999999999998,"15min":0.59099999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.73499999999999999}]},{"osd":7,"last update":"Mon Nov 27 15:16:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67800000000000005,"15min":0.67200000000000004},"min":{"1min":0.434,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.46100000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.31}]},{"osd":8,"last update":"Mon Nov 27 15:15:26 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.77600000000000002}]},{"osd":9,"last update":"Mon Nov 27 15:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.71899999999999997}]},{"osd":10,"last update":"Mon Nov 27 15:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.69299999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411378,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7164,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":93737988,"statfs":{"total":95995035648,"available":95987699712,"internally_reserved":0,"allocated":905216,"data_stored":427894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.49099999999999999,"15min":0.47099999999999997},"min":{"1min":0.38500000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.79000000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.48899999999999999,"15min":0.47699999999999998},"min":{"1min":0.35499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.45800000000000002}]},{"osd":1,"last update":"Mon Nov 27 15:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52000000000000002,"15min":0.50600000000000001},"min":{"1min":0.34399999999999997,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.503,"15min":0.48599999999999999},"min":{"1min":0.36399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.60399999999999998}]},{"osd":3,"last update":"Mon Nov 27 15:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.55600000000000005,"15min":0.53200000000000003},"min":{"1min":0.56000000000000005,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.55700000000000005,"15min":0.53300000000000003},"min":{"1min":0.46200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.55000000000000004}]},{"osd":4,"last update":"Mon Nov 27 15:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.38400000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.76200000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.35799999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79200000000000004,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.51000000000000001}]},{"osd":5,"last update":"Mon Nov 27 15:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.41699999999999998,"15min":0.39500000000000002},"min":{"1min":0.32200000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.41899999999999998,"15min":0.40699999999999997},"min":{"1min":0.35999999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.36199999999999999}]},{"osd":6,"last update":"Mon Nov 27 15:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.435,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.629,"15min":0.61799999999999999},"min":{"1min":0.495,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.441}]},{"osd":7,"last update":"Mon Nov 27 15:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.39800000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.44},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69199999999999995,"15min":0.69599999999999995},"min":{"1min":0.46300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93300000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.51100000000000001}]},{"osd":8,"last update":"Mon Nov 27 15:15:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.379},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.42999999999999999}]},{"osd":9,"last update":"Mon Nov 27 15:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.57299999999999995}]},{"osd":10,"last update":"Mon Nov 27 15:16:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.623},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.67000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6520,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738632,"statfs":{"total":95995035648,"available":95988359168,"internally_reserved":0,"allocated":901120,"data_stored":426490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.34999999999999998,"15min":0.33000000000000002},"min":{"1min":0.28299999999999997,"5min":0.22,"15min":0.22},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.36799999999999999,"15min":0.35199999999999998},"min":{"1min":0.33700000000000002,"5min":0.214,"15min":0.214},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.38800000000000001}]},{"osd":1,"last update":"Mon Nov 27 15:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.52500000000000002,"15min":0.501},"min":{"1min":0.432,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81999999999999995,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.53200000000000003,"15min":0.52200000000000002},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.44600000000000001}]},{"osd":2,"last update":"Mon Nov 27 15:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.57299999999999995,"15min":0.54100000000000004},"min":{"1min":0.48399999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.56399999999999995,"15min":0.53800000000000003},"min":{"1min":0.47099999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.63300000000000001}]},{"osd":4,"last update":"Mon Nov 27 15:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.54300000000000004,"15min":0.52400000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.56000000000000005,"15min":0.52200000000000002},"min":{"1min":0.495,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.52800000000000002}]},{"osd":5,"last update":"Mon Nov 27 15:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.47099999999999997,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.72099999999999997}]},{"osd":6,"last update":"Mon Nov 27 15:15:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.56899999999999995,"15min":0.55000000000000004},"min":{"1min":0.46899999999999997,"5min":0.37,"15min":0.37},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.56000000000000005,"15min":0.54400000000000004},"min":{"1min":0.42299999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.51300000000000001}]},{"osd":7,"last update":"Mon Nov 27 15:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.496},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.72499999999999998,"15min":0.70899999999999996},"min":{"1min":0.59599999999999997,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.70099999999999996}]},{"osd":8,"last update":"Mon Nov 27 15:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.628}]},{"osd":9,"last update":"Mon Nov 27 15:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68799999999999994}]},{"osd":10,"last update":"Mon Nov 27 15:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.67600000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920811,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6392,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738760,"statfs":{"total":95995035648,"available":95988490240,"internally_reserved":0,"allocated":901120,"data_stored":427282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.48099999999999998,"15min":0.46800000000000003},"min":{"1min":0.35299999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.52200000000000002,"15min":0.50700000000000001},"min":{"1min":0.40300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.432}]},{"osd":1,"last update":"Mon Nov 27 15:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.36899999999999999,"15min":0.34799999999999998},"min":{"1min":0.30299999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.36899999999999999,"15min":0.34699999999999998},"min":{"1min":0.28399999999999997,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.318}]},{"osd":2,"last update":"Mon Nov 27 15:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.50700000000000001,"15min":0.48899999999999999},"min":{"1min":0.36699999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.54500000000000004,"15min":0.53500000000000003},"min":{"1min":0.40200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.50800000000000001}]},{"osd":3,"last update":"Mon Nov 27 15:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.53900000000000003,"15min":0.51400000000000001},"min":{"1min":0.45100000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.40999999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80700000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.48899999999999999}]},{"osd":5,"last update":"Mon Nov 27 15:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.378,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.375,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.60399999999999998}]},{"osd":6,"last update":"Mon Nov 27 15:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59199999999999997,"15min":0.58599999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.628,"15min":0.622},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.54000000000000004}]},{"osd":7,"last update":"Mon Nov 27 15:16:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.34000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.376},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.34899999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.39400000000000002}]},{"osd":8,"last update":"Mon Nov 27 15:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.65400000000000003}]},{"osd":9,"last update":"Mon Nov 27 15:15:49 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.51900000000000002}]},{"osd":10,"last update":"Mon Nov 27 15:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.40899999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]}]},{"osd":5,"up_from":38,"seq":163208757286,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6520,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738632,"statfs":{"total":95995035648,"available":95988359168,"internally_reserved":0,"allocated":966656,"data_stored":1018092,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.501,"15min":0.49399999999999999},"min":{"1min":0.36099999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.47799999999999998,"15min":0.46999999999999997},"min":{"1min":0.33900000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.36699999999999999}]},{"osd":1,"last update":"Mon Nov 27 15:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.53400000000000003,"15min":0.52400000000000002},"min":{"1min":0.433,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.52200000000000002,"15min":0.50900000000000001},"min":{"1min":0.41099999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.54700000000000004}]},{"osd":2,"last update":"Mon Nov 27 15:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.39100000000000001,"15min":0.38700000000000001},"min":{"1min":0.30199999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.442,"5min":0.40600000000000003,"15min":0.40000000000000002},"min":{"1min":0.28000000000000003,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.42299999999999999}]},{"osd":3,"last update":"Mon Nov 27 15:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55000000000000004,"15min":0.54700000000000004},"min":{"1min":0.374,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.38600000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.443}]},{"osd":4,"last update":"Mon Nov 27 15:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.55400000000000005,"15min":0.54800000000000004},"min":{"1min":0.434,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.58199999999999996,"15min":0.57699999999999996},"min":{"1min":0.44600000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.40799999999999997}]},{"osd":6,"last update":"Mon Nov 27 15:15:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58899999999999997,"15min":0.58699999999999997},"min":{"1min":0.45300000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.86399999999999999,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.85599999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.64000000000000001}]},{"osd":7,"last update":"Mon Nov 27 15:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.50600000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.48099999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.56100000000000005}]},{"osd":8,"last update":"Mon Nov 27 15:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.54000000000000004}]},{"osd":9,"last update":"Mon Nov 27 15:15:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.499},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.67100000000000004}]},{"osd":10,"last update":"Mon Nov 27 15:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.59899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34455,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34455,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":18929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-27T15:16:17.907 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-27T15:16:17.907 INFO:tasks.cephadm:Setting up client nodes... 2023-11-27T15:16:17.907 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-11-27T15:16:18.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:17 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/801939919' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:16:18.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:17 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/745563203' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:16:18.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:17 smithi150 ceph-mon[100559]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:16:18.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:17 smithi150 ceph-mon[100559]: pgmap v260: 97 pgs: 97 active+clean; 581 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 975 B/s rd, 3.7 KiB/s wr, 14 op/s 2023-11-27T15:16:18.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:17 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/801939919' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:16:18.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:17 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/745563203' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:16:18.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:17 smithi102 ceph-mon[100296]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:16:18.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:17 smithi102 ceph-mon[100296]: pgmap v260: 97 pgs: 97 active+clean; 581 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 975 B/s rd, 3.7 KiB/s wr, 14 op/s 2023-11-27T15:16:18.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:17 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/801939919' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:16:18.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:17 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/745563203' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:16:18.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:17 smithi052 ceph-mon[99056]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:16:18.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:17 smithi052 ceph-mon[99056]: pgmap v260: 97 pgs: 97 active+clean; 581 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 975 B/s rd, 3.7 KiB/s wr, 14 op/s 2023-11-27T15:16:18.589 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:19.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:18 smithi150 ceph-mon[100559]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:16:19.040 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:18 smithi052 ceph-mon[99056]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:16:19.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:18 smithi102 ceph-mon[100296]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:16:19.957 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.951+0000 7fb1b179f700 1 -- 172.21.15.52:0/1487679829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1ac0feff0 msgr2=0x7fb1ac0ff400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:19.957 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.951+0000 7fb1b179f700 1 --2- 172.21.15.52:0/1487679829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1ac0feff0 0x7fb1ac0ff400 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb19c009990 tx=0x7fb19c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:16:19.957 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.952+0000 7fb1b179f700 1 -- 172.21.15.52:0/1487679829 shutdown_connections 2023-11-27T15:16:19.957 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.952+0000 7fb1b179f700 1 --2- 172.21.15.52:0/1487679829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1ac100b80 0x7fb1ac107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:19.957 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.952+0000 7fb1b179f700 1 --2- 172.21.15.52:0/1487679829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1ac0feff0 0x7fb1ac0ff400 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:19.957 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.952+0000 7fb1b179f700 1 --2- 172.21.15.52:0/1487679829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1ac1001f0 0x7fb1ac100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:19.958 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.952+0000 7fb1b179f700 1 -- 172.21.15.52:0/1487679829 >> 172.21.15.52:0/1487679829 conn(0x7fb1ac0fa620 msgr2=0x7fb1ac0fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:19.958 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.952+0000 7fb1b179f700 1 -- 172.21.15.52:0/1487679829 shutdown_connections 2023-11-27T15:16:19.958 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.952+0000 7fb1b179f700 1 -- 172.21.15.52:0/1487679829 wait complete. 2023-11-27T15:16:19.958 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.952+0000 7fb1b179f700 1 Processor -- start 2023-11-27T15:16:19.958 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.952+0000 7fb1b179f700 1 -- start start 2023-11-27T15:16:19.960 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.953+0000 7fb1b179f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1ac0feff0 0x7fb1ac069170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:19.960 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.953+0000 7fb1b179f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1ac1001f0 0x7fb1ac19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:19.960 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.953+0000 7fb1b179f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1ac100b80 0x7fb1ac19d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:19.960 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.953+0000 7fb1b179f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac19d970 con 0x7fb1ac0feff0 2023-11-27T15:16:19.960 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.953+0000 7fb1b179f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac19dae0 con 0x7fb1ac100b80 2023-11-27T15:16:19.960 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.953+0000 7fb1b179f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac19dca0 con 0x7fb1ac1001f0 2023-11-27T15:16:19.960 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.953+0000 7fb1abfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1ac0feff0 0x7fb1ac069170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:19.960 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.953+0000 7fb1abfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1ac0feff0 0x7fb1ac069170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50242/0 (socket says 172.21.15.52:50242) 2023-11-27T15:16:19.961 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.953+0000 7fb1abfff700 1 -- 172.21.15.52:0/4254089956 learned_addr learned my addr 172.21.15.52:0/4254089956 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:19.961 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.953+0000 7fb1b0f9e700 1 --2- 172.21.15.52:0/4254089956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1ac100b80 0x7fb1ac19d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:19.961 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.953+0000 7fb1ab7fe700 1 --2- 172.21.15.52:0/4254089956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1ac1001f0 0x7fb1ac19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:19.961 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.953+0000 7fb1abfff700 1 -- 172.21.15.52:0/4254089956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1ac1001f0 msgr2=0x7fb1ac19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:19.961 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.953+0000 7fb1abfff700 1 --2- 172.21.15.52:0/4254089956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1ac1001f0 0x7fb1ac19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:19.961 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.953+0000 7fb1abfff700 1 -- 172.21.15.52:0/4254089956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1ac100b80 msgr2=0x7fb1ac19d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:19.961 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.953+0000 7fb1abfff700 1 --2- 172.21.15.52:0/4254089956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1ac100b80 0x7fb1ac19d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:19.961 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.953+0000 7fb1abfff700 1 -- 172.21.15.52:0/4254089956 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19c0095f0 con 0x7fb1ac0feff0 2023-11-27T15:16:19.961 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.954+0000 7fb1abfff700 1 --2- 172.21.15.52:0/4254089956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1ac0feff0 0x7fb1ac069170 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb19c000c00 tx=0x7fb19c00f700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:19.961 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.954+0000 7fb1a97fa700 1 -- 172.21.15.52:0/4254089956 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c01d070 con 0x7fb1ac0feff0 2023-11-27T15:16:19.961 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.954+0000 7fb1b179f700 1 -- 172.21.15.52:0/4254089956 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1ac10a9c0 con 0x7fb1ac0feff0 2023-11-27T15:16:19.962 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.954+0000 7fb1a97fa700 1 -- 172.21.15.52:0/4254089956 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb19c022470 con 0x7fb1ac0feff0 2023-11-27T15:16:19.962 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.954+0000 7fb1a97fa700 1 -- 172.21.15.52:0/4254089956 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c017630 con 0x7fb1ac0feff0 2023-11-27T15:16:19.962 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.954+0000 7fb1b179f700 1 -- 172.21.15.52:0/4254089956 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1ac10a830 con 0x7fb1ac0feff0 2023-11-27T15:16:19.962 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.955+0000 7fb1b179f700 1 -- 172.21.15.52:0/4254089956 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1ac1065c0 con 0x7fb1ac0feff0 2023-11-27T15:16:19.962 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.957+0000 7fb1a97fa700 1 -- 172.21.15.52:0/4254089956 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb19c0228e0 con 0x7fb1ac0feff0 2023-11-27T15:16:19.963 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.957+0000 7fb1a97fa700 1 --2- 172.21.15.52:0/4254089956 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb1940a11e0 0x7fb1940a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:19.963 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.958+0000 7fb1ab7fe700 1 --2- 172.21.15.52:0/4254089956 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb1940a11e0 0x7fb1940a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:19.963 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.958+0000 7fb1a97fa700 1 -- 172.21.15.52:0/4254089956 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb19c0ce230 con 0x7fb1ac0feff0 2023-11-27T15:16:19.963 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.958+0000 7fb1ab7fe700 1 --2- 172.21.15.52:0/4254089956 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb1940a11e0 0x7fb1940a3690 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0009a10 tx=0x7fb1a0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:19.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:19.961+0000 7fb1a97fa700 1 -- 172.21.15.52:0/4254089956 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb19c097960 con 0x7fb1ac0feff0 2023-11-27T15:16:20.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:19 smithi150 ceph-mon[100559]: pgmap v261: 97 pgs: 97 active+clean; 616 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.7 KiB/s wr, 18 op/s 2023-11-27T15:16:20.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:19 smithi102 ceph-mon[100296]: pgmap v261: 97 pgs: 97 active+clean; 616 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.7 KiB/s wr, 18 op/s 2023-11-27T15:16:20.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:19 smithi052 ceph-mon[99056]: pgmap v261: 97 pgs: 97 active+clean; 616 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.7 KiB/s wr, 18 op/s 2023-11-27T15:16:20.243 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:20.239+0000 7fb1b179f700 1 -- 172.21.15.52:0/4254089956 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fb1ac061700 con 0x7fb1ac0feff0 2023-11-27T15:16:20.247 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:20.243+0000 7fb1a97fa700 1 -- 172.21.15.52:0/4254089956 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7fb19c09af80 con 0x7fb1ac0feff0 2023-11-27T15:16:20.248 INFO:teuthology.orchestra.run.smithi052.stdout:[client.0] 2023-11-27T15:16:20.248 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQDEsmRlA7pRDhAAifhxHn3WZObrZWjX9znn/w== 2023-11-27T15:16:20.250 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:20.245+0000 7fb1b179f700 1 -- 172.21.15.52:0/4254089956 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb1940a11e0 msgr2=0x7fb1940a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:20.250 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:20.245+0000 7fb1b179f700 1 --2- 172.21.15.52:0/4254089956 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb1940a11e0 0x7fb1940a3690 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0009a10 tx=0x7fb1a0009190 comp rx=0 tx=0).stop 2023-11-27T15:16:20.250 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:20.245+0000 7fb1b179f700 1 -- 172.21.15.52:0/4254089956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1ac0feff0 msgr2=0x7fb1ac069170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:20.250 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:20.245+0000 7fb1b179f700 1 --2- 172.21.15.52:0/4254089956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1ac0feff0 0x7fb1ac069170 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb19c000c00 tx=0x7fb19c00f700 comp rx=0 tx=0).stop 2023-11-27T15:16:20.250 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:20.246+0000 7fb1b179f700 1 -- 172.21.15.52:0/4254089956 shutdown_connections 2023-11-27T15:16:20.250 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:20.246+0000 7fb1b179f700 1 --2- 172.21.15.52:0/4254089956 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb1940a11e0 0x7fb1940a3690 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:20.251 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:20.246+0000 7fb1b179f700 1 --2- 172.21.15.52:0/4254089956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1ac100b80 0x7fb1ac19d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:20.251 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:20.246+0000 7fb1b179f700 1 --2- 172.21.15.52:0/4254089956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1ac0feff0 0x7fb1ac069170 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:20.251 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:20.246+0000 7fb1b179f700 1 --2- 172.21.15.52:0/4254089956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1ac1001f0 0x7fb1ac19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:20.251 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:20.246+0000 7fb1b179f700 1 -- 172.21.15.52:0/4254089956 >> 172.21.15.52:0/4254089956 conn(0x7fb1ac0fa620 msgr2=0x7fb1ac103700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:20.251 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:20.246+0000 7fb1b179f700 1 -- 172.21.15.52:0/4254089956 shutdown_connections 2023-11-27T15:16:20.251 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:20.246+0000 7fb1b179f700 1 -- 172.21.15.52:0/4254089956 wait complete. 2023-11-27T15:16:20.649 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2023-11-27T15:16:20.649 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-11-27T15:16:20.649 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-27T15:16:20.691 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-27T15:16:20.691 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-11-27T15:16:20.692 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph mgr dump --format=json 2023-11-27T15:16:21.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:20 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/4254089956' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-11-27T15:16:21.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:20 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/4254089956' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-11-27T15:16:21.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:20 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/4254089956' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-11-27T15:16:21.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:20 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/4254089956' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-11-27T15:16:21.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:20 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/4254089956' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-11-27T15:16:21.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:20 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/4254089956' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-11-27T15:16:21.384 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:21.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:21 smithi052 ceph-mon[99056]: pgmap v262: 97 pgs: 97 active+clean; 616 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.8 KiB/s wr, 17 op/s 2023-11-27T15:16:22.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:21 smithi150 ceph-mon[100559]: pgmap v262: 97 pgs: 97 active+clean; 616 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.8 KiB/s wr, 17 op/s 2023-11-27T15:16:22.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:21 smithi102 ceph-mon[100296]: pgmap v262: 97 pgs: 97 active+clean; 616 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.8 KiB/s wr, 17 op/s 2023-11-27T15:16:22.740 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.734+0000 7fd9f0e4b700 1 -- 172.21.15.52:0/3323881271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec0ffe00 msgr2=0x7fd9ec103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:22.741 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.734+0000 7fd9f0e4b700 1 --2- 172.21.15.52:0/3323881271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec0ffe00 0x7fd9ec103dd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc0099c0 tx=0x7fd9dc009cd0 comp rx=0 tx=0).stop 2023-11-27T15:16:22.741 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.734+0000 7fd9f0e4b700 1 -- 172.21.15.52:0/3323881271 shutdown_connections 2023-11-27T15:16:22.741 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.734+0000 7fd9f0e4b700 1 --2- 172.21.15.52:0/3323881271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9ec104500 0x7fd9ec10c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:22.741 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.734+0000 7fd9f0e4b700 1 --2- 172.21.15.52:0/3323881271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9ec0ff460 0x7fd9ec0ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:22.741 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.734+0000 7fd9f0e4b700 1 --2- 172.21.15.52:0/3323881271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec0ffe00 0x7fd9ec103dd0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:22.741 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.734+0000 7fd9f0e4b700 1 -- 172.21.15.52:0/3323881271 >> 172.21.15.52:0/3323881271 conn(0x7fd9ec0770a0 msgr2=0x7fd9ec0774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:22.741 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.734+0000 7fd9f0e4b700 1 -- 172.21.15.52:0/3323881271 shutdown_connections 2023-11-27T15:16:22.741 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.734+0000 7fd9f0e4b700 1 -- 172.21.15.52:0/3323881271 wait complete. 2023-11-27T15:16:22.741 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.734+0000 7fd9f0e4b700 1 Processor -- start 2023-11-27T15:16:22.742 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.735+0000 7fd9f0e4b700 1 -- start start 2023-11-27T15:16:22.742 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.735+0000 7fd9f0e4b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9ec0ff460 0x7fd9ec1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:22.742 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.735+0000 7fd9f0e4b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9ec0ffe00 0x7fd9ec198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:22.742 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.735+0000 7fd9f0e4b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec104500 0x7fd9ec19cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:22.742 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.735+0000 7fd9f0e4b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec19d560 con 0x7fd9ec104500 2023-11-27T15:16:22.742 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.735+0000 7fd9f0e4b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec19d6d0 con 0x7fd9ec0ff460 2023-11-27T15:16:22.742 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.735+0000 7fd9f0e4b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec19d890 con 0x7fd9ec0ffe00 2023-11-27T15:16:22.742 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.735+0000 7fd9ea59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9ec0ff460 0x7fd9ec1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:22.742 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.735+0000 7fd9ead9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec104500 0x7fd9ec19cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:22.742 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.735+0000 7fd9ea59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9ec0ff460 0x7fd9ec1985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:39914/0 (socket says 172.21.15.52:39914) 2023-11-27T15:16:22.742 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.735+0000 7fd9ea59c700 1 -- 172.21.15.52:0/3580175173 learned_addr learned my addr 172.21.15.52:0/3580175173 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:22.743 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.735+0000 7fd9ead9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec104500 0x7fd9ec19cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50260/0 (socket says 172.21.15.52:50260) 2023-11-27T15:16:22.743 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.735+0000 7fd9e9d9b700 1 --2- 172.21.15.52:0/3580175173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9ec0ffe00 0x7fd9ec198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:22.743 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.735+0000 7fd9ead9d700 1 -- 172.21.15.52:0/3580175173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9ec0ffe00 msgr2=0x7fd9ec198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:22.743 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.735+0000 7fd9ead9d700 1 --2- 172.21.15.52:0/3580175173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9ec0ffe00 0x7fd9ec198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:22.743 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.735+0000 7fd9ead9d700 1 -- 172.21.15.52:0/3580175173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9ec0ff460 msgr2=0x7fd9ec1985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:22.743 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.735+0000 7fd9ead9d700 1 --2- 172.21.15.52:0/3580175173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9ec0ff460 0x7fd9ec1985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:22.743 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.735+0000 7fd9ead9d700 1 -- 172.21.15.52:0/3580175173 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9dc0095f0 con 0x7fd9ec104500 2023-11-27T15:16:22.743 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.736+0000 7fd9ead9d700 1 --2- 172.21.15.52:0/3580175173 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec104500 0x7fd9ec19cf50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd9e400ab40 tx=0x7fd9e400ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:22.743 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.736+0000 7fd9e37fe700 1 -- 172.21.15.52:0/3580175173 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e400f070 con 0x7fd9ec104500 2023-11-27T15:16:22.743 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.736+0000 7fd9f0e4b700 1 -- 172.21.15.52:0/3580175173 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9ec10e190 con 0x7fd9ec104500 2023-11-27T15:16:22.743 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.736+0000 7fd9e37fe700 1 -- 172.21.15.52:0/3580175173 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9e4007a70 con 0x7fd9ec104500 2023-11-27T15:16:22.743 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.736+0000 7fd9e37fe700 1 -- 172.21.15.52:0/3580175173 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4005280 con 0x7fd9ec104500 2023-11-27T15:16:22.744 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.736+0000 7fd9f0e4b700 1 -- 172.21.15.52:0/3580175173 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9ec10e000 con 0x7fd9ec104500 2023-11-27T15:16:22.744 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.737+0000 7fd9f0e4b700 1 -- 172.21.15.52:0/3580175173 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9ec10e320 con 0x7fd9ec104500 2023-11-27T15:16:22.745 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.738+0000 7fd9e37fe700 1 -- 172.21.15.52:0/3580175173 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fd9e4004ad0 con 0x7fd9ec104500 2023-11-27T15:16:22.745 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.739+0000 7fd9e37fe700 1 --2- 172.21.15.52:0/3580175173 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd9d80a11c0 0x7fd9d80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:22.745 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.739+0000 7fd9ea59c700 1 --2- 172.21.15.52:0/3580175173 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd9d80a11c0 0x7fd9d80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:22.745 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.739+0000 7fd9e37fe700 1 -- 172.21.15.52:0/3580175173 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd9e40cbcc0 con 0x7fd9ec104500 2023-11-27T15:16:22.745 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.739+0000 7fd9ea59c700 1 --2- 172.21.15.52:0/3580175173 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd9d80a11c0 0x7fd9d80a3670 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400ad20 tx=0x7fd9d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:22.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.743+0000 7fd9e37fe700 1 -- 172.21.15.52:0/3580175173 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd9e40954f0 con 0x7fd9ec104500 2023-11-27T15:16:23.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:22.997+0000 7fd9f0e4b700 1 -- 172.21.15.52:0/3580175173 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fd9ec1a5620 con 0x7fd9ec104500 2023-11-27T15:16:23.005 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:23.001+0000 7fd9e37fe700 1 -- 172.21.15.52:0/3580175173 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260126 (secure 0 0 0) 0x7fd9e4098b10 con 0x7fd9ec104500 2023-11-27T15:16:23.009 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:16:23.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:23.004+0000 7fd9f0e4b700 1 -- 172.21.15.52:0/3580175173 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd9d80a11c0 msgr2=0x7fd9d80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:23.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:23.004+0000 7fd9f0e4b700 1 --2- 172.21.15.52:0/3580175173 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd9d80a11c0 0x7fd9d80a3670 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400ad20 tx=0x7fd9d4005f90 comp rx=0 tx=0).stop 2023-11-27T15:16:23.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:23.004+0000 7fd9f0e4b700 1 -- 172.21.15.52:0/3580175173 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec104500 msgr2=0x7fd9ec19cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:23.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:23.004+0000 7fd9f0e4b700 1 --2- 172.21.15.52:0/3580175173 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec104500 0x7fd9ec19cf50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd9e400ab40 tx=0x7fd9e400ae50 comp rx=0 tx=0).stop 2023-11-27T15:16:23.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:23.004+0000 7fd9f0e4b700 1 -- 172.21.15.52:0/3580175173 shutdown_connections 2023-11-27T15:16:23.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:23.004+0000 7fd9f0e4b700 1 --2- 172.21.15.52:0/3580175173 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd9d80a11c0 0x7fd9d80a3670 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:23.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:23.004+0000 7fd9f0e4b700 1 --2- 172.21.15.52:0/3580175173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9ec0ff460 0x7fd9ec1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:23.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:23.004+0000 7fd9f0e4b700 1 --2- 172.21.15.52:0/3580175173 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec104500 0x7fd9ec19cf50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:23.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:23.004+0000 7fd9f0e4b700 1 --2- 172.21.15.52:0/3580175173 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9ec0ffe00 0x7fd9ec198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:23.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:23.004+0000 7fd9f0e4b700 1 -- 172.21.15.52:0/3580175173 >> 172.21.15.52:0/3580175173 conn(0x7fd9ec0770a0 msgr2=0x7fd9ec0fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:23.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:23.004+0000 7fd9f0e4b700 1 -- 172.21.15.52:0/3580175173 shutdown_connections 2023-11-27T15:16:23.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:23.005+0000 7fd9f0e4b700 1 -- 172.21.15.52:0/3580175173 wait complete. 2023-11-27T15:16:23.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:23 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3580175173' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-11-27T15:16:23.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:23 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3580175173' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-11-27T15:16:23.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:23 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3580175173' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-11-27T15:16:23.532 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":17,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2640036585},{"type":"v1","addr":"172.21.15.52:6801","nonce":2640036585}]},"active_addr":"172.21.15.52:6801/2640036585","active_change":"2023-11-27T15:10:03.644990+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.52:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.52:0","nonce":2690561203}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.52:0","nonce":4081249946}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.52:0","nonce":4025191157}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.52:0","nonce":2454989397}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.52:0","nonce":3635306215}]}]} 2023-11-27T15:16:23.536 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-11-27T15:16:23.537 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-11-27T15:16:23.537 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph osd dump --format=json 2023-11-27T15:16:24.197 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:24.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:24 smithi102 ceph-mon[100296]: pgmap v263: 97 pgs: 97 active+clean; 621 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.8 KiB/s wr, 17 op/s 2023-11-27T15:16:24.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:24 smithi052 ceph-mon[99056]: pgmap v263: 97 pgs: 97 active+clean; 621 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.8 KiB/s wr, 17 op/s 2023-11-27T15:16:24.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:24 smithi150 ceph-mon[100559]: pgmap v263: 97 pgs: 97 active+clean; 621 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.8 KiB/s wr, 17 op/s 2023-11-27T15:16:25.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.768+0000 7f769e0ae700 1 -- 172.21.15.52:0/1785610581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76980ffe00 msgr2=0x7f7698103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:25.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.768+0000 7f769e0ae700 1 --2- 172.21.15.52:0/1785610581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76980ffe00 0x7f7698103dd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f768c0099c0 tx=0x7f768c009cd0 comp rx=0 tx=0).stop 2023-11-27T15:16:25.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.768+0000 7f769e0ae700 1 -- 172.21.15.52:0/1785610581 shutdown_connections 2023-11-27T15:16:25.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.768+0000 7f769e0ae700 1 --2- 172.21.15.52:0/1785610581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76980ff460 0x7f76980ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:25.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.768+0000 7f769e0ae700 1 --2- 172.21.15.52:0/1785610581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7698104500 0x7f769810c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:25.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.768+0000 7f769e0ae700 1 --2- 172.21.15.52:0/1785610581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76980ffe00 0x7f7698103dd0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:25.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.768+0000 7f769e0ae700 1 -- 172.21.15.52:0/1785610581 >> 172.21.15.52:0/1785610581 conn(0x7f76980770a0 msgr2=0x7f76980774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:25.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.768+0000 7f769e0ae700 1 -- 172.21.15.52:0/1785610581 shutdown_connections 2023-11-27T15:16:25.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.768+0000 7f769e0ae700 1 -- 172.21.15.52:0/1785610581 wait complete. 2023-11-27T15:16:25.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.769+0000 7f769e0ae700 1 Processor -- start 2023-11-27T15:16:25.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.769+0000 7f769e0ae700 1 -- start start 2023-11-27T15:16:25.775 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.769+0000 7f769e0ae700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76980ff460 0x7f7698195330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:25.775 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.769+0000 7f769e0ae700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76980ffe00 0x7f7698195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:25.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.769+0000 7f769e0ae700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7698104500 0x7f7698199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:25.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.769+0000 7f769e0ae700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769819a2f0 con 0x7f76980ffe00 2023-11-27T15:16:25.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.769+0000 7f769e0ae700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769819a460 con 0x7f76980ff460 2023-11-27T15:16:25.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.769+0000 7f769e0ae700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769819a620 con 0x7f7698104500 2023-11-27T15:16:25.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.769+0000 7f7696ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76980ffe00 0x7f7698195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:25.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.769+0000 7f7696ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76980ffe00 0x7f7698195870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50270/0 (socket says 172.21.15.52:50270) 2023-11-27T15:16:25.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.769+0000 7f7696ffd700 1 -- 172.21.15.52:0/3946502738 learned_addr learned my addr 172.21.15.52:0/3946502738 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:25.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.770+0000 7f7697fff700 1 --2- 172.21.15.52:0/3946502738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7698104500 0x7f7698199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:25.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.770+0000 7f76977fe700 1 --2- 172.21.15.52:0/3946502738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76980ff460 0x7f7698195330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:25.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.770+0000 7f7696ffd700 1 -- 172.21.15.52:0/3946502738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7698104500 msgr2=0x7f7698199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:25.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.770+0000 7f7696ffd700 1 --2- 172.21.15.52:0/3946502738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7698104500 0x7f7698199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:25.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.770+0000 7f7696ffd700 1 -- 172.21.15.52:0/3946502738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76980ff460 msgr2=0x7f7698195330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:25.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.770+0000 7f7696ffd700 1 --2- 172.21.15.52:0/3946502738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76980ff460 0x7f7698195330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:25.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.770+0000 7f7696ffd700 1 -- 172.21.15.52:0/3946502738 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f768c0095f0 con 0x7f76980ffe00 2023-11-27T15:16:25.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.770+0000 7f76977fe700 1 --2- 172.21.15.52:0/3946502738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76980ff460 0x7f7698195330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:16:25.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.770+0000 7f7696ffd700 1 --2- 172.21.15.52:0/3946502738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76980ffe00 0x7f7698195870 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f768c009990 tx=0x7f768c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:25.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.770+0000 7f7694ff9700 1 -- 172.21.15.52:0/3946502738 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c01d070 con 0x7f76980ffe00 2023-11-27T15:16:25.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.770+0000 7f769e0ae700 1 -- 172.21.15.52:0/3946502738 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f769810f290 con 0x7f76980ffe00 2023-11-27T15:16:25.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.770+0000 7f7694ff9700 1 -- 172.21.15.52:0/3946502738 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f768c0054f0 con 0x7f76980ffe00 2023-11-27T15:16:25.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.770+0000 7f7694ff9700 1 -- 172.21.15.52:0/3946502738 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c003ad0 con 0x7f76980ffe00 2023-11-27T15:16:25.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.770+0000 7f769e0ae700 1 -- 172.21.15.52:0/3946502738 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f769810f100 con 0x7f76980ffe00 2023-11-27T15:16:25.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.771+0000 7f769e0ae700 1 -- 172.21.15.52:0/3946502738 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f769810f420 con 0x7f76980ffe00 2023-11-27T15:16:25.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.772+0000 7f7694ff9700 1 -- 172.21.15.52:0/3946502738 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f768c0047a0 con 0x7f76980ffe00 2023-11-27T15:16:25.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.773+0000 7f7694ff9700 1 --2- 172.21.15.52:0/3946502738 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f76800a1250 0x7f76800a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:25.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.773+0000 7f7694ff9700 1 -- 172.21.15.52:0/3946502738 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f768c0cd6a0 con 0x7f76980ffe00 2023-11-27T15:16:25.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.773+0000 7f76977fe700 1 --2- 172.21.15.52:0/3946502738 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f76800a1250 0x7f76800a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:25.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.773+0000 7f76977fe700 1 --2- 172.21.15.52:0/3946502738 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f76800a1250 0x7f76800a3700 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f76880099d0 tx=0x7f7688008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:25.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.777+0000 7f7694ff9700 1 -- 172.21.15.52:0/3946502738 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f768c096ed0 con 0x7f76980ffe00 2023-11-27T15:16:25.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.975+0000 7f769e0ae700 1 -- 172.21.15.52:0/3946502738 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7698066ea0 con 0x7f76980ffe00 2023-11-27T15:16:25.981 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.976+0000 7f7694ff9700 1 -- 172.21.15.52:0/3946502738 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22130 (secure 0 0 0) 0x7f768c09a4f0 con 0x7f76980ffe00 2023-11-27T15:16:25.981 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:16:25.981 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":83,"fsid":"c6c7df88-8d36-11ee-95a2-87774f69a715","created":"2023-11-27T15:09:03.333345+0000","modified":"2023-11-27T15:16:10.321506+0000","last_up_change":"2023-11-27T15:15:27.656134+0000","last_in_change":"2023-11-27T15:15:10.671153+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:12:14.177523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:16:07.842251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:16:08.728520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58d69ede-c293-49f5-abbe-8187793f2b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6803","nonce":608216415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6805","nonce":608216415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6809","nonce":608216415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6807","nonce":608216415}]},"public_addr":"172.21.15.52:6803/608216415","cluster_addr":"172.21.15.52:6805/608216415","heartbeat_back_addr":"172.21.15.52:6809/608216415","heartbeat_front_addr":"172.21.15.52:6807/608216415","state":["exists","up"]},{"osd":1,"uuid":"d6a46b91-adc8-45ef-80f5-566ae400911f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6801","nonce":2538384087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6803","nonce":2538384087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6807","nonce":2538384087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6805","nonce":2538384087}]},"public_addr":"172.21.15.102:6801/2538384087","cluster_addr":"172.21.15.102:6803/2538384087","heartbeat_back_addr":"172.21.15.102:6807/2538384087","heartbeat_front_addr":"172.21.15.102:6805/2538384087","state":["exists","up"]},{"osd":2,"uuid":"846ba125-11f8-46b2-8941-bd5e1947cc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6801","nonce":4258038221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6803","nonce":4258038221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6807","nonce":4258038221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6805","nonce":4258038221}]},"public_addr":"172.21.15.150:6801/4258038221","cluster_addr":"172.21.15.150:6803/4258038221","heartbeat_back_addr":"172.21.15.150:6807/4258038221","heartbeat_front_addr":"172.21.15.150:6805/4258038221","state":["exists","up"]},{"osd":3,"uuid":"950fcd67-1057-4288-a389-714f1ec94261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6811","nonce":2622100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6813","nonce":2622100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6817","nonce":2622100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6815","nonce":2622100752}]},"public_addr":"172.21.15.52:6811/2622100752","cluster_addr":"172.21.15.52:6813/2622100752","heartbeat_back_addr":"172.21.15.52:6817/2622100752","heartbeat_front_addr":"172.21.15.52:6815/2622100752","state":["exists","up"]},{"osd":4,"uuid":"1c96a676-cea9-4deb-bad1-f7fd240c3d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6809","nonce":1546794763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6811","nonce":1546794763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6815","nonce":1546794763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6813","nonce":1546794763}]},"public_addr":"172.21.15.102:6809/1546794763","cluster_addr":"172.21.15.102:6811/1546794763","heartbeat_back_addr":"172.21.15.102:6815/1546794763","heartbeat_front_addr":"172.21.15.102:6813/1546794763","state":["exists","up"]},{"osd":5,"uuid":"38a75af1-fd4d-433e-b579-0609109c851c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6809","nonce":461511375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6811","nonce":461511375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6815","nonce":461511375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6813","nonce":461511375}]},"public_addr":"172.21.15.150:6809/461511375","cluster_addr":"172.21.15.150:6811/461511375","heartbeat_back_addr":"172.21.15.150:6815/461511375","heartbeat_front_addr":"172.21.15.150:6813/461511375","state":["exists","up"]},{"osd":6,"uuid":"2f0ede4a-d175-4b4a-b876-c0f442048901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6819","nonce":1002457464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6821","nonce":1002457464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6825","nonce":1002457464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6823","nonce":1002457464}]},"public_addr":"172.21.15.52:6819/1002457464","cluster_addr":"172.21.15.52:6821/1002457464","heartbeat_back_addr":"172.21.15.52:6825/1002457464","heartbeat_front_addr":"172.21.15.52:6823/1002457464","state":["exists","up"]},{"osd":7,"uuid":"6b635192-abcf-4bbf-bf78-a019b351bca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6817","nonce":3249840064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6819","nonce":3249840064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6823","nonce":3249840064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6821","nonce":3249840064}]},"public_addr":"172.21.15.102:6817/3249840064","cluster_addr":"172.21.15.102:6819/3249840064","heartbeat_back_addr":"172.21.15.102:6823/3249840064","heartbeat_front_addr":"172.21.15.102:6821/3249840064","state":["exists","up"]},{"osd":8,"uuid":"c7ca8007-7170-47bf-956c-3f00edf27e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6817","nonce":3672583574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6819","nonce":3672583574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6823","nonce":3672583574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6821","nonce":3672583574}]},"public_addr":"172.21.15.150:6817/3672583574","cluster_addr":"172.21.15.150:6819/3672583574","heartbeat_back_addr":"172.21.15.150:6823/3672583574","heartbeat_front_addr":"172.21.15.150:6821/3672583574","state":["exists","up"]},{"osd":9,"uuid":"0e252c2c-01a4-48ba-832d-067ed940fefb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6827","nonce":4287934320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6829","nonce":4287934320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6833","nonce":4287934320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6831","nonce":4287934320}]},"public_addr":"172.21.15.52:6827/4287934320","cluster_addr":"172.21.15.52:6829/4287934320","heartbeat_back_addr":"172.21.15.52:6833/4287934320","heartbeat_front_addr":"172.21.15.52:6831/4287934320","state":["exists","up"]},{"osd":10,"uuid":"bb540142-7563-44a4-b7a3-cb662714c128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6825","nonce":3056034201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6827","nonce":3056034201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6831","nonce":3056034201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6829","nonce":3056034201}]},"public_addr":"172.21.15.102:6825/3056034201","cluster_addr":"172.21.15.102:6827/3056034201","heartbeat_back_addr":"172.21.15.102:6831/3056034201","heartbeat_front_addr":"172.21.15.102:6829/3056034201","state":["exists","up"]},{"osd":11,"uuid":"3156528e-e0f8-4d44-9946-c8b6339c2bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6825","nonce":3207559739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6827","nonce":3207559739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6831","nonce":3207559739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6829","nonce":3207559739}]},"public_addr":"172.21.15.150:6825/3207559739","cluster_addr":"172.21.15.150:6827/3207559739","heartbeat_back_addr":"172.21.15.150:6831/3207559739","heartbeat_front_addr":"172.21.15.150:6829/3207559739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:26.826099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:48.331371+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:10.459738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:32.356472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:54.043170+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:15.648582+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:37.166463+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:59.240819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:20.710097+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:43.364563+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:05.279433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:26.344573+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/1392282110":"2023-11-28T15:10:03.644870+0000","172.21.15.52:0/1479993479":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6800/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/961562652":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/1290665390":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2410529742":"2023-11-28T15:09:37.118690+0000","172.21.15.52:6800/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2130037044":"2023-11-28T15:10:03.644870+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:16:25.983 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.979+0000 7f769e0ae700 1 -- 172.21.15.52:0/3946502738 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f76800a1250 msgr2=0x7f76800a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:25.983 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.979+0000 7f769e0ae700 1 --2- 172.21.15.52:0/3946502738 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f76800a1250 0x7f76800a3700 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f76880099d0 tx=0x7f7688008040 comp rx=0 tx=0).stop 2023-11-27T15:16:25.983 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.979+0000 7f769e0ae700 1 -- 172.21.15.52:0/3946502738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76980ffe00 msgr2=0x7f7698195870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:25.983 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.979+0000 7f769e0ae700 1 --2- 172.21.15.52:0/3946502738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76980ffe00 0x7f7698195870 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f768c009990 tx=0x7f768c004730 comp rx=0 tx=0).stop 2023-11-27T15:16:25.983 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.979+0000 7f769e0ae700 1 -- 172.21.15.52:0/3946502738 shutdown_connections 2023-11-27T15:16:25.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.979+0000 7f769e0ae700 1 --2- 172.21.15.52:0/3946502738 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f76800a1250 0x7f76800a3700 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:25.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.979+0000 7f769e0ae700 1 --2- 172.21.15.52:0/3946502738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76980ff460 0x7f7698195330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:25.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.979+0000 7f769e0ae700 1 --2- 172.21.15.52:0/3946502738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7698104500 0x7f7698199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:25.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.979+0000 7f769e0ae700 1 --2- 172.21.15.52:0/3946502738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76980ffe00 0x7f7698195870 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:25.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.979+0000 7f769e0ae700 1 -- 172.21.15.52:0/3946502738 >> 172.21.15.52:0/3946502738 conn(0x7f76980770a0 msgr2=0x7f76980fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:25.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.979+0000 7f769e0ae700 1 -- 172.21.15.52:0/3946502738 shutdown_connections 2023-11-27T15:16:25.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:25.979+0000 7f769e0ae700 1 -- 172.21.15.52:0/3946502738 wait complete. 2023-11-27T15:16:26.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:25 smithi102 ceph-mon[100296]: pgmap v264: 97 pgs: 97 active+clean; 621 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2023-11-27T15:16:26.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:25 smithi052 ceph-mon[99056]: pgmap v264: 97 pgs: 97 active+clean; 621 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2023-11-27T15:16:26.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:25 smithi150 ceph-mon[100559]: pgmap v264: 97 pgs: 97 active+clean; 621 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2023-11-27T15:16:26.459 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-11-27T15:16:26.460 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph osd dump --format=json 2023-11-27T15:16:27.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:26 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3946502738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:16:27.116 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:27.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:26 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3946502738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:16:27.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:26 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3946502738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:16:28.011 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:27 smithi052 ceph-mon[99056]: pgmap v265: 97 pgs: 97 active+clean; 621 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2023-11-27T15:16:28.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:27 smithi102 ceph-mon[100296]: pgmap v265: 97 pgs: 97 active+clean; 621 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2023-11-27T15:16:28.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:27 smithi150 ceph-mon[100559]: pgmap v265: 97 pgs: 97 active+clean; 621 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2023-11-27T15:16:28.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.643+0000 7fee509d7700 1 -- 172.21.15.52:0/3844151022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee4c0ff460 msgr2=0x7fee4c0ff830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:28.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.643+0000 7fee509d7700 1 --2- 172.21.15.52:0/3844151022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee4c0ff460 0x7fee4c0ff830 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0099c0 tx=0x7fee3c009cd0 comp rx=0 tx=0).stop 2023-11-27T15:16:28.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.643+0000 7fee509d7700 1 -- 172.21.15.52:0/3844151022 shutdown_connections 2023-11-27T15:16:28.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.643+0000 7fee509d7700 1 --2- 172.21.15.52:0/3844151022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee4c104500 0x7fee4c10c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:28.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.643+0000 7fee509d7700 1 --2- 172.21.15.52:0/3844151022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee4c0ff460 0x7fee4c0ff830 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:28.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.643+0000 7fee509d7700 1 --2- 172.21.15.52:0/3844151022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee4c0ffe00 0x7fee4c103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:28.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.643+0000 7fee509d7700 1 -- 172.21.15.52:0/3844151022 >> 172.21.15.52:0/3844151022 conn(0x7fee4c0770a0 msgr2=0x7fee4c0774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:28.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.643+0000 7fee509d7700 1 -- 172.21.15.52:0/3844151022 shutdown_connections 2023-11-27T15:16:28.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.643+0000 7fee509d7700 1 -- 172.21.15.52:0/3844151022 wait complete. 2023-11-27T15:16:28.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.644+0000 7fee509d7700 1 Processor -- start 2023-11-27T15:16:28.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.644+0000 7fee509d7700 1 -- start start 2023-11-27T15:16:28.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.644+0000 7fee509d7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee4c0ff460 0x7fee4c1953a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:28.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.644+0000 7fee509d7700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee4c0ffe00 0x7fee4c1958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:28.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.644+0000 7fee509d7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee4c104500 0x7fee4c199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:28.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.644+0000 7fee509d7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c19a360 con 0x7fee4c0ff460 2023-11-27T15:16:28.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.644+0000 7fee509d7700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c19a4d0 con 0x7fee4c0ffe00 2023-11-27T15:16:28.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.644+0000 7fee509d7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c19a690 con 0x7fee4c104500 2023-11-27T15:16:28.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.644+0000 7fee4a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee4c0ff460 0x7fee4c1953a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:28.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.644+0000 7fee4a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee4c0ff460 0x7fee4c1953a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50282/0 (socket says 172.21.15.52:50282) 2023-11-27T15:16:28.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.645+0000 7fee4a59c700 1 -- 172.21.15.52:0/3128317360 learned_addr learned my addr 172.21.15.52:0/3128317360 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:28.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.645+0000 7fee49d9b700 1 --2- 172.21.15.52:0/3128317360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee4c0ffe00 0x7fee4c1958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:28.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.645+0000 7fee4ad9d700 1 --2- 172.21.15.52:0/3128317360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee4c104500 0x7fee4c199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:28.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.645+0000 7fee4a59c700 1 -- 172.21.15.52:0/3128317360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee4c104500 msgr2=0x7fee4c199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:28.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.645+0000 7fee4a59c700 1 --2- 172.21.15.52:0/3128317360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee4c104500 0x7fee4c199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:28.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.645+0000 7fee4a59c700 1 -- 172.21.15.52:0/3128317360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee4c0ffe00 msgr2=0x7fee4c1958e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:28.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.645+0000 7fee4a59c700 1 --2- 172.21.15.52:0/3128317360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee4c0ffe00 0x7fee4c1958e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:28.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.645+0000 7fee4a59c700 1 -- 172.21.15.52:0/3128317360 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3c0095f0 con 0x7fee4c0ff460 2023-11-27T15:16:28.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.645+0000 7fee4a59c700 1 --2- 172.21.15.52:0/3128317360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee4c0ff460 0x7fee4c1953a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0098e0 tx=0x7fee3c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:28.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.645+0000 7fee437fe700 1 -- 172.21.15.52:0/3128317360 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3c01d070 con 0x7fee4c0ff460 2023-11-27T15:16:28.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.645+0000 7fee437fe700 1 -- 172.21.15.52:0/3128317360 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee3c004060 con 0x7fee4c0ff460 2023-11-27T15:16:28.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.645+0000 7fee509d7700 1 -- 172.21.15.52:0/3128317360 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee4c10f240 con 0x7fee4c0ff460 2023-11-27T15:16:28.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.645+0000 7fee437fe700 1 -- 172.21.15.52:0/3128317360 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3c022dd0 con 0x7fee4c0ff460 2023-11-27T15:16:28.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.645+0000 7fee509d7700 1 -- 172.21.15.52:0/3128317360 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee4c10f0b0 con 0x7fee4c0ff460 2023-11-27T15:16:28.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.646+0000 7fee509d7700 1 -- 172.21.15.52:0/3128317360 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee4c10f3d0 con 0x7fee4c0ff460 2023-11-27T15:16:28.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.647+0000 7fee437fe700 1 -- 172.21.15.52:0/3128317360 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fee3c022500 con 0x7fee4c0ff460 2023-11-27T15:16:28.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.648+0000 7fee437fe700 1 --2- 172.21.15.52:0/3128317360 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fee380a11e0 0x7fee380a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:28.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.648+0000 7fee437fe700 1 -- 172.21.15.52:0/3128317360 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fee3c0cdd10 con 0x7fee4c0ff460 2023-11-27T15:16:28.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.648+0000 7fee49d9b700 1 --2- 172.21.15.52:0/3128317360 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fee380a11e0 0x7fee380a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:28.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.648+0000 7fee49d9b700 1 --2- 172.21.15.52:0/3128317360 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fee380a11e0 0x7fee380a3690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fee4c1969c0 tx=0x7fee3400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:28.656 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.651+0000 7fee437fe700 1 -- 172.21.15.52:0/3128317360 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fee3c097540 con 0x7fee4c0ff460 2023-11-27T15:16:28.855 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.850+0000 7fee509d7700 1 -- 172.21.15.52:0/3128317360 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee4c196700 con 0x7fee4c0ff460 2023-11-27T15:16:28.856 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.852+0000 7fee437fe700 1 -- 172.21.15.52:0/3128317360 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22130 (secure 0 0 0) 0x7fee3c005c00 con 0x7fee4c0ff460 2023-11-27T15:16:28.856 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:16:28.857 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":83,"fsid":"c6c7df88-8d36-11ee-95a2-87774f69a715","created":"2023-11-27T15:09:03.333345+0000","modified":"2023-11-27T15:16:10.321506+0000","last_up_change":"2023-11-27T15:15:27.656134+0000","last_in_change":"2023-11-27T15:15:10.671153+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:12:14.177523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:16:07.842251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:16:08.728520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58d69ede-c293-49f5-abbe-8187793f2b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6803","nonce":608216415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6805","nonce":608216415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6809","nonce":608216415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6807","nonce":608216415}]},"public_addr":"172.21.15.52:6803/608216415","cluster_addr":"172.21.15.52:6805/608216415","heartbeat_back_addr":"172.21.15.52:6809/608216415","heartbeat_front_addr":"172.21.15.52:6807/608216415","state":["exists","up"]},{"osd":1,"uuid":"d6a46b91-adc8-45ef-80f5-566ae400911f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6801","nonce":2538384087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6803","nonce":2538384087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6807","nonce":2538384087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6805","nonce":2538384087}]},"public_addr":"172.21.15.102:6801/2538384087","cluster_addr":"172.21.15.102:6803/2538384087","heartbeat_back_addr":"172.21.15.102:6807/2538384087","heartbeat_front_addr":"172.21.15.102:6805/2538384087","state":["exists","up"]},{"osd":2,"uuid":"846ba125-11f8-46b2-8941-bd5e1947cc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6801","nonce":4258038221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6803","nonce":4258038221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6807","nonce":4258038221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6805","nonce":4258038221}]},"public_addr":"172.21.15.150:6801/4258038221","cluster_addr":"172.21.15.150:6803/4258038221","heartbeat_back_addr":"172.21.15.150:6807/4258038221","heartbeat_front_addr":"172.21.15.150:6805/4258038221","state":["exists","up"]},{"osd":3,"uuid":"950fcd67-1057-4288-a389-714f1ec94261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6811","nonce":2622100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6813","nonce":2622100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6817","nonce":2622100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6815","nonce":2622100752}]},"public_addr":"172.21.15.52:6811/2622100752","cluster_addr":"172.21.15.52:6813/2622100752","heartbeat_back_addr":"172.21.15.52:6817/2622100752","heartbeat_front_addr":"172.21.15.52:6815/2622100752","state":["exists","up"]},{"osd":4,"uuid":"1c96a676-cea9-4deb-bad1-f7fd240c3d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6809","nonce":1546794763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6811","nonce":1546794763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6815","nonce":1546794763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6813","nonce":1546794763}]},"public_addr":"172.21.15.102:6809/1546794763","cluster_addr":"172.21.15.102:6811/1546794763","heartbeat_back_addr":"172.21.15.102:6815/1546794763","heartbeat_front_addr":"172.21.15.102:6813/1546794763","state":["exists","up"]},{"osd":5,"uuid":"38a75af1-fd4d-433e-b579-0609109c851c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6809","nonce":461511375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6811","nonce":461511375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6815","nonce":461511375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6813","nonce":461511375}]},"public_addr":"172.21.15.150:6809/461511375","cluster_addr":"172.21.15.150:6811/461511375","heartbeat_back_addr":"172.21.15.150:6815/461511375","heartbeat_front_addr":"172.21.15.150:6813/461511375","state":["exists","up"]},{"osd":6,"uuid":"2f0ede4a-d175-4b4a-b876-c0f442048901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6819","nonce":1002457464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6821","nonce":1002457464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6825","nonce":1002457464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6823","nonce":1002457464}]},"public_addr":"172.21.15.52:6819/1002457464","cluster_addr":"172.21.15.52:6821/1002457464","heartbeat_back_addr":"172.21.15.52:6825/1002457464","heartbeat_front_addr":"172.21.15.52:6823/1002457464","state":["exists","up"]},{"osd":7,"uuid":"6b635192-abcf-4bbf-bf78-a019b351bca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6817","nonce":3249840064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6819","nonce":3249840064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6823","nonce":3249840064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6821","nonce":3249840064}]},"public_addr":"172.21.15.102:6817/3249840064","cluster_addr":"172.21.15.102:6819/3249840064","heartbeat_back_addr":"172.21.15.102:6823/3249840064","heartbeat_front_addr":"172.21.15.102:6821/3249840064","state":["exists","up"]},{"osd":8,"uuid":"c7ca8007-7170-47bf-956c-3f00edf27e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6817","nonce":3672583574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6819","nonce":3672583574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6823","nonce":3672583574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6821","nonce":3672583574}]},"public_addr":"172.21.15.150:6817/3672583574","cluster_addr":"172.21.15.150:6819/3672583574","heartbeat_back_addr":"172.21.15.150:6823/3672583574","heartbeat_front_addr":"172.21.15.150:6821/3672583574","state":["exists","up"]},{"osd":9,"uuid":"0e252c2c-01a4-48ba-832d-067ed940fefb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6827","nonce":4287934320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6829","nonce":4287934320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6833","nonce":4287934320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6831","nonce":4287934320}]},"public_addr":"172.21.15.52:6827/4287934320","cluster_addr":"172.21.15.52:6829/4287934320","heartbeat_back_addr":"172.21.15.52:6833/4287934320","heartbeat_front_addr":"172.21.15.52:6831/4287934320","state":["exists","up"]},{"osd":10,"uuid":"bb540142-7563-44a4-b7a3-cb662714c128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6825","nonce":3056034201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6827","nonce":3056034201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6831","nonce":3056034201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6829","nonce":3056034201}]},"public_addr":"172.21.15.102:6825/3056034201","cluster_addr":"172.21.15.102:6827/3056034201","heartbeat_back_addr":"172.21.15.102:6831/3056034201","heartbeat_front_addr":"172.21.15.102:6829/3056034201","state":["exists","up"]},{"osd":11,"uuid":"3156528e-e0f8-4d44-9946-c8b6339c2bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6825","nonce":3207559739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6827","nonce":3207559739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6831","nonce":3207559739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6829","nonce":3207559739}]},"public_addr":"172.21.15.150:6825/3207559739","cluster_addr":"172.21.15.150:6827/3207559739","heartbeat_back_addr":"172.21.15.150:6831/3207559739","heartbeat_front_addr":"172.21.15.150:6829/3207559739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:26.826099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:48.331371+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:10.459738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:32.356472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:54.043170+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:15.648582+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:37.166463+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:59.240819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:20.710097+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:43.364563+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:05.279433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:26.344573+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/1392282110":"2023-11-28T15:10:03.644870+0000","172.21.15.52:0/1479993479":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6800/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/961562652":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/1290665390":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2410529742":"2023-11-28T15:09:37.118690+0000","172.21.15.52:6800/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2130037044":"2023-11-28T15:10:03.644870+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:16:28.858 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.854+0000 7fee509d7700 1 -- 172.21.15.52:0/3128317360 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fee380a11e0 msgr2=0x7fee380a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:28.858 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.854+0000 7fee509d7700 1 --2- 172.21.15.52:0/3128317360 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fee380a11e0 0x7fee380a3690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fee4c1969c0 tx=0x7fee3400b410 comp rx=0 tx=0).stop 2023-11-27T15:16:28.858 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.854+0000 7fee509d7700 1 -- 172.21.15.52:0/3128317360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee4c0ff460 msgr2=0x7fee4c1953a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:28.859 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.854+0000 7fee509d7700 1 --2- 172.21.15.52:0/3128317360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee4c0ff460 0x7fee4c1953a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0098e0 tx=0x7fee3c004730 comp rx=0 tx=0).stop 2023-11-27T15:16:28.859 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.854+0000 7fee509d7700 1 -- 172.21.15.52:0/3128317360 shutdown_connections 2023-11-27T15:16:28.859 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.854+0000 7fee509d7700 1 --2- 172.21.15.52:0/3128317360 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fee380a11e0 0x7fee380a3690 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:28.859 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.854+0000 7fee509d7700 1 --2- 172.21.15.52:0/3128317360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee4c104500 0x7fee4c199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:28.859 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.854+0000 7fee509d7700 1 --2- 172.21.15.52:0/3128317360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee4c0ff460 0x7fee4c1953a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:28.859 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.854+0000 7fee509d7700 1 --2- 172.21.15.52:0/3128317360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee4c0ffe00 0x7fee4c1958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:28.859 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.854+0000 7fee509d7700 1 -- 172.21.15.52:0/3128317360 >> 172.21.15.52:0/3128317360 conn(0x7fee4c0770a0 msgr2=0x7fee4c102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:28.859 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.854+0000 7fee509d7700 1 -- 172.21.15.52:0/3128317360 shutdown_connections 2023-11-27T15:16:28.859 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:28.854+0000 7fee509d7700 1 -- 172.21.15.52:0/3128317360 wait complete. 2023-11-27T15:16:29.367 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2023-11-27T15:16:29.368 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2023-11-27T15:16:29.368 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2023-11-27T15:16:29.368 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2023-11-27T15:16:29.369 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2023-11-27T15:16:29.369 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2023-11-27T15:16:29.370 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2023-11-27T15:16:29.370 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2023-11-27T15:16:29.370 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2023-11-27T15:16:29.371 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2023-11-27T15:16:29.371 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2023-11-27T15:16:29.372 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2023-11-27T15:16:30.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:29 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3128317360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:16:30.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:29 smithi102 ceph-mon[100296]: pgmap v266: 97 pgs: 97 active+clean; 633 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2023-11-27T15:16:30.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:29 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3128317360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:16:30.118 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:29 smithi052 ceph-mon[99056]: pgmap v266: 97 pgs: 97 active+clean; 633 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2023-11-27T15:16:30.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:29 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3128317360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:16:30.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:29 smithi150 ceph-mon[100559]: pgmap v266: 97 pgs: 97 active+clean; 633 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2023-11-27T15:16:30.476 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:30.502 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:30.665 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:30.987 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:31.794 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:32.019 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:31 smithi052 ceph-mon[99056]: pgmap v267: 97 pgs: 97 active+clean; 633 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2023-11-27T15:16:32.029 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:32.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:31 smithi102 ceph-mon[100296]: pgmap v267: 97 pgs: 97 active+clean; 633 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2023-11-27T15:16:32.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:31 smithi150 ceph-mon[100559]: pgmap v267: 97 pgs: 97 active+clean; 633 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2023-11-27T15:16:32.480 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:32.760 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:32.871 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:32.871 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:34.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:33 smithi102 ceph-mon[100296]: pgmap v268: 97 pgs: 97 active+clean; 633 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-11-27T15:16:34.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:33 smithi052 ceph-mon[99056]: pgmap v268: 97 pgs: 97 active+clean; 633 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-11-27T15:16:34.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:33 smithi150 ceph-mon[100559]: pgmap v268: 97 pgs: 97 active+clean; 633 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-11-27T15:16:34.343 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:34.450 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:34.745 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.740+0000 7f183ce99700 1 -- 172.21.15.52:0/4120080755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18380ffde0 msgr2=0x7f1838103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:34.745 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.740+0000 7f183ce99700 1 --2- 172.21.15.52:0/4120080755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18380ffde0 0x7f1838103db0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f182c0099c0 tx=0x7f182c009cd0 comp rx=0 tx=0).stop 2023-11-27T15:16:34.745 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.740+0000 7f183ce99700 1 -- 172.21.15.52:0/4120080755 shutdown_connections 2023-11-27T15:16:34.745 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.740+0000 7f183ce99700 1 --2- 172.21.15.52:0/4120080755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18380ff440 0x7f18380ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:34.745 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.740+0000 7f183ce99700 1 --2- 172.21.15.52:0/4120080755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18381044e0 0x7f183810c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:34.745 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.740+0000 7f183ce99700 1 --2- 172.21.15.52:0/4120080755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18380ffde0 0x7f1838103db0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:34.745 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.740+0000 7f183ce99700 1 -- 172.21.15.52:0/4120080755 >> 172.21.15.52:0/4120080755 conn(0x7f18380facc0 msgr2=0x7f18380fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:34.746 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.740+0000 7f183ce99700 1 -- 172.21.15.52:0/4120080755 shutdown_connections 2023-11-27T15:16:34.746 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.740+0000 7f183ce99700 1 -- 172.21.15.52:0/4120080755 wait complete. 2023-11-27T15:16:34.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.741+0000 7f183ce99700 1 Processor -- start 2023-11-27T15:16:34.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.741+0000 7f183ce99700 1 -- start start 2023-11-27T15:16:34.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.742+0000 7f183ce99700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18380ff440 0x7f1838197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:34.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.742+0000 7f183ce99700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18380ffde0 0x7f1838197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:34.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.742+0000 7f183ce99700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18381044e0 0x7f18381a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:34.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.742+0000 7f183ce99700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18381980c0 con 0x7f18380ff440 2023-11-27T15:16:34.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.742+0000 7f183ce99700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18381a2800 con 0x7f18380ffde0 2023-11-27T15:16:34.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.742+0000 7f183ce99700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18381a29c0 con 0x7f18381044e0 2023-11-27T15:16:34.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.742+0000 7f183659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18380ff440 0x7f1838197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:34.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.742+0000 7f183659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18380ff440 0x7f1838197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50310/0 (socket says 172.21.15.52:50310) 2023-11-27T15:16:34.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.742+0000 7f1835d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18380ffde0 0x7f1838197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:34.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.742+0000 7f183659c700 1 -- 172.21.15.52:0/561879651 learned_addr learned my addr 172.21.15.52:0/561879651 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:34.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.742+0000 7f1836d9d700 1 --2- 172.21.15.52:0/561879651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18381044e0 0x7f18381a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:34.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.742+0000 7f1835d9b700 1 -- 172.21.15.52:0/561879651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18381044e0 msgr2=0x7f18381a22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:34.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.742+0000 7f1835d9b700 1 --2- 172.21.15.52:0/561879651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18381044e0 0x7f18381a22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:34.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.742+0000 7f1835d9b700 1 -- 172.21.15.52:0/561879651 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18380ff440 msgr2=0x7f1838197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:34.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.742+0000 7f1835d9b700 1 --2- 172.21.15.52:0/561879651 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18380ff440 0x7f1838197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:34.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.742+0000 7f1835d9b700 1 -- 172.21.15.52:0/561879651 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c0095f0 con 0x7f18380ffde0 2023-11-27T15:16:34.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.742+0000 7f183659c700 1 --2- 172.21.15.52:0/561879651 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18380ff440 0x7f1838197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:16:34.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.743+0000 7f1835d9b700 1 --2- 172.21.15.52:0/561879651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18380ffde0 0x7f1838197780 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f182c0058e0 tx=0x7f182c00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:34.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.743+0000 7f18277fe700 1 -- 172.21.15.52:0/561879651 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c01d070 con 0x7f18380ffde0 2023-11-27T15:16:34.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.743+0000 7f183ce99700 1 -- 172.21.15.52:0/561879651 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f183810f280 con 0x7f18380ffde0 2023-11-27T15:16:34.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.743+0000 7f18277fe700 1 -- 172.21.15.52:0/561879651 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f182c005020 con 0x7f18380ffde0 2023-11-27T15:16:34.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.743+0000 7f18277fe700 1 -- 172.21.15.52:0/561879651 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c0049e0 con 0x7f18380ffde0 2023-11-27T15:16:34.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.743+0000 7f183ce99700 1 -- 172.21.15.52:0/561879651 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f183810f0f0 con 0x7f18380ffde0 2023-11-27T15:16:34.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.744+0000 7f183ce99700 1 -- 172.21.15.52:0/561879651 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f18380fc110 con 0x7f18380ffde0 2023-11-27T15:16:34.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.745+0000 7f18277fe700 1 -- 172.21.15.52:0/561879651 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f182c005190 con 0x7f18380ffde0 2023-11-27T15:16:34.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.746+0000 7f18277fe700 1 --2- 172.21.15.52:0/561879651 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f18200a11e0 0x7f18200a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:34.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.746+0000 7f183659c700 1 --2- 172.21.15.52:0/561879651 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f18200a11e0 0x7f18200a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:34.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.746+0000 7f18277fe700 1 -- 172.21.15.52:0/561879651 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f182c0cd830 con 0x7f18380ffde0 2023-11-27T15:16:34.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.746+0000 7f18277fe700 1 --2- 172.21.15.52:0/561879651 >> [v2:172.21.15.102:6816/3249840064,v1:172.21.15.102:6817/3249840064] conn(0x7f18200aa880 0x7f18200acc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:34.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.746+0000 7f18277fe700 1 -- 172.21.15.52:0/561879651 --> [v2:172.21.15.102:6816/3249840064,v1:172.21.15.102:6817/3249840064] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f18200ad340 con 0x7f18200aa880 2023-11-27T15:16:34.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.746+0000 7f18277fe700 1 -- 172.21.15.52:0/561879651 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f182c0cdba0 con 0x7f18380ffde0 2023-11-27T15:16:34.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.746+0000 7f183659c700 1 --2- 172.21.15.52:0/561879651 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f18200a11e0 0x7f18200a3690 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1828011550 tx=0x7f1828009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:34.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.746+0000 7f1836d9d700 1 --2- 172.21.15.52:0/561879651 >> [v2:172.21.15.102:6816/3249840064,v1:172.21.15.102:6817/3249840064] conn(0x7f18200aa880 0x7f18200acc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:34.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.747+0000 7f1836d9d700 1 --2- 172.21.15.52:0/561879651 >> [v2:172.21.15.102:6816/3249840064,v1:172.21.15.102:6817/3249840064] conn(0x7f18200aa880 0x7f18200acc90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:34.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.748+0000 7f18277fe700 1 -- 172.21.15.52:0/561879651 <== osd.7 v2:172.21.15.102:6816/3249840064 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f18200ad340 con 0x7f18200aa880 2023-11-27T15:16:34.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.768+0000 7f183ce99700 1 -- 172.21.15.52:0/561879651 --> [v2:172.21.15.102:6816/3249840064,v1:172.21.15.102:6817/3249840064] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f183810f410 con 0x7f18200aa880 2023-11-27T15:16:34.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.768+0000 7f18277fe700 1 -- 172.21.15.52:0/561879651 <== osd.7 v2:172.21.15.102:6816/3249840064 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f183810f410 con 0x7f18200aa880 2023-11-27T15:16:34.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.769+0000 7f183ce99700 1 -- 172.21.15.52:0/561879651 >> [v2:172.21.15.102:6816/3249840064,v1:172.21.15.102:6817/3249840064] conn(0x7f18200aa880 msgr2=0x7f18200acc90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:34.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.769+0000 7f183ce99700 1 --2- 172.21.15.52:0/561879651 >> [v2:172.21.15.102:6816/3249840064,v1:172.21.15.102:6817/3249840064] conn(0x7f18200aa880 0x7f18200acc90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:34.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.769+0000 7f183ce99700 1 -- 172.21.15.52:0/561879651 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f18200a11e0 msgr2=0x7f18200a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:34.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.769+0000 7f183ce99700 1 --2- 172.21.15.52:0/561879651 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f18200a11e0 0x7f18200a3690 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1828011550 tx=0x7f1828009260 comp rx=0 tx=0).stop 2023-11-27T15:16:34.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.769+0000 7f183ce99700 1 -- 172.21.15.52:0/561879651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18380ffde0 msgr2=0x7f1838197780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:34.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.769+0000 7f183ce99700 1 --2- 172.21.15.52:0/561879651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18380ffde0 0x7f1838197780 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f182c0058e0 tx=0x7f182c00f690 comp rx=0 tx=0).stop 2023-11-27T15:16:34.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.769+0000 7f1836d9d700 1 -- 172.21.15.52:0/561879651 reap_dead start 2023-11-27T15:16:34.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.769+0000 7f183ce99700 1 -- 172.21.15.52:0/561879651 shutdown_connections 2023-11-27T15:16:34.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.769+0000 7f183ce99700 1 -- 172.21.15.52:0/561879651 >> 172.21.15.52:0/561879651 conn(0x7f18380facc0 msgr2=0x7f18380fb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:34.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.769+0000 7f183ce99700 1 -- 172.21.15.52:0/561879651 shutdown_connections 2023-11-27T15:16:34.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.769+0000 7f183ce99700 1 -- 172.21.15.52:0/561879651 wait complete. 2023-11-27T15:16:35.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.702+0000 7f3aca0c5700 1 -- 172.21.15.52:0/2472777089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ac4109ca0 msgr2=0x7f3ac410c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:35.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.702+0000 7f3aca0c5700 1 --2- 172.21.15.52:0/2472777089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ac4109ca0 0x7f3ac410c080 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00b0a0 tx=0x7f3abc00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:16:35.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.702+0000 7f3aca0c5700 1 -- 172.21.15.52:0/2472777089 shutdown_connections 2023-11-27T15:16:35.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.702+0000 7f3aca0c5700 1 --2- 172.21.15.52:0/2472777089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ac4109ca0 0x7f3ac410c080 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:35.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.702+0000 7f3aca0c5700 1 --2- 172.21.15.52:0/2472777089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ac41006f0 0x7f3ac4100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:35.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.702+0000 7f3aca0c5700 1 --2- 172.21.15.52:0/2472777089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ac4101000 0x7f3ac4109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:35.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.702+0000 7f3aca0c5700 1 -- 172.21.15.52:0/2472777089 >> 172.21.15.52:0/2472777089 conn(0x7f3ac4074ba0 msgr2=0x7f3ac4074fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:35.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.702+0000 7f3aca0c5700 1 -- 172.21.15.52:0/2472777089 shutdown_connections 2023-11-27T15:16:35.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.702+0000 7f3aca0c5700 1 -- 172.21.15.52:0/2472777089 wait complete. 2023-11-27T15:16:35.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.703+0000 7f3aca0c5700 1 Processor -- start 2023-11-27T15:16:35.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.705+0000 7f3aca0c5700 1 -- start start 2023-11-27T15:16:35.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.706+0000 7f3aca0c5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ac41006f0 0x7f3ac4195360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:35.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.706+0000 7f3aca0c5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ac4101000 0x7f3ac41958a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:35.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.706+0000 7f3aca0c5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ac4109ca0 0x7f3ac4199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:35.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.706+0000 7f3aca0c5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac419a320 con 0x7f3ac4101000 2023-11-27T15:16:35.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.706+0000 7f3aca0c5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac419a490 con 0x7f3ac41006f0 2023-11-27T15:16:35.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.706+0000 7f3aca0c5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac419a650 con 0x7f3ac4109ca0 2023-11-27T15:16:35.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.706+0000 7f3ac2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ac4101000 0x7f3ac41958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:35.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.706+0000 7f3ac2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ac4101000 0x7f3ac41958a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50296/0 (socket says 172.21.15.52:50296) 2023-11-27T15:16:35.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.706+0000 7f3ac2ffd700 1 -- 172.21.15.52:0/3160809991 learned_addr learned my addr 172.21.15.52:0/3160809991 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:35.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.706+0000 7f3ac37fe700 1 --2- 172.21.15.52:0/3160809991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ac41006f0 0x7f3ac4195360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:35.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.706+0000 7f3ac3fff700 1 --2- 172.21.15.52:0/3160809991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ac4109ca0 0x7f3ac4199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:35.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.706+0000 7f3ac2ffd700 1 -- 172.21.15.52:0/3160809991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ac4109ca0 msgr2=0x7f3ac4199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:35.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.706+0000 7f3ac2ffd700 1 --2- 172.21.15.52:0/3160809991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ac4109ca0 0x7f3ac4199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:35.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.706+0000 7f3ac2ffd700 1 -- 172.21.15.52:0/3160809991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ac41006f0 msgr2=0x7f3ac4195360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:35.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.706+0000 7f3ac2ffd700 1 --2- 172.21.15.52:0/3160809991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ac41006f0 0x7f3ac4195360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:35.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.706+0000 7f3ac2ffd700 1 -- 172.21.15.52:0/3160809991 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3abc009d00 con 0x7f3ac4101000 2023-11-27T15:16:35.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.706+0000 7f3ac2ffd700 1 --2- 172.21.15.52:0/3160809991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ac4101000 0x7f3ac41958a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800d950 tx=0x7f3ab800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:35.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.707+0000 7f3ac0ff9700 1 -- 172.21.15.52:0/3160809991 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8009940 con 0x7f3ac4101000 2023-11-27T15:16:35.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.707+0000 7f3ac0ff9700 1 -- 172.21.15.52:0/3160809991 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ab8010460 con 0x7f3ac4101000 2023-11-27T15:16:35.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.707+0000 7f3aca0c5700 1 -- 172.21.15.52:0/3160809991 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac410f2e0 con 0x7f3ac4101000 2023-11-27T15:16:35.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.707+0000 7f3ac0ff9700 1 -- 172.21.15.52:0/3160809991 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab800f620 con 0x7f3ac4101000 2023-11-27T15:16:35.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.707+0000 7f3aca0c5700 1 -- 172.21.15.52:0/3160809991 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac410f150 con 0x7f3ac4101000 2023-11-27T15:16:35.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.708+0000 7f3aca0c5700 1 -- 172.21.15.52:0/3160809991 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3ac40ff980 con 0x7f3ac4101000 2023-11-27T15:16:35.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.709+0000 7f3ac0ff9700 1 -- 172.21.15.52:0/3160809991 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3ab80108d0 con 0x7f3ac4101000 2023-11-27T15:16:35.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.710+0000 7f3ac0ff9700 1 --2- 172.21.15.52:0/3160809991 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f3aac0a11e0 0x7f3aac0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:35.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.710+0000 7f3ac37fe700 1 --2- 172.21.15.52:0/3160809991 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f3aac0a11e0 0x7f3aac0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:35.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.710+0000 7f3ac0ff9700 1 -- 172.21.15.52:0/3160809991 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3ab80ccd30 con 0x7f3ac4101000 2023-11-27T15:16:35.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.710+0000 7f3ac0ff9700 1 --2- 172.21.15.52:0/3160809991 >> [v2:172.21.15.102:6808/1546794763,v1:172.21.15.102:6809/1546794763] conn(0x7f3aac0aa9b0 0x7f3aac0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:35.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.710+0000 7f3ac0ff9700 1 -- 172.21.15.52:0/3160809991 --> [v2:172.21.15.102:6808/1546794763,v1:172.21.15.102:6809/1546794763] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3aac0ad470 con 0x7f3aac0aa9b0 2023-11-27T15:16:35.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.710+0000 7f3ac0ff9700 1 -- 172.21.15.52:0/3160809991 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ab80cd1d0 con 0x7f3ac4101000 2023-11-27T15:16:35.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.710+0000 7f3ac3fff700 1 --2- 172.21.15.52:0/3160809991 >> [v2:172.21.15.102:6808/1546794763,v1:172.21.15.102:6809/1546794763] conn(0x7f3aac0aa9b0 0x7f3aac0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:35.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.710+0000 7f3ac37fe700 1 --2- 172.21.15.52:0/3160809991 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f3aac0a11e0 0x7f3aac0a3690 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4005950 tx=0x7f3ab400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:35.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.711+0000 7f3ac3fff700 1 --2- 172.21.15.52:0/3160809991 >> [v2:172.21.15.102:6808/1546794763,v1:172.21.15.102:6809/1546794763] conn(0x7f3aac0aa9b0 0x7f3aac0acdc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:35.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.711+0000 7f3ac0ff9700 1 -- 172.21.15.52:0/3160809991 <== osd.4 v2:172.21.15.102:6808/1546794763 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3aac0ad470 con 0x7f3aac0aa9b0 2023-11-27T15:16:35.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.730+0000 7f3aca0c5700 1 -- 172.21.15.52:0/3160809991 --> [v2:172.21.15.102:6808/1546794763,v1:172.21.15.102:6809/1546794763] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3ac410f470 con 0x7f3aac0aa9b0 2023-11-27T15:16:35.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.731+0000 7f3ac0ff9700 1 -- 172.21.15.52:0/3160809991 <== osd.4 v2:172.21.15.102:6808/1546794763 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3ac410f470 con 0x7f3aac0aa9b0 2023-11-27T15:16:35.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.731+0000 7f3aca0c5700 1 -- 172.21.15.52:0/3160809991 >> [v2:172.21.15.102:6808/1546794763,v1:172.21.15.102:6809/1546794763] conn(0x7f3aac0aa9b0 msgr2=0x7f3aac0acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:35.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.731+0000 7f3aca0c5700 1 --2- 172.21.15.52:0/3160809991 >> [v2:172.21.15.102:6808/1546794763,v1:172.21.15.102:6809/1546794763] conn(0x7f3aac0aa9b0 0x7f3aac0acdc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:35.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.731+0000 7f3aca0c5700 1 -- 172.21.15.52:0/3160809991 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f3aac0a11e0 msgr2=0x7f3aac0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:35.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.731+0000 7f3aca0c5700 1 --2- 172.21.15.52:0/3160809991 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f3aac0a11e0 0x7f3aac0a3690 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4005950 tx=0x7f3ab400b410 comp rx=0 tx=0).stop 2023-11-27T15:16:35.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.731+0000 7f3aca0c5700 1 -- 172.21.15.52:0/3160809991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ac4101000 msgr2=0x7f3ac41958a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:35.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.731+0000 7f3aca0c5700 1 --2- 172.21.15.52:0/3160809991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ac4101000 0x7f3ac41958a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800d950 tx=0x7f3ab800dc60 comp rx=0 tx=0).stop 2023-11-27T15:16:35.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.731+0000 7f3ac3fff700 1 -- 172.21.15.52:0/3160809991 reap_dead start 2023-11-27T15:16:35.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.731+0000 7f3aca0c5700 1 -- 172.21.15.52:0/3160809991 shutdown_connections 2023-11-27T15:16:35.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.731+0000 7f3aca0c5700 1 -- 172.21.15.52:0/3160809991 >> 172.21.15.52:0/3160809991 conn(0x7f3ac4074ba0 msgr2=0x7f3ac41016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:35.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.731+0000 7f3aca0c5700 1 -- 172.21.15.52:0/3160809991 shutdown_connections 2023-11-27T15:16:35.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:34.731+0000 7f3aca0c5700 1 -- 172.21.15.52:0/3160809991 wait complete. 2023-11-27T15:16:35.926 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:35 smithi052 ceph-mon[99056]: pgmap v269: 97 pgs: 97 active+clean; 633 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-11-27T15:16:36.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:35 smithi102 ceph-mon[100296]: pgmap v269: 97 pgs: 97 active+clean; 633 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-11-27T15:16:36.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:35 smithi150 ceph-mon[100559]: pgmap v269: 97 pgs: 97 active+clean; 633 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-11-27T15:16:36.420 INFO:teuthology.orchestra.run.smithi052.stdout:214748364833 2023-11-27T15:16:36.420 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.7 2023-11-27T15:16:36.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.450+0000 7f520fb57700 1 -- 172.21.15.52:0/3268889518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52080ff290 msgr2=0x7f52080ff660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:36.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.450+0000 7f520fb57700 1 --2- 172.21.15.52:0/3268889518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52080ff290 0x7f52080ff660 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5204009990 tx=0x7f5204009ca0 comp rx=0 tx=0).stop 2023-11-27T15:16:36.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.450+0000 7f520fb57700 1 -- 172.21.15.52:0/3268889518 shutdown_connections 2023-11-27T15:16:36.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.450+0000 7f520fb57700 1 --2- 172.21.15.52:0/3268889518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52081046a0 0x7f5208104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:36.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.450+0000 7f520fb57700 1 --2- 172.21.15.52:0/3268889518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52080ff290 0x7f52080ff660 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:36.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.450+0000 7f520fb57700 1 --2- 172.21.15.52:0/3268889518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52080ffba0 0x7f5208104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:36.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.450+0000 7f520fb57700 1 -- 172.21.15.52:0/3268889518 >> 172.21.15.52:0/3268889518 conn(0x7f52080facc0 msgr2=0x7f52080fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:36.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.450+0000 7f520fb57700 1 -- 172.21.15.52:0/3268889518 shutdown_connections 2023-11-27T15:16:36.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.450+0000 7f520fb57700 1 -- 172.21.15.52:0/3268889518 wait complete. 2023-11-27T15:16:36.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.451+0000 7f520fb57700 1 Processor -- start 2023-11-27T15:16:36.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.451+0000 7f520fb57700 1 -- start start 2023-11-27T15:16:36.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.451+0000 7f520fb57700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52080ff290 0x7f52081a26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:36.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.451+0000 7f520fb57700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52080ffba0 0x7f52081a2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:36.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.452+0000 7f520fb57700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52081046a0 0x7f520819c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:36.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.452+0000 7f520fb57700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520819cde0 con 0x7f52081046a0 2023-11-27T15:16:36.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.452+0000 7f520fb57700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520819cf50 con 0x7f52080ffba0 2023-11-27T15:16:36.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.452+0000 7f520fb57700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520819d110 con 0x7f52080ff290 2023-11-27T15:16:36.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.452+0000 7f520e0f4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52081046a0 0x7f520819c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:36.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.452+0000 7f520e0f4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52081046a0 0x7f520819c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50328/0 (socket says 172.21.15.52:50328) 2023-11-27T15:16:36.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.452+0000 7f520e0f4700 1 -- 172.21.15.52:0/3008824315 learned_addr learned my addr 172.21.15.52:0/3008824315 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:36.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.452+0000 7f520d8f3700 1 --2- 172.21.15.52:0/3008824315 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52080ff290 0x7f52081a26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:36.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.452+0000 7f520d0f2700 1 --2- 172.21.15.52:0/3008824315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52080ffba0 0x7f52081a2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:36.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.452+0000 7f520e0f4700 1 -- 172.21.15.52:0/3008824315 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52080ff290 msgr2=0x7f52081a26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:36.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.452+0000 7f520e0f4700 1 --2- 172.21.15.52:0/3008824315 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52080ff290 0x7f52081a26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:36.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.452+0000 7f520e0f4700 1 -- 172.21.15.52:0/3008824315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52080ffba0 msgr2=0x7f52081a2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:36.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.452+0000 7f520e0f4700 1 --2- 172.21.15.52:0/3008824315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52080ffba0 0x7f52081a2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:36.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.452+0000 7f520e0f4700 1 -- 172.21.15.52:0/3008824315 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52040095f0 con 0x7f52081046a0 2023-11-27T15:16:36.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.452+0000 7f520e0f4700 1 --2- 172.21.15.52:0/3008824315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52081046a0 0x7f520819c770 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5200007ee0 tx=0x7f520000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:36.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.452+0000 7f51feffd700 1 -- 172.21.15.52:0/3008824315 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5200017070 con 0x7f52081046a0 2023-11-27T15:16:36.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.453+0000 7f520fb57700 1 -- 172.21.15.52:0/3008824315 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52081168e0 con 0x7f52081046a0 2023-11-27T15:16:36.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.453+0000 7f51feffd700 1 -- 172.21.15.52:0/3008824315 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f520000f040 con 0x7f52081046a0 2023-11-27T15:16:36.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.453+0000 7f51feffd700 1 -- 172.21.15.52:0/3008824315 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5200005580 con 0x7f52081046a0 2023-11-27T15:16:36.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.453+0000 7f520fb57700 1 -- 172.21.15.52:0/3008824315 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5208116750 con 0x7f52081046a0 2023-11-27T15:16:36.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.454+0000 7f520fb57700 1 -- 172.21.15.52:0/3008824315 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5208103370 con 0x7f52081046a0 2023-11-27T15:16:36.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.456+0000 7f51feffd700 1 -- 172.21.15.52:0/3008824315 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f520000db20 con 0x7f52081046a0 2023-11-27T15:16:36.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.457+0000 7f51feffd700 1 --2- 172.21.15.52:0/3008824315 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f51f40a11e0 0x7f51f40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:36.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.457+0000 7f520d8f3700 1 --2- 172.21.15.52:0/3008824315 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f51f40a11e0 0x7f51f40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:36.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.457+0000 7f51feffd700 1 -- 172.21.15.52:0/3008824315 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f52000cc710 con 0x7f52081046a0 2023-11-27T15:16:36.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.457+0000 7f51feffd700 1 --2- 172.21.15.52:0/3008824315 >> [v2:172.21.15.52:6818/1002457464,v1:172.21.15.52:6819/1002457464] conn(0x7f51f40aa9b0 0x7f51f40acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:36.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.457+0000 7f51feffd700 1 -- 172.21.15.52:0/3008824315 --> [v2:172.21.15.52:6818/1002457464,v1:172.21.15.52:6819/1002457464] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f51f40ad470 con 0x7f51f40aa9b0 2023-11-27T15:16:36.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.457+0000 7f51feffd700 1 -- 172.21.15.52:0/3008824315 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52000cca80 con 0x7f52081046a0 2023-11-27T15:16:36.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.457+0000 7f520d0f2700 1 --2- 172.21.15.52:0/3008824315 >> [v2:172.21.15.52:6818/1002457464,v1:172.21.15.52:6819/1002457464] conn(0x7f51f40aa9b0 0x7f51f40acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:36.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.457+0000 7f520d8f3700 1 --2- 172.21.15.52:0/3008824315 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f51f40a11e0 0x7f51f40a3690 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5204000c00 tx=0x7f5204005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:36.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.458+0000 7f520d0f2700 1 --2- 172.21.15.52:0/3008824315 >> [v2:172.21.15.52:6818/1002457464,v1:172.21.15.52:6819/1002457464] conn(0x7f51f40aa9b0 0x7f51f40acdc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:36.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.458+0000 7f51feffd700 1 -- 172.21.15.52:0/3008824315 <== osd.6 v2:172.21.15.52:6818/1002457464 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f51f40ad470 con 0x7f51f40aa9b0 2023-11-27T15:16:36.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.491+0000 7f520fb57700 1 -- 172.21.15.52:0/3008824315 --> [v2:172.21.15.52:6818/1002457464,v1:172.21.15.52:6819/1002457464] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5208116a70 con 0x7f51f40aa9b0 2023-11-27T15:16:36.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.491+0000 7f51feffd700 1 -- 172.21.15.52:0/3008824315 <== osd.6 v2:172.21.15.52:6818/1002457464 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5208116a70 con 0x7f51f40aa9b0 2023-11-27T15:16:36.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.492+0000 7f520fb57700 1 -- 172.21.15.52:0/3008824315 >> [v2:172.21.15.52:6818/1002457464,v1:172.21.15.52:6819/1002457464] conn(0x7f51f40aa9b0 msgr2=0x7f51f40acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:36.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.492+0000 7f520fb57700 1 --2- 172.21.15.52:0/3008824315 >> [v2:172.21.15.52:6818/1002457464,v1:172.21.15.52:6819/1002457464] conn(0x7f51f40aa9b0 0x7f51f40acdc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:36.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.492+0000 7f520fb57700 1 -- 172.21.15.52:0/3008824315 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f51f40a11e0 msgr2=0x7f51f40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:36.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.492+0000 7f520fb57700 1 --2- 172.21.15.52:0/3008824315 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f51f40a11e0 0x7f51f40a3690 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5204000c00 tx=0x7f5204005960 comp rx=0 tx=0).stop 2023-11-27T15:16:36.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.492+0000 7f520fb57700 1 -- 172.21.15.52:0/3008824315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52081046a0 msgr2=0x7f520819c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:36.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.492+0000 7f520fb57700 1 --2- 172.21.15.52:0/3008824315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52081046a0 0x7f520819c770 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5200007ee0 tx=0x7f520000d3b0 comp rx=0 tx=0).stop 2023-11-27T15:16:36.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.492+0000 7f520e0f4700 1 -- 172.21.15.52:0/3008824315 reap_dead start 2023-11-27T15:16:36.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.492+0000 7f520fb57700 1 -- 172.21.15.52:0/3008824315 shutdown_connections 2023-11-27T15:16:36.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.492+0000 7f520fb57700 1 -- 172.21.15.52:0/3008824315 >> 172.21.15.52:0/3008824315 conn(0x7f52080facc0 msgr2=0x7f52080fcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:36.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.492+0000 7f520fb57700 1 -- 172.21.15.52:0/3008824315 shutdown_connections 2023-11-27T15:16:36.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.492+0000 7f520fb57700 1 -- 172.21.15.52:0/3008824315 wait complete. 2023-11-27T15:16:36.908 INFO:teuthology.orchestra.run.smithi052.stdout:141733920815 2023-11-27T15:16:36.908 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.4 2023-11-27T15:16:37.866 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:37 smithi052 ceph-mon[99056]: pgmap v270: 97 pgs: 97 active+clean; 633 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-11-27T15:16:38.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:37 smithi102 ceph-mon[100296]: pgmap v270: 97 pgs: 97 active+clean; 633 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-11-27T15:16:38.176 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.535+0000 7f554a031700 1 -- 172.21.15.52:0/3264365223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55440ff9e0 msgr2=0x7f554410c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:38.176 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.535+0000 7f554a031700 1 --2- 172.21.15.52:0/3264365223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55440ff9e0 0x7f554410c740 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5538009990 tx=0x7f5538009ca0 comp rx=0 tx=0).stop 2023-11-27T15:16:38.176 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.535+0000 7f554a031700 1 -- 172.21.15.52:0/3264365223 shutdown_connections 2023-11-27T15:16:38.176 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.535+0000 7f554a031700 1 --2- 172.21.15.52:0/3264365223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55440ff0d0 0x7f55440ff4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:38.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.535+0000 7f554a031700 1 --2- 172.21.15.52:0/3264365223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f554410cc80 0x7f554410f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:38.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.535+0000 7f554a031700 1 --2- 172.21.15.52:0/3264365223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55440ff9e0 0x7f554410c740 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:38.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.535+0000 7f554a031700 1 -- 172.21.15.52:0/3264365223 >> 172.21.15.52:0/3264365223 conn(0x7f55440770a0 msgr2=0x7f55440774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:38.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.535+0000 7f554a031700 1 -- 172.21.15.52:0/3264365223 shutdown_connections 2023-11-27T15:16:38.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.535+0000 7f554a031700 1 -- 172.21.15.52:0/3264365223 wait complete. 2023-11-27T15:16:38.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.536+0000 7f554a031700 1 Processor -- start 2023-11-27T15:16:38.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.536+0000 7f554a031700 1 -- start start 2023-11-27T15:16:38.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.536+0000 7f554a031700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55440ff0d0 0x7f55441986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:38.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.536+0000 7f554a031700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55440ff9e0 0x7f5544198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:38.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.536+0000 7f554a031700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f554410cc80 0x7f554419bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:38.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.536+0000 7f554a031700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554419c5e0 con 0x7f55440ff9e0 2023-11-27T15:16:38.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.536+0000 7f554a031700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554419c750 con 0x7f554410cc80 2023-11-27T15:16:38.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.536+0000 7f554a031700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554419c910 con 0x7f55440ff0d0 2023-11-27T15:16:38.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.536+0000 7f5542ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55440ff9e0 0x7f5544198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:38.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.536+0000 7f5542ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55440ff9e0 0x7f5544198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50342/0 (socket says 172.21.15.52:50342) 2023-11-27T15:16:38.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.536+0000 7f5542ffd700 1 -- 172.21.15.52:0/139060746 learned_addr learned my addr 172.21.15.52:0/139060746 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:38.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.536+0000 7f5543fff700 1 --2- 172.21.15.52:0/139060746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f554410cc80 0x7f554419bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:38.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.536+0000 7f55437fe700 1 --2- 172.21.15.52:0/139060746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55440ff0d0 0x7f55441986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:38.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.537+0000 7f5542ffd700 1 -- 172.21.15.52:0/139060746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55440ff0d0 msgr2=0x7f55441986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:38.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.537+0000 7f5542ffd700 1 --2- 172.21.15.52:0/139060746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55440ff0d0 0x7f55441986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:38.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.537+0000 7f5542ffd700 1 -- 172.21.15.52:0/139060746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f554410cc80 msgr2=0x7f554419bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:38.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.537+0000 7f5542ffd700 1 --2- 172.21.15.52:0/139060746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f554410cc80 0x7f554419bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:38.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.537+0000 7f5542ffd700 1 -- 172.21.15.52:0/139060746 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55380095f0 con 0x7f55440ff9e0 2023-11-27T15:16:38.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.537+0000 7f5543fff700 1 --2- 172.21.15.52:0/139060746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f554410cc80 0x7f554419bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:16:38.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.537+0000 7f5542ffd700 1 --2- 172.21.15.52:0/139060746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55440ff9e0 0x7f5544198c20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f553800b680 tx=0x7f5538004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:38.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.537+0000 7f5540ff9700 1 -- 172.21.15.52:0/139060746 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553801d070 con 0x7f55440ff9e0 2023-11-27T15:16:38.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.537+0000 7f5540ff9700 1 -- 172.21.15.52:0/139060746 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5538005030 con 0x7f55440ff9e0 2023-11-27T15:16:38.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.537+0000 7f554a031700 1 -- 172.21.15.52:0/139060746 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5544112570 con 0x7f55440ff9e0 2023-11-27T15:16:38.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.537+0000 7f5540ff9700 1 -- 172.21.15.52:0/139060746 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5538004c40 con 0x7f55440ff9e0 2023-11-27T15:16:38.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.537+0000 7f554a031700 1 -- 172.21.15.52:0/139060746 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55441123e0 con 0x7f55440ff9e0 2023-11-27T15:16:38.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.538+0000 7f554a031700 1 -- 172.21.15.52:0/139060746 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f554410ea50 con 0x7f55440ff9e0 2023-11-27T15:16:38.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.539+0000 7f5540ff9700 1 -- 172.21.15.52:0/139060746 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f55380054a0 con 0x7f55440ff9e0 2023-11-27T15:16:38.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.540+0000 7f5540ff9700 1 --2- 172.21.15.52:0/139060746 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f552c0a11e0 0x7f552c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:38.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.540+0000 7f55437fe700 1 --2- 172.21.15.52:0/139060746 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f552c0a11e0 0x7f552c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:38.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.540+0000 7f5540ff9700 1 -- 172.21.15.52:0/139060746 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f55380ce070 con 0x7f55440ff9e0 2023-11-27T15:16:38.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.540+0000 7f5540ff9700 1 --2- 172.21.15.52:0/139060746 >> [v2:172.21.15.150:6816/3672583574,v1:172.21.15.150:6817/3672583574] conn(0x7f552c0aa9b0 0x7f552c0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:38.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.540+0000 7f5540ff9700 1 -- 172.21.15.52:0/139060746 --> [v2:172.21.15.150:6816/3672583574,v1:172.21.15.150:6817/3672583574] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f552c0ad470 con 0x7f552c0aa9b0 2023-11-27T15:16:38.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.540+0000 7f5540ff9700 1 -- 172.21.15.52:0/139060746 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55380ce510 con 0x7f55440ff9e0 2023-11-27T15:16:38.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.541+0000 7f55437fe700 1 --2- 172.21.15.52:0/139060746 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f552c0a11e0 0x7f552c0a3690 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f553400f540 tx=0x7f553400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:38.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.541+0000 7f5543fff700 1 --2- 172.21.15.52:0/139060746 >> [v2:172.21.15.150:6816/3672583574,v1:172.21.15.150:6817/3672583574] conn(0x7f552c0aa9b0 0x7f552c0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:38.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.541+0000 7f5543fff700 1 --2- 172.21.15.52:0/139060746 >> [v2:172.21.15.150:6816/3672583574,v1:172.21.15.150:6817/3672583574] conn(0x7f552c0aa9b0 0x7f552c0acdc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:38.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.542+0000 7f5540ff9700 1 -- 172.21.15.52:0/139060746 <== osd.8 v2:172.21.15.150:6816/3672583574 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f552c0ad470 con 0x7f552c0aa9b0 2023-11-27T15:16:38.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.558+0000 7f554a031700 1 -- 172.21.15.52:0/139060746 --> [v2:172.21.15.150:6816/3672583574,v1:172.21.15.150:6817/3672583574] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5544112700 con 0x7f552c0aa9b0 2023-11-27T15:16:38.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.558+0000 7f5540ff9700 1 -- 172.21.15.52:0/139060746 <== osd.8 v2:172.21.15.150:6816/3672583574 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5544112700 con 0x7f552c0aa9b0 2023-11-27T15:16:38.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.559+0000 7f554a031700 1 -- 172.21.15.52:0/139060746 >> [v2:172.21.15.150:6816/3672583574,v1:172.21.15.150:6817/3672583574] conn(0x7f552c0aa9b0 msgr2=0x7f552c0acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:38.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.559+0000 7f554a031700 1 --2- 172.21.15.52:0/139060746 >> [v2:172.21.15.150:6816/3672583574,v1:172.21.15.150:6817/3672583574] conn(0x7f552c0aa9b0 0x7f552c0acdc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:38.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.559+0000 7f554a031700 1 -- 172.21.15.52:0/139060746 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f552c0a11e0 msgr2=0x7f552c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:38.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.559+0000 7f554a031700 1 --2- 172.21.15.52:0/139060746 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f552c0a11e0 0x7f552c0a3690 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f553400f540 tx=0x7f553400b040 comp rx=0 tx=0).stop 2023-11-27T15:16:38.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.559+0000 7f554a031700 1 -- 172.21.15.52:0/139060746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55440ff9e0 msgr2=0x7f5544198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:38.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.559+0000 7f554a031700 1 --2- 172.21.15.52:0/139060746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55440ff9e0 0x7f5544198c20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f553800b680 tx=0x7f5538004730 comp rx=0 tx=0).stop 2023-11-27T15:16:38.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.559+0000 7f5543fff700 1 -- 172.21.15.52:0/139060746 reap_dead start 2023-11-27T15:16:38.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.559+0000 7f554a031700 1 -- 172.21.15.52:0/139060746 shutdown_connections 2023-11-27T15:16:38.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.559+0000 7f554a031700 1 -- 172.21.15.52:0/139060746 >> 172.21.15.52:0/139060746 conn(0x7f55440770a0 msgr2=0x7f554410e080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:38.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.559+0000 7f554a031700 1 -- 172.21.15.52:0/139060746 shutdown_connections 2023-11-27T15:16:38.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:36.559+0000 7f554a031700 1 -- 172.21.15.52:0/139060746 wait complete. 2023-11-27T15:16:38.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:37 smithi150 ceph-mon[100559]: pgmap v270: 97 pgs: 97 active+clean; 633 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-11-27T15:16:38.507 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.502+0000 7f768a64a700 1 -- 172.21.15.52:0/2219444154 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76840ffe10 msgr2=0x7f7684103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:38.507 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.502+0000 7f768a64a700 1 --2- 172.21.15.52:0/2219444154 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76840ffe10 0x7f7684103de0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f76780099c0 tx=0x7f7678009cd0 comp rx=0 tx=0).stop 2023-11-27T15:16:38.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.502+0000 7f768a64a700 1 -- 172.21.15.52:0/2219444154 shutdown_connections 2023-11-27T15:16:38.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.502+0000 7f768a64a700 1 --2- 172.21.15.52:0/2219444154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76840ff470 0x7f76840ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:38.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.502+0000 7f768a64a700 1 --2- 172.21.15.52:0/2219444154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7684104510 0x7f768410c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:38.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.502+0000 7f768a64a700 1 --2- 172.21.15.52:0/2219444154 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76840ffe10 0x7f7684103de0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:38.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.502+0000 7f768a64a700 1 -- 172.21.15.52:0/2219444154 >> 172.21.15.52:0/2219444154 conn(0x7f76840770d0 msgr2=0x7f76840774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:38.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.502+0000 7f768a64a700 1 -- 172.21.15.52:0/2219444154 shutdown_connections 2023-11-27T15:16:38.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.502+0000 7f768a64a700 1 -- 172.21.15.52:0/2219444154 wait complete. 2023-11-27T15:16:38.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.503+0000 7f768a64a700 1 Processor -- start 2023-11-27T15:16:38.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.503+0000 7f768a64a700 1 -- start start 2023-11-27T15:16:38.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.503+0000 7f768a64a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76840ff470 0x7f7684195020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:38.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.503+0000 7f768a64a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76840ffe10 0x7f7684195560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:38.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.504+0000 7f768a64a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7684104510 0x7f7684195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:38.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.504+0000 7f768a64a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76841a4860 con 0x7f76840ffe10 2023-11-27T15:16:38.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.504+0000 7f768a64a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76841a49d0 con 0x7f76840ff470 2023-11-27T15:16:38.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.504+0000 7f768a64a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76841a4b90 con 0x7f7684104510 2023-11-27T15:16:38.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.504+0000 7f76837fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76840ffe10 0x7f7684195560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:38.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.504+0000 7f76837fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76840ffe10 0x7f7684195560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50356/0 (socket says 172.21.15.52:50356) 2023-11-27T15:16:38.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.504+0000 7f76837fe700 1 -- 172.21.15.52:0/2835893121 learned_addr learned my addr 172.21.15.52:0/2835893121 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:38.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.504+0000 7f7683fff700 1 --2- 172.21.15.52:0/2835893121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76840ff470 0x7f7684195020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:38.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.504+0000 7f7688be7700 1 --2- 172.21.15.52:0/2835893121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7684104510 0x7f7684195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:38.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.504+0000 7f76837fe700 1 -- 172.21.15.52:0/2835893121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7684104510 msgr2=0x7f7684195aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:38.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.504+0000 7f76837fe700 1 --2- 172.21.15.52:0/2835893121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7684104510 0x7f7684195aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:38.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.504+0000 7f76837fe700 1 -- 172.21.15.52:0/2835893121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76840ff470 msgr2=0x7f7684195020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:38.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.504+0000 7f76837fe700 1 --2- 172.21.15.52:0/2835893121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76840ff470 0x7f7684195020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:38.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.504+0000 7f76837fe700 1 -- 172.21.15.52:0/2835893121 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76780095f0 con 0x7f76840ffe10 2023-11-27T15:16:38.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.504+0000 7f7683fff700 1 --2- 172.21.15.52:0/2835893121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76840ff470 0x7f7684195020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:16:38.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.504+0000 7f76837fe700 1 --2- 172.21.15.52:0/2835893121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76840ffe10 0x7f7684195560 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7678009fd0 tx=0x7f767800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:38.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.505+0000 7f76817fa700 1 -- 172.21.15.52:0/2835893121 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767801d070 con 0x7f76840ffe10 2023-11-27T15:16:38.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.505+0000 7f76817fa700 1 -- 172.21.15.52:0/2835893121 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7678005050 con 0x7f76840ffe10 2023-11-27T15:16:38.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.505+0000 7f768a64a700 1 -- 172.21.15.52:0/2835893121 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f768410f290 con 0x7f76840ffe10 2023-11-27T15:16:38.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.505+0000 7f76817fa700 1 -- 172.21.15.52:0/2835893121 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7678004a90 con 0x7f76840ffe10 2023-11-27T15:16:38.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.505+0000 7f768a64a700 1 -- 172.21.15.52:0/2835893121 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f768410f100 con 0x7f76840ffe10 2023-11-27T15:16:38.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.507+0000 7f768a64a700 1 -- 172.21.15.52:0/2835893121 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f76840fd660 con 0x7f76840ffe10 2023-11-27T15:16:38.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.508+0000 7f76817fa700 1 -- 172.21.15.52:0/2835893121 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f7678005470 con 0x7f76840ffe10 2023-11-27T15:16:38.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.509+0000 7f76817fa700 1 --2- 172.21.15.52:0/2835893121 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f766c0a11e0 0x7f766c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:38.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.509+0000 7f76817fa700 1 -- 172.21.15.52:0/2835893121 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f76780cd620 con 0x7f76840ffe10 2023-11-27T15:16:38.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.509+0000 7f7683fff700 1 --2- 172.21.15.52:0/2835893121 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f766c0a11e0 0x7f766c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:38.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.509+0000 7f76817fa700 1 --2- 172.21.15.52:0/2835893121 >> [v2:172.21.15.52:6802/608216415,v1:172.21.15.52:6803/608216415] conn(0x7f766c0aa9b0 0x7f766c0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:38.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.509+0000 7f76817fa700 1 -- 172.21.15.52:0/2835893121 --> [v2:172.21.15.52:6802/608216415,v1:172.21.15.52:6803/608216415] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f766c0ad470 con 0x7f766c0aa9b0 2023-11-27T15:16:38.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.509+0000 7f76817fa700 1 -- 172.21.15.52:0/2835893121 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76780cfd30 con 0x7f76840ffe10 2023-11-27T15:16:38.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.510+0000 7f7688be7700 1 --2- 172.21.15.52:0/2835893121 >> [v2:172.21.15.52:6802/608216415,v1:172.21.15.52:6803/608216415] conn(0x7f766c0aa9b0 0x7f766c0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:38.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.510+0000 7f7683fff700 1 --2- 172.21.15.52:0/2835893121 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f766c0a11e0 0x7f766c0a3690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7674011490 tx=0x7f7674009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:38.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.510+0000 7f7688be7700 1 --2- 172.21.15.52:0/2835893121 >> [v2:172.21.15.52:6802/608216415,v1:172.21.15.52:6803/608216415] conn(0x7f766c0aa9b0 0x7f766c0acdc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:38.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.511+0000 7f76817fa700 1 -- 172.21.15.52:0/2835893121 <== osd.0 v2:172.21.15.52:6802/608216415 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f766c0ad470 con 0x7f766c0aa9b0 2023-11-27T15:16:38.534 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.530+0000 7f768a64a700 1 -- 172.21.15.52:0/2835893121 --> [v2:172.21.15.52:6802/608216415,v1:172.21.15.52:6803/608216415] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f768410f420 con 0x7f766c0aa9b0 2023-11-27T15:16:38.534 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.530+0000 7f76817fa700 1 -- 172.21.15.52:0/2835893121 <== osd.0 v2:172.21.15.52:6802/608216415 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f768410f420 con 0x7f766c0aa9b0 2023-11-27T15:16:38.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.531+0000 7f768a64a700 1 -- 172.21.15.52:0/2835893121 >> [v2:172.21.15.52:6802/608216415,v1:172.21.15.52:6803/608216415] conn(0x7f766c0aa9b0 msgr2=0x7f766c0acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:38.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.531+0000 7f768a64a700 1 --2- 172.21.15.52:0/2835893121 >> [v2:172.21.15.52:6802/608216415,v1:172.21.15.52:6803/608216415] conn(0x7f766c0aa9b0 0x7f766c0acdc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:38.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.531+0000 7f768a64a700 1 -- 172.21.15.52:0/2835893121 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f766c0a11e0 msgr2=0x7f766c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:38.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.531+0000 7f768a64a700 1 --2- 172.21.15.52:0/2835893121 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f766c0a11e0 0x7f766c0a3690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7674011490 tx=0x7f7674009260 comp rx=0 tx=0).stop 2023-11-27T15:16:38.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.531+0000 7f768a64a700 1 -- 172.21.15.52:0/2835893121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76840ffe10 msgr2=0x7f7684195560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:38.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.531+0000 7f768a64a700 1 --2- 172.21.15.52:0/2835893121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76840ffe10 0x7f7684195560 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7678009fd0 tx=0x7f767800f690 comp rx=0 tx=0).stop 2023-11-27T15:16:38.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.531+0000 7f7688be7700 1 -- 172.21.15.52:0/2835893121 reap_dead start 2023-11-27T15:16:38.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.531+0000 7f768a64a700 1 -- 172.21.15.52:0/2835893121 shutdown_connections 2023-11-27T15:16:38.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.531+0000 7f768a64a700 1 -- 172.21.15.52:0/2835893121 >> 172.21.15.52:0/2835893121 conn(0x7f76840770d0 msgr2=0x7f76840fcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:38.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.531+0000 7f768a64a700 1 -- 172.21.15.52:0/2835893121 shutdown_connections 2023-11-27T15:16:38.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.531+0000 7f768a64a700 1 -- 172.21.15.52:0/2835893121 wait complete. 2023-11-27T15:16:38.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.680+0000 7f0642f63700 1 -- 172.21.15.52:0/1971625841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f063c104510 msgr2=0x7f063c10c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:38.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.680+0000 7f0642f63700 1 --2- 172.21.15.52:0/1971625841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f063c104510 0x7f063c10c090 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f063400b0d0 tx=0x7f063400b3e0 comp rx=0 tx=0).stop 2023-11-27T15:16:38.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.680+0000 7f0642f63700 1 -- 172.21.15.52:0/1971625841 shutdown_connections 2023-11-27T15:16:38.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.680+0000 7f0642f63700 1 --2- 172.21.15.52:0/1971625841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f063c0ff470 0x7f063c0ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:38.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.680+0000 7f0642f63700 1 --2- 172.21.15.52:0/1971625841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f063c104510 0x7f063c10c090 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:38.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.680+0000 7f0642f63700 1 --2- 172.21.15.52:0/1971625841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f063c0ffe10 0x7f063c103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:38.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.680+0000 7f0642f63700 1 -- 172.21.15.52:0/1971625841 >> 172.21.15.52:0/1971625841 conn(0x7f063c0770d0 msgr2=0x7f063c0774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:38.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.680+0000 7f0642f63700 1 -- 172.21.15.52:0/1971625841 shutdown_connections 2023-11-27T15:16:38.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.680+0000 7f0642f63700 1 -- 172.21.15.52:0/1971625841 wait complete. 2023-11-27T15:16:38.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.681+0000 7f0642f63700 1 Processor -- start 2023-11-27T15:16:38.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.681+0000 7f0642f63700 1 -- start start 2023-11-27T15:16:38.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.681+0000 7f0642f63700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f063c0ff470 0x7f063c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:38.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.681+0000 7f0642f63700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f063c0ffe10 0x7f063c198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:38.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.681+0000 7f0642f63700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f063c104510 0x7f063c19bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:38.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.681+0000 7f0642f63700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063c19c5f0 con 0x7f063c104510 2023-11-27T15:16:38.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.681+0000 7f0642f63700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063c19c760 con 0x7f063c0ffe10 2023-11-27T15:16:38.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.681+0000 7f0642f63700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063c19c920 con 0x7f063c0ff470 2023-11-27T15:16:38.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.681+0000 7f0641500700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f063c104510 0x7f063c19bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:38.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.681+0000 7f0641500700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f063c104510 0x7f063c19bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50376/0 (socket says 172.21.15.52:50376) 2023-11-27T15:16:38.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.681+0000 7f0641500700 1 -- 172.21.15.52:0/2750508088 learned_addr learned my addr 172.21.15.52:0/2750508088 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:38.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.681+0000 7f063bfff700 1 --2- 172.21.15.52:0/2750508088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f063c0ffe10 0x7f063c198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:38.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.681+0000 7f0640cff700 1 --2- 172.21.15.52:0/2750508088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f063c0ff470 0x7f063c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:38.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.681+0000 7f0641500700 1 -- 172.21.15.52:0/2750508088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f063c0ff470 msgr2=0x7f063c1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:38.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.681+0000 7f0641500700 1 --2- 172.21.15.52:0/2750508088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f063c0ff470 0x7f063c1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:38.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.681+0000 7f0641500700 1 -- 172.21.15.52:0/2750508088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f063c0ffe10 msgr2=0x7f063c198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:38.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.681+0000 7f0641500700 1 --2- 172.21.15.52:0/2750508088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f063c0ffe10 0x7f063c198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:38.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.682+0000 7f0641500700 1 -- 172.21.15.52:0/2750508088 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0634009d00 con 0x7f063c104510 2023-11-27T15:16:38.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.682+0000 7f0641500700 1 --2- 172.21.15.52:0/2750508088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f063c104510 0x7f063c19bff0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0634007870 tx=0x7f06340078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:38.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.682+0000 7f0639ffb700 1 -- 172.21.15.52:0/2750508088 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063400e040 con 0x7f063c104510 2023-11-27T15:16:38.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.682+0000 7f0639ffb700 1 -- 172.21.15.52:0/2750508088 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0634003e10 con 0x7f063c104510 2023-11-27T15:16:38.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.682+0000 7f0642f63700 1 -- 172.21.15.52:0/2750508088 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f063c10f270 con 0x7f063c104510 2023-11-27T15:16:38.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.682+0000 7f0639ffb700 1 -- 172.21.15.52:0/2750508088 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063401ba10 con 0x7f063c104510 2023-11-27T15:16:38.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.682+0000 7f0642f63700 1 -- 172.21.15.52:0/2750508088 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f063c10f0e0 con 0x7f063c104510 2023-11-27T15:16:38.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.683+0000 7f0642f63700 1 -- 172.21.15.52:0/2750508088 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f063c069890 con 0x7f063c104510 2023-11-27T15:16:38.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.684+0000 7f0639ffb700 1 -- 172.21.15.52:0/2750508088 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f0634005090 con 0x7f063c104510 2023-11-27T15:16:38.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.685+0000 7f0639ffb700 1 --2- 172.21.15.52:0/2750508088 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f062c0a1260 0x7f062c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:38.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.685+0000 7f0640cff700 1 --2- 172.21.15.52:0/2750508088 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f062c0a1260 0x7f062c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:38.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.685+0000 7f0639ffb700 1 -- 172.21.15.52:0/2750508088 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f06340ce1e0 con 0x7f063c104510 2023-11-27T15:16:38.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.685+0000 7f0639ffb700 1 --2- 172.21.15.52:0/2750508088 >> [v2:172.21.15.102:6800/2538384087,v1:172.21.15.102:6801/2538384087] conn(0x7f062c0aaa30 0x7f062c0ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:38.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.685+0000 7f0639ffb700 1 -- 172.21.15.52:0/2750508088 --> [v2:172.21.15.102:6800/2538384087,v1:172.21.15.102:6801/2538384087] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f062c0ad4f0 con 0x7f062c0aaa30 2023-11-27T15:16:38.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.685+0000 7f0639ffb700 1 -- 172.21.15.52:0/2750508088 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06340ce680 con 0x7f063c104510 2023-11-27T15:16:38.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.685+0000 7f0640cff700 1 --2- 172.21.15.52:0/2750508088 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f062c0a1260 0x7f062c0a3710 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0630005950 tx=0x7f063000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:38.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.685+0000 7f063bfff700 1 --2- 172.21.15.52:0/2750508088 >> [v2:172.21.15.102:6800/2538384087,v1:172.21.15.102:6801/2538384087] conn(0x7f062c0aaa30 0x7f062c0ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:38.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.686+0000 7f063bfff700 1 --2- 172.21.15.52:0/2750508088 >> [v2:172.21.15.102:6800/2538384087,v1:172.21.15.102:6801/2538384087] conn(0x7f062c0aaa30 0x7f062c0ace40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:38.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.687+0000 7f0639ffb700 1 -- 172.21.15.52:0/2750508088 <== osd.1 v2:172.21.15.102:6800/2538384087 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f062c0ad4f0 con 0x7f062c0aaa30 2023-11-27T15:16:38.707 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.703+0000 7f0642f63700 1 -- 172.21.15.52:0/2750508088 --> [v2:172.21.15.102:6800/2538384087,v1:172.21.15.102:6801/2538384087] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f063c10f400 con 0x7f062c0aaa30 2023-11-27T15:16:38.707 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.703+0000 7f0639ffb700 1 -- 172.21.15.52:0/2750508088 <== osd.1 v2:172.21.15.102:6800/2538384087 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f063c10f400 con 0x7f062c0aaa30 2023-11-27T15:16:38.708 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.703+0000 7f0642f63700 1 -- 172.21.15.52:0/2750508088 >> [v2:172.21.15.102:6800/2538384087,v1:172.21.15.102:6801/2538384087] conn(0x7f062c0aaa30 msgr2=0x7f062c0ace40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:38.708 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.704+0000 7f0642f63700 1 --2- 172.21.15.52:0/2750508088 >> [v2:172.21.15.102:6800/2538384087,v1:172.21.15.102:6801/2538384087] conn(0x7f062c0aaa30 0x7f062c0ace40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:38.708 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.704+0000 7f0642f63700 1 -- 172.21.15.52:0/2750508088 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f062c0a1260 msgr2=0x7f062c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:38.708 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.704+0000 7f0642f63700 1 --2- 172.21.15.52:0/2750508088 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f062c0a1260 0x7f062c0a3710 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0630005950 tx=0x7f063000a300 comp rx=0 tx=0).stop 2023-11-27T15:16:38.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.704+0000 7f0642f63700 1 -- 172.21.15.52:0/2750508088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f063c104510 msgr2=0x7f063c19bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:38.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.704+0000 7f0642f63700 1 --2- 172.21.15.52:0/2750508088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f063c104510 0x7f063c19bff0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0634007870 tx=0x7f06340078a0 comp rx=0 tx=0).stop 2023-11-27T15:16:38.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.704+0000 7f0641500700 1 -- 172.21.15.52:0/2750508088 reap_dead start 2023-11-27T15:16:38.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.704+0000 7f0642f63700 1 -- 172.21.15.52:0/2750508088 shutdown_connections 2023-11-27T15:16:38.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.704+0000 7f0642f63700 1 -- 172.21.15.52:0/2750508088 >> 172.21.15.52:0/2750508088 conn(0x7f063c0770d0 msgr2=0x7f063c068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:38.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.704+0000 7f0642f63700 1 -- 172.21.15.52:0/2750508088 shutdown_connections 2023-11-27T15:16:38.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:38.704+0000 7f0642f63700 1 -- 172.21.15.52:0/2750508088 wait complete. 2023-11-27T15:16:39.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.196+0000 7f4346e35700 1 -- 172.21.15.52:0/2648744442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43400ff460 msgr2=0x7f43400ff830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:39.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.196+0000 7f4346e35700 1 --2- 172.21.15.52:0/2648744442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43400ff460 0x7f43400ff830 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f43340099c0 tx=0x7f4334009cd0 comp rx=0 tx=0).stop 2023-11-27T15:16:39.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.196+0000 7f4346e35700 1 -- 172.21.15.52:0/2648744442 shutdown_connections 2023-11-27T15:16:39.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.196+0000 7f4346e35700 1 --2- 172.21.15.52:0/2648744442 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4340104500 0x7f434010c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:39.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.196+0000 7f4346e35700 1 --2- 172.21.15.52:0/2648744442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43400ff460 0x7f43400ff830 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:39.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.196+0000 7f4346e35700 1 --2- 172.21.15.52:0/2648744442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43400ffe00 0x7f4340103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:39.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.196+0000 7f4346e35700 1 -- 172.21.15.52:0/2648744442 >> 172.21.15.52:0/2648744442 conn(0x7f43400770a0 msgr2=0x7f43400774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:39.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.196+0000 7f4346e35700 1 -- 172.21.15.52:0/2648744442 shutdown_connections 2023-11-27T15:16:39.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.196+0000 7f4346e35700 1 -- 172.21.15.52:0/2648744442 wait complete. 2023-11-27T15:16:39.203 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.197+0000 7f4346e35700 1 Processor -- start 2023-11-27T15:16:39.203 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.197+0000 7f4346e35700 1 -- start start 2023-11-27T15:16:39.203 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.197+0000 7f4346e35700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43400ff460 0x7f4340197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:39.204 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.197+0000 7f4346e35700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43400ffe00 0x7f4340197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:39.204 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.197+0000 7f4346e35700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4340104500 0x7f43401a22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:39.204 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.197+0000 7f4346e35700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43401980b0 con 0x7f4340104500 2023-11-27T15:16:39.204 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.197+0000 7f4346e35700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43401a27f0 con 0x7f43400ffe00 2023-11-27T15:16:39.204 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.197+0000 7f4346e35700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43401a29b0 con 0x7f43400ff460 2023-11-27T15:16:39.204 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.197+0000 7f43453d2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4340104500 0x7f43401a22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:39.204 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.197+0000 7f43453d2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4340104500 0x7f43401a22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50392/0 (socket says 172.21.15.52:50392) 2023-11-27T15:16:39.204 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.197+0000 7f43453d2700 1 -- 172.21.15.52:0/2112413367 learned_addr learned my addr 172.21.15.52:0/2112413367 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:39.204 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.197+0000 7f4344bd1700 1 --2- 172.21.15.52:0/2112413367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43400ff460 0x7f4340197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:39.204 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.198+0000 7f433ffff700 1 --2- 172.21.15.52:0/2112413367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43400ffe00 0x7f4340197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:39.204 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.198+0000 7f43453d2700 1 -- 172.21.15.52:0/2112413367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43400ff460 msgr2=0x7f4340197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:39.204 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.198+0000 7f43453d2700 1 --2- 172.21.15.52:0/2112413367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43400ff460 0x7f4340197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:39.205 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.198+0000 7f43453d2700 1 -- 172.21.15.52:0/2112413367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43400ffe00 msgr2=0x7f4340197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:39.205 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.198+0000 7f43453d2700 1 --2- 172.21.15.52:0/2112413367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43400ffe00 0x7f4340197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:39.205 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.198+0000 7f43453d2700 1 -- 172.21.15.52:0/2112413367 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43340095f0 con 0x7f4340104500 2023-11-27T15:16:39.205 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.198+0000 7f43453d2700 1 --2- 172.21.15.52:0/2112413367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4340104500 0x7f43401a22b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4338007ee0 tx=0x7f433800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:39.205 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.198+0000 7f433dffb700 1 -- 172.21.15.52:0/2112413367 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4338017070 con 0x7f4340104500 2023-11-27T15:16:39.205 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.198+0000 7f433dffb700 1 -- 172.21.15.52:0/2112413367 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f433800f040 con 0x7f4340104500 2023-11-27T15:16:39.205 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.198+0000 7f4346e35700 1 -- 172.21.15.52:0/2112413367 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f434010f240 con 0x7f4340104500 2023-11-27T15:16:39.205 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.198+0000 7f433dffb700 1 -- 172.21.15.52:0/2112413367 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4338005580 con 0x7f4340104500 2023-11-27T15:16:39.205 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.198+0000 7f4346e35700 1 -- 172.21.15.52:0/2112413367 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f434010f0b0 con 0x7f4340104500 2023-11-27T15:16:39.205 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.199+0000 7f4346e35700 1 -- 172.21.15.52:0/2112413367 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f43400fd6e0 con 0x7f4340104500 2023-11-27T15:16:39.206 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.200+0000 7f433dffb700 1 -- 172.21.15.52:0/2112413367 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f433800db20 con 0x7f4340104500 2023-11-27T15:16:39.207 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.201+0000 7f433dffb700 1 --2- 172.21.15.52:0/2112413367 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f43300a1260 0x7f43300a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:39.207 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.201+0000 7f4344bd1700 1 --2- 172.21.15.52:0/2112413367 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f43300a1260 0x7f43300a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:39.207 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.201+0000 7f433dffb700 1 -- 172.21.15.52:0/2112413367 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f43380cc690 con 0x7f4340104500 2023-11-27T15:16:39.207 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.201+0000 7f433dffb700 1 --2- 172.21.15.52:0/2112413367 >> [v2:172.21.15.150:6808/461511375,v1:172.21.15.150:6809/461511375] conn(0x7f43300aaa30 0x7f43300ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:39.207 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.201+0000 7f433dffb700 1 -- 172.21.15.52:0/2112413367 --> [v2:172.21.15.150:6808/461511375,v1:172.21.15.150:6809/461511375] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f43300ad4f0 con 0x7f43300aaa30 2023-11-27T15:16:39.207 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.201+0000 7f433dffb700 1 -- 172.21.15.52:0/2112413367 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43380cca00 con 0x7f4340104500 2023-11-27T15:16:39.207 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.201+0000 7f4344bd1700 1 --2- 172.21.15.52:0/2112413367 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f43300a1260 0x7f43300a3710 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4334000c00 tx=0x7f4334003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:39.207 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.201+0000 7f433ffff700 1 --2- 172.21.15.52:0/2112413367 >> [v2:172.21.15.150:6808/461511375,v1:172.21.15.150:6809/461511375] conn(0x7f43300aaa30 0x7f43300ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:39.207 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.202+0000 7f433ffff700 1 --2- 172.21.15.52:0/2112413367 >> [v2:172.21.15.150:6808/461511375,v1:172.21.15.150:6809/461511375] conn(0x7f43300aaa30 0x7f43300ace40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:39.207 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.203+0000 7f433dffb700 1 -- 172.21.15.52:0/2112413367 <== osd.5 v2:172.21.15.150:6808/461511375 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f43300ad4f0 con 0x7f43300aaa30 2023-11-27T15:16:39.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.219+0000 7f4346e35700 1 -- 172.21.15.52:0/2112413367 --> [v2:172.21.15.150:6808/461511375,v1:172.21.15.150:6809/461511375] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f434010f3d0 con 0x7f43300aaa30 2023-11-27T15:16:39.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.219+0000 7f433dffb700 1 -- 172.21.15.52:0/2112413367 <== osd.5 v2:172.21.15.150:6808/461511375 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f434010f3d0 con 0x7f43300aaa30 2023-11-27T15:16:39.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.219+0000 7f4346e35700 1 -- 172.21.15.52:0/2112413367 >> [v2:172.21.15.150:6808/461511375,v1:172.21.15.150:6809/461511375] conn(0x7f43300aaa30 msgr2=0x7f43300ace40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:39.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.219+0000 7f4346e35700 1 --2- 172.21.15.52:0/2112413367 >> [v2:172.21.15.150:6808/461511375,v1:172.21.15.150:6809/461511375] conn(0x7f43300aaa30 0x7f43300ace40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:39.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.220+0000 7f4346e35700 1 -- 172.21.15.52:0/2112413367 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f43300a1260 msgr2=0x7f43300a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:39.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.220+0000 7f4346e35700 1 --2- 172.21.15.52:0/2112413367 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f43300a1260 0x7f43300a3710 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4334000c00 tx=0x7f4334003680 comp rx=0 tx=0).stop 2023-11-27T15:16:39.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.220+0000 7f4346e35700 1 -- 172.21.15.52:0/2112413367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4340104500 msgr2=0x7f43401a22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:39.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.220+0000 7f4346e35700 1 --2- 172.21.15.52:0/2112413367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4340104500 0x7f43401a22b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4338007ee0 tx=0x7f433800d3b0 comp rx=0 tx=0).stop 2023-11-27T15:16:39.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.220+0000 7f43453d2700 1 -- 172.21.15.52:0/2112413367 reap_dead start 2023-11-27T15:16:39.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.220+0000 7f4346e35700 1 -- 172.21.15.52:0/2112413367 shutdown_connections 2023-11-27T15:16:39.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.220+0000 7f4346e35700 1 -- 172.21.15.52:0/2112413367 >> 172.21.15.52:0/2112413367 conn(0x7f43400770a0 msgr2=0x7f43400fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:39.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.220+0000 7f4346e35700 1 -- 172.21.15.52:0/2112413367 shutdown_connections 2023-11-27T15:16:39.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:39.220+0000 7f4346e35700 1 -- 172.21.15.52:0/2112413367 wait complete. 2023-11-27T15:16:40.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:39 smithi150 ceph-mon[100559]: pgmap v271: 97 pgs: 97 active+clean; 637 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-11-27T15:16:40.037 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:39 smithi052 ceph-mon[99056]: pgmap v271: 97 pgs: 97 active+clean; 637 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-11-27T15:16:40.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:39 smithi102 ceph-mon[100296]: pgmap v271: 97 pgs: 97 active+clean; 637 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-11-27T15:16:40.388 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.381+0000 7f55cef66700 1 -- 172.21.15.52:0/396926673 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55c8104500 msgr2=0x7f55c810c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:40.388 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.381+0000 7f55cef66700 1 --2- 172.21.15.52:0/396926673 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55c8104500 0x7f55c810c080 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f55c000b0d0 tx=0x7f55c000b3e0 comp rx=0 tx=0).stop 2023-11-27T15:16:40.388 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.381+0000 7f55cef66700 1 -- 172.21.15.52:0/396926673 shutdown_connections 2023-11-27T15:16:40.388 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.381+0000 7f55cef66700 1 --2- 172.21.15.52:0/396926673 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55c8104500 0x7f55c810c080 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:40.388 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.381+0000 7f55cef66700 1 --2- 172.21.15.52:0/396926673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55c80ff460 0x7f55c80ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:40.388 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.381+0000 7f55cef66700 1 --2- 172.21.15.52:0/396926673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55c80ffe00 0x7f55c8103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:40.388 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.381+0000 7f55cef66700 1 -- 172.21.15.52:0/396926673 >> 172.21.15.52:0/396926673 conn(0x7f55c80770a0 msgr2=0x7f55c80774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:40.389 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.381+0000 7f55cef66700 1 -- 172.21.15.52:0/396926673 shutdown_connections 2023-11-27T15:16:40.389 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.381+0000 7f55cef66700 1 -- 172.21.15.52:0/396926673 wait complete. 2023-11-27T15:16:40.389 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.381+0000 7f55cef66700 1 Processor -- start 2023-11-27T15:16:40.389 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.381+0000 7f55cef66700 1 -- start start 2023-11-27T15:16:40.389 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.382+0000 7f55cef66700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55c80ff460 0x7f55c8197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:40.389 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.382+0000 7f55cef66700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55c80ffe00 0x7f55c81978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:40.389 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.382+0000 7f55cef66700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55c8104500 0x7f55c81a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:40.389 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.382+0000 7f55cef66700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c81981b0 con 0x7f55c80ffe00 2023-11-27T15:16:40.389 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.382+0000 7f55cef66700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c81a2920 con 0x7f55c8104500 2023-11-27T15:16:40.389 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.382+0000 7f55cef66700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c81a2a90 con 0x7f55c80ff460 2023-11-27T15:16:40.389 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.382+0000 7f55c7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55c80ffe00 0x7f55c81978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:40.389 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.382+0000 7f55c7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55c80ffe00 0x7f55c81978a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50406/0 (socket says 172.21.15.52:50406) 2023-11-27T15:16:40.389 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.382+0000 7f55c7fff700 1 -- 172.21.15.52:0/2423987102 learned_addr learned my addr 172.21.15.52:0/2423987102 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:40.390 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.382+0000 7f55ccd02700 1 --2- 172.21.15.52:0/2423987102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55c80ff460 0x7f55c8197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:40.390 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.382+0000 7f55cd503700 1 --2- 172.21.15.52:0/2423987102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55c8104500 0x7f55c81a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:40.390 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.382+0000 7f55c7fff700 1 -- 172.21.15.52:0/2423987102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55c80ff460 msgr2=0x7f55c8197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:40.390 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.382+0000 7f55c7fff700 1 --2- 172.21.15.52:0/2423987102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55c80ff460 0x7f55c8197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:40.390 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.382+0000 7f55c7fff700 1 -- 172.21.15.52:0/2423987102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55c8104500 msgr2=0x7f55c81a23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:40.390 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.382+0000 7f55c7fff700 1 --2- 172.21.15.52:0/2423987102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55c8104500 0x7f55c81a23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:40.390 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.382+0000 7f55c7fff700 1 -- 172.21.15.52:0/2423987102 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c0009d00 con 0x7f55c80ffe00 2023-11-27T15:16:40.390 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.382+0000 7f55cd503700 1 --2- 172.21.15.52:0/2423987102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55c8104500 0x7f55c81a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:16:40.390 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.382+0000 7f55c7fff700 1 --2- 172.21.15.52:0/2423987102 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55c80ffe00 0x7f55c81978a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f55b000e970 tx=0x7f55b000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:40.390 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.382+0000 7f55c5ffb700 1 -- 172.21.15.52:0/2423987102 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b0009790 con 0x7f55c80ffe00 2023-11-27T15:16:40.391 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.382+0000 7f55c5ffb700 1 -- 172.21.15.52:0/2423987102 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55b000eec0 con 0x7f55c80ffe00 2023-11-27T15:16:40.391 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.382+0000 7f55cef66700 1 -- 172.21.15.52:0/2423987102 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c810f240 con 0x7f55c80ffe00 2023-11-27T15:16:40.391 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.383+0000 7f55c5ffb700 1 -- 172.21.15.52:0/2423987102 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b0005590 con 0x7f55c80ffe00 2023-11-27T15:16:40.391 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.383+0000 7f55cef66700 1 -- 172.21.15.52:0/2423987102 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c810f0b0 con 0x7f55c80ffe00 2023-11-27T15:16:40.391 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.383+0000 7f55cef66700 1 -- 172.21.15.52:0/2423987102 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f55c80fd610 con 0x7f55c80ffe00 2023-11-27T15:16:40.392 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.385+0000 7f55c5ffb700 1 -- 172.21.15.52:0/2423987102 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f55b000cbd0 con 0x7f55c80ffe00 2023-11-27T15:16:40.392 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.385+0000 7f55c5ffb700 1 --2- 172.21.15.52:0/2423987102 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f55b40a11e0 0x7f55b40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:40.392 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.385+0000 7f55c5ffb700 1 -- 172.21.15.52:0/2423987102 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f55b0014070 con 0x7f55c80ffe00 2023-11-27T15:16:40.392 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.385+0000 7f55ccd02700 1 --2- 172.21.15.52:0/2423987102 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f55b40a11e0 0x7f55b40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:40.392 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.386+0000 7f55c5ffb700 1 --2- 172.21.15.52:0/2423987102 >> [v2:172.21.15.102:6824/3056034201,v1:172.21.15.102:6825/3056034201] conn(0x7f55b40aa9b0 0x7f55b40acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:40.392 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.386+0000 7f55c5ffb700 1 -- 172.21.15.52:0/2423987102 --> [v2:172.21.15.102:6824/3056034201,v1:172.21.15.102:6825/3056034201] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f55b40ad470 con 0x7f55b40aa9b0 2023-11-27T15:16:40.392 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.386+0000 7f55c5ffb700 1 -- 172.21.15.52:0/2423987102 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55b00cd190 con 0x7f55c80ffe00 2023-11-27T15:16:40.393 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.386+0000 7f55ccd02700 1 --2- 172.21.15.52:0/2423987102 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f55b40a11e0 0x7f55b40a3690 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f55b8005950 tx=0x7f55b800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:40.393 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.386+0000 7f55cd503700 1 --2- 172.21.15.52:0/2423987102 >> [v2:172.21.15.102:6824/3056034201,v1:172.21.15.102:6825/3056034201] conn(0x7f55b40aa9b0 0x7f55b40acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:40.393 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.386+0000 7f55cd503700 1 --2- 172.21.15.52:0/2423987102 >> [v2:172.21.15.102:6824/3056034201,v1:172.21.15.102:6825/3056034201] conn(0x7f55b40aa9b0 0x7f55b40acdc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:40.393 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.387+0000 7f55c5ffb700 1 -- 172.21.15.52:0/2423987102 <== osd.10 v2:172.21.15.102:6824/3056034201 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f55b40ad470 con 0x7f55b40aa9b0 2023-11-27T15:16:40.408 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.403+0000 7f55cef66700 1 -- 172.21.15.52:0/2423987102 --> [v2:172.21.15.102:6824/3056034201,v1:172.21.15.102:6825/3056034201] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f55c810f3d0 con 0x7f55b40aa9b0 2023-11-27T15:16:40.408 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.404+0000 7f55c5ffb700 1 -- 172.21.15.52:0/2423987102 <== osd.10 v2:172.21.15.102:6824/3056034201 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f55c810f3d0 con 0x7f55b40aa9b0 2023-11-27T15:16:40.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.404+0000 7f55cef66700 1 -- 172.21.15.52:0/2423987102 >> [v2:172.21.15.102:6824/3056034201,v1:172.21.15.102:6825/3056034201] conn(0x7f55b40aa9b0 msgr2=0x7f55b40acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:40.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.404+0000 7f55cef66700 1 --2- 172.21.15.52:0/2423987102 >> [v2:172.21.15.102:6824/3056034201,v1:172.21.15.102:6825/3056034201] conn(0x7f55b40aa9b0 0x7f55b40acdc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:40.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.404+0000 7f55cef66700 1 -- 172.21.15.52:0/2423987102 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f55b40a11e0 msgr2=0x7f55b40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:40.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.404+0000 7f55cef66700 1 --2- 172.21.15.52:0/2423987102 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f55b40a11e0 0x7f55b40a3690 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f55b8005950 tx=0x7f55b800b410 comp rx=0 tx=0).stop 2023-11-27T15:16:40.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.404+0000 7f55cef66700 1 -- 172.21.15.52:0/2423987102 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55c80ffe00 msgr2=0x7f55c81978a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:40.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.404+0000 7f55cef66700 1 --2- 172.21.15.52:0/2423987102 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55c80ffe00 0x7f55c81978a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f55b000e970 tx=0x7f55b000ec80 comp rx=0 tx=0).stop 2023-11-27T15:16:40.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.404+0000 7f55cd503700 1 -- 172.21.15.52:0/2423987102 reap_dead start 2023-11-27T15:16:40.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.404+0000 7f55cef66700 1 -- 172.21.15.52:0/2423987102 shutdown_connections 2023-11-27T15:16:40.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.404+0000 7f55cef66700 1 -- 172.21.15.52:0/2423987102 >> 172.21.15.52:0/2423987102 conn(0x7f55c80770a0 msgr2=0x7f55c80fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:40.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.404+0000 7f55cef66700 1 -- 172.21.15.52:0/2423987102 shutdown_connections 2023-11-27T15:16:40.410 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:40.404+0000 7f55cef66700 1 -- 172.21.15.52:0/2423987102 wait complete. 2023-11-27T15:16:40.856 INFO:teuthology.orchestra.run.smithi052.stdout:55834574909 2023-11-27T15:16:40.857 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.1 2023-11-27T15:16:40.858 INFO:teuthology.orchestra.run.smithi052.stdout:34359738432 2023-11-27T15:16:40.858 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.0 2023-11-27T15:16:41.404 INFO:teuthology.orchestra.run.smithi052.stdout:249108103198 2023-11-27T15:16:41.405 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.8 2023-11-27T15:16:41.755 INFO:teuthology.orchestra.run.smithi052.stdout:193273528359 2023-11-27T15:16:41.755 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.6 2023-11-27T15:16:41.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.578+0000 7fb968b22700 1 -- 172.21.15.52:0/4169116047 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb964101e50 msgr2=0x7fb96410a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:41.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.578+0000 7fb968b22700 1 --2- 172.21.15.52:0/4169116047 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb964101e50 0x7fb96410a350 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb958009990 tx=0x7fb958009ca0 comp rx=0 tx=0).stop 2023-11-27T15:16:41.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.579+0000 7fb968b22700 1 -- 172.21.15.52:0/4169116047 shutdown_connections 2023-11-27T15:16:41.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.579+0000 7fb968b22700 1 --2- 172.21.15.52:0/4169116047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb96410a890 0x7fb96410ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:41.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.579+0000 7fb968b22700 1 --2- 172.21.15.52:0/4169116047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb964101540 0x7fb964101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:41.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.579+0000 7fb968b22700 1 --2- 172.21.15.52:0/4169116047 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb964101e50 0x7fb96410a350 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:41.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.579+0000 7fb968b22700 1 -- 172.21.15.52:0/4169116047 >> 172.21.15.52:0/4169116047 conn(0x7fb9640770a0 msgr2=0x7fb9640774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:41.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.579+0000 7fb968b22700 1 -- 172.21.15.52:0/4169116047 shutdown_connections 2023-11-27T15:16:41.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.579+0000 7fb968b22700 1 -- 172.21.15.52:0/4169116047 wait complete. 2023-11-27T15:16:41.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.579+0000 7fb968b22700 1 Processor -- start 2023-11-27T15:16:41.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.579+0000 7fb968b22700 1 -- start start 2023-11-27T15:16:41.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.580+0000 7fb968b22700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb964101540 0x7fb9640ff1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:41.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.580+0000 7fb968b22700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb964101e50 0x7fb9640ff6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:41.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.580+0000 7fb968b22700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb96410a890 0x7fb9640ffc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:41.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.580+0000 7fb968b22700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9641a4900 con 0x7fb964101540 2023-11-27T15:16:41.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.580+0000 7fb968b22700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9641a4a70 con 0x7fb96410a890 2023-11-27T15:16:41.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.580+0000 7fb968b22700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9641a4c30 con 0x7fb964101e50 2023-11-27T15:16:41.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.580+0000 7fb96259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb964101540 0x7fb9640ff1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:41.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.580+0000 7fb96259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb964101540 0x7fb9640ff1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50432/0 (socket says 172.21.15.52:50432) 2023-11-27T15:16:41.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.580+0000 7fb96259c700 1 -- 172.21.15.52:0/3837128610 learned_addr learned my addr 172.21.15.52:0/3837128610 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:41.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.580+0000 7fb961d9b700 1 --2- 172.21.15.52:0/3837128610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb964101e50 0x7fb9640ff6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:41.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.580+0000 7fb962d9d700 1 --2- 172.21.15.52:0/3837128610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb96410a890 0x7fb9640ffc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:41.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.580+0000 7fb96259c700 1 -- 172.21.15.52:0/3837128610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb964101e50 msgr2=0x7fb9640ff6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:41.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.580+0000 7fb96259c700 1 --2- 172.21.15.52:0/3837128610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb964101e50 0x7fb9640ff6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:41.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.580+0000 7fb96259c700 1 -- 172.21.15.52:0/3837128610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb96410a890 msgr2=0x7fb9640ffc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:41.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.580+0000 7fb96259c700 1 --2- 172.21.15.52:0/3837128610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb96410a890 0x7fb9640ffc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:41.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.580+0000 7fb96259c700 1 -- 172.21.15.52:0/3837128610 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9580095f0 con 0x7fb964101540 2023-11-27T15:16:41.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.580+0000 7fb96259c700 1 --2- 172.21.15.52:0/3837128610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb964101540 0x7fb9640ff1a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb95400d950 tx=0x7fb95400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:41.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.580+0000 7fb9537fe700 1 -- 172.21.15.52:0/3837128610 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb954009940 con 0x7fb964101540 2023-11-27T15:16:41.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.581+0000 7fb9537fe700 1 -- 172.21.15.52:0/3837128610 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb954010460 con 0x7fb964101540 2023-11-27T15:16:41.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.581+0000 7fb968b22700 1 -- 172.21.15.52:0/3837128610 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb964110360 con 0x7fb964101540 2023-11-27T15:16:41.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.581+0000 7fb9537fe700 1 -- 172.21.15.52:0/3837128610 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95400f620 con 0x7fb964101540 2023-11-27T15:16:41.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.581+0000 7fb968b22700 1 -- 172.21.15.52:0/3837128610 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9641101d0 con 0x7fb964101540 2023-11-27T15:16:41.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.581+0000 7fb968b22700 1 -- 172.21.15.52:0/3837128610 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb9640fd860 con 0x7fb964101540 2023-11-27T15:16:41.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.583+0000 7fb9537fe700 1 -- 172.21.15.52:0/3837128610 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb9540108d0 con 0x7fb964101540 2023-11-27T15:16:41.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.583+0000 7fb9537fe700 1 --2- 172.21.15.52:0/3837128610 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb94c0a11e0 0x7fb94c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:41.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.583+0000 7fb961d9b700 1 --2- 172.21.15.52:0/3837128610 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb94c0a11e0 0x7fb94c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:41.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.583+0000 7fb9537fe700 1 -- 172.21.15.52:0/3837128610 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb9540ccc10 con 0x7fb964101540 2023-11-27T15:16:41.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.584+0000 7fb9537fe700 1 --2- 172.21.15.52:0/3837128610 >> [v2:172.21.15.150:6824/3207559739,v1:172.21.15.150:6825/3207559739] conn(0x7fb94c0aa9b0 0x7fb94c0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:41.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.584+0000 7fb9537fe700 1 -- 172.21.15.52:0/3837128610 --> [v2:172.21.15.150:6824/3207559739,v1:172.21.15.150:6825/3207559739] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb94c0ad470 con 0x7fb94c0aa9b0 2023-11-27T15:16:41.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.584+0000 7fb9537fe700 1 -- 172.21.15.52:0/3837128610 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9540cd0b0 con 0x7fb964101540 2023-11-27T15:16:41.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.584+0000 7fb961d9b700 1 --2- 172.21.15.52:0/3837128610 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb94c0a11e0 0x7fb94c0a3690 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb964100990 tx=0x7fb958005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:41.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.584+0000 7fb962d9d700 1 --2- 172.21.15.52:0/3837128610 >> [v2:172.21.15.150:6824/3207559739,v1:172.21.15.150:6825/3207559739] conn(0x7fb94c0aa9b0 0x7fb94c0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:41.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.584+0000 7fb962d9d700 1 --2- 172.21.15.52:0/3837128610 >> [v2:172.21.15.150:6824/3207559739,v1:172.21.15.150:6825/3207559739] conn(0x7fb94c0aa9b0 0x7fb94c0acdc0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:41.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.585+0000 7fb9537fe700 1 -- 172.21.15.52:0/3837128610 <== osd.11 v2:172.21.15.150:6824/3207559739 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb94c0ad470 con 0x7fb94c0aa9b0 2023-11-27T15:16:41.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.601+0000 7fb968b22700 1 -- 172.21.15.52:0/3837128610 --> [v2:172.21.15.150:6824/3207559739,v1:172.21.15.150:6825/3207559739] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb9641104f0 con 0x7fb94c0aa9b0 2023-11-27T15:16:41.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.602+0000 7fb9537fe700 1 -- 172.21.15.52:0/3837128610 <== osd.11 v2:172.21.15.150:6824/3207559739 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb9641104f0 con 0x7fb94c0aa9b0 2023-11-27T15:16:41.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.602+0000 7fb968b22700 1 -- 172.21.15.52:0/3837128610 >> [v2:172.21.15.150:6824/3207559739,v1:172.21.15.150:6825/3207559739] conn(0x7fb94c0aa9b0 msgr2=0x7fb94c0acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:41.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.602+0000 7fb968b22700 1 --2- 172.21.15.52:0/3837128610 >> [v2:172.21.15.150:6824/3207559739,v1:172.21.15.150:6825/3207559739] conn(0x7fb94c0aa9b0 0x7fb94c0acdc0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:41.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.602+0000 7fb968b22700 1 -- 172.21.15.52:0/3837128610 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb94c0a11e0 msgr2=0x7fb94c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:41.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.602+0000 7fb968b22700 1 --2- 172.21.15.52:0/3837128610 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb94c0a11e0 0x7fb94c0a3690 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb964100990 tx=0x7fb958005fb0 comp rx=0 tx=0).stop 2023-11-27T15:16:41.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.602+0000 7fb968b22700 1 -- 172.21.15.52:0/3837128610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb964101540 msgr2=0x7fb9640ff1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:41.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.603+0000 7fb968b22700 1 --2- 172.21.15.52:0/3837128610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb964101540 0x7fb9640ff1a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb95400d950 tx=0x7fb95400dc60 comp rx=0 tx=0).stop 2023-11-27T15:16:41.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.603+0000 7fb962d9d700 1 -- 172.21.15.52:0/3837128610 reap_dead start 2023-11-27T15:16:41.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.603+0000 7fb968b22700 1 -- 172.21.15.52:0/3837128610 shutdown_connections 2023-11-27T15:16:41.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.603+0000 7fb968b22700 1 -- 172.21.15.52:0/3837128610 >> 172.21.15.52:0/3837128610 conn(0x7fb9640770a0 msgr2=0x7fb9640fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:41.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.603+0000 7fb968b22700 1 -- 172.21.15.52:0/3837128610 shutdown_connections 2023-11-27T15:16:41.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.603+0000 7fb968b22700 1 -- 172.21.15.52:0/3837128610 wait complete. 2023-11-27T15:16:41.933 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.080+0000 7fc7b77b1700 1 -- 172.21.15.52:0/1658959516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7b0101ca0 msgr2=0x7fc7b010a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:41.934 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.080+0000 7fc7b77b1700 1 --2- 172.21.15.52:0/1658959516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7b0101ca0 0x7fc7b010a370 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0009990 tx=0x7fc7a0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:16:41.934 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.080+0000 7fc7b77b1700 1 -- 172.21.15.52:0/1658959516 shutdown_connections 2023-11-27T15:16:41.934 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.080+0000 7fc7b77b1700 1 --2- 172.21.15.52:0/1658959516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7b0101390 0x7fc7b0101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:41.934 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.080+0000 7fc7b77b1700 1 --2- 172.21.15.52:0/1658959516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7b010a8b0 0x7fc7b010ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:41.934 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.080+0000 7fc7b77b1700 1 --2- 172.21.15.52:0/1658959516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7b0101ca0 0x7fc7b010a370 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:41.934 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.080+0000 7fc7b77b1700 1 -- 172.21.15.52:0/1658959516 >> 172.21.15.52:0/1658959516 conn(0x7fc7b00facc0 msgr2=0x7fc7b00fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:41.934 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.080+0000 7fc7b77b1700 1 -- 172.21.15.52:0/1658959516 shutdown_connections 2023-11-27T15:16:41.934 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.080+0000 7fc7b77b1700 1 -- 172.21.15.52:0/1658959516 wait complete. 2023-11-27T15:16:41.934 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.081+0000 7fc7b77b1700 1 Processor -- start 2023-11-27T15:16:41.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.081+0000 7fc7b77b1700 1 -- start start 2023-11-27T15:16:41.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.081+0000 7fc7b77b1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7b0101390 0x7fc7b0198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:41.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.081+0000 7fc7b77b1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7b0101ca0 0x7fc7b0198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:41.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.081+0000 7fc7b77b1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7b010a8b0 0x7fc7b019cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:41.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.081+0000 7fc7b77b1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b019d550 con 0x7fc7b010a8b0 2023-11-27T15:16:41.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.081+0000 7fc7b77b1700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b019d6c0 con 0x7fc7b0101390 2023-11-27T15:16:41.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.081+0000 7fc7b77b1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b019d880 con 0x7fc7b0101ca0 2023-11-27T15:16:41.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.081+0000 7fc7b5d4e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7b010a8b0 0x7fc7b019cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:41.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.081+0000 7fc7b5d4e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7b010a8b0 0x7fc7b019cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50422/0 (socket says 172.21.15.52:50422) 2023-11-27T15:16:41.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.081+0000 7fc7b5d4e700 1 -- 172.21.15.52:0/2757450262 learned_addr learned my addr 172.21.15.52:0/2757450262 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:41.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.081+0000 7fc7b554d700 1 --2- 172.21.15.52:0/2757450262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7b0101390 0x7fc7b0198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:41.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.081+0000 7fc7b4d4c700 1 --2- 172.21.15.52:0/2757450262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7b0101ca0 0x7fc7b0198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:41.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.082+0000 7fc7b5d4e700 1 -- 172.21.15.52:0/2757450262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7b0101ca0 msgr2=0x7fc7b0198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:41.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.082+0000 7fc7b5d4e700 1 --2- 172.21.15.52:0/2757450262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7b0101ca0 0x7fc7b0198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:41.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.082+0000 7fc7b5d4e700 1 -- 172.21.15.52:0/2757450262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7b0101390 msgr2=0x7fc7b0198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:41.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.082+0000 7fc7b5d4e700 1 --2- 172.21.15.52:0/2757450262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7b0101390 0x7fc7b0198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:41.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.082+0000 7fc7b5d4e700 1 -- 172.21.15.52:0/2757450262 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a00095f0 con 0x7fc7b010a8b0 2023-11-27T15:16:41.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.082+0000 7fc7b554d700 1 --2- 172.21.15.52:0/2757450262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7b0101390 0x7fc7b0198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:16:41.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.082+0000 7fc7b5d4e700 1 --2- 172.21.15.52:0/2757450262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7b010a8b0 0x7fc7b019cf40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8007d40 tx=0x7fc7a800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:41.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.082+0000 7fc7a67fc700 1 -- 172.21.15.52:0/2757450262 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a800ac90 con 0x7fc7b010a8b0 2023-11-27T15:16:41.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.082+0000 7fc7a67fc700 1 -- 172.21.15.52:0/2757450262 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7a8007e90 con 0x7fc7b010a8b0 2023-11-27T15:16:41.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.082+0000 7fc7b77b1700 1 -- 172.21.15.52:0/2757450262 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b0110330 con 0x7fc7b010a8b0 2023-11-27T15:16:41.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.082+0000 7fc7a67fc700 1 -- 172.21.15.52:0/2757450262 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a80115e0 con 0x7fc7b010a8b0 2023-11-27T15:16:41.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.082+0000 7fc7b77b1700 1 -- 172.21.15.52:0/2757450262 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b01101a0 con 0x7fc7b010a8b0 2023-11-27T15:16:41.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.083+0000 7fc7b77b1700 1 -- 172.21.15.52:0/2757450262 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc7b00fca70 con 0x7fc7b010a8b0 2023-11-27T15:16:41.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.084+0000 7fc7a67fc700 1 -- 172.21.15.52:0/2757450262 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc7a8012420 con 0x7fc7b010a8b0 2023-11-27T15:16:41.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.085+0000 7fc7a67fc700 1 --2- 172.21.15.52:0/2757450262 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fc79c0a11e0 0x7fc79c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:41.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.085+0000 7fc7b554d700 1 --2- 172.21.15.52:0/2757450262 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fc79c0a11e0 0x7fc79c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:41.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.085+0000 7fc7a67fc700 1 -- 172.21.15.52:0/2757450262 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc7a80cd000 con 0x7fc7b010a8b0 2023-11-27T15:16:41.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.085+0000 7fc7a67fc700 1 --2- 172.21.15.52:0/2757450262 >> [v2:172.21.15.52:6810/2622100752,v1:172.21.15.52:6811/2622100752] conn(0x7fc79c0aa9b0 0x7fc79c0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:41.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.085+0000 7fc7a67fc700 1 -- 172.21.15.52:0/2757450262 --> [v2:172.21.15.52:6810/2622100752,v1:172.21.15.52:6811/2622100752] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc79c0ad470 con 0x7fc79c0aa9b0 2023-11-27T15:16:41.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.085+0000 7fc7a67fc700 1 -- 172.21.15.52:0/2757450262 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7a80cd370 con 0x7fc7b010a8b0 2023-11-27T15:16:41.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.085+0000 7fc7b4d4c700 1 --2- 172.21.15.52:0/2757450262 >> [v2:172.21.15.52:6810/2622100752,v1:172.21.15.52:6811/2622100752] conn(0x7fc79c0aa9b0 0x7fc79c0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:41.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.085+0000 7fc7b554d700 1 --2- 172.21.15.52:0/2757450262 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fc79c0a11e0 0x7fc79c0a3690 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac005950 tx=0x7fc7ac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:41.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.086+0000 7fc7b4d4c700 1 --2- 172.21.15.52:0/2757450262 >> [v2:172.21.15.52:6810/2622100752,v1:172.21.15.52:6811/2622100752] conn(0x7fc79c0aa9b0 0x7fc79c0acdc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:41.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.086+0000 7fc7a67fc700 1 -- 172.21.15.52:0/2757450262 <== osd.3 v2:172.21.15.52:6810/2622100752 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc79c0ad470 con 0x7fc79c0aa9b0 2023-11-27T15:16:41.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.102+0000 7fc7b77b1700 1 -- 172.21.15.52:0/2757450262 --> [v2:172.21.15.52:6810/2622100752,v1:172.21.15.52:6811/2622100752] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc7b01104c0 con 0x7fc79c0aa9b0 2023-11-27T15:16:41.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.103+0000 7fc7a67fc700 1 -- 172.21.15.52:0/2757450262 <== osd.3 v2:172.21.15.52:6810/2622100752 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc7b01104c0 con 0x7fc79c0aa9b0 2023-11-27T15:16:41.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.103+0000 7fc7b77b1700 1 -- 172.21.15.52:0/2757450262 >> [v2:172.21.15.52:6810/2622100752,v1:172.21.15.52:6811/2622100752] conn(0x7fc79c0aa9b0 msgr2=0x7fc79c0acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:41.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.103+0000 7fc7b77b1700 1 --2- 172.21.15.52:0/2757450262 >> [v2:172.21.15.52:6810/2622100752,v1:172.21.15.52:6811/2622100752] conn(0x7fc79c0aa9b0 0x7fc79c0acdc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:41.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.103+0000 7fc7b77b1700 1 -- 172.21.15.52:0/2757450262 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fc79c0a11e0 msgr2=0x7fc79c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:41.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.103+0000 7fc7b77b1700 1 --2- 172.21.15.52:0/2757450262 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fc79c0a11e0 0x7fc79c0a3690 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac005950 tx=0x7fc7ac00b410 comp rx=0 tx=0).stop 2023-11-27T15:16:41.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.103+0000 7fc7b77b1700 1 -- 172.21.15.52:0/2757450262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7b010a8b0 msgr2=0x7fc7b019cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:41.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.103+0000 7fc7b77b1700 1 --2- 172.21.15.52:0/2757450262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc7b010a8b0 0x7fc7b019cf40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8007d40 tx=0x7fc7a800a340 comp rx=0 tx=0).stop 2023-11-27T15:16:41.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.103+0000 7fc7b5d4e700 1 -- 172.21.15.52:0/2757450262 reap_dead start 2023-11-27T15:16:41.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.104+0000 7fc7b77b1700 1 -- 172.21.15.52:0/2757450262 shutdown_connections 2023-11-27T15:16:41.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.104+0000 7fc7b77b1700 1 -- 172.21.15.52:0/2757450262 >> 172.21.15.52:0/2757450262 conn(0x7fc7b00facc0 msgr2=0x7fc7b00fb090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:41.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.104+0000 7fc7b77b1700 1 -- 172.21.15.52:0/2757450262 shutdown_connections 2023-11-27T15:16:41.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:41.104+0000 7fc7b77b1700 1 -- 172.21.15.52:0/2757450262 wait complete. 2023-11-27T15:16:41.997 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:41 smithi052 ceph-mon[99056]: pgmap v272: 97 pgs: 97 active+clean; 637 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2023-11-27T15:16:41.998 INFO:teuthology.orchestra.run.smithi052.stdout:163208757292 2023-11-27T15:16:41.999 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.5 2023-11-27T15:16:42.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:41 smithi102 ceph-mon[100296]: pgmap v272: 97 pgs: 97 active+clean; 637 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2023-11-27T15:16:42.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:41 smithi150 ceph-mon[100559]: pgmap v272: 97 pgs: 97 active+clean; 637 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2023-11-27T15:16:42.988 INFO:teuthology.orchestra.run.smithi052.stdout:107374182452 2023-11-27T15:16:42.988 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.3 2023-11-27T15:16:42.998 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.994+0000 7f18b877f700 1 -- 172.21.15.52:0/2011427767 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18b00ff9e0 msgr2=0x7f18b010c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:42.998 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.994+0000 7f18b877f700 1 --2- 172.21.15.52:0/2011427767 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18b00ff9e0 0x7f18b010c740 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009990 tx=0x7f18a0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:16:42.998 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.994+0000 7f18b877f700 1 -- 172.21.15.52:0/2011427767 shutdown_connections 2023-11-27T15:16:42.998 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.994+0000 7f18b877f700 1 --2- 172.21.15.52:0/2011427767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18b00ff0d0 0x7f18b00ff4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:42.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.994+0000 7f18b877f700 1 --2- 172.21.15.52:0/2011427767 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18b010cc80 0x7f18b010f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:42.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.994+0000 7f18b877f700 1 --2- 172.21.15.52:0/2011427767 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18b00ff9e0 0x7f18b010c740 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:42.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.994+0000 7f18b877f700 1 -- 172.21.15.52:0/2011427767 >> 172.21.15.52:0/2011427767 conn(0x7f18b00770a0 msgr2=0x7f18b00774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:42.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.994+0000 7f18b877f700 1 -- 172.21.15.52:0/2011427767 shutdown_connections 2023-11-27T15:16:42.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.994+0000 7f18b877f700 1 -- 172.21.15.52:0/2011427767 wait complete. 2023-11-27T15:16:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.995+0000 7f18b877f700 1 Processor -- start 2023-11-27T15:16:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.995+0000 7f18b877f700 1 -- start start 2023-11-27T15:16:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.995+0000 7f18b877f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18b00ff0d0 0x7f18b0198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.995+0000 7f18b877f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18b00ff9e0 0x7f18b0198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.995+0000 7f18b877f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18b010cc80 0x7f18b019bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.995+0000 7f18b877f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b019c590 con 0x7f18b00ff0d0 2023-11-27T15:16:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.995+0000 7f18b877f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b019c700 con 0x7f18b010cc80 2023-11-27T15:16:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.995+0000 7f18b877f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b019c8c0 con 0x7f18b00ff9e0 2023-11-27T15:16:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.995+0000 7f18b651b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18b00ff0d0 0x7f18b0198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.995+0000 7f18b651b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18b00ff0d0 0x7f18b0198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50446/0 (socket says 172.21.15.52:50446) 2023-11-27T15:16:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.995+0000 7f18b651b700 1 -- 172.21.15.52:0/3102814475 learned_addr learned my addr 172.21.15.52:0/3102814475 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.995+0000 7f18b5d1a700 1 --2- 172.21.15.52:0/3102814475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18b00ff9e0 0x7f18b0198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.995+0000 7f18b6d1c700 1 --2- 172.21.15.52:0/3102814475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18b010cc80 0x7f18b019bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.995+0000 7f18b651b700 1 -- 172.21.15.52:0/3102814475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18b00ff9e0 msgr2=0x7f18b0198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.995+0000 7f18b651b700 1 --2- 172.21.15.52:0/3102814475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18b00ff9e0 0x7f18b0198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.995+0000 7f18b651b700 1 -- 172.21.15.52:0/3102814475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18b010cc80 msgr2=0x7f18b019bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.995+0000 7f18b651b700 1 --2- 172.21.15.52:0/3102814475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18b010cc80 0x7f18b019bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.995+0000 7f18b651b700 1 -- 172.21.15.52:0/3102814475 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a00095f0 con 0x7f18b00ff0d0 2023-11-27T15:16:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.996+0000 7f18b651b700 1 --2- 172.21.15.52:0/3102814475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18b00ff0d0 0x7f18b0198690 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f18ac00d950 tx=0x7f18ac00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:43.003 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.996+0000 7f18a77fe700 1 -- 172.21.15.52:0/3102814475 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ac009940 con 0x7f18b00ff0d0 2023-11-27T15:16:43.003 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.996+0000 7f18b877f700 1 -- 172.21.15.52:0/3102814475 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b01125c0 con 0x7f18b00ff0d0 2023-11-27T15:16:43.003 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.996+0000 7f18a77fe700 1 -- 172.21.15.52:0/3102814475 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18ac010460 con 0x7f18b00ff0d0 2023-11-27T15:16:43.003 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.996+0000 7f18a77fe700 1 -- 172.21.15.52:0/3102814475 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ac00f620 con 0x7f18b00ff0d0 2023-11-27T15:16:43.003 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.996+0000 7f18b877f700 1 -- 172.21.15.52:0/3102814475 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b0112430 con 0x7f18b00ff0d0 2023-11-27T15:16:43.003 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.997+0000 7f18b877f700 1 -- 172.21.15.52:0/3102814475 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f18b010eae0 con 0x7f18b00ff0d0 2023-11-27T15:16:43.004 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.998+0000 7f18a77fe700 1 -- 172.21.15.52:0/3102814475 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f18ac0108d0 con 0x7f18b00ff0d0 2023-11-27T15:16:43.004 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.999+0000 7f18a77fe700 1 --2- 172.21.15.52:0/3102814475 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f189c0a1260 0x7f189c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:43.004 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.999+0000 7f18a77fe700 1 -- 172.21.15.52:0/3102814475 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f18ac0ccb50 con 0x7f18b00ff0d0 2023-11-27T15:16:43.004 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.999+0000 7f18b5d1a700 1 --2- 172.21.15.52:0/3102814475 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f189c0a1260 0x7f189c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:43.004 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.999+0000 7f18a77fe700 1 --2- 172.21.15.52:0/3102814475 >> [v2:172.21.15.150:6800/4258038221,v1:172.21.15.150:6801/4258038221] conn(0x7f189c0aaa30 0x7f189c0ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:43.004 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.999+0000 7f18a77fe700 1 -- 172.21.15.52:0/3102814475 --> [v2:172.21.15.150:6800/4258038221,v1:172.21.15.150:6801/4258038221] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f189c0ad4f0 con 0x7f189c0aaa30 2023-11-27T15:16:43.004 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.999+0000 7f18a77fe700 1 -- 172.21.15.52:0/3102814475 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18ac0ccff0 con 0x7f18b00ff0d0 2023-11-27T15:16:43.004 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.999+0000 7f18b6d1c700 1 --2- 172.21.15.52:0/3102814475 >> [v2:172.21.15.150:6800/4258038221,v1:172.21.15.150:6801/4258038221] conn(0x7f189c0aaa30 0x7f189c0ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:43.004 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:42.999+0000 7f18b5d1a700 1 --2- 172.21.15.52:0/3102814475 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f189c0a1260 0x7f189c0a3710 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f18b0199cb0 tx=0x7f18a0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:43.005 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.000+0000 7f18b6d1c700 1 --2- 172.21.15.52:0/3102814475 >> [v2:172.21.15.150:6800/4258038221,v1:172.21.15.150:6801/4258038221] conn(0x7f189c0aaa30 0x7f189c0ace40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:43.005 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.001+0000 7f18a77fe700 1 -- 172.21.15.52:0/3102814475 <== osd.2 v2:172.21.15.150:6800/4258038221 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f189c0ad4f0 con 0x7f189c0aaa30 2023-11-27T15:16:43.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.020+0000 7f18b877f700 1 -- 172.21.15.52:0/3102814475 --> [v2:172.21.15.150:6800/4258038221,v1:172.21.15.150:6801/4258038221] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f18b0112750 con 0x7f189c0aaa30 2023-11-27T15:16:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.020+0000 7f18a77fe700 1 -- 172.21.15.52:0/3102814475 <== osd.2 v2:172.21.15.150:6800/4258038221 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f18b0112750 con 0x7f189c0aaa30 2023-11-27T15:16:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.021+0000 7f18b877f700 1 -- 172.21.15.52:0/3102814475 >> [v2:172.21.15.150:6800/4258038221,v1:172.21.15.150:6801/4258038221] conn(0x7f189c0aaa30 msgr2=0x7f189c0ace40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.021+0000 7f18b877f700 1 --2- 172.21.15.52:0/3102814475 >> [v2:172.21.15.150:6800/4258038221,v1:172.21.15.150:6801/4258038221] conn(0x7f189c0aaa30 0x7f189c0ace40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.021+0000 7f18b877f700 1 -- 172.21.15.52:0/3102814475 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f189c0a1260 msgr2=0x7f189c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.021+0000 7f18b877f700 1 --2- 172.21.15.52:0/3102814475 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f189c0a1260 0x7f189c0a3710 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f18b0199cb0 tx=0x7f18a0005fb0 comp rx=0 tx=0).stop 2023-11-27T15:16:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.021+0000 7f18b877f700 1 -- 172.21.15.52:0/3102814475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18b00ff0d0 msgr2=0x7f18b0198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.021+0000 7f18b877f700 1 --2- 172.21.15.52:0/3102814475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18b00ff0d0 0x7f18b0198690 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f18ac00d950 tx=0x7f18ac00dc60 comp rx=0 tx=0).stop 2023-11-27T15:16:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.021+0000 7f18b6d1c700 1 -- 172.21.15.52:0/3102814475 reap_dead start 2023-11-27T15:16:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.021+0000 7f18b877f700 1 -- 172.21.15.52:0/3102814475 shutdown_connections 2023-11-27T15:16:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.021+0000 7f18b877f700 1 -- 172.21.15.52:0/3102814475 >> 172.21.15.52:0/3102814475 conn(0x7f18b00770a0 msgr2=0x7f18b010e090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.021+0000 7f18b877f700 1 -- 172.21.15.52:0/3102814475 shutdown_connections 2023-11-27T15:16:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.021+0000 7f18b877f700 1 -- 172.21.15.52:0/3102814475 wait complete. 2023-11-27T15:16:43.212 INFO:teuthology.orchestra.run.smithi052.stdout:335007449106 2023-11-27T15:16:43.212 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.11 2023-11-27T15:16:43.610 INFO:teuthology.orchestra.run.smithi052.stdout:304942678038 2023-11-27T15:16:43.610 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.10 2023-11-27T15:16:43.697 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.693+0000 7fdf6ae3e700 1 -- 172.21.15.52:0/817285960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf64104500 msgr2=0x7fdf6410c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:43.697 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.693+0000 7fdf6ae3e700 1 --2- 172.21.15.52:0/817285960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf64104500 0x7fdf6410c080 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00b0d0 tx=0x7fdf5c00b3e0 comp rx=0 tx=0).stop 2023-11-27T15:16:43.697 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.693+0000 7fdf6ae3e700 1 -- 172.21.15.52:0/817285960 shutdown_connections 2023-11-27T15:16:43.698 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.693+0000 7fdf6ae3e700 1 --2- 172.21.15.52:0/817285960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf64104500 0x7fdf6410c080 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:43.698 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.693+0000 7fdf6ae3e700 1 --2- 172.21.15.52:0/817285960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf640ff460 0x7fdf640ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:43.698 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.693+0000 7fdf6ae3e700 1 --2- 172.21.15.52:0/817285960 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf640ffe00 0x7fdf64103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:43.698 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.693+0000 7fdf6ae3e700 1 -- 172.21.15.52:0/817285960 >> 172.21.15.52:0/817285960 conn(0x7fdf640770a0 msgr2=0x7fdf640774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:43.698 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.693+0000 7fdf6ae3e700 1 -- 172.21.15.52:0/817285960 shutdown_connections 2023-11-27T15:16:43.698 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.693+0000 7fdf6ae3e700 1 -- 172.21.15.52:0/817285960 wait complete. 2023-11-27T15:16:43.699 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.694+0000 7fdf6ae3e700 1 Processor -- start 2023-11-27T15:16:43.699 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.694+0000 7fdf6ae3e700 1 -- start start 2023-11-27T15:16:43.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.694+0000 7fdf6ae3e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf640ff460 0x7fdf64197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:43.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.694+0000 7fdf6ae3e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf640ffe00 0x7fdf64197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:43.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.694+0000 7fdf6ae3e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf64104500 0x7fdf641a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:43.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.694+0000 7fdf6ae3e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf64198270 con 0x7fdf640ffe00 2023-11-27T15:16:43.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.694+0000 7fdf6ae3e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf641a29e0 con 0x7fdf64104500 2023-11-27T15:16:43.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.694+0000 7fdf6ae3e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf641a2b50 con 0x7fdf640ff460 2023-11-27T15:16:43.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.695+0000 7fdf63fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf640ffe00 0x7fdf64197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:43.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.695+0000 7fdf63fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf640ffe00 0x7fdf64197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50464/0 (socket says 172.21.15.52:50464) 2023-11-27T15:16:43.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.695+0000 7fdf63fff700 1 -- 172.21.15.52:0/2857933815 learned_addr learned my addr 172.21.15.52:0/2857933815 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:43.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.695+0000 7fdf68bda700 1 --2- 172.21.15.52:0/2857933815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf640ff460 0x7fdf64197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:43.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.695+0000 7fdf693db700 1 --2- 172.21.15.52:0/2857933815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf64104500 0x7fdf641a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:43.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.695+0000 7fdf63fff700 1 -- 172.21.15.52:0/2857933815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf640ff460 msgr2=0x7fdf64197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:43.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.695+0000 7fdf63fff700 1 --2- 172.21.15.52:0/2857933815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf640ff460 0x7fdf64197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:43.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.695+0000 7fdf63fff700 1 -- 172.21.15.52:0/2857933815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf64104500 msgr2=0x7fdf641a24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:43.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.695+0000 7fdf63fff700 1 --2- 172.21.15.52:0/2857933815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf64104500 0x7fdf641a24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:43.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.695+0000 7fdf63fff700 1 -- 172.21.15.52:0/2857933815 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5c009d00 con 0x7fdf640ffe00 2023-11-27T15:16:43.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.695+0000 7fdf693db700 1 --2- 172.21.15.52:0/2857933815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf64104500 0x7fdf641a24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:16:43.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.695+0000 7fdf68bda700 1 --2- 172.21.15.52:0/2857933815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf640ff460 0x7fdf64197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:16:43.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.695+0000 7fdf63fff700 1 --2- 172.21.15.52:0/2857933815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf640ffe00 0x7fdf64197960 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000e970 tx=0x7fdf5000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:43.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.696+0000 7fdf61ffb700 1 -- 172.21.15.52:0/2857933815 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf50009790 con 0x7fdf640ffe00 2023-11-27T15:16:43.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.696+0000 7fdf6ae3e700 1 -- 172.21.15.52:0/2857933815 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf6410f2c0 con 0x7fdf640ffe00 2023-11-27T15:16:43.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.696+0000 7fdf61ffb700 1 -- 172.21.15.52:0/2857933815 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf50004cf0 con 0x7fdf640ffe00 2023-11-27T15:16:43.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.696+0000 7fdf61ffb700 1 -- 172.21.15.52:0/2857933815 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf50005680 con 0x7fdf640ffe00 2023-11-27T15:16:43.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.696+0000 7fdf6ae3e700 1 -- 172.21.15.52:0/2857933815 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf6410f130 con 0x7fdf640ffe00 2023-11-27T15:16:43.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.697+0000 7fdf6ae3e700 1 -- 172.21.15.52:0/2857933815 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdf640fd630 con 0x7fdf640ffe00 2023-11-27T15:16:43.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.699+0000 7fdf61ffb700 1 -- 172.21.15.52:0/2857933815 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fdf50004750 con 0x7fdf640ffe00 2023-11-27T15:16:43.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.699+0000 7fdf61ffb700 1 --2- 172.21.15.52:0/2857933815 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fdf540a11e0 0x7fdf540a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:43.704 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.700+0000 7fdf61ffb700 1 -- 172.21.15.52:0/2857933815 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdf50014070 con 0x7fdf640ffe00 2023-11-27T15:16:43.704 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.700+0000 7fdf61ffb700 1 --2- 172.21.15.52:0/2857933815 >> [v2:172.21.15.52:6826/4287934320,v1:172.21.15.52:6827/4287934320] conn(0x7fdf540aa9b0 0x7fdf540acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:43.704 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.700+0000 7fdf61ffb700 1 -- 172.21.15.52:0/2857933815 --> [v2:172.21.15.52:6826/4287934320,v1:172.21.15.52:6827/4287934320] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf540ad470 con 0x7fdf540aa9b0 2023-11-27T15:16:43.704 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.700+0000 7fdf61ffb700 1 -- 172.21.15.52:0/2857933815 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf500d12c0 con 0x7fdf640ffe00 2023-11-27T15:16:43.705 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.700+0000 7fdf68bda700 1 --2- 172.21.15.52:0/2857933815 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fdf540a11e0 0x7fdf540a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:43.705 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.701+0000 7fdf68bda700 1 --2- 172.21.15.52:0/2857933815 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fdf540a11e0 0x7fdf540a3690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdf580099d0 tx=0x7fdf58008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:43.705 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.702+0000 7fdf693db700 1 --2- 172.21.15.52:0/2857933815 >> [v2:172.21.15.52:6826/4287934320,v1:172.21.15.52:6827/4287934320] conn(0x7fdf540aa9b0 0x7fdf540acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:43.706 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.702+0000 7fdf693db700 1 --2- 172.21.15.52:0/2857933815 >> [v2:172.21.15.52:6826/4287934320,v1:172.21.15.52:6827/4287934320] conn(0x7fdf540aa9b0 0x7fdf540acdc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:43.707 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.703+0000 7fdf61ffb700 1 -- 172.21.15.52:0/2857933815 <== osd.9 v2:172.21.15.52:6826/4287934320 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fdf540ad470 con 0x7fdf540aa9b0 2023-11-27T15:16:43.728 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.723+0000 7fdf6ae3e700 1 -- 172.21.15.52:0/2857933815 --> [v2:172.21.15.52:6826/4287934320,v1:172.21.15.52:6827/4287934320] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdf6410f450 con 0x7fdf540aa9b0 2023-11-27T15:16:43.728 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.724+0000 7fdf61ffb700 1 -- 172.21.15.52:0/2857933815 <== osd.9 v2:172.21.15.52:6826/4287934320 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fdf6410f450 con 0x7fdf540aa9b0 2023-11-27T15:16:43.729 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.724+0000 7fdf6ae3e700 1 -- 172.21.15.52:0/2857933815 >> [v2:172.21.15.52:6826/4287934320,v1:172.21.15.52:6827/4287934320] conn(0x7fdf540aa9b0 msgr2=0x7fdf540acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:43.729 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.724+0000 7fdf6ae3e700 1 --2- 172.21.15.52:0/2857933815 >> [v2:172.21.15.52:6826/4287934320,v1:172.21.15.52:6827/4287934320] conn(0x7fdf540aa9b0 0x7fdf540acdc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:43.729 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.724+0000 7fdf6ae3e700 1 -- 172.21.15.52:0/2857933815 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fdf540a11e0 msgr2=0x7fdf540a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:43.729 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.724+0000 7fdf6ae3e700 1 --2- 172.21.15.52:0/2857933815 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fdf540a11e0 0x7fdf540a3690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdf580099d0 tx=0x7fdf58008040 comp rx=0 tx=0).stop 2023-11-27T15:16:43.729 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.725+0000 7fdf6ae3e700 1 -- 172.21.15.52:0/2857933815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf640ffe00 msgr2=0x7fdf64197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:43.729 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.725+0000 7fdf6ae3e700 1 --2- 172.21.15.52:0/2857933815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf640ffe00 0x7fdf64197960 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000e970 tx=0x7fdf5000ed30 comp rx=0 tx=0).stop 2023-11-27T15:16:43.729 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.725+0000 7fdf693db700 1 -- 172.21.15.52:0/2857933815 reap_dead start 2023-11-27T15:16:43.730 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.725+0000 7fdf6ae3e700 1 -- 172.21.15.52:0/2857933815 shutdown_connections 2023-11-27T15:16:43.730 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.725+0000 7fdf6ae3e700 1 -- 172.21.15.52:0/2857933815 >> 172.21.15.52:0/2857933815 conn(0x7fdf640770a0 msgr2=0x7fdf640fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:43.730 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.725+0000 7fdf6ae3e700 1 -- 172.21.15.52:0/2857933815 shutdown_connections 2023-11-27T15:16:43.730 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:43.725+0000 7fdf6ae3e700 1 -- 172.21.15.52:0/2857933815 wait complete. 2023-11-27T15:16:43.866 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:43 smithi052 ceph-mon[99056]: pgmap v273: 97 pgs: 97 active+clean; 637 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2023-11-27T15:16:44.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:43 smithi102 ceph-mon[100296]: pgmap v273: 97 pgs: 97 active+clean; 637 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2023-11-27T15:16:44.148 INFO:teuthology.orchestra.run.smithi052.stdout:77309411385 2023-11-27T15:16:44.149 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.2 2023-11-27T15:16:44.250 INFO:teuthology.orchestra.run.smithi052.stdout:283467841563 2023-11-27T15:16:44.250 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.9 2023-11-27T15:16:44.256 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:44.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:43 smithi150 ceph-mon[100559]: pgmap v273: 97 pgs: 97 active+clean; 637 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2023-11-27T15:16:44.291 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:44.384 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:44.385 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:44.529 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:44.530 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:45.193 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:45.194 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:45.194 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:45.889 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:45 smithi052 ceph-mon[99056]: pgmap v274: 97 pgs: 97 active+clean; 637 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2023-11-27T15:16:46.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:45 smithi102 ceph-mon[100296]: pgmap v274: 97 pgs: 97 active+clean; 637 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2023-11-27T15:16:46.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:45 smithi150 ceph-mon[100559]: pgmap v274: 97 pgs: 97 active+clean; 637 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2023-11-27T15:16:46.694 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:48.057 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:47 smithi052 ceph-mon[99056]: pgmap v275: 97 pgs: 97 active+clean; 637 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2023-11-27T15:16:48.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:47 smithi102 ceph-mon[100296]: pgmap v275: 97 pgs: 97 active+clean; 637 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2023-11-27T15:16:48.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:47 smithi150 ceph-mon[100559]: pgmap v275: 97 pgs: 97 active+clean; 637 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2023-11-27T15:16:48.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.489+0000 7f5d8103e700 1 -- 172.21.15.52:0/878948947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d7c1055e0 msgr2=0x7f5d7c10d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:48.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.489+0000 7f5d8103e700 1 --2- 172.21.15.52:0/878948947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d7c1055e0 0x7f5d7c10d160 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5d7400b0d0 tx=0x7f5d7400b3e0 comp rx=0 tx=0).stop 2023-11-27T15:16:48.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.489+0000 7f5d8103e700 1 -- 172.21.15.52:0/878948947 shutdown_connections 2023-11-27T15:16:48.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.489+0000 7f5d8103e700 1 --2- 172.21.15.52:0/878948947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d7c1055e0 0x7f5d7c10d160 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:48.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.489+0000 7f5d8103e700 1 --2- 172.21.15.52:0/878948947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d7c100540 0x7f5d7c100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:48.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.489+0000 7f5d8103e700 1 --2- 172.21.15.52:0/878948947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d7c100ee0 0x7f5d7c104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:48.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.489+0000 7f5d8103e700 1 -- 172.21.15.52:0/878948947 >> 172.21.15.52:0/878948947 conn(0x7f5d7c0fbdc0 msgr2=0x7f5d7c0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:48.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.489+0000 7f5d8103e700 1 -- 172.21.15.52:0/878948947 shutdown_connections 2023-11-27T15:16:48.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.489+0000 7f5d8103e700 1 -- 172.21.15.52:0/878948947 wait complete. 2023-11-27T15:16:48.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.490+0000 7f5d8103e700 1 Processor -- start 2023-11-27T15:16:48.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.490+0000 7f5d8103e700 1 -- start start 2023-11-27T15:16:48.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.490+0000 7f5d8103e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d7c100540 0x7f5d7c072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:48.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.490+0000 7f5d8103e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d7c100ee0 0x7f5d7c073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:48.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.490+0000 7f5d8103e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d7c1055e0 0x7f5d7c1a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:48.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.490+0000 7f5d8103e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c073c20 con 0x7f5d7c100ee0 2023-11-27T15:16:48.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.490+0000 7f5d8103e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c1a4c10 con 0x7f5d7c100540 2023-11-27T15:16:48.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.490+0000 7f5d8103e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c1a4d80 con 0x7f5d7c1055e0 2023-11-27T15:16:48.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.491+0000 7f5d7a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d7c100ee0 0x7f5d7c073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:48.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.491+0000 7f5d7a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d7c100ee0 0x7f5d7c073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50478/0 (socket says 172.21.15.52:50478) 2023-11-27T15:16:48.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.491+0000 7f5d7a59c700 1 -- 172.21.15.52:0/1626264145 learned_addr learned my addr 172.21.15.52:0/1626264145 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:48.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.491+0000 7f5d7ad9d700 1 --2- 172.21.15.52:0/1626264145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d7c100540 0x7f5d7c072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:48.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.491+0000 7f5d7a59c700 1 -- 172.21.15.52:0/1626264145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d7c1055e0 msgr2=0x7f5d7c1a46d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-27T15:16:48.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.491+0000 7f5d7a59c700 1 --2- 172.21.15.52:0/1626264145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d7c1055e0 0x7f5d7c1a46d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:48.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.491+0000 7f5d7a59c700 1 -- 172.21.15.52:0/1626264145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d7c100540 msgr2=0x7f5d7c072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:48.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.491+0000 7f5d7a59c700 1 --2- 172.21.15.52:0/1626264145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d7c100540 0x7f5d7c072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:48.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.491+0000 7f5d7a59c700 1 -- 172.21.15.52:0/1626264145 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d74009d00 con 0x7f5d7c100ee0 2023-11-27T15:16:48.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.491+0000 7f5d7a59c700 1 --2- 172.21.15.52:0/1626264145 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d7c100ee0 0x7f5d7c073330 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000d980 tx=0x7f5d7000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:48.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.491+0000 7f5d63fff700 1 -- 172.21.15.52:0/1626264145 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70009930 con 0x7f5d7c100ee0 2023-11-27T15:16:48.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.491+0000 7f5d8103e700 1 -- 172.21.15.52:0/1626264145 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d7c110320 con 0x7f5d7c100ee0 2023-11-27T15:16:48.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.491+0000 7f5d63fff700 1 -- 172.21.15.52:0/1626264145 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d70010460 con 0x7f5d7c100ee0 2023-11-27T15:16:48.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.491+0000 7f5d63fff700 1 -- 172.21.15.52:0/1626264145 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7000f620 con 0x7f5d7c100ee0 2023-11-27T15:16:48.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.492+0000 7f5d8103e700 1 -- 172.21.15.52:0/1626264145 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d7c110190 con 0x7f5d7c100ee0 2023-11-27T15:16:48.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.492+0000 7f5d8103e700 1 -- 172.21.15.52:0/1626264145 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d7c1104b0 con 0x7f5d7c100ee0 2023-11-27T15:16:48.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.495+0000 7f5d63fff700 1 -- 172.21.15.52:0/1626264145 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f5d700108d0 con 0x7f5d7c100ee0 2023-11-27T15:16:48.501 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.496+0000 7f5d63fff700 1 --2- 172.21.15.52:0/1626264145 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f5d640a11e0 0x7f5d640a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:48.501 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.496+0000 7f5d7ad9d700 1 --2- 172.21.15.52:0/1626264145 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f5d640a11e0 0x7f5d640a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:48.501 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.496+0000 7f5d63fff700 1 -- 172.21.15.52:0/1626264145 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5d700cc2c0 con 0x7f5d7c100ee0 2023-11-27T15:16:48.501 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.496+0000 7f5d7ad9d700 1 --2- 172.21.15.52:0/1626264145 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f5d640a11e0 0x7f5d640a3690 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c005950 tx=0x7f5d6c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:48.504 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.499+0000 7f5d63fff700 1 -- 172.21.15.52:0/1626264145 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5d70095a60 con 0x7f5d7c100ee0 2023-11-27T15:16:48.724 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.719+0000 7f5d8103e700 1 -- 172.21.15.52:0/1626264145 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f5d7c066ea0 con 0x7f5d7c100ee0 2023-11-27T15:16:48.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.719+0000 7f5d63fff700 1 -- 172.21.15.52:0/1626264145 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5d70017020 con 0x7f5d7c100ee0 2023-11-27T15:16:48.725 INFO:teuthology.orchestra.run.smithi052.stdout:141733920818 2023-11-27T15:16:48.726 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.721+0000 7f5d8103e700 1 -- 172.21.15.52:0/1626264145 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f5d640a11e0 msgr2=0x7f5d640a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:48.726 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.721+0000 7f5d8103e700 1 --2- 172.21.15.52:0/1626264145 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f5d640a11e0 0x7f5d640a3690 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c005950 tx=0x7f5d6c00a300 comp rx=0 tx=0).stop 2023-11-27T15:16:48.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.721+0000 7f5d8103e700 1 -- 172.21.15.52:0/1626264145 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d7c100ee0 msgr2=0x7f5d7c073330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:48.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.721+0000 7f5d8103e700 1 --2- 172.21.15.52:0/1626264145 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d7c100ee0 0x7f5d7c073330 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000d980 tx=0x7f5d7000dc90 comp rx=0 tx=0).stop 2023-11-27T15:16:48.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.721+0000 7f5d8103e700 1 -- 172.21.15.52:0/1626264145 shutdown_connections 2023-11-27T15:16:48.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.721+0000 7f5d8103e700 1 --2- 172.21.15.52:0/1626264145 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f5d640a11e0 0x7f5d640a3690 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:48.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.721+0000 7f5d8103e700 1 --2- 172.21.15.52:0/1626264145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d7c100540 0x7f5d7c072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:48.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.721+0000 7f5d8103e700 1 --2- 172.21.15.52:0/1626264145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d7c1055e0 0x7f5d7c1a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:48.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.721+0000 7f5d8103e700 1 --2- 172.21.15.52:0/1626264145 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d7c100ee0 0x7f5d7c073330 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:48.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.721+0000 7f5d8103e700 1 -- 172.21.15.52:0/1626264145 >> 172.21.15.52:0/1626264145 conn(0x7f5d7c0fbdc0 msgr2=0x7f5d7c0fc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:48.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.721+0000 7f5d8103e700 1 -- 172.21.15.52:0/1626264145 shutdown_connections 2023-11-27T15:16:48.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:48.721+0000 7f5d8103e700 1 -- 172.21.15.52:0/1626264145 wait complete. 2023-11-27T15:16:49.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:48 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1626264145' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-11-27T15:16:49.094 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:48 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1626264145' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-11-27T15:16:49.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:48 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1626264145' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-11-27T15:16:49.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.610+0000 7f2d6f295700 1 -- 172.21.15.52:0/2909047498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d680ffe00 msgr2=0x7f2d68103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:49.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.610+0000 7f2d6f295700 1 --2- 172.21.15.52:0/2909047498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d680ffe00 0x7f2d68103dd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2d640099c0 tx=0x7f2d64009cd0 comp rx=0 tx=0).stop 2023-11-27T15:16:49.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.610+0000 7f2d6f295700 1 -- 172.21.15.52:0/2909047498 shutdown_connections 2023-11-27T15:16:49.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.610+0000 7f2d6f295700 1 --2- 172.21.15.52:0/2909047498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d68104500 0x7f2d6810c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:49.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.610+0000 7f2d6f295700 1 --2- 172.21.15.52:0/2909047498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d680ff460 0x7f2d680ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:49.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.610+0000 7f2d6f295700 1 --2- 172.21.15.52:0/2909047498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d680ffe00 0x7f2d68103dd0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:49.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.610+0000 7f2d6f295700 1 -- 172.21.15.52:0/2909047498 >> 172.21.15.52:0/2909047498 conn(0x7f2d680770a0 msgr2=0x7f2d680774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:49.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.610+0000 7f2d6f295700 1 -- 172.21.15.52:0/2909047498 shutdown_connections 2023-11-27T15:16:49.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.610+0000 7f2d6f295700 1 -- 172.21.15.52:0/2909047498 wait complete. 2023-11-27T15:16:49.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.611+0000 7f2d6f295700 1 Processor -- start 2023-11-27T15:16:49.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.611+0000 7f2d6f295700 1 -- start start 2023-11-27T15:16:49.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.611+0000 7f2d6f295700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d680ff460 0x7f2d68196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:49.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.611+0000 7f2d6f295700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d680ffe00 0x7f2d681969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:49.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.611+0000 7f2d6f295700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d68104500 0x7f2d6819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:49.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.611+0000 7f2d6f295700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6819b420 con 0x7f2d680ff460 2023-11-27T15:16:49.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.611+0000 7f2d6f295700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6819b590 con 0x7f2d680ffe00 2023-11-27T15:16:49.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.611+0000 7f2d6f295700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6819b750 con 0x7f2d68104500 2023-11-27T15:16:49.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.611+0000 7f2d6d031700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d680ff460 0x7f2d68196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:49.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.611+0000 7f2d6d031700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d680ff460 0x7f2d68196460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50488/0 (socket says 172.21.15.52:50488) 2023-11-27T15:16:49.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.611+0000 7f2d6d031700 1 -- 172.21.15.52:0/1589415594 learned_addr learned my addr 172.21.15.52:0/1589415594 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:49.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.611+0000 7f2d6c830700 1 --2- 172.21.15.52:0/1589415594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d680ffe00 0x7f2d681969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:49.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.611+0000 7f2d6d832700 1 --2- 172.21.15.52:0/1589415594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d68104500 0x7f2d6819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:49.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.612+0000 7f2d6d031700 1 -- 172.21.15.52:0/1589415594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d68104500 msgr2=0x7f2d6819ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:49.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.612+0000 7f2d6d031700 1 --2- 172.21.15.52:0/1589415594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d68104500 0x7f2d6819ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:49.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.612+0000 7f2d6d031700 1 -- 172.21.15.52:0/1589415594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d680ffe00 msgr2=0x7f2d681969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:49.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.612+0000 7f2d6d031700 1 --2- 172.21.15.52:0/1589415594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d680ffe00 0x7f2d681969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:49.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.612+0000 7f2d6d031700 1 -- 172.21.15.52:0/1589415594 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d640095f0 con 0x7f2d680ff460 2023-11-27T15:16:49.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.612+0000 7f2d6d031700 1 --2- 172.21.15.52:0/1589415594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d680ff460 0x7f2d68196460 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800b530 tx=0x7f2d5800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:49.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.612+0000 7f2d5e7fc700 1 -- 172.21.15.52:0/1589415594 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d58011830 con 0x7f2d680ff460 2023-11-27T15:16:49.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.612+0000 7f2d6f295700 1 -- 172.21.15.52:0/1589415594 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d6810f290 con 0x7f2d680ff460 2023-11-27T15:16:49.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.612+0000 7f2d5e7fc700 1 -- 172.21.15.52:0/1589415594 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d58011e50 con 0x7f2d680ff460 2023-11-27T15:16:49.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.612+0000 7f2d5e7fc700 1 -- 172.21.15.52:0/1589415594 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5800f2e0 con 0x7f2d680ff460 2023-11-27T15:16:49.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.612+0000 7f2d6f295700 1 -- 172.21.15.52:0/1589415594 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6810f100 con 0x7f2d680ff460 2023-11-27T15:16:49.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.613+0000 7f2d6f295700 1 -- 172.21.15.52:0/1589415594 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d6810f420 con 0x7f2d680ff460 2023-11-27T15:16:49.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.614+0000 7f2d5e7fc700 1 -- 172.21.15.52:0/1589415594 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f2d580105a0 con 0x7f2d680ff460 2023-11-27T15:16:49.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.615+0000 7f2d5e7fc700 1 --2- 172.21.15.52:0/1589415594 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f2d540a11e0 0x7f2d540a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:49.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.615+0000 7f2d6c830700 1 --2- 172.21.15.52:0/1589415594 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f2d540a11e0 0x7f2d540a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:49.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.615+0000 7f2d5e7fc700 1 -- 172.21.15.52:0/1589415594 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2d580cc180 con 0x7f2d680ff460 2023-11-27T15:16:49.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.615+0000 7f2d6c830700 1 --2- 172.21.15.52:0/1589415594 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f2d540a11e0 0x7f2d540a3690 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2d64009fd0 tx=0x7f2d64005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:49.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.619+0000 7f2d5e7fc700 1 -- 172.21.15.52:0/1589415594 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2d58095a30 con 0x7f2d680ff460 2023-11-27T15:16:49.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.836+0000 7f2d6f295700 1 -- 172.21.15.52:0/1589415594 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f2d68066ea0 con 0x7f2d680ff460 2023-11-27T15:16:49.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.836+0000 7f2d5e7fc700 1 -- 172.21.15.52:0/1589415594 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2d58099050 con 0x7f2d680ff460 2023-11-27T15:16:49.842 INFO:teuthology.orchestra.run.smithi052.stdout:214748364836 2023-11-27T15:16:49.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.838+0000 7f2d6f295700 1 -- 172.21.15.52:0/1589415594 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f2d540a11e0 msgr2=0x7f2d540a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:49.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.838+0000 7f2d6f295700 1 --2- 172.21.15.52:0/1589415594 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f2d540a11e0 0x7f2d540a3690 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2d64009fd0 tx=0x7f2d64005fb0 comp rx=0 tx=0).stop 2023-11-27T15:16:49.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.838+0000 7f2d6f295700 1 -- 172.21.15.52:0/1589415594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d680ff460 msgr2=0x7f2d68196460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:49.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.838+0000 7f2d6f295700 1 --2- 172.21.15.52:0/1589415594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d680ff460 0x7f2d68196460 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800b530 tx=0x7f2d5800b840 comp rx=0 tx=0).stop 2023-11-27T15:16:49.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.839+0000 7f2d6f295700 1 -- 172.21.15.52:0/1589415594 shutdown_connections 2023-11-27T15:16:49.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.839+0000 7f2d6f295700 1 --2- 172.21.15.52:0/1589415594 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f2d540a11e0 0x7f2d540a3690 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:49.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.839+0000 7f2d6f295700 1 --2- 172.21.15.52:0/1589415594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d68104500 0x7f2d6819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:49.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.839+0000 7f2d6f295700 1 --2- 172.21.15.52:0/1589415594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d680ff460 0x7f2d68196460 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:49.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.839+0000 7f2d6f295700 1 --2- 172.21.15.52:0/1589415594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d680ffe00 0x7f2d681969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:49.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.839+0000 7f2d6f295700 1 -- 172.21.15.52:0/1589415594 >> 172.21.15.52:0/1589415594 conn(0x7f2d680770a0 msgr2=0x7f2d68102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:49.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.839+0000 7f2d6f295700 1 -- 172.21.15.52:0/1589415594 shutdown_connections 2023-11-27T15:16:49.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:49.839+0000 7f2d6f295700 1 -- 172.21.15.52:0/1589415594 wait complete. 2023-11-27T15:16:50.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:49 smithi102 ceph-mon[100296]: pgmap v276: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 7 op/s 2023-11-27T15:16:50.094 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:49 smithi052 ceph-mon[99056]: pgmap v276: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 7 op/s 2023-11-27T15:16:50.120 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920815 got 141733920818 for osd.4 2023-11-27T15:16:50.121 DEBUG:teuthology.parallel:result is None 2023-11-27T15:16:50.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.165+0000 7f92d3c79700 1 -- 172.21.15.52:0/3388829960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92cc104510 msgr2=0x7f92cc10c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:50.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.165+0000 7f92d3c79700 1 --2- 172.21.15.52:0/3388829960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92cc104510 0x7f92cc10c090 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f92c400b0d0 tx=0x7f92c400b3e0 comp rx=0 tx=0).stop 2023-11-27T15:16:50.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.165+0000 7f92d3c79700 1 -- 172.21.15.52:0/3388829960 shutdown_connections 2023-11-27T15:16:50.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.165+0000 7f92d3c79700 1 --2- 172.21.15.52:0/3388829960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92cc104510 0x7f92cc10c090 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:50.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.165+0000 7f92d3c79700 1 --2- 172.21.15.52:0/3388829960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92cc0ff470 0x7f92cc0ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:50.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.165+0000 7f92d3c79700 1 --2- 172.21.15.52:0/3388829960 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92cc0ffe10 0x7f92cc103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:50.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.165+0000 7f92d3c79700 1 -- 172.21.15.52:0/3388829960 >> 172.21.15.52:0/3388829960 conn(0x7f92cc0770d0 msgr2=0x7f92cc0774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:50.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.165+0000 7f92d3c79700 1 -- 172.21.15.52:0/3388829960 shutdown_connections 2023-11-27T15:16:50.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.165+0000 7f92d3c79700 1 -- 172.21.15.52:0/3388829960 wait complete. 2023-11-27T15:16:50.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.166+0000 7f92d3c79700 1 Processor -- start 2023-11-27T15:16:50.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.166+0000 7f92d3c79700 1 -- start start 2023-11-27T15:16:50.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.166+0000 7f92d3c79700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92cc0ff470 0x7f92cc1953f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:50.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.166+0000 7f92d3c79700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92cc0ffe10 0x7f92cc195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:50.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.166+0000 7f92d3c79700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92cc104510 0x7f92cc199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:50.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.166+0000 7f92d3c79700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc19a3b0 con 0x7f92cc0ff470 2023-11-27T15:16:50.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.166+0000 7f92d3c79700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc19a520 con 0x7f92cc0ffe10 2023-11-27T15:16:50.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.166+0000 7f92d3c79700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc19a6e0 con 0x7f92cc104510 2023-11-27T15:16:50.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.167+0000 7f92d1a15700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92cc0ff470 0x7f92cc1953f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:50.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.167+0000 7f92d1a15700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92cc0ff470 0x7f92cc1953f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50502/0 (socket says 172.21.15.52:50502) 2023-11-27T15:16:50.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.167+0000 7f92d1a15700 1 -- 172.21.15.52:0/3174674516 learned_addr learned my addr 172.21.15.52:0/3174674516 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:50.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.167+0000 7f92d1214700 1 --2- 172.21.15.52:0/3174674516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92cc0ffe10 0x7f92cc195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:50.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.167+0000 7f92d2216700 1 --2- 172.21.15.52:0/3174674516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92cc104510 0x7f92cc199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:50.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.167+0000 7f92d1a15700 1 -- 172.21.15.52:0/3174674516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92cc104510 msgr2=0x7f92cc199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:50.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.167+0000 7f92d1a15700 1 --2- 172.21.15.52:0/3174674516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92cc104510 0x7f92cc199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:50.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.167+0000 7f92d1a15700 1 -- 172.21.15.52:0/3174674516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92cc0ffe10 msgr2=0x7f92cc195930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:50.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.167+0000 7f92d1a15700 1 --2- 172.21.15.52:0/3174674516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92cc0ffe10 0x7f92cc195930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:50.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.167+0000 7f92d1a15700 1 -- 172.21.15.52:0/3174674516 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c4009d00 con 0x7f92cc0ff470 2023-11-27T15:16:50.175 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.167+0000 7f92d2216700 1 --2- 172.21.15.52:0/3174674516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92cc104510 0x7f92cc199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:16:50.175 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.167+0000 7f92d1a15700 1 --2- 172.21.15.52:0/3174674516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92cc0ff470 0x7f92cc1953f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f92c80098f0 tx=0x7f92c8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:50.175 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.167+0000 7f92c2ffd700 1 -- 172.21.15.52:0/3174674516 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c8012070 con 0x7f92cc0ff470 2023-11-27T15:16:50.175 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.167+0000 7f92d3c79700 1 -- 172.21.15.52:0/3174674516 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92cc10f270 con 0x7f92cc0ff470 2023-11-27T15:16:50.175 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.167+0000 7f92c2ffd700 1 -- 172.21.15.52:0/3174674516 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92c8004cf0 con 0x7f92cc0ff470 2023-11-27T15:16:50.175 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.167+0000 7f92c2ffd700 1 -- 172.21.15.52:0/3174674516 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c80055f0 con 0x7f92cc0ff470 2023-11-27T15:16:50.175 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.167+0000 7f92d3c79700 1 -- 172.21.15.52:0/3174674516 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92cc10f0e0 con 0x7f92cc0ff470 2023-11-27T15:16:50.176 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.168+0000 7f92d3c79700 1 -- 172.21.15.52:0/3174674516 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92cc10f400 con 0x7f92cc0ff470 2023-11-27T15:16:50.176 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.170+0000 7f92c2ffd700 1 -- 172.21.15.52:0/3174674516 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f92c8016070 con 0x7f92cc0ff470 2023-11-27T15:16:50.176 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.170+0000 7f92c2ffd700 1 --2- 172.21.15.52:0/3174674516 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f92b80a55f0 0x7f92b80a7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:50.176 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.170+0000 7f92d1214700 1 --2- 172.21.15.52:0/3174674516 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f92b80a55f0 0x7f92b80a7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:50.176 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.170+0000 7f92c2ffd700 1 -- 172.21.15.52:0/3174674516 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f92c80cbfb0 con 0x7f92cc0ff470 2023-11-27T15:16:50.176 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.171+0000 7f92d1214700 1 --2- 172.21.15.52:0/3174674516 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f92b80a55f0 0x7f92b80a7aa0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f92cc196a10 tx=0x7f92bc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:50.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.174+0000 7f92c2ffd700 1 -- 172.21.15.52:0/3174674516 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f92c8095750 con 0x7f92cc0ff470 2023-11-27T15:16:50.186 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:50.187 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:50.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:49 smithi150 ceph-mon[100559]: pgmap v276: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 7 op/s 2023-11-27T15:16:50.410 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.404+0000 7f92d3c79700 1 -- 172.21.15.52:0/3174674516 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f92cc196870 con 0x7f92cc0ff470 2023-11-27T15:16:50.410 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.405+0000 7f92c2ffd700 1 -- 172.21.15.52:0/3174674516 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f92cc196870 con 0x7f92cc0ff470 2023-11-27T15:16:50.410 INFO:teuthology.orchestra.run.smithi052.stdout:163208757294 2023-11-27T15:16:50.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.407+0000 7f92d3c79700 1 -- 172.21.15.52:0/3174674516 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f92b80a55f0 msgr2=0x7f92b80a7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:50.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.407+0000 7f92d3c79700 1 --2- 172.21.15.52:0/3174674516 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f92b80a55f0 0x7f92b80a7aa0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f92cc196a10 tx=0x7f92bc00b410 comp rx=0 tx=0).stop 2023-11-27T15:16:50.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.407+0000 7f92d3c79700 1 -- 172.21.15.52:0/3174674516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92cc0ff470 msgr2=0x7f92cc1953f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:50.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.407+0000 7f92d3c79700 1 --2- 172.21.15.52:0/3174674516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92cc0ff470 0x7f92cc1953f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f92c80098f0 tx=0x7f92c8009c00 comp rx=0 tx=0).stop 2023-11-27T15:16:50.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.407+0000 7f92d3c79700 1 -- 172.21.15.52:0/3174674516 shutdown_connections 2023-11-27T15:16:50.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.407+0000 7f92d3c79700 1 --2- 172.21.15.52:0/3174674516 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f92b80a55f0 0x7f92b80a7aa0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:50.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.407+0000 7f92d3c79700 1 --2- 172.21.15.52:0/3174674516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92cc104510 0x7f92cc199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:50.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.407+0000 7f92d3c79700 1 --2- 172.21.15.52:0/3174674516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92cc0ff470 0x7f92cc1953f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:50.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.407+0000 7f92d3c79700 1 --2- 172.21.15.52:0/3174674516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92cc0ffe10 0x7f92cc195930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:50.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.407+0000 7f92d3c79700 1 -- 172.21.15.52:0/3174674516 >> 172.21.15.52:0/3174674516 conn(0x7f92cc0770d0 msgr2=0x7f92cc0fcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:50.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.407+0000 7f92d3c79700 1 -- 172.21.15.52:0/3174674516 shutdown_connections 2023-11-27T15:16:50.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:50.407+0000 7f92d3c79700 1 -- 172.21.15.52:0/3174674516 wait complete. 2023-11-27T15:16:51.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:50 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1589415594' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-11-27T15:16:51.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:50 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3174674516' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-11-27T15:16:51.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:50 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1589415594' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-11-27T15:16:51.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:50 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3174674516' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-11-27T15:16:51.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:50 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1589415594' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-11-27T15:16:51.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:50 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3174674516' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-11-27T15:16:51.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.465+0000 7fd560351700 1 -- 172.21.15.52:0/2687327852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5581013b0 msgr2=0x7fd558101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:51.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.465+0000 7fd560351700 1 --2- 172.21.15.52:0/2687327852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5581013b0 0x7fd558101780 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd554009990 tx=0x7fd554009ca0 comp rx=0 tx=0).stop 2023-11-27T15:16:51.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.465+0000 7fd560351700 1 -- 172.21.15.52:0/2687327852 shutdown_connections 2023-11-27T15:16:51.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.465+0000 7fd560351700 1 --2- 172.21.15.52:0/2687327852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd55810a8d0 0x7fd55810ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:51.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.465+0000 7fd560351700 1 --2- 172.21.15.52:0/2687327852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5581013b0 0x7fd558101780 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:51.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.465+0000 7fd560351700 1 --2- 172.21.15.52:0/2687327852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd558101cc0 0x7fd55810a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:51.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.465+0000 7fd560351700 1 -- 172.21.15.52:0/2687327852 >> 172.21.15.52:0/2687327852 conn(0x7fd5580770a0 msgr2=0x7fd5580774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:51.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.465+0000 7fd560351700 1 -- 172.21.15.52:0/2687327852 shutdown_connections 2023-11-27T15:16:51.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.465+0000 7fd560351700 1 -- 172.21.15.52:0/2687327852 wait complete. 2023-11-27T15:16:51.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.466+0000 7fd560351700 1 Processor -- start 2023-11-27T15:16:51.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.466+0000 7fd560351700 1 -- start start 2023-11-27T15:16:51.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.466+0000 7fd560351700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5581013b0 0x7fd5580ff1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:51.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.466+0000 7fd560351700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd558101cc0 0x7fd5580ff720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:51.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.466+0000 7fd560351700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd55810a8d0 0x7fd5580ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:51.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.466+0000 7fd560351700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5581a4860 con 0x7fd5581013b0 2023-11-27T15:16:51.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.466+0000 7fd560351700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5581a49d0 con 0x7fd558101cc0 2023-11-27T15:16:51.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.466+0000 7fd560351700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5581a4b90 con 0x7fd55810a8d0 2023-11-27T15:16:51.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.466+0000 7fd55e0ed700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5581013b0 0x7fd5580ff1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:51.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.466+0000 7fd55e0ed700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5581013b0 0x7fd5580ff1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50514/0 (socket says 172.21.15.52:50514) 2023-11-27T15:16:51.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.466+0000 7fd55e0ed700 1 -- 172.21.15.52:0/584281834 learned_addr learned my addr 172.21.15.52:0/584281834 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:51.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.466+0000 7fd55d8ec700 1 --2- 172.21.15.52:0/584281834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd558101cc0 0x7fd5580ff720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:51.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.467+0000 7fd55e8ee700 1 --2- 172.21.15.52:0/584281834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd55810a8d0 0x7fd5580ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:51.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.467+0000 7fd55e0ed700 1 -- 172.21.15.52:0/584281834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd55810a8d0 msgr2=0x7fd5580ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:51.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.467+0000 7fd55e0ed700 1 --2- 172.21.15.52:0/584281834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd55810a8d0 0x7fd5580ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:51.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.467+0000 7fd55e0ed700 1 -- 172.21.15.52:0/584281834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd558101cc0 msgr2=0x7fd5580ff720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:51.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.467+0000 7fd55e0ed700 1 --2- 172.21.15.52:0/584281834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd558101cc0 0x7fd5580ff720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:51.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.467+0000 7fd55e0ed700 1 -- 172.21.15.52:0/584281834 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5540095f0 con 0x7fd5581013b0 2023-11-27T15:16:51.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.467+0000 7fd55e0ed700 1 --2- 172.21.15.52:0/584281834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5581013b0 0x7fd5580ff1e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd5540095c0 tx=0x7fd55400f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:51.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.467+0000 7fd54f7fe700 1 -- 172.21.15.52:0/584281834 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55401d070 con 0x7fd5581013b0 2023-11-27T15:16:51.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.467+0000 7fd54f7fe700 1 -- 172.21.15.52:0/584281834 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd55400fb50 con 0x7fd5581013b0 2023-11-27T15:16:51.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.467+0000 7fd54f7fe700 1 -- 172.21.15.52:0/584281834 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5540173f0 con 0x7fd5581013b0 2023-11-27T15:16:51.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.467+0000 7fd560351700 1 -- 172.21.15.52:0/584281834 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd558110340 con 0x7fd5581013b0 2023-11-27T15:16:51.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.467+0000 7fd560351700 1 -- 172.21.15.52:0/584281834 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5581101b0 con 0x7fd5581013b0 2023-11-27T15:16:51.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.468+0000 7fd560351700 1 -- 172.21.15.52:0/584281834 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5581104d0 con 0x7fd5581013b0 2023-11-27T15:16:51.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.470+0000 7fd54f7fe700 1 -- 172.21.15.52:0/584281834 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fd55400fcc0 con 0x7fd5581013b0 2023-11-27T15:16:51.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.471+0000 7fd54f7fe700 1 --2- 172.21.15.52:0/584281834 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd5440a11e0 0x7fd5440a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:51.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.471+0000 7fd55d8ec700 1 --2- 172.21.15.52:0/584281834 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd5440a11e0 0x7fd5440a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:51.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.471+0000 7fd54f7fe700 1 -- 172.21.15.52:0/584281834 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd5540cdc10 con 0x7fd5581013b0 2023-11-27T15:16:51.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.471+0000 7fd55d8ec700 1 --2- 172.21.15.52:0/584281834 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd5440a11e0 0x7fd5440a3690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd5581009d0 tx=0x7fd54800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:51.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.474+0000 7fd54f7fe700 1 -- 172.21.15.52:0/584281834 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd554097440 con 0x7fd5581013b0 2023-11-27T15:16:51.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.506+0000 7f5614695700 1 -- 172.21.15.52:0/3335474461 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f560c0ff0d0 msgr2=0x7f560c0ff4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:51.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.506+0000 7f5614695700 1 --2- 172.21.15.52:0/3335474461 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f560c0ff0d0 0x7f560c0ff4a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5608009990 tx=0x7f5608009ca0 comp rx=0 tx=0).stop 2023-11-27T15:16:51.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.506+0000 7f5614695700 1 -- 172.21.15.52:0/3335474461 shutdown_connections 2023-11-27T15:16:51.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.506+0000 7f5614695700 1 --2- 172.21.15.52:0/3335474461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f560c10cc80 0x7f560c10f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:51.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.506+0000 7f5614695700 1 --2- 172.21.15.52:0/3335474461 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f560c0ff0d0 0x7f560c0ff4a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:51.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.506+0000 7f5614695700 1 --2- 172.21.15.52:0/3335474461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f560c0ff9e0 0x7f560c10c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:51.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.506+0000 7f5614695700 1 -- 172.21.15.52:0/3335474461 >> 172.21.15.52:0/3335474461 conn(0x7f560c0770a0 msgr2=0x7f560c0774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:51.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.506+0000 7f5614695700 1 -- 172.21.15.52:0/3335474461 shutdown_connections 2023-11-27T15:16:51.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.507+0000 7f5614695700 1 -- 172.21.15.52:0/3335474461 wait complete. 2023-11-27T15:16:51.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.507+0000 7f5614695700 1 Processor -- start 2023-11-27T15:16:51.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.507+0000 7f5614695700 1 -- start start 2023-11-27T15:16:51.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.507+0000 7f5614695700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f560c0ff0d0 0x7f560c198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:51.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.507+0000 7f5614695700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f560c0ff9e0 0x7f560c198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:51.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.507+0000 7f5614695700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f560c10cc80 0x7f560c19bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:51.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.507+0000 7f5614695700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c19c580 con 0x7f560c10cc80 2023-11-27T15:16:51.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.507+0000 7f5614695700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c19c6f0 con 0x7f560c0ff9e0 2023-11-27T15:16:51.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.507+0000 7f5614695700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c19c8b0 con 0x7f560c0ff0d0 2023-11-27T15:16:51.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.507+0000 7f5612c32700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f560c10cc80 0x7f560c19bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:51.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.507+0000 7f5612c32700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f560c10cc80 0x7f560c19bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50532/0 (socket says 172.21.15.52:50532) 2023-11-27T15:16:51.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.507+0000 7f5612c32700 1 -- 172.21.15.52:0/3195396694 learned_addr learned my addr 172.21.15.52:0/3195396694 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:51.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.508+0000 7f5611c30700 1 --2- 172.21.15.52:0/3195396694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f560c0ff9e0 0x7f560c198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:51.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.508+0000 7f5612431700 1 --2- 172.21.15.52:0/3195396694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f560c0ff0d0 0x7f560c198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:51.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.508+0000 7f5612c32700 1 -- 172.21.15.52:0/3195396694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f560c0ff0d0 msgr2=0x7f560c198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:51.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.508+0000 7f5612c32700 1 --2- 172.21.15.52:0/3195396694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f560c0ff0d0 0x7f560c198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:51.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.508+0000 7f5612c32700 1 -- 172.21.15.52:0/3195396694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f560c0ff9e0 msgr2=0x7f560c198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:51.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.508+0000 7f5612c32700 1 --2- 172.21.15.52:0/3195396694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f560c0ff9e0 0x7f560c198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:51.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.508+0000 7f5612c32700 1 -- 172.21.15.52:0/3195396694 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56080095f0 con 0x7f560c10cc80 2023-11-27T15:16:51.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.508+0000 7f5612431700 1 --2- 172.21.15.52:0/3195396694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f560c0ff0d0 0x7f560c198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:16:51.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.508+0000 7f5612c32700 1 --2- 172.21.15.52:0/3195396694 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f560c10cc80 0x7f560c19bf80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5604009fd0 tx=0x7f560400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:51.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.508+0000 7f56037fe700 1 -- 172.21.15.52:0/3195396694 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5604018070 con 0x7f560c10cc80 2023-11-27T15:16:51.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.508+0000 7f56037fe700 1 -- 172.21.15.52:0/3195396694 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f560400f040 con 0x7f560c10cc80 2023-11-27T15:16:51.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.508+0000 7f5614695700 1 -- 172.21.15.52:0/3195396694 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f560c1125b0 con 0x7f560c10cc80 2023-11-27T15:16:51.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.508+0000 7f56037fe700 1 -- 172.21.15.52:0/3195396694 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560400cdb0 con 0x7f560c10cc80 2023-11-27T15:16:51.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.508+0000 7f5614695700 1 -- 172.21.15.52:0/3195396694 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f560c112420 con 0x7f560c10cc80 2023-11-27T15:16:51.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.509+0000 7f5614695700 1 -- 172.21.15.52:0/3195396694 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f560c112740 con 0x7f560c10cc80 2023-11-27T15:16:51.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.510+0000 7f56037fe700 1 -- 172.21.15.52:0/3195396694 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f5604004ad0 con 0x7f560c10cc80 2023-11-27T15:16:51.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.511+0000 7f56037fe700 1 --2- 172.21.15.52:0/3195396694 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f55f80a11e0 0x7f55f80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:51.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.511+0000 7f5612431700 1 --2- 172.21.15.52:0/3195396694 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f55f80a11e0 0x7f55f80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:51.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.511+0000 7f56037fe700 1 -- 172.21.15.52:0/3195396694 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f56040cccb0 con 0x7f560c10cc80 2023-11-27T15:16:51.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.512+0000 7f5612431700 1 --2- 172.21.15.52:0/3195396694 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f55f80a11e0 0x7f55f80a3690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5608000c00 tx=0x7f5608005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:51.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.514+0000 7f56037fe700 1 -- 172.21.15.52:0/3195396694 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f56040964e0 con 0x7f560c10cc80 2023-11-27T15:16:51.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.681+0000 7fd560351700 1 -- 172.21.15.52:0/584281834 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fd558066ea0 con 0x7fd5581013b0 2023-11-27T15:16:51.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.681+0000 7fd54f7fe700 1 -- 172.21.15.52:0/584281834 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fd55409aa60 con 0x7fd5581013b0 2023-11-27T15:16:51.686 INFO:teuthology.orchestra.run.smithi052.stdout:304942678040 2023-11-27T15:16:51.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.683+0000 7fd560351700 1 -- 172.21.15.52:0/584281834 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd5440a11e0 msgr2=0x7fd5440a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:51.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.683+0000 7fd560351700 1 --2- 172.21.15.52:0/584281834 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd5440a11e0 0x7fd5440a3690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd5581009d0 tx=0x7fd54800a300 comp rx=0 tx=0).stop 2023-11-27T15:16:51.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.683+0000 7fd560351700 1 -- 172.21.15.52:0/584281834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5581013b0 msgr2=0x7fd5580ff1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:51.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.683+0000 7fd560351700 1 --2- 172.21.15.52:0/584281834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5581013b0 0x7fd5580ff1e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd5540095c0 tx=0x7fd55400f710 comp rx=0 tx=0).stop 2023-11-27T15:16:51.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.683+0000 7fd560351700 1 -- 172.21.15.52:0/584281834 shutdown_connections 2023-11-27T15:16:51.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.683+0000 7fd560351700 1 --2- 172.21.15.52:0/584281834 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd5440a11e0 0x7fd5440a3690 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:51.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.683+0000 7fd560351700 1 --2- 172.21.15.52:0/584281834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd55810a8d0 0x7fd5580ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:51.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.683+0000 7fd560351700 1 --2- 172.21.15.52:0/584281834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5581013b0 0x7fd5580ff1e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:51.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.683+0000 7fd560351700 1 --2- 172.21.15.52:0/584281834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd558101cc0 0x7fd5580ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:51.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.683+0000 7fd560351700 1 -- 172.21.15.52:0/584281834 >> 172.21.15.52:0/584281834 conn(0x7fd5580770a0 msgr2=0x7fd5580fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:51.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.683+0000 7fd560351700 1 -- 172.21.15.52:0/584281834 shutdown_connections 2023-11-27T15:16:51.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.683+0000 7fd560351700 1 -- 172.21.15.52:0/584281834 wait complete. 2023-11-27T15:16:51.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.722+0000 7f5614695700 1 -- 172.21.15.52:0/3195396694 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f560c066ea0 con 0x7f560c10cc80 2023-11-27T15:16:51.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.722+0000 7f56037fe700 1 -- 172.21.15.52:0/3195396694 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5604099b00 con 0x7f560c10cc80 2023-11-27T15:16:51.728 INFO:teuthology.orchestra.run.smithi052.stdout:34359738435 2023-11-27T15:16:51.730 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.724+0000 7f5614695700 1 -- 172.21.15.52:0/3195396694 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f55f80a11e0 msgr2=0x7f55f80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:51.730 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.724+0000 7f5614695700 1 --2- 172.21.15.52:0/3195396694 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f55f80a11e0 0x7f55f80a3690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5608000c00 tx=0x7f5608005fb0 comp rx=0 tx=0).stop 2023-11-27T15:16:51.730 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.724+0000 7f5614695700 1 -- 172.21.15.52:0/3195396694 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f560c10cc80 msgr2=0x7f560c19bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:51.730 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.724+0000 7f5614695700 1 --2- 172.21.15.52:0/3195396694 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f560c10cc80 0x7f560c19bf80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5604009fd0 tx=0x7f560400ef90 comp rx=0 tx=0).stop 2023-11-27T15:16:51.730 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.724+0000 7f5614695700 1 -- 172.21.15.52:0/3195396694 shutdown_connections 2023-11-27T15:16:51.730 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.724+0000 7f5614695700 1 --2- 172.21.15.52:0/3195396694 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f55f80a11e0 0x7f55f80a3690 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:51.730 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.724+0000 7f5614695700 1 --2- 172.21.15.52:0/3195396694 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f560c10cc80 0x7f560c19bf80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:51.730 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.724+0000 7f5614695700 1 --2- 172.21.15.52:0/3195396694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f560c0ff0d0 0x7f560c198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:51.731 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.724+0000 7f5614695700 1 --2- 172.21.15.52:0/3195396694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f560c0ff9e0 0x7f560c198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:51.731 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.724+0000 7f5614695700 1 -- 172.21.15.52:0/3195396694 >> 172.21.15.52:0/3195396694 conn(0x7f560c0770a0 msgr2=0x7f560c10e120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:51.731 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.724+0000 7f5614695700 1 -- 172.21.15.52:0/3195396694 shutdown_connections 2023-11-27T15:16:51.731 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:51.725+0000 7f5614695700 1 -- 172.21.15.52:0/3195396694 wait complete. 2023-11-27T15:16:51.983 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:51 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/584281834' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-11-27T15:16:51.983 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:51 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3195396694' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-11-27T15:16:51.983 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:51 smithi052 ceph-mon[99056]: pgmap v277: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-11-27T15:16:52.064 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364833 got 214748364836 for osd.7 2023-11-27T15:16:52.064 DEBUG:teuthology.parallel:result is None 2023-11-27T15:16:52.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:51 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/584281834' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-11-27T15:16:52.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:51 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3195396694' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-11-27T15:16:52.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:51 smithi102 ceph-mon[100296]: pgmap v277: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-11-27T15:16:52.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:51 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/584281834' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-11-27T15:16:52.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:51 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3195396694' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-11-27T15:16:52.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:51 smithi150 ceph-mon[100559]: pgmap v277: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-11-27T15:16:52.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.730+0000 7f26feae1700 1 -- 172.21.15.52:0/148532687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26f8101540 msgr2=0x7f26f8101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:52.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.730+0000 7f26feae1700 1 --2- 172.21.15.52:0/148532687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26f8101540 0x7f26f8101910 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f26ec009990 tx=0x7f26ec009ca0 comp rx=0 tx=0).stop 2023-11-27T15:16:52.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.730+0000 7f26feae1700 1 -- 172.21.15.52:0/148532687 shutdown_connections 2023-11-27T15:16:52.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.730+0000 7f26feae1700 1 --2- 172.21.15.52:0/148532687 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26f810a890 0x7f26f810ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:52.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.730+0000 7f26feae1700 1 --2- 172.21.15.52:0/148532687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26f8101540 0x7f26f8101910 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:52.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.730+0000 7f26feae1700 1 --2- 172.21.15.52:0/148532687 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26f8101e50 0x7f26f810a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:52.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.730+0000 7f26feae1700 1 -- 172.21.15.52:0/148532687 >> 172.21.15.52:0/148532687 conn(0x7f26f80770a0 msgr2=0x7f26f80774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:52.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.731+0000 7f26feae1700 1 -- 172.21.15.52:0/148532687 shutdown_connections 2023-11-27T15:16:52.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.731+0000 7f26feae1700 1 -- 172.21.15.52:0/148532687 wait complete. 2023-11-27T15:16:52.739 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.731+0000 7f26feae1700 1 Processor -- start 2023-11-27T15:16:52.739 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.731+0000 7f26feae1700 1 -- start start 2023-11-27T15:16:52.739 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.732+0000 7f26feae1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26f8101540 0x7f26f81963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:52.739 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.732+0000 7f26feae1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26f8101e50 0x7f26f8196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:52.739 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.732+0000 7f26feae1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26f810a890 0x7f26f819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:52.739 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.732+0000 7f26feae1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f819b3b0 con 0x7f26f810a890 2023-11-27T15:16:52.739 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.732+0000 7f26feae1700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f819b520 con 0x7f26f8101540 2023-11-27T15:16:52.739 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.732+0000 7f26feae1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f819b6e0 con 0x7f26f8101e50 2023-11-27T15:16:52.740 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.732+0000 7f26fd07e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26f810a890 0x7f26f819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:52.740 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.732+0000 7f26fd07e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26f810a890 0x7f26f819ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50560/0 (socket says 172.21.15.52:50560) 2023-11-27T15:16:52.740 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.732+0000 7f26fc87d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26f8101540 0x7f26f81963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:52.740 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.732+0000 7f26fd07e700 1 -- 172.21.15.52:0/2756289018 learned_addr learned my addr 172.21.15.52:0/2756289018 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:52.740 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.732+0000 7f26f7fff700 1 --2- 172.21.15.52:0/2756289018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26f8101e50 0x7f26f8196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:52.740 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.732+0000 7f26fc87d700 1 -- 172.21.15.52:0/2756289018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26f8101e50 msgr2=0x7f26f8196930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:52.740 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.732+0000 7f26fc87d700 1 --2- 172.21.15.52:0/2756289018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26f8101e50 0x7f26f8196930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:52.740 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.732+0000 7f26fc87d700 1 -- 172.21.15.52:0/2756289018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26f810a890 msgr2=0x7f26f819ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:52.740 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.732+0000 7f26fc87d700 1 --2- 172.21.15.52:0/2756289018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26f810a890 0x7f26f819ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:52.740 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.732+0000 7f26fc87d700 1 -- 172.21.15.52:0/2756289018 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ec0095f0 con 0x7f26f8101540 2023-11-27T15:16:52.740 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.733+0000 7f26fc87d700 1 --2- 172.21.15.52:0/2756289018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26f8101540 0x7f26f81963f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00b5c0 tx=0x7f26ec00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:52.741 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.733+0000 7f26f5ffb700 1 -- 172.21.15.52:0/2756289018 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec01d070 con 0x7f26f8101540 2023-11-27T15:16:52.741 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.733+0000 7f26feae1700 1 -- 172.21.15.52:0/2756289018 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f8110310 con 0x7f26f8101540 2023-11-27T15:16:52.741 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.733+0000 7f26f5ffb700 1 -- 172.21.15.52:0/2756289018 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26ec022470 con 0x7f26f8101540 2023-11-27T15:16:52.741 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.733+0000 7f26feae1700 1 -- 172.21.15.52:0/2756289018 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f8110180 con 0x7f26f8101540 2023-11-27T15:16:52.741 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.733+0000 7f26f5ffb700 1 -- 172.21.15.52:0/2756289018 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec017650 con 0x7f26f8101540 2023-11-27T15:16:52.742 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.735+0000 7f26feae1700 1 -- 172.21.15.52:0/2756289018 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f81104a0 con 0x7f26f8101540 2023-11-27T15:16:52.742 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.736+0000 7f26f5ffb700 1 -- 172.21.15.52:0/2756289018 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f26ec0227c0 con 0x7f26f8101540 2023-11-27T15:16:52.742 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.736+0000 7f26f5ffb700 1 --2- 172.21.15.52:0/2756289018 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f26e80a11e0 0x7f26e80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:52.742 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.736+0000 7f26f7fff700 1 --2- 172.21.15.52:0/2756289018 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f26e80a11e0 0x7f26e80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:52.742 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.736+0000 7f26f5ffb700 1 -- 172.21.15.52:0/2756289018 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f26ec0cda00 con 0x7f26f8101540 2023-11-27T15:16:52.742 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.737+0000 7f26f7fff700 1 --2- 172.21.15.52:0/2756289018 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f26e80a11e0 0x7f26e80a3690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f26f8197a10 tx=0x7f26e400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:52.746 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.741+0000 7f26f5ffb700 1 -- 172.21.15.52:0/2756289018 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f26ec097230 con 0x7f26f8101540 2023-11-27T15:16:52.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.941+0000 7f26feae1700 1 -- 172.21.15.52:0/2756289018 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f26f81a3470 con 0x7f26f8101540 2023-11-27T15:16:52.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.942+0000 7f26f5ffb700 1 -- 172.21.15.52:0/2756289018 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f26ec09a850 con 0x7f26f8101540 2023-11-27T15:16:52.947 INFO:teuthology.orchestra.run.smithi052.stdout:107374182454 2023-11-27T15:16:52.949 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.944+0000 7f26feae1700 1 -- 172.21.15.52:0/2756289018 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f26e80a11e0 msgr2=0x7f26e80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:52.949 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.944+0000 7f26feae1700 1 --2- 172.21.15.52:0/2756289018 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f26e80a11e0 0x7f26e80a3690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f26f8197a10 tx=0x7f26e400b410 comp rx=0 tx=0).stop 2023-11-27T15:16:52.949 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.944+0000 7f26feae1700 1 -- 172.21.15.52:0/2756289018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26f8101540 msgr2=0x7f26f81963f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:52.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.944+0000 7f26feae1700 1 --2- 172.21.15.52:0/2756289018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26f8101540 0x7f26f81963f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00b5c0 tx=0x7f26ec00f690 comp rx=0 tx=0).stop 2023-11-27T15:16:52.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.944+0000 7f26feae1700 1 -- 172.21.15.52:0/2756289018 shutdown_connections 2023-11-27T15:16:52.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.944+0000 7f26feae1700 1 --2- 172.21.15.52:0/2756289018 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f26e80a11e0 0x7f26e80a3690 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:52.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.944+0000 7f26feae1700 1 --2- 172.21.15.52:0/2756289018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26f8101540 0x7f26f81963f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:52.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.944+0000 7f26feae1700 1 --2- 172.21.15.52:0/2756289018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26f810a890 0x7f26f819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:52.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.944+0000 7f26feae1700 1 --2- 172.21.15.52:0/2756289018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26f8101e50 0x7f26f8196930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:52.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.944+0000 7f26feae1700 1 -- 172.21.15.52:0/2756289018 >> 172.21.15.52:0/2756289018 conn(0x7f26f80770a0 msgr2=0x7f26f80fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:52.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.944+0000 7f26feae1700 1 -- 172.21.15.52:0/2756289018 shutdown_connections 2023-11-27T15:16:52.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.944+0000 7f26feae1700 1 -- 172.21.15.52:0/2756289018 wait complete. 2023-11-27T15:16:52.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:52 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1563599924' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-11-27T15:16:53.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.147+0000 7f32cc72b700 1 -- 172.21.15.52:0/2810319812 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32c4104500 msgr2=0x7f32c410c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:53.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.147+0000 7f32cc72b700 1 --2- 172.21.15.52:0/2810319812 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32c4104500 0x7f32c410c080 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00b0d0 tx=0x7f32bc00b3e0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.147+0000 7f32cc72b700 1 -- 172.21.15.52:0/2810319812 shutdown_connections 2023-11-27T15:16:53.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.147+0000 7f32cc72b700 1 --2- 172.21.15.52:0/2810319812 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32c4104500 0x7f32c410c080 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.147+0000 7f32cc72b700 1 --2- 172.21.15.52:0/2810319812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32c40ff460 0x7f32c40ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.147+0000 7f32cc72b700 1 --2- 172.21.15.52:0/2810319812 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32c40ffe00 0x7f32c4103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.147+0000 7f32cc72b700 1 -- 172.21.15.52:0/2810319812 >> 172.21.15.52:0/2810319812 conn(0x7f32c40770a0 msgr2=0x7f32c40774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:53.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.147+0000 7f32cc72b700 1 -- 172.21.15.52:0/2810319812 shutdown_connections 2023-11-27T15:16:53.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.147+0000 7f32cc72b700 1 -- 172.21.15.52:0/2810319812 wait complete. 2023-11-27T15:16:53.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.148+0000 7f32cc72b700 1 Processor -- start 2023-11-27T15:16:53.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.148+0000 7f32cc72b700 1 -- start start 2023-11-27T15:16:53.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.148+0000 7f32cc72b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32c40ff460 0x7f32c41972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:53.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.148+0000 7f32cc72b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32c40ffe00 0x7f32c4197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:53.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.148+0000 7f32cc72b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32c4104500 0x7f32c41a2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:53.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.148+0000 7f32cc72b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c4198150 con 0x7f32c4104500 2023-11-27T15:16:53.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.148+0000 7f32cc72b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c41a2890 con 0x7f32c40ffe00 2023-11-27T15:16:53.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.148+0000 7f32cc72b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c41a2a50 con 0x7f32c40ff460 2023-11-27T15:16:53.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.148+0000 7f32cacc8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32c4104500 0x7f32c41a2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:53.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.148+0000 7f32cacc8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32c4104500 0x7f32c41a2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50546/0 (socket says 172.21.15.52:50546) 2023-11-27T15:16:53.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.148+0000 7f32cacc8700 1 -- 172.21.15.52:0/1563599924 learned_addr learned my addr 172.21.15.52:0/1563599924 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:53.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.149+0000 7f32ca4c7700 1 --2- 172.21.15.52:0/1563599924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32c40ff460 0x7f32c41972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:53.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.149+0000 7f32c9cc6700 1 --2- 172.21.15.52:0/1563599924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32c40ffe00 0x7f32c4197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:53.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.149+0000 7f32cacc8700 1 -- 172.21.15.52:0/1563599924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32c40ff460 msgr2=0x7f32c41972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:53.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.149+0000 7f32cacc8700 1 --2- 172.21.15.52:0/1563599924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32c40ff460 0x7f32c41972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.149+0000 7f32cacc8700 1 -- 172.21.15.52:0/1563599924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32c40ffe00 msgr2=0x7f32c4197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:53.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.149+0000 7f32cacc8700 1 --2- 172.21.15.52:0/1563599924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32c40ffe00 0x7f32c4197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.149+0000 7f32cacc8700 1 -- 172.21.15.52:0/1563599924 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32bc009d00 con 0x7f32c4104500 2023-11-27T15:16:53.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.149+0000 7f32cacc8700 1 --2- 172.21.15.52:0/1563599924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32c4104500 0x7f32c41a2350 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f32bc007870 tx=0x7f32bc007950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:53.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.149+0000 7f32bb7fe700 1 -- 172.21.15.52:0/1563599924 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32bc00e040 con 0x7f32c4104500 2023-11-27T15:16:53.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.149+0000 7f32cc72b700 1 -- 172.21.15.52:0/1563599924 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c410f240 con 0x7f32c4104500 2023-11-27T15:16:53.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.149+0000 7f32bb7fe700 1 -- 172.21.15.52:0/1563599924 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32bc00be30 con 0x7f32c4104500 2023-11-27T15:16:53.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.149+0000 7f32bb7fe700 1 -- 172.21.15.52:0/1563599924 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32bc01b750 con 0x7f32c4104500 2023-11-27T15:16:53.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.149+0000 7f32cc72b700 1 -- 172.21.15.52:0/1563599924 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c410f0b0 con 0x7f32c4104500 2023-11-27T15:16:53.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.150+0000 7f32cc72b700 1 -- 172.21.15.52:0/1563599924 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c410f3d0 con 0x7f32c4104500 2023-11-27T15:16:53.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.151+0000 7f32bb7fe700 1 -- 172.21.15.52:0/1563599924 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f32bc005090 con 0x7f32c4104500 2023-11-27T15:16:53.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.152+0000 7f32bb7fe700 1 --2- 172.21.15.52:0/1563599924 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f32b00a11e0 0x7f32b00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:53.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.152+0000 7f32ca4c7700 1 --2- 172.21.15.52:0/1563599924 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f32b00a11e0 0x7f32b00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:53.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.152+0000 7f32bb7fe700 1 -- 172.21.15.52:0/1563599924 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f32bc0ce420 con 0x7f32c4104500 2023-11-27T15:16:53.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.152+0000 7f32ca4c7700 1 --2- 172.21.15.52:0/1563599924 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f32b00a11e0 0x7f32b00a3690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f32c0005ea0 tx=0x7f32c0005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:53.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.155+0000 7f32bb7fe700 1 -- 172.21.15.52:0/1563599924 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f32bc097b50 con 0x7f32c4104500 2023-11-27T15:16:53.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.358+0000 7f32cc72b700 1 -- 172.21.15.52:0/1563599924 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f32c4066ea0 con 0x7f32c4104500 2023-11-27T15:16:53.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.359+0000 7f32bb7fe700 1 -- 172.21.15.52:0/1563599924 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f32bc09b170 con 0x7f32c4104500 2023-11-27T15:16:53.020 INFO:teuthology.orchestra.run.smithi052.stdout:249108103201 2023-11-27T15:16:53.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.361+0000 7f32cc72b700 1 -- 172.21.15.52:0/1563599924 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f32b00a11e0 msgr2=0x7f32b00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:53.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.361+0000 7f32cc72b700 1 --2- 172.21.15.52:0/1563599924 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f32b00a11e0 0x7f32b00a3690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f32c0005ea0 tx=0x7f32c0005e30 comp rx=0 tx=0).stop 2023-11-27T15:16:53.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.361+0000 7f32cc72b700 1 -- 172.21.15.52:0/1563599924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32c4104500 msgr2=0x7f32c41a2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:53.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.361+0000 7f32cc72b700 1 --2- 172.21.15.52:0/1563599924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32c4104500 0x7f32c41a2350 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f32bc007870 tx=0x7f32bc007950 comp rx=0 tx=0).stop 2023-11-27T15:16:53.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.361+0000 7f32cc72b700 1 -- 172.21.15.52:0/1563599924 shutdown_connections 2023-11-27T15:16:53.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.361+0000 7f32cc72b700 1 --2- 172.21.15.52:0/1563599924 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f32b00a11e0 0x7f32b00a3690 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.361+0000 7f32cc72b700 1 --2- 172.21.15.52:0/1563599924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32c4104500 0x7f32c41a2350 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.361+0000 7f32cc72b700 1 --2- 172.21.15.52:0/1563599924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32c40ff460 0x7f32c41972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.361+0000 7f32cc72b700 1 --2- 172.21.15.52:0/1563599924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32c40ffe00 0x7f32c4197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.361+0000 7f32cc72b700 1 -- 172.21.15.52:0/1563599924 >> 172.21.15.52:0/1563599924 conn(0x7f32c40770a0 msgr2=0x7f32c40fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:53.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.361+0000 7f32cc72b700 1 -- 172.21.15.52:0/1563599924 shutdown_connections 2023-11-27T15:16:53.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:52.361+0000 7f32cc72b700 1 -- 172.21.15.52:0/1563599924 wait complete. 2023-11-27T15:16:53.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:52 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1563599924' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-11-27T15:16:53.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:52 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1563599924' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-11-27T15:16:53.386 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757292 got 163208757294 for osd.5 2023-11-27T15:16:53.387 DEBUG:teuthology.parallel:result is None 2023-11-27T15:16:53.410 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678038 got 304942678040 for osd.10 2023-11-27T15:16:53.411 DEBUG:teuthology.parallel:result is None 2023-11-27T15:16:53.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.686+0000 7f74b7098700 1 -- 172.21.15.52:0/1723147678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74b00ff1e0 msgr2=0x7f74b010d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:53.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.686+0000 7f74b7098700 1 --2- 172.21.15.52:0/1723147678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74b00ff1e0 0x7f74b010d150 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f74a800b0a0 tx=0x7f74a800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.686+0000 7f74b7098700 1 -- 172.21.15.52:0/1723147678 shutdown_connections 2023-11-27T15:16:53.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.686+0000 7f74b7098700 1 --2- 172.21.15.52:0/1723147678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74b00ff1e0 0x7f74b010d150 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.686+0000 7f74b7098700 1 --2- 172.21.15.52:0/1723147678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74b0105290 0x7f74b0105660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.686+0000 7f74b7098700 1 --2- 172.21.15.52:0/1723147678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74b00fe830 0x7f74b00feca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.686+0000 7f74b7098700 1 -- 172.21.15.52:0/1723147678 >> 172.21.15.52:0/1723147678 conn(0x7f74b00758a0 msgr2=0x7f74b0075ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:53.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.686+0000 7f74b7098700 1 -- 172.21.15.52:0/1723147678 shutdown_connections 2023-11-27T15:16:53.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.686+0000 7f74b7098700 1 -- 172.21.15.52:0/1723147678 wait complete. 2023-11-27T15:16:53.694 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.687+0000 7f74b7098700 1 Processor -- start 2023-11-27T15:16:53.694 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.687+0000 7f74b7098700 1 -- start start 2023-11-27T15:16:53.694 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.687+0000 7f74b7098700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74b00fe830 0x7f74b00ffa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:53.694 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.687+0000 7f74b7098700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74b00ff1e0 0x7f74b00fff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:53.694 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.687+0000 7f74b7098700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74b0105290 0x7f74b01004d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:53.694 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.687+0000 7f74b7098700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b01a4940 con 0x7f74b00ff1e0 2023-11-27T15:16:53.694 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.687+0000 7f74b7098700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b01a4ab0 con 0x7f74b00fe830 2023-11-27T15:16:53.694 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.687+0000 7f74b7098700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b01a4c70 con 0x7f74b0105290 2023-11-27T15:16:53.694 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.687+0000 7f74affff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74b00ff1e0 0x7f74b00fff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:53.695 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.687+0000 7f74affff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74b00ff1e0 0x7f74b00fff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50572/0 (socket says 172.21.15.52:50572) 2023-11-27T15:16:53.695 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.687+0000 7f74affff700 1 -- 172.21.15.52:0/4013131858 learned_addr learned my addr 172.21.15.52:0/4013131858 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:53.695 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.687+0000 7f74b4e34700 1 --2- 172.21.15.52:0/4013131858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74b00fe830 0x7f74b00ffa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:53.695 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.687+0000 7f74b5635700 1 --2- 172.21.15.52:0/4013131858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74b0105290 0x7f74b01004d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:53.695 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.687+0000 7f74affff700 1 -- 172.21.15.52:0/4013131858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74b0105290 msgr2=0x7f74b01004d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:53.695 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.687+0000 7f74affff700 1 --2- 172.21.15.52:0/4013131858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74b0105290 0x7f74b01004d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.695 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.687+0000 7f74affff700 1 -- 172.21.15.52:0/4013131858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74b00fe830 msgr2=0x7f74b00ffa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:53.695 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.687+0000 7f74affff700 1 --2- 172.21.15.52:0/4013131858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74b00fe830 0x7f74b00ffa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.695 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.687+0000 7f74affff700 1 -- 172.21.15.52:0/4013131858 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a8009d00 con 0x7f74b00ff1e0 2023-11-27T15:16:53.695 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.687+0000 7f74b5635700 1 --2- 172.21.15.52:0/4013131858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74b0105290 0x7f74b01004d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:16:53.695 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.688+0000 7f74affff700 1 --2- 172.21.15.52:0/4013131858 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74b00ff1e0 0x7f74b00fff50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f749c00e9a0 tx=0x7f749c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:53.695 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.688+0000 7f74adffb700 1 -- 172.21.15.52:0/4013131858 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c009790 con 0x7f74b00ff1e0 2023-11-27T15:16:53.696 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.688+0000 7f74adffb700 1 -- 172.21.15.52:0/4013131858 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f749c00cbd0 con 0x7f74b00ff1e0 2023-11-27T15:16:53.696 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.688+0000 7f74b7098700 1 -- 172.21.15.52:0/4013131858 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b0110460 con 0x7f74b00ff1e0 2023-11-27T15:16:53.696 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.688+0000 7f74adffb700 1 -- 172.21.15.52:0/4013131858 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c018680 con 0x7f74b00ff1e0 2023-11-27T15:16:53.696 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.688+0000 7f74b7098700 1 -- 172.21.15.52:0/4013131858 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b01102d0 con 0x7f74b00ff1e0 2023-11-27T15:16:53.697 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.689+0000 7f74b7098700 1 -- 172.21.15.52:0/4013131858 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74b01105f0 con 0x7f74b00ff1e0 2023-11-27T15:16:53.697 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.691+0000 7f74adffb700 1 -- 172.21.15.52:0/4013131858 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f749c00cd40 con 0x7f74b00ff1e0 2023-11-27T15:16:53.697 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.691+0000 7f74adffb700 1 --2- 172.21.15.52:0/4013131858 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f74a00a1260 0x7f74a00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:53.697 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.691+0000 7f74adffb700 1 -- 172.21.15.52:0/4013131858 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f749c014070 con 0x7f74b00ff1e0 2023-11-27T15:16:53.697 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.691+0000 7f74b4e34700 1 --2- 172.21.15.52:0/4013131858 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f74a00a1260 0x7f74a00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:53.697 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.692+0000 7f74b4e34700 1 --2- 172.21.15.52:0/4013131858 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f74a00a1260 0x7f74a00a3710 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f74a4005950 tx=0x7f74a400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:53.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.695+0000 7f74adffb700 1 -- 172.21.15.52:0/4013131858 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f749c0964f0 con 0x7f74b00ff1e0 2023-11-27T15:16:53.728 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103198 got 249108103201 for osd.8 2023-11-27T15:16:53.729 DEBUG:teuthology.parallel:result is None 2023-11-27T15:16:53.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.789+0000 7f6ea03c1700 1 -- 172.21.15.52:0/2325213320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e980ff9e0 msgr2=0x7f6e9810c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:53.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.789+0000 7f6ea03c1700 1 --2- 172.21.15.52:0/2325213320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e980ff9e0 0x7f6e9810c740 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6e88009990 tx=0x7f6e88009ca0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.789+0000 7f6ea03c1700 1 -- 172.21.15.52:0/2325213320 shutdown_connections 2023-11-27T15:16:53.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.789+0000 7f6ea03c1700 1 --2- 172.21.15.52:0/2325213320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e980ff0d0 0x7f6e980ff4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.789+0000 7f6ea03c1700 1 --2- 172.21.15.52:0/2325213320 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e9810cc80 0x7f6e9810f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.789+0000 7f6ea03c1700 1 --2- 172.21.15.52:0/2325213320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e980ff9e0 0x7f6e9810c740 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.790+0000 7f6ea03c1700 1 -- 172.21.15.52:0/2325213320 >> 172.21.15.52:0/2325213320 conn(0x7f6e980770a0 msgr2=0x7f6e980774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:53.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.790+0000 7f6ea03c1700 1 -- 172.21.15.52:0/2325213320 shutdown_connections 2023-11-27T15:16:53.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.790+0000 7f6ea03c1700 1 -- 172.21.15.52:0/2325213320 wait complete. 2023-11-27T15:16:53.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.790+0000 7f6ea03c1700 1 Processor -- start 2023-11-27T15:16:53.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.790+0000 7f6ea03c1700 1 -- start start 2023-11-27T15:16:53.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.791+0000 7f6ea03c1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e980ff0d0 0x7f6e98198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:53.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.791+0000 7f6ea03c1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e980ff9e0 0x7f6e98198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:53.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.791+0000 7f6ea03c1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e9810cc80 0x7f6e9819bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:53.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.791+0000 7f6ea03c1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9819c590 con 0x7f6e980ff9e0 2023-11-27T15:16:53.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.791+0000 7f6ea03c1700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9819c700 con 0x7f6e980ff0d0 2023-11-27T15:16:53.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.791+0000 7f6ea03c1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9819c8c0 con 0x7f6e9810cc80 2023-11-27T15:16:53.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.791+0000 7f6e9d95c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e980ff9e0 0x7f6e98198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:53.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.791+0000 7f6e9d95c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e980ff9e0 0x7f6e98198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50586/0 (socket says 172.21.15.52:50586) 2023-11-27T15:16:53.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.791+0000 7f6e9d95c700 1 -- 172.21.15.52:0/3214145371 learned_addr learned my addr 172.21.15.52:0/3214145371 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:53.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.791+0000 7f6e9e15d700 1 --2- 172.21.15.52:0/3214145371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e980ff0d0 0x7f6e98198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:53.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.791+0000 7f6e9e95e700 1 --2- 172.21.15.52:0/3214145371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e9810cc80 0x7f6e9819bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:53.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.791+0000 7f6e9d95c700 1 -- 172.21.15.52:0/3214145371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e9810cc80 msgr2=0x7f6e9819bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:53.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.791+0000 7f6e9d95c700 1 --2- 172.21.15.52:0/3214145371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e9810cc80 0x7f6e9819bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.791+0000 7f6e9d95c700 1 -- 172.21.15.52:0/3214145371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e980ff0d0 msgr2=0x7f6e98198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:53.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.791+0000 7f6e9d95c700 1 --2- 172.21.15.52:0/3214145371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e980ff0d0 0x7f6e98198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.791+0000 7f6e9d95c700 1 -- 172.21.15.52:0/3214145371 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e880095f0 con 0x7f6e980ff9e0 2023-11-27T15:16:53.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.792+0000 7f6e9d95c700 1 --2- 172.21.15.52:0/3214145371 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e980ff9e0 0x7f6e98198bd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800b680 tx=0x7f6e88004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:53.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.792+0000 7f6e8f7fe700 1 -- 172.21.15.52:0/3214145371 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8801d070 con 0x7f6e980ff9e0 2023-11-27T15:16:53.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.792+0000 7f6e8f7fe700 1 -- 172.21.15.52:0/3214145371 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e88005030 con 0x7f6e980ff9e0 2023-11-27T15:16:53.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.792+0000 7f6e8f7fe700 1 -- 172.21.15.52:0/3214145371 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e88004c40 con 0x7f6e980ff9e0 2023-11-27T15:16:53.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.792+0000 7f6ea03c1700 1 -- 172.21.15.52:0/3214145371 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e98112570 con 0x7f6e980ff9e0 2023-11-27T15:16:53.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.792+0000 7f6ea03c1700 1 -- 172.21.15.52:0/3214145371 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e981123e0 con 0x7f6e980ff9e0 2023-11-27T15:16:53.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.793+0000 7f6ea03c1700 1 -- 172.21.15.52:0/3214145371 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e98112700 con 0x7f6e980ff9e0 2023-11-27T15:16:53.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.794+0000 7f6e8f7fe700 1 -- 172.21.15.52:0/3214145371 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f6e880054a0 con 0x7f6e980ff9e0 2023-11-27T15:16:53.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.795+0000 7f6e8f7fe700 1 --2- 172.21.15.52:0/3214145371 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6e840a11e0 0x7f6e840a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:53.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.795+0000 7f6e9e15d700 1 --2- 172.21.15.52:0/3214145371 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6e840a11e0 0x7f6e840a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:53.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.795+0000 7f6e8f7fe700 1 -- 172.21.15.52:0/3214145371 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6e880cdd60 con 0x7f6e980ff9e0 2023-11-27T15:16:53.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.796+0000 7f6e9e15d700 1 --2- 172.21.15.52:0/3214145371 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6e840a11e0 0x7f6e840a3690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6e94007be0 tx=0x7f6e940073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:53.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.799+0000 7f6e8f7fe700 1 -- 172.21.15.52:0/3214145371 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6e88097590 con 0x7f6e980ff9e0 2023-11-27T15:16:53.909 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.904+0000 7f74b7098700 1 -- 172.21.15.52:0/4013131858 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f74b0066ea0 con 0x7f74b00ff1e0 2023-11-27T15:16:53.909 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.904+0000 7f74adffb700 1 -- 172.21.15.52:0/4013131858 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f749c026090 con 0x7f74b00ff1e0 2023-11-27T15:16:53.910 INFO:teuthology.orchestra.run.smithi052.stdout:193273528362 2023-11-27T15:16:53.911 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.906+0000 7f74b7098700 1 -- 172.21.15.52:0/4013131858 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f74a00a1260 msgr2=0x7f74a00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:53.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.906+0000 7f74b7098700 1 --2- 172.21.15.52:0/4013131858 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f74a00a1260 0x7f74a00a3710 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f74a4005950 tx=0x7f74a400b410 comp rx=0 tx=0).stop 2023-11-27T15:16:53.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.906+0000 7f74b7098700 1 -- 172.21.15.52:0/4013131858 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74b00ff1e0 msgr2=0x7f74b00fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:53.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.906+0000 7f74b7098700 1 --2- 172.21.15.52:0/4013131858 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74b00ff1e0 0x7f74b00fff50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f749c00e9a0 tx=0x7f749c00ecb0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.906+0000 7f74b7098700 1 -- 172.21.15.52:0/4013131858 shutdown_connections 2023-11-27T15:16:53.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.906+0000 7f74b7098700 1 --2- 172.21.15.52:0/4013131858 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f74a00a1260 0x7f74a00a3710 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.906+0000 7f74b7098700 1 --2- 172.21.15.52:0/4013131858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74b00fe830 0x7f74b00ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.906+0000 7f74b7098700 1 --2- 172.21.15.52:0/4013131858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74b0105290 0x7f74b01004d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.906+0000 7f74b7098700 1 --2- 172.21.15.52:0/4013131858 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74b00ff1e0 0x7f74b00fff50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.906+0000 7f74b7098700 1 -- 172.21.15.52:0/4013131858 >> 172.21.15.52:0/4013131858 conn(0x7f74b00758a0 msgr2=0x7f74b00fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:53.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.906+0000 7f74b7098700 1 -- 172.21.15.52:0/4013131858 shutdown_connections 2023-11-27T15:16:53.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:53.906+0000 7f74b7098700 1 -- 172.21.15.52:0/4013131858 wait complete. 2023-11-27T15:16:54.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.012+0000 7f6ea03c1700 1 -- 172.21.15.52:0/3214145371 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f6e9819ce00 con 0x7f6e980ff9e0 2023-11-27T15:16:54.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.013+0000 7f6e8f7fe700 1 -- 172.21.15.52:0/3214145371 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f6e88095e60 con 0x7f6e980ff9e0 2023-11-27T15:16:54.019 INFO:teuthology.orchestra.run.smithi052.stdout:335007449109 2023-11-27T15:16:54.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.015+0000 7f6ea03c1700 1 -- 172.21.15.52:0/3214145371 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6e840a11e0 msgr2=0x7f6e840a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:54.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.015+0000 7f6ea03c1700 1 --2- 172.21.15.52:0/3214145371 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6e840a11e0 0x7f6e840a3690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6e94007be0 tx=0x7f6e940073d0 comp rx=0 tx=0).stop 2023-11-27T15:16:54.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.015+0000 7f6ea03c1700 1 -- 172.21.15.52:0/3214145371 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e980ff9e0 msgr2=0x7f6e98198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:54.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.015+0000 7f6ea03c1700 1 --2- 172.21.15.52:0/3214145371 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e980ff9e0 0x7f6e98198bd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800b680 tx=0x7f6e88004730 comp rx=0 tx=0).stop 2023-11-27T15:16:54.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.016+0000 7f6ea03c1700 1 -- 172.21.15.52:0/3214145371 shutdown_connections 2023-11-27T15:16:54.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.016+0000 7f6ea03c1700 1 --2- 172.21.15.52:0/3214145371 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6e840a11e0 0x7f6e840a3690 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:54.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.016+0000 7f6ea03c1700 1 --2- 172.21.15.52:0/3214145371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e980ff0d0 0x7f6e98198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:54.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.016+0000 7f6ea03c1700 1 --2- 172.21.15.52:0/3214145371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e9810cc80 0x7f6e9819bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:54.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.016+0000 7f6ea03c1700 1 --2- 172.21.15.52:0/3214145371 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e980ff9e0 0x7f6e98198bd0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:54.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.016+0000 7f6ea03c1700 1 -- 172.21.15.52:0/3214145371 >> 172.21.15.52:0/3214145371 conn(0x7f6e980770a0 msgr2=0x7f6e9810e050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:54.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.016+0000 7f6ea03c1700 1 -- 172.21.15.52:0/3214145371 shutdown_connections 2023-11-27T15:16:54.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.016+0000 7f6ea03c1700 1 -- 172.21.15.52:0/3214145371 wait complete. 2023-11-27T15:16:54.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:53 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2756289018' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-11-27T15:16:54.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:53 smithi102 ceph-mon[100296]: pgmap v278: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-11-27T15:16:54.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:53 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2756289018' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-11-27T15:16:54.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:53 smithi052 ceph-mon[99056]: pgmap v278: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-11-27T15:16:54.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.176+0000 7f014e029700 1 -- 172.21.15.52:0/1041314718 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f014810cc80 msgr2=0x7f014810f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:54.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.176+0000 7f014e029700 1 --2- 172.21.15.52:0/1041314718 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f014810cc80 0x7f014810f060 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f014000b0a0 tx=0x7f014000b3b0 comp rx=0 tx=0).stop 2023-11-27T15:16:54.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.177+0000 7f014e029700 1 -- 172.21.15.52:0/1041314718 shutdown_connections 2023-11-27T15:16:54.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.177+0000 7f014e029700 1 --2- 172.21.15.52:0/1041314718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01480ff0d0 0x7f01480ff4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:54.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.177+0000 7f014e029700 1 --2- 172.21.15.52:0/1041314718 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f014810cc80 0x7f014810f060 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:54.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.177+0000 7f014e029700 1 --2- 172.21.15.52:0/1041314718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01480ff9e0 0x7f014810c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:54.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.177+0000 7f014e029700 1 -- 172.21.15.52:0/1041314718 >> 172.21.15.52:0/1041314718 conn(0x7f01480770a0 msgr2=0x7f01480774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:54.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.177+0000 7f014e029700 1 -- 172.21.15.52:0/1041314718 shutdown_connections 2023-11-27T15:16:54.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.177+0000 7f014e029700 1 -- 172.21.15.52:0/1041314718 wait complete. 2023-11-27T15:16:54.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.177+0000 7f014e029700 1 Processor -- start 2023-11-27T15:16:54.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.177+0000 7f014e029700 1 -- start start 2023-11-27T15:16:54.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.178+0000 7f014e029700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01480ff0d0 0x7f014819e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:54.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.178+0000 7f014e029700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01480ff9e0 0x7f014819ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:54.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.178+0000 7f014e029700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f014810cc80 0x7f01481985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:54.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.178+0000 7f014e029700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0148198c20 con 0x7f01480ff9e0 2023-11-27T15:16:54.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.178+0000 7f014e029700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0148198d90 con 0x7f01480ff0d0 2023-11-27T15:16:54.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.178+0000 7f014e029700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0148198f50 con 0x7f014810cc80 2023-11-27T15:16:54.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.178+0000 7f0146ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01480ff9e0 0x7f014819ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:54.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.178+0000 7f0146ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01480ff9e0 0x7f014819ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50600/0 (socket says 172.21.15.52:50600) 2023-11-27T15:16:54.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.178+0000 7f0146ffd700 1 -- 172.21.15.52:0/1275569814 learned_addr learned my addr 172.21.15.52:0/1275569814 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:54.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.178+0000 7f0147fff700 1 --2- 172.21.15.52:0/1275569814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f014810cc80 0x7f01481985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:54.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.178+0000 7f01477fe700 1 --2- 172.21.15.52:0/1275569814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01480ff0d0 0x7f014819e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:54.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.178+0000 7f0146ffd700 1 -- 172.21.15.52:0/1275569814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f014810cc80 msgr2=0x7f01481985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:54.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.178+0000 7f0146ffd700 1 --2- 172.21.15.52:0/1275569814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f014810cc80 0x7f01481985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:54.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.178+0000 7f0146ffd700 1 -- 172.21.15.52:0/1275569814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01480ff0d0 msgr2=0x7f014819e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:54.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.178+0000 7f0146ffd700 1 --2- 172.21.15.52:0/1275569814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01480ff0d0 0x7f014819e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:54.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.178+0000 7f0146ffd700 1 -- 172.21.15.52:0/1275569814 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0140009d00 con 0x7f01480ff9e0 2023-11-27T15:16:54.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.178+0000 7f0147fff700 1 --2- 172.21.15.52:0/1275569814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f014810cc80 0x7f01481985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:16:54.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.178+0000 7f0146ffd700 1 --2- 172.21.15.52:0/1275569814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01480ff9e0 0x7f014819ea80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f013c00c9b0 tx=0x7f013c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:54.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.178+0000 7f0144ff9700 1 -- 172.21.15.52:0/1275569814 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c007b00 con 0x7f01480ff9e0 2023-11-27T15:16:54.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.178+0000 7f0144ff9700 1 -- 172.21.15.52:0/1275569814 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f013c007c60 con 0x7f01480ff9e0 2023-11-27T15:16:54.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.178+0000 7f014e029700 1 -- 172.21.15.52:0/1275569814 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0148112520 con 0x7f01480ff9e0 2023-11-27T15:16:54.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.178+0000 7f0144ff9700 1 -- 172.21.15.52:0/1275569814 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c018660 con 0x7f01480ff9e0 2023-11-27T15:16:54.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.179+0000 7f014e029700 1 -- 172.21.15.52:0/1275569814 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0148112390 con 0x7f01480ff9e0 2023-11-27T15:16:54.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.179+0000 7f014e029700 1 -- 172.21.15.52:0/1275569814 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01481126b0 con 0x7f01480ff9e0 2023-11-27T15:16:54.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.180+0000 7f0144ff9700 1 -- 172.21.15.52:0/1275569814 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f013c018880 con 0x7f01480ff9e0 2023-11-27T15:16:54.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.181+0000 7f0144ff9700 1 --2- 172.21.15.52:0/1275569814 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f01300a11e0 0x7f01300a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:54.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.181+0000 7f01477fe700 1 --2- 172.21.15.52:0/1275569814 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f01300a11e0 0x7f01300a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:54.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.181+0000 7f0144ff9700 1 -- 172.21.15.52:0/1275569814 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f013c0cc940 con 0x7f01480ff9e0 2023-11-27T15:16:54.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.182+0000 7f01477fe700 1 --2- 172.21.15.52:0/1275569814 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f01300a11e0 0x7f01300a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0138005fd0 tx=0x7f0138005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:54.191 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.186+0000 7f0144ff9700 1 -- 172.21.15.52:0/1275569814 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f013c0960e0 con 0x7f01480ff9e0 2023-11-27T15:16:54.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:53 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2756289018' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-11-27T15:16:54.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:53 smithi150 ceph-mon[100559]: pgmap v278: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2023-11-27T15:16:54.393 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.387+0000 7f014e029700 1 -- 172.21.15.52:0/1275569814 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0148066ea0 con 0x7f01480ff9e0 2023-11-27T15:16:54.393 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.387+0000 7f0144ff9700 1 -- 172.21.15.52:0/1275569814 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f013c099700 con 0x7f01480ff9e0 2023-11-27T15:16:54.393 INFO:teuthology.orchestra.run.smithi052.stdout:55834574912 2023-11-27T15:16:54.395 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.390+0000 7f014e029700 1 -- 172.21.15.52:0/1275569814 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f01300a11e0 msgr2=0x7f01300a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:54.395 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.390+0000 7f014e029700 1 --2- 172.21.15.52:0/1275569814 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f01300a11e0 0x7f01300a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0138005fd0 tx=0x7f0138005f60 comp rx=0 tx=0).stop 2023-11-27T15:16:54.395 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.390+0000 7f014e029700 1 -- 172.21.15.52:0/1275569814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01480ff9e0 msgr2=0x7f014819ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:54.395 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.390+0000 7f014e029700 1 --2- 172.21.15.52:0/1275569814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01480ff9e0 0x7f014819ea80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f013c00c9b0 tx=0x7f013c00cd70 comp rx=0 tx=0).stop 2023-11-27T15:16:54.396 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.390+0000 7f014e029700 1 -- 172.21.15.52:0/1275569814 shutdown_connections 2023-11-27T15:16:54.396 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.390+0000 7f014e029700 1 --2- 172.21.15.52:0/1275569814 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f01300a11e0 0x7f01300a3690 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:54.396 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.390+0000 7f014e029700 1 --2- 172.21.15.52:0/1275569814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01480ff0d0 0x7f014819e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:54.396 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.390+0000 7f014e029700 1 --2- 172.21.15.52:0/1275569814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f014810cc80 0x7f01481985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:54.396 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.390+0000 7f014e029700 1 --2- 172.21.15.52:0/1275569814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01480ff9e0 0x7f014819ea80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:54.396 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.390+0000 7f014e029700 1 -- 172.21.15.52:0/1275569814 >> 172.21.15.52:0/1275569814 conn(0x7f01480770a0 msgr2=0x7f014810df40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:54.396 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.390+0000 7f014e029700 1 -- 172.21.15.52:0/1275569814 shutdown_connections 2023-11-27T15:16:54.396 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:54.390+0000 7f014e029700 1 -- 172.21.15.52:0/1275569814 wait complete. 2023-11-27T15:16:54.723 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738432 got 34359738435 for osd.0 2023-11-27T15:16:54.723 DEBUG:teuthology.parallel:result is None 2023-11-27T15:16:54.776 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528359 got 193273528362 for osd.6 2023-11-27T15:16:54.776 DEBUG:teuthology.parallel:result is None 2023-11-27T15:16:55.062 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:54 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/4013131858' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-11-27T15:16:55.062 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:54 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3214145371' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-11-27T15:16:55.062 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:54 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1275569814' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-11-27T15:16:55.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:54 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/4013131858' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-11-27T15:16:55.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:54 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3214145371' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-11-27T15:16:55.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:54 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1275569814' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-11-27T15:16:55.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:54 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/4013131858' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-11-27T15:16:55.308 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:54 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3214145371' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-11-27T15:16:55.309 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:54 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1275569814' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-11-27T15:16:55.310 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574909 got 55834574912 for osd.1 2023-11-27T15:16:55.310 DEBUG:teuthology.parallel:result is None 2023-11-27T15:16:55.904 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182452 got 107374182454 for osd.3 2023-11-27T15:16:55.904 DEBUG:teuthology.parallel:result is None 2023-11-27T15:16:55.992 INFO:tasks.cephadm.ceph_manager.ceph:need seq 335007449106 got 335007449109 for osd.11 2023-11-27T15:16:55.992 DEBUG:teuthology.parallel:result is None 2023-11-27T15:16:56.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:55 smithi102 ceph-mon[100296]: pgmap v279: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2023-11-27T15:16:56.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:55 smithi052 ceph-mon[99056]: pgmap v279: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2023-11-27T15:16:56.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.173+0000 7fc312a79700 1 -- 172.21.15.52:0/3446943507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc30c100550 msgr2=0x7fc30c100920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:56.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.173+0000 7fc312a79700 1 --2- 172.21.15.52:0/3446943507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc30c100550 0x7fc30c100920 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc3000099c0 tx=0x7fc300009cd0 comp rx=0 tx=0).stop 2023-11-27T15:16:56.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.173+0000 7fc312a79700 1 -- 172.21.15.52:0/3446943507 shutdown_connections 2023-11-27T15:16:56.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.173+0000 7fc312a79700 1 --2- 172.21.15.52:0/3446943507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc30c1055f0 0x7fc30c10d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:56.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.173+0000 7fc312a79700 1 --2- 172.21.15.52:0/3446943507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc30c100550 0x7fc30c100920 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:56.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.173+0000 7fc312a79700 1 --2- 172.21.15.52:0/3446943507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc30c100ef0 0x7fc30c104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:56.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.173+0000 7fc312a79700 1 -- 172.21.15.52:0/3446943507 >> 172.21.15.52:0/3446943507 conn(0x7fc30c0fbdf0 msgr2=0x7fc30c0fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:56.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.173+0000 7fc312a79700 1 -- 172.21.15.52:0/3446943507 shutdown_connections 2023-11-27T15:16:56.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.173+0000 7fc312a79700 1 -- 172.21.15.52:0/3446943507 wait complete. 2023-11-27T15:16:56.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.173+0000 7fc312a79700 1 Processor -- start 2023-11-27T15:16:56.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.173+0000 7fc312a79700 1 -- start start 2023-11-27T15:16:56.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.174+0000 7fc312a79700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc30c100550 0x7fc30c1963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:56.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.174+0000 7fc312a79700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc30c100ef0 0x7fc30c1968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:56.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.174+0000 7fc312a79700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc30c1055f0 0x7fc30c19ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:56.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.174+0000 7fc312a79700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c19b370 con 0x7fc30c100ef0 2023-11-27T15:16:56.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.174+0000 7fc312a79700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c19b4e0 con 0x7fc30c1055f0 2023-11-27T15:16:56.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.174+0000 7fc312a79700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c19b6a0 con 0x7fc30c100550 2023-11-27T15:16:56.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.174+0000 7fc30bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc30c100ef0 0x7fc30c1968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:56.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.174+0000 7fc30bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc30c100ef0 0x7fc30c1968f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50612/0 (socket says 172.21.15.52:50612) 2023-11-27T15:16:56.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.174+0000 7fc30bfff700 1 -- 172.21.15.52:0/2452216882 learned_addr learned my addr 172.21.15.52:0/2452216882 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:56.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.174+0000 7fc310815700 1 --2- 172.21.15.52:0/2452216882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc30c100550 0x7fc30c1963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:56.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.174+0000 7fc311016700 1 --2- 172.21.15.52:0/2452216882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc30c1055f0 0x7fc30c19ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:56.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.174+0000 7fc30bfff700 1 -- 172.21.15.52:0/2452216882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc30c100550 msgr2=0x7fc30c1963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:56.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.174+0000 7fc30bfff700 1 --2- 172.21.15.52:0/2452216882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc30c100550 0x7fc30c1963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:56.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.174+0000 7fc30bfff700 1 -- 172.21.15.52:0/2452216882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc30c1055f0 msgr2=0x7fc30c19ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:56.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.174+0000 7fc30bfff700 1 --2- 172.21.15.52:0/2452216882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc30c1055f0 0x7fc30c19ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:56.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.174+0000 7fc30bfff700 1 -- 172.21.15.52:0/2452216882 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3000095f0 con 0x7fc30c100ef0 2023-11-27T15:16:56.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.174+0000 7fc30bfff700 1 --2- 172.21.15.52:0/2452216882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc30c100ef0 0x7fc30c1968f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc2f800c9e0 tx=0x7fc2f800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:56.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.174+0000 7fc309ffb700 1 -- 172.21.15.52:0/2452216882 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8007aa0 con 0x7fc30c100ef0 2023-11-27T15:16:56.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.174+0000 7fc309ffb700 1 -- 172.21.15.52:0/2452216882 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2f800ce80 con 0x7fc30c100ef0 2023-11-27T15:16:56.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.174+0000 7fc312a79700 1 -- 172.21.15.52:0/2452216882 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc30c110330 con 0x7fc30c100ef0 2023-11-27T15:16:56.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.175+0000 7fc309ffb700 1 -- 172.21.15.52:0/2452216882 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8018660 con 0x7fc30c100ef0 2023-11-27T15:16:56.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.175+0000 7fc312a79700 1 -- 172.21.15.52:0/2452216882 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc30c1101a0 con 0x7fc30c100ef0 2023-11-27T15:16:56.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.176+0000 7fc312a79700 1 -- 172.21.15.52:0/2452216882 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc30c1104c0 con 0x7fc30c100ef0 2023-11-27T15:16:56.184 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.177+0000 7fc309ffb700 1 -- 172.21.15.52:0/2452216882 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc2f800f450 con 0x7fc30c100ef0 2023-11-27T15:16:56.184 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.177+0000 7fc309ffb700 1 --2- 172.21.15.52:0/2452216882 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fc2fc0a11e0 0x7fc2fc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:56.184 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.177+0000 7fc310815700 1 --2- 172.21.15.52:0/2452216882 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fc2fc0a11e0 0x7fc2fc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:56.184 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.177+0000 7fc309ffb700 1 -- 172.21.15.52:0/2452216882 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc2f80cc870 con 0x7fc30c100ef0 2023-11-27T15:16:56.184 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.178+0000 7fc310815700 1 --2- 172.21.15.52:0/2452216882 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fc2fc0a11e0 0x7fc2fc0a3690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc300000c00 tx=0x7fc300003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:56.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.181+0000 7fc309ffb700 1 -- 172.21.15.52:0/2452216882 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc2f8096010 con 0x7fc30c100ef0 2023-11-27T15:16:56.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:55 smithi150 ceph-mon[100559]: pgmap v279: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2023-11-27T15:16:56.395 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.390+0000 7fc312a79700 1 -- 172.21.15.52:0/2452216882 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fc30c1a34f0 con 0x7fc30c100ef0 2023-11-27T15:16:56.395 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.390+0000 7fc309ffb700 1 -- 172.21.15.52:0/2452216882 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc2f800fe60 con 0x7fc30c100ef0 2023-11-27T15:16:56.396 INFO:teuthology.orchestra.run.smithi052.stdout:283467841565 2023-11-27T15:16:56.398 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.393+0000 7fc312a79700 1 -- 172.21.15.52:0/2452216882 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fc2fc0a11e0 msgr2=0x7fc2fc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:56.398 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.393+0000 7fc312a79700 1 --2- 172.21.15.52:0/2452216882 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fc2fc0a11e0 0x7fc2fc0a3690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc300000c00 tx=0x7fc300003680 comp rx=0 tx=0).stop 2023-11-27T15:16:56.398 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.393+0000 7fc312a79700 1 -- 172.21.15.52:0/2452216882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc30c100ef0 msgr2=0x7fc30c1968f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:56.399 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.393+0000 7fc312a79700 1 --2- 172.21.15.52:0/2452216882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc30c100ef0 0x7fc30c1968f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc2f800c9e0 tx=0x7fc2f800ccf0 comp rx=0 tx=0).stop 2023-11-27T15:16:56.399 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.393+0000 7fc312a79700 1 -- 172.21.15.52:0/2452216882 shutdown_connections 2023-11-27T15:16:56.399 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.393+0000 7fc312a79700 1 --2- 172.21.15.52:0/2452216882 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fc2fc0a11e0 0x7fc2fc0a3690 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:56.399 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.393+0000 7fc312a79700 1 --2- 172.21.15.52:0/2452216882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc30c1055f0 0x7fc30c19ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:56.399 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.393+0000 7fc312a79700 1 --2- 172.21.15.52:0/2452216882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc30c100550 0x7fc30c1963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:56.399 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.393+0000 7fc312a79700 1 --2- 172.21.15.52:0/2452216882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc30c100ef0 0x7fc30c1968f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:56.399 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.393+0000 7fc312a79700 1 -- 172.21.15.52:0/2452216882 >> 172.21.15.52:0/2452216882 conn(0x7fc30c0fbdf0 msgr2=0x7fc30c0fc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:56.399 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.393+0000 7fc312a79700 1 -- 172.21.15.52:0/2452216882 shutdown_connections 2023-11-27T15:16:56.399 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.393+0000 7fc312a79700 1 -- 172.21.15.52:0/2452216882 wait complete. 2023-11-27T15:16:56.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.470+0000 7f0aa5354700 1 -- 172.21.15.52:0/2979696471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0aa0100ee0 msgr2=0x7f0aa0104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:56.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.470+0000 7f0aa5354700 1 --2- 172.21.15.52:0/2979696471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0aa0100ee0 0x7f0aa0104eb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0a940099c0 tx=0x7f0a94009cd0 comp rx=0 tx=0).stop 2023-11-27T15:16:56.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.470+0000 7f0aa5354700 1 -- 172.21.15.52:0/2979696471 shutdown_connections 2023-11-27T15:16:56.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.470+0000 7f0aa5354700 1 --2- 172.21.15.52:0/2979696471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0aa0100540 0x7f0aa0100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:56.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.470+0000 7f0aa5354700 1 --2- 172.21.15.52:0/2979696471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0aa01055e0 0x7f0aa010d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:56.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.470+0000 7f0aa5354700 1 --2- 172.21.15.52:0/2979696471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0aa0100ee0 0x7f0aa0104eb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:56.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.470+0000 7f0aa5354700 1 -- 172.21.15.52:0/2979696471 >> 172.21.15.52:0/2979696471 conn(0x7f0aa00fbdc0 msgr2=0x7f0aa00fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:56.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.470+0000 7f0aa5354700 1 -- 172.21.15.52:0/2979696471 shutdown_connections 2023-11-27T15:16:56.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.470+0000 7f0aa5354700 1 -- 172.21.15.52:0/2979696471 wait complete. 2023-11-27T15:16:56.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.471+0000 7f0aa5354700 1 Processor -- start 2023-11-27T15:16:56.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.471+0000 7f0aa5354700 1 -- start start 2023-11-27T15:16:56.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.471+0000 7f0aa5354700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0aa0100540 0x7f0aa0196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:56.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.471+0000 7f0aa5354700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0aa0100ee0 0x7f0aa01967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:56.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.471+0000 7f0aa5354700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0aa01055e0 0x7f0aa019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:56.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.471+0000 7f0aa5354700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa019b250 con 0x7f0aa01055e0 2023-11-27T15:16:56.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.471+0000 7f0aa5354700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa019b3c0 con 0x7f0aa0100540 2023-11-27T15:16:56.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.471+0000 7f0aa5354700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa019b580 con 0x7f0aa0100ee0 2023-11-27T15:16:56.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.471+0000 7f0a9f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0aa01055e0 0x7f0aa019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:56.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.471+0000 7f0a9effd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0aa0100540 0x7f0aa0196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:56.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.471+0000 7f0a9f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0aa01055e0 0x7f0aa019ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50628/0 (socket says 172.21.15.52:50628) 2023-11-27T15:16:56.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.471+0000 7f0a9f7fe700 1 -- 172.21.15.52:0/3654372801 learned_addr learned my addr 172.21.15.52:0/3654372801 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:56.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.471+0000 7f0a9effd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0aa0100540 0x7f0aa0196290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:40282/0 (socket says 172.21.15.52:40282) 2023-11-27T15:16:56.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.471+0000 7f0a9e7fc700 1 --2- 172.21.15.52:0/3654372801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0aa0100ee0 0x7f0aa01967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:56.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.472+0000 7f0a9f7fe700 1 -- 172.21.15.52:0/3654372801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0aa0100ee0 msgr2=0x7f0aa01967d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:56.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.472+0000 7f0a9f7fe700 1 --2- 172.21.15.52:0/3654372801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0aa0100ee0 0x7f0aa01967d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:56.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.472+0000 7f0a9f7fe700 1 -- 172.21.15.52:0/3654372801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0aa0100540 msgr2=0x7f0aa0196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:56.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.472+0000 7f0a9f7fe700 1 --2- 172.21.15.52:0/3654372801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0aa0100540 0x7f0aa0196290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:56.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.472+0000 7f0a9f7fe700 1 -- 172.21.15.52:0/3654372801 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a940095f0 con 0x7f0aa01055e0 2023-11-27T15:16:56.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.472+0000 7f0a9effd700 1 --2- 172.21.15.52:0/3654372801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0aa0100540 0x7f0aa0196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:16:56.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.472+0000 7f0a9f7fe700 1 --2- 172.21.15.52:0/3654372801 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0aa01055e0 0x7f0aa019ac40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0a98007d70 tx=0x7f0a9800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:56.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.472+0000 7f0a87fff700 1 -- 172.21.15.52:0/3654372801 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9800ac90 con 0x7f0aa01055e0 2023-11-27T15:16:56.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.472+0000 7f0aa5354700 1 -- 172.21.15.52:0/3654372801 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa0110310 con 0x7f0aa01055e0 2023-11-27T15:16:56.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.472+0000 7f0a87fff700 1 -- 172.21.15.52:0/3654372801 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a9800adf0 con 0x7f0aa01055e0 2023-11-27T15:16:56.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.472+0000 7f0a87fff700 1 -- 172.21.15.52:0/3654372801 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a980115e0 con 0x7f0aa01055e0 2023-11-27T15:16:56.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.472+0000 7f0aa5354700 1 -- 172.21.15.52:0/3654372801 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa0110180 con 0x7f0aa01055e0 2023-11-27T15:16:56.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.473+0000 7f0aa5354700 1 -- 172.21.15.52:0/3654372801 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa01104a0 con 0x7f0aa01055e0 2023-11-27T15:16:56.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.475+0000 7f0a87fff700 1 -- 172.21.15.52:0/3654372801 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f0a98012420 con 0x7f0aa01055e0 2023-11-27T15:16:56.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.475+0000 7f0a87fff700 1 --2- 172.21.15.52:0/3654372801 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0a880a1260 0x7f0a880a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:56.482 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.476+0000 7f0a9effd700 1 --2- 172.21.15.52:0/3654372801 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0a880a1260 0x7f0a880a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:56.482 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.476+0000 7f0a87fff700 1 -- 172.21.15.52:0/3654372801 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0a980cccb0 con 0x7f0aa01055e0 2023-11-27T15:16:56.482 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.476+0000 7f0a9effd700 1 --2- 172.21.15.52:0/3654372801 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0a880a1260 0x7f0a880a3710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0a9000afd0 tx=0x7f0a9000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:56.484 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.479+0000 7f0a87fff700 1 -- 172.21.15.52:0/3654372801 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0a98096560 con 0x7f0aa01055e0 2023-11-27T15:16:56.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.685+0000 7f0aa5354700 1 -- 172.21.15.52:0/3654372801 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0aa0066ea0 con 0x7f0aa01055e0 2023-11-27T15:16:56.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.685+0000 7f0a87fff700 1 -- 172.21.15.52:0/3654372801 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0a98016090 con 0x7f0aa01055e0 2023-11-27T15:16:56.691 INFO:teuthology.orchestra.run.smithi052.stdout:77309411387 2023-11-27T15:16:56.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.687+0000 7f0aa5354700 1 -- 172.21.15.52:0/3654372801 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0a880a1260 msgr2=0x7f0a880a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:56.693 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.687+0000 7f0aa5354700 1 --2- 172.21.15.52:0/3654372801 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0a880a1260 0x7f0a880a3710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0a9000afd0 tx=0x7f0a9000a380 comp rx=0 tx=0).stop 2023-11-27T15:16:56.693 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.687+0000 7f0aa5354700 1 -- 172.21.15.52:0/3654372801 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0aa01055e0 msgr2=0x7f0aa019ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:56.693 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.687+0000 7f0aa5354700 1 --2- 172.21.15.52:0/3654372801 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0aa01055e0 0x7f0aa019ac40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0a98007d70 tx=0x7f0a9800a340 comp rx=0 tx=0).stop 2023-11-27T15:16:56.693 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.687+0000 7f0aa5354700 1 -- 172.21.15.52:0/3654372801 shutdown_connections 2023-11-27T15:16:56.693 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.687+0000 7f0aa5354700 1 --2- 172.21.15.52:0/3654372801 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0a880a1260 0x7f0a880a3710 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:56.693 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.687+0000 7f0aa5354700 1 --2- 172.21.15.52:0/3654372801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0aa0100540 0x7f0aa0196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:56.693 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.687+0000 7f0aa5354700 1 --2- 172.21.15.52:0/3654372801 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0aa01055e0 0x7f0aa019ac40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:56.693 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.687+0000 7f0aa5354700 1 --2- 172.21.15.52:0/3654372801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0aa0100ee0 0x7f0aa01967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:56.693 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.687+0000 7f0aa5354700 1 -- 172.21.15.52:0/3654372801 >> 172.21.15.52:0/3654372801 conn(0x7f0aa00fbdc0 msgr2=0x7f0aa00fc760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:56.693 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.687+0000 7f0aa5354700 1 -- 172.21.15.52:0/3654372801 shutdown_connections 2023-11-27T15:16:56.693 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:56.687+0000 7f0aa5354700 1 -- 172.21.15.52:0/3654372801 wait complete. 2023-11-27T15:16:56.863 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841563 got 283467841565 for osd.9 2023-11-27T15:16:56.864 DEBUG:teuthology.parallel:result is None 2023-11-27T15:16:57.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:56 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2452216882' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-11-27T15:16:57.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:56 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3654372801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-11-27T15:16:57.105 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:56 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2452216882' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-11-27T15:16:57.105 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:56 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3654372801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-11-27T15:16:57.194 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411385 got 77309411387 for osd.2 2023-11-27T15:16:57.194 DEBUG:teuthology.parallel:result is None 2023-11-27T15:16:57.194 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-11-27T15:16:57.194 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph pg dump --format=json 2023-11-27T15:16:57.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:56 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2452216882' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-11-27T15:16:57.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:56 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3654372801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-11-27T15:16:57.852 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:58.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:57 smithi102 ceph-mon[100296]: pgmap v280: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-11-27T15:16:58.096 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:57 smithi052 ceph-mon[99056]: pgmap v280: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-11-27T15:16:58.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:57 smithi150 ceph-mon[100559]: pgmap v280: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-11-27T15:16:59.405 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.400+0000 7efcdb326700 1 -- 172.21.15.52:0/3637965374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcd4104510 msgr2=0x7efcd410c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:59.405 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.400+0000 7efcdb326700 1 --2- 172.21.15.52:0/3637965374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcd4104510 0x7efcd410c090 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7efccc00b0d0 tx=0x7efccc00b3e0 comp rx=0 tx=0).stop 2023-11-27T15:16:59.405 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.400+0000 7efcdb326700 1 -- 172.21.15.52:0/3637965374 shutdown_connections 2023-11-27T15:16:59.406 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.400+0000 7efcdb326700 1 --2- 172.21.15.52:0/3637965374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcd4104510 0x7efcd410c090 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:59.406 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.400+0000 7efcdb326700 1 --2- 172.21.15.52:0/3637965374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efcd40ff470 0x7efcd40ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:59.406 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.400+0000 7efcdb326700 1 --2- 172.21.15.52:0/3637965374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcd40ffe10 0x7efcd4103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:59.406 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.400+0000 7efcdb326700 1 -- 172.21.15.52:0/3637965374 >> 172.21.15.52:0/3637965374 conn(0x7efcd40770d0 msgr2=0x7efcd40774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:59.406 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.400+0000 7efcdb326700 1 -- 172.21.15.52:0/3637965374 shutdown_connections 2023-11-27T15:16:59.406 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.400+0000 7efcdb326700 1 -- 172.21.15.52:0/3637965374 wait complete. 2023-11-27T15:16:59.408 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.400+0000 7efcdb326700 1 Processor -- start 2023-11-27T15:16:59.408 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.400+0000 7efcdb326700 1 -- start start 2023-11-27T15:16:59.408 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.401+0000 7efcdb326700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcd40ff470 0x7efcd4196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:59.408 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.401+0000 7efcdb326700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efcd40ffe10 0x7efcd41969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:59.408 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.401+0000 7efcdb326700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcd4104510 0x7efcd419ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:59.408 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.401+0000 7efcdb326700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd419b450 con 0x7efcd40ff470 2023-11-27T15:16:59.408 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.401+0000 7efcdb326700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd419b5c0 con 0x7efcd4104510 2023-11-27T15:16:59.408 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.401+0000 7efcdb326700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd419b780 con 0x7efcd40ffe10 2023-11-27T15:16:59.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.401+0000 7efcd90c2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcd40ff470 0x7efcd4196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:59.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.401+0000 7efcd90c2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcd40ff470 0x7efcd4196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50636/0 (socket says 172.21.15.52:50636) 2023-11-27T15:16:59.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.401+0000 7efcd90c2700 1 -- 172.21.15.52:0/475806631 learned_addr learned my addr 172.21.15.52:0/475806631 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:16:59.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.401+0000 7efcd88c1700 1 --2- 172.21.15.52:0/475806631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efcd40ffe10 0x7efcd41969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:59.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.401+0000 7efcd98c3700 1 --2- 172.21.15.52:0/475806631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcd4104510 0x7efcd419ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:59.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.401+0000 7efcd90c2700 1 -- 172.21.15.52:0/475806631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efcd40ffe10 msgr2=0x7efcd41969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:59.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.401+0000 7efcd90c2700 1 --2- 172.21.15.52:0/475806631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efcd40ffe10 0x7efcd41969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:59.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.401+0000 7efcd90c2700 1 -- 172.21.15.52:0/475806631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcd4104510 msgr2=0x7efcd419ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:59.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.401+0000 7efcd90c2700 1 --2- 172.21.15.52:0/475806631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcd4104510 0x7efcd419ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:59.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.401+0000 7efcd90c2700 1 -- 172.21.15.52:0/475806631 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efccc009d00 con 0x7efcd40ff470 2023-11-27T15:16:59.410 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.401+0000 7efcd98c3700 1 --2- 172.21.15.52:0/475806631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcd4104510 0x7efcd419ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:16:59.410 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.401+0000 7efcd90c2700 1 --2- 172.21.15.52:0/475806631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcd40ff470 0x7efcd4196490 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efcc400d950 tx=0x7efcc400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:59.410 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.401+0000 7efcca7fc700 1 -- 172.21.15.52:0/475806631 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc4009940 con 0x7efcd40ff470 2023-11-27T15:16:59.410 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.402+0000 7efcca7fc700 1 -- 172.21.15.52:0/475806631 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcc4010460 con 0x7efcd40ff470 2023-11-27T15:16:59.410 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.402+0000 7efcdb326700 1 -- 172.21.15.52:0/475806631 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd41103f0 con 0x7efcd40ff470 2023-11-27T15:16:59.410 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.402+0000 7efcca7fc700 1 -- 172.21.15.52:0/475806631 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc400f620 con 0x7efcd40ff470 2023-11-27T15:16:59.410 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.402+0000 7efcdb326700 1 -- 172.21.15.52:0/475806631 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd4110260 con 0x7efcd40ff470 2023-11-27T15:16:59.410 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.402+0000 7efcdb326700 1 -- 172.21.15.52:0/475806631 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcd4110580 con 0x7efcd40ff470 2023-11-27T15:16:59.411 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.404+0000 7efcca7fc700 1 -- 172.21.15.52:0/475806631 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7efcc40108d0 con 0x7efcd40ff470 2023-11-27T15:16:59.411 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.404+0000 7efcca7fc700 1 --2- 172.21.15.52:0/475806631 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7efcc00a11e0 0x7efcc00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:59.411 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.405+0000 7efcd88c1700 1 --2- 172.21.15.52:0/475806631 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7efcc00a11e0 0x7efcc00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:59.411 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.405+0000 7efcca7fc700 1 -- 172.21.15.52:0/475806631 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7efcc40cc880 con 0x7efcd40ff470 2023-11-27T15:16:59.411 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.405+0000 7efcd88c1700 1 --2- 172.21.15.52:0/475806631 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7efcc00a11e0 0x7efcc00a3690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efcd4197ab0 tx=0x7efcd000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:59.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.408+0000 7efcca7fc700 1 -- 172.21.15.52:0/475806631 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7efcc4095f20 con 0x7efcd40ff470 2023-11-27T15:16:59.605 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.600+0000 7efcdb326700 1 -- 172.21.15.52:0/475806631 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efcd40fd360 con 0x7efcc00a11e0 2023-11-27T15:16:59.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.612+0000 7efcca7fc700 1 -- 172.21.15.52:0/475806631 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+339939 (secure 0 0 0) 0x7efcd40fd360 con 0x7efcc00a11e0 2023-11-27T15:16:59.621 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:16:59.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.615+0000 7efcdb326700 1 -- 172.21.15.52:0/475806631 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7efcc00a11e0 msgr2=0x7efcc00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:59.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.615+0000 7efcdb326700 1 --2- 172.21.15.52:0/475806631 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7efcc00a11e0 0x7efcc00a3690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efcd4197ab0 tx=0x7efcd000b410 comp rx=0 tx=0).stop 2023-11-27T15:16:59.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.615+0000 7efcdb326700 1 -- 172.21.15.52:0/475806631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcd40ff470 msgr2=0x7efcd4196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:59.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.615+0000 7efcdb326700 1 --2- 172.21.15.52:0/475806631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcd40ff470 0x7efcd4196490 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efcc400d950 tx=0x7efcc400dc60 comp rx=0 tx=0).stop 2023-11-27T15:16:59.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.615+0000 7efcdb326700 1 -- 172.21.15.52:0/475806631 shutdown_connections 2023-11-27T15:16:59.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.615+0000 7efcdb326700 1 --2- 172.21.15.52:0/475806631 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7efcc00a11e0 0x7efcc00a3690 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:59.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.615+0000 7efcdb326700 1 --2- 172.21.15.52:0/475806631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efcd4104510 0x7efcd419ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:59.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.615+0000 7efcdb326700 1 --2- 172.21.15.52:0/475806631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efcd40ff470 0x7efcd4196490 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:59.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.615+0000 7efcdb326700 1 --2- 172.21.15.52:0/475806631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efcd40ffe10 0x7efcd41969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:59.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.615+0000 7efcdb326700 1 -- 172.21.15.52:0/475806631 >> 172.21.15.52:0/475806631 conn(0x7efcd40770d0 msgr2=0x7efcd40fcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:59.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.615+0000 7efcdb326700 1 -- 172.21.15.52:0/475806631 shutdown_connections 2023-11-27T15:16:59.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:16:59.615+0000 7efcdb326700 1 -- 172.21.15.52:0/475806631 wait complete. 2023-11-27T15:16:59.624 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2023-11-27T15:17:00.013 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:59 smithi052 ceph-mon[99056]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:17:00.013 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:16:59 smithi052 ceph-mon[99056]: pgmap v281: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-27T15:17:00.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:59 smithi150 ceph-mon[100559]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:17:00.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:16:59 smithi150 ceph-mon[100559]: pgmap v281: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-27T15:17:00.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:59 smithi102 ceph-mon[100296]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:17:00.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:16:59 smithi102 ceph-mon[100296]: pgmap v281: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-27T15:17:00.107 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2023-11-27T15:16:57.734498+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653435,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":406,"num_read_kb":319,"num_write":334,"num_write_kb":3044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":272,"ondisk_log_size":272,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":78608,"kb_used_data":10992,"kb_used_omap":0,"kb_used_meta":67520,"kb_avail":1124863216,"statfs":{"total":1151940427776,"available":1151859933184,"internally_reserved":0,"allocated":11255808,"data_stored":7075857,"data_compressed":108282,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":69140480},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":32,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003112"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329249+0000","last_change":"2023-11-27T15:16:09.343007+0000","last_active":"2023-11-27T15:16:10.329249+0000","last_peered":"2023-11-27T15:16:10.329249+0000","last_clean":"2023-11-27T15:16:10.329249+0000","last_became_active":"2023-11-27T15:16:09.342125+0000","last_became_peered":"2023-11-27T15:16:09.342125+0000","last_unstale":"2023-11-27T15:16:10.329249+0000","last_undegraded":"2023-11-27T15:16:10.329249+0000","last_fullsized":"2023-11-27T15:16:10.329249+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:10:56.620347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.865187+0000","last_change":"2023-11-27T15:16:09.338942+0000","last_active":"2023-11-27T15:16:10.865187+0000","last_peered":"2023-11-27T15:16:10.865187+0000","last_clean":"2023-11-27T15:16:10.865187+0000","last_became_active":"2023-11-27T15:16:09.338636+0000","last_became_peered":"2023-11-27T15:16:09.338636+0000","last_unstale":"2023-11-27T15:16:10.865187+0000","last_undegraded":"2023-11-27T15:16:10.865187+0000","last_fullsized":"2023-11-27T15:16:10.865187+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:30:06.448581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.865094+0000","last_change":"2023-11-27T15:16:09.347845+0000","last_active":"2023-11-27T15:16:10.865094+0000","last_peered":"2023-11-27T15:16:10.865094+0000","last_clean":"2023-11-27T15:16:10.865094+0000","last_became_active":"2023-11-27T15:16:09.347533+0000","last_became_peered":"2023-11-27T15:16:09.347533+0000","last_unstale":"2023-11-27T15:16:10.865094+0000","last_undegraded":"2023-11-27T15:16:10.865094+0000","last_fullsized":"2023-11-27T15:16:10.865094+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:01:40.820155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.019299+0000","last_change":"2023-11-27T15:16:09.333251+0000","last_active":"2023-11-27T15:16:11.019299+0000","last_peered":"2023-11-27T15:16:11.019299+0000","last_clean":"2023-11-27T15:16:11.019299+0000","last_became_active":"2023-11-27T15:16:09.331510+0000","last_became_peered":"2023-11-27T15:16:09.331510+0000","last_unstale":"2023-11-27T15:16:11.019299+0000","last_undegraded":"2023-11-27T15:16:11.019299+0000","last_fullsized":"2023-11-27T15:16:11.019299+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:34:53.253074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018096+0000","last_change":"2023-11-27T15:16:09.349097+0000","last_active":"2023-11-27T15:16:11.018096+0000","last_peered":"2023-11-27T15:16:11.018096+0000","last_clean":"2023-11-27T15:16:11.018096+0000","last_became_active":"2023-11-27T15:16:09.348793+0000","last_became_peered":"2023-11-27T15:16:09.348793+0000","last_unstale":"2023-11-27T15:16:11.018096+0000","last_undegraded":"2023-11-27T15:16:11.018096+0000","last_fullsized":"2023-11-27T15:16:11.018096+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:55:31.214202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.019174+0000","last_change":"2023-11-27T15:16:09.338246+0000","last_active":"2023-11-27T15:16:11.019174+0000","last_peered":"2023-11-27T15:16:11.019174+0000","last_clean":"2023-11-27T15:16:11.019174+0000","last_became_active":"2023-11-27T15:16:09.338003+0000","last_became_peered":"2023-11-27T15:16:09.338003+0000","last_unstale":"2023-11-27T15:16:11.019174+0000","last_undegraded":"2023-11-27T15:16:11.019174+0000","last_fullsized":"2023-11-27T15:16:11.019174+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:25:15.821446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017705+0000","last_change":"2023-11-27T15:16:09.342131+0000","last_active":"2023-11-27T15:16:11.017705+0000","last_peered":"2023-11-27T15:16:11.017705+0000","last_clean":"2023-11-27T15:16:11.017705+0000","last_became_active":"2023-11-27T15:16:09.341834+0000","last_became_peered":"2023-11-27T15:16:09.341834+0000","last_unstale":"2023-11-27T15:16:11.017705+0000","last_undegraded":"2023-11-27T15:16:11.017705+0000","last_fullsized":"2023-11-27T15:16:11.017705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:06:07.633604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018355+0000","last_change":"2023-11-27T15:16:09.337308+0000","last_active":"2023-11-27T15:16:11.018355+0000","last_peered":"2023-11-27T15:16:11.018355+0000","last_clean":"2023-11-27T15:16:11.018355+0000","last_became_active":"2023-11-27T15:16:09.336958+0000","last_became_peered":"2023-11-27T15:16:09.336958+0000","last_unstale":"2023-11-27T15:16:11.018355+0000","last_undegraded":"2023-11-27T15:16:11.018355+0000","last_fullsized":"2023-11-27T15:16:11.018355+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:00:54.104008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.332789+0000","last_change":"2023-11-27T15:16:09.340646+0000","last_active":"2023-11-27T15:16:10.332789+0000","last_peered":"2023-11-27T15:16:10.332789+0000","last_clean":"2023-11-27T15:16:10.332789+0000","last_became_active":"2023-11-27T15:16:09.340084+0000","last_became_peered":"2023-11-27T15:16:09.340084+0000","last_unstale":"2023-11-27T15:16:10.332789+0000","last_undegraded":"2023-11-27T15:16:10.332789+0000","last_fullsized":"2023-11-27T15:16:10.332789+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:33:29.581983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.019348+0000","last_change":"2023-11-27T15:16:09.336871+0000","last_active":"2023-11-27T15:16:11.019348+0000","last_peered":"2023-11-27T15:16:11.019348+0000","last_clean":"2023-11-27T15:16:11.019348+0000","last_became_active":"2023-11-27T15:16:09.336496+0000","last_became_peered":"2023-11-27T15:16:09.336496+0000","last_unstale":"2023-11-27T15:16:11.019348+0000","last_undegraded":"2023-11-27T15:16:11.019348+0000","last_fullsized":"2023-11-27T15:16:11.019348+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:37:00.015095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018026+0000","last_change":"2023-11-27T15:16:09.343120+0000","last_active":"2023-11-27T15:16:11.018026+0000","last_peered":"2023-11-27T15:16:11.018026+0000","last_clean":"2023-11-27T15:16:11.018026+0000","last_became_active":"2023-11-27T15:16:09.342797+0000","last_became_peered":"2023-11-27T15:16:09.342797+0000","last_unstale":"2023-11-27T15:16:11.018026+0000","last_undegraded":"2023-11-27T15:16:11.018026+0000","last_fullsized":"2023-11-27T15:16:11.018026+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:39:21.071837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.019248+0000","last_change":"2023-11-27T15:16:09.336521+0000","last_active":"2023-11-27T15:16:11.019248+0000","last_peered":"2023-11-27T15:16:11.019248+0000","last_clean":"2023-11-27T15:16:11.019248+0000","last_became_active":"2023-11-27T15:16:09.336183+0000","last_became_peered":"2023-11-27T15:16:09.336183+0000","last_unstale":"2023-11-27T15:16:11.019248+0000","last_undegraded":"2023-11-27T15:16:11.019248+0000","last_fullsized":"2023-11-27T15:16:11.019248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:25:55.091755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018282+0000","last_change":"2023-11-27T15:16:09.346839+0000","last_active":"2023-11-27T15:16:11.018282+0000","last_peered":"2023-11-27T15:16:11.018282+0000","last_clean":"2023-11-27T15:16:11.018282+0000","last_became_active":"2023-11-27T15:16:09.345230+0000","last_became_peered":"2023-11-27T15:16:09.345230+0000","last_unstale":"2023-11-27T15:16:11.018282+0000","last_undegraded":"2023-11-27T15:16:11.018282+0000","last_fullsized":"2023-11-27T15:16:11.018282+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:22:56.772626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.864881+0000","last_change":"2023-11-27T15:16:09.342380+0000","last_active":"2023-11-27T15:16:10.864881+0000","last_peered":"2023-11-27T15:16:10.864881+0000","last_clean":"2023-11-27T15:16:10.864881+0000","last_became_active":"2023-11-27T15:16:09.341967+0000","last_became_peered":"2023-11-27T15:16:09.341967+0000","last_unstale":"2023-11-27T15:16:10.864881+0000","last_undegraded":"2023-11-27T15:16:10.864881+0000","last_fullsized":"2023-11-27T15:16:10.864881+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:35:13.999971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.864779+0000","last_change":"2023-11-27T15:16:09.349738+0000","last_active":"2023-11-27T15:16:10.864779+0000","last_peered":"2023-11-27T15:16:10.864779+0000","last_clean":"2023-11-27T15:16:10.864779+0000","last_became_active":"2023-11-27T15:16:09.349349+0000","last_became_peered":"2023-11-27T15:16:09.349349+0000","last_unstale":"2023-11-27T15:16:10.864779+0000","last_undegraded":"2023-11-27T15:16:10.864779+0000","last_fullsized":"2023-11-27T15:16:10.864779+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:13:03.443885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.332227+0000","last_change":"2023-11-27T15:16:09.339171+0000","last_active":"2023-11-27T15:16:10.332227+0000","last_peered":"2023-11-27T15:16:10.332227+0000","last_clean":"2023-11-27T15:16:10.332227+0000","last_became_active":"2023-11-27T15:16:09.338887+0000","last_became_peered":"2023-11-27T15:16:09.338887+0000","last_unstale":"2023-11-27T15:16:10.332227+0000","last_undegraded":"2023-11-27T15:16:10.332227+0000","last_fullsized":"2023-11-27T15:16:10.332227+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:42:45.286967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329413+0000","last_change":"2023-11-27T15:16:09.343202+0000","last_active":"2023-11-27T15:16:10.329413+0000","last_peered":"2023-11-27T15:16:10.329413+0000","last_clean":"2023-11-27T15:16:10.329413+0000","last_became_active":"2023-11-27T15:16:09.342539+0000","last_became_peered":"2023-11-27T15:16:09.342539+0000","last_unstale":"2023-11-27T15:16:10.329413+0000","last_undegraded":"2023-11-27T15:16:10.329413+0000","last_fullsized":"2023-11-27T15:16:10.329413+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:31:31.313017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018587+0000","last_change":"2023-11-27T15:16:09.339915+0000","last_active":"2023-11-27T15:16:11.018587+0000","last_peered":"2023-11-27T15:16:11.018587+0000","last_clean":"2023-11-27T15:16:11.018587+0000","last_became_active":"2023-11-27T15:16:09.339637+0000","last_became_peered":"2023-11-27T15:16:09.339637+0000","last_unstale":"2023-11-27T15:16:11.018587+0000","last_undegraded":"2023-11-27T15:16:11.018587+0000","last_fullsized":"2023-11-27T15:16:11.018587+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:37:13.698277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329316+0000","last_change":"2023-11-27T15:16:09.347852+0000","last_active":"2023-11-27T15:16:10.329316+0000","last_peered":"2023-11-27T15:16:10.329316+0000","last_clean":"2023-11-27T15:16:10.329316+0000","last_became_active":"2023-11-27T15:16:09.347281+0000","last_became_peered":"2023-11-27T15:16:09.347281+0000","last_unstale":"2023-11-27T15:16:10.329316+0000","last_undegraded":"2023-11-27T15:16:10.329316+0000","last_fullsized":"2023-11-27T15:16:10.329316+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:54:00.346911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:13.751316+0000","last_change":"2023-11-27T15:16:08.912206+0000","last_active":"2023-11-27T15:16:13.751316+0000","last_peered":"2023-11-27T15:16:13.751316+0000","last_clean":"2023-11-27T15:16:13.751316+0000","last_became_active":"2023-11-27T15:16:08.911846+0000","last_became_peered":"2023-11-27T15:16:08.911846+0000","last_unstale":"2023-11-27T15:16:13.751316+0000","last_undegraded":"2023-11-27T15:16:13.751316+0000","last_fullsized":"2023-11-27T15:16:13.751316+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:21:37.095544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018680+0000","last_change":"2023-11-27T15:16:09.336485+0000","last_active":"2023-11-27T15:16:11.018680+0000","last_peered":"2023-11-27T15:16:11.018680+0000","last_clean":"2023-11-27T15:16:11.018680+0000","last_became_active":"2023-11-27T15:16:09.336236+0000","last_became_peered":"2023-11-27T15:16:09.336236+0000","last_unstale":"2023-11-27T15:16:11.018680+0000","last_undegraded":"2023-11-27T15:16:11.018680+0000","last_fullsized":"2023-11-27T15:16:11.018680+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:00:31.457583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018872+0000","last_change":"2023-11-27T15:16:08.921198+0000","last_active":"2023-11-27T15:16:11.018872+0000","last_peered":"2023-11-27T15:16:11.018872+0000","last_clean":"2023-11-27T15:16:11.018872+0000","last_became_active":"2023-11-27T15:16:08.920804+0000","last_became_peered":"2023-11-27T15:16:08.920804+0000","last_unstale":"2023-11-27T15:16:11.018872+0000","last_undegraded":"2023-11-27T15:16:11.018872+0000","last_fullsized":"2023-11-27T15:16:11.018872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:25:26.133784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.865444+0000","last_change":"2023-11-27T15:16:09.340642+0000","last_active":"2023-11-27T15:16:10.865444+0000","last_peered":"2023-11-27T15:16:10.865444+0000","last_clean":"2023-11-27T15:16:10.865444+0000","last_became_active":"2023-11-27T15:16:09.340328+0000","last_became_peered":"2023-11-27T15:16:09.340328+0000","last_unstale":"2023-11-27T15:16:10.865444+0000","last_undegraded":"2023-11-27T15:16:10.865444+0000","last_fullsized":"2023-11-27T15:16:10.865444+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:48:12.279600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:13.748861+0000","last_change":"2023-11-27T15:16:08.915317+0000","last_active":"2023-11-27T15:16:13.748861+0000","last_peered":"2023-11-27T15:16:13.748861+0000","last_clean":"2023-11-27T15:16:13.748861+0000","last_became_active":"2023-11-27T15:16:08.914949+0000","last_became_peered":"2023-11-27T15:16:08.914949+0000","last_unstale":"2023-11-27T15:16:13.748861+0000","last_undegraded":"2023-11-27T15:16:13.748861+0000","last_fullsized":"2023-11-27T15:16:13.748861+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:35:47.443716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.332402+0000","last_change":"2023-11-27T15:16:09.348483+0000","last_active":"2023-11-27T15:16:10.332402+0000","last_peered":"2023-11-27T15:16:10.332402+0000","last_clean":"2023-11-27T15:16:10.332402+0000","last_became_active":"2023-11-27T15:16:09.348185+0000","last_became_peered":"2023-11-27T15:16:09.348185+0000","last_unstale":"2023-11-27T15:16:10.332402+0000","last_undegraded":"2023-11-27T15:16:10.332402+0000","last_fullsized":"2023-11-27T15:16:10.332402+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:23:59.373819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018251+0000","last_change":"2023-11-27T15:16:08.911018+0000","last_active":"2023-11-27T15:16:11.018251+0000","last_peered":"2023-11-27T15:16:11.018251+0000","last_clean":"2023-11-27T15:16:11.018251+0000","last_became_active":"2023-11-27T15:16:08.910684+0000","last_became_peered":"2023-11-27T15:16:08.910684+0000","last_unstale":"2023-11-27T15:16:11.018251+0000","last_undegraded":"2023-11-27T15:16:11.018251+0000","last_fullsized":"2023-11-27T15:16:11.018251+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:17:37.062173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018419+0000","last_change":"2023-11-27T15:16:09.341414+0000","last_active":"2023-11-27T15:16:11.018419+0000","last_peered":"2023-11-27T15:16:11.018419+0000","last_clean":"2023-11-27T15:16:11.018419+0000","last_became_active":"2023-11-27T15:16:09.341068+0000","last_became_peered":"2023-11-27T15:16:09.341068+0000","last_unstale":"2023-11-27T15:16:11.018419+0000","last_undegraded":"2023-11-27T15:16:11.018419+0000","last_fullsized":"2023-11-27T15:16:11.018419+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:08:37.252539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":63,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:51.797116+0000","last_change":"2023-11-27T15:16:08.915093+0000","last_active":"2023-11-27T15:16:51.797116+0000","last_peered":"2023-11-27T15:16:51.797116+0000","last_clean":"2023-11-27T15:16:51.797116+0000","last_became_active":"2023-11-27T15:16:08.914761+0000","last_became_peered":"2023-11-27T15:16:08.914761+0000","last_unstale":"2023-11-27T15:16:51.797116+0000","last_undegraded":"2023-11-27T15:16:51.797116+0000","last_fullsized":"2023-11-27T15:16:51.797116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:36:34.653519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018143+0000","last_change":"2023-11-27T15:16:09.337330+0000","last_active":"2023-11-27T15:16:11.018143+0000","last_peered":"2023-11-27T15:16:11.018143+0000","last_clean":"2023-11-27T15:16:11.018143+0000","last_became_active":"2023-11-27T15:16:09.336991+0000","last_became_peered":"2023-11-27T15:16:09.336991+0000","last_unstale":"2023-11-27T15:16:11.018143+0000","last_undegraded":"2023-11-27T15:16:11.018143+0000","last_fullsized":"2023-11-27T15:16:11.018143+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:12:42.839632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:13.748619+0000","last_change":"2023-11-27T15:16:08.912208+0000","last_active":"2023-11-27T15:16:13.748619+0000","last_peered":"2023-11-27T15:16:13.748619+0000","last_clean":"2023-11-27T15:16:13.748619+0000","last_became_active":"2023-11-27T15:16:08.911879+0000","last_became_peered":"2023-11-27T15:16:08.911879+0000","last_unstale":"2023-11-27T15:16:13.748619+0000","last_undegraded":"2023-11-27T15:16:13.748619+0000","last_fullsized":"2023-11-27T15:16:13.748619+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:05:04.291584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018417+0000","last_change":"2023-11-27T15:16:09.339660+0000","last_active":"2023-11-27T15:16:11.018417+0000","last_peered":"2023-11-27T15:16:11.018417+0000","last_clean":"2023-11-27T15:16:11.018417+0000","last_became_active":"2023-11-27T15:16:09.339323+0000","last_became_peered":"2023-11-27T15:16:09.339323+0000","last_unstale":"2023-11-27T15:16:11.018417+0000","last_undegraded":"2023-11-27T15:16:11.018417+0000","last_fullsized":"2023-11-27T15:16:11.018417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:54:38.570422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:13.750343+0000","last_change":"2023-11-27T15:16:08.911103+0000","last_active":"2023-11-27T15:16:13.750343+0000","last_peered":"2023-11-27T15:16:13.750343+0000","last_clean":"2023-11-27T15:16:13.750343+0000","last_became_active":"2023-11-27T15:16:08.910765+0000","last_became_peered":"2023-11-27T15:16:08.910765+0000","last_unstale":"2023-11-27T15:16:13.750343+0000","last_undegraded":"2023-11-27T15:16:13.750343+0000","last_fullsized":"2023-11-27T15:16:13.750343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:35:58.046428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018483+0000","last_change":"2023-11-27T15:16:09.351741+0000","last_active":"2023-11-27T15:16:11.018483+0000","last_peered":"2023-11-27T15:16:11.018483+0000","last_clean":"2023-11-27T15:16:11.018483+0000","last_became_active":"2023-11-27T15:16:09.351280+0000","last_became_peered":"2023-11-27T15:16:09.351280+0000","last_unstale":"2023-11-27T15:16:11.018483+0000","last_undegraded":"2023-11-27T15:16:11.018483+0000","last_fullsized":"2023-11-27T15:16:11.018483+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:23:19.955510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018060+0000","last_change":"2023-11-27T15:16:08.915630+0000","last_active":"2023-11-27T15:16:11.018060+0000","last_peered":"2023-11-27T15:16:11.018060+0000","last_clean":"2023-11-27T15:16:11.018060+0000","last_became_active":"2023-11-27T15:16:08.915294+0000","last_became_peered":"2023-11-27T15:16:08.915294+0000","last_unstale":"2023-11-27T15:16:11.018060+0000","last_undegraded":"2023-11-27T15:16:11.018060+0000","last_fullsized":"2023-11-27T15:16:11.018060+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:48:25.055189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018460+0000","last_change":"2023-11-27T15:16:09.338436+0000","last_active":"2023-11-27T15:16:11.018460+0000","last_peered":"2023-11-27T15:16:11.018460+0000","last_clean":"2023-11-27T15:16:11.018460+0000","last_became_active":"2023-11-27T15:16:09.338148+0000","last_became_peered":"2023-11-27T15:16:09.338148+0000","last_unstale":"2023-11-27T15:16:11.018460+0000","last_undegraded":"2023-11-27T15:16:11.018460+0000","last_fullsized":"2023-11-27T15:16:11.018460+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:11:53.119360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":57,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:51.796830+0000","last_change":"2023-11-27T15:16:08.913072+0000","last_active":"2023-11-27T15:16:51.796830+0000","last_peered":"2023-11-27T15:16:51.796830+0000","last_clean":"2023-11-27T15:16:51.796830+0000","last_became_active":"2023-11-27T15:16:08.912757+0000","last_became_peered":"2023-11-27T15:16:08.912757+0000","last_unstale":"2023-11-27T15:16:51.796830+0000","last_undegraded":"2023-11-27T15:16:51.796830+0000","last_fullsized":"2023-11-27T15:16:51.796830+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:41:31.325128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.332666+0000","last_change":"2023-11-27T15:16:09.340380+0000","last_active":"2023-11-27T15:16:10.332666+0000","last_peered":"2023-11-27T15:16:10.332666+0000","last_clean":"2023-11-27T15:16:10.332666+0000","last_became_active":"2023-11-27T15:16:09.339581+0000","last_became_peered":"2023-11-27T15:16:09.339581+0000","last_unstale":"2023-11-27T15:16:10.332666+0000","last_undegraded":"2023-11-27T15:16:10.332666+0000","last_fullsized":"2023-11-27T15:16:10.332666+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:00:41.810407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.743621+0000","last_change":"2023-11-27T15:16:08.917924+0000","last_active":"2023-11-27T15:16:12.743621+0000","last_peered":"2023-11-27T15:16:12.743621+0000","last_clean":"2023-11-27T15:16:12.743621+0000","last_became_active":"2023-11-27T15:16:08.917618+0000","last_became_peered":"2023-11-27T15:16:08.917618+0000","last_unstale":"2023-11-27T15:16:12.743621+0000","last_undegraded":"2023-11-27T15:16:12.743621+0000","last_fullsized":"2023-11-27T15:16:12.743621+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:05:48.700431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.865352+0000","last_change":"2023-11-27T15:16:09.346020+0000","last_active":"2023-11-27T15:16:10.865352+0000","last_peered":"2023-11-27T15:16:10.865352+0000","last_clean":"2023-11-27T15:16:10.865352+0000","last_became_active":"2023-11-27T15:16:09.345822+0000","last_became_peered":"2023-11-27T15:16:09.345822+0000","last_unstale":"2023-11-27T15:16:10.865352+0000","last_undegraded":"2023-11-27T15:16:10.865352+0000","last_fullsized":"2023-11-27T15:16:10.865352+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:20:28.114199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.742118+0000","last_change":"2023-11-27T15:16:08.921810+0000","last_active":"2023-11-27T15:16:12.742118+0000","last_peered":"2023-11-27T15:16:12.742118+0000","last_clean":"2023-11-27T15:16:12.742118+0000","last_became_active":"2023-11-27T15:16:08.921518+0000","last_became_peered":"2023-11-27T15:16:08.921518+0000","last_unstale":"2023-11-27T15:16:12.742118+0000","last_undegraded":"2023-11-27T15:16:12.742118+0000","last_fullsized":"2023-11-27T15:16:12.742118+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:14:14.359239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.332512+0000","last_change":"2023-11-27T15:16:09.351083+0000","last_active":"2023-11-27T15:16:10.332512+0000","last_peered":"2023-11-27T15:16:10.332512+0000","last_clean":"2023-11-27T15:16:10.332512+0000","last_became_active":"2023-11-27T15:16:09.350540+0000","last_became_peered":"2023-11-27T15:16:09.350540+0000","last_unstale":"2023-11-27T15:16:10.332512+0000","last_undegraded":"2023-11-27T15:16:10.332512+0000","last_fullsized":"2023-11-27T15:16:10.332512+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:16:25.645689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'28","reported_seq":130,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:52.798354+0000","last_change":"2023-11-27T15:16:08.913023+0000","last_active":"2023-11-27T15:16:52.798354+0000","last_peered":"2023-11-27T15:16:52.798354+0000","last_clean":"2023-11-27T15:16:52.798354+0000","last_became_active":"2023-11-27T15:16:08.912693+0000","last_became_peered":"2023-11-27T15:16:08.912693+0000","last_unstale":"2023-11-27T15:16:52.798354+0000","last_undegraded":"2023-11-27T15:16:52.798354+0000","last_fullsized":"2023-11-27T15:16:52.798354+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:02:06.453103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58527,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":88,"num_read_kb":100,"num_write":29,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018683+0000","last_change":"2023-11-27T15:16:09.343446+0000","last_active":"2023-11-27T15:16:11.018683+0000","last_peered":"2023-11-27T15:16:11.018683+0000","last_clean":"2023-11-27T15:16:11.018683+0000","last_became_active":"2023-11-27T15:16:09.341980+0000","last_became_peered":"2023-11-27T15:16:09.341980+0000","last_unstale":"2023-11-27T15:16:11.018683+0000","last_undegraded":"2023-11-27T15:16:11.018683+0000","last_fullsized":"2023-11-27T15:16:11.018683+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:03:09.462489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:13.749211+0000","last_change":"2023-11-27T15:16:08.914181+0000","last_active":"2023-11-27T15:16:13.749211+0000","last_peered":"2023-11-27T15:16:13.749211+0000","last_clean":"2023-11-27T15:16:13.749211+0000","last_became_active":"2023-11-27T15:16:08.913836+0000","last_became_peered":"2023-11-27T15:16:08.913836+0000","last_unstale":"2023-11-27T15:16:13.749211+0000","last_undegraded":"2023-11-27T15:16:13.749211+0000","last_fullsized":"2023-11-27T15:16:13.749211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:28:48.961843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018010+0000","last_change":"2023-11-27T15:16:09.344446+0000","last_active":"2023-11-27T15:16:11.018010+0000","last_peered":"2023-11-27T15:16:11.018010+0000","last_clean":"2023-11-27T15:16:11.018010+0000","last_became_active":"2023-11-27T15:16:09.344104+0000","last_became_peered":"2023-11-27T15:16:09.344104+0000","last_unstale":"2023-11-27T15:16:11.018010+0000","last_undegraded":"2023-11-27T15:16:11.018010+0000","last_fullsized":"2023-11-27T15:16:11.018010+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:09:38.847885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.744575+0000","last_change":"2023-11-27T15:16:08.915597+0000","last_active":"2023-11-27T15:16:12.744575+0000","last_peered":"2023-11-27T15:16:12.744575+0000","last_clean":"2023-11-27T15:16:12.744575+0000","last_became_active":"2023-11-27T15:16:08.914727+0000","last_became_peered":"2023-11-27T15:16:08.914727+0000","last_unstale":"2023-11-27T15:16:12.744575+0000","last_undegraded":"2023-11-27T15:16:12.744575+0000","last_fullsized":"2023-11-27T15:16:12.744575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:32:36.396885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017899+0000","last_change":"2023-11-27T15:16:09.352643+0000","last_active":"2023-11-27T15:16:11.017899+0000","last_peered":"2023-11-27T15:16:11.017899+0000","last_clean":"2023-11-27T15:16:11.017899+0000","last_became_active":"2023-11-27T15:16:09.352336+0000","last_became_peered":"2023-11-27T15:16:09.352336+0000","last_unstale":"2023-11-27T15:16:11.017899+0000","last_undegraded":"2023-11-27T15:16:11.017899+0000","last_fullsized":"2023-11-27T15:16:11.017899+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:27:00.130120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'4","reported_seq":63,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:54.825055+0000","last_change":"2023-11-27T15:16:08.914303+0000","last_active":"2023-11-27T15:16:54.825055+0000","last_peered":"2023-11-27T15:16:54.825055+0000","last_clean":"2023-11-27T15:16:54.825055+0000","last_became_active":"2023-11-27T15:16:08.913601+0000","last_became_peered":"2023-11-27T15:16:08.913601+0000","last_unstale":"2023-11-27T15:16:54.825055+0000","last_undegraded":"2023-11-27T15:16:54.825055+0000","last_fullsized":"2023-11-27T15:16:54.825055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:35:53.186395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017672+0000","last_change":"2023-11-27T15:16:09.341342+0000","last_active":"2023-11-27T15:16:11.017672+0000","last_peered":"2023-11-27T15:16:11.017672+0000","last_clean":"2023-11-27T15:16:11.017672+0000","last_became_active":"2023-11-27T15:16:09.341100+0000","last_became_peered":"2023-11-27T15:16:09.341100+0000","last_unstale":"2023-11-27T15:16:11.017672+0000","last_undegraded":"2023-11-27T15:16:11.017672+0000","last_fullsized":"2023-11-27T15:16:11.017672+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:52:19.668174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:13.750916+0000","last_change":"2023-11-27T15:16:08.921771+0000","last_active":"2023-11-27T15:16:13.750916+0000","last_peered":"2023-11-27T15:16:13.750916+0000","last_clean":"2023-11-27T15:16:13.750916+0000","last_became_active":"2023-11-27T15:16:08.921396+0000","last_became_peered":"2023-11-27T15:16:08.921396+0000","last_unstale":"2023-11-27T15:16:13.750916+0000","last_undegraded":"2023-11-27T15:16:13.750916+0000","last_fullsized":"2023-11-27T15:16:13.750916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:32:00.774950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018456+0000","last_change":"2023-11-27T15:16:09.341258+0000","last_active":"2023-11-27T15:16:11.018456+0000","last_peered":"2023-11-27T15:16:11.018456+0000","last_clean":"2023-11-27T15:16:11.018456+0000","last_became_active":"2023-11-27T15:16:09.340921+0000","last_became_peered":"2023-11-27T15:16:09.340921+0000","last_unstale":"2023-11-27T15:16:11.018456+0000","last_undegraded":"2023-11-27T15:16:11.018456+0000","last_fullsized":"2023-11-27T15:16:11.018456+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:59:25.144109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:13.738069+0000","last_change":"2023-11-27T15:16:08.911219+0000","last_active":"2023-11-27T15:16:13.738069+0000","last_peered":"2023-11-27T15:16:13.738069+0000","last_clean":"2023-11-27T15:16:13.738069+0000","last_became_active":"2023-11-27T15:16:08.910788+0000","last_became_peered":"2023-11-27T15:16:08.910788+0000","last_unstale":"2023-11-27T15:16:13.738069+0000","last_undegraded":"2023-11-27T15:16:13.738069+0000","last_fullsized":"2023-11-27T15:16:13.738069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:02:11.224624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329287+0000","last_change":"2023-11-27T15:16:09.343793+0000","last_active":"2023-11-27T15:16:10.329287+0000","last_peered":"2023-11-27T15:16:10.329287+0000","last_clean":"2023-11-27T15:16:10.329287+0000","last_became_active":"2023-11-27T15:16:09.343480+0000","last_became_peered":"2023-11-27T15:16:09.343480+0000","last_unstale":"2023-11-27T15:16:10.329287+0000","last_undegraded":"2023-11-27T15:16:10.329287+0000","last_fullsized":"2023-11-27T15:16:10.329287+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:23:15.936537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018282+0000","last_change":"2023-11-27T15:16:08.920651+0000","last_active":"2023-11-27T15:16:11.018282+0000","last_peered":"2023-11-27T15:16:11.018282+0000","last_clean":"2023-11-27T15:16:11.018282+0000","last_became_active":"2023-11-27T15:16:08.920294+0000","last_became_peered":"2023-11-27T15:16:08.920294+0000","last_unstale":"2023-11-27T15:16:11.018282+0000","last_undegraded":"2023-11-27T15:16:11.018282+0000","last_fullsized":"2023-11-27T15:16:11.018282+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:57:51.796562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:13.149060+0000","last_change":"2023-11-27T15:16:09.348279+0000","last_active":"2023-11-27T15:16:13.149060+0000","last_peered":"2023-11-27T15:16:13.149060+0000","last_clean":"2023-11-27T15:16:13.149060+0000","last_became_active":"2023-11-27T15:16:09.347760+0000","last_became_peered":"2023-11-27T15:16:09.347760+0000","last_unstale":"2023-11-27T15:16:13.149060+0000","last_undegraded":"2023-11-27T15:16:13.149060+0000","last_fullsized":"2023-11-27T15:16:13.149060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:35:32.477154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.170186+0000","last_change":"2023-11-27T15:16:08.915622+0000","last_active":"2023-11-27T15:16:11.170186+0000","last_peered":"2023-11-27T15:16:11.170186+0000","last_clean":"2023-11-27T15:16:11.170186+0000","last_became_active":"2023-11-27T15:16:08.915293+0000","last_became_peered":"2023-11-27T15:16:08.915293+0000","last_unstale":"2023-11-27T15:16:11.170186+0000","last_undegraded":"2023-11-27T15:16:11.170186+0000","last_fullsized":"2023-11-27T15:16:11.170186+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:42:31.199595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.019331+0000","last_change":"2023-11-27T15:16:09.337096+0000","last_active":"2023-11-27T15:16:11.019331+0000","last_peered":"2023-11-27T15:16:11.019331+0000","last_clean":"2023-11-27T15:16:11.019331+0000","last_became_active":"2023-11-27T15:16:09.336359+0000","last_became_peered":"2023-11-27T15:16:09.336359+0000","last_unstale":"2023-11-27T15:16:11.019331+0000","last_undegraded":"2023-11-27T15:16:11.019331+0000","last_fullsized":"2023-11-27T15:16:11.019331+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:01:29.480942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018063+0000","last_change":"2023-11-27T15:16:09.340478+0000","last_active":"2023-11-27T15:16:11.018063+0000","last_peered":"2023-11-27T15:16:11.018063+0000","last_clean":"2023-11-27T15:16:11.018063+0000","last_became_active":"2023-11-27T15:16:09.340214+0000","last_became_peered":"2023-11-27T15:16:09.340214+0000","last_unstale":"2023-11-27T15:16:11.018063+0000","last_undegraded":"2023-11-27T15:16:11.018063+0000","last_fullsized":"2023-11-27T15:16:11.018063+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:09:06.499851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":58,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:52.819769+0000","last_change":"2023-11-27T15:16:08.920387+0000","last_active":"2023-11-27T15:16:52.819769+0000","last_peered":"2023-11-27T15:16:52.819769+0000","last_clean":"2023-11-27T15:16:52.819769+0000","last_became_active":"2023-11-27T15:16:08.920004+0000","last_became_peered":"2023-11-27T15:16:08.920004+0000","last_unstale":"2023-11-27T15:16:52.819769+0000","last_undegraded":"2023-11-27T15:16:52.819769+0000","last_fullsized":"2023-11-27T15:16:52.819769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:02:23.895980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018879+0000","last_change":"2023-11-27T15:16:09.352291+0000","last_active":"2023-11-27T15:16:11.018879+0000","last_peered":"2023-11-27T15:16:11.018879+0000","last_clean":"2023-11-27T15:16:11.018879+0000","last_became_active":"2023-11-27T15:16:09.351944+0000","last_became_peered":"2023-11-27T15:16:09.351944+0000","last_unstale":"2023-11-27T15:16:11.018879+0000","last_undegraded":"2023-11-27T15:16:11.018879+0000","last_fullsized":"2023-11-27T15:16:11.018879+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:59:40.185533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:13.748748+0000","last_change":"2023-11-27T15:16:08.909694+0000","last_active":"2023-11-27T15:16:13.748748+0000","last_peered":"2023-11-27T15:16:13.748748+0000","last_clean":"2023-11-27T15:16:13.748748+0000","last_became_active":"2023-11-27T15:16:08.909324+0000","last_became_peered":"2023-11-27T15:16:08.909324+0000","last_unstale":"2023-11-27T15:16:13.748748+0000","last_undegraded":"2023-11-27T15:16:13.748748+0000","last_fullsized":"2023-11-27T15:16:13.748748+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:37:52.187833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":340,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017780+0000","last_change":"2023-11-27T15:15:10.678805+0000","last_active":"2023-11-27T15:16:11.017780+0000","last_peered":"2023-11-27T15:16:11.017780+0000","last_clean":"2023-11-27T15:16:11.017780+0000","last_became_active":"2023-11-27T15:15:10.678267+0000","last_became_peered":"2023-11-27T15:15:10.678267+0000","last_unstale":"2023-11-27T15:16:11.017780+0000","last_undegraded":"2023-11-27T15:16:11.017780+0000","last_fullsized":"2023-11-27T15:16:11.017780+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:12:14.179315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:12:14.179315+0000","last_clean_scrub_stamp":"2023-11-27T15:12:14.179315+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:53:29.305985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017431+0000","last_change":"2023-11-27T15:16:09.346622+0000","last_active":"2023-11-27T15:16:11.017431+0000","last_peered":"2023-11-27T15:16:11.017431+0000","last_clean":"2023-11-27T15:16:11.017431+0000","last_became_active":"2023-11-27T15:16:09.346287+0000","last_became_peered":"2023-11-27T15:16:09.346287+0000","last_unstale":"2023-11-27T15:16:11.017431+0000","last_undegraded":"2023-11-27T15:16:11.017431+0000","last_fullsized":"2023-11-27T15:16:11.017431+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:06:51.424504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:13.749235+0000","last_change":"2023-11-27T15:16:08.915598+0000","last_active":"2023-11-27T15:16:13.749235+0000","last_peered":"2023-11-27T15:16:13.749235+0000","last_clean":"2023-11-27T15:16:13.749235+0000","last_became_active":"2023-11-27T15:16:08.915255+0000","last_became_peered":"2023-11-27T15:16:08.915255+0000","last_unstale":"2023-11-27T15:16:13.749235+0000","last_undegraded":"2023-11-27T15:16:13.749235+0000","last_fullsized":"2023-11-27T15:16:13.749235+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:59:36.190517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.743609+0000","last_change":"2023-11-27T15:16:08.921855+0000","last_active":"2023-11-27T15:16:12.743609+0000","last_peered":"2023-11-27T15:16:12.743609+0000","last_clean":"2023-11-27T15:16:12.743609+0000","last_became_active":"2023-11-27T15:16:08.921525+0000","last_became_peered":"2023-11-27T15:16:08.921525+0000","last_unstale":"2023-11-27T15:16:12.743609+0000","last_undegraded":"2023-11-27T15:16:12.743609+0000","last_fullsized":"2023-11-27T15:16:12.743609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:18:46.190023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017617+0000","last_change":"2023-11-27T15:16:09.345119+0000","last_active":"2023-11-27T15:16:11.017617+0000","last_peered":"2023-11-27T15:16:11.017617+0000","last_clean":"2023-11-27T15:16:11.017617+0000","last_became_active":"2023-11-27T15:16:09.344807+0000","last_became_peered":"2023-11-27T15:16:09.344807+0000","last_unstale":"2023-11-27T15:16:11.017617+0000","last_undegraded":"2023-11-27T15:16:11.017617+0000","last_fullsized":"2023-11-27T15:16:11.017617+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:35:43.938240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:19.643866+0000","last_change":"2023-11-27T15:16:08.911564+0000","last_active":"2023-11-27T15:16:19.643866+0000","last_peered":"2023-11-27T15:16:19.643866+0000","last_clean":"2023-11-27T15:16:19.643866+0000","last_became_active":"2023-11-27T15:16:08.911057+0000","last_became_peered":"2023-11-27T15:16:08.911057+0000","last_unstale":"2023-11-27T15:16:19.643866+0000","last_undegraded":"2023-11-27T15:16:19.643866+0000","last_fullsized":"2023-11-27T15:16:19.643866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:12:28.660065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018236+0000","last_change":"2023-11-27T15:16:09.339085+0000","last_active":"2023-11-27T15:16:11.018236+0000","last_peered":"2023-11-27T15:16:11.018236+0000","last_clean":"2023-11-27T15:16:11.018236+0000","last_became_active":"2023-11-27T15:16:09.338777+0000","last_became_peered":"2023-11-27T15:16:09.338777+0000","last_unstale":"2023-11-27T15:16:11.018236+0000","last_undegraded":"2023-11-27T15:16:11.018236+0000","last_fullsized":"2023-11-27T15:16:11.018236+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:37:58.241374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.744087+0000","last_change":"2023-11-27T15:16:08.913939+0000","last_active":"2023-11-27T15:16:12.744087+0000","last_peered":"2023-11-27T15:16:12.744087+0000","last_clean":"2023-11-27T15:16:12.744087+0000","last_became_active":"2023-11-27T15:16:08.913629+0000","last_became_peered":"2023-11-27T15:16:08.913629+0000","last_unstale":"2023-11-27T15:16:12.744087+0000","last_undegraded":"2023-11-27T15:16:12.744087+0000","last_fullsized":"2023-11-27T15:16:12.744087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:46:10.524140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.865269+0000","last_change":"2023-11-27T15:16:09.350712+0000","last_active":"2023-11-27T15:16:10.865269+0000","last_peered":"2023-11-27T15:16:10.865269+0000","last_clean":"2023-11-27T15:16:10.865269+0000","last_became_active":"2023-11-27T15:16:09.350359+0000","last_became_peered":"2023-11-27T15:16:09.350359+0000","last_unstale":"2023-11-27T15:16:10.865269+0000","last_undegraded":"2023-11-27T15:16:10.865269+0000","last_fullsized":"2023-11-27T15:16:10.865269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:30:08.385906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.744120+0000","last_change":"2023-11-27T15:16:08.919501+0000","last_active":"2023-11-27T15:16:12.744120+0000","last_peered":"2023-11-27T15:16:12.744120+0000","last_clean":"2023-11-27T15:16:12.744120+0000","last_became_active":"2023-11-27T15:16:08.919258+0000","last_became_peered":"2023-11-27T15:16:08.919258+0000","last_unstale":"2023-11-27T15:16:12.744120+0000","last_undegraded":"2023-11-27T15:16:12.744120+0000","last_fullsized":"2023-11-27T15:16:12.744120+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:12:35.863121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017921+0000","last_change":"2023-11-27T15:16:09.344066+0000","last_active":"2023-11-27T15:16:11.017921+0000","last_peered":"2023-11-27T15:16:11.017921+0000","last_clean":"2023-11-27T15:16:11.017921+0000","last_became_active":"2023-11-27T15:16:09.343757+0000","last_became_peered":"2023-11-27T15:16:09.343757+0000","last_unstale":"2023-11-27T15:16:11.017921+0000","last_undegraded":"2023-11-27T15:16:11.017921+0000","last_fullsized":"2023-11-27T15:16:11.017921+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:45:10.744579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.744748+0000","last_change":"2023-11-27T15:16:08.906205+0000","last_active":"2023-11-27T15:16:12.744748+0000","last_peered":"2023-11-27T15:16:12.744748+0000","last_clean":"2023-11-27T15:16:12.744748+0000","last_became_active":"2023-11-27T15:16:08.905885+0000","last_became_peered":"2023-11-27T15:16:08.905885+0000","last_unstale":"2023-11-27T15:16:12.744748+0000","last_undegraded":"2023-11-27T15:16:12.744748+0000","last_fullsized":"2023-11-27T15:16:12.744748+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:15:37.488708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.328941+0000","last_change":"2023-11-27T15:16:09.349642+0000","last_active":"2023-11-27T15:16:10.328941+0000","last_peered":"2023-11-27T15:16:10.328941+0000","last_clean":"2023-11-27T15:16:10.328941+0000","last_became_active":"2023-11-27T15:16:09.349086+0000","last_became_peered":"2023-11-27T15:16:09.349086+0000","last_unstale":"2023-11-27T15:16:10.328941+0000","last_undegraded":"2023-11-27T15:16:10.328941+0000","last_fullsized":"2023-11-27T15:16:10.328941+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:54:34.111407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.796803+0000","last_change":"2023-11-27T15:16:08.917835+0000","last_active":"2023-11-27T15:16:12.796803+0000","last_peered":"2023-11-27T15:16:12.796803+0000","last_clean":"2023-11-27T15:16:12.796803+0000","last_became_active":"2023-11-27T15:16:08.917532+0000","last_became_peered":"2023-11-27T15:16:08.917532+0000","last_unstale":"2023-11-27T15:16:12.796803+0000","last_undegraded":"2023-11-27T15:16:12.796803+0000","last_fullsized":"2023-11-27T15:16:12.796803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:35:27.771748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.019054+0000","last_change":"2023-11-27T15:16:09.338459+0000","last_active":"2023-11-27T15:16:11.019054+0000","last_peered":"2023-11-27T15:16:11.019054+0000","last_clean":"2023-11-27T15:16:11.019054+0000","last_became_active":"2023-11-27T15:16:09.338148+0000","last_became_peered":"2023-11-27T15:16:09.338148+0000","last_unstale":"2023-11-27T15:16:11.019054+0000","last_undegraded":"2023-11-27T15:16:11.019054+0000","last_fullsized":"2023-11-27T15:16:11.019054+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:50:17.721161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.743072+0000","last_change":"2023-11-27T15:16:08.915401+0000","last_active":"2023-11-27T15:16:12.743072+0000","last_peered":"2023-11-27T15:16:12.743072+0000","last_clean":"2023-11-27T15:16:12.743072+0000","last_became_active":"2023-11-27T15:16:08.915012+0000","last_became_peered":"2023-11-27T15:16:08.915012+0000","last_unstale":"2023-11-27T15:16:12.743072+0000","last_undegraded":"2023-11-27T15:16:12.743072+0000","last_fullsized":"2023-11-27T15:16:12.743072+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:25:39.081595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018186+0000","last_change":"2023-11-27T15:16:09.352309+0000","last_active":"2023-11-27T15:16:11.018186+0000","last_peered":"2023-11-27T15:16:11.018186+0000","last_clean":"2023-11-27T15:16:11.018186+0000","last_became_active":"2023-11-27T15:16:09.352039+0000","last_became_peered":"2023-11-27T15:16:09.352039+0000","last_unstale":"2023-11-27T15:16:11.018186+0000","last_undegraded":"2023-11-27T15:16:11.018186+0000","last_fullsized":"2023-11-27T15:16:11.018186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:02:54.926789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:13.751357+0000","last_change":"2023-11-27T15:16:08.920446+0000","last_active":"2023-11-27T15:16:13.751357+0000","last_peered":"2023-11-27T15:16:13.751357+0000","last_clean":"2023-11-27T15:16:13.751357+0000","last_became_active":"2023-11-27T15:16:08.920232+0000","last_became_peered":"2023-11-27T15:16:08.920232+0000","last_unstale":"2023-11-27T15:16:13.751357+0000","last_undegraded":"2023-11-27T15:16:13.751357+0000","last_fullsized":"2023-11-27T15:16:13.751357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:28:22.523270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018226+0000","last_change":"2023-11-27T15:16:09.344328+0000","last_active":"2023-11-27T15:16:11.018226+0000","last_peered":"2023-11-27T15:16:11.018226+0000","last_clean":"2023-11-27T15:16:11.018226+0000","last_became_active":"2023-11-27T15:16:09.344047+0000","last_became_peered":"2023-11-27T15:16:09.344047+0000","last_unstale":"2023-11-27T15:16:11.018226+0000","last_undegraded":"2023-11-27T15:16:11.018226+0000","last_fullsized":"2023-11-27T15:16:11.018226+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:10:10.564306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018219+0000","last_change":"2023-11-27T15:16:08.909865+0000","last_active":"2023-11-27T15:16:11.018219+0000","last_peered":"2023-11-27T15:16:11.018219+0000","last_clean":"2023-11-27T15:16:11.018219+0000","last_became_active":"2023-11-27T15:16:08.909559+0000","last_became_peered":"2023-11-27T15:16:08.909559+0000","last_unstale":"2023-11-27T15:16:11.018219+0000","last_undegraded":"2023-11-27T15:16:11.018219+0000","last_fullsized":"2023-11-27T15:16:11.018219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:21:20.674879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.170220+0000","last_change":"2023-11-27T15:16:09.342525+0000","last_active":"2023-11-27T15:16:11.170220+0000","last_peered":"2023-11-27T15:16:11.170220+0000","last_clean":"2023-11-27T15:16:11.170220+0000","last_became_active":"2023-11-27T15:16:09.340987+0000","last_became_peered":"2023-11-27T15:16:09.340987+0000","last_unstale":"2023-11-27T15:16:11.170220+0000","last_undegraded":"2023-11-27T15:16:11.170220+0000","last_fullsized":"2023-11-27T15:16:11.170220+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:57:33.064763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017934+0000","last_change":"2023-11-27T15:16:08.914088+0000","last_active":"2023-11-27T15:16:11.017934+0000","last_peered":"2023-11-27T15:16:11.017934+0000","last_clean":"2023-11-27T15:16:11.017934+0000","last_became_active":"2023-11-27T15:16:08.913750+0000","last_became_peered":"2023-11-27T15:16:08.913750+0000","last_unstale":"2023-11-27T15:16:11.017934+0000","last_undegraded":"2023-11-27T15:16:11.017934+0000","last_fullsized":"2023-11-27T15:16:11.017934+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:52:50.990435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018281+0000","last_change":"2023-11-27T15:16:09.344403+0000","last_active":"2023-11-27T15:16:11.018281+0000","last_peered":"2023-11-27T15:16:11.018281+0000","last_clean":"2023-11-27T15:16:11.018281+0000","last_became_active":"2023-11-27T15:16:09.344120+0000","last_became_peered":"2023-11-27T15:16:09.344120+0000","last_unstale":"2023-11-27T15:16:11.018281+0000","last_undegraded":"2023-11-27T15:16:11.018281+0000","last_fullsized":"2023-11-27T15:16:11.018281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:47:52.787834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.328999+0000","last_change":"2023-11-27T15:16:09.349780+0000","last_active":"2023-11-27T15:16:10.328999+0000","last_peered":"2023-11-27T15:16:10.328999+0000","last_clean":"2023-11-27T15:16:10.328999+0000","last_became_active":"2023-11-27T15:16:09.349493+0000","last_became_peered":"2023-11-27T15:16:09.349493+0000","last_unstale":"2023-11-27T15:16:10.328999+0000","last_undegraded":"2023-11-27T15:16:10.328999+0000","last_fullsized":"2023-11-27T15:16:10.328999+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:11:07.759158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329634+0000","last_change":"2023-11-27T15:16:09.340528+0000","last_active":"2023-11-27T15:16:10.329634+0000","last_peered":"2023-11-27T15:16:10.329634+0000","last_clean":"2023-11-27T15:16:10.329634+0000","last_became_active":"2023-11-27T15:16:09.339435+0000","last_became_peered":"2023-11-27T15:16:09.339435+0000","last_unstale":"2023-11-27T15:16:10.329634+0000","last_undegraded":"2023-11-27T15:16:10.329634+0000","last_fullsized":"2023-11-27T15:16:10.329634+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:08:23.254070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329091+0000","last_change":"2023-11-27T15:16:09.337989+0000","last_active":"2023-11-27T15:16:10.329091+0000","last_peered":"2023-11-27T15:16:10.329091+0000","last_clean":"2023-11-27T15:16:10.329091+0000","last_became_active":"2023-11-27T15:16:09.336712+0000","last_became_peered":"2023-11-27T15:16:09.336712+0000","last_unstale":"2023-11-27T15:16:10.329091+0000","last_undegraded":"2023-11-27T15:16:10.329091+0000","last_fullsized":"2023-11-27T15:16:10.329091+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:53:19.057499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.865031+0000","last_change":"2023-11-27T15:16:09.347077+0000","last_active":"2023-11-27T15:16:10.865031+0000","last_peered":"2023-11-27T15:16:10.865031+0000","last_clean":"2023-11-27T15:16:10.865031+0000","last_became_active":"2023-11-27T15:16:09.346722+0000","last_became_peered":"2023-11-27T15:16:09.346722+0000","last_unstale":"2023-11-27T15:16:10.865031+0000","last_undegraded":"2023-11-27T15:16:10.865031+0000","last_fullsized":"2023-11-27T15:16:10.865031+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:09:22.987678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017743+0000","last_change":"2023-11-27T15:16:09.344317+0000","last_active":"2023-11-27T15:16:11.017743+0000","last_peered":"2023-11-27T15:16:11.017743+0000","last_clean":"2023-11-27T15:16:11.017743+0000","last_became_active":"2023-11-27T15:16:09.343962+0000","last_became_peered":"2023-11-27T15:16:09.343962+0000","last_unstale":"2023-11-27T15:16:11.017743+0000","last_undegraded":"2023-11-27T15:16:11.017743+0000","last_fullsized":"2023-11-27T15:16:11.017743+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:49:36.883005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329146+0000","last_change":"2023-11-27T15:16:09.344690+0000","last_active":"2023-11-27T15:16:10.329146+0000","last_peered":"2023-11-27T15:16:10.329146+0000","last_clean":"2023-11-27T15:16:10.329146+0000","last_became_active":"2023-11-27T15:16:09.344259+0000","last_became_peered":"2023-11-27T15:16:09.344259+0000","last_unstale":"2023-11-27T15:16:10.329146+0000","last_undegraded":"2023-11-27T15:16:10.329146+0000","last_fullsized":"2023-11-27T15:16:10.329146+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:47:31.777045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329769+0000","last_change":"2023-11-27T15:16:09.340459+0000","last_active":"2023-11-27T15:16:10.329769+0000","last_peered":"2023-11-27T15:16:10.329769+0000","last_clean":"2023-11-27T15:16:10.329769+0000","last_became_active":"2023-11-27T15:16:09.340140+0000","last_became_peered":"2023-11-27T15:16:09.340140+0000","last_unstale":"2023-11-27T15:16:10.329769+0000","last_undegraded":"2023-11-27T15:16:10.329769+0000","last_fullsized":"2023-11-27T15:16:10.329769+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:51:52.323186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018191+0000","last_change":"2023-11-27T15:16:09.336625+0000","last_active":"2023-11-27T15:16:11.018191+0000","last_peered":"2023-11-27T15:16:11.018191+0000","last_clean":"2023-11-27T15:16:11.018191+0000","last_became_active":"2023-11-27T15:16:09.336287+0000","last_became_peered":"2023-11-27T15:16:09.336287+0000","last_unstale":"2023-11-27T15:16:11.018191+0000","last_undegraded":"2023-11-27T15:16:11.018191+0000","last_fullsized":"2023-11-27T15:16:11.018191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:19:26.038950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017922+0000","last_change":"2023-11-27T15:16:09.335480+0000","last_active":"2023-11-27T15:16:11.017922+0000","last_peered":"2023-11-27T15:16:11.017922+0000","last_clean":"2023-11-27T15:16:11.017922+0000","last_became_active":"2023-11-27T15:16:09.335163+0000","last_became_peered":"2023-11-27T15:16:09.335163+0000","last_unstale":"2023-11-27T15:16:11.017922+0000","last_undegraded":"2023-11-27T15:16:11.017922+0000","last_fullsized":"2023-11-27T15:16:11.017922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:27:19.391538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017703+0000","last_change":"2023-11-27T15:16:09.336879+0000","last_active":"2023-11-27T15:16:11.017703+0000","last_peered":"2023-11-27T15:16:11.017703+0000","last_clean":"2023-11-27T15:16:11.017703+0000","last_became_active":"2023-11-27T15:16:09.336580+0000","last_became_peered":"2023-11-27T15:16:09.336580+0000","last_unstale":"2023-11-27T15:16:11.017703+0000","last_undegraded":"2023-11-27T15:16:11.017703+0000","last_fullsized":"2023-11-27T15:16:11.017703+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:45:37.465813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.332430+0000","last_change":"2023-11-27T15:16:09.341400+0000","last_active":"2023-11-27T15:16:10.332430+0000","last_peered":"2023-11-27T15:16:10.332430+0000","last_clean":"2023-11-27T15:16:10.332430+0000","last_became_active":"2023-11-27T15:16:09.341103+0000","last_became_peered":"2023-11-27T15:16:09.341103+0000","last_unstale":"2023-11-27T15:16:10.332430+0000","last_undegraded":"2023-11-27T15:16:10.332430+0000","last_fullsized":"2023-11-27T15:16:10.332430+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:59:30.634393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018153+0000","last_change":"2023-11-27T15:16:09.345938+0000","last_active":"2023-11-27T15:16:11.018153+0000","last_peered":"2023-11-27T15:16:11.018153+0000","last_clean":"2023-11-27T15:16:11.018153+0000","last_became_active":"2023-11-27T15:16:09.345624+0000","last_became_peered":"2023-11-27T15:16:09.345624+0000","last_unstale":"2023-11-27T15:16:11.018153+0000","last_undegraded":"2023-11-27T15:16:11.018153+0000","last_fullsized":"2023-11-27T15:16:11.018153+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:03:33.299715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329231+0000","last_change":"2023-11-27T15:16:09.343896+0000","last_active":"2023-11-27T15:16:10.329231+0000","last_peered":"2023-11-27T15:16:10.329231+0000","last_clean":"2023-11-27T15:16:10.329231+0000","last_became_active":"2023-11-27T15:16:09.343603+0000","last_became_peered":"2023-11-27T15:16:09.343603+0000","last_unstale":"2023-11-27T15:16:10.329231+0000","last_undegraded":"2023-11-27T15:16:10.329231+0000","last_fullsized":"2023-11-27T15:16:10.329231+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:58:06.401944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":249380,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62821,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":270,"num_read_kb":200,"num_write":151,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":188463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449109,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5936,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":93739216,"statfs":{"total":95995035648,"available":95988957184,"internally_reserved":0,"allocated":892928,"data_stored":426878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.45500000000000002}]},{"osd":1,"last update":"Mon Nov 27 15:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.64800000000000002}]},{"osd":2,"last update":"Mon Nov 27 15:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.41199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":4,"last update":"Mon Nov 27 15:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.50900000000000001}]},{"osd":5,"last update":"Mon Nov 27 15:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.436},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.69199999999999995}]},{"osd":6,"last update":"Mon Nov 27 15:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.59199999999999997}]},{"osd":7,"last update":"Mon Nov 27 15:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.61299999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":9,"last update":"Mon Nov 27 15:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.77100000000000002}]},{"osd":10,"last update":"Mon Nov 27 15:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.706,"5min":1.706,"15min":1.706},"last":0.68600000000000005}]}]},{"osd":10,"up_from":71,"seq":304942678041,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6404,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738748,"statfs":{"total":95995035648,"available":95988477952,"internally_reserved":0,"allocated":1044480,"data_stored":1077234,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":1,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.41699999999999998}]},{"osd":2,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.71199999999999997}]},{"osd":3,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.44},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.58999999999999997}]},{"osd":4,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.501},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.48499999999999999}]},{"osd":5,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.67400000000000004}]},{"osd":6,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.53700000000000003}]},{"osd":7,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.53300000000000003}]},{"osd":8,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.66100000000000003}]},{"osd":9,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.61399999999999999}]},{"osd":11,"last update":"Mon Nov 27 15:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.73499999999999999}]}]},{"osd":9,"up_from":66,"seq":283467841566,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6080,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739072,"statfs":{"total":95995035648,"available":95988809728,"internally_reserved":0,"allocated":909312,"data_stored":427114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.437,"15min":0.443},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.65300000000000002,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44500000000000001,"15min":0.44800000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.33500000000000002}]},{"osd":1,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.61099999999999999,"15min":0.61499999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.434},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.628,"15min":0.63800000000000001},"min":{"1min":0.42499999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77900000000000003,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.47799999999999998}]},{"osd":2,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64100000000000001,"15min":0.64200000000000002},"min":{"1min":0.42899999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.46300000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.81999999999999995,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.58999999999999997}]},{"osd":3,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.52900000000000003,"15min":0.53500000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.378},{"interface":"front","average":{"1min":0.495,"5min":0.59499999999999997,"15min":0.61199999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65500000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.52200000000000002}]},{"osd":4,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.65200000000000002,"15min":0.66600000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79300000000000004,"5min":1.236,"15min":1.236},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.46400000000000002}]},{"osd":5,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.70899999999999996,"15min":0.72699999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.82499999999999996,"5min":1.274,"15min":1.274},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.68500000000000005,"15min":0.68700000000000006},"min":{"1min":0.439,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.53700000000000003}]},{"osd":6,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.69499999999999995,"15min":0.70899999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.71199999999999997,"15min":0.73399999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.77100000000000002,"5min":1.351,"15min":1.351},"last":0.5}]},{"osd":7,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.75800000000000001,"15min":0.76200000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.77900000000000003,"15min":0.79500000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.92000000000000004,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.64900000000000002}]},{"osd":8,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79600000000000004,"15min":0.79900000000000004},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.78500000000000003,"15min":0.80300000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.86699999999999999,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.57899999999999996}]},{"osd":10,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.54700000000000004}]}]},{"osd":8,"up_from":58,"seq":249108103202,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6272,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738880,"statfs":{"total":95995035648,"available":95988613120,"internally_reserved":0,"allocated":909312,"data_stored":427034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.499,"15min":0.495},"min":{"1min":0.35599999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.503,"15min":0.496},"min":{"1min":0.375,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.56499999999999995}]},{"osd":1,"last update":"Mon Nov 27 15:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.55700000000000005,"15min":0.55100000000000005},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56599999999999995,"15min":0.56100000000000005},"min":{"1min":0.41899999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.77300000000000002}]},{"osd":2,"last update":"Mon Nov 27 15:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.45600000000000002,"15min":0.44700000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.44800000000000001,"15min":0.441},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.82599999999999996}]},{"osd":3,"last update":"Mon Nov 27 15:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53800000000000003,"15min":0.53100000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.621},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.56399999999999995,"15min":0.55600000000000005},"min":{"1min":0.40899999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.69599999999999995}]},{"osd":4,"last update":"Mon Nov 27 15:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.443,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63400000000000001,"15min":0.628},"min":{"1min":0.48699999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94499999999999995,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.84899999999999998}]},{"osd":5,"last update":"Mon Nov 27 15:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.752},{"interface":"front","average":{"1min":0.625,"5min":0.55700000000000005,"15min":0.54600000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.86599999999999999}]},{"osd":6,"last update":"Mon Nov 27 15:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.50700000000000001,"5min":0.434,"15min":0.434},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.65000000000000002,"15min":0.64500000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.89300000000000002}]},{"osd":7,"last update":"Mon Nov 27 15:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67400000000000004,"15min":0.66900000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.68999999999999995,"15min":0.68200000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.91400000000000003}]},{"osd":9,"last update":"Mon Nov 27 15:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.72899999999999998,"15min":0.71699999999999997},"min":{"1min":0.52600000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.74399999999999999,"15min":0.73099999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.93899999999999995}]},{"osd":10,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.96699999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]}]},{"osd":7,"up_from":50,"seq":214748364838,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6384,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738768,"statfs":{"total":95995035648,"available":95988498432,"internally_reserved":0,"allocated":958464,"data_stored":1019263,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.35599999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.76300000000000001,"5min":1.052,"15min":1.052},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.51800000000000002,"15min":0.503},"min":{"1min":0.315,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.38700000000000001}]},{"osd":1,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.39400000000000002,"15min":0.38200000000000001},"min":{"1min":0.33400000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.60499999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.40999999999999998,"15min":0.39900000000000002},"min":{"1min":0.28199999999999997,"5min":0.254,"15min":0.254},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.71399999999999997}]},{"osd":2,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.39500000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.55300000000000005,"15min":0.54100000000000004},"min":{"1min":0.41499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.77500000000000002}]},{"osd":3,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.54000000000000004,"15min":0.52400000000000002},"min":{"1min":0.38800000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.623},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.59299999999999997,"15min":0.57999999999999996},"min":{"1min":0.45700000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88300000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.59899999999999998}]},{"osd":4,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.49299999999999999,"15min":0.47499999999999998},"min":{"1min":0.43099999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.48999999999999999,"15min":0.47399999999999998},"min":{"1min":0.41399999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.57599999999999996}]},{"osd":5,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.61799999999999999,"15min":0.59499999999999997},"min":{"1min":0.47099999999999997,"5min":0.375,"15min":0.375},"max":{"1min":2.3719999999999999,"5min":2.3719999999999999,"15min":2.3719999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.64100000000000001,"15min":0.61199999999999999},"min":{"1min":0.44700000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":3.1059999999999999,"5min":3.1059999999999999,"15min":3.1059999999999999},"last":0.68600000000000005}]},{"osd":6,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.66600000000000004,"15min":0.65400000000000003},"min":{"1min":0.51200000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.95299999999999996,"5min":1.008,"15min":1.008},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.625,"15min":0.61099999999999999},"min":{"1min":0.52900000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.70199999999999996}]},{"osd":8,"last update":"Mon Nov 27 15:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.66500000000000004,"15min":0.65300000000000002},"min":{"1min":0.51700000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.96099999999999997,"5min":1.157,"15min":1.157},"last":0.871},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.72799999999999998,"15min":0.69999999999999996},"min":{"1min":0.501,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":3.7559999999999998,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":0.88200000000000001}]},{"osd":9,"last update":"Mon Nov 27 15:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.752,"15min":0.75},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.73999999999999999,"15min":0.72999999999999998},"min":{"1min":0.50900000000000001,"5min":0.497,"15min":0.497},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":0.89300000000000002}]},{"osd":10,"last update":"Mon Nov 27 15:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.85399999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]}]},{"osd":6,"up_from":45,"seq":193273528363,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6260,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738892,"statfs":{"total":95995035648,"available":95988625408,"internally_reserved":0,"allocated":897024,"data_stored":426509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.438,"15min":0.44500000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.86099999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.503,"15min":0.53200000000000003},"min":{"1min":0.29899999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.83699999999999997,"5min":2.508,"15min":2.508},"last":0.499}]},{"osd":1,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59599999999999997,"15min":0.58699999999999997},"min":{"1min":0.45000000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.61499999999999999,"15min":0.63800000000000001},"min":{"1min":0.41099999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73799999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":0.89800000000000002}]},{"osd":2,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.65300000000000002,"15min":0.67500000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91000000000000003,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.64600000000000002,"15min":0.65000000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.626}]},{"osd":3,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52900000000000003,"15min":0.53100000000000003},"min":{"1min":0.377,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89600000000000002,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.54500000000000004,"15min":0.58699999999999997},"min":{"1min":0.33000000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.879,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":0.53400000000000003}]},{"osd":4,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.64300000000000002,"15min":0.65800000000000003},"min":{"1min":0.42699999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95299999999999996,"5min":1.51,"15min":1.51},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65500000000000003,"15min":0.64300000000000002},"min":{"1min":0.495,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.012,"5min":1.054,"15min":1.054},"last":0.70799999999999996}]},{"osd":5,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.69299999999999995,"15min":0.73099999999999998},"min":{"1min":0.47399999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77700000000000002,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.70199999999999996,"15min":0.71899999999999997},"min":{"1min":0.51600000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.93300000000000005,"5min":1.599,"15min":1.599},"last":0.749}]},{"osd":7,"last update":"Mon Nov 27 15:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72599999999999998,"15min":0.72499999999999998},"min":{"1min":0.46000000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.995,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.73999999999999999,"15min":0.73599999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.77400000000000002}]},{"osd":8,"last update":"Mon Nov 27 15:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.79700000000000004,"15min":0.81200000000000006},"min":{"1min":0.57199999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.99299999999999999,"5min":1.155,"15min":1.155},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.77700000000000002,"15min":0.80500000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.75900000000000001,"5min":2.766,"15min":2.766},"last":0.67000000000000004}]},{"osd":9,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.71599999999999997,"15min":0.74099999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.77800000000000002,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.75,"15min":0.77000000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90000000000000002,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.66000000000000003}]},{"osd":10,"last update":"Mon Nov 27 15:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.85299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7280,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":6336,"kb_avail":93737872,"statfs":{"total":95995035648,"available":95987580928,"internally_reserved":0,"allocated":958464,"data_stored":484823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6488064},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.502,"15min":0.46999999999999997},"min":{"1min":0.34000000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77000000000000002,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.435},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.505,"15min":0.46500000000000002},"min":{"1min":0.35499999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.51000000000000001}]},{"osd":2,"last update":"Mon Nov 27 15:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.39900000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53900000000000003,"15min":0.52600000000000002},"min":{"1min":0.36699999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76800000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.72499999999999998}]},{"osd":3,"last update":"Mon Nov 27 15:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56999999999999995,"15min":0.53400000000000003},"min":{"1min":0.38100000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.96099999999999997,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56399999999999995,"15min":0.55700000000000005},"min":{"1min":0.32200000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.78300000000000003,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.65800000000000003}]},{"osd":4,"last update":"Mon Nov 27 15:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.44,"15min":0.42799999999999999},"min":{"1min":0.314,"5min":0.251,"15min":0.251},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.45500000000000002,"15min":0.441},"min":{"1min":0.26400000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.54900000000000004}]},{"osd":5,"last update":"Mon Nov 27 15:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.59799999999999998,"15min":0.58899999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.63,"15min":0.623},"min":{"1min":0.53100000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.66900000000000004}]},{"osd":6,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.60599999999999998,"15min":0.58299999999999996},"min":{"1min":0.40699999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.63600000000000001,"15min":0.626},"min":{"1min":0.434,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95499999999999996,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.753}]},{"osd":7,"last update":"Mon Nov 27 15:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.53600000000000003,"15min":0.52600000000000002},"min":{"1min":0.38,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.58199999999999996,"15min":0.57299999999999995},"min":{"1min":0.432,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.46500000000000002}]},{"osd":8,"last update":"Mon Nov 27 15:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.66000000000000003,"15min":0.64800000000000002},"min":{"1min":0.52200000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.63500000000000001,"15min":0.624},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.625}]},{"osd":9,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72899999999999998,"15min":0.72099999999999997},"min":{"1min":0.57899999999999996,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.72899999999999998,"15min":0.72499999999999998},"min":{"1min":0.53400000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.73999999999999999}]},{"osd":10,"last update":"Mon Nov 27 15:16:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.76600000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7176,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":93737976,"statfs":{"total":95995035648,"available":95987687424,"internally_reserved":0,"allocated":917504,"data_stored":427774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 27 15:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.58199999999999996,"15min":0.54600000000000004},"min":{"1min":0.45900000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.97099999999999997,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.60499999999999998,"15min":0.55900000000000005},"min":{"1min":0.51000000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.018,"5min":1.55,"15min":1.55},"last":0.54000000000000004}]},{"osd":2,"last update":"Mon Nov 27 15:16:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56699999999999995,"15min":0.54000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77100000000000002,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53700000000000003,"15min":0.51700000000000002},"min":{"1min":0.39800000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.67100000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.51300000000000001}]},{"osd":3,"last update":"Mon Nov 27 15:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.38900000000000001,"15min":0.371},"min":{"1min":0.26000000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.441,"15min":0.42899999999999999},"min":{"1min":0.30399999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.34399999999999997}]},{"osd":4,"last update":"Mon Nov 27 15:16:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.57699999999999996,"15min":0.55000000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.96099999999999997,"5min":0.999,"15min":0.999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.58899999999999997,"15min":0.55900000000000005},"min":{"1min":0.38300000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.498}]},{"osd":5,"last update":"Mon Nov 27 15:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.621,"15min":0.61399999999999999},"min":{"1min":0.40799999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.749,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.61799999999999999,"15min":0.60799999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.81100000000000005,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.61199999999999999}]},{"osd":6,"last update":"Mon Nov 27 15:16:43 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.308,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.63900000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.754,"5min":1.032,"15min":1.032},"last":0.52600000000000002}]},{"osd":7,"last update":"Mon Nov 27 15:16:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67800000000000005,"15min":0.67200000000000004},"min":{"1min":0.434,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.46100000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.31}]},{"osd":8,"last update":"Mon Nov 27 15:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.68300000000000005,"15min":0.70299999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.72799999999999998,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.73199999999999998,"15min":0.746},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84299999999999997,"5min":1.004,"15min":1.004},"last":0.754}]},{"osd":9,"last update":"Mon Nov 27 15:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.73499999999999999,"15min":0.76100000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88800000000000001,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.72299999999999998,"15min":0.74299999999999999},"min":{"1min":0.438,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.45400000000000001}]},{"osd":10,"last update":"Mon Nov 27 15:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.69999999999999996}]},{"osd":11,"last update":"Mon Nov 27 15:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.68600000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7368,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737784,"statfs":{"total":95995035648,"available":95987490816,"internally_reserved":0,"allocated":983040,"data_stored":486515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.51100000000000001,"15min":0.47699999999999998},"min":{"1min":0.36099999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.499,"15min":0.48099999999999998},"min":{"1min":0.36499999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.48799999999999999}]},{"osd":1,"last update":"Mon Nov 27 15:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.53200000000000003,"15min":0.51000000000000001},"min":{"1min":0.34899999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51100000000000001,"15min":0.48899999999999999},"min":{"1min":0.377,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.53600000000000003}]},{"osd":3,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.59899999999999998,"15min":0.54600000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.2170000000000001,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.59599999999999997,"15min":0.54600000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":3.278,"5min":3.278,"15min":3.278},"last":0.68799999999999994}]},{"osd":4,"last update":"Mon Nov 27 15:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.38400000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.76200000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.35799999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79200000000000004,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.65400000000000003}]},{"osd":5,"last update":"Mon Nov 27 15:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.434,"15min":0.40100000000000002},"min":{"1min":0.29599999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.75},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.436,"15min":0.41199999999999998},"min":{"1min":0.311,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.59299999999999997}]},{"osd":6,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.58599999999999997,"15min":0.56499999999999995},"min":{"1min":0.44900000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.70399999999999996,"15min":0.64300000000000002},"min":{"1min":0.41599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":4.5800000000000001,"5min":4.5800000000000001,"15min":4.5800000000000001},"last":0.82099999999999995}]},{"osd":7,"last update":"Mon Nov 27 15:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.39800000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.44},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69199999999999995,"15min":0.69599999999999995},"min":{"1min":0.46300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93300000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.51100000000000001}]},{"osd":8,"last update":"Mon Nov 27 15:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.64800000000000002,"15min":0.65700000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57499999999999996,"15min":0.57399999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.73999999999999999}]},{"osd":9,"last update":"Mon Nov 27 15:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.72199999999999998,"15min":0.70199999999999996},"min":{"1min":0.49399999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":3.633,"5min":3.633,"15min":3.633},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.036,"5min":0.79500000000000004,"15min":0.755},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":7.6369999999999996,"5min":7.6369999999999996,"15min":7.6369999999999996},"last":0.77700000000000002}]},{"osd":10,"last update":"Mon Nov 27 15:16:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.79000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]}]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6532,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738620,"statfs":{"total":95995035648,"available":95988346880,"internally_reserved":0,"allocated":913408,"data_stored":426826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.36599999999999999,"15min":0.33500000000000002},"min":{"1min":0.254,"5min":0.22,"15min":0.22},"max":{"1min":0.60499999999999998,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.38500000000000001,"15min":0.35699999999999998},"min":{"1min":0.27300000000000002,"5min":0.214,"15min":0.214},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.38400000000000001}]},{"osd":1,"last update":"Mon Nov 27 15:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53800000000000003,"15min":0.505},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78500000000000003,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53800000000000003,"15min":0.52400000000000002},"min":{"1min":0.376,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82599999999999996,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.73099999999999998}]},{"osd":2,"last update":"Mon Nov 27 15:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58699999999999997,"15min":0.54500000000000004},"min":{"1min":0.40200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80100000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58099999999999996,"15min":0.54400000000000004},"min":{"1min":0.41699999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.81100000000000005,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.52600000000000002}]},{"osd":4,"last update":"Mon Nov 27 15:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.54300000000000004,"15min":0.52400000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.56000000000000005,"15min":0.52200000000000002},"min":{"1min":0.495,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.61299999999999999}]},{"osd":5,"last update":"Mon Nov 27 15:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.60499999999999998,"15min":0.57599999999999996},"min":{"1min":0.42899999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.92900000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.61799999999999999,"15min":0.58799999999999997},"min":{"1min":0.44700000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89000000000000001,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48699999999999999}]},{"osd":6,"last update":"Mon Nov 27 15:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56999999999999995,"15min":0.55000000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.91100000000000003,"5min":1.089,"15min":1.089},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55900000000000005,"15min":0.54300000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.49299999999999999}]},{"osd":7,"last update":"Mon Nov 27 15:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.72499999999999998,"15min":0.70899999999999996},"min":{"1min":0.59599999999999997,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.77200000000000002}]},{"osd":8,"last update":"Mon Nov 27 15:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.628}]},{"osd":9,"last update":"Mon Nov 27 15:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.80600000000000005,"15min":0.82599999999999996},"min":{"1min":0.49299999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.083,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.73799999999999999,"15min":0.75600000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95399999999999996,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.66800000000000004}]},{"osd":10,"last update":"Mon Nov 27 15:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.80000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]}]},{"osd":4,"up_from":33,"seq":141733920820,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6392,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738760,"statfs":{"total":95995035648,"available":95988490240,"internally_reserved":0,"allocated":901120,"data_stored":427282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.48099999999999998,"15min":0.46800000000000003},"min":{"1min":0.35299999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.52200000000000002,"15min":0.50700000000000001},"min":{"1min":0.40300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.432}]},{"osd":1,"last update":"Mon Nov 27 15:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.36899999999999999,"15min":0.34799999999999998},"min":{"1min":0.30299999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.36899999999999999,"15min":0.34699999999999998},"min":{"1min":0.28399999999999997,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.68799999999999994}]},{"osd":2,"last update":"Mon Nov 27 15:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.50700000000000001,"15min":0.48899999999999999},"min":{"1min":0.36699999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.54500000000000004,"15min":0.53500000000000003},"min":{"1min":0.40200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.64600000000000002}]},{"osd":3,"last update":"Mon Nov 27 15:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.53900000000000003,"15min":0.51400000000000001},"min":{"1min":0.45100000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.40999999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80700000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.626}]},{"osd":5,"last update":"Mon Nov 27 15:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.56100000000000005,"15min":0.53500000000000003},"min":{"1min":0.42099999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.57699999999999996,"15min":0.55900000000000005},"min":{"1min":0.45300000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.76000000000000001}]},{"osd":6,"last update":"Mon Nov 27 15:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.63500000000000001,"15min":0.59999999999999998},"min":{"1min":0.46500000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":5.1040000000000001,"5min":5.1040000000000001,"15min":5.1040000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.67400000000000004,"15min":0.63800000000000001},"min":{"1min":0.46700000000000003,"5min":0.441,"15min":0.441},"max":{"1min":5.1139999999999999,"5min":5.1139999999999999,"15min":5.1139999999999999},"last":0.82999999999999996}]},{"osd":7,"last update":"Mon Nov 27 15:16:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.34000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.34899999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.81200000000000006}]},{"osd":8,"last update":"Mon Nov 27 15:16:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.66900000000000004,"15min":0.66000000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.70299999999999996,"15min":0.69499999999999995},"min":{"1min":0.49099999999999999,"5min":0.432,"15min":0.432},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.85599999999999998}]},{"osd":9,"last update":"Mon Nov 27 15:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.78100000000000003,"15min":0.75900000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":4.8780000000000001,"5min":4.8780000000000001,"15min":4.8780000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.747,"15min":0.72299999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":4.9169999999999998,"5min":4.9169999999999998,"15min":4.9169999999999998},"last":0.79300000000000004}]},{"osd":10,"last update":"Mon Nov 27 15:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.74199999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757295,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6524,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738628,"statfs":{"total":95995035648,"available":95988355072,"internally_reserved":0,"allocated":970752,"data_stored":1018605,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.498,"15min":0.49299999999999999},"min":{"1min":0.32600000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67100000000000004,"5min":0.752,"15min":0.752},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.47599999999999998,"15min":0.46999999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.64500000000000002,"5min":0.755,"15min":0.755},"last":0.51300000000000001}]},{"osd":1,"last update":"Mon Nov 27 15:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54600000000000004,"15min":0.52800000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.54600000000000004,"15min":0.51700000000000002},"min":{"1min":0.41199999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.747}]},{"osd":2,"last update":"Mon Nov 27 15:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.40400000000000003,"15min":0.39100000000000001},"min":{"1min":0.27400000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.60999999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.42099999999999999,"15min":0.40500000000000003},"min":{"1min":0.28999999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.67100000000000004,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.66400000000000003}]},{"osd":3,"last update":"Mon Nov 27 15:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71799999999999997,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54100000000000004,"15min":0.53500000000000003},"min":{"1min":0.38200000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.73399999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.621}]},{"osd":4,"last update":"Mon Nov 27 15:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.56999999999999995,"15min":0.55300000000000005},"min":{"1min":0.46600000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.68200000000000005}]},{"osd":6,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":0.68500000000000005,"15min":0.61899999999999999},"min":{"1min":0.50800000000000001,"5min":0.436,"15min":0.436},"max":{"1min":8.0280000000000005,"5min":8.0280000000000005,"15min":8.0280000000000005},"last":0.65100000000000002},{"interface":"front","average":{"1min":1.1319999999999999,"5min":0.73899999999999999,"15min":0.67100000000000004},"min":{"1min":0.51800000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":8.0389999999999997,"5min":8.0389999999999997,"15min":8.0389999999999997},"last":0.82099999999999995}]},{"osd":7,"last update":"Mon Nov 27 15:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.50600000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.48099999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.80700000000000005}]},{"osd":8,"last update":"Mon Nov 27 15:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.64500000000000002,"15min":0.63700000000000001},"min":{"1min":0.41599999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.39100000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80100000000000005,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.71499999999999997}]},{"osd":9,"last update":"Mon Nov 27 15:16:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":0.747,"15min":0.69199999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":7.8070000000000004,"5min":7.8070000000000004,"15min":7.8070000000000004},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.194,"5min":0.81899999999999995,"15min":0.75700000000000001},"min":{"1min":0.56599999999999995,"5min":0.503,"15min":0.503},"max":{"1min":7.7690000000000001,"5min":7.7690000000000001,"15min":7.7690000000000001},"last":0.88500000000000001}]},{"osd":10,"last update":"Mon Nov 27 15:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.77400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34455,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34455,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-27T15:17:00.113 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph pg dump --format=json 2023-11-27T15:17:00.770 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:17:01.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:01 smithi052 ceph-mon[99056]: pgmap v282: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:17:02.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:01 smithi102 ceph-mon[100296]: pgmap v282: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:17:02.216 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.210+0000 7f1bc507c700 1 -- 172.21.15.52:0/394095814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bc010a8b0 msgr2=0x7f1bc010ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:02.216 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.210+0000 7f1bc507c700 1 --2- 172.21.15.52:0/394095814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bc010a8b0 0x7f1bc010ccf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800b0a0 tx=0x7f1bb800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:17:02.216 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.210+0000 7f1bc507c700 1 -- 172.21.15.52:0/394095814 shutdown_connections 2023-11-27T15:17:02.216 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.210+0000 7f1bc507c700 1 --2- 172.21.15.52:0/394095814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bc0101390 0x7f1bc0101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:02.216 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.210+0000 7f1bc507c700 1 --2- 172.21.15.52:0/394095814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bc010a8b0 0x7f1bc010ccf0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:02.216 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.210+0000 7f1bc507c700 1 --2- 172.21.15.52:0/394095814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bc0101ca0 0x7f1bc010a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:02.217 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.210+0000 7f1bc507c700 1 -- 172.21.15.52:0/394095814 >> 172.21.15.52:0/394095814 conn(0x7f1bc00770a0 msgr2=0x7f1bc00774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:02.217 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.210+0000 7f1bc507c700 1 -- 172.21.15.52:0/394095814 shutdown_connections 2023-11-27T15:17:02.217 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.210+0000 7f1bc507c700 1 -- 172.21.15.52:0/394095814 wait complete. 2023-11-27T15:17:02.219 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.211+0000 7f1bc507c700 1 Processor -- start 2023-11-27T15:17:02.219 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.211+0000 7f1bc507c700 1 -- start start 2023-11-27T15:17:02.219 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.212+0000 7f1bc507c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bc0101390 0x7f1bc00ff1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:02.220 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.212+0000 7f1bc507c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bc0101ca0 0x7f1bc00ff700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:02.220 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.212+0000 7f1bc507c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bc010a8b0 0x7f1bc00ffc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:02.220 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.212+0000 7f1bc507c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc01a48c0 con 0x7f1bc010a8b0 2023-11-27T15:17:02.220 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.212+0000 7f1bc507c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc01a4a30 con 0x7f1bc0101390 2023-11-27T15:17:02.220 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.212+0000 7f1bc507c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc01a4bf0 con 0x7f1bc0101ca0 2023-11-27T15:17:02.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.212+0000 7f1bbf59e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bc010a8b0 0x7f1bc00ffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:02.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.212+0000 7f1bbf59e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bc010a8b0 0x7f1bc00ffc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50654/0 (socket says 172.21.15.52:50654) 2023-11-27T15:17:02.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.212+0000 7f1bbf59e700 1 -- 172.21.15.52:0/3774371236 learned_addr learned my addr 172.21.15.52:0/3774371236 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:02.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.212+0000 7f1bbed9d700 1 --2- 172.21.15.52:0/3774371236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bc0101390 0x7f1bc00ff1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:02.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.212+0000 7f1bbe59c700 1 --2- 172.21.15.52:0/3774371236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bc0101ca0 0x7f1bc00ff700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:02.222 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.212+0000 7f1bbf59e700 1 -- 172.21.15.52:0/3774371236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bc0101ca0 msgr2=0x7f1bc00ff700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:02.222 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.212+0000 7f1bbf59e700 1 --2- 172.21.15.52:0/3774371236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bc0101ca0 0x7f1bc00ff700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:02.222 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.212+0000 7f1bbf59e700 1 -- 172.21.15.52:0/3774371236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bc0101390 msgr2=0x7f1bc00ff1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:02.222 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.212+0000 7f1bbf59e700 1 --2- 172.21.15.52:0/3774371236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bc0101390 0x7f1bc00ff1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:02.222 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.212+0000 7f1bbf59e700 1 -- 172.21.15.52:0/3774371236 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb8009d00 con 0x7f1bc010a8b0 2023-11-27T15:17:02.222 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.212+0000 7f1bbe59c700 1 --2- 172.21.15.52:0/3774371236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bc0101ca0 0x7f1bc00ff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:17:02.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.212+0000 7f1bbf59e700 1 --2- 172.21.15.52:0/3774371236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bc010a8b0 0x7f1bc00ffc80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800b830 tx=0x7f1bb80079b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:02.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.212+0000 7f1bb7fff700 1 -- 172.21.15.52:0/3774371236 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb800e040 con 0x7f1bc010a8b0 2023-11-27T15:17:02.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.213+0000 7f1bc507c700 1 -- 172.21.15.52:0/3774371236 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc0110310 con 0x7f1bc010a8b0 2023-11-27T15:17:02.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.213+0000 7f1bb7fff700 1 -- 172.21.15.52:0/3774371236 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bb8004d60 con 0x7f1bc010a8b0 2023-11-27T15:17:02.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.213+0000 7f1bb7fff700 1 -- 172.21.15.52:0/3774371236 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb801d480 con 0x7f1bc010a8b0 2023-11-27T15:17:02.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.213+0000 7f1bc507c700 1 -- 172.21.15.52:0/3774371236 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc0110180 con 0x7f1bc010a8b0 2023-11-27T15:17:02.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.214+0000 7f1bc507c700 1 -- 172.21.15.52:0/3774371236 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bc01104a0 con 0x7f1bc010a8b0 2023-11-27T15:17:02.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.215+0000 7f1bb7fff700 1 -- 172.21.15.52:0/3774371236 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f1bb8003a00 con 0x7f1bc010a8b0 2023-11-27T15:17:02.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.216+0000 7f1bb7fff700 1 --2- 172.21.15.52:0/3774371236 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f1bac0a11e0 0x7f1bac0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:02.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.216+0000 7f1bbed9d700 1 --2- 172.21.15.52:0/3774371236 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f1bac0a11e0 0x7f1bac0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:02.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.216+0000 7f1bb7fff700 1 -- 172.21.15.52:0/3774371236 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1bb80ce790 con 0x7f1bc010a8b0 2023-11-27T15:17:02.226 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.216+0000 7f1bbed9d700 1 --2- 172.21.15.52:0/3774371236 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f1bac0a11e0 0x7f1bac0a3690 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0005950 tx=0x7f1bb000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:02.226 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.219+0000 7f1bb7fff700 1 -- 172.21.15.52:0/3774371236 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1bb8097ec0 con 0x7f1bc010a8b0 2023-11-27T15:17:02.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:01 smithi150 ceph-mon[100559]: pgmap v282: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:17:02.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.411+0000 7f1bc507c700 1 -- 172.21.15.52:0/3774371236 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1bc01006f0 con 0x7f1bac0a11e0 2023-11-27T15:17:02.428 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.423+0000 7f1bb7fff700 1 -- 172.21.15.52:0/3774371236 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+339850 (secure 0 0 0) 0x7f1bc01006f0 con 0x7f1bac0a11e0 2023-11-27T15:17:02.433 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:17:02.436 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.426+0000 7f1bc507c700 1 -- 172.21.15.52:0/3774371236 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f1bac0a11e0 msgr2=0x7f1bac0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:02.436 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.426+0000 7f1bc507c700 1 --2- 172.21.15.52:0/3774371236 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f1bac0a11e0 0x7f1bac0a3690 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0005950 tx=0x7f1bb000b410 comp rx=0 tx=0).stop 2023-11-27T15:17:02.436 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.426+0000 7f1bc507c700 1 -- 172.21.15.52:0/3774371236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bc010a8b0 msgr2=0x7f1bc00ffc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:02.437 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.426+0000 7f1bc507c700 1 --2- 172.21.15.52:0/3774371236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bc010a8b0 0x7f1bc00ffc80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800b830 tx=0x7f1bb80079b0 comp rx=0 tx=0).stop 2023-11-27T15:17:02.437 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.426+0000 7f1bc507c700 1 -- 172.21.15.52:0/3774371236 shutdown_connections 2023-11-27T15:17:02.437 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.426+0000 7f1bc507c700 1 --2- 172.21.15.52:0/3774371236 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f1bac0a11e0 0x7f1bac0a3690 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:02.437 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.426+0000 7f1bc507c700 1 --2- 172.21.15.52:0/3774371236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bc0101390 0x7f1bc00ff1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:02.437 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.426+0000 7f1bc507c700 1 --2- 172.21.15.52:0/3774371236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bc010a8b0 0x7f1bc00ffc80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:02.437 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.426+0000 7f1bc507c700 1 --2- 172.21.15.52:0/3774371236 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bc0101ca0 0x7f1bc00ff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:02.437 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.426+0000 7f1bc507c700 1 -- 172.21.15.52:0/3774371236 >> 172.21.15.52:0/3774371236 conn(0x7f1bc00770a0 msgr2=0x7f1bc0102310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:02.438 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.426+0000 7f1bc507c700 1 -- 172.21.15.52:0/3774371236 shutdown_connections 2023-11-27T15:17:02.438 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:02.426+0000 7f1bc507c700 1 -- 172.21.15.52:0/3774371236 wait complete. 2023-11-27T15:17:02.438 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2023-11-27T15:17:02.835 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2023-11-27T15:17:01.735686+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653435,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":436,"num_read_kb":334,"num_write":334,"num_write_kb":3044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":272,"ondisk_log_size":272,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":78608,"kb_used_data":10992,"kb_used_omap":0,"kb_used_meta":67520,"kb_avail":1124863216,"statfs":{"total":1151940427776,"available":1151859933184,"internally_reserved":0,"allocated":11255808,"data_stored":7075857,"data_compressed":108282,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":69140480},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003401"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329249+0000","last_change":"2023-11-27T15:16:09.343007+0000","last_active":"2023-11-27T15:16:10.329249+0000","last_peered":"2023-11-27T15:16:10.329249+0000","last_clean":"2023-11-27T15:16:10.329249+0000","last_became_active":"2023-11-27T15:16:09.342125+0000","last_became_peered":"2023-11-27T15:16:09.342125+0000","last_unstale":"2023-11-27T15:16:10.329249+0000","last_undegraded":"2023-11-27T15:16:10.329249+0000","last_fullsized":"2023-11-27T15:16:10.329249+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:10:56.620347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.865187+0000","last_change":"2023-11-27T15:16:09.338942+0000","last_active":"2023-11-27T15:16:10.865187+0000","last_peered":"2023-11-27T15:16:10.865187+0000","last_clean":"2023-11-27T15:16:10.865187+0000","last_became_active":"2023-11-27T15:16:09.338636+0000","last_became_peered":"2023-11-27T15:16:09.338636+0000","last_unstale":"2023-11-27T15:16:10.865187+0000","last_undegraded":"2023-11-27T15:16:10.865187+0000","last_fullsized":"2023-11-27T15:16:10.865187+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:30:06.448581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.865094+0000","last_change":"2023-11-27T15:16:09.347845+0000","last_active":"2023-11-27T15:16:10.865094+0000","last_peered":"2023-11-27T15:16:10.865094+0000","last_clean":"2023-11-27T15:16:10.865094+0000","last_became_active":"2023-11-27T15:16:09.347533+0000","last_became_peered":"2023-11-27T15:16:09.347533+0000","last_unstale":"2023-11-27T15:16:10.865094+0000","last_undegraded":"2023-11-27T15:16:10.865094+0000","last_fullsized":"2023-11-27T15:16:10.865094+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:01:40.820155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.019299+0000","last_change":"2023-11-27T15:16:09.333251+0000","last_active":"2023-11-27T15:16:11.019299+0000","last_peered":"2023-11-27T15:16:11.019299+0000","last_clean":"2023-11-27T15:16:11.019299+0000","last_became_active":"2023-11-27T15:16:09.331510+0000","last_became_peered":"2023-11-27T15:16:09.331510+0000","last_unstale":"2023-11-27T15:16:11.019299+0000","last_undegraded":"2023-11-27T15:16:11.019299+0000","last_fullsized":"2023-11-27T15:16:11.019299+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:34:53.253074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018096+0000","last_change":"2023-11-27T15:16:09.349097+0000","last_active":"2023-11-27T15:16:11.018096+0000","last_peered":"2023-11-27T15:16:11.018096+0000","last_clean":"2023-11-27T15:16:11.018096+0000","last_became_active":"2023-11-27T15:16:09.348793+0000","last_became_peered":"2023-11-27T15:16:09.348793+0000","last_unstale":"2023-11-27T15:16:11.018096+0000","last_undegraded":"2023-11-27T15:16:11.018096+0000","last_fullsized":"2023-11-27T15:16:11.018096+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:55:31.214202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.019174+0000","last_change":"2023-11-27T15:16:09.338246+0000","last_active":"2023-11-27T15:16:11.019174+0000","last_peered":"2023-11-27T15:16:11.019174+0000","last_clean":"2023-11-27T15:16:11.019174+0000","last_became_active":"2023-11-27T15:16:09.338003+0000","last_became_peered":"2023-11-27T15:16:09.338003+0000","last_unstale":"2023-11-27T15:16:11.019174+0000","last_undegraded":"2023-11-27T15:16:11.019174+0000","last_fullsized":"2023-11-27T15:16:11.019174+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:25:15.821446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017705+0000","last_change":"2023-11-27T15:16:09.342131+0000","last_active":"2023-11-27T15:16:11.017705+0000","last_peered":"2023-11-27T15:16:11.017705+0000","last_clean":"2023-11-27T15:16:11.017705+0000","last_became_active":"2023-11-27T15:16:09.341834+0000","last_became_peered":"2023-11-27T15:16:09.341834+0000","last_unstale":"2023-11-27T15:16:11.017705+0000","last_undegraded":"2023-11-27T15:16:11.017705+0000","last_fullsized":"2023-11-27T15:16:11.017705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:06:07.633604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018355+0000","last_change":"2023-11-27T15:16:09.337308+0000","last_active":"2023-11-27T15:16:11.018355+0000","last_peered":"2023-11-27T15:16:11.018355+0000","last_clean":"2023-11-27T15:16:11.018355+0000","last_became_active":"2023-11-27T15:16:09.336958+0000","last_became_peered":"2023-11-27T15:16:09.336958+0000","last_unstale":"2023-11-27T15:16:11.018355+0000","last_undegraded":"2023-11-27T15:16:11.018355+0000","last_fullsized":"2023-11-27T15:16:11.018355+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:00:54.104008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.332789+0000","last_change":"2023-11-27T15:16:09.340646+0000","last_active":"2023-11-27T15:16:10.332789+0000","last_peered":"2023-11-27T15:16:10.332789+0000","last_clean":"2023-11-27T15:16:10.332789+0000","last_became_active":"2023-11-27T15:16:09.340084+0000","last_became_peered":"2023-11-27T15:16:09.340084+0000","last_unstale":"2023-11-27T15:16:10.332789+0000","last_undegraded":"2023-11-27T15:16:10.332789+0000","last_fullsized":"2023-11-27T15:16:10.332789+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:33:29.581983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.019348+0000","last_change":"2023-11-27T15:16:09.336871+0000","last_active":"2023-11-27T15:16:11.019348+0000","last_peered":"2023-11-27T15:16:11.019348+0000","last_clean":"2023-11-27T15:16:11.019348+0000","last_became_active":"2023-11-27T15:16:09.336496+0000","last_became_peered":"2023-11-27T15:16:09.336496+0000","last_unstale":"2023-11-27T15:16:11.019348+0000","last_undegraded":"2023-11-27T15:16:11.019348+0000","last_fullsized":"2023-11-27T15:16:11.019348+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:37:00.015095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018026+0000","last_change":"2023-11-27T15:16:09.343120+0000","last_active":"2023-11-27T15:16:11.018026+0000","last_peered":"2023-11-27T15:16:11.018026+0000","last_clean":"2023-11-27T15:16:11.018026+0000","last_became_active":"2023-11-27T15:16:09.342797+0000","last_became_peered":"2023-11-27T15:16:09.342797+0000","last_unstale":"2023-11-27T15:16:11.018026+0000","last_undegraded":"2023-11-27T15:16:11.018026+0000","last_fullsized":"2023-11-27T15:16:11.018026+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:39:21.071837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.019248+0000","last_change":"2023-11-27T15:16:09.336521+0000","last_active":"2023-11-27T15:16:11.019248+0000","last_peered":"2023-11-27T15:16:11.019248+0000","last_clean":"2023-11-27T15:16:11.019248+0000","last_became_active":"2023-11-27T15:16:09.336183+0000","last_became_peered":"2023-11-27T15:16:09.336183+0000","last_unstale":"2023-11-27T15:16:11.019248+0000","last_undegraded":"2023-11-27T15:16:11.019248+0000","last_fullsized":"2023-11-27T15:16:11.019248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:25:55.091755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018282+0000","last_change":"2023-11-27T15:16:09.346839+0000","last_active":"2023-11-27T15:16:11.018282+0000","last_peered":"2023-11-27T15:16:11.018282+0000","last_clean":"2023-11-27T15:16:11.018282+0000","last_became_active":"2023-11-27T15:16:09.345230+0000","last_became_peered":"2023-11-27T15:16:09.345230+0000","last_unstale":"2023-11-27T15:16:11.018282+0000","last_undegraded":"2023-11-27T15:16:11.018282+0000","last_fullsized":"2023-11-27T15:16:11.018282+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:22:56.772626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.864881+0000","last_change":"2023-11-27T15:16:09.342380+0000","last_active":"2023-11-27T15:16:10.864881+0000","last_peered":"2023-11-27T15:16:10.864881+0000","last_clean":"2023-11-27T15:16:10.864881+0000","last_became_active":"2023-11-27T15:16:09.341967+0000","last_became_peered":"2023-11-27T15:16:09.341967+0000","last_unstale":"2023-11-27T15:16:10.864881+0000","last_undegraded":"2023-11-27T15:16:10.864881+0000","last_fullsized":"2023-11-27T15:16:10.864881+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:35:13.999971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.864779+0000","last_change":"2023-11-27T15:16:09.349738+0000","last_active":"2023-11-27T15:16:10.864779+0000","last_peered":"2023-11-27T15:16:10.864779+0000","last_clean":"2023-11-27T15:16:10.864779+0000","last_became_active":"2023-11-27T15:16:09.349349+0000","last_became_peered":"2023-11-27T15:16:09.349349+0000","last_unstale":"2023-11-27T15:16:10.864779+0000","last_undegraded":"2023-11-27T15:16:10.864779+0000","last_fullsized":"2023-11-27T15:16:10.864779+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:13:03.443885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.332227+0000","last_change":"2023-11-27T15:16:09.339171+0000","last_active":"2023-11-27T15:16:10.332227+0000","last_peered":"2023-11-27T15:16:10.332227+0000","last_clean":"2023-11-27T15:16:10.332227+0000","last_became_active":"2023-11-27T15:16:09.338887+0000","last_became_peered":"2023-11-27T15:16:09.338887+0000","last_unstale":"2023-11-27T15:16:10.332227+0000","last_undegraded":"2023-11-27T15:16:10.332227+0000","last_fullsized":"2023-11-27T15:16:10.332227+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:42:45.286967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329413+0000","last_change":"2023-11-27T15:16:09.343202+0000","last_active":"2023-11-27T15:16:10.329413+0000","last_peered":"2023-11-27T15:16:10.329413+0000","last_clean":"2023-11-27T15:16:10.329413+0000","last_became_active":"2023-11-27T15:16:09.342539+0000","last_became_peered":"2023-11-27T15:16:09.342539+0000","last_unstale":"2023-11-27T15:16:10.329413+0000","last_undegraded":"2023-11-27T15:16:10.329413+0000","last_fullsized":"2023-11-27T15:16:10.329413+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:31:31.313017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018587+0000","last_change":"2023-11-27T15:16:09.339915+0000","last_active":"2023-11-27T15:16:11.018587+0000","last_peered":"2023-11-27T15:16:11.018587+0000","last_clean":"2023-11-27T15:16:11.018587+0000","last_became_active":"2023-11-27T15:16:09.339637+0000","last_became_peered":"2023-11-27T15:16:09.339637+0000","last_unstale":"2023-11-27T15:16:11.018587+0000","last_undegraded":"2023-11-27T15:16:11.018587+0000","last_fullsized":"2023-11-27T15:16:11.018587+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:37:13.698277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329316+0000","last_change":"2023-11-27T15:16:09.347852+0000","last_active":"2023-11-27T15:16:10.329316+0000","last_peered":"2023-11-27T15:16:10.329316+0000","last_clean":"2023-11-27T15:16:10.329316+0000","last_became_active":"2023-11-27T15:16:09.347281+0000","last_became_peered":"2023-11-27T15:16:09.347281+0000","last_unstale":"2023-11-27T15:16:10.329316+0000","last_undegraded":"2023-11-27T15:16:10.329316+0000","last_fullsized":"2023-11-27T15:16:10.329316+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:54:00.346911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:13.751316+0000","last_change":"2023-11-27T15:16:08.912206+0000","last_active":"2023-11-27T15:16:13.751316+0000","last_peered":"2023-11-27T15:16:13.751316+0000","last_clean":"2023-11-27T15:16:13.751316+0000","last_became_active":"2023-11-27T15:16:08.911846+0000","last_became_peered":"2023-11-27T15:16:08.911846+0000","last_unstale":"2023-11-27T15:16:13.751316+0000","last_undegraded":"2023-11-27T15:16:13.751316+0000","last_fullsized":"2023-11-27T15:16:13.751316+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:21:37.095544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018680+0000","last_change":"2023-11-27T15:16:09.336485+0000","last_active":"2023-11-27T15:16:11.018680+0000","last_peered":"2023-11-27T15:16:11.018680+0000","last_clean":"2023-11-27T15:16:11.018680+0000","last_became_active":"2023-11-27T15:16:09.336236+0000","last_became_peered":"2023-11-27T15:16:09.336236+0000","last_unstale":"2023-11-27T15:16:11.018680+0000","last_undegraded":"2023-11-27T15:16:11.018680+0000","last_fullsized":"2023-11-27T15:16:11.018680+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:00:31.457583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018872+0000","last_change":"2023-11-27T15:16:08.921198+0000","last_active":"2023-11-27T15:16:11.018872+0000","last_peered":"2023-11-27T15:16:11.018872+0000","last_clean":"2023-11-27T15:16:11.018872+0000","last_became_active":"2023-11-27T15:16:08.920804+0000","last_became_peered":"2023-11-27T15:16:08.920804+0000","last_unstale":"2023-11-27T15:16:11.018872+0000","last_undegraded":"2023-11-27T15:16:11.018872+0000","last_fullsized":"2023-11-27T15:16:11.018872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:25:26.133784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.865444+0000","last_change":"2023-11-27T15:16:09.340642+0000","last_active":"2023-11-27T15:16:10.865444+0000","last_peered":"2023-11-27T15:16:10.865444+0000","last_clean":"2023-11-27T15:16:10.865444+0000","last_became_active":"2023-11-27T15:16:09.340328+0000","last_became_peered":"2023-11-27T15:16:09.340328+0000","last_unstale":"2023-11-27T15:16:10.865444+0000","last_undegraded":"2023-11-27T15:16:10.865444+0000","last_fullsized":"2023-11-27T15:16:10.865444+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:48:12.279600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:13.748861+0000","last_change":"2023-11-27T15:16:08.915317+0000","last_active":"2023-11-27T15:16:13.748861+0000","last_peered":"2023-11-27T15:16:13.748861+0000","last_clean":"2023-11-27T15:16:13.748861+0000","last_became_active":"2023-11-27T15:16:08.914949+0000","last_became_peered":"2023-11-27T15:16:08.914949+0000","last_unstale":"2023-11-27T15:16:13.748861+0000","last_undegraded":"2023-11-27T15:16:13.748861+0000","last_fullsized":"2023-11-27T15:16:13.748861+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:35:47.443716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.332402+0000","last_change":"2023-11-27T15:16:09.348483+0000","last_active":"2023-11-27T15:16:10.332402+0000","last_peered":"2023-11-27T15:16:10.332402+0000","last_clean":"2023-11-27T15:16:10.332402+0000","last_became_active":"2023-11-27T15:16:09.348185+0000","last_became_peered":"2023-11-27T15:16:09.348185+0000","last_unstale":"2023-11-27T15:16:10.332402+0000","last_undegraded":"2023-11-27T15:16:10.332402+0000","last_fullsized":"2023-11-27T15:16:10.332402+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:23:59.373819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018251+0000","last_change":"2023-11-27T15:16:08.911018+0000","last_active":"2023-11-27T15:16:11.018251+0000","last_peered":"2023-11-27T15:16:11.018251+0000","last_clean":"2023-11-27T15:16:11.018251+0000","last_became_active":"2023-11-27T15:16:08.910684+0000","last_became_peered":"2023-11-27T15:16:08.910684+0000","last_unstale":"2023-11-27T15:16:11.018251+0000","last_undegraded":"2023-11-27T15:16:11.018251+0000","last_fullsized":"2023-11-27T15:16:11.018251+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:17:37.062173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018419+0000","last_change":"2023-11-27T15:16:09.341414+0000","last_active":"2023-11-27T15:16:11.018419+0000","last_peered":"2023-11-27T15:16:11.018419+0000","last_clean":"2023-11-27T15:16:11.018419+0000","last_became_active":"2023-11-27T15:16:09.341068+0000","last_became_peered":"2023-11-27T15:16:09.341068+0000","last_unstale":"2023-11-27T15:16:11.018419+0000","last_undegraded":"2023-11-27T15:16:11.018419+0000","last_fullsized":"2023-11-27T15:16:11.018419+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:08:37.252539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":68,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:56.804475+0000","last_change":"2023-11-27T15:16:08.915093+0000","last_active":"2023-11-27T15:16:56.804475+0000","last_peered":"2023-11-27T15:16:56.804475+0000","last_clean":"2023-11-27T15:16:56.804475+0000","last_became_active":"2023-11-27T15:16:08.914761+0000","last_became_peered":"2023-11-27T15:16:08.914761+0000","last_unstale":"2023-11-27T15:16:56.804475+0000","last_undegraded":"2023-11-27T15:16:56.804475+0000","last_fullsized":"2023-11-27T15:16:56.804475+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:36:34.653519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018143+0000","last_change":"2023-11-27T15:16:09.337330+0000","last_active":"2023-11-27T15:16:11.018143+0000","last_peered":"2023-11-27T15:16:11.018143+0000","last_clean":"2023-11-27T15:16:11.018143+0000","last_became_active":"2023-11-27T15:16:09.336991+0000","last_became_peered":"2023-11-27T15:16:09.336991+0000","last_unstale":"2023-11-27T15:16:11.018143+0000","last_undegraded":"2023-11-27T15:16:11.018143+0000","last_fullsized":"2023-11-27T15:16:11.018143+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:12:42.839632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:13.748619+0000","last_change":"2023-11-27T15:16:08.912208+0000","last_active":"2023-11-27T15:16:13.748619+0000","last_peered":"2023-11-27T15:16:13.748619+0000","last_clean":"2023-11-27T15:16:13.748619+0000","last_became_active":"2023-11-27T15:16:08.911879+0000","last_became_peered":"2023-11-27T15:16:08.911879+0000","last_unstale":"2023-11-27T15:16:13.748619+0000","last_undegraded":"2023-11-27T15:16:13.748619+0000","last_fullsized":"2023-11-27T15:16:13.748619+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:05:04.291584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018417+0000","last_change":"2023-11-27T15:16:09.339660+0000","last_active":"2023-11-27T15:16:11.018417+0000","last_peered":"2023-11-27T15:16:11.018417+0000","last_clean":"2023-11-27T15:16:11.018417+0000","last_became_active":"2023-11-27T15:16:09.339323+0000","last_became_peered":"2023-11-27T15:16:09.339323+0000","last_unstale":"2023-11-27T15:16:11.018417+0000","last_undegraded":"2023-11-27T15:16:11.018417+0000","last_fullsized":"2023-11-27T15:16:11.018417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:54:38.570422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:13.750343+0000","last_change":"2023-11-27T15:16:08.911103+0000","last_active":"2023-11-27T15:16:13.750343+0000","last_peered":"2023-11-27T15:16:13.750343+0000","last_clean":"2023-11-27T15:16:13.750343+0000","last_became_active":"2023-11-27T15:16:08.910765+0000","last_became_peered":"2023-11-27T15:16:08.910765+0000","last_unstale":"2023-11-27T15:16:13.750343+0000","last_undegraded":"2023-11-27T15:16:13.750343+0000","last_fullsized":"2023-11-27T15:16:13.750343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:35:58.046428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018483+0000","last_change":"2023-11-27T15:16:09.351741+0000","last_active":"2023-11-27T15:16:11.018483+0000","last_peered":"2023-11-27T15:16:11.018483+0000","last_clean":"2023-11-27T15:16:11.018483+0000","last_became_active":"2023-11-27T15:16:09.351280+0000","last_became_peered":"2023-11-27T15:16:09.351280+0000","last_unstale":"2023-11-27T15:16:11.018483+0000","last_undegraded":"2023-11-27T15:16:11.018483+0000","last_fullsized":"2023-11-27T15:16:11.018483+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:23:19.955510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018060+0000","last_change":"2023-11-27T15:16:08.915630+0000","last_active":"2023-11-27T15:16:11.018060+0000","last_peered":"2023-11-27T15:16:11.018060+0000","last_clean":"2023-11-27T15:16:11.018060+0000","last_became_active":"2023-11-27T15:16:08.915294+0000","last_became_peered":"2023-11-27T15:16:08.915294+0000","last_unstale":"2023-11-27T15:16:11.018060+0000","last_undegraded":"2023-11-27T15:16:11.018060+0000","last_fullsized":"2023-11-27T15:16:11.018060+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:48:25.055189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018460+0000","last_change":"2023-11-27T15:16:09.338436+0000","last_active":"2023-11-27T15:16:11.018460+0000","last_peered":"2023-11-27T15:16:11.018460+0000","last_clean":"2023-11-27T15:16:11.018460+0000","last_became_active":"2023-11-27T15:16:09.338148+0000","last_became_peered":"2023-11-27T15:16:09.338148+0000","last_unstale":"2023-11-27T15:16:11.018460+0000","last_undegraded":"2023-11-27T15:16:11.018460+0000","last_fullsized":"2023-11-27T15:16:11.018460+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:11:53.119360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":62,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:56.804098+0000","last_change":"2023-11-27T15:16:08.913072+0000","last_active":"2023-11-27T15:16:56.804098+0000","last_peered":"2023-11-27T15:16:56.804098+0000","last_clean":"2023-11-27T15:16:56.804098+0000","last_became_active":"2023-11-27T15:16:08.912757+0000","last_became_peered":"2023-11-27T15:16:08.912757+0000","last_unstale":"2023-11-27T15:16:56.804098+0000","last_undegraded":"2023-11-27T15:16:56.804098+0000","last_fullsized":"2023-11-27T15:16:56.804098+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:41:31.325128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":45,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.332666+0000","last_change":"2023-11-27T15:16:09.340380+0000","last_active":"2023-11-27T15:16:10.332666+0000","last_peered":"2023-11-27T15:16:10.332666+0000","last_clean":"2023-11-27T15:16:10.332666+0000","last_became_active":"2023-11-27T15:16:09.339581+0000","last_became_peered":"2023-11-27T15:16:09.339581+0000","last_unstale":"2023-11-27T15:16:10.332666+0000","last_undegraded":"2023-11-27T15:16:10.332666+0000","last_fullsized":"2023-11-27T15:16:10.332666+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:00:41.810407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.743621+0000","last_change":"2023-11-27T15:16:08.917924+0000","last_active":"2023-11-27T15:16:12.743621+0000","last_peered":"2023-11-27T15:16:12.743621+0000","last_clean":"2023-11-27T15:16:12.743621+0000","last_became_active":"2023-11-27T15:16:08.917618+0000","last_became_peered":"2023-11-27T15:16:08.917618+0000","last_unstale":"2023-11-27T15:16:12.743621+0000","last_undegraded":"2023-11-27T15:16:12.743621+0000","last_fullsized":"2023-11-27T15:16:12.743621+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:05:48.700431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.865352+0000","last_change":"2023-11-27T15:16:09.346020+0000","last_active":"2023-11-27T15:16:10.865352+0000","last_peered":"2023-11-27T15:16:10.865352+0000","last_clean":"2023-11-27T15:16:10.865352+0000","last_became_active":"2023-11-27T15:16:09.345822+0000","last_became_peered":"2023-11-27T15:16:09.345822+0000","last_unstale":"2023-11-27T15:16:10.865352+0000","last_undegraded":"2023-11-27T15:16:10.865352+0000","last_fullsized":"2023-11-27T15:16:10.865352+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:20:28.114199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.742118+0000","last_change":"2023-11-27T15:16:08.921810+0000","last_active":"2023-11-27T15:16:12.742118+0000","last_peered":"2023-11-27T15:16:12.742118+0000","last_clean":"2023-11-27T15:16:12.742118+0000","last_became_active":"2023-11-27T15:16:08.921518+0000","last_became_peered":"2023-11-27T15:16:08.921518+0000","last_unstale":"2023-11-27T15:16:12.742118+0000","last_undegraded":"2023-11-27T15:16:12.742118+0000","last_fullsized":"2023-11-27T15:16:12.742118+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:14:14.359239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.332512+0000","last_change":"2023-11-27T15:16:09.351083+0000","last_active":"2023-11-27T15:16:10.332512+0000","last_peered":"2023-11-27T15:16:10.332512+0000","last_clean":"2023-11-27T15:16:10.332512+0000","last_became_active":"2023-11-27T15:16:09.350540+0000","last_became_peered":"2023-11-27T15:16:09.350540+0000","last_unstale":"2023-11-27T15:16:10.332512+0000","last_undegraded":"2023-11-27T15:16:10.332512+0000","last_fullsized":"2023-11-27T15:16:10.332512+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:16:25.645689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'28","reported_seq":140,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:57.805578+0000","last_change":"2023-11-27T15:16:08.913023+0000","last_active":"2023-11-27T15:16:57.805578+0000","last_peered":"2023-11-27T15:16:57.805578+0000","last_clean":"2023-11-27T15:16:57.805578+0000","last_became_active":"2023-11-27T15:16:08.912693+0000","last_became_peered":"2023-11-27T15:16:08.912693+0000","last_unstale":"2023-11-27T15:16:57.805578+0000","last_undegraded":"2023-11-27T15:16:57.805578+0000","last_fullsized":"2023-11-27T15:16:57.805578+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:02:06.453103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58527,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":98,"num_read_kb":105,"num_write":29,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018683+0000","last_change":"2023-11-27T15:16:09.343446+0000","last_active":"2023-11-27T15:16:11.018683+0000","last_peered":"2023-11-27T15:16:11.018683+0000","last_clean":"2023-11-27T15:16:11.018683+0000","last_became_active":"2023-11-27T15:16:09.341980+0000","last_became_peered":"2023-11-27T15:16:09.341980+0000","last_unstale":"2023-11-27T15:16:11.018683+0000","last_undegraded":"2023-11-27T15:16:11.018683+0000","last_fullsized":"2023-11-27T15:16:11.018683+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:03:09.462489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:13.749211+0000","last_change":"2023-11-27T15:16:08.914181+0000","last_active":"2023-11-27T15:16:13.749211+0000","last_peered":"2023-11-27T15:16:13.749211+0000","last_clean":"2023-11-27T15:16:13.749211+0000","last_became_active":"2023-11-27T15:16:08.913836+0000","last_became_peered":"2023-11-27T15:16:08.913836+0000","last_unstale":"2023-11-27T15:16:13.749211+0000","last_undegraded":"2023-11-27T15:16:13.749211+0000","last_fullsized":"2023-11-27T15:16:13.749211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:28:48.961843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018010+0000","last_change":"2023-11-27T15:16:09.344446+0000","last_active":"2023-11-27T15:16:11.018010+0000","last_peered":"2023-11-27T15:16:11.018010+0000","last_clean":"2023-11-27T15:16:11.018010+0000","last_became_active":"2023-11-27T15:16:09.344104+0000","last_became_peered":"2023-11-27T15:16:09.344104+0000","last_unstale":"2023-11-27T15:16:11.018010+0000","last_undegraded":"2023-11-27T15:16:11.018010+0000","last_fullsized":"2023-11-27T15:16:11.018010+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:09:38.847885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.744575+0000","last_change":"2023-11-27T15:16:08.915597+0000","last_active":"2023-11-27T15:16:12.744575+0000","last_peered":"2023-11-27T15:16:12.744575+0000","last_clean":"2023-11-27T15:16:12.744575+0000","last_became_active":"2023-11-27T15:16:08.914727+0000","last_became_peered":"2023-11-27T15:16:08.914727+0000","last_unstale":"2023-11-27T15:16:12.744575+0000","last_undegraded":"2023-11-27T15:16:12.744575+0000","last_fullsized":"2023-11-27T15:16:12.744575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:32:36.396885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017899+0000","last_change":"2023-11-27T15:16:09.352643+0000","last_active":"2023-11-27T15:16:11.017899+0000","last_peered":"2023-11-27T15:16:11.017899+0000","last_clean":"2023-11-27T15:16:11.017899+0000","last_became_active":"2023-11-27T15:16:09.352336+0000","last_became_peered":"2023-11-27T15:16:09.352336+0000","last_unstale":"2023-11-27T15:16:11.017899+0000","last_undegraded":"2023-11-27T15:16:11.017899+0000","last_fullsized":"2023-11-27T15:16:11.017899+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:27:00.130120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'4","reported_seq":68,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:59.834254+0000","last_change":"2023-11-27T15:16:08.914303+0000","last_active":"2023-11-27T15:16:59.834254+0000","last_peered":"2023-11-27T15:16:59.834254+0000","last_clean":"2023-11-27T15:16:59.834254+0000","last_became_active":"2023-11-27T15:16:08.913601+0000","last_became_peered":"2023-11-27T15:16:08.913601+0000","last_unstale":"2023-11-27T15:16:59.834254+0000","last_undegraded":"2023-11-27T15:16:59.834254+0000","last_fullsized":"2023-11-27T15:16:59.834254+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:35:53.186395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017672+0000","last_change":"2023-11-27T15:16:09.341342+0000","last_active":"2023-11-27T15:16:11.017672+0000","last_peered":"2023-11-27T15:16:11.017672+0000","last_clean":"2023-11-27T15:16:11.017672+0000","last_became_active":"2023-11-27T15:16:09.341100+0000","last_became_peered":"2023-11-27T15:16:09.341100+0000","last_unstale":"2023-11-27T15:16:11.017672+0000","last_undegraded":"2023-11-27T15:16:11.017672+0000","last_fullsized":"2023-11-27T15:16:11.017672+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:52:19.668174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:13.750916+0000","last_change":"2023-11-27T15:16:08.921771+0000","last_active":"2023-11-27T15:16:13.750916+0000","last_peered":"2023-11-27T15:16:13.750916+0000","last_clean":"2023-11-27T15:16:13.750916+0000","last_became_active":"2023-11-27T15:16:08.921396+0000","last_became_peered":"2023-11-27T15:16:08.921396+0000","last_unstale":"2023-11-27T15:16:13.750916+0000","last_undegraded":"2023-11-27T15:16:13.750916+0000","last_fullsized":"2023-11-27T15:16:13.750916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:32:00.774950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018456+0000","last_change":"2023-11-27T15:16:09.341258+0000","last_active":"2023-11-27T15:16:11.018456+0000","last_peered":"2023-11-27T15:16:11.018456+0000","last_clean":"2023-11-27T15:16:11.018456+0000","last_became_active":"2023-11-27T15:16:09.340921+0000","last_became_peered":"2023-11-27T15:16:09.340921+0000","last_unstale":"2023-11-27T15:16:11.018456+0000","last_undegraded":"2023-11-27T15:16:11.018456+0000","last_fullsized":"2023-11-27T15:16:11.018456+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:59:25.144109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:13.738069+0000","last_change":"2023-11-27T15:16:08.911219+0000","last_active":"2023-11-27T15:16:13.738069+0000","last_peered":"2023-11-27T15:16:13.738069+0000","last_clean":"2023-11-27T15:16:13.738069+0000","last_became_active":"2023-11-27T15:16:08.910788+0000","last_became_peered":"2023-11-27T15:16:08.910788+0000","last_unstale":"2023-11-27T15:16:13.738069+0000","last_undegraded":"2023-11-27T15:16:13.738069+0000","last_fullsized":"2023-11-27T15:16:13.738069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:02:11.224624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329287+0000","last_change":"2023-11-27T15:16:09.343793+0000","last_active":"2023-11-27T15:16:10.329287+0000","last_peered":"2023-11-27T15:16:10.329287+0000","last_clean":"2023-11-27T15:16:10.329287+0000","last_became_active":"2023-11-27T15:16:09.343480+0000","last_became_peered":"2023-11-27T15:16:09.343480+0000","last_unstale":"2023-11-27T15:16:10.329287+0000","last_undegraded":"2023-11-27T15:16:10.329287+0000","last_fullsized":"2023-11-27T15:16:10.329287+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:23:15.936537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018282+0000","last_change":"2023-11-27T15:16:08.920651+0000","last_active":"2023-11-27T15:16:11.018282+0000","last_peered":"2023-11-27T15:16:11.018282+0000","last_clean":"2023-11-27T15:16:11.018282+0000","last_became_active":"2023-11-27T15:16:08.920294+0000","last_became_peered":"2023-11-27T15:16:08.920294+0000","last_unstale":"2023-11-27T15:16:11.018282+0000","last_undegraded":"2023-11-27T15:16:11.018282+0000","last_fullsized":"2023-11-27T15:16:11.018282+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:57:51.796562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:13.149060+0000","last_change":"2023-11-27T15:16:09.348279+0000","last_active":"2023-11-27T15:16:13.149060+0000","last_peered":"2023-11-27T15:16:13.149060+0000","last_clean":"2023-11-27T15:16:13.149060+0000","last_became_active":"2023-11-27T15:16:09.347760+0000","last_became_peered":"2023-11-27T15:16:09.347760+0000","last_unstale":"2023-11-27T15:16:13.149060+0000","last_undegraded":"2023-11-27T15:16:13.149060+0000","last_fullsized":"2023-11-27T15:16:13.149060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:35:32.477154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.170186+0000","last_change":"2023-11-27T15:16:08.915622+0000","last_active":"2023-11-27T15:16:11.170186+0000","last_peered":"2023-11-27T15:16:11.170186+0000","last_clean":"2023-11-27T15:16:11.170186+0000","last_became_active":"2023-11-27T15:16:08.915293+0000","last_became_peered":"2023-11-27T15:16:08.915293+0000","last_unstale":"2023-11-27T15:16:11.170186+0000","last_undegraded":"2023-11-27T15:16:11.170186+0000","last_fullsized":"2023-11-27T15:16:11.170186+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:42:31.199595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.019331+0000","last_change":"2023-11-27T15:16:09.337096+0000","last_active":"2023-11-27T15:16:11.019331+0000","last_peered":"2023-11-27T15:16:11.019331+0000","last_clean":"2023-11-27T15:16:11.019331+0000","last_became_active":"2023-11-27T15:16:09.336359+0000","last_became_peered":"2023-11-27T15:16:09.336359+0000","last_unstale":"2023-11-27T15:16:11.019331+0000","last_undegraded":"2023-11-27T15:16:11.019331+0000","last_fullsized":"2023-11-27T15:16:11.019331+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:01:29.480942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018063+0000","last_change":"2023-11-27T15:16:09.340478+0000","last_active":"2023-11-27T15:16:11.018063+0000","last_peered":"2023-11-27T15:16:11.018063+0000","last_clean":"2023-11-27T15:16:11.018063+0000","last_became_active":"2023-11-27T15:16:09.340214+0000","last_became_peered":"2023-11-27T15:16:09.340214+0000","last_unstale":"2023-11-27T15:16:11.018063+0000","last_undegraded":"2023-11-27T15:16:11.018063+0000","last_fullsized":"2023-11-27T15:16:11.018063+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:09:06.499851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":63,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:57.829199+0000","last_change":"2023-11-27T15:16:08.920387+0000","last_active":"2023-11-27T15:16:57.829199+0000","last_peered":"2023-11-27T15:16:57.829199+0000","last_clean":"2023-11-27T15:16:57.829199+0000","last_became_active":"2023-11-27T15:16:08.920004+0000","last_became_peered":"2023-11-27T15:16:08.920004+0000","last_unstale":"2023-11-27T15:16:57.829199+0000","last_undegraded":"2023-11-27T15:16:57.829199+0000","last_fullsized":"2023-11-27T15:16:57.829199+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:02:23.895980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":45,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018879+0000","last_change":"2023-11-27T15:16:09.352291+0000","last_active":"2023-11-27T15:16:11.018879+0000","last_peered":"2023-11-27T15:16:11.018879+0000","last_clean":"2023-11-27T15:16:11.018879+0000","last_became_active":"2023-11-27T15:16:09.351944+0000","last_became_peered":"2023-11-27T15:16:09.351944+0000","last_unstale":"2023-11-27T15:16:11.018879+0000","last_undegraded":"2023-11-27T15:16:11.018879+0000","last_fullsized":"2023-11-27T15:16:11.018879+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:59:40.185533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:13.748748+0000","last_change":"2023-11-27T15:16:08.909694+0000","last_active":"2023-11-27T15:16:13.748748+0000","last_peered":"2023-11-27T15:16:13.748748+0000","last_clean":"2023-11-27T15:16:13.748748+0000","last_became_active":"2023-11-27T15:16:08.909324+0000","last_became_peered":"2023-11-27T15:16:08.909324+0000","last_unstale":"2023-11-27T15:16:13.748748+0000","last_undegraded":"2023-11-27T15:16:13.748748+0000","last_fullsized":"2023-11-27T15:16:13.748748+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:37:52.187833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":340,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017780+0000","last_change":"2023-11-27T15:15:10.678805+0000","last_active":"2023-11-27T15:16:11.017780+0000","last_peered":"2023-11-27T15:16:11.017780+0000","last_clean":"2023-11-27T15:16:11.017780+0000","last_became_active":"2023-11-27T15:15:10.678267+0000","last_became_peered":"2023-11-27T15:15:10.678267+0000","last_unstale":"2023-11-27T15:16:11.017780+0000","last_undegraded":"2023-11-27T15:16:11.017780+0000","last_fullsized":"2023-11-27T15:16:11.017780+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:12:14.179315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:12:14.179315+0000","last_clean_scrub_stamp":"2023-11-27T15:12:14.179315+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:53:29.305985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017431+0000","last_change":"2023-11-27T15:16:09.346622+0000","last_active":"2023-11-27T15:16:11.017431+0000","last_peered":"2023-11-27T15:16:11.017431+0000","last_clean":"2023-11-27T15:16:11.017431+0000","last_became_active":"2023-11-27T15:16:09.346287+0000","last_became_peered":"2023-11-27T15:16:09.346287+0000","last_unstale":"2023-11-27T15:16:11.017431+0000","last_undegraded":"2023-11-27T15:16:11.017431+0000","last_fullsized":"2023-11-27T15:16:11.017431+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:06:51.424504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:13.749235+0000","last_change":"2023-11-27T15:16:08.915598+0000","last_active":"2023-11-27T15:16:13.749235+0000","last_peered":"2023-11-27T15:16:13.749235+0000","last_clean":"2023-11-27T15:16:13.749235+0000","last_became_active":"2023-11-27T15:16:08.915255+0000","last_became_peered":"2023-11-27T15:16:08.915255+0000","last_unstale":"2023-11-27T15:16:13.749235+0000","last_undegraded":"2023-11-27T15:16:13.749235+0000","last_fullsized":"2023-11-27T15:16:13.749235+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:59:36.190517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.743609+0000","last_change":"2023-11-27T15:16:08.921855+0000","last_active":"2023-11-27T15:16:12.743609+0000","last_peered":"2023-11-27T15:16:12.743609+0000","last_clean":"2023-11-27T15:16:12.743609+0000","last_became_active":"2023-11-27T15:16:08.921525+0000","last_became_peered":"2023-11-27T15:16:08.921525+0000","last_unstale":"2023-11-27T15:16:12.743609+0000","last_undegraded":"2023-11-27T15:16:12.743609+0000","last_fullsized":"2023-11-27T15:16:12.743609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:18:46.190023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017617+0000","last_change":"2023-11-27T15:16:09.345119+0000","last_active":"2023-11-27T15:16:11.017617+0000","last_peered":"2023-11-27T15:16:11.017617+0000","last_clean":"2023-11-27T15:16:11.017617+0000","last_became_active":"2023-11-27T15:16:09.344807+0000","last_became_peered":"2023-11-27T15:16:09.344807+0000","last_unstale":"2023-11-27T15:16:11.017617+0000","last_undegraded":"2023-11-27T15:16:11.017617+0000","last_fullsized":"2023-11-27T15:16:11.017617+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:35:43.938240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:19.643866+0000","last_change":"2023-11-27T15:16:08.911564+0000","last_active":"2023-11-27T15:16:19.643866+0000","last_peered":"2023-11-27T15:16:19.643866+0000","last_clean":"2023-11-27T15:16:19.643866+0000","last_became_active":"2023-11-27T15:16:08.911057+0000","last_became_peered":"2023-11-27T15:16:08.911057+0000","last_unstale":"2023-11-27T15:16:19.643866+0000","last_undegraded":"2023-11-27T15:16:19.643866+0000","last_fullsized":"2023-11-27T15:16:19.643866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:12:28.660065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018236+0000","last_change":"2023-11-27T15:16:09.339085+0000","last_active":"2023-11-27T15:16:11.018236+0000","last_peered":"2023-11-27T15:16:11.018236+0000","last_clean":"2023-11-27T15:16:11.018236+0000","last_became_active":"2023-11-27T15:16:09.338777+0000","last_became_peered":"2023-11-27T15:16:09.338777+0000","last_unstale":"2023-11-27T15:16:11.018236+0000","last_undegraded":"2023-11-27T15:16:11.018236+0000","last_fullsized":"2023-11-27T15:16:11.018236+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:37:58.241374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.744087+0000","last_change":"2023-11-27T15:16:08.913939+0000","last_active":"2023-11-27T15:16:12.744087+0000","last_peered":"2023-11-27T15:16:12.744087+0000","last_clean":"2023-11-27T15:16:12.744087+0000","last_became_active":"2023-11-27T15:16:08.913629+0000","last_became_peered":"2023-11-27T15:16:08.913629+0000","last_unstale":"2023-11-27T15:16:12.744087+0000","last_undegraded":"2023-11-27T15:16:12.744087+0000","last_fullsized":"2023-11-27T15:16:12.744087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:46:10.524140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.865269+0000","last_change":"2023-11-27T15:16:09.350712+0000","last_active":"2023-11-27T15:16:10.865269+0000","last_peered":"2023-11-27T15:16:10.865269+0000","last_clean":"2023-11-27T15:16:10.865269+0000","last_became_active":"2023-11-27T15:16:09.350359+0000","last_became_peered":"2023-11-27T15:16:09.350359+0000","last_unstale":"2023-11-27T15:16:10.865269+0000","last_undegraded":"2023-11-27T15:16:10.865269+0000","last_fullsized":"2023-11-27T15:16:10.865269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:30:08.385906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.744120+0000","last_change":"2023-11-27T15:16:08.919501+0000","last_active":"2023-11-27T15:16:12.744120+0000","last_peered":"2023-11-27T15:16:12.744120+0000","last_clean":"2023-11-27T15:16:12.744120+0000","last_became_active":"2023-11-27T15:16:08.919258+0000","last_became_peered":"2023-11-27T15:16:08.919258+0000","last_unstale":"2023-11-27T15:16:12.744120+0000","last_undegraded":"2023-11-27T15:16:12.744120+0000","last_fullsized":"2023-11-27T15:16:12.744120+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:12:35.863121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017921+0000","last_change":"2023-11-27T15:16:09.344066+0000","last_active":"2023-11-27T15:16:11.017921+0000","last_peered":"2023-11-27T15:16:11.017921+0000","last_clean":"2023-11-27T15:16:11.017921+0000","last_became_active":"2023-11-27T15:16:09.343757+0000","last_became_peered":"2023-11-27T15:16:09.343757+0000","last_unstale":"2023-11-27T15:16:11.017921+0000","last_undegraded":"2023-11-27T15:16:11.017921+0000","last_fullsized":"2023-11-27T15:16:11.017921+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:45:10.744579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.744748+0000","last_change":"2023-11-27T15:16:08.906205+0000","last_active":"2023-11-27T15:16:12.744748+0000","last_peered":"2023-11-27T15:16:12.744748+0000","last_clean":"2023-11-27T15:16:12.744748+0000","last_became_active":"2023-11-27T15:16:08.905885+0000","last_became_peered":"2023-11-27T15:16:08.905885+0000","last_unstale":"2023-11-27T15:16:12.744748+0000","last_undegraded":"2023-11-27T15:16:12.744748+0000","last_fullsized":"2023-11-27T15:16:12.744748+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:15:37.488708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.328941+0000","last_change":"2023-11-27T15:16:09.349642+0000","last_active":"2023-11-27T15:16:10.328941+0000","last_peered":"2023-11-27T15:16:10.328941+0000","last_clean":"2023-11-27T15:16:10.328941+0000","last_became_active":"2023-11-27T15:16:09.349086+0000","last_became_peered":"2023-11-27T15:16:09.349086+0000","last_unstale":"2023-11-27T15:16:10.328941+0000","last_undegraded":"2023-11-27T15:16:10.328941+0000","last_fullsized":"2023-11-27T15:16:10.328941+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:54:34.111407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.796803+0000","last_change":"2023-11-27T15:16:08.917835+0000","last_active":"2023-11-27T15:16:12.796803+0000","last_peered":"2023-11-27T15:16:12.796803+0000","last_clean":"2023-11-27T15:16:12.796803+0000","last_became_active":"2023-11-27T15:16:08.917532+0000","last_became_peered":"2023-11-27T15:16:08.917532+0000","last_unstale":"2023-11-27T15:16:12.796803+0000","last_undegraded":"2023-11-27T15:16:12.796803+0000","last_fullsized":"2023-11-27T15:16:12.796803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:35:27.771748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.019054+0000","last_change":"2023-11-27T15:16:09.338459+0000","last_active":"2023-11-27T15:16:11.019054+0000","last_peered":"2023-11-27T15:16:11.019054+0000","last_clean":"2023-11-27T15:16:11.019054+0000","last_became_active":"2023-11-27T15:16:09.338148+0000","last_became_peered":"2023-11-27T15:16:09.338148+0000","last_unstale":"2023-11-27T15:16:11.019054+0000","last_undegraded":"2023-11-27T15:16:11.019054+0000","last_fullsized":"2023-11-27T15:16:11.019054+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:50:17.721161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:12.743072+0000","last_change":"2023-11-27T15:16:08.915401+0000","last_active":"2023-11-27T15:16:12.743072+0000","last_peered":"2023-11-27T15:16:12.743072+0000","last_clean":"2023-11-27T15:16:12.743072+0000","last_became_active":"2023-11-27T15:16:08.915012+0000","last_became_peered":"2023-11-27T15:16:08.915012+0000","last_unstale":"2023-11-27T15:16:12.743072+0000","last_undegraded":"2023-11-27T15:16:12.743072+0000","last_fullsized":"2023-11-27T15:16:12.743072+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:25:39.081595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018186+0000","last_change":"2023-11-27T15:16:09.352309+0000","last_active":"2023-11-27T15:16:11.018186+0000","last_peered":"2023-11-27T15:16:11.018186+0000","last_clean":"2023-11-27T15:16:11.018186+0000","last_became_active":"2023-11-27T15:16:09.352039+0000","last_became_peered":"2023-11-27T15:16:09.352039+0000","last_unstale":"2023-11-27T15:16:11.018186+0000","last_undegraded":"2023-11-27T15:16:11.018186+0000","last_fullsized":"2023-11-27T15:16:11.018186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:02:54.926789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:13.751357+0000","last_change":"2023-11-27T15:16:08.920446+0000","last_active":"2023-11-27T15:16:13.751357+0000","last_peered":"2023-11-27T15:16:13.751357+0000","last_clean":"2023-11-27T15:16:13.751357+0000","last_became_active":"2023-11-27T15:16:08.920232+0000","last_became_peered":"2023-11-27T15:16:08.920232+0000","last_unstale":"2023-11-27T15:16:13.751357+0000","last_undegraded":"2023-11-27T15:16:13.751357+0000","last_fullsized":"2023-11-27T15:16:13.751357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:28:22.523270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018226+0000","last_change":"2023-11-27T15:16:09.344328+0000","last_active":"2023-11-27T15:16:11.018226+0000","last_peered":"2023-11-27T15:16:11.018226+0000","last_clean":"2023-11-27T15:16:11.018226+0000","last_became_active":"2023-11-27T15:16:09.344047+0000","last_became_peered":"2023-11-27T15:16:09.344047+0000","last_unstale":"2023-11-27T15:16:11.018226+0000","last_undegraded":"2023-11-27T15:16:11.018226+0000","last_fullsized":"2023-11-27T15:16:11.018226+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:10:10.564306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018219+0000","last_change":"2023-11-27T15:16:08.909865+0000","last_active":"2023-11-27T15:16:11.018219+0000","last_peered":"2023-11-27T15:16:11.018219+0000","last_clean":"2023-11-27T15:16:11.018219+0000","last_became_active":"2023-11-27T15:16:08.909559+0000","last_became_peered":"2023-11-27T15:16:08.909559+0000","last_unstale":"2023-11-27T15:16:11.018219+0000","last_undegraded":"2023-11-27T15:16:11.018219+0000","last_fullsized":"2023-11-27T15:16:11.018219+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:21:20.674879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.170220+0000","last_change":"2023-11-27T15:16:09.342525+0000","last_active":"2023-11-27T15:16:11.170220+0000","last_peered":"2023-11-27T15:16:11.170220+0000","last_clean":"2023-11-27T15:16:11.170220+0000","last_became_active":"2023-11-27T15:16:09.340987+0000","last_became_peered":"2023-11-27T15:16:09.340987+0000","last_unstale":"2023-11-27T15:16:11.170220+0000","last_undegraded":"2023-11-27T15:16:11.170220+0000","last_fullsized":"2023-11-27T15:16:11.170220+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:57:33.064763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017934+0000","last_change":"2023-11-27T15:16:08.914088+0000","last_active":"2023-11-27T15:16:11.017934+0000","last_peered":"2023-11-27T15:16:11.017934+0000","last_clean":"2023-11-27T15:16:11.017934+0000","last_became_active":"2023-11-27T15:16:08.913750+0000","last_became_peered":"2023-11-27T15:16:08.913750+0000","last_unstale":"2023-11-27T15:16:11.017934+0000","last_undegraded":"2023-11-27T15:16:11.017934+0000","last_fullsized":"2023-11-27T15:16:11.017934+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:52:50.990435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018281+0000","last_change":"2023-11-27T15:16:09.344403+0000","last_active":"2023-11-27T15:16:11.018281+0000","last_peered":"2023-11-27T15:16:11.018281+0000","last_clean":"2023-11-27T15:16:11.018281+0000","last_became_active":"2023-11-27T15:16:09.344120+0000","last_became_peered":"2023-11-27T15:16:09.344120+0000","last_unstale":"2023-11-27T15:16:11.018281+0000","last_undegraded":"2023-11-27T15:16:11.018281+0000","last_fullsized":"2023-11-27T15:16:11.018281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:47:52.787834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.328999+0000","last_change":"2023-11-27T15:16:09.349780+0000","last_active":"2023-11-27T15:16:10.328999+0000","last_peered":"2023-11-27T15:16:10.328999+0000","last_clean":"2023-11-27T15:16:10.328999+0000","last_became_active":"2023-11-27T15:16:09.349493+0000","last_became_peered":"2023-11-27T15:16:09.349493+0000","last_unstale":"2023-11-27T15:16:10.328999+0000","last_undegraded":"2023-11-27T15:16:10.328999+0000","last_fullsized":"2023-11-27T15:16:10.328999+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:11:07.759158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329634+0000","last_change":"2023-11-27T15:16:09.340528+0000","last_active":"2023-11-27T15:16:10.329634+0000","last_peered":"2023-11-27T15:16:10.329634+0000","last_clean":"2023-11-27T15:16:10.329634+0000","last_became_active":"2023-11-27T15:16:09.339435+0000","last_became_peered":"2023-11-27T15:16:09.339435+0000","last_unstale":"2023-11-27T15:16:10.329634+0000","last_undegraded":"2023-11-27T15:16:10.329634+0000","last_fullsized":"2023-11-27T15:16:10.329634+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:08:23.254070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329091+0000","last_change":"2023-11-27T15:16:09.337989+0000","last_active":"2023-11-27T15:16:10.329091+0000","last_peered":"2023-11-27T15:16:10.329091+0000","last_clean":"2023-11-27T15:16:10.329091+0000","last_became_active":"2023-11-27T15:16:09.336712+0000","last_became_peered":"2023-11-27T15:16:09.336712+0000","last_unstale":"2023-11-27T15:16:10.329091+0000","last_undegraded":"2023-11-27T15:16:10.329091+0000","last_fullsized":"2023-11-27T15:16:10.329091+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:53:19.057499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.865031+0000","last_change":"2023-11-27T15:16:09.347077+0000","last_active":"2023-11-27T15:16:10.865031+0000","last_peered":"2023-11-27T15:16:10.865031+0000","last_clean":"2023-11-27T15:16:10.865031+0000","last_became_active":"2023-11-27T15:16:09.346722+0000","last_became_peered":"2023-11-27T15:16:09.346722+0000","last_unstale":"2023-11-27T15:16:10.865031+0000","last_undegraded":"2023-11-27T15:16:10.865031+0000","last_fullsized":"2023-11-27T15:16:10.865031+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:09:22.987678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017743+0000","last_change":"2023-11-27T15:16:09.344317+0000","last_active":"2023-11-27T15:16:11.017743+0000","last_peered":"2023-11-27T15:16:11.017743+0000","last_clean":"2023-11-27T15:16:11.017743+0000","last_became_active":"2023-11-27T15:16:09.343962+0000","last_became_peered":"2023-11-27T15:16:09.343962+0000","last_unstale":"2023-11-27T15:16:11.017743+0000","last_undegraded":"2023-11-27T15:16:11.017743+0000","last_fullsized":"2023-11-27T15:16:11.017743+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:49:36.883005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329146+0000","last_change":"2023-11-27T15:16:09.344690+0000","last_active":"2023-11-27T15:16:10.329146+0000","last_peered":"2023-11-27T15:16:10.329146+0000","last_clean":"2023-11-27T15:16:10.329146+0000","last_became_active":"2023-11-27T15:16:09.344259+0000","last_became_peered":"2023-11-27T15:16:09.344259+0000","last_unstale":"2023-11-27T15:16:10.329146+0000","last_undegraded":"2023-11-27T15:16:10.329146+0000","last_fullsized":"2023-11-27T15:16:10.329146+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:47:31.777045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329769+0000","last_change":"2023-11-27T15:16:09.340459+0000","last_active":"2023-11-27T15:16:10.329769+0000","last_peered":"2023-11-27T15:16:10.329769+0000","last_clean":"2023-11-27T15:16:10.329769+0000","last_became_active":"2023-11-27T15:16:09.340140+0000","last_became_peered":"2023-11-27T15:16:09.340140+0000","last_unstale":"2023-11-27T15:16:10.329769+0000","last_undegraded":"2023-11-27T15:16:10.329769+0000","last_fullsized":"2023-11-27T15:16:10.329769+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:51:52.323186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018191+0000","last_change":"2023-11-27T15:16:09.336625+0000","last_active":"2023-11-27T15:16:11.018191+0000","last_peered":"2023-11-27T15:16:11.018191+0000","last_clean":"2023-11-27T15:16:11.018191+0000","last_became_active":"2023-11-27T15:16:09.336287+0000","last_became_peered":"2023-11-27T15:16:09.336287+0000","last_unstale":"2023-11-27T15:16:11.018191+0000","last_undegraded":"2023-11-27T15:16:11.018191+0000","last_fullsized":"2023-11-27T15:16:11.018191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:19:26.038950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017922+0000","last_change":"2023-11-27T15:16:09.335480+0000","last_active":"2023-11-27T15:16:11.017922+0000","last_peered":"2023-11-27T15:16:11.017922+0000","last_clean":"2023-11-27T15:16:11.017922+0000","last_became_active":"2023-11-27T15:16:09.335163+0000","last_became_peered":"2023-11-27T15:16:09.335163+0000","last_unstale":"2023-11-27T15:16:11.017922+0000","last_undegraded":"2023-11-27T15:16:11.017922+0000","last_fullsized":"2023-11-27T15:16:11.017922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:27:19.391538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.017703+0000","last_change":"2023-11-27T15:16:09.336879+0000","last_active":"2023-11-27T15:16:11.017703+0000","last_peered":"2023-11-27T15:16:11.017703+0000","last_clean":"2023-11-27T15:16:11.017703+0000","last_became_active":"2023-11-27T15:16:09.336580+0000","last_became_peered":"2023-11-27T15:16:09.336580+0000","last_unstale":"2023-11-27T15:16:11.017703+0000","last_undegraded":"2023-11-27T15:16:11.017703+0000","last_fullsized":"2023-11-27T15:16:11.017703+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:45:37.465813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.332430+0000","last_change":"2023-11-27T15:16:09.341400+0000","last_active":"2023-11-27T15:16:10.332430+0000","last_peered":"2023-11-27T15:16:10.332430+0000","last_clean":"2023-11-27T15:16:10.332430+0000","last_became_active":"2023-11-27T15:16:09.341103+0000","last_became_peered":"2023-11-27T15:16:09.341103+0000","last_unstale":"2023-11-27T15:16:10.332430+0000","last_undegraded":"2023-11-27T15:16:10.332430+0000","last_fullsized":"2023-11-27T15:16:10.332430+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:59:30.634393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:11.018153+0000","last_change":"2023-11-27T15:16:09.345938+0000","last_active":"2023-11-27T15:16:11.018153+0000","last_peered":"2023-11-27T15:16:11.018153+0000","last_clean":"2023-11-27T15:16:11.018153+0000","last_became_active":"2023-11-27T15:16:09.345624+0000","last_became_peered":"2023-11-27T15:16:09.345624+0000","last_unstale":"2023-11-27T15:16:11.018153+0000","last_undegraded":"2023-11-27T15:16:11.018153+0000","last_fullsized":"2023-11-27T15:16:11.018153+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:03:33.299715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-27T15:16:10.329231+0000","last_change":"2023-11-27T15:16:09.343896+0000","last_active":"2023-11-27T15:16:10.329231+0000","last_peered":"2023-11-27T15:16:10.329231+0000","last_clean":"2023-11-27T15:16:10.329231+0000","last_became_active":"2023-11-27T15:16:09.343603+0000","last_became_peered":"2023-11-27T15:16:09.343603+0000","last_unstale":"2023-11-27T15:16:10.329231+0000","last_undegraded":"2023-11-27T15:16:10.329231+0000","last_fullsized":"2023-11-27T15:16:10.329231+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:58:06.401944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":249380,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62821,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":300,"num_read_kb":215,"num_write":151,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":188463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449110,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5936,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":93739216,"statfs":{"total":95995035648,"available":95988957184,"internally_reserved":0,"allocated":892928,"data_stored":426878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.436},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.33400000000000002}]},{"osd":1,"last update":"Mon Nov 27 15:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.51500000000000001}]},{"osd":2,"last update":"Mon Nov 27 15:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.66400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":4,"last update":"Mon Nov 27 15:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.57099999999999995}]},{"osd":5,"last update":"Mon Nov 27 15:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.72099999999999997}]},{"osd":6,"last update":"Mon Nov 27 15:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.53000000000000003}]},{"osd":7,"last update":"Mon Nov 27 15:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.746},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.629}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":9,"last update":"Mon Nov 27 15:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.80900000000000005}]},{"osd":10,"last update":"Mon Nov 27 15:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.706,"5min":1.706,"15min":1.706},"last":0.64000000000000001}]}]},{"osd":10,"up_from":71,"seq":304942678042,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6404,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738748,"statfs":{"total":95995035648,"available":95988477952,"internally_reserved":0,"allocated":1044480,"data_stored":1077234,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":1,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.52600000000000002}]},{"osd":2,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.747},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.76600000000000001}]},{"osd":3,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.66400000000000003}]},{"osd":4,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.64600000000000002}]},{"osd":5,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.81699999999999995}]},{"osd":6,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.876},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.68300000000000005}]},{"osd":7,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.53300000000000003}]},{"osd":8,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.80000000000000004}]},{"osd":9,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.85299999999999998}]},{"osd":11,"last update":"Mon Nov 27 15:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.90300000000000002}]}]},{"osd":9,"up_from":66,"seq":283467841567,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6080,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739072,"statfs":{"total":95995035648,"available":95988809728,"internally_reserved":0,"allocated":909312,"data_stored":427114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.437,"15min":0.443},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.65300000000000002,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44500000000000001,"15min":0.44800000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.34599999999999997}]},{"osd":1,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.61099999999999999,"15min":0.61499999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.628,"15min":0.63800000000000001},"min":{"1min":0.42499999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77900000000000003,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.52300000000000002}]},{"osd":2,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64100000000000001,"15min":0.64200000000000002},"min":{"1min":0.42899999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.46300000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.81999999999999995,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.48499999999999999}]},{"osd":3,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.52900000000000003,"15min":0.53500000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.59499999999999997,"15min":0.61199999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65500000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.376}]},{"osd":4,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.65200000000000002,"15min":0.66600000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79300000000000004,"5min":1.236,"15min":1.236},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.70299999999999996}]},{"osd":5,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.70899999999999996,"15min":0.72699999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.82499999999999996,"5min":1.274,"15min":1.274},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.68500000000000005,"15min":0.68700000000000006},"min":{"1min":0.439,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.73299999999999998}]},{"osd":6,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.69499999999999995,"15min":0.70899999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":2.9020000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.71199999999999997,"15min":0.73399999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.77100000000000002,"5min":1.351,"15min":1.351},"last":0.55400000000000005}]},{"osd":7,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.75800000000000001,"15min":0.76200000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.77900000000000003,"15min":0.79500000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.92000000000000004,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.69299999999999995}]},{"osd":8,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79600000000000004,"15min":0.79900000000000004},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.78500000000000003,"15min":0.80300000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.86699999999999999,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.60799999999999998}]},{"osd":10,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.622},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.67100000000000004}]}]},{"osd":8,"up_from":58,"seq":249108103203,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6272,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738880,"statfs":{"total":95995035648,"available":95988613120,"internally_reserved":0,"allocated":909312,"data_stored":427034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.499,"15min":0.495},"min":{"1min":0.35599999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.503,"15min":0.496},"min":{"1min":0.375,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.55900000000000005}]},{"osd":1,"last update":"Mon Nov 27 15:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.55700000000000005,"15min":0.55100000000000005},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56599999999999995,"15min":0.56100000000000005},"min":{"1min":0.41899999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.72199999999999998}]},{"osd":2,"last update":"Mon Nov 27 15:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.45600000000000002,"15min":0.44700000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.44800000000000001,"15min":0.441},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.52100000000000002}]},{"osd":3,"last update":"Mon Nov 27 15:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53800000000000003,"15min":0.53100000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.56399999999999995,"15min":0.55600000000000005},"min":{"1min":0.40899999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.61799999999999999}]},{"osd":4,"last update":"Mon Nov 27 15:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.58999999999999997,"15min":0.58199999999999996},"min":{"1min":0.443,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.75},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63400000000000001,"15min":0.628},"min":{"1min":0.48699999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94499999999999995,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.77500000000000002}]},{"osd":5,"last update":"Mon Nov 27 15:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.55700000000000005,"15min":0.54600000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.64100000000000001}]},{"osd":6,"last update":"Mon Nov 27 15:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.66100000000000003,"15min":0.65300000000000002},"min":{"1min":0.50700000000000001,"5min":0.434,"15min":0.434},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.65000000000000002,"15min":0.64500000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.73199999999999998}]},{"osd":7,"last update":"Mon Nov 27 15:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67400000000000004,"15min":0.66900000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.68999999999999995,"15min":0.68200000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.78500000000000003}]},{"osd":9,"last update":"Mon Nov 27 15:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.72899999999999998,"15min":0.71699999999999997},"min":{"1min":0.52600000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.74399999999999999,"15min":0.73099999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.82199999999999995}]},{"osd":10,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.85699999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]}]},{"osd":7,"up_from":50,"seq":214748364839,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6384,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738768,"statfs":{"total":95995035648,"available":95988498432,"internally_reserved":0,"allocated":958464,"data_stored":1019263,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.35599999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.76300000000000001,"5min":1.052,"15min":1.052},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.51800000000000002,"15min":0.503},"min":{"1min":0.315,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.38700000000000001}]},{"osd":1,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.39400000000000002,"15min":0.38200000000000001},"min":{"1min":0.33400000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.60499999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.40999999999999998,"15min":0.39900000000000002},"min":{"1min":0.28199999999999997,"5min":0.254,"15min":0.254},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.371}]},{"osd":2,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.39500000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.55300000000000005,"15min":0.54100000000000004},"min":{"1min":0.41499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.47399999999999998}]},{"osd":3,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.54000000000000004,"15min":0.52400000000000002},"min":{"1min":0.38800000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.59299999999999997,"15min":0.57999999999999996},"min":{"1min":0.45700000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88300000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.67500000000000004}]},{"osd":4,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.49299999999999999,"15min":0.47499999999999998},"min":{"1min":0.43099999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.495},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.48999999999999999,"15min":0.47399999999999998},"min":{"1min":0.41399999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.39800000000000002}]},{"osd":5,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.61799999999999999,"15min":0.59499999999999997},"min":{"1min":0.47099999999999997,"5min":0.375,"15min":0.375},"max":{"1min":2.3719999999999999,"5min":2.3719999999999999,"15min":2.3719999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.64100000000000001,"15min":0.61199999999999999},"min":{"1min":0.44700000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":3.1059999999999999,"5min":3.1059999999999999,"15min":3.1059999999999999},"last":0.68700000000000006}]},{"osd":6,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.66600000000000004,"15min":0.65400000000000003},"min":{"1min":0.51200000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.95299999999999996,"5min":1.008,"15min":1.008},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.625,"15min":0.61099999999999999},"min":{"1min":0.52900000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.66300000000000003}]},{"osd":8,"last update":"Mon Nov 27 15:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.66500000000000004,"15min":0.65300000000000002},"min":{"1min":0.51700000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.96099999999999997,"5min":1.157,"15min":1.157},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.72799999999999998,"15min":0.69999999999999996},"min":{"1min":0.501,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":3.7559999999999998,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":0.63500000000000001}]},{"osd":9,"last update":"Mon Nov 27 15:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.752,"15min":0.75},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":0.748},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.73999999999999999,"15min":0.72999999999999998},"min":{"1min":0.50900000000000001,"5min":0.497,"15min":0.497},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":0.70099999999999996}]},{"osd":10,"last update":"Mon Nov 27 15:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.51400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]}]},{"osd":6,"up_from":45,"seq":193273528364,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6260,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738892,"statfs":{"total":95995035648,"available":95988625408,"internally_reserved":0,"allocated":897024,"data_stored":426509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.438,"15min":0.44500000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.86099999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.503,"15min":0.53200000000000003},"min":{"1min":0.29899999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.83699999999999997,"5min":2.508,"15min":2.508},"last":0.38200000000000001}]},{"osd":1,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59599999999999997,"15min":0.58699999999999997},"min":{"1min":0.45000000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.61499999999999999,"15min":0.63800000000000001},"min":{"1min":0.41099999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73799999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":0.60099999999999998}]},{"osd":2,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.65300000000000002,"15min":0.67500000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91000000000000003,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.64600000000000002,"15min":0.65000000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.56499999999999995}]},{"osd":3,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52900000000000003,"15min":0.53100000000000003},"min":{"1min":0.377,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89600000000000002,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.54500000000000004,"15min":0.58699999999999997},"min":{"1min":0.33000000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.879,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":0.373}]},{"osd":4,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.64300000000000002,"15min":0.65800000000000003},"min":{"1min":0.42699999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95299999999999996,"5min":1.51,"15min":1.51},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65500000000000003,"15min":0.64300000000000002},"min":{"1min":0.495,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.012,"5min":1.054,"15min":1.054},"last":0.61299999999999999}]},{"osd":5,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.69299999999999995,"15min":0.73099999999999998},"min":{"1min":0.47399999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77700000000000002,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.70199999999999996,"15min":0.71899999999999997},"min":{"1min":0.51600000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.93300000000000005,"5min":1.599,"15min":1.599},"last":0.65000000000000002}]},{"osd":7,"last update":"Mon Nov 27 15:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72599999999999998,"15min":0.72499999999999998},"min":{"1min":0.46000000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.995,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.73999999999999999,"15min":0.73599999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.77700000000000002}]},{"osd":8,"last update":"Mon Nov 27 15:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.79700000000000004,"15min":0.81200000000000006},"min":{"1min":0.57199999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.99299999999999999,"5min":1.155,"15min":1.155},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.77700000000000002,"15min":0.80500000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.75900000000000001,"5min":2.766,"15min":2.766},"last":0.67100000000000004}]},{"osd":9,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.71599999999999997,"15min":0.74099999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.77800000000000002,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.75,"15min":0.77000000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90000000000000002,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.59399999999999997}]},{"osd":10,"last update":"Mon Nov 27 15:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.755}]}]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7280,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":6336,"kb_avail":93737872,"statfs":{"total":95995035648,"available":95987580928,"internally_reserved":0,"allocated":958464,"data_stored":484823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6488064},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51200000000000001,"15min":0.47299999999999998},"min":{"1min":0.32700000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.497},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.52000000000000002,"15min":0.46999999999999997},"min":{"1min":0.33900000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.86799999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.40799999999999997}]},{"osd":2,"last update":"Mon Nov 27 15:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.39900000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53900000000000003,"15min":0.52600000000000002},"min":{"1min":0.36699999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76800000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.56799999999999995}]},{"osd":3,"last update":"Mon Nov 27 15:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56999999999999995,"15min":0.53400000000000003},"min":{"1min":0.38100000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.96099999999999997,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56399999999999995,"15min":0.55700000000000005},"min":{"1min":0.32200000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.78300000000000003,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.46000000000000002}]},{"osd":4,"last update":"Mon Nov 27 15:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.44,"15min":0.42799999999999999},"min":{"1min":0.314,"5min":0.251,"15min":0.251},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.45500000000000002,"15min":0.441},"min":{"1min":0.26400000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.38600000000000001}]},{"osd":5,"last update":"Mon Nov 27 15:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.59799999999999998,"15min":0.58899999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.63,"15min":0.623},"min":{"1min":0.53100000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.67600000000000005}]},{"osd":6,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.60599999999999998,"15min":0.58299999999999996},"min":{"1min":0.40699999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.63600000000000001,"15min":0.626},"min":{"1min":0.434,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95499999999999996,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.47299999999999998}]},{"osd":7,"last update":"Mon Nov 27 15:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.53600000000000003,"15min":0.52600000000000002},"min":{"1min":0.38,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.436},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.58199999999999996,"15min":0.57299999999999995},"min":{"1min":0.432,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.44600000000000001}]},{"osd":8,"last update":"Mon Nov 27 15:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.66000000000000003,"15min":0.64800000000000002},"min":{"1min":0.52200000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.63500000000000001,"15min":0.624},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.65200000000000002}]},{"osd":9,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72899999999999998,"15min":0.72099999999999997},"min":{"1min":0.57899999999999996,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.72899999999999998,"15min":0.72499999999999998},"min":{"1min":0.53400000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.54000000000000004}]},{"osd":10,"last update":"Mon Nov 27 15:16:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.76900000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7176,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":93737976,"statfs":{"total":95995035648,"available":95987687424,"internally_reserved":0,"allocated":917504,"data_stored":427774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 27 15:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59299999999999997,"15min":0.54900000000000004},"min":{"1min":0.45400000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79000000000000004,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.60999999999999999,"15min":0.56100000000000005},"min":{"1min":0.432,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.70099999999999996,"5min":1.55,"15min":1.55},"last":0.45200000000000001}]},{"osd":2,"last update":"Mon Nov 27 15:16:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56699999999999995,"15min":0.54000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77100000000000002,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53700000000000003,"15min":0.51700000000000002},"min":{"1min":0.39800000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.67100000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.95899999999999996}]},{"osd":3,"last update":"Mon Nov 27 15:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.38900000000000001,"15min":0.371},"min":{"1min":0.26000000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.502},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.441,"15min":0.42899999999999999},"min":{"1min":0.30399999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.35099999999999998}]},{"osd":4,"last update":"Mon Nov 27 15:16:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.57699999999999996,"15min":0.55000000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.96099999999999997,"5min":0.999,"15min":0.999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.58899999999999997,"15min":0.55900000000000005},"min":{"1min":0.38300000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.57099999999999995}]},{"osd":5,"last update":"Mon Nov 27 15:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.621,"15min":0.61399999999999999},"min":{"1min":0.40799999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.749,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.61799999999999999,"15min":0.60799999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.81100000000000005,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.64100000000000001}]},{"osd":6,"last update":"Mon Nov 27 15:16:43 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.308,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.63900000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.754,"5min":1.032,"15min":1.032},"last":0.42699999999999999}]},{"osd":7,"last update":"Mon Nov 27 15:16:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67800000000000005,"15min":0.67200000000000004},"min":{"1min":0.434,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.46100000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.31}]},{"osd":8,"last update":"Mon Nov 27 15:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.68300000000000005,"15min":0.70299999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.72799999999999998,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.629},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.73199999999999998,"15min":0.746},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84299999999999997,"5min":1.004,"15min":1.004},"last":0.52000000000000002}]},{"osd":9,"last update":"Mon Nov 27 15:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.73499999999999999,"15min":0.76100000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88800000000000001,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.72299999999999998,"15min":0.74299999999999999},"min":{"1min":0.438,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.48799999999999999}]},{"osd":10,"last update":"Mon Nov 27 15:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.622}]},{"osd":11,"last update":"Mon Nov 27 15:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.64700000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7368,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737784,"statfs":{"total":95995035648,"available":95987490816,"internally_reserved":0,"allocated":983040,"data_stored":486515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.51100000000000001,"15min":0.47699999999999998},"min":{"1min":0.36099999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.499,"15min":0.48099999999999998},"min":{"1min":0.36499999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.48799999999999999}]},{"osd":1,"last update":"Mon Nov 27 15:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.53200000000000003,"15min":0.51000000000000001},"min":{"1min":0.34899999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51100000000000001,"15min":0.48899999999999999},"min":{"1min":0.377,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.53600000000000003}]},{"osd":3,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.59899999999999998,"15min":0.54600000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.2170000000000001,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.59599999999999997,"15min":0.54600000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":3.278,"5min":3.278,"15min":3.278},"last":0.68799999999999994}]},{"osd":4,"last update":"Mon Nov 27 15:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.38400000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.76200000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.35799999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79200000000000004,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.65400000000000003}]},{"osd":5,"last update":"Mon Nov 27 15:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.434,"15min":0.40100000000000002},"min":{"1min":0.29599999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.75},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.436,"15min":0.41199999999999998},"min":{"1min":0.311,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.59299999999999997}]},{"osd":6,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.58599999999999997,"15min":0.56499999999999995},"min":{"1min":0.44900000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.70399999999999996,"15min":0.64300000000000002},"min":{"1min":0.41599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":4.5800000000000001,"5min":4.5800000000000001,"15min":4.5800000000000001},"last":0.82099999999999995}]},{"osd":7,"last update":"Mon Nov 27 15:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.39800000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.44},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69199999999999995,"15min":0.69599999999999995},"min":{"1min":0.46300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93300000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.51100000000000001}]},{"osd":8,"last update":"Mon Nov 27 15:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.64800000000000002,"15min":0.65700000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57499999999999996,"15min":0.57399999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.73999999999999999}]},{"osd":9,"last update":"Mon Nov 27 15:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.72199999999999998,"15min":0.70199999999999996},"min":{"1min":0.49399999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":3.633,"5min":3.633,"15min":3.633},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.036,"5min":0.79500000000000004,"15min":0.755},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":7.6369999999999996,"5min":7.6369999999999996,"15min":7.6369999999999996},"last":0.77700000000000002}]},{"osd":10,"last update":"Mon Nov 27 15:16:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.79000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]}]},{"osd":3,"up_from":25,"seq":107374182456,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6532,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738620,"statfs":{"total":95995035648,"available":95988346880,"internally_reserved":0,"allocated":913408,"data_stored":426826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.36599999999999999,"15min":0.33500000000000002},"min":{"1min":0.254,"5min":0.22,"15min":0.22},"max":{"1min":0.60499999999999998,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.308},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.38500000000000001,"15min":0.35699999999999998},"min":{"1min":0.27300000000000002,"5min":0.214,"15min":0.214},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.32300000000000001}]},{"osd":1,"last update":"Mon Nov 27 15:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53800000000000003,"15min":0.505},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78500000000000003,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53800000000000003,"15min":0.52400000000000002},"min":{"1min":0.376,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82599999999999996,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.47799999999999998}]},{"osd":2,"last update":"Mon Nov 27 15:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58699999999999997,"15min":0.54500000000000004},"min":{"1min":0.40200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80100000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58099999999999996,"15min":0.54400000000000004},"min":{"1min":0.41699999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.81100000000000005,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.53400000000000003}]},{"osd":4,"last update":"Mon Nov 27 15:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.54300000000000004,"15min":0.52400000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.56000000000000005,"15min":0.52200000000000002},"min":{"1min":0.495,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.51900000000000002}]},{"osd":5,"last update":"Mon Nov 27 15:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.60499999999999998,"15min":0.57599999999999996},"min":{"1min":0.42899999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.92900000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.61799999999999999,"15min":0.58799999999999997},"min":{"1min":0.44700000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89000000000000001,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.60799999999999998}]},{"osd":6,"last update":"Mon Nov 27 15:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56999999999999995,"15min":0.55000000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.91100000000000003,"5min":1.089,"15min":1.089},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55900000000000005,"15min":0.54300000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.45000000000000001}]},{"osd":7,"last update":"Mon Nov 27 15:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.72499999999999998,"15min":0.70899999999999996},"min":{"1min":0.59599999999999997,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.66300000000000003}]},{"osd":8,"last update":"Mon Nov 27 15:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.628}]},{"osd":9,"last update":"Mon Nov 27 15:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.80600000000000005,"15min":0.82599999999999996},"min":{"1min":0.49299999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.083,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.73799999999999999,"15min":0.75600000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95399999999999996,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.50900000000000001}]},{"osd":10,"last update":"Mon Nov 27 15:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.70299999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]}]},{"osd":4,"up_from":33,"seq":141733920821,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6392,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738760,"statfs":{"total":95995035648,"available":95988490240,"internally_reserved":0,"allocated":901120,"data_stored":427282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.48099999999999998,"15min":0.46800000000000003},"min":{"1min":0.35299999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.52200000000000002,"15min":0.50700000000000001},"min":{"1min":0.40300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.432}]},{"osd":1,"last update":"Mon Nov 27 15:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.36899999999999999,"15min":0.34799999999999998},"min":{"1min":0.30299999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.36899999999999999,"15min":0.34699999999999998},"min":{"1min":0.28399999999999997,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.90500000000000003}]},{"osd":2,"last update":"Mon Nov 27 15:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.50700000000000001,"15min":0.48899999999999999},"min":{"1min":0.36699999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.996},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.54500000000000004,"15min":0.53500000000000003},"min":{"1min":0.40200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.97299999999999998}]},{"osd":3,"last update":"Mon Nov 27 15:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.53900000000000003,"15min":0.51400000000000001},"min":{"1min":0.45100000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":1.127},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.40999999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80700000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.872}]},{"osd":5,"last update":"Mon Nov 27 15:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.56100000000000005,"15min":0.53500000000000003},"min":{"1min":0.42099999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":1.038},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.57699999999999996,"15min":0.55900000000000005},"min":{"1min":0.45300000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":1.052}]},{"osd":6,"last update":"Mon Nov 27 15:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.63500000000000001,"15min":0.59999999999999998},"min":{"1min":0.46500000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":5.1040000000000001,"5min":5.1040000000000001,"15min":5.1040000000000001},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.67400000000000004,"15min":0.63800000000000001},"min":{"1min":0.46700000000000003,"5min":0.441,"15min":0.441},"max":{"1min":5.1139999999999999,"5min":5.1139999999999999,"15min":5.1139999999999999},"last":1.0800000000000001}]},{"osd":7,"last update":"Mon Nov 27 15:16:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.34000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.34899999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.83899999999999997}]},{"osd":8,"last update":"Mon Nov 27 15:16:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.66900000000000004,"15min":0.66000000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.70299999999999996,"15min":0.69499999999999995},"min":{"1min":0.49099999999999999,"5min":0.432,"15min":0.432},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":1.022}]},{"osd":9,"last update":"Mon Nov 27 15:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.78100000000000003,"15min":0.75900000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":4.8780000000000001,"5min":4.8780000000000001,"15min":4.8780000000000001},"last":1.1499999999999999},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.747,"15min":0.72299999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":4.9169999999999998,"5min":4.9169999999999998,"15min":4.9169999999999998},"last":0.92600000000000005}]},{"osd":10,"last update":"Mon Nov 27 15:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.78600000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]}]},{"osd":5,"up_from":38,"seq":163208757296,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6524,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738628,"statfs":{"total":95995035648,"available":95988355072,"internally_reserved":0,"allocated":970752,"data_stored":1018605,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.498,"15min":0.49299999999999999},"min":{"1min":0.32600000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67100000000000004,"5min":0.752,"15min":0.752},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.47599999999999998,"15min":0.46999999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.64500000000000002,"5min":0.755,"15min":0.755},"last":0.65300000000000002}]},{"osd":1,"last update":"Mon Nov 27 15:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54600000000000004,"15min":0.52800000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.54600000000000004,"15min":0.51700000000000002},"min":{"1min":0.41199999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.63500000000000001}]},{"osd":2,"last update":"Mon Nov 27 15:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.40400000000000003,"15min":0.39100000000000001},"min":{"1min":0.27400000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.60999999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.42099999999999999,"15min":0.40500000000000003},"min":{"1min":0.28999999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.67100000000000004,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.55600000000000005}]},{"osd":3,"last update":"Mon Nov 27 15:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71799999999999997,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54100000000000004,"15min":0.53500000000000003},"min":{"1min":0.38200000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.73399999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.71299999999999997}]},{"osd":4,"last update":"Mon Nov 27 15:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.56999999999999995,"15min":0.55300000000000005},"min":{"1min":0.46600000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.61799999999999999}]},{"osd":6,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":0.68500000000000005,"15min":0.61899999999999999},"min":{"1min":0.50800000000000001,"5min":0.436,"15min":0.436},"max":{"1min":8.0280000000000005,"5min":8.0280000000000005,"15min":8.0280000000000005},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.1319999999999999,"5min":0.73899999999999999,"15min":0.67100000000000004},"min":{"1min":0.51800000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":8.0389999999999997,"5min":8.0389999999999997,"15min":8.0389999999999997},"last":0.875}]},{"osd":7,"last update":"Mon Nov 27 15:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.50600000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.48099999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.80200000000000005}]},{"osd":8,"last update":"Mon Nov 27 15:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.64500000000000002,"15min":0.63700000000000001},"min":{"1min":0.41599999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.39100000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80100000000000005,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.69199999999999995}]},{"osd":9,"last update":"Mon Nov 27 15:16:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":0.747,"15min":0.69199999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":7.8070000000000004,"5min":7.8070000000000004,"15min":7.8070000000000004},"last":0.754},{"interface":"front","average":{"1min":1.194,"5min":0.81899999999999995,"15min":0.75700000000000001},"min":{"1min":0.56599999999999995,"5min":0.503,"15min":0.503},"max":{"1min":7.7690000000000001,"5min":7.7690000000000001,"15min":7.7690000000000001},"last":0.94399999999999995}]},{"osd":10,"last update":"Mon Nov 27 15:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.82899999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34455,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34455,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-27T15:17:02.841 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-11-27T15:17:02.842 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-27T15:17:02.842 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-11-27T15:17:02.842 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-11-27T15:17:03.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:02 smithi102 ceph-mon[100296]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:17:03.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:02 smithi052 ceph-mon[99056]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:17:03.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:02 smithi150 ceph-mon[100559]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:17:03.496 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:17:04.049 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:03 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]: dispatch 2023-11-27T15:17:04.049 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:03 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2023-11-27T15:17:04.049 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:03 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-11-27T15:17:04.049 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:03 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2023-11-27T15:17:04.049 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:03 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2023-11-27T15:17:04.049 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:03 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]: dispatch 2023-11-27T15:17:04.049 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:03 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2023-11-27T15:17:04.049 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:03 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2023-11-27T15:17:04.049 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:03 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 7]}]: dispatch 2023-11-27T15:17:04.049 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:03 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2023-11-27T15:17:04.049 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:03 smithi052 ceph-mon[99056]: pgmap v283: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-27T15:17:04.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:03 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]: dispatch 2023-11-27T15:17:04.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:03 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2023-11-27T15:17:04.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:03 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-11-27T15:17:04.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:03 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2023-11-27T15:17:04.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:03 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2023-11-27T15:17:04.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:03 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]: dispatch 2023-11-27T15:17:04.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:03 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2023-11-27T15:17:04.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:03 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2023-11-27T15:17:04.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:03 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 7]}]: dispatch 2023-11-27T15:17:04.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:03 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2023-11-27T15:17:04.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:03 smithi102 ceph-mon[100296]: pgmap v283: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-27T15:17:04.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:03 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]: dispatch 2023-11-27T15:17:04.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:03 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2023-11-27T15:17:04.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:03 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-11-27T15:17:04.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:03 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2023-11-27T15:17:04.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:03 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2023-11-27T15:17:04.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:03 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]: dispatch 2023-11-27T15:17:04.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:03 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2023-11-27T15:17:04.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:03 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2023-11-27T15:17:04.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:03 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 7]}]: dispatch 2023-11-27T15:17:04.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:03 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2023-11-27T15:17:04.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:03 smithi150 ceph-mon[100559]: pgmap v283: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-27T15:17:04.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.819+0000 7f1a562d8700 1 -- 172.21.15.52:0/1369878087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a50108400 msgr2=0x7f1a50112e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:04.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.819+0000 7f1a562d8700 1 --2- 172.21.15.52:0/1369878087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a50108400 0x7f1a50112e70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1a400098f0 tx=0x7f1a40009c00 comp rx=0 tx=0).stop 2023-11-27T15:17:04.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.819+0000 7f1a562d8700 1 -- 172.21.15.52:0/1369878087 shutdown_connections 2023-11-27T15:17:04.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.819+0000 7f1a562d8700 1 --2- 172.21.15.52:0/1369878087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a501133b0 0x7f1a50115790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:04.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.819+0000 7f1a562d8700 1 --2- 172.21.15.52:0/1369878087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a50107af0 0x7f1a50107ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:04.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.819+0000 7f1a562d8700 1 --2- 172.21.15.52:0/1369878087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a50108400 0x7f1a50112e70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:04.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.819+0000 7f1a562d8700 1 -- 172.21.15.52:0/1369878087 >> 172.21.15.52:0/1369878087 conn(0x7f1a5006c600 msgr2=0x7f1a500717b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:04.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.819+0000 7f1a562d8700 1 -- 172.21.15.52:0/1369878087 shutdown_connections 2023-11-27T15:17:04.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.819+0000 7f1a562d8700 1 -- 172.21.15.52:0/1369878087 wait complete. 2023-11-27T15:17:04.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.819+0000 7f1a562d8700 1 Processor -- start 2023-11-27T15:17:04.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.819+0000 7f1a562d8700 1 -- start start 2023-11-27T15:17:04.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.820+0000 7f1a562d8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a50107af0 0x7f1a50108cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:04.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.820+0000 7f1a562d8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a50108400 0x7f1a50109210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:04.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.820+0000 7f1a562d8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a501133b0 0x7f1a50109790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:04.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.820+0000 7f1a562d8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a501ad080 con 0x7f1a501133b0 2023-11-27T15:17:04.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.820+0000 7f1a562d8700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a501ad1f0 con 0x7f1a50107af0 2023-11-27T15:17:04.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.820+0000 7f1a562d8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a501ad3b0 con 0x7f1a50108400 2023-11-27T15:17:04.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.820+0000 7f1a54875700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a501133b0 0x7f1a50109790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:04.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.820+0000 7f1a54875700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a501133b0 0x7f1a50109790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50666/0 (socket says 172.21.15.52:50666) 2023-11-27T15:17:04.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.820+0000 7f1a54875700 1 -- 172.21.15.52:0/1159524004 learned_addr learned my addr 172.21.15.52:0/1159524004 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:04.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.820+0000 7f1a4f7fe700 1 --2- 172.21.15.52:0/1159524004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a50108400 0x7f1a50109210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:04.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.820+0000 7f1a4ffff700 1 --2- 172.21.15.52:0/1159524004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a50107af0 0x7f1a50108cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:04.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.820+0000 7f1a54875700 1 -- 172.21.15.52:0/1159524004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a50108400 msgr2=0x7f1a50109210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:04.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.820+0000 7f1a54875700 1 --2- 172.21.15.52:0/1159524004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a50108400 0x7f1a50109210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:04.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.820+0000 7f1a54875700 1 -- 172.21.15.52:0/1159524004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a50107af0 msgr2=0x7f1a50108cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:04.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.820+0000 7f1a54875700 1 --2- 172.21.15.52:0/1159524004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a50107af0 0x7f1a50108cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:04.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.820+0000 7f1a54875700 1 -- 172.21.15.52:0/1159524004 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a40009520 con 0x7f1a501133b0 2023-11-27T15:17:04.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.820+0000 7f1a54875700 1 --2- 172.21.15.52:0/1159524004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a501133b0 0x7f1a50109790 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1a4800b9e0 tx=0x7f1a4800bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:04.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.821+0000 7f1a4d7fa700 1 -- 172.21.15.52:0/1159524004 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4800c870 con 0x7f1a501133b0 2023-11-27T15:17:04.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.821+0000 7f1a562d8700 1 -- 172.21.15.52:0/1159524004 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a50118bd0 con 0x7f1a501133b0 2023-11-27T15:17:04.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.821+0000 7f1a4d7fa700 1 -- 172.21.15.52:0/1159524004 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a4800ce90 con 0x7f1a501133b0 2023-11-27T15:17:04.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.821+0000 7f1a4d7fa700 1 -- 172.21.15.52:0/1159524004 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a48012340 con 0x7f1a501133b0 2023-11-27T15:17:04.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.821+0000 7f1a562d8700 1 -- 172.21.15.52:0/1159524004 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a50118a40 con 0x7f1a501133b0 2023-11-27T15:17:04.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.822+0000 7f1a562d8700 1 -- 172.21.15.52:0/1159524004 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a50118d60 con 0x7f1a501133b0 2023-11-27T15:17:04.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.823+0000 7f1a4d7fa700 1 -- 172.21.15.52:0/1159524004 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f1a4800c9d0 con 0x7f1a501133b0 2023-11-27T15:17:04.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.823+0000 7f1a4d7fa700 1 --2- 172.21.15.52:0/1159524004 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f1a380a11e0 0x7f1a380a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:04.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.823+0000 7f1a4ffff700 1 --2- 172.21.15.52:0/1159524004 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f1a380a11e0 0x7f1a380a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:04.832 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.823+0000 7f1a4d7fa700 1 -- 172.21.15.52:0/1159524004 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f1a48006e40 con 0x7f1a501133b0 2023-11-27T15:17:04.832 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.824+0000 7f1a4ffff700 1 --2- 172.21.15.52:0/1159524004 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f1a380a11e0 0x7f1a380a3690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1a44005fd0 tx=0x7f1a44009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:04.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:04.828+0000 7f1a4d7fa700 1 -- 172.21.15.52:0/1159524004 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1a480956e0 con 0x7f1a501133b0 2023-11-27T15:17:04.866 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:04 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]': finished 2023-11-27T15:17:04.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:04 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2023-11-27T15:17:04.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:04 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-11-27T15:17:04.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:04 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2023-11-27T15:17:04.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:04 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2023-11-27T15:17:04.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:04 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2023-11-27T15:17:04.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:04 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2023-11-27T15:17:04.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:04 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2023-11-27T15:17:04.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:04 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 7]}]': finished 2023-11-27T15:17:04.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:04 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2023-11-27T15:17:04.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:04 smithi052 ceph-mon[99056]: osdmap e84: 12 total, 12 up, 12 in 2023-11-27T15:17:05.074 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.069+0000 7f1a562d8700 1 -- 172.21.15.52:0/1159524004 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f1a5010dcd0 con 0x7f1a501133b0 2023-11-27T15:17:05.074 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.069+0000 7f1a4d7fa700 1 -- 172.21.15.52:0/1159524004 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f1a48098d00 con 0x7f1a501133b0 2023-11-27T15:17:05.075 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:17:05.075 INFO:teuthology.orchestra.run.smithi052.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-27T15:17:05.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.071+0000 7f1a562d8700 1 -- 172.21.15.52:0/1159524004 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f1a380a11e0 msgr2=0x7f1a380a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:05.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.071+0000 7f1a562d8700 1 --2- 172.21.15.52:0/1159524004 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f1a380a11e0 0x7f1a380a3690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1a44005fd0 tx=0x7f1a44009310 comp rx=0 tx=0).stop 2023-11-27T15:17:05.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.071+0000 7f1a562d8700 1 -- 172.21.15.52:0/1159524004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a501133b0 msgr2=0x7f1a50109790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:05.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.071+0000 7f1a562d8700 1 --2- 172.21.15.52:0/1159524004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a501133b0 0x7f1a50109790 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1a4800b9e0 tx=0x7f1a4800bcf0 comp rx=0 tx=0).stop 2023-11-27T15:17:05.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.071+0000 7f1a562d8700 1 -- 172.21.15.52:0/1159524004 shutdown_connections 2023-11-27T15:17:05.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.071+0000 7f1a562d8700 1 --2- 172.21.15.52:0/1159524004 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f1a380a11e0 0x7f1a380a3690 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:05.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.071+0000 7f1a562d8700 1 --2- 172.21.15.52:0/1159524004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a50107af0 0x7f1a50108cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:05.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.071+0000 7f1a562d8700 1 --2- 172.21.15.52:0/1159524004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a501133b0 0x7f1a50109790 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:05.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.071+0000 7f1a562d8700 1 --2- 172.21.15.52:0/1159524004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a50108400 0x7f1a50109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:05.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.071+0000 7f1a562d8700 1 -- 172.21.15.52:0/1159524004 >> 172.21.15.52:0/1159524004 conn(0x7f1a5006c600 msgr2=0x7f1a5010ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:05.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.071+0000 7f1a562d8700 1 -- 172.21.15.52:0/1159524004 shutdown_connections 2023-11-27T15:17:05.078 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.071+0000 7f1a562d8700 1 -- 172.21.15.52:0/1159524004 wait complete. 2023-11-27T15:17:05.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:04 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]': finished 2023-11-27T15:17:05.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:04 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2023-11-27T15:17:05.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:04 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-11-27T15:17:05.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:04 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2023-11-27T15:17:05.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:04 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2023-11-27T15:17:05.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:04 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2023-11-27T15:17:05.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:04 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2023-11-27T15:17:05.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:04 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2023-11-27T15:17:05.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:04 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 7]}]': finished 2023-11-27T15:17:05.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:04 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2023-11-27T15:17:05.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:04 smithi102 ceph-mon[100296]: osdmap e84: 12 total, 12 up, 12 in 2023-11-27T15:17:05.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:04 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]': finished 2023-11-27T15:17:05.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:04 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2023-11-27T15:17:05.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:04 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-11-27T15:17:05.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:04 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2023-11-27T15:17:05.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:04 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2023-11-27T15:17:05.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:04 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2023-11-27T15:17:05.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:04 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2023-11-27T15:17:05.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:04 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2023-11-27T15:17:05.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:04 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 7]}]': finished 2023-11-27T15:17:05.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:04 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2023-11-27T15:17:05.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:04 smithi150 ceph-mon[100559]: osdmap e84: 12 total, 12 up, 12 in 2023-11-27T15:17:05.558 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-11-27T15:17:05.558 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:17:05.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.683+0000 7f3ee5947700 1 -- 172.21.15.52:0/2373595804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ee01003a0 msgr2=0x7f3ee01007f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:05.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.683+0000 7f3ee5947700 1 --2- 172.21.15.52:0/2373595804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ee01003a0 0x7f3ee01007f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8009990 tx=0x7f3ec8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:05.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.683+0000 7f3ee5947700 1 -- 172.21.15.52:0/2373595804 shutdown_connections 2023-11-27T15:17:05.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.683+0000 7f3ee5947700 1 --2- 172.21.15.52:0/2373595804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ee00ff1b0 0x7f3ee00ff5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:05.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.683+0000 7f3ee5947700 1 --2- 172.21.15.52:0/2373595804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ee0100d30 0x7f3ee01075f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:05.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.683+0000 7f3ee5947700 1 --2- 172.21.15.52:0/2373595804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ee01003a0 0x7f3ee01007f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:05.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.683+0000 7f3ee5947700 1 -- 172.21.15.52:0/2373595804 >> 172.21.15.52:0/2373595804 conn(0x7f3ee00fa790 msgr2=0x7f3ee00fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:05.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.683+0000 7f3ee5947700 1 -- 172.21.15.52:0/2373595804 shutdown_connections 2023-11-27T15:17:05.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.683+0000 7f3ee5947700 1 -- 172.21.15.52:0/2373595804 wait complete. 2023-11-27T15:17:05.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.683+0000 7f3ee5947700 1 Processor -- start 2023-11-27T15:17:05.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.684+0000 7f3ee5947700 1 -- start start 2023-11-27T15:17:05.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.684+0000 7f3ee5947700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ee00ff1b0 0x7f3ee0195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:05.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.684+0000 7f3ee5947700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ee01003a0 0x7f3ee0196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:05.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.684+0000 7f3ee5947700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ee0100d30 0x7f3ee019d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:05.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.684+0000 7f3ee5947700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee019db50 con 0x7f3ee01003a0 2023-11-27T15:17:05.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.684+0000 7f3ee5947700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee019dc90 con 0x7f3ee00ff1b0 2023-11-27T15:17:05.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.684+0000 7f3ee5947700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee019de20 con 0x7f3ee0100d30 2023-11-27T15:17:05.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.684+0000 7f3ede7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ee01003a0 0x7f3ee0196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:05.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.684+0000 7f3ede7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ee01003a0 0x7f3ee0196510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50676/0 (socket says 172.21.15.52:50676) 2023-11-27T15:17:05.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.684+0000 7f3ede7fc700 1 -- 172.21.15.52:0/1473486715 learned_addr learned my addr 172.21.15.52:0/1473486715 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:05.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.684+0000 7f3edeffd700 1 --2- 172.21.15.52:0/1473486715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ee00ff1b0 0x7f3ee0195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:05.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.684+0000 7f3edf7fe700 1 --2- 172.21.15.52:0/1473486715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ee0100d30 0x7f3ee019d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:05.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.684+0000 7f3ede7fc700 1 -- 172.21.15.52:0/1473486715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ee0100d30 msgr2=0x7f3ee019d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:05.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.684+0000 7f3ede7fc700 1 --2- 172.21.15.52:0/1473486715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ee0100d30 0x7f3ee019d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:05.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.684+0000 7f3ede7fc700 1 -- 172.21.15.52:0/1473486715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ee00ff1b0 msgr2=0x7f3ee0195fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:05.693 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.684+0000 7f3ede7fc700 1 --2- 172.21.15.52:0/1473486715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ee00ff1b0 0x7f3ee0195fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:05.693 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.684+0000 7f3ede7fc700 1 -- 172.21.15.52:0/1473486715 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec80095f0 con 0x7f3ee01003a0 2023-11-27T15:17:05.693 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.684+0000 7f3ede7fc700 1 --2- 172.21.15.52:0/1473486715 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ee01003a0 0x7f3ee0196510 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8006010 tx=0x7f3ec8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:05.693 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.685+0000 7f3ee4945700 1 -- 172.21.15.52:0/1473486715 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec801d070 con 0x7f3ee01003a0 2023-11-27T15:17:05.693 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.685+0000 7f3ee5947700 1 -- 172.21.15.52:0/1473486715 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee010bc80 con 0x7f3ee01003a0 2023-11-27T15:17:05.693 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.685+0000 7f3ee4945700 1 -- 172.21.15.52:0/1473486715 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ec8003f60 con 0x7f3ee01003a0 2023-11-27T15:17:05.693 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.685+0000 7f3ee4945700 1 -- 172.21.15.52:0/1473486715 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8022c40 con 0x7f3ee01003a0 2023-11-27T15:17:05.693 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.685+0000 7f3ee5947700 1 -- 172.21.15.52:0/1473486715 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee010baf0 con 0x7f3ee01003a0 2023-11-27T15:17:05.693 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.686+0000 7f3ee5947700 1 -- 172.21.15.52:0/1473486715 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee019e420 con 0x7f3ee01003a0 2023-11-27T15:17:05.694 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.687+0000 7f3ee4945700 1 -- 172.21.15.52:0/1473486715 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3ec80040d0 con 0x7f3ee01003a0 2023-11-27T15:17:05.694 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.688+0000 7f3ee4945700 1 --2- 172.21.15.52:0/1473486715 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f3ecc0a11e0 0x7f3ecc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:05.694 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.688+0000 7f3edeffd700 1 --2- 172.21.15.52:0/1473486715 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f3ecc0a11e0 0x7f3ecc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:05.694 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.688+0000 7f3ee4945700 1 -- 172.21.15.52:0/1473486715 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3ec80ce220 con 0x7f3ee01003a0 2023-11-27T15:17:05.694 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.688+0000 7f3edeffd700 1 --2- 172.21.15.52:0/1473486715 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f3ecc0a11e0 0x7f3ecc0a3690 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0100200 tx=0x7f3ed000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:05.696 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.691+0000 7f3ee4945700 1 -- 172.21.15.52:0/1473486715 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3ec8097880 con 0x7f3ee01003a0 2023-11-27T15:17:05.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.893+0000 7f3ee5947700 1 -- 172.21.15.52:0/1473486715 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ee0106100 con 0x7f3ee01003a0 2023-11-27T15:17:05.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.895+0000 7f3ee4945700 1 -- 172.21.15.52:0/1473486715 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22657 (secure 0 0 0) 0x7f3ec809aea0 con 0x7f3ee01003a0 2023-11-27T15:17:05.901 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:17:05.901 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":85,"fsid":"c6c7df88-8d36-11ee-95a2-87774f69a715","created":"2023-11-27T15:09:03.333345+0000","modified":"2023-11-27T15:17:04.797892+0000","last_up_change":"2023-11-27T15:15:27.656134+0000","last_in_change":"2023-11-27T15:15:10.671153+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:12:14.177523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:16:07.842251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:16:08.728520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58d69ede-c293-49f5-abbe-8187793f2b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6803","nonce":608216415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6805","nonce":608216415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6809","nonce":608216415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6807","nonce":608216415}]},"public_addr":"172.21.15.52:6803/608216415","cluster_addr":"172.21.15.52:6805/608216415","heartbeat_back_addr":"172.21.15.52:6809/608216415","heartbeat_front_addr":"172.21.15.52:6807/608216415","state":["exists","up"]},{"osd":1,"uuid":"d6a46b91-adc8-45ef-80f5-566ae400911f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6801","nonce":2538384087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6803","nonce":2538384087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6807","nonce":2538384087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6805","nonce":2538384087}]},"public_addr":"172.21.15.102:6801/2538384087","cluster_addr":"172.21.15.102:6803/2538384087","heartbeat_back_addr":"172.21.15.102:6807/2538384087","heartbeat_front_addr":"172.21.15.102:6805/2538384087","state":["exists","up"]},{"osd":2,"uuid":"846ba125-11f8-46b2-8941-bd5e1947cc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6801","nonce":4258038221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6803","nonce":4258038221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6807","nonce":4258038221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6805","nonce":4258038221}]},"public_addr":"172.21.15.150:6801/4258038221","cluster_addr":"172.21.15.150:6803/4258038221","heartbeat_back_addr":"172.21.15.150:6807/4258038221","heartbeat_front_addr":"172.21.15.150:6805/4258038221","state":["exists","up"]},{"osd":3,"uuid":"950fcd67-1057-4288-a389-714f1ec94261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6811","nonce":2622100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6813","nonce":2622100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6817","nonce":2622100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6815","nonce":2622100752}]},"public_addr":"172.21.15.52:6811/2622100752","cluster_addr":"172.21.15.52:6813/2622100752","heartbeat_back_addr":"172.21.15.52:6817/2622100752","heartbeat_front_addr":"172.21.15.52:6815/2622100752","state":["exists","up"]},{"osd":4,"uuid":"1c96a676-cea9-4deb-bad1-f7fd240c3d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6809","nonce":1546794763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6811","nonce":1546794763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6815","nonce":1546794763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6813","nonce":1546794763}]},"public_addr":"172.21.15.102:6809/1546794763","cluster_addr":"172.21.15.102:6811/1546794763","heartbeat_back_addr":"172.21.15.102:6815/1546794763","heartbeat_front_addr":"172.21.15.102:6813/1546794763","state":["exists","up"]},{"osd":5,"uuid":"38a75af1-fd4d-433e-b579-0609109c851c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6809","nonce":461511375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6811","nonce":461511375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6815","nonce":461511375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6813","nonce":461511375}]},"public_addr":"172.21.15.150:6809/461511375","cluster_addr":"172.21.15.150:6811/461511375","heartbeat_back_addr":"172.21.15.150:6815/461511375","heartbeat_front_addr":"172.21.15.150:6813/461511375","state":["exists","up"]},{"osd":6,"uuid":"2f0ede4a-d175-4b4a-b876-c0f442048901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6819","nonce":1002457464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6821","nonce":1002457464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6825","nonce":1002457464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6823","nonce":1002457464}]},"public_addr":"172.21.15.52:6819/1002457464","cluster_addr":"172.21.15.52:6821/1002457464","heartbeat_back_addr":"172.21.15.52:6825/1002457464","heartbeat_front_addr":"172.21.15.52:6823/1002457464","state":["exists","up"]},{"osd":7,"uuid":"6b635192-abcf-4bbf-bf78-a019b351bca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6817","nonce":3249840064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6819","nonce":3249840064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6823","nonce":3249840064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6821","nonce":3249840064}]},"public_addr":"172.21.15.102:6817/3249840064","cluster_addr":"172.21.15.102:6819/3249840064","heartbeat_back_addr":"172.21.15.102:6823/3249840064","heartbeat_front_addr":"172.21.15.102:6821/3249840064","state":["exists","up"]},{"osd":8,"uuid":"c7ca8007-7170-47bf-956c-3f00edf27e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6817","nonce":3672583574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6819","nonce":3672583574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6823","nonce":3672583574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6821","nonce":3672583574}]},"public_addr":"172.21.15.150:6817/3672583574","cluster_addr":"172.21.15.150:6819/3672583574","heartbeat_back_addr":"172.21.15.150:6823/3672583574","heartbeat_front_addr":"172.21.15.150:6821/3672583574","state":["exists","up"]},{"osd":9,"uuid":"0e252c2c-01a4-48ba-832d-067ed940fefb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6827","nonce":4287934320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6829","nonce":4287934320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6833","nonce":4287934320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6831","nonce":4287934320}]},"public_addr":"172.21.15.52:6827/4287934320","cluster_addr":"172.21.15.52:6829/4287934320","heartbeat_back_addr":"172.21.15.52:6833/4287934320","heartbeat_front_addr":"172.21.15.52:6831/4287934320","state":["exists","up"]},{"osd":10,"uuid":"bb540142-7563-44a4-b7a3-cb662714c128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6825","nonce":3056034201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6827","nonce":3056034201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6831","nonce":3056034201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6829","nonce":3056034201}]},"public_addr":"172.21.15.102:6825/3056034201","cluster_addr":"172.21.15.102:6827/3056034201","heartbeat_back_addr":"172.21.15.102:6831/3056034201","heartbeat_front_addr":"172.21.15.102:6829/3056034201","state":["exists","up"]},{"osd":11,"uuid":"3156528e-e0f8-4d44-9946-c8b6339c2bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6825","nonce":3207559739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6827","nonce":3207559739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6831","nonce":3207559739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6829","nonce":3207559739}]},"public_addr":"172.21.15.150:6825/3207559739","cluster_addr":"172.21.15.150:6827/3207559739","heartbeat_back_addr":"172.21.15.150:6831/3207559739","heartbeat_front_addr":"172.21.15.150:6829/3207559739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:26.826099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:48.331371+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:10.459738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:32.356472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:54.043170+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:15.648582+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:37.166463+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:59.240819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:20.710097+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:43.364563+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:05.279433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:26.344573+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":5,"to":7}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/1392282110":"2023-11-28T15:10:03.644870+0000","172.21.15.52:0/1479993479":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6800/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/961562652":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/1290665390":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2410529742":"2023-11-28T15:09:37.118690+0000","172.21.15.52:6800/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2130037044":"2023-11-28T15:10:03.644870+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:17:05.902 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.897+0000 7f3ee5947700 1 -- 172.21.15.52:0/1473486715 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f3ecc0a11e0 msgr2=0x7f3ecc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:05.903 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.897+0000 7f3ee5947700 1 --2- 172.21.15.52:0/1473486715 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f3ecc0a11e0 0x7f3ecc0a3690 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0100200 tx=0x7f3ed000b410 comp rx=0 tx=0).stop 2023-11-27T15:17:05.903 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.897+0000 7f3ee5947700 1 -- 172.21.15.52:0/1473486715 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ee01003a0 msgr2=0x7f3ee0196510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:05.903 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.897+0000 7f3ee5947700 1 --2- 172.21.15.52:0/1473486715 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ee01003a0 0x7f3ee0196510 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8006010 tx=0x7f3ec8004730 comp rx=0 tx=0).stop 2023-11-27T15:17:05.903 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.897+0000 7f3ee5947700 1 -- 172.21.15.52:0/1473486715 shutdown_connections 2023-11-27T15:17:05.903 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.897+0000 7f3ee5947700 1 --2- 172.21.15.52:0/1473486715 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f3ecc0a11e0 0x7f3ecc0a3690 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:05.903 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.897+0000 7f3ee5947700 1 --2- 172.21.15.52:0/1473486715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3ee00ff1b0 0x7f3ee0195fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:05.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.897+0000 7f3ee5947700 1 --2- 172.21.15.52:0/1473486715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ee0100d30 0x7f3ee019d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:05.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.897+0000 7f3ee5947700 1 --2- 172.21.15.52:0/1473486715 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ee01003a0 0x7f3ee0196510 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:05.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.897+0000 7f3ee5947700 1 -- 172.21.15.52:0/1473486715 >> 172.21.15.52:0/1473486715 conn(0x7f3ee00fa790 msgr2=0x7f3ee00fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:05.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.897+0000 7f3ee5947700 1 -- 172.21.15.52:0/1473486715 shutdown_connections 2023-11-27T15:17:05.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:05.897+0000 7f3ee5947700 1 -- 172.21.15.52:0/1473486715 wait complete. 2023-11-27T15:17:05.915 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:12:14.177523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-27T15:16:07.842251+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-27T15:16:08.728520+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-27T15:17:05.916 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:17:06.046 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.039+0000 7f0ee3822700 1 -- 172.21.15.52:0/3474354325 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0edc101ed0 msgr2=0x7f0edc108790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:06.047 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.039+0000 7f0ee3822700 1 --2- 172.21.15.52:0/3474354325 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0edc101ed0 0x7f0edc108790 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0ed400b0a0 tx=0x7f0ed400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.047 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.039+0000 7f0ee3822700 1 -- 172.21.15.52:0/3474354325 shutdown_connections 2023-11-27T15:17:06.047 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.039+0000 7f0ee3822700 1 --2- 172.21.15.52:0/3474354325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0edc0fe570 0x7f0edc0fe980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.047 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.039+0000 7f0ee3822700 1 --2- 172.21.15.52:0/3474354325 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0edc101ed0 0x7f0edc108790 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.047 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.039+0000 7f0ee3822700 1 --2- 172.21.15.52:0/3474354325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0edc101540 0x7f0edc101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.047 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.039+0000 7f0ee3822700 1 -- 172.21.15.52:0/3474354325 >> 172.21.15.52:0/3474354325 conn(0x7f0edc0f9f30 msgr2=0x7f0edc0fc380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:06.047 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.039+0000 7f0ee3822700 1 -- 172.21.15.52:0/3474354325 shutdown_connections 2023-11-27T15:17:06.047 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.039+0000 7f0ee3822700 1 -- 172.21.15.52:0/3474354325 wait complete. 2023-11-27T15:17:06.047 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.039+0000 7f0ee3822700 1 Processor -- start 2023-11-27T15:17:06.047 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.040+0000 7f0ee3822700 1 -- start start 2023-11-27T15:17:06.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.040+0000 7f0ee3822700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0edc0fe570 0x7f0edc196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:06.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.040+0000 7f0ee3822700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0edc101540 0x7f0edc1966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:06.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.040+0000 7f0ee3822700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0edc101ed0 0x7f0edc19d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:06.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.040+0000 7f0ee3822700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc19dd10 con 0x7f0edc0fe570 2023-11-27T15:17:06.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.040+0000 7f0ee3822700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc19de50 con 0x7f0edc101540 2023-11-27T15:17:06.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.040+0000 7f0ee3822700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc19dfe0 con 0x7f0edc101ed0 2023-11-27T15:17:06.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.040+0000 7f0ee15be700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0edc0fe570 0x7f0edc196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:06.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.040+0000 7f0ee15be700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0edc0fe570 0x7f0edc196190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50688/0 (socket says 172.21.15.52:50688) 2023-11-27T15:17:06.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.040+0000 7f0ee15be700 1 -- 172.21.15.52:0/1700595058 learned_addr learned my addr 172.21.15.52:0/1700595058 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:06.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.040+0000 7f0ee0dbd700 1 --2- 172.21.15.52:0/1700595058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0edc101540 0x7f0edc1966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:06.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.040+0000 7f0ee1dbf700 1 --2- 172.21.15.52:0/1700595058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0edc101ed0 0x7f0edc19d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:06.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.040+0000 7f0ee15be700 1 -- 172.21.15.52:0/1700595058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0edc101ed0 msgr2=0x7f0edc19d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:06.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.040+0000 7f0ee15be700 1 --2- 172.21.15.52:0/1700595058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0edc101ed0 0x7f0edc19d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.040+0000 7f0ee15be700 1 -- 172.21.15.52:0/1700595058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0edc101540 msgr2=0x7f0edc1966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:06.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.040+0000 7f0ee15be700 1 --2- 172.21.15.52:0/1700595058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0edc101540 0x7f0edc1966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.040+0000 7f0ee15be700 1 -- 172.21.15.52:0/1700595058 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed4009d00 con 0x7f0edc0fe570 2023-11-27T15:17:06.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.040+0000 7f0ee1dbf700 1 --2- 172.21.15.52:0/1700595058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0edc101ed0 0x7f0edc19d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:17:06.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.040+0000 7f0ee15be700 1 --2- 172.21.15.52:0/1700595058 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0edc0fe570 0x7f0edc196190 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc00e970 tx=0x7f0ecc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:06.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.041+0000 7f0ed27fc700 1 -- 172.21.15.52:0/1700595058 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc009790 con 0x7f0edc0fe570 2023-11-27T15:17:06.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.041+0000 7f0ed27fc700 1 -- 172.21.15.52:0/1700595058 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ecc00eec0 con 0x7f0edc0fe570 2023-11-27T15:17:06.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.041+0000 7f0ee3822700 1 -- 172.21.15.52:0/1700595058 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0edc10bca0 con 0x7f0edc0fe570 2023-11-27T15:17:06.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.041+0000 7f0ed27fc700 1 -- 172.21.15.52:0/1700595058 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc003cf0 con 0x7f0edc0fe570 2023-11-27T15:17:06.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.041+0000 7f0ee3822700 1 -- 172.21.15.52:0/1700595058 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0edc10bb10 con 0x7f0edc0fe570 2023-11-27T15:17:06.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.041+0000 7f0ee3822700 1 -- 172.21.15.52:0/1700595058 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0edc107280 con 0x7f0edc0fe570 2023-11-27T15:17:06.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.043+0000 7f0ed27fc700 1 -- 172.21.15.52:0/1700595058 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f0ecc00ccd0 con 0x7f0edc0fe570 2023-11-27T15:17:06.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.043+0000 7f0ed27fc700 1 --2- 172.21.15.52:0/1700595058 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0ec80a11e0 0x7f0ec80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:06.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.044+0000 7f0ee0dbd700 1 --2- 172.21.15.52:0/1700595058 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0ec80a11e0 0x7f0ec80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:06.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.044+0000 7f0ed27fc700 1 -- 172.21.15.52:0/1700595058 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0ecc014070 con 0x7f0edc0fe570 2023-11-27T15:17:06.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.044+0000 7f0ee0dbd700 1 --2- 172.21.15.52:0/1700595058 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0ec80a11e0 0x7f0ec80a3690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8009a10 tx=0x7f0ed8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:06.052 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.047+0000 7f0ed27fc700 1 -- 172.21.15.52:0/1700595058 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0ecc096420 con 0x7f0edc0fe570 2023-11-27T15:17:06.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:05 smithi102 ceph-mon[100296]: osdmap e85: 12 total, 12 up, 12 in 2023-11-27T15:17:06.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:05 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1159524004' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-11-27T15:17:06.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:05 smithi102 ceph-mon[100296]: pgmap v286: 97 pgs: 2 peering, 95 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-11-27T15:17:06.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:05 smithi052 ceph-mon[99056]: osdmap e85: 12 total, 12 up, 12 in 2023-11-27T15:17:06.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:05 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1159524004' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-11-27T15:17:06.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:05 smithi052 ceph-mon[99056]: pgmap v286: 97 pgs: 2 peering, 95 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-11-27T15:17:06.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.248+0000 7f0ee3822700 1 -- 172.21.15.52:0/1700595058 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0edc061700 con 0x7f0edc0fe570 2023-11-27T15:17:06.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.248+0000 7f0ed27fc700 1 -- 172.21.15.52:0/1700595058 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0ecc099a40 con 0x7f0edc0fe570 2023-11-27T15:17:06.254 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2023-11-27T15:17:06.256 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.250+0000 7f0ee3822700 1 -- 172.21.15.52:0/1700595058 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0ec80a11e0 msgr2=0x7f0ec80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:06.256 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.250+0000 7f0ee3822700 1 --2- 172.21.15.52:0/1700595058 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0ec80a11e0 0x7f0ec80a3690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8009a10 tx=0x7f0ed8009190 comp rx=0 tx=0).stop 2023-11-27T15:17:06.257 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.250+0000 7f0ee3822700 1 -- 172.21.15.52:0/1700595058 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0edc0fe570 msgr2=0x7f0edc196190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:06.257 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.250+0000 7f0ee3822700 1 --2- 172.21.15.52:0/1700595058 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0edc0fe570 0x7f0edc196190 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc00e970 tx=0x7f0ecc00ec80 comp rx=0 tx=0).stop 2023-11-27T15:17:06.257 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.250+0000 7f0ee3822700 1 -- 172.21.15.52:0/1700595058 shutdown_connections 2023-11-27T15:17:06.257 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.250+0000 7f0ee3822700 1 --2- 172.21.15.52:0/1700595058 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0ec80a11e0 0x7f0ec80a3690 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.250+0000 7f0ee3822700 1 --2- 172.21.15.52:0/1700595058 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0edc101ed0 0x7f0edc19d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.250+0000 7f0ee3822700 1 --2- 172.21.15.52:0/1700595058 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0edc0fe570 0x7f0edc196190 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.250+0000 7f0ee3822700 1 --2- 172.21.15.52:0/1700595058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0edc101540 0x7f0edc1966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.250+0000 7f0ee3822700 1 -- 172.21.15.52:0/1700595058 >> 172.21.15.52:0/1700595058 conn(0x7f0edc0f9f30 msgr2=0x7f0edc0fc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:06.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.251+0000 7f0ee3822700 1 -- 172.21.15.52:0/1700595058 shutdown_connections 2023-11-27T15:17:06.259 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.251+0000 7f0ee3822700 1 -- 172.21.15.52:0/1700595058 wait complete. 2023-11-27T15:17:06.268 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-27T15:17:06.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:05 smithi150 ceph-mon[100559]: osdmap e85: 12 total, 12 up, 12 in 2023-11-27T15:17:06.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:05 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1159524004' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-11-27T15:17:06.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:05 smithi150 ceph-mon[100559]: pgmap v286: 97 pgs: 2 peering, 95 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-11-27T15:17:06.399 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.392+0000 7ff6fbdde700 1 -- 172.21.15.52:0/4155341035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6f4103680 msgr2=0x7ff6f4105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:06.399 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.392+0000 7ff6fbdde700 1 --2- 172.21.15.52:0/4155341035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6f4103680 0x7ff6f4105a60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0009990 tx=0x7ff6f0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.399 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.392+0000 7ff6fbdde700 1 -- 172.21.15.52:0/4155341035 shutdown_connections 2023-11-27T15:17:06.399 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.392+0000 7ff6fbdde700 1 --2- 172.21.15.52:0/4155341035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6f4105fa0 0x7ff6f4108400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.399 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.392+0000 7ff6fbdde700 1 --2- 172.21.15.52:0/4155341035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6f4100d60 0x7ff6f4103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.399 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.392+0000 7ff6fbdde700 1 --2- 172.21.15.52:0/4155341035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6f4103680 0x7ff6f4105a60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.399 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.392+0000 7ff6fbdde700 1 -- 172.21.15.52:0/4155341035 >> 172.21.15.52:0/4155341035 conn(0x7ff6f40fa800 msgr2=0x7ff6f40fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:06.399 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.392+0000 7ff6fbdde700 1 -- 172.21.15.52:0/4155341035 shutdown_connections 2023-11-27T15:17:06.400 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.393+0000 7ff6fbdde700 1 -- 172.21.15.52:0/4155341035 wait complete. 2023-11-27T15:17:06.400 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.393+0000 7ff6fbdde700 1 Processor -- start 2023-11-27T15:17:06.400 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.393+0000 7ff6fbdde700 1 -- start start 2023-11-27T15:17:06.400 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.393+0000 7ff6fbdde700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6f4100d60 0x7ff6f4196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:06.400 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.393+0000 7ff6fbdde700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6f4103680 0x7ff6f4196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:06.400 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.393+0000 7ff6fbdde700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6f4105fa0 0x7ff6f419d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:06.400 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.393+0000 7ff6fbdde700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f419dba0 con 0x7ff6f4103680 2023-11-27T15:17:06.400 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.393+0000 7ff6fbdde700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f419dce0 con 0x7ff6f4100d60 2023-11-27T15:17:06.400 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.393+0000 7ff6fbdde700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f419de70 con 0x7ff6f4105fa0 2023-11-27T15:17:06.400 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.394+0000 7ff6f9b7a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6f4100d60 0x7ff6f4196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:06.400 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.394+0000 7ff6fa37b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6f4105fa0 0x7ff6f419d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:06.400 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.394+0000 7ff6f9b7a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6f4100d60 0x7ff6f4196020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:40360/0 (socket says 172.21.15.52:40360) 2023-11-27T15:17:06.400 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.394+0000 7ff6f9b7a700 1 -- 172.21.15.52:0/988349439 learned_addr learned my addr 172.21.15.52:0/988349439 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:06.401 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.394+0000 7ff6f9379700 1 --2- 172.21.15.52:0/988349439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6f4103680 0x7ff6f4196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:06.402 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.394+0000 7ff6f9379700 1 -- 172.21.15.52:0/988349439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6f4105fa0 msgr2=0x7ff6f419d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:06.402 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.394+0000 7ff6f9379700 1 --2- 172.21.15.52:0/988349439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6f4105fa0 0x7ff6f419d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.402 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.394+0000 7ff6f9379700 1 -- 172.21.15.52:0/988349439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6f4100d60 msgr2=0x7ff6f4196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:06.402 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.394+0000 7ff6f9379700 1 --2- 172.21.15.52:0/988349439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6f4100d60 0x7ff6f4196020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.402 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.394+0000 7ff6f9379700 1 -- 172.21.15.52:0/988349439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f00095f0 con 0x7ff6f4103680 2023-11-27T15:17:06.402 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.394+0000 7ff6f9b7a700 1 --2- 172.21.15.52:0/988349439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6f4100d60 0x7ff6f4196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:17:06.402 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.394+0000 7ff6f9379700 1 --2- 172.21.15.52:0/988349439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6f4103680 0x7ff6f4196560 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff6f000b5c0 tx=0x7ff6f000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:06.402 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.394+0000 7ff6eaffd700 1 -- 172.21.15.52:0/988349439 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f001d070 con 0x7ff6f4103680 2023-11-27T15:17:06.402 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.394+0000 7ff6eaffd700 1 -- 172.21.15.52:0/988349439 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6f0005030 con 0x7ff6f4103680 2023-11-27T15:17:06.403 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.394+0000 7ff6fbdde700 1 -- 172.21.15.52:0/988349439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f410bca0 con 0x7ff6f4103680 2023-11-27T15:17:06.403 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.395+0000 7ff6eaffd700 1 -- 172.21.15.52:0/988349439 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0004ab0 con 0x7ff6f4103680 2023-11-27T15:17:06.403 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.395+0000 7ff6fbdde700 1 -- 172.21.15.52:0/988349439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f410bb10 con 0x7ff6f4103680 2023-11-27T15:17:06.403 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.395+0000 7ff6fbdde700 1 -- 172.21.15.52:0/988349439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6f40fc370 con 0x7ff6f4103680 2023-11-27T15:17:06.403 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.397+0000 7ff6eaffd700 1 -- 172.21.15.52:0/988349439 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff6f0005430 con 0x7ff6f4103680 2023-11-27T15:17:06.405 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.397+0000 7ff6eaffd700 1 --2- 172.21.15.52:0/988349439 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff6e00a11e0 0x7ff6e00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:06.405 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.397+0000 7ff6f9b7a700 1 --2- 172.21.15.52:0/988349439 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff6e00a11e0 0x7ff6e00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:06.405 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.397+0000 7ff6eaffd700 1 -- 172.21.15.52:0/988349439 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff6f00cdd70 con 0x7ff6f4103680 2023-11-27T15:17:06.406 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.398+0000 7ff6f9b7a700 1 --2- 172.21.15.52:0/988349439 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff6e00a11e0 0x7ff6e00a3690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4005950 tx=0x7ff6e400b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:06.406 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.401+0000 7ff6eaffd700 1 -- 172.21.15.52:0/988349439 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff6f00973d0 con 0x7ff6f4103680 2023-11-27T15:17:06.603 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.598+0000 7ff6fbdde700 1 -- 172.21.15.52:0/988349439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff6f4061700 con 0x7ff6f4103680 2023-11-27T15:17:06.603 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.598+0000 7ff6eaffd700 1 -- 172.21.15.52:0/988349439 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff6f009a9f0 con 0x7ff6f4103680 2023-11-27T15:17:06.603 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2023-11-27T15:17:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.600+0000 7ff6fbdde700 1 -- 172.21.15.52:0/988349439 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff6e00a11e0 msgr2=0x7ff6e00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.600+0000 7ff6fbdde700 1 --2- 172.21.15.52:0/988349439 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff6e00a11e0 0x7ff6e00a3690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4005950 tx=0x7ff6e400b500 comp rx=0 tx=0).stop 2023-11-27T15:17:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.600+0000 7ff6fbdde700 1 -- 172.21.15.52:0/988349439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6f4103680 msgr2=0x7ff6f4196560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.600+0000 7ff6fbdde700 1 --2- 172.21.15.52:0/988349439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6f4103680 0x7ff6f4196560 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff6f000b5c0 tx=0x7ff6f000f690 comp rx=0 tx=0).stop 2023-11-27T15:17:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.600+0000 7ff6fbdde700 1 -- 172.21.15.52:0/988349439 shutdown_connections 2023-11-27T15:17:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.600+0000 7ff6fbdde700 1 --2- 172.21.15.52:0/988349439 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff6e00a11e0 0x7ff6e00a3690 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.600+0000 7ff6fbdde700 1 --2- 172.21.15.52:0/988349439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6f4100d60 0x7ff6f4196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.600+0000 7ff6fbdde700 1 --2- 172.21.15.52:0/988349439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6f4105fa0 0x7ff6f419d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.607 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.600+0000 7ff6fbdde700 1 --2- 172.21.15.52:0/988349439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6f4103680 0x7ff6f4196560 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.607 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.600+0000 7ff6fbdde700 1 -- 172.21.15.52:0/988349439 >> 172.21.15.52:0/988349439 conn(0x7ff6f40fa800 msgr2=0x7ff6f41019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:06.607 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.601+0000 7ff6fbdde700 1 -- 172.21.15.52:0/988349439 shutdown_connections 2023-11-27T15:17:06.607 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.601+0000 7ff6fbdde700 1 -- 172.21.15.52:0/988349439 wait complete. 2023-11-27T15:17:06.618 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-27T15:17:06.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.742+0000 7f20e155c700 1 -- 172.21.15.52:0/1138278027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20dc1002d0 msgr2=0x7f20dc1006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:06.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.742+0000 7f20e155c700 1 --2- 172.21.15.52:0/1138278027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20dc1002d0 0x7f20dc1006e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f20d0009990 tx=0x7f20d0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.742+0000 7f20e155c700 1 -- 172.21.15.52:0/1138278027 shutdown_connections 2023-11-27T15:17:06.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.742+0000 7f20e155c700 1 --2- 172.21.15.52:0/1138278027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20dc101e50 0x7f20dc108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.742+0000 7f20e155c700 1 --2- 172.21.15.52:0/1138278027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20dc1002d0 0x7f20dc1006e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.742+0000 7f20e155c700 1 --2- 172.21.15.52:0/1138278027 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20dc1014c0 0x7f20dc101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.742+0000 7f20e155c700 1 -- 172.21.15.52:0/1138278027 >> 172.21.15.52:0/1138278027 conn(0x7f20dc0fb8d0 msgr2=0x7f20dc0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:06.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.742+0000 7f20e155c700 1 -- 172.21.15.52:0/1138278027 shutdown_connections 2023-11-27T15:17:06.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.742+0000 7f20e155c700 1 -- 172.21.15.52:0/1138278027 wait complete. 2023-11-27T15:17:06.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.742+0000 7f20e155c700 1 Processor -- start 2023-11-27T15:17:06.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.743+0000 7f20e155c700 1 -- start start 2023-11-27T15:17:06.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.743+0000 7f20e155c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20dc1002d0 0x7f20dc196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:06.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.743+0000 7f20e155c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20dc1014c0 0x7f20dc196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:06.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.743+0000 7f20e155c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20dc101e50 0x7f20dc19d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:06.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.743+0000 7f20e155c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc19dba0 con 0x7f20dc101e50 2023-11-27T15:17:06.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.743+0000 7f20e155c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc19dce0 con 0x7f20dc1002d0 2023-11-27T15:17:06.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.743+0000 7f20e155c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc19de70 con 0x7f20dc1014c0 2023-11-27T15:17:06.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.743+0000 7f20db7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20dc101e50 0x7f20dc19d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:06.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.743+0000 7f20db7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20dc101e50 0x7f20dc19d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50720/0 (socket says 172.21.15.52:50720) 2023-11-27T15:17:06.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.743+0000 7f20db7fe700 1 -- 172.21.15.52:0/3400017259 learned_addr learned my addr 172.21.15.52:0/3400017259 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:06.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.743+0000 7f20daffd700 1 --2- 172.21.15.52:0/3400017259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20dc1002d0 0x7f20dc196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:06.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.743+0000 7f20da7fc700 1 --2- 172.21.15.52:0/3400017259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20dc1014c0 0x7f20dc196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:06.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.743+0000 7f20db7fe700 1 -- 172.21.15.52:0/3400017259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20dc1014c0 msgr2=0x7f20dc196560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:06.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.743+0000 7f20db7fe700 1 --2- 172.21.15.52:0/3400017259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20dc1014c0 0x7f20dc196560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.743+0000 7f20db7fe700 1 -- 172.21.15.52:0/3400017259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20dc1002d0 msgr2=0x7f20dc196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:06.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.743+0000 7f20db7fe700 1 --2- 172.21.15.52:0/3400017259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20dc1002d0 0x7f20dc196020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.743+0000 7f20db7fe700 1 -- 172.21.15.52:0/3400017259 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d00095f0 con 0x7f20dc101e50 2023-11-27T15:17:06.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.743+0000 7f20daffd700 1 --2- 172.21.15.52:0/3400017259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20dc1002d0 0x7f20dc196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:17:06.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.743+0000 7f20db7fe700 1 --2- 172.21.15.52:0/3400017259 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20dc101e50 0x7f20dc19d5d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f20d4007ee0 tx=0x7f20d400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:06.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.744+0000 7f20c3fff700 1 -- 172.21.15.52:0/3400017259 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d400ddd0 con 0x7f20dc101e50 2023-11-27T15:17:06.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.744+0000 7f20c3fff700 1 -- 172.21.15.52:0/3400017259 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20d400f040 con 0x7f20dc101e50 2023-11-27T15:17:06.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.744+0000 7f20e155c700 1 -- 172.21.15.52:0/3400017259 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20dc10bc30 con 0x7f20dc101e50 2023-11-27T15:17:06.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.744+0000 7f20c3fff700 1 -- 172.21.15.52:0/3400017259 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d4015400 con 0x7f20dc101e50 2023-11-27T15:17:06.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.744+0000 7f20e155c700 1 -- 172.21.15.52:0/3400017259 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20dc10baa0 con 0x7f20dc101e50 2023-11-27T15:17:06.754 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.745+0000 7f20e155c700 1 -- 172.21.15.52:0/3400017259 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20dc19e560 con 0x7f20dc101e50 2023-11-27T15:17:06.754 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.746+0000 7f20c3fff700 1 -- 172.21.15.52:0/3400017259 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f20d4004020 con 0x7f20dc101e50 2023-11-27T15:17:06.754 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.746+0000 7f20c3fff700 1 --2- 172.21.15.52:0/3400017259 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f20c40a1260 0x7f20c40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:06.754 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.746+0000 7f20daffd700 1 --2- 172.21.15.52:0/3400017259 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f20c40a1260 0x7f20c40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:06.754 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.746+0000 7f20c3fff700 1 -- 172.21.15.52:0/3400017259 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f20d40cc3e0 con 0x7f20dc101e50 2023-11-27T15:17:06.754 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.747+0000 7f20daffd700 1 --2- 172.21.15.52:0/3400017259 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f20c40a1260 0x7f20c40a3710 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f20d0006010 tx=0x7f20d000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:06.756 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.751+0000 7f20c3fff700 1 -- 172.21.15.52:0/3400017259 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f20d404e730 con 0x7f20dc101e50 2023-11-27T15:17:06.959 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.954+0000 7f20e155c700 1 -- 172.21.15.52:0/3400017259 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f20dc1071a0 con 0x7f20dc101e50 2023-11-27T15:17:06.959 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.954+0000 7f20c3fff700 1 -- 172.21.15.52:0/3400017259 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f20d40959b0 con 0x7f20dc101e50 2023-11-27T15:17:06.959 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2023-11-27T15:17:06.961 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.956+0000 7f20e155c700 1 -- 172.21.15.52:0/3400017259 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f20c40a1260 msgr2=0x7f20c40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:06.962 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.956+0000 7f20e155c700 1 --2- 172.21.15.52:0/3400017259 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f20c40a1260 0x7f20c40a3710 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f20d0006010 tx=0x7f20d000b540 comp rx=0 tx=0).stop 2023-11-27T15:17:06.962 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.956+0000 7f20e155c700 1 -- 172.21.15.52:0/3400017259 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20dc101e50 msgr2=0x7f20dc19d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:06.962 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.956+0000 7f20e155c700 1 --2- 172.21.15.52:0/3400017259 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20dc101e50 0x7f20dc19d5d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f20d4007ee0 tx=0x7f20d400d3b0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.962 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.956+0000 7f20e155c700 1 -- 172.21.15.52:0/3400017259 shutdown_connections 2023-11-27T15:17:06.962 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.956+0000 7f20e155c700 1 --2- 172.21.15.52:0/3400017259 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f20c40a1260 0x7f20c40a3710 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.962 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.956+0000 7f20e155c700 1 --2- 172.21.15.52:0/3400017259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20dc1002d0 0x7f20dc196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.962 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.956+0000 7f20e155c700 1 --2- 172.21.15.52:0/3400017259 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20dc101e50 0x7f20dc19d5d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.962 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.956+0000 7f20e155c700 1 --2- 172.21.15.52:0/3400017259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20dc1014c0 0x7f20dc196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:06.962 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.956+0000 7f20e155c700 1 -- 172.21.15.52:0/3400017259 >> 172.21.15.52:0/3400017259 conn(0x7f20dc0fb8d0 msgr2=0x7f20dc0fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:06.962 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.956+0000 7f20e155c700 1 -- 172.21.15.52:0/3400017259 shutdown_connections 2023-11-27T15:17:06.962 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:06.956+0000 7f20e155c700 1 -- 172.21.15.52:0/3400017259 wait complete. 2023-11-27T15:17:06.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:06 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1473486715' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:17:06.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:06 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1700595058' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:17:06.973 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:06 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/988349439' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:17:06.974 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:17:07.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:06 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1473486715' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:17:07.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:06 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1700595058' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:17:07.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:06 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/988349439' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:17:07.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.094+0000 7f36c2d76700 1 -- 172.21.15.52:0/1476884695 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36bc100db0 msgr2=0x7f36bc107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:07.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.094+0000 7f36c2d76700 1 --2- 172.21.15.52:0/1476884695 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36bc100db0 0x7f36bc107650 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f36b400b0a0 tx=0x7f36b400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:17:07.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.094+0000 7f36c2d76700 1 -- 172.21.15.52:0/1476884695 shutdown_connections 2023-11-27T15:17:07.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.094+0000 7f36c2d76700 1 --2- 172.21.15.52:0/1476884695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36bc0ff170 0x7f36bc0ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:07.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.094+0000 7f36c2d76700 1 --2- 172.21.15.52:0/1476884695 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36bc100db0 0x7f36bc107650 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:07.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.094+0000 7f36c2d76700 1 --2- 172.21.15.52:0/1476884695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36bc100400 0x7f36bc100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:07.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.094+0000 7f36c2d76700 1 -- 172.21.15.52:0/1476884695 >> 172.21.15.52:0/1476884695 conn(0x7f36bc0fa750 msgr2=0x7f36bc0fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:07.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.094+0000 7f36c2d76700 1 -- 172.21.15.52:0/1476884695 shutdown_connections 2023-11-27T15:17:07.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.094+0000 7f36c2d76700 1 -- 172.21.15.52:0/1476884695 wait complete. 2023-11-27T15:17:07.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.094+0000 7f36c2d76700 1 Processor -- start 2023-11-27T15:17:07.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.094+0000 7f36c2d76700 1 -- start start 2023-11-27T15:17:07.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.095+0000 7f36c2d76700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36bc0ff170 0x7f36bc194f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:07.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.095+0000 7f36c2d76700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36bc100400 0x7f36bc195480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:07.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.095+0000 7f36c2d76700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36bc100db0 0x7f36bc19c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:07.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.095+0000 7f36c2d76700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc19cac0 con 0x7f36bc100db0 2023-11-27T15:17:07.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.095+0000 7f36c2d76700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc19cc00 con 0x7f36bc100400 2023-11-27T15:17:07.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.095+0000 7f36c2d76700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc19cd90 con 0x7f36bc0ff170 2023-11-27T15:17:07.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.095+0000 7f36c1313700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36bc100db0 0x7f36bc19c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:07.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.095+0000 7f36c1313700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36bc100db0 0x7f36bc19c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50734/0 (socket says 172.21.15.52:50734) 2023-11-27T15:17:07.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.095+0000 7f36c1313700 1 -- 172.21.15.52:0/498031252 learned_addr learned my addr 172.21.15.52:0/498031252 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:07.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.095+0000 7f36bbfff700 1 --2- 172.21.15.52:0/498031252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36bc100400 0x7f36bc195480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:07.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.095+0000 7f36c0b12700 1 --2- 172.21.15.52:0/498031252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36bc0ff170 0x7f36bc194f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:07.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.095+0000 7f36c1313700 1 -- 172.21.15.52:0/498031252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36bc0ff170 msgr2=0x7f36bc194f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:07.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.095+0000 7f36c1313700 1 --2- 172.21.15.52:0/498031252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36bc0ff170 0x7f36bc194f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:07.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.095+0000 7f36c1313700 1 -- 172.21.15.52:0/498031252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36bc100400 msgr2=0x7f36bc195480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:07.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.095+0000 7f36c1313700 1 --2- 172.21.15.52:0/498031252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36bc100400 0x7f36bc195480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:07.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.095+0000 7f36c1313700 1 -- 172.21.15.52:0/498031252 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b4009d00 con 0x7f36bc100db0 2023-11-27T15:17:07.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.095+0000 7f36c0b12700 1 --2- 172.21.15.52:0/498031252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36bc0ff170 0x7f36bc194f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:17:07.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.095+0000 7f36c1313700 1 --2- 172.21.15.52:0/498031252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36bc100db0 0x7f36bc19c4f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f36b40062a0 tx=0x7f36b4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:07.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.095+0000 7f36b9ffb700 1 -- 172.21.15.52:0/498031252 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b400e040 con 0x7f36bc100db0 2023-11-27T15:17:07.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.095+0000 7f36b9ffb700 1 -- 172.21.15.52:0/498031252 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36b400cec0 con 0x7f36bc100db0 2023-11-27T15:17:07.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.095+0000 7f36c2d76700 1 -- 172.21.15.52:0/498031252 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36bc10ab10 con 0x7f36bc100db0 2023-11-27T15:17:07.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.095+0000 7f36b9ffb700 1 -- 172.21.15.52:0/498031252 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b4004020 con 0x7f36bc100db0 2023-11-27T15:17:07.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.096+0000 7f36c2d76700 1 -- 172.21.15.52:0/498031252 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36bc10a980 con 0x7f36bc100db0 2023-11-27T15:17:07.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.097+0000 7f36c2d76700 1 -- 172.21.15.52:0/498031252 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36bc106140 con 0x7f36bc100db0 2023-11-27T15:17:07.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.098+0000 7f36b9ffb700 1 -- 172.21.15.52:0/498031252 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f36b401d690 con 0x7f36bc100db0 2023-11-27T15:17:07.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.098+0000 7f36b9ffb700 1 --2- 172.21.15.52:0/498031252 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f36ac0a1260 0x7f36ac0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:07.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.098+0000 7f36b9ffb700 1 -- 172.21.15.52:0/498031252 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f36b40ce220 con 0x7f36bc100db0 2023-11-27T15:17:07.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.098+0000 7f36c0b12700 1 --2- 172.21.15.52:0/498031252 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f36ac0a1260 0x7f36ac0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:07.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.099+0000 7f36c0b12700 1 --2- 172.21.15.52:0/498031252 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f36ac0a1260 0x7f36ac0a3710 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f36b0005fd0 tx=0x7f36b0005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:07.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.103+0000 7f36b9ffb700 1 -- 172.21.15.52:0/498031252 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f36b4097880 con 0x7f36bc100db0 2023-11-27T15:17:07.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:06 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1473486715' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:17:07.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:06 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1700595058' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:17:07.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:06 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/988349439' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:17:07.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.347+0000 7f36c2d76700 1 -- 172.21.15.52:0/498031252 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36bc02cfe0 con 0x7f36bc100db0 2023-11-27T15:17:07.353 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.348+0000 7f36b9ffb700 1 -- 172.21.15.52:0/498031252 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10804 (secure 0 0 0) 0x7f36b409aea0 con 0x7f36bc100db0 2023-11-27T15:17:07.353 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:17:07.353 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2328881376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2328881376},{"type":"v1","addr":"172.21.15.52:6839","nonce":2328881376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2799551570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2799551570},{"type":"v1","addr":"172.21.15.52:6841","nonce":2799551570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:16:09.319630+0000","modified":"2023-11-27T15:16:14.742405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24494,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/215725838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":215725838},{"type":"v1","addr":"172.21.15.150:6839","nonce":215725838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3571830559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3571830559},{"type":"v1","addr":"172.21.15.102:6839","nonce":3571830559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:17:07.356 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.350+0000 7f36c2d76700 1 -- 172.21.15.52:0/498031252 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f36ac0a1260 msgr2=0x7f36ac0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:07.356 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.350+0000 7f36c2d76700 1 --2- 172.21.15.52:0/498031252 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f36ac0a1260 0x7f36ac0a3710 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f36b0005fd0 tx=0x7f36b0005f60 comp rx=0 tx=0).stop 2023-11-27T15:17:07.356 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.350+0000 7f36c2d76700 1 -- 172.21.15.52:0/498031252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36bc100db0 msgr2=0x7f36bc19c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:07.356 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.350+0000 7f36c2d76700 1 --2- 172.21.15.52:0/498031252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36bc100db0 0x7f36bc19c4f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f36b40062a0 tx=0x7f36b4004d00 comp rx=0 tx=0).stop 2023-11-27T15:17:07.356 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.350+0000 7f36c2d76700 1 -- 172.21.15.52:0/498031252 shutdown_connections 2023-11-27T15:17:07.356 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.350+0000 7f36c2d76700 1 --2- 172.21.15.52:0/498031252 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f36ac0a1260 0x7f36ac0a3710 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:07.356 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.350+0000 7f36c2d76700 1 --2- 172.21.15.52:0/498031252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36bc100db0 0x7f36bc19c4f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:07.356 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.350+0000 7f36c2d76700 1 --2- 172.21.15.52:0/498031252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36bc0ff170 0x7f36bc194f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:07.356 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.350+0000 7f36c2d76700 1 --2- 172.21.15.52:0/498031252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36bc100400 0x7f36bc195480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:07.357 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.350+0000 7f36c2d76700 1 -- 172.21.15.52:0/498031252 >> 172.21.15.52:0/498031252 conn(0x7f36bc0fa750 msgr2=0x7f36bc0fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:07.357 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.351+0000 7f36c2d76700 1 -- 172.21.15.52:0/498031252 shutdown_connections 2023-11-27T15:17:07.357 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.351+0000 7f36c2d76700 1 -- 172.21.15.52:0/498031252 wait complete. 2023-11-27T15:17:07.357 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 23 2023-11-27T15:17:07.368 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:17:07.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.488+0000 7f8e3ade9700 1 -- 172.21.15.52:0/3152717325 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e34100d40 msgr2=0x7f8e34107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:07.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.488+0000 7f8e3ade9700 1 --2- 172.21.15.52:0/3152717325 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e34100d40 0x7f8e34107600 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c00b0a0 tx=0x7f8e2c00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:17:07.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.489+0000 7f8e3ade9700 1 -- 172.21.15.52:0/3152717325 shutdown_connections 2023-11-27T15:17:07.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.489+0000 7f8e3ade9700 1 --2- 172.21.15.52:0/3152717325 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e34100d40 0x7f8e34107600 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:07.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.489+0000 7f8e3ade9700 1 --2- 172.21.15.52:0/3152717325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e340ff1c0 0x7f8e340ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:07.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.489+0000 7f8e3ade9700 1 --2- 172.21.15.52:0/3152717325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e341003b0 0x7f8e34100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:07.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.489+0000 7f8e3ade9700 1 -- 172.21.15.52:0/3152717325 >> 172.21.15.52:0/3152717325 conn(0x7f8e340fa7a0 msgr2=0x7f8e340fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:07.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.489+0000 7f8e3ade9700 1 -- 172.21.15.52:0/3152717325 shutdown_connections 2023-11-27T15:17:07.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.489+0000 7f8e3ade9700 1 -- 172.21.15.52:0/3152717325 wait complete. 2023-11-27T15:17:07.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.489+0000 7f8e3ade9700 1 Processor -- start 2023-11-27T15:17:07.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.489+0000 7f8e3ade9700 1 -- start start 2023-11-27T15:17:07.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.489+0000 7f8e3ade9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e340ff1c0 0x7f8e34069950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:07.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.489+0000 7f8e3ade9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e341003b0 0x7f8e3419ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:07.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.489+0000 7f8e3ade9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e34100d40 0x7f8e3419d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:07.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.489+0000 7f8e3ade9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3419db80 con 0x7f8e340ff1c0 2023-11-27T15:17:07.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.489+0000 7f8e3ade9700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3419dcf0 con 0x7f8e341003b0 2023-11-27T15:17:07.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.489+0000 7f8e3ade9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3419deb0 con 0x7f8e34100d40 2023-11-27T15:17:07.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.489+0000 7f8e38b85700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e340ff1c0 0x7f8e34069950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:07.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.490+0000 7f8e38b85700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e340ff1c0 0x7f8e34069950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50744/0 (socket says 172.21.15.52:50744) 2023-11-27T15:17:07.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.490+0000 7f8e38b85700 1 -- 172.21.15.52:0/3940266873 learned_addr learned my addr 172.21.15.52:0/3940266873 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:07.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.490+0000 7f8e33fff700 1 --2- 172.21.15.52:0/3940266873 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e341003b0 0x7f8e3419ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:07.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.490+0000 7f8e39386700 1 --2- 172.21.15.52:0/3940266873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e34100d40 0x7f8e3419d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:07.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.490+0000 7f8e38b85700 1 -- 172.21.15.52:0/3940266873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e34100d40 msgr2=0x7f8e3419d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:07.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.490+0000 7f8e38b85700 1 --2- 172.21.15.52:0/3940266873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e34100d40 0x7f8e3419d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:07.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.490+0000 7f8e38b85700 1 -- 172.21.15.52:0/3940266873 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e341003b0 msgr2=0x7f8e3419ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:07.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.490+0000 7f8e38b85700 1 --2- 172.21.15.52:0/3940266873 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e341003b0 0x7f8e3419ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:07.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.490+0000 7f8e38b85700 1 -- 172.21.15.52:0/3940266873 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2c009d00 con 0x7f8e340ff1c0 2023-11-27T15:17:07.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.490+0000 7f8e33fff700 1 --2- 172.21.15.52:0/3940266873 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e341003b0 0x7f8e3419ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:17:07.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.490+0000 7f8e38b85700 1 --2- 172.21.15.52:0/3940266873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e340ff1c0 0x7f8e34069950 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800d980 tx=0x7f8e2800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:07.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.490+0000 7f8e31ffb700 1 -- 172.21.15.52:0/3940266873 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2800bb60 con 0x7f8e340ff1c0 2023-11-27T15:17:07.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.490+0000 7f8e3ade9700 1 -- 172.21.15.52:0/3940266873 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e3410ab40 con 0x7f8e340ff1c0 2023-11-27T15:17:07.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.490+0000 7f8e31ffb700 1 -- 172.21.15.52:0/3940266873 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e28007620 con 0x7f8e340ff1c0 2023-11-27T15:17:07.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.490+0000 7f8e31ffb700 1 -- 172.21.15.52:0/3940266873 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2800f620 con 0x7f8e340ff1c0 2023-11-27T15:17:07.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.490+0000 7f8e3ade9700 1 -- 172.21.15.52:0/3940266873 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e3410a9b0 con 0x7f8e340ff1c0 2023-11-27T15:17:07.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.491+0000 7f8e3ade9700 1 -- 172.21.15.52:0/3940266873 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e341066e0 con 0x7f8e340ff1c0 2023-11-27T15:17:07.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.492+0000 7f8e31ffb700 1 -- 172.21.15.52:0/3940266873 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8e28010460 con 0x7f8e340ff1c0 2023-11-27T15:17:07.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.493+0000 7f8e31ffb700 1 --2- 172.21.15.52:0/3940266873 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f8e240a11e0 0x7f8e240a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:07.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.493+0000 7f8e31ffb700 1 -- 172.21.15.52:0/3940266873 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8e280cd380 con 0x7f8e340ff1c0 2023-11-27T15:17:07.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.493+0000 7f8e33fff700 1 --2- 172.21.15.52:0/3940266873 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f8e240a11e0 0x7f8e240a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:07.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.493+0000 7f8e33fff700 1 --2- 172.21.15.52:0/3940266873 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f8e240a11e0 0x7f8e240a3690 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8e340696c0 tx=0x7f8e20009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:07.501 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.496+0000 7f8e31ffb700 1 -- 172.21.15.52:0/3940266873 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8e280968f0 con 0x7f8e340ff1c0 2023-11-27T15:17:07.705 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.700+0000 7f8e3ade9700 1 -- 172.21.15.52:0/3940266873 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e3402cfe0 con 0x7f8e340ff1c0 2023-11-27T15:17:07.707 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.701+0000 7f8e31ffb700 1 -- 172.21.15.52:0/3940266873 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22657 (secure 0 0 0) 0x7f8e2800de60 con 0x7f8e340ff1c0 2023-11-27T15:17:07.707 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:17:07.707 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":85,"fsid":"c6c7df88-8d36-11ee-95a2-87774f69a715","created":"2023-11-27T15:09:03.333345+0000","modified":"2023-11-27T15:17:04.797892+0000","last_up_change":"2023-11-27T15:15:27.656134+0000","last_in_change":"2023-11-27T15:15:10.671153+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:12:14.177523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:16:07.842251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:16:08.728520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58d69ede-c293-49f5-abbe-8187793f2b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6803","nonce":608216415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6805","nonce":608216415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6809","nonce":608216415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6807","nonce":608216415}]},"public_addr":"172.21.15.52:6803/608216415","cluster_addr":"172.21.15.52:6805/608216415","heartbeat_back_addr":"172.21.15.52:6809/608216415","heartbeat_front_addr":"172.21.15.52:6807/608216415","state":["exists","up"]},{"osd":1,"uuid":"d6a46b91-adc8-45ef-80f5-566ae400911f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6801","nonce":2538384087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6803","nonce":2538384087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6807","nonce":2538384087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6805","nonce":2538384087}]},"public_addr":"172.21.15.102:6801/2538384087","cluster_addr":"172.21.15.102:6803/2538384087","heartbeat_back_addr":"172.21.15.102:6807/2538384087","heartbeat_front_addr":"172.21.15.102:6805/2538384087","state":["exists","up"]},{"osd":2,"uuid":"846ba125-11f8-46b2-8941-bd5e1947cc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6801","nonce":4258038221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6803","nonce":4258038221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6807","nonce":4258038221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6805","nonce":4258038221}]},"public_addr":"172.21.15.150:6801/4258038221","cluster_addr":"172.21.15.150:6803/4258038221","heartbeat_back_addr":"172.21.15.150:6807/4258038221","heartbeat_front_addr":"172.21.15.150:6805/4258038221","state":["exists","up"]},{"osd":3,"uuid":"950fcd67-1057-4288-a389-714f1ec94261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6811","nonce":2622100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6813","nonce":2622100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6817","nonce":2622100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6815","nonce":2622100752}]},"public_addr":"172.21.15.52:6811/2622100752","cluster_addr":"172.21.15.52:6813/2622100752","heartbeat_back_addr":"172.21.15.52:6817/2622100752","heartbeat_front_addr":"172.21.15.52:6815/2622100752","state":["exists","up"]},{"osd":4,"uuid":"1c96a676-cea9-4deb-bad1-f7fd240c3d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6809","nonce":1546794763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6811","nonce":1546794763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6815","nonce":1546794763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6813","nonce":1546794763}]},"public_addr":"172.21.15.102:6809/1546794763","cluster_addr":"172.21.15.102:6811/1546794763","heartbeat_back_addr":"172.21.15.102:6815/1546794763","heartbeat_front_addr":"172.21.15.102:6813/1546794763","state":["exists","up"]},{"osd":5,"uuid":"38a75af1-fd4d-433e-b579-0609109c851c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6809","nonce":461511375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6811","nonce":461511375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6815","nonce":461511375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6813","nonce":461511375}]},"public_addr":"172.21.15.150:6809/461511375","cluster_addr":"172.21.15.150:6811/461511375","heartbeat_back_addr":"172.21.15.150:6815/461511375","heartbeat_front_addr":"172.21.15.150:6813/461511375","state":["exists","up"]},{"osd":6,"uuid":"2f0ede4a-d175-4b4a-b876-c0f442048901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6819","nonce":1002457464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6821","nonce":1002457464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6825","nonce":1002457464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6823","nonce":1002457464}]},"public_addr":"172.21.15.52:6819/1002457464","cluster_addr":"172.21.15.52:6821/1002457464","heartbeat_back_addr":"172.21.15.52:6825/1002457464","heartbeat_front_addr":"172.21.15.52:6823/1002457464","state":["exists","up"]},{"osd":7,"uuid":"6b635192-abcf-4bbf-bf78-a019b351bca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6817","nonce":3249840064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6819","nonce":3249840064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6823","nonce":3249840064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6821","nonce":3249840064}]},"public_addr":"172.21.15.102:6817/3249840064","cluster_addr":"172.21.15.102:6819/3249840064","heartbeat_back_addr":"172.21.15.102:6823/3249840064","heartbeat_front_addr":"172.21.15.102:6821/3249840064","state":["exists","up"]},{"osd":8,"uuid":"c7ca8007-7170-47bf-956c-3f00edf27e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6817","nonce":3672583574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6819","nonce":3672583574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6823","nonce":3672583574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6821","nonce":3672583574}]},"public_addr":"172.21.15.150:6817/3672583574","cluster_addr":"172.21.15.150:6819/3672583574","heartbeat_back_addr":"172.21.15.150:6823/3672583574","heartbeat_front_addr":"172.21.15.150:6821/3672583574","state":["exists","up"]},{"osd":9,"uuid":"0e252c2c-01a4-48ba-832d-067ed940fefb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6827","nonce":4287934320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6829","nonce":4287934320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6833","nonce":4287934320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6831","nonce":4287934320}]},"public_addr":"172.21.15.52:6827/4287934320","cluster_addr":"172.21.15.52:6829/4287934320","heartbeat_back_addr":"172.21.15.52:6833/4287934320","heartbeat_front_addr":"172.21.15.52:6831/4287934320","state":["exists","up"]},{"osd":10,"uuid":"bb540142-7563-44a4-b7a3-cb662714c128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6825","nonce":3056034201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6827","nonce":3056034201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6831","nonce":3056034201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6829","nonce":3056034201}]},"public_addr":"172.21.15.102:6825/3056034201","cluster_addr":"172.21.15.102:6827/3056034201","heartbeat_back_addr":"172.21.15.102:6831/3056034201","heartbeat_front_addr":"172.21.15.102:6829/3056034201","state":["exists","up"]},{"osd":11,"uuid":"3156528e-e0f8-4d44-9946-c8b6339c2bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6825","nonce":3207559739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6827","nonce":3207559739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6831","nonce":3207559739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6829","nonce":3207559739}]},"public_addr":"172.21.15.150:6825/3207559739","cluster_addr":"172.21.15.150:6827/3207559739","heartbeat_back_addr":"172.21.15.150:6831/3207559739","heartbeat_front_addr":"172.21.15.150:6829/3207559739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:26.826099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:48.331371+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:10.459738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:32.356472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:54.043170+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:15.648582+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:37.166463+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:59.240819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:20.710097+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:43.364563+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:05.279433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:26.344573+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":5,"to":7}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/1392282110":"2023-11-28T15:10:03.644870+0000","172.21.15.52:0/1479993479":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6800/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/961562652":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/1290665390":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2410529742":"2023-11-28T15:09:37.118690+0000","172.21.15.52:6800/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2130037044":"2023-11-28T15:10:03.644870+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:17:07.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.704+0000 7f8e3ade9700 1 -- 172.21.15.52:0/3940266873 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f8e240a11e0 msgr2=0x7f8e240a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:07.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.704+0000 7f8e3ade9700 1 --2- 172.21.15.52:0/3940266873 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f8e240a11e0 0x7f8e240a3690 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8e340696c0 tx=0x7f8e20009260 comp rx=0 tx=0).stop 2023-11-27T15:17:07.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.704+0000 7f8e3ade9700 1 -- 172.21.15.52:0/3940266873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e340ff1c0 msgr2=0x7f8e34069950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:07.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.704+0000 7f8e3ade9700 1 --2- 172.21.15.52:0/3940266873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e340ff1c0 0x7f8e34069950 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800d980 tx=0x7f8e2800dd40 comp rx=0 tx=0).stop 2023-11-27T15:17:07.710 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.704+0000 7f8e3ade9700 1 -- 172.21.15.52:0/3940266873 shutdown_connections 2023-11-27T15:17:07.710 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.704+0000 7f8e3ade9700 1 --2- 172.21.15.52:0/3940266873 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f8e240a11e0 0x7f8e240a3690 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:07.710 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.704+0000 7f8e3ade9700 1 --2- 172.21.15.52:0/3940266873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e34100d40 0x7f8e3419d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:07.710 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.704+0000 7f8e3ade9700 1 --2- 172.21.15.52:0/3940266873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e340ff1c0 0x7f8e34069950 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:07.710 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.704+0000 7f8e3ade9700 1 --2- 172.21.15.52:0/3940266873 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8e341003b0 0x7f8e3419ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:07.710 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.704+0000 7f8e3ade9700 1 -- 172.21.15.52:0/3940266873 >> 172.21.15.52:0/3940266873 conn(0x7f8e340fa7a0 msgr2=0x7f8e34103820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:07.710 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.704+0000 7f8e3ade9700 1 -- 172.21.15.52:0/3940266873 shutdown_connections 2023-11-27T15:17:07.710 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.704+0000 7f8e3ade9700 1 -- 172.21.15.52:0/3940266873 wait complete. 2023-11-27T15:17:07.722 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:12:14.177523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-27T15:16:07.842251+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-27T15:16:08.728520+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-27T15:17:07.722 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:17:07.853 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.846+0000 7f953408e700 1 -- 172.21.15.52:0/853270489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f952c1002f0 msgr2=0x7f952c100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:07.854 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.846+0000 7f953408e700 1 --2- 172.21.15.52:0/853270489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f952c1002f0 0x7f952c100700 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f951c009990 tx=0x7f951c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:07.854 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.846+0000 7f953408e700 1 -- 172.21.15.52:0/853270489 shutdown_connections 2023-11-27T15:17:07.854 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.846+0000 7f953408e700 1 --2- 172.21.15.52:0/853270489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f952c101e70 0x7f952c108730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:07.854 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.846+0000 7f953408e700 1 --2- 172.21.15.52:0/853270489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f952c1002f0 0x7f952c100700 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:07.854 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.846+0000 7f953408e700 1 --2- 172.21.15.52:0/853270489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f952c1014e0 0x7f952c101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:07.854 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.846+0000 7f953408e700 1 -- 172.21.15.52:0/853270489 >> 172.21.15.52:0/853270489 conn(0x7f952c0fb930 msgr2=0x7f952c0fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:07.854 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.846+0000 7f953408e700 1 -- 172.21.15.52:0/853270489 shutdown_connections 2023-11-27T15:17:07.854 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.846+0000 7f953408e700 1 -- 172.21.15.52:0/853270489 wait complete. 2023-11-27T15:17:07.854 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.847+0000 7f953408e700 1 Processor -- start 2023-11-27T15:17:07.854 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.847+0000 7f953408e700 1 -- start start 2023-11-27T15:17:07.854 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.847+0000 7f953408e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f952c1002f0 0x7f952c1960c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:07.855 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.847+0000 7f953408e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f952c1014e0 0x7f952c196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:07.855 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.847+0000 7f953408e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f952c101e70 0x7f952c19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:07.855 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.847+0000 7f953408e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c19dc40 con 0x7f952c1014e0 2023-11-27T15:17:07.855 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.847+0000 7f953408e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c19dd80 con 0x7f952c101e70 2023-11-27T15:17:07.855 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.847+0000 7f953408e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c19df10 con 0x7f952c1002f0 2023-11-27T15:17:07.855 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.847+0000 7f9531629700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f952c1014e0 0x7f952c196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:07.855 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.847+0000 7f9531629700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f952c1014e0 0x7f952c196600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50758/0 (socket says 172.21.15.52:50758) 2023-11-27T15:17:07.856 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.847+0000 7f9531629700 1 -- 172.21.15.52:0/3511366180 learned_addr learned my addr 172.21.15.52:0/3511366180 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:07.856 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.847+0000 7f953262b700 1 --2- 172.21.15.52:0/3511366180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f952c101e70 0x7f952c19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:07.856 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.847+0000 7f9531e2a700 1 --2- 172.21.15.52:0/3511366180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f952c1002f0 0x7f952c1960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:07.856 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.848+0000 7f9531629700 1 -- 172.21.15.52:0/3511366180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f952c1002f0 msgr2=0x7f952c1960c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:07.856 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.848+0000 7f9531629700 1 --2- 172.21.15.52:0/3511366180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f952c1002f0 0x7f952c1960c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:07.856 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.848+0000 7f9531629700 1 -- 172.21.15.52:0/3511366180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f952c101e70 msgr2=0x7f952c19d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:07.856 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.848+0000 7f9531629700 1 --2- 172.21.15.52:0/3511366180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f952c101e70 0x7f952c19d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:07.856 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.848+0000 7f9531629700 1 -- 172.21.15.52:0/3511366180 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c0095f0 con 0x7f952c1014e0 2023-11-27T15:17:07.856 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.848+0000 7f9531629700 1 --2- 172.21.15.52:0/3511366180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f952c1014e0 0x7f952c196600 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f952800d950 tx=0x7f952800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:07.856 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.848+0000 7f9522ffd700 1 -- 172.21.15.52:0/3511366180 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9528009b40 con 0x7f952c1014e0 2023-11-27T15:17:07.856 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.848+0000 7f9522ffd700 1 -- 172.21.15.52:0/3511366180 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9528009ca0 con 0x7f952c1014e0 2023-11-27T15:17:07.857 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.848+0000 7f953408e700 1 -- 172.21.15.52:0/3511366180 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952c10bc70 con 0x7f952c1014e0 2023-11-27T15:17:07.857 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.848+0000 7f9522ffd700 1 -- 172.21.15.52:0/3511366180 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952800f620 con 0x7f952c1014e0 2023-11-27T15:17:07.857 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.848+0000 7f953408e700 1 -- 172.21.15.52:0/3511366180 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952c10bae0 con 0x7f952c1014e0 2023-11-27T15:17:07.857 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.849+0000 7f953408e700 1 -- 172.21.15.52:0/3511366180 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f952c107890 con 0x7f952c1014e0 2023-11-27T15:17:07.858 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.850+0000 7f9522ffd700 1 -- 172.21.15.52:0/3511366180 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9528010460 con 0x7f952c1014e0 2023-11-27T15:17:07.858 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.850+0000 7f9522ffd700 1 --2- 172.21.15.52:0/3511366180 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f95180a1260 0x7f95180a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:07.858 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.851+0000 7f9531e2a700 1 --2- 172.21.15.52:0/3511366180 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f95180a1260 0x7f95180a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:07.858 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.851+0000 7f9522ffd700 1 -- 172.21.15.52:0/3511366180 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f95280ccdc0 con 0x7f952c1014e0 2023-11-27T15:17:07.858 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.851+0000 7f9531e2a700 1 --2- 172.21.15.52:0/3511366180 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f95180a1260 0x7f95180a3710 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f951c006010 tx=0x7f951c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:07.860 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:07.855+0000 7f9522ffd700 1 -- 172.21.15.52:0/3511366180 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9528096390 con 0x7f952c1014e0 2023-11-27T15:17:08.062 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.057+0000 7f953408e700 1 -- 172.21.15.52:0/3511366180 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f952c061700 con 0x7f952c1014e0 2023-11-27T15:17:08.062 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.057+0000 7f9522ffd700 1 -- 172.21.15.52:0/3511366180 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f95280200a0 con 0x7f952c1014e0 2023-11-27T15:17:08.063 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2023-11-27T15:17:08.065 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.060+0000 7f953408e700 1 -- 172.21.15.52:0/3511366180 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f95180a1260 msgr2=0x7f95180a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:08.065 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.060+0000 7f953408e700 1 --2- 172.21.15.52:0/3511366180 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f95180a1260 0x7f95180a3710 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f951c006010 tx=0x7f951c00b540 comp rx=0 tx=0).stop 2023-11-27T15:17:08.065 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.060+0000 7f953408e700 1 -- 172.21.15.52:0/3511366180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f952c1014e0 msgr2=0x7f952c196600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:08.065 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.060+0000 7f953408e700 1 --2- 172.21.15.52:0/3511366180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f952c1014e0 0x7f952c196600 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f952800d950 tx=0x7f952800dd10 comp rx=0 tx=0).stop 2023-11-27T15:17:08.065 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.060+0000 7f953408e700 1 -- 172.21.15.52:0/3511366180 shutdown_connections 2023-11-27T15:17:08.065 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.060+0000 7f953408e700 1 --2- 172.21.15.52:0/3511366180 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f95180a1260 0x7f95180a3710 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.066 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.060+0000 7f953408e700 1 --2- 172.21.15.52:0/3511366180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f952c101e70 0x7f952c19d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.066 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.060+0000 7f953408e700 1 --2- 172.21.15.52:0/3511366180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f952c1002f0 0x7f952c1960c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.066 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.060+0000 7f953408e700 1 --2- 172.21.15.52:0/3511366180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f952c1014e0 0x7f952c196600 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.066 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.060+0000 7f953408e700 1 -- 172.21.15.52:0/3511366180 >> 172.21.15.52:0/3511366180 conn(0x7f952c0fb930 msgr2=0x7f952c1049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:08.066 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.060+0000 7f953408e700 1 -- 172.21.15.52:0/3511366180 shutdown_connections 2023-11-27T15:17:08.066 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.060+0000 7f953408e700 1 -- 172.21.15.52:0/3511366180 wait complete. 2023-11-27T15:17:08.076 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:07 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3400017259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:17:08.076 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:07 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/498031252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:17:08.076 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:07 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3940266873' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:17:08.076 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:07 smithi052 ceph-mon[99056]: pgmap v287: 97 pgs: 2 peering, 95 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2023-11-27T15:17:08.077 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-27T15:17:08.209 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.201+0000 7f54e6374700 1 -- 172.21.15.52:0/2639461061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54e01003b0 msgr2=0x7f54e0100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:08.209 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.201+0000 7f54e6374700 1 --2- 172.21.15.52:0/2639461061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54e01003b0 0x7f54e0100800 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f54c8009990 tx=0x7f54c8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.209 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.201+0000 7f54e6374700 1 -- 172.21.15.52:0/2639461061 shutdown_connections 2023-11-27T15:17:08.209 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.201+0000 7f54e6374700 1 --2- 172.21.15.52:0/2639461061 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54e00ff1c0 0x7f54e00ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.209 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.201+0000 7f54e6374700 1 --2- 172.21.15.52:0/2639461061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54e0100d40 0x7f54e0107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.209 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.201+0000 7f54e6374700 1 --2- 172.21.15.52:0/2639461061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54e01003b0 0x7f54e0100800 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.209 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.201+0000 7f54e6374700 1 -- 172.21.15.52:0/2639461061 >> 172.21.15.52:0/2639461061 conn(0x7f54e00fa7a0 msgr2=0x7f54e00fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:08.209 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.201+0000 7f54e6374700 1 -- 172.21.15.52:0/2639461061 shutdown_connections 2023-11-27T15:17:08.209 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.201+0000 7f54e6374700 1 -- 172.21.15.52:0/2639461061 wait complete. 2023-11-27T15:17:08.209 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.202+0000 7f54e6374700 1 Processor -- start 2023-11-27T15:17:08.210 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.202+0000 7f54e6374700 1 -- start start 2023-11-27T15:17:08.210 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.202+0000 7f54e6374700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54e00ff1c0 0x7f54e0103150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:08.210 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.202+0000 7f54e6374700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54e01003b0 0x7f54e0101790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:08.210 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.202+0000 7f54e6374700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54e0100d40 0x7f54e0101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:08.210 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.202+0000 7f54e6374700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e0102470 con 0x7f54e00ff1c0 2023-11-27T15:17:08.210 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.202+0000 7f54e6374700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e01025e0 con 0x7f54e0100d40 2023-11-27T15:17:08.210 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.202+0000 7f54e6374700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e01a00c0 con 0x7f54e01003b0 2023-11-27T15:17:08.210 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.202+0000 7f54dffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54e00ff1c0 0x7f54e0103150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:08.210 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.202+0000 7f54dffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54e00ff1c0 0x7f54e0103150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50770/0 (socket says 172.21.15.52:50770) 2023-11-27T15:17:08.210 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.202+0000 7f54dffff700 1 -- 172.21.15.52:0/996056965 learned_addr learned my addr 172.21.15.52:0/996056965 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:08.210 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.202+0000 7f54d7fff700 1 --2- 172.21.15.52:0/996056965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54e01003b0 0x7f54e0101790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:08.211 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.202+0000 7f54e4911700 1 --2- 172.21.15.52:0/996056965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54e0100d40 0x7f54e0101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:08.211 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.202+0000 7f54dffff700 1 -- 172.21.15.52:0/996056965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54e01003b0 msgr2=0x7f54e0101790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:08.211 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.202+0000 7f54dffff700 1 --2- 172.21.15.52:0/996056965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54e01003b0 0x7f54e0101790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.211 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.202+0000 7f54dffff700 1 -- 172.21.15.52:0/996056965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54e0100d40 msgr2=0x7f54e0101cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:08.211 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.202+0000 7f54dffff700 1 --2- 172.21.15.52:0/996056965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54e0100d40 0x7f54e0101cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.211 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.202+0000 7f54dffff700 1 -- 172.21.15.52:0/996056965 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c80095f0 con 0x7f54e00ff1c0 2023-11-27T15:17:08.211 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.202+0000 7f54e4911700 1 --2- 172.21.15.52:0/996056965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54e0100d40 0x7f54e0101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:17:08.211 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.202+0000 7f54dffff700 1 --2- 172.21.15.52:0/996056965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54e00ff1c0 0x7f54e0103150 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f54d000d980 tx=0x7f54d000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:08.211 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.203+0000 7f54ddffb700 1 -- 172.21.15.52:0/996056965 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d0004020 con 0x7f54e00ff1c0 2023-11-27T15:17:08.211 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.203+0000 7f54ddffb700 1 -- 172.21.15.52:0/996056965 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54d0009730 con 0x7f54e00ff1c0 2023-11-27T15:17:08.211 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.203+0000 7f54e6374700 1 -- 172.21.15.52:0/996056965 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e010ab70 con 0x7f54e00ff1c0 2023-11-27T15:17:08.211 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.203+0000 7f54ddffb700 1 -- 172.21.15.52:0/996056965 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d0003a80 con 0x7f54e00ff1c0 2023-11-27T15:17:08.212 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.203+0000 7f54e6374700 1 -- 172.21.15.52:0/996056965 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e010a9e0 con 0x7f54e00ff1c0 2023-11-27T15:17:08.212 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.204+0000 7f54e6374700 1 -- 172.21.15.52:0/996056965 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54e01a0860 con 0x7f54e00ff1c0 2023-11-27T15:17:08.212 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.205+0000 7f54ddffb700 1 -- 172.21.15.52:0/996056965 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f54d00098a0 con 0x7f54e00ff1c0 2023-11-27T15:17:08.212 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.205+0000 7f54ddffb700 1 --2- 172.21.15.52:0/996056965 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f54cc0a1260 0x7f54cc0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:08.213 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.206+0000 7f54ddffb700 1 -- 172.21.15.52:0/996056965 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f54d00cca80 con 0x7f54e00ff1c0 2023-11-27T15:17:08.213 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.206+0000 7f54d7fff700 1 --2- 172.21.15.52:0/996056965 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f54cc0a1260 0x7f54cc0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:08.213 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.206+0000 7f54d7fff700 1 --2- 172.21.15.52:0/996056965 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f54cc0a1260 0x7f54cc0a3710 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f54c800b5c0 tx=0x7f54c8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:08.214 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.209+0000 7f54ddffb700 1 -- 172.21.15.52:0/996056965 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f54d00960d0 con 0x7f54e00ff1c0 2023-11-27T15:17:08.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:07 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3400017259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:17:08.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:07 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/498031252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:17:08.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:07 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3940266873' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:17:08.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:07 smithi150 ceph-mon[100559]: pgmap v287: 97 pgs: 2 peering, 95 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2023-11-27T15:17:08.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:07 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3400017259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:17:08.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:07 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/498031252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:17:08.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:07 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3940266873' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:17:08.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:07 smithi102 ceph-mon[100296]: pgmap v287: 97 pgs: 2 peering, 95 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2023-11-27T15:17:08.414 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.409+0000 7f54e6374700 1 -- 172.21.15.52:0/996056965 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f54e01a0e10 con 0x7f54e00ff1c0 2023-11-27T15:17:08.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.409+0000 7f54ddffb700 1 -- 172.21.15.52:0/996056965 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f54d00996f0 con 0x7f54e00ff1c0 2023-11-27T15:17:08.415 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2023-11-27T15:17:08.417 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.411+0000 7f54e6374700 1 -- 172.21.15.52:0/996056965 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f54cc0a1260 msgr2=0x7f54cc0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:08.417 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.411+0000 7f54e6374700 1 --2- 172.21.15.52:0/996056965 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f54cc0a1260 0x7f54cc0a3710 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f54c800b5c0 tx=0x7f54c8005fb0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.417 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.411+0000 7f54e6374700 1 -- 172.21.15.52:0/996056965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54e00ff1c0 msgr2=0x7f54e0103150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:08.417 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.411+0000 7f54e6374700 1 --2- 172.21.15.52:0/996056965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54e00ff1c0 0x7f54e0103150 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f54d000d980 tx=0x7f54d000dc90 comp rx=0 tx=0).stop 2023-11-27T15:17:08.417 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.412+0000 7f54e6374700 1 -- 172.21.15.52:0/996056965 shutdown_connections 2023-11-27T15:17:08.417 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.412+0000 7f54e6374700 1 --2- 172.21.15.52:0/996056965 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f54cc0a1260 0x7f54cc0a3710 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.412+0000 7f54e6374700 1 --2- 172.21.15.52:0/996056965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54e0100d40 0x7f54e0101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.412+0000 7f54e6374700 1 --2- 172.21.15.52:0/996056965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54e00ff1c0 0x7f54e0103150 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.412+0000 7f54e6374700 1 --2- 172.21.15.52:0/996056965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54e01003b0 0x7f54e0101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.412+0000 7f54e6374700 1 -- 172.21.15.52:0/996056965 >> 172.21.15.52:0/996056965 conn(0x7f54e00fa7a0 msgr2=0x7f54e0068eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:08.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.412+0000 7f54e6374700 1 -- 172.21.15.52:0/996056965 shutdown_connections 2023-11-27T15:17:08.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.412+0000 7f54e6374700 1 -- 172.21.15.52:0/996056965 wait complete. 2023-11-27T15:17:08.429 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-27T15:17:08.560 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.553+0000 7f372ee48700 1 -- 172.21.15.52:0/1932180753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37281003b0 msgr2=0x7f3728100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:08.560 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.553+0000 7f372ee48700 1 --2- 172.21.15.52:0/1932180753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37281003b0 0x7f3728100800 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3714009990 tx=0x7f3714009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.560 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.553+0000 7f372ee48700 1 -- 172.21.15.52:0/1932180753 shutdown_connections 2023-11-27T15:17:08.560 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.553+0000 7f372ee48700 1 --2- 172.21.15.52:0/1932180753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f37280ff1c0 0x7f37280ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.560 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.553+0000 7f372ee48700 1 --2- 172.21.15.52:0/1932180753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3728100d40 0x7f3728107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.553+0000 7f372ee48700 1 --2- 172.21.15.52:0/1932180753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37281003b0 0x7f3728100800 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.553+0000 7f372ee48700 1 -- 172.21.15.52:0/1932180753 >> 172.21.15.52:0/1932180753 conn(0x7f37280fa7c0 msgr2=0x7f37280fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:08.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.553+0000 7f372ee48700 1 -- 172.21.15.52:0/1932180753 shutdown_connections 2023-11-27T15:17:08.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.553+0000 7f372ee48700 1 -- 172.21.15.52:0/1932180753 wait complete. 2023-11-27T15:17:08.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.554+0000 7f372ee48700 1 Processor -- start 2023-11-27T15:17:08.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.554+0000 7f372ee48700 1 -- start start 2023-11-27T15:17:08.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.554+0000 7f372ee48700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37280ff1c0 0x7f3728103150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:08.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.554+0000 7f372ee48700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f37281003b0 0x7f3728101790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:08.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.554+0000 7f372ee48700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3728100d40 0x7f3728101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:08.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.554+0000 7f372ee48700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3728102470 con 0x7f3728100d40 2023-11-27T15:17:08.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.554+0000 7f372ee48700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37281025e0 con 0x7f37281003b0 2023-11-27T15:17:08.562 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.554+0000 7f372ee48700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37281a0160 con 0x7f37280ff1c0 2023-11-27T15:17:08.562 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.554+0000 7f372d3e5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3728100d40 0x7f3728101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:08.562 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.554+0000 7f372d3e5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3728100d40 0x7f3728101cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50788/0 (socket says 172.21.15.52:50788) 2023-11-27T15:17:08.562 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.554+0000 7f372d3e5700 1 -- 172.21.15.52:0/1632455815 learned_addr learned my addr 172.21.15.52:0/1632455815 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:08.562 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.554+0000 7f372cbe4700 1 --2- 172.21.15.52:0/1632455815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37280ff1c0 0x7f3728103150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:08.562 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.554+0000 7f3727fff700 1 --2- 172.21.15.52:0/1632455815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f37281003b0 0x7f3728101790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:08.562 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.554+0000 7f372d3e5700 1 -- 172.21.15.52:0/1632455815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37280ff1c0 msgr2=0x7f3728103150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:08.562 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.554+0000 7f372d3e5700 1 --2- 172.21.15.52:0/1632455815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37280ff1c0 0x7f3728103150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.562 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.554+0000 7f372d3e5700 1 -- 172.21.15.52:0/1632455815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f37281003b0 msgr2=0x7f3728101790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:08.562 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.554+0000 7f372d3e5700 1 --2- 172.21.15.52:0/1632455815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f37281003b0 0x7f3728101790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.562 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.554+0000 7f372d3e5700 1 -- 172.21.15.52:0/1632455815 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37140095f0 con 0x7f3728100d40 2023-11-27T15:17:08.563 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.554+0000 7f372cbe4700 1 --2- 172.21.15.52:0/1632455815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37280ff1c0 0x7f3728103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:17:08.563 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.554+0000 7f372d3e5700 1 --2- 172.21.15.52:0/1632455815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3728100d40 0x7f3728101cd0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3720007ee0 tx=0x7f372000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:08.563 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.555+0000 7f3725ffb700 1 -- 172.21.15.52:0/1632455815 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3720017070 con 0x7f3728100d40 2023-11-27T15:17:08.563 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.555+0000 7f372ee48700 1 -- 172.21.15.52:0/1632455815 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f372810ab70 con 0x7f3728100d40 2023-11-27T15:17:08.563 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.555+0000 7f3725ffb700 1 -- 172.21.15.52:0/1632455815 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f372000f040 con 0x7f3728100d40 2023-11-27T15:17:08.563 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.555+0000 7f3725ffb700 1 -- 172.21.15.52:0/1632455815 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3720005740 con 0x7f3728100d40 2023-11-27T15:17:08.563 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.555+0000 7f372ee48700 1 -- 172.21.15.52:0/1632455815 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f372810a9e0 con 0x7f3728100d40 2023-11-27T15:17:08.564 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.556+0000 7f372ee48700 1 -- 172.21.15.52:0/1632455815 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37281a0800 con 0x7f3728100d40 2023-11-27T15:17:08.564 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.557+0000 7f3725ffb700 1 -- 172.21.15.52:0/1632455815 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f37200058a0 con 0x7f3728100d40 2023-11-27T15:17:08.564 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.557+0000 7f3725ffb700 1 --2- 172.21.15.52:0/1632455815 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f37180a11e0 0x7f37180a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:08.564 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.558+0000 7f372cbe4700 1 --2- 172.21.15.52:0/1632455815 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f37180a11e0 0x7f37180a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:08.564 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.558+0000 7f3725ffb700 1 -- 172.21.15.52:0/1632455815 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f37200cc380 con 0x7f3728100d40 2023-11-27T15:17:08.564 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.558+0000 7f372cbe4700 1 --2- 172.21.15.52:0/1632455815 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f37180a11e0 0x7f37180a3690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3728100210 tx=0x7f371c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:08.567 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.562+0000 7f3725ffb700 1 -- 172.21.15.52:0/1632455815 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3720095950 con 0x7f3728100d40 2023-11-27T15:17:08.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.761+0000 7f372ee48700 1 -- 172.21.15.52:0/1632455815 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f37281a0db0 con 0x7f3728100d40 2023-11-27T15:17:08.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.762+0000 7f3725ffb700 1 -- 172.21.15.52:0/1632455815 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3720082e60 con 0x7f3728100d40 2023-11-27T15:17:08.767 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2023-11-27T15:17:08.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.764+0000 7f372ee48700 1 -- 172.21.15.52:0/1632455815 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f37180a11e0 msgr2=0x7f37180a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:08.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.764+0000 7f372ee48700 1 --2- 172.21.15.52:0/1632455815 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f37180a11e0 0x7f37180a3690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3728100210 tx=0x7f371c009260 comp rx=0 tx=0).stop 2023-11-27T15:17:08.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.764+0000 7f372ee48700 1 -- 172.21.15.52:0/1632455815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3728100d40 msgr2=0x7f3728101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:08.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.764+0000 7f372ee48700 1 --2- 172.21.15.52:0/1632455815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3728100d40 0x7f3728101cd0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3720007ee0 tx=0x7f372000d3b0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.764+0000 7f372ee48700 1 -- 172.21.15.52:0/1632455815 shutdown_connections 2023-11-27T15:17:08.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.764+0000 7f372ee48700 1 --2- 172.21.15.52:0/1632455815 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f37180a11e0 0x7f37180a3690 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.764+0000 7f372ee48700 1 --2- 172.21.15.52:0/1632455815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3728100d40 0x7f3728101cd0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.764+0000 7f372ee48700 1 --2- 172.21.15.52:0/1632455815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37280ff1c0 0x7f3728103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.764+0000 7f372ee48700 1 --2- 172.21.15.52:0/1632455815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f37281003b0 0x7f3728101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.764+0000 7f372ee48700 1 -- 172.21.15.52:0/1632455815 >> 172.21.15.52:0/1632455815 conn(0x7f37280fa7c0 msgr2=0x7f3728067dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:08.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.764+0000 7f372ee48700 1 -- 172.21.15.52:0/1632455815 shutdown_connections 2023-11-27T15:17:08.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.764+0000 7f372ee48700 1 -- 172.21.15.52:0/1632455815 wait complete. 2023-11-27T15:17:08.781 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:17:08.908 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.903+0000 7fe88d1ce700 1 -- 172.21.15.52:0/130262086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe888100400 msgr2=0x7fe888100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:08.909 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.903+0000 7fe88d1ce700 1 --2- 172.21.15.52:0/130262086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe888100400 0x7fe888100870 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe870009990 tx=0x7fe870009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.909 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.903+0000 7fe88d1ce700 1 -- 172.21.15.52:0/130262086 shutdown_connections 2023-11-27T15:17:08.909 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.903+0000 7fe88d1ce700 1 --2- 172.21.15.52:0/130262086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8880ff170 0x7fe8880ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.909 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.903+0000 7fe88d1ce700 1 --2- 172.21.15.52:0/130262086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe888100db0 0x7fe888107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.909 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.903+0000 7fe88d1ce700 1 --2- 172.21.15.52:0/130262086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe888100400 0x7fe888100870 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.909 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.903+0000 7fe88d1ce700 1 -- 172.21.15.52:0/130262086 >> 172.21.15.52:0/130262086 conn(0x7fe8880fa750 msgr2=0x7fe8880fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:08.909 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.903+0000 7fe88d1ce700 1 -- 172.21.15.52:0/130262086 shutdown_connections 2023-11-27T15:17:08.909 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.903+0000 7fe88d1ce700 1 -- 172.21.15.52:0/130262086 wait complete. 2023-11-27T15:17:08.911 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.904+0000 7fe88d1ce700 1 Processor -- start 2023-11-27T15:17:08.911 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.904+0000 7fe88d1ce700 1 -- start start 2023-11-27T15:17:08.911 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.904+0000 7fe88d1ce700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8880ff170 0x7fe888068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.904+0000 7fe88d1ce700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe888100400 0x7fe88819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.904+0000 7fe88d1ce700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe888100db0 0x7fe88819d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.904+0000 7fe88d1ce700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88819da20 con 0x7fe888100db0 2023-11-27T15:17:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.904+0000 7fe88d1ce700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88819db90 con 0x7fe8880ff170 2023-11-27T15:17:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.904+0000 7fe88d1ce700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88819dd50 con 0x7fe888100400 2023-11-27T15:17:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.904+0000 7fe88759e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe888100db0 0x7fe88819d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.904+0000 7fe88759e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe888100db0 0x7fe88819d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50802/0 (socket says 172.21.15.52:50802) 2023-11-27T15:17:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.904+0000 7fe88759e700 1 -- 172.21.15.52:0/3519635373 learned_addr learned my addr 172.21.15.52:0/3519635373 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.904+0000 7fe88659c700 1 --2- 172.21.15.52:0/3519635373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe888100400 0x7fe88819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.904+0000 7fe886d9d700 1 --2- 172.21.15.52:0/3519635373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8880ff170 0x7fe888068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.905+0000 7fe88759e700 1 -- 172.21.15.52:0/3519635373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe888100400 msgr2=0x7fe88819ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.905+0000 7fe88759e700 1 --2- 172.21.15.52:0/3519635373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe888100400 0x7fe88819ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.913 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.905+0000 7fe88759e700 1 -- 172.21.15.52:0/3519635373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8880ff170 msgr2=0x7fe888068960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:08.913 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.905+0000 7fe88759e700 1 --2- 172.21.15.52:0/3519635373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8880ff170 0x7fe888068960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:08.913 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.905+0000 7fe88759e700 1 -- 172.21.15.52:0/3519635373 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8700095f0 con 0x7fe888100db0 2023-11-27T15:17:08.913 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.905+0000 7fe88759e700 1 --2- 172.21.15.52:0/3519635373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe888100db0 0x7fe88819d250 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe880007ee0 tx=0x7fe88000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:08.913 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.905+0000 7fe87ffff700 1 -- 172.21.15.52:0/3519635373 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe880017070 con 0x7fe888100db0 2023-11-27T15:17:08.913 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.905+0000 7fe87ffff700 1 -- 172.21.15.52:0/3519635373 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe88000f040 con 0x7fe888100db0 2023-11-27T15:17:08.913 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.905+0000 7fe88d1ce700 1 -- 172.21.15.52:0/3519635373 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe88810ab10 con 0x7fe888100db0 2023-11-27T15:17:08.913 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.905+0000 7fe87ffff700 1 -- 172.21.15.52:0/3519635373 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe880005580 con 0x7fe888100db0 2023-11-27T15:17:08.913 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.905+0000 7fe88d1ce700 1 -- 172.21.15.52:0/3519635373 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe88810a980 con 0x7fe888100db0 2023-11-27T15:17:08.913 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.906+0000 7fe88d1ce700 1 -- 172.21.15.52:0/3519635373 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe88819e330 con 0x7fe888100db0 2023-11-27T15:17:08.914 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.907+0000 7fe87ffff700 1 -- 172.21.15.52:0/3519635373 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fe88000db20 con 0x7fe888100db0 2023-11-27T15:17:08.914 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.908+0000 7fe87ffff700 1 --2- 172.21.15.52:0/3519635373 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fe8740a11e0 0x7fe8740a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:08.914 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.908+0000 7fe886d9d700 1 --2- 172.21.15.52:0/3519635373 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fe8740a11e0 0x7fe8740a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:08.914 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.908+0000 7fe87ffff700 1 -- 172.21.15.52:0/3519635373 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe8800cc800 con 0x7fe888100db0 2023-11-27T15:17:08.914 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.908+0000 7fe886d9d700 1 --2- 172.21.15.52:0/3519635373 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fe8740a11e0 0x7fe8740a3690 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe87800ac50 tx=0x7fe87800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:08.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:08.911+0000 7fe87ffff700 1 -- 172.21.15.52:0/3519635373 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe880095dd0 con 0x7fe888100db0 2023-11-27T15:17:09.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:08 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3511366180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:17:09.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:08 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/996056965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:17:09.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:08 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1632455815' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:17:09.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.161+0000 7fe88d1ce700 1 -- 172.21.15.52:0/3519635373 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe888106110 con 0x7fe888100db0 2023-11-27T15:17:09.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.161+0000 7fe87ffff700 1 -- 172.21.15.52:0/3519635373 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10804 (secure 0 0 0) 0x7fe8800993f0 con 0x7fe888100db0 2023-11-27T15:17:09.167 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:17:09.167 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2328881376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2328881376},{"type":"v1","addr":"172.21.15.52:6839","nonce":2328881376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2799551570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2799551570},{"type":"v1","addr":"172.21.15.52:6841","nonce":2799551570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:16:09.319630+0000","modified":"2023-11-27T15:16:14.742405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24494,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/215725838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":215725838},{"type":"v1","addr":"172.21.15.150:6839","nonce":215725838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3571830559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3571830559},{"type":"v1","addr":"172.21.15.102:6839","nonce":3571830559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:17:09.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.163+0000 7fe88d1ce700 1 -- 172.21.15.52:0/3519635373 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fe8740a11e0 msgr2=0x7fe8740a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:09.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.163+0000 7fe88d1ce700 1 --2- 172.21.15.52:0/3519635373 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fe8740a11e0 0x7fe8740a3690 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe87800ac50 tx=0x7fe87800a380 comp rx=0 tx=0).stop 2023-11-27T15:17:09.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.163+0000 7fe88d1ce700 1 -- 172.21.15.52:0/3519635373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe888100db0 msgr2=0x7fe88819d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:09.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.163+0000 7fe88d1ce700 1 --2- 172.21.15.52:0/3519635373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe888100db0 0x7fe88819d250 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe880007ee0 tx=0x7fe88000d3b0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.163+0000 7fe88d1ce700 1 -- 172.21.15.52:0/3519635373 shutdown_connections 2023-11-27T15:17:09.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.163+0000 7fe88d1ce700 1 --2- 172.21.15.52:0/3519635373 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fe8740a11e0 0x7fe8740a3690 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.163+0000 7fe88d1ce700 1 --2- 172.21.15.52:0/3519635373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8880ff170 0x7fe888068960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.163+0000 7fe88d1ce700 1 --2- 172.21.15.52:0/3519635373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe888100db0 0x7fe88819d250 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.163+0000 7fe88d1ce700 1 --2- 172.21.15.52:0/3519635373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe888100400 0x7fe88819ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.163+0000 7fe88d1ce700 1 -- 172.21.15.52:0/3519635373 >> 172.21.15.52:0/3519635373 conn(0x7fe8880fa750 msgr2=0x7fe8880fc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:09.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.163+0000 7fe88d1ce700 1 -- 172.21.15.52:0/3519635373 shutdown_connections 2023-11-27T15:17:09.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.163+0000 7fe88d1ce700 1 -- 172.21.15.52:0/3519635373 wait complete. 2023-11-27T15:17:09.170 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 23 2023-11-27T15:17:09.181 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:17:09.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:08 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3511366180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:17:09.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:08 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/996056965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:17:09.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:08 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1632455815' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:17:09.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.302+0000 7efd3f441700 1 -- 172.21.15.52:0/4250211423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd38103380 msgr2=0x7efd381057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:09.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.302+0000 7efd3f441700 1 --2- 172.21.15.52:0/4250211423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd38103380 0x7efd381057c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efd28009990 tx=0x7efd28009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.302+0000 7efd3f441700 1 -- 172.21.15.52:0/4250211423 shutdown_connections 2023-11-27T15:17:09.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.302+0000 7efd3f441700 1 --2- 172.21.15.52:0/4250211423 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd38105e50 0x7efd38108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.302+0000 7efd3f441700 1 --2- 172.21.15.52:0/4250211423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd38068ad0 0x7efd38102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.302+0000 7efd3f441700 1 --2- 172.21.15.52:0/4250211423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd38103380 0x7efd381057c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.302+0000 7efd3f441700 1 -- 172.21.15.52:0/4250211423 >> 172.21.15.52:0/4250211423 conn(0x7efd380fa750 msgr2=0x7efd380fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:09.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.302+0000 7efd3f441700 1 -- 172.21.15.52:0/4250211423 shutdown_connections 2023-11-27T15:17:09.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.302+0000 7efd3f441700 1 -- 172.21.15.52:0/4250211423 wait complete. 2023-11-27T15:17:09.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.303+0000 7efd3f441700 1 Processor -- start 2023-11-27T15:17:09.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.303+0000 7efd3f441700 1 -- start start 2023-11-27T15:17:09.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.303+0000 7efd3f441700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd38068ad0 0x7efd38195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:09.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.303+0000 7efd3f441700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd38103380 0x7efd381964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:09.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.303+0000 7efd3f441700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd38105e50 0x7efd3819d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:09.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.303+0000 7efd3f441700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3819daf0 con 0x7efd38105e50 2023-11-27T15:17:09.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.303+0000 7efd3f441700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3819dc30 con 0x7efd38068ad0 2023-11-27T15:17:09.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.303+0000 7efd3f441700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3819ddc0 con 0x7efd38103380 2023-11-27T15:17:09.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.303+0000 7efd3d9de700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd38105e50 0x7efd3819d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:09.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.303+0000 7efd3d9de700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd38105e50 0x7efd3819d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50814/0 (socket says 172.21.15.52:50814) 2023-11-27T15:17:09.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.303+0000 7efd3d9de700 1 -- 172.21.15.52:0/3399983092 learned_addr learned my addr 172.21.15.52:0/3399983092 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:09.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.303+0000 7efd3d1dd700 1 --2- 172.21.15.52:0/3399983092 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd38068ad0 0x7efd38195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:09.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.303+0000 7efd3c9dc700 1 --2- 172.21.15.52:0/3399983092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd38103380 0x7efd381964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:09.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.304+0000 7efd3d9de700 1 -- 172.21.15.52:0/3399983092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd38103380 msgr2=0x7efd381964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:09.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.304+0000 7efd3d9de700 1 --2- 172.21.15.52:0/3399983092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd38103380 0x7efd381964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.304+0000 7efd3d9de700 1 -- 172.21.15.52:0/3399983092 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd38068ad0 msgr2=0x7efd38195f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:09.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.304+0000 7efd3d9de700 1 --2- 172.21.15.52:0/3399983092 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd38068ad0 0x7efd38195f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.304+0000 7efd3d9de700 1 -- 172.21.15.52:0/3399983092 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd280095f0 con 0x7efd38105e50 2023-11-27T15:17:09.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.304+0000 7efd3d9de700 1 --2- 172.21.15.52:0/3399983092 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd38105e50 0x7efd3819d520 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7efd3000ef30 tx=0x7efd3000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:09.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.304+0000 7efd2e7fc700 1 -- 172.21.15.52:0/3399983092 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd30018070 con 0x7efd38105e50 2023-11-27T15:17:09.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.304+0000 7efd2e7fc700 1 -- 172.21.15.52:0/3399983092 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd3000f040 con 0x7efd38105e50 2023-11-27T15:17:09.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.304+0000 7efd3f441700 1 -- 172.21.15.52:0/3399983092 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd3810bc10 con 0x7efd38105e50 2023-11-27T15:17:09.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.304+0000 7efd2e7fc700 1 -- 172.21.15.52:0/3399983092 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3000cd00 con 0x7efd38105e50 2023-11-27T15:17:09.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.304+0000 7efd3f441700 1 -- 172.21.15.52:0/3399983092 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd3810ba80 con 0x7efd38105e50 2023-11-27T15:17:09.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.305+0000 7efd3f441700 1 -- 172.21.15.52:0/3399983092 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd380fc2c0 con 0x7efd38105e50 2023-11-27T15:17:09.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.306+0000 7efd2e7fc700 1 -- 172.21.15.52:0/3399983092 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7efd30003a00 con 0x7efd38105e50 2023-11-27T15:17:09.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.307+0000 7efd2e7fc700 1 --2- 172.21.15.52:0/3399983092 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7efd240a1260 0x7efd240a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:09.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.307+0000 7efd3d1dd700 1 --2- 172.21.15.52:0/3399983092 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7efd240a1260 0x7efd240a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:09.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.307+0000 7efd2e7fc700 1 -- 172.21.15.52:0/3399983092 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7efd300ccf20 con 0x7efd38105e50 2023-11-27T15:17:09.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.307+0000 7efd3d1dd700 1 --2- 172.21.15.52:0/3399983092 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7efd240a1260 0x7efd240a3710 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efd380687f0 tx=0x7efd34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:09.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.310+0000 7efd2e7fc700 1 -- 172.21.15.52:0/3399983092 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7efd30096520 con 0x7efd38105e50 2023-11-27T15:17:09.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:08 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3511366180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:17:09.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:08 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/996056965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:17:09.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:08 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1632455815' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:17:09.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.510+0000 7efd3f441700 1 -- 172.21.15.52:0/3399983092 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd3802cfe0 con 0x7efd38105e50 2023-11-27T15:17:09.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.512+0000 7efd2e7fc700 1 -- 172.21.15.52:0/3399983092 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22657 (secure 0 0 0) 0x7efd30099b40 con 0x7efd38105e50 2023-11-27T15:17:09.517 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:17:09.517 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":85,"fsid":"c6c7df88-8d36-11ee-95a2-87774f69a715","created":"2023-11-27T15:09:03.333345+0000","modified":"2023-11-27T15:17:04.797892+0000","last_up_change":"2023-11-27T15:15:27.656134+0000","last_in_change":"2023-11-27T15:15:10.671153+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:12:14.177523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:16:07.842251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:16:08.728520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58d69ede-c293-49f5-abbe-8187793f2b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6803","nonce":608216415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6805","nonce":608216415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6809","nonce":608216415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6807","nonce":608216415}]},"public_addr":"172.21.15.52:6803/608216415","cluster_addr":"172.21.15.52:6805/608216415","heartbeat_back_addr":"172.21.15.52:6809/608216415","heartbeat_front_addr":"172.21.15.52:6807/608216415","state":["exists","up"]},{"osd":1,"uuid":"d6a46b91-adc8-45ef-80f5-566ae400911f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6801","nonce":2538384087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6803","nonce":2538384087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6807","nonce":2538384087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6805","nonce":2538384087}]},"public_addr":"172.21.15.102:6801/2538384087","cluster_addr":"172.21.15.102:6803/2538384087","heartbeat_back_addr":"172.21.15.102:6807/2538384087","heartbeat_front_addr":"172.21.15.102:6805/2538384087","state":["exists","up"]},{"osd":2,"uuid":"846ba125-11f8-46b2-8941-bd5e1947cc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6801","nonce":4258038221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6803","nonce":4258038221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6807","nonce":4258038221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6805","nonce":4258038221}]},"public_addr":"172.21.15.150:6801/4258038221","cluster_addr":"172.21.15.150:6803/4258038221","heartbeat_back_addr":"172.21.15.150:6807/4258038221","heartbeat_front_addr":"172.21.15.150:6805/4258038221","state":["exists","up"]},{"osd":3,"uuid":"950fcd67-1057-4288-a389-714f1ec94261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6811","nonce":2622100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6813","nonce":2622100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6817","nonce":2622100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6815","nonce":2622100752}]},"public_addr":"172.21.15.52:6811/2622100752","cluster_addr":"172.21.15.52:6813/2622100752","heartbeat_back_addr":"172.21.15.52:6817/2622100752","heartbeat_front_addr":"172.21.15.52:6815/2622100752","state":["exists","up"]},{"osd":4,"uuid":"1c96a676-cea9-4deb-bad1-f7fd240c3d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6809","nonce":1546794763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6811","nonce":1546794763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6815","nonce":1546794763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6813","nonce":1546794763}]},"public_addr":"172.21.15.102:6809/1546794763","cluster_addr":"172.21.15.102:6811/1546794763","heartbeat_back_addr":"172.21.15.102:6815/1546794763","heartbeat_front_addr":"172.21.15.102:6813/1546794763","state":["exists","up"]},{"osd":5,"uuid":"38a75af1-fd4d-433e-b579-0609109c851c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6809","nonce":461511375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6811","nonce":461511375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6815","nonce":461511375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6813","nonce":461511375}]},"public_addr":"172.21.15.150:6809/461511375","cluster_addr":"172.21.15.150:6811/461511375","heartbeat_back_addr":"172.21.15.150:6815/461511375","heartbeat_front_addr":"172.21.15.150:6813/461511375","state":["exists","up"]},{"osd":6,"uuid":"2f0ede4a-d175-4b4a-b876-c0f442048901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6819","nonce":1002457464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6821","nonce":1002457464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6825","nonce":1002457464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6823","nonce":1002457464}]},"public_addr":"172.21.15.52:6819/1002457464","cluster_addr":"172.21.15.52:6821/1002457464","heartbeat_back_addr":"172.21.15.52:6825/1002457464","heartbeat_front_addr":"172.21.15.52:6823/1002457464","state":["exists","up"]},{"osd":7,"uuid":"6b635192-abcf-4bbf-bf78-a019b351bca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6817","nonce":3249840064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6819","nonce":3249840064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6823","nonce":3249840064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6821","nonce":3249840064}]},"public_addr":"172.21.15.102:6817/3249840064","cluster_addr":"172.21.15.102:6819/3249840064","heartbeat_back_addr":"172.21.15.102:6823/3249840064","heartbeat_front_addr":"172.21.15.102:6821/3249840064","state":["exists","up"]},{"osd":8,"uuid":"c7ca8007-7170-47bf-956c-3f00edf27e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6817","nonce":3672583574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6819","nonce":3672583574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6823","nonce":3672583574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6821","nonce":3672583574}]},"public_addr":"172.21.15.150:6817/3672583574","cluster_addr":"172.21.15.150:6819/3672583574","heartbeat_back_addr":"172.21.15.150:6823/3672583574","heartbeat_front_addr":"172.21.15.150:6821/3672583574","state":["exists","up"]},{"osd":9,"uuid":"0e252c2c-01a4-48ba-832d-067ed940fefb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6827","nonce":4287934320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6829","nonce":4287934320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6833","nonce":4287934320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6831","nonce":4287934320}]},"public_addr":"172.21.15.52:6827/4287934320","cluster_addr":"172.21.15.52:6829/4287934320","heartbeat_back_addr":"172.21.15.52:6833/4287934320","heartbeat_front_addr":"172.21.15.52:6831/4287934320","state":["exists","up"]},{"osd":10,"uuid":"bb540142-7563-44a4-b7a3-cb662714c128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6825","nonce":3056034201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6827","nonce":3056034201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6831","nonce":3056034201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6829","nonce":3056034201}]},"public_addr":"172.21.15.102:6825/3056034201","cluster_addr":"172.21.15.102:6827/3056034201","heartbeat_back_addr":"172.21.15.102:6831/3056034201","heartbeat_front_addr":"172.21.15.102:6829/3056034201","state":["exists","up"]},{"osd":11,"uuid":"3156528e-e0f8-4d44-9946-c8b6339c2bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6825","nonce":3207559739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6827","nonce":3207559739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6831","nonce":3207559739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6829","nonce":3207559739}]},"public_addr":"172.21.15.150:6825/3207559739","cluster_addr":"172.21.15.150:6827/3207559739","heartbeat_back_addr":"172.21.15.150:6831/3207559739","heartbeat_front_addr":"172.21.15.150:6829/3207559739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:26.826099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:48.331371+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:10.459738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:32.356472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:54.043170+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:15.648582+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:37.166463+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:59.240819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:20.710097+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:43.364563+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:05.279433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:26.344573+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":5,"to":7}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/1392282110":"2023-11-28T15:10:03.644870+0000","172.21.15.52:0/1479993479":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6800/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/961562652":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/1290665390":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2410529742":"2023-11-28T15:09:37.118690+0000","172.21.15.52:6800/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2130037044":"2023-11-28T15:10:03.644870+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:17:09.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.514+0000 7efd3f441700 1 -- 172.21.15.52:0/3399983092 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7efd240a1260 msgr2=0x7efd240a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:09.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.514+0000 7efd3f441700 1 --2- 172.21.15.52:0/3399983092 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7efd240a1260 0x7efd240a3710 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efd380687f0 tx=0x7efd34009260 comp rx=0 tx=0).stop 2023-11-27T15:17:09.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.514+0000 7efd3f441700 1 -- 172.21.15.52:0/3399983092 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd38105e50 msgr2=0x7efd3819d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:09.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.514+0000 7efd3f441700 1 --2- 172.21.15.52:0/3399983092 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd38105e50 0x7efd3819d520 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7efd3000ef30 tx=0x7efd3000c3a0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.514+0000 7efd3f441700 1 -- 172.21.15.52:0/3399983092 shutdown_connections 2023-11-27T15:17:09.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.514+0000 7efd3f441700 1 --2- 172.21.15.52:0/3399983092 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7efd240a1260 0x7efd240a3710 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.514+0000 7efd3f441700 1 --2- 172.21.15.52:0/3399983092 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd38068ad0 0x7efd38195f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.514+0000 7efd3f441700 1 --2- 172.21.15.52:0/3399983092 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd38105e50 0x7efd3819d520 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.514+0000 7efd3f441700 1 --2- 172.21.15.52:0/3399983092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd38103380 0x7efd381964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.514+0000 7efd3f441700 1 -- 172.21.15.52:0/3399983092 >> 172.21.15.52:0/3399983092 conn(0x7efd380fa750 msgr2=0x7efd381016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:09.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.514+0000 7efd3f441700 1 -- 172.21.15.52:0/3399983092 shutdown_connections 2023-11-27T15:17:09.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.514+0000 7efd3f441700 1 -- 172.21.15.52:0/3399983092 wait complete. 2023-11-27T15:17:09.532 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:17:09.658 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.652+0000 7ff85d2b5700 1 -- 172.21.15.52:0/2091721315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8580ff440 msgr2=0x7ff8580ff8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:09.658 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.652+0000 7ff85d2b5700 1 --2- 172.21.15.52:0/2091721315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8580ff440 0x7ff8580ff8b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff84c009990 tx=0x7ff84c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.658 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.652+0000 7ff85d2b5700 1 -- 172.21.15.52:0/2091721315 shutdown_connections 2023-11-27T15:17:09.658 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.652+0000 7ff85d2b5700 1 --2- 172.21.15.52:0/2091721315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8581007f0 0x7ff858100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.658 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.652+0000 7ff85d2b5700 1 --2- 172.21.15.52:0/2091721315 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8580ffdf0 0x7ff8581076a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.658 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.652+0000 7ff85d2b5700 1 --2- 172.21.15.52:0/2091721315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8580ff440 0x7ff8580ff8b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.658 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.652+0000 7ff85d2b5700 1 -- 172.21.15.52:0/2091721315 >> 172.21.15.52:0/2091721315 conn(0x7ff8580f9fa0 msgr2=0x7ff8580fc410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:09.659 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.652+0000 7ff85d2b5700 1 -- 172.21.15.52:0/2091721315 shutdown_connections 2023-11-27T15:17:09.659 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.652+0000 7ff85d2b5700 1 -- 172.21.15.52:0/2091721315 wait complete. 2023-11-27T15:17:09.659 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.653+0000 7ff85d2b5700 1 Processor -- start 2023-11-27T15:17:09.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.653+0000 7ff85d2b5700 1 -- start start 2023-11-27T15:17:09.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.653+0000 7ff85d2b5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8580ff440 0x7ff8581031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:09.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.653+0000 7ff85d2b5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8580ffdf0 0x7ff858101830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:09.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.653+0000 7ff85d2b5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8581007f0 0x7ff858101d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:09.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.653+0000 7ff85d2b5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff858102510 con 0x7ff8581007f0 2023-11-27T15:17:09.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.653+0000 7ff85d2b5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff858102680 con 0x7ff8580ff440 2023-11-27T15:17:09.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.653+0000 7ff85d2b5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85819ff90 con 0x7ff8580ffdf0 2023-11-27T15:17:09.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.653+0000 7ff8577fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8581007f0 0x7ff858101d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:09.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.654+0000 7ff8577fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8581007f0 0x7ff858101d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50830/0 (socket says 172.21.15.52:50830) 2023-11-27T15:17:09.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.654+0000 7ff8577fe700 1 -- 172.21.15.52:0/2900609164 learned_addr learned my addr 172.21.15.52:0/2900609164 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:09.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.654+0000 7ff8567fc700 1 --2- 172.21.15.52:0/2900609164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8580ffdf0 0x7ff858101830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:09.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.654+0000 7ff856ffd700 1 --2- 172.21.15.52:0/2900609164 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8580ff440 0x7ff8581031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:09.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.654+0000 7ff8577fe700 1 -- 172.21.15.52:0/2900609164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8580ffdf0 msgr2=0x7ff858101830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:09.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.654+0000 7ff8577fe700 1 --2- 172.21.15.52:0/2900609164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8580ffdf0 0x7ff858101830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.654+0000 7ff8577fe700 1 -- 172.21.15.52:0/2900609164 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8580ff440 msgr2=0x7ff8581031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:09.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.654+0000 7ff8577fe700 1 --2- 172.21.15.52:0/2900609164 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8580ff440 0x7ff8581031f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.654+0000 7ff8577fe700 1 -- 172.21.15.52:0/2900609164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff84c0095f0 con 0x7ff8581007f0 2023-11-27T15:17:09.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.654+0000 7ff8577fe700 1 --2- 172.21.15.52:0/2900609164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8581007f0 0x7ff858101d70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff85000ef30 tx=0x7ff85000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:09.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.654+0000 7ff83ffff700 1 -- 172.21.15.52:0/2900609164 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff850018070 con 0x7ff8581007f0 2023-11-27T15:17:09.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.654+0000 7ff83ffff700 1 -- 172.21.15.52:0/2900609164 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff85000f040 con 0x7ff8581007f0 2023-11-27T15:17:09.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.654+0000 7ff85d2b5700 1 -- 172.21.15.52:0/2900609164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff85810ab20 con 0x7ff8581007f0 2023-11-27T15:17:09.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.654+0000 7ff83ffff700 1 -- 172.21.15.52:0/2900609164 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85000cd00 con 0x7ff8581007f0 2023-11-27T15:17:09.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.654+0000 7ff85d2b5700 1 -- 172.21.15.52:0/2900609164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff85810a990 con 0x7ff8581007f0 2023-11-27T15:17:09.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.655+0000 7ff85d2b5700 1 -- 172.21.15.52:0/2900609164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff858067eb0 con 0x7ff8581007f0 2023-11-27T15:17:09.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.656+0000 7ff83ffff700 1 -- 172.21.15.52:0/2900609164 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff850003a00 con 0x7ff8581007f0 2023-11-27T15:17:09.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.657+0000 7ff83ffff700 1 --2- 172.21.15.52:0/2900609164 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff8400a11e0 0x7ff8400a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:09.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.657+0000 7ff83ffff700 1 -- 172.21.15.52:0/2900609164 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff8500ccfc0 con 0x7ff8581007f0 2023-11-27T15:17:09.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.657+0000 7ff856ffd700 1 --2- 172.21.15.52:0/2900609164 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff8400a11e0 0x7ff8400a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:09.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.657+0000 7ff856ffd700 1 --2- 172.21.15.52:0/2900609164 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff8400a11e0 0x7ff8400a3690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff848009c60 tx=0x7ff848009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:09.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.660+0000 7ff83ffff700 1 -- 172.21.15.52:0/2900609164 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff850096620 con 0x7ff8581007f0 2023-11-27T15:17:09.913 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.907+0000 7ff85d2b5700 1 -- 172.21.15.52:0/2900609164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8581a09f0 con 0x7ff8581007f0 2023-11-27T15:17:09.913 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.908+0000 7ff83ffff700 1 -- 172.21.15.52:0/2900609164 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10804 (secure 0 0 0) 0x7ff850099c40 con 0x7ff8581007f0 2023-11-27T15:17:09.913 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:17:09.914 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2328881376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2328881376},{"type":"v1","addr":"172.21.15.52:6839","nonce":2328881376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2799551570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2799551570},{"type":"v1","addr":"172.21.15.52:6841","nonce":2799551570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:16:09.319630+0000","modified":"2023-11-27T15:16:14.742405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24494,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/215725838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":215725838},{"type":"v1","addr":"172.21.15.150:6839","nonce":215725838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3571830559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3571830559},{"type":"v1","addr":"172.21.15.102:6839","nonce":3571830559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:17:09.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.910+0000 7ff85d2b5700 1 -- 172.21.15.52:0/2900609164 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff8400a11e0 msgr2=0x7ff8400a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:09.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.910+0000 7ff85d2b5700 1 --2- 172.21.15.52:0/2900609164 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff8400a11e0 0x7ff8400a3690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff848009c60 tx=0x7ff848009260 comp rx=0 tx=0).stop 2023-11-27T15:17:09.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.910+0000 7ff85d2b5700 1 -- 172.21.15.52:0/2900609164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8581007f0 msgr2=0x7ff858101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:09.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.910+0000 7ff85d2b5700 1 --2- 172.21.15.52:0/2900609164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8581007f0 0x7ff858101d70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff85000ef30 tx=0x7ff85000c3a0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.910+0000 7ff85d2b5700 1 -- 172.21.15.52:0/2900609164 shutdown_connections 2023-11-27T15:17:09.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.910+0000 7ff85d2b5700 1 --2- 172.21.15.52:0/2900609164 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff8400a11e0 0x7ff8400a3690 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.910+0000 7ff85d2b5700 1 --2- 172.21.15.52:0/2900609164 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8580ff440 0x7ff8581031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.910+0000 7ff85d2b5700 1 --2- 172.21.15.52:0/2900609164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8581007f0 0x7ff858101d70 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.910+0000 7ff85d2b5700 1 --2- 172.21.15.52:0/2900609164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8580ffdf0 0x7ff858101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.910+0000 7ff85d2b5700 1 -- 172.21.15.52:0/2900609164 >> 172.21.15.52:0/2900609164 conn(0x7ff8580f9fa0 msgr2=0x7ff8580fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:09.917 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.910+0000 7ff85d2b5700 1 -- 172.21.15.52:0/2900609164 shutdown_connections 2023-11-27T15:17:09.917 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:09.910+0000 7ff85d2b5700 1 -- 172.21.15.52:0/2900609164 wait complete. 2023-11-27T15:17:09.917 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 23 2023-11-27T15:17:09.928 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-11-27T15:16:09.319630+0000', 'modified': '2023-11-27T15:16:14.742405+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24536, 'mds_1': 24494, 'mds_2': 24472}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.150:6835/617413375', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6834', 'nonce': 617413375}, {'type': 'v1', 'addr': '172.21.15.150:6835', 'nonce': 617413375}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24472': {'gid': 24472, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.150:6837/805360541', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6836', 'nonce': 805360541}, {'type': 'v1', 'addr': '172.21.15.150:6837', 'nonce': 805360541}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24473': {'gid': 24473, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.102:6835/2398692013', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6834', 'nonce': 2398692013}, {'type': 'v1', 'addr': '172.21.15.102:6835', 'nonce': 2398692013}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.150:6839/215725838', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6838', 'nonce': 215725838}, {'type': 'v1', 'addr': '172.21.15.150:6839', 'nonce': 215725838}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24494': {'gid': 24494, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.102:6837/2141793919', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6836', 'nonce': 2141793919}, {'type': 'v1', 'addr': '172.21.15.102:6837', 'nonce': 2141793919}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24536': {'gid': 24536, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.102:6839/3571830559', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6838', 'nonce': 3571830559}, {'type': 'v1', 'addr': '172.21.15.102:6839', 'nonce': 3571830559}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-27T15:17:09.928 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-11-27T15:17:09.928 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-27T15:17:10.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.050+0000 7f909d46c700 1 -- 172.21.15.52:0/1644867738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9098100340 msgr2=0x7f9098100790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:10.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.050+0000 7f909d46c700 1 --2- 172.21.15.52:0/1644867738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9098100340 0x7f9098100790 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9088009990 tx=0x7f9088009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.050+0000 7f909d46c700 1 -- 172.21.15.52:0/1644867738 shutdown_connections 2023-11-27T15:17:10.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.050+0000 7f909d46c700 1 --2- 172.21.15.52:0/1644867738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90980ff150 0x7f90980ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.050+0000 7f909d46c700 1 --2- 172.21.15.52:0/1644867738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9098100cd0 0x7f9098107590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.050+0000 7f909d46c700 1 --2- 172.21.15.52:0/1644867738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9098100340 0x7f9098100790 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.050+0000 7f909d46c700 1 -- 172.21.15.52:0/1644867738 >> 172.21.15.52:0/1644867738 conn(0x7f90980fa750 msgr2=0x7f90980fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:10.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.050+0000 7f909d46c700 1 -- 172.21.15.52:0/1644867738 shutdown_connections 2023-11-27T15:17:10.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.050+0000 7f909d46c700 1 -- 172.21.15.52:0/1644867738 wait complete. 2023-11-27T15:17:10.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.050+0000 7f909d46c700 1 Processor -- start 2023-11-27T15:17:10.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.050+0000 7f909d46c700 1 -- start start 2023-11-27T15:17:10.058 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.051+0000 7f909d46c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90980ff150 0x7f9098198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.058 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.051+0000 7f909d46c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9098100340 0x7f9098198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.058 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.051+0000 7f909d46c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9098100cd0 0x7f909819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.058 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.051+0000 7f909d46c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909819fc90 con 0x7f9098100340 2023-11-27T15:17:10.058 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.051+0000 7f909d46c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909819fdd0 con 0x7f9098100cd0 2023-11-27T15:17:10.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.051+0000 7f909d46c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909819ff60 con 0x7f90980ff150 2023-11-27T15:17:10.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.051+0000 7f90967fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9098100340 0x7f9098198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.051+0000 7f90967fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9098100340 0x7f9098198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50842/0 (socket says 172.21.15.52:50842) 2023-11-27T15:17:10.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.051+0000 7f90967fc700 1 -- 172.21.15.52:0/3400867382 learned_addr learned my addr 172.21.15.52:0/3400867382 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:10.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.051+0000 7f9096ffd700 1 --2- 172.21.15.52:0/3400867382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90980ff150 0x7f9098198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.051+0000 7f90977fe700 1 --2- 172.21.15.52:0/3400867382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9098100cd0 0x7f909819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.051+0000 7f90967fc700 1 -- 172.21.15.52:0/3400867382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90980ff150 msgr2=0x7f9098198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:10.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.051+0000 7f90967fc700 1 --2- 172.21.15.52:0/3400867382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90980ff150 0x7f9098198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.051+0000 7f90967fc700 1 -- 172.21.15.52:0/3400867382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9098100cd0 msgr2=0x7f909819f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:10.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.051+0000 7f90967fc700 1 --2- 172.21.15.52:0/3400867382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9098100cd0 0x7f909819f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.051+0000 7f90967fc700 1 -- 172.21.15.52:0/3400867382 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90880095f0 con 0x7f9098100340 2023-11-27T15:17:10.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.051+0000 7f90977fe700 1 --2- 172.21.15.52:0/3400867382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9098100cd0 0x7f909819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:17:10.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.051+0000 7f9096ffd700 1 --2- 172.21.15.52:0/3400867382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90980ff150 0x7f9098198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:17:10.060 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.051+0000 7f90967fc700 1 --2- 172.21.15.52:0/3400867382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9098100340 0x7f9098198650 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9088005770 tx=0x7f9088005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:10.060 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.051+0000 7f907ffff700 1 -- 172.21.15.52:0/3400867382 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908801d070 con 0x7f9098100340 2023-11-27T15:17:10.060 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.051+0000 7f909d46c700 1 -- 172.21.15.52:0/3400867382 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f909810aab0 con 0x7f9098100340 2023-11-27T15:17:10.060 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.051+0000 7f907ffff700 1 -- 172.21.15.52:0/3400867382 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90880040e0 con 0x7f9098100340 2023-11-27T15:17:10.060 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.051+0000 7f907ffff700 1 -- 172.21.15.52:0/3400867382 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908800f460 con 0x7f9098100340 2023-11-27T15:17:10.060 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.052+0000 7f909d46c700 1 -- 172.21.15.52:0/3400867382 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f909810a920 con 0x7f9098100340 2023-11-27T15:17:10.062 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.054+0000 7f907ffff700 1 -- 172.21.15.52:0/3400867382 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9088022600 con 0x7f9098100340 2023-11-27T15:17:10.063 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.055+0000 7f907ffff700 1 --2- 172.21.15.52:0/3400867382 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f90800a11e0 0x7f90800a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.063 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.055+0000 7f9096ffd700 1 --2- 172.21.15.52:0/3400867382 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f90800a11e0 0x7f90800a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.063 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.055+0000 7f907ffff700 1 -- 172.21.15.52:0/3400867382 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f90880cdff0 con 0x7f9098100340 2023-11-27T15:17:10.063 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.055+0000 7f9096ffd700 1 --2- 172.21.15.52:0/3400867382 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f90800a11e0 0x7f90800a3690 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f908c005950 tx=0x7f908c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:10.063 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.056+0000 7f90a95f8b80 1 Processor -- start 2023-11-27T15:17:10.063 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.056+0000 7f90a95f8b80 1 -- start start 2023-11-27T15:17:10.063 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.056+0000 7f90a95f8b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565cfcc5bb0 0x5565cfcc5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.063 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.056+0000 7f90a95f8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfcc6500 0x5565cfcc6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.063 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.056+0000 7f90a95f8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565cfcc70c0 0x5565cfcc7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.063 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.056+0000 7f90a95f8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565cfbcb010 con 0x5565cfcc6500 2023-11-27T15:17:10.063 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.056+0000 7f90a95f8b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565cfc1d850 con 0x5565cfcc5bb0 2023-11-27T15:17:10.063 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.056+0000 7f90a95f8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565cfc16ce0 con 0x5565cfcc70c0 2023-11-27T15:17:10.064 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.056+0000 7f90967fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfcc6500 0x5565cfcc6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.064 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.056+0000 7f90967fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfcc6500 0x5565cfcc6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50850/0 (socket says 172.21.15.52:50850) 2023-11-27T15:17:10.064 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.056+0000 7f90967fc700 1 -- 172.21.15.52:0/2255972916 learned_addr learned my addr 172.21.15.52:0/2255972916 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:10.064 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.056+0000 7f90977fe700 1 --2- 172.21.15.52:0/2255972916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565cfcc70c0 0x5565cfcc7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.064 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.056+0000 7f9096ffd700 1 --2- 172.21.15.52:0/2255972916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565cfcc5bb0 0x5565cfcc5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.064 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.056+0000 7f90967fc700 1 -- 172.21.15.52:0/2255972916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565cfcc70c0 msgr2=0x5565cfcc7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:10.064 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.056+0000 7f90967fc700 1 --2- 172.21.15.52:0/2255972916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565cfcc70c0 0x5565cfcc7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.064 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.056+0000 7f90967fc700 1 -- 172.21.15.52:0/2255972916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565cfcc5bb0 msgr2=0x5565cfcc5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:10.064 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.056+0000 7f90967fc700 1 --2- 172.21.15.52:0/2255972916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565cfcc5bb0 0x5565cfcc5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.064 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.056+0000 7f90967fc700 1 -- 172.21.15.52:0/2255972916 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565cfa985e0 con 0x5565cfcc6500 2023-11-27T15:17:10.064 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.057+0000 7f90967fc700 1 --2- 172.21.15.52:0/2255972916 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfcc6500 0x5565cfcc6ae0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9088097d20 tx=0x7f9088009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd9983ef9d0acab server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:10.064 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.057+0000 7f907d7fa700 1 -- 172.21.15.52:0/2255972916 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908809ac10 con 0x5565cfcc6500 2023-11-27T15:17:10.064 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.057+0000 7f907d7fa700 1 -- 172.21.15.52:0/2255972916 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f908809b2b0 con 0x5565cfcc6500 2023-11-27T15:17:10.064 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.057+0000 7f907d7fa700 1 -- 172.21.15.52:0/2255972916 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908809a7a0 con 0x5565cfcc6500 2023-11-27T15:17:10.065 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.057+0000 7f90a95f8b80 1 -- 172.21.15.52:0/2255972916 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfcc6500 msgr2=0x5565cfcc6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:10.065 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.057+0000 7f90a95f8b80 1 --2- 172.21.15.52:0/2255972916 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfcc6500 0x5565cfcc6ae0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9088097d20 tx=0x7f9088009310 comp rx=0 tx=0).stop 2023-11-27T15:17:10.065 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.057+0000 7f90a95f8b80 1 -- 172.21.15.52:0/2255972916 shutdown_connections 2023-11-27T15:17:10.065 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.057+0000 7f90a95f8b80 1 --2- 172.21.15.52:0/2255972916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565cfcc5bb0 0x5565cfcc5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.065 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.057+0000 7f90a95f8b80 1 --2- 172.21.15.52:0/2255972916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565cfcc70c0 0x5565cfcc7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.065 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.057+0000 7f90a95f8b80 1 --2- 172.21.15.52:0/2255972916 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfcc6500 0x5565cfcc6ae0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.065 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.058+0000 7f90a95f8b80 1 -- 172.21.15.52:0/2255972916 >> 172.21.15.52:0/2255972916 conn(0x5565cfcbb310 msgr2=0x5565cfcc0620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:10.065 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.058+0000 7f90a95f8b80 1 -- 172.21.15.52:0/2255972916 shutdown_connections 2023-11-27T15:17:10.065 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.058+0000 7f90a95f8b80 1 -- 172.21.15.52:0/2255972916 wait complete. 2023-11-27T15:17:10.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.058+0000 7f90a95f8b80 1 -- start start 2023-11-27T15:17:10.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.058+0000 7f90a95f8b80 1 Processor -- start 2023-11-27T15:17:10.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.058+0000 7f90a95f8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfcc5bb0 0x5565cfccf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.058+0000 7f90a95f8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565cfcc6500 0x5565cfccf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.058+0000 7f90a95f8b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565cfcc70c0 0x5565cfcd67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.058+0000 7f90a95f8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565cfbb8890 con 0x5565cfcc5bb0 2023-11-27T15:17:10.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.058+0000 7f90a95f8b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565cfbb8530 con 0x5565cfcc70c0 2023-11-27T15:17:10.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.058+0000 7f90a95f8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565cfa9a8d0 con 0x5565cfcc6500 2023-11-27T15:17:10.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.058+0000 7f90a95f8b80 10 client.0 _ll_register_callbacks cb 0x5565cfbd6730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:17:10.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.058+0000 7f9096ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfcc5bb0 0x5565cfccf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.058+0000 7f9096ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfcc5bb0 0x5565cfccf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50854/0 (socket says 172.21.15.52:50854) 2023-11-27T15:17:10.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.058+0000 7f9096ffd700 1 -- 172.21.15.52:0/1028589099 learned_addr learned my addr 172.21.15.52:0/1028589099 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:10.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.058+0000 7f90977fe700 1 --2- 172.21.15.52:0/1028589099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565cfcc70c0 0x5565cfcd67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.058+0000 7f90967fc700 1 --2- 172.21.15.52:0/1028589099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565cfcc6500 0x5565cfccf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.059+0000 7f9096ffd700 1 -- 172.21.15.52:0/1028589099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565cfcc6500 msgr2=0x5565cfccf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:10.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.059+0000 7f9096ffd700 1 --2- 172.21.15.52:0/1028589099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565cfcc6500 0x5565cfccf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.059+0000 7f9096ffd700 1 -- 172.21.15.52:0/1028589099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565cfcc70c0 msgr2=0x5565cfcd67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:10.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.059+0000 7f9096ffd700 1 --2- 172.21.15.52:0/1028589099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565cfcc70c0 0x5565cfcd67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.059+0000 7f9096ffd700 1 -- 172.21.15.52:0/1028589099 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f909810a920 con 0x5565cfcc5bb0 2023-11-27T15:17:10.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.059+0000 7f90967fc700 1 --2- 172.21.15.52:0/1028589099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565cfcc6500 0x5565cfccf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:17:10.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.059+0000 7f9096ffd700 1 --2- 172.21.15.52:0/1028589099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfcc5bb0 0x5565cfccf250 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f908c011490 tx=0x7f908c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95243fd0a94d901 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:10.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.059+0000 7f907aff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:17:10.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.059+0000 7f907aff5700 1 -- 172.21.15.52:0/1028589099 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c01f630 con 0x5565cfcc5bb0 2023-11-27T15:17:10.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.059+0000 7f90a95f8b80 1 -- 172.21.15.52:0/1028589099 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565cfcb8ad0 con 0x5565cfcc5bb0 2023-11-27T15:17:10.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.059+0000 7f907aff5700 1 -- 172.21.15.52:0/1028589099 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f908c01f790 con 0x5565cfcc5bb0 2023-11-27T15:17:10.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.059+0000 7f907aff5700 1 -- 172.21.15.52:0/1028589099 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f908c01f950 con 0x5565cfcc5bb0 2023-11-27T15:17:10.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.059+0000 7f90a95f8b80 10 client.15006 fetch_fsmap learned FSMap version 23 2023-11-27T15:17:10.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.059+0000 7f90a95f8b80 1 -- 172.21.15.52:0/1028589099 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5565cfbdd140 con 0x5565cfcc5bb0 2023-11-27T15:17:10.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.059+0000 7f907aff5700 1 -- 172.21.15.52:0/1028589099 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5565cfcb8ad0 con 0x5565cfcc5bb0 2023-11-27T15:17:10.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.059+0000 7f907aff5700 1 -- 172.21.15.52:0/1028589099 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f908c030050 con 0x5565cfcc5bb0 2023-11-27T15:17:10.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.059+0000 7f90a95f8b80 10 client.15006 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:17:10.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.060+0000 7f90a95f8b80 10 client.15006 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-27T15:17:10.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.060+0000 7f90a95f8b80 20 client.15006 populate_metadata read hostname 'smithi052' 2023-11-27T15:17:10.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.060+0000 7f90a95f8b80 1 --2- 172.21.15.52:0/1028589099 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5565cfcda580 0x5565cfcd9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.060+0000 7f90a95f8b80 4 client.15006 mds_command: new command op to 24472 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:17:10.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.060+0000 7f90a95f8b80 1 -- 172.21.15.52:0/1028589099 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565cfbd9050 con 0x5565cfcda580 2023-11-27T15:17:10.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.060+0000 7f90967fc700 1 --2- 172.21.15.52:0/1028589099 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5565cfcda580 0x5565cfcd9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.060+0000 7f90967fc700 1 --2- 172.21.15.52:0/1028589099 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5565cfcda580 0x5565cfcd9980 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fe8df2c9384921fe server_cookie=8082d2fe17cb46c9 in_seq=0 out_seq=0 2023-11-27T15:17:10.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.060+0000 7f907aff5700 10 client.15006 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:17:10.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.061+0000 7f907aff5700 1 -- 172.21.15.52:0/1028589099 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5565cfbd9050 con 0x5565cfcda580 2023-11-27T15:17:10.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.061+0000 7f907aff5700 10 client.15006 handle_command_reply: tid=0 2023-11-27T15:17:10.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.061+0000 7f90a95f8b80 1 client.15006 shutdown 2023-11-27T15:17:10.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.061+0000 7f90a95f8b80 1 -- 172.21.15.52:0/1028589099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfcc5bb0 msgr2=0x5565cfccf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:10.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.061+0000 7f90a95f8b80 1 --2- 172.21.15.52:0/1028589099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfcc5bb0 0x5565cfccf250 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f908c011490 tx=0x7f908c004750 comp rx=0 tx=0).stop 2023-11-27T15:17:10.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.061+0000 7f90a95f8b80 1 -- 172.21.15.52:0/1028589099 shutdown_connections 2023-11-27T15:17:10.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.061+0000 7f90a95f8b80 1 --2- 172.21.15.52:0/1028589099 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5565cfcda580 0x5565cfcd9980 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.061+0000 7f90a95f8b80 1 --2- 172.21.15.52:0/1028589099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565cfcc70c0 0x5565cfcd67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.061+0000 7f90a95f8b80 1 --2- 172.21.15.52:0/1028589099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfcc5bb0 0x5565cfccf250 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.061+0000 7f907aff5700 0 client.15006 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:17:10.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.061+0000 7f90a95f8b80 1 --2- 172.21.15.52:0/1028589099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565cfcc6500 0x5565cfccf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.061+0000 7f90a95f8b80 1 -- 172.21.15.52:0/1028589099 >> 172.21.15.52:0/1028589099 conn(0x5565cfcbb310 msgr2=0x5565cfcbfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:10.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.061+0000 7f90a95f8b80 1 -- 172.21.15.52:0/1028589099 shutdown_connections 2023-11-27T15:17:10.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.061+0000 7f90a95f8b80 1 -- 172.21.15.52:0/1028589099 wait complete. 2023-11-27T15:17:10.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.061+0000 7f90a95f8b80 20 client.15006 trim_cache size 0 max 16384 2023-11-27T15:17:10.089 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.079+0000 7f90a95f8b80 1 Processor -- start 2023-11-27T15:17:10.089 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.079+0000 7f90a95f8b80 1 -- start start 2023-11-27T15:17:10.089 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.079+0000 7f90a95f8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfcc5bb0 0x5565cfcd9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.090 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.079+0000 7f90a95f8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565cfcc6500 0x5565cfcd9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.090 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.079+0000 7f90a95f8b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565cfcc70c0 0x5565cfcd6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.090 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.079+0000 7f90a95f8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565cfc1d850 con 0x5565cfcc5bb0 2023-11-27T15:17:10.090 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.079+0000 7f90a95f8b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565cfbd9050 con 0x5565cfcc70c0 2023-11-27T15:17:10.090 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.079+0000 7f90a95f8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565cfb22ed0 con 0x5565cfcc6500 2023-11-27T15:17:10.090 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.079+0000 7f9096ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfcc5bb0 0x5565cfcd9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.090 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.080+0000 7f9096ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfcc5bb0 0x5565cfcd9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50862/0 (socket says 172.21.15.52:50862) 2023-11-27T15:17:10.090 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.080+0000 7f9096ffd700 1 -- 172.21.15.52:0/302169 learned_addr learned my addr 172.21.15.52:0/302169 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:10.090 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.080+0000 7f90977fe700 1 --2- 172.21.15.52:0/302169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565cfcc70c0 0x5565cfcd6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.090 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.080+0000 7f9096ffd700 1 -- 172.21.15.52:0/302169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565cfcc6500 msgr2=0x5565cfcd9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:10.090 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.080+0000 7f90967fc700 1 --2- 172.21.15.52:0/302169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565cfcc6500 0x5565cfcd9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.090 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.080+0000 7f9096ffd700 1 --2- 172.21.15.52:0/302169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565cfcc6500 0x5565cfcd9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.080+0000 7f9096ffd700 1 -- 172.21.15.52:0/302169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565cfcc70c0 msgr2=0x5565cfcd6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:10.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.080+0000 7f9096ffd700 1 --2- 172.21.15.52:0/302169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565cfcc70c0 0x5565cfcd6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.080+0000 7f9096ffd700 1 -- 172.21.15.52:0/302169 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565cfcbcab0 con 0x5565cfcc5bb0 2023-11-27T15:17:10.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.080+0000 7f90977fe700 1 --2- 172.21.15.52:0/302169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565cfcc70c0 0x5565cfcd6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:17:10.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.080+0000 7f9096ffd700 1 --2- 172.21.15.52:0/302169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfcc5bb0 0x5565cfcd9980 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f908c014040 tx=0x7f908c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef470ae505201cc5 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:10.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.080+0000 7f907dffb700 1 -- 172.21.15.52:0/302169 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c011b30 con 0x5565cfcc5bb0 2023-11-27T15:17:10.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.080+0000 7f907dffb700 1 -- 172.21.15.52:0/302169 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f908c0044e0 con 0x5565cfcc5bb0 2023-11-27T15:17:10.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.080+0000 7f907dffb700 1 -- 172.21.15.52:0/302169 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c003a80 con 0x5565cfcc5bb0 2023-11-27T15:17:10.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.081+0000 7f90a95f8b80 1 -- 172.21.15.52:0/302169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfcc5bb0 msgr2=0x5565cfcd9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:10.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.081+0000 7f90a95f8b80 1 --2- 172.21.15.52:0/302169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfcc5bb0 0x5565cfcd9980 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f908c014040 tx=0x7f908c00b4a0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.081+0000 7f90a95f8b80 1 -- 172.21.15.52:0/302169 shutdown_connections 2023-11-27T15:17:10.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.081+0000 7f90a95f8b80 1 --2- 172.21.15.52:0/302169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565cfcc70c0 0x5565cfcd6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.081+0000 7f90a95f8b80 1 --2- 172.21.15.52:0/302169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfcc5bb0 0x5565cfcd9980 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.081+0000 7f90a95f8b80 1 --2- 172.21.15.52:0/302169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565cfcc6500 0x5565cfcd9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.081+0000 7f90a95f8b80 1 -- 172.21.15.52:0/302169 >> 172.21.15.52:0/302169 conn(0x5565cfcbb310 msgr2=0x5565cfccd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:10.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.081+0000 7f90a95f8b80 1 -- 172.21.15.52:0/302169 shutdown_connections 2023-11-27T15:17:10.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.081+0000 7f90a95f8b80 1 -- 172.21.15.52:0/302169 wait complete. 2023-11-27T15:17:10.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.081+0000 7f90a95f8b80 1 -- start start 2023-11-27T15:17:10.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.081+0000 7f90a95f8b80 1 Processor -- start 2023-11-27T15:17:10.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.081+0000 7f90a95f8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfcc5bb0 0x5565cfcd1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.081+0000 7f90a95f8b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565cfcc6500 0x5565cfcd65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.081+0000 7f90a95f8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565cfcc70c0 0x5565cfcd15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.082+0000 7f90a95f8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565cfcd1c80 con 0x5565cfcc5bb0 2023-11-27T15:17:10.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.082+0000 7f90a95f8b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565cfd2e420 con 0x5565cfcc6500 2023-11-27T15:17:10.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.082+0000 7f90a95f8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565cfd2e560 con 0x5565cfcc70c0 2023-11-27T15:17:10.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.082+0000 7f90a95f8b80 10 client.0 _ll_register_callbacks cb 0x7f908c00fbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:17:10.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.082+0000 7f9096ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfcc5bb0 0x5565cfcd1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.082+0000 7f9096ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfcc5bb0 0x5565cfcd1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50868/0 (socket says 172.21.15.52:50868) 2023-11-27T15:17:10.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.082+0000 7f9096ffd700 1 -- 172.21.15.52:0/4280776104 learned_addr learned my addr 172.21.15.52:0/4280776104 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:10.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.082+0000 7f90967fc700 1 --2- 172.21.15.52:0/4280776104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565cfcc6500 0x5565cfcd65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.082+0000 7f9096ffd700 1 -- 172.21.15.52:0/4280776104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565cfcc70c0 msgr2=0x5565cfcd15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:10.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.082+0000 7f9096ffd700 1 --2- 172.21.15.52:0/4280776104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565cfcc70c0 0x5565cfcd15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.082+0000 7f9096ffd700 1 -- 172.21.15.52:0/4280776104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565cfcc6500 msgr2=0x5565cfcd65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:10.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.082+0000 7f9096ffd700 1 --2- 172.21.15.52:0/4280776104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565cfcc6500 0x5565cfcd65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.082+0000 7f9096ffd700 1 -- 172.21.15.52:0/4280776104 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565cfbdd140 con 0x5565cfcc5bb0 2023-11-27T15:17:10.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.082+0000 7f9096ffd700 1 --2- 172.21.15.52:0/4280776104 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfcc5bb0 0x5565cfcd1060 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f908c032880 tx=0x7f908c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caaf92e06eecb6ca server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:10.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.082+0000 7f907aff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:17:10.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.082+0000 7f907aff5700 1 -- 172.21.15.52:0/4280776104 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c011b30 con 0x5565cfcc5bb0 2023-11-27T15:17:10.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.082+0000 7f907aff5700 1 -- 172.21.15.52:0/4280776104 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f908c003800 con 0x5565cfcc5bb0 2023-11-27T15:17:10.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.082+0000 7f90a95f8b80 1 -- 172.21.15.52:0/4280776104 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f908809b2b0 con 0x5565cfcc5bb0 2023-11-27T15:17:10.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.082+0000 7f907aff5700 1 -- 172.21.15.52:0/4280776104 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f908c0039c0 con 0x5565cfcc5bb0 2023-11-27T15:17:10.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.082+0000 7f90a95f8b80 10 client.15012 fetch_fsmap learned FSMap version 23 2023-11-27T15:17:10.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.082+0000 7f90a95f8b80 1 -- 172.21.15.52:0/4280776104 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5565cfa44d90 con 0x5565cfcc5bb0 2023-11-27T15:17:10.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.083+0000 7f907aff5700 1 -- 172.21.15.52:0/4280776104 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f908c052020 con 0x5565cfcc5bb0 2023-11-27T15:17:10.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.083+0000 7f907aff5700 1 -- 172.21.15.52:0/4280776104 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f908c039030 con 0x5565cfcc5bb0 2023-11-27T15:17:10.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.083+0000 7f90a95f8b80 10 client.15012 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:17:10.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.083+0000 7f90a95f8b80 10 client.15012 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-27T15:17:10.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.083+0000 7f90a95f8b80 20 client.15012 populate_metadata read hostname 'smithi052' 2023-11-27T15:17:10.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.083+0000 7f90a95f8b80 1 --2- 172.21.15.52:0/4280776104 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5565cfd32d50 0x5565cfd2f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.083+0000 7f90a95f8b80 4 client.15012 mds_command: new command op to 24472 tid=0[{"prefix": "status"}] 2023-11-27T15:17:10.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.083+0000 7f90a95f8b80 1 -- 172.21.15.52:0/4280776104 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5565cfbe5610 con 0x5565cfd32d50 2023-11-27T15:17:10.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.083+0000 7f90967fc700 1 --2- 172.21.15.52:0/4280776104 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5565cfd32d50 0x5565cfd2f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.083+0000 7f907aff5700 10 client.15012 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:17:10.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.083+0000 7f90967fc700 1 --2- 172.21.15.52:0/4280776104 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5565cfd32d50 0x5565cfd2f780 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=57c6a240847348c7 server_cookie=dbd6e514035ae5c5 in_seq=0 out_seq=0 2023-11-27T15:17:10.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.084+0000 7f907aff5700 1 -- 172.21.15.52:0/4280776104 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x5565cfbe5610 con 0x5565cfd32d50 2023-11-27T15:17:10.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.084+0000 7f907aff5700 10 client.15012 handle_command_reply: tid=0 2023-11-27T15:17:10.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.084+0000 7f90a95f8b80 1 client.15012 shutdown 2023-11-27T15:17:10.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.084+0000 7f90a95f8b80 1 -- 172.21.15.52:0/4280776104 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfcc5bb0 msgr2=0x5565cfcd1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:10.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.084+0000 7f90a95f8b80 1 --2- 172.21.15.52:0/4280776104 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfcc5bb0 0x5565cfcd1060 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f908c032880 tx=0x7f908c0047c0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.084+0000 7f90a95f8b80 1 -- 172.21.15.52:0/4280776104 shutdown_connections 2023-11-27T15:17:10.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.084+0000 7f90a95f8b80 1 --2- 172.21.15.52:0/4280776104 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5565cfd32d50 0x5565cfd2f780 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.084+0000 7f90a95f8b80 1 --2- 172.21.15.52:0/4280776104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5565cfcc70c0 0x5565cfcd15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.084+0000 7f90a95f8b80 1 --2- 172.21.15.52:0/4280776104 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565cfcc5bb0 0x5565cfcd1060 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.084+0000 7f907aff5700 0 client.15012 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:17:10.096 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.084+0000 7f90a95f8b80 1 --2- 172.21.15.52:0/4280776104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565cfcc6500 0x5565cfcd65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.096 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.084+0000 7f90a95f8b80 1 -- 172.21.15.52:0/4280776104 >> 172.21.15.52:0/4280776104 conn(0x5565cfcbb310 msgr2=0x5565cfcd9980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:10.096 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.084+0000 7f90a95f8b80 1 -- 172.21.15.52:0/4280776104 shutdown_connections 2023-11-27T15:17:10.096 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.084+0000 7f90a95f8b80 1 -- 172.21.15.52:0/4280776104 wait complete. 2023-11-27T15:17:10.096 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.084+0000 7f90a95f8b80 20 client.15012 trim_cache size 0 max 16384 2023-11-27T15:17:10.096 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:17:10.096 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "c6c7df88-8d36-11ee-95a2-87774f69a715", 2023-11-27T15:17:10.096 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 2, 2023-11-27T15:17:10.096 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24472, 2023-11-27T15:17:10.097 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2023-11-27T15:17:10.097 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2023-11-27T15:17:10.097 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2023-11-27T15:17:10.097 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 56.345825366, 2023-11-27T15:17:10.097 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 23, 2023-11-27T15:17:10.097 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 85, 2023-11-27T15:17:10.097 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 83, 2023-11-27T15:17:10.097 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 77.222658104000004 2023-11-27T15:17:10.097 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:17:10.098 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.084+0000 7f909d46c700 1 -- 172.21.15.52:0/3400867382 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f90800a11e0 msgr2=0x7f90800a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:10.098 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.084+0000 7f909d46c700 1 --2- 172.21.15.52:0/3400867382 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f90800a11e0 0x7f90800a3690 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f908c005950 tx=0x7f908c0058e0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.098 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.084+0000 7f909d46c700 1 -- 172.21.15.52:0/3400867382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9098100340 msgr2=0x7f9098198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:10.098 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.084+0000 7f909d46c700 1 --2- 172.21.15.52:0/3400867382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9098100340 0x7f9098198650 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9088005770 tx=0x7f9088005ef0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.098 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.084+0000 7f909d46c700 1 -- 172.21.15.52:0/3400867382 shutdown_connections 2023-11-27T15:17:10.098 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.084+0000 7f909d46c700 1 --2- 172.21.15.52:0/3400867382 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f90800a11e0 0x7f90800a3690 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.098 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.084+0000 7f909d46c700 1 --2- 172.21.15.52:0/3400867382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9098100cd0 0x7f909819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.098 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.084+0000 7f909d46c700 1 --2- 172.21.15.52:0/3400867382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90980ff150 0x7f9098198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.098 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.084+0000 7f909d46c700 1 --2- 172.21.15.52:0/3400867382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9098100340 0x7f9098198650 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.099 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.084+0000 7f909d46c700 1 -- 172.21.15.52:0/3400867382 >> 172.21.15.52:0/3400867382 conn(0x7f90980fa750 msgr2=0x7f90980690d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:10.099 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.085+0000 7f909d46c700 1 -- 172.21.15.52:0/3400867382 shutdown_connections 2023-11-27T15:17:10.099 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.085+0000 7f909d46c700 1 -- 172.21.15.52:0/3400867382 wait complete. 2023-11-27T15:17:10.101 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:09 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3519635373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:17:10.101 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:09 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3399983092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:17:10.101 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:09 smithi052 ceph-mon[99056]: pgmap v288: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 5 op/s; 127 B/s, 1 keys/s, 0 objects/s recovering 2023-11-27T15:17:10.102 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-11-27T15:17:10.229 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.223+0000 7f2e4c977700 1 -- 172.21.15.52:0/3235752775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e48100260 msgr2=0x7f2e48100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:10.229 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.223+0000 7f2e4c977700 1 --2- 172.21.15.52:0/3235752775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e48100260 0x7f2e48100670 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009990 tx=0x7f2e38009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.229 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.223+0000 7f2e4c977700 1 -- 172.21.15.52:0/3235752775 shutdown_connections 2023-11-27T15:17:10.229 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.223+0000 7f2e4c977700 1 --2- 172.21.15.52:0/3235752775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e48101de0 0x7f2e481086a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.229 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.223+0000 7f2e4c977700 1 --2- 172.21.15.52:0/3235752775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e48100260 0x7f2e48100670 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.229 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.223+0000 7f2e4c977700 1 --2- 172.21.15.52:0/3235752775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e48101450 0x7f2e481018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.229 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.223+0000 7f2e4c977700 1 -- 172.21.15.52:0/3235752775 >> 172.21.15.52:0/3235752775 conn(0x7f2e480fb840 msgr2=0x7f2e480fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:10.230 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.223+0000 7f2e4c977700 1 -- 172.21.15.52:0/3235752775 shutdown_connections 2023-11-27T15:17:10.230 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.223+0000 7f2e4c977700 1 -- 172.21.15.52:0/3235752775 wait complete. 2023-11-27T15:17:10.232 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.224+0000 7f2e4c977700 1 Processor -- start 2023-11-27T15:17:10.232 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.224+0000 7f2e4c977700 1 -- start start 2023-11-27T15:17:10.232 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.224+0000 7f2e4c977700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e48100260 0x7f2e48195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.232 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.224+0000 7f2e4c977700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e48101450 0x7f2e481964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.232 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.224+0000 7f2e4c977700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e48101de0 0x7f2e4819d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.232 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.224+0000 7f2e4c977700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4819db10 con 0x7f2e48100260 2023-11-27T15:17:10.233 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.224+0000 7f2e4c977700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4819dc50 con 0x7f2e48101450 2023-11-27T15:17:10.233 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.224+0000 7f2e4c977700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4819dde0 con 0x7f2e48101de0 2023-11-27T15:17:10.233 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.224+0000 7f2e4659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e48100260 0x7f2e48195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.233 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.224+0000 7f2e4659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e48100260 0x7f2e48195f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50882/0 (socket says 172.21.15.52:50882) 2023-11-27T15:17:10.233 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.224+0000 7f2e4659c700 1 -- 172.21.15.52:0/16455538 learned_addr learned my addr 172.21.15.52:0/16455538 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:10.233 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.224+0000 7f2e45d9b700 1 --2- 172.21.15.52:0/16455538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e48101450 0x7f2e481964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.234 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.225+0000 7f2e46d9d700 1 --2- 172.21.15.52:0/16455538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e48101de0 0x7f2e4819d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.234 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.225+0000 7f2e4659c700 1 -- 172.21.15.52:0/16455538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e48101de0 msgr2=0x7f2e4819d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:10.234 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.225+0000 7f2e4659c700 1 --2- 172.21.15.52:0/16455538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e48101de0 0x7f2e4819d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.234 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.225+0000 7f2e4659c700 1 -- 172.21.15.52:0/16455538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e48101450 msgr2=0x7f2e481964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:10.234 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.225+0000 7f2e4659c700 1 --2- 172.21.15.52:0/16455538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e48101450 0x7f2e481964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.234 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.225+0000 7f2e4659c700 1 -- 172.21.15.52:0/16455538 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e380095f0 con 0x7f2e48100260 2023-11-27T15:17:10.234 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.225+0000 7f2e4659c700 1 --2- 172.21.15.52:0/16455538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e48100260 0x7f2e48195f90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800b620 tx=0x7f2e38005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:10.235 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.225+0000 7f2e3f7fe700 1 -- 172.21.15.52:0/16455538 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3801d070 con 0x7f2e48100260 2023-11-27T15:17:10.235 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.225+0000 7f2e3f7fe700 1 -- 172.21.15.52:0/16455538 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e380047e0 con 0x7f2e48100260 2023-11-27T15:17:10.235 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.225+0000 7f2e3f7fe700 1 -- 172.21.15.52:0/16455538 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e38005560 con 0x7f2e48100260 2023-11-27T15:17:10.235 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.225+0000 7f2e4c977700 1 -- 172.21.15.52:0/16455538 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e4810bbc0 con 0x7f2e48100260 2023-11-27T15:17:10.235 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.225+0000 7f2e4c977700 1 -- 172.21.15.52:0/16455538 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e4810ba30 con 0x7f2e48100260 2023-11-27T15:17:10.241 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.228+0000 7f2e3f7fe700 1 -- 172.21.15.52:0/16455538 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f2e38004950 con 0x7f2e48100260 2023-11-27T15:17:10.241 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.228+0000 7f2e3f7fe700 1 --2- 172.21.15.52:0/16455538 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f2e340a1260 0x7f2e340a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.242 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.228+0000 7f2e3f7fe700 1 -- 172.21.15.52:0/16455538 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f2e380cdbe0 con 0x7f2e48100260 2023-11-27T15:17:10.242 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.228+0000 7f2e45d9b700 1 --2- 172.21.15.52:0/16455538 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f2e340a1260 0x7f2e340a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.242 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.229+0000 7f2e45d9b700 1 --2- 172.21.15.52:0/16455538 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f2e340a1260 0x7f2e340a3710 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2e30009bf0 tx=0x7f2e30009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:10.242 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.229+0000 7f2e58b03b80 1 Processor -- start 2023-11-27T15:17:10.242 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.229+0000 7f2e58b03b80 1 -- start start 2023-11-27T15:17:10.242 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.229+0000 7f2e58b03b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef43ddf000 0x55ef43ddf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.243 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.229+0000 7f2e58b03b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef43ddf950 0x55ef43ddfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.243 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.229+0000 7f2e58b03b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ef43de04c0 0x55ef43de0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.243 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.229+0000 7f2e58b03b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef43ce0010 con 0x55ef43ddf000 2023-11-27T15:17:10.243 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.230+0000 7f2e58b03b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef43d32850 con 0x55ef43de04c0 2023-11-27T15:17:10.243 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.230+0000 7f2e58b03b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef43d2bce0 con 0x55ef43ddf950 2023-11-27T15:17:10.243 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.230+0000 7f2e4659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef43ddf000 0x55ef43ddf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.243 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.230+0000 7f2e4659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef43ddf000 0x55ef43ddf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50890/0 (socket says 172.21.15.52:50890) 2023-11-27T15:17:10.244 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.230+0000 7f2e4659c700 1 -- 172.21.15.52:0/1639576059 learned_addr learned my addr 172.21.15.52:0/1639576059 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:10.244 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.230+0000 7f2e45d9b700 1 --2- 172.21.15.52:0/1639576059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef43ddf950 0x55ef43ddfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.244 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.230+0000 7f2e4659c700 1 -- 172.21.15.52:0/1639576059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef43ddf950 msgr2=0x55ef43ddfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:10.244 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.230+0000 7f2e46d9d700 1 --2- 172.21.15.52:0/1639576059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ef43de04c0 0x55ef43de0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.244 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.230+0000 7f2e4659c700 1 --2- 172.21.15.52:0/1639576059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef43ddf950 0x55ef43ddfee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.244 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.230+0000 7f2e4659c700 1 -- 172.21.15.52:0/1639576059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ef43de04c0 msgr2=0x55ef43de0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:10.244 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.230+0000 7f2e4659c700 1 --2- 172.21.15.52:0/1639576059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ef43de04c0 0x55ef43de0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.245 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.230+0000 7f2e4659c700 1 -- 172.21.15.52:0/1639576059 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef43bad5e0 con 0x55ef43ddf000 2023-11-27T15:17:10.245 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.230+0000 7f2e45d9b700 1 --2- 172.21.15.52:0/1639576059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef43ddf950 0x55ef43ddfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:17:10.245 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.230+0000 7f2e46d9d700 1 --2- 172.21.15.52:0/1639576059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ef43de04c0 0x55ef43de0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:17:10.245 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.230+0000 7f2e4659c700 1 --2- 172.21.15.52:0/1639576059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef43ddf000 0x55ef43ddf410 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2e3809ae90 tx=0x7f2e3809af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=391ff59e5dedd936 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:10.245 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.230+0000 7f2e3cff9700 1 -- 172.21.15.52:0/1639576059 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e380992a0 con 0x55ef43ddf000 2023-11-27T15:17:10.245 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.230+0000 7f2e3cff9700 1 -- 172.21.15.52:0/1639576059 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e380cb7c0 con 0x55ef43ddf000 2023-11-27T15:17:10.246 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.230+0000 7f2e3cff9700 1 -- 172.21.15.52:0/1639576059 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e380a2e30 con 0x55ef43ddf000 2023-11-27T15:17:10.246 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.231+0000 7f2e58b03b80 1 -- 172.21.15.52:0/1639576059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef43ddf000 msgr2=0x55ef43ddf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:10.246 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.231+0000 7f2e58b03b80 1 --2- 172.21.15.52:0/1639576059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef43ddf000 0x55ef43ddf410 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2e3809ae90 tx=0x7f2e3809af80 comp rx=0 tx=0).stop 2023-11-27T15:17:10.246 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.231+0000 7f2e58b03b80 1 -- 172.21.15.52:0/1639576059 shutdown_connections 2023-11-27T15:17:10.246 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.231+0000 7f2e58b03b80 1 --2- 172.21.15.52:0/1639576059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ef43de04c0 0x55ef43de0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.246 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.231+0000 7f2e58b03b80 1 --2- 172.21.15.52:0/1639576059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef43ddf000 0x55ef43ddf410 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.246 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.231+0000 7f2e58b03b80 1 --2- 172.21.15.52:0/1639576059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef43ddf950 0x55ef43ddfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.231+0000 7f2e58b03b80 1 -- 172.21.15.52:0/1639576059 >> 172.21.15.52:0/1639576059 conn(0x55ef43dd0310 msgr2=0x55ef43dd5620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.231+0000 7f2e58b03b80 1 -- 172.21.15.52:0/1639576059 shutdown_connections 2023-11-27T15:17:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.231+0000 7f2e58b03b80 1 -- 172.21.15.52:0/1639576059 wait complete. 2023-11-27T15:17:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.231+0000 7f2e58b03b80 1 -- start start 2023-11-27T15:17:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.231+0000 7f2e58b03b80 1 Processor -- start 2023-11-27T15:17:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.232+0000 7f2e58b03b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef43ddf000 0x55ef43de8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.247 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.232+0000 7f2e58b03b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ef43ddf950 0x55ef43de8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.232+0000 7f2e58b03b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef43de04c0 0x55ef43defb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.232+0000 7f2e58b03b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef43ccd890 con 0x55ef43ddf000 2023-11-27T15:17:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.232+0000 7f2e58b03b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef43ccd530 con 0x55ef43ddf950 2023-11-27T15:17:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.232+0000 7f2e58b03b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef43baf8d0 con 0x55ef43de04c0 2023-11-27T15:17:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.232+0000 7f2e58b03b80 10 client.0 _ll_register_callbacks cb 0x55ef43ceb730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:17:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.232+0000 7f2e4659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef43ddf000 0x55ef43de8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.248 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.232+0000 7f2e4659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef43ddf000 0x55ef43de8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50896/0 (socket says 172.21.15.52:50896) 2023-11-27T15:17:10.249 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.232+0000 7f2e4659c700 1 -- 172.21.15.52:0/4273230422 learned_addr learned my addr 172.21.15.52:0/4273230422 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:10.249 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.232+0000 7f2e46d9d700 1 --2- 172.21.15.52:0/4273230422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef43de04c0 0x55ef43defb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.249 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.232+0000 7f2e45d9b700 1 --2- 172.21.15.52:0/4273230422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ef43ddf950 0x55ef43de8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.249 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.232+0000 7f2e4659c700 1 -- 172.21.15.52:0/4273230422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef43de04c0 msgr2=0x55ef43defb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:10.249 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.232+0000 7f2e4659c700 1 --2- 172.21.15.52:0/4273230422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef43de04c0 0x55ef43defb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.249 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.232+0000 7f2e4659c700 1 -- 172.21.15.52:0/4273230422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ef43ddf950 msgr2=0x55ef43de8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:10.249 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.232+0000 7f2e4659c700 1 --2- 172.21.15.52:0/4273230422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ef43ddf950 0x55ef43de8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.250 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.232+0000 7f2e4659c700 1 -- 172.21.15.52:0/4273230422 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e4810ba30 con 0x55ef43ddf000 2023-11-27T15:17:10.250 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.232+0000 7f2e45d9b700 1 --2- 172.21.15.52:0/4273230422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ef43ddf950 0x55ef43de8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:17:10.250 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.232+0000 7f2e4659c700 1 --2- 172.21.15.52:0/4273230422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef43ddf000 0x55ef43de8590 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2e380a2d90 tx=0x7f2e3809aac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9860c4759083524f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:10.250 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.232+0000 7f2e2dffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:17:10.250 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.232+0000 7f2e2dffb700 1 -- 172.21.15.52:0/4273230422 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e380cbb80 con 0x55ef43ddf000 2023-11-27T15:17:10.250 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.232+0000 7f2e58b03b80 1 -- 172.21.15.52:0/4273230422 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ef43dcdad0 con 0x55ef43ddf000 2023-11-27T15:17:10.250 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.232+0000 7f2e2dffb700 1 -- 172.21.15.52:0/4273230422 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f2e380a0070 con 0x55ef43ddf000 2023-11-27T15:17:10.251 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.232+0000 7f2e2dffb700 1 -- 172.21.15.52:0/4273230422 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e380b9250 con 0x55ef43ddf000 2023-11-27T15:17:10.251 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.232+0000 7f2e58b03b80 10 client.15024 fetch_fsmap learned FSMap version 23 2023-11-27T15:17:10.251 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.232+0000 7f2e58b03b80 1 -- 172.21.15.52:0/4273230422 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ef43cf2140 con 0x55ef43ddf000 2023-11-27T15:17:10.251 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.233+0000 7f2e2dffb700 1 -- 172.21.15.52:0/4273230422 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e380cb020 con 0x55ef43ddf000 2023-11-27T15:17:10.251 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.233+0000 7f2e2dffb700 1 -- 172.21.15.52:0/4273230422 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2e380aed00 con 0x55ef43ddf000 2023-11-27T15:17:10.251 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.233+0000 7f2e58b03b80 10 client.15024 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:17:10.251 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.233+0000 7f2e58b03b80 10 client.15024 resolve_mds: resolved name 'h' to daemon mds.h 2023-11-27T15:17:10.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.233+0000 7f2e58b03b80 20 client.15024 populate_metadata read hostname 'smithi052' 2023-11-27T15:17:10.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.233+0000 7f2e58b03b80 1 --2- 172.21.15.52:0/4273230422 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55ef43df3330 0x55ef43df29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.233+0000 7f2e58b03b80 4 client.15024 mds_command: new command op to 24494 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:17:10.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.233+0000 7f2e58b03b80 1 -- 172.21.15.52:0/4273230422 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ef43cee050 con 0x55ef43df3330 2023-11-27T15:17:10.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.233+0000 7f2e45d9b700 1 --2- 172.21.15.52:0/4273230422 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55ef43df3330 0x55ef43df29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.233+0000 7f2e2dffb700 10 client.15024 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:17:10.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.233+0000 7f2e45d9b700 1 --2- 172.21.15.52:0/4273230422 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55ef43df3330 0x55ef43df29b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bf672058995fb640 server_cookie=c4abd20346259817 in_seq=0 out_seq=0 2023-11-27T15:17:10.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.234+0000 7f2e2dffb700 1 -- 172.21.15.52:0/4273230422 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ef43cee050 con 0x55ef43df3330 2023-11-27T15:17:10.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.234+0000 7f2e2dffb700 10 client.15024 handle_command_reply: tid=0 2023-11-27T15:17:10.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.234+0000 7f2e58b03b80 1 client.15024 shutdown 2023-11-27T15:17:10.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.234+0000 7f2e58b03b80 1 -- 172.21.15.52:0/4273230422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef43ddf000 msgr2=0x55ef43de8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:10.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.234+0000 7f2e58b03b80 1 --2- 172.21.15.52:0/4273230422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef43ddf000 0x55ef43de8590 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2e380a2d90 tx=0x7f2e3809aac0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.234+0000 7f2e58b03b80 1 -- 172.21.15.52:0/4273230422 shutdown_connections 2023-11-27T15:17:10.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.234+0000 7f2e58b03b80 1 --2- 172.21.15.52:0/4273230422 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55ef43df3330 0x55ef43df29b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.234+0000 7f2e58b03b80 1 --2- 172.21.15.52:0/4273230422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef43de04c0 0x55ef43defb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.234+0000 7f2e58b03b80 1 --2- 172.21.15.52:0/4273230422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef43ddf000 0x55ef43de8590 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.234+0000 7f2e58b03b80 1 --2- 172.21.15.52:0/4273230422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ef43ddf950 0x55ef43de8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.234+0000 7f2e2dffb700 0 client.15024 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:17:10.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.234+0000 7f2e58b03b80 1 -- 172.21.15.52:0/4273230422 >> 172.21.15.52:0/4273230422 conn(0x55ef43dd0310 msgr2=0x55ef43dd4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:10.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.235+0000 7f2e58b03b80 1 -- 172.21.15.52:0/4273230422 shutdown_connections 2023-11-27T15:17:10.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.235+0000 7f2e58b03b80 1 -- 172.21.15.52:0/4273230422 wait complete. 2023-11-27T15:17:10.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.235+0000 7f2e58b03b80 20 client.15024 trim_cache size 0 max 16384 2023-11-27T15:17:10.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.253+0000 7f2e58b03b80 1 Processor -- start 2023-11-27T15:17:10.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.253+0000 7f2e58b03b80 1 -- start start 2023-11-27T15:17:10.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.253+0000 7f2e58b03b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef43ddf000 0x55ef43de7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.253+0000 7f2e58b03b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ef43ddf950 0x55ef43defad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.253+0000 7f2e58b03b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef43de04c0 0x55ef43df29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.253+0000 7f2e58b03b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef43d32850 con 0x55ef43de04c0 2023-11-27T15:17:10.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.253+0000 7f2e58b03b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef43de9490 con 0x55ef43ddf950 2023-11-27T15:17:10.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.253+0000 7f2e58b03b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef43cee050 con 0x55ef43ddf000 2023-11-27T15:17:10.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.253+0000 7f2e46d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef43de04c0 0x55ef43df29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.253+0000 7f2e46d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef43de04c0 0x55ef43df29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50908/0 (socket says 172.21.15.52:50908) 2023-11-27T15:17:10.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.253+0000 7f2e4659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef43ddf000 0x55ef43de7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.253+0000 7f2e46d9d700 1 -- 172.21.15.52:0/1110827290 learned_addr learned my addr 172.21.15.52:0/1110827290 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:10.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.253+0000 7f2e45d9b700 1 --2- 172.21.15.52:0/1110827290 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ef43ddf950 0x55ef43defad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.253+0000 7f2e46d9d700 1 -- 172.21.15.52:0/1110827290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef43ddf000 msgr2=0x55ef43de7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:10.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.253+0000 7f2e46d9d700 1 --2- 172.21.15.52:0/1110827290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef43ddf000 0x55ef43de7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.253+0000 7f2e46d9d700 1 -- 172.21.15.52:0/1110827290 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ef43ddf950 msgr2=0x55ef43defad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:10.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.253+0000 7f2e46d9d700 1 --2- 172.21.15.52:0/1110827290 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ef43ddf950 0x55ef43defad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.253+0000 7f2e46d9d700 1 -- 172.21.15.52:0/1110827290 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef43cf1e90 con 0x55ef43de04c0 2023-11-27T15:17:10.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.253+0000 7f2e45d9b700 1 --2- 172.21.15.52:0/1110827290 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ef43ddf950 0x55ef43defad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:17:10.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.254+0000 7f2e46d9d700 1 --2- 172.21.15.52:0/1110827290 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef43de04c0 0x55ef43df29b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2e400094a0 tx=0x7f2e4000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158ce82355859c85 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:10.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.254+0000 7f2e3d7fa700 1 -- 172.21.15.52:0/1110827290 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4000e030 con 0x55ef43de04c0 2023-11-27T15:17:10.267 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.254+0000 7f2e3d7fa700 1 -- 172.21.15.52:0/1110827290 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e40004020 con 0x55ef43de04c0 2023-11-27T15:17:10.267 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.254+0000 7f2e3d7fa700 1 -- 172.21.15.52:0/1110827290 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e400166c0 con 0x55ef43de04c0 2023-11-27T15:17:10.267 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.254+0000 7f2e58b03b80 1 -- 172.21.15.52:0/1110827290 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef43de04c0 msgr2=0x55ef43df29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:10.267 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.254+0000 7f2e58b03b80 1 --2- 172.21.15.52:0/1110827290 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef43de04c0 0x55ef43df29b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2e400094a0 tx=0x7f2e4000b040 comp rx=0 tx=0).stop 2023-11-27T15:17:10.267 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.254+0000 7f2e58b03b80 1 -- 172.21.15.52:0/1110827290 shutdown_connections 2023-11-27T15:17:10.267 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.254+0000 7f2e58b03b80 1 --2- 172.21.15.52:0/1110827290 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef43de04c0 0x55ef43df29b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.267 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.254+0000 7f2e58b03b80 1 --2- 172.21.15.52:0/1110827290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef43ddf000 0x55ef43de7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.268 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.254+0000 7f2e58b03b80 1 --2- 172.21.15.52:0/1110827290 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ef43ddf950 0x55ef43defad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.268 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.254+0000 7f2e58b03b80 1 -- 172.21.15.52:0/1110827290 >> 172.21.15.52:0/1110827290 conn(0x55ef43dd0310 msgr2=0x55ef43dfc060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:10.268 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.254+0000 7f2e58b03b80 1 -- 172.21.15.52:0/1110827290 shutdown_connections 2023-11-27T15:17:10.268 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.254+0000 7f2e58b03b80 1 -- 172.21.15.52:0/1110827290 wait complete. 2023-11-27T15:17:10.268 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.255+0000 7f2e58b03b80 1 -- start start 2023-11-27T15:17:10.268 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.255+0000 7f2e58b03b80 1 Processor -- start 2023-11-27T15:17:10.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.255+0000 7f2e58b03b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef43ddf000 0x55ef43dea290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.255+0000 7f2e58b03b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef43ddf950 0x55ef43dea7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.255+0000 7f2e58b03b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ef43de04c0 0x55ef43def640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.255+0000 7f2e58b03b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef43defb80 con 0x55ef43ddf000 2023-11-27T15:17:10.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.255+0000 7f2e58b03b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef43defcf0 con 0x55ef43de04c0 2023-11-27T15:17:10.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.255+0000 7f2e58b03b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef43defeb0 con 0x55ef43ddf950 2023-11-27T15:17:10.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.255+0000 7f2e58b03b80 10 client.0 _ll_register_callbacks cb 0x55ef43ceb730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:17:10.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.255+0000 7f2e4659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef43ddf000 0x55ef43dea290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.255+0000 7f2e4659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef43ddf000 0x55ef43dea290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50910/0 (socket says 172.21.15.52:50910) 2023-11-27T15:17:10.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.255+0000 7f2e4659c700 1 -- 172.21.15.52:0/1685798515 learned_addr learned my addr 172.21.15.52:0/1685798515 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:10.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.255+0000 7f2e46d9d700 1 --2- 172.21.15.52:0/1685798515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ef43de04c0 0x55ef43def640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.255+0000 7f2e45d9b700 1 --2- 172.21.15.52:0/1685798515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef43ddf950 0x55ef43dea7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.255+0000 7f2e4659c700 1 -- 172.21.15.52:0/1685798515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef43ddf950 msgr2=0x55ef43dea7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.256+0000 7f2e4659c700 1 --2- 172.21.15.52:0/1685798515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef43ddf950 0x55ef43dea7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.256+0000 7f2e4659c700 1 -- 172.21.15.52:0/1685798515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ef43de04c0 msgr2=0x55ef43def640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.256+0000 7f2e4659c700 1 --2- 172.21.15.52:0/1685798515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ef43de04c0 0x55ef43def640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.256+0000 7f2e4659c700 1 -- 172.21.15.52:0/1685798515 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e40009150 con 0x55ef43ddf000 2023-11-27T15:17:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.256+0000 7f2e4659c700 1 --2- 172.21.15.52:0/1685798515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef43ddf000 0x55ef43dea290 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2e38006010 tx=0x7f2e38095f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c3269fbf2fcdc8 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.256+0000 7f2e2dffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:17:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.256+0000 7f2e2dffb700 1 -- 172.21.15.52:0/1685798515 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e380cbd80 con 0x55ef43ddf000 2023-11-27T15:17:10.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.256+0000 7f2e2dffb700 1 -- 172.21.15.52:0/1685798515 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f2e380a0040 con 0x55ef43ddf000 2023-11-27T15:17:10.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.256+0000 7f2e58b03b80 1 -- 172.21.15.52:0/1685798515 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2e380cb7c0 con 0x55ef43ddf000 2023-11-27T15:17:10.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.256+0000 7f2e2dffb700 1 -- 172.21.15.52:0/1685798515 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e380b9250 con 0x55ef43ddf000 2023-11-27T15:17:10.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.256+0000 7f2e58b03b80 10 client.15030 fetch_fsmap learned FSMap version 23 2023-11-27T15:17:10.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.256+0000 7f2e58b03b80 1 -- 172.21.15.52:0/1685798515 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ef43b42c10 con 0x55ef43ddf000 2023-11-27T15:17:10.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.256+0000 7f2e2dffb700 1 -- 172.21.15.52:0/1685798515 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e380cb7c0 con 0x55ef43ddf000 2023-11-27T15:17:10.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.256+0000 7f2e2dffb700 1 -- 172.21.15.52:0/1685798515 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2e38099e60 con 0x55ef43ddf000 2023-11-27T15:17:10.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.256+0000 7f2e58b03b80 10 client.15030 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:17:10.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.256+0000 7f2e58b03b80 10 client.15030 resolve_mds: resolved name 'h' to daemon mds.h 2023-11-27T15:17:10.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.256+0000 7f2e58b03b80 20 client.15030 populate_metadata read hostname 'smithi052' 2023-11-27T15:17:10.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.256+0000 7f2e58b03b80 1 --2- 172.21.15.52:0/1685798515 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55ef43e4b6d0 0x55ef43e48120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.256+0000 7f2e58b03b80 4 client.15030 mds_command: new command op to 24494 tid=0[{"prefix": "status"}] 2023-11-27T15:17:10.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.256+0000 7f2e58b03b80 1 -- 172.21.15.52:0/1685798515 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ef43cfa610 con 0x55ef43e4b6d0 2023-11-27T15:17:10.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.257+0000 7f2e45d9b700 1 --2- 172.21.15.52:0/1685798515 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55ef43e4b6d0 0x55ef43e48120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.257+0000 7f2e45d9b700 1 --2- 172.21.15.52:0/1685798515 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55ef43e4b6d0 0x55ef43e48120 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b962875101c513d server_cookie=d2e23ec23b871487 in_seq=0 out_seq=0 2023-11-27T15:17:10.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.257+0000 7f2e2dffb700 10 client.15030 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:17:10.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.257+0000 7f2e2dffb700 1 -- 172.21.15.52:0/1685798515 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+320 (crc 0 0 0) 0x55ef43cfa610 con 0x55ef43e4b6d0 2023-11-27T15:17:10.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.257+0000 7f2e2dffb700 10 client.15030 handle_command_reply: tid=0 2023-11-27T15:17:10.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.257+0000 7f2e58b03b80 1 client.15030 shutdown 2023-11-27T15:17:10.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.257+0000 7f2e58b03b80 1 -- 172.21.15.52:0/1685798515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef43ddf000 msgr2=0x55ef43dea290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:10.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.257+0000 7f2e58b03b80 1 --2- 172.21.15.52:0/1685798515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef43ddf000 0x55ef43dea290 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2e38006010 tx=0x7f2e38095f90 comp rx=0 tx=0).stop 2023-11-27T15:17:10.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.257+0000 7f2e58b03b80 1 -- 172.21.15.52:0/1685798515 shutdown_connections 2023-11-27T15:17:10.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.257+0000 7f2e58b03b80 1 --2- 172.21.15.52:0/1685798515 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55ef43e4b6d0 0x55ef43e48120 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.257+0000 7f2e58b03b80 1 --2- 172.21.15.52:0/1685798515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ef43de04c0 0x55ef43def640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.257+0000 7f2e2dffb700 0 client.15030 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:17:10.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.257+0000 7f2e58b03b80 1 --2- 172.21.15.52:0/1685798515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef43ddf000 0x55ef43dea290 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.257+0000 7f2e58b03b80 1 --2- 172.21.15.52:0/1685798515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ef43ddf950 0x55ef43dea7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.257+0000 7f2e58b03b80 1 -- 172.21.15.52:0/1685798515 >> 172.21.15.52:0/1685798515 conn(0x55ef43dd0310 msgr2=0x55ef43de7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:10.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.257+0000 7f2e58b03b80 1 -- 172.21.15.52:0/1685798515 shutdown_connections 2023-11-27T15:17:10.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.257+0000 7f2e58b03b80 1 -- 172.21.15.52:0/1685798515 wait complete. 2023-11-27T15:17:10.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.258+0000 7f2e58b03b80 20 client.15030 trim_cache size 0 max 16384 2023-11-27T15:17:10.276 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:17:10.276 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "c6c7df88-8d36-11ee-95a2-87774f69a715", 2023-11-27T15:17:10.277 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 1, 2023-11-27T15:17:10.277 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24494, 2023-11-27T15:17:10.277 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2023-11-27T15:17:10.277 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2023-11-27T15:17:10.277 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2023-11-27T15:17:10.277 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 57.524465518, 2023-11-27T15:17:10.277 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 23, 2023-11-27T15:17:10.277 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 85, 2023-11-27T15:17:10.278 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 83, 2023-11-27T15:17:10.278 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 75.226523254 2023-11-27T15:17:10.278 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:17:10.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.258+0000 7f2e4c977700 1 -- 172.21.15.52:0/16455538 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f2e340a1260 msgr2=0x7f2e340a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:10.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.258+0000 7f2e4c977700 1 --2- 172.21.15.52:0/16455538 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f2e340a1260 0x7f2e340a3710 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2e30009bf0 tx=0x7f2e30009260 comp rx=0 tx=0).stop 2023-11-27T15:17:10.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.258+0000 7f2e4c977700 1 -- 172.21.15.52:0/16455538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e48100260 msgr2=0x7f2e48195f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:10.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.258+0000 7f2e4c977700 1 --2- 172.21.15.52:0/16455538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e48100260 0x7f2e48195f90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800b620 tx=0x7f2e38005ef0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.258+0000 7f2e4c977700 1 -- 172.21.15.52:0/16455538 shutdown_connections 2023-11-27T15:17:10.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.258+0000 7f2e4c977700 1 --2- 172.21.15.52:0/16455538 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f2e340a1260 0x7f2e340a3710 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.258+0000 7f2e4c977700 1 --2- 172.21.15.52:0/16455538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e48101de0 0x7f2e4819d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.258+0000 7f2e4c977700 1 --2- 172.21.15.52:0/16455538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e48100260 0x7f2e48195f90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.258+0000 7f2e4c977700 1 --2- 172.21.15.52:0/16455538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2e48101450 0x7f2e481964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.258+0000 7f2e4c977700 1 -- 172.21.15.52:0/16455538 >> 172.21.15.52:0/16455538 conn(0x7f2e480fb840 msgr2=0x7f2e480fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:10.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.258+0000 7f2e4c977700 1 -- 172.21.15.52:0/16455538 shutdown_connections 2023-11-27T15:17:10.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.258+0000 7f2e4c977700 1 -- 172.21.15.52:0/16455538 wait complete. 2023-11-27T15:17:10.281 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-27T15:17:10.282 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:09 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3519635373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:17:10.282 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:09 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3399983092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:17:10.282 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:09 smithi150 ceph-mon[100559]: pgmap v288: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 5 op/s; 127 B/s, 1 keys/s, 0 objects/s recovering 2023-11-27T15:17:10.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:09 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3519635373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:17:10.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:09 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3399983092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:17:10.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:09 smithi102 ceph-mon[100296]: pgmap v288: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 5 op/s; 127 B/s, 1 keys/s, 0 objects/s recovering 2023-11-27T15:17:10.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.403+0000 7f6d76500700 1 -- 172.21.15.52:0/1732490060 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d70100350 msgr2=0x7f6d701007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:10.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.403+0000 7f6d76500700 1 --2- 172.21.15.52:0/1732490060 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d70100350 0x7f6d701007a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009990 tx=0x7f6d58009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.403+0000 7f6d76500700 1 -- 172.21.15.52:0/1732490060 shutdown_connections 2023-11-27T15:17:10.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.403+0000 7f6d76500700 1 --2- 172.21.15.52:0/1732490060 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d70100ce0 0x7f6d701075a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.403+0000 7f6d76500700 1 --2- 172.21.15.52:0/1732490060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d700ff160 0x7f6d700ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.403+0000 7f6d76500700 1 --2- 172.21.15.52:0/1732490060 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d70100350 0x7f6d701007a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.403+0000 7f6d76500700 1 -- 172.21.15.52:0/1732490060 >> 172.21.15.52:0/1732490060 conn(0x7f6d700fa780 msgr2=0x7f6d700fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:10.410 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.403+0000 7f6d76500700 1 -- 172.21.15.52:0/1732490060 shutdown_connections 2023-11-27T15:17:10.410 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.403+0000 7f6d76500700 1 -- 172.21.15.52:0/1732490060 wait complete. 2023-11-27T15:17:10.410 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.404+0000 7f6d76500700 1 Processor -- start 2023-11-27T15:17:10.410 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.404+0000 7f6d76500700 1 -- start start 2023-11-27T15:17:10.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.404+0000 7f6d76500700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d700ff160 0x7f6d70195f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.404+0000 7f6d76500700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d70100350 0x7f6d70196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.404+0000 7f6d76500700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d70100ce0 0x7f6d7019d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.404+0000 7f6d76500700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7019dad0 con 0x7f6d70100350 2023-11-27T15:17:10.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.404+0000 7f6d76500700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7019dc10 con 0x7f6d70100ce0 2023-11-27T15:17:10.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.404+0000 7f6d76500700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7019dda0 con 0x7f6d700ff160 2023-11-27T15:17:10.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.404+0000 7f6d6f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d70100350 0x7f6d70196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.404+0000 7f6d6f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d70100350 0x7f6d70196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50926/0 (socket says 172.21.15.52:50926) 2023-11-27T15:17:10.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.404+0000 7f6d6f7fe700 1 -- 172.21.15.52:0/971436810 learned_addr learned my addr 172.21.15.52:0/971436810 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:10.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.404+0000 7f6d74a9d700 1 --2- 172.21.15.52:0/971436810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d70100ce0 0x7f6d7019d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.404+0000 7f6d6ffff700 1 --2- 172.21.15.52:0/971436810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d700ff160 0x7f6d70195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.414 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.404+0000 7f6d6f7fe700 1 -- 172.21.15.52:0/971436810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d700ff160 msgr2=0x7f6d70195f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:10.414 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.404+0000 7f6d6f7fe700 1 --2- 172.21.15.52:0/971436810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d700ff160 0x7f6d70195f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.414 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.404+0000 7f6d6f7fe700 1 -- 172.21.15.52:0/971436810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d70100ce0 msgr2=0x7f6d7019d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:10.414 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.404+0000 7f6d6f7fe700 1 --2- 172.21.15.52:0/971436810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d70100ce0 0x7f6d7019d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.414 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.404+0000 7f6d6f7fe700 1 -- 172.21.15.52:0/971436810 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d580095f0 con 0x7f6d70100350 2023-11-27T15:17:10.414 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.405+0000 7f6d6f7fe700 1 --2- 172.21.15.52:0/971436810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d70100350 0x7f6d70196490 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6d58004760 tx=0x7f6d58004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:10.414 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.405+0000 7f6d6d7fa700 1 -- 172.21.15.52:0/971436810 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5801d070 con 0x7f6d70100350 2023-11-27T15:17:10.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.405+0000 7f6d6d7fa700 1 -- 172.21.15.52:0/971436810 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d58004cf0 con 0x7f6d70100350 2023-11-27T15:17:10.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.405+0000 7f6d76500700 1 -- 172.21.15.52:0/971436810 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d7010aac0 con 0x7f6d70100350 2023-11-27T15:17:10.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.405+0000 7f6d6d7fa700 1 -- 172.21.15.52:0/971436810 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d58005280 con 0x7f6d70100350 2023-11-27T15:17:10.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.405+0000 7f6d76500700 1 -- 172.21.15.52:0/971436810 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d7010a930 con 0x7f6d70100350 2023-11-27T15:17:10.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.407+0000 7f6d6d7fa700 1 -- 172.21.15.52:0/971436810 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f6d58005420 con 0x7f6d70100350 2023-11-27T15:17:10.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.408+0000 7f6d6d7fa700 1 --2- 172.21.15.52:0/971436810 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6d5c0a1260 0x7f6d5c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.408+0000 7f6d6ffff700 1 --2- 172.21.15.52:0/971436810 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6d5c0a1260 0x7f6d5c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.408+0000 7f6d6d7fa700 1 -- 172.21.15.52:0/971436810 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6d580cdb60 con 0x7f6d70100350 2023-11-27T15:17:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.408+0000 7f6d6ffff700 1 --2- 172.21.15.52:0/971436810 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6d5c0a1260 0x7f6d5c0a3710 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6d60005fd0 tx=0x7f6d60005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.408+0000 7f6d8268cb80 1 Processor -- start 2023-11-27T15:17:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.408+0000 7f6d8268cb80 1 -- start start 2023-11-27T15:17:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.408+0000 7f6d8268cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629498a1000 0x5629498a1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.409+0000 7f6d8268cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629498a1950 0x5629498a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.409+0000 7f6d8268cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629498a24c0 0x5629498a2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.409+0000 7f6d8268cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629497a2010 con 0x5629498a1000 2023-11-27T15:17:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.409+0000 7f6d8268cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629497f4850 con 0x5629498a24c0 2023-11-27T15:17:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.409+0000 7f6d8268cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629497edce0 con 0x5629498a1950 2023-11-27T15:17:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.409+0000 7f6d6ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629498a1000 0x5629498a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.423 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.409+0000 7f6d6ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629498a1000 0x5629498a1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50932/0 (socket says 172.21.15.52:50932) 2023-11-27T15:17:10.423 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.409+0000 7f6d6ffff700 1 -- 172.21.15.52:0/610392355 learned_addr learned my addr 172.21.15.52:0/610392355 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:10.423 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.409+0000 7f6d74a9d700 1 --2- 172.21.15.52:0/610392355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629498a24c0 0x5629498a2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.423 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.409+0000 7f6d6f7fe700 1 --2- 172.21.15.52:0/610392355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629498a1950 0x5629498a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.423 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.409+0000 7f6d6ffff700 1 -- 172.21.15.52:0/610392355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629498a1950 msgr2=0x5629498a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:10.423 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.409+0000 7f6d6ffff700 1 --2- 172.21.15.52:0/610392355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629498a1950 0x5629498a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.423 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.409+0000 7f6d6ffff700 1 -- 172.21.15.52:0/610392355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629498a24c0 msgr2=0x5629498a2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:10.423 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.409+0000 7f6d6ffff700 1 --2- 172.21.15.52:0/610392355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629498a24c0 0x5629498a2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.423 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.409+0000 7f6d6ffff700 1 -- 172.21.15.52:0/610392355 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56294966f5e0 con 0x5629498a1000 2023-11-27T15:17:10.423 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.409+0000 7f6d74a9d700 1 --2- 172.21.15.52:0/610392355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629498a24c0 0x5629498a2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:17:10.423 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.409+0000 7f6d6ffff700 1 --2- 172.21.15.52:0/610392355 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629498a1000 0x5629498a1410 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6d6000ec60 tx=0x7f6d60011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad308eede939150 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:10.424 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.409+0000 7f6d667fc700 1 -- 172.21.15.52:0/610392355 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d60010cd0 con 0x5629498a1000 2023-11-27T15:17:10.424 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.409+0000 7f6d667fc700 1 -- 172.21.15.52:0/610392355 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d60010e30 con 0x5629498a1000 2023-11-27T15:17:10.424 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.409+0000 7f6d667fc700 1 -- 172.21.15.52:0/610392355 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6000f6b0 con 0x5629498a1000 2023-11-27T15:17:10.424 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.410+0000 7f6d8268cb80 1 -- 172.21.15.52:0/610392355 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629498a1000 msgr2=0x5629498a1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:10.424 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.410+0000 7f6d8268cb80 1 --2- 172.21.15.52:0/610392355 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629498a1000 0x5629498a1410 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6d6000ec60 tx=0x7f6d60011000 comp rx=0 tx=0).stop 2023-11-27T15:17:10.424 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.410+0000 7f6d8268cb80 1 -- 172.21.15.52:0/610392355 shutdown_connections 2023-11-27T15:17:10.424 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.410+0000 7f6d8268cb80 1 --2- 172.21.15.52:0/610392355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629498a24c0 0x5629498a2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.424 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.410+0000 7f6d8268cb80 1 --2- 172.21.15.52:0/610392355 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629498a1000 0x5629498a1410 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.424 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.410+0000 7f6d8268cb80 1 --2- 172.21.15.52:0/610392355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629498a1950 0x5629498a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.424 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.410+0000 7f6d8268cb80 1 -- 172.21.15.52:0/610392355 >> 172.21.15.52:0/610392355 conn(0x562949892310 msgr2=0x562949897620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:10.424 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.410+0000 7f6d8268cb80 1 -- 172.21.15.52:0/610392355 shutdown_connections 2023-11-27T15:17:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.410+0000 7f6d8268cb80 1 -- 172.21.15.52:0/610392355 wait complete. 2023-11-27T15:17:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.410+0000 7f6d8268cb80 1 -- start start 2023-11-27T15:17:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.411+0000 7f6d8268cb80 1 Processor -- start 2023-11-27T15:17:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.411+0000 7f6d8268cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629498a1000 0x5629498aa950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.411+0000 7f6d8268cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629498a1950 0x5629498aae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.411+0000 7f6d8268cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629498a24c0 0x5629498b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.411+0000 7f6d8268cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294978f890 con 0x5629498a1000 2023-11-27T15:17:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.411+0000 7f6d8268cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294978f530 con 0x5629498a1950 2023-11-27T15:17:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.411+0000 7f6d8268cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629496718d0 con 0x5629498a24c0 2023-11-27T15:17:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.411+0000 7f6d6ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629498a1000 0x5629498aa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.411+0000 7f6d8268cb80 10 client.0 _ll_register_callbacks cb 0x5629497ad730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:17:10.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.411+0000 7f6d6ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629498a1000 0x5629498aa950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50938/0 (socket says 172.21.15.52:50938) 2023-11-27T15:17:10.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.411+0000 7f6d6ffff700 1 -- 172.21.15.52:0/3717466575 learned_addr learned my addr 172.21.15.52:0/3717466575 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:10.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.411+0000 7f6d6f7fe700 1 --2- 172.21.15.52:0/3717466575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629498a1950 0x5629498aae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.411+0000 7f6d74a9d700 1 --2- 172.21.15.52:0/3717466575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629498a24c0 0x5629498b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.411+0000 7f6d6ffff700 1 -- 172.21.15.52:0/3717466575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629498a24c0 msgr2=0x5629498b1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:10.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.411+0000 7f6d6ffff700 1 --2- 172.21.15.52:0/3717466575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629498a24c0 0x5629498b1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.411+0000 7f6d6ffff700 1 -- 172.21.15.52:0/3717466575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629498a1950 msgr2=0x5629498aae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:10.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.411+0000 7f6d6ffff700 1 --2- 172.21.15.52:0/3717466575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629498a1950 0x5629498aae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.411+0000 7f6d6ffff700 1 -- 172.21.15.52:0/3717466575 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d6000e910 con 0x5629498a1000 2023-11-27T15:17:10.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.412+0000 7f6d74a9d700 1 --2- 172.21.15.52:0/3717466575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629498a24c0 0x5629498b1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:17:10.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.412+0000 7f6d6f7fe700 1 --2- 172.21.15.52:0/3717466575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629498a1950 0x5629498aae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:17:10.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.412+0000 7f6d6ffff700 1 --2- 172.21.15.52:0/3717466575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629498a1000 0x5629498aa950 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6d6000a4a0 tx=0x7f6d60010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97bd084598193c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:10.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.412+0000 7f6d4f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:17:10.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.412+0000 7f6d4f7fe700 1 -- 172.21.15.52:0/3717466575 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6002d070 con 0x5629498a1000 2023-11-27T15:17:10.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.412+0000 7f6d4f7fe700 1 -- 172.21.15.52:0/3717466575 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6d60004aa0 con 0x5629498a1000 2023-11-27T15:17:10.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.412+0000 7f6d8268cb80 1 -- 172.21.15.52:0/3717466575 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56294988fad0 con 0x5629498a1000 2023-11-27T15:17:10.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.412+0000 7f6d4f7fe700 1 -- 172.21.15.52:0/3717466575 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d60004c60 con 0x5629498a1000 2023-11-27T15:17:10.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.412+0000 7f6d8268cb80 10 client.15042 fetch_fsmap learned FSMap version 23 2023-11-27T15:17:10.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.412+0000 7f6d8268cb80 1 -- 172.21.15.52:0/3717466575 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5629497b4140 con 0x5629498a1000 2023-11-27T15:17:10.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.412+0000 7f6d4f7fe700 1 -- 172.21.15.52:0/3717466575 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56294988fad0 con 0x5629498a1000 2023-11-27T15:17:10.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.412+0000 7f6d4f7fe700 1 -- 172.21.15.52:0/3717466575 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6d60003ac0 con 0x5629498a1000 2023-11-27T15:17:10.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.412+0000 7f6d8268cb80 10 client.15042 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:17:10.428 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.412+0000 7f6d8268cb80 10 client.15042 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-27T15:17:10.428 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.412+0000 7f6d8268cb80 20 client.15042 populate_metadata read hostname 'smithi052' 2023-11-27T15:17:10.428 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.412+0000 7f6d8268cb80 1 --2- 172.21.15.52:0/3717466575 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5629498b5a00 0x5629498b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.428 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.412+0000 7f6d8268cb80 4 client.15042 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:17:10.428 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.412+0000 7f6d8268cb80 1 -- 172.21.15.52:0/3717466575 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629497b0050 con 0x5629498b5a00 2023-11-27T15:17:10.428 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.413+0000 7f6d6f7fe700 1 --2- 172.21.15.52:0/3717466575 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5629498b5a00 0x5629498b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.428 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.413+0000 7f6d6f7fe700 1 --2- 172.21.15.52:0/3717466575 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5629498b5a00 0x5629498b4e30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac5a8ebba939f2ae server_cookie=d0776364da62d085 in_seq=0 out_seq=0 2023-11-27T15:17:10.428 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.413+0000 7f6d4f7fe700 10 client.15042 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:17:10.428 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.414+0000 7f6d4f7fe700 1 -- 172.21.15.52:0/3717466575 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5629497b0050 con 0x5629498b5a00 2023-11-27T15:17:10.428 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.414+0000 7f6d4f7fe700 10 client.15042 handle_command_reply: tid=0 2023-11-27T15:17:10.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.414+0000 7f6d8268cb80 1 client.15042 shutdown 2023-11-27T15:17:10.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.414+0000 7f6d8268cb80 1 -- 172.21.15.52:0/3717466575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629498a1000 msgr2=0x5629498aa950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:10.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.414+0000 7f6d8268cb80 1 --2- 172.21.15.52:0/3717466575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629498a1000 0x5629498aa950 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6d6000a4a0 tx=0x7f6d60010c20 comp rx=0 tx=0).stop 2023-11-27T15:17:10.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.414+0000 7f6d8268cb80 1 -- 172.21.15.52:0/3717466575 shutdown_connections 2023-11-27T15:17:10.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.414+0000 7f6d8268cb80 1 --2- 172.21.15.52:0/3717466575 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5629498b5a00 0x5629498b4e30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.414+0000 7f6d8268cb80 1 --2- 172.21.15.52:0/3717466575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629498a24c0 0x5629498b1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.414+0000 7f6d8268cb80 1 --2- 172.21.15.52:0/3717466575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629498a1000 0x5629498aa950 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.414+0000 7f6d4f7fe700 0 client.15042 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:17:10.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.414+0000 7f6d8268cb80 1 --2- 172.21.15.52:0/3717466575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629498a1950 0x5629498aae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.414+0000 7f6d8268cb80 1 -- 172.21.15.52:0/3717466575 >> 172.21.15.52:0/3717466575 conn(0x562949892310 msgr2=0x562949896d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:10.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.414+0000 7f6d8268cb80 1 -- 172.21.15.52:0/3717466575 shutdown_connections 2023-11-27T15:17:10.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.414+0000 7f6d8268cb80 1 -- 172.21.15.52:0/3717466575 wait complete. 2023-11-27T15:17:10.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.414+0000 7f6d8268cb80 20 client.15042 trim_cache size 0 max 16384 2023-11-27T15:17:10.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.432+0000 7f6d8268cb80 1 Processor -- start 2023-11-27T15:17:10.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.432+0000 7f6d8268cb80 1 -- start start 2023-11-27T15:17:10.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.432+0000 7f6d8268cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629498a1000 0x5629498a8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.432+0000 7f6d8268cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629498a1950 0x5629498a8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.432+0000 7f6d8268cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629498a24c0 0x5629498b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.432+0000 7f6d8268cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x562949797730 con 0x5629498a1000 2023-11-27T15:17:10.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.432+0000 7f6d8268cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629496f9ed0 con 0x5629498a24c0 2023-11-27T15:17:10.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.432+0000 7f6d8268cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629497abfa0 con 0x5629498a1950 2023-11-27T15:17:10.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.433+0000 7f6d6ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629498a1000 0x5629498a8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.433+0000 7f6d6ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629498a1000 0x5629498a8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50946/0 (socket says 172.21.15.52:50946) 2023-11-27T15:17:10.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.433+0000 7f6d6ffff700 1 -- 172.21.15.52:0/230423011 learned_addr learned my addr 172.21.15.52:0/230423011 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:10.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.433+0000 7f6d6f7fe700 1 --2- 172.21.15.52:0/230423011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629498a1950 0x5629498a8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.433+0000 7f6d6ffff700 1 -- 172.21.15.52:0/230423011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629498a1950 msgr2=0x5629498a8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:10.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.433+0000 7f6d6ffff700 1 --2- 172.21.15.52:0/230423011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629498a1950 0x5629498a8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.433+0000 7f6d6ffff700 1 -- 172.21.15.52:0/230423011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629498a24c0 msgr2=0x5629498b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:10.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.433+0000 7f6d6ffff700 1 --2- 172.21.15.52:0/230423011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629498a24c0 0x5629498b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.433+0000 7f6d6ffff700 1 -- 172.21.15.52:0/230423011 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629497b3e90 con 0x5629498a1000 2023-11-27T15:17:10.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.433+0000 7f6d6f7fe700 1 --2- 172.21.15.52:0/230423011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629498a1950 0x5629498a8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:17:10.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.433+0000 7f6d6ffff700 1 --2- 172.21.15.52:0/230423011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629498a1000 0x5629498a8880 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6d60005830 tx=0x7f6d60010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dae37c6bae7549f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:10.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.433+0000 7f6d66ffd700 1 -- 172.21.15.52:0/230423011 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6002d070 con 0x5629498a1000 2023-11-27T15:17:10.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.433+0000 7f6d66ffd700 1 -- 172.21.15.52:0/230423011 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d6002e040 con 0x5629498a1000 2023-11-27T15:17:10.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.433+0000 7f6d66ffd700 1 -- 172.21.15.52:0/230423011 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d60003680 con 0x5629498a1000 2023-11-27T15:17:10.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.434+0000 7f6d8268cb80 1 -- 172.21.15.52:0/230423011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629498a1000 msgr2=0x5629498a8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:10.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.434+0000 7f6d8268cb80 1 --2- 172.21.15.52:0/230423011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629498a1000 0x5629498a8880 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6d60005830 tx=0x7f6d60010dc0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.434+0000 7f6d8268cb80 1 -- 172.21.15.52:0/230423011 shutdown_connections 2023-11-27T15:17:10.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.434+0000 7f6d8268cb80 1 --2- 172.21.15.52:0/230423011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629498a24c0 0x5629498b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.434+0000 7f6d8268cb80 1 --2- 172.21.15.52:0/230423011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629498a1000 0x5629498a8880 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.434+0000 7f6d8268cb80 1 --2- 172.21.15.52:0/230423011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629498a1950 0x5629498a8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.434+0000 7f6d8268cb80 1 -- 172.21.15.52:0/230423011 >> 172.21.15.52:0/230423011 conn(0x562949892310 msgr2=0x5629498b4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:10.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.434+0000 7f6d8268cb80 1 -- 172.21.15.52:0/230423011 shutdown_connections 2023-11-27T15:17:10.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.434+0000 7f6d8268cb80 1 -- 172.21.15.52:0/230423011 wait complete. 2023-11-27T15:17:10.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.434+0000 7f6d8268cb80 1 -- start start 2023-11-27T15:17:10.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.434+0000 7f6d8268cb80 1 Processor -- start 2023-11-27T15:17:10.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.434+0000 7f6d8268cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629498a1000 0x5629498acc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.434+0000 7f6d8268cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629498a1950 0x5629498ad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.435+0000 7f6d8268cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629498a24c0 0x5629499098a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.435+0000 7f6d8268cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629498ad710 con 0x5629498a1000 2023-11-27T15:17:10.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.435+0000 7f6d8268cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629498ad880 con 0x5629498a1950 2023-11-27T15:17:10.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.435+0000 7f6d8268cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562949909de0 con 0x5629498a24c0 2023-11-27T15:17:10.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.435+0000 7f6d8268cb80 10 client.0 _ll_register_callbacks cb 0x7f6d68003a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:17:10.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.435+0000 7f6d6ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629498a1000 0x5629498acc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.435+0000 7f6d6ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629498a1000 0x5629498acc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50952/0 (socket says 172.21.15.52:50952) 2023-11-27T15:17:10.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.435+0000 7f6d6ffff700 1 -- 172.21.15.52:0/1529436487 learned_addr learned my addr 172.21.15.52:0/1529436487 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:10.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.435+0000 7f6d6f7fe700 1 --2- 172.21.15.52:0/1529436487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629498a1950 0x5629498ad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.435+0000 7f6d74a9d700 1 --2- 172.21.15.52:0/1529436487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629498a24c0 0x5629499098a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.435+0000 7f6d6ffff700 1 -- 172.21.15.52:0/1529436487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629498a24c0 msgr2=0x5629499098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:10.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.435+0000 7f6d6ffff700 1 --2- 172.21.15.52:0/1529436487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629498a24c0 0x5629499098a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.435+0000 7f6d6ffff700 1 -- 172.21.15.52:0/1529436487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629498a1950 msgr2=0x5629498ad1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:10.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.435+0000 7f6d6ffff700 1 --2- 172.21.15.52:0/1529436487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629498a1950 0x5629498ad1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.435+0000 7f6d6ffff700 1 -- 172.21.15.52:0/1529436487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629497b4140 con 0x5629498a1000 2023-11-27T15:17:10.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.435+0000 7f6d74a9d700 1 --2- 172.21.15.52:0/1529436487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629498a24c0 0x5629499098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:17:10.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.435+0000 7f6d6ffff700 1 --2- 172.21.15.52:0/1529436487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629498a1000 0x5629498acc90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6d6000a4a0 tx=0x7f6d600053c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c829f21b54b3ef server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:10.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.435+0000 7f6d4f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:17:10.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.435+0000 7f6d4f7fe700 1 -- 172.21.15.52:0/1529436487 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6002d040 con 0x5629498a1000 2023-11-27T15:17:10.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.435+0000 7f6d4f7fe700 1 -- 172.21.15.52:0/1529436487 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6d60004aa0 con 0x5629498a1000 2023-11-27T15:17:10.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.435+0000 7f6d8268cb80 1 -- 172.21.15.52:0/1529436487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d60010e30 con 0x5629498a1000 2023-11-27T15:17:10.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.435+0000 7f6d4f7fe700 1 -- 172.21.15.52:0/1529436487 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d60004ca0 con 0x5629498a1000 2023-11-27T15:17:10.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.435+0000 7f6d8268cb80 10 client.15048 fetch_fsmap learned FSMap version 23 2023-11-27T15:17:10.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.435+0000 7f6d8268cb80 1 -- 172.21.15.52:0/1529436487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x562949604c10 con 0x5629498a1000 2023-11-27T15:17:10.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.436+0000 7f6d4f7fe700 1 -- 172.21.15.52:0/1529436487 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d60010e30 con 0x5629498a1000 2023-11-27T15:17:10.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.436+0000 7f6d4f7fe700 1 -- 172.21.15.52:0/1529436487 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6d6001d9f0 con 0x5629498a1000 2023-11-27T15:17:10.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.436+0000 7f6d8268cb80 10 client.15048 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:17:10.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.436+0000 7f6d8268cb80 10 client.15048 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-27T15:17:10.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.436+0000 7f6d8268cb80 20 client.15048 populate_metadata read hostname 'smithi052' 2023-11-27T15:17:10.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.436+0000 7f6d8268cb80 1 --2- 172.21.15.52:0/1529436487 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x56294990e670 0x56294990b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.436+0000 7f6d8268cb80 4 client.15048 mds_command: new command op to 24536 tid=0[{"prefix": "status"}] 2023-11-27T15:17:10.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.436+0000 7f6d8268cb80 1 -- 172.21.15.52:0/1529436487 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5629497bc610 con 0x56294990e670 2023-11-27T15:17:10.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.436+0000 7f6d6f7fe700 1 --2- 172.21.15.52:0/1529436487 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x56294990e670 0x56294990b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.450 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.436+0000 7f6d4f7fe700 10 client.15048 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:17:10.450 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.436+0000 7f6d6f7fe700 1 --2- 172.21.15.52:0/1529436487 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x56294990e670 0x56294990b020 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7488a2a91abd3fa1 server_cookie=9a492b980a77cbbd in_seq=0 out_seq=0 2023-11-27T15:17:10.450 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.437+0000 7f6d4f7fe700 1 -- 172.21.15.52:0/1529436487 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5629497bc610 con 0x56294990e670 2023-11-27T15:17:10.450 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.437+0000 7f6d4f7fe700 10 client.15048 handle_command_reply: tid=0 2023-11-27T15:17:10.450 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.437+0000 7f6d8268cb80 1 client.15048 shutdown 2023-11-27T15:17:10.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.437+0000 7f6d8268cb80 1 -- 172.21.15.52:0/1529436487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629498a1000 msgr2=0x5629498acc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:10.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.437+0000 7f6d8268cb80 1 --2- 172.21.15.52:0/1529436487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629498a1000 0x5629498acc90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6d6000a4a0 tx=0x7f6d600053c0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.437+0000 7f6d8268cb80 1 -- 172.21.15.52:0/1529436487 shutdown_connections 2023-11-27T15:17:10.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.437+0000 7f6d8268cb80 1 --2- 172.21.15.52:0/1529436487 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x56294990e670 0x56294990b020 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.437+0000 7f6d8268cb80 1 --2- 172.21.15.52:0/1529436487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5629498a24c0 0x5629499098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.437+0000 7f6d4f7fe700 0 client.15048 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:17:10.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.437+0000 7f6d8268cb80 1 --2- 172.21.15.52:0/1529436487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5629498a1000 0x5629498acc90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.437+0000 7f6d8268cb80 1 --2- 172.21.15.52:0/1529436487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629498a1950 0x5629498ad1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.437+0000 7f6d8268cb80 1 -- 172.21.15.52:0/1529436487 >> 172.21.15.52:0/1529436487 conn(0x562949892310 msgr2=0x5629498a8880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:10.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.437+0000 7f6d8268cb80 1 -- 172.21.15.52:0/1529436487 shutdown_connections 2023-11-27T15:17:10.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.437+0000 7f6d8268cb80 1 -- 172.21.15.52:0/1529436487 wait complete. 2023-11-27T15:17:10.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.437+0000 7f6d8268cb80 20 client.15048 trim_cache size 0 max 16384 2023-11-27T15:17:10.452 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:17:10.453 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "c6c7df88-8d36-11ee-95a2-87774f69a715", 2023-11-27T15:17:10.453 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 0, 2023-11-27T15:17:10.453 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24536, 2023-11-27T15:17:10.453 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2023-11-27T15:17:10.453 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2023-11-27T15:17:10.453 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2023-11-27T15:17:10.453 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 61.115873215000001, 2023-11-27T15:17:10.453 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 23, 2023-11-27T15:17:10.454 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 85, 2023-11-27T15:17:10.454 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 83, 2023-11-27T15:17:10.454 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 65.798432786000006 2023-11-27T15:17:10.454 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:17:10.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.437+0000 7f6d76500700 1 -- 172.21.15.52:0/971436810 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6d5c0a1260 msgr2=0x7f6d5c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:10.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.437+0000 7f6d76500700 1 --2- 172.21.15.52:0/971436810 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6d5c0a1260 0x7f6d5c0a3710 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6d60005fd0 tx=0x7f6d60005dd0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.437+0000 7f6d76500700 1 -- 172.21.15.52:0/971436810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d70100350 msgr2=0x7f6d70196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:10.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.437+0000 7f6d76500700 1 --2- 172.21.15.52:0/971436810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d70100350 0x7f6d70196490 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6d58004760 tx=0x7f6d58004790 comp rx=0 tx=0).stop 2023-11-27T15:17:10.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.437+0000 7f6d76500700 1 -- 172.21.15.52:0/971436810 shutdown_connections 2023-11-27T15:17:10.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.438+0000 7f6d76500700 1 --2- 172.21.15.52:0/971436810 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6d5c0a1260 0x7f6d5c0a3710 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.438+0000 7f6d76500700 1 --2- 172.21.15.52:0/971436810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d70100ce0 0x7f6d7019d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.438+0000 7f6d76500700 1 --2- 172.21.15.52:0/971436810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d700ff160 0x7f6d70195f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.438+0000 7f6d76500700 1 --2- 172.21.15.52:0/971436810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d70100350 0x7f6d70196490 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.438+0000 7f6d76500700 1 -- 172.21.15.52:0/971436810 >> 172.21.15.52:0/971436810 conn(0x7f6d700fa780 msgr2=0x7f6d70068fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:10.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.438+0000 7f6d76500700 1 -- 172.21.15.52:0/971436810 shutdown_connections 2023-11-27T15:17:10.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:10.438+0000 7f6d76500700 1 -- 172.21.15.52:0/971436810 wait complete. 2023-11-27T15:17:10.457 INFO:tasks.cephadm:Setup complete, yielding 2023-11-27T15:17:10.458 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-11-27T15:17:10.467 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi052.front.sepia.ceph.com 2023-11-27T15:17:10.467 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- bash -c 'ceph orch status' 2023-11-27T15:17:11.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:10 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2900609164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:17:11.120 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:17:11.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:10 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2900609164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:17:11.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:10 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2900609164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:17:12.065 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:11 smithi052 ceph-mon[99056]: pgmap v289: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3 op/s; 127 B/s, 1 keys/s, 0 objects/s recovering 2023-11-27T15:17:12.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:11 smithi150 ceph-mon[100559]: pgmap v289: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3 op/s; 127 B/s, 1 keys/s, 0 objects/s recovering 2023-11-27T15:17:12.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:11 smithi102 ceph-mon[100296]: pgmap v289: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3 op/s; 127 B/s, 1 keys/s, 0 objects/s recovering 2023-11-27T15:17:12.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.564+0000 7ff40d30a700 1 -- 172.21.15.52:0/1110009383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4081002f0 msgr2=0x7ff408100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:12.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.564+0000 7ff40d30a700 1 --2- 172.21.15.52:0/1110009383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4081002f0 0x7ff408100700 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8009990 tx=0x7ff3f8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:12.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.565+0000 7ff40d30a700 1 -- 172.21.15.52:0/1110009383 shutdown_connections 2023-11-27T15:17:12.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.565+0000 7ff40d30a700 1 --2- 172.21.15.52:0/1110009383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff408101e80 0x7ff408108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:12.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.565+0000 7ff40d30a700 1 --2- 172.21.15.52:0/1110009383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4081002f0 0x7ff408100700 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:12.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.565+0000 7ff40d30a700 1 --2- 172.21.15.52:0/1110009383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4081014f0 0x7ff408101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:12.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.565+0000 7ff40d30a700 1 -- 172.21.15.52:0/1110009383 >> 172.21.15.52:0/1110009383 conn(0x7ff4080fb900 msgr2=0x7ff4080fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:12.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.565+0000 7ff40d30a700 1 -- 172.21.15.52:0/1110009383 shutdown_connections 2023-11-27T15:17:12.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.565+0000 7ff40d30a700 1 -- 172.21.15.52:0/1110009383 wait complete. 2023-11-27T15:17:12.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.565+0000 7ff40d30a700 1 Processor -- start 2023-11-27T15:17:12.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.565+0000 7ff40d30a700 1 -- start start 2023-11-27T15:17:12.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.566+0000 7ff40d30a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4081002f0 0x7ff408074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:12.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.566+0000 7ff40d30a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4081014f0 0x7ff408072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:12.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.566+0000 7ff40d30a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff408101e80 0x7ff408073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:12.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.566+0000 7ff40d30a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff408073ba0 con 0x7ff408101e80 2023-11-27T15:17:12.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.566+0000 7ff40d30a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff408073d10 con 0x7ff4081014f0 2023-11-27T15:17:12.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.566+0000 7ff40d30a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4081a0090 con 0x7ff4081002f0 2023-11-27T15:17:12.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.566+0000 7ff4077fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff408101e80 0x7ff408073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:12.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.566+0000 7ff4077fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff408101e80 0x7ff408073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50970/0 (socket says 172.21.15.52:50970) 2023-11-27T15:17:12.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.566+0000 7ff4077fe700 1 -- 172.21.15.52:0/3624074810 learned_addr learned my addr 172.21.15.52:0/3624074810 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:12.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.566+0000 7ff406ffd700 1 --2- 172.21.15.52:0/3624074810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4081002f0 0x7ff408074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:12.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.566+0000 7ff4067fc700 1 --2- 172.21.15.52:0/3624074810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4081014f0 0x7ff408072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:12.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.566+0000 7ff4077fe700 1 -- 172.21.15.52:0/3624074810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4081002f0 msgr2=0x7ff408074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:12.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.566+0000 7ff4077fe700 1 --2- 172.21.15.52:0/3624074810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4081002f0 0x7ff408074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:12.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.566+0000 7ff4077fe700 1 -- 172.21.15.52:0/3624074810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4081014f0 msgr2=0x7ff408072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:12.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.566+0000 7ff4077fe700 1 --2- 172.21.15.52:0/3624074810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4081014f0 0x7ff408072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:12.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.566+0000 7ff4077fe700 1 -- 172.21.15.52:0/3624074810 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f80095f0 con 0x7ff408101e80 2023-11-27T15:17:12.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.566+0000 7ff4077fe700 1 --2- 172.21.15.52:0/3624074810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff408101e80 0x7ff408073420 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff400007ee0 tx=0x7ff40000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:12.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.566+0000 7ff3effff700 1 -- 172.21.15.52:0/3624074810 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff400017070 con 0x7ff408101e80 2023-11-27T15:17:12.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.566+0000 7ff40d30a700 1 -- 172.21.15.52:0/3624074810 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff40810bc40 con 0x7ff408101e80 2023-11-27T15:17:12.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.566+0000 7ff3effff700 1 -- 172.21.15.52:0/3624074810 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff40000f040 con 0x7ff408101e80 2023-11-27T15:17:12.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.566+0000 7ff3effff700 1 -- 172.21.15.52:0/3624074810 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff400005580 con 0x7ff408101e80 2023-11-27T15:17:12.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.566+0000 7ff40d30a700 1 -- 172.21.15.52:0/3624074810 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff40810bab0 con 0x7ff408101e80 2023-11-27T15:17:12.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.567+0000 7ff40d30a700 1 -- 172.21.15.52:0/3624074810 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4081a0770 con 0x7ff408101e80 2023-11-27T15:17:12.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.568+0000 7ff3effff700 1 -- 172.21.15.52:0/3624074810 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff40000db20 con 0x7ff408101e80 2023-11-27T15:17:12.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.569+0000 7ff3effff700 1 --2- 172.21.15.52:0/3624074810 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff3f00a11e0 0x7ff3f00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:12.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.569+0000 7ff406ffd700 1 --2- 172.21.15.52:0/3624074810 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff3f00a11e0 0x7ff3f00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:12.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.569+0000 7ff3effff700 1 -- 172.21.15.52:0/3624074810 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff4000cc830 con 0x7ff408101e80 2023-11-27T15:17:12.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.570+0000 7ff406ffd700 1 --2- 172.21.15.52:0/3624074810 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff3f00a11e0 0x7ff3f00a3690 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800b2e0 tx=0x7ff3f8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:12.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.572+0000 7ff3effff700 1 -- 172.21.15.52:0/3624074810 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff400095e00 con 0x7ff408101e80 2023-11-27T15:17:12.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.793+0000 7ff40d30a700 1 -- 172.21.15.52:0/3624074810 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff408060f30 con 0x7ff3f00a11e0 2023-11-27T15:17:12.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.794+0000 7ff3effff700 1 -- 172.21.15.52:0/3624074810 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7ff408060f30 con 0x7ff3f00a11e0 2023-11-27T15:17:12.799 INFO:teuthology.orchestra.run.smithi052.stdout:Backend: cephadm 2023-11-27T15:17:12.799 INFO:teuthology.orchestra.run.smithi052.stdout:Available: Yes 2023-11-27T15:17:12.799 INFO:teuthology.orchestra.run.smithi052.stdout:Paused: No 2023-11-27T15:17:12.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.796+0000 7ff40d30a700 1 -- 172.21.15.52:0/3624074810 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff3f00a11e0 msgr2=0x7ff3f00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:12.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.796+0000 7ff40d30a700 1 --2- 172.21.15.52:0/3624074810 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff3f00a11e0 0x7ff3f00a3690 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800b2e0 tx=0x7ff3f8005fb0 comp rx=0 tx=0).stop 2023-11-27T15:17:12.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.796+0000 7ff40d30a700 1 -- 172.21.15.52:0/3624074810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff408101e80 msgr2=0x7ff408073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:12.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.796+0000 7ff40d30a700 1 --2- 172.21.15.52:0/3624074810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff408101e80 0x7ff408073420 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff400007ee0 tx=0x7ff40000d3b0 comp rx=0 tx=0).stop 2023-11-27T15:17:12.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.796+0000 7ff40d30a700 1 -- 172.21.15.52:0/3624074810 shutdown_connections 2023-11-27T15:17:12.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.796+0000 7ff40d30a700 1 --2- 172.21.15.52:0/3624074810 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff3f00a11e0 0x7ff3f00a3690 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:12.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.796+0000 7ff40d30a700 1 --2- 172.21.15.52:0/3624074810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff408101e80 0x7ff408073420 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:12.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.796+0000 7ff40d30a700 1 --2- 172.21.15.52:0/3624074810 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4081002f0 0x7ff408074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:12.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.796+0000 7ff40d30a700 1 --2- 172.21.15.52:0/3624074810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4081014f0 0x7ff408072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:12.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.796+0000 7ff40d30a700 1 -- 172.21.15.52:0/3624074810 >> 172.21.15.52:0/3624074810 conn(0x7ff4080fb900 msgr2=0x7ff4080fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:12.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.796+0000 7ff40d30a700 1 -- 172.21.15.52:0/3624074810 shutdown_connections 2023-11-27T15:17:12.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:12.796+0000 7ff40d30a700 1 -- 172.21.15.52:0/3624074810 wait complete. 2023-11-27T15:17:13.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:12 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:13.188 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- bash -c 'ceph orch ps' 2023-11-27T15:17:13.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:12 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:13.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:12 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:13.899 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:17:14.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:13 smithi052 ceph-mon[99056]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:14.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:13 smithi052 ceph-mon[99056]: pgmap v290: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4 op/s; 102 B/s, 1 keys/s, 0 objects/s recovering 2023-11-27T15:17:14.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:13 smithi150 ceph-mon[100559]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:14.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:13 smithi150 ceph-mon[100559]: pgmap v290: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4 op/s; 102 B/s, 1 keys/s, 0 objects/s recovering 2023-11-27T15:17:14.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:13 smithi102 ceph-mon[100296]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:14.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:13 smithi102 ceph-mon[100296]: pgmap v290: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4 op/s; 102 B/s, 1 keys/s, 0 objects/s recovering 2023-11-27T15:17:15.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.454+0000 7f4c47396700 1 -- 172.21.15.52:0/3808316002 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c400ff510 msgr2=0x7f4c400ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:15.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.454+0000 7f4c47396700 1 --2- 172.21.15.52:0/3808316002 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c400ff510 0x7f4c400ff980 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4c30009990 tx=0x7f4c30009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:15.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.454+0000 7f4c47396700 1 -- 172.21.15.52:0/3808316002 shutdown_connections 2023-11-27T15:17:15.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.454+0000 7f4c47396700 1 --2- 172.21.15.52:0/3808316002 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c400febc0 0x7f4c400fefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:15.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.454+0000 7f4c47396700 1 --2- 172.21.15.52:0/3808316002 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c400ffec0 0x7f4c40100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:15.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.454+0000 7f4c47396700 1 --2- 172.21.15.52:0/3808316002 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c400ff510 0x7f4c400ff980 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:15.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.454+0000 7f4c47396700 1 -- 172.21.15.52:0/3808316002 >> 172.21.15.52:0/3808316002 conn(0x7f4c400fa7f0 msgr2=0x7f4c400fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:15.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.454+0000 7f4c47396700 1 -- 172.21.15.52:0/3808316002 shutdown_connections 2023-11-27T15:17:15.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.454+0000 7f4c47396700 1 -- 172.21.15.52:0/3808316002 wait complete. 2023-11-27T15:17:15.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.455+0000 7f4c47396700 1 Processor -- start 2023-11-27T15:17:15.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.455+0000 7f4c47396700 1 -- start start 2023-11-27T15:17:15.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.455+0000 7f4c47396700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c400febc0 0x7f4c401981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:15.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.455+0000 7f4c47396700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c400ff510 0x7f4c40198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:15.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.455+0000 7f4c47396700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c400ffec0 0x7f4c4019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:15.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.455+0000 7f4c47396700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4019fd70 con 0x7f4c400ff510 2023-11-27T15:17:15.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.455+0000 7f4c47396700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4019feb0 con 0x7f4c400febc0 2023-11-27T15:17:15.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.455+0000 7f4c47396700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c401a0040 con 0x7f4c400ffec0 2023-11-27T15:17:15.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.455+0000 7f4c44931700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c400ff510 0x7f4c40198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:15.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.455+0000 7f4c45132700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c400febc0 0x7f4c401981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:15.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.455+0000 7f4c44931700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c400ff510 0x7f4c40198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50980/0 (socket says 172.21.15.52:50980) 2023-11-27T15:17:15.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.455+0000 7f4c44931700 1 -- 172.21.15.52:0/1822626183 learned_addr learned my addr 172.21.15.52:0/1822626183 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:15.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.456+0000 7f4c45933700 1 --2- 172.21.15.52:0/1822626183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c400ffec0 0x7f4c4019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:15.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.456+0000 7f4c45132700 1 -- 172.21.15.52:0/1822626183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c400ffec0 msgr2=0x7f4c4019f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:15.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.456+0000 7f4c45132700 1 --2- 172.21.15.52:0/1822626183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c400ffec0 0x7f4c4019f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:15.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.456+0000 7f4c45132700 1 -- 172.21.15.52:0/1822626183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c400ff510 msgr2=0x7f4c40198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:15.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.456+0000 7f4c45132700 1 --2- 172.21.15.52:0/1822626183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c400ff510 0x7f4c40198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:15.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.456+0000 7f4c45132700 1 -- 172.21.15.52:0/1822626183 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c300095f0 con 0x7f4c400febc0 2023-11-27T15:17:15.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.456+0000 7f4c44931700 1 --2- 172.21.15.52:0/1822626183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c400ff510 0x7f4c40198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:17:15.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.456+0000 7f4c45132700 1 --2- 172.21.15.52:0/1822626183 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c400febc0 0x7f4c401981e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c00d950 tx=0x7f4c3c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:15.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.456+0000 7f4c367fc700 1 -- 172.21.15.52:0/1822626183 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c00f890 con 0x7f4c400febc0 2023-11-27T15:17:15.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.456+0000 7f4c47396700 1 -- 172.21.15.52:0/1822626183 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c4010de30 con 0x7f4c400febc0 2023-11-27T15:17:15.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.456+0000 7f4c367fc700 1 -- 172.21.15.52:0/1822626183 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c3c010460 con 0x7f4c400febc0 2023-11-27T15:17:15.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.456+0000 7f4c367fc700 1 -- 172.21.15.52:0/1822626183 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c00e5f0 con 0x7f4c400febc0 2023-11-27T15:17:15.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.456+0000 7f4c47396700 1 -- 172.21.15.52:0/1822626183 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c4010dca0 con 0x7f4c400febc0 2023-11-27T15:17:15.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.457+0000 7f4c47396700 1 -- 172.21.15.52:0/1822626183 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c401099a0 con 0x7f4c400febc0 2023-11-27T15:17:15.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.458+0000 7f4c367fc700 1 -- 172.21.15.52:0/1822626183 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f4c3c0105d0 con 0x7f4c400febc0 2023-11-27T15:17:15.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.459+0000 7f4c367fc700 1 --2- 172.21.15.52:0/1822626183 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4c2c0a11e0 0x7f4c2c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:15.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.459+0000 7f4c44931700 1 --2- 172.21.15.52:0/1822626183 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4c2c0a11e0 0x7f4c2c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:15.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.459+0000 7f4c367fc700 1 -- 172.21.15.52:0/1822626183 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4c3c0cc8a0 con 0x7f4c400febc0 2023-11-27T15:17:15.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.459+0000 7f4c44931700 1 --2- 172.21.15.52:0/1822626183 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4c2c0a11e0 0x7f4c2c0a3690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4c30006010 tx=0x7f4c30005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:15.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.462+0000 7f4c367fc700 1 -- 172.21.15.52:0/1822626183 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4c3c095e10 con 0x7f4c400febc0 2023-11-27T15:17:15.683 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.677+0000 7f4c47396700 1 -- 172.21.15.52:0/1822626183 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c40061aa0 con 0x7f4c2c0a11e0 2023-11-27T15:17:15.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.698+0000 7f4c367fc700 1 -- 172.21.15.52:0/1822626183 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f4c40061aa0 con 0x7f4c2c0a11e0 2023-11-27T15:17:15.704 INFO:teuthology.orchestra.run.smithi052.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-11-27T15:17:15.704 INFO:teuthology.orchestra.run.smithi052.stdout:mds.a smithi052 running (73s) 0s ago 73s 16.5M - 18.0.0-3906-g1c956514 ace6904b066b 3bfad5603ceb 2023-11-27T15:17:15.704 INFO:teuthology.orchestra.run.smithi052.stdout:mds.b smithi102 running (71s) 0s ago 70s 30.0M - 18.0.0-3906-g1c956514 ace6904b066b ef6fb055b075 2023-11-27T15:17:15.704 INFO:teuthology.orchestra.run.smithi052.stdout:mds.c smithi150 running (83s) 0s ago 82s 19.3M - 18.0.0-3906-g1c956514 ace6904b066b e7d32af36f97 2023-11-27T15:17:15.704 INFO:teuthology.orchestra.run.smithi052.stdout:mds.d smithi052 running (89s) 0s ago 89s 15.2M - 18.0.0-3906-g1c956514 ace6904b066b 6ccfc6682dee 2023-11-27T15:17:15.704 INFO:teuthology.orchestra.run.smithi052.stdout:mds.e smithi102 running (87s) 0s ago 87s 17.8M - 18.0.0-3906-g1c956514 ace6904b066b 2d123c3448f5 2023-11-27T15:17:15.704 INFO:teuthology.orchestra.run.smithi052.stdout:mds.f smithi150 running (85s) 0s ago 84s 20.2M - 18.0.0-3906-g1c956514 ace6904b066b f6f622f9962d 2023-11-27T15:17:15.704 INFO:teuthology.orchestra.run.smithi052.stdout:mds.g smithi052 running (76s) 0s ago 75s 17.5M - 18.0.0-3906-g1c956514 ace6904b066b a85c57f9d522 2023-11-27T15:17:15.704 INFO:teuthology.orchestra.run.smithi052.stdout:mds.h smithi102 running (81s) 0s ago 80s 19.8M - 18.0.0-3906-g1c956514 ace6904b066b b36eecfb3698 2023-11-27T15:17:15.704 INFO:teuthology.orchestra.run.smithi052.stdout:mds.i smithi150 running (78s) 0s ago 78s 21.6M - 18.0.0-3906-g1c956514 ace6904b066b 9d006c406e6d 2023-11-27T15:17:15.704 INFO:teuthology.orchestra.run.smithi052.stdout:mds.j smithi052 running (91s) 0s ago 91s 17.4M - 18.0.0-3906-g1c956514 ace6904b066b dc2294d91b6c 2023-11-27T15:17:15.705 INFO:teuthology.orchestra.run.smithi052.stdout:mds.k smithi102 running (94s) 0s ago 93s 17.5M - 18.0.0-3906-g1c956514 ace6904b066b 9454bf594bec 2023-11-27T15:17:15.705 INFO:teuthology.orchestra.run.smithi052.stdout:mds.l smithi150 running (96s) 0s ago 95s 14.0M - 18.0.0-3906-g1c956514 ace6904b066b 10c71557d7b7 2023-11-27T15:17:15.705 INFO:teuthology.orchestra.run.smithi052.stdout:mgr.x smithi052 *:9283,8765 running (8m) 0s ago 8m 516M - 18.0.0-3906-g1c956514 ace6904b066b 228134862db3 2023-11-27T15:17:15.705 INFO:teuthology.orchestra.run.smithi052.stdout:mgr.y smithi102 *:8443,8765 running (6m) 0s ago 6m 445M - 18.0.0-3906-g1c956514 ace6904b066b 9c5d94e7dac3 2023-11-27T15:17:15.705 INFO:teuthology.orchestra.run.smithi052.stdout:mgr.z smithi150 *:8443,8765 running (6m) 0s ago 6m 448M - 18.0.0-3906-g1c956514 ace6904b066b 41bf55970cbc 2023-11-27T15:17:15.705 INFO:teuthology.orchestra.run.smithi052.stdout:mon.a smithi052 running (8m) 0s ago 8m 94.9M 2048M 18.0.0-3906-g1c956514 ace6904b066b 39f489ecf31e 2023-11-27T15:17:15.705 INFO:teuthology.orchestra.run.smithi052.stdout:mon.b smithi102 running (6m) 0s ago 6m 63.6M 2048M 18.0.0-3906-g1c956514 ace6904b066b 9e5952c69650 2023-11-27T15:17:15.705 INFO:teuthology.orchestra.run.smithi052.stdout:mon.c smithi150 running (6m) 0s ago 6m 61.2M 2048M 18.0.0-3906-g1c956514 ace6904b066b 4f1f45ffa809 2023-11-27T15:17:15.705 INFO:teuthology.orchestra.run.smithi052.stdout:osd.0 smithi052 running (5m) 0s ago 5m 70.2M 4293M 18.0.0-3906-g1c956514 ace6904b066b f82e14161c58 2023-11-27T15:17:15.705 INFO:teuthology.orchestra.run.smithi052.stdout:osd.1 smithi102 running (5m) 0s ago 5m 75.1M 4292M 18.0.0-3906-g1c956514 ace6904b066b 7b1ac98cf97f 2023-11-27T15:17:15.705 INFO:teuthology.orchestra.run.smithi052.stdout:osd.2 smithi150 running (5m) 0s ago 5m 73.6M 4292M 18.0.0-3906-g1c956514 ace6904b066b a72ea08caa25 2023-11-27T15:17:15.706 INFO:teuthology.orchestra.run.smithi052.stdout:osd.3 smithi052 running (4m) 0s ago 4m 65.2M 4293M 18.0.0-3906-g1c956514 ace6904b066b 663b7752b66b 2023-11-27T15:17:15.706 INFO:teuthology.orchestra.run.smithi052.stdout:osd.4 smithi102 running (4m) 0s ago 4m 58.2M 4292M 18.0.0-3906-g1c956514 ace6904b066b 90f947253a38 2023-11-27T15:17:15.706 INFO:teuthology.orchestra.run.smithi052.stdout:osd.5 smithi150 running (4m) 0s ago 4m 64.6M 4292M 18.0.0-3906-g1c956514 ace6904b066b 40efced07ff1 2023-11-27T15:17:15.706 INFO:teuthology.orchestra.run.smithi052.stdout:osd.6 smithi052 running (3m) 0s ago 3m 58.2M 4293M 18.0.0-3906-g1c956514 ace6904b066b f2d33c20e63f 2023-11-27T15:17:15.706 INFO:teuthology.orchestra.run.smithi052.stdout:osd.7 smithi102 running (3m) 0s ago 3m 62.2M 4292M 18.0.0-3906-g1c956514 ace6904b066b 4e56a76d67fc 2023-11-27T15:17:15.706 INFO:teuthology.orchestra.run.smithi052.stdout:osd.8 smithi150 running (3m) 0s ago 3m 59.2M 4292M 18.0.0-3906-g1c956514 ace6904b066b b4c4d47ac098 2023-11-27T15:17:15.706 INFO:teuthology.orchestra.run.smithi052.stdout:osd.9 smithi052 running (2m) 0s ago 2m 55.2M 4293M 18.0.0-3906-g1c956514 ace6904b066b d8c295d4becf 2023-11-27T15:17:15.707 INFO:teuthology.orchestra.run.smithi052.stdout:osd.10 smithi102 running (2m) 0s ago 2m 64.0M 4292M 18.0.0-3906-g1c956514 ace6904b066b 8601b9079926 2023-11-27T15:17:15.707 INFO:teuthology.orchestra.run.smithi052.stdout:osd.11 smithi150 running (114s) 0s ago 114s 52.9M 4292M 18.0.0-3906-g1c956514 ace6904b066b 691fa975dfb2 2023-11-27T15:17:15.707 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.700+0000 7f4c47396700 1 -- 172.21.15.52:0/1822626183 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4c2c0a11e0 msgr2=0x7f4c2c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:15.707 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.700+0000 7f4c47396700 1 --2- 172.21.15.52:0/1822626183 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4c2c0a11e0 0x7f4c2c0a3690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4c30006010 tx=0x7f4c30005960 comp rx=0 tx=0).stop 2023-11-27T15:17:15.708 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.700+0000 7f4c47396700 1 -- 172.21.15.52:0/1822626183 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c400febc0 msgr2=0x7f4c401981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:15.708 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.700+0000 7f4c47396700 1 --2- 172.21.15.52:0/1822626183 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c400febc0 0x7f4c401981e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c00d950 tx=0x7f4c3c00dc60 comp rx=0 tx=0).stop 2023-11-27T15:17:15.708 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.701+0000 7f4c47396700 1 -- 172.21.15.52:0/1822626183 shutdown_connections 2023-11-27T15:17:15.708 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.701+0000 7f4c47396700 1 --2- 172.21.15.52:0/1822626183 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4c2c0a11e0 0x7f4c2c0a3690 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:15.708 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.701+0000 7f4c47396700 1 --2- 172.21.15.52:0/1822626183 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c400febc0 0x7f4c401981e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:15.708 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.701+0000 7f4c47396700 1 --2- 172.21.15.52:0/1822626183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c400ffec0 0x7f4c4019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:15.708 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.701+0000 7f4c47396700 1 --2- 172.21.15.52:0/1822626183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c400ff510 0x7f4c40198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:15.708 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.701+0000 7f4c47396700 1 -- 172.21.15.52:0/1822626183 >> 172.21.15.52:0/1822626183 conn(0x7f4c400fa7f0 msgr2=0x7f4c400fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:15.708 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.701+0000 7f4c47396700 1 -- 172.21.15.52:0/1822626183 shutdown_connections 2023-11-27T15:17:15.708 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:15.701+0000 7f4c47396700 1 -- 172.21.15.52:0/1822626183 wait complete. 2023-11-27T15:17:16.103 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:15 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:17:16.103 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:15 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:17:16.103 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:15 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:17:16.103 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:15 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:17:16.103 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:15 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:17:16.103 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:15 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:17:16.103 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:15 smithi052 ceph-mon[99056]: from='client.24941 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:16.103 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:15 smithi052 ceph-mon[99056]: pgmap v291: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 93 B/s, 0 keys/s, 0 objects/s recovering 2023-11-27T15:17:16.166 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- bash -c 'ceph orch ls' 2023-11-27T15:17:16.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:15 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:17:16.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:15 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:17:16.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:15 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:17:16.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:15 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:17:16.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:15 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:17:16.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:15 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:17:16.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:15 smithi150 ceph-mon[100559]: from='client.24941 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:16.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:15 smithi150 ceph-mon[100559]: pgmap v291: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 93 B/s, 0 keys/s, 0 objects/s recovering 2023-11-27T15:17:16.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:15 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:17:16.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:15 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:17:16.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:15 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:17:16.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:15 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:17:16.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:15 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:17:16.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:15 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:17:16.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:15 smithi102 ceph-mon[100296]: from='client.24941 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:16.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:15 smithi102 ceph-mon[100296]: pgmap v291: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 93 B/s, 0 keys/s, 0 objects/s recovering 2023-11-27T15:17:16.824 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:17:17.252 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:16 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:17.252 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:16 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:17.252 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:16 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:17:17.253 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:16 smithi052 ceph-mon[99056]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:17:17.253 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:16 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:17:17.253 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:16 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:17:17.253 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:16 smithi052 ceph-mon[99056]: Service mds.all has no related daemon already placed. 2023-11-27T15:17:17.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:16 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:17.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:16 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:17.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:16 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:17:17.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:16 smithi150 ceph-mon[100559]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:17:17.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:16 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:17:17.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:16 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:17:17.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:16 smithi150 ceph-mon[100559]: Service mds.all has no related daemon already placed. 2023-11-27T15:17:17.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:16 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:17.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:16 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:17.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:16 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:17:17.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:16 smithi102 ceph-mon[100296]: from='mgr.14152 172.21.15.52:0/1817126667' entity='mgr.x' 2023-11-27T15:17:17.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:16 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:17:17.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:16 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:17:17.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:16 smithi102 ceph-mon[100296]: Service mds.all has no related daemon already placed. 2023-11-27T15:17:18.147 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.140+0000 7fd69b209700 1 -- 172.21.15.52:0/4205382370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6940ff1e0 msgr2=0x7fd6940ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:18.147 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.140+0000 7fd69b209700 1 --2- 172.21.15.52:0/4205382370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6940ff1e0 0x7fd6940ff5f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd688009990 tx=0x7fd688009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:18.147 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.140+0000 7fd69b209700 1 -- 172.21.15.52:0/4205382370 shutdown_connections 2023-11-27T15:17:18.148 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.140+0000 7fd69b209700 1 --2- 172.21.15.52:0/4205382370 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd694100d70 0x7fd694107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:18.148 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.140+0000 7fd69b209700 1 --2- 172.21.15.52:0/4205382370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6940ff1e0 0x7fd6940ff5f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:18.148 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.140+0000 7fd69b209700 1 --2- 172.21.15.52:0/4205382370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6941003e0 0x7fd694100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:18.148 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.140+0000 7fd69b209700 1 -- 172.21.15.52:0/4205382370 >> 172.21.15.52:0/4205382370 conn(0x7fd6940fa7f0 msgr2=0x7fd6940fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:18.148 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.140+0000 7fd69b209700 1 -- 172.21.15.52:0/4205382370 shutdown_connections 2023-11-27T15:17:18.148 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.140+0000 7fd69b209700 1 -- 172.21.15.52:0/4205382370 wait complete. 2023-11-27T15:17:18.148 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.141+0000 7fd69b209700 1 Processor -- start 2023-11-27T15:17:18.148 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.141+0000 7fd69b209700 1 -- start start 2023-11-27T15:17:18.148 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.141+0000 7fd69b209700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6940ff1e0 0x7fd694194f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:18.148 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.141+0000 7fd69b209700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6941003e0 0x7fd694195450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:18.148 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.141+0000 7fd69b209700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd694100d70 0x7fd69419c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:18.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.141+0000 7fd69b209700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69419caa0 con 0x7fd694100d70 2023-11-27T15:17:18.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.141+0000 7fd69b209700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69419cbe0 con 0x7fd6941003e0 2023-11-27T15:17:18.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.141+0000 7fd69b209700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69419cd70 con 0x7fd6940ff1e0 2023-11-27T15:17:18.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.141+0000 7fd6997a6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd694100d70 0x7fd69419c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:18.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.141+0000 7fd6997a6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd694100d70 0x7fd69419c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50994/0 (socket says 172.21.15.52:50994) 2023-11-27T15:17:18.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.141+0000 7fd6997a6700 1 -- 172.21.15.52:0/795312991 learned_addr learned my addr 172.21.15.52:0/795312991 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:18.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.141+0000 7fd693fff700 1 --2- 172.21.15.52:0/795312991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6941003e0 0x7fd694195450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:18.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.141+0000 7fd698fa5700 1 --2- 172.21.15.52:0/795312991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6940ff1e0 0x7fd694194f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:18.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.142+0000 7fd6997a6700 1 -- 172.21.15.52:0/795312991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6940ff1e0 msgr2=0x7fd694194f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:18.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.142+0000 7fd6997a6700 1 --2- 172.21.15.52:0/795312991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6940ff1e0 0x7fd694194f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:18.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.142+0000 7fd6997a6700 1 -- 172.21.15.52:0/795312991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6941003e0 msgr2=0x7fd694195450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:18.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.142+0000 7fd6997a6700 1 --2- 172.21.15.52:0/795312991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6941003e0 0x7fd694195450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:18.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.142+0000 7fd6997a6700 1 -- 172.21.15.52:0/795312991 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6880095f0 con 0x7fd694100d70 2023-11-27T15:17:18.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.142+0000 7fd6997a6700 1 --2- 172.21.15.52:0/795312991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd694100d70 0x7fd69419c4d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd68c007f10 tx=0x7fd68c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:18.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.142+0000 7fd691ffb700 1 -- 172.21.15.52:0/795312991 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c017070 con 0x7fd694100d70 2023-11-27T15:17:18.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.142+0000 7fd691ffb700 1 -- 172.21.15.52:0/795312991 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd68c00f040 con 0x7fd694100d70 2023-11-27T15:17:18.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.142+0000 7fd69b209700 1 -- 172.21.15.52:0/795312991 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd69410ab60 con 0x7fd694100d70 2023-11-27T15:17:18.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.142+0000 7fd691ffb700 1 -- 172.21.15.52:0/795312991 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c005610 con 0x7fd694100d70 2023-11-27T15:17:18.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.142+0000 7fd69b209700 1 -- 172.21.15.52:0/795312991 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd69410a9d0 con 0x7fd694100d70 2023-11-27T15:17:18.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.143+0000 7fd69b209700 1 -- 172.21.15.52:0/795312991 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd69419d390 con 0x7fd694100d70 2023-11-27T15:17:18.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.144+0000 7fd691ffb700 1 -- 172.21.15.52:0/795312991 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fd68c004ad0 con 0x7fd694100d70 2023-11-27T15:17:18.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.145+0000 7fd691ffb700 1 --2- 172.21.15.52:0/795312991 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd6840a11e0 0x7fd6840a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:18.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.145+0000 7fd698fa5700 1 --2- 172.21.15.52:0/795312991 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd6840a11e0 0x7fd6840a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:18.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.145+0000 7fd691ffb700 1 -- 172.21.15.52:0/795312991 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd68c0cc200 con 0x7fd694100d70 2023-11-27T15:17:18.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.145+0000 7fd698fa5700 1 --2- 172.21.15.52:0/795312991 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd6840a11e0 0x7fd6840a3690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd68800b5c0 tx=0x7fd688005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:18.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.150+0000 7fd691ffb700 1 -- 172.21.15.52:0/795312991 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd68c095770 con 0x7fd694100d70 2023-11-27T15:17:18.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:17 smithi150 ceph-mon[100559]: pgmap v292: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 84 B/s, 0 keys/s, 0 objects/s recovering 2023-11-27T15:17:18.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:17 smithi102 ceph-mon[100296]: pgmap v292: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 84 B/s, 0 keys/s, 0 objects/s recovering 2023-11-27T15:17:18.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.354+0000 7fd69b209700 1 -- 172.21.15.52:0/795312991 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd694060f30 con 0x7fd6840a11e0 2023-11-27T15:17:18.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:17 smithi052 ceph-mon[99056]: pgmap v292: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 84 B/s, 0 keys/s, 0 objects/s recovering 2023-11-27T15:17:18.367 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.361+0000 7fd691ffb700 1 -- 172.21.15.52:0/795312991 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+985 (secure 0 0 0) 0x7fd694060f30 con 0x7fd6840a11e0 2023-11-27T15:17:18.367 INFO:teuthology.orchestra.run.smithi052.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-11-27T15:17:18.367 INFO:teuthology.orchestra.run.smithi052.stdout:mds.all 12/12 3s ago 100s smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi102=b;smithi102=e;smithi102=h;smithi102=k;smithi150=c;smithi150=f;smithi150=i;smithi150=l;count:12 2023-11-27T15:17:18.367 INFO:teuthology.orchestra.run.smithi052.stdout:mgr 3/3 3s ago 6m smithi052=x;smithi102=y;smithi150=z;count:3 2023-11-27T15:17:18.367 INFO:teuthology.orchestra.run.smithi052.stdout:mon 3/3 3s ago 6m smithi052:172.21.15.52=a;smithi102:172.21.15.102=b;smithi150:172.21.15.150=c;count:3 2023-11-27T15:17:18.367 INFO:teuthology.orchestra.run.smithi052.stdout:osd 12 3s ago - 2023-11-27T15:17:18.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.364+0000 7fd69b209700 1 -- 172.21.15.52:0/795312991 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd6840a11e0 msgr2=0x7fd6840a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:18.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.364+0000 7fd69b209700 1 --2- 172.21.15.52:0/795312991 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd6840a11e0 0x7fd6840a3690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd68800b5c0 tx=0x7fd688005fb0 comp rx=0 tx=0).stop 2023-11-27T15:17:18.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.364+0000 7fd69b209700 1 -- 172.21.15.52:0/795312991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd694100d70 msgr2=0x7fd69419c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:18.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.364+0000 7fd69b209700 1 --2- 172.21.15.52:0/795312991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd694100d70 0x7fd69419c4d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd68c007f10 tx=0x7fd68c00d3b0 comp rx=0 tx=0).stop 2023-11-27T15:17:18.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.364+0000 7fd69b209700 1 -- 172.21.15.52:0/795312991 shutdown_connections 2023-11-27T15:17:18.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.364+0000 7fd69b209700 1 --2- 172.21.15.52:0/795312991 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd6840a11e0 0x7fd6840a3690 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:18.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.364+0000 7fd69b209700 1 --2- 172.21.15.52:0/795312991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd694100d70 0x7fd69419c4d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:18.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.364+0000 7fd69b209700 1 --2- 172.21.15.52:0/795312991 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6940ff1e0 0x7fd694194f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:18.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.364+0000 7fd69b209700 1 --2- 172.21.15.52:0/795312991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6941003e0 0x7fd694195450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:18.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.364+0000 7fd69b209700 1 -- 172.21.15.52:0/795312991 >> 172.21.15.52:0/795312991 conn(0x7fd6940fa7f0 msgr2=0x7fd6940fcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:18.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.364+0000 7fd69b209700 1 -- 172.21.15.52:0/795312991 shutdown_connections 2023-11-27T15:17:18.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:18.364+0000 7fd69b209700 1 -- 172.21.15.52:0/795312991 wait complete. 2023-11-27T15:17:18.883 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- bash -c 'ceph orch host ls' 2023-11-27T15:17:19.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:18 smithi052 ceph-mon[99056]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:19.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:18 smithi150 ceph-mon[100559]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:19.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:18 smithi102 ceph-mon[100296]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:19.541 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:17:20.105 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:19 smithi052 ceph-mon[99056]: pgmap v293: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s; 84 B/s, 0 keys/s, 0 objects/s recovering 2023-11-27T15:17:20.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:19 smithi150 ceph-mon[100559]: pgmap v293: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s; 84 B/s, 0 keys/s, 0 objects/s recovering 2023-11-27T15:17:20.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:19 smithi102 ceph-mon[100296]: pgmap v293: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s; 84 B/s, 0 keys/s, 0 objects/s recovering 2023-11-27T15:17:20.864 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.858+0000 7f7c7f0dd700 1 -- 172.21.15.52:0/1483895223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c78106010 msgr2=0x7f7c78108410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:20.864 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.858+0000 7f7c7f0dd700 1 --2- 172.21.15.52:0/1483895223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c78106010 0x7f7c78108410 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000b0a0 tx=0x7f7c7000b3b0 comp rx=0 tx=0).stop 2023-11-27T15:17:20.864 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.858+0000 7f7c7f0dd700 1 -- 172.21.15.52:0/1483895223 shutdown_connections 2023-11-27T15:17:20.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.858+0000 7f7c7f0dd700 1 --2- 172.21.15.52:0/1483895223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c78100dd0 0x7f7c781031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:20.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.858+0000 7f7c7f0dd700 1 --2- 172.21.15.52:0/1483895223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c78106010 0x7f7c78108410 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:20.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.858+0000 7f7c7f0dd700 1 --2- 172.21.15.52:0/1483895223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c781036f0 0x7f7c78105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:20.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.858+0000 7f7c7f0dd700 1 -- 172.21.15.52:0/1483895223 >> 172.21.15.52:0/1483895223 conn(0x7f7c780fa7f0 msgr2=0x7f7c780fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:20.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.858+0000 7f7c7f0dd700 1 -- 172.21.15.52:0/1483895223 shutdown_connections 2023-11-27T15:17:20.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.858+0000 7f7c7f0dd700 1 -- 172.21.15.52:0/1483895223 wait complete. 2023-11-27T15:17:20.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.859+0000 7f7c7f0dd700 1 Processor -- start 2023-11-27T15:17:20.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.859+0000 7f7c7f0dd700 1 -- start start 2023-11-27T15:17:20.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.859+0000 7f7c7f0dd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c78100dd0 0x7f7c78100730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:20.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.859+0000 7f7c7f0dd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c781036f0 0x7f7c780fed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:20.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.859+0000 7f7c7f0dd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c78106010 0x7f7c780ff2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:20.866 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.859+0000 7f7c7f0dd700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c780ffa40 con 0x7f7c78106010 2023-11-27T15:17:20.866 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.859+0000 7f7c7f0dd700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c780ffbb0 con 0x7f7c78100dd0 2023-11-27T15:17:20.866 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.859+0000 7f7c7f0dd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c781a02a0 con 0x7f7c781036f0 2023-11-27T15:17:20.866 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.859+0000 7f7c7d67a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c78106010 0x7f7c780ff2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:20.866 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.859+0000 7f7c7d67a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c78106010 0x7f7c780ff2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51006/0 (socket says 172.21.15.52:51006) 2023-11-27T15:17:20.866 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.859+0000 7f7c7d67a700 1 -- 172.21.15.52:0/2379591130 learned_addr learned my addr 172.21.15.52:0/2379591130 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:20.866 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.859+0000 7f7c77fff700 1 --2- 172.21.15.52:0/2379591130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c781036f0 0x7f7c780fed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:20.866 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.859+0000 7f7c7ce79700 1 --2- 172.21.15.52:0/2379591130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c78100dd0 0x7f7c78100730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:20.866 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.859+0000 7f7c7d67a700 1 -- 172.21.15.52:0/2379591130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c781036f0 msgr2=0x7f7c780fed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:20.866 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.859+0000 7f7c7d67a700 1 --2- 172.21.15.52:0/2379591130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c781036f0 0x7f7c780fed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:20.866 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.859+0000 7f7c7d67a700 1 -- 172.21.15.52:0/2379591130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c78100dd0 msgr2=0x7f7c78100730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:20.866 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.859+0000 7f7c7d67a700 1 --2- 172.21.15.52:0/2379591130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c78100dd0 0x7f7c78100730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:20.867 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.859+0000 7f7c7d67a700 1 -- 172.21.15.52:0/2379591130 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c70009d00 con 0x7f7c78106010 2023-11-27T15:17:20.867 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.859+0000 7f7c7ce79700 1 --2- 172.21.15.52:0/2379591130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c78100dd0 0x7f7c78100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:17:20.867 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.859+0000 7f7c7d67a700 1 --2- 172.21.15.52:0/2379591130 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c78106010 0x7f7c780ff2c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7c70009fd0 tx=0x7f7c700095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:20.867 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.859+0000 7f7c75ffb700 1 -- 172.21.15.52:0/2379591130 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7000e040 con 0x7f7c78106010 2023-11-27T15:17:20.867 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.860+0000 7f7c75ffb700 1 -- 172.21.15.52:0/2379591130 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c70004020 con 0x7f7c78106010 2023-11-27T15:17:20.867 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.860+0000 7f7c75ffb700 1 -- 172.21.15.52:0/2379591130 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7001d6a0 con 0x7f7c78106010 2023-11-27T15:17:20.867 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.860+0000 7f7c7f0dd700 1 -- 172.21.15.52:0/2379591130 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c7810bd00 con 0x7f7c78106010 2023-11-27T15:17:20.867 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.860+0000 7f7c7f0dd700 1 -- 172.21.15.52:0/2379591130 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c7810bb70 con 0x7f7c78106010 2023-11-27T15:17:20.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.861+0000 7f7c7f0dd700 1 -- 172.21.15.52:0/2379591130 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c780fc360 con 0x7f7c78106010 2023-11-27T15:17:20.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.862+0000 7f7c75ffb700 1 -- 172.21.15.52:0/2379591130 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f7c70004470 con 0x7f7c78106010 2023-11-27T15:17:20.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.862+0000 7f7c75ffb700 1 --2- 172.21.15.52:0/2379591130 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7c680a1260 0x7f7c680a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:20.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.862+0000 7f7c75ffb700 1 -- 172.21.15.52:0/2379591130 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7c700ce0d0 con 0x7f7c78106010 2023-11-27T15:17:20.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.862+0000 7f7c7ce79700 1 --2- 172.21.15.52:0/2379591130 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7c680a1260 0x7f7c680a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:20.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.863+0000 7f7c7ce79700 1 --2- 172.21.15.52:0/2379591130 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7c680a1260 0x7f7c680a3710 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c005fd0 tx=0x7f7c6c005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:20.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:20.867+0000 7f7c75ffb700 1 -- 172.21.15.52:0/2379591130 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7c70097730 con 0x7f7c78106010 2023-11-27T15:17:21.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:21.071+0000 7f7c7f0dd700 1 -- 172.21.15.52:0/2379591130 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c78061ae0 con 0x7f7c680a1260 2023-11-27T15:17:21.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:21.073+0000 7f7c75ffb700 1 -- 172.21.15.52:0/2379591130 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f7c78061ae0 con 0x7f7c680a1260 2023-11-27T15:17:21.077 INFO:teuthology.orchestra.run.smithi052.stdout:HOST ADDR LABELS STATUS 2023-11-27T15:17:21.077 INFO:teuthology.orchestra.run.smithi052.stdout:smithi052 172.21.15.52 2023-11-27T15:17:21.077 INFO:teuthology.orchestra.run.smithi052.stdout:smithi102 172.21.15.102 2023-11-27T15:17:21.077 INFO:teuthology.orchestra.run.smithi052.stdout:smithi150 172.21.15.150 2023-11-27T15:17:21.077 INFO:teuthology.orchestra.run.smithi052.stdout:3 hosts in cluster 2023-11-27T15:17:21.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:21.074+0000 7f7c7f0dd700 1 -- 172.21.15.52:0/2379591130 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7c680a1260 msgr2=0x7f7c680a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:21.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:21.074+0000 7f7c7f0dd700 1 --2- 172.21.15.52:0/2379591130 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7c680a1260 0x7f7c680a3710 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c005fd0 tx=0x7f7c6c005dc0 comp rx=0 tx=0).stop 2023-11-27T15:17:21.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:21.074+0000 7f7c7f0dd700 1 -- 172.21.15.52:0/2379591130 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c78106010 msgr2=0x7f7c780ff2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:21.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:21.074+0000 7f7c7f0dd700 1 --2- 172.21.15.52:0/2379591130 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c78106010 0x7f7c780ff2c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7c70009fd0 tx=0x7f7c700095c0 comp rx=0 tx=0).stop 2023-11-27T15:17:21.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:21.075+0000 7f7c7f0dd700 1 -- 172.21.15.52:0/2379591130 shutdown_connections 2023-11-27T15:17:21.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:21.075+0000 7f7c7f0dd700 1 --2- 172.21.15.52:0/2379591130 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7c680a1260 0x7f7c680a3710 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:21.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:21.075+0000 7f7c7f0dd700 1 --2- 172.21.15.52:0/2379591130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7c78100dd0 0x7f7c78100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:21.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:21.075+0000 7f7c7f0dd700 1 --2- 172.21.15.52:0/2379591130 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c78106010 0x7f7c780ff2c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:21.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:21.075+0000 7f7c7f0dd700 1 --2- 172.21.15.52:0/2379591130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7c781036f0 0x7f7c780fed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:21.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:21.075+0000 7f7c7f0dd700 1 -- 172.21.15.52:0/2379591130 >> 172.21.15.52:0/2379591130 conn(0x7f7c780fa7f0 msgr2=0x7f7c78101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:21.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:21.075+0000 7f7c7f0dd700 1 -- 172.21.15.52:0/2379591130 shutdown_connections 2023-11-27T15:17:21.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:21.075+0000 7f7c7f0dd700 1 -- 172.21.15.52:0/2379591130 wait complete. 2023-11-27T15:17:21.488 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- bash -c 'ceph orch device ls' 2023-11-27T15:17:22.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:21 smithi102 ceph-mon[100296]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:22.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:21 smithi102 ceph-mon[100296]: pgmap v294: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:17:22.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:21 smithi052 ceph-mon[99056]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:22.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:21 smithi052 ceph-mon[99056]: pgmap v294: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:17:22.143 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:17:22.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:21 smithi150 ceph-mon[100559]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:22.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:21 smithi150 ceph-mon[100559]: pgmap v294: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:17:23.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.436+0000 7fac64f0a700 1 -- 172.21.15.52:0/511436379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac60100d70 msgr2=0x7fac60107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:23.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.436+0000 7fac64f0a700 1 --2- 172.21.15.52:0/511436379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac60100d70 0x7fac60107630 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fac5800b0a0 tx=0x7fac5800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:17:23.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.437+0000 7fac64f0a700 1 -- 172.21.15.52:0/511436379 shutdown_connections 2023-11-27T15:17:23.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.437+0000 7fac64f0a700 1 --2- 172.21.15.52:0/511436379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac60100d70 0x7fac60107630 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:23.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.437+0000 7fac64f0a700 1 --2- 172.21.15.52:0/511436379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac600ff1e0 0x7fac600ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:23.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.437+0000 7fac64f0a700 1 --2- 172.21.15.52:0/511436379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac601003e0 0x7fac60100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:23.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.437+0000 7fac64f0a700 1 -- 172.21.15.52:0/511436379 >> 172.21.15.52:0/511436379 conn(0x7fac600fa7f0 msgr2=0x7fac600fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:23.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.437+0000 7fac64f0a700 1 -- 172.21.15.52:0/511436379 shutdown_connections 2023-11-27T15:17:23.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.437+0000 7fac64f0a700 1 -- 172.21.15.52:0/511436379 wait complete. 2023-11-27T15:17:23.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.437+0000 7fac64f0a700 1 Processor -- start 2023-11-27T15:17:23.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.437+0000 7fac64f0a700 1 -- start start 2023-11-27T15:17:23.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.438+0000 7fac64f0a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac600ff1e0 0x7fac60069110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:23.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.438+0000 7fac64f0a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac601003e0 0x7fac6019d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:23.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.438+0000 7fac64f0a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac60100d70 0x7fac6019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:23.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.438+0000 7fac64f0a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6019fe40 con 0x7fac60100d70 2023-11-27T15:17:23.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.438+0000 7fac64f0a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6019ffb0 con 0x7fac600ff1e0 2023-11-27T15:17:23.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.438+0000 7fac64f0a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac601a0170 con 0x7fac601003e0 2023-11-27T15:17:23.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.438+0000 7fac5ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac60100d70 0x7fac6019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:23.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.438+0000 7fac5ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac60100d70 0x7fac6019f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51020/0 (socket says 172.21.15.52:51020) 2023-11-27T15:17:23.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.438+0000 7fac5ed9d700 1 -- 172.21.15.52:0/2777065379 learned_addr learned my addr 172.21.15.52:0/2777065379 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:23.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.438+0000 7fac5e59c700 1 --2- 172.21.15.52:0/2777065379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac600ff1e0 0x7fac60069110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:23.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.438+0000 7fac5dd9b700 1 --2- 172.21.15.52:0/2777065379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac601003e0 0x7fac6019d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:23.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.438+0000 7fac5ed9d700 1 -- 172.21.15.52:0/2777065379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac601003e0 msgr2=0x7fac6019d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:23.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.438+0000 7fac5ed9d700 1 --2- 172.21.15.52:0/2777065379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac601003e0 0x7fac6019d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:23.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.438+0000 7fac5ed9d700 1 -- 172.21.15.52:0/2777065379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac600ff1e0 msgr2=0x7fac60069110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:23.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.438+0000 7fac5ed9d700 1 --2- 172.21.15.52:0/2777065379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac600ff1e0 0x7fac60069110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:23.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.438+0000 7fac5ed9d700 1 -- 172.21.15.52:0/2777065379 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac58009d00 con 0x7fac60100d70 2023-11-27T15:17:23.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.438+0000 7fac5ed9d700 1 --2- 172.21.15.52:0/2777065379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac60100d70 0x7fac6019f790 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fac5800b830 tx=0x7fac580094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:23.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.438+0000 7fac577fe700 1 -- 172.21.15.52:0/2777065379 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5800e040 con 0x7fac60100d70 2023-11-27T15:17:23.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.438+0000 7fac64f0a700 1 -- 172.21.15.52:0/2777065379 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac6010ab30 con 0x7fac60100d70 2023-11-27T15:17:23.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.438+0000 7fac577fe700 1 -- 172.21.15.52:0/2777065379 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac58004de0 con 0x7fac60100d70 2023-11-27T15:17:23.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.438+0000 7fac577fe700 1 -- 172.21.15.52:0/2777065379 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5801d480 con 0x7fac60100d70 2023-11-27T15:17:23.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.438+0000 7fac64f0a700 1 -- 172.21.15.52:0/2777065379 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac6010a9a0 con 0x7fac60100d70 2023-11-27T15:17:23.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.439+0000 7fac64f0a700 1 -- 172.21.15.52:0/2777065379 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac60066ea0 con 0x7fac60100d70 2023-11-27T15:17:23.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.441+0000 7fac577fe700 1 -- 172.21.15.52:0/2777065379 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fac58003a00 con 0x7fac60100d70 2023-11-27T15:17:23.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.442+0000 7fac577fe700 1 --2- 172.21.15.52:0/2777065379 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fac4c0a11e0 0x7fac4c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:23.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.442+0000 7fac5e59c700 1 --2- 172.21.15.52:0/2777065379 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fac4c0a11e0 0x7fac4c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:23.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.442+0000 7fac577fe700 1 -- 172.21.15.52:0/2777065379 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fac580ced00 con 0x7fac60100d70 2023-11-27T15:17:23.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.443+0000 7fac5e59c700 1 --2- 172.21.15.52:0/2777065379 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fac4c0a11e0 0x7fac4c0a3690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fac60100240 tx=0x7fac5000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:23.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.445+0000 7fac577fe700 1 -- 172.21.15.52:0/2777065379 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fac58098360 con 0x7fac60100d70 2023-11-27T15:17:23.656 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.651+0000 7fac64f0a700 1 -- 172.21.15.52:0/2777065379 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fac60060f30 con 0x7fac4c0a11e0 2023-11-27T15:17:23.658 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.654+0000 7fac577fe700 1 -- 172.21.15.52:0/2777065379 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+492 (secure 0 0 0) 0x7fac60060f30 con 0x7fac4c0a11e0 2023-11-27T15:17:23.658 INFO:teuthology.orchestra.run.smithi052.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-11-27T15:17:23.658 INFO:teuthology.orchestra.run.smithi052.stdout:smithi052 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513010N480DGN 447G No 2m ago LVM detected, locked 2023-11-27T15:17:23.658 INFO:teuthology.orchestra.run.smithi052.stdout:smithi102 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400VK400BGN 372G No 2m ago LVM detected, locked 2023-11-27T15:17:23.658 INFO:teuthology.orchestra.run.smithi052.stdout:smithi150 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GX400BGN 372G No 116s ago LVM detected, locked 2023-11-27T15:17:23.660 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.655+0000 7fac64f0a700 1 -- 172.21.15.52:0/2777065379 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fac4c0a11e0 msgr2=0x7fac4c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:23.660 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.655+0000 7fac64f0a700 1 --2- 172.21.15.52:0/2777065379 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fac4c0a11e0 0x7fac4c0a3690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fac60100240 tx=0x7fac5000a3b0 comp rx=0 tx=0).stop 2023-11-27T15:17:23.660 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.655+0000 7fac64f0a700 1 -- 172.21.15.52:0/2777065379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac60100d70 msgr2=0x7fac6019f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:23.660 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.655+0000 7fac64f0a700 1 --2- 172.21.15.52:0/2777065379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac60100d70 0x7fac6019f790 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fac5800b830 tx=0x7fac580094e0 comp rx=0 tx=0).stop 2023-11-27T15:17:23.660 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.656+0000 7fac64f0a700 1 -- 172.21.15.52:0/2777065379 shutdown_connections 2023-11-27T15:17:23.660 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.656+0000 7fac64f0a700 1 --2- 172.21.15.52:0/2777065379 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fac4c0a11e0 0x7fac4c0a3690 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:23.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.656+0000 7fac64f0a700 1 --2- 172.21.15.52:0/2777065379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac600ff1e0 0x7fac60069110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:23.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.656+0000 7fac64f0a700 1 --2- 172.21.15.52:0/2777065379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac60100d70 0x7fac6019f790 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:23.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.656+0000 7fac64f0a700 1 --2- 172.21.15.52:0/2777065379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac601003e0 0x7fac6019d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:23.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.656+0000 7fac64f0a700 1 -- 172.21.15.52:0/2777065379 >> 172.21.15.52:0/2777065379 conn(0x7fac600fa7f0 msgr2=0x7fac600fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:23.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.656+0000 7fac64f0a700 1 -- 172.21.15.52:0/2777065379 shutdown_connections 2023-11-27T15:17:23.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:23.656+0000 7fac64f0a700 1 -- 172.21.15.52:0/2777065379 wait complete. 2023-11-27T15:17:24.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:23 smithi102 ceph-mon[100296]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:24.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:23 smithi102 ceph-mon[100296]: pgmap v295: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-27T15:17:24.108 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:23 smithi052 ceph-mon[99056]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:24.108 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:23 smithi052 ceph-mon[99056]: pgmap v295: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-27T15:17:24.191 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-11-27T15:17:24.201 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi052.front.sepia.ceph.com 2023-11-27T15:17:24.201 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- bash -c 'ceph fs dump' 2023-11-27T15:17:24.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:23 smithi150 ceph-mon[100559]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:24.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:23 smithi150 ceph-mon[100559]: pgmap v295: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-27T15:17:24.855 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:17:25.979 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:25 smithi052 ceph-mon[99056]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-27T15:17:26.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:25 smithi102 ceph-mon[100296]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-27T15:17:26.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.190+0000 7f9e363e9700 1 -- 172.21.15.52:0/3766392260 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e30101e80 msgr2=0x7f9e30108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:26.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.190+0000 7f9e363e9700 1 --2- 172.21.15.52:0/3766392260 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e30101e80 0x7f9e30108740 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9e2800b0a0 tx=0x7f9e2800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:17:26.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.190+0000 7f9e363e9700 1 -- 172.21.15.52:0/3766392260 shutdown_connections 2023-11-27T15:17:26.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.190+0000 7f9e363e9700 1 --2- 172.21.15.52:0/3766392260 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e30101e80 0x7f9e30108740 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:26.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.190+0000 7f9e363e9700 1 --2- 172.21.15.52:0/3766392260 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e301002f0 0x7f9e30100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:26.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.190+0000 7f9e363e9700 1 --2- 172.21.15.52:0/3766392260 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e301014f0 0x7f9e30101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:26.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.190+0000 7f9e363e9700 1 -- 172.21.15.52:0/3766392260 >> 172.21.15.52:0/3766392260 conn(0x7f9e300fb900 msgr2=0x7f9e300fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:26.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.190+0000 7f9e363e9700 1 -- 172.21.15.52:0/3766392260 shutdown_connections 2023-11-27T15:17:26.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.190+0000 7f9e363e9700 1 -- 172.21.15.52:0/3766392260 wait complete. 2023-11-27T15:17:26.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.191+0000 7f9e363e9700 1 Processor -- start 2023-11-27T15:17:26.196 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.191+0000 7f9e363e9700 1 -- start start 2023-11-27T15:17:26.197 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.191+0000 7f9e363e9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e301002f0 0x7f9e30196080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:26.197 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.191+0000 7f9e363e9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e301014f0 0x7f9e301965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:26.198 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.191+0000 7f9e363e9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e30101e80 0x7f9e3019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:26.198 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.191+0000 7f9e363e9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3019dc10 con 0x7f9e30101e80 2023-11-27T15:17:26.198 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.191+0000 7f9e363e9700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3019dd50 con 0x7f9e301002f0 2023-11-27T15:17:26.198 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.191+0000 7f9e363e9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3019dee0 con 0x7f9e301014f0 2023-11-27T15:17:26.198 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.191+0000 7f9e34986700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e30101e80 0x7f9e3019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:26.198 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.191+0000 7f9e34986700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e30101e80 0x7f9e3019d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51034/0 (socket says 172.21.15.52:51034) 2023-11-27T15:17:26.198 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.191+0000 7f9e34986700 1 -- 172.21.15.52:0/2330743001 learned_addr learned my addr 172.21.15.52:0/2330743001 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:26.198 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.191+0000 7f9e2ffff700 1 --2- 172.21.15.52:0/2330743001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e301002f0 0x7f9e30196080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:26.198 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.191+0000 7f9e2f7fe700 1 --2- 172.21.15.52:0/2330743001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e301014f0 0x7f9e301965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:26.198 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.191+0000 7f9e34986700 1 -- 172.21.15.52:0/2330743001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e301014f0 msgr2=0x7f9e301965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:26.198 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.191+0000 7f9e34986700 1 --2- 172.21.15.52:0/2330743001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e301014f0 0x7f9e301965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:26.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.191+0000 7f9e34986700 1 -- 172.21.15.52:0/2330743001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e301002f0 msgr2=0x7f9e30196080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:26.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.191+0000 7f9e34986700 1 --2- 172.21.15.52:0/2330743001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e301002f0 0x7f9e30196080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:26.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.191+0000 7f9e34986700 1 -- 172.21.15.52:0/2330743001 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e28009d00 con 0x7f9e30101e80 2023-11-27T15:17:26.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.191+0000 7f9e2ffff700 1 --2- 172.21.15.52:0/2330743001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e301002f0 0x7f9e30196080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:17:26.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.191+0000 7f9e2f7fe700 1 --2- 172.21.15.52:0/2330743001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e301014f0 0x7f9e301965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:17:26.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.192+0000 7f9e34986700 1 --2- 172.21.15.52:0/2330743001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e30101e80 0x7f9e3019d640 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9e28004d00 tx=0x7f9e28004de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:26.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.192+0000 7f9e2d7fa700 1 -- 172.21.15.52:0/2330743001 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2800e040 con 0x7f9e30101e80 2023-11-27T15:17:26.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.192+0000 7f9e363e9700 1 -- 172.21.15.52:0/2330743001 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e3010bc50 con 0x7f9e30101e80 2023-11-27T15:17:26.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.192+0000 7f9e2d7fa700 1 -- 172.21.15.52:0/2330743001 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e2801bd20 con 0x7f9e30101e80 2023-11-27T15:17:26.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.192+0000 7f9e2d7fa700 1 -- 172.21.15.52:0/2330743001 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e28005940 con 0x7f9e30101e80 2023-11-27T15:17:26.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.192+0000 7f9e363e9700 1 -- 172.21.15.52:0/2330743001 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e3010bac0 con 0x7f9e30101e80 2023-11-27T15:17:26.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.193+0000 7f9e363e9700 1 -- 172.21.15.52:0/2330743001 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e30066ea0 con 0x7f9e30101e80 2023-11-27T15:17:26.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.195+0000 7f9e2d7fa700 1 -- 172.21.15.52:0/2330743001 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9e28005040 con 0x7f9e30101e80 2023-11-27T15:17:26.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.195+0000 7f9e2d7fa700 1 --2- 172.21.15.52:0/2330743001 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f9e180a11e0 0x7f9e180a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:26.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.195+0000 7f9e2d7fa700 1 -- 172.21.15.52:0/2330743001 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9e280cee20 con 0x7f9e30101e80 2023-11-27T15:17:26.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.195+0000 7f9e2ffff700 1 --2- 172.21.15.52:0/2330743001 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f9e180a11e0 0x7f9e180a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:26.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.196+0000 7f9e2ffff700 1 --2- 172.21.15.52:0/2330743001 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f9e180a11e0 0x7f9e180a3690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9e30101350 tx=0x7f9e20005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:26.203 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.199+0000 7f9e2d7fa700 1 -- 172.21.15.52:0/2330743001 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9e28098480 con 0x7f9e30101e80 2023-11-27T15:17:26.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:25 smithi150 ceph-mon[100559]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-27T15:17:26.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.439+0000 7f9e363e9700 1 -- 172.21.15.52:0/2330743001 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9e30107110 con 0x7f9e30101e80 2023-11-27T15:17:26.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.440+0000 7f9e2d7fa700 1 -- 172.21.15.52:0/2330743001 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+2897 (secure 0 0 0) 0x7f9e28020b70 con 0x7f9e30101e80 2023-11-27T15:17:26.444 INFO:teuthology.orchestra.run.smithi052.stdout:e23 2023-11-27T15:17:26.445 INFO:teuthology.orchestra.run.smithi052.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-11-27T15:17:26.445 INFO:teuthology.orchestra.run.smithi052.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-27T15:17:26.445 INFO:teuthology.orchestra.run.smithi052.stdout:legacy client fscid: 1 2023-11-27T15:17:26.445 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:17:26.445 INFO:teuthology.orchestra.run.smithi052.stdout:Filesystem 'cephfs' (1) 2023-11-27T15:17:26.445 INFO:teuthology.orchestra.run.smithi052.stdout:fs_name cephfs 2023-11-27T15:17:26.445 INFO:teuthology.orchestra.run.smithi052.stdout:epoch 23 2023-11-27T15:17:26.445 INFO:teuthology.orchestra.run.smithi052.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-11-27T15:17:26.445 INFO:teuthology.orchestra.run.smithi052.stdout:created 2023-11-27T15:16:09.319630+0000 2023-11-27T15:17:26.446 INFO:teuthology.orchestra.run.smithi052.stdout:modified 2023-11-27T15:16:14.742405+0000 2023-11-27T15:17:26.446 INFO:teuthology.orchestra.run.smithi052.stdout:tableserver 0 2023-11-27T15:17:26.446 INFO:teuthology.orchestra.run.smithi052.stdout:root 0 2023-11-27T15:17:26.446 INFO:teuthology.orchestra.run.smithi052.stdout:session_timeout 300 2023-11-27T15:17:26.446 INFO:teuthology.orchestra.run.smithi052.stdout:session_autoclose 300 2023-11-27T15:17:26.446 INFO:teuthology.orchestra.run.smithi052.stdout:max_file_size 1099511627776 2023-11-27T15:17:26.446 INFO:teuthology.orchestra.run.smithi052.stdout:max_xattr_size 65536 2023-11-27T15:17:26.446 INFO:teuthology.orchestra.run.smithi052.stdout:required_client_features {} 2023-11-27T15:17:26.447 INFO:teuthology.orchestra.run.smithi052.stdout:last_failure 0 2023-11-27T15:17:26.447 INFO:teuthology.orchestra.run.smithi052.stdout:last_failure_osd_epoch 0 2023-11-27T15:17:26.447 INFO:teuthology.orchestra.run.smithi052.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-27T15:17:26.447 INFO:teuthology.orchestra.run.smithi052.stdout:max_mds 3 2023-11-27T15:17:26.447 INFO:teuthology.orchestra.run.smithi052.stdout:in 0,1,2 2023-11-27T15:17:26.447 INFO:teuthology.orchestra.run.smithi052.stdout:up {0=24536,1=24494,2=24472} 2023-11-27T15:17:26.447 INFO:teuthology.orchestra.run.smithi052.stdout:failed 2023-11-27T15:17:26.447 INFO:teuthology.orchestra.run.smithi052.stdout:damaged 2023-11-27T15:17:26.448 INFO:teuthology.orchestra.run.smithi052.stdout:stopped 2023-11-27T15:17:26.448 INFO:teuthology.orchestra.run.smithi052.stdout:data_pools [3] 2023-11-27T15:17:26.448 INFO:teuthology.orchestra.run.smithi052.stdout:metadata_pool 2 2023-11-27T15:17:26.448 INFO:teuthology.orchestra.run.smithi052.stdout:inline_data disabled 2023-11-27T15:17:26.448 INFO:teuthology.orchestra.run.smithi052.stdout:balancer 2023-11-27T15:17:26.448 INFO:teuthology.orchestra.run.smithi052.stdout:bal_rank_mask -1 2023-11-27T15:17:26.448 INFO:teuthology.orchestra.run.smithi052.stdout:standby_count_wanted 0 2023-11-27T15:17:26.449 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.b{0:24536} state up:active seq 3 addr [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:17:26.449 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.i{0:24493} state up:standby-replay seq 1 addr [v2:172.21.15.150:6838/215725838,v1:172.21.15.150:6839/215725838] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:17:26.449 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.h{1:24494} state up:active seq 6 addr [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:17:26.449 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.e{1:24473} state up:standby-replay seq 1 addr [v2:172.21.15.102:6834/2398692013,v1:172.21.15.102:6835/2398692013] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:17:26.449 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.c{2:24472} state up:active seq 7 addr [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:17:26.449 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.f{2:24460} state up:standby-replay seq 1 addr [v2:172.21.15.150:6834/617413375,v1:172.21.15.150:6835/617413375] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:17:26.449 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:17:26.449 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:17:26.450 INFO:teuthology.orchestra.run.smithi052.stdout:Standby daemons: 2023-11-27T15:17:26.450 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:17:26.450 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.j{-1:14529} state up:standby seq 1 addr [v2:172.21.15.52:6834/3561136724,v1:172.21.15.52:6835/3561136724] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:17:26.450 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.d{-1:14535} state up:standby seq 1 addr [v2:172.21.15.52:6836/3583014325,v1:172.21.15.52:6837/3583014325] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:17:26.450 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.g{-1:14586} state up:standby seq 1 addr [v2:172.21.15.52:6838/2328881376,v1:172.21.15.52:6839/2328881376] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:17:26.450 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.a{-1:14598} state up:standby seq 1 addr [v2:172.21.15.52:6840/2799551570,v1:172.21.15.52:6841/2799551570] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:17:26.450 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.l{-1:24418} state up:standby seq 1 addr [v2:172.21.15.150:6832/1857157307,v1:172.21.15.150:6833/1857157307] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:17:26.450 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.k{-1:24446} state up:standby seq 1 addr [v2:172.21.15.102:6832/883247933,v1:172.21.15.102:6833/883247933] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:17:26.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.442+0000 7f9e363e9700 1 -- 172.21.15.52:0/2330743001 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f9e180a11e0 msgr2=0x7f9e180a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:26.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.442+0000 7f9e363e9700 1 --2- 172.21.15.52:0/2330743001 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f9e180a11e0 0x7f9e180a3690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9e30101350 tx=0x7f9e20005f90 comp rx=0 tx=0).stop 2023-11-27T15:17:26.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.442+0000 7f9e363e9700 1 -- 172.21.15.52:0/2330743001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e30101e80 msgr2=0x7f9e3019d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:26.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.442+0000 7f9e363e9700 1 --2- 172.21.15.52:0/2330743001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e30101e80 0x7f9e3019d640 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9e28004d00 tx=0x7f9e28004de0 comp rx=0 tx=0).stop 2023-11-27T15:17:26.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.442+0000 7f9e363e9700 1 -- 172.21.15.52:0/2330743001 shutdown_connections 2023-11-27T15:17:26.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.442+0000 7f9e363e9700 1 --2- 172.21.15.52:0/2330743001 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f9e180a11e0 0x7f9e180a3690 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:26.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.442+0000 7f9e363e9700 1 --2- 172.21.15.52:0/2330743001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e301002f0 0x7f9e30196080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:26.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.442+0000 7f9e363e9700 1 --2- 172.21.15.52:0/2330743001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e30101e80 0x7f9e3019d640 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:26.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.442+0000 7f9e363e9700 1 --2- 172.21.15.52:0/2330743001 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e301014f0 0x7f9e301965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:26.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.442+0000 7f9e363e9700 1 -- 172.21.15.52:0/2330743001 >> 172.21.15.52:0/2330743001 conn(0x7f9e300fb900 msgr2=0x7f9e300fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:26.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.442+0000 7f9e363e9700 1 -- 172.21.15.52:0/2330743001 shutdown_connections 2023-11-27T15:17:26.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:26.442+0000 7f9e363e9700 1 -- 172.21.15.52:0/2330743001 wait complete. 2023-11-27T15:17:26.453 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 23 2023-11-27T15:17:26.918 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- bash -c 'ceph osd dump' 2023-11-27T15:17:27.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:26 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2330743001' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-11-27T15:17:27.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:26 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2330743001' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-11-27T15:17:27.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:26 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2330743001' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-11-27T15:17:27.570 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:17:28.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:27 smithi102 ceph-mon[100296]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:17:28.113 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:27 smithi052 ceph-mon[99056]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:17:28.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:27 smithi150 ceph-mon[100559]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:17:28.879 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.874+0000 7fcb3e488700 1 -- 172.21.15.52:0/1717159928 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb38105fb0 msgr2=0x7fcb38108410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:28.879 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.874+0000 7fcb3e488700 1 --2- 172.21.15.52:0/1717159928 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb38105fb0 0x7fcb38108410 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcb3000b0a0 tx=0x7fcb3000b3b0 comp rx=0 tx=0).stop 2023-11-27T15:17:28.879 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.874+0000 7fcb3e488700 1 -- 172.21.15.52:0/1717159928 shutdown_connections 2023-11-27T15:17:28.879 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.874+0000 7fcb3e488700 1 --2- 172.21.15.52:0/1717159928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb38100d70 0x7fcb38103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:28.879 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.874+0000 7fcb3e488700 1 --2- 172.21.15.52:0/1717159928 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb38105fb0 0x7fcb38108410 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:28.880 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.874+0000 7fcb3e488700 1 --2- 172.21.15.52:0/1717159928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb38103690 0x7fcb38105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:28.880 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.874+0000 7fcb3e488700 1 -- 172.21.15.52:0/1717159928 >> 172.21.15.52:0/1717159928 conn(0x7fcb380fa7f0 msgr2=0x7fcb380fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:28.880 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.874+0000 7fcb3e488700 1 -- 172.21.15.52:0/1717159928 shutdown_connections 2023-11-27T15:17:28.880 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.874+0000 7fcb3e488700 1 -- 172.21.15.52:0/1717159928 wait complete. 2023-11-27T15:17:28.880 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.875+0000 7fcb3e488700 1 Processor -- start 2023-11-27T15:17:28.880 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.875+0000 7fcb3e488700 1 -- start start 2023-11-27T15:17:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.875+0000 7fcb3e488700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb38100d70 0x7fcb38196140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.875+0000 7fcb3e488700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb38103690 0x7fcb38196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.875+0000 7fcb3e488700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb38105fb0 0x7fcb3819d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.875+0000 7fcb3e488700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3819dcd0 con 0x7fcb38105fb0 2023-11-27T15:17:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.875+0000 7fcb3e488700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3819de10 con 0x7fcb38103690 2023-11-27T15:17:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.875+0000 7fcb3e488700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3819dfa0 con 0x7fcb38100d70 2023-11-27T15:17:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.875+0000 7fcb3ca25700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb38105fb0 0x7fcb3819d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.875+0000 7fcb3ca25700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb38105fb0 0x7fcb3819d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51046/0 (socket says 172.21.15.52:51046) 2023-11-27T15:17:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.875+0000 7fcb3ca25700 1 -- 172.21.15.52:0/2888755312 learned_addr learned my addr 172.21.15.52:0/2888755312 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.876+0000 7fcb377fe700 1 --2- 172.21.15.52:0/2888755312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb38103690 0x7fcb38196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.876+0000 7fcb37fff700 1 --2- 172.21.15.52:0/2888755312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb38100d70 0x7fcb38196140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:28.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.876+0000 7fcb3ca25700 1 -- 172.21.15.52:0/2888755312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb38100d70 msgr2=0x7fcb38196140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:28.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.876+0000 7fcb3ca25700 1 --2- 172.21.15.52:0/2888755312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb38100d70 0x7fcb38196140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:28.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.876+0000 7fcb3ca25700 1 -- 172.21.15.52:0/2888755312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb38103690 msgr2=0x7fcb38196680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:28.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.876+0000 7fcb3ca25700 1 --2- 172.21.15.52:0/2888755312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb38103690 0x7fcb38196680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:28.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.876+0000 7fcb3ca25700 1 -- 172.21.15.52:0/2888755312 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb30009d00 con 0x7fcb38105fb0 2023-11-27T15:17:28.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.876+0000 7fcb37fff700 1 --2- 172.21.15.52:0/2888755312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb38100d70 0x7fcb38196140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:17:28.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.876+0000 7fcb3ca25700 1 --2- 172.21.15.52:0/2888755312 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb38105fb0 0x7fcb3819d700 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcb3000b070 tx=0x7fcb30004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:28.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.876+0000 7fcb357fa700 1 -- 172.21.15.52:0/2888755312 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3000e040 con 0x7fcb38105fb0 2023-11-27T15:17:28.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.876+0000 7fcb357fa700 1 -- 172.21.15.52:0/2888755312 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb30005710 con 0x7fcb38105fb0 2023-11-27T15:17:28.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.876+0000 7fcb3e488700 1 -- 172.21.15.52:0/2888755312 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb3810bca0 con 0x7fcb38105fb0 2023-11-27T15:17:28.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.876+0000 7fcb357fa700 1 -- 172.21.15.52:0/2888755312 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3001dbf0 con 0x7fcb38105fb0 2023-11-27T15:17:28.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.876+0000 7fcb3e488700 1 -- 172.21.15.52:0/2888755312 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb3810bb10 con 0x7fcb38105fb0 2023-11-27T15:17:28.884 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.877+0000 7fcb3e488700 1 -- 172.21.15.52:0/2888755312 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb380fc360 con 0x7fcb38105fb0 2023-11-27T15:17:28.884 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.878+0000 7fcb357fa700 1 -- 172.21.15.52:0/2888755312 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fcb30005880 con 0x7fcb38105fb0 2023-11-27T15:17:28.884 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.879+0000 7fcb357fa700 1 --2- 172.21.15.52:0/2888755312 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fcb240a1260 0x7fcb240a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:28.885 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.879+0000 7fcb37fff700 1 --2- 172.21.15.52:0/2888755312 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fcb240a1260 0x7fcb240a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:28.885 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.879+0000 7fcb357fa700 1 -- 172.21.15.52:0/2888755312 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fcb300ce2f0 con 0x7fcb38105fb0 2023-11-27T15:17:28.885 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.879+0000 7fcb37fff700 1 --2- 172.21.15.52:0/2888755312 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fcb240a1260 0x7fcb240a3710 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcb20005fd0 tx=0x7fcb20005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:28.887 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:28.883+0000 7fcb357fa700 1 -- 172.21.15.52:0/2888755312 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcb30097950 con 0x7fcb38105fb0 2023-11-27T15:17:29.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.080+0000 7fcb3e488700 1 -- 172.21.15.52:0/2888755312 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fcb3802cfe0 con 0x7fcb38105fb0 2023-11-27T15:17:29.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.081+0000 7fcb357fa700 1 -- 172.21.15.52:0/2888755312 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v85) v1 ==== 56+0+5337 (secure 0 0 0) 0x7fcb3009af70 con 0x7fcb38105fb0 2023-11-27T15:17:29.085 INFO:teuthology.orchestra.run.smithi052.stdout:epoch 85 2023-11-27T15:17:29.086 INFO:teuthology.orchestra.run.smithi052.stdout:fsid c6c7df88-8d36-11ee-95a2-87774f69a715 2023-11-27T15:17:29.086 INFO:teuthology.orchestra.run.smithi052.stdout:created 2023-11-27T15:09:03.333345+0000 2023-11-27T15:17:29.086 INFO:teuthology.orchestra.run.smithi052.stdout:modified 2023-11-27T15:17:04.797892+0000 2023-11-27T15:17:29.086 INFO:teuthology.orchestra.run.smithi052.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-11-27T15:17:29.086 INFO:teuthology.orchestra.run.smithi052.stdout:crush_version 26 2023-11-27T15:17:29.086 INFO:teuthology.orchestra.run.smithi052.stdout:full_ratio 0.95 2023-11-27T15:17:29.086 INFO:teuthology.orchestra.run.smithi052.stdout:backfillfull_ratio 0.9 2023-11-27T15:17:29.086 INFO:teuthology.orchestra.run.smithi052.stdout:nearfull_ratio 0.85 2023-11-27T15:17:29.086 INFO:teuthology.orchestra.run.smithi052.stdout:require_min_compat_client luminous 2023-11-27T15:17:29.086 INFO:teuthology.orchestra.run.smithi052.stdout:min_compat_client luminous 2023-11-27T15:17:29.086 INFO:teuthology.orchestra.run.smithi052.stdout:require_osd_release reef 2023-11-27T15:17:29.087 INFO:teuthology.orchestra.run.smithi052.stdout:stretch_mode_enabled false 2023-11-27T15:17:29.087 INFO:teuthology.orchestra.run.smithi052.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-11-27T15:17:29.087 INFO:teuthology.orchestra.run.smithi052.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-11-27T15:17:29.087 INFO:teuthology.orchestra.run.smithi052.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2023-11-27T15:17:29.087 INFO:teuthology.orchestra.run.smithi052.stdout:max_osd 12 2023-11-27T15:17:29.087 INFO:teuthology.orchestra.run.smithi052.stdout:osd.0 up in weight 1 up_from 8 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.52:6802/608216415,v1:172.21.15.52:6803/608216415] [v2:172.21.15.52:6804/608216415,v1:172.21.15.52:6805/608216415] exists,up 58d69ede-c293-49f5-abbe-8187793f2b8e 2023-11-27T15:17:29.087 INFO:teuthology.orchestra.run.smithi052.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6800/2538384087,v1:172.21.15.102:6801/2538384087] [v2:172.21.15.102:6802/2538384087,v1:172.21.15.102:6803/2538384087] exists,up d6a46b91-adc8-45ef-80f5-566ae400911f 2023-11-27T15:17:29.087 INFO:teuthology.orchestra.run.smithi052.stdout:osd.2 up in weight 1 up_from 18 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6800/4258038221,v1:172.21.15.150:6801/4258038221] [v2:172.21.15.150:6802/4258038221,v1:172.21.15.150:6803/4258038221] exists,up 846ba125-11f8-46b2-8941-bd5e1947cc8a 2023-11-27T15:17:29.087 INFO:teuthology.orchestra.run.smithi052.stdout:osd.3 up in weight 1 up_from 25 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.52:6810/2622100752,v1:172.21.15.52:6811/2622100752] [v2:172.21.15.52:6812/2622100752,v1:172.21.15.52:6813/2622100752] exists,up 950fcd67-1057-4288-a389-714f1ec94261 2023-11-27T15:17:29.087 INFO:teuthology.orchestra.run.smithi052.stdout:osd.4 up in weight 1 up_from 33 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6808/1546794763,v1:172.21.15.102:6809/1546794763] [v2:172.21.15.102:6810/1546794763,v1:172.21.15.102:6811/1546794763] exists,up 1c96a676-cea9-4deb-bad1-f7fd240c3d95 2023-11-27T15:17:29.087 INFO:teuthology.orchestra.run.smithi052.stdout:osd.5 up in weight 1 up_from 38 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6808/461511375,v1:172.21.15.150:6809/461511375] [v2:172.21.15.150:6810/461511375,v1:172.21.15.150:6811/461511375] exists,up 38a75af1-fd4d-433e-b579-0609109c851c 2023-11-27T15:17:29.088 INFO:teuthology.orchestra.run.smithi052.stdout:osd.6 up in weight 1 up_from 45 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.52:6818/1002457464,v1:172.21.15.52:6819/1002457464] [v2:172.21.15.52:6820/1002457464,v1:172.21.15.52:6821/1002457464] exists,up 2f0ede4a-d175-4b4a-b876-c0f442048901 2023-11-27T15:17:29.088 INFO:teuthology.orchestra.run.smithi052.stdout:osd.7 up in weight 1 up_from 50 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6816/3249840064,v1:172.21.15.102:6817/3249840064] [v2:172.21.15.102:6818/3249840064,v1:172.21.15.102:6819/3249840064] exists,up 6b635192-abcf-4bbf-bf78-a019b351bca5 2023-11-27T15:17:29.088 INFO:teuthology.orchestra.run.smithi052.stdout:osd.8 up in weight 1 up_from 58 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6816/3672583574,v1:172.21.15.150:6817/3672583574] [v2:172.21.15.150:6818/3672583574,v1:172.21.15.150:6819/3672583574] exists,up c7ca8007-7170-47bf-956c-3f00edf27e74 2023-11-27T15:17:29.088 INFO:teuthology.orchestra.run.smithi052.stdout:osd.9 up in weight 1 up_from 66 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.52:6826/4287934320,v1:172.21.15.52:6827/4287934320] [v2:172.21.15.52:6828/4287934320,v1:172.21.15.52:6829/4287934320] exists,up 0e252c2c-01a4-48ba-832d-067ed940fefb 2023-11-27T15:17:29.088 INFO:teuthology.orchestra.run.smithi052.stdout:osd.10 up in weight 1 up_from 71 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6824/3056034201,v1:172.21.15.102:6825/3056034201] [v2:172.21.15.102:6826/3056034201,v1:172.21.15.102:6827/3056034201] exists,up bb540142-7563-44a4-b7a3-cb662714c128 2023-11-27T15:17:29.088 INFO:teuthology.orchestra.run.smithi052.stdout:osd.11 up in weight 1 up_from 78 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6824/3207559739,v1:172.21.15.150:6825/3207559739] [v2:172.21.15.150:6826/3207559739,v1:172.21.15.150:6827/3207559739] exists,up 3156528e-e0f8-4d44-9946-c8b6339c2bd4 2023-11-27T15:17:29.088 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 2.0 [8,1] 2023-11-27T15:17:29.088 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 2.5 [8,11] 2023-11-27T15:17:29.088 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 2.13 [2,11] 2023-11-27T15:17:29.088 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 2.1b [2,4] 2023-11-27T15:17:29.088 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 3.2 [3,9] 2023-11-27T15:17:29.089 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 3.12 [3,9] 2023-11-27T15:17:29.089 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 3.21 [4,6] 2023-11-27T15:17:29.089 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 3.23 [4,8] 2023-11-27T15:17:29.089 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 3.37 [5,7] 2023-11-27T15:17:29.089 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 3.3c [4,9] 2023-11-27T15:17:29.089 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:0/1392282110 expires 2023-11-28T15:10:03.644870+0000 2023-11-27T15:17:29.089 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:0/1479993479 expires 2023-11-28T15:10:03.644870+0000 2023-11-27T15:17:29.089 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:6801/3470359117 expires 2023-11-28T15:10:03.644870+0000 2023-11-27T15:17:29.089 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:6800/3470359117 expires 2023-11-28T15:10:03.644870+0000 2023-11-27T15:17:29.089 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:6801/960704660 expires 2023-11-28T15:09:37.118690+0000 2023-11-27T15:17:29.089 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:0/961562652 expires 2023-11-28T15:09:37.118690+0000 2023-11-27T15:17:29.089 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:0/1290665390 expires 2023-11-28T15:09:37.118690+0000 2023-11-27T15:17:29.089 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:0/2410529742 expires 2023-11-28T15:09:37.118690+0000 2023-11-27T15:17:29.090 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:6800/960704660 expires 2023-11-28T15:09:37.118690+0000 2023-11-27T15:17:29.090 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:0/2130037044 expires 2023-11-28T15:10:03.644870+0000 2023-11-27T15:17:29.090 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.083+0000 7fcb3e488700 1 -- 172.21.15.52:0/2888755312 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fcb240a1260 msgr2=0x7fcb240a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:29.090 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.083+0000 7fcb3e488700 1 --2- 172.21.15.52:0/2888755312 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fcb240a1260 0x7fcb240a3710 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcb20005fd0 tx=0x7fcb20005dc0 comp rx=0 tx=0).stop 2023-11-27T15:17:29.090 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.083+0000 7fcb3e488700 1 -- 172.21.15.52:0/2888755312 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb38105fb0 msgr2=0x7fcb3819d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:29.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.083+0000 7fcb3e488700 1 --2- 172.21.15.52:0/2888755312 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb38105fb0 0x7fcb3819d700 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcb3000b070 tx=0x7fcb30004d30 comp rx=0 tx=0).stop 2023-11-27T15:17:29.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.084+0000 7fcb3e488700 1 -- 172.21.15.52:0/2888755312 shutdown_connections 2023-11-27T15:17:29.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.084+0000 7fcb3e488700 1 --2- 172.21.15.52:0/2888755312 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fcb240a1260 0x7fcb240a3710 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:29.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.084+0000 7fcb3e488700 1 --2- 172.21.15.52:0/2888755312 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb38105fb0 0x7fcb3819d700 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:29.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.084+0000 7fcb3e488700 1 --2- 172.21.15.52:0/2888755312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb38100d70 0x7fcb38196140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:29.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.084+0000 7fcb3e488700 1 --2- 172.21.15.52:0/2888755312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb38103690 0x7fcb38196680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:29.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.084+0000 7fcb3e488700 1 -- 172.21.15.52:0/2888755312 >> 172.21.15.52:0/2888755312 conn(0x7fcb380fa7f0 msgr2=0x7fcb381019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:29.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.084+0000 7fcb3e488700 1 -- 172.21.15.52:0/2888755312 shutdown_connections 2023-11-27T15:17:29.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.084+0000 7fcb3e488700 1 -- 172.21.15.52:0/2888755312 wait complete. 2023-11-27T15:17:29.518 INFO:teuthology.run_tasks:Running task fs.ready... 2023-11-27T15:17:29.530 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:17:29.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.656+0000 7fbf68868700 1 -- 172.21.15.52:0/3113589298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf64100db0 msgr2=0x7fbf64107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:29.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.656+0000 7fbf68868700 1 --2- 172.21.15.52:0/3113589298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf64100db0 0x7fbf64107650 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00b0a0 tx=0x7fbf5c00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:17:29.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.656+0000 7fbf68868700 1 -- 172.21.15.52:0/3113589298 shutdown_connections 2023-11-27T15:17:29.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.656+0000 7fbf68868700 1 --2- 172.21.15.52:0/3113589298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf640ff170 0x7fbf640ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:29.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.656+0000 7fbf68868700 1 --2- 172.21.15.52:0/3113589298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf64100db0 0x7fbf64107650 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:29.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.656+0000 7fbf68868700 1 --2- 172.21.15.52:0/3113589298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf64100400 0x7fbf64100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:29.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.656+0000 7fbf68868700 1 -- 172.21.15.52:0/3113589298 >> 172.21.15.52:0/3113589298 conn(0x7fbf640fa750 msgr2=0x7fbf640fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:29.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.656+0000 7fbf68868700 1 -- 172.21.15.52:0/3113589298 shutdown_connections 2023-11-27T15:17:29.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.656+0000 7fbf68868700 1 -- 172.21.15.52:0/3113589298 wait complete. 2023-11-27T15:17:29.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.657+0000 7fbf68868700 1 Processor -- start 2023-11-27T15:17:29.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.657+0000 7fbf68868700 1 -- start start 2023-11-27T15:17:29.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.657+0000 7fbf68868700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf640ff170 0x7fbf64195f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:29.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.657+0000 7fbf68868700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf64100400 0x7fbf64196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:29.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.657+0000 7fbf68868700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf64100db0 0x7fbf6419d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:29.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.657+0000 7fbf68868700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6419dad0 con 0x7fbf640ff170 2023-11-27T15:17:29.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.657+0000 7fbf68868700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6419dc10 con 0x7fbf64100db0 2023-11-27T15:17:29.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.657+0000 7fbf68868700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6419dda0 con 0x7fbf64100400 2023-11-27T15:17:29.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.657+0000 7fbf6259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf640ff170 0x7fbf64195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:29.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.657+0000 7fbf6259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf640ff170 0x7fbf64195f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51056/0 (socket says 172.21.15.52:51056) 2023-11-27T15:17:29.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.657+0000 7fbf6259c700 1 -- 172.21.15.52:0/3025933463 learned_addr learned my addr 172.21.15.52:0/3025933463 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:29.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.657+0000 7fbf62d9d700 1 --2- 172.21.15.52:0/3025933463 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf64100db0 0x7fbf6419d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:29.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.657+0000 7fbf61d9b700 1 --2- 172.21.15.52:0/3025933463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf64100400 0x7fbf64196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:29.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.657+0000 7fbf6259c700 1 -- 172.21.15.52:0/3025933463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf64100400 msgr2=0x7fbf64196490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:29.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.657+0000 7fbf6259c700 1 --2- 172.21.15.52:0/3025933463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf64100400 0x7fbf64196490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:29.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.657+0000 7fbf6259c700 1 -- 172.21.15.52:0/3025933463 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf64100db0 msgr2=0x7fbf6419d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:29.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.657+0000 7fbf6259c700 1 --2- 172.21.15.52:0/3025933463 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf64100db0 0x7fbf6419d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:29.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.657+0000 7fbf6259c700 1 -- 172.21.15.52:0/3025933463 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5c009d00 con 0x7fbf640ff170 2023-11-27T15:17:29.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.657+0000 7fbf61d9b700 1 --2- 172.21.15.52:0/3025933463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf64100400 0x7fbf64196490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:17:29.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.657+0000 7fbf6259c700 1 --2- 172.21.15.52:0/3025933463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf640ff170 0x7fbf64195f50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbf5400c9b0 tx=0x7fbf5400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:29.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.658+0000 7fbf5b7fe700 1 -- 172.21.15.52:0/3025933463 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf54007b00 con 0x7fbf640ff170 2023-11-27T15:17:29.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.658+0000 7fbf5b7fe700 1 -- 172.21.15.52:0/3025933463 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf54007c60 con 0x7fbf640ff170 2023-11-27T15:17:29.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.658+0000 7fbf68868700 1 -- 172.21.15.52:0/3025933463 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf6410ab70 con 0x7fbf640ff170 2023-11-27T15:17:29.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.658+0000 7fbf5b7fe700 1 -- 172.21.15.52:0/3025933463 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf54018660 con 0x7fbf640ff170 2023-11-27T15:17:29.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.658+0000 7fbf68868700 1 -- 172.21.15.52:0/3025933463 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf6410a9e0 con 0x7fbf640ff170 2023-11-27T15:17:29.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.658+0000 7fbf68868700 1 -- 172.21.15.52:0/3025933463 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf64107180 con 0x7fbf640ff170 2023-11-27T15:17:29.666 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.660+0000 7fbf5b7fe700 1 -- 172.21.15.52:0/3025933463 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fbf5400f450 con 0x7fbf640ff170 2023-11-27T15:17:29.666 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.660+0000 7fbf5b7fe700 1 --2- 172.21.15.52:0/3025933463 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fbf500a1260 0x7fbf500a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:29.666 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.660+0000 7fbf5b7fe700 1 -- 172.21.15.52:0/3025933463 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fbf540cccd0 con 0x7fbf640ff170 2023-11-27T15:17:29.666 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.660+0000 7fbf61d9b700 1 --2- 172.21.15.52:0/3025933463 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fbf500a1260 0x7fbf500a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:29.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.661+0000 7fbf61d9b700 1 --2- 172.21.15.52:0/3025933463 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fbf500a1260 0x7fbf500a3710 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c005fd0 tx=0x7fbf4c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:29.668 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.664+0000 7fbf5b7fe700 1 -- 172.21.15.52:0/3025933463 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbf540963b0 con 0x7fbf640ff170 2023-11-27T15:17:29.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.864+0000 7fbf68868700 1 -- 172.21.15.52:0/3025933463 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf6402cfe0 con 0x7fbf640ff170 2023-11-27T15:17:29.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.866+0000 7fbf5b7fe700 1 -- 172.21.15.52:0/3025933463 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22657 (secure 0 0 0) 0x7fbf540999d0 con 0x7fbf640ff170 2023-11-27T15:17:29.870 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:17:29.870 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":85,"fsid":"c6c7df88-8d36-11ee-95a2-87774f69a715","created":"2023-11-27T15:09:03.333345+0000","modified":"2023-11-27T15:17:04.797892+0000","last_up_change":"2023-11-27T15:15:27.656134+0000","last_in_change":"2023-11-27T15:15:10.671153+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:12:14.177523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:16:07.842251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:16:08.728520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58d69ede-c293-49f5-abbe-8187793f2b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6803","nonce":608216415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6805","nonce":608216415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6809","nonce":608216415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6807","nonce":608216415}]},"public_addr":"172.21.15.52:6803/608216415","cluster_addr":"172.21.15.52:6805/608216415","heartbeat_back_addr":"172.21.15.52:6809/608216415","heartbeat_front_addr":"172.21.15.52:6807/608216415","state":["exists","up"]},{"osd":1,"uuid":"d6a46b91-adc8-45ef-80f5-566ae400911f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6801","nonce":2538384087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6803","nonce":2538384087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6807","nonce":2538384087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6805","nonce":2538384087}]},"public_addr":"172.21.15.102:6801/2538384087","cluster_addr":"172.21.15.102:6803/2538384087","heartbeat_back_addr":"172.21.15.102:6807/2538384087","heartbeat_front_addr":"172.21.15.102:6805/2538384087","state":["exists","up"]},{"osd":2,"uuid":"846ba125-11f8-46b2-8941-bd5e1947cc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6801","nonce":4258038221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6803","nonce":4258038221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6807","nonce":4258038221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6805","nonce":4258038221}]},"public_addr":"172.21.15.150:6801/4258038221","cluster_addr":"172.21.15.150:6803/4258038221","heartbeat_back_addr":"172.21.15.150:6807/4258038221","heartbeat_front_addr":"172.21.15.150:6805/4258038221","state":["exists","up"]},{"osd":3,"uuid":"950fcd67-1057-4288-a389-714f1ec94261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6811","nonce":2622100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6813","nonce":2622100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6817","nonce":2622100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6815","nonce":2622100752}]},"public_addr":"172.21.15.52:6811/2622100752","cluster_addr":"172.21.15.52:6813/2622100752","heartbeat_back_addr":"172.21.15.52:6817/2622100752","heartbeat_front_addr":"172.21.15.52:6815/2622100752","state":["exists","up"]},{"osd":4,"uuid":"1c96a676-cea9-4deb-bad1-f7fd240c3d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6809","nonce":1546794763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6811","nonce":1546794763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6815","nonce":1546794763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6813","nonce":1546794763}]},"public_addr":"172.21.15.102:6809/1546794763","cluster_addr":"172.21.15.102:6811/1546794763","heartbeat_back_addr":"172.21.15.102:6815/1546794763","heartbeat_front_addr":"172.21.15.102:6813/1546794763","state":["exists","up"]},{"osd":5,"uuid":"38a75af1-fd4d-433e-b579-0609109c851c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6809","nonce":461511375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6811","nonce":461511375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6815","nonce":461511375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6813","nonce":461511375}]},"public_addr":"172.21.15.150:6809/461511375","cluster_addr":"172.21.15.150:6811/461511375","heartbeat_back_addr":"172.21.15.150:6815/461511375","heartbeat_front_addr":"172.21.15.150:6813/461511375","state":["exists","up"]},{"osd":6,"uuid":"2f0ede4a-d175-4b4a-b876-c0f442048901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6819","nonce":1002457464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6821","nonce":1002457464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6825","nonce":1002457464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6823","nonce":1002457464}]},"public_addr":"172.21.15.52:6819/1002457464","cluster_addr":"172.21.15.52:6821/1002457464","heartbeat_back_addr":"172.21.15.52:6825/1002457464","heartbeat_front_addr":"172.21.15.52:6823/1002457464","state":["exists","up"]},{"osd":7,"uuid":"6b635192-abcf-4bbf-bf78-a019b351bca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6817","nonce":3249840064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6819","nonce":3249840064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6823","nonce":3249840064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6821","nonce":3249840064}]},"public_addr":"172.21.15.102:6817/3249840064","cluster_addr":"172.21.15.102:6819/3249840064","heartbeat_back_addr":"172.21.15.102:6823/3249840064","heartbeat_front_addr":"172.21.15.102:6821/3249840064","state":["exists","up"]},{"osd":8,"uuid":"c7ca8007-7170-47bf-956c-3f00edf27e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6817","nonce":3672583574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6819","nonce":3672583574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6823","nonce":3672583574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6821","nonce":3672583574}]},"public_addr":"172.21.15.150:6817/3672583574","cluster_addr":"172.21.15.150:6819/3672583574","heartbeat_back_addr":"172.21.15.150:6823/3672583574","heartbeat_front_addr":"172.21.15.150:6821/3672583574","state":["exists","up"]},{"osd":9,"uuid":"0e252c2c-01a4-48ba-832d-067ed940fefb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6827","nonce":4287934320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6829","nonce":4287934320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6833","nonce":4287934320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6831","nonce":4287934320}]},"public_addr":"172.21.15.52:6827/4287934320","cluster_addr":"172.21.15.52:6829/4287934320","heartbeat_back_addr":"172.21.15.52:6833/4287934320","heartbeat_front_addr":"172.21.15.52:6831/4287934320","state":["exists","up"]},{"osd":10,"uuid":"bb540142-7563-44a4-b7a3-cb662714c128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6825","nonce":3056034201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6827","nonce":3056034201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6831","nonce":3056034201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6829","nonce":3056034201}]},"public_addr":"172.21.15.102:6825/3056034201","cluster_addr":"172.21.15.102:6827/3056034201","heartbeat_back_addr":"172.21.15.102:6831/3056034201","heartbeat_front_addr":"172.21.15.102:6829/3056034201","state":["exists","up"]},{"osd":11,"uuid":"3156528e-e0f8-4d44-9946-c8b6339c2bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6825","nonce":3207559739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6827","nonce":3207559739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6831","nonce":3207559739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6829","nonce":3207559739}]},"public_addr":"172.21.15.150:6825/3207559739","cluster_addr":"172.21.15.150:6827/3207559739","heartbeat_back_addr":"172.21.15.150:6831/3207559739","heartbeat_front_addr":"172.21.15.150:6829/3207559739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:26.826099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:48.331371+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:10.459738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:32.356472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:54.043170+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:15.648582+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:37.166463+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:59.240819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:20.710097+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:43.364563+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:05.279433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:26.344573+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":5,"to":7}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/1392282110":"2023-11-28T15:10:03.644870+0000","172.21.15.52:0/1479993479":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6800/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/961562652":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/1290665390":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2410529742":"2023-11-28T15:09:37.118690+0000","172.21.15.52:6800/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2130037044":"2023-11-28T15:10:03.644870+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:17:29.872 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.868+0000 7fbf68868700 1 -- 172.21.15.52:0/3025933463 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fbf500a1260 msgr2=0x7fbf500a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:29.872 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.868+0000 7fbf68868700 1 --2- 172.21.15.52:0/3025933463 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fbf500a1260 0x7fbf500a3710 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c005fd0 tx=0x7fbf4c005e30 comp rx=0 tx=0).stop 2023-11-27T15:17:29.872 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.868+0000 7fbf68868700 1 -- 172.21.15.52:0/3025933463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf640ff170 msgr2=0x7fbf64195f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:29.873 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.868+0000 7fbf68868700 1 --2- 172.21.15.52:0/3025933463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf640ff170 0x7fbf64195f50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbf5400c9b0 tx=0x7fbf5400ccc0 comp rx=0 tx=0).stop 2023-11-27T15:17:29.873 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.868+0000 7fbf68868700 1 -- 172.21.15.52:0/3025933463 shutdown_connections 2023-11-27T15:17:29.873 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.868+0000 7fbf68868700 1 --2- 172.21.15.52:0/3025933463 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fbf500a1260 0x7fbf500a3710 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:29.873 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.868+0000 7fbf68868700 1 --2- 172.21.15.52:0/3025933463 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf64100db0 0x7fbf6419d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:29.873 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.868+0000 7fbf68868700 1 --2- 172.21.15.52:0/3025933463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf640ff170 0x7fbf64195f50 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:29.873 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.868+0000 7fbf68868700 1 --2- 172.21.15.52:0/3025933463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf64100400 0x7fbf64196490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:29.873 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.868+0000 7fbf68868700 1 -- 172.21.15.52:0/3025933463 >> 172.21.15.52:0/3025933463 conn(0x7fbf640fa750 msgr2=0x7fbf640fc880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:29.873 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.868+0000 7fbf68868700 1 -- 172.21.15.52:0/3025933463 shutdown_connections 2023-11-27T15:17:29.873 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:29.868+0000 7fbf68868700 1 -- 172.21.15.52:0/3025933463 wait complete. 2023-11-27T15:17:29.885 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:12:14.177523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-27T15:16:07.842251+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-27T15:16:08.728520+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-27T15:17:29.885 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:17:30.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.010+0000 7f575ff4e700 1 -- 172.21.15.52:0/948290744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5758100d70 msgr2=0x7f5758107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:30.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.010+0000 7f575ff4e700 1 --2- 172.21.15.52:0/948290744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5758100d70 0x7f5758107630 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f575000b0a0 tx=0x7f575000b3b0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.010+0000 7f575ff4e700 1 -- 172.21.15.52:0/948290744 shutdown_connections 2023-11-27T15:17:30.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.010+0000 7f575ff4e700 1 --2- 172.21.15.52:0/948290744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5758100d70 0x7f5758107630 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.010+0000 7f575ff4e700 1 --2- 172.21.15.52:0/948290744 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57580ff1f0 0x7f57580ff600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.010+0000 7f575ff4e700 1 --2- 172.21.15.52:0/948290744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f57581003e0 0x7f5758100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.010+0000 7f575ff4e700 1 -- 172.21.15.52:0/948290744 >> 172.21.15.52:0/948290744 conn(0x7f57580fa810 msgr2=0x7f57580fcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:30.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.010+0000 7f575ff4e700 1 -- 172.21.15.52:0/948290744 shutdown_connections 2023-11-27T15:17:30.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.010+0000 7f575ff4e700 1 -- 172.21.15.52:0/948290744 wait complete. 2023-11-27T15:17:30.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.011+0000 7f575ff4e700 1 Processor -- start 2023-11-27T15:17:30.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.011+0000 7f575ff4e700 1 -- start start 2023-11-27T15:17:30.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.011+0000 7f575ff4e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57580ff1f0 0x7f5758103180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:30.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.011+0000 7f575ff4e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57581003e0 0x7f57581017c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:30.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.011+0000 7f575ff4e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5758100d70 0x7f5758101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:30.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.011+0000 7f575ff4e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57581024a0 con 0x7f57580ff1f0 2023-11-27T15:17:30.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.011+0000 7f575ff4e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5758102610 con 0x7f5758100d70 2023-11-27T15:17:30.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.011+0000 7f575ff4e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57581a01c0 con 0x7f57581003e0 2023-11-27T15:17:30.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.011+0000 7f575dcea700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57580ff1f0 0x7f5758103180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:30.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.011+0000 7f575dcea700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57580ff1f0 0x7f5758103180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51070/0 (socket says 172.21.15.52:51070) 2023-11-27T15:17:30.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.011+0000 7f575dcea700 1 -- 172.21.15.52:0/857023045 learned_addr learned my addr 172.21.15.52:0/857023045 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:30.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.011+0000 7f575e4eb700 1 --2- 172.21.15.52:0/857023045 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5758100d70 0x7f5758101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:30.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.011+0000 7f575d4e9700 1 --2- 172.21.15.52:0/857023045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57581003e0 0x7f57581017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:30.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.011+0000 7f575dcea700 1 -- 172.21.15.52:0/857023045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57581003e0 msgr2=0x7f57581017c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:30.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.011+0000 7f575dcea700 1 --2- 172.21.15.52:0/857023045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57581003e0 0x7f57581017c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.011+0000 7f575dcea700 1 -- 172.21.15.52:0/857023045 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5758100d70 msgr2=0x7f5758101d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:30.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.011+0000 7f575dcea700 1 --2- 172.21.15.52:0/857023045 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5758100d70 0x7f5758101d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.011+0000 7f575dcea700 1 -- 172.21.15.52:0/857023045 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5750009d00 con 0x7f57580ff1f0 2023-11-27T15:17:30.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.012+0000 7f575dcea700 1 --2- 172.21.15.52:0/857023045 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57580ff1f0 0x7f5758103180 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f574800d980 tx=0x7f574800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:30.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.012+0000 7f574effd700 1 -- 172.21.15.52:0/857023045 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5748004020 con 0x7f57580ff1f0 2023-11-27T15:17:30.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.012+0000 7f574effd700 1 -- 172.21.15.52:0/857023045 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f574800f690 con 0x7f57580ff1f0 2023-11-27T15:17:30.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.012+0000 7f575ff4e700 1 -- 172.21.15.52:0/857023045 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f575810abb0 con 0x7f57580ff1f0 2023-11-27T15:17:30.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.012+0000 7f574effd700 1 -- 172.21.15.52:0/857023045 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5748003a60 con 0x7f57580ff1f0 2023-11-27T15:17:30.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.012+0000 7f575ff4e700 1 -- 172.21.15.52:0/857023045 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f575810aa20 con 0x7f57580ff1f0 2023-11-27T15:17:30.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.013+0000 7f575ff4e700 1 -- 172.21.15.52:0/857023045 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57581a0930 con 0x7f57580ff1f0 2023-11-27T15:17:30.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.014+0000 7f574effd700 1 -- 172.21.15.52:0/857023045 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f574800fb00 con 0x7f57580ff1f0 2023-11-27T15:17:30.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.015+0000 7f574effd700 1 --2- 172.21.15.52:0/857023045 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f57440a1260 0x7f57440a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:30.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.015+0000 7f575d4e9700 1 --2- 172.21.15.52:0/857023045 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f57440a1260 0x7f57440a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:30.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.015+0000 7f574effd700 1 -- 172.21.15.52:0/857023045 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f57480cc970 con 0x7f57580ff1f0 2023-11-27T15:17:30.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.015+0000 7f575d4e9700 1 --2- 172.21.15.52:0/857023045 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f57440a1260 0x7f57440a3710 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5754009c60 tx=0x7f5754009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:30.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.018+0000 7f574effd700 1 -- 172.21.15.52:0/857023045 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5748095fc0 con 0x7f57580ff1f0 2023-11-27T15:17:30.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:29 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2888755312' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-11-27T15:17:30.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:29 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2888755312' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-11-27T15:17:30.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:29 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2888755312' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-11-27T15:17:30.220 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.216+0000 7f575ff4e700 1 -- 172.21.15.52:0/857023045 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f57581a0ee0 con 0x7f57580ff1f0 2023-11-27T15:17:30.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.216+0000 7f574effd700 1 -- 172.21.15.52:0/857023045 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f57480995e0 con 0x7f57580ff1f0 2023-11-27T15:17:30.221 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2023-11-27T15:17:30.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.219+0000 7f575ff4e700 1 -- 172.21.15.52:0/857023045 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f57440a1260 msgr2=0x7f57440a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:30.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.219+0000 7f575ff4e700 1 --2- 172.21.15.52:0/857023045 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f57440a1260 0x7f57440a3710 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5754009c60 tx=0x7f5754009260 comp rx=0 tx=0).stop 2023-11-27T15:17:30.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.219+0000 7f575ff4e700 1 -- 172.21.15.52:0/857023045 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57580ff1f0 msgr2=0x7f5758103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:30.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.219+0000 7f575ff4e700 1 --2- 172.21.15.52:0/857023045 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57580ff1f0 0x7f5758103180 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f574800d980 tx=0x7f574800dc90 comp rx=0 tx=0).stop 2023-11-27T15:17:30.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.219+0000 7f575ff4e700 1 -- 172.21.15.52:0/857023045 shutdown_connections 2023-11-27T15:17:30.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.219+0000 7f575ff4e700 1 --2- 172.21.15.52:0/857023045 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f57440a1260 0x7f57440a3710 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.219+0000 7f575ff4e700 1 --2- 172.21.15.52:0/857023045 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5758100d70 0x7f5758101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.219+0000 7f575ff4e700 1 --2- 172.21.15.52:0/857023045 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57580ff1f0 0x7f5758103180 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.219+0000 7f575ff4e700 1 --2- 172.21.15.52:0/857023045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57581003e0 0x7f57581017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.219+0000 7f575ff4e700 1 -- 172.21.15.52:0/857023045 >> 172.21.15.52:0/857023045 conn(0x7f57580fa810 msgr2=0x7f5758068e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:30.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.219+0000 7f575ff4e700 1 -- 172.21.15.52:0/857023045 shutdown_connections 2023-11-27T15:17:30.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.219+0000 7f575ff4e700 1 -- 172.21.15.52:0/857023045 wait complete. 2023-11-27T15:17:30.236 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-27T15:17:30.366 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.360+0000 7f762af07700 1 -- 172.21.15.52:0/3002756261 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7624100d40 msgr2=0x7f7624107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:30.367 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.360+0000 7f762af07700 1 --2- 172.21.15.52:0/3002756261 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7624100d40 0x7f7624107600 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f761c00b0a0 tx=0x7f761c00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.367 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.360+0000 7f762af07700 1 -- 172.21.15.52:0/3002756261 shutdown_connections 2023-11-27T15:17:30.367 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.360+0000 7f762af07700 1 --2- 172.21.15.52:0/3002756261 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7624100d40 0x7f7624107600 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.367 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.360+0000 7f762af07700 1 --2- 172.21.15.52:0/3002756261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76240ff1c0 0x7f76240ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.367 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.360+0000 7f762af07700 1 --2- 172.21.15.52:0/3002756261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76241003b0 0x7f7624100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.367 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.360+0000 7f762af07700 1 -- 172.21.15.52:0/3002756261 >> 172.21.15.52:0/3002756261 conn(0x7f76240fa7c0 msgr2=0x7f76240fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:30.367 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.360+0000 7f762af07700 1 -- 172.21.15.52:0/3002756261 shutdown_connections 2023-11-27T15:17:30.367 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.360+0000 7f762af07700 1 -- 172.21.15.52:0/3002756261 wait complete. 2023-11-27T15:17:30.367 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.361+0000 7f762af07700 1 Processor -- start 2023-11-27T15:17:30.367 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.361+0000 7f762af07700 1 -- start start 2023-11-27T15:17:30.368 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.361+0000 7f762af07700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76240ff1c0 0x7f7624068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:30.368 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.361+0000 7f762af07700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76241003b0 0x7f762419ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:30.368 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.361+0000 7f762af07700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7624100d40 0x7f762419d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:30.368 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.361+0000 7f762af07700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762419dcb0 con 0x7f76240ff1c0 2023-11-27T15:17:30.368 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.361+0000 7f762af07700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762419de20 con 0x7f7624100d40 2023-11-27T15:17:30.368 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.361+0000 7f762af07700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762419dfe0 con 0x7f76241003b0 2023-11-27T15:17:30.368 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.361+0000 7f7628ca3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76240ff1c0 0x7f7624068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:30.368 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.361+0000 7f7628ca3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76240ff1c0 0x7f7624068960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51084/0 (socket says 172.21.15.52:51084) 2023-11-27T15:17:30.368 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.361+0000 7f7628ca3700 1 -- 172.21.15.52:0/1668210466 learned_addr learned my addr 172.21.15.52:0/1668210466 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:30.368 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.361+0000 7f7623fff700 1 --2- 172.21.15.52:0/1668210466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76241003b0 0x7f762419ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:30.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.361+0000 7f76294a4700 1 --2- 172.21.15.52:0/1668210466 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7624100d40 0x7f762419d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:30.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.361+0000 7f7628ca3700 1 -- 172.21.15.52:0/1668210466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76241003b0 msgr2=0x7f762419ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:30.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.361+0000 7f7628ca3700 1 --2- 172.21.15.52:0/1668210466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76241003b0 0x7f762419ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.361+0000 7f7628ca3700 1 -- 172.21.15.52:0/1668210466 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7624100d40 msgr2=0x7f762419d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:30.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.361+0000 7f7628ca3700 1 --2- 172.21.15.52:0/1668210466 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7624100d40 0x7f762419d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.361+0000 7f7628ca3700 1 -- 172.21.15.52:0/1668210466 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f761c009d00 con 0x7f76240ff1c0 2023-11-27T15:17:30.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.362+0000 7f7628ca3700 1 --2- 172.21.15.52:0/1668210466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76240ff1c0 0x7f7624068960 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f761800d980 tx=0x7f761800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:30.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.362+0000 7f7621ffb700 1 -- 172.21.15.52:0/1668210466 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761800f890 con 0x7f76240ff1c0 2023-11-27T15:17:30.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.362+0000 7f7621ffb700 1 -- 172.21.15.52:0/1668210466 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7618010460 con 0x7f76240ff1c0 2023-11-27T15:17:30.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.362+0000 7f7621ffb700 1 -- 172.21.15.52:0/1668210466 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76180195c0 con 0x7f76240ff1c0 2023-11-27T15:17:30.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.362+0000 7f762af07700 1 -- 172.21.15.52:0/1668210466 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f762410ab40 con 0x7f76240ff1c0 2023-11-27T15:17:30.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.362+0000 7f762af07700 1 -- 172.21.15.52:0/1668210466 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f762410a9b0 con 0x7f76240ff1c0 2023-11-27T15:17:30.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.363+0000 7f762af07700 1 -- 172.21.15.52:0/1668210466 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f762419e730 con 0x7f76240ff1c0 2023-11-27T15:17:30.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.364+0000 7f7621ffb700 1 -- 172.21.15.52:0/1668210466 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f76180105d0 con 0x7f76240ff1c0 2023-11-27T15:17:30.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.365+0000 7f7621ffb700 1 --2- 172.21.15.52:0/1668210466 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f76140a1260 0x7f76140a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:30.371 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.365+0000 7f7621ffb700 1 -- 172.21.15.52:0/1668210466 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f76180cc7a0 con 0x7f76240ff1c0 2023-11-27T15:17:30.371 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.365+0000 7f7623fff700 1 --2- 172.21.15.52:0/1668210466 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f76140a1260 0x7f76140a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:30.371 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.365+0000 7f7623fff700 1 --2- 172.21.15.52:0/1668210466 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f76140a1260 0x7f76140a3710 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f76240686d0 tx=0x7f7610009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:30.373 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.369+0000 7f7621ffb700 1 -- 172.21.15.52:0/1668210466 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7618095d70 con 0x7f76240ff1c0 2023-11-27T15:17:30.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.567+0000 7f762af07700 1 -- 172.21.15.52:0/1668210466 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7624106070 con 0x7f76240ff1c0 2023-11-27T15:17:30.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.568+0000 7f7621ffb700 1 -- 172.21.15.52:0/1668210466 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7618099390 con 0x7f76240ff1c0 2023-11-27T15:17:30.572 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2023-11-27T15:17:30.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.570+0000 7f762af07700 1 -- 172.21.15.52:0/1668210466 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f76140a1260 msgr2=0x7f76140a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:30.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.570+0000 7f762af07700 1 --2- 172.21.15.52:0/1668210466 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f76140a1260 0x7f76140a3710 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f76240686d0 tx=0x7f7610009260 comp rx=0 tx=0).stop 2023-11-27T15:17:30.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.570+0000 7f762af07700 1 -- 172.21.15.52:0/1668210466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76240ff1c0 msgr2=0x7f7624068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:30.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.570+0000 7f762af07700 1 --2- 172.21.15.52:0/1668210466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76240ff1c0 0x7f7624068960 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f761800d980 tx=0x7f761800dc90 comp rx=0 tx=0).stop 2023-11-27T15:17:30.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.570+0000 7f762af07700 1 -- 172.21.15.52:0/1668210466 shutdown_connections 2023-11-27T15:17:30.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.570+0000 7f762af07700 1 --2- 172.21.15.52:0/1668210466 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f76140a1260 0x7f76140a3710 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.570+0000 7f762af07700 1 --2- 172.21.15.52:0/1668210466 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7624100d40 0x7f762419d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.570+0000 7f762af07700 1 --2- 172.21.15.52:0/1668210466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76240ff1c0 0x7f7624068960 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.570+0000 7f762af07700 1 --2- 172.21.15.52:0/1668210466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76241003b0 0x7f762419ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.570+0000 7f762af07700 1 -- 172.21.15.52:0/1668210466 >> 172.21.15.52:0/1668210466 conn(0x7f76240fa7c0 msgr2=0x7f76240fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:30.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.570+0000 7f762af07700 1 -- 172.21.15.52:0/1668210466 shutdown_connections 2023-11-27T15:17:30.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.570+0000 7f762af07700 1 -- 172.21.15.52:0/1668210466 wait complete. 2023-11-27T15:17:30.587 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-27T15:17:30.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.712+0000 7f7e8e6e1700 1 -- 172.21.15.52:0/3778367610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e881003b0 msgr2=0x7f7e88100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:30.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.712+0000 7f7e8e6e1700 1 --2- 172.21.15.52:0/3778367610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e881003b0 0x7f7e88100800 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009990 tx=0x7f7e70009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.712+0000 7f7e8e6e1700 1 -- 172.21.15.52:0/3778367610 shutdown_connections 2023-11-27T15:17:30.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.712+0000 7f7e8e6e1700 1 --2- 172.21.15.52:0/3778367610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e880ff1c0 0x7f7e880ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.712+0000 7f7e8e6e1700 1 --2- 172.21.15.52:0/3778367610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e88100d40 0x7f7e88107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.712+0000 7f7e8e6e1700 1 --2- 172.21.15.52:0/3778367610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e881003b0 0x7f7e88100800 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.712+0000 7f7e8e6e1700 1 -- 172.21.15.52:0/3778367610 >> 172.21.15.52:0/3778367610 conn(0x7f7e880fa7c0 msgr2=0x7f7e880fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:30.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.712+0000 7f7e8e6e1700 1 -- 172.21.15.52:0/3778367610 shutdown_connections 2023-11-27T15:17:30.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.712+0000 7f7e8e6e1700 1 -- 172.21.15.52:0/3778367610 wait complete. 2023-11-27T15:17:30.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.713+0000 7f7e8e6e1700 1 Processor -- start 2023-11-27T15:17:30.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.713+0000 7f7e8e6e1700 1 -- start start 2023-11-27T15:17:30.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.713+0000 7f7e8e6e1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e880ff1c0 0x7f7e88068940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:30.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.713+0000 7f7e8e6e1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e881003b0 0x7f7e8819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:30.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.713+0000 7f7e8e6e1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e88100d40 0x7f7e8819d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:30.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.713+0000 7f7e8e6e1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8819dbd0 con 0x7f7e880ff1c0 2023-11-27T15:17:30.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.713+0000 7f7e8e6e1700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8819dd40 con 0x7f7e88100d40 2023-11-27T15:17:30.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.713+0000 7f7e8e6e1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8819df00 con 0x7f7e881003b0 2023-11-27T15:17:30.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.713+0000 7f7e87fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e880ff1c0 0x7f7e88068940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:30.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.713+0000 7f7e87fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e880ff1c0 0x7f7e88068940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51098/0 (socket says 172.21.15.52:51098) 2023-11-27T15:17:30.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.713+0000 7f7e87fff700 1 -- 172.21.15.52:0/29354191 learned_addr learned my addr 172.21.15.52:0/29354191 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:30.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.713+0000 7f7e8cc7e700 1 --2- 172.21.15.52:0/29354191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e88100d40 0x7f7e8819d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:30.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.713+0000 7f7e877fe700 1 --2- 172.21.15.52:0/29354191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e881003b0 0x7f7e8819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:30.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.713+0000 7f7e87fff700 1 -- 172.21.15.52:0/29354191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e881003b0 msgr2=0x7f7e8819adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:30.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.713+0000 7f7e87fff700 1 --2- 172.21.15.52:0/29354191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e881003b0 0x7f7e8819adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.713+0000 7f7e87fff700 1 -- 172.21.15.52:0/29354191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e88100d40 msgr2=0x7f7e8819d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:30.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.713+0000 7f7e87fff700 1 --2- 172.21.15.52:0/29354191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e88100d40 0x7f7e8819d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.713+0000 7f7e87fff700 1 -- 172.21.15.52:0/29354191 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e700095f0 con 0x7f7e880ff1c0 2023-11-27T15:17:30.721 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.713+0000 7f7e877fe700 1 --2- 172.21.15.52:0/29354191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e881003b0 0x7f7e8819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:17:30.721 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.714+0000 7f7e87fff700 1 --2- 172.21.15.52:0/29354191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e880ff1c0 0x7f7e88068940 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800e970 tx=0x7f7e7800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:30.721 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.714+0000 7f7e857fa700 1 -- 172.21.15.52:0/29354191 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e78009790 con 0x7f7e880ff1c0 2023-11-27T15:17:30.721 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.714+0000 7f7e857fa700 1 -- 172.21.15.52:0/29354191 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e7800eec0 con 0x7f7e880ff1c0 2023-11-27T15:17:30.721 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.714+0000 7f7e8e6e1700 1 -- 172.21.15.52:0/29354191 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e8810ab30 con 0x7f7e880ff1c0 2023-11-27T15:17:30.721 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.714+0000 7f7e857fa700 1 -- 172.21.15.52:0/29354191 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e78003cf0 con 0x7f7e880ff1c0 2023-11-27T15:17:30.721 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.714+0000 7f7e8e6e1700 1 -- 172.21.15.52:0/29354191 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e8810a9a0 con 0x7f7e880ff1c0 2023-11-27T15:17:30.722 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.715+0000 7f7e8e6e1700 1 -- 172.21.15.52:0/29354191 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e8819e650 con 0x7f7e880ff1c0 2023-11-27T15:17:30.722 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.716+0000 7f7e857fa700 1 -- 172.21.15.52:0/29354191 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f7e7800ccd0 con 0x7f7e880ff1c0 2023-11-27T15:17:30.722 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.716+0000 7f7e857fa700 1 --2- 172.21.15.52:0/29354191 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7e740a11e0 0x7f7e740a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:30.722 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.717+0000 7f7e877fe700 1 --2- 172.21.15.52:0/29354191 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7e740a11e0 0x7f7e740a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:30.722 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.717+0000 7f7e857fa700 1 -- 172.21.15.52:0/29354191 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7e78014070 con 0x7f7e880ff1c0 2023-11-27T15:17:30.722 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.717+0000 7f7e877fe700 1 --2- 172.21.15.52:0/29354191 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7e740a11e0 0x7f7e740a3690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000b2e0 tx=0x7f7e7000b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:30.724 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.720+0000 7f7e857fa700 1 -- 172.21.15.52:0/29354191 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7e78096420 con 0x7f7e880ff1c0 2023-11-27T15:17:30.866 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:30 smithi052 ceph-mon[99056]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:17:30.866 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:30 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3025933463' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:17:30.866 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:30 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/857023045' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:17:30.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:30 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1668210466' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:17:30.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.921+0000 7f7e8e6e1700 1 -- 172.21.15.52:0/29354191 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7e88106160 con 0x7f7e880ff1c0 2023-11-27T15:17:30.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.921+0000 7f7e857fa700 1 -- 172.21.15.52:0/29354191 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7e78099a40 con 0x7f7e880ff1c0 2023-11-27T15:17:30.926 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2023-11-27T15:17:30.928 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.923+0000 7f7e8e6e1700 1 -- 172.21.15.52:0/29354191 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7e740a11e0 msgr2=0x7f7e740a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:30.928 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.923+0000 7f7e8e6e1700 1 --2- 172.21.15.52:0/29354191 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7e740a11e0 0x7f7e740a3690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000b2e0 tx=0x7f7e7000b5b0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.928 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.923+0000 7f7e8e6e1700 1 -- 172.21.15.52:0/29354191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e880ff1c0 msgr2=0x7f7e88068940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:30.928 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.923+0000 7f7e8e6e1700 1 --2- 172.21.15.52:0/29354191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e880ff1c0 0x7f7e88068940 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800e970 tx=0x7f7e7800ec80 comp rx=0 tx=0).stop 2023-11-27T15:17:30.928 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.923+0000 7f7e8e6e1700 1 -- 172.21.15.52:0/29354191 shutdown_connections 2023-11-27T15:17:30.928 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.923+0000 7f7e8e6e1700 1 --2- 172.21.15.52:0/29354191 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7e740a11e0 0x7f7e740a3690 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.928 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.923+0000 7f7e8e6e1700 1 --2- 172.21.15.52:0/29354191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e88100d40 0x7f7e8819d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.928 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.923+0000 7f7e8e6e1700 1 --2- 172.21.15.52:0/29354191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e880ff1c0 0x7f7e88068940 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.928 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.923+0000 7f7e8e6e1700 1 --2- 172.21.15.52:0/29354191 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e881003b0 0x7f7e8819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:30.929 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.923+0000 7f7e8e6e1700 1 -- 172.21.15.52:0/29354191 >> 172.21.15.52:0/29354191 conn(0x7f7e880fa7c0 msgr2=0x7f7e880fcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:30.929 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.923+0000 7f7e8e6e1700 1 -- 172.21.15.52:0/29354191 shutdown_connections 2023-11-27T15:17:30.929 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:30.923+0000 7f7e8e6e1700 1 -- 172.21.15.52:0/29354191 wait complete. 2023-11-27T15:17:30.940 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:17:31.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:30 smithi150 ceph-mon[100559]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:17:31.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:30 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3025933463' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:17:31.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:30 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/857023045' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:17:31.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:30 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1668210466' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:17:31.067 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.061+0000 7f4b961ad700 1 -- 172.21.15.52:0/4095986656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b901036b0 msgr2=0x7f4b90105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:31.067 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.061+0000 7f4b961ad700 1 --2- 172.21.15.52:0/4095986656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b901036b0 0x7f4b90105a90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4b78009990 tx=0x7f4b78009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.061+0000 7f4b961ad700 1 -- 172.21.15.52:0/4095986656 shutdown_connections 2023-11-27T15:17:31.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.061+0000 7f4b961ad700 1 --2- 172.21.15.52:0/4095986656 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b90105fd0 0x7f4b901083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.061+0000 7f4b961ad700 1 --2- 172.21.15.52:0/4095986656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b90100d90 0x7f4b90103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.061+0000 7f4b961ad700 1 --2- 172.21.15.52:0/4095986656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b901036b0 0x7f4b90105a90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.061+0000 7f4b961ad700 1 -- 172.21.15.52:0/4095986656 >> 172.21.15.52:0/4095986656 conn(0x7f4b900fa750 msgr2=0x7f4b900fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:31.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.061+0000 7f4b961ad700 1 -- 172.21.15.52:0/4095986656 shutdown_connections 2023-11-27T15:17:31.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.061+0000 7f4b961ad700 1 -- 172.21.15.52:0/4095986656 wait complete. 2023-11-27T15:17:31.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.062+0000 7f4b961ad700 1 Processor -- start 2023-11-27T15:17:31.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.062+0000 7f4b961ad700 1 -- start start 2023-11-27T15:17:31.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.062+0000 7f4b961ad700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b90100d90 0x7f4b90100700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:31.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.062+0000 7f4b961ad700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b901036b0 0x7f4b900fed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:31.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.062+0000 7f4b961ad700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b90105fd0 0x7f4b900ff280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:31.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.062+0000 7f4b961ad700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b900ffa20 con 0x7f4b90100d90 2023-11-27T15:17:31.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.062+0000 7f4b961ad700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b900ffb90 con 0x7f4b901036b0 2023-11-27T15:17:31.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.062+0000 7f4b961ad700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b901a0040 con 0x7f4b90105fd0 2023-11-27T15:17:31.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.062+0000 7f4b8f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b90100d90 0x7f4b90100700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:31.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.062+0000 7f4b8f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b90100d90 0x7f4b90100700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51110/0 (socket says 172.21.15.52:51110) 2023-11-27T15:17:31.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.062+0000 7f4b8f7fe700 1 -- 172.21.15.52:0/767475245 learned_addr learned my addr 172.21.15.52:0/767475245 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:31.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.062+0000 7f4b8ffff700 1 --2- 172.21.15.52:0/767475245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b90105fd0 0x7f4b900ff280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:31.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.062+0000 7f4b8effd700 1 --2- 172.21.15.52:0/767475245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b901036b0 0x7f4b900fed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:31.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.062+0000 7f4b8f7fe700 1 -- 172.21.15.52:0/767475245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b90105fd0 msgr2=0x7f4b900ff280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:31.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.062+0000 7f4b8f7fe700 1 --2- 172.21.15.52:0/767475245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b90105fd0 0x7f4b900ff280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.062+0000 7f4b8f7fe700 1 -- 172.21.15.52:0/767475245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b901036b0 msgr2=0x7f4b900fed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:31.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.062+0000 7f4b8f7fe700 1 --2- 172.21.15.52:0/767475245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b901036b0 0x7f4b900fed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.062+0000 7f4b8f7fe700 1 -- 172.21.15.52:0/767475245 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b780095f0 con 0x7f4b90100d90 2023-11-27T15:17:31.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.063+0000 7f4b8f7fe700 1 --2- 172.21.15.52:0/767475245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b90100d90 0x7f4b90100700 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000d950 tx=0x7f4b8000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:31.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.063+0000 7f4b8cff9700 1 -- 172.21.15.52:0/767475245 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8000bb60 con 0x7f4b90100d90 2023-11-27T15:17:31.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.063+0000 7f4b8cff9700 1 -- 172.21.15.52:0/767475245 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b80007620 con 0x7f4b90100d90 2023-11-27T15:17:31.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.063+0000 7f4b961ad700 1 -- 172.21.15.52:0/767475245 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b9010bc60 con 0x7f4b90100d90 2023-11-27T15:17:31.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.063+0000 7f4b8cff9700 1 -- 172.21.15.52:0/767475245 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b80005560 con 0x7f4b90100d90 2023-11-27T15:17:31.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.063+0000 7f4b961ad700 1 -- 172.21.15.52:0/767475245 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b9010bad0 con 0x7f4b90100d90 2023-11-27T15:17:31.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.064+0000 7f4b961ad700 1 -- 172.21.15.52:0/767475245 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b900fc2c0 con 0x7f4b90100d90 2023-11-27T15:17:31.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.066+0000 7f4b8cff9700 1 -- 172.21.15.52:0/767475245 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f4b80004750 con 0x7f4b90100d90 2023-11-27T15:17:31.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.066+0000 7f4b8cff9700 1 --2- 172.21.15.52:0/767475245 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4b7c0a1260 0x7f4b7c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:31.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.066+0000 7f4b8cff9700 1 -- 172.21.15.52:0/767475245 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4b800ccb50 con 0x7f4b90100d90 2023-11-27T15:17:31.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.066+0000 7f4b8effd700 1 --2- 172.21.15.52:0/767475245 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4b7c0a1260 0x7f4b7c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:31.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.067+0000 7f4b8effd700 1 --2- 172.21.15.52:0/767475245 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4b7c0a1260 0x7f4b7c0a3710 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800b2e0 tx=0x7f4b78005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:31.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.069+0000 7f4b8cff9700 1 -- 172.21.15.52:0/767475245 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4b800960c0 con 0x7f4b90100d90 2023-11-27T15:17:31.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:30 smithi102 ceph-mon[100296]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:17:31.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:30 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3025933463' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:17:31.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:30 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/857023045' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:17:31.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:30 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1668210466' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:17:31.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.312+0000 7f4b961ad700 1 -- 172.21.15.52:0/767475245 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b9002cfe0 con 0x7f4b90100d90 2023-11-27T15:17:31.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.312+0000 7f4b8cff9700 1 -- 172.21.15.52:0/767475245 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10804 (secure 0 0 0) 0x7f4b80016020 con 0x7f4b90100d90 2023-11-27T15:17:31.317 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:17:31.317 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2328881376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2328881376},{"type":"v1","addr":"172.21.15.52:6839","nonce":2328881376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2799551570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2799551570},{"type":"v1","addr":"172.21.15.52:6841","nonce":2799551570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:16:09.319630+0000","modified":"2023-11-27T15:16:14.742405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24494,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/215725838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":215725838},{"type":"v1","addr":"172.21.15.150:6839","nonce":215725838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3571830559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3571830559},{"type":"v1","addr":"172.21.15.102:6839","nonce":3571830559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:17:31.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.314+0000 7f4b961ad700 1 -- 172.21.15.52:0/767475245 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4b7c0a1260 msgr2=0x7f4b7c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:31.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.314+0000 7f4b961ad700 1 --2- 172.21.15.52:0/767475245 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4b7c0a1260 0x7f4b7c0a3710 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800b2e0 tx=0x7f4b78005fb0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.314+0000 7f4b961ad700 1 -- 172.21.15.52:0/767475245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b90100d90 msgr2=0x7f4b90100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:31.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.314+0000 7f4b961ad700 1 --2- 172.21.15.52:0/767475245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b90100d90 0x7f4b90100700 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000d950 tx=0x7f4b8000dd10 comp rx=0 tx=0).stop 2023-11-27T15:17:31.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.314+0000 7f4b961ad700 1 -- 172.21.15.52:0/767475245 shutdown_connections 2023-11-27T15:17:31.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.314+0000 7f4b961ad700 1 --2- 172.21.15.52:0/767475245 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4b7c0a1260 0x7f4b7c0a3710 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.314+0000 7f4b961ad700 1 --2- 172.21.15.52:0/767475245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b90105fd0 0x7f4b900ff280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.314+0000 7f4b961ad700 1 --2- 172.21.15.52:0/767475245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b90100d90 0x7f4b90100700 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.314+0000 7f4b961ad700 1 --2- 172.21.15.52:0/767475245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b901036b0 0x7f4b900fed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.314+0000 7f4b961ad700 1 -- 172.21.15.52:0/767475245 >> 172.21.15.52:0/767475245 conn(0x7f4b900fa750 msgr2=0x7f4b901019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:31.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.315+0000 7f4b961ad700 1 -- 172.21.15.52:0/767475245 shutdown_connections 2023-11-27T15:17:31.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.315+0000 7f4b961ad700 1 -- 172.21.15.52:0/767475245 wait complete. 2023-11-27T15:17:31.320 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 23 2023-11-27T15:17:31.332 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:17:31.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.453+0000 7f8d94fc4700 1 -- 172.21.15.52:0/1715148733 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d90100400 msgr2=0x7f8d90100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:31.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.453+0000 7f8d94fc4700 1 --2- 172.21.15.52:0/1715148733 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d90100400 0x7f8d90100870 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8d84009990 tx=0x7f8d84009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.453+0000 7f8d94fc4700 1 -- 172.21.15.52:0/1715148733 shutdown_connections 2023-11-27T15:17:31.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.453+0000 7f8d94fc4700 1 --2- 172.21.15.52:0/1715148733 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d90100db0 0x7f8d90107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.453+0000 7f8d94fc4700 1 --2- 172.21.15.52:0/1715148733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d900ff170 0x7f8d900ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.453+0000 7f8d94fc4700 1 --2- 172.21.15.52:0/1715148733 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d90100400 0x7f8d90100870 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.453+0000 7f8d94fc4700 1 -- 172.21.15.52:0/1715148733 >> 172.21.15.52:0/1715148733 conn(0x7f8d900fa750 msgr2=0x7f8d900fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:31.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.453+0000 7f8d94fc4700 1 -- 172.21.15.52:0/1715148733 shutdown_connections 2023-11-27T15:17:31.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.453+0000 7f8d94fc4700 1 -- 172.21.15.52:0/1715148733 wait complete. 2023-11-27T15:17:31.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.454+0000 7f8d94fc4700 1 Processor -- start 2023-11-27T15:17:31.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.454+0000 7f8d94fc4700 1 -- start start 2023-11-27T15:17:31.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.454+0000 7f8d94fc4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d900ff170 0x7f8d90068940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:31.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.454+0000 7f8d94fc4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d90100400 0x7f8d9019d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:31.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.454+0000 7f8d94fc4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d90100db0 0x7f8d9019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:31.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.454+0000 7f8d94fc4700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9019fd70 con 0x7f8d90100400 2023-11-27T15:17:31.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.454+0000 7f8d94fc4700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9019fee0 con 0x7f8d900ff170 2023-11-27T15:17:31.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.454+0000 7f8d94fc4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d901a00a0 con 0x7f8d90100db0 2023-11-27T15:17:31.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.454+0000 7f8d8dd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d90100400 0x7f8d9019d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:31.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.454+0000 7f8d8dd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d90100400 0x7f8d9019d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51126/0 (socket says 172.21.15.52:51126) 2023-11-27T15:17:31.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.454+0000 7f8d8e59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d900ff170 0x7f8d90068940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:31.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.454+0000 7f8d8dd9b700 1 -- 172.21.15.52:0/1501540922 learned_addr learned my addr 172.21.15.52:0/1501540922 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:31.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.454+0000 7f8d8ed9d700 1 --2- 172.21.15.52:0/1501540922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d90100db0 0x7f8d9019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:31.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.455+0000 7f8d8dd9b700 1 -- 172.21.15.52:0/1501540922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d90100db0 msgr2=0x7f8d9019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:31.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.455+0000 7f8d8dd9b700 1 --2- 172.21.15.52:0/1501540922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d90100db0 0x7f8d9019f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.455+0000 7f8d8dd9b700 1 -- 172.21.15.52:0/1501540922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d900ff170 msgr2=0x7f8d90068940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:31.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.455+0000 7f8d8dd9b700 1 --2- 172.21.15.52:0/1501540922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d900ff170 0x7f8d90068940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.455+0000 7f8d8dd9b700 1 -- 172.21.15.52:0/1501540922 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d840095f0 con 0x7f8d90100400 2023-11-27T15:17:31.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.455+0000 7f8d8dd9b700 1 --2- 172.21.15.52:0/1501540922 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d90100400 0x7f8d9019d170 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8d84000c00 tx=0x7f8d8400f780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:31.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.455+0000 7f8d7f7fe700 1 -- 172.21.15.52:0/1501540922 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8401d070 con 0x7f8d90100400 2023-11-27T15:17:31.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.455+0000 7f8d7f7fe700 1 -- 172.21.15.52:0/1501540922 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d8400fc30 con 0x7f8d90100400 2023-11-27T15:17:31.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.455+0000 7f8d94fc4700 1 -- 172.21.15.52:0/1501540922 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d9010abd0 con 0x7f8d90100400 2023-11-27T15:17:31.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.455+0000 7f8d7f7fe700 1 -- 172.21.15.52:0/1501540922 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d84017650 con 0x7f8d90100400 2023-11-27T15:17:31.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.455+0000 7f8d94fc4700 1 -- 172.21.15.52:0/1501540922 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d9010aa40 con 0x7f8d90100400 2023-11-27T15:17:31.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.456+0000 7f8d94fc4700 1 -- 172.21.15.52:0/1501540922 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d90106160 con 0x7f8d90100400 2023-11-27T15:17:31.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.457+0000 7f8d7f7fe700 1 -- 172.21.15.52:0/1501540922 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8d84022470 con 0x7f8d90100400 2023-11-27T15:17:31.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.457+0000 7f8d7f7fe700 1 --2- 172.21.15.52:0/1501540922 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f8d780a1260 0x7f8d780a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:31.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.458+0000 7f8d8e59c700 1 --2- 172.21.15.52:0/1501540922 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f8d780a1260 0x7f8d780a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:31.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.458+0000 7f8d7f7fe700 1 -- 172.21.15.52:0/1501540922 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8d840cde50 con 0x7f8d90100400 2023-11-27T15:17:31.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.458+0000 7f8d8e59c700 1 --2- 172.21.15.52:0/1501540922 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f8d780a1260 0x7f8d780a3710 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8d80009a10 tx=0x7f8d80009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:31.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.462+0000 7f8d7f7fe700 1 -- 172.21.15.52:0/1501540922 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8d8405c730 con 0x7f8d90100400 2023-11-27T15:17:31.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.663+0000 7f8d94fc4700 1 -- 172.21.15.52:0/1501540922 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d9002cfe0 con 0x7f8d90100400 2023-11-27T15:17:31.669 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.665+0000 7f8d7f7fe700 1 -- 172.21.15.52:0/1501540922 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22657 (secure 0 0 0) 0x7f8d84097450 con 0x7f8d90100400 2023-11-27T15:17:31.669 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:17:31.669 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":85,"fsid":"c6c7df88-8d36-11ee-95a2-87774f69a715","created":"2023-11-27T15:09:03.333345+0000","modified":"2023-11-27T15:17:04.797892+0000","last_up_change":"2023-11-27T15:15:27.656134+0000","last_in_change":"2023-11-27T15:15:10.671153+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:12:14.177523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:16:07.842251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:16:08.728520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58d69ede-c293-49f5-abbe-8187793f2b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6803","nonce":608216415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6805","nonce":608216415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6809","nonce":608216415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6807","nonce":608216415}]},"public_addr":"172.21.15.52:6803/608216415","cluster_addr":"172.21.15.52:6805/608216415","heartbeat_back_addr":"172.21.15.52:6809/608216415","heartbeat_front_addr":"172.21.15.52:6807/608216415","state":["exists","up"]},{"osd":1,"uuid":"d6a46b91-adc8-45ef-80f5-566ae400911f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6801","nonce":2538384087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6803","nonce":2538384087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6807","nonce":2538384087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6805","nonce":2538384087}]},"public_addr":"172.21.15.102:6801/2538384087","cluster_addr":"172.21.15.102:6803/2538384087","heartbeat_back_addr":"172.21.15.102:6807/2538384087","heartbeat_front_addr":"172.21.15.102:6805/2538384087","state":["exists","up"]},{"osd":2,"uuid":"846ba125-11f8-46b2-8941-bd5e1947cc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6801","nonce":4258038221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6803","nonce":4258038221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6807","nonce":4258038221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6805","nonce":4258038221}]},"public_addr":"172.21.15.150:6801/4258038221","cluster_addr":"172.21.15.150:6803/4258038221","heartbeat_back_addr":"172.21.15.150:6807/4258038221","heartbeat_front_addr":"172.21.15.150:6805/4258038221","state":["exists","up"]},{"osd":3,"uuid":"950fcd67-1057-4288-a389-714f1ec94261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6811","nonce":2622100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6813","nonce":2622100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6817","nonce":2622100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6815","nonce":2622100752}]},"public_addr":"172.21.15.52:6811/2622100752","cluster_addr":"172.21.15.52:6813/2622100752","heartbeat_back_addr":"172.21.15.52:6817/2622100752","heartbeat_front_addr":"172.21.15.52:6815/2622100752","state":["exists","up"]},{"osd":4,"uuid":"1c96a676-cea9-4deb-bad1-f7fd240c3d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6809","nonce":1546794763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6811","nonce":1546794763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6815","nonce":1546794763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6813","nonce":1546794763}]},"public_addr":"172.21.15.102:6809/1546794763","cluster_addr":"172.21.15.102:6811/1546794763","heartbeat_back_addr":"172.21.15.102:6815/1546794763","heartbeat_front_addr":"172.21.15.102:6813/1546794763","state":["exists","up"]},{"osd":5,"uuid":"38a75af1-fd4d-433e-b579-0609109c851c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6809","nonce":461511375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6811","nonce":461511375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6815","nonce":461511375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6813","nonce":461511375}]},"public_addr":"172.21.15.150:6809/461511375","cluster_addr":"172.21.15.150:6811/461511375","heartbeat_back_addr":"172.21.15.150:6815/461511375","heartbeat_front_addr":"172.21.15.150:6813/461511375","state":["exists","up"]},{"osd":6,"uuid":"2f0ede4a-d175-4b4a-b876-c0f442048901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6819","nonce":1002457464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6821","nonce":1002457464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6825","nonce":1002457464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6823","nonce":1002457464}]},"public_addr":"172.21.15.52:6819/1002457464","cluster_addr":"172.21.15.52:6821/1002457464","heartbeat_back_addr":"172.21.15.52:6825/1002457464","heartbeat_front_addr":"172.21.15.52:6823/1002457464","state":["exists","up"]},{"osd":7,"uuid":"6b635192-abcf-4bbf-bf78-a019b351bca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6817","nonce":3249840064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6819","nonce":3249840064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6823","nonce":3249840064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6821","nonce":3249840064}]},"public_addr":"172.21.15.102:6817/3249840064","cluster_addr":"172.21.15.102:6819/3249840064","heartbeat_back_addr":"172.21.15.102:6823/3249840064","heartbeat_front_addr":"172.21.15.102:6821/3249840064","state":["exists","up"]},{"osd":8,"uuid":"c7ca8007-7170-47bf-956c-3f00edf27e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6817","nonce":3672583574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6819","nonce":3672583574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6823","nonce":3672583574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6821","nonce":3672583574}]},"public_addr":"172.21.15.150:6817/3672583574","cluster_addr":"172.21.15.150:6819/3672583574","heartbeat_back_addr":"172.21.15.150:6823/3672583574","heartbeat_front_addr":"172.21.15.150:6821/3672583574","state":["exists","up"]},{"osd":9,"uuid":"0e252c2c-01a4-48ba-832d-067ed940fefb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6827","nonce":4287934320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6829","nonce":4287934320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6833","nonce":4287934320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6831","nonce":4287934320}]},"public_addr":"172.21.15.52:6827/4287934320","cluster_addr":"172.21.15.52:6829/4287934320","heartbeat_back_addr":"172.21.15.52:6833/4287934320","heartbeat_front_addr":"172.21.15.52:6831/4287934320","state":["exists","up"]},{"osd":10,"uuid":"bb540142-7563-44a4-b7a3-cb662714c128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6825","nonce":3056034201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6827","nonce":3056034201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6831","nonce":3056034201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6829","nonce":3056034201}]},"public_addr":"172.21.15.102:6825/3056034201","cluster_addr":"172.21.15.102:6827/3056034201","heartbeat_back_addr":"172.21.15.102:6831/3056034201","heartbeat_front_addr":"172.21.15.102:6829/3056034201","state":["exists","up"]},{"osd":11,"uuid":"3156528e-e0f8-4d44-9946-c8b6339c2bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6825","nonce":3207559739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6827","nonce":3207559739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6831","nonce":3207559739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6829","nonce":3207559739}]},"public_addr":"172.21.15.150:6825/3207559739","cluster_addr":"172.21.15.150:6827/3207559739","heartbeat_back_addr":"172.21.15.150:6831/3207559739","heartbeat_front_addr":"172.21.15.150:6829/3207559739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:26.826099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:48.331371+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:10.459738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:32.356472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:54.043170+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:15.648582+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:37.166463+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:59.240819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:20.710097+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:43.364563+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:05.279433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:26.344573+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":5,"to":7}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/1392282110":"2023-11-28T15:10:03.644870+0000","172.21.15.52:0/1479993479":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6800/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/961562652":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/1290665390":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2410529742":"2023-11-28T15:09:37.118690+0000","172.21.15.52:6800/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2130037044":"2023-11-28T15:10:03.644870+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:17:31.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.667+0000 7f8d94fc4700 1 -- 172.21.15.52:0/1501540922 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f8d780a1260 msgr2=0x7f8d780a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:31.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.667+0000 7f8d94fc4700 1 --2- 172.21.15.52:0/1501540922 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f8d780a1260 0x7f8d780a3710 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8d80009a10 tx=0x7f8d80009190 comp rx=0 tx=0).stop 2023-11-27T15:17:31.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.667+0000 7f8d94fc4700 1 -- 172.21.15.52:0/1501540922 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d90100400 msgr2=0x7f8d9019d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:31.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.667+0000 7f8d94fc4700 1 --2- 172.21.15.52:0/1501540922 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d90100400 0x7f8d9019d170 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8d84000c00 tx=0x7f8d8400f780 comp rx=0 tx=0).stop 2023-11-27T15:17:31.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.667+0000 7f8d94fc4700 1 -- 172.21.15.52:0/1501540922 shutdown_connections 2023-11-27T15:17:31.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.667+0000 7f8d94fc4700 1 --2- 172.21.15.52:0/1501540922 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f8d780a1260 0x7f8d780a3710 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.667+0000 7f8d94fc4700 1 --2- 172.21.15.52:0/1501540922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d900ff170 0x7f8d90068940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.667+0000 7f8d94fc4700 1 --2- 172.21.15.52:0/1501540922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d90100db0 0x7f8d9019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.667+0000 7f8d94fc4700 1 --2- 172.21.15.52:0/1501540922 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d90100400 0x7f8d9019d170 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.667+0000 7f8d94fc4700 1 -- 172.21.15.52:0/1501540922 >> 172.21.15.52:0/1501540922 conn(0x7f8d900fa750 msgr2=0x7f8d900fca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:31.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.667+0000 7f8d94fc4700 1 -- 172.21.15.52:0/1501540922 shutdown_connections 2023-11-27T15:17:31.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.667+0000 7f8d94fc4700 1 -- 172.21.15.52:0/1501540922 wait complete. 2023-11-27T15:17:31.684 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:12:14.177523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-27T15:16:07.842251+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-27T15:16:08.728520+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-27T15:17:31.684 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:17:31.817 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.810+0000 7f32b8396700 1 -- 172.21.15.52:0/648212382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32b00ff1c0 msgr2=0x7f32b00ff5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:31.817 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.810+0000 7f32b8396700 1 --2- 172.21.15.52:0/648212382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32b00ff1c0 0x7f32b00ff5d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00b0a0 tx=0x7f32ac00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.817 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.810+0000 7f32b8396700 1 -- 172.21.15.52:0/648212382 shutdown_connections 2023-11-27T15:17:31.817 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.810+0000 7f32b8396700 1 --2- 172.21.15.52:0/648212382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32b0100d40 0x7f32b0107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.817 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.810+0000 7f32b8396700 1 --2- 172.21.15.52:0/648212382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32b00ff1c0 0x7f32b00ff5d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.817 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.810+0000 7f32b8396700 1 --2- 172.21.15.52:0/648212382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b01003b0 0x7f32b0100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.817 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.810+0000 7f32b8396700 1 -- 172.21.15.52:0/648212382 >> 172.21.15.52:0/648212382 conn(0x7f32b00fa7c0 msgr2=0x7f32b00fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:31.817 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.810+0000 7f32b8396700 1 -- 172.21.15.52:0/648212382 shutdown_connections 2023-11-27T15:17:31.817 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.810+0000 7f32b8396700 1 -- 172.21.15.52:0/648212382 wait complete. 2023-11-27T15:17:31.817 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.811+0000 7f32b8396700 1 Processor -- start 2023-11-27T15:17:31.818 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.811+0000 7f32b8396700 1 -- start start 2023-11-27T15:17:31.818 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.811+0000 7f32b8396700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b00ff1c0 0x7f32b0068940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:31.818 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.811+0000 7f32b8396700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32b01003b0 0x7f32b019af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:31.818 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.811+0000 7f32b8396700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32b0100d40 0x7f32b019d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:31.818 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.811+0000 7f32b8396700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b019dce0 con 0x7f32b01003b0 2023-11-27T15:17:31.818 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.811+0000 7f32b8396700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b019de50 con 0x7f32b00ff1c0 2023-11-27T15:17:31.818 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.811+0000 7f32b8396700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b019e010 con 0x7f32b0100d40 2023-11-27T15:17:31.818 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.811+0000 7f32b5931700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32b01003b0 0x7f32b019af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:31.818 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.811+0000 7f32b5931700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32b01003b0 0x7f32b019af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51140/0 (socket says 172.21.15.52:51140) 2023-11-27T15:17:31.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.811+0000 7f32b5931700 1 -- 172.21.15.52:0/4238903684 learned_addr learned my addr 172.21.15.52:0/4238903684 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:31.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.811+0000 7f32b6132700 1 --2- 172.21.15.52:0/4238903684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b00ff1c0 0x7f32b0068940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:31.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.811+0000 7f32b6933700 1 --2- 172.21.15.52:0/4238903684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32b0100d40 0x7f32b019d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:31.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.811+0000 7f32b5931700 1 -- 172.21.15.52:0/4238903684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32b0100d40 msgr2=0x7f32b019d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:31.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.811+0000 7f32b5931700 1 --2- 172.21.15.52:0/4238903684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32b0100d40 0x7f32b019d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.811+0000 7f32b5931700 1 -- 172.21.15.52:0/4238903684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b00ff1c0 msgr2=0x7f32b0068940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:31.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.811+0000 7f32b5931700 1 --2- 172.21.15.52:0/4238903684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b00ff1c0 0x7f32b0068940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.811+0000 7f32b5931700 1 -- 172.21.15.52:0/4238903684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ac009d00 con 0x7f32b01003b0 2023-11-27T15:17:31.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.812+0000 7f32b5931700 1 --2- 172.21.15.52:0/4238903684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32b01003b0 0x7f32b019af00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f32a000d950 tx=0x7f32a000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:31.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.812+0000 7f32a77fe700 1 -- 172.21.15.52:0/4238903684 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a0009940 con 0x7f32b01003b0 2023-11-27T15:17:31.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.812+0000 7f32a77fe700 1 -- 172.21.15.52:0/4238903684 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32a0010460 con 0x7f32b01003b0 2023-11-27T15:17:31.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.812+0000 7f32b8396700 1 -- 172.21.15.52:0/4238903684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b010ab20 con 0x7f32b01003b0 2023-11-27T15:17:31.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.812+0000 7f32a77fe700 1 -- 172.21.15.52:0/4238903684 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a000f620 con 0x7f32b01003b0 2023-11-27T15:17:31.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.812+0000 7f32b8396700 1 -- 172.21.15.52:0/4238903684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b010a990 con 0x7f32b01003b0 2023-11-27T15:17:31.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.813+0000 7f32b8396700 1 -- 172.21.15.52:0/4238903684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b0067b00 con 0x7f32b01003b0 2023-11-27T15:17:31.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.814+0000 7f32a77fe700 1 -- 172.21.15.52:0/4238903684 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f32a00108d0 con 0x7f32b01003b0 2023-11-27T15:17:31.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.814+0000 7f32a77fe700 1 --2- 172.21.15.52:0/4238903684 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f329c0a11e0 0x7f329c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:31.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.815+0000 7f32b6132700 1 --2- 172.21.15.52:0/4238903684 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f329c0a11e0 0x7f329c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:31.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.815+0000 7f32a77fe700 1 -- 172.21.15.52:0/4238903684 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f32a00cc950 con 0x7f32b01003b0 2023-11-27T15:17:31.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.815+0000 7f32b6132700 1 --2- 172.21.15.52:0/4238903684 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f329c0a11e0 0x7f329c0a3690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f32ac009fd0 tx=0x7f32ac00b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:31.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:31.818+0000 7f32a77fe700 1 -- 172.21.15.52:0/4238903684 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f32a0095f20 con 0x7f32b01003b0 2023-11-27T15:17:32.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.016+0000 7f32b8396700 1 -- 172.21.15.52:0/4238903684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f32b01060e0 con 0x7f32b01003b0 2023-11-27T15:17:32.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.017+0000 7f32a77fe700 1 -- 172.21.15.52:0/4238903684 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f32a00160a0 con 0x7f32b01003b0 2023-11-27T15:17:32.021 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2023-11-27T15:17:32.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.018+0000 7f32b8396700 1 -- 172.21.15.52:0/4238903684 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f329c0a11e0 msgr2=0x7f329c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:32.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.018+0000 7f32b8396700 1 --2- 172.21.15.52:0/4238903684 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f329c0a11e0 0x7f329c0a3690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f32ac009fd0 tx=0x7f32ac00b7b0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.018+0000 7f32b8396700 1 -- 172.21.15.52:0/4238903684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32b01003b0 msgr2=0x7f32b019af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:32.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.018+0000 7f32b8396700 1 --2- 172.21.15.52:0/4238903684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32b01003b0 0x7f32b019af00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f32a000d950 tx=0x7f32a000dc60 comp rx=0 tx=0).stop 2023-11-27T15:17:32.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.019+0000 7f32b8396700 1 -- 172.21.15.52:0/4238903684 shutdown_connections 2023-11-27T15:17:32.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.019+0000 7f32b8396700 1 --2- 172.21.15.52:0/4238903684 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f329c0a11e0 0x7f329c0a3690 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.019+0000 7f32b8396700 1 --2- 172.21.15.52:0/4238903684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32b00ff1c0 0x7f32b0068940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.019+0000 7f32b8396700 1 --2- 172.21.15.52:0/4238903684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32b0100d40 0x7f32b019d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.019+0000 7f32b8396700 1 --2- 172.21.15.52:0/4238903684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32b01003b0 0x7f32b019af00 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.019+0000 7f32b8396700 1 -- 172.21.15.52:0/4238903684 >> 172.21.15.52:0/4238903684 conn(0x7f32b00fa7c0 msgr2=0x7f32b00fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:32.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.019+0000 7f32b8396700 1 -- 172.21.15.52:0/4238903684 shutdown_connections 2023-11-27T15:17:32.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.019+0000 7f32b8396700 1 -- 172.21.15.52:0/4238903684 wait complete. 2023-11-27T15:17:32.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:31 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/29354191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:17:32.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:31 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/767475245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:17:32.030 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:31 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1501540922' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:17:32.034 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:31 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/29354191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:17:32.034 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:31 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/767475245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:17:32.034 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:31 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1501540922' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:17:32.035 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-27T15:17:32.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:31 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/29354191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:17:32.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:31 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/767475245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:17:32.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:31 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1501540922' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:17:32.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.159+0000 7fcafd4bb700 1 -- 172.21.15.52:0/2239663547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaf80ff1c0 msgr2=0x7fcaf80ff5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:32.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.159+0000 7fcafd4bb700 1 --2- 172.21.15.52:0/2239663547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaf80ff1c0 0x7fcaf80ff5d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcae8009990 tx=0x7fcae8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.160+0000 7fcafd4bb700 1 -- 172.21.15.52:0/2239663547 shutdown_connections 2023-11-27T15:17:32.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.160+0000 7fcafd4bb700 1 --2- 172.21.15.52:0/2239663547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcaf8100d40 0x7fcaf8107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.160+0000 7fcafd4bb700 1 --2- 172.21.15.52:0/2239663547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaf80ff1c0 0x7fcaf80ff5d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.160+0000 7fcafd4bb700 1 --2- 172.21.15.52:0/2239663547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcaf81003b0 0x7fcaf8100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.160+0000 7fcafd4bb700 1 -- 172.21.15.52:0/2239663547 >> 172.21.15.52:0/2239663547 conn(0x7fcaf80fa7c0 msgr2=0x7fcaf80fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:32.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.160+0000 7fcafd4bb700 1 -- 172.21.15.52:0/2239663547 shutdown_connections 2023-11-27T15:17:32.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.160+0000 7fcafd4bb700 1 -- 172.21.15.52:0/2239663547 wait complete. 2023-11-27T15:17:32.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.160+0000 7fcafd4bb700 1 Processor -- start 2023-11-27T15:17:32.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.161+0000 7fcafd4bb700 1 -- start start 2023-11-27T15:17:32.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.161+0000 7fcafd4bb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcaf80ff1c0 0x7fcaf8193f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:32.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.161+0000 7fcafd4bb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcaf81003b0 0x7fcaf8194460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:32.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.161+0000 7fcafd4bb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaf8100d40 0x7fcaf819b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:32.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.161+0000 7fcafd4bb700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf819baa0 con 0x7fcaf8100d40 2023-11-27T15:17:32.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.161+0000 7fcafd4bb700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf819bbe0 con 0x7fcaf80ff1c0 2023-11-27T15:17:32.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.161+0000 7fcafd4bb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf819bd70 con 0x7fcaf81003b0 2023-11-27T15:17:32.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.161+0000 7fcaf77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaf8100d40 0x7fcaf819b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:32.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.161+0000 7fcaf77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaf8100d40 0x7fcaf819b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51156/0 (socket says 172.21.15.52:51156) 2023-11-27T15:17:32.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.161+0000 7fcaf77fe700 1 -- 172.21.15.52:0/3806607992 learned_addr learned my addr 172.21.15.52:0/3806607992 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:32.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.161+0000 7fcaf6ffd700 1 --2- 172.21.15.52:0/3806607992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcaf80ff1c0 0x7fcaf8193f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:32.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.161+0000 7fcaf67fc700 1 --2- 172.21.15.52:0/3806607992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcaf81003b0 0x7fcaf8194460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:32.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.161+0000 7fcaf77fe700 1 -- 172.21.15.52:0/3806607992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcaf81003b0 msgr2=0x7fcaf8194460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:32.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.161+0000 7fcaf77fe700 1 --2- 172.21.15.52:0/3806607992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcaf81003b0 0x7fcaf8194460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.161+0000 7fcaf77fe700 1 -- 172.21.15.52:0/3806607992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcaf80ff1c0 msgr2=0x7fcaf8193f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:32.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.161+0000 7fcaf77fe700 1 --2- 172.21.15.52:0/3806607992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcaf80ff1c0 0x7fcaf8193f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.161+0000 7fcaf77fe700 1 -- 172.21.15.52:0/3806607992 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae80095f0 con 0x7fcaf8100d40 2023-11-27T15:17:32.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.161+0000 7fcaf77fe700 1 --2- 172.21.15.52:0/3806607992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaf8100d40 0x7fcaf819b4d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcaf000f0d0 tx=0x7fcaf000f3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:32.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.162+0000 7fcadffff700 1 -- 172.21.15.52:0/3806607992 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0019070 con 0x7fcaf8100d40 2023-11-27T15:17:32.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.162+0000 7fcadffff700 1 -- 172.21.15.52:0/3806607992 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcaf0011040 con 0x7fcaf8100d40 2023-11-27T15:17:32.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.162+0000 7fcafd4bb700 1 -- 172.21.15.52:0/3806607992 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf8109b40 con 0x7fcaf8100d40 2023-11-27T15:17:32.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.162+0000 7fcadffff700 1 -- 172.21.15.52:0/3806607992 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0004380 con 0x7fcaf8100d40 2023-11-27T15:17:32.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.162+0000 7fcafd4bb700 1 -- 172.21.15.52:0/3806607992 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf81099b0 con 0x7fcaf8100d40 2023-11-27T15:17:32.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.163+0000 7fcafd4bb700 1 -- 172.21.15.52:0/3806607992 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf819c3a0 con 0x7fcaf8100d40 2023-11-27T15:17:32.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.164+0000 7fcadffff700 1 -- 172.21.15.52:0/3806607992 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fcaf0004ad0 con 0x7fcaf8100d40 2023-11-27T15:17:32.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.165+0000 7fcadffff700 1 --2- 172.21.15.52:0/3806607992 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fcae00a1260 0x7fcae00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:32.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.165+0000 7fcadffff700 1 -- 172.21.15.52:0/3806607992 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fcaf00ce770 con 0x7fcaf8100d40 2023-11-27T15:17:32.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.165+0000 7fcaf6ffd700 1 --2- 172.21.15.52:0/3806607992 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fcae00a1260 0x7fcae00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:32.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.166+0000 7fcaf6ffd700 1 --2- 172.21.15.52:0/3806607992 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fcae00a1260 0x7fcae00a3710 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcae8000c00 tx=0x7fcae8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:32.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.168+0000 7fcadffff700 1 -- 172.21.15.52:0/3806607992 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcaf0097dd0 con 0x7fcaf8100d40 2023-11-27T15:17:32.371 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.367+0000 7fcafd4bb700 1 -- 172.21.15.52:0/3806607992 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcaf8106110 con 0x7fcaf8100d40 2023-11-27T15:17:32.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.368+0000 7fcadffff700 1 -- 172.21.15.52:0/3806607992 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcaf009b3f0 con 0x7fcaf8100d40 2023-11-27T15:17:32.372 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2023-11-27T15:17:32.374 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.370+0000 7fcafd4bb700 1 -- 172.21.15.52:0/3806607992 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fcae00a1260 msgr2=0x7fcae00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:32.374 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.370+0000 7fcafd4bb700 1 --2- 172.21.15.52:0/3806607992 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fcae00a1260 0x7fcae00a3710 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcae8000c00 tx=0x7fcae8005fb0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.374 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.370+0000 7fcafd4bb700 1 -- 172.21.15.52:0/3806607992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaf8100d40 msgr2=0x7fcaf819b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:32.374 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.370+0000 7fcafd4bb700 1 --2- 172.21.15.52:0/3806607992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaf8100d40 0x7fcaf819b4d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcaf000f0d0 tx=0x7fcaf000f3e0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.370+0000 7fcafd4bb700 1 -- 172.21.15.52:0/3806607992 shutdown_connections 2023-11-27T15:17:32.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.370+0000 7fcafd4bb700 1 --2- 172.21.15.52:0/3806607992 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fcae00a1260 0x7fcae00a3710 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.370+0000 7fcafd4bb700 1 --2- 172.21.15.52:0/3806607992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcaf80ff1c0 0x7fcaf8193f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.370+0000 7fcafd4bb700 1 --2- 172.21.15.52:0/3806607992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcaf8100d40 0x7fcaf819b4d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.370+0000 7fcafd4bb700 1 --2- 172.21.15.52:0/3806607992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcaf81003b0 0x7fcaf8194460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.370+0000 7fcafd4bb700 1 -- 172.21.15.52:0/3806607992 >> 172.21.15.52:0/3806607992 conn(0x7fcaf80fa7c0 msgr2=0x7fcaf80fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:32.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.370+0000 7fcafd4bb700 1 -- 172.21.15.52:0/3806607992 shutdown_connections 2023-11-27T15:17:32.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.370+0000 7fcafd4bb700 1 -- 172.21.15.52:0/3806607992 wait complete. 2023-11-27T15:17:32.387 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-27T15:17:32.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.510+0000 7f58e3d03700 1 -- 172.21.15.52:0/2104893387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58dc101540 msgr2=0x7f58dc101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:32.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.510+0000 7f58e3d03700 1 --2- 172.21.15.52:0/2104893387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58dc101540 0x7f58dc101990 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f58cc009990 tx=0x7f58cc009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.511+0000 7f58e3d03700 1 -- 172.21.15.52:0/2104893387 shutdown_connections 2023-11-27T15:17:32.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.511+0000 7f58e3d03700 1 --2- 172.21.15.52:0/2104893387 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58dc101ed0 0x7f58dc108790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.511+0000 7f58e3d03700 1 --2- 172.21.15.52:0/2104893387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58dc0fe570 0x7f58dc0fe980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.511+0000 7f58e3d03700 1 --2- 172.21.15.52:0/2104893387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58dc101540 0x7f58dc101990 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.511+0000 7f58e3d03700 1 -- 172.21.15.52:0/2104893387 >> 172.21.15.52:0/2104893387 conn(0x7f58dc0f9f30 msgr2=0x7f58dc0fc380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:32.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.511+0000 7f58e3d03700 1 -- 172.21.15.52:0/2104893387 shutdown_connections 2023-11-27T15:17:32.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.511+0000 7f58e3d03700 1 -- 172.21.15.52:0/2104893387 wait complete. 2023-11-27T15:17:32.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.511+0000 7f58e3d03700 1 Processor -- start 2023-11-27T15:17:32.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.511+0000 7f58e3d03700 1 -- start start 2023-11-27T15:17:32.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.512+0000 7f58e3d03700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58dc0fe570 0x7f58dc195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:32.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.512+0000 7f58e3d03700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58dc101540 0x7f58dc196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:32.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.512+0000 7f58e3d03700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58dc101ed0 0x7f58dc19d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:32.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.512+0000 7f58e3d03700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc19da90 con 0x7f58dc101ed0 2023-11-27T15:17:32.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.512+0000 7f58e3d03700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc19dbd0 con 0x7f58dc101540 2023-11-27T15:17:32.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.512+0000 7f58e3d03700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc19dd60 con 0x7f58dc0fe570 2023-11-27T15:17:32.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.512+0000 7f58e22a0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58dc101ed0 0x7f58dc19d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:32.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.512+0000 7f58e22a0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58dc101ed0 0x7f58dc19d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51170/0 (socket says 172.21.15.52:51170) 2023-11-27T15:17:32.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.512+0000 7f58e22a0700 1 -- 172.21.15.52:0/3527307527 learned_addr learned my addr 172.21.15.52:0/3527307527 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:32.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.512+0000 7f58e1a9f700 1 --2- 172.21.15.52:0/3527307527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58dc0fe570 0x7f58dc195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:32.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.512+0000 7f58e22a0700 1 -- 172.21.15.52:0/3527307527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58dc0fe570 msgr2=0x7f58dc195f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:32.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.512+0000 7f58e22a0700 1 --2- 172.21.15.52:0/3527307527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58dc0fe570 0x7f58dc195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.512+0000 7f58e129e700 1 --2- 172.21.15.52:0/3527307527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58dc101540 0x7f58dc196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:32.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.512+0000 7f58e22a0700 1 -- 172.21.15.52:0/3527307527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58dc101540 msgr2=0x7f58dc196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:32.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.512+0000 7f58e22a0700 1 --2- 172.21.15.52:0/3527307527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58dc101540 0x7f58dc196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.512+0000 7f58e22a0700 1 -- 172.21.15.52:0/3527307527 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58cc0095f0 con 0x7f58dc101ed0 2023-11-27T15:17:32.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.512+0000 7f58e22a0700 1 --2- 172.21.15.52:0/3527307527 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58dc101ed0 0x7f58dc19d4c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f58d4007d70 tx=0x7f58d400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:32.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.512+0000 7f58d2ffd700 1 -- 172.21.15.52:0/3527307527 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d400ad40 con 0x7f58dc101ed0 2023-11-27T15:17:32.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.513+0000 7f58d2ffd700 1 -- 172.21.15.52:0/3527307527 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58d40128e0 con 0x7f58dc101ed0 2023-11-27T15:17:32.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.513+0000 7f58e3d03700 1 -- 172.21.15.52:0/3527307527 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58dc10bc20 con 0x7f58dc101ed0 2023-11-27T15:17:32.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.513+0000 7f58d2ffd700 1 -- 172.21.15.52:0/3527307527 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d4011780 con 0x7f58dc101ed0 2023-11-27T15:17:32.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.513+0000 7f58e3d03700 1 -- 172.21.15.52:0/3527307527 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58dc10ba90 con 0x7f58dc101ed0 2023-11-27T15:17:32.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.513+0000 7f58e3d03700 1 -- 172.21.15.52:0/3527307527 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58dc107190 con 0x7f58dc101ed0 2023-11-27T15:17:32.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.515+0000 7f58d2ffd700 1 -- 172.21.15.52:0/3527307527 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f58d4012a50 con 0x7f58dc101ed0 2023-11-27T15:17:32.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.515+0000 7f58d2ffd700 1 --2- 172.21.15.52:0/3527307527 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f58c80a11e0 0x7f58c80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:32.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.515+0000 7f58d2ffd700 1 -- 172.21.15.52:0/3527307527 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f58d40ccd10 con 0x7f58dc101ed0 2023-11-27T15:17:32.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.515+0000 7f58e1a9f700 1 --2- 172.21.15.52:0/3527307527 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f58c80a11e0 0x7f58c80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:32.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.516+0000 7f58e1a9f700 1 --2- 172.21.15.52:0/3527307527 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f58c80a11e0 0x7f58c80a3690 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f58dc067bd0 tx=0x7f58d8005d70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:32.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.519+0000 7f58d2ffd700 1 -- 172.21.15.52:0/3527307527 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f58d40963f0 con 0x7f58dc101ed0 2023-11-27T15:17:32.723 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.719+0000 7f58e3d03700 1 -- 172.21.15.52:0/3527307527 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f58dc0619b0 con 0x7f58dc101ed0 2023-11-27T15:17:32.723 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.719+0000 7f58d2ffd700 1 -- 172.21.15.52:0/3527307527 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f58d40160a0 con 0x7f58dc101ed0 2023-11-27T15:17:32.724 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2023-11-27T15:17:32.726 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.721+0000 7f58e3d03700 1 -- 172.21.15.52:0/3527307527 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f58c80a11e0 msgr2=0x7f58c80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:32.726 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.721+0000 7f58e3d03700 1 --2- 172.21.15.52:0/3527307527 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f58c80a11e0 0x7f58c80a3690 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f58dc067bd0 tx=0x7f58d8005d70 comp rx=0 tx=0).stop 2023-11-27T15:17:32.726 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.721+0000 7f58e3d03700 1 -- 172.21.15.52:0/3527307527 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58dc101ed0 msgr2=0x7f58dc19d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:32.726 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.721+0000 7f58e3d03700 1 --2- 172.21.15.52:0/3527307527 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58dc101ed0 0x7f58dc19d4c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f58d4007d70 tx=0x7f58d400a340 comp rx=0 tx=0).stop 2023-11-27T15:17:32.726 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.721+0000 7f58e3d03700 1 -- 172.21.15.52:0/3527307527 shutdown_connections 2023-11-27T15:17:32.726 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.721+0000 7f58e3d03700 1 --2- 172.21.15.52:0/3527307527 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f58c80a11e0 0x7f58c80a3690 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.726 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.721+0000 7f58e3d03700 1 --2- 172.21.15.52:0/3527307527 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f58dc101ed0 0x7f58dc19d4c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.721+0000 7f58e3d03700 1 --2- 172.21.15.52:0/3527307527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58dc0fe570 0x7f58dc195f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.721+0000 7f58e3d03700 1 --2- 172.21.15.52:0/3527307527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58dc101540 0x7f58dc196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.721+0000 7f58e3d03700 1 -- 172.21.15.52:0/3527307527 >> 172.21.15.52:0/3527307527 conn(0x7f58dc0f9f30 msgr2=0x7f58dc0fc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:32.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.721+0000 7f58e3d03700 1 -- 172.21.15.52:0/3527307527 shutdown_connections 2023-11-27T15:17:32.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.721+0000 7f58e3d03700 1 -- 172.21.15.52:0/3527307527 wait complete. 2023-11-27T15:17:32.738 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:17:32.867 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.860+0000 7ff4d7442700 1 -- 172.21.15.52:0/1255713856 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4d01003a0 msgr2=0x7ff4d01007f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:32.867 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.860+0000 7ff4d7442700 1 --2- 172.21.15.52:0/1255713856 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4d01003a0 0x7ff4d01007f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009990 tx=0x7ff4cc009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.867 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.860+0000 7ff4d7442700 1 -- 172.21.15.52:0/1255713856 shutdown_connections 2023-11-27T15:17:32.867 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.860+0000 7ff4d7442700 1 --2- 172.21.15.52:0/1255713856 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4d0100d30 0x7ff4d01075f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.867 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.860+0000 7ff4d7442700 1 --2- 172.21.15.52:0/1255713856 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4d00ff1b0 0x7ff4d00ff5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.867 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.860+0000 7ff4d7442700 1 --2- 172.21.15.52:0/1255713856 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4d01003a0 0x7ff4d01007f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.867 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.860+0000 7ff4d7442700 1 -- 172.21.15.52:0/1255713856 >> 172.21.15.52:0/1255713856 conn(0x7ff4d00fa790 msgr2=0x7ff4d00fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:32.867 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.860+0000 7ff4d7442700 1 -- 172.21.15.52:0/1255713856 shutdown_connections 2023-11-27T15:17:32.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.860+0000 7ff4d7442700 1 -- 172.21.15.52:0/1255713856 wait complete. 2023-11-27T15:17:32.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.861+0000 7ff4d7442700 1 Processor -- start 2023-11-27T15:17:32.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.861+0000 7ff4d7442700 1 -- start start 2023-11-27T15:17:32.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.861+0000 7ff4d7442700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4d00ff1b0 0x7ff4d0069950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:32.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.861+0000 7ff4d7442700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4d01003a0 0x7ff4d019d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:32.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.861+0000 7ff4d7442700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4d0100d30 0x7ff4d019f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:32.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.861+0000 7ff4d7442700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d019fc70 con 0x7ff4d00ff1b0 2023-11-27T15:17:32.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.861+0000 7ff4d7442700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d019fde0 con 0x7ff4d01003a0 2023-11-27T15:17:32.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.861+0000 7ff4d7442700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d019ffa0 con 0x7ff4d0100d30 2023-11-27T15:17:32.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.861+0000 7ff4d51de700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4d00ff1b0 0x7ff4d0069950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:32.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.861+0000 7ff4d51de700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4d00ff1b0 0x7ff4d0069950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51180/0 (socket says 172.21.15.52:51180) 2023-11-27T15:17:32.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.861+0000 7ff4d51de700 1 -- 172.21.15.52:0/411445713 learned_addr learned my addr 172.21.15.52:0/411445713 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:32.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.861+0000 7ff4d49dd700 1 --2- 172.21.15.52:0/411445713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4d01003a0 0x7ff4d019d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:32.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.861+0000 7ff4d59df700 1 --2- 172.21.15.52:0/411445713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4d0100d30 0x7ff4d019f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:32.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.861+0000 7ff4d51de700 1 -- 172.21.15.52:0/411445713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4d0100d30 msgr2=0x7ff4d019f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:32.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.861+0000 7ff4d51de700 1 --2- 172.21.15.52:0/411445713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4d0100d30 0x7ff4d019f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.861+0000 7ff4d51de700 1 -- 172.21.15.52:0/411445713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4d01003a0 msgr2=0x7ff4d019d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:32.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.861+0000 7ff4d51de700 1 --2- 172.21.15.52:0/411445713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4d01003a0 0x7ff4d019d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:32.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.861+0000 7ff4d51de700 1 -- 172.21.15.52:0/411445713 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4cc0095f0 con 0x7ff4d00ff1b0 2023-11-27T15:17:32.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.861+0000 7ff4d59df700 1 --2- 172.21.15.52:0/411445713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4d0100d30 0x7ff4d019f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:17:32.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.862+0000 7ff4d51de700 1 --2- 172.21.15.52:0/411445713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4d00ff1b0 0x7ff4d0069950 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff4c000c970 tx=0x7ff4c000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:32.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.862+0000 7ff4c67fc700 1 -- 172.21.15.52:0/411445713 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0004220 con 0x7ff4d00ff1b0 2023-11-27T15:17:32.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.862+0000 7ff4c67fc700 1 -- 172.21.15.52:0/411445713 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4c0007d60 con 0x7ff4d00ff1b0 2023-11-27T15:17:32.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.862+0000 7ff4d7442700 1 -- 172.21.15.52:0/411445713 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d010ab20 con 0x7ff4d00ff1b0 2023-11-27T15:17:32.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.862+0000 7ff4c67fc700 1 -- 172.21.15.52:0/411445713 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0003c90 con 0x7ff4d00ff1b0 2023-11-27T15:17:32.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.862+0000 7ff4d7442700 1 -- 172.21.15.52:0/411445713 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d010a990 con 0x7ff4d00ff1b0 2023-11-27T15:17:32.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.863+0000 7ff4d7442700 1 -- 172.21.15.52:0/411445713 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4d0106100 con 0x7ff4d00ff1b0 2023-11-27T15:17:32.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.864+0000 7ff4c67fc700 1 -- 172.21.15.52:0/411445713 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff4c0004380 con 0x7ff4d00ff1b0 2023-11-27T15:17:32.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.864+0000 7ff4c67fc700 1 --2- 172.21.15.52:0/411445713 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff4bc0a11e0 0x7ff4bc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:32.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.865+0000 7ff4d49dd700 1 --2- 172.21.15.52:0/411445713 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff4bc0a11e0 0x7ff4bc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:32.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.865+0000 7ff4c67fc700 1 -- 172.21.15.52:0/411445713 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff4c00cca90 con 0x7ff4d00ff1b0 2023-11-27T15:17:32.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.865+0000 7ff4d49dd700 1 --2- 172.21.15.52:0/411445713 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff4bc0a11e0 0x7ff4bc0a3690 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00b5c0 tx=0x7ff4cc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:32.872 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:32.868+0000 7ff4c67fc700 1 -- 172.21.15.52:0/411445713 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff4c0096000 con 0x7ff4d00ff1b0 2023-11-27T15:17:33.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:32 smithi150 ceph-mon[100559]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:17:33.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:32 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/4238903684' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:17:33.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:32 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3806607992' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:17:33.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:32 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3527307527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:17:33.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:32 smithi102 ceph-mon[100296]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:17:33.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:32 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/4238903684' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:17:33.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:32 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3806607992' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:17:33.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:32 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3527307527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:17:33.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:32 smithi052 ceph-mon[99056]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:17:33.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:32 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/4238903684' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:17:33.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:32 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3806607992' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:17:33.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:32 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3527307527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:17:33.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.114+0000 7ff4d7442700 1 -- 172.21.15.52:0/411445713 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4d0061700 con 0x7ff4d00ff1b0 2023-11-27T15:17:33.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.115+0000 7ff4c67fc700 1 -- 172.21.15.52:0/411445713 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10804 (secure 0 0 0) 0x7ff4c001d090 con 0x7ff4d00ff1b0 2023-11-27T15:17:33.119 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:17:33.119 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2328881376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2328881376},{"type":"v1","addr":"172.21.15.52:6839","nonce":2328881376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2799551570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2799551570},{"type":"v1","addr":"172.21.15.52:6841","nonce":2799551570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:16:09.319630+0000","modified":"2023-11-27T15:16:14.742405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24494,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/215725838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":215725838},{"type":"v1","addr":"172.21.15.150:6839","nonce":215725838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3571830559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3571830559},{"type":"v1","addr":"172.21.15.102:6839","nonce":3571830559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:17:33.121 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.117+0000 7ff4d7442700 1 -- 172.21.15.52:0/411445713 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff4bc0a11e0 msgr2=0x7ff4bc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:33.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.117+0000 7ff4d7442700 1 --2- 172.21.15.52:0/411445713 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff4bc0a11e0 0x7ff4bc0a3690 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00b5c0 tx=0x7ff4cc005960 comp rx=0 tx=0).stop 2023-11-27T15:17:33.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.117+0000 7ff4d7442700 1 -- 172.21.15.52:0/411445713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4d00ff1b0 msgr2=0x7ff4d0069950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:33.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.117+0000 7ff4d7442700 1 --2- 172.21.15.52:0/411445713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4d00ff1b0 0x7ff4d0069950 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff4c000c970 tx=0x7ff4c000cc80 comp rx=0 tx=0).stop 2023-11-27T15:17:33.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.117+0000 7ff4d7442700 1 -- 172.21.15.52:0/411445713 shutdown_connections 2023-11-27T15:17:33.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.117+0000 7ff4d7442700 1 --2- 172.21.15.52:0/411445713 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff4bc0a11e0 0x7ff4bc0a3690 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.117+0000 7ff4d7442700 1 --2- 172.21.15.52:0/411445713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4d0100d30 0x7ff4d019f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.117+0000 7ff4d7442700 1 --2- 172.21.15.52:0/411445713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4d00ff1b0 0x7ff4d0069950 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.123 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.117+0000 7ff4d7442700 1 --2- 172.21.15.52:0/411445713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4d01003a0 0x7ff4d019d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.123 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.117+0000 7ff4d7442700 1 -- 172.21.15.52:0/411445713 >> 172.21.15.52:0/411445713 conn(0x7ff4d00fa790 msgr2=0x7ff4d00fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:33.123 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.117+0000 7ff4d7442700 1 -- 172.21.15.52:0/411445713 shutdown_connections 2023-11-27T15:17:33.123 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.117+0000 7ff4d7442700 1 -- 172.21.15.52:0/411445713 wait complete. 2023-11-27T15:17:33.123 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 23 2023-11-27T15:17:33.134 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:17:33.261 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.254+0000 7ff08a8e5700 1 -- 172.21.15.52:0/3727264908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0840ff460 msgr2=0x7ff0840ff8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:33.261 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.254+0000 7ff08a8e5700 1 --2- 172.21.15.52:0/3727264908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0840ff460 0x7ff0840ff8d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff0780099c0 tx=0x7ff078009cd0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.261 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.254+0000 7ff08a8e5700 1 -- 172.21.15.52:0/3727264908 shutdown_connections 2023-11-27T15:17:33.261 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.254+0000 7ff08a8e5700 1 --2- 172.21.15.52:0/3727264908 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff084100770 0x7ff084100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.261 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.254+0000 7ff08a8e5700 1 --2- 172.21.15.52:0/3727264908 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0840ffe10 0x7ff0841076c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.254+0000 7ff08a8e5700 1 --2- 172.21.15.52:0/3727264908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0840ff460 0x7ff0840ff8d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.254+0000 7ff08a8e5700 1 -- 172.21.15.52:0/3727264908 >> 172.21.15.52:0/3727264908 conn(0x7ff0840f9ee0 msgr2=0x7ff0840fc350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:33.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.255+0000 7ff08a8e5700 1 -- 172.21.15.52:0/3727264908 shutdown_connections 2023-11-27T15:17:33.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.255+0000 7ff08a8e5700 1 -- 172.21.15.52:0/3727264908 wait complete. 2023-11-27T15:17:33.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.255+0000 7ff08a8e5700 1 Processor -- start 2023-11-27T15:17:33.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.255+0000 7ff08a8e5700 1 -- start start 2023-11-27T15:17:33.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.255+0000 7ff08a8e5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0840ff460 0x7ff084103210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.255+0000 7ff08a8e5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0840ffe10 0x7ff084101850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.255+0000 7ff08a8e5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff084100770 0x7ff084101d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.255+0000 7ff08a8e5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff084102530 con 0x7ff084100770 2023-11-27T15:17:33.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.255+0000 7ff08a8e5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0841026a0 con 0x7ff0840ffe10 2023-11-27T15:17:33.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.255+0000 7ff08a8e5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08419fee0 con 0x7ff0840ff460 2023-11-27T15:17:33.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.256+0000 7ff088e82700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff084100770 0x7ff084101d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.256+0000 7ff088e82700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff084100770 0x7ff084101d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51196/0 (socket says 172.21.15.52:51196) 2023-11-27T15:17:33.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.256+0000 7ff088e82700 1 -- 172.21.15.52:0/1392250186 learned_addr learned my addr 172.21.15.52:0/1392250186 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:33.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.256+0000 7ff083fff700 1 --2- 172.21.15.52:0/1392250186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0840ff460 0x7ff084103210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.256+0000 7ff0837fe700 1 --2- 172.21.15.52:0/1392250186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0840ffe10 0x7ff084101850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.256+0000 7ff088e82700 1 -- 172.21.15.52:0/1392250186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0840ff460 msgr2=0x7ff084103210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:33.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.256+0000 7ff088e82700 1 --2- 172.21.15.52:0/1392250186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0840ff460 0x7ff084103210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.256+0000 7ff088e82700 1 -- 172.21.15.52:0/1392250186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0840ffe10 msgr2=0x7ff084101850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:33.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.256+0000 7ff088e82700 1 --2- 172.21.15.52:0/1392250186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0840ffe10 0x7ff084101850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.256+0000 7ff088e82700 1 -- 172.21.15.52:0/1392250186 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0780095f0 con 0x7ff084100770 2023-11-27T15:17:33.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.256+0000 7ff088e82700 1 --2- 172.21.15.52:0/1392250186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff084100770 0x7ff084101d90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff07c007d70 tx=0x7ff07c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:33.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.256+0000 7ff0817fa700 1 -- 172.21.15.52:0/1392250186 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c00ad40 con 0x7ff084100770 2023-11-27T15:17:33.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.256+0000 7ff0817fa700 1 -- 172.21.15.52:0/1392250186 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff07c0128e0 con 0x7ff084100770 2023-11-27T15:17:33.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.256+0000 7ff08a8e5700 1 -- 172.21.15.52:0/1392250186 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff08410ab00 con 0x7ff084100770 2023-11-27T15:17:33.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.256+0000 7ff0817fa700 1 -- 172.21.15.52:0/1392250186 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c0116b0 con 0x7ff084100770 2023-11-27T15:17:33.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.256+0000 7ff08a8e5700 1 -- 172.21.15.52:0/1392250186 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff08410a970 con 0x7ff084100770 2023-11-27T15:17:33.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.257+0000 7ff08a8e5700 1 -- 172.21.15.52:0/1392250186 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0841a05e0 con 0x7ff084100770 2023-11-27T15:17:33.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.259+0000 7ff0817fa700 1 -- 172.21.15.52:0/1392250186 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff07c012420 con 0x7ff084100770 2023-11-27T15:17:33.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.259+0000 7ff0817fa700 1 --2- 172.21.15.52:0/1392250186 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff06c0a11e0 0x7ff06c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.259+0000 7ff083fff700 1 --2- 172.21.15.52:0/1392250186 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff06c0a11e0 0x7ff06c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.260+0000 7ff0817fa700 1 -- 172.21.15.52:0/1392250186 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff07c0ccc30 con 0x7ff084100770 2023-11-27T15:17:33.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.260+0000 7ff083fff700 1 --2- 172.21.15.52:0/1392250186 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff06c0a11e0 0x7ff06c0a3690 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff07400ac50 tx=0x7ff07400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:33.268 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.264+0000 7ff0817fa700 1 -- 172.21.15.52:0/1392250186 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff07c096310 con 0x7ff084100770 2023-11-27T15:17:33.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.466+0000 7ff08a8e5700 1 -- 172.21.15.52:0/1392250186 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0841a0b90 con 0x7ff084100770 2023-11-27T15:17:33.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.468+0000 7ff0817fa700 1 -- 172.21.15.52:0/1392250186 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22657 (secure 0 0 0) 0x7ff07c099930 con 0x7ff084100770 2023-11-27T15:17:33.473 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:17:33.473 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":85,"fsid":"c6c7df88-8d36-11ee-95a2-87774f69a715","created":"2023-11-27T15:09:03.333345+0000","modified":"2023-11-27T15:17:04.797892+0000","last_up_change":"2023-11-27T15:15:27.656134+0000","last_in_change":"2023-11-27T15:15:10.671153+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:12:14.177523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:16:07.842251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:16:08.728520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58d69ede-c293-49f5-abbe-8187793f2b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6803","nonce":608216415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6805","nonce":608216415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6809","nonce":608216415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6807","nonce":608216415}]},"public_addr":"172.21.15.52:6803/608216415","cluster_addr":"172.21.15.52:6805/608216415","heartbeat_back_addr":"172.21.15.52:6809/608216415","heartbeat_front_addr":"172.21.15.52:6807/608216415","state":["exists","up"]},{"osd":1,"uuid":"d6a46b91-adc8-45ef-80f5-566ae400911f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6801","nonce":2538384087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6803","nonce":2538384087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6807","nonce":2538384087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6805","nonce":2538384087}]},"public_addr":"172.21.15.102:6801/2538384087","cluster_addr":"172.21.15.102:6803/2538384087","heartbeat_back_addr":"172.21.15.102:6807/2538384087","heartbeat_front_addr":"172.21.15.102:6805/2538384087","state":["exists","up"]},{"osd":2,"uuid":"846ba125-11f8-46b2-8941-bd5e1947cc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6801","nonce":4258038221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6803","nonce":4258038221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6807","nonce":4258038221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6805","nonce":4258038221}]},"public_addr":"172.21.15.150:6801/4258038221","cluster_addr":"172.21.15.150:6803/4258038221","heartbeat_back_addr":"172.21.15.150:6807/4258038221","heartbeat_front_addr":"172.21.15.150:6805/4258038221","state":["exists","up"]},{"osd":3,"uuid":"950fcd67-1057-4288-a389-714f1ec94261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6811","nonce":2622100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6813","nonce":2622100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6817","nonce":2622100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6815","nonce":2622100752}]},"public_addr":"172.21.15.52:6811/2622100752","cluster_addr":"172.21.15.52:6813/2622100752","heartbeat_back_addr":"172.21.15.52:6817/2622100752","heartbeat_front_addr":"172.21.15.52:6815/2622100752","state":["exists","up"]},{"osd":4,"uuid":"1c96a676-cea9-4deb-bad1-f7fd240c3d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6809","nonce":1546794763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6811","nonce":1546794763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6815","nonce":1546794763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6813","nonce":1546794763}]},"public_addr":"172.21.15.102:6809/1546794763","cluster_addr":"172.21.15.102:6811/1546794763","heartbeat_back_addr":"172.21.15.102:6815/1546794763","heartbeat_front_addr":"172.21.15.102:6813/1546794763","state":["exists","up"]},{"osd":5,"uuid":"38a75af1-fd4d-433e-b579-0609109c851c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6809","nonce":461511375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6811","nonce":461511375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6815","nonce":461511375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6813","nonce":461511375}]},"public_addr":"172.21.15.150:6809/461511375","cluster_addr":"172.21.15.150:6811/461511375","heartbeat_back_addr":"172.21.15.150:6815/461511375","heartbeat_front_addr":"172.21.15.150:6813/461511375","state":["exists","up"]},{"osd":6,"uuid":"2f0ede4a-d175-4b4a-b876-c0f442048901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6819","nonce":1002457464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6821","nonce":1002457464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6825","nonce":1002457464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6823","nonce":1002457464}]},"public_addr":"172.21.15.52:6819/1002457464","cluster_addr":"172.21.15.52:6821/1002457464","heartbeat_back_addr":"172.21.15.52:6825/1002457464","heartbeat_front_addr":"172.21.15.52:6823/1002457464","state":["exists","up"]},{"osd":7,"uuid":"6b635192-abcf-4bbf-bf78-a019b351bca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6817","nonce":3249840064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6819","nonce":3249840064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6823","nonce":3249840064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6821","nonce":3249840064}]},"public_addr":"172.21.15.102:6817/3249840064","cluster_addr":"172.21.15.102:6819/3249840064","heartbeat_back_addr":"172.21.15.102:6823/3249840064","heartbeat_front_addr":"172.21.15.102:6821/3249840064","state":["exists","up"]},{"osd":8,"uuid":"c7ca8007-7170-47bf-956c-3f00edf27e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6817","nonce":3672583574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6819","nonce":3672583574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6823","nonce":3672583574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6821","nonce":3672583574}]},"public_addr":"172.21.15.150:6817/3672583574","cluster_addr":"172.21.15.150:6819/3672583574","heartbeat_back_addr":"172.21.15.150:6823/3672583574","heartbeat_front_addr":"172.21.15.150:6821/3672583574","state":["exists","up"]},{"osd":9,"uuid":"0e252c2c-01a4-48ba-832d-067ed940fefb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6827","nonce":4287934320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6829","nonce":4287934320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6833","nonce":4287934320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6831","nonce":4287934320}]},"public_addr":"172.21.15.52:6827/4287934320","cluster_addr":"172.21.15.52:6829/4287934320","heartbeat_back_addr":"172.21.15.52:6833/4287934320","heartbeat_front_addr":"172.21.15.52:6831/4287934320","state":["exists","up"]},{"osd":10,"uuid":"bb540142-7563-44a4-b7a3-cb662714c128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6825","nonce":3056034201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6827","nonce":3056034201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6831","nonce":3056034201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6829","nonce":3056034201}]},"public_addr":"172.21.15.102:6825/3056034201","cluster_addr":"172.21.15.102:6827/3056034201","heartbeat_back_addr":"172.21.15.102:6831/3056034201","heartbeat_front_addr":"172.21.15.102:6829/3056034201","state":["exists","up"]},{"osd":11,"uuid":"3156528e-e0f8-4d44-9946-c8b6339c2bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6825","nonce":3207559739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6827","nonce":3207559739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6831","nonce":3207559739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6829","nonce":3207559739}]},"public_addr":"172.21.15.150:6825/3207559739","cluster_addr":"172.21.15.150:6827/3207559739","heartbeat_back_addr":"172.21.15.150:6831/3207559739","heartbeat_front_addr":"172.21.15.150:6829/3207559739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:26.826099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:48.331371+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:10.459738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:32.356472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:54.043170+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:15.648582+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:37.166463+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:59.240819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:20.710097+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:43.364563+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:05.279433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:26.344573+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":5,"to":7}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/1392282110":"2023-11-28T15:10:03.644870+0000","172.21.15.52:0/1479993479":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6800/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/961562652":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/1290665390":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2410529742":"2023-11-28T15:09:37.118690+0000","172.21.15.52:6800/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2130037044":"2023-11-28T15:10:03.644870+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:17:33.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.470+0000 7ff08a8e5700 1 -- 172.21.15.52:0/1392250186 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff06c0a11e0 msgr2=0x7ff06c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:33.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.470+0000 7ff08a8e5700 1 --2- 172.21.15.52:0/1392250186 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff06c0a11e0 0x7ff06c0a3690 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff07400ac50 tx=0x7ff07400a380 comp rx=0 tx=0).stop 2023-11-27T15:17:33.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.470+0000 7ff08a8e5700 1 -- 172.21.15.52:0/1392250186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff084100770 msgr2=0x7ff084101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:33.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.470+0000 7ff08a8e5700 1 --2- 172.21.15.52:0/1392250186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff084100770 0x7ff084101d90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff07c007d70 tx=0x7ff07c00a340 comp rx=0 tx=0).stop 2023-11-27T15:17:33.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.470+0000 7ff08a8e5700 1 -- 172.21.15.52:0/1392250186 shutdown_connections 2023-11-27T15:17:33.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.470+0000 7ff08a8e5700 1 --2- 172.21.15.52:0/1392250186 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7ff06c0a11e0 0x7ff06c0a3690 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.470+0000 7ff08a8e5700 1 --2- 172.21.15.52:0/1392250186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff084100770 0x7ff084101d90 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.470+0000 7ff08a8e5700 1 --2- 172.21.15.52:0/1392250186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0840ff460 0x7ff084103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.470+0000 7ff08a8e5700 1 --2- 172.21.15.52:0/1392250186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0840ffe10 0x7ff084101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.470+0000 7ff08a8e5700 1 -- 172.21.15.52:0/1392250186 >> 172.21.15.52:0/1392250186 conn(0x7ff0840f9ee0 msgr2=0x7ff0840fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:33.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.470+0000 7ff08a8e5700 1 -- 172.21.15.52:0/1392250186 shutdown_connections 2023-11-27T15:17:33.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.470+0000 7ff08a8e5700 1 -- 172.21.15.52:0/1392250186 wait complete. 2023-11-27T15:17:33.487 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-11-27T15:16:09.319630+0000', 'modified': '2023-11-27T15:16:14.742405+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24536, 'mds_1': 24494, 'mds_2': 24472}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.150:6835/617413375', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6834', 'nonce': 617413375}, {'type': 'v1', 'addr': '172.21.15.150:6835', 'nonce': 617413375}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24472': {'gid': 24472, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.150:6837/805360541', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6836', 'nonce': 805360541}, {'type': 'v1', 'addr': '172.21.15.150:6837', 'nonce': 805360541}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24473': {'gid': 24473, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.102:6835/2398692013', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6834', 'nonce': 2398692013}, {'type': 'v1', 'addr': '172.21.15.102:6835', 'nonce': 2398692013}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.150:6839/215725838', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6838', 'nonce': 215725838}, {'type': 'v1', 'addr': '172.21.15.150:6839', 'nonce': 215725838}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24494': {'gid': 24494, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.102:6837/2141793919', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6836', 'nonce': 2141793919}, {'type': 'v1', 'addr': '172.21.15.102:6837', 'nonce': 2141793919}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24536': {'gid': 24536, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.102:6839/3571830559', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6838', 'nonce': 3571830559}, {'type': 'v1', 'addr': '172.21.15.102:6839', 'nonce': 3571830559}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-27T15:17:33.487 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-11-27T15:17:33.488 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-27T15:17:33.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.610+0000 7f9f5fc4a700 1 -- 172.21.15.52:0/2405241678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f58100cc0 msgr2=0x7f9f58107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:33.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.610+0000 7f9f5fc4a700 1 --2- 172.21.15.52:0/2405241678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f58100cc0 0x7f9f58107580 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000b0a0 tx=0x7f9f5000b3b0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.610+0000 7f9f5fc4a700 1 -- 172.21.15.52:0/2405241678 shutdown_connections 2023-11-27T15:17:33.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.610+0000 7f9f5fc4a700 1 --2- 172.21.15.52:0/2405241678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f580ff140 0x7f9f580ff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.610+0000 7f9f5fc4a700 1 --2- 172.21.15.52:0/2405241678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f58100cc0 0x7f9f58107580 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.610+0000 7f9f5fc4a700 1 --2- 172.21.15.52:0/2405241678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f58100330 0x7f9f58100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.610+0000 7f9f5fc4a700 1 -- 172.21.15.52:0/2405241678 >> 172.21.15.52:0/2405241678 conn(0x7f9f580fa720 msgr2=0x7f9f580fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:33.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.610+0000 7f9f5fc4a700 1 -- 172.21.15.52:0/2405241678 shutdown_connections 2023-11-27T15:17:33.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.610+0000 7f9f5fc4a700 1 -- 172.21.15.52:0/2405241678 wait complete. 2023-11-27T15:17:33.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.610+0000 7f9f5fc4a700 1 Processor -- start 2023-11-27T15:17:33.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.610+0000 7f9f5fc4a700 1 -- start start 2023-11-27T15:17:33.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.611+0000 7f9f5fc4a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f580ff140 0x7f9f58194fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.611+0000 7f9f5fc4a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f58100330 0x7f9f58195520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.611+0000 7f9f5fc4a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f58100cc0 0x7f9f5819c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.611+0000 7f9f5fc4a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5819cb60 con 0x7f9f58100cc0 2023-11-27T15:17:33.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.611+0000 7f9f5fc4a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5819cca0 con 0x7f9f58100330 2023-11-27T15:17:33.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.611+0000 7f9f5fc4a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5819ce30 con 0x7f9f580ff140 2023-11-27T15:17:33.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.611+0000 7f9f5e1e7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f58100cc0 0x7f9f5819c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.611+0000 7f9f5e1e7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f58100cc0 0x7f9f5819c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51208/0 (socket says 172.21.15.52:51208) 2023-11-27T15:17:33.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.611+0000 7f9f5e1e7700 1 -- 172.21.15.52:0/3234237383 learned_addr learned my addr 172.21.15.52:0/3234237383 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:33.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.611+0000 7f9f5d9e6700 1 --2- 172.21.15.52:0/3234237383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f580ff140 0x7f9f58194fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.611+0000 7f9f5d1e5700 1 --2- 172.21.15.52:0/3234237383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f58100330 0x7f9f58195520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.611+0000 7f9f5e1e7700 1 -- 172.21.15.52:0/3234237383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f580ff140 msgr2=0x7f9f58194fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:33.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.611+0000 7f9f5e1e7700 1 --2- 172.21.15.52:0/3234237383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f580ff140 0x7f9f58194fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.611+0000 7f9f5e1e7700 1 -- 172.21.15.52:0/3234237383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f58100330 msgr2=0x7f9f58195520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:33.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.611+0000 7f9f5e1e7700 1 --2- 172.21.15.52:0/3234237383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f58100330 0x7f9f58195520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.611+0000 7f9f5e1e7700 1 -- 172.21.15.52:0/3234237383 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f50009d00 con 0x7f9f58100cc0 2023-11-27T15:17:33.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.611+0000 7f9f5e1e7700 1 --2- 172.21.15.52:0/3234237383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f58100cc0 0x7f9f5819c590 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000d010 tx=0x7f9f50007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:33.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.611+0000 7f9f4effd700 1 -- 172.21.15.52:0/3234237383 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5000e040 con 0x7f9f58100cc0 2023-11-27T15:17:33.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.611+0000 7f9f4effd700 1 -- 172.21.15.52:0/3234237383 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f5001d480 con 0x7f9f58100cc0 2023-11-27T15:17:33.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.611+0000 7f9f5fc4a700 1 -- 172.21.15.52:0/3234237383 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5810ab10 con 0x7f9f58100cc0 2023-11-27T15:17:33.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.611+0000 7f9f4effd700 1 -- 172.21.15.52:0/3234237383 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f50012630 con 0x7f9f58100cc0 2023-11-27T15:17:33.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.611+0000 7f9f5fc4a700 1 -- 172.21.15.52:0/3234237383 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5810a980 con 0x7f9f58100cc0 2023-11-27T15:17:33.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.613+0000 7f9f4effd700 1 -- 172.21.15.52:0/3234237383 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9f5001d8f0 con 0x7f9f58100cc0 2023-11-27T15:17:33.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.614+0000 7f9f4effd700 1 --2- 172.21.15.52:0/3234237383 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f9f440a11e0 0x7f9f440a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.614+0000 7f9f4effd700 1 -- 172.21.15.52:0/3234237383 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9f500cd6b0 con 0x7f9f58100cc0 2023-11-27T15:17:33.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.614+0000 7f9f5d9e6700 1 --2- 172.21.15.52:0/3234237383 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f9f440a11e0 0x7f9f440a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.615+0000 7f9f5d9e6700 1 --2- 172.21.15.52:0/3234237383 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f9f440a11e0 0x7f9f440a3690 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9f58100190 tx=0x7f9f54007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:33.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.615+0000 7f9f6bdd6b80 1 Processor -- start 2023-11-27T15:17:33.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.615+0000 7f9f6bdd6b80 1 -- start start 2023-11-27T15:17:33.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.615+0000 7f9f6bdd6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a350f01bf0 0x55a350f02000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.615+0000 7f9f6bdd6b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a350f02540 0x55a350f02ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.615+0000 7f9f6bdd6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a350f03070 0x55a350f03510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.615+0000 7f9f6bdd6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a350e07010 con 0x55a350f03070 2023-11-27T15:17:33.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.615+0000 7f9f6bdd6b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a350e59850 con 0x55a350f02540 2023-11-27T15:17:33.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.615+0000 7f9f6bdd6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a350e52ce0 con 0x55a350f01bf0 2023-11-27T15:17:33.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.615+0000 7f9f5e1e7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a350f03070 0x55a350f03510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.615+0000 7f9f5e1e7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a350f03070 0x55a350f03510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51216/0 (socket says 172.21.15.52:51216) 2023-11-27T15:17:33.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.615+0000 7f9f5e1e7700 1 -- 172.21.15.52:0/4241598934 learned_addr learned my addr 172.21.15.52:0/4241598934 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:33.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.615+0000 7f9f5d1e5700 1 --2- 172.21.15.52:0/4241598934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a350f02540 0x55a350f02ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.616+0000 7f9f5d9e6700 1 --2- 172.21.15.52:0/4241598934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a350f01bf0 0x55a350f02000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.616+0000 7f9f5e1e7700 1 -- 172.21.15.52:0/4241598934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a350f01bf0 msgr2=0x55a350f02000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:33.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.616+0000 7f9f5e1e7700 1 --2- 172.21.15.52:0/4241598934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a350f01bf0 0x55a350f02000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.616+0000 7f9f5e1e7700 1 -- 172.21.15.52:0/4241598934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a350f02540 msgr2=0x55a350f02ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:33.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.616+0000 7f9f5e1e7700 1 --2- 172.21.15.52:0/4241598934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a350f02540 0x55a350f02ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.616+0000 7f9f5e1e7700 1 -- 172.21.15.52:0/4241598934 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a350cd45e0 con 0x55a350f03070 2023-11-27T15:17:33.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.616+0000 7f9f5e1e7700 1 --2- 172.21.15.52:0/4241598934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a350f03070 0x55a350f03510 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000b070 tx=0x7f9f500099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1d8eb2f6934365 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:33.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.616+0000 7f9f43fff700 1 -- 172.21.15.52:0/4241598934 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5009a510 con 0x55a350f03070 2023-11-27T15:17:33.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.616+0000 7f9f43fff700 1 -- 172.21.15.52:0/4241598934 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f5009a330 con 0x55a350f03070 2023-11-27T15:17:33.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.616+0000 7f9f43fff700 1 -- 172.21.15.52:0/4241598934 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5009ac30 con 0x55a350f03070 2023-11-27T15:17:33.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.617+0000 7f9f6bdd6b80 1 -- 172.21.15.52:0/4241598934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a350f03070 msgr2=0x55a350f03510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:33.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.617+0000 7f9f6bdd6b80 1 --2- 172.21.15.52:0/4241598934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a350f03070 0x55a350f03510 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000b070 tx=0x7f9f500099e0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.617+0000 7f9f6bdd6b80 1 -- 172.21.15.52:0/4241598934 shutdown_connections 2023-11-27T15:17:33.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.617+0000 7f9f6bdd6b80 1 --2- 172.21.15.52:0/4241598934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a350f03070 0x55a350f03510 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.617+0000 7f9f6bdd6b80 1 --2- 172.21.15.52:0/4241598934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a350f01bf0 0x55a350f02000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.617+0000 7f9f6bdd6b80 1 --2- 172.21.15.52:0/4241598934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a350f02540 0x55a350f02ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.617+0000 7f9f6bdd6b80 1 -- 172.21.15.52:0/4241598934 >> 172.21.15.52:0/4241598934 conn(0x55a350ef7310 msgr2=0x55a350efc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:33.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.617+0000 7f9f6bdd6b80 1 -- 172.21.15.52:0/4241598934 shutdown_connections 2023-11-27T15:17:33.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.617+0000 7f9f6bdd6b80 1 -- 172.21.15.52:0/4241598934 wait complete. 2023-11-27T15:17:33.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.617+0000 7f9f6bdd6b80 1 -- start start 2023-11-27T15:17:33.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.617+0000 7f9f6bdd6b80 1 Processor -- start 2023-11-27T15:17:33.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.618+0000 7f9f6bdd6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a350f01bf0 0x55a350f0f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.618+0000 7f9f6bdd6b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a350f02540 0x55a350f0fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.618+0000 7f9f6bdd6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a350f03070 0x55a350f16d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.618+0000 7f9f6bdd6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a350df4890 con 0x55a350f03070 2023-11-27T15:17:33.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.618+0000 7f9f6bdd6b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a350df4530 con 0x55a350f02540 2023-11-27T15:17:33.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.618+0000 7f9f6bdd6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a350cd68d0 con 0x55a350f01bf0 2023-11-27T15:17:33.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.618+0000 7f9f6bdd6b80 10 client.0 _ll_register_callbacks cb 0x55a350e12730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:17:33.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.618+0000 7f9f5e1e7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a350f03070 0x55a350f16d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.618+0000 7f9f5e1e7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a350f03070 0x55a350f16d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51222/0 (socket says 172.21.15.52:51222) 2023-11-27T15:17:33.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.618+0000 7f9f5e1e7700 1 -- 172.21.15.52:0/219766831 learned_addr learned my addr 172.21.15.52:0/219766831 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:33.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.618+0000 7f9f5d1e5700 1 --2- 172.21.15.52:0/219766831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a350f02540 0x55a350f0fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.618+0000 7f9f5d9e6700 1 --2- 172.21.15.52:0/219766831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a350f01bf0 0x55a350f0f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.618+0000 7f9f5e1e7700 1 -- 172.21.15.52:0/219766831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a350f01bf0 msgr2=0x55a350f0f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:33.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.618+0000 7f9f5e1e7700 1 --2- 172.21.15.52:0/219766831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a350f01bf0 0x55a350f0f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.618+0000 7f9f5e1e7700 1 -- 172.21.15.52:0/219766831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a350f02540 msgr2=0x55a350f0fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:33.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.618+0000 7f9f5e1e7700 1 --2- 172.21.15.52:0/219766831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a350f02540 0x55a350f0fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.618+0000 7f9f5e1e7700 1 -- 172.21.15.52:0/219766831 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5810a980 con 0x55a350f03070 2023-11-27T15:17:33.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.618+0000 7f9f5e1e7700 1 --2- 172.21.15.52:0/219766831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a350f03070 0x55a350f16d90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9f500cb0c0 tx=0x7f9f500cb4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adcad4f1079d2a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:33.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.618+0000 7f9f417fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:17:33.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.618+0000 7f9f417fa700 1 -- 172.21.15.52:0/219766831 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5009a510 con 0x55a350f03070 2023-11-27T15:17:33.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.618+0000 7f9f417fa700 1 -- 172.21.15.52:0/219766831 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9f5009a9d0 con 0x55a350f03070 2023-11-27T15:17:33.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.618+0000 7f9f6bdd6b80 1 -- 172.21.15.52:0/219766831 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a350ef4ad0 con 0x55a350f03070 2023-11-27T15:17:33.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.619+0000 7f9f417fa700 1 -- 172.21.15.52:0/219766831 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f5001d5f0 con 0x55a350f03070 2023-11-27T15:17:33.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.619+0000 7f9f6bdd6b80 10 client.15168 fetch_fsmap learned FSMap version 23 2023-11-27T15:17:33.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.619+0000 7f9f6bdd6b80 1 -- 172.21.15.52:0/219766831 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55a350e19140 con 0x55a350f03070 2023-11-27T15:17:33.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.619+0000 7f9f417fa700 1 -- 172.21.15.52:0/219766831 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a350ef4ad0 con 0x55a350f03070 2023-11-27T15:17:33.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.619+0000 7f9f417fa700 1 -- 172.21.15.52:0/219766831 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9f500967c0 con 0x55a350f03070 2023-11-27T15:17:33.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.619+0000 7f9f6bdd6b80 10 client.15168 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:17:33.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.619+0000 7f9f6bdd6b80 10 client.15168 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-27T15:17:33.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.619+0000 7f9f6bdd6b80 20 client.15168 populate_metadata read hostname 'smithi052' 2023-11-27T15:17:33.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.619+0000 7f9f6bdd6b80 1 --2- 172.21.15.52:0/219766831 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55a350f1a690 0x55a350f19cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.619+0000 7f9f6bdd6b80 4 client.15168 mds_command: new command op to 24472 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:17:33.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.619+0000 7f9f6bdd6b80 1 -- 172.21.15.52:0/219766831 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a350e15050 con 0x55a350f1a690 2023-11-27T15:17:33.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.619+0000 7f9f5d9e6700 1 --2- 172.21.15.52:0/219766831 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55a350f1a690 0x55a350f19cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.620+0000 7f9f5d9e6700 1 --2- 172.21.15.52:0/219766831 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55a350f1a690 0x55a350f19cc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dd9e968736e24d7c server_cookie=4610a01bc2bda8ea in_seq=0 out_seq=0 2023-11-27T15:17:33.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.620+0000 7f9f417fa700 10 client.15168 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:17:33.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.620+0000 7f9f417fa700 1 -- 172.21.15.52:0/219766831 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a350e15050 con 0x55a350f1a690 2023-11-27T15:17:33.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.620+0000 7f9f417fa700 10 client.15168 handle_command_reply: tid=0 2023-11-27T15:17:33.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.620+0000 7f9f6bdd6b80 1 client.15168 shutdown 2023-11-27T15:17:33.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.621+0000 7f9f6bdd6b80 1 -- 172.21.15.52:0/219766831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a350f03070 msgr2=0x55a350f16d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:33.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.621+0000 7f9f6bdd6b80 1 --2- 172.21.15.52:0/219766831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a350f03070 0x55a350f16d90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9f500cb0c0 tx=0x7f9f500cb4c0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.621+0000 7f9f6bdd6b80 1 -- 172.21.15.52:0/219766831 shutdown_connections 2023-11-27T15:17:33.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.621+0000 7f9f6bdd6b80 1 --2- 172.21.15.52:0/219766831 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55a350f1a690 0x55a350f19cc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.621+0000 7f9f6bdd6b80 1 --2- 172.21.15.52:0/219766831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a350f03070 0x55a350f16d90 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.621+0000 7f9f6bdd6b80 1 --2- 172.21.15.52:0/219766831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a350f01bf0 0x55a350f0f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.621+0000 7f9f6bdd6b80 1 --2- 172.21.15.52:0/219766831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a350f02540 0x55a350f0fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.621+0000 7f9f417fa700 0 client.15168 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:17:33.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.621+0000 7f9f6bdd6b80 1 -- 172.21.15.52:0/219766831 >> 172.21.15.52:0/219766831 conn(0x55a350ef7310 msgr2=0x55a350efbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:33.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.621+0000 7f9f6bdd6b80 1 -- 172.21.15.52:0/219766831 shutdown_connections 2023-11-27T15:17:33.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.621+0000 7f9f6bdd6b80 1 -- 172.21.15.52:0/219766831 wait complete. 2023-11-27T15:17:33.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.621+0000 7f9f6bdd6b80 20 client.15168 trim_cache size 0 max 16384 2023-11-27T15:17:33.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.639+0000 7f9f6bdd6b80 1 Processor -- start 2023-11-27T15:17:33.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.639+0000 7f9f6bdd6b80 1 -- start start 2023-11-27T15:17:33.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.639+0000 7f9f6bdd6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a350f01bf0 0x55a350f19cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.639+0000 7f9f6bdd6b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a350f02540 0x55a350f16d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.639+0000 7f9f6bdd6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a350f03070 0x55a350f0d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.639+0000 7f9f6bdd6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a350e59850 con 0x55a350f03070 2023-11-27T15:17:33.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.639+0000 7f9f6bdd6b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a350e07010 con 0x55a350f02540 2023-11-27T15:17:33.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.639+0000 7f9f6bdd6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a350f105a0 con 0x55a350f01bf0 2023-11-27T15:17:33.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.639+0000 7f9f5e1e7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a350f03070 0x55a350f0d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.639+0000 7f9f5e1e7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a350f03070 0x55a350f0d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51230/0 (socket says 172.21.15.52:51230) 2023-11-27T15:17:33.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.639+0000 7f9f5e1e7700 1 -- 172.21.15.52:0/3365239818 learned_addr learned my addr 172.21.15.52:0/3365239818 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:33.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.639+0000 7f9f5d9e6700 1 --2- 172.21.15.52:0/3365239818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a350f01bf0 0x55a350f19cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.639+0000 7f9f5d1e5700 1 --2- 172.21.15.52:0/3365239818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a350f02540 0x55a350f16d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.639+0000 7f9f5e1e7700 1 -- 172.21.15.52:0/3365239818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a350f01bf0 msgr2=0x55a350f19cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:33.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.639+0000 7f9f5e1e7700 1 --2- 172.21.15.52:0/3365239818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a350f01bf0 0x55a350f19cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.639+0000 7f9f5e1e7700 1 -- 172.21.15.52:0/3365239818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a350f02540 msgr2=0x55a350f16d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:33.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.639+0000 7f9f5e1e7700 1 --2- 172.21.15.52:0/3365239818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a350f02540 0x55a350f16d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.639+0000 7f9f5e1e7700 1 -- 172.21.15.52:0/3365239818 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a350e18e90 con 0x55a350f03070 2023-11-27T15:17:33.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.639+0000 7f9f5d1e5700 1 --2- 172.21.15.52:0/3365239818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a350f02540 0x55a350f16d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:17:33.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.639+0000 7f9f5e1e7700 1 --2- 172.21.15.52:0/3365239818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a350f03070 0x55a350f0d680 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9f5009a960 tx=0x7f9f500099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3af3eaf3dd7a31 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:33.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.640+0000 7f9f4cdac700 1 -- 172.21.15.52:0/3365239818 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f500a2e00 con 0x55a350f03070 2023-11-27T15:17:33.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.640+0000 7f9f4cdac700 1 -- 172.21.15.52:0/3365239818 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f5009a510 con 0x55a350f03070 2023-11-27T15:17:33.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.640+0000 7f9f4cdac700 1 -- 172.21.15.52:0/3365239818 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f50098d70 con 0x55a350f03070 2023-11-27T15:17:33.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.640+0000 7f9f6bdd6b80 1 -- 172.21.15.52:0/3365239818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a350f03070 msgr2=0x55a350f0d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:33.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.640+0000 7f9f6bdd6b80 1 --2- 172.21.15.52:0/3365239818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a350f03070 0x55a350f0d680 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9f5009a960 tx=0x7f9f500099e0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.640+0000 7f9f6bdd6b80 1 -- 172.21.15.52:0/3365239818 shutdown_connections 2023-11-27T15:17:33.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.640+0000 7f9f6bdd6b80 1 --2- 172.21.15.52:0/3365239818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a350f03070 0x55a350f0d680 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.640+0000 7f9f6bdd6b80 1 --2- 172.21.15.52:0/3365239818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a350f01bf0 0x55a350f19cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.640+0000 7f9f6bdd6b80 1 --2- 172.21.15.52:0/3365239818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a350f02540 0x55a350f16d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.640+0000 7f9f6bdd6b80 1 -- 172.21.15.52:0/3365239818 >> 172.21.15.52:0/3365239818 conn(0x55a350ef7310 msgr2=0x55a350f0e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:33.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.640+0000 7f9f6bdd6b80 1 -- 172.21.15.52:0/3365239818 shutdown_connections 2023-11-27T15:17:33.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.640+0000 7f9f6bdd6b80 1 -- 172.21.15.52:0/3365239818 wait complete. 2023-11-27T15:17:33.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.641+0000 7f9f6bdd6b80 1 -- start start 2023-11-27T15:17:33.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.641+0000 7f9f6bdd6b80 1 Processor -- start 2023-11-27T15:17:33.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.641+0000 7f9f6bdd6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a350f01bf0 0x55a350f11720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.641+0000 7f9f6bdd6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a350f02540 0x55a350f16c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.641+0000 7f9f6bdd6b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a350f03070 0x55a350f11cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.641+0000 7f9f6bdd6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a350f6e4f0 con 0x55a350f01bf0 2023-11-27T15:17:33.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.641+0000 7f9f6bdd6b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a350f6e630 con 0x55a350f03070 2023-11-27T15:17:33.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.641+0000 7f9f6bdd6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a350f6e770 con 0x55a350f02540 2023-11-27T15:17:33.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.641+0000 7f9f6bdd6b80 10 client.0 _ll_register_callbacks cb 0x7f9f48009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:17:33.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.641+0000 7f9f5d9e6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a350f01bf0 0x55a350f11720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.641+0000 7f9f5d9e6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a350f01bf0 0x55a350f11720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51232/0 (socket says 172.21.15.52:51232) 2023-11-27T15:17:33.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.641+0000 7f9f5d9e6700 1 -- 172.21.15.52:0/928216169 learned_addr learned my addr 172.21.15.52:0/928216169 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:33.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.641+0000 7f9f5e1e7700 1 --2- 172.21.15.52:0/928216169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a350f03070 0x55a350f11cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.641+0000 7f9f5d1e5700 1 --2- 172.21.15.52:0/928216169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a350f02540 0x55a350f16c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.641+0000 7f9f5d9e6700 1 -- 172.21.15.52:0/928216169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a350f02540 msgr2=0x55a350f16c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:33.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.641+0000 7f9f5d9e6700 1 --2- 172.21.15.52:0/928216169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a350f02540 0x55a350f16c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.641+0000 7f9f5d9e6700 1 -- 172.21.15.52:0/928216169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a350f03070 msgr2=0x55a350f11cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:33.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.641+0000 7f9f5d9e6700 1 --2- 172.21.15.52:0/928216169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a350f03070 0x55a350f11cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.641+0000 7f9f5d9e6700 1 -- 172.21.15.52:0/928216169 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a350e19140 con 0x55a350f01bf0 2023-11-27T15:17:33.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.642+0000 7f9f5d9e6700 1 --2- 172.21.15.52:0/928216169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a350f01bf0 0x55a350f11720 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x55a350f10570 tx=0x7f9f54013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0e4c4cab30987d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:33.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.642+0000 7f9f417fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:17:33.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.642+0000 7f9f417fa700 1 -- 172.21.15.52:0/928216169 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f54026040 con 0x55a350f01bf0 2023-11-27T15:17:33.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.642+0000 7f9f417fa700 1 -- 172.21.15.52:0/928216169 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9f54017070 con 0x55a350f01bf0 2023-11-27T15:17:33.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.642+0000 7f9f6bdd6b80 1 -- 172.21.15.52:0/928216169 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f5009a330 con 0x55a350f01bf0 2023-11-27T15:17:33.655 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.642+0000 7f9f417fa700 1 -- 172.21.15.52:0/928216169 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f54011e70 con 0x55a350f01bf0 2023-11-27T15:17:33.655 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.642+0000 7f9f6bdd6b80 10 client.15174 fetch_fsmap learned FSMap version 23 2023-11-27T15:17:33.655 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.642+0000 7f9f6bdd6b80 1 -- 172.21.15.52:0/928216169 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55a350c69c10 con 0x55a350f01bf0 2023-11-27T15:17:33.655 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.642+0000 7f9f417fa700 1 -- 172.21.15.52:0/928216169 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f5009a330 con 0x55a350f01bf0 2023-11-27T15:17:33.655 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.642+0000 7f9f417fa700 1 -- 172.21.15.52:0/928216169 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9f54026b90 con 0x55a350f01bf0 2023-11-27T15:17:33.655 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.642+0000 7f9f6bdd6b80 10 client.15174 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:17:33.655 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.642+0000 7f9f6bdd6b80 10 client.15174 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-27T15:17:33.655 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.642+0000 7f9f6bdd6b80 20 client.15174 populate_metadata read hostname 'smithi052' 2023-11-27T15:17:33.655 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.642+0000 7f9f6bdd6b80 1 --2- 172.21.15.52:0/928216169 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55a350f73530 0x55a350f6fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.655 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.642+0000 7f9f6bdd6b80 4 client.15174 mds_command: new command op to 24472 tid=0[{"prefix": "status"}] 2023-11-27T15:17:33.656 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.642+0000 7f9f6bdd6b80 1 -- 172.21.15.52:0/928216169 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a350e21610 con 0x55a350f73530 2023-11-27T15:17:33.656 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.643+0000 7f9f5d1e5700 1 --2- 172.21.15.52:0/928216169 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55a350f73530 0x55a350f6fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.656 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.643+0000 7f9f417fa700 10 client.15174 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:17:33.656 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.643+0000 7f9f5d1e5700 1 --2- 172.21.15.52:0/928216169 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55a350f73530 0x55a350f6fea0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d947bf9505ca0f6f server_cookie=17dff15a7f3d6dcd in_seq=0 out_seq=0 2023-11-27T15:17:33.656 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.643+0000 7f9f417fa700 1 -- 172.21.15.52:0/928216169 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55a350e21610 con 0x55a350f73530 2023-11-27T15:17:33.656 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.643+0000 7f9f417fa700 10 client.15174 handle_command_reply: tid=0 2023-11-27T15:17:33.656 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.643+0000 7f9f6bdd6b80 1 client.15174 shutdown 2023-11-27T15:17:33.656 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.643+0000 7f9f6bdd6b80 1 -- 172.21.15.52:0/928216169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a350f01bf0 msgr2=0x55a350f11720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:33.656 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.643+0000 7f9f6bdd6b80 1 --2- 172.21.15.52:0/928216169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a350f01bf0 0x55a350f11720 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x55a350f10570 tx=0x7f9f54013070 comp rx=0 tx=0).stop 2023-11-27T15:17:33.656 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.643+0000 7f9f6bdd6b80 1 -- 172.21.15.52:0/928216169 shutdown_connections 2023-11-27T15:17:33.657 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.643+0000 7f9f6bdd6b80 1 --2- 172.21.15.52:0/928216169 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55a350f73530 0x55a350f6fea0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.657 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.643+0000 7f9f6bdd6b80 1 --2- 172.21.15.52:0/928216169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a350f03070 0x55a350f11cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.657 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.643+0000 7f9f6bdd6b80 1 --2- 172.21.15.52:0/928216169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a350f01bf0 0x55a350f11720 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.657 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.643+0000 7f9f6bdd6b80 1 --2- 172.21.15.52:0/928216169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a350f02540 0x55a350f16c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.657 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.643+0000 7f9f417fa700 0 client.15174 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:17:33.657 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.643+0000 7f9f6bdd6b80 1 -- 172.21.15.52:0/928216169 >> 172.21.15.52:0/928216169 conn(0x55a350ef7310 msgr2=0x55a350f0a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:33.657 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.644+0000 7f9f6bdd6b80 1 -- 172.21.15.52:0/928216169 shutdown_connections 2023-11-27T15:17:33.657 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.644+0000 7f9f6bdd6b80 1 -- 172.21.15.52:0/928216169 wait complete. 2023-11-27T15:17:33.657 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.644+0000 7f9f6bdd6b80 20 client.15174 trim_cache size 0 max 16384 2023-11-27T15:17:33.658 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:17:33.658 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "c6c7df88-8d36-11ee-95a2-87774f69a715", 2023-11-27T15:17:33.658 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 2, 2023-11-27T15:17:33.658 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24472, 2023-11-27T15:17:33.658 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2023-11-27T15:17:33.658 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2023-11-27T15:17:33.658 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2023-11-27T15:17:33.658 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 79.904453509999996, 2023-11-27T15:17:33.659 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 23, 2023-11-27T15:17:33.659 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 85, 2023-11-27T15:17:33.659 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 83, 2023-11-27T15:17:33.659 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 100.78128637899999 2023-11-27T15:17:33.659 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:17:33.660 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.644+0000 7f9f5fc4a700 1 -- 172.21.15.52:0/3234237383 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f9f440a11e0 msgr2=0x7f9f440a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:33.660 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.644+0000 7f9f5fc4a700 1 --2- 172.21.15.52:0/3234237383 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f9f440a11e0 0x7f9f440a3690 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9f58100190 tx=0x7f9f54007400 comp rx=0 tx=0).stop 2023-11-27T15:17:33.660 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.644+0000 7f9f5fc4a700 1 -- 172.21.15.52:0/3234237383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f58100cc0 msgr2=0x7f9f5819c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:33.660 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.644+0000 7f9f5fc4a700 1 --2- 172.21.15.52:0/3234237383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f58100cc0 0x7f9f5819c590 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000d010 tx=0x7f9f50007df0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.660 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.644+0000 7f9f5fc4a700 1 -- 172.21.15.52:0/3234237383 shutdown_connections 2023-11-27T15:17:33.660 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.644+0000 7f9f5fc4a700 1 --2- 172.21.15.52:0/3234237383 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f9f440a11e0 0x7f9f440a3690 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.660 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.644+0000 7f9f5fc4a700 1 --2- 172.21.15.52:0/3234237383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9f58100cc0 0x7f9f5819c590 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.644+0000 7f9f5fc4a700 1 --2- 172.21.15.52:0/3234237383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f580ff140 0x7f9f58194fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.644+0000 7f9f5fc4a700 1 --2- 172.21.15.52:0/3234237383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f58100330 0x7f9f58195520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.644+0000 7f9f5fc4a700 1 -- 172.21.15.52:0/3234237383 >> 172.21.15.52:0/3234237383 conn(0x7f9f580fa720 msgr2=0x7f9f580fc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:33.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.644+0000 7f9f5fc4a700 1 -- 172.21.15.52:0/3234237383 shutdown_connections 2023-11-27T15:17:33.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.644+0000 7f9f5fc4a700 1 -- 172.21.15.52:0/3234237383 wait complete. 2023-11-27T15:17:33.662 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-11-27T15:17:33.810 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.806+0000 7f358026c700 1 -- 172.21.15.52:0/3533072803 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35780ff160 msgr2=0x7f35780ff570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:33.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.806+0000 7f358026c700 1 --2- 172.21.15.52:0/3533072803 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35780ff160 0x7f35780ff570 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3574009990 tx=0x7f3574009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.806+0000 7f358026c700 1 -- 172.21.15.52:0/3533072803 shutdown_connections 2023-11-27T15:17:33.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.806+0000 7f358026c700 1 --2- 172.21.15.52:0/3533072803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3578100ce0 0x7f35781075a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.806+0000 7f358026c700 1 --2- 172.21.15.52:0/3533072803 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35780ff160 0x7f35780ff570 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.806+0000 7f358026c700 1 --2- 172.21.15.52:0/3533072803 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3578100350 0x7f35781007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.806+0000 7f358026c700 1 -- 172.21.15.52:0/3533072803 >> 172.21.15.52:0/3533072803 conn(0x7f35780fa740 msgr2=0x7f35780fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:33.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.806+0000 7f358026c700 1 -- 172.21.15.52:0/3533072803 shutdown_connections 2023-11-27T15:17:33.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.806+0000 7f358026c700 1 -- 172.21.15.52:0/3533072803 wait complete. 2023-11-27T15:17:33.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.807+0000 7f358026c700 1 Processor -- start 2023-11-27T15:17:33.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.807+0000 7f358026c700 1 -- start start 2023-11-27T15:17:33.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.807+0000 7f358026c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35780ff160 0x7f3578194e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.807+0000 7f358026c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3578100350 0x7f35781953d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.807+0000 7f358026c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3578100ce0 0x7f357819c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.807+0000 7f358026c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357819ca10 con 0x7f35780ff160 2023-11-27T15:17:33.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.807+0000 7f358026c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357819cb50 con 0x7f3578100350 2023-11-27T15:17:33.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.807+0000 7f358026c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357819cce0 con 0x7f3578100ce0 2023-11-27T15:17:33.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.807+0000 7f357e008700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35780ff160 0x7f3578194e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.807+0000 7f357e008700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35780ff160 0x7f3578194e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51246/0 (socket says 172.21.15.52:51246) 2023-11-27T15:17:33.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.807+0000 7f357e008700 1 -- 172.21.15.52:0/3397448605 learned_addr learned my addr 172.21.15.52:0/3397448605 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:33.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.807+0000 7f357d807700 1 --2- 172.21.15.52:0/3397448605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3578100350 0x7f35781953d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.807+0000 7f357e809700 1 --2- 172.21.15.52:0/3397448605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3578100ce0 0x7f357819c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.807+0000 7f357e008700 1 -- 172.21.15.52:0/3397448605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3578100ce0 msgr2=0x7f357819c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:33.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.807+0000 7f357e008700 1 --2- 172.21.15.52:0/3397448605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3578100ce0 0x7f357819c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.807+0000 7f357e008700 1 -- 172.21.15.52:0/3397448605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3578100350 msgr2=0x7f35781953d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:33.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.807+0000 7f357e008700 1 --2- 172.21.15.52:0/3397448605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3578100350 0x7f35781953d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.807+0000 7f357e008700 1 -- 172.21.15.52:0/3397448605 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35740095f0 con 0x7f35780ff160 2023-11-27T15:17:33.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.807+0000 7f357e809700 1 --2- 172.21.15.52:0/3397448605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3578100ce0 0x7f357819c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:17:33.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.808+0000 7f357e008700 1 --2- 172.21.15.52:0/3397448605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35780ff160 0x7f3578194e90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3574004760 tx=0x7f3574009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:33.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.808+0000 7f356f7fe700 1 -- 172.21.15.52:0/3397448605 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357401d070 con 0x7f35780ff160 2023-11-27T15:17:33.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.808+0000 7f358026c700 1 -- 172.21.15.52:0/3397448605 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f357810ab10 con 0x7f35780ff160 2023-11-27T15:17:33.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.808+0000 7f356f7fe700 1 -- 172.21.15.52:0/3397448605 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3574004cd0 con 0x7f35780ff160 2023-11-27T15:17:33.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.808+0000 7f356f7fe700 1 -- 172.21.15.52:0/3397448605 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3574005290 con 0x7f35780ff160 2023-11-27T15:17:33.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.808+0000 7f358026c700 1 -- 172.21.15.52:0/3397448605 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f357810a980 con 0x7f35780ff160 2023-11-27T15:17:33.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.810+0000 7f356f7fe700 1 -- 172.21.15.52:0/3397448605 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3574003b60 con 0x7f35780ff160 2023-11-27T15:17:33.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.811+0000 7f356f7fe700 1 --2- 172.21.15.52:0/3397448605 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f35640a11e0 0x7f35640a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.811+0000 7f357d807700 1 --2- 172.21.15.52:0/3397448605 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f35640a11e0 0x7f35640a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.811+0000 7f356f7fe700 1 -- 172.21.15.52:0/3397448605 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f35740cdab0 con 0x7f35780ff160 2023-11-27T15:17:33.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.811+0000 7f357d807700 1 --2- 172.21.15.52:0/3397448605 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f35640a11e0 0x7f35640a3690 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3568005950 tx=0x7f356800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:33.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.811+0000 7f358c3f8b80 1 Processor -- start 2023-11-27T15:17:33.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.812+0000 7f358c3f8b80 1 -- start start 2023-11-27T15:17:33.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.812+0000 7f358c3f8b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d21cc01e50 0x55d21cc02260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.812+0000 7f358c3f8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d21cc02cf0 0x55d21cc03160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.812+0000 7f358c3f8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d21cc036a0 0x55d21cc09f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.812+0000 7f358c3f8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d21cb05010 con 0x55d21cc02cf0 2023-11-27T15:17:33.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.812+0000 7f358c3f8b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d21cb57850 con 0x55d21cc01e50 2023-11-27T15:17:33.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.812+0000 7f358c3f8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d21cb50ce0 con 0x55d21cc036a0 2023-11-27T15:17:33.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.812+0000 7f357d807700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d21cc02cf0 0x55d21cc03160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.812+0000 7f357d807700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d21cc02cf0 0x55d21cc03160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51256/0 (socket says 172.21.15.52:51256) 2023-11-27T15:17:33.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.812+0000 7f357d807700 1 -- 172.21.15.52:0/130888449 learned_addr learned my addr 172.21.15.52:0/130888449 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:33.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.812+0000 7f357e809700 1 --2- 172.21.15.52:0/130888449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d21cc036a0 0x55d21cc09f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.812+0000 7f357e008700 1 --2- 172.21.15.52:0/130888449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d21cc01e50 0x55d21cc02260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.812+0000 7f357d807700 1 -- 172.21.15.52:0/130888449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d21cc036a0 msgr2=0x55d21cc09f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:33.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.812+0000 7f357d807700 1 --2- 172.21.15.52:0/130888449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d21cc036a0 0x55d21cc09f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.812+0000 7f357d807700 1 -- 172.21.15.52:0/130888449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d21cc01e50 msgr2=0x55d21cc02260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:33.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.812+0000 7f357d807700 1 --2- 172.21.15.52:0/130888449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d21cc01e50 0x55d21cc02260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.812+0000 7f357d807700 1 -- 172.21.15.52:0/130888449 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d21c9d25e0 con 0x55d21cc02cf0 2023-11-27T15:17:33.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.812+0000 7f357e008700 1 --2- 172.21.15.52:0/130888449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d21cc01e50 0x55d21cc02260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:17:33.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.812+0000 7f357d807700 1 --2- 172.21.15.52:0/130888449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d21cc02cf0 0x55d21cc03160 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3568010fa0 tx=0x7f3568013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d74e71bf60fda9d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:33.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.813+0000 7f356cdac700 1 -- 172.21.15.52:0/130888449 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3568011db0 con 0x55d21cc02cf0 2023-11-27T15:17:33.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.813+0000 7f356cdac700 1 -- 172.21.15.52:0/130888449 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3568015070 con 0x55d21cc02cf0 2023-11-27T15:17:33.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.813+0000 7f356cdac700 1 -- 172.21.15.52:0/130888449 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356801c610 con 0x55d21cc02cf0 2023-11-27T15:17:33.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.813+0000 7f358c3f8b80 1 -- 172.21.15.52:0/130888449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d21cc02cf0 msgr2=0x55d21cc03160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:33.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.813+0000 7f358c3f8b80 1 --2- 172.21.15.52:0/130888449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d21cc02cf0 0x55d21cc03160 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3568010fa0 tx=0x7f3568013000 comp rx=0 tx=0).stop 2023-11-27T15:17:33.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.813+0000 7f358c3f8b80 1 -- 172.21.15.52:0/130888449 shutdown_connections 2023-11-27T15:17:33.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.813+0000 7f358c3f8b80 1 --2- 172.21.15.52:0/130888449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d21cc01e50 0x55d21cc02260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.813+0000 7f358c3f8b80 1 --2- 172.21.15.52:0/130888449 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d21cc036a0 0x55d21cc09f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.813+0000 7f358c3f8b80 1 --2- 172.21.15.52:0/130888449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d21cc02cf0 0x55d21cc03160 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.813+0000 7f358c3f8b80 1 -- 172.21.15.52:0/130888449 >> 172.21.15.52:0/130888449 conn(0x55d21cbf5310 msgr2=0x55d21cbfa620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:33.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.813+0000 7f358c3f8b80 1 -- 172.21.15.52:0/130888449 shutdown_connections 2023-11-27T15:17:33.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.814+0000 7f358c3f8b80 1 -- 172.21.15.52:0/130888449 wait complete. 2023-11-27T15:17:33.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.814+0000 7f358c3f8b80 1 -- start start 2023-11-27T15:17:33.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.814+0000 7f358c3f8b80 1 Processor -- start 2023-11-27T15:17:33.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.814+0000 7f358c3f8b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d21cc01e50 0x55d21cc0b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.814+0000 7f358c3f8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d21cc02cf0 0x55d21cc0bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.814+0000 7f358c3f8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d21cc036a0 0x55d21cc12da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.814+0000 7f358c3f8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d21caf2890 con 0x55d21cc02cf0 2023-11-27T15:17:33.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.814+0000 7f358c3f8b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d21caf2530 con 0x55d21cc01e50 2023-11-27T15:17:33.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.814+0000 7f358c3f8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d21c9d48d0 con 0x55d21cc036a0 2023-11-27T15:17:33.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.814+0000 7f358c3f8b80 10 client.0 _ll_register_callbacks cb 0x55d21cb10730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:17:33.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.814+0000 7f357d807700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d21cc02cf0 0x55d21cc0bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.814+0000 7f357e008700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d21cc01e50 0x55d21cc0b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.814+0000 7f357d807700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d21cc02cf0 0x55d21cc0bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51262/0 (socket says 172.21.15.52:51262) 2023-11-27T15:17:33.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.814+0000 7f357d807700 1 -- 172.21.15.52:0/2506204752 learned_addr learned my addr 172.21.15.52:0/2506204752 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:33.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.814+0000 7f357e809700 1 --2- 172.21.15.52:0/2506204752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d21cc036a0 0x55d21cc12da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.815+0000 7f357d807700 1 -- 172.21.15.52:0/2506204752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d21cc036a0 msgr2=0x55d21cc12da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:33.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.815+0000 7f357d807700 1 --2- 172.21.15.52:0/2506204752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d21cc036a0 0x55d21cc12da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.815+0000 7f357d807700 1 -- 172.21.15.52:0/2506204752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d21cc01e50 msgr2=0x55d21cc0b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:33.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.815+0000 7f357d807700 1 --2- 172.21.15.52:0/2506204752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d21cc01e50 0x55d21cc0b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.815+0000 7f357d807700 1 -- 172.21.15.52:0/2506204752 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3568010c50 con 0x55d21cc02cf0 2023-11-27T15:17:33.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.815+0000 7f357d807700 1 --2- 172.21.15.52:0/2506204752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d21cc02cf0 0x55d21cc0bd60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3568005d90 tx=0x7f3568011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb449b517d90557 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:33.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.815+0000 7f3561ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:17:33.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.815+0000 7f3561ffb700 1 -- 172.21.15.52:0/2506204752 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356802c070 con 0x55d21cc02cf0 2023-11-27T15:17:33.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.815+0000 7f3561ffb700 1 -- 172.21.15.52:0/2506204752 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3568017070 con 0x55d21cc02cf0 2023-11-27T15:17:33.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.815+0000 7f358c3f8b80 1 -- 172.21.15.52:0/2506204752 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d21cbf2ad0 con 0x55d21cc02cf0 2023-11-27T15:17:33.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.815+0000 7f3561ffb700 1 -- 172.21.15.52:0/2506204752 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f356801e630 con 0x55d21cc02cf0 2023-11-27T15:17:33.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.815+0000 7f358c3f8b80 10 client.15186 fetch_fsmap learned FSMap version 23 2023-11-27T15:17:33.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.815+0000 7f358c3f8b80 1 -- 172.21.15.52:0/2506204752 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d21cb17140 con 0x55d21cc02cf0 2023-11-27T15:17:33.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.815+0000 7f3561ffb700 1 -- 172.21.15.52:0/2506204752 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f356803f080 con 0x55d21cc02cf0 2023-11-27T15:17:33.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.815+0000 7f3561ffb700 1 -- 172.21.15.52:0/2506204752 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3568039030 con 0x55d21cc02cf0 2023-11-27T15:17:33.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.816+0000 7f358c3f8b80 10 client.15186 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:17:33.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.816+0000 7f358c3f8b80 10 client.15186 resolve_mds: resolved name 'h' to daemon mds.h 2023-11-27T15:17:33.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.816+0000 7f358c3f8b80 20 client.15186 populate_metadata read hostname 'smithi052' 2023-11-27T15:17:33.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.816+0000 7f358c3f8b80 1 --2- 172.21.15.52:0/2506204752 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55d21cc16930 0x55d21cc15de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.816+0000 7f358c3f8b80 4 client.15186 mds_command: new command op to 24494 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:17:33.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.816+0000 7f358c3f8b80 1 -- 172.21.15.52:0/2506204752 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d21cb13050 con 0x55d21cc16930 2023-11-27T15:17:33.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.816+0000 7f357e008700 1 --2- 172.21.15.52:0/2506204752 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55d21cc16930 0x55d21cc15de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.816+0000 7f3561ffb700 10 client.15186 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:17:33.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.816+0000 7f357e008700 1 --2- 172.21.15.52:0/2506204752 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55d21cc16930 0x55d21cc15de0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7165aee7cfe703a0 server_cookie=1ca0967bf36d52dc in_seq=0 out_seq=0 2023-11-27T15:17:33.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.817+0000 7f3561ffb700 1 -- 172.21.15.52:0/2506204752 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d21cb13050 con 0x55d21cc16930 2023-11-27T15:17:33.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.817+0000 7f3561ffb700 10 client.15186 handle_command_reply: tid=0 2023-11-27T15:17:33.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.817+0000 7f358c3f8b80 1 client.15186 shutdown 2023-11-27T15:17:33.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.817+0000 7f358c3f8b80 1 -- 172.21.15.52:0/2506204752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d21cc02cf0 msgr2=0x55d21cc0bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:33.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.817+0000 7f358c3f8b80 1 --2- 172.21.15.52:0/2506204752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d21cc02cf0 0x55d21cc0bd60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3568005d90 tx=0x7f3568011de0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.817+0000 7f358c3f8b80 1 -- 172.21.15.52:0/2506204752 shutdown_connections 2023-11-27T15:17:33.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.817+0000 7f358c3f8b80 1 --2- 172.21.15.52:0/2506204752 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55d21cc16930 0x55d21cc15de0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.817+0000 7f358c3f8b80 1 --2- 172.21.15.52:0/2506204752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d21cc01e50 0x55d21cc0b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.817+0000 7f358c3f8b80 1 --2- 172.21.15.52:0/2506204752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d21cc036a0 0x55d21cc12da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.817+0000 7f358c3f8b80 1 --2- 172.21.15.52:0/2506204752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d21cc02cf0 0x55d21cc0bd60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.817+0000 7f3561ffb700 0 client.15186 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:17:33.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.817+0000 7f358c3f8b80 1 -- 172.21.15.52:0/2506204752 >> 172.21.15.52:0/2506204752 conn(0x55d21cbf5310 msgr2=0x55d21cbf9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:33.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.817+0000 7f358c3f8b80 1 -- 172.21.15.52:0/2506204752 shutdown_connections 2023-11-27T15:17:33.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.817+0000 7f358c3f8b80 1 -- 172.21.15.52:0/2506204752 wait complete. 2023-11-27T15:17:33.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.818+0000 7f358c3f8b80 20 client.15186 trim_cache size 0 max 16384 2023-11-27T15:17:33.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.835+0000 7f358c3f8b80 1 Processor -- start 2023-11-27T15:17:33.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.835+0000 7f358c3f8b80 1 -- start start 2023-11-27T15:17:33.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.835+0000 7f358c3f8b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d21cc01e50 0x55d21cc15de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.835+0000 7f358c3f8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d21cc02cf0 0x55d21cc12cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.835+0000 7f358c3f8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d21cc036a0 0x55d21cc06650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.835+0000 7f358c3f8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d21cbfb270 con 0x55d21cc036a0 2023-11-27T15:17:33.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.835+0000 7f358c3f8b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d21cc0ca10 con 0x55d21cc01e50 2023-11-27T15:17:33.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.835+0000 7f358c3f8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d21ca5ced0 con 0x55d21cc02cf0 2023-11-27T15:17:33.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.835+0000 7f357e809700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d21cc036a0 0x55d21cc06650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.835+0000 7f357e809700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d21cc036a0 0x55d21cc06650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51272/0 (socket says 172.21.15.52:51272) 2023-11-27T15:17:33.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.835+0000 7f357e809700 1 -- 172.21.15.52:0/3983297366 learned_addr learned my addr 172.21.15.52:0/3983297366 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:33.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.836+0000 7f357e008700 1 --2- 172.21.15.52:0/3983297366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d21cc01e50 0x55d21cc15de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.836+0000 7f357e809700 1 -- 172.21.15.52:0/3983297366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d21cc02cf0 msgr2=0x55d21cc12cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:33.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.836+0000 7f357e809700 1 --2- 172.21.15.52:0/3983297366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d21cc02cf0 0x55d21cc12cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.836+0000 7f357e809700 1 -- 172.21.15.52:0/3983297366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d21cc01e50 msgr2=0x55d21cc15de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:33.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.836+0000 7f357e809700 1 --2- 172.21.15.52:0/3983297366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d21cc01e50 0x55d21cc15de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.836+0000 7f357e809700 1 -- 172.21.15.52:0/3983297366 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d21cb16e90 con 0x55d21cc036a0 2023-11-27T15:17:33.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.836+0000 7f357e809700 1 --2- 172.21.15.52:0/3983297366 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d21cc036a0 0x55d21cc06650 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3570009fd0 tx=0x7f3570007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d4745590f53089 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:33.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.836+0000 7f356d5ad700 1 -- 172.21.15.52:0/3983297366 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3570017410 con 0x55d21cc036a0 2023-11-27T15:17:33.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.836+0000 7f356d5ad700 1 -- 172.21.15.52:0/3983297366 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f357000f040 con 0x55d21cc036a0 2023-11-27T15:17:33.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.836+0000 7f356d5ad700 1 -- 172.21.15.52:0/3983297366 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3570016a00 con 0x55d21cc036a0 2023-11-27T15:17:33.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.836+0000 7f358c3f8b80 1 -- 172.21.15.52:0/3983297366 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d21cc036a0 msgr2=0x55d21cc06650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:33.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.836+0000 7f358c3f8b80 1 --2- 172.21.15.52:0/3983297366 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d21cc036a0 0x55d21cc06650 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3570009fd0 tx=0x7f3570007500 comp rx=0 tx=0).stop 2023-11-27T15:17:33.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.836+0000 7f358c3f8b80 1 -- 172.21.15.52:0/3983297366 shutdown_connections 2023-11-27T15:17:33.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.836+0000 7f358c3f8b80 1 --2- 172.21.15.52:0/3983297366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d21cc01e50 0x55d21cc15de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.836+0000 7f358c3f8b80 1 --2- 172.21.15.52:0/3983297366 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d21cc036a0 0x55d21cc06650 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.836+0000 7f358c3f8b80 1 --2- 172.21.15.52:0/3983297366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d21cc02cf0 0x55d21cc12cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.836+0000 7f358c3f8b80 1 -- 172.21.15.52:0/3983297366 >> 172.21.15.52:0/3983297366 conn(0x55d21cbf5310 msgr2=0x55d21cc0a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:33.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.837+0000 7f358c3f8b80 1 -- 172.21.15.52:0/3983297366 shutdown_connections 2023-11-27T15:17:33.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.837+0000 7f358c3f8b80 1 -- 172.21.15.52:0/3983297366 wait complete. 2023-11-27T15:17:33.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.837+0000 7f358c3f8b80 1 -- start start 2023-11-27T15:17:33.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.837+0000 7f358c3f8b80 1 Processor -- start 2023-11-27T15:17:33.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.837+0000 7f358c3f8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d21cc01e50 0x55d21cc0fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.837+0000 7f358c3f8b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d21cc02cf0 0x55d21cc10140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.837+0000 7f358c3f8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d21cc036a0 0x55d21cc6a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.837+0000 7f358c3f8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d21cc10870 con 0x55d21cc01e50 2023-11-27T15:17:33.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.837+0000 7f358c3f8b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d21cc6ad00 con 0x55d21cc02cf0 2023-11-27T15:17:33.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.837+0000 7f358c3f8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d21cc6aec0 con 0x55d21cc036a0 2023-11-27T15:17:33.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.837+0000 7f358c3f8b80 10 client.0 _ll_register_callbacks cb 0x7f3570009250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:17:33.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.837+0000 7f357e008700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d21cc01e50 0x55d21cc0fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.837+0000 7f357e008700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d21cc01e50 0x55d21cc0fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51274/0 (socket says 172.21.15.52:51274) 2023-11-27T15:17:33.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.837+0000 7f357e008700 1 -- 172.21.15.52:0/1315625586 learned_addr learned my addr 172.21.15.52:0/1315625586 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:33.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.837+0000 7f357d807700 1 --2- 172.21.15.52:0/1315625586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d21cc02cf0 0x55d21cc10140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.837+0000 7f357e809700 1 --2- 172.21.15.52:0/1315625586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d21cc036a0 0x55d21cc6a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.837+0000 7f357e008700 1 -- 172.21.15.52:0/1315625586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d21cc036a0 msgr2=0x55d21cc6a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:33.848 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.837+0000 7f357e008700 1 --2- 172.21.15.52:0/1315625586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d21cc036a0 0x55d21cc6a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.848 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.837+0000 7f357e008700 1 -- 172.21.15.52:0/1315625586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d21cc02cf0 msgr2=0x55d21cc10140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:33.848 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.837+0000 7f357e008700 1 --2- 172.21.15.52:0/1315625586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d21cc02cf0 0x55d21cc10140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.848 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.837+0000 7f357e008700 1 -- 172.21.15.52:0/1315625586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3570009c50 con 0x55d21cc01e50 2023-11-27T15:17:33.848 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.838+0000 7f357d807700 1 --2- 172.21.15.52:0/1315625586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d21cc02cf0 0x55d21cc10140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:17:33.848 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.838+0000 7f357e008700 1 --2- 172.21.15.52:0/1315625586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d21cc01e50 0x55d21cc0fc00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f357409aad0 tx=0x7f357409af30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c98575638c18c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:33.848 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.838+0000 7f3561ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:17:33.848 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.838+0000 7f3561ffb700 1 -- 172.21.15.52:0/1315625586 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35740a3190 con 0x55d21cc01e50 2023-11-27T15:17:33.848 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.838+0000 7f3561ffb700 1 -- 172.21.15.52:0/1315625586 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3574004e40 con 0x55d21cc01e50 2023-11-27T15:17:33.848 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.838+0000 7f358c3f8b80 1 -- 172.21.15.52:0/1315625586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3568015070 con 0x55d21cc01e50 2023-11-27T15:17:33.848 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.838+0000 7f3561ffb700 1 -- 172.21.15.52:0/1315625586 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f357409a2c0 con 0x55d21cc01e50 2023-11-27T15:17:33.848 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.838+0000 7f358c3f8b80 10 client.15192 fetch_fsmap learned FSMap version 23 2023-11-27T15:17:33.849 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.838+0000 7f358c3f8b80 1 -- 172.21.15.52:0/1315625586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d21c967c10 con 0x55d21cc01e50 2023-11-27T15:17:33.849 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.838+0000 7f3561ffb700 1 -- 172.21.15.52:0/1315625586 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3568015070 con 0x55d21cc01e50 2023-11-27T15:17:33.849 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.838+0000 7f3561ffb700 1 -- 172.21.15.52:0/1315625586 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f35740b3250 con 0x55d21cc01e50 2023-11-27T15:17:33.849 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.838+0000 7f358c3f8b80 10 client.15192 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:17:33.849 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.838+0000 7f358c3f8b80 10 client.15192 resolve_mds: resolved name 'h' to daemon mds.h 2023-11-27T15:17:33.849 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.838+0000 7f358c3f8b80 20 client.15192 populate_metadata read hostname 'smithi052' 2023-11-27T15:17:33.849 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.838+0000 7f358c3f8b80 1 --2- 172.21.15.52:0/1315625586 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55d21cc6f940 0x55d21cc6c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.849 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.838+0000 7f358c3f8b80 4 client.15192 mds_command: new command op to 24494 tid=0[{"prefix": "status"}] 2023-11-27T15:17:33.849 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.838+0000 7f358c3f8b80 1 -- 172.21.15.52:0/1315625586 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d21cb1f610 con 0x55d21cc6f940 2023-11-27T15:17:33.849 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.839+0000 7f357d807700 1 --2- 172.21.15.52:0/1315625586 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55d21cc6f940 0x55d21cc6c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.849 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.839+0000 7f357d807700 1 --2- 172.21.15.52:0/1315625586 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55d21cc6f940 0x55d21cc6c2a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2191de4f58ed5591 server_cookie=c1dc3a9f3baba64d in_seq=0 out_seq=0 2023-11-27T15:17:33.849 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.839+0000 7f3561ffb700 10 client.15192 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:17:33.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.839+0000 7f3561ffb700 1 -- 172.21.15.52:0/1315625586 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55d21cb1f610 con 0x55d21cc6f940 2023-11-27T15:17:33.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.839+0000 7f3561ffb700 10 client.15192 handle_command_reply: tid=0 2023-11-27T15:17:33.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.839+0000 7f358c3f8b80 1 client.15192 shutdown 2023-11-27T15:17:33.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.839+0000 7f358c3f8b80 1 -- 172.21.15.52:0/1315625586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d21cc01e50 msgr2=0x55d21cc0fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:33.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.839+0000 7f358c3f8b80 1 --2- 172.21.15.52:0/1315625586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d21cc01e50 0x55d21cc0fc00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f357409aad0 tx=0x7f357409af30 comp rx=0 tx=0).stop 2023-11-27T15:17:33.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.839+0000 7f358c3f8b80 1 -- 172.21.15.52:0/1315625586 shutdown_connections 2023-11-27T15:17:33.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.839+0000 7f358c3f8b80 1 --2- 172.21.15.52:0/1315625586 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55d21cc6f940 0x55d21cc6c2a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.839+0000 7f358c3f8b80 1 --2- 172.21.15.52:0/1315625586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d21cc036a0 0x55d21cc6a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.839+0000 7f3561ffb700 0 client.15192 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:17:33.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.839+0000 7f358c3f8b80 1 --2- 172.21.15.52:0/1315625586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d21cc01e50 0x55d21cc0fc00 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.839+0000 7f358c3f8b80 1 --2- 172.21.15.52:0/1315625586 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d21cc02cf0 0x55d21cc10140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.851 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.839+0000 7f358c3f8b80 1 -- 172.21.15.52:0/1315625586 >> 172.21.15.52:0/1315625586 conn(0x55d21cbf5310 msgr2=0x55d21cc15de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:33.851 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.839+0000 7f358c3f8b80 1 -- 172.21.15.52:0/1315625586 shutdown_connections 2023-11-27T15:17:33.851 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.839+0000 7f358c3f8b80 1 -- 172.21.15.52:0/1315625586 wait complete. 2023-11-27T15:17:33.851 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.839+0000 7f358c3f8b80 20 client.15192 trim_cache size 0 max 16384 2023-11-27T15:17:33.851 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:17:33.851 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "c6c7df88-8d36-11ee-95a2-87774f69a715", 2023-11-27T15:17:33.851 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 1, 2023-11-27T15:17:33.851 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24494, 2023-11-27T15:17:33.851 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2023-11-27T15:17:33.851 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2023-11-27T15:17:33.852 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2023-11-27T15:17:33.852 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 81.105348898000003, 2023-11-27T15:17:33.852 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 23, 2023-11-27T15:17:33.852 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 85, 2023-11-27T15:17:33.852 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 83, 2023-11-27T15:17:33.852 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 98.807406055000001 2023-11-27T15:17:33.852 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:17:33.853 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.840+0000 7f358026c700 1 -- 172.21.15.52:0/3397448605 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f35640a11e0 msgr2=0x7f35640a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:33.853 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.840+0000 7f358026c700 1 --2- 172.21.15.52:0/3397448605 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f35640a11e0 0x7f35640a3690 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3568005950 tx=0x7f356800b410 comp rx=0 tx=0).stop 2023-11-27T15:17:33.853 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.840+0000 7f358026c700 1 -- 172.21.15.52:0/3397448605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35780ff160 msgr2=0x7f3578194e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:33.853 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.840+0000 7f358026c700 1 --2- 172.21.15.52:0/3397448605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35780ff160 0x7f3578194e90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3574004760 tx=0x7f3574009580 comp rx=0 tx=0).stop 2023-11-27T15:17:33.853 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.840+0000 7f358026c700 1 -- 172.21.15.52:0/3397448605 shutdown_connections 2023-11-27T15:17:33.853 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.840+0000 7f358026c700 1 --2- 172.21.15.52:0/3397448605 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f35640a11e0 0x7f35640a3690 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.853 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.840+0000 7f358026c700 1 --2- 172.21.15.52:0/3397448605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3578100ce0 0x7f357819c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.853 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.840+0000 7f358026c700 1 --2- 172.21.15.52:0/3397448605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35780ff160 0x7f3578194e90 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.853 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.840+0000 7f358026c700 1 --2- 172.21.15.52:0/3397448605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3578100350 0x7f35781953d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.854 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.840+0000 7f358026c700 1 -- 172.21.15.52:0/3397448605 >> 172.21.15.52:0/3397448605 conn(0x7f35780fa740 msgr2=0x7f35780fc930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:33.854 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.840+0000 7f358026c700 1 -- 172.21.15.52:0/3397448605 shutdown_connections 2023-11-27T15:17:33.854 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.840+0000 7f358026c700 1 -- 172.21.15.52:0/3397448605 wait complete. 2023-11-27T15:17:33.856 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-27T15:17:33.985 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.979+0000 7efee2ea9700 1 -- 172.21.15.52:0/1729504937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efedc103620 msgr2=0x7efedc105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:33.985 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.979+0000 7efee2ea9700 1 --2- 172.21.15.52:0/1729504937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efedc103620 0x7efedc105a00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7efec8009990 tx=0x7efec8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.985 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.979+0000 7efee2ea9700 1 -- 172.21.15.52:0/1729504937 shutdown_connections 2023-11-27T15:17:33.985 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.979+0000 7efee2ea9700 1 --2- 172.21.15.52:0/1729504937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efedc100d00 0x7efedc1030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.979+0000 7efee2ea9700 1 --2- 172.21.15.52:0/1729504937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efedc105f40 0x7efedc108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.979+0000 7efee2ea9700 1 --2- 172.21.15.52:0/1729504937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efedc103620 0x7efedc105a00 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.979+0000 7efee2ea9700 1 -- 172.21.15.52:0/1729504937 >> 172.21.15.52:0/1729504937 conn(0x7efedc0fa700 msgr2=0x7efedc0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:33.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.979+0000 7efee2ea9700 1 -- 172.21.15.52:0/1729504937 shutdown_connections 2023-11-27T15:17:33.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.979+0000 7efee2ea9700 1 -- 172.21.15.52:0/1729504937 wait complete. 2023-11-27T15:17:33.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.979+0000 7efee2ea9700 1 Processor -- start 2023-11-27T15:17:33.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.980+0000 7efee2ea9700 1 -- start start 2023-11-27T15:17:33.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.980+0000 7efee2ea9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efedc100d00 0x7efedc100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.980+0000 7efee2ea9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efedc103620 0x7efedc0fecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.980+0000 7efee2ea9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efedc105f40 0x7efedc0ff1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.980+0000 7efee2ea9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efedc0ff990 con 0x7efedc105f40 2023-11-27T15:17:33.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.980+0000 7efee2ea9700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efedc0ffb00 con 0x7efedc100d00 2023-11-27T15:17:33.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.980+0000 7efee2ea9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efedc19ff80 con 0x7efedc103620 2023-11-27T15:17:33.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.980+0000 7efee1446700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efedc105f40 0x7efedc0ff1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.980+0000 7efee1446700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efedc105f40 0x7efedc0ff1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51292/0 (socket says 172.21.15.52:51292) 2023-11-27T15:17:33.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.980+0000 7efee1446700 1 -- 172.21.15.52:0/1917637784 learned_addr learned my addr 172.21.15.52:0/1917637784 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:33.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.980+0000 7efee0c45700 1 --2- 172.21.15.52:0/1917637784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efedc100d00 0x7efedc100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.980+0000 7efedbfff700 1 --2- 172.21.15.52:0/1917637784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efedc103620 0x7efedc0fecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.980+0000 7efee1446700 1 -- 172.21.15.52:0/1917637784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efedc103620 msgr2=0x7efedc0fecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:33.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.980+0000 7efee1446700 1 --2- 172.21.15.52:0/1917637784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efedc103620 0x7efedc0fecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.980+0000 7efee1446700 1 -- 172.21.15.52:0/1917637784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efedc100d00 msgr2=0x7efedc100670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:33.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.980+0000 7efee1446700 1 --2- 172.21.15.52:0/1917637784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efedc100d00 0x7efedc100670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.980+0000 7efee1446700 1 -- 172.21.15.52:0/1917637784 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec80095f0 con 0x7efedc105f40 2023-11-27T15:17:33.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.980+0000 7efedbfff700 1 --2- 172.21.15.52:0/1917637784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efedc103620 0x7efedc0fecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:17:33.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.980+0000 7efee1446700 1 --2- 172.21.15.52:0/1917637784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efedc105f40 0x7efedc0ff1f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7efed4007ee0 tx=0x7efed400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:33.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.980+0000 7efed9ffb700 1 -- 172.21.15.52:0/1917637784 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed4017070 con 0x7efedc105f40 2023-11-27T15:17:33.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.980+0000 7efed9ffb700 1 -- 172.21.15.52:0/1917637784 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efed400f040 con 0x7efedc105f40 2023-11-27T15:17:33.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.980+0000 7efee2ea9700 1 -- 172.21.15.52:0/1917637784 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efedc10bb80 con 0x7efedc105f40 2023-11-27T15:17:33.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.980+0000 7efed9ffb700 1 -- 172.21.15.52:0/1917637784 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed4005740 con 0x7efedc105f40 2023-11-27T15:17:33.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.981+0000 7efee2ea9700 1 -- 172.21.15.52:0/1917637784 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efedc10b9f0 con 0x7efedc105f40 2023-11-27T15:17:33.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.983+0000 7efed9ffb700 1 -- 172.21.15.52:0/1917637784 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7efed400dc60 con 0x7efedc105f40 2023-11-27T15:17:33.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.983+0000 7efed9ffb700 1 --2- 172.21.15.52:0/1917637784 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7efecc0a11e0 0x7efecc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.983+0000 7efee0c45700 1 --2- 172.21.15.52:0/1917637784 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7efecc0a11e0 0x7efecc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.983+0000 7efed9ffb700 1 -- 172.21.15.52:0/1917637784 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7efed40cc300 con 0x7efedc105f40 2023-11-27T15:17:33.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.984+0000 7efee0c45700 1 --2- 172.21.15.52:0/1917637784 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7efecc0a11e0 0x7efecc0a3690 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efed0009c60 tx=0x7efed0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:33.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.984+0000 7efeef035b80 1 Processor -- start 2023-11-27T15:17:33.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.984+0000 7efeef035b80 1 -- start start 2023-11-27T15:17:33.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.984+0000 7efeef035b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c35da55bb0 0x55c35da55fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.984+0000 7efeef035b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c35da56500 0x55c35da56ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.984+0000 7efeef035b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c35da570c0 0x55c35da57530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.984+0000 7efeef035b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35d95b010 con 0x55c35da56500 2023-11-27T15:17:33.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.984+0000 7efeef035b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35d9ad850 con 0x55c35da55bb0 2023-11-27T15:17:33.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.984+0000 7efeef035b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35d9a6ce0 con 0x55c35da570c0 2023-11-27T15:17:33.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.984+0000 7efedbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c35da56500 0x55c35da56ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.984+0000 7efedbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c35da56500 0x55c35da56ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51298/0 (socket says 172.21.15.52:51298) 2023-11-27T15:17:33.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.984+0000 7efedbfff700 1 -- 172.21.15.52:0/945302657 learned_addr learned my addr 172.21.15.52:0/945302657 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:33.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.985+0000 7efee0c45700 1 --2- 172.21.15.52:0/945302657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c35da55bb0 0x55c35da55fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.985+0000 7efee1446700 1 --2- 172.21.15.52:0/945302657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c35da570c0 0x55c35da57530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.985+0000 7efedbfff700 1 -- 172.21.15.52:0/945302657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c35da570c0 msgr2=0x55c35da57530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:33.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.985+0000 7efedbfff700 1 --2- 172.21.15.52:0/945302657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c35da570c0 0x55c35da57530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.985+0000 7efedbfff700 1 -- 172.21.15.52:0/945302657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c35da55bb0 msgr2=0x55c35da55fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:33.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.985+0000 7efedbfff700 1 --2- 172.21.15.52:0/945302657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c35da55bb0 0x55c35da55fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.985+0000 7efedbfff700 1 -- 172.21.15.52:0/945302657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c35d8285e0 con 0x55c35da56500 2023-11-27T15:17:33.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.985+0000 7efedbfff700 1 --2- 172.21.15.52:0/945302657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c35da56500 0x55c35da56ae0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efec8015000 tx=0x7efec8005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2eb96c02663a9b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:33.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.985+0000 7efec6ffd700 1 -- 172.21.15.52:0/945302657 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec800bee0 con 0x55c35da56500 2023-11-27T15:17:33.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.985+0000 7efec6ffd700 1 -- 172.21.15.52:0/945302657 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efec801e040 con 0x55c35da56500 2023-11-27T15:17:33.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.985+0000 7efec6ffd700 1 -- 172.21.15.52:0/945302657 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec8004d20 con 0x55c35da56500 2023-11-27T15:17:33.997 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.986+0000 7efeef035b80 1 -- 172.21.15.52:0/945302657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c35da56500 msgr2=0x55c35da56ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:33.997 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.986+0000 7efeef035b80 1 --2- 172.21.15.52:0/945302657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c35da56500 0x55c35da56ae0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efec8015000 tx=0x7efec8005c00 comp rx=0 tx=0).stop 2023-11-27T15:17:33.997 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.986+0000 7efeef035b80 1 -- 172.21.15.52:0/945302657 shutdown_connections 2023-11-27T15:17:33.997 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.986+0000 7efeef035b80 1 --2- 172.21.15.52:0/945302657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c35da55bb0 0x55c35da55fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.997 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.986+0000 7efeef035b80 1 --2- 172.21.15.52:0/945302657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c35da570c0 0x55c35da57530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.997 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.986+0000 7efeef035b80 1 --2- 172.21.15.52:0/945302657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c35da56500 0x55c35da56ae0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.997 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.986+0000 7efeef035b80 1 -- 172.21.15.52:0/945302657 >> 172.21.15.52:0/945302657 conn(0x55c35da4b310 msgr2=0x55c35da50620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:33.997 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.986+0000 7efeef035b80 1 -- 172.21.15.52:0/945302657 shutdown_connections 2023-11-27T15:17:33.997 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.986+0000 7efeef035b80 1 -- 172.21.15.52:0/945302657 wait complete. 2023-11-27T15:17:33.997 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.986+0000 7efeef035b80 1 -- start start 2023-11-27T15:17:33.997 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.986+0000 7efeef035b80 1 Processor -- start 2023-11-27T15:17:33.997 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.987+0000 7efeef035b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c35da55bb0 0x55c35da5f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.998 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.987+0000 7efeef035b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c35da56500 0x55c35da5fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.998 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.987+0000 7efeef035b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c35da570c0 0x55c35da66ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:33.998 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.987+0000 7efeef035b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35d948890 con 0x55c35da55bb0 2023-11-27T15:17:33.998 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.987+0000 7efeef035b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35d948530 con 0x55c35da570c0 2023-11-27T15:17:33.998 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.987+0000 7efeef035b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35d82a8d0 con 0x55c35da56500 2023-11-27T15:17:33.998 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.987+0000 7efeef035b80 10 client.0 _ll_register_callbacks cb 0x55c35d966730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:17:33.998 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.987+0000 7efee0c45700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c35da55bb0 0x55c35da5f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.998 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.987+0000 7efee0c45700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c35da55bb0 0x55c35da5f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51302/0 (socket says 172.21.15.52:51302) 2023-11-27T15:17:33.998 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.987+0000 7efee0c45700 1 -- 172.21.15.52:0/1351909784 learned_addr learned my addr 172.21.15.52:0/1351909784 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:33.998 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.987+0000 7efee1446700 1 --2- 172.21.15.52:0/1351909784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c35da570c0 0x55c35da66ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.998 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.987+0000 7efedbfff700 1 --2- 172.21.15.52:0/1351909784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c35da56500 0x55c35da5fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:33.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.987+0000 7efee0c45700 1 -- 172.21.15.52:0/1351909784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c35da56500 msgr2=0x55c35da5fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:33.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.987+0000 7efee0c45700 1 --2- 172.21.15.52:0/1351909784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c35da56500 0x55c35da5fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.987+0000 7efee0c45700 1 -- 172.21.15.52:0/1351909784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c35da570c0 msgr2=0x55c35da66ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:33.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.987+0000 7efee0c45700 1 --2- 172.21.15.52:0/1351909784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c35da570c0 0x55c35da66ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:33.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.987+0000 7efee0c45700 1 -- 172.21.15.52:0/1351909784 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efedc108f30 con 0x55c35da55bb0 2023-11-27T15:17:33.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.987+0000 7efee0c45700 1 --2- 172.21.15.52:0/1351909784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c35da55bb0 0x55c35da5f530 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7efed001ebc0 tx=0x7efed00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3877c99b1ecb96ac server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:33.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.987+0000 7efebbfff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:17:33.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.987+0000 7efebbfff700 1 -- 172.21.15.52:0/1351909784 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed000ebd0 con 0x55c35da55bb0 2023-11-27T15:17:33.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.988+0000 7efebbfff700 1 -- 172.21.15.52:0/1351909784 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7efed000ed30 con 0x55c35da55bb0 2023-11-27T15:17:33.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.988+0000 7efeef035b80 1 -- 172.21.15.52:0/1351909784 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c35da48ad0 con 0x55c35da55bb0 2023-11-27T15:17:33.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.988+0000 7efebbfff700 1 -- 172.21.15.52:0/1351909784 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7efed00283f0 con 0x55c35da55bb0 2023-11-27T15:17:33.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.988+0000 7efeef035b80 10 client.15204 fetch_fsmap learned FSMap version 23 2023-11-27T15:17:33.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.988+0000 7efeef035b80 1 -- 172.21.15.52:0/1351909784 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c35d96d140 con 0x55c35da55bb0 2023-11-27T15:17:33.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.988+0000 7efebbfff700 1 -- 172.21.15.52:0/1351909784 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c35da48ad0 con 0x55c35da55bb0 2023-11-27T15:17:34.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.988+0000 7efebbfff700 1 -- 172.21.15.52:0/1351909784 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efed001c5d0 con 0x55c35da55bb0 2023-11-27T15:17:34.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.988+0000 7efeef035b80 10 client.15204 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:17:34.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.988+0000 7efeef035b80 10 client.15204 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-27T15:17:34.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.988+0000 7efeef035b80 20 client.15204 populate_metadata read hostname 'smithi052' 2023-11-27T15:17:34.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.988+0000 7efeef035b80 1 --2- 172.21.15.52:0/1351909784 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c35da6a8b0 0x55c35da69c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:34.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.988+0000 7efeef035b80 4 client.15204 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:17:34.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.988+0000 7efeef035b80 1 -- 172.21.15.52:0/1351909784 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c35d969050 con 0x55c35da6a8b0 2023-11-27T15:17:34.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.988+0000 7efedbfff700 1 --2- 172.21.15.52:0/1351909784 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c35da6a8b0 0x55c35da69c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:34.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.989+0000 7efebbfff700 10 client.15204 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:17:34.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.989+0000 7efedbfff700 1 --2- 172.21.15.52:0/1351909784 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c35da6a8b0 0x55c35da69c60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82606b5886cfa25c server_cookie=3ce8702a8c972f35 in_seq=0 out_seq=0 2023-11-27T15:17:34.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.989+0000 7efebbfff700 1 -- 172.21.15.52:0/1351909784 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c35d969050 con 0x55c35da6a8b0 2023-11-27T15:17:34.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.989+0000 7efebbfff700 10 client.15204 handle_command_reply: tid=0 2023-11-27T15:17:34.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.989+0000 7efeef035b80 1 client.15204 shutdown 2023-11-27T15:17:34.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.989+0000 7efeef035b80 1 -- 172.21.15.52:0/1351909784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c35da55bb0 msgr2=0x55c35da5f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:34.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.989+0000 7efeef035b80 1 --2- 172.21.15.52:0/1351909784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c35da55bb0 0x55c35da5f530 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7efed001ebc0 tx=0x7efed00047c0 comp rx=0 tx=0).stop 2023-11-27T15:17:34.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.989+0000 7efeef035b80 1 -- 172.21.15.52:0/1351909784 shutdown_connections 2023-11-27T15:17:34.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.989+0000 7efeef035b80 1 --2- 172.21.15.52:0/1351909784 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c35da6a8b0 0x55c35da69c60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:34.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.989+0000 7efeef035b80 1 --2- 172.21.15.52:0/1351909784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c35da570c0 0x55c35da66ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:34.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.989+0000 7efebbfff700 0 client.15204 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:17:34.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.989+0000 7efeef035b80 1 --2- 172.21.15.52:0/1351909784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c35da55bb0 0x55c35da5f530 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:34.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.989+0000 7efeef035b80 1 --2- 172.21.15.52:0/1351909784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c35da56500 0x55c35da5fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:34.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.989+0000 7efeef035b80 1 -- 172.21.15.52:0/1351909784 >> 172.21.15.52:0/1351909784 conn(0x55c35da4b310 msgr2=0x55c35da4fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:34.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.990+0000 7efeef035b80 1 -- 172.21.15.52:0/1351909784 shutdown_connections 2023-11-27T15:17:34.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.990+0000 7efeef035b80 1 -- 172.21.15.52:0/1351909784 wait complete. 2023-11-27T15:17:34.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:33.990+0000 7efeef035b80 20 client.15204 trim_cache size 0 max 16384 2023-11-27T15:17:34.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.008+0000 7efeef035b80 1 Processor -- start 2023-11-27T15:17:34.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.008+0000 7efeef035b80 1 -- start start 2023-11-27T15:17:34.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.008+0000 7efeef035b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c35da55bb0 0x55c35da5d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:34.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.008+0000 7efeef035b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c35da56500 0x55c35da5d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:34.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.008+0000 7efeef035b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c35da570c0 0x55c35da66a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:34.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.008+0000 7efeef035b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35da579b0 con 0x55c35da570c0 2023-11-27T15:17:34.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.008+0000 7efeef035b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35d942e10 con 0x55c35da56500 2023-11-27T15:17:34.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.008+0000 7efeef035b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35d82ca30 con 0x55c35da55bb0 2023-11-27T15:17:34.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.008+0000 7efee1446700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c35da570c0 0x55c35da66a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:34.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.008+0000 7efee1446700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c35da570c0 0x55c35da66a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51314/0 (socket says 172.21.15.52:51314) 2023-11-27T15:17:34.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.008+0000 7efee1446700 1 -- 172.21.15.52:0/204158246 learned_addr learned my addr 172.21.15.52:0/204158246 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:34.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.008+0000 7efee0c45700 1 --2- 172.21.15.52:0/204158246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c35da55bb0 0x55c35da5d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:34.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.008+0000 7efedbfff700 1 --2- 172.21.15.52:0/204158246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c35da56500 0x55c35da5d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:34.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.008+0000 7efee1446700 1 -- 172.21.15.52:0/204158246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c35da55bb0 msgr2=0x55c35da5d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:34.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.008+0000 7efee1446700 1 --2- 172.21.15.52:0/204158246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c35da55bb0 0x55c35da5d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:34.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.008+0000 7efee1446700 1 -- 172.21.15.52:0/204158246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c35da56500 msgr2=0x55c35da5d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:34.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.008+0000 7efee1446700 1 --2- 172.21.15.52:0/204158246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c35da56500 0x55c35da5d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:34.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.008+0000 7efee1446700 1 -- 172.21.15.52:0/204158246 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c35d96ce90 con 0x55c35da570c0 2023-11-27T15:17:34.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.008+0000 7efedbfff700 1 --2- 172.21.15.52:0/204158246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c35da56500 0x55c35da5d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:17:34.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.008+0000 7efee1446700 1 --2- 172.21.15.52:0/204158246 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c35da570c0 0x55c35da66a70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7efed4096000 tx=0x7efed4084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286a849330a2383f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:34.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.009+0000 7efec77fe700 1 -- 172.21.15.52:0/204158246 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed4099590 con 0x55c35da570c0 2023-11-27T15:17:34.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.009+0000 7efec77fe700 1 -- 172.21.15.52:0/204158246 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efed409d5c0 con 0x55c35da570c0 2023-11-27T15:17:34.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.009+0000 7efec77fe700 1 -- 172.21.15.52:0/204158246 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed40058a0 con 0x55c35da570c0 2023-11-27T15:17:34.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.009+0000 7efeef035b80 1 -- 172.21.15.52:0/204158246 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c35da570c0 msgr2=0x55c35da66a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:34.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.009+0000 7efeef035b80 1 --2- 172.21.15.52:0/204158246 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c35da570c0 0x55c35da66a70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7efed4096000 tx=0x7efed4084040 comp rx=0 tx=0).stop 2023-11-27T15:17:34.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.009+0000 7efeef035b80 1 -- 172.21.15.52:0/204158246 shutdown_connections 2023-11-27T15:17:34.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.009+0000 7efeef035b80 1 --2- 172.21.15.52:0/204158246 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c35da570c0 0x55c35da66a70 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:34.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.009+0000 7efeef035b80 1 --2- 172.21.15.52:0/204158246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c35da55bb0 0x55c35da5d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:34.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.009+0000 7efeef035b80 1 --2- 172.21.15.52:0/204158246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c35da56500 0x55c35da5d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:34.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.009+0000 7efeef035b80 1 -- 172.21.15.52:0/204158246 >> 172.21.15.52:0/204158246 conn(0x55c35da4b310 msgr2=0x55c35da73640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:34.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.009+0000 7efeef035b80 1 -- 172.21.15.52:0/204158246 shutdown_connections 2023-11-27T15:17:34.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.009+0000 7efeef035b80 1 -- 172.21.15.52:0/204158246 wait complete. 2023-11-27T15:17:34.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.010+0000 7efeef035b80 1 -- start start 2023-11-27T15:17:34.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.010+0000 7efeef035b80 1 Processor -- start 2023-11-27T15:17:34.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.010+0000 7efeef035b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c35da55bb0 0x55c35da61520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:34.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.010+0000 7efeef035b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c35da56500 0x55c35da61a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:34.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.010+0000 7efeef035b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c35da570c0 0x55c35dabe770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:34.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.010+0000 7efeef035b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35da66a90 con 0x55c35da55bb0 2023-11-27T15:17:34.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.010+0000 7efeef035b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35da66c00 con 0x55c35da570c0 2023-11-27T15:17:34.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.010+0000 7efeef035b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35da66dc0 con 0x55c35da56500 2023-11-27T15:17:34.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.010+0000 7efeef035b80 10 client.0 _ll_register_callbacks cb 0x7efed001e7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:17:34.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.010+0000 7efee0c45700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c35da55bb0 0x55c35da61520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:34.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.010+0000 7efee0c45700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c35da55bb0 0x55c35da61520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51316/0 (socket says 172.21.15.52:51316) 2023-11-27T15:17:34.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.010+0000 7efee0c45700 1 -- 172.21.15.52:0/3107741600 learned_addr learned my addr 172.21.15.52:0/3107741600 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:34.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.010+0000 7efee1446700 1 --2- 172.21.15.52:0/3107741600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c35da570c0 0x55c35dabe770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:34.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.010+0000 7efedbfff700 1 --2- 172.21.15.52:0/3107741600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c35da56500 0x55c35da61a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:34.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.010+0000 7efee1446700 1 -- 172.21.15.52:0/3107741600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c35da56500 msgr2=0x55c35da61a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:34.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.010+0000 7efee1446700 1 --2- 172.21.15.52:0/3107741600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c35da56500 0x55c35da61a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:34.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.010+0000 7efee1446700 1 -- 172.21.15.52:0/3107741600 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c35da55bb0 msgr2=0x55c35da61520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:34.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.010+0000 7efee1446700 1 --2- 172.21.15.52:0/3107741600 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c35da55bb0 0x55c35da61520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:34.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.010+0000 7efee1446700 1 -- 172.21.15.52:0/3107741600 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c35d8285e0 con 0x55c35da570c0 2023-11-27T15:17:34.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.011+0000 7efee0c45700 1 --2- 172.21.15.52:0/3107741600 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c35da55bb0 0x55c35da61520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:17:34.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.011+0000 7efee1446700 1 --2- 172.21.15.52:0/3107741600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c35da570c0 0x55c35dabe770 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efed40c9d10 tx=0x7efed400dbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc6f12ac0ead1df7 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:34.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.011+0000 7efebbfff700 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2023-11-27T15:17:34.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.011+0000 7efebbfff700 1 -- 172.21.15.52:0/3107741600 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed4005aa0 con 0x55c35da570c0 2023-11-27T15:17:34.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.011+0000 7efebbfff700 1 -- 172.21.15.52:0/3107741600 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7efed40a19e0 con 0x55c35da570c0 2023-11-27T15:17:34.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.011+0000 7efeef035b80 1 -- 172.21.15.52:0/3107741600 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efec801e040 con 0x55c35da570c0 2023-11-27T15:17:34.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.011+0000 7efebbfff700 1 -- 172.21.15.52:0/3107741600 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7efed4084290 con 0x55c35da570c0 2023-11-27T15:17:34.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.011+0000 7efeef035b80 10 client.25055 fetch_fsmap learned FSMap version 23 2023-11-27T15:17:34.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.011+0000 7efeef035b80 1 -- 172.21.15.52:0/3107741600 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7efedc10b9f0 con 0x55c35da570c0 2023-11-27T15:17:34.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.011+0000 7efebbfff700 1 -- 172.21.15.52:0/3107741600 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efec801e040 con 0x55c35da570c0 2023-11-27T15:17:34.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.011+0000 7efebbfff700 1 -- 172.21.15.52:0/3107741600 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efed4098a80 con 0x55c35da570c0 2023-11-27T15:17:34.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.011+0000 7efeef035b80 10 client.25055 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:17:34.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.011+0000 7efeef035b80 10 client.25055 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-27T15:17:34.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.011+0000 7efeef035b80 20 client.25055 populate_metadata read hostname 'smithi052' 2023-11-27T15:17:34.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.011+0000 7efeef035b80 1 --2- 172.21.15.52:0/3107741600 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c35dac3470 0x55c35dabff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:34.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.011+0000 7efeef035b80 4 client.25055 mds_command: new command op to 24536 tid=0[{"prefix": "status"}] 2023-11-27T15:17:34.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.011+0000 7efeef035b80 1 -- 172.21.15.52:0/3107741600 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c35d975610 con 0x55c35dac3470 2023-11-27T15:17:34.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.012+0000 7efee0c45700 1 --2- 172.21.15.52:0/3107741600 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c35dac3470 0x55c35dabff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:34.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.012+0000 7efebbfff700 10 client.25055 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:17:34.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.012+0000 7efee0c45700 1 --2- 172.21.15.52:0/3107741600 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c35dac3470 0x55c35dabff90 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e46c32a76549116 server_cookie=69de5ff9e566611b in_seq=0 out_seq=0 2023-11-27T15:17:34.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.012+0000 7efebbfff700 1 -- 172.21.15.52:0/3107741600 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55c35d975610 con 0x55c35dac3470 2023-11-27T15:17:34.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.012+0000 7efebbfff700 10 client.25055 handle_command_reply: tid=0 2023-11-27T15:17:34.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.012+0000 7efeef035b80 1 client.25055 shutdown 2023-11-27T15:17:34.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.012+0000 7efeef035b80 1 -- 172.21.15.52:0/3107741600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c35da570c0 msgr2=0x55c35dabe770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:34.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.012+0000 7efeef035b80 1 --2- 172.21.15.52:0/3107741600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c35da570c0 0x55c35dabe770 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efed40c9d10 tx=0x7efed400dbd0 comp rx=0 tx=0).stop 2023-11-27T15:17:34.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.012+0000 7efeef035b80 1 -- 172.21.15.52:0/3107741600 shutdown_connections 2023-11-27T15:17:34.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.012+0000 7efeef035b80 1 --2- 172.21.15.52:0/3107741600 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c35dac3470 0x55c35dabff90 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:34.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.012+0000 7efeef035b80 1 --2- 172.21.15.52:0/3107741600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c35da570c0 0x55c35dabe770 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:34.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.012+0000 7efeef035b80 1 --2- 172.21.15.52:0/3107741600 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c35da55bb0 0x55c35da61520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:34.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.012+0000 7efeef035b80 1 --2- 172.21.15.52:0/3107741600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c35da56500 0x55c35da61a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:34.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.012+0000 7efebbfff700 0 client.25055 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:17:34.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.012+0000 7efeef035b80 1 -- 172.21.15.52:0/3107741600 >> 172.21.15.52:0/3107741600 conn(0x55c35da4b310 msgr2=0x55c35da69c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:34.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.013+0000 7efeef035b80 1 -- 172.21.15.52:0/3107741600 shutdown_connections 2023-11-27T15:17:34.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.013+0000 7efeef035b80 1 -- 172.21.15.52:0/3107741600 wait complete. 2023-11-27T15:17:34.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.013+0000 7efeef035b80 20 client.25055 trim_cache size 0 max 16384 2023-11-27T15:17:34.024 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:17:34.025 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "c6c7df88-8d36-11ee-95a2-87774f69a715", 2023-11-27T15:17:34.025 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 0, 2023-11-27T15:17:34.025 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24536, 2023-11-27T15:17:34.025 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2023-11-27T15:17:34.025 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2023-11-27T15:17:34.025 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2023-11-27T15:17:34.025 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 84.690500048000004, 2023-11-27T15:17:34.025 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 23, 2023-11-27T15:17:34.025 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 85, 2023-11-27T15:17:34.025 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 83, 2023-11-27T15:17:34.025 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 89.373058717999996 2023-11-27T15:17:34.025 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:17:34.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.013+0000 7efee2ea9700 1 -- 172.21.15.52:0/1917637784 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7efecc0a11e0 msgr2=0x7efecc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:34.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.013+0000 7efee2ea9700 1 --2- 172.21.15.52:0/1917637784 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7efecc0a11e0 0x7efecc0a3690 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efed0009c60 tx=0x7efed0009260 comp rx=0 tx=0).stop 2023-11-27T15:17:34.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.013+0000 7efee2ea9700 1 -- 172.21.15.52:0/1917637784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efedc105f40 msgr2=0x7efedc0ff1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:34.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.013+0000 7efee2ea9700 1 --2- 172.21.15.52:0/1917637784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efedc105f40 0x7efedc0ff1f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7efed4007ee0 tx=0x7efed400d3b0 comp rx=0 tx=0).stop 2023-11-27T15:17:34.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.013+0000 7efee2ea9700 1 -- 172.21.15.52:0/1917637784 shutdown_connections 2023-11-27T15:17:34.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.013+0000 7efee2ea9700 1 --2- 172.21.15.52:0/1917637784 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7efecc0a11e0 0x7efecc0a3690 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:34.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.013+0000 7efee2ea9700 1 --2- 172.21.15.52:0/1917637784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efedc100d00 0x7efedc100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:34.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.013+0000 7efee2ea9700 1 --2- 172.21.15.52:0/1917637784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efedc105f40 0x7efedc0ff1f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:34.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.013+0000 7efee2ea9700 1 --2- 172.21.15.52:0/1917637784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efedc103620 0x7efedc0fecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:34.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.013+0000 7efee2ea9700 1 -- 172.21.15.52:0/1917637784 >> 172.21.15.52:0/1917637784 conn(0x7efedc0fa700 msgr2=0x7efedc102190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:34.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.013+0000 7efee2ea9700 1 -- 172.21.15.52:0/1917637784 shutdown_connections 2023-11-27T15:17:34.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:34.013+0000 7efee2ea9700 1 -- 172.21.15.52:0/1917637784 wait complete. 2023-11-27T15:17:34.029 INFO:teuthology.run_tasks:Running task kclient... 2023-11-27T15:17:34.051 INFO:tasks.kclient:Mounting kernel clients... 2023-11-27T15:17:34.051 INFO:tasks.kclient:config is {'client.0': {}} 2023-11-27T15:17:34.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:17:34.052 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2023-11-27T15:17:34.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:17:34.069 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link delete ceph-brx 2023-11-27T15:17:34.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:33 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/411445713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:17:34.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:33 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1392250186' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:17:34.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:33 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/411445713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:17:34.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:33 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1392250186' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:17:34.138 INFO:teuthology.orchestra.run.smithi052.stderr:Cannot find device "ceph-brx" 2023-11-27T15:17:34.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T15:17:34.139 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2023-11-27T15:17:34.140 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-27T15:17:34.140 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-27T15:17:34.140 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph fs ls 2023-11-27T15:17:34.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:33 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/411445713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:17:34.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:33 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1392250186' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:17:34.832 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:17:35.043 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:34 smithi052 ceph-mon[99056]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-27T15:17:35.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:34 smithi102 ceph-mon[100296]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-27T15:17:35.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:34 smithi150 ceph-mon[100559]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-27T15:17:36.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:35 smithi102 ceph-mon[100296]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-11-27T15:17:36.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:35 smithi052 ceph-mon[99056]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-11-27T15:17:36.196 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.192+0000 7f68f9c68700 1 -- 172.21.15.52:0/2982453137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68f4104500 msgr2=0x7f68f410c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:36.196 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.192+0000 7f68f9c68700 1 --2- 172.21.15.52:0/2982453137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68f4104500 0x7f68f410c080 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f68ec00b0d0 tx=0x7f68ec00b3e0 comp rx=0 tx=0).stop 2023-11-27T15:17:36.196 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.192+0000 7f68f9c68700 1 -- 172.21.15.52:0/2982453137 shutdown_connections 2023-11-27T15:17:36.196 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.192+0000 7f68f9c68700 1 --2- 172.21.15.52:0/2982453137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68f4104500 0x7f68f410c080 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:36.197 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.192+0000 7f68f9c68700 1 --2- 172.21.15.52:0/2982453137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68f40ff460 0x7f68f40ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:36.197 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.192+0000 7f68f9c68700 1 --2- 172.21.15.52:0/2982453137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68f40ffe00 0x7f68f4103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:36.197 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.192+0000 7f68f9c68700 1 -- 172.21.15.52:0/2982453137 >> 172.21.15.52:0/2982453137 conn(0x7f68f40770a0 msgr2=0x7f68f40774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:36.197 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.192+0000 7f68f9c68700 1 -- 172.21.15.52:0/2982453137 shutdown_connections 2023-11-27T15:17:36.197 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.192+0000 7f68f9c68700 1 -- 172.21.15.52:0/2982453137 wait complete. 2023-11-27T15:17:36.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.192+0000 7f68f9c68700 1 Processor -- start 2023-11-27T15:17:36.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.193+0000 7f68f9c68700 1 -- start start 2023-11-27T15:17:36.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.193+0000 7f68f9c68700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68f40ff460 0x7f68f4195380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:36.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.193+0000 7f68f9c68700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68f40ffe00 0x7f68f41958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:36.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.193+0000 7f68f9c68700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68f4104500 0x7f68f4199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:36.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.193+0000 7f68f9c68700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f419a340 con 0x7f68f40ff460 2023-11-27T15:17:36.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.193+0000 7f68f9c68700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f419a4b0 con 0x7f68f40ffe00 2023-11-27T15:17:36.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.193+0000 7f68f9c68700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f419a670 con 0x7f68f4104500 2023-11-27T15:17:36.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.193+0000 7f68f37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68f40ff460 0x7f68f4195380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:36.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.193+0000 7f68f37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68f40ff460 0x7f68f4195380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51330/0 (socket says 172.21.15.52:51330) 2023-11-27T15:17:36.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.193+0000 7f68f37fe700 1 -- 172.21.15.52:0/2010040537 learned_addr learned my addr 172.21.15.52:0/2010040537 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:36.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.193+0000 7f68f2ffd700 1 --2- 172.21.15.52:0/2010040537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68f40ffe00 0x7f68f41958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:36.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.193+0000 7f68f3fff700 1 --2- 172.21.15.52:0/2010040537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68f4104500 0x7f68f4199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:36.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.193+0000 7f68f37fe700 1 -- 172.21.15.52:0/2010040537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68f4104500 msgr2=0x7f68f4199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:36.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.193+0000 7f68f37fe700 1 --2- 172.21.15.52:0/2010040537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68f4104500 0x7f68f4199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:36.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.193+0000 7f68f37fe700 1 -- 172.21.15.52:0/2010040537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68f40ffe00 msgr2=0x7f68f41958c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:36.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.193+0000 7f68f37fe700 1 --2- 172.21.15.52:0/2010040537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68f40ffe00 0x7f68f41958c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:36.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.193+0000 7f68f37fe700 1 -- 172.21.15.52:0/2010040537 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ec009d00 con 0x7f68f40ff460 2023-11-27T15:17:36.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.193+0000 7f68f3fff700 1 --2- 172.21.15.52:0/2010040537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68f4104500 0x7f68f4199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:17:36.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.193+0000 7f68f37fe700 1 --2- 172.21.15.52:0/2010040537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68f40ff460 0x7f68f4195380 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f68e400d950 tx=0x7f68e400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:36.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.194+0000 7f68f0ff9700 1 -- 172.21.15.52:0/2010040537 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e4009940 con 0x7f68f40ff460 2023-11-27T15:17:36.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.194+0000 7f68f0ff9700 1 -- 172.21.15.52:0/2010040537 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68e4010460 con 0x7f68f40ff460 2023-11-27T15:17:36.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.194+0000 7f68f9c68700 1 -- 172.21.15.52:0/2010040537 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f410f240 con 0x7f68f40ff460 2023-11-27T15:17:36.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.194+0000 7f68f0ff9700 1 -- 172.21.15.52:0/2010040537 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e400f620 con 0x7f68f40ff460 2023-11-27T15:17:36.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.194+0000 7f68f9c68700 1 -- 172.21.15.52:0/2010040537 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f410f0b0 con 0x7f68f40ff460 2023-11-27T15:17:36.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.195+0000 7f68f9c68700 1 -- 172.21.15.52:0/2010040537 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f410f3d0 con 0x7f68f40ff460 2023-11-27T15:17:36.202 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.196+0000 7f68f0ff9700 1 -- 172.21.15.52:0/2010040537 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f68e40108d0 con 0x7f68f40ff460 2023-11-27T15:17:36.202 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.196+0000 7f68f0ff9700 1 --2- 172.21.15.52:0/2010040537 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f68e00a1150 0x7f68e00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:36.202 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.197+0000 7f68f2ffd700 1 --2- 172.21.15.52:0/2010040537 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f68e00a1150 0x7f68e00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:36.202 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.197+0000 7f68f0ff9700 1 -- 172.21.15.52:0/2010040537 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f68e40cc7c0 con 0x7f68f40ff460 2023-11-27T15:17:36.202 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.197+0000 7f68f2ffd700 1 --2- 172.21.15.52:0/2010040537 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f68e00a1150 0x7f68e00a3600 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f68f41969a0 tx=0x7f68dc00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:36.204 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.200+0000 7f68f0ff9700 1 -- 172.21.15.52:0/2010040537 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f68e4095ef0 con 0x7f68f40ff460 2023-11-27T15:17:36.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:35 smithi150 ceph-mon[100559]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-11-27T15:17:36.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.444+0000 7f68f9c68700 1 -- 172.21.15.52:0/2010040537 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f68f41a2400 con 0x7f68f40ff460 2023-11-27T15:17:36.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.444+0000 7f68f0ff9700 1 -- 172.21.15.52:0/2010040537 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7f68e4016070 con 0x7f68f40ff460 2023-11-27T15:17:36.449 INFO:teuthology.orchestra.run.smithi052.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-27T15:17:36.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.446+0000 7f68f9c68700 1 -- 172.21.15.52:0/2010040537 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f68e00a1150 msgr2=0x7f68e00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:36.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.446+0000 7f68f9c68700 1 --2- 172.21.15.52:0/2010040537 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f68e00a1150 0x7f68e00a3600 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f68f41969a0 tx=0x7f68dc00a380 comp rx=0 tx=0).stop 2023-11-27T15:17:36.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.446+0000 7f68f9c68700 1 -- 172.21.15.52:0/2010040537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68f40ff460 msgr2=0x7f68f4195380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:36.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.446+0000 7f68f9c68700 1 --2- 172.21.15.52:0/2010040537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68f40ff460 0x7f68f4195380 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f68e400d950 tx=0x7f68e400dc60 comp rx=0 tx=0).stop 2023-11-27T15:17:36.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.446+0000 7f68f9c68700 1 -- 172.21.15.52:0/2010040537 shutdown_connections 2023-11-27T15:17:36.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.446+0000 7f68f9c68700 1 --2- 172.21.15.52:0/2010040537 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f68e00a1150 0x7f68e00a3600 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:36.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.446+0000 7f68f9c68700 1 --2- 172.21.15.52:0/2010040537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68f4104500 0x7f68f4199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:36.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.446+0000 7f68f9c68700 1 --2- 172.21.15.52:0/2010040537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68f40ff460 0x7f68f4195380 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:36.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.446+0000 7f68f9c68700 1 --2- 172.21.15.52:0/2010040537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68f40ffe00 0x7f68f41958c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:36.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.446+0000 7f68f9c68700 1 -- 172.21.15.52:0/2010040537 >> 172.21.15.52:0/2010040537 conn(0x7f68f40770a0 msgr2=0x7f68f40fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:36.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.447+0000 7f68f9c68700 1 -- 172.21.15.52:0/2010040537 shutdown_connections 2023-11-27T15:17:36.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:36.447+0000 7f68f9c68700 1 -- 172.21.15.52:0/2010040537 wait complete. 2023-11-27T15:17:36.993 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-27T15:17:36.993 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-27T15:17:36.993 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2023-11-27T15:17:36.993 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-27T15:17:36.993 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-27T15:17:36.993 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-27T15:17:36.993 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-27T15:17:36.993 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-27T15:17:36.993 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-27T15:17:36.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:17:36.994 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2023-11-27T15:17:37.012 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-27T15:17:37.012 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-27T15:17:37.012 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2023-11-27T15:17:37.012 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2023-11-27T15:17:37.013 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2023-11-27T15:17:37.013 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2023-11-27T15:17:37.013 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-27T15:17:37.013 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2023-11-27T15:17:37.013 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-27T15:17:37.013 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2023-11-27T15:17:37.013 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-27T15:17:37.013 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2023-11-27T15:17:37.013 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-27T15:17:37.013 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 41722sec preferred_lft 41722sec 2023-11-27T15:17:37.013 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link 2023-11-27T15:17:37.014 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2023-11-27T15:17:37.014 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-27T15:17:37.014 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2023-11-27T15:17:37.014 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-27T15:17:37.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:17:37.014 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec bash -c ' 2023-11-27T15:17:37.014 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2023-11-27T15:17:37.015 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link add name ceph-brx type bridge 2023-11-27T15:17:37.015 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip addr flush dev ceph-brx 2023-11-27T15:17:37.015 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set ceph-brx up 2023-11-27T15:17:37.015 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-27T15:17:37.015 DEBUG:teuthology.orchestra.run.smithi052:> ') 2023-11-27T15:17:37.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:36 smithi102 ceph-mon[100296]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2023-11-27T15:17:37.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:36 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2010040537' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-27T15:17:37.093 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:36 smithi052 ceph-mon[99056]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2023-11-27T15:17:37.093 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:36 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2010040537' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-27T15:17:37.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:17:37.188 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-27T15:17:37.256 INFO:teuthology.orchestra.run.smithi052.stdout:1 2023-11-27T15:17:37.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:17:37.258 DEBUG:teuthology.orchestra.run.smithi052:> route 2023-11-27T15:17:37.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:36 smithi150 ceph-mon[100559]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2023-11-27T15:17:37.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:36 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2010040537' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-27T15:17:37.328 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2023-11-27T15:17:37.328 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-27T15:17:37.328 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-27T15:17:37.328 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-27T15:17:37.328 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-27T15:17:37.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:17:37.329 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec bash -c ' 2023-11-27T15:17:37.329 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2023-11-27T15:17:37.329 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-27T15:17:37.329 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-27T15:17:37.329 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-27T15:17:37.329 DEBUG:teuthology.orchestra.run.smithi052:> ') 2023-11-27T15:17:37.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:17:37.579 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2023-11-27T15:17:37.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:17:37.597 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list-id 2023-11-27T15:17:37.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:17:37.654 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec bash -c ' 2023-11-27T15:17:37.654 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2023-11-27T15:17:37.654 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-27T15:17:37.655 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-27T15:17:37.655 DEBUG:teuthology.orchestra.run.smithi052:> ') 2023-11-27T15:17:37.752 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-27T15:17:37.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:17:37.752 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec bash -c ' 2023-11-27T15:17:37.753 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2023-11-27T15:17:37.753 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-27T15:17:37.753 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-27T15:17:37.753 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-27T15:17:37.753 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-27T15:17:37.753 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-27T15:17:37.753 DEBUG:teuthology.orchestra.run.smithi052:> ') 2023-11-27T15:17:37.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:17:37.967 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec bash -c ' 2023-11-27T15:17:37.967 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2023-11-27T15:17:37.967 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.0 up 2023-11-27T15:17:37.967 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set dev brx.0 master ceph-brx 2023-11-27T15:17:37.967 DEBUG:teuthology.orchestra.run.smithi052:> ') 2023-11-27T15:17:38.068 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-27T15:17:38.068 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-27T15:17:38.085 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-27T15:17:38.086 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-27T15:17:38.086 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-27T15:17:38.142 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-11-27T15:17:38.143 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-11-27T15:17:38.143 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/_nogroup/sv_1/fdbc3811-1284-497a-bc65-bbb78ba5ec15 2023-11-27T15:17:38.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:17:38.143 DEBUG:teuthology.orchestra.run.smithi052:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/_nogroup/sv_1/fdbc3811-1284-497a-bc65-bbb78ba5ec15 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2023-11-27T15:17:38.255 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2023-11-27T15:17:38.256 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync". 2023-11-27T15:17:38.447 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2023-11-27T15:17:38.447 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync". 2023-11-27T15:17:38.447 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: resolved to: "172.21.15.52:6789,172.21.15.102:6789,172.21.15.150:6789" 2023-11-27T15:17:38.448 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: trying mount with new device syntax: 0@c6c7df88-8d36-11ee-95a2-87774f69a715.cephfs=/volumes/_nogroup/sv_1/fdbc3811-1284-497a-bc65-bbb78ba5ec15 2023-11-27T15:17:38.448 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync,name=0,key=0,mon_addr=172.21.15.52:6789/172.21.15.102:6789/172.21.15.150:6789" will pass to kernel 2023-11-27T15:17:38.448 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: v2 mount support check returned 2 2023-11-27T15:17:38.448 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-27T15:17:38.448 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.52:6789,172.21.15.102:6789,172.21.15.150:6789:/volumes/_nogroup/sv_1/fdbc3811-1284-497a-bc65-bbb78ba5ec15 2023-11-27T15:17:38.448 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync,name=0,key=0,mds_namespace=cephfs,fsid=c6c7df88-8d36-11ee-95a2-87774f69a715" will pass to kernel 2023-11-27T15:17:38.450 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-27T15:17:38.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:17:38.451 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-27T15:17:38.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:17:38.480 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-27T15:17:38.554 INFO:teuthology.orchestra.run.smithi052.stdout:client15219 2023-11-27T15:17:38.554 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2023-11-27T15:17:38.554 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-27T15:17:38.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:17:38.612 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-27T15:17:38.685 INFO:teuthology.orchestra.run.smithi052.stdout:client15219 2023-11-27T15:17:38.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:17:38.686 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-27T15:17:38.759 INFO:teuthology.orchestra.run.smithi052.stdout:client15219 2023-11-27T15:17:38.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:17:38.759 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715.client15219/status') 2023-11-27T15:17:38.842 INFO:teuthology.orchestra.run.smithi052.stdout:instance: client.15219 (3)192.168.0.1:0/3273606005 2023-11-27T15:17:38.842 INFO:teuthology.orchestra.run.smithi052.stdout:blocklisted: false 2023-11-27T15:17:38.842 INFO:teuthology.orchestra.run.smithi052.stderr:0+1 records in 2023-11-27T15:17:38.842 INFO:teuthology.orchestra.run.smithi052.stderr:0+1 records out 2023-11-27T15:17:38.842 INFO:teuthology.orchestra.run.smithi052.stderr:70 bytes copied, 2.5284e-05 s, 2.8 MB/s 2023-11-27T15:17:38.844 INFO:teuthology.run_tasks:Running task check-counter... 2023-11-27T15:17:38.856 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-11-27T15:17:38.868 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:17:39.014 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.007+0000 7fb375856700 1 -- 172.21.15.52:0/914376483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3700ff420 msgr2=0x7fb3700ff870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:39.014 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.007+0000 7fb375856700 1 --2- 172.21.15.52:0/914376483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3700ff420 0x7fb3700ff870 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb364009990 tx=0x7fb364009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.014 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.007+0000 7fb375856700 1 -- 172.21.15.52:0/914376483 shutdown_connections 2023-11-27T15:17:39.014 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.007+0000 7fb375856700 1 --2- 172.21.15.52:0/914376483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb370100780 0x7fb370100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.014 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.007+0000 7fb375856700 1 --2- 172.21.15.52:0/914376483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3700ffdb0 0x7fb370107680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.015 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.007+0000 7fb375856700 1 --2- 172.21.15.52:0/914376483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3700ff420 0x7fb3700ff870 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.015 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.007+0000 7fb375856700 1 -- 172.21.15.52:0/914376483 >> 172.21.15.52:0/914376483 conn(0x7fb3700f9f00 msgr2=0x7fb3700fc370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:39.015 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.007+0000 7fb375856700 1 -- 172.21.15.52:0/914376483 shutdown_connections 2023-11-27T15:17:39.015 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.007+0000 7fb375856700 1 -- 172.21.15.52:0/914376483 wait complete. 2023-11-27T15:17:39.015 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.008+0000 7fb375856700 1 Processor -- start 2023-11-27T15:17:39.015 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.008+0000 7fb375856700 1 -- start start 2023-11-27T15:17:39.015 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.008+0000 7fb375856700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3700ff420 0x7fb3700feea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:39.015 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.008+0000 7fb375856700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3700ffdb0 0x7fb37019acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:39.015 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.008+0000 7fb375856700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb370100780 0x7fb37019d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:39.015 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.008+0000 7fb375856700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37019daf0 con 0x7fb3700ff420 2023-11-27T15:17:39.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.008+0000 7fb375856700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37019dc60 con 0x7fb3700ffdb0 2023-11-27T15:17:39.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.008+0000 7fb375856700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37019de20 con 0x7fb370100780 2023-11-27T15:17:39.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.008+0000 7fb36effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3700ff420 0x7fb3700feea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:39.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.008+0000 7fb36effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3700ff420 0x7fb3700feea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51348/0 (socket says 172.21.15.52:51348) 2023-11-27T15:17:39.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.008+0000 7fb36effd700 1 -- 172.21.15.52:0/1482747392 learned_addr learned my addr 172.21.15.52:0/1482747392 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:39.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.009+0000 7fb36e7fc700 1 --2- 172.21.15.52:0/1482747392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3700ffdb0 0x7fb37019acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:39.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.009+0000 7fb36f7fe700 1 --2- 172.21.15.52:0/1482747392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb370100780 0x7fb37019d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:39.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.009+0000 7fb36effd700 1 -- 172.21.15.52:0/1482747392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb370100780 msgr2=0x7fb37019d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:39.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.009+0000 7fb36effd700 1 --2- 172.21.15.52:0/1482747392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb370100780 0x7fb37019d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.009+0000 7fb36effd700 1 -- 172.21.15.52:0/1482747392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3700ffdb0 msgr2=0x7fb37019acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:39.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.009+0000 7fb36effd700 1 --2- 172.21.15.52:0/1482747392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3700ffdb0 0x7fb37019acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.009+0000 7fb36effd700 1 -- 172.21.15.52:0/1482747392 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3640095f0 con 0x7fb3700ff420 2023-11-27T15:17:39.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.009+0000 7fb36f7fe700 1 --2- 172.21.15.52:0/1482747392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb370100780 0x7fb37019d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:17:39.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.009+0000 7fb36effd700 1 --2- 172.21.15.52:0/1482747392 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3700ff420 0x7fb3700feea0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb36000d980 tx=0x7fb36000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:39.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.009+0000 7fb374854700 1 -- 172.21.15.52:0/1482747392 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36000b950 con 0x7fb3700ff420 2023-11-27T15:17:39.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.009+0000 7fb374854700 1 -- 172.21.15.52:0/1482747392 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb360007620 con 0x7fb3700ff420 2023-11-27T15:17:39.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.009+0000 7fb374854700 1 -- 172.21.15.52:0/1482747392 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36000f3c0 con 0x7fb3700ff420 2023-11-27T15:17:39.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.009+0000 7fb375856700 1 -- 172.21.15.52:0/1482747392 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb37010ab70 con 0x7fb3700ff420 2023-11-27T15:17:39.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.009+0000 7fb375856700 1 -- 172.21.15.52:0/1482747392 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb37010a9e0 con 0x7fb3700ff420 2023-11-27T15:17:39.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.010+0000 7fb375856700 1 -- 172.21.15.52:0/1482747392 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3701067e0 con 0x7fb3700ff420 2023-11-27T15:17:39.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.011+0000 7fb374854700 1 -- 172.21.15.52:0/1482747392 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb360010640 con 0x7fb3700ff420 2023-11-27T15:17:39.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.012+0000 7fb374854700 1 --2- 172.21.15.52:0/1482747392 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb3580a1150 0x7fb3580a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:39.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.012+0000 7fb374854700 1 -- 172.21.15.52:0/1482747392 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb3600ccff0 con 0x7fb3700ff420 2023-11-27T15:17:39.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.012+0000 7fb36e7fc700 1 --2- 172.21.15.52:0/1482747392 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb3580a1150 0x7fb3580a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:39.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.012+0000 7fb36e7fc700 1 --2- 172.21.15.52:0/1482747392 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb3580a1150 0x7fb3580a3600 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb3700fec10 tx=0x7fb364005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:39.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.015+0000 7fb374854700 1 -- 172.21.15.52:0/1482747392 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb360096720 con 0x7fb3700ff420 2023-11-27T15:17:39.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:38 smithi102 ceph-mon[100296]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-27T15:17:39.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:38 smithi052 ceph-mon[99056]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-27T15:17:39.218 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.214+0000 7fb375856700 1 -- 172.21.15.52:0/1482747392 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb37002cfe0 con 0x7fb3700ff420 2023-11-27T15:17:39.220 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.216+0000 7fb374854700 1 -- 172.21.15.52:0/1482747392 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22657 (secure 0 0 0) 0x7fb36000ee60 con 0x7fb3700ff420 2023-11-27T15:17:39.220 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:17:39.221 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":85,"fsid":"c6c7df88-8d36-11ee-95a2-87774f69a715","created":"2023-11-27T15:09:03.333345+0000","modified":"2023-11-27T15:17:04.797892+0000","last_up_change":"2023-11-27T15:15:27.656134+0000","last_in_change":"2023-11-27T15:15:10.671153+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:12:14.177523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:16:07.842251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:16:08.728520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58d69ede-c293-49f5-abbe-8187793f2b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6803","nonce":608216415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6805","nonce":608216415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6809","nonce":608216415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6807","nonce":608216415}]},"public_addr":"172.21.15.52:6803/608216415","cluster_addr":"172.21.15.52:6805/608216415","heartbeat_back_addr":"172.21.15.52:6809/608216415","heartbeat_front_addr":"172.21.15.52:6807/608216415","state":["exists","up"]},{"osd":1,"uuid":"d6a46b91-adc8-45ef-80f5-566ae400911f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6801","nonce":2538384087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6803","nonce":2538384087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6807","nonce":2538384087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6805","nonce":2538384087}]},"public_addr":"172.21.15.102:6801/2538384087","cluster_addr":"172.21.15.102:6803/2538384087","heartbeat_back_addr":"172.21.15.102:6807/2538384087","heartbeat_front_addr":"172.21.15.102:6805/2538384087","state":["exists","up"]},{"osd":2,"uuid":"846ba125-11f8-46b2-8941-bd5e1947cc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6801","nonce":4258038221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6803","nonce":4258038221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6807","nonce":4258038221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6805","nonce":4258038221}]},"public_addr":"172.21.15.150:6801/4258038221","cluster_addr":"172.21.15.150:6803/4258038221","heartbeat_back_addr":"172.21.15.150:6807/4258038221","heartbeat_front_addr":"172.21.15.150:6805/4258038221","state":["exists","up"]},{"osd":3,"uuid":"950fcd67-1057-4288-a389-714f1ec94261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6811","nonce":2622100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6813","nonce":2622100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6817","nonce":2622100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6815","nonce":2622100752}]},"public_addr":"172.21.15.52:6811/2622100752","cluster_addr":"172.21.15.52:6813/2622100752","heartbeat_back_addr":"172.21.15.52:6817/2622100752","heartbeat_front_addr":"172.21.15.52:6815/2622100752","state":["exists","up"]},{"osd":4,"uuid":"1c96a676-cea9-4deb-bad1-f7fd240c3d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6809","nonce":1546794763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6811","nonce":1546794763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6815","nonce":1546794763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6813","nonce":1546794763}]},"public_addr":"172.21.15.102:6809/1546794763","cluster_addr":"172.21.15.102:6811/1546794763","heartbeat_back_addr":"172.21.15.102:6815/1546794763","heartbeat_front_addr":"172.21.15.102:6813/1546794763","state":["exists","up"]},{"osd":5,"uuid":"38a75af1-fd4d-433e-b579-0609109c851c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6809","nonce":461511375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6811","nonce":461511375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6815","nonce":461511375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6813","nonce":461511375}]},"public_addr":"172.21.15.150:6809/461511375","cluster_addr":"172.21.15.150:6811/461511375","heartbeat_back_addr":"172.21.15.150:6815/461511375","heartbeat_front_addr":"172.21.15.150:6813/461511375","state":["exists","up"]},{"osd":6,"uuid":"2f0ede4a-d175-4b4a-b876-c0f442048901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6819","nonce":1002457464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6821","nonce":1002457464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6825","nonce":1002457464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6823","nonce":1002457464}]},"public_addr":"172.21.15.52:6819/1002457464","cluster_addr":"172.21.15.52:6821/1002457464","heartbeat_back_addr":"172.21.15.52:6825/1002457464","heartbeat_front_addr":"172.21.15.52:6823/1002457464","state":["exists","up"]},{"osd":7,"uuid":"6b635192-abcf-4bbf-bf78-a019b351bca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6817","nonce":3249840064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6819","nonce":3249840064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6823","nonce":3249840064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6821","nonce":3249840064}]},"public_addr":"172.21.15.102:6817/3249840064","cluster_addr":"172.21.15.102:6819/3249840064","heartbeat_back_addr":"172.21.15.102:6823/3249840064","heartbeat_front_addr":"172.21.15.102:6821/3249840064","state":["exists","up"]},{"osd":8,"uuid":"c7ca8007-7170-47bf-956c-3f00edf27e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6817","nonce":3672583574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6819","nonce":3672583574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6823","nonce":3672583574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6821","nonce":3672583574}]},"public_addr":"172.21.15.150:6817/3672583574","cluster_addr":"172.21.15.150:6819/3672583574","heartbeat_back_addr":"172.21.15.150:6823/3672583574","heartbeat_front_addr":"172.21.15.150:6821/3672583574","state":["exists","up"]},{"osd":9,"uuid":"0e252c2c-01a4-48ba-832d-067ed940fefb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6827","nonce":4287934320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6829","nonce":4287934320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6833","nonce":4287934320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6831","nonce":4287934320}]},"public_addr":"172.21.15.52:6827/4287934320","cluster_addr":"172.21.15.52:6829/4287934320","heartbeat_back_addr":"172.21.15.52:6833/4287934320","heartbeat_front_addr":"172.21.15.52:6831/4287934320","state":["exists","up"]},{"osd":10,"uuid":"bb540142-7563-44a4-b7a3-cb662714c128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6825","nonce":3056034201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6827","nonce":3056034201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6831","nonce":3056034201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6829","nonce":3056034201}]},"public_addr":"172.21.15.102:6825/3056034201","cluster_addr":"172.21.15.102:6827/3056034201","heartbeat_back_addr":"172.21.15.102:6831/3056034201","heartbeat_front_addr":"172.21.15.102:6829/3056034201","state":["exists","up"]},{"osd":11,"uuid":"3156528e-e0f8-4d44-9946-c8b6339c2bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6825","nonce":3207559739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6827","nonce":3207559739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6831","nonce":3207559739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6829","nonce":3207559739}]},"public_addr":"172.21.15.150:6825/3207559739","cluster_addr":"172.21.15.150:6827/3207559739","heartbeat_back_addr":"172.21.15.150:6831/3207559739","heartbeat_front_addr":"172.21.15.150:6829/3207559739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:26.826099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:48.331371+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:10.459738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:32.356472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:54.043170+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:15.648582+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:37.166463+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:59.240819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:20.710097+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:43.364563+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:05.279433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:26.344573+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":5,"to":7}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/1392282110":"2023-11-28T15:10:03.644870+0000","172.21.15.52:0/1479993479":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6800/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/961562652":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/1290665390":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2410529742":"2023-11-28T15:09:37.118690+0000","172.21.15.52:6800/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2130037044":"2023-11-28T15:10:03.644870+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:17:39.222 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.218+0000 7fb375856700 1 -- 172.21.15.52:0/1482747392 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb3580a1150 msgr2=0x7fb3580a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:39.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.218+0000 7fb375856700 1 --2- 172.21.15.52:0/1482747392 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb3580a1150 0x7fb3580a3600 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb3700fec10 tx=0x7fb364005fb0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.218+0000 7fb375856700 1 -- 172.21.15.52:0/1482747392 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3700ff420 msgr2=0x7fb3700feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:39.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.218+0000 7fb375856700 1 --2- 172.21.15.52:0/1482747392 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3700ff420 0x7fb3700feea0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb36000d980 tx=0x7fb36000dc90 comp rx=0 tx=0).stop 2023-11-27T15:17:39.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.218+0000 7fb375856700 1 -- 172.21.15.52:0/1482747392 shutdown_connections 2023-11-27T15:17:39.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.218+0000 7fb375856700 1 --2- 172.21.15.52:0/1482747392 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb3580a1150 0x7fb3580a3600 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.218+0000 7fb375856700 1 --2- 172.21.15.52:0/1482747392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb370100780 0x7fb37019d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.218+0000 7fb375856700 1 --2- 172.21.15.52:0/1482747392 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3700ff420 0x7fb3700feea0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.218+0000 7fb375856700 1 --2- 172.21.15.52:0/1482747392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3700ffdb0 0x7fb37019acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.218+0000 7fb375856700 1 -- 172.21.15.52:0/1482747392 >> 172.21.15.52:0/1482747392 conn(0x7fb3700f9f00 msgr2=0x7fb370103920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:39.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.218+0000 7fb375856700 1 -- 172.21.15.52:0/1482747392 shutdown_connections 2023-11-27T15:17:39.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.218+0000 7fb375856700 1 -- 172.21.15.52:0/1482747392 wait complete. 2023-11-27T15:17:39.235 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:12:14.177523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-27T15:16:07.842251+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-27T15:16:08.728520+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-27T15:17:39.235 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:17:39.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:38 smithi150 ceph-mon[100559]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-27T15:17:39.364 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.360+0000 7f4fdad6b700 1 -- 172.21.15.52:0/59090563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fd40ff1c0 msgr2=0x7f4fd40ff5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:39.364 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.360+0000 7f4fdad6b700 1 --2- 172.21.15.52:0/59090563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fd40ff1c0 0x7f4fd40ff5d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009990 tx=0x7f4fc8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.364 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.360+0000 7f4fdad6b700 1 -- 172.21.15.52:0/59090563 shutdown_connections 2023-11-27T15:17:39.365 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.360+0000 7f4fdad6b700 1 --2- 172.21.15.52:0/59090563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4fd4100d40 0x7f4fd4107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.365 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.360+0000 7f4fdad6b700 1 --2- 172.21.15.52:0/59090563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fd40ff1c0 0x7f4fd40ff5d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.365 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.360+0000 7f4fdad6b700 1 --2- 172.21.15.52:0/59090563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fd41003b0 0x7f4fd4100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.365 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.360+0000 7f4fdad6b700 1 -- 172.21.15.52:0/59090563 >> 172.21.15.52:0/59090563 conn(0x7f4fd40fa7a0 msgr2=0x7f4fd40fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:39.365 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.360+0000 7f4fdad6b700 1 -- 172.21.15.52:0/59090563 shutdown_connections 2023-11-27T15:17:39.365 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.360+0000 7f4fdad6b700 1 -- 172.21.15.52:0/59090563 wait complete. 2023-11-27T15:17:39.367 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.361+0000 7f4fdad6b700 1 Processor -- start 2023-11-27T15:17:39.367 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.361+0000 7f4fdad6b700 1 -- start start 2023-11-27T15:17:39.367 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.361+0000 7f4fdad6b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fd40ff1c0 0x7f4fd4194fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:39.367 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.361+0000 7f4fdad6b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fd41003b0 0x7f4fd41954e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:39.368 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.361+0000 7f4fdad6b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4fd4100d40 0x7f4fd419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:39.368 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.361+0000 7f4fdad6b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd419cb20 con 0x7f4fd40ff1c0 2023-11-27T15:17:39.368 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.361+0000 7f4fdad6b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd419cc60 con 0x7f4fd4100d40 2023-11-27T15:17:39.368 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.361+0000 7f4fdad6b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd419cdf0 con 0x7f4fd41003b0 2023-11-27T15:17:39.368 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.361+0000 7f4fd8b07700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fd40ff1c0 0x7f4fd4194fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:39.368 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.361+0000 7f4fd8b07700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fd40ff1c0 0x7f4fd4194fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51362/0 (socket says 172.21.15.52:51362) 2023-11-27T15:17:39.368 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.361+0000 7f4fd8b07700 1 -- 172.21.15.52:0/2349031457 learned_addr learned my addr 172.21.15.52:0/2349031457 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:39.368 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.361+0000 7f4fd3fff700 1 --2- 172.21.15.52:0/2349031457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fd41003b0 0x7f4fd41954e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:39.368 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.361+0000 7f4fd8b07700 1 -- 172.21.15.52:0/2349031457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fd41003b0 msgr2=0x7f4fd41954e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:39.368 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.361+0000 7f4fd9308700 1 --2- 172.21.15.52:0/2349031457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4fd4100d40 0x7f4fd419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:39.368 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.361+0000 7f4fd8b07700 1 --2- 172.21.15.52:0/2349031457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fd41003b0 0x7f4fd41954e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.368 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.361+0000 7f4fd8b07700 1 -- 172.21.15.52:0/2349031457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4fd4100d40 msgr2=0x7f4fd419c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:39.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.361+0000 7f4fd8b07700 1 --2- 172.21.15.52:0/2349031457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4fd4100d40 0x7f4fd419c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.361+0000 7f4fd8b07700 1 -- 172.21.15.52:0/2349031457 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc80095f0 con 0x7f4fd40ff1c0 2023-11-27T15:17:39.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.361+0000 7f4fd3fff700 1 --2- 172.21.15.52:0/2349031457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fd41003b0 0x7f4fd41954e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:17:39.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.361+0000 7f4fd9308700 1 --2- 172.21.15.52:0/2349031457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4fd4100d40 0x7f4fd419c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:17:39.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.362+0000 7f4fd8b07700 1 --2- 172.21.15.52:0/2349031457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fd40ff1c0 0x7f4fd4194fa0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8000c00 tx=0x7f4fc800f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:39.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.362+0000 7f4fd1ffb700 1 -- 172.21.15.52:0/2349031457 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc801d070 con 0x7f4fd40ff1c0 2023-11-27T15:17:39.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.362+0000 7f4fd1ffb700 1 -- 172.21.15.52:0/2349031457 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fc800fc60 con 0x7f4fd40ff1c0 2023-11-27T15:17:39.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.362+0000 7f4fdad6b700 1 -- 172.21.15.52:0/2349031457 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd410ab20 con 0x7f4fd40ff1c0 2023-11-27T15:17:39.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.362+0000 7f4fd1ffb700 1 -- 172.21.15.52:0/2349031457 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8017650 con 0x7f4fd40ff1c0 2023-11-27T15:17:39.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.362+0000 7f4fdad6b700 1 -- 172.21.15.52:0/2349031457 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd410a990 con 0x7f4fd40ff1c0 2023-11-27T15:17:39.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.363+0000 7f4fdad6b700 1 -- 172.21.15.52:0/2349031457 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fd419d390 con 0x7f4fd40ff1c0 2023-11-27T15:17:39.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.364+0000 7f4fd1ffb700 1 -- 172.21.15.52:0/2349031457 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4fc8004830 con 0x7f4fd40ff1c0 2023-11-27T15:17:39.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.365+0000 7f4fd1ffb700 1 --2- 172.21.15.52:0/2349031457 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4fc40a1150 0x7f4fc40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:39.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.365+0000 7f4fd1ffb700 1 -- 172.21.15.52:0/2349031457 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4fc80cdff0 con 0x7f4fd40ff1c0 2023-11-27T15:17:39.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.365+0000 7f4fd3fff700 1 --2- 172.21.15.52:0/2349031457 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4fc40a1150 0x7f4fc40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:39.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.365+0000 7f4fd3fff700 1 --2- 172.21.15.52:0/2349031457 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4fc40a1150 0x7f4fc40a3600 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0009be0 tx=0x7f4fc0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:39.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.368+0000 7f4fd1ffb700 1 -- 172.21.15.52:0/2349031457 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4fc8097790 con 0x7f4fd40ff1c0 2023-11-27T15:17:39.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.566+0000 7f4fdad6b700 1 -- 172.21.15.52:0/2349031457 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4fd4106120 con 0x7f4fd40ff1c0 2023-11-27T15:17:39.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.567+0000 7f4fd1ffb700 1 -- 172.21.15.52:0/2349031457 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4fc809adb0 con 0x7f4fd40ff1c0 2023-11-27T15:17:39.571 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2023-11-27T15:17:39.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.569+0000 7f4fdad6b700 1 -- 172.21.15.52:0/2349031457 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4fc40a1150 msgr2=0x7f4fc40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:39.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.569+0000 7f4fdad6b700 1 --2- 172.21.15.52:0/2349031457 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4fc40a1150 0x7f4fc40a3600 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0009be0 tx=0x7f4fc0009260 comp rx=0 tx=0).stop 2023-11-27T15:17:39.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.569+0000 7f4fdad6b700 1 -- 172.21.15.52:0/2349031457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fd40ff1c0 msgr2=0x7f4fd4194fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:39.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.569+0000 7f4fdad6b700 1 --2- 172.21.15.52:0/2349031457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fd40ff1c0 0x7f4fd4194fa0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8000c00 tx=0x7f4fc800f510 comp rx=0 tx=0).stop 2023-11-27T15:17:39.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.569+0000 7f4fdad6b700 1 -- 172.21.15.52:0/2349031457 shutdown_connections 2023-11-27T15:17:39.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.569+0000 7f4fdad6b700 1 --2- 172.21.15.52:0/2349031457 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4fc40a1150 0x7f4fc40a3600 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.569+0000 7f4fdad6b700 1 --2- 172.21.15.52:0/2349031457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4fd4100d40 0x7f4fd419c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.569+0000 7f4fdad6b700 1 --2- 172.21.15.52:0/2349031457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fd40ff1c0 0x7f4fd4194fa0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.569+0000 7f4fdad6b700 1 --2- 172.21.15.52:0/2349031457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fd41003b0 0x7f4fd41954e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.569+0000 7f4fdad6b700 1 -- 172.21.15.52:0/2349031457 >> 172.21.15.52:0/2349031457 conn(0x7f4fd40fa7a0 msgr2=0x7f4fd40fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:39.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.569+0000 7f4fdad6b700 1 -- 172.21.15.52:0/2349031457 shutdown_connections 2023-11-27T15:17:39.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.569+0000 7f4fdad6b700 1 -- 172.21.15.52:0/2349031457 wait complete. 2023-11-27T15:17:39.585 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-27T15:17:39.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.710+0000 7f1616989700 1 -- 172.21.15.52:0/3471827814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1610101e50 msgr2=0x7f1610108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:39.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.710+0000 7f1616989700 1 --2- 172.21.15.52:0/3471827814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1610101e50 0x7f1610108710 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f160800b0a0 tx=0x7f160800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.710+0000 7f1616989700 1 -- 172.21.15.52:0/3471827814 shutdown_connections 2023-11-27T15:17:39.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.710+0000 7f1616989700 1 --2- 172.21.15.52:0/3471827814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1610101e50 0x7f1610108710 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.710+0000 7f1616989700 1 --2- 172.21.15.52:0/3471827814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16101002d0 0x7f16101006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.710+0000 7f1616989700 1 --2- 172.21.15.52:0/3471827814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16101014c0 0x7f1610101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.710+0000 7f1616989700 1 -- 172.21.15.52:0/3471827814 >> 172.21.15.52:0/3471827814 conn(0x7f16100fb8b0 msgr2=0x7f16100fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:39.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.710+0000 7f1616989700 1 -- 172.21.15.52:0/3471827814 shutdown_connections 2023-11-27T15:17:39.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.710+0000 7f1616989700 1 -- 172.21.15.52:0/3471827814 wait complete. 2023-11-27T15:17:39.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.711+0000 7f1616989700 1 Processor -- start 2023-11-27T15:17:39.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.711+0000 7f1616989700 1 -- start start 2023-11-27T15:17:39.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.711+0000 7f1616989700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16101002d0 0x7f1610196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:39.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.711+0000 7f1616989700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16101014c0 0x7f1610196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:39.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.711+0000 7f1616989700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1610101e50 0x7f161019d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:39.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.711+0000 7f1616989700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161019dcb0 con 0x7f1610101e50 2023-11-27T15:17:39.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.711+0000 7f1616989700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161019ddf0 con 0x7f16101002d0 2023-11-27T15:17:39.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.711+0000 7f1616989700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161019df80 con 0x7f16101014c0 2023-11-27T15:17:39.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.711+0000 7f1614f26700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1610101e50 0x7f161019d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:39.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.711+0000 7f1614f26700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1610101e50 0x7f161019d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51378/0 (socket says 172.21.15.52:51378) 2023-11-27T15:17:39.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.711+0000 7f1614f26700 1 -- 172.21.15.52:0/4109129937 learned_addr learned my addr 172.21.15.52:0/4109129937 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:39.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.711+0000 7f160f7fe700 1 --2- 172.21.15.52:0/4109129937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16101014c0 0x7f1610196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:39.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.711+0000 7f160ffff700 1 --2- 172.21.15.52:0/4109129937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16101002d0 0x7f1610196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:39.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.711+0000 7f1614f26700 1 -- 172.21.15.52:0/4109129937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16101014c0 msgr2=0x7f1610196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:39.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.711+0000 7f1614f26700 1 --2- 172.21.15.52:0/4109129937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16101014c0 0x7f1610196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.711+0000 7f1614f26700 1 -- 172.21.15.52:0/4109129937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16101002d0 msgr2=0x7f1610196130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:39.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.711+0000 7f1614f26700 1 --2- 172.21.15.52:0/4109129937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16101002d0 0x7f1610196130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.711+0000 7f1614f26700 1 -- 172.21.15.52:0/4109129937 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1608009d00 con 0x7f1610101e50 2023-11-27T15:17:39.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.711+0000 7f160f7fe700 1 --2- 172.21.15.52:0/4109129937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16101014c0 0x7f1610196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:17:39.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.712+0000 7f1614f26700 1 --2- 172.21.15.52:0/4109129937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1610101e50 0x7f161019d6e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f160800b830 tx=0x7f1608004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:39.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.712+0000 7f160d7fa700 1 -- 172.21.15.52:0/4109129937 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160800e040 con 0x7f1610101e50 2023-11-27T15:17:39.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.712+0000 7f160d7fa700 1 -- 172.21.15.52:0/4109129937 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16080094c0 con 0x7f1610101e50 2023-11-27T15:17:39.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.712+0000 7f1616989700 1 -- 172.21.15.52:0/4109129937 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f161010bc60 con 0x7f1610101e50 2023-11-27T15:17:39.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.712+0000 7f160d7fa700 1 -- 172.21.15.52:0/4109129937 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160801bc70 con 0x7f1610101e50 2023-11-27T15:17:39.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.712+0000 7f1616989700 1 -- 172.21.15.52:0/4109129937 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f161010bad0 con 0x7f1610101e50 2023-11-27T15:17:39.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.713+0000 7f1616989700 1 -- 172.21.15.52:0/4109129937 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16101077f0 con 0x7f1610101e50 2023-11-27T15:17:39.721 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.714+0000 7f160d7fa700 1 -- 172.21.15.52:0/4109129937 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f160801d890 con 0x7f1610101e50 2023-11-27T15:17:39.722 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.714+0000 7f160d7fa700 1 --2- 172.21.15.52:0/4109129937 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f15fc0a11d0 0x7f15fc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:39.722 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.714+0000 7f160ffff700 1 --2- 172.21.15.52:0/4109129937 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f15fc0a11d0 0x7f15fc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:39.722 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.714+0000 7f160d7fa700 1 -- 172.21.15.52:0/4109129937 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f16080ceb40 con 0x7f1610101e50 2023-11-27T15:17:39.722 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.715+0000 7f160ffff700 1 --2- 172.21.15.52:0/4109129937 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f15fc0a11d0 0x7f15fc0a3680 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1600005fd0 tx=0x7f160000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:39.722 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.718+0000 7f160d7fa700 1 -- 172.21.15.52:0/4109129937 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1608098380 con 0x7f1610101e50 2023-11-27T15:17:39.866 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:39 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1482747392' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:17:39.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:39 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2349031457' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:17:39.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.920+0000 7f1616989700 1 -- 172.21.15.52:0/4109129937 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1610061700 con 0x7f1610101e50 2023-11-27T15:17:39.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.920+0000 7f160d7fa700 1 -- 172.21.15.52:0/4109129937 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1608021020 con 0x7f1610101e50 2023-11-27T15:17:39.925 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2023-11-27T15:17:39.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.922+0000 7f1616989700 1 -- 172.21.15.52:0/4109129937 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f15fc0a11d0 msgr2=0x7f15fc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:39.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.922+0000 7f1616989700 1 --2- 172.21.15.52:0/4109129937 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f15fc0a11d0 0x7f15fc0a3680 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1600005fd0 tx=0x7f160000a560 comp rx=0 tx=0).stop 2023-11-27T15:17:39.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.922+0000 7f1616989700 1 -- 172.21.15.52:0/4109129937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1610101e50 msgr2=0x7f161019d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:39.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.922+0000 7f1616989700 1 --2- 172.21.15.52:0/4109129937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1610101e50 0x7f161019d6e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f160800b830 tx=0x7f1608004d30 comp rx=0 tx=0).stop 2023-11-27T15:17:39.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.922+0000 7f1616989700 1 -- 172.21.15.52:0/4109129937 shutdown_connections 2023-11-27T15:17:39.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.922+0000 7f1616989700 1 --2- 172.21.15.52:0/4109129937 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f15fc0a11d0 0x7f15fc0a3680 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.928 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.922+0000 7f1616989700 1 --2- 172.21.15.52:0/4109129937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16101002d0 0x7f1610196130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.928 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.922+0000 7f1616989700 1 --2- 172.21.15.52:0/4109129937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1610101e50 0x7f161019d6e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.928 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.922+0000 7f1616989700 1 --2- 172.21.15.52:0/4109129937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16101014c0 0x7f1610196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:39.928 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.922+0000 7f1616989700 1 -- 172.21.15.52:0/4109129937 >> 172.21.15.52:0/4109129937 conn(0x7f16100fb8b0 msgr2=0x7f1610104930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:39.929 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.922+0000 7f1616989700 1 -- 172.21.15.52:0/4109129937 shutdown_connections 2023-11-27T15:17:39.929 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:39.922+0000 7f1616989700 1 -- 172.21.15.52:0/4109129937 wait complete. 2023-11-27T15:17:39.939 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-27T15:17:40.067 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.063+0000 7f621d2b6700 1 -- 172.21.15.52:0/1091562638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6218100d40 msgr2=0x7f6218107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:40.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.063+0000 7f621d2b6700 1 --2- 172.21.15.52:0/1091562638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6218100d40 0x7f6218107600 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f621000b0a0 tx=0x7f621000b3b0 comp rx=0 tx=0).stop 2023-11-27T15:17:40.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.063+0000 7f621d2b6700 1 -- 172.21.15.52:0/1091562638 shutdown_connections 2023-11-27T15:17:40.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.063+0000 7f621d2b6700 1 --2- 172.21.15.52:0/1091562638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6218100d40 0x7f6218107600 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:40.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.063+0000 7f621d2b6700 1 --2- 172.21.15.52:0/1091562638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62180ff1c0 0x7f62180ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:40.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.063+0000 7f621d2b6700 1 --2- 172.21.15.52:0/1091562638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62181003b0 0x7f6218100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:40.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.063+0000 7f621d2b6700 1 -- 172.21.15.52:0/1091562638 >> 172.21.15.52:0/1091562638 conn(0x7f62180fa7c0 msgr2=0x7f62180fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:40.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.063+0000 7f621d2b6700 1 -- 172.21.15.52:0/1091562638 shutdown_connections 2023-11-27T15:17:40.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.063+0000 7f621d2b6700 1 -- 172.21.15.52:0/1091562638 wait complete. 2023-11-27T15:17:40.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.064+0000 7f621d2b6700 1 Processor -- start 2023-11-27T15:17:40.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.064+0000 7f621d2b6700 1 -- start start 2023-11-27T15:17:40.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.064+0000 7f621d2b6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62180ff1c0 0x7f62181961d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:40.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.064+0000 7f621d2b6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62181003b0 0x7f6218196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:40.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.064+0000 7f621d2b6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6218100d40 0x7f621819d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:40.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.064+0000 7f621d2b6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621819dd50 con 0x7f62181003b0 2023-11-27T15:17:40.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.064+0000 7f621d2b6700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621819de90 con 0x7f62180ff1c0 2023-11-27T15:17:40.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.064+0000 7f621d2b6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621819e020 con 0x7f6218100d40 2023-11-27T15:17:40.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.064+0000 7f62167fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62181003b0 0x7f6218196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:40.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.064+0000 7f62167fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62181003b0 0x7f6218196710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51390/0 (socket says 172.21.15.52:51390) 2023-11-27T15:17:40.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.064+0000 7f62167fc700 1 -- 172.21.15.52:0/437175937 learned_addr learned my addr 172.21.15.52:0/437175937 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:40.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.064+0000 7f6216ffd700 1 --2- 172.21.15.52:0/437175937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62180ff1c0 0x7f62181961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:40.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.064+0000 7f62177fe700 1 --2- 172.21.15.52:0/437175937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6218100d40 0x7f621819d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:40.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.064+0000 7f62167fc700 1 -- 172.21.15.52:0/437175937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6218100d40 msgr2=0x7f621819d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:40.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.064+0000 7f62167fc700 1 --2- 172.21.15.52:0/437175937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6218100d40 0x7f621819d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:40.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.065+0000 7f62167fc700 1 -- 172.21.15.52:0/437175937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62180ff1c0 msgr2=0x7f62181961d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:40.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.065+0000 7f62167fc700 1 --2- 172.21.15.52:0/437175937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62180ff1c0 0x7f62181961d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:40.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.065+0000 7f62167fc700 1 -- 172.21.15.52:0/437175937 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6210009d00 con 0x7f62181003b0 2023-11-27T15:17:40.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.065+0000 7f62167fc700 1 --2- 172.21.15.52:0/437175937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62181003b0 0x7f6218196710 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f620c00d950 tx=0x7f620c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:40.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.065+0000 7f61fffff700 1 -- 172.21.15.52:0/437175937 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c009940 con 0x7f62181003b0 2023-11-27T15:17:40.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.065+0000 7f61fffff700 1 -- 172.21.15.52:0/437175937 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f620c010460 con 0x7f62181003b0 2023-11-27T15:17:40.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.065+0000 7f621d2b6700 1 -- 172.21.15.52:0/437175937 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f621810ab40 con 0x7f62181003b0 2023-11-27T15:17:40.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.065+0000 7f61fffff700 1 -- 172.21.15.52:0/437175937 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c00f620 con 0x7f62181003b0 2023-11-27T15:17:40.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.065+0000 7f621d2b6700 1 -- 172.21.15.52:0/437175937 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f621810a9b0 con 0x7f62181003b0 2023-11-27T15:17:40.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.066+0000 7f621d2b6700 1 -- 172.21.15.52:0/437175937 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f621819e710 con 0x7f62181003b0 2023-11-27T15:17:40.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.067+0000 7f61fffff700 1 -- 172.21.15.52:0/437175937 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f620c0108d0 con 0x7f62181003b0 2023-11-27T15:17:40.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.068+0000 7f61fffff700 1 --2- 172.21.15.52:0/437175937 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f62000a1150 0x7f62000a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:40.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.068+0000 7f6216ffd700 1 --2- 172.21.15.52:0/437175937 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f62000a1150 0x7f62000a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:40.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.068+0000 7f61fffff700 1 -- 172.21.15.52:0/437175937 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f620c0cc780 con 0x7f62181003b0 2023-11-27T15:17:40.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.068+0000 7f6216ffd700 1 --2- 172.21.15.52:0/437175937 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f62000a1150 0x7f62000a3600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6218100210 tx=0x7f620800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:40.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.071+0000 7f61fffff700 1 -- 172.21.15.52:0/437175937 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f620c095e50 con 0x7f62181003b0 2023-11-27T15:17:40.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:39 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1482747392' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:17:40.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:39 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2349031457' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:17:40.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.266+0000 7f621d2b6700 1 -- 172.21.15.52:0/437175937 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f621819ecc0 con 0x7f62181003b0 2023-11-27T15:17:40.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.267+0000 7f61fffff700 1 -- 172.21.15.52:0/437175937 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f620c020020 con 0x7f62181003b0 2023-11-27T15:17:40.271 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2023-11-27T15:17:40.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.269+0000 7f621d2b6700 1 -- 172.21.15.52:0/437175937 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f62000a1150 msgr2=0x7f62000a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:40.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.269+0000 7f621d2b6700 1 --2- 172.21.15.52:0/437175937 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f62000a1150 0x7f62000a3600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6218100210 tx=0x7f620800a380 comp rx=0 tx=0).stop 2023-11-27T15:17:40.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.269+0000 7f621d2b6700 1 -- 172.21.15.52:0/437175937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62181003b0 msgr2=0x7f6218196710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:40.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.269+0000 7f621d2b6700 1 --2- 172.21.15.52:0/437175937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62181003b0 0x7f6218196710 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f620c00d950 tx=0x7f620c00dc60 comp rx=0 tx=0).stop 2023-11-27T15:17:40.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.269+0000 7f621d2b6700 1 -- 172.21.15.52:0/437175937 shutdown_connections 2023-11-27T15:17:40.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.269+0000 7f621d2b6700 1 --2- 172.21.15.52:0/437175937 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f62000a1150 0x7f62000a3600 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:40.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.269+0000 7f621d2b6700 1 --2- 172.21.15.52:0/437175937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62180ff1c0 0x7f62181961d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:40.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.269+0000 7f621d2b6700 1 --2- 172.21.15.52:0/437175937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6218100d40 0x7f621819d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:40.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.269+0000 7f621d2b6700 1 --2- 172.21.15.52:0/437175937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62181003b0 0x7f6218196710 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:40.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.269+0000 7f621d2b6700 1 -- 172.21.15.52:0/437175937 >> 172.21.15.52:0/437175937 conn(0x7f62180fa7c0 msgr2=0x7f6218067dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:40.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.269+0000 7f621d2b6700 1 -- 172.21.15.52:0/437175937 shutdown_connections 2023-11-27T15:17:40.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.269+0000 7f621d2b6700 1 -- 172.21.15.52:0/437175937 wait complete. 2023-11-27T15:17:40.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:39 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1482747392' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:17:40.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:39 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2349031457' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:17:40.286 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:17:40.414 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.408+0000 7f33b1990700 1 -- 172.21.15.52:0/1229866344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33ac1003b0 msgr2=0x7f33ac100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:40.414 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.408+0000 7f33b1990700 1 --2- 172.21.15.52:0/1229866344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33ac1003b0 0x7f33ac100800 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f339c009990 tx=0x7f339c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:40.414 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.408+0000 7f33b1990700 1 -- 172.21.15.52:0/1229866344 shutdown_connections 2023-11-27T15:17:40.414 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.408+0000 7f33b1990700 1 --2- 172.21.15.52:0/1229866344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33ac0ff1c0 0x7f33ac0ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:40.414 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.408+0000 7f33b1990700 1 --2- 172.21.15.52:0/1229866344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33ac100d40 0x7f33ac107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:40.414 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.408+0000 7f33b1990700 1 --2- 172.21.15.52:0/1229866344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33ac1003b0 0x7f33ac100800 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:40.414 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.408+0000 7f33b1990700 1 -- 172.21.15.52:0/1229866344 >> 172.21.15.52:0/1229866344 conn(0x7f33ac0fa7c0 msgr2=0x7f33ac0fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:40.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.408+0000 7f33b1990700 1 -- 172.21.15.52:0/1229866344 shutdown_connections 2023-11-27T15:17:40.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.408+0000 7f33b1990700 1 -- 172.21.15.52:0/1229866344 wait complete. 2023-11-27T15:17:40.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.408+0000 7f33b1990700 1 Processor -- start 2023-11-27T15:17:40.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.409+0000 7f33b1990700 1 -- start start 2023-11-27T15:17:40.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.409+0000 7f33b1990700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33ac0ff1c0 0x7f33ac069b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:40.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.409+0000 7f33b1990700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33ac1003b0 0x7f33ac19acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:40.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.409+0000 7f33b1990700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33ac100d40 0x7f33ac19d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:40.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.409+0000 7f33b1990700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac19dad0 con 0x7f33ac0ff1c0 2023-11-27T15:17:40.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.409+0000 7f33b1990700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac19dc40 con 0x7f33ac100d40 2023-11-27T15:17:40.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.409+0000 7f33b1990700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac19de00 con 0x7f33ac1003b0 2023-11-27T15:17:40.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.409+0000 7f33aaffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33ac0ff1c0 0x7f33ac069b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:40.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.409+0000 7f33aaffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33ac0ff1c0 0x7f33ac069b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51402/0 (socket says 172.21.15.52:51402) 2023-11-27T15:17:40.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.409+0000 7f33aaffd700 1 -- 172.21.15.52:0/627233561 learned_addr learned my addr 172.21.15.52:0/627233561 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:40.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.409+0000 7f33aa7fc700 1 --2- 172.21.15.52:0/627233561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33ac1003b0 0x7f33ac19acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:40.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.409+0000 7f33ab7fe700 1 --2- 172.21.15.52:0/627233561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33ac100d40 0x7f33ac19d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:40.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.409+0000 7f33aaffd700 1 -- 172.21.15.52:0/627233561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33ac1003b0 msgr2=0x7f33ac19acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:40.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.409+0000 7f33aaffd700 1 --2- 172.21.15.52:0/627233561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33ac1003b0 0x7f33ac19acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:40.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.409+0000 7f33aaffd700 1 -- 172.21.15.52:0/627233561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33ac100d40 msgr2=0x7f33ac19d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:40.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.409+0000 7f33aaffd700 1 --2- 172.21.15.52:0/627233561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33ac100d40 0x7f33ac19d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:40.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.409+0000 7f33aaffd700 1 -- 172.21.15.52:0/627233561 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f339c0095f0 con 0x7f33ac0ff1c0 2023-11-27T15:17:40.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.409+0000 7f33ab7fe700 1 --2- 172.21.15.52:0/627233561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33ac100d40 0x7f33ac19d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:17:40.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.409+0000 7f33aaffd700 1 --2- 172.21.15.52:0/627233561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33ac0ff1c0 0x7f33ac069b80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f33a000d980 tx=0x7f33a000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:40.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.410+0000 7f33b098e700 1 -- 172.21.15.52:0/627233561 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a000b950 con 0x7f33ac0ff1c0 2023-11-27T15:17:40.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.410+0000 7f33b1990700 1 -- 172.21.15.52:0/627233561 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33ac10ab30 con 0x7f33ac0ff1c0 2023-11-27T15:17:40.417 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.410+0000 7f33b098e700 1 -- 172.21.15.52:0/627233561 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33a0007620 con 0x7f33ac0ff1c0 2023-11-27T15:17:40.417 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.410+0000 7f33b098e700 1 -- 172.21.15.52:0/627233561 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a000f3c0 con 0x7f33ac0ff1c0 2023-11-27T15:17:40.417 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.410+0000 7f33b1990700 1 -- 172.21.15.52:0/627233561 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33ac10a9a0 con 0x7f33ac0ff1c0 2023-11-27T15:17:40.417 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.411+0000 7f33b1990700 1 -- 172.21.15.52:0/627233561 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33ac106780 con 0x7f33ac0ff1c0 2023-11-27T15:17:40.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.412+0000 7f33b098e700 1 -- 172.21.15.52:0/627233561 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f33a0010640 con 0x7f33ac0ff1c0 2023-11-27T15:17:40.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.413+0000 7f33b098e700 1 --2- 172.21.15.52:0/627233561 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f33940a1150 0x7f33940a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:40.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.413+0000 7f33aa7fc700 1 --2- 172.21.15.52:0/627233561 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f33940a1150 0x7f33940a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:40.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.413+0000 7f33b098e700 1 -- 172.21.15.52:0/627233561 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f33a00ccff0 con 0x7f33ac0ff1c0 2023-11-27T15:17:40.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.413+0000 7f33aa7fc700 1 --2- 172.21.15.52:0/627233561 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f33940a1150 0x7f33940a3600 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f339c00b5c0 tx=0x7f339c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:40.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.416+0000 7f33b098e700 1 -- 172.21.15.52:0/627233561 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f33a0096720 con 0x7f33ac0ff1c0 2023-11-27T15:17:40.622 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.618+0000 7f33b1990700 1 -- 172.21.15.52:0/627233561 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33ac02cfe0 con 0x7f33ac0ff1c0 2023-11-27T15:17:40.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.620+0000 7f33b098e700 1 -- 172.21.15.52:0/627233561 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22657 (secure 0 0 0) 0x7f33a000ee60 con 0x7f33ac0ff1c0 2023-11-27T15:17:40.624 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:17:40.624 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":85,"fsid":"c6c7df88-8d36-11ee-95a2-87774f69a715","created":"2023-11-27T15:09:03.333345+0000","modified":"2023-11-27T15:17:04.797892+0000","last_up_change":"2023-11-27T15:15:27.656134+0000","last_in_change":"2023-11-27T15:15:10.671153+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:12:14.177523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:16:07.842251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:16:08.728520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58d69ede-c293-49f5-abbe-8187793f2b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6803","nonce":608216415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6805","nonce":608216415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6809","nonce":608216415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6807","nonce":608216415}]},"public_addr":"172.21.15.52:6803/608216415","cluster_addr":"172.21.15.52:6805/608216415","heartbeat_back_addr":"172.21.15.52:6809/608216415","heartbeat_front_addr":"172.21.15.52:6807/608216415","state":["exists","up"]},{"osd":1,"uuid":"d6a46b91-adc8-45ef-80f5-566ae400911f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6801","nonce":2538384087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6803","nonce":2538384087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6807","nonce":2538384087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6805","nonce":2538384087}]},"public_addr":"172.21.15.102:6801/2538384087","cluster_addr":"172.21.15.102:6803/2538384087","heartbeat_back_addr":"172.21.15.102:6807/2538384087","heartbeat_front_addr":"172.21.15.102:6805/2538384087","state":["exists","up"]},{"osd":2,"uuid":"846ba125-11f8-46b2-8941-bd5e1947cc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6801","nonce":4258038221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6803","nonce":4258038221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6807","nonce":4258038221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6805","nonce":4258038221}]},"public_addr":"172.21.15.150:6801/4258038221","cluster_addr":"172.21.15.150:6803/4258038221","heartbeat_back_addr":"172.21.15.150:6807/4258038221","heartbeat_front_addr":"172.21.15.150:6805/4258038221","state":["exists","up"]},{"osd":3,"uuid":"950fcd67-1057-4288-a389-714f1ec94261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6811","nonce":2622100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6813","nonce":2622100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6817","nonce":2622100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6815","nonce":2622100752}]},"public_addr":"172.21.15.52:6811/2622100752","cluster_addr":"172.21.15.52:6813/2622100752","heartbeat_back_addr":"172.21.15.52:6817/2622100752","heartbeat_front_addr":"172.21.15.52:6815/2622100752","state":["exists","up"]},{"osd":4,"uuid":"1c96a676-cea9-4deb-bad1-f7fd240c3d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6809","nonce":1546794763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6811","nonce":1546794763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6815","nonce":1546794763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6813","nonce":1546794763}]},"public_addr":"172.21.15.102:6809/1546794763","cluster_addr":"172.21.15.102:6811/1546794763","heartbeat_back_addr":"172.21.15.102:6815/1546794763","heartbeat_front_addr":"172.21.15.102:6813/1546794763","state":["exists","up"]},{"osd":5,"uuid":"38a75af1-fd4d-433e-b579-0609109c851c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6809","nonce":461511375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6811","nonce":461511375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6815","nonce":461511375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6813","nonce":461511375}]},"public_addr":"172.21.15.150:6809/461511375","cluster_addr":"172.21.15.150:6811/461511375","heartbeat_back_addr":"172.21.15.150:6815/461511375","heartbeat_front_addr":"172.21.15.150:6813/461511375","state":["exists","up"]},{"osd":6,"uuid":"2f0ede4a-d175-4b4a-b876-c0f442048901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6819","nonce":1002457464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6821","nonce":1002457464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6825","nonce":1002457464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6823","nonce":1002457464}]},"public_addr":"172.21.15.52:6819/1002457464","cluster_addr":"172.21.15.52:6821/1002457464","heartbeat_back_addr":"172.21.15.52:6825/1002457464","heartbeat_front_addr":"172.21.15.52:6823/1002457464","state":["exists","up"]},{"osd":7,"uuid":"6b635192-abcf-4bbf-bf78-a019b351bca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6817","nonce":3249840064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6819","nonce":3249840064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6823","nonce":3249840064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6821","nonce":3249840064}]},"public_addr":"172.21.15.102:6817/3249840064","cluster_addr":"172.21.15.102:6819/3249840064","heartbeat_back_addr":"172.21.15.102:6823/3249840064","heartbeat_front_addr":"172.21.15.102:6821/3249840064","state":["exists","up"]},{"osd":8,"uuid":"c7ca8007-7170-47bf-956c-3f00edf27e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6817","nonce":3672583574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6819","nonce":3672583574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6823","nonce":3672583574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6821","nonce":3672583574}]},"public_addr":"172.21.15.150:6817/3672583574","cluster_addr":"172.21.15.150:6819/3672583574","heartbeat_back_addr":"172.21.15.150:6823/3672583574","heartbeat_front_addr":"172.21.15.150:6821/3672583574","state":["exists","up"]},{"osd":9,"uuid":"0e252c2c-01a4-48ba-832d-067ed940fefb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6827","nonce":4287934320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6829","nonce":4287934320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6833","nonce":4287934320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6831","nonce":4287934320}]},"public_addr":"172.21.15.52:6827/4287934320","cluster_addr":"172.21.15.52:6829/4287934320","heartbeat_back_addr":"172.21.15.52:6833/4287934320","heartbeat_front_addr":"172.21.15.52:6831/4287934320","state":["exists","up"]},{"osd":10,"uuid":"bb540142-7563-44a4-b7a3-cb662714c128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6825","nonce":3056034201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6827","nonce":3056034201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6831","nonce":3056034201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6829","nonce":3056034201}]},"public_addr":"172.21.15.102:6825/3056034201","cluster_addr":"172.21.15.102:6827/3056034201","heartbeat_back_addr":"172.21.15.102:6831/3056034201","heartbeat_front_addr":"172.21.15.102:6829/3056034201","state":["exists","up"]},{"osd":11,"uuid":"3156528e-e0f8-4d44-9946-c8b6339c2bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6825","nonce":3207559739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6827","nonce":3207559739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6831","nonce":3207559739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6829","nonce":3207559739}]},"public_addr":"172.21.15.150:6825/3207559739","cluster_addr":"172.21.15.150:6827/3207559739","heartbeat_back_addr":"172.21.15.150:6831/3207559739","heartbeat_front_addr":"172.21.15.150:6829/3207559739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:26.826099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:48.331371+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:10.459738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:32.356472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:54.043170+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:15.648582+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:37.166463+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:59.240819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:20.710097+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:43.364563+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:05.279433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:26.344573+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":5,"to":7}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/1392282110":"2023-11-28T15:10:03.644870+0000","172.21.15.52:0/1479993479":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6800/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/961562652":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/1290665390":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2410529742":"2023-11-28T15:09:37.118690+0000","172.21.15.52:6800/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2130037044":"2023-11-28T15:10:03.644870+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:17:40.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.622+0000 7f33b1990700 1 -- 172.21.15.52:0/627233561 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f33940a1150 msgr2=0x7f33940a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:40.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.622+0000 7f33b1990700 1 --2- 172.21.15.52:0/627233561 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f33940a1150 0x7f33940a3600 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f339c00b5c0 tx=0x7f339c005960 comp rx=0 tx=0).stop 2023-11-27T15:17:40.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.622+0000 7f33b1990700 1 -- 172.21.15.52:0/627233561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33ac0ff1c0 msgr2=0x7f33ac069b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:40.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.622+0000 7f33b1990700 1 --2- 172.21.15.52:0/627233561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33ac0ff1c0 0x7f33ac069b80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f33a000d980 tx=0x7f33a000dc90 comp rx=0 tx=0).stop 2023-11-27T15:17:40.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.622+0000 7f33b1990700 1 -- 172.21.15.52:0/627233561 shutdown_connections 2023-11-27T15:17:40.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.622+0000 7f33b1990700 1 --2- 172.21.15.52:0/627233561 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f33940a1150 0x7f33940a3600 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:40.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.622+0000 7f33b1990700 1 --2- 172.21.15.52:0/627233561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33ac100d40 0x7f33ac19d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:40.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.622+0000 7f33b1990700 1 --2- 172.21.15.52:0/627233561 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33ac0ff1c0 0x7f33ac069b80 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:40.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.622+0000 7f33b1990700 1 --2- 172.21.15.52:0/627233561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33ac1003b0 0x7f33ac19acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:40.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.622+0000 7f33b1990700 1 -- 172.21.15.52:0/627233561 >> 172.21.15.52:0/627233561 conn(0x7f33ac0fa7c0 msgr2=0x7f33ac1038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:40.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.622+0000 7f33b1990700 1 -- 172.21.15.52:0/627233561 shutdown_connections 2023-11-27T15:17:40.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.622+0000 7f33b1990700 1 -- 172.21.15.52:0/627233561 wait complete. 2023-11-27T15:17:40.639 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:12:14.177523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-27T15:16:07.842251+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-27T15:16:08.728520+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-27T15:17:40.639 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:17:40.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.764+0000 7f2218c27700 1 -- 172.21.15.52:0/3349624506 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22141002d0 msgr2=0x7f22141006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:40.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.764+0000 7f2218c27700 1 --2- 172.21.15.52:0/3349624506 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22141002d0 0x7f22141006e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2204009990 tx=0x7f2204009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:40.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.764+0000 7f2218c27700 1 -- 172.21.15.52:0/3349624506 shutdown_connections 2023-11-27T15:17:40.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.764+0000 7f2218c27700 1 --2- 172.21.15.52:0/3349624506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2214101e50 0x7f2214108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:40.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.764+0000 7f2218c27700 1 --2- 172.21.15.52:0/3349624506 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22141002d0 0x7f22141006e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:40.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.764+0000 7f2218c27700 1 --2- 172.21.15.52:0/3349624506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22141014c0 0x7f2214101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:40.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.764+0000 7f2218c27700 1 -- 172.21.15.52:0/3349624506 >> 172.21.15.52:0/3349624506 conn(0x7f22140fb8b0 msgr2=0x7f22140fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:40.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.764+0000 7f2218c27700 1 -- 172.21.15.52:0/3349624506 shutdown_connections 2023-11-27T15:17:40.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.764+0000 7f2218c27700 1 -- 172.21.15.52:0/3349624506 wait complete. 2023-11-27T15:17:40.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.764+0000 7f2218c27700 1 Processor -- start 2023-11-27T15:17:40.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.764+0000 7f2218c27700 1 -- start start 2023-11-27T15:17:40.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.765+0000 7f2218c27700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22141002d0 0x7f22141981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:40.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.765+0000 7f2218c27700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22141014c0 0x7f2214198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:40.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.765+0000 7f2218c27700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2214101e50 0x7f221419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:40.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.765+0000 7f2218c27700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221419fd70 con 0x7f22141014c0 2023-11-27T15:17:40.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.765+0000 7f2218c27700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221419feb0 con 0x7f2214101e50 2023-11-27T15:17:40.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.765+0000 7f2218c27700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22141a0040 con 0x7f22141002d0 2023-11-27T15:17:40.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.765+0000 7f2211d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22141014c0 0x7f2214198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:40.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.765+0000 7f2211d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22141014c0 0x7f2214198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51418/0 (socket says 172.21.15.52:51418) 2023-11-27T15:17:40.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.765+0000 7f2211d9b700 1 -- 172.21.15.52:0/1764572379 learned_addr learned my addr 172.21.15.52:0/1764572379 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:40.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.765+0000 7f2212d9d700 1 --2- 172.21.15.52:0/1764572379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2214101e50 0x7f221419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:40.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.765+0000 7f221259c700 1 --2- 172.21.15.52:0/1764572379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22141002d0 0x7f22141981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:40.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.765+0000 7f2211d9b700 1 -- 172.21.15.52:0/1764572379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22141002d0 msgr2=0x7f22141981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:40.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.765+0000 7f2211d9b700 1 --2- 172.21.15.52:0/1764572379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22141002d0 0x7f22141981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:40.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.765+0000 7f2211d9b700 1 -- 172.21.15.52:0/1764572379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2214101e50 msgr2=0x7f221419f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:40.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.765+0000 7f2211d9b700 1 --2- 172.21.15.52:0/1764572379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2214101e50 0x7f221419f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:40.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.765+0000 7f2211d9b700 1 -- 172.21.15.52:0/1764572379 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22040095f0 con 0x7f22141014c0 2023-11-27T15:17:40.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.765+0000 7f2211d9b700 1 --2- 172.21.15.52:0/1764572379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22141014c0 0x7f2214198730 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f220800d950 tx=0x7f220800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:40.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.765+0000 7f22037fe700 1 -- 172.21.15.52:0/1764572379 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220800f890 con 0x7f22141014c0 2023-11-27T15:17:40.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.765+0000 7f22037fe700 1 -- 172.21.15.52:0/1764572379 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2208010460 con 0x7f22141014c0 2023-11-27T15:17:40.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.765+0000 7f2218c27700 1 -- 172.21.15.52:0/1764572379 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f221410bc30 con 0x7f22141014c0 2023-11-27T15:17:40.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.766+0000 7f2218c27700 1 -- 172.21.15.52:0/1764572379 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f221410baa0 con 0x7f22141014c0 2023-11-27T15:17:40.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.766+0000 7f22037fe700 1 -- 172.21.15.52:0/1764572379 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220800e610 con 0x7f22141014c0 2023-11-27T15:17:40.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.766+0000 7f2218c27700 1 -- 172.21.15.52:0/1764572379 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22141a0780 con 0x7f22141014c0 2023-11-27T15:17:40.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.768+0000 7f22037fe700 1 -- 172.21.15.52:0/1764572379 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f22080108d0 con 0x7f22141014c0 2023-11-27T15:17:40.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.769+0000 7f22037fe700 1 --2- 172.21.15.52:0/1764572379 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f21fc0a1150 0x7f21fc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:40.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.769+0000 7f221259c700 1 --2- 172.21.15.52:0/1764572379 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f21fc0a1150 0x7f21fc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:40.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.769+0000 7f22037fe700 1 -- 172.21.15.52:0/1764572379 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f22080ccba0 con 0x7f22141014c0 2023-11-27T15:17:40.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.769+0000 7f221259c700 1 --2- 172.21.15.52:0/1764572379 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f21fc0a1150 0x7f21fc0a3600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f220400b5c0 tx=0x7f2204005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:40.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.772+0000 7f22037fe700 1 -- 172.21.15.52:0/1764572379 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f22080cce60 con 0x7f22141014c0 2023-11-27T15:17:40.975 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.971+0000 7f2218c27700 1 -- 172.21.15.52:0/1764572379 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f22141a0d30 con 0x7f22141014c0 2023-11-27T15:17:40.975 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.971+0000 7f22037fe700 1 -- 172.21.15.52:0/1764572379 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f22080160a0 con 0x7f22141014c0 2023-11-27T15:17:40.976 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2023-11-27T15:17:40.977 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.973+0000 7f2218c27700 1 -- 172.21.15.52:0/1764572379 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f21fc0a1150 msgr2=0x7f21fc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:40.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.973+0000 7f2218c27700 1 --2- 172.21.15.52:0/1764572379 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f21fc0a1150 0x7f21fc0a3600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f220400b5c0 tx=0x7f2204005960 comp rx=0 tx=0).stop 2023-11-27T15:17:40.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.973+0000 7f2218c27700 1 -- 172.21.15.52:0/1764572379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22141014c0 msgr2=0x7f2214198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:40.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.973+0000 7f2218c27700 1 --2- 172.21.15.52:0/1764572379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22141014c0 0x7f2214198730 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f220800d950 tx=0x7f220800dc60 comp rx=0 tx=0).stop 2023-11-27T15:17:40.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.973+0000 7f2218c27700 1 -- 172.21.15.52:0/1764572379 shutdown_connections 2023-11-27T15:17:40.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.973+0000 7f2218c27700 1 --2- 172.21.15.52:0/1764572379 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f21fc0a1150 0x7f21fc0a3600 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:40.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.973+0000 7f2218c27700 1 --2- 172.21.15.52:0/1764572379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2214101e50 0x7f221419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:40.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.973+0000 7f2218c27700 1 --2- 172.21.15.52:0/1764572379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22141002d0 0x7f22141981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:40.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.973+0000 7f2218c27700 1 --2- 172.21.15.52:0/1764572379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22141014c0 0x7f2214198730 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:40.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.973+0000 7f2218c27700 1 -- 172.21.15.52:0/1764572379 >> 172.21.15.52:0/1764572379 conn(0x7f22140fb8b0 msgr2=0x7f22140fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:40.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.973+0000 7f2218c27700 1 -- 172.21.15.52:0/1764572379 shutdown_connections 2023-11-27T15:17:40.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:40.973+0000 7f2218c27700 1 -- 172.21.15.52:0/1764572379 wait complete. 2023-11-27T15:17:40.989 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:40 smithi052 ceph-mon[99056]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2023-11-27T15:17:40.989 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:40 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/4109129937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:17:40.989 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:40 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/437175937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:17:40.989 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:40 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/627233561' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:17:40.990 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-27T15:17:41.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:40 smithi102 ceph-mon[100296]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2023-11-27T15:17:41.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:40 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/4109129937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:17:41.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:40 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/437175937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:17:41.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:40 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/627233561' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:17:41.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.114+0000 7f0d64b4b700 1 -- 172.21.15.52:0/1226474351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d601002d0 msgr2=0x7f0d601006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:41.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.114+0000 7f0d64b4b700 1 --2- 172.21.15.52:0/1226474351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d601002d0 0x7f0d601006e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0d54009990 tx=0x7f0d54009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.114+0000 7f0d64b4b700 1 -- 172.21.15.52:0/1226474351 shutdown_connections 2023-11-27T15:17:41.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.114+0000 7f0d64b4b700 1 --2- 172.21.15.52:0/1226474351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d60101e50 0x7f0d60108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.114+0000 7f0d64b4b700 1 --2- 172.21.15.52:0/1226474351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d601002d0 0x7f0d601006e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.114+0000 7f0d64b4b700 1 --2- 172.21.15.52:0/1226474351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d601014c0 0x7f0d60101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.114+0000 7f0d64b4b700 1 -- 172.21.15.52:0/1226474351 >> 172.21.15.52:0/1226474351 conn(0x7f0d600fb8d0 msgr2=0x7f0d600fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:41.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.114+0000 7f0d64b4b700 1 -- 172.21.15.52:0/1226474351 shutdown_connections 2023-11-27T15:17:41.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.114+0000 7f0d64b4b700 1 -- 172.21.15.52:0/1226474351 wait complete. 2023-11-27T15:17:41.121 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.115+0000 7f0d64b4b700 1 Processor -- start 2023-11-27T15:17:41.121 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.115+0000 7f0d64b4b700 1 -- start start 2023-11-27T15:17:41.121 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.115+0000 7f0d64b4b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d601002d0 0x7f0d60196080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:41.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.115+0000 7f0d64b4b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d601014c0 0x7f0d601965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:41.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.115+0000 7f0d64b4b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d60101e50 0x7f0d6019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:41.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.115+0000 7f0d64b4b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6019dc00 con 0x7f0d601002d0 2023-11-27T15:17:41.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.115+0000 7f0d64b4b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6019dd40 con 0x7f0d601014c0 2023-11-27T15:17:41.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.115+0000 7f0d64b4b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6019ded0 con 0x7f0d60101e50 2023-11-27T15:17:41.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.115+0000 7f0d5e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d601002d0 0x7f0d60196080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:41.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.115+0000 7f0d5e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d601002d0 0x7f0d60196080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51430/0 (socket says 172.21.15.52:51430) 2023-11-27T15:17:41.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.115+0000 7f0d5e59c700 1 -- 172.21.15.52:0/932086727 learned_addr learned my addr 172.21.15.52:0/932086727 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:41.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.115+0000 7f0d5ed9d700 1 --2- 172.21.15.52:0/932086727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d60101e50 0x7f0d6019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:41.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.116+0000 7f0d5dd9b700 1 --2- 172.21.15.52:0/932086727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d601014c0 0x7f0d601965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:41.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.116+0000 7f0d5e59c700 1 -- 172.21.15.52:0/932086727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d60101e50 msgr2=0x7f0d6019d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:41.123 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.116+0000 7f0d5e59c700 1 --2- 172.21.15.52:0/932086727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d60101e50 0x7f0d6019d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.123 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.116+0000 7f0d5e59c700 1 -- 172.21.15.52:0/932086727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d601014c0 msgr2=0x7f0d601965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:41.123 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.116+0000 7f0d5e59c700 1 --2- 172.21.15.52:0/932086727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d601014c0 0x7f0d601965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.123 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.116+0000 7f0d5e59c700 1 -- 172.21.15.52:0/932086727 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d540095f0 con 0x7f0d601002d0 2023-11-27T15:17:41.123 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.116+0000 7f0d5e59c700 1 --2- 172.21.15.52:0/932086727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d601002d0 0x7f0d60196080 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0d540059d0 tx=0x7f0d54005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:41.123 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.116+0000 7f0d4f7fe700 1 -- 172.21.15.52:0/932086727 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5401d070 con 0x7f0d601002d0 2023-11-27T15:17:41.123 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.116+0000 7f0d4f7fe700 1 -- 172.21.15.52:0/932086727 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d54004020 con 0x7f0d601002d0 2023-11-27T15:17:41.123 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.116+0000 7f0d64b4b700 1 -- 172.21.15.52:0/932086727 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d6010bc30 con 0x7f0d601002d0 2023-11-27T15:17:41.123 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.116+0000 7f0d4f7fe700 1 -- 172.21.15.52:0/932086727 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d54022d00 con 0x7f0d601002d0 2023-11-27T15:17:41.123 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.116+0000 7f0d64b4b700 1 -- 172.21.15.52:0/932086727 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d6010baa0 con 0x7f0d601002d0 2023-11-27T15:17:41.123 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.117+0000 7f0d64b4b700 1 -- 172.21.15.52:0/932086727 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d60107810 con 0x7f0d601002d0 2023-11-27T15:17:41.124 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.118+0000 7f0d4f7fe700 1 -- 172.21.15.52:0/932086727 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0d54022470 con 0x7f0d601002d0 2023-11-27T15:17:41.124 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.119+0000 7f0d4f7fe700 1 --2- 172.21.15.52:0/932086727 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0d480a11d0 0x7f0d480a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:41.124 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.119+0000 7f0d5dd9b700 1 --2- 172.21.15.52:0/932086727 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0d480a11d0 0x7f0d480a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:41.124 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.119+0000 7f0d4f7fe700 1 -- 172.21.15.52:0/932086727 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0d540ce450 con 0x7f0d601002d0 2023-11-27T15:17:41.124 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.119+0000 7f0d5dd9b700 1 --2- 172.21.15.52:0/932086727 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0d480a11d0 0x7f0d480a3680 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0d50005fd0 tx=0x7f0d5000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:41.126 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.122+0000 7f0d4f7fe700 1 -- 172.21.15.52:0/932086727 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0d54097bb0 con 0x7f0d601002d0 2023-11-27T15:17:41.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:40 smithi150 ceph-mon[100559]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2023-11-27T15:17:41.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:40 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/4109129937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:17:41.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:40 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/437175937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:17:41.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:40 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/627233561' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:17:41.323 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.319+0000 7f0d64b4b700 1 -- 172.21.15.52:0/932086727 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0d60061700 con 0x7f0d601002d0 2023-11-27T15:17:41.323 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.319+0000 7f0d4f7fe700 1 -- 172.21.15.52:0/932086727 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0d54022e60 con 0x7f0d601002d0 2023-11-27T15:17:41.323 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2023-11-27T15:17:41.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.321+0000 7f0d64b4b700 1 -- 172.21.15.52:0/932086727 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0d480a11d0 msgr2=0x7f0d480a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:41.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.321+0000 7f0d64b4b700 1 --2- 172.21.15.52:0/932086727 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0d480a11d0 0x7f0d480a3680 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0d50005fd0 tx=0x7f0d5000a560 comp rx=0 tx=0).stop 2023-11-27T15:17:41.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.321+0000 7f0d64b4b700 1 -- 172.21.15.52:0/932086727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d601002d0 msgr2=0x7f0d60196080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:41.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.321+0000 7f0d64b4b700 1 --2- 172.21.15.52:0/932086727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d601002d0 0x7f0d60196080 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0d540059d0 tx=0x7f0d54005740 comp rx=0 tx=0).stop 2023-11-27T15:17:41.327 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.321+0000 7f0d64b4b700 1 -- 172.21.15.52:0/932086727 shutdown_connections 2023-11-27T15:17:41.327 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.321+0000 7f0d64b4b700 1 --2- 172.21.15.52:0/932086727 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0d480a11d0 0x7f0d480a3680 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.327 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.321+0000 7f0d64b4b700 1 --2- 172.21.15.52:0/932086727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d60101e50 0x7f0d6019d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.327 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.321+0000 7f0d64b4b700 1 --2- 172.21.15.52:0/932086727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d601002d0 0x7f0d60196080 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.327 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.321+0000 7f0d64b4b700 1 --2- 172.21.15.52:0/932086727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d601014c0 0x7f0d601965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.327 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.321+0000 7f0d64b4b700 1 -- 172.21.15.52:0/932086727 >> 172.21.15.52:0/932086727 conn(0x7f0d600fb8d0 msgr2=0x7f0d60104950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:41.327 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.322+0000 7f0d64b4b700 1 -- 172.21.15.52:0/932086727 shutdown_connections 2023-11-27T15:17:41.327 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.322+0000 7f0d64b4b700 1 -- 172.21.15.52:0/932086727 wait complete. 2023-11-27T15:17:41.338 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-27T15:17:41.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.462+0000 7f3b44952700 1 -- 172.21.15.52:0/1221958752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b400ff1f0 msgr2=0x7f3b400ff600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:41.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.462+0000 7f3b44952700 1 --2- 172.21.15.52:0/1221958752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b400ff1f0 0x7f3b400ff600 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009990 tx=0x7f3b30009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.462+0000 7f3b44952700 1 -- 172.21.15.52:0/1221958752 shutdown_connections 2023-11-27T15:17:41.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.462+0000 7f3b44952700 1 --2- 172.21.15.52:0/1221958752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b40100d70 0x7f3b40107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.462+0000 7f3b44952700 1 --2- 172.21.15.52:0/1221958752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b400ff1f0 0x7f3b400ff600 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.462+0000 7f3b44952700 1 --2- 172.21.15.52:0/1221958752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b401003e0 0x7f3b40100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.462+0000 7f3b44952700 1 -- 172.21.15.52:0/1221958752 >> 172.21.15.52:0/1221958752 conn(0x7f3b400fa810 msgr2=0x7f3b400fcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:41.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.462+0000 7f3b44952700 1 -- 172.21.15.52:0/1221958752 shutdown_connections 2023-11-27T15:17:41.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.463+0000 7f3b44952700 1 -- 172.21.15.52:0/1221958752 wait complete. 2023-11-27T15:17:41.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.463+0000 7f3b44952700 1 Processor -- start 2023-11-27T15:17:41.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.463+0000 7f3b44952700 1 -- start start 2023-11-27T15:17:41.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.463+0000 7f3b44952700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b400ff1f0 0x7f3b401034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:41.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.463+0000 7f3b44952700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b401003e0 0x7f3b40101af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:41.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.463+0000 7f3b44952700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b40100d70 0x7f3b40102030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:41.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.463+0000 7f3b44952700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b401027d0 con 0x7f3b401003e0 2023-11-27T15:17:41.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.463+0000 7f3b44952700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b40102940 con 0x7f3b400ff1f0 2023-11-27T15:17:41.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.463+0000 7f3b44952700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b401a00c0 con 0x7f3b40100d70 2023-11-27T15:17:41.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.464+0000 7f3b3dd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b401003e0 0x7f3b40101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:41.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.464+0000 7f3b3dd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b401003e0 0x7f3b40101af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51446/0 (socket says 172.21.15.52:51446) 2023-11-27T15:17:41.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.464+0000 7f3b3dd9b700 1 -- 172.21.15.52:0/2582010134 learned_addr learned my addr 172.21.15.52:0/2582010134 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:41.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.464+0000 7f3b3e59c700 1 --2- 172.21.15.52:0/2582010134 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b400ff1f0 0x7f3b401034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:41.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.464+0000 7f3b3ed9d700 1 --2- 172.21.15.52:0/2582010134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b40100d70 0x7f3b40102030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:41.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.464+0000 7f3b3dd9b700 1 -- 172.21.15.52:0/2582010134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b40100d70 msgr2=0x7f3b40102030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:41.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.464+0000 7f3b3dd9b700 1 --2- 172.21.15.52:0/2582010134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b40100d70 0x7f3b40102030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.464+0000 7f3b3dd9b700 1 -- 172.21.15.52:0/2582010134 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b400ff1f0 msgr2=0x7f3b401034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:41.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.464+0000 7f3b3dd9b700 1 --2- 172.21.15.52:0/2582010134 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b400ff1f0 0x7f3b401034b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.464+0000 7f3b3dd9b700 1 -- 172.21.15.52:0/2582010134 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b300095f0 con 0x7f3b401003e0 2023-11-27T15:17:41.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.464+0000 7f3b3ed9d700 1 --2- 172.21.15.52:0/2582010134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b40100d70 0x7f3b40102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:17:41.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.464+0000 7f3b3dd9b700 1 --2- 172.21.15.52:0/2582010134 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b401003e0 0x7f3b40101af0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3b2800e9a0 tx=0x7f3b2800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:41.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.464+0000 7f3b377fe700 1 -- 172.21.15.52:0/2582010134 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b28009790 con 0x7f3b401003e0 2023-11-27T15:17:41.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.464+0000 7f3b377fe700 1 -- 172.21.15.52:0/2582010134 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b2800cbd0 con 0x7f3b401003e0 2023-11-27T15:17:41.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.464+0000 7f3b44952700 1 -- 172.21.15.52:0/2582010134 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b4010ab60 con 0x7f3b401003e0 2023-11-27T15:17:41.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.464+0000 7f3b377fe700 1 -- 172.21.15.52:0/2582010134 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b28018680 con 0x7f3b401003e0 2023-11-27T15:17:41.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.464+0000 7f3b44952700 1 -- 172.21.15.52:0/2582010134 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b4010a9d0 con 0x7f3b401003e0 2023-11-27T15:17:41.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.465+0000 7f3b44952700 1 -- 172.21.15.52:0/2582010134 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b401072f0 con 0x7f3b401003e0 2023-11-27T15:17:41.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.467+0000 7f3b377fe700 1 -- 172.21.15.52:0/2582010134 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3b2800cd40 con 0x7f3b401003e0 2023-11-27T15:17:41.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.468+0000 7f3b377fe700 1 --2- 172.21.15.52:0/2582010134 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f3b2c0a1150 0x7f3b2c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:41.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.468+0000 7f3b377fe700 1 -- 172.21.15.52:0/2582010134 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3b28014070 con 0x7f3b401003e0 2023-11-27T15:17:41.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.468+0000 7f3b3e59c700 1 --2- 172.21.15.52:0/2582010134 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f3b2c0a1150 0x7f3b2c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:41.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.468+0000 7f3b3e59c700 1 --2- 172.21.15.52:0/2582010134 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f3b2c0a1150 0x7f3b2c0a3600 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3b30006010 tx=0x7f3b3000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:41.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.471+0000 7f3b377fe700 1 -- 172.21.15.52:0/2582010134 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3b28096850 con 0x7f3b401003e0 2023-11-27T15:17:41.676 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.671+0000 7f3b44952700 1 -- 172.21.15.52:0/2582010134 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3b40061700 con 0x7f3b401003e0 2023-11-27T15:17:41.676 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.672+0000 7f3b377fe700 1 -- 172.21.15.52:0/2582010134 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3b28099e70 con 0x7f3b401003e0 2023-11-27T15:17:41.676 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2023-11-27T15:17:41.678 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.674+0000 7f3b44952700 1 -- 172.21.15.52:0/2582010134 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f3b2c0a1150 msgr2=0x7f3b2c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:41.678 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.674+0000 7f3b44952700 1 --2- 172.21.15.52:0/2582010134 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f3b2c0a1150 0x7f3b2c0a3600 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3b30006010 tx=0x7f3b3000b260 comp rx=0 tx=0).stop 2023-11-27T15:17:41.678 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.674+0000 7f3b44952700 1 -- 172.21.15.52:0/2582010134 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b401003e0 msgr2=0x7f3b40101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:41.679 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.674+0000 7f3b44952700 1 --2- 172.21.15.52:0/2582010134 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b401003e0 0x7f3b40101af0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3b2800e9a0 tx=0x7f3b2800ed60 comp rx=0 tx=0).stop 2023-11-27T15:17:41.679 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.674+0000 7f3b44952700 1 -- 172.21.15.52:0/2582010134 shutdown_connections 2023-11-27T15:17:41.679 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.674+0000 7f3b44952700 1 --2- 172.21.15.52:0/2582010134 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f3b2c0a1150 0x7f3b2c0a3600 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.679 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.674+0000 7f3b44952700 1 --2- 172.21.15.52:0/2582010134 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3b400ff1f0 0x7f3b401034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.679 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.674+0000 7f3b44952700 1 --2- 172.21.15.52:0/2582010134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3b40100d70 0x7f3b40102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.679 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.674+0000 7f3b44952700 1 --2- 172.21.15.52:0/2582010134 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b401003e0 0x7f3b40101af0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.679 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.674+0000 7f3b44952700 1 -- 172.21.15.52:0/2582010134 >> 172.21.15.52:0/2582010134 conn(0x7f3b400fa810 msgr2=0x7f3b400fca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:41.679 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.674+0000 7f3b44952700 1 -- 172.21.15.52:0/2582010134 shutdown_connections 2023-11-27T15:17:41.679 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.674+0000 7f3b44952700 1 -- 172.21.15.52:0/2582010134 wait complete. 2023-11-27T15:17:41.691 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-11-27T15:17:41.691 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:17:41.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.813+0000 7fb2346d8700 1 -- 172.21.15.52:0/3563078382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb22c0ff170 msgr2=0x7fb22c0ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:41.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.813+0000 7fb2346d8700 1 --2- 172.21.15.52:0/3563078382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb22c0ff170 0x7fb22c0ff580 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb228009990 tx=0x7fb228009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.813+0000 7fb2346d8700 1 -- 172.21.15.52:0/3563078382 shutdown_connections 2023-11-27T15:17:41.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.813+0000 7fb2346d8700 1 --2- 172.21.15.52:0/3563078382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb22c100db0 0x7fb22c107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.813+0000 7fb2346d8700 1 --2- 172.21.15.52:0/3563078382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb22c0ff170 0x7fb22c0ff580 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.813+0000 7fb2346d8700 1 --2- 172.21.15.52:0/3563078382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb22c100400 0x7fb22c100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.813+0000 7fb2346d8700 1 -- 172.21.15.52:0/3563078382 >> 172.21.15.52:0/3563078382 conn(0x7fb22c0fa750 msgr2=0x7fb22c0fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:41.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.813+0000 7fb2346d8700 1 -- 172.21.15.52:0/3563078382 shutdown_connections 2023-11-27T15:17:41.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.813+0000 7fb2346d8700 1 -- 172.21.15.52:0/3563078382 wait complete. 2023-11-27T15:17:41.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.813+0000 7fb2346d8700 1 Processor -- start 2023-11-27T15:17:41.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.813+0000 7fb2346d8700 1 -- start start 2023-11-27T15:17:41.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.814+0000 7fb2346d8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb22c0ff170 0x7fb22c195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:41.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.814+0000 7fb2346d8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb22c100400 0x7fb22c1964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:41.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.814+0000 7fb2346d8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb22c100db0 0x7fb22c19d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:41.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.814+0000 7fb2346d8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c19db30 con 0x7fb22c100400 2023-11-27T15:17:41.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.814+0000 7fb2346d8700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c19dc70 con 0x7fb22c0ff170 2023-11-27T15:17:41.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.814+0000 7fb2346d8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c19de00 con 0x7fb22c100db0 2023-11-27T15:17:41.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.814+0000 7fb231c73700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb22c100400 0x7fb22c1964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:41.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.814+0000 7fb231c73700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb22c100400 0x7fb22c1964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51460/0 (socket says 172.21.15.52:51460) 2023-11-27T15:17:41.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.814+0000 7fb231c73700 1 -- 172.21.15.52:0/3372519384 learned_addr learned my addr 172.21.15.52:0/3372519384 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:41.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.814+0000 7fb232474700 1 --2- 172.21.15.52:0/3372519384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb22c0ff170 0x7fb22c195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:41.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.814+0000 7fb232c75700 1 --2- 172.21.15.52:0/3372519384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb22c100db0 0x7fb22c19d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:41.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.814+0000 7fb231c73700 1 -- 172.21.15.52:0/3372519384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb22c100db0 msgr2=0x7fb22c19d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:41.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.814+0000 7fb231c73700 1 --2- 172.21.15.52:0/3372519384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb22c100db0 0x7fb22c19d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.814+0000 7fb231c73700 1 -- 172.21.15.52:0/3372519384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb22c0ff170 msgr2=0x7fb22c195fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:41.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.814+0000 7fb231c73700 1 --2- 172.21.15.52:0/3372519384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb22c0ff170 0x7fb22c195fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.814+0000 7fb231c73700 1 -- 172.21.15.52:0/3372519384 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2280095f0 con 0x7fb22c100400 2023-11-27T15:17:41.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.814+0000 7fb232c75700 1 --2- 172.21.15.52:0/3372519384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb22c100db0 0x7fb22c19d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:17:41.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.814+0000 7fb231c73700 1 --2- 172.21.15.52:0/3372519384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb22c100400 0x7fb22c1964f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb21c00d950 tx=0x7fb21c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:41.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.814+0000 7fb2237fe700 1 -- 172.21.15.52:0/3372519384 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c009940 con 0x7fb22c100400 2023-11-27T15:17:41.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.814+0000 7fb2237fe700 1 -- 172.21.15.52:0/3372519384 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb21c010460 con 0x7fb22c100400 2023-11-27T15:17:41.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.814+0000 7fb2346d8700 1 -- 172.21.15.52:0/3372519384 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb22c10ab60 con 0x7fb22c100400 2023-11-27T15:17:41.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.814+0000 7fb2237fe700 1 -- 172.21.15.52:0/3372519384 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c00f620 con 0x7fb22c100400 2023-11-27T15:17:41.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.814+0000 7fb2346d8700 1 -- 172.21.15.52:0/3372519384 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb22c10a9d0 con 0x7fb22c100400 2023-11-27T15:17:41.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.815+0000 7fb2346d8700 1 -- 172.21.15.52:0/3372519384 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb22c107310 con 0x7fb22c100400 2023-11-27T15:17:41.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.817+0000 7fb2237fe700 1 -- 172.21.15.52:0/3372519384 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb21c0108d0 con 0x7fb22c100400 2023-11-27T15:17:41.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.818+0000 7fb2237fe700 1 --2- 172.21.15.52:0/3372519384 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb2180a11d0 0x7fb2180a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:41.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.818+0000 7fb232474700 1 --2- 172.21.15.52:0/3372519384 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb2180a11d0 0x7fb2180a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:41.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.818+0000 7fb2237fe700 1 -- 172.21.15.52:0/3372519384 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb21c0cc790 con 0x7fb22c100400 2023-11-27T15:17:41.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.818+0000 7fb232474700 1 --2- 172.21.15.52:0/3372519384 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb2180a11d0 0x7fb2180a3680 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb22800b2e0 tx=0x7fb228005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:41.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:41.821+0000 7fb2237fe700 1 -- 172.21.15.52:0/3372519384 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb21c095e60 con 0x7fb22c100400 2023-11-27T15:17:42.067 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.062+0000 7fb2346d8700 1 -- 172.21.15.52:0/3372519384 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb22c061700 con 0x7fb22c100400 2023-11-27T15:17:42.067 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.063+0000 7fb2237fe700 1 -- 172.21.15.52:0/3372519384 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10804 (secure 0 0 0) 0x7fb21c020090 con 0x7fb22c100400 2023-11-27T15:17:42.067 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:17:42.068 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2328881376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2328881376},{"type":"v1","addr":"172.21.15.52:6839","nonce":2328881376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2799551570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2799551570},{"type":"v1","addr":"172.21.15.52:6841","nonce":2799551570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:16:09.319630+0000","modified":"2023-11-27T15:16:14.742405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24494,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/215725838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":215725838},{"type":"v1","addr":"172.21.15.150:6839","nonce":215725838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3571830559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3571830559},{"type":"v1","addr":"172.21.15.102:6839","nonce":3571830559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:17:42.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.065+0000 7fb2346d8700 1 -- 172.21.15.52:0/3372519384 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb2180a11d0 msgr2=0x7fb2180a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:42.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.065+0000 7fb2346d8700 1 --2- 172.21.15.52:0/3372519384 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb2180a11d0 0x7fb2180a3680 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb22800b2e0 tx=0x7fb228005fb0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.065+0000 7fb2346d8700 1 -- 172.21.15.52:0/3372519384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb22c100400 msgr2=0x7fb22c1964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:42.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.065+0000 7fb2346d8700 1 --2- 172.21.15.52:0/3372519384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb22c100400 0x7fb22c1964f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb21c00d950 tx=0x7fb21c00dc60 comp rx=0 tx=0).stop 2023-11-27T15:17:42.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.065+0000 7fb2346d8700 1 -- 172.21.15.52:0/3372519384 shutdown_connections 2023-11-27T15:17:42.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.065+0000 7fb2346d8700 1 --2- 172.21.15.52:0/3372519384 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fb2180a11d0 0x7fb2180a3680 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.065+0000 7fb2346d8700 1 --2- 172.21.15.52:0/3372519384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb22c0ff170 0x7fb22c195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.065+0000 7fb2346d8700 1 --2- 172.21.15.52:0/3372519384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb22c100db0 0x7fb22c19d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.065+0000 7fb2346d8700 1 --2- 172.21.15.52:0/3372519384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb22c100400 0x7fb22c1964f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.065+0000 7fb2346d8700 1 -- 172.21.15.52:0/3372519384 >> 172.21.15.52:0/3372519384 conn(0x7fb22c0fa750 msgr2=0x7fb22c0fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:42.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.065+0000 7fb2346d8700 1 -- 172.21.15.52:0/3372519384 shutdown_connections 2023-11-27T15:17:42.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.065+0000 7fb2346d8700 1 -- 172.21.15.52:0/3372519384 wait complete. 2023-11-27T15:17:42.071 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 23 2023-11-27T15:17:42.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:41 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1764572379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:17:42.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:41 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/932086727' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:17:42.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:41 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2582010134' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:17:42.080 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:41 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1764572379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:17:42.081 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:41 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/932086727' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:17:42.081 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:41 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2582010134' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:17:42.082 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-11-27T15:17:42.082 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-11-27T15:17:42.082 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-27T15:17:42.210 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.204+0000 7faa54ea0700 1 -- 172.21.15.52:0/2516934807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa50101480 msgr2=0x7faa501018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:42.210 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.204+0000 7faa54ea0700 1 --2- 172.21.15.52:0/2516934807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa50101480 0x7faa501018d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7faa44009990 tx=0x7faa44009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.211 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.204+0000 7faa54ea0700 1 -- 172.21.15.52:0/2516934807 shutdown_connections 2023-11-27T15:17:42.211 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.204+0000 7faa54ea0700 1 --2- 172.21.15.52:0/2516934807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa50100290 0x7faa501006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.211 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.204+0000 7faa54ea0700 1 --2- 172.21.15.52:0/2516934807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa50101e10 0x7faa501086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.211 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.204+0000 7faa54ea0700 1 --2- 172.21.15.52:0/2516934807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa50101480 0x7faa501018d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.211 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.204+0000 7faa54ea0700 1 -- 172.21.15.52:0/2516934807 >> 172.21.15.52:0/2516934807 conn(0x7faa500fb870 msgr2=0x7faa500fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:42.211 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.204+0000 7faa54ea0700 1 -- 172.21.15.52:0/2516934807 shutdown_connections 2023-11-27T15:17:42.211 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.204+0000 7faa54ea0700 1 -- 172.21.15.52:0/2516934807 wait complete. 2023-11-27T15:17:42.211 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.205+0000 7faa54ea0700 1 Processor -- start 2023-11-27T15:17:42.211 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.205+0000 7faa54ea0700 1 -- start start 2023-11-27T15:17:42.211 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.205+0000 7faa54ea0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa50100290 0x7faa50195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:42.211 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.205+0000 7faa54ea0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa50101480 0x7faa501964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:42.212 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.205+0000 7faa54ea0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa50101e10 0x7faa5019d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:42.212 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.205+0000 7faa54ea0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5019db20 con 0x7faa50101e10 2023-11-27T15:17:42.212 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.205+0000 7faa54ea0700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5019dc60 con 0x7faa50101480 2023-11-27T15:17:42.212 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.205+0000 7faa54ea0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5019ddf0 con 0x7faa50100290 2023-11-27T15:17:42.212 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.205+0000 7faa4ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa50101e10 0x7faa5019d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:42.212 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.205+0000 7faa4ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa50101e10 0x7faa5019d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51476/0 (socket says 172.21.15.52:51476) 2023-11-27T15:17:42.212 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.205+0000 7faa4ed9d700 1 -- 172.21.15.52:0/972915198 learned_addr learned my addr 172.21.15.52:0/972915198 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:42.212 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.205+0000 7faa4e59c700 1 --2- 172.21.15.52:0/972915198 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa50100290 0x7faa50195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:42.212 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.205+0000 7faa4dd9b700 1 --2- 172.21.15.52:0/972915198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa50101480 0x7faa501964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:42.212 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.205+0000 7faa4ed9d700 1 -- 172.21.15.52:0/972915198 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa50100290 msgr2=0x7faa50195fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:42.212 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.205+0000 7faa4ed9d700 1 --2- 172.21.15.52:0/972915198 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa50100290 0x7faa50195fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.212 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.205+0000 7faa4ed9d700 1 -- 172.21.15.52:0/972915198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa50101480 msgr2=0x7faa501964e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:42.213 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.205+0000 7faa4ed9d700 1 --2- 172.21.15.52:0/972915198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa50101480 0x7faa501964e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.213 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.205+0000 7faa4ed9d700 1 -- 172.21.15.52:0/972915198 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa440095f0 con 0x7faa50101e10 2023-11-27T15:17:42.213 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.205+0000 7faa4dd9b700 1 --2- 172.21.15.52:0/972915198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa50101480 0x7faa501964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:17:42.213 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.205+0000 7faa4ed9d700 1 --2- 172.21.15.52:0/972915198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa50101e10 0x7faa5019d550 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7faa4800ef30 tx=0x7faa4800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:42.213 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.206+0000 7faa3f7fe700 1 -- 172.21.15.52:0/972915198 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa48018070 con 0x7faa50101e10 2023-11-27T15:17:42.213 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.206+0000 7faa3f7fe700 1 -- 172.21.15.52:0/972915198 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa4800f040 con 0x7faa50101e10 2023-11-27T15:17:42.213 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.206+0000 7faa54ea0700 1 -- 172.21.15.52:0/972915198 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa5010bc50 con 0x7faa50101e10 2023-11-27T15:17:42.213 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.206+0000 7faa3f7fe700 1 -- 172.21.15.52:0/972915198 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4800cdb0 con 0x7faa50101e10 2023-11-27T15:17:42.213 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.206+0000 7faa54ea0700 1 -- 172.21.15.52:0/972915198 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa5010bac0 con 0x7faa50101e10 2023-11-27T15:17:42.214 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.207+0000 7faa54ea0700 1 -- 172.21.15.52:0/972915198 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa5019e4d0 con 0x7faa50101e10 2023-11-27T15:17:42.214 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.208+0000 7faa3f7fe700 1 -- 172.21.15.52:0/972915198 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7faa48003a00 con 0x7faa50101e10 2023-11-27T15:17:42.214 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.209+0000 7faa3f7fe700 1 --2- 172.21.15.52:0/972915198 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7faa380a11d0 0x7faa380a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:42.215 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.209+0000 7faa4e59c700 1 --2- 172.21.15.52:0/972915198 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7faa380a11d0 0x7faa380a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:42.215 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.209+0000 7faa3f7fe700 1 -- 172.21.15.52:0/972915198 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7faa480ccde0 con 0x7faa50101e10 2023-11-27T15:17:42.215 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.209+0000 7faa4e59c700 1 --2- 172.21.15.52:0/972915198 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7faa380a11d0 0x7faa380a3680 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faa501012e0 tx=0x7faa40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:42.216 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.212+0000 7faa3f7fe700 1 -- 172.21.15.52:0/972915198 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7faa480965a0 con 0x7faa50101e10 2023-11-27T15:17:42.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:41 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1764572379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:17:42.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:41 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/932086727' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:17:42.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:41 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2582010134' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:17:42.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.409+0000 7faa54ea0700 1 -- 172.21.15.52:0/972915198 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faa50060f30 con 0x7faa380a11d0 2023-11-27T15:17:42.425 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.421+0000 7faa3f7fe700 1 -- 172.21.15.52:0/972915198 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344013 (secure 0 0 0) 0x7faa50060f30 con 0x7faa380a11d0 2023-11-27T15:17:42.430 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:17:42.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.425+0000 7faa54ea0700 1 -- 172.21.15.52:0/972915198 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7faa380a11d0 msgr2=0x7faa380a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:42.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.425+0000 7faa54ea0700 1 --2- 172.21.15.52:0/972915198 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7faa380a11d0 0x7faa380a3680 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faa501012e0 tx=0x7faa40009260 comp rx=0 tx=0).stop 2023-11-27T15:17:42.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.425+0000 7faa54ea0700 1 -- 172.21.15.52:0/972915198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa50101e10 msgr2=0x7faa5019d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:42.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.425+0000 7faa54ea0700 1 --2- 172.21.15.52:0/972915198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa50101e10 0x7faa5019d550 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7faa4800ef30 tx=0x7faa4800c3a0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.425+0000 7faa54ea0700 1 -- 172.21.15.52:0/972915198 shutdown_connections 2023-11-27T15:17:42.433 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.425+0000 7faa54ea0700 1 --2- 172.21.15.52:0/972915198 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7faa380a11d0 0x7faa380a3680 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.433 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.425+0000 7faa54ea0700 1 --2- 172.21.15.52:0/972915198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa50101e10 0x7faa5019d550 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.433 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.425+0000 7faa54ea0700 1 --2- 172.21.15.52:0/972915198 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa50100290 0x7faa50195fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.433 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.425+0000 7faa54ea0700 1 --2- 172.21.15.52:0/972915198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa50101480 0x7faa501964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.433 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.425+0000 7faa54ea0700 1 -- 172.21.15.52:0/972915198 >> 172.21.15.52:0/972915198 conn(0x7faa500fb870 msgr2=0x7faa500fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:42.433 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.425+0000 7faa54ea0700 1 -- 172.21.15.52:0/972915198 shutdown_connections 2023-11-27T15:17:42.433 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.425+0000 7faa54ea0700 1 -- 172.21.15.52:0/972915198 wait complete. 2023-11-27T15:17:42.433 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2023-11-27T15:17:42.442 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2023-11-27T15:17:41.748543+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654284,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":669,"num_read_kb":455,"num_write":346,"num_write_kb":3053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181756,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":782,"ondisk_log_size":782,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":80372,"kb_used_data":11284,"kb_used_omap":0,"kb_used_meta":68992,"kb_avail":1124861452,"statfs":{"total":1151940427776,"available":1151858126848,"internally_reserved":0,"allocated":11554816,"data_stored":7294767,"data_compressed":177726,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":0,"internal_metadata":70647808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":849,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":34,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005611"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.765734+0000","last_change":"2023-11-27T15:16:09.343007+0000","last_active":"2023-11-27T15:17:33.765734+0000","last_peered":"2023-11-27T15:17:33.765734+0000","last_clean":"2023-11-27T15:17:33.765734+0000","last_became_active":"2023-11-27T15:16:09.342125+0000","last_became_peered":"2023-11-27T15:16:09.342125+0000","last_unstale":"2023-11-27T15:17:33.765734+0000","last_undegraded":"2023-11-27T15:17:33.765734+0000","last_fullsized":"2023-11-27T15:17:33.765734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:24:26.323974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.764972+0000","last_change":"2023-11-27T15:16:09.338942+0000","last_active":"2023-11-27T15:17:33.764972+0000","last_peered":"2023-11-27T15:17:33.764972+0000","last_clean":"2023-11-27T15:17:33.764972+0000","last_became_active":"2023-11-27T15:16:09.338636+0000","last_became_peered":"2023-11-27T15:16:09.338636+0000","last_unstale":"2023-11-27T15:17:33.764972+0000","last_undegraded":"2023-11-27T15:17:33.764972+0000","last_fullsized":"2023-11-27T15:17:33.764972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:40:36.907284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.765902+0000","last_change":"2023-11-27T15:16:09.347845+0000","last_active":"2023-11-27T15:17:33.765902+0000","last_peered":"2023-11-27T15:17:33.765902+0000","last_clean":"2023-11-27T15:17:33.765902+0000","last_became_active":"2023-11-27T15:16:09.347533+0000","last_became_peered":"2023-11-27T15:16:09.347533+0000","last_unstale":"2023-11-27T15:17:33.765902+0000","last_undegraded":"2023-11-27T15:17:33.765902+0000","last_fullsized":"2023-11-27T15:17:33.765902+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:30:09.473655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.762843+0000","last_change":"2023-11-27T15:17:04.811819+0000","last_active":"2023-11-27T15:17:33.762843+0000","last_peered":"2023-11-27T15:17:33.762843+0000","last_clean":"2023-11-27T15:17:33.762843+0000","last_became_active":"2023-11-27T15:17:04.811443+0000","last_became_peered":"2023-11-27T15:17:04.811443+0000","last_unstale":"2023-11-27T15:17:33.762843+0000","last_undegraded":"2023-11-27T15:17:33.762843+0000","last_fullsized":"2023-11-27T15:17:33.762843+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:03:57.997950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.772322+0000","last_change":"2023-11-27T15:16:09.349097+0000","last_active":"2023-11-27T15:17:33.772322+0000","last_peered":"2023-11-27T15:17:33.772322+0000","last_clean":"2023-11-27T15:17:33.772322+0000","last_became_active":"2023-11-27T15:16:09.348793+0000","last_became_peered":"2023-11-27T15:16:09.348793+0000","last_unstale":"2023-11-27T15:17:33.772322+0000","last_undegraded":"2023-11-27T15:17:33.772322+0000","last_fullsized":"2023-11-27T15:17:33.772322+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:26:40.373634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.765958+0000","last_change":"2023-11-27T15:16:09.338246+0000","last_active":"2023-11-27T15:17:33.765958+0000","last_peered":"2023-11-27T15:17:33.765958+0000","last_clean":"2023-11-27T15:17:33.765958+0000","last_became_active":"2023-11-27T15:16:09.338003+0000","last_became_peered":"2023-11-27T15:16:09.338003+0000","last_unstale":"2023-11-27T15:17:33.765958+0000","last_undegraded":"2023-11-27T15:17:33.765958+0000","last_fullsized":"2023-11-27T15:17:33.765958+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:50:11.348331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.773179+0000","last_change":"2023-11-27T15:16:09.342131+0000","last_active":"2023-11-27T15:17:33.773179+0000","last_peered":"2023-11-27T15:17:33.773179+0000","last_clean":"2023-11-27T15:17:33.773179+0000","last_became_active":"2023-11-27T15:16:09.341834+0000","last_became_peered":"2023-11-27T15:16:09.341834+0000","last_unstale":"2023-11-27T15:17:33.773179+0000","last_undegraded":"2023-11-27T15:17:33.773179+0000","last_fullsized":"2023-11-27T15:17:33.773179+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:22:39.182216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.776357+0000","last_change":"2023-11-27T15:16:09.337308+0000","last_active":"2023-11-27T15:17:33.776357+0000","last_peered":"2023-11-27T15:17:33.776357+0000","last_clean":"2023-11-27T15:17:33.776357+0000","last_became_active":"2023-11-27T15:16:09.336958+0000","last_became_peered":"2023-11-27T15:16:09.336958+0000","last_unstale":"2023-11-27T15:17:33.776357+0000","last_undegraded":"2023-11-27T15:17:33.776357+0000","last_fullsized":"2023-11-27T15:17:33.776357+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:00:00.299148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.771230+0000","last_change":"2023-11-27T15:17:04.810530+0000","last_active":"2023-11-27T15:17:33.771230+0000","last_peered":"2023-11-27T15:17:33.771230+0000","last_clean":"2023-11-27T15:17:33.771230+0000","last_became_active":"2023-11-27T15:17:04.810197+0000","last_became_peered":"2023-11-27T15:17:04.810197+0000","last_unstale":"2023-11-27T15:17:33.771230+0000","last_undegraded":"2023-11-27T15:17:33.771230+0000","last_fullsized":"2023-11-27T15:17:33.771230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:58:40.385968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,11],"acting":[1,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.771264+0000","last_change":"2023-11-27T15:16:09.336871+0000","last_active":"2023-11-27T15:17:33.771264+0000","last_peered":"2023-11-27T15:17:33.771264+0000","last_clean":"2023-11-27T15:17:33.771264+0000","last_became_active":"2023-11-27T15:16:09.336496+0000","last_became_peered":"2023-11-27T15:16:09.336496+0000","last_unstale":"2023-11-27T15:17:33.771264+0000","last_undegraded":"2023-11-27T15:17:33.771264+0000","last_fullsized":"2023-11-27T15:17:33.771264+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:12:27.254009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.776130+0000","last_change":"2023-11-27T15:16:09.343120+0000","last_active":"2023-11-27T15:17:33.776130+0000","last_peered":"2023-11-27T15:17:33.776130+0000","last_clean":"2023-11-27T15:17:33.776130+0000","last_became_active":"2023-11-27T15:16:09.342797+0000","last_became_peered":"2023-11-27T15:16:09.342797+0000","last_unstale":"2023-11-27T15:17:33.776130+0000","last_undegraded":"2023-11-27T15:17:33.776130+0000","last_fullsized":"2023-11-27T15:17:33.776130+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:22:16.706218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.759194+0000","last_change":"2023-11-27T15:16:09.336521+0000","last_active":"2023-11-27T15:17:33.759194+0000","last_peered":"2023-11-27T15:17:33.759194+0000","last_clean":"2023-11-27T15:17:33.759194+0000","last_became_active":"2023-11-27T15:16:09.336183+0000","last_became_peered":"2023-11-27T15:16:09.336183+0000","last_unstale":"2023-11-27T15:17:33.759194+0000","last_undegraded":"2023-11-27T15:17:33.759194+0000","last_fullsized":"2023-11-27T15:17:33.759194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:54:55.143503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.775471+0000","last_change":"2023-11-27T15:16:09.346839+0000","last_active":"2023-11-27T15:17:33.775471+0000","last_peered":"2023-11-27T15:17:33.775471+0000","last_clean":"2023-11-27T15:17:33.775471+0000","last_became_active":"2023-11-27T15:16:09.345230+0000","last_became_peered":"2023-11-27T15:16:09.345230+0000","last_unstale":"2023-11-27T15:17:33.775471+0000","last_undegraded":"2023-11-27T15:17:33.775471+0000","last_fullsized":"2023-11-27T15:17:33.775471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:35:36.915491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.758451+0000","last_change":"2023-11-27T15:16:09.342380+0000","last_active":"2023-11-27T15:17:33.758451+0000","last_peered":"2023-11-27T15:17:33.758451+0000","last_clean":"2023-11-27T15:17:33.758451+0000","last_became_active":"2023-11-27T15:16:09.341967+0000","last_became_peered":"2023-11-27T15:16:09.341967+0000","last_unstale":"2023-11-27T15:17:33.758451+0000","last_undegraded":"2023-11-27T15:17:33.758451+0000","last_fullsized":"2023-11-27T15:17:33.758451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:12:37.584284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.754779+0000","last_change":"2023-11-27T15:16:09.349738+0000","last_active":"2023-11-27T15:17:33.754779+0000","last_peered":"2023-11-27T15:17:33.754779+0000","last_clean":"2023-11-27T15:17:33.754779+0000","last_became_active":"2023-11-27T15:16:09.349349+0000","last_became_peered":"2023-11-27T15:16:09.349349+0000","last_unstale":"2023-11-27T15:17:33.754779+0000","last_undegraded":"2023-11-27T15:17:33.754779+0000","last_fullsized":"2023-11-27T15:17:33.754779+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:59:30.065309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.776017+0000","last_change":"2023-11-27T15:16:09.339171+0000","last_active":"2023-11-27T15:17:33.776017+0000","last_peered":"2023-11-27T15:17:33.776017+0000","last_clean":"2023-11-27T15:17:33.776017+0000","last_became_active":"2023-11-27T15:16:09.338887+0000","last_became_peered":"2023-11-27T15:16:09.338887+0000","last_unstale":"2023-11-27T15:17:33.776017+0000","last_undegraded":"2023-11-27T15:17:33.776017+0000","last_fullsized":"2023-11-27T15:17:33.776017+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:08:50.297424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.768341+0000","last_change":"2023-11-27T15:16:09.343202+0000","last_active":"2023-11-27T15:17:33.768341+0000","last_peered":"2023-11-27T15:17:33.768341+0000","last_clean":"2023-11-27T15:17:33.768341+0000","last_became_active":"2023-11-27T15:16:09.342539+0000","last_became_peered":"2023-11-27T15:16:09.342539+0000","last_unstale":"2023-11-27T15:17:33.768341+0000","last_undegraded":"2023-11-27T15:17:33.768341+0000","last_fullsized":"2023-11-27T15:17:33.768341+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:53:20.106298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.772964+0000","last_change":"2023-11-27T15:16:09.339915+0000","last_active":"2023-11-27T15:17:33.772964+0000","last_peered":"2023-11-27T15:17:33.772964+0000","last_clean":"2023-11-27T15:17:33.772964+0000","last_became_active":"2023-11-27T15:16:09.339637+0000","last_became_peered":"2023-11-27T15:16:09.339637+0000","last_unstale":"2023-11-27T15:17:33.772964+0000","last_undegraded":"2023-11-27T15:17:33.772964+0000","last_fullsized":"2023-11-27T15:17:33.772964+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:04:48.736761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.773767+0000","last_change":"2023-11-27T15:16:09.347852+0000","last_active":"2023-11-27T15:17:33.773767+0000","last_peered":"2023-11-27T15:17:33.773767+0000","last_clean":"2023-11-27T15:17:33.773767+0000","last_became_active":"2023-11-27T15:16:09.347281+0000","last_became_peered":"2023-11-27T15:16:09.347281+0000","last_unstale":"2023-11-27T15:17:33.773767+0000","last_undegraded":"2023-11-27T15:17:33.773767+0000","last_fullsized":"2023-11-27T15:17:33.773767+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:29:49.527742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.806625+0000","last_change":"2023-11-27T15:16:08.912206+0000","last_active":"2023-11-27T15:17:04.806625+0000","last_peered":"2023-11-27T15:17:04.806625+0000","last_clean":"2023-11-27T15:17:04.806625+0000","last_became_active":"2023-11-27T15:16:08.911846+0000","last_became_peered":"2023-11-27T15:16:08.911846+0000","last_unstale":"2023-11-27T15:17:04.806625+0000","last_undegraded":"2023-11-27T15:17:04.806625+0000","last_fullsized":"2023-11-27T15:17:04.806625+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:38:19.552326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.770773+0000","last_change":"2023-11-27T15:16:09.336485+0000","last_active":"2023-11-27T15:17:33.770773+0000","last_peered":"2023-11-27T15:17:33.770773+0000","last_clean":"2023-11-27T15:17:33.770773+0000","last_became_active":"2023-11-27T15:16:09.336236+0000","last_became_peered":"2023-11-27T15:16:09.336236+0000","last_unstale":"2023-11-27T15:17:33.770773+0000","last_undegraded":"2023-11-27T15:17:33.770773+0000","last_fullsized":"2023-11-27T15:17:33.770773+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:56:02.954043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.808190+0000","last_change":"2023-11-27T15:16:08.921198+0000","last_active":"2023-11-27T15:17:04.808190+0000","last_peered":"2023-11-27T15:17:04.808190+0000","last_clean":"2023-11-27T15:17:04.808190+0000","last_became_active":"2023-11-27T15:16:08.920804+0000","last_became_peered":"2023-11-27T15:16:08.920804+0000","last_unstale":"2023-11-27T15:17:04.808190+0000","last_undegraded":"2023-11-27T15:17:04.808190+0000","last_fullsized":"2023-11-27T15:17:04.808190+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:29:23.298656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.767388+0000","last_change":"2023-11-27T15:16:09.340642+0000","last_active":"2023-11-27T15:17:33.767388+0000","last_peered":"2023-11-27T15:17:33.767388+0000","last_clean":"2023-11-27T15:17:33.767388+0000","last_became_active":"2023-11-27T15:16:09.340328+0000","last_became_peered":"2023-11-27T15:16:09.340328+0000","last_unstale":"2023-11-27T15:17:33.767388+0000","last_undegraded":"2023-11-27T15:17:33.767388+0000","last_fullsized":"2023-11-27T15:17:33.767388+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:46:04.794567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.810213+0000","last_change":"2023-11-27T15:16:08.915317+0000","last_active":"2023-11-27T15:17:04.810213+0000","last_peered":"2023-11-27T15:17:04.810213+0000","last_clean":"2023-11-27T15:17:04.810213+0000","last_became_active":"2023-11-27T15:16:08.914949+0000","last_became_peered":"2023-11-27T15:16:08.914949+0000","last_unstale":"2023-11-27T15:17:04.810213+0000","last_undegraded":"2023-11-27T15:17:04.810213+0000","last_fullsized":"2023-11-27T15:17:04.810213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:06:23.361651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.773082+0000","last_change":"2023-11-27T15:16:09.348483+0000","last_active":"2023-11-27T15:17:33.773082+0000","last_peered":"2023-11-27T15:17:33.773082+0000","last_clean":"2023-11-27T15:17:33.773082+0000","last_became_active":"2023-11-27T15:16:09.348185+0000","last_became_peered":"2023-11-27T15:16:09.348185+0000","last_unstale":"2023-11-27T15:17:33.773082+0000","last_undegraded":"2023-11-27T15:17:33.773082+0000","last_fullsized":"2023-11-27T15:17:33.773082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:48:03.154051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.809047+0000","last_change":"2023-11-27T15:16:08.911018+0000","last_active":"2023-11-27T15:17:04.809047+0000","last_peered":"2023-11-27T15:17:04.809047+0000","last_clean":"2023-11-27T15:17:04.809047+0000","last_became_active":"2023-11-27T15:16:08.910684+0000","last_became_peered":"2023-11-27T15:16:08.910684+0000","last_unstale":"2023-11-27T15:17:04.809047+0000","last_undegraded":"2023-11-27T15:17:04.809047+0000","last_fullsized":"2023-11-27T15:17:04.809047+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:08:38.910735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.774391+0000","last_change":"2023-11-27T15:16:09.341414+0000","last_active":"2023-11-27T15:17:33.774391+0000","last_peered":"2023-11-27T15:17:33.774391+0000","last_clean":"2023-11-27T15:17:33.774391+0000","last_became_active":"2023-11-27T15:16:09.341068+0000","last_became_peered":"2023-11-27T15:16:09.341068+0000","last_unstale":"2023-11-27T15:17:33.774391+0000","last_undegraded":"2023-11-27T15:17:33.774391+0000","last_fullsized":"2023-11-27T15:17:33.774391+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:24:59.603494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'7","reported_seq":125,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:36.892827+0000","last_change":"2023-11-27T15:17:04.849470+0000","last_active":"2023-11-27T15:17:36.892827+0000","last_peered":"2023-11-27T15:17:36.892827+0000","last_clean":"2023-11-27T15:17:36.892827+0000","last_became_active":"2023-11-27T15:17:04.845827+0000","last_became_peered":"2023-11-27T15:17:04.845827+0000","last_unstale":"2023-11-27T15:17:36.892827+0000","last_undegraded":"2023-11-27T15:17:36.892827+0000","last_fullsized":"2023-11-27T15:17:36.892827+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:59:52.295722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1105,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":964,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.758486+0000","last_change":"2023-11-27T15:16:09.337330+0000","last_active":"2023-11-27T15:17:33.758486+0000","last_peered":"2023-11-27T15:17:33.758486+0000","last_clean":"2023-11-27T15:17:33.758486+0000","last_became_active":"2023-11-27T15:16:09.336991+0000","last_became_peered":"2023-11-27T15:16:09.336991+0000","last_unstale":"2023-11-27T15:17:33.758486+0000","last_undegraded":"2023-11-27T15:17:33.758486+0000","last_fullsized":"2023-11-27T15:17:33.758486+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:35:01.499613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.809306+0000","last_change":"2023-11-27T15:16:08.912208+0000","last_active":"2023-11-27T15:17:04.809306+0000","last_peered":"2023-11-27T15:17:04.809306+0000","last_clean":"2023-11-27T15:17:04.809306+0000","last_became_active":"2023-11-27T15:16:08.911879+0000","last_became_peered":"2023-11-27T15:16:08.911879+0000","last_unstale":"2023-11-27T15:17:04.809306+0000","last_undegraded":"2023-11-27T15:17:04.809306+0000","last_fullsized":"2023-11-27T15:17:04.809306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:34:55.140701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.772328+0000","last_change":"2023-11-27T15:16:09.339660+0000","last_active":"2023-11-27T15:17:33.772328+0000","last_peered":"2023-11-27T15:17:33.772328+0000","last_clean":"2023-11-27T15:17:33.772328+0000","last_became_active":"2023-11-27T15:16:09.339323+0000","last_became_peered":"2023-11-27T15:16:09.339323+0000","last_unstale":"2023-11-27T15:17:33.772328+0000","last_undegraded":"2023-11-27T15:17:33.772328+0000","last_fullsized":"2023-11-27T15:17:33.772328+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:31:05.598344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.803818+0000","last_change":"2023-11-27T15:16:08.911103+0000","last_active":"2023-11-27T15:17:04.803818+0000","last_peered":"2023-11-27T15:17:04.803818+0000","last_clean":"2023-11-27T15:17:04.803818+0000","last_became_active":"2023-11-27T15:16:08.910765+0000","last_became_peered":"2023-11-27T15:16:08.910765+0000","last_unstale":"2023-11-27T15:17:04.803818+0000","last_undegraded":"2023-11-27T15:17:04.803818+0000","last_fullsized":"2023-11-27T15:17:04.803818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:08:20.782772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.774114+0000","last_change":"2023-11-27T15:16:09.351741+0000","last_active":"2023-11-27T15:17:33.774114+0000","last_peered":"2023-11-27T15:17:33.774114+0000","last_clean":"2023-11-27T15:17:33.774114+0000","last_became_active":"2023-11-27T15:16:09.351280+0000","last_became_peered":"2023-11-27T15:16:09.351280+0000","last_unstale":"2023-11-27T15:17:33.774114+0000","last_undegraded":"2023-11-27T15:17:33.774114+0000","last_fullsized":"2023-11-27T15:17:33.774114+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:29:29.349396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.802775+0000","last_change":"2023-11-27T15:16:08.915630+0000","last_active":"2023-11-27T15:17:04.802775+0000","last_peered":"2023-11-27T15:17:04.802775+0000","last_clean":"2023-11-27T15:17:04.802775+0000","last_became_active":"2023-11-27T15:16:08.915294+0000","last_became_peered":"2023-11-27T15:16:08.915294+0000","last_unstale":"2023-11-27T15:17:04.802775+0000","last_undegraded":"2023-11-27T15:17:04.802775+0000","last_fullsized":"2023-11-27T15:17:04.802775+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:58:19.686545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.777372+0000","last_change":"2023-11-27T15:16:09.338436+0000","last_active":"2023-11-27T15:17:33.777372+0000","last_peered":"2023-11-27T15:17:33.777372+0000","last_clean":"2023-11-27T15:17:33.777372+0000","last_became_active":"2023-11-27T15:16:09.338148+0000","last_became_peered":"2023-11-27T15:16:09.338148+0000","last_unstale":"2023-11-27T15:17:33.777372+0000","last_undegraded":"2023-11-27T15:17:33.777372+0000","last_fullsized":"2023-11-27T15:17:33.777372+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:49:40.837798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":106,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:36.892579+0000","last_change":"2023-11-27T15:16:08.913072+0000","last_active":"2023-11-27T15:17:36.892579+0000","last_peered":"2023-11-27T15:17:36.892579+0000","last_clean":"2023-11-27T15:17:36.892579+0000","last_became_active":"2023-11-27T15:16:08.912757+0000","last_became_peered":"2023-11-27T15:16:08.912757+0000","last_unstale":"2023-11-27T15:17:36.892579+0000","last_undegraded":"2023-11-27T15:17:36.892579+0000","last_fullsized":"2023-11-27T15:17:36.892579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:48:31.534178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.772079+0000","last_change":"2023-11-27T15:16:09.340380+0000","last_active":"2023-11-27T15:17:33.772079+0000","last_peered":"2023-11-27T15:17:33.772079+0000","last_clean":"2023-11-27T15:17:33.772079+0000","last_became_active":"2023-11-27T15:16:09.339581+0000","last_became_peered":"2023-11-27T15:16:09.339581+0000","last_unstale":"2023-11-27T15:17:33.772079+0000","last_undegraded":"2023-11-27T15:17:33.772079+0000","last_fullsized":"2023-11-27T15:17:33.772079+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:21:44.281165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.807944+0000","last_change":"2023-11-27T15:16:08.917924+0000","last_active":"2023-11-27T15:17:04.807944+0000","last_peered":"2023-11-27T15:17:04.807944+0000","last_clean":"2023-11-27T15:17:04.807944+0000","last_became_active":"2023-11-27T15:16:08.917618+0000","last_became_peered":"2023-11-27T15:16:08.917618+0000","last_unstale":"2023-11-27T15:17:04.807944+0000","last_undegraded":"2023-11-27T15:17:04.807944+0000","last_fullsized":"2023-11-27T15:17:04.807944+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:26:02.008425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.767487+0000","last_change":"2023-11-27T15:16:09.346020+0000","last_active":"2023-11-27T15:17:33.767487+0000","last_peered":"2023-11-27T15:17:33.767487+0000","last_clean":"2023-11-27T15:17:33.767487+0000","last_became_active":"2023-11-27T15:16:09.345822+0000","last_became_peered":"2023-11-27T15:16:09.345822+0000","last_unstale":"2023-11-27T15:17:33.767487+0000","last_undegraded":"2023-11-27T15:17:33.767487+0000","last_fullsized":"2023-11-27T15:17:33.767487+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:09:13.031060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.803585+0000","last_change":"2023-11-27T15:16:08.921810+0000","last_active":"2023-11-27T15:17:04.803585+0000","last_peered":"2023-11-27T15:17:04.803585+0000","last_clean":"2023-11-27T15:17:04.803585+0000","last_became_active":"2023-11-27T15:16:08.921518+0000","last_became_peered":"2023-11-27T15:16:08.921518+0000","last_unstale":"2023-11-27T15:17:04.803585+0000","last_undegraded":"2023-11-27T15:17:04.803585+0000","last_fullsized":"2023-11-27T15:17:04.803585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:57:19.901063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.772314+0000","last_change":"2023-11-27T15:16:09.351083+0000","last_active":"2023-11-27T15:17:33.772314+0000","last_peered":"2023-11-27T15:17:33.772314+0000","last_clean":"2023-11-27T15:17:33.772314+0000","last_became_active":"2023-11-27T15:16:09.350540+0000","last_became_peered":"2023-11-27T15:16:09.350540+0000","last_unstale":"2023-11-27T15:17:33.772314+0000","last_undegraded":"2023-11-27T15:17:33.772314+0000","last_fullsized":"2023-11-27T15:17:33.772314+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:08:16.021485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'31","reported_seq":226,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:37.894182+0000","last_change":"2023-11-27T15:16:08.913023+0000","last_active":"2023-11-27T15:17:37.894182+0000","last_peered":"2023-11-27T15:17:37.894182+0000","last_clean":"2023-11-27T15:17:37.894182+0000","last_became_active":"2023-11-27T15:16:08.912693+0000","last_became_peered":"2023-11-27T15:16:08.912693+0000","last_unstale":"2023-11-27T15:17:37.894182+0000","last_undegraded":"2023-11-27T15:17:37.894182+0000","last_fullsized":"2023-11-27T15:17:37.894182+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:33:54.579228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58776,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":177,"num_read_kb":145,"num_write":32,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.775109+0000","last_change":"2023-11-27T15:16:09.343446+0000","last_active":"2023-11-27T15:17:33.775109+0000","last_peered":"2023-11-27T15:17:33.775109+0000","last_clean":"2023-11-27T15:17:33.775109+0000","last_became_active":"2023-11-27T15:16:09.341980+0000","last_became_peered":"2023-11-27T15:16:09.341980+0000","last_unstale":"2023-11-27T15:17:33.775109+0000","last_undegraded":"2023-11-27T15:17:33.775109+0000","last_fullsized":"2023-11-27T15:17:33.775109+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:48:41.501519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.822854+0000","last_change":"2023-11-27T15:17:04.822416+0000","last_active":"2023-11-27T15:17:04.822854+0000","last_peered":"2023-11-27T15:17:04.822854+0000","last_clean":"2023-11-27T15:17:04.822854+0000","last_became_active":"2023-11-27T15:17:04.810377+0000","last_became_peered":"2023-11-27T15:17:04.810377+0000","last_unstale":"2023-11-27T15:17:04.822854+0000","last_undegraded":"2023-11-27T15:17:04.822854+0000","last_fullsized":"2023-11-27T15:17:04.822854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:03:53.866628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.775297+0000","last_change":"2023-11-27T15:17:04.809248+0000","last_active":"2023-11-27T15:17:33.775297+0000","last_peered":"2023-11-27T15:17:33.775297+0000","last_clean":"2023-11-27T15:17:33.775297+0000","last_became_active":"2023-11-27T15:17:04.808813+0000","last_became_peered":"2023-11-27T15:17:04.808813+0000","last_unstale":"2023-11-27T15:17:33.775297+0000","last_undegraded":"2023-11-27T15:17:33.775297+0000","last_fullsized":"2023-11-27T15:17:33.775297+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:58:27.173973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.803065+0000","last_change":"2023-11-27T15:16:08.915597+0000","last_active":"2023-11-27T15:17:04.803065+0000","last_peered":"2023-11-27T15:17:04.803065+0000","last_clean":"2023-11-27T15:17:04.803065+0000","last_became_active":"2023-11-27T15:16:08.914727+0000","last_became_peered":"2023-11-27T15:16:08.914727+0000","last_unstale":"2023-11-27T15:17:04.803065+0000","last_undegraded":"2023-11-27T15:17:04.803065+0000","last_fullsized":"2023-11-27T15:17:04.803065+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:24:30.347721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.777426+0000","last_change":"2023-11-27T15:16:09.352643+0000","last_active":"2023-11-27T15:17:33.777426+0000","last_peered":"2023-11-27T15:17:33.777426+0000","last_clean":"2023-11-27T15:17:33.777426+0000","last_became_active":"2023-11-27T15:16:09.352336+0000","last_became_peered":"2023-11-27T15:16:09.352336+0000","last_unstale":"2023-11-27T15:17:33.777426+0000","last_undegraded":"2023-11-27T15:17:33.777426+0000","last_fullsized":"2023-11-27T15:17:33.777426+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:26:11.095933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'6","reported_seq":116,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:39.898232+0000","last_change":"2023-11-27T15:16:08.914303+0000","last_active":"2023-11-27T15:17:39.898232+0000","last_peered":"2023-11-27T15:17:39.898232+0000","last_clean":"2023-11-27T15:17:39.898232+0000","last_became_active":"2023-11-27T15:16:08.913601+0000","last_became_peered":"2023-11-27T15:16:08.913601+0000","last_unstale":"2023-11-27T15:17:39.898232+0000","last_undegraded":"2023-11-27T15:17:39.898232+0000","last_fullsized":"2023-11-27T15:17:39.898232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:32:26.143895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1389,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.775199+0000","last_change":"2023-11-27T15:16:09.341342+0000","last_active":"2023-11-27T15:17:33.775199+0000","last_peered":"2023-11-27T15:17:33.775199+0000","last_clean":"2023-11-27T15:17:33.775199+0000","last_became_active":"2023-11-27T15:16:09.341100+0000","last_became_peered":"2023-11-27T15:16:09.341100+0000","last_unstale":"2023-11-27T15:17:33.775199+0000","last_undegraded":"2023-11-27T15:17:33.775199+0000","last_fullsized":"2023-11-27T15:17:33.775199+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:11:05.729312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.808459+0000","last_change":"2023-11-27T15:16:08.921771+0000","last_active":"2023-11-27T15:17:04.808459+0000","last_peered":"2023-11-27T15:17:04.808459+0000","last_clean":"2023-11-27T15:17:04.808459+0000","last_became_active":"2023-11-27T15:16:08.921396+0000","last_became_peered":"2023-11-27T15:16:08.921396+0000","last_unstale":"2023-11-27T15:17:04.808459+0000","last_undegraded":"2023-11-27T15:17:04.808459+0000","last_fullsized":"2023-11-27T15:17:04.808459+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:05:46.765380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.771089+0000","last_change":"2023-11-27T15:16:09.341258+0000","last_active":"2023-11-27T15:17:33.771089+0000","last_peered":"2023-11-27T15:17:33.771089+0000","last_clean":"2023-11-27T15:17:33.771089+0000","last_became_active":"2023-11-27T15:16:09.340921+0000","last_became_peered":"2023-11-27T15:16:09.340921+0000","last_unstale":"2023-11-27T15:17:33.771089+0000","last_undegraded":"2023-11-27T15:17:33.771089+0000","last_fullsized":"2023-11-27T15:17:33.771089+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:12:25.330319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.809915+0000","last_change":"2023-11-27T15:16:08.911219+0000","last_active":"2023-11-27T15:17:04.809915+0000","last_peered":"2023-11-27T15:17:04.809915+0000","last_clean":"2023-11-27T15:17:04.809915+0000","last_became_active":"2023-11-27T15:16:08.910788+0000","last_became_peered":"2023-11-27T15:16:08.910788+0000","last_unstale":"2023-11-27T15:17:04.809915+0000","last_undegraded":"2023-11-27T15:17:04.809915+0000","last_fullsized":"2023-11-27T15:17:04.809915+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:39:39.465171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.765541+0000","last_change":"2023-11-27T15:16:09.343793+0000","last_active":"2023-11-27T15:17:33.765541+0000","last_peered":"2023-11-27T15:17:33.765541+0000","last_clean":"2023-11-27T15:17:33.765541+0000","last_became_active":"2023-11-27T15:16:09.343480+0000","last_became_peered":"2023-11-27T15:16:09.343480+0000","last_unstale":"2023-11-27T15:17:33.765541+0000","last_undegraded":"2023-11-27T15:17:33.765541+0000","last_fullsized":"2023-11-27T15:17:33.765541+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:37:38.699371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.806936+0000","last_change":"2023-11-27T15:16:08.920651+0000","last_active":"2023-11-27T15:17:04.806936+0000","last_peered":"2023-11-27T15:17:04.806936+0000","last_clean":"2023-11-27T15:17:04.806936+0000","last_became_active":"2023-11-27T15:16:08.920294+0000","last_became_peered":"2023-11-27T15:16:08.920294+0000","last_unstale":"2023-11-27T15:17:04.806936+0000","last_undegraded":"2023-11-27T15:17:04.806936+0000","last_fullsized":"2023-11-27T15:17:04.806936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:41:12.965663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.773421+0000","last_change":"2023-11-27T15:16:09.348279+0000","last_active":"2023-11-27T15:17:33.773421+0000","last_peered":"2023-11-27T15:17:33.773421+0000","last_clean":"2023-11-27T15:17:33.773421+0000","last_became_active":"2023-11-27T15:16:09.347760+0000","last_became_peered":"2023-11-27T15:16:09.347760+0000","last_unstale":"2023-11-27T15:17:33.773421+0000","last_undegraded":"2023-11-27T15:17:33.773421+0000","last_fullsized":"2023-11-27T15:17:33.773421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:19:36.809531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.807809+0000","last_change":"2023-11-27T15:16:08.915622+0000","last_active":"2023-11-27T15:17:04.807809+0000","last_peered":"2023-11-27T15:17:04.807809+0000","last_clean":"2023-11-27T15:17:04.807809+0000","last_became_active":"2023-11-27T15:16:08.915293+0000","last_became_peered":"2023-11-27T15:16:08.915293+0000","last_unstale":"2023-11-27T15:17:04.807809+0000","last_undegraded":"2023-11-27T15:17:04.807809+0000","last_fullsized":"2023-11-27T15:17:04.807809+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:31:34.850798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.769972+0000","last_change":"2023-11-27T15:16:09.337096+0000","last_active":"2023-11-27T15:17:33.769972+0000","last_peered":"2023-11-27T15:17:33.769972+0000","last_clean":"2023-11-27T15:17:33.769972+0000","last_became_active":"2023-11-27T15:16:09.336359+0000","last_became_peered":"2023-11-27T15:16:09.336359+0000","last_unstale":"2023-11-27T15:17:33.769972+0000","last_undegraded":"2023-11-27T15:17:33.769972+0000","last_fullsized":"2023-11-27T15:17:33.769972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:23:45.942895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.774368+0000","last_change":"2023-11-27T15:16:09.340478+0000","last_active":"2023-11-27T15:17:33.774368+0000","last_peered":"2023-11-27T15:17:33.774368+0000","last_clean":"2023-11-27T15:17:33.774368+0000","last_became_active":"2023-11-27T15:16:09.340214+0000","last_became_peered":"2023-11-27T15:16:09.340214+0000","last_unstale":"2023-11-27T15:17:33.774368+0000","last_undegraded":"2023-11-27T15:17:33.774368+0000","last_fullsized":"2023-11-27T15:17:33.774368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:51:37.081922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":108,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:37.893754+0000","last_change":"2023-11-27T15:16:08.920387+0000","last_active":"2023-11-27T15:17:37.893754+0000","last_peered":"2023-11-27T15:17:37.893754+0000","last_clean":"2023-11-27T15:17:37.893754+0000","last_became_active":"2023-11-27T15:16:08.920004+0000","last_became_peered":"2023-11-27T15:16:08.920004+0000","last_unstale":"2023-11-27T15:17:37.893754+0000","last_undegraded":"2023-11-27T15:17:37.893754+0000","last_fullsized":"2023-11-27T15:17:37.893754+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:31:37.559127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":85,"num_read_kb":85,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.770288+0000","last_change":"2023-11-27T15:17:04.808609+0000","last_active":"2023-11-27T15:17:33.770288+0000","last_peered":"2023-11-27T15:17:33.770288+0000","last_clean":"2023-11-27T15:17:33.770288+0000","last_became_active":"2023-11-27T15:17:04.808268+0000","last_became_peered":"2023-11-27T15:17:04.808268+0000","last_unstale":"2023-11-27T15:17:33.770288+0000","last_undegraded":"2023-11-27T15:17:33.770288+0000","last_fullsized":"2023-11-27T15:17:33.770288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:31:40.207340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.845316+0000","last_change":"2023-11-27T15:16:08.909694+0000","last_active":"2023-11-27T15:17:04.845316+0000","last_peered":"2023-11-27T15:17:04.845316+0000","last_clean":"2023-11-27T15:17:04.845316+0000","last_became_active":"2023-11-27T15:16:08.909324+0000","last_became_peered":"2023-11-27T15:16:08.909324+0000","last_unstale":"2023-11-27T15:17:04.845316+0000","last_undegraded":"2023-11-27T15:17:04.845316+0000","last_fullsized":"2023-11-27T15:17:04.845316+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:50:22.630279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":344,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.807076+0000","last_change":"2023-11-27T15:15:10.678805+0000","last_active":"2023-11-27T15:17:04.807076+0000","last_peered":"2023-11-27T15:17:04.807076+0000","last_clean":"2023-11-27T15:17:04.807076+0000","last_became_active":"2023-11-27T15:15:10.678267+0000","last_became_peered":"2023-11-27T15:15:10.678267+0000","last_unstale":"2023-11-27T15:17:04.807076+0000","last_undegraded":"2023-11-27T15:17:04.807076+0000","last_fullsized":"2023-11-27T15:17:04.807076+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:12:14.179315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:12:14.179315+0000","last_clean_scrub_stamp":"2023-11-27T15:12:14.179315+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:46:50.658179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.770453+0000","last_change":"2023-11-27T15:16:09.346622+0000","last_active":"2023-11-27T15:17:33.770453+0000","last_peered":"2023-11-27T15:17:33.770453+0000","last_clean":"2023-11-27T15:17:33.770453+0000","last_became_active":"2023-11-27T15:16:09.346287+0000","last_became_peered":"2023-11-27T15:16:09.346287+0000","last_unstale":"2023-11-27T15:17:33.770453+0000","last_undegraded":"2023-11-27T15:17:33.770453+0000","last_fullsized":"2023-11-27T15:17:33.770453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:45:31.217367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.814144+0000","last_change":"2023-11-27T15:17:04.814144+0000","last_active":"2023-11-27T15:17:04.814144+0000","last_peered":"2023-11-27T15:17:04.814144+0000","last_clean":"2023-11-27T15:17:04.814144+0000","last_became_active":"2023-11-27T15:17:04.810509+0000","last_became_peered":"2023-11-27T15:17:04.810509+0000","last_unstale":"2023-11-27T15:17:04.814144+0000","last_undegraded":"2023-11-27T15:17:04.814144+0000","last_fullsized":"2023-11-27T15:17:04.814144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:54:31.956866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.815900+0000","last_change":"2023-11-27T15:17:04.815501+0000","last_active":"2023-11-27T15:17:04.815900+0000","last_peered":"2023-11-27T15:17:04.815900+0000","last_clean":"2023-11-27T15:17:04.815900+0000","last_became_active":"2023-11-27T15:17:04.810004+0000","last_became_peered":"2023-11-27T15:17:04.810004+0000","last_unstale":"2023-11-27T15:17:04.815900+0000","last_undegraded":"2023-11-27T15:17:04.815900+0000","last_fullsized":"2023-11-27T15:17:04.815900+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:15:11.028146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.767894+0000","last_change":"2023-11-27T15:16:09.345119+0000","last_active":"2023-11-27T15:17:33.767894+0000","last_peered":"2023-11-27T15:17:33.767894+0000","last_clean":"2023-11-27T15:17:33.767894+0000","last_became_active":"2023-11-27T15:16:09.344807+0000","last_became_peered":"2023-11-27T15:16:09.344807+0000","last_unstale":"2023-11-27T15:17:33.767894+0000","last_undegraded":"2023-11-27T15:17:33.767894+0000","last_fullsized":"2023-11-27T15:17:33.767894+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:16:43.558443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'7","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:34.645800+0000","last_change":"2023-11-27T15:16:08.911564+0000","last_active":"2023-11-27T15:17:34.645800+0000","last_peered":"2023-11-27T15:17:34.645800+0000","last_clean":"2023-11-27T15:17:34.645800+0000","last_became_active":"2023-11-27T15:16:08.911057+0000","last_became_peered":"2023-11-27T15:16:08.911057+0000","last_unstale":"2023-11-27T15:17:34.645800+0000","last_undegraded":"2023-11-27T15:17:34.645800+0000","last_fullsized":"2023-11-27T15:17:34.645800+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:45:19.804625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.777021+0000","last_change":"2023-11-27T15:16:09.339085+0000","last_active":"2023-11-27T15:17:33.777021+0000","last_peered":"2023-11-27T15:17:33.777021+0000","last_clean":"2023-11-27T15:17:33.777021+0000","last_became_active":"2023-11-27T15:16:09.338777+0000","last_became_peered":"2023-11-27T15:16:09.338777+0000","last_unstale":"2023-11-27T15:17:33.777021+0000","last_undegraded":"2023-11-27T15:17:33.777021+0000","last_fullsized":"2023-11-27T15:17:33.777021+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:08:45.733215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.845177+0000","last_change":"2023-11-27T15:16:08.913939+0000","last_active":"2023-11-27T15:17:04.845177+0000","last_peered":"2023-11-27T15:17:04.845177+0000","last_clean":"2023-11-27T15:17:04.845177+0000","last_became_active":"2023-11-27T15:16:08.913629+0000","last_became_peered":"2023-11-27T15:16:08.913629+0000","last_unstale":"2023-11-27T15:17:04.845177+0000","last_undegraded":"2023-11-27T15:17:04.845177+0000","last_fullsized":"2023-11-27T15:17:04.845177+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:52:20.803089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.766880+0000","last_change":"2023-11-27T15:16:09.350712+0000","last_active":"2023-11-27T15:17:33.766880+0000","last_peered":"2023-11-27T15:17:33.766880+0000","last_clean":"2023-11-27T15:17:33.766880+0000","last_became_active":"2023-11-27T15:16:09.350359+0000","last_became_peered":"2023-11-27T15:16:09.350359+0000","last_unstale":"2023-11-27T15:17:33.766880+0000","last_undegraded":"2023-11-27T15:17:33.766880+0000","last_fullsized":"2023-11-27T15:17:33.766880+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:17:11.861223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.805284+0000","last_change":"2023-11-27T15:16:08.919501+0000","last_active":"2023-11-27T15:17:04.805284+0000","last_peered":"2023-11-27T15:17:04.805284+0000","last_clean":"2023-11-27T15:17:04.805284+0000","last_became_active":"2023-11-27T15:16:08.919258+0000","last_became_peered":"2023-11-27T15:16:08.919258+0000","last_unstale":"2023-11-27T15:17:04.805284+0000","last_undegraded":"2023-11-27T15:17:04.805284+0000","last_fullsized":"2023-11-27T15:17:04.805284+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:04:56.583314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.734625+0000","last_change":"2023-11-27T15:16:09.344066+0000","last_active":"2023-11-27T15:17:33.734625+0000","last_peered":"2023-11-27T15:17:33.734625+0000","last_clean":"2023-11-27T15:17:33.734625+0000","last_became_active":"2023-11-27T15:16:09.343757+0000","last_became_peered":"2023-11-27T15:16:09.343757+0000","last_unstale":"2023-11-27T15:17:33.734625+0000","last_undegraded":"2023-11-27T15:17:33.734625+0000","last_fullsized":"2023-11-27T15:17:33.734625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:13:29.747291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.808214+0000","last_change":"2023-11-27T15:16:08.906205+0000","last_active":"2023-11-27T15:17:04.808214+0000","last_peered":"2023-11-27T15:17:04.808214+0000","last_clean":"2023-11-27T15:17:04.808214+0000","last_became_active":"2023-11-27T15:16:08.905885+0000","last_became_peered":"2023-11-27T15:16:08.905885+0000","last_unstale":"2023-11-27T15:17:04.808214+0000","last_undegraded":"2023-11-27T15:17:04.808214+0000","last_fullsized":"2023-11-27T15:17:04.808214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:10:14.001695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.761768+0000","last_change":"2023-11-27T15:16:09.349642+0000","last_active":"2023-11-27T15:17:33.761768+0000","last_peered":"2023-11-27T15:17:33.761768+0000","last_clean":"2023-11-27T15:17:33.761768+0000","last_became_active":"2023-11-27T15:16:09.349086+0000","last_became_peered":"2023-11-27T15:16:09.349086+0000","last_unstale":"2023-11-27T15:17:33.761768+0000","last_undegraded":"2023-11-27T15:17:33.761768+0000","last_fullsized":"2023-11-27T15:17:33.761768+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:36:33.604216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.806096+0000","last_change":"2023-11-27T15:16:08.917835+0000","last_active":"2023-11-27T15:17:04.806096+0000","last_peered":"2023-11-27T15:17:04.806096+0000","last_clean":"2023-11-27T15:17:04.806096+0000","last_became_active":"2023-11-27T15:16:08.917532+0000","last_became_peered":"2023-11-27T15:16:08.917532+0000","last_unstale":"2023-11-27T15:17:04.806096+0000","last_undegraded":"2023-11-27T15:17:04.806096+0000","last_fullsized":"2023-11-27T15:17:04.806096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:29:51.697143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.774919+0000","last_change":"2023-11-27T15:16:09.338459+0000","last_active":"2023-11-27T15:17:33.774919+0000","last_peered":"2023-11-27T15:17:33.774919+0000","last_clean":"2023-11-27T15:17:33.774919+0000","last_became_active":"2023-11-27T15:16:09.338148+0000","last_became_peered":"2023-11-27T15:16:09.338148+0000","last_unstale":"2023-11-27T15:17:33.774919+0000","last_undegraded":"2023-11-27T15:17:33.774919+0000","last_fullsized":"2023-11-27T15:17:33.774919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:19:16.710021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.809804+0000","last_change":"2023-11-27T15:16:08.915401+0000","last_active":"2023-11-27T15:17:04.809804+0000","last_peered":"2023-11-27T15:17:04.809804+0000","last_clean":"2023-11-27T15:17:04.809804+0000","last_became_active":"2023-11-27T15:16:08.915012+0000","last_became_peered":"2023-11-27T15:16:08.915012+0000","last_unstale":"2023-11-27T15:17:04.809804+0000","last_undegraded":"2023-11-27T15:17:04.809804+0000","last_fullsized":"2023-11-27T15:17:04.809804+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:03:26.905621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.765523+0000","last_change":"2023-11-27T15:16:09.352309+0000","last_active":"2023-11-27T15:17:33.765523+0000","last_peered":"2023-11-27T15:17:33.765523+0000","last_clean":"2023-11-27T15:17:33.765523+0000","last_became_active":"2023-11-27T15:16:09.352039+0000","last_became_peered":"2023-11-27T15:16:09.352039+0000","last_unstale":"2023-11-27T15:17:33.765523+0000","last_undegraded":"2023-11-27T15:17:33.765523+0000","last_fullsized":"2023-11-27T15:17:33.765523+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:20:09.773389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.805829+0000","last_change":"2023-11-27T15:16:08.920446+0000","last_active":"2023-11-27T15:17:04.805829+0000","last_peered":"2023-11-27T15:17:04.805829+0000","last_clean":"2023-11-27T15:17:04.805829+0000","last_became_active":"2023-11-27T15:16:08.920232+0000","last_became_peered":"2023-11-27T15:16:08.920232+0000","last_unstale":"2023-11-27T15:17:04.805829+0000","last_undegraded":"2023-11-27T15:17:04.805829+0000","last_fullsized":"2023-11-27T15:17:04.805829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:08:07.892335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.775022+0000","last_change":"2023-11-27T15:16:09.344328+0000","last_active":"2023-11-27T15:17:33.775022+0000","last_peered":"2023-11-27T15:17:33.775022+0000","last_clean":"2023-11-27T15:17:33.775022+0000","last_became_active":"2023-11-27T15:16:09.344047+0000","last_became_peered":"2023-11-27T15:16:09.344047+0000","last_unstale":"2023-11-27T15:17:33.775022+0000","last_undegraded":"2023-11-27T15:17:33.775022+0000","last_fullsized":"2023-11-27T15:17:33.775022+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:00:56.015107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.803420+0000","last_change":"2023-11-27T15:16:08.909865+0000","last_active":"2023-11-27T15:17:04.803420+0000","last_peered":"2023-11-27T15:17:04.803420+0000","last_clean":"2023-11-27T15:17:04.803420+0000","last_became_active":"2023-11-27T15:16:08.909559+0000","last_became_peered":"2023-11-27T15:16:08.909559+0000","last_unstale":"2023-11-27T15:17:04.803420+0000","last_undegraded":"2023-11-27T15:17:04.803420+0000","last_fullsized":"2023-11-27T15:17:04.803420+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:59:09.511504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.775967+0000","last_change":"2023-11-27T15:16:09.342525+0000","last_active":"2023-11-27T15:17:33.775967+0000","last_peered":"2023-11-27T15:17:33.775967+0000","last_clean":"2023-11-27T15:17:33.775967+0000","last_became_active":"2023-11-27T15:16:09.340987+0000","last_became_peered":"2023-11-27T15:16:09.340987+0000","last_unstale":"2023-11-27T15:17:33.775967+0000","last_undegraded":"2023-11-27T15:17:33.775967+0000","last_fullsized":"2023-11-27T15:17:33.775967+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:37:38.155819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.804148+0000","last_change":"2023-11-27T15:16:08.914088+0000","last_active":"2023-11-27T15:17:04.804148+0000","last_peered":"2023-11-27T15:17:04.804148+0000","last_clean":"2023-11-27T15:17:04.804148+0000","last_became_active":"2023-11-27T15:16:08.913750+0000","last_became_peered":"2023-11-27T15:16:08.913750+0000","last_unstale":"2023-11-27T15:17:04.804148+0000","last_undegraded":"2023-11-27T15:17:04.804148+0000","last_fullsized":"2023-11-27T15:17:04.804148+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:57:55.114032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.774914+0000","last_change":"2023-11-27T15:16:09.344403+0000","last_active":"2023-11-27T15:17:33.774914+0000","last_peered":"2023-11-27T15:17:33.774914+0000","last_clean":"2023-11-27T15:17:33.774914+0000","last_became_active":"2023-11-27T15:16:09.344120+0000","last_became_peered":"2023-11-27T15:16:09.344120+0000","last_unstale":"2023-11-27T15:17:33.774914+0000","last_undegraded":"2023-11-27T15:17:33.774914+0000","last_fullsized":"2023-11-27T15:17:33.774914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:47:27.200812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.762743+0000","last_change":"2023-11-27T15:16:09.349780+0000","last_active":"2023-11-27T15:17:33.762743+0000","last_peered":"2023-11-27T15:17:33.762743+0000","last_clean":"2023-11-27T15:17:33.762743+0000","last_became_active":"2023-11-27T15:16:09.349493+0000","last_became_peered":"2023-11-27T15:16:09.349493+0000","last_unstale":"2023-11-27T15:17:33.762743+0000","last_undegraded":"2023-11-27T15:17:33.762743+0000","last_fullsized":"2023-11-27T15:17:33.762743+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:14:43.605196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.773008+0000","last_change":"2023-11-27T15:17:04.809959+0000","last_active":"2023-11-27T15:17:33.773008+0000","last_peered":"2023-11-27T15:17:33.773008+0000","last_clean":"2023-11-27T15:17:33.773008+0000","last_became_active":"2023-11-27T15:17:04.809529+0000","last_became_peered":"2023-11-27T15:17:04.809529+0000","last_unstale":"2023-11-27T15:17:33.773008+0000","last_undegraded":"2023-11-27T15:17:33.773008+0000","last_fullsized":"2023-11-27T15:17:33.773008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:19:17.711075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.770374+0000","last_change":"2023-11-27T15:16:09.337989+0000","last_active":"2023-11-27T15:17:33.770374+0000","last_peered":"2023-11-27T15:17:33.770374+0000","last_clean":"2023-11-27T15:17:33.770374+0000","last_became_active":"2023-11-27T15:16:09.336712+0000","last_became_peered":"2023-11-27T15:16:09.336712+0000","last_unstale":"2023-11-27T15:17:33.770374+0000","last_undegraded":"2023-11-27T15:17:33.770374+0000","last_fullsized":"2023-11-27T15:17:33.770374+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:25:41.941448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.768395+0000","last_change":"2023-11-27T15:17:04.812727+0000","last_active":"2023-11-27T15:17:33.768395+0000","last_peered":"2023-11-27T15:17:33.768395+0000","last_clean":"2023-11-27T15:17:33.768395+0000","last_became_active":"2023-11-27T15:17:04.811852+0000","last_became_peered":"2023-11-27T15:17:04.811852+0000","last_unstale":"2023-11-27T15:17:33.768395+0000","last_undegraded":"2023-11-27T15:17:33.768395+0000","last_fullsized":"2023-11-27T15:17:33.768395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:03:44.659440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.771893+0000","last_change":"2023-11-27T15:16:09.344317+0000","last_active":"2023-11-27T15:17:33.771893+0000","last_peered":"2023-11-27T15:17:33.771893+0000","last_clean":"2023-11-27T15:17:33.771893+0000","last_became_active":"2023-11-27T15:16:09.343962+0000","last_became_peered":"2023-11-27T15:16:09.343962+0000","last_unstale":"2023-11-27T15:17:33.771893+0000","last_undegraded":"2023-11-27T15:17:33.771893+0000","last_fullsized":"2023-11-27T15:17:33.771893+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:15:39.862196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.766088+0000","last_change":"2023-11-27T15:16:09.344690+0000","last_active":"2023-11-27T15:17:33.766088+0000","last_peered":"2023-11-27T15:17:33.766088+0000","last_clean":"2023-11-27T15:17:33.766088+0000","last_became_active":"2023-11-27T15:16:09.344259+0000","last_became_peered":"2023-11-27T15:16:09.344259+0000","last_unstale":"2023-11-27T15:17:33.766088+0000","last_undegraded":"2023-11-27T15:17:33.766088+0000","last_fullsized":"2023-11-27T15:17:33.766088+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:48:11.448071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.773804+0000","last_change":"2023-11-27T15:16:09.340459+0000","last_active":"2023-11-27T15:17:33.773804+0000","last_peered":"2023-11-27T15:17:33.773804+0000","last_clean":"2023-11-27T15:17:33.773804+0000","last_became_active":"2023-11-27T15:16:09.340140+0000","last_became_peered":"2023-11-27T15:16:09.340140+0000","last_unstale":"2023-11-27T15:17:33.773804+0000","last_undegraded":"2023-11-27T15:17:33.773804+0000","last_fullsized":"2023-11-27T15:17:33.773804+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:50:28.936856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.770718+0000","last_change":"2023-11-27T15:16:09.336625+0000","last_active":"2023-11-27T15:17:33.770718+0000","last_peered":"2023-11-27T15:17:33.770718+0000","last_clean":"2023-11-27T15:17:33.770718+0000","last_became_active":"2023-11-27T15:16:09.336287+0000","last_became_peered":"2023-11-27T15:16:09.336287+0000","last_unstale":"2023-11-27T15:17:33.770718+0000","last_undegraded":"2023-11-27T15:17:33.770718+0000","last_fullsized":"2023-11-27T15:17:33.770718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:09:58.745009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.772059+0000","last_change":"2023-11-27T15:16:09.335480+0000","last_active":"2023-11-27T15:17:33.772059+0000","last_peered":"2023-11-27T15:17:33.772059+0000","last_clean":"2023-11-27T15:17:33.772059+0000","last_became_active":"2023-11-27T15:16:09.335163+0000","last_became_peered":"2023-11-27T15:16:09.335163+0000","last_unstale":"2023-11-27T15:17:33.772059+0000","last_undegraded":"2023-11-27T15:17:33.772059+0000","last_fullsized":"2023-11-27T15:17:33.772059+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:57:09.686634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.775133+0000","last_change":"2023-11-27T15:16:09.336879+0000","last_active":"2023-11-27T15:17:33.775133+0000","last_peered":"2023-11-27T15:17:33.775133+0000","last_clean":"2023-11-27T15:17:33.775133+0000","last_became_active":"2023-11-27T15:16:09.336580+0000","last_became_peered":"2023-11-27T15:16:09.336580+0000","last_unstale":"2023-11-27T15:17:33.775133+0000","last_undegraded":"2023-11-27T15:17:33.775133+0000","last_fullsized":"2023-11-27T15:17:33.775133+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:18:10.805656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.773451+0000","last_change":"2023-11-27T15:16:09.341400+0000","last_active":"2023-11-27T15:17:33.773451+0000","last_peered":"2023-11-27T15:17:33.773451+0000","last_clean":"2023-11-27T15:17:33.773451+0000","last_became_active":"2023-11-27T15:16:09.341103+0000","last_became_peered":"2023-11-27T15:16:09.341103+0000","last_unstale":"2023-11-27T15:17:33.773451+0000","last_undegraded":"2023-11-27T15:17:33.773451+0000","last_fullsized":"2023-11-27T15:17:33.773451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:51:15.698428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.776500+0000","last_change":"2023-11-27T15:16:09.345938+0000","last_active":"2023-11-27T15:17:33.776500+0000","last_peered":"2023-11-27T15:17:33.776500+0000","last_clean":"2023-11-27T15:17:33.776500+0000","last_became_active":"2023-11-27T15:16:09.345624+0000","last_became_peered":"2023-11-27T15:16:09.345624+0000","last_unstale":"2023-11-27T15:17:33.776500+0000","last_undegraded":"2023-11-27T15:17:33.776500+0000","last_fullsized":"2023-11-27T15:17:33.776500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:29:31.241865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.765022+0000","last_change":"2023-11-27T15:16:09.343896+0000","last_active":"2023-11-27T15:17:33.765022+0000","last_peered":"2023-11-27T15:17:33.765022+0000","last_clean":"2023-11-27T15:17:33.765022+0000","last_became_active":"2023-11-27T15:16:09.343603+0000","last_became_peered":"2023-11-27T15:16:09.343603+0000","last_unstale":"2023-11-27T15:17:33.765022+0000","last_undegraded":"2023-11-27T15:17:33.765022+0000","last_fullsized":"2023-11-27T15:17:33.765022+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:20:29.881258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":249380,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63670,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":533,"num_read_kb":336,"num_write":163,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1020,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":196389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449118,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6096,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739056,"statfs":{"total":95995035648,"available":95988793344,"internally_reserved":0,"allocated":925696,"data_stored":444516,"data_compressed":5787,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54000000000000004,"15min":0.53600000000000003},"min":{"1min":0.39100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51300000000000001,"15min":0.51600000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.68000000000000005,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.48199999999999998}]},{"osd":1,"last update":"Mon Nov 27 15:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.63100000000000001,"15min":0.625},"min":{"1min":0.51900000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.80500000000000005,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81399999999999995,"5min":1.262,"15min":1.262},"last":0.58799999999999997}]},{"osd":2,"last update":"Mon Nov 27 15:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.46000000000000002,"15min":0.45700000000000002},"min":{"1min":0.32800000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.58099999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.48799999999999999,"15min":0.48699999999999999},"min":{"1min":0.373,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.66400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.504}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":4,"last update":"Mon Nov 27 15:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.64900000000000002,"15min":0.64400000000000002},"min":{"1min":0.57099999999999995,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83299999999999996,"5min":1.155,"15min":1.155},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82199999999999995,"5min":1.048,"15min":1.048},"last":0.69199999999999995}]},{"osd":5,"last update":"Mon Nov 27 15:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52700000000000002,"15min":0.52600000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.63800000000000001,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58999999999999997,"15min":0.58499999999999996},"min":{"1min":0.434,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82999999999999996,"5min":1.02,"15min":1.02},"last":0.755}]},{"osd":6,"last update":"Mon Nov 27 15:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.61899999999999999}]},{"osd":7,"last update":"Mon Nov 27 15:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54200000000000004,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.91900000000000004,"5min":1.232,"15min":1.232},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.71899999999999997,"15min":0.71799999999999997},"min":{"1min":0.51500000000000001,"5min":0.503,"15min":0.503},"max":{"1min":0.85699999999999998,"5min":1.302,"15min":1.302},"last":0.78500000000000003}]},{"osd":8,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.73399999999999999}]},{"osd":9,"last update":"Mon Nov 27 15:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72199999999999998,"15min":0.72499999999999998},"min":{"1min":0.51600000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.97399999999999998,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.75600000000000001,"15min":0.752},"min":{"1min":0.628,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.96199999999999997,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.876}]},{"osd":10,"last update":"Mon Nov 27 15:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.76300000000000001,"15min":0.76100000000000001},"min":{"1min":0.59799999999999998,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.88600000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.628,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.92000000000000004,"5min":1.706,"15min":1.706},"last":0.77100000000000002}]}]},{"osd":10,"up_from":71,"seq":304942678050,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6556,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738596,"statfs":{"total":95995035648,"available":95988322304,"internally_reserved":0,"allocated":1069056,"data_stored":1095206,"data_compressed":41881,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.496},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.433}]},{"osd":1,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.44500000000000001,"15min":0.442},"min":{"1min":0.248,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.318},{"interface":"front","average":{"1min":0.434,"5min":0.45800000000000002,"15min":0.46200000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.58499999999999996,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.36499999999999999}]},{"osd":2,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.61499999999999999,"15min":0.62},"min":{"1min":0.40100000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84199999999999997,"5min":0.995,"15min":0.995},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76600000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.65700000000000003}]},{"osd":3,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.58799999999999997,"15min":0.60299999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72999999999999998,"5min":1.071,"15min":1.071},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63600000000000001,"15min":0.64500000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.80100000000000005,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.38600000000000001}]},{"osd":4,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53100000000000003,"15min":0.53700000000000003},"min":{"1min":0.34499999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.71599999999999997,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.53100000000000003,"15min":0.53800000000000003},"min":{"1min":0.35599999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71199999999999997,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.34499999999999997}]},{"osd":5,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.629,"15min":0.627},"min":{"1min":0.47499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77700000000000002,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.66500000000000004,"15min":0.65900000000000003},"min":{"1min":0.46800000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.61899999999999999}]},{"osd":6,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.71199999999999997,"15min":0.71799999999999997},"min":{"1min":0.45000000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.876,"5min":1.304,"15min":1.304},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.67000000000000004,"15min":0.68200000000000005},"min":{"1min":0.42499999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82399999999999995,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.52500000000000002}]},{"osd":7,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.53300000000000003}]},{"osd":8,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.70699999999999996,"15min":0.71299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85799999999999998,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.72999999999999998,"15min":0.73399999999999999},"min":{"1min":0.504,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.98899999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.55900000000000005}]},{"osd":9,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.73999999999999999,"15min":0.752},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.85299999999999998,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.76900000000000002,"15min":0.78300000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.85299999999999998,"5min":1.109,"15min":1.109},"last":0.48399999999999999}]},{"osd":11,"last update":"Mon Nov 27 15:17:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71699999999999997,"15min":0.71499999999999997},"min":{"1min":0.63,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89000000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.72199999999999998,"15min":0.71899999999999997},"min":{"1min":0.57999999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.63500000000000001}]}]},{"osd":9,"up_from":66,"seq":283467841575,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6232,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93738920,"statfs":{"total":95995035648,"available":95988654080,"internally_reserved":0,"allocated":933888,"data_stored":444696,"data_compressed":5787,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.437,"15min":0.443},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.65300000000000002,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44500000000000001,"15min":0.44800000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.39400000000000002}]},{"osd":1,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.61099999999999999,"15min":0.61499999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.501},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.628,"15min":0.63800000000000001},"min":{"1min":0.42499999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77900000000000003,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.51200000000000001}]},{"osd":2,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64100000000000001,"15min":0.64200000000000002},"min":{"1min":0.42899999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.441},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.46300000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.81999999999999995,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.54900000000000004}]},{"osd":3,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.52900000000000003,"15min":0.53500000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.59499999999999997,"15min":0.61199999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65500000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.32500000000000001}]},{"osd":4,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.65200000000000002,"15min":0.66600000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79300000000000004,"5min":1.236,"15min":1.236},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.52700000000000002}]},{"osd":5,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.70899999999999996,"15min":0.72699999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.82499999999999996,"5min":1.274,"15min":1.274},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.68500000000000005,"15min":0.68700000000000006},"min":{"1min":0.439,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.47899999999999998}]},{"osd":6,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.69499999999999995,"15min":0.70899999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.71199999999999997,"15min":0.73399999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.77100000000000002,"5min":1.351,"15min":1.351},"last":0.61399999999999999}]},{"osd":7,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.75800000000000001,"15min":0.76200000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.77900000000000003,"15min":0.79500000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.92000000000000004,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.58399999999999996}]},{"osd":8,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79600000000000004,"15min":0.79900000000000004},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.78500000000000003,"15min":0.80300000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.86699999999999999,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.68899999999999995}]},{"osd":10,"last update":"Mon Nov 27 15:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.74399999999999999,"15min":0.753},"min":{"1min":0.51200000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.95399999999999996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.746,"15min":0.75800000000000001},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.88800000000000001,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.70599999999999996}]}]},{"osd":8,"up_from":58,"seq":249108103211,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6420,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738732,"statfs":{"total":95995035648,"available":95988461568,"internally_reserved":0,"allocated":929792,"data_stored":444594,"data_compressed":5787,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.34899999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51500000000000001,"15min":0.5},"min":{"1min":0.40899999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.52400000000000002}]},{"osd":1,"last update":"Mon Nov 27 15:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.57199999999999995,"15min":0.55600000000000005},"min":{"1min":0.48399999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.81100000000000005,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.59099999999999997,"15min":0.56899999999999995},"min":{"1min":0.52300000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.92000000000000004,"5min":1.004,"15min":1.004},"last":0.67400000000000004}]},{"osd":2,"last update":"Mon Nov 27 15:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46200000000000002,"15min":0.44900000000000001},"min":{"1min":0.33100000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.68000000000000005,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.46800000000000003,"15min":0.44800000000000001},"min":{"1min":0.32900000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.38400000000000001}]},{"osd":3,"last update":"Mon Nov 27 15:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55200000000000005,"15min":0.53600000000000003},"min":{"1min":0.41599999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.57899999999999996,"15min":0.56100000000000005},"min":{"1min":0.44,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.56200000000000006}]},{"osd":4,"last update":"Mon Nov 27 15:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.60699999999999998,"15min":0.58699999999999997},"min":{"1min":0.53800000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79800000000000004,"5min":1.014,"15min":1.014},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.64800000000000002,"15min":0.63300000000000001},"min":{"1min":0.54000000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.84999999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.622}]},{"osd":5,"last update":"Mon Nov 27 15:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.40000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77500000000000002,"5min":0.873,"15min":0.873},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56599999999999995,"15min":0.54900000000000004},"min":{"1min":0.376,"5min":0.315,"15min":0.315},"max":{"1min":0.86599999999999999,"5min":1.056,"15min":1.056},"last":0.57999999999999996}]},{"osd":6,"last update":"Mon Nov 27 15:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66500000000000004,"15min":0.65500000000000003},"min":{"1min":0.46100000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.66500000000000004,"15min":0.65000000000000002},"min":{"1min":0.51300000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.59799999999999998}]},{"osd":7,"last update":"Mon Nov 27 15:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.69099999999999995,"15min":0.67500000000000004},"min":{"1min":0.53800000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.70799999999999996,"15min":0.68799999999999994},"min":{"1min":0.63700000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.98599999999999999,"5min":1.115,"15min":1.115},"last":0.72899999999999998}]},{"osd":9,"last update":"Mon Nov 27 15:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.72899999999999998,"15min":0.71699999999999997},"min":{"1min":0.52600000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.74399999999999999,"15min":0.73099999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.75700000000000001}]},{"osd":10,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.68400000000000005,"15min":0.66800000000000004},"min":{"1min":0.58599999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1,"5min":1,"15min":1},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.72399999999999998,"15min":0.70699999999999996},"min":{"1min":0.60599999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.80300000000000005}]},{"osd":11,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.78000000000000003}]}]},{"osd":7,"up_from":50,"seq":214748364847,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6536,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738616,"statfs":{"total":95995035648,"available":95988342784,"internally_reserved":0,"allocated":983040,"data_stored":1037445,"data_compressed":41881,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.35599999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.76300000000000001,"5min":1.052,"15min":1.052},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.51800000000000002,"15min":0.503},"min":{"1min":0.315,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.38700000000000001}]},{"osd":1,"last update":"Mon Nov 27 15:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.40500000000000003,"15min":0.38500000000000001},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.497},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.42799999999999999,"15min":0.40500000000000003},"min":{"1min":0.27900000000000003,"5min":0.254,"15min":0.254},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.59899999999999998}]},{"osd":2,"last update":"Mon Nov 27 15:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58299999999999996,"15min":0.56599999999999995},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56399999999999995,"15min":0.54400000000000004},"min":{"1min":0.38,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77500000000000002,"5min":1.077,"15min":1.077},"last":0.58199999999999996}]},{"osd":3,"last update":"Mon Nov 27 15:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54800000000000004,"15min":0.52700000000000002},"min":{"1min":0.36199999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71399999999999997,"5min":1.026,"15min":1.026},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59299999999999997,"15min":0.57999999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79100000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.628}]},{"osd":4,"last update":"Mon Nov 27 15:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.505,"15min":0.47899999999999998},"min":{"1min":0.35499999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.871,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.498,"15min":0.47699999999999998},"min":{"1min":0.318,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.73799999999999999}]},{"osd":5,"last update":"Mon Nov 27 15:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63500000000000001,"15min":0.60099999999999998},"min":{"1min":0.45600000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.96399999999999997,"5min":2.3719999999999999,"15min":2.3719999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65300000000000002,"15min":0.61599999999999999},"min":{"1min":0.47199999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76000000000000001,"5min":3.1059999999999999,"15min":3.1059999999999999},"last":0.70299999999999996}]},{"osd":6,"last update":"Mon Nov 27 15:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67200000000000004,"15min":0.65600000000000003},"min":{"1min":0.495,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.63,"15min":0.61199999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.77500000000000002}]},{"osd":8,"last update":"Mon Nov 27 15:17:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67800000000000005,"15min":0.65800000000000003},"min":{"1min":0.46500000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.97299999999999998,"5min":1.157,"15min":1.157},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.72799999999999998,"15min":0.69999999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.89100000000000001,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":0.80700000000000005}]},{"osd":9,"last update":"Mon Nov 27 15:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.752,"15min":0.75},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":0.754},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.73999999999999999,"15min":0.72999999999999998},"min":{"1min":0.50900000000000001,"5min":0.497,"15min":0.497},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":0.86399999999999999}]},{"osd":10,"last update":"Mon Nov 27 15:17:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.755,"15min":0.76500000000000001},"min":{"1min":0.52600000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.83099999999999996,"5min":1.214,"15min":1.214},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.77100000000000002,"15min":0.78900000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93400000000000005,"5min":1.095,"15min":1.095},"last":0.79500000000000004}]},{"osd":11,"last update":"Mon Nov 27 15:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.84199999999999997}]}]},{"osd":6,"up_from":45,"seq":193273528372,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6348,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738804,"statfs":{"total":95995035648,"available":95988535296,"internally_reserved":0,"allocated":921600,"data_stored":444091,"data_compressed":5787,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.438,"15min":0.44500000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.86099999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.503,"15min":0.53200000000000003},"min":{"1min":0.29899999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.83699999999999997,"5min":2.508,"15min":2.508},"last":0.36899999999999999}]},{"osd":1,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59599999999999997,"15min":0.58699999999999997},"min":{"1min":0.45000000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.61499999999999999,"15min":0.63800000000000001},"min":{"1min":0.41099999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73799999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":0.48499999999999999}]},{"osd":2,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.65300000000000002,"15min":0.67500000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91000000000000003,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.497},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.64600000000000002,"15min":0.65000000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.58399999999999996}]},{"osd":3,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52900000000000003,"15min":0.53100000000000003},"min":{"1min":0.377,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89600000000000002,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.433,"5min":0.54500000000000004,"15min":0.58699999999999997},"min":{"1min":0.33000000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.879,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":0.39000000000000001}]},{"osd":4,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.64300000000000002,"15min":0.65800000000000003},"min":{"1min":0.42699999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95299999999999996,"5min":1.51,"15min":1.51},"last":0.437},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65500000000000003,"15min":0.64300000000000002},"min":{"1min":0.495,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.012,"5min":1.054,"15min":1.054},"last":0.54800000000000004}]},{"osd":5,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.69299999999999995,"15min":0.73099999999999998},"min":{"1min":0.47399999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77700000000000002,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.70199999999999996,"15min":0.71899999999999997},"min":{"1min":0.51600000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.93300000000000005,"5min":1.599,"15min":1.599},"last":0.53000000000000003}]},{"osd":7,"last update":"Mon Nov 27 15:17:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.71199999999999997,"15min":0.71999999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88700000000000001,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73799999999999999,"15min":0.73499999999999999},"min":{"1min":0.56200000000000006,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.96099999999999997,"5min":1.206,"15min":1.206},"last":0.56899999999999995}]},{"osd":8,"last update":"Mon Nov 27 15:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.81000000000000005,"15min":0.81599999999999995},"min":{"1min":0.58699999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":4.0449999999999999,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.74299999999999999,"15min":0.79300000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.89200000000000002,"5min":2.766,"15min":2.766},"last":0.66400000000000003}]},{"osd":9,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.71599999999999997,"15min":0.74099999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.77800000000000002,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.75,"15min":0.77000000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90000000000000002,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.59999999999999998}]},{"osd":10,"last update":"Mon Nov 27 15:17:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.68700000000000006,"15min":0.68500000000000005},"min":{"1min":0.51200000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.95499999999999996,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.79300000000000004,"15min":0.79900000000000004},"min":{"1min":0.64100000000000001,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.93700000000000006,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.63300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7436,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737716,"statfs":{"total":95995035648,"available":95987421184,"internally_reserved":0,"allocated":987136,"data_stored":508351,"data_compressed":5787,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[0,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51200000000000001,"15min":0.47299999999999998},"min":{"1min":0.32700000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.52000000000000002,"15min":0.46999999999999997},"min":{"1min":0.33900000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.86799999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.41899999999999998}]},{"osd":2,"last update":"Mon Nov 27 15:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.56699999999999995,"15min":0.54700000000000004},"min":{"1min":0.40200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80900000000000005,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.622},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.55800000000000005,"15min":0.53200000000000003},"min":{"1min":0.41699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.66700000000000004}]},{"osd":3,"last update":"Mon Nov 27 15:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56999999999999995,"15min":0.53400000000000003},"min":{"1min":0.38100000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.96099999999999997,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56399999999999995,"15min":0.55700000000000005},"min":{"1min":0.32200000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.78300000000000003,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.55200000000000005}]},{"osd":4,"last update":"Mon Nov 27 15:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.45100000000000001,"15min":0.43099999999999999},"min":{"1min":0.307,"5min":0.251,"15min":0.251},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.46600000000000003,"15min":0.44400000000000001},"min":{"1min":0.33000000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.63600000000000001,"5min":0.754,"15min":0.754},"last":0.629}]},{"osd":5,"last update":"Mon Nov 27 15:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.59799999999999998,"15min":0.58899999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.63,"15min":0.623},"min":{"1min":0.53100000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.72699999999999998}]},{"osd":6,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.60599999999999998,"15min":0.58299999999999996},"min":{"1min":0.40699999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.63600000000000001,"15min":0.626},"min":{"1min":0.434,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95499999999999996,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.54200000000000004}]},{"osd":7,"last update":"Mon Nov 27 15:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54100000000000004,"15min":0.52800000000000002},"min":{"1min":0.38300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.90900000000000003,"5min":1.02,"15min":1.02},"last":0.499},{"interface":"front","average":{"1min":0.621,"5min":0.59199999999999997,"15min":0.57599999999999996},"min":{"1min":0.33900000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0549999999999999,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.58099999999999996}]},{"osd":8,"last update":"Mon Nov 27 15:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.67800000000000005,"15min":0.65400000000000003},"min":{"1min":0.52900000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91800000000000004,"5min":1.177,"15min":1.177},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.65300000000000002,"15min":0.63},"min":{"1min":0.53400000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.875,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.68200000000000005}]},{"osd":9,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72899999999999998,"15min":0.72099999999999997},"min":{"1min":0.57899999999999996,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.72899999999999998,"15min":0.72499999999999998},"min":{"1min":0.53400000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.60399999999999998}]},{"osd":10,"last update":"Mon Nov 27 15:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71699999999999997,"15min":0.71799999999999997},"min":{"1min":0.48299999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0149999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.71899999999999997,"15min":0.71299999999999997},"min":{"1min":0.57299999999999995,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.93799999999999994,"5min":0.995,"15min":0.995},"last":0.69399999999999995}]},{"osd":11,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.751}]}]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7328,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737824,"statfs":{"total":95995035648,"available":95987531776,"internally_reserved":0,"allocated":942080,"data_stored":445356,"data_compressed":5787,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 27 15:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59299999999999997,"15min":0.54900000000000004},"min":{"1min":0.45400000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79000000000000004,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.60999999999999999,"15min":0.56100000000000005},"min":{"1min":0.432,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.70099999999999996,"5min":1.55,"15min":1.55},"last":0.38900000000000001}]},{"osd":2,"last update":"Mon Nov 27 15:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58299999999999996,"15min":0.54500000000000004},"min":{"1min":0.42599999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54800000000000004,"15min":0.52100000000000002},"min":{"1min":0.35999999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.41099999999999998}]},{"osd":3,"last update":"Mon Nov 27 15:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.438},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.372}]},{"osd":4,"last update":"Mon Nov 27 15:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.56999999999999995,"15min":0.54800000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68300000000000005,"5min":0.999,"15min":0.999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.58599999999999997,"15min":0.55800000000000005},"min":{"1min":0.40500000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.67700000000000005,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.497}]},{"osd":5,"last update":"Mon Nov 27 15:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.61499999999999999},"min":{"1min":0.47699999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85499999999999998,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61799999999999999,"15min":0.60799999999999998},"min":{"1min":0.41899999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71899999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.65000000000000002}]},{"osd":6,"last update":"Mon Nov 27 15:16:43 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.308,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.63900000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.754,"5min":1.032,"15min":1.032},"last":0.61799999999999999}]},{"osd":7,"last update":"Mon Nov 27 15:16:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67800000000000005,"15min":0.67200000000000004},"min":{"1min":0.434,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.46100000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.31}]},{"osd":8,"last update":"Mon Nov 27 15:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.66000000000000003,"15min":0.69499999999999995},"min":{"1min":0.441,"5min":0.434,"15min":0.434},"max":{"1min":0.73699999999999999,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.70899999999999996,"15min":0.73899999999999999},"min":{"1min":0.45500000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.78200000000000003,"5min":1.004,"15min":1.004},"last":0.55300000000000005}]},{"osd":9,"last update":"Mon Nov 27 15:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.73499999999999999,"15min":0.76100000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88800000000000001,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.72299999999999998,"15min":0.74299999999999999},"min":{"1min":0.438,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.46899999999999997}]},{"osd":10,"last update":"Mon Nov 27 15:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.74399999999999999,"15min":0.75800000000000001},"min":{"1min":0.53000000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.81399999999999995,"5min":1.145,"15min":1.145},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.77100000000000002,"15min":0.78700000000000003},"min":{"1min":0.503,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.95399999999999996,"5min":1.248,"15min":1.248},"last":0.57499999999999996}]},{"osd":11,"last update":"Mon Nov 27 15:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65000000000000002,"15min":0.64900000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72699999999999998,"15min":0.73199999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.82399999999999995,"5min":0.879,"15min":0.879},"last":0.66300000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7508,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":6528,"kb_avail":93737644,"statfs":{"total":95995035648,"available":95987347456,"internally_reserved":0,"allocated":995328,"data_stored":503348,"data_compressed":5787,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":6684672},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.61199999999999999,"15min":0.51100000000000001},"min":{"1min":0.35199999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":8.5069999999999997,"5min":8.5069999999999997,"15min":8.5069999999999997},"last":0.442},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.50900000000000001,"15min":0.48399999999999999},"min":{"1min":0.36199999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.47699999999999998}]},{"osd":1,"last update":"Mon Nov 27 15:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.56899999999999995,"15min":0.52200000000000002},"min":{"1min":0.45900000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.53900000000000003,"15min":0.498},"min":{"1min":0.48099999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.57999999999999996}]},{"osd":3,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.59899999999999998,"15min":0.54600000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.2170000000000001,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.59599999999999997,"15min":0.54600000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":3.278,"5min":3.278,"15min":3.278},"last":0.50900000000000001}]},{"osd":4,"last update":"Mon Nov 27 15:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.63600000000000001}]},{"osd":5,"last update":"Mon Nov 27 15:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.47099999999999997,"15min":0.41299999999999998},"min":{"1min":0.26600000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.86799999999999999,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.48099999999999998,"15min":0.42699999999999999},"min":{"1min":0.33100000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.61299999999999999}]},{"osd":6,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.58599999999999997,"15min":0.56499999999999995},"min":{"1min":0.44900000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.70399999999999996,"15min":0.64300000000000002},"min":{"1min":0.41599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":4.5800000000000001,"5min":4.5800000000000001,"15min":4.5800000000000001},"last":0.754}]},{"osd":7,"last update":"Mon Nov 27 15:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.39800000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.44},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69199999999999995,"15min":0.69599999999999995},"min":{"1min":0.46300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93300000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.51100000000000001}]},{"osd":8,"last update":"Mon Nov 27 15:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.65900000000000003,"15min":0.66100000000000003},"min":{"1min":0.35099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.94799999999999995,"5min":0.997,"15min":0.997},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.59999999999999998,"15min":0.58199999999999996},"min":{"1min":0.41999999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.66800000000000004}]},{"osd":9,"last update":"Mon Nov 27 15:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.72199999999999998,"15min":0.70199999999999996},"min":{"1min":0.49399999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":3.633,"5min":3.633,"15min":3.633},"last":0.71699999999999997},{"interface":"front","average":{"1min":1.036,"5min":0.79500000000000004,"15min":0.755},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":7.6369999999999996,"5min":7.6369999999999996,"15min":7.6369999999999996},"last":0.67900000000000005}]},{"osd":10,"last update":"Mon Nov 27 15:17:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.67700000000000005,"15min":0.66200000000000003},"min":{"1min":0.59299999999999997,"5min":0.496,"15min":0.496},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.70299999999999996,"15min":0.67900000000000005},"min":{"1min":0.67300000000000004,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.76300000000000001}]},{"osd":11,"last update":"Mon Nov 27 15:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.78300000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182464,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6684,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738468,"statfs":{"total":95995035648,"available":95988191232,"internally_reserved":0,"allocated":937984,"data_stored":444408,"data_compressed":5787,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.36599999999999999,"15min":0.33500000000000002},"min":{"1min":0.254,"5min":0.22,"15min":0.22},"max":{"1min":0.60499999999999998,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.38500000000000001,"15min":0.35699999999999998},"min":{"1min":0.27300000000000002,"5min":0.214,"15min":0.214},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.35899999999999999}]},{"osd":1,"last update":"Mon Nov 27 15:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53800000000000003,"15min":0.505},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78500000000000003,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53800000000000003,"15min":0.52400000000000002},"min":{"1min":0.376,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82599999999999996,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.499}]},{"osd":2,"last update":"Mon Nov 27 15:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58699999999999997,"15min":0.54500000000000004},"min":{"1min":0.40200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80100000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58099999999999996,"15min":0.54400000000000004},"min":{"1min":0.41699999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.81100000000000005,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.47999999999999998}]},{"osd":4,"last update":"Mon Nov 27 15:17:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55300000000000005,"15min":0.52700000000000002},"min":{"1min":0.41799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78900000000000003,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.52800000000000002},"min":{"1min":0.40300000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79700000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.56999999999999995}]},{"osd":5,"last update":"Mon Nov 27 15:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.624,"15min":0.58199999999999996},"min":{"1min":0.44900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.64300000000000002,"15min":0.59599999999999997},"min":{"1min":0.47299999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":0.51100000000000001}]},{"osd":6,"last update":"Mon Nov 27 15:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56999999999999995,"15min":0.55000000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.91100000000000003,"5min":1.089,"15min":1.089},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55900000000000005,"15min":0.54300000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.42899999999999999}]},{"osd":7,"last update":"Mon Nov 27 15:17:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.45900000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.80100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72599999999999998,"15min":0.70899999999999996},"min":{"1min":0.57299999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.83299999999999996,"5min":1.034,"15min":1.034},"last":0.67300000000000004}]},{"osd":8,"last update":"Mon Nov 27 15:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.628}]},{"osd":9,"last update":"Mon Nov 27 15:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.80600000000000005,"15min":0.82599999999999996},"min":{"1min":0.49299999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.083,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.625},{"interface":"front","average":{"1min":0.63,"5min":0.73799999999999999,"15min":0.75600000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95399999999999996,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.56200000000000006}]},{"osd":10,"last update":"Mon Nov 27 15:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.71799999999999997,"15min":0.73199999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.77200000000000002,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.84399999999999997,"15min":0.86199999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.86899999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.65200000000000002}]},{"osd":11,"last update":"Mon Nov 27 15:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.64000000000000001}]}]},{"osd":4,"up_from":33,"seq":141733920829,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6552,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738600,"statfs":{"total":95995035648,"available":95988326400,"internally_reserved":0,"allocated":933888,"data_stored":446428,"data_compressed":5787,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.48099999999999998,"15min":0.46800000000000003},"min":{"1min":0.35299999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.52200000000000002,"15min":0.50700000000000001},"min":{"1min":0.40300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.432}]},{"osd":1,"last update":"Mon Nov 27 15:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.39400000000000002,"15min":0.35599999999999998},"min":{"1min":0.29999999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.77200000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.39900000000000002,"15min":0.35699999999999998},"min":{"1min":0.318,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.90500000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.55600000000000005}]},{"osd":2,"last update":"Mon Nov 27 15:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.52800000000000002,"15min":0.496},"min":{"1min":0.36599999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.56299999999999994,"15min":0.54100000000000004},"min":{"1min":0.42599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.65000000000000002}]},{"osd":3,"last update":"Mon Nov 27 15:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.56299999999999994,"15min":0.52200000000000002},"min":{"1min":0.48799999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.872,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.71899999999999997}]},{"osd":5,"last update":"Mon Nov 27 15:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.58399999999999996,"15min":0.54300000000000004},"min":{"1min":0.47599999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.60299999999999998,"15min":0.56799999999999995},"min":{"1min":0.56100000000000005,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.74399999999999999}]},{"osd":6,"last update":"Mon Nov 27 15:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.63500000000000001,"15min":0.59999999999999998},"min":{"1min":0.46500000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":5.1040000000000001,"5min":5.1040000000000001,"15min":5.1040000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.67400000000000004,"15min":0.63800000000000001},"min":{"1min":0.46700000000000003,"5min":0.441,"15min":0.441},"max":{"1min":5.1139999999999999,"5min":5.1139999999999999,"15min":5.1139999999999999},"last":0.76700000000000002}]},{"osd":7,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.56000000000000005,"15min":0.53300000000000003},"min":{"1min":0.376,"5min":0.316,"15min":0.316},"max":{"1min":0.90300000000000002,"5min":1.018,"15min":1.018},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58799999999999997,"15min":0.55500000000000005},"min":{"1min":0.39400000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.873,"5min":1.115,"15min":1.115},"last":0.81599999999999995}]},{"osd":8,"last update":"Mon Nov 27 15:17:25 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.68799999999999994,"15min":0.66600000000000004},"min":{"1min":0.51900000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94299999999999995,"5min":1.054,"15min":1.054},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.71999999999999997,"15min":0.70099999999999996},"min":{"1min":0.63100000000000001,"5min":0.432,"15min":0.432},"max":{"1min":1.022,"5min":1.179,"15min":1.179},"last":0.84999999999999998}]},{"osd":9,"last update":"Mon Nov 27 15:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.78100000000000003,"15min":0.75900000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":4.8780000000000001,"5min":4.8780000000000001,"15min":4.8780000000000001},"last":0.878},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.747,"15min":0.72299999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":4.9169999999999998,"5min":4.9169999999999998,"15min":4.9169999999999998},"last":0.86099999999999999}]},{"osd":10,"last update":"Mon Nov 27 15:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.75900000000000001,"15min":0.76000000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95899999999999996,"5min":1.103,"15min":1.103},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.91200000000000003}]},{"osd":11,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.92500000000000004}]}]},{"osd":5,"up_from":38,"seq":163208757304,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6676,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738476,"statfs":{"total":95995035648,"available":95988199424,"internally_reserved":0,"allocated":995328,"data_stored":1036328,"data_compressed":41881,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.50900000000000001,"15min":0.497},"min":{"1min":0.37,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.56000000000000005,"15min":0.498},"min":{"1min":0.39900000000000002,"5min":0.312,"15min":0.312},"max":{"1min":7.673,"5min":7.673,"15min":7.673},"last":0.44}]},{"osd":1,"last update":"Mon Nov 27 15:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.56899999999999995,"15min":0.53500000000000003},"min":{"1min":0.44700000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.57099999999999995,"15min":0.52500000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.60599999999999998}]},{"osd":2,"last update":"Mon Nov 27 15:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.41899999999999998,"15min":0.39600000000000002},"min":{"1min":0.29499999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.60499999999999998,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.628},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.44500000000000001,"15min":0.41299999999999998},"min":{"1min":0.28399999999999997,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.70599999999999996,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.54500000000000004}]},{"osd":3,"last update":"Mon Nov 27 15:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.623,"15min":0.57099999999999995},"min":{"1min":0.433,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":7.875,"5min":7.875,"15min":7.875},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.621,"15min":0.56200000000000006},"min":{"1min":0.46500000000000002,"5min":0.372,"15min":0.372},"max":{"1min":7.6059999999999999,"5min":7.6059999999999999,"15min":7.6059999999999999},"last":0.51700000000000002}]},{"osd":4,"last update":"Mon Nov 27 15:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.60299999999999998,"15min":0.56399999999999995},"min":{"1min":0.50600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.58199999999999996},"min":{"1min":0.42699999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.64700000000000002}]},{"osd":6,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":0.68500000000000005,"15min":0.61899999999999999},"min":{"1min":0.50800000000000001,"5min":0.436,"15min":0.436},"max":{"1min":8.0280000000000005,"5min":8.0280000000000005,"15min":8.0280000000000005},"last":0.69899999999999995},{"interface":"front","average":{"1min":1.1319999999999999,"5min":0.73899999999999999,"15min":0.67100000000000004},"min":{"1min":0.51800000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":8.0389999999999997,"5min":8.0389999999999997,"15min":8.0389999999999997},"last":0.68799999999999994}]},{"osd":7,"last update":"Mon Nov 27 15:17:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.67800000000000005,"15min":0.65900000000000003},"min":{"1min":0.498,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67000000000000004,"15min":0.66000000000000003},"min":{"1min":0.56100000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90400000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.73799999999999999}]},{"osd":8,"last update":"Mon Nov 27 15:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.66400000000000003,"15min":0.64400000000000002},"min":{"1min":0.38600000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.752},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.60899999999999999,"15min":0.59699999999999998},"min":{"1min":0.39700000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80900000000000005,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.71399999999999997}]},{"osd":9,"last update":"Mon Nov 27 15:16:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":0.747,"15min":0.69199999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":7.8070000000000004,"5min":7.8070000000000004,"15min":7.8070000000000004},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.194,"5min":0.81899999999999995,"15min":0.75700000000000001},"min":{"1min":0.56599999999999995,"5min":0.503,"15min":0.503},"max":{"1min":7.7690000000000001,"5min":7.7690000000000001,"15min":7.7690000000000001},"last":0.79200000000000004}]},{"osd":10,"last update":"Mon Nov 27 15:17:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.76000000000000001,"15min":0.747},"min":{"1min":0.63900000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.75,"5min":0.69599999999999995,"15min":0.68700000000000006},"min":{"1min":0.57099999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.77900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34455,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34455,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":59470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-27T15:17:42.448 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-27T15:17:42.608 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.604+0000 7f366d1b8700 1 -- 172.21.15.52:0/243787266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3668101480 msgr2=0x7f36681018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:42.608 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.604+0000 7f366d1b8700 1 --2- 172.21.15.52:0/243787266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3668101480 0x7f36681018d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f365c009990 tx=0x7f365c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.608 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.604+0000 7f366d1b8700 1 -- 172.21.15.52:0/243787266 shutdown_connections 2023-11-27T15:17:42.608 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.604+0000 7f366d1b8700 1 --2- 172.21.15.52:0/243787266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3668101e10 0x7f36681086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.604+0000 7f366d1b8700 1 --2- 172.21.15.52:0/243787266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3668100290 0x7f36681006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.604+0000 7f366d1b8700 1 --2- 172.21.15.52:0/243787266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3668101480 0x7f36681018d0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.604+0000 7f366d1b8700 1 -- 172.21.15.52:0/243787266 >> 172.21.15.52:0/243787266 conn(0x7f36680fb870 msgr2=0x7f36680fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:42.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.604+0000 7f366d1b8700 1 -- 172.21.15.52:0/243787266 shutdown_connections 2023-11-27T15:17:42.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.604+0000 7f366d1b8700 1 -- 172.21.15.52:0/243787266 wait complete. 2023-11-27T15:17:42.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.604+0000 7f366d1b8700 1 Processor -- start 2023-11-27T15:17:42.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.605+0000 7f366d1b8700 1 -- start start 2023-11-27T15:17:42.612 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.605+0000 7f366d1b8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3668100290 0x7f36680748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:42.612 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.605+0000 7f366d1b8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3668101480 0x7f3668072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:42.612 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.605+0000 7f366d1b8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3668101e10 0x7f3668073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:42.612 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.605+0000 7f366d1b8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3668073bc0 con 0x7f3668101480 2023-11-27T15:17:42.612 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.605+0000 7f366d1b8700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3668073d30 con 0x7f3668101e10 2023-11-27T15:17:42.612 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.605+0000 7f366d1b8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366819ffc0 con 0x7f3668100290 2023-11-27T15:17:42.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.605+0000 7f366659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3668101480 0x7f3668072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:42.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.605+0000 7f366659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3668101480 0x7f3668072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51488/0 (socket says 172.21.15.52:51488) 2023-11-27T15:17:42.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.605+0000 7f366659c700 1 -- 172.21.15.52:0/4117522761 learned_addr learned my addr 172.21.15.52:0/4117522761 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:42.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.605+0000 7f3666d9d700 1 --2- 172.21.15.52:0/4117522761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3668100290 0x7f36680748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:42.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.605+0000 7f366759e700 1 --2- 172.21.15.52:0/4117522761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3668101e10 0x7f3668073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:42.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.605+0000 7f366659c700 1 -- 172.21.15.52:0/4117522761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3668100290 msgr2=0x7f36680748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:42.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.605+0000 7f366659c700 1 --2- 172.21.15.52:0/4117522761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3668100290 0x7f36680748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.605+0000 7f366659c700 1 -- 172.21.15.52:0/4117522761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3668101e10 msgr2=0x7f3668073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:42.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.605+0000 7f366659c700 1 --2- 172.21.15.52:0/4117522761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3668101e10 0x7f3668073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.605+0000 7f366659c700 1 -- 172.21.15.52:0/4117522761 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365c0095f0 con 0x7f3668101480 2023-11-27T15:17:42.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.605+0000 7f3666d9d700 1 --2- 172.21.15.52:0/4117522761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3668100290 0x7f36680748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:17:42.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.606+0000 7f366659c700 1 --2- 172.21.15.52:0/4117522761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3668101480 0x7f3668072ee0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f365c000c00 tx=0x7f365c0047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:42.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.606+0000 7f364ffff700 1 -- 172.21.15.52:0/4117522761 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c01d070 con 0x7f3668101480 2023-11-27T15:17:42.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.606+0000 7f366d1b8700 1 -- 172.21.15.52:0/4117522761 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f366810bc00 con 0x7f3668101480 2023-11-27T15:17:42.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.606+0000 7f364ffff700 1 -- 172.21.15.52:0/4117522761 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f365c022470 con 0x7f3668101480 2023-11-27T15:17:42.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.606+0000 7f364ffff700 1 -- 172.21.15.52:0/4117522761 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c017650 con 0x7f3668101480 2023-11-27T15:17:42.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.606+0000 7f366d1b8700 1 -- 172.21.15.52:0/4117522761 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f366810ba70 con 0x7f3668101480 2023-11-27T15:17:42.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.607+0000 7f366d1b8700 1 -- 172.21.15.52:0/4117522761 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36681a0570 con 0x7f3668101480 2023-11-27T15:17:42.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.608+0000 7f364ffff700 1 -- 172.21.15.52:0/4117522761 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f365c0225e0 con 0x7f3668101480 2023-11-27T15:17:42.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.608+0000 7f364ffff700 1 --2- 172.21.15.52:0/4117522761 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f36500a1150 0x7f36500a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:42.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.609+0000 7f364ffff700 1 -- 172.21.15.52:0/4117522761 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f365c0ce2e0 con 0x7f3668101480 2023-11-27T15:17:42.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.609+0000 7f3666d9d700 1 --2- 172.21.15.52:0/4117522761 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f36500a1150 0x7f36500a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:42.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.609+0000 7f3666d9d700 1 --2- 172.21.15.52:0/4117522761 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f36500a1150 0x7f36500a3600 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3658005950 tx=0x7f36580058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:42.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.612+0000 7f364ffff700 1 -- 172.21.15.52:0/4117522761 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f365c097aa0 con 0x7f3668101480 2023-11-27T15:17:42.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.809+0000 7f366d1b8700 1 -- 172.21.15.52:0/4117522761 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3668060f30 con 0x7f36500a1150 2023-11-27T15:17:42.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.822+0000 7f364ffff700 1 -- 172.21.15.52:0/4117522761 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344013 (secure 0 0 0) 0x7f3668060f30 con 0x7f36500a1150 2023-11-27T15:17:42.830 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:17:42.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.825+0000 7f366d1b8700 1 -- 172.21.15.52:0/4117522761 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f36500a1150 msgr2=0x7f36500a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:42.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.825+0000 7f366d1b8700 1 --2- 172.21.15.52:0/4117522761 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f36500a1150 0x7f36500a3600 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3658005950 tx=0x7f36580058e0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.825+0000 7f366d1b8700 1 -- 172.21.15.52:0/4117522761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3668101480 msgr2=0x7f3668072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:42.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.825+0000 7f366d1b8700 1 --2- 172.21.15.52:0/4117522761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3668101480 0x7f3668072ee0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f365c000c00 tx=0x7f365c0047a0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.826+0000 7f366d1b8700 1 -- 172.21.15.52:0/4117522761 shutdown_connections 2023-11-27T15:17:42.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.826+0000 7f366d1b8700 1 --2- 172.21.15.52:0/4117522761 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f36500a1150 0x7f36500a3600 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.826+0000 7f366d1b8700 1 --2- 172.21.15.52:0/4117522761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3668101e10 0x7f3668073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.826+0000 7f366d1b8700 1 --2- 172.21.15.52:0/4117522761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3668100290 0x7f36680748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.826+0000 7f366d1b8700 1 --2- 172.21.15.52:0/4117522761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3668101480 0x7f3668072ee0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.826+0000 7f366d1b8700 1 -- 172.21.15.52:0/4117522761 >> 172.21.15.52:0/4117522761 conn(0x7f36680fb870 msgr2=0x7f36680fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:42.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.826+0000 7f366d1b8700 1 -- 172.21.15.52:0/4117522761 shutdown_connections 2023-11-27T15:17:42.834 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:42.826+0000 7f366d1b8700 1 -- 172.21.15.52:0/4117522761 wait complete. 2023-11-27T15:17:42.834 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2023-11-27T15:17:42.842 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2023-11-27T15:17:41.748543+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654284,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":669,"num_read_kb":455,"num_write":346,"num_write_kb":3053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181756,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":782,"ondisk_log_size":782,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":80372,"kb_used_data":11284,"kb_used_omap":0,"kb_used_meta":68992,"kb_avail":1124861452,"statfs":{"total":1151940427776,"available":1151858126848,"internally_reserved":0,"allocated":11554816,"data_stored":7294767,"data_compressed":177726,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":0,"internal_metadata":70647808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":849,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":34,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005611"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.765734+0000","last_change":"2023-11-27T15:16:09.343007+0000","last_active":"2023-11-27T15:17:33.765734+0000","last_peered":"2023-11-27T15:17:33.765734+0000","last_clean":"2023-11-27T15:17:33.765734+0000","last_became_active":"2023-11-27T15:16:09.342125+0000","last_became_peered":"2023-11-27T15:16:09.342125+0000","last_unstale":"2023-11-27T15:17:33.765734+0000","last_undegraded":"2023-11-27T15:17:33.765734+0000","last_fullsized":"2023-11-27T15:17:33.765734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:24:26.323974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.764972+0000","last_change":"2023-11-27T15:16:09.338942+0000","last_active":"2023-11-27T15:17:33.764972+0000","last_peered":"2023-11-27T15:17:33.764972+0000","last_clean":"2023-11-27T15:17:33.764972+0000","last_became_active":"2023-11-27T15:16:09.338636+0000","last_became_peered":"2023-11-27T15:16:09.338636+0000","last_unstale":"2023-11-27T15:17:33.764972+0000","last_undegraded":"2023-11-27T15:17:33.764972+0000","last_fullsized":"2023-11-27T15:17:33.764972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:40:36.907284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.765902+0000","last_change":"2023-11-27T15:16:09.347845+0000","last_active":"2023-11-27T15:17:33.765902+0000","last_peered":"2023-11-27T15:17:33.765902+0000","last_clean":"2023-11-27T15:17:33.765902+0000","last_became_active":"2023-11-27T15:16:09.347533+0000","last_became_peered":"2023-11-27T15:16:09.347533+0000","last_unstale":"2023-11-27T15:17:33.765902+0000","last_undegraded":"2023-11-27T15:17:33.765902+0000","last_fullsized":"2023-11-27T15:17:33.765902+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:30:09.473655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.762843+0000","last_change":"2023-11-27T15:17:04.811819+0000","last_active":"2023-11-27T15:17:33.762843+0000","last_peered":"2023-11-27T15:17:33.762843+0000","last_clean":"2023-11-27T15:17:33.762843+0000","last_became_active":"2023-11-27T15:17:04.811443+0000","last_became_peered":"2023-11-27T15:17:04.811443+0000","last_unstale":"2023-11-27T15:17:33.762843+0000","last_undegraded":"2023-11-27T15:17:33.762843+0000","last_fullsized":"2023-11-27T15:17:33.762843+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:03:57.997950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.772322+0000","last_change":"2023-11-27T15:16:09.349097+0000","last_active":"2023-11-27T15:17:33.772322+0000","last_peered":"2023-11-27T15:17:33.772322+0000","last_clean":"2023-11-27T15:17:33.772322+0000","last_became_active":"2023-11-27T15:16:09.348793+0000","last_became_peered":"2023-11-27T15:16:09.348793+0000","last_unstale":"2023-11-27T15:17:33.772322+0000","last_undegraded":"2023-11-27T15:17:33.772322+0000","last_fullsized":"2023-11-27T15:17:33.772322+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:26:40.373634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.765958+0000","last_change":"2023-11-27T15:16:09.338246+0000","last_active":"2023-11-27T15:17:33.765958+0000","last_peered":"2023-11-27T15:17:33.765958+0000","last_clean":"2023-11-27T15:17:33.765958+0000","last_became_active":"2023-11-27T15:16:09.338003+0000","last_became_peered":"2023-11-27T15:16:09.338003+0000","last_unstale":"2023-11-27T15:17:33.765958+0000","last_undegraded":"2023-11-27T15:17:33.765958+0000","last_fullsized":"2023-11-27T15:17:33.765958+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:50:11.348331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.773179+0000","last_change":"2023-11-27T15:16:09.342131+0000","last_active":"2023-11-27T15:17:33.773179+0000","last_peered":"2023-11-27T15:17:33.773179+0000","last_clean":"2023-11-27T15:17:33.773179+0000","last_became_active":"2023-11-27T15:16:09.341834+0000","last_became_peered":"2023-11-27T15:16:09.341834+0000","last_unstale":"2023-11-27T15:17:33.773179+0000","last_undegraded":"2023-11-27T15:17:33.773179+0000","last_fullsized":"2023-11-27T15:17:33.773179+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:22:39.182216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.776357+0000","last_change":"2023-11-27T15:16:09.337308+0000","last_active":"2023-11-27T15:17:33.776357+0000","last_peered":"2023-11-27T15:17:33.776357+0000","last_clean":"2023-11-27T15:17:33.776357+0000","last_became_active":"2023-11-27T15:16:09.336958+0000","last_became_peered":"2023-11-27T15:16:09.336958+0000","last_unstale":"2023-11-27T15:17:33.776357+0000","last_undegraded":"2023-11-27T15:17:33.776357+0000","last_fullsized":"2023-11-27T15:17:33.776357+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:00:00.299148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.771230+0000","last_change":"2023-11-27T15:17:04.810530+0000","last_active":"2023-11-27T15:17:33.771230+0000","last_peered":"2023-11-27T15:17:33.771230+0000","last_clean":"2023-11-27T15:17:33.771230+0000","last_became_active":"2023-11-27T15:17:04.810197+0000","last_became_peered":"2023-11-27T15:17:04.810197+0000","last_unstale":"2023-11-27T15:17:33.771230+0000","last_undegraded":"2023-11-27T15:17:33.771230+0000","last_fullsized":"2023-11-27T15:17:33.771230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:58:40.385968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,11],"acting":[1,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.771264+0000","last_change":"2023-11-27T15:16:09.336871+0000","last_active":"2023-11-27T15:17:33.771264+0000","last_peered":"2023-11-27T15:17:33.771264+0000","last_clean":"2023-11-27T15:17:33.771264+0000","last_became_active":"2023-11-27T15:16:09.336496+0000","last_became_peered":"2023-11-27T15:16:09.336496+0000","last_unstale":"2023-11-27T15:17:33.771264+0000","last_undegraded":"2023-11-27T15:17:33.771264+0000","last_fullsized":"2023-11-27T15:17:33.771264+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:12:27.254009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.776130+0000","last_change":"2023-11-27T15:16:09.343120+0000","last_active":"2023-11-27T15:17:33.776130+0000","last_peered":"2023-11-27T15:17:33.776130+0000","last_clean":"2023-11-27T15:17:33.776130+0000","last_became_active":"2023-11-27T15:16:09.342797+0000","last_became_peered":"2023-11-27T15:16:09.342797+0000","last_unstale":"2023-11-27T15:17:33.776130+0000","last_undegraded":"2023-11-27T15:17:33.776130+0000","last_fullsized":"2023-11-27T15:17:33.776130+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:22:16.706218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.759194+0000","last_change":"2023-11-27T15:16:09.336521+0000","last_active":"2023-11-27T15:17:33.759194+0000","last_peered":"2023-11-27T15:17:33.759194+0000","last_clean":"2023-11-27T15:17:33.759194+0000","last_became_active":"2023-11-27T15:16:09.336183+0000","last_became_peered":"2023-11-27T15:16:09.336183+0000","last_unstale":"2023-11-27T15:17:33.759194+0000","last_undegraded":"2023-11-27T15:17:33.759194+0000","last_fullsized":"2023-11-27T15:17:33.759194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:54:55.143503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.775471+0000","last_change":"2023-11-27T15:16:09.346839+0000","last_active":"2023-11-27T15:17:33.775471+0000","last_peered":"2023-11-27T15:17:33.775471+0000","last_clean":"2023-11-27T15:17:33.775471+0000","last_became_active":"2023-11-27T15:16:09.345230+0000","last_became_peered":"2023-11-27T15:16:09.345230+0000","last_unstale":"2023-11-27T15:17:33.775471+0000","last_undegraded":"2023-11-27T15:17:33.775471+0000","last_fullsized":"2023-11-27T15:17:33.775471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:35:36.915491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.758451+0000","last_change":"2023-11-27T15:16:09.342380+0000","last_active":"2023-11-27T15:17:33.758451+0000","last_peered":"2023-11-27T15:17:33.758451+0000","last_clean":"2023-11-27T15:17:33.758451+0000","last_became_active":"2023-11-27T15:16:09.341967+0000","last_became_peered":"2023-11-27T15:16:09.341967+0000","last_unstale":"2023-11-27T15:17:33.758451+0000","last_undegraded":"2023-11-27T15:17:33.758451+0000","last_fullsized":"2023-11-27T15:17:33.758451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:12:37.584284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.754779+0000","last_change":"2023-11-27T15:16:09.349738+0000","last_active":"2023-11-27T15:17:33.754779+0000","last_peered":"2023-11-27T15:17:33.754779+0000","last_clean":"2023-11-27T15:17:33.754779+0000","last_became_active":"2023-11-27T15:16:09.349349+0000","last_became_peered":"2023-11-27T15:16:09.349349+0000","last_unstale":"2023-11-27T15:17:33.754779+0000","last_undegraded":"2023-11-27T15:17:33.754779+0000","last_fullsized":"2023-11-27T15:17:33.754779+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:59:30.065309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.776017+0000","last_change":"2023-11-27T15:16:09.339171+0000","last_active":"2023-11-27T15:17:33.776017+0000","last_peered":"2023-11-27T15:17:33.776017+0000","last_clean":"2023-11-27T15:17:33.776017+0000","last_became_active":"2023-11-27T15:16:09.338887+0000","last_became_peered":"2023-11-27T15:16:09.338887+0000","last_unstale":"2023-11-27T15:17:33.776017+0000","last_undegraded":"2023-11-27T15:17:33.776017+0000","last_fullsized":"2023-11-27T15:17:33.776017+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:08:50.297424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.768341+0000","last_change":"2023-11-27T15:16:09.343202+0000","last_active":"2023-11-27T15:17:33.768341+0000","last_peered":"2023-11-27T15:17:33.768341+0000","last_clean":"2023-11-27T15:17:33.768341+0000","last_became_active":"2023-11-27T15:16:09.342539+0000","last_became_peered":"2023-11-27T15:16:09.342539+0000","last_unstale":"2023-11-27T15:17:33.768341+0000","last_undegraded":"2023-11-27T15:17:33.768341+0000","last_fullsized":"2023-11-27T15:17:33.768341+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:53:20.106298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.772964+0000","last_change":"2023-11-27T15:16:09.339915+0000","last_active":"2023-11-27T15:17:33.772964+0000","last_peered":"2023-11-27T15:17:33.772964+0000","last_clean":"2023-11-27T15:17:33.772964+0000","last_became_active":"2023-11-27T15:16:09.339637+0000","last_became_peered":"2023-11-27T15:16:09.339637+0000","last_unstale":"2023-11-27T15:17:33.772964+0000","last_undegraded":"2023-11-27T15:17:33.772964+0000","last_fullsized":"2023-11-27T15:17:33.772964+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:04:48.736761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.773767+0000","last_change":"2023-11-27T15:16:09.347852+0000","last_active":"2023-11-27T15:17:33.773767+0000","last_peered":"2023-11-27T15:17:33.773767+0000","last_clean":"2023-11-27T15:17:33.773767+0000","last_became_active":"2023-11-27T15:16:09.347281+0000","last_became_peered":"2023-11-27T15:16:09.347281+0000","last_unstale":"2023-11-27T15:17:33.773767+0000","last_undegraded":"2023-11-27T15:17:33.773767+0000","last_fullsized":"2023-11-27T15:17:33.773767+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:29:49.527742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.806625+0000","last_change":"2023-11-27T15:16:08.912206+0000","last_active":"2023-11-27T15:17:04.806625+0000","last_peered":"2023-11-27T15:17:04.806625+0000","last_clean":"2023-11-27T15:17:04.806625+0000","last_became_active":"2023-11-27T15:16:08.911846+0000","last_became_peered":"2023-11-27T15:16:08.911846+0000","last_unstale":"2023-11-27T15:17:04.806625+0000","last_undegraded":"2023-11-27T15:17:04.806625+0000","last_fullsized":"2023-11-27T15:17:04.806625+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:38:19.552326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.770773+0000","last_change":"2023-11-27T15:16:09.336485+0000","last_active":"2023-11-27T15:17:33.770773+0000","last_peered":"2023-11-27T15:17:33.770773+0000","last_clean":"2023-11-27T15:17:33.770773+0000","last_became_active":"2023-11-27T15:16:09.336236+0000","last_became_peered":"2023-11-27T15:16:09.336236+0000","last_unstale":"2023-11-27T15:17:33.770773+0000","last_undegraded":"2023-11-27T15:17:33.770773+0000","last_fullsized":"2023-11-27T15:17:33.770773+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:56:02.954043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.808190+0000","last_change":"2023-11-27T15:16:08.921198+0000","last_active":"2023-11-27T15:17:04.808190+0000","last_peered":"2023-11-27T15:17:04.808190+0000","last_clean":"2023-11-27T15:17:04.808190+0000","last_became_active":"2023-11-27T15:16:08.920804+0000","last_became_peered":"2023-11-27T15:16:08.920804+0000","last_unstale":"2023-11-27T15:17:04.808190+0000","last_undegraded":"2023-11-27T15:17:04.808190+0000","last_fullsized":"2023-11-27T15:17:04.808190+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:29:23.298656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.767388+0000","last_change":"2023-11-27T15:16:09.340642+0000","last_active":"2023-11-27T15:17:33.767388+0000","last_peered":"2023-11-27T15:17:33.767388+0000","last_clean":"2023-11-27T15:17:33.767388+0000","last_became_active":"2023-11-27T15:16:09.340328+0000","last_became_peered":"2023-11-27T15:16:09.340328+0000","last_unstale":"2023-11-27T15:17:33.767388+0000","last_undegraded":"2023-11-27T15:17:33.767388+0000","last_fullsized":"2023-11-27T15:17:33.767388+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:46:04.794567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.810213+0000","last_change":"2023-11-27T15:16:08.915317+0000","last_active":"2023-11-27T15:17:04.810213+0000","last_peered":"2023-11-27T15:17:04.810213+0000","last_clean":"2023-11-27T15:17:04.810213+0000","last_became_active":"2023-11-27T15:16:08.914949+0000","last_became_peered":"2023-11-27T15:16:08.914949+0000","last_unstale":"2023-11-27T15:17:04.810213+0000","last_undegraded":"2023-11-27T15:17:04.810213+0000","last_fullsized":"2023-11-27T15:17:04.810213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:06:23.361651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.773082+0000","last_change":"2023-11-27T15:16:09.348483+0000","last_active":"2023-11-27T15:17:33.773082+0000","last_peered":"2023-11-27T15:17:33.773082+0000","last_clean":"2023-11-27T15:17:33.773082+0000","last_became_active":"2023-11-27T15:16:09.348185+0000","last_became_peered":"2023-11-27T15:16:09.348185+0000","last_unstale":"2023-11-27T15:17:33.773082+0000","last_undegraded":"2023-11-27T15:17:33.773082+0000","last_fullsized":"2023-11-27T15:17:33.773082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:48:03.154051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.809047+0000","last_change":"2023-11-27T15:16:08.911018+0000","last_active":"2023-11-27T15:17:04.809047+0000","last_peered":"2023-11-27T15:17:04.809047+0000","last_clean":"2023-11-27T15:17:04.809047+0000","last_became_active":"2023-11-27T15:16:08.910684+0000","last_became_peered":"2023-11-27T15:16:08.910684+0000","last_unstale":"2023-11-27T15:17:04.809047+0000","last_undegraded":"2023-11-27T15:17:04.809047+0000","last_fullsized":"2023-11-27T15:17:04.809047+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:08:38.910735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.774391+0000","last_change":"2023-11-27T15:16:09.341414+0000","last_active":"2023-11-27T15:17:33.774391+0000","last_peered":"2023-11-27T15:17:33.774391+0000","last_clean":"2023-11-27T15:17:33.774391+0000","last_became_active":"2023-11-27T15:16:09.341068+0000","last_became_peered":"2023-11-27T15:16:09.341068+0000","last_unstale":"2023-11-27T15:17:33.774391+0000","last_undegraded":"2023-11-27T15:17:33.774391+0000","last_fullsized":"2023-11-27T15:17:33.774391+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:24:59.603494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'7","reported_seq":125,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:36.892827+0000","last_change":"2023-11-27T15:17:04.849470+0000","last_active":"2023-11-27T15:17:36.892827+0000","last_peered":"2023-11-27T15:17:36.892827+0000","last_clean":"2023-11-27T15:17:36.892827+0000","last_became_active":"2023-11-27T15:17:04.845827+0000","last_became_peered":"2023-11-27T15:17:04.845827+0000","last_unstale":"2023-11-27T15:17:36.892827+0000","last_undegraded":"2023-11-27T15:17:36.892827+0000","last_fullsized":"2023-11-27T15:17:36.892827+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:59:52.295722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1105,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":964,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.758486+0000","last_change":"2023-11-27T15:16:09.337330+0000","last_active":"2023-11-27T15:17:33.758486+0000","last_peered":"2023-11-27T15:17:33.758486+0000","last_clean":"2023-11-27T15:17:33.758486+0000","last_became_active":"2023-11-27T15:16:09.336991+0000","last_became_peered":"2023-11-27T15:16:09.336991+0000","last_unstale":"2023-11-27T15:17:33.758486+0000","last_undegraded":"2023-11-27T15:17:33.758486+0000","last_fullsized":"2023-11-27T15:17:33.758486+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:35:01.499613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.809306+0000","last_change":"2023-11-27T15:16:08.912208+0000","last_active":"2023-11-27T15:17:04.809306+0000","last_peered":"2023-11-27T15:17:04.809306+0000","last_clean":"2023-11-27T15:17:04.809306+0000","last_became_active":"2023-11-27T15:16:08.911879+0000","last_became_peered":"2023-11-27T15:16:08.911879+0000","last_unstale":"2023-11-27T15:17:04.809306+0000","last_undegraded":"2023-11-27T15:17:04.809306+0000","last_fullsized":"2023-11-27T15:17:04.809306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:34:55.140701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.772328+0000","last_change":"2023-11-27T15:16:09.339660+0000","last_active":"2023-11-27T15:17:33.772328+0000","last_peered":"2023-11-27T15:17:33.772328+0000","last_clean":"2023-11-27T15:17:33.772328+0000","last_became_active":"2023-11-27T15:16:09.339323+0000","last_became_peered":"2023-11-27T15:16:09.339323+0000","last_unstale":"2023-11-27T15:17:33.772328+0000","last_undegraded":"2023-11-27T15:17:33.772328+0000","last_fullsized":"2023-11-27T15:17:33.772328+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:31:05.598344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.803818+0000","last_change":"2023-11-27T15:16:08.911103+0000","last_active":"2023-11-27T15:17:04.803818+0000","last_peered":"2023-11-27T15:17:04.803818+0000","last_clean":"2023-11-27T15:17:04.803818+0000","last_became_active":"2023-11-27T15:16:08.910765+0000","last_became_peered":"2023-11-27T15:16:08.910765+0000","last_unstale":"2023-11-27T15:17:04.803818+0000","last_undegraded":"2023-11-27T15:17:04.803818+0000","last_fullsized":"2023-11-27T15:17:04.803818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:08:20.782772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.774114+0000","last_change":"2023-11-27T15:16:09.351741+0000","last_active":"2023-11-27T15:17:33.774114+0000","last_peered":"2023-11-27T15:17:33.774114+0000","last_clean":"2023-11-27T15:17:33.774114+0000","last_became_active":"2023-11-27T15:16:09.351280+0000","last_became_peered":"2023-11-27T15:16:09.351280+0000","last_unstale":"2023-11-27T15:17:33.774114+0000","last_undegraded":"2023-11-27T15:17:33.774114+0000","last_fullsized":"2023-11-27T15:17:33.774114+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:29:29.349396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.802775+0000","last_change":"2023-11-27T15:16:08.915630+0000","last_active":"2023-11-27T15:17:04.802775+0000","last_peered":"2023-11-27T15:17:04.802775+0000","last_clean":"2023-11-27T15:17:04.802775+0000","last_became_active":"2023-11-27T15:16:08.915294+0000","last_became_peered":"2023-11-27T15:16:08.915294+0000","last_unstale":"2023-11-27T15:17:04.802775+0000","last_undegraded":"2023-11-27T15:17:04.802775+0000","last_fullsized":"2023-11-27T15:17:04.802775+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:58:19.686545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.777372+0000","last_change":"2023-11-27T15:16:09.338436+0000","last_active":"2023-11-27T15:17:33.777372+0000","last_peered":"2023-11-27T15:17:33.777372+0000","last_clean":"2023-11-27T15:17:33.777372+0000","last_became_active":"2023-11-27T15:16:09.338148+0000","last_became_peered":"2023-11-27T15:16:09.338148+0000","last_unstale":"2023-11-27T15:17:33.777372+0000","last_undegraded":"2023-11-27T15:17:33.777372+0000","last_fullsized":"2023-11-27T15:17:33.777372+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:49:40.837798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":106,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:36.892579+0000","last_change":"2023-11-27T15:16:08.913072+0000","last_active":"2023-11-27T15:17:36.892579+0000","last_peered":"2023-11-27T15:17:36.892579+0000","last_clean":"2023-11-27T15:17:36.892579+0000","last_became_active":"2023-11-27T15:16:08.912757+0000","last_became_peered":"2023-11-27T15:16:08.912757+0000","last_unstale":"2023-11-27T15:17:36.892579+0000","last_undegraded":"2023-11-27T15:17:36.892579+0000","last_fullsized":"2023-11-27T15:17:36.892579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:48:31.534178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.772079+0000","last_change":"2023-11-27T15:16:09.340380+0000","last_active":"2023-11-27T15:17:33.772079+0000","last_peered":"2023-11-27T15:17:33.772079+0000","last_clean":"2023-11-27T15:17:33.772079+0000","last_became_active":"2023-11-27T15:16:09.339581+0000","last_became_peered":"2023-11-27T15:16:09.339581+0000","last_unstale":"2023-11-27T15:17:33.772079+0000","last_undegraded":"2023-11-27T15:17:33.772079+0000","last_fullsized":"2023-11-27T15:17:33.772079+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:21:44.281165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.807944+0000","last_change":"2023-11-27T15:16:08.917924+0000","last_active":"2023-11-27T15:17:04.807944+0000","last_peered":"2023-11-27T15:17:04.807944+0000","last_clean":"2023-11-27T15:17:04.807944+0000","last_became_active":"2023-11-27T15:16:08.917618+0000","last_became_peered":"2023-11-27T15:16:08.917618+0000","last_unstale":"2023-11-27T15:17:04.807944+0000","last_undegraded":"2023-11-27T15:17:04.807944+0000","last_fullsized":"2023-11-27T15:17:04.807944+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:26:02.008425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.767487+0000","last_change":"2023-11-27T15:16:09.346020+0000","last_active":"2023-11-27T15:17:33.767487+0000","last_peered":"2023-11-27T15:17:33.767487+0000","last_clean":"2023-11-27T15:17:33.767487+0000","last_became_active":"2023-11-27T15:16:09.345822+0000","last_became_peered":"2023-11-27T15:16:09.345822+0000","last_unstale":"2023-11-27T15:17:33.767487+0000","last_undegraded":"2023-11-27T15:17:33.767487+0000","last_fullsized":"2023-11-27T15:17:33.767487+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:09:13.031060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.803585+0000","last_change":"2023-11-27T15:16:08.921810+0000","last_active":"2023-11-27T15:17:04.803585+0000","last_peered":"2023-11-27T15:17:04.803585+0000","last_clean":"2023-11-27T15:17:04.803585+0000","last_became_active":"2023-11-27T15:16:08.921518+0000","last_became_peered":"2023-11-27T15:16:08.921518+0000","last_unstale":"2023-11-27T15:17:04.803585+0000","last_undegraded":"2023-11-27T15:17:04.803585+0000","last_fullsized":"2023-11-27T15:17:04.803585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:57:19.901063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.772314+0000","last_change":"2023-11-27T15:16:09.351083+0000","last_active":"2023-11-27T15:17:33.772314+0000","last_peered":"2023-11-27T15:17:33.772314+0000","last_clean":"2023-11-27T15:17:33.772314+0000","last_became_active":"2023-11-27T15:16:09.350540+0000","last_became_peered":"2023-11-27T15:16:09.350540+0000","last_unstale":"2023-11-27T15:17:33.772314+0000","last_undegraded":"2023-11-27T15:17:33.772314+0000","last_fullsized":"2023-11-27T15:17:33.772314+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:08:16.021485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'31","reported_seq":226,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:37.894182+0000","last_change":"2023-11-27T15:16:08.913023+0000","last_active":"2023-11-27T15:17:37.894182+0000","last_peered":"2023-11-27T15:17:37.894182+0000","last_clean":"2023-11-27T15:17:37.894182+0000","last_became_active":"2023-11-27T15:16:08.912693+0000","last_became_peered":"2023-11-27T15:16:08.912693+0000","last_unstale":"2023-11-27T15:17:37.894182+0000","last_undegraded":"2023-11-27T15:17:37.894182+0000","last_fullsized":"2023-11-27T15:17:37.894182+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:33:54.579228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58776,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":177,"num_read_kb":145,"num_write":32,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.775109+0000","last_change":"2023-11-27T15:16:09.343446+0000","last_active":"2023-11-27T15:17:33.775109+0000","last_peered":"2023-11-27T15:17:33.775109+0000","last_clean":"2023-11-27T15:17:33.775109+0000","last_became_active":"2023-11-27T15:16:09.341980+0000","last_became_peered":"2023-11-27T15:16:09.341980+0000","last_unstale":"2023-11-27T15:17:33.775109+0000","last_undegraded":"2023-11-27T15:17:33.775109+0000","last_fullsized":"2023-11-27T15:17:33.775109+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:48:41.501519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.822854+0000","last_change":"2023-11-27T15:17:04.822416+0000","last_active":"2023-11-27T15:17:04.822854+0000","last_peered":"2023-11-27T15:17:04.822854+0000","last_clean":"2023-11-27T15:17:04.822854+0000","last_became_active":"2023-11-27T15:17:04.810377+0000","last_became_peered":"2023-11-27T15:17:04.810377+0000","last_unstale":"2023-11-27T15:17:04.822854+0000","last_undegraded":"2023-11-27T15:17:04.822854+0000","last_fullsized":"2023-11-27T15:17:04.822854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:03:53.866628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.775297+0000","last_change":"2023-11-27T15:17:04.809248+0000","last_active":"2023-11-27T15:17:33.775297+0000","last_peered":"2023-11-27T15:17:33.775297+0000","last_clean":"2023-11-27T15:17:33.775297+0000","last_became_active":"2023-11-27T15:17:04.808813+0000","last_became_peered":"2023-11-27T15:17:04.808813+0000","last_unstale":"2023-11-27T15:17:33.775297+0000","last_undegraded":"2023-11-27T15:17:33.775297+0000","last_fullsized":"2023-11-27T15:17:33.775297+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:58:27.173973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.803065+0000","last_change":"2023-11-27T15:16:08.915597+0000","last_active":"2023-11-27T15:17:04.803065+0000","last_peered":"2023-11-27T15:17:04.803065+0000","last_clean":"2023-11-27T15:17:04.803065+0000","last_became_active":"2023-11-27T15:16:08.914727+0000","last_became_peered":"2023-11-27T15:16:08.914727+0000","last_unstale":"2023-11-27T15:17:04.803065+0000","last_undegraded":"2023-11-27T15:17:04.803065+0000","last_fullsized":"2023-11-27T15:17:04.803065+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:24:30.347721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.777426+0000","last_change":"2023-11-27T15:16:09.352643+0000","last_active":"2023-11-27T15:17:33.777426+0000","last_peered":"2023-11-27T15:17:33.777426+0000","last_clean":"2023-11-27T15:17:33.777426+0000","last_became_active":"2023-11-27T15:16:09.352336+0000","last_became_peered":"2023-11-27T15:16:09.352336+0000","last_unstale":"2023-11-27T15:17:33.777426+0000","last_undegraded":"2023-11-27T15:17:33.777426+0000","last_fullsized":"2023-11-27T15:17:33.777426+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:26:11.095933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'6","reported_seq":116,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:39.898232+0000","last_change":"2023-11-27T15:16:08.914303+0000","last_active":"2023-11-27T15:17:39.898232+0000","last_peered":"2023-11-27T15:17:39.898232+0000","last_clean":"2023-11-27T15:17:39.898232+0000","last_became_active":"2023-11-27T15:16:08.913601+0000","last_became_peered":"2023-11-27T15:16:08.913601+0000","last_unstale":"2023-11-27T15:17:39.898232+0000","last_undegraded":"2023-11-27T15:17:39.898232+0000","last_fullsized":"2023-11-27T15:17:39.898232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:32:26.143895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1389,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.775199+0000","last_change":"2023-11-27T15:16:09.341342+0000","last_active":"2023-11-27T15:17:33.775199+0000","last_peered":"2023-11-27T15:17:33.775199+0000","last_clean":"2023-11-27T15:17:33.775199+0000","last_became_active":"2023-11-27T15:16:09.341100+0000","last_became_peered":"2023-11-27T15:16:09.341100+0000","last_unstale":"2023-11-27T15:17:33.775199+0000","last_undegraded":"2023-11-27T15:17:33.775199+0000","last_fullsized":"2023-11-27T15:17:33.775199+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:11:05.729312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.808459+0000","last_change":"2023-11-27T15:16:08.921771+0000","last_active":"2023-11-27T15:17:04.808459+0000","last_peered":"2023-11-27T15:17:04.808459+0000","last_clean":"2023-11-27T15:17:04.808459+0000","last_became_active":"2023-11-27T15:16:08.921396+0000","last_became_peered":"2023-11-27T15:16:08.921396+0000","last_unstale":"2023-11-27T15:17:04.808459+0000","last_undegraded":"2023-11-27T15:17:04.808459+0000","last_fullsized":"2023-11-27T15:17:04.808459+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:05:46.765380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.771089+0000","last_change":"2023-11-27T15:16:09.341258+0000","last_active":"2023-11-27T15:17:33.771089+0000","last_peered":"2023-11-27T15:17:33.771089+0000","last_clean":"2023-11-27T15:17:33.771089+0000","last_became_active":"2023-11-27T15:16:09.340921+0000","last_became_peered":"2023-11-27T15:16:09.340921+0000","last_unstale":"2023-11-27T15:17:33.771089+0000","last_undegraded":"2023-11-27T15:17:33.771089+0000","last_fullsized":"2023-11-27T15:17:33.771089+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:12:25.330319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.809915+0000","last_change":"2023-11-27T15:16:08.911219+0000","last_active":"2023-11-27T15:17:04.809915+0000","last_peered":"2023-11-27T15:17:04.809915+0000","last_clean":"2023-11-27T15:17:04.809915+0000","last_became_active":"2023-11-27T15:16:08.910788+0000","last_became_peered":"2023-11-27T15:16:08.910788+0000","last_unstale":"2023-11-27T15:17:04.809915+0000","last_undegraded":"2023-11-27T15:17:04.809915+0000","last_fullsized":"2023-11-27T15:17:04.809915+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:39:39.465171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.765541+0000","last_change":"2023-11-27T15:16:09.343793+0000","last_active":"2023-11-27T15:17:33.765541+0000","last_peered":"2023-11-27T15:17:33.765541+0000","last_clean":"2023-11-27T15:17:33.765541+0000","last_became_active":"2023-11-27T15:16:09.343480+0000","last_became_peered":"2023-11-27T15:16:09.343480+0000","last_unstale":"2023-11-27T15:17:33.765541+0000","last_undegraded":"2023-11-27T15:17:33.765541+0000","last_fullsized":"2023-11-27T15:17:33.765541+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:37:38.699371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.806936+0000","last_change":"2023-11-27T15:16:08.920651+0000","last_active":"2023-11-27T15:17:04.806936+0000","last_peered":"2023-11-27T15:17:04.806936+0000","last_clean":"2023-11-27T15:17:04.806936+0000","last_became_active":"2023-11-27T15:16:08.920294+0000","last_became_peered":"2023-11-27T15:16:08.920294+0000","last_unstale":"2023-11-27T15:17:04.806936+0000","last_undegraded":"2023-11-27T15:17:04.806936+0000","last_fullsized":"2023-11-27T15:17:04.806936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:41:12.965663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.773421+0000","last_change":"2023-11-27T15:16:09.348279+0000","last_active":"2023-11-27T15:17:33.773421+0000","last_peered":"2023-11-27T15:17:33.773421+0000","last_clean":"2023-11-27T15:17:33.773421+0000","last_became_active":"2023-11-27T15:16:09.347760+0000","last_became_peered":"2023-11-27T15:16:09.347760+0000","last_unstale":"2023-11-27T15:17:33.773421+0000","last_undegraded":"2023-11-27T15:17:33.773421+0000","last_fullsized":"2023-11-27T15:17:33.773421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:19:36.809531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.807809+0000","last_change":"2023-11-27T15:16:08.915622+0000","last_active":"2023-11-27T15:17:04.807809+0000","last_peered":"2023-11-27T15:17:04.807809+0000","last_clean":"2023-11-27T15:17:04.807809+0000","last_became_active":"2023-11-27T15:16:08.915293+0000","last_became_peered":"2023-11-27T15:16:08.915293+0000","last_unstale":"2023-11-27T15:17:04.807809+0000","last_undegraded":"2023-11-27T15:17:04.807809+0000","last_fullsized":"2023-11-27T15:17:04.807809+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:31:34.850798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.769972+0000","last_change":"2023-11-27T15:16:09.337096+0000","last_active":"2023-11-27T15:17:33.769972+0000","last_peered":"2023-11-27T15:17:33.769972+0000","last_clean":"2023-11-27T15:17:33.769972+0000","last_became_active":"2023-11-27T15:16:09.336359+0000","last_became_peered":"2023-11-27T15:16:09.336359+0000","last_unstale":"2023-11-27T15:17:33.769972+0000","last_undegraded":"2023-11-27T15:17:33.769972+0000","last_fullsized":"2023-11-27T15:17:33.769972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:23:45.942895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.774368+0000","last_change":"2023-11-27T15:16:09.340478+0000","last_active":"2023-11-27T15:17:33.774368+0000","last_peered":"2023-11-27T15:17:33.774368+0000","last_clean":"2023-11-27T15:17:33.774368+0000","last_became_active":"2023-11-27T15:16:09.340214+0000","last_became_peered":"2023-11-27T15:16:09.340214+0000","last_unstale":"2023-11-27T15:17:33.774368+0000","last_undegraded":"2023-11-27T15:17:33.774368+0000","last_fullsized":"2023-11-27T15:17:33.774368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:51:37.081922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":108,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:37.893754+0000","last_change":"2023-11-27T15:16:08.920387+0000","last_active":"2023-11-27T15:17:37.893754+0000","last_peered":"2023-11-27T15:17:37.893754+0000","last_clean":"2023-11-27T15:17:37.893754+0000","last_became_active":"2023-11-27T15:16:08.920004+0000","last_became_peered":"2023-11-27T15:16:08.920004+0000","last_unstale":"2023-11-27T15:17:37.893754+0000","last_undegraded":"2023-11-27T15:17:37.893754+0000","last_fullsized":"2023-11-27T15:17:37.893754+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:31:37.559127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":85,"num_read_kb":85,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.770288+0000","last_change":"2023-11-27T15:17:04.808609+0000","last_active":"2023-11-27T15:17:33.770288+0000","last_peered":"2023-11-27T15:17:33.770288+0000","last_clean":"2023-11-27T15:17:33.770288+0000","last_became_active":"2023-11-27T15:17:04.808268+0000","last_became_peered":"2023-11-27T15:17:04.808268+0000","last_unstale":"2023-11-27T15:17:33.770288+0000","last_undegraded":"2023-11-27T15:17:33.770288+0000","last_fullsized":"2023-11-27T15:17:33.770288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:31:40.207340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.845316+0000","last_change":"2023-11-27T15:16:08.909694+0000","last_active":"2023-11-27T15:17:04.845316+0000","last_peered":"2023-11-27T15:17:04.845316+0000","last_clean":"2023-11-27T15:17:04.845316+0000","last_became_active":"2023-11-27T15:16:08.909324+0000","last_became_peered":"2023-11-27T15:16:08.909324+0000","last_unstale":"2023-11-27T15:17:04.845316+0000","last_undegraded":"2023-11-27T15:17:04.845316+0000","last_fullsized":"2023-11-27T15:17:04.845316+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:50:22.630279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":344,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.807076+0000","last_change":"2023-11-27T15:15:10.678805+0000","last_active":"2023-11-27T15:17:04.807076+0000","last_peered":"2023-11-27T15:17:04.807076+0000","last_clean":"2023-11-27T15:17:04.807076+0000","last_became_active":"2023-11-27T15:15:10.678267+0000","last_became_peered":"2023-11-27T15:15:10.678267+0000","last_unstale":"2023-11-27T15:17:04.807076+0000","last_undegraded":"2023-11-27T15:17:04.807076+0000","last_fullsized":"2023-11-27T15:17:04.807076+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:12:14.179315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:12:14.179315+0000","last_clean_scrub_stamp":"2023-11-27T15:12:14.179315+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:46:50.658179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.770453+0000","last_change":"2023-11-27T15:16:09.346622+0000","last_active":"2023-11-27T15:17:33.770453+0000","last_peered":"2023-11-27T15:17:33.770453+0000","last_clean":"2023-11-27T15:17:33.770453+0000","last_became_active":"2023-11-27T15:16:09.346287+0000","last_became_peered":"2023-11-27T15:16:09.346287+0000","last_unstale":"2023-11-27T15:17:33.770453+0000","last_undegraded":"2023-11-27T15:17:33.770453+0000","last_fullsized":"2023-11-27T15:17:33.770453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:45:31.217367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.814144+0000","last_change":"2023-11-27T15:17:04.814144+0000","last_active":"2023-11-27T15:17:04.814144+0000","last_peered":"2023-11-27T15:17:04.814144+0000","last_clean":"2023-11-27T15:17:04.814144+0000","last_became_active":"2023-11-27T15:17:04.810509+0000","last_became_peered":"2023-11-27T15:17:04.810509+0000","last_unstale":"2023-11-27T15:17:04.814144+0000","last_undegraded":"2023-11-27T15:17:04.814144+0000","last_fullsized":"2023-11-27T15:17:04.814144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:54:31.956866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.815900+0000","last_change":"2023-11-27T15:17:04.815501+0000","last_active":"2023-11-27T15:17:04.815900+0000","last_peered":"2023-11-27T15:17:04.815900+0000","last_clean":"2023-11-27T15:17:04.815900+0000","last_became_active":"2023-11-27T15:17:04.810004+0000","last_became_peered":"2023-11-27T15:17:04.810004+0000","last_unstale":"2023-11-27T15:17:04.815900+0000","last_undegraded":"2023-11-27T15:17:04.815900+0000","last_fullsized":"2023-11-27T15:17:04.815900+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:15:11.028146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.767894+0000","last_change":"2023-11-27T15:16:09.345119+0000","last_active":"2023-11-27T15:17:33.767894+0000","last_peered":"2023-11-27T15:17:33.767894+0000","last_clean":"2023-11-27T15:17:33.767894+0000","last_became_active":"2023-11-27T15:16:09.344807+0000","last_became_peered":"2023-11-27T15:16:09.344807+0000","last_unstale":"2023-11-27T15:17:33.767894+0000","last_undegraded":"2023-11-27T15:17:33.767894+0000","last_fullsized":"2023-11-27T15:17:33.767894+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:16:43.558443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'7","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:34.645800+0000","last_change":"2023-11-27T15:16:08.911564+0000","last_active":"2023-11-27T15:17:34.645800+0000","last_peered":"2023-11-27T15:17:34.645800+0000","last_clean":"2023-11-27T15:17:34.645800+0000","last_became_active":"2023-11-27T15:16:08.911057+0000","last_became_peered":"2023-11-27T15:16:08.911057+0000","last_unstale":"2023-11-27T15:17:34.645800+0000","last_undegraded":"2023-11-27T15:17:34.645800+0000","last_fullsized":"2023-11-27T15:17:34.645800+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:45:19.804625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.777021+0000","last_change":"2023-11-27T15:16:09.339085+0000","last_active":"2023-11-27T15:17:33.777021+0000","last_peered":"2023-11-27T15:17:33.777021+0000","last_clean":"2023-11-27T15:17:33.777021+0000","last_became_active":"2023-11-27T15:16:09.338777+0000","last_became_peered":"2023-11-27T15:16:09.338777+0000","last_unstale":"2023-11-27T15:17:33.777021+0000","last_undegraded":"2023-11-27T15:17:33.777021+0000","last_fullsized":"2023-11-27T15:17:33.777021+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:08:45.733215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.845177+0000","last_change":"2023-11-27T15:16:08.913939+0000","last_active":"2023-11-27T15:17:04.845177+0000","last_peered":"2023-11-27T15:17:04.845177+0000","last_clean":"2023-11-27T15:17:04.845177+0000","last_became_active":"2023-11-27T15:16:08.913629+0000","last_became_peered":"2023-11-27T15:16:08.913629+0000","last_unstale":"2023-11-27T15:17:04.845177+0000","last_undegraded":"2023-11-27T15:17:04.845177+0000","last_fullsized":"2023-11-27T15:17:04.845177+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:52:20.803089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.766880+0000","last_change":"2023-11-27T15:16:09.350712+0000","last_active":"2023-11-27T15:17:33.766880+0000","last_peered":"2023-11-27T15:17:33.766880+0000","last_clean":"2023-11-27T15:17:33.766880+0000","last_became_active":"2023-11-27T15:16:09.350359+0000","last_became_peered":"2023-11-27T15:16:09.350359+0000","last_unstale":"2023-11-27T15:17:33.766880+0000","last_undegraded":"2023-11-27T15:17:33.766880+0000","last_fullsized":"2023-11-27T15:17:33.766880+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:17:11.861223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.805284+0000","last_change":"2023-11-27T15:16:08.919501+0000","last_active":"2023-11-27T15:17:04.805284+0000","last_peered":"2023-11-27T15:17:04.805284+0000","last_clean":"2023-11-27T15:17:04.805284+0000","last_became_active":"2023-11-27T15:16:08.919258+0000","last_became_peered":"2023-11-27T15:16:08.919258+0000","last_unstale":"2023-11-27T15:17:04.805284+0000","last_undegraded":"2023-11-27T15:17:04.805284+0000","last_fullsized":"2023-11-27T15:17:04.805284+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:04:56.583314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.734625+0000","last_change":"2023-11-27T15:16:09.344066+0000","last_active":"2023-11-27T15:17:33.734625+0000","last_peered":"2023-11-27T15:17:33.734625+0000","last_clean":"2023-11-27T15:17:33.734625+0000","last_became_active":"2023-11-27T15:16:09.343757+0000","last_became_peered":"2023-11-27T15:16:09.343757+0000","last_unstale":"2023-11-27T15:17:33.734625+0000","last_undegraded":"2023-11-27T15:17:33.734625+0000","last_fullsized":"2023-11-27T15:17:33.734625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:13:29.747291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.808214+0000","last_change":"2023-11-27T15:16:08.906205+0000","last_active":"2023-11-27T15:17:04.808214+0000","last_peered":"2023-11-27T15:17:04.808214+0000","last_clean":"2023-11-27T15:17:04.808214+0000","last_became_active":"2023-11-27T15:16:08.905885+0000","last_became_peered":"2023-11-27T15:16:08.905885+0000","last_unstale":"2023-11-27T15:17:04.808214+0000","last_undegraded":"2023-11-27T15:17:04.808214+0000","last_fullsized":"2023-11-27T15:17:04.808214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:10:14.001695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.761768+0000","last_change":"2023-11-27T15:16:09.349642+0000","last_active":"2023-11-27T15:17:33.761768+0000","last_peered":"2023-11-27T15:17:33.761768+0000","last_clean":"2023-11-27T15:17:33.761768+0000","last_became_active":"2023-11-27T15:16:09.349086+0000","last_became_peered":"2023-11-27T15:16:09.349086+0000","last_unstale":"2023-11-27T15:17:33.761768+0000","last_undegraded":"2023-11-27T15:17:33.761768+0000","last_fullsized":"2023-11-27T15:17:33.761768+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:36:33.604216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.806096+0000","last_change":"2023-11-27T15:16:08.917835+0000","last_active":"2023-11-27T15:17:04.806096+0000","last_peered":"2023-11-27T15:17:04.806096+0000","last_clean":"2023-11-27T15:17:04.806096+0000","last_became_active":"2023-11-27T15:16:08.917532+0000","last_became_peered":"2023-11-27T15:16:08.917532+0000","last_unstale":"2023-11-27T15:17:04.806096+0000","last_undegraded":"2023-11-27T15:17:04.806096+0000","last_fullsized":"2023-11-27T15:17:04.806096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:29:51.697143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.774919+0000","last_change":"2023-11-27T15:16:09.338459+0000","last_active":"2023-11-27T15:17:33.774919+0000","last_peered":"2023-11-27T15:17:33.774919+0000","last_clean":"2023-11-27T15:17:33.774919+0000","last_became_active":"2023-11-27T15:16:09.338148+0000","last_became_peered":"2023-11-27T15:16:09.338148+0000","last_unstale":"2023-11-27T15:17:33.774919+0000","last_undegraded":"2023-11-27T15:17:33.774919+0000","last_fullsized":"2023-11-27T15:17:33.774919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:19:16.710021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.809804+0000","last_change":"2023-11-27T15:16:08.915401+0000","last_active":"2023-11-27T15:17:04.809804+0000","last_peered":"2023-11-27T15:17:04.809804+0000","last_clean":"2023-11-27T15:17:04.809804+0000","last_became_active":"2023-11-27T15:16:08.915012+0000","last_became_peered":"2023-11-27T15:16:08.915012+0000","last_unstale":"2023-11-27T15:17:04.809804+0000","last_undegraded":"2023-11-27T15:17:04.809804+0000","last_fullsized":"2023-11-27T15:17:04.809804+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:03:26.905621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.765523+0000","last_change":"2023-11-27T15:16:09.352309+0000","last_active":"2023-11-27T15:17:33.765523+0000","last_peered":"2023-11-27T15:17:33.765523+0000","last_clean":"2023-11-27T15:17:33.765523+0000","last_became_active":"2023-11-27T15:16:09.352039+0000","last_became_peered":"2023-11-27T15:16:09.352039+0000","last_unstale":"2023-11-27T15:17:33.765523+0000","last_undegraded":"2023-11-27T15:17:33.765523+0000","last_fullsized":"2023-11-27T15:17:33.765523+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:20:09.773389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.805829+0000","last_change":"2023-11-27T15:16:08.920446+0000","last_active":"2023-11-27T15:17:04.805829+0000","last_peered":"2023-11-27T15:17:04.805829+0000","last_clean":"2023-11-27T15:17:04.805829+0000","last_became_active":"2023-11-27T15:16:08.920232+0000","last_became_peered":"2023-11-27T15:16:08.920232+0000","last_unstale":"2023-11-27T15:17:04.805829+0000","last_undegraded":"2023-11-27T15:17:04.805829+0000","last_fullsized":"2023-11-27T15:17:04.805829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:08:07.892335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.775022+0000","last_change":"2023-11-27T15:16:09.344328+0000","last_active":"2023-11-27T15:17:33.775022+0000","last_peered":"2023-11-27T15:17:33.775022+0000","last_clean":"2023-11-27T15:17:33.775022+0000","last_became_active":"2023-11-27T15:16:09.344047+0000","last_became_peered":"2023-11-27T15:16:09.344047+0000","last_unstale":"2023-11-27T15:17:33.775022+0000","last_undegraded":"2023-11-27T15:17:33.775022+0000","last_fullsized":"2023-11-27T15:17:33.775022+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:00:56.015107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.803420+0000","last_change":"2023-11-27T15:16:08.909865+0000","last_active":"2023-11-27T15:17:04.803420+0000","last_peered":"2023-11-27T15:17:04.803420+0000","last_clean":"2023-11-27T15:17:04.803420+0000","last_became_active":"2023-11-27T15:16:08.909559+0000","last_became_peered":"2023-11-27T15:16:08.909559+0000","last_unstale":"2023-11-27T15:17:04.803420+0000","last_undegraded":"2023-11-27T15:17:04.803420+0000","last_fullsized":"2023-11-27T15:17:04.803420+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:59:09.511504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.775967+0000","last_change":"2023-11-27T15:16:09.342525+0000","last_active":"2023-11-27T15:17:33.775967+0000","last_peered":"2023-11-27T15:17:33.775967+0000","last_clean":"2023-11-27T15:17:33.775967+0000","last_became_active":"2023-11-27T15:16:09.340987+0000","last_became_peered":"2023-11-27T15:16:09.340987+0000","last_unstale":"2023-11-27T15:17:33.775967+0000","last_undegraded":"2023-11-27T15:17:33.775967+0000","last_fullsized":"2023-11-27T15:17:33.775967+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:37:38.155819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.804148+0000","last_change":"2023-11-27T15:16:08.914088+0000","last_active":"2023-11-27T15:17:04.804148+0000","last_peered":"2023-11-27T15:17:04.804148+0000","last_clean":"2023-11-27T15:17:04.804148+0000","last_became_active":"2023-11-27T15:16:08.913750+0000","last_became_peered":"2023-11-27T15:16:08.913750+0000","last_unstale":"2023-11-27T15:17:04.804148+0000","last_undegraded":"2023-11-27T15:17:04.804148+0000","last_fullsized":"2023-11-27T15:17:04.804148+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:57:55.114032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.774914+0000","last_change":"2023-11-27T15:16:09.344403+0000","last_active":"2023-11-27T15:17:33.774914+0000","last_peered":"2023-11-27T15:17:33.774914+0000","last_clean":"2023-11-27T15:17:33.774914+0000","last_became_active":"2023-11-27T15:16:09.344120+0000","last_became_peered":"2023-11-27T15:16:09.344120+0000","last_unstale":"2023-11-27T15:17:33.774914+0000","last_undegraded":"2023-11-27T15:17:33.774914+0000","last_fullsized":"2023-11-27T15:17:33.774914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:47:27.200812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.762743+0000","last_change":"2023-11-27T15:16:09.349780+0000","last_active":"2023-11-27T15:17:33.762743+0000","last_peered":"2023-11-27T15:17:33.762743+0000","last_clean":"2023-11-27T15:17:33.762743+0000","last_became_active":"2023-11-27T15:16:09.349493+0000","last_became_peered":"2023-11-27T15:16:09.349493+0000","last_unstale":"2023-11-27T15:17:33.762743+0000","last_undegraded":"2023-11-27T15:17:33.762743+0000","last_fullsized":"2023-11-27T15:17:33.762743+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:14:43.605196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.773008+0000","last_change":"2023-11-27T15:17:04.809959+0000","last_active":"2023-11-27T15:17:33.773008+0000","last_peered":"2023-11-27T15:17:33.773008+0000","last_clean":"2023-11-27T15:17:33.773008+0000","last_became_active":"2023-11-27T15:17:04.809529+0000","last_became_peered":"2023-11-27T15:17:04.809529+0000","last_unstale":"2023-11-27T15:17:33.773008+0000","last_undegraded":"2023-11-27T15:17:33.773008+0000","last_fullsized":"2023-11-27T15:17:33.773008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:19:17.711075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.770374+0000","last_change":"2023-11-27T15:16:09.337989+0000","last_active":"2023-11-27T15:17:33.770374+0000","last_peered":"2023-11-27T15:17:33.770374+0000","last_clean":"2023-11-27T15:17:33.770374+0000","last_became_active":"2023-11-27T15:16:09.336712+0000","last_became_peered":"2023-11-27T15:16:09.336712+0000","last_unstale":"2023-11-27T15:17:33.770374+0000","last_undegraded":"2023-11-27T15:17:33.770374+0000","last_fullsized":"2023-11-27T15:17:33.770374+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:25:41.941448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.768395+0000","last_change":"2023-11-27T15:17:04.812727+0000","last_active":"2023-11-27T15:17:33.768395+0000","last_peered":"2023-11-27T15:17:33.768395+0000","last_clean":"2023-11-27T15:17:33.768395+0000","last_became_active":"2023-11-27T15:17:04.811852+0000","last_became_peered":"2023-11-27T15:17:04.811852+0000","last_unstale":"2023-11-27T15:17:33.768395+0000","last_undegraded":"2023-11-27T15:17:33.768395+0000","last_fullsized":"2023-11-27T15:17:33.768395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:03:44.659440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.771893+0000","last_change":"2023-11-27T15:16:09.344317+0000","last_active":"2023-11-27T15:17:33.771893+0000","last_peered":"2023-11-27T15:17:33.771893+0000","last_clean":"2023-11-27T15:17:33.771893+0000","last_became_active":"2023-11-27T15:16:09.343962+0000","last_became_peered":"2023-11-27T15:16:09.343962+0000","last_unstale":"2023-11-27T15:17:33.771893+0000","last_undegraded":"2023-11-27T15:17:33.771893+0000","last_fullsized":"2023-11-27T15:17:33.771893+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:15:39.862196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.766088+0000","last_change":"2023-11-27T15:16:09.344690+0000","last_active":"2023-11-27T15:17:33.766088+0000","last_peered":"2023-11-27T15:17:33.766088+0000","last_clean":"2023-11-27T15:17:33.766088+0000","last_became_active":"2023-11-27T15:16:09.344259+0000","last_became_peered":"2023-11-27T15:16:09.344259+0000","last_unstale":"2023-11-27T15:17:33.766088+0000","last_undegraded":"2023-11-27T15:17:33.766088+0000","last_fullsized":"2023-11-27T15:17:33.766088+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:48:11.448071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.773804+0000","last_change":"2023-11-27T15:16:09.340459+0000","last_active":"2023-11-27T15:17:33.773804+0000","last_peered":"2023-11-27T15:17:33.773804+0000","last_clean":"2023-11-27T15:17:33.773804+0000","last_became_active":"2023-11-27T15:16:09.340140+0000","last_became_peered":"2023-11-27T15:16:09.340140+0000","last_unstale":"2023-11-27T15:17:33.773804+0000","last_undegraded":"2023-11-27T15:17:33.773804+0000","last_fullsized":"2023-11-27T15:17:33.773804+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:50:28.936856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.770718+0000","last_change":"2023-11-27T15:16:09.336625+0000","last_active":"2023-11-27T15:17:33.770718+0000","last_peered":"2023-11-27T15:17:33.770718+0000","last_clean":"2023-11-27T15:17:33.770718+0000","last_became_active":"2023-11-27T15:16:09.336287+0000","last_became_peered":"2023-11-27T15:16:09.336287+0000","last_unstale":"2023-11-27T15:17:33.770718+0000","last_undegraded":"2023-11-27T15:17:33.770718+0000","last_fullsized":"2023-11-27T15:17:33.770718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:09:58.745009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.772059+0000","last_change":"2023-11-27T15:16:09.335480+0000","last_active":"2023-11-27T15:17:33.772059+0000","last_peered":"2023-11-27T15:17:33.772059+0000","last_clean":"2023-11-27T15:17:33.772059+0000","last_became_active":"2023-11-27T15:16:09.335163+0000","last_became_peered":"2023-11-27T15:16:09.335163+0000","last_unstale":"2023-11-27T15:17:33.772059+0000","last_undegraded":"2023-11-27T15:17:33.772059+0000","last_fullsized":"2023-11-27T15:17:33.772059+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:57:09.686634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.775133+0000","last_change":"2023-11-27T15:16:09.336879+0000","last_active":"2023-11-27T15:17:33.775133+0000","last_peered":"2023-11-27T15:17:33.775133+0000","last_clean":"2023-11-27T15:17:33.775133+0000","last_became_active":"2023-11-27T15:16:09.336580+0000","last_became_peered":"2023-11-27T15:16:09.336580+0000","last_unstale":"2023-11-27T15:17:33.775133+0000","last_undegraded":"2023-11-27T15:17:33.775133+0000","last_fullsized":"2023-11-27T15:17:33.775133+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:18:10.805656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.773451+0000","last_change":"2023-11-27T15:16:09.341400+0000","last_active":"2023-11-27T15:17:33.773451+0000","last_peered":"2023-11-27T15:17:33.773451+0000","last_clean":"2023-11-27T15:17:33.773451+0000","last_became_active":"2023-11-27T15:16:09.341103+0000","last_became_peered":"2023-11-27T15:16:09.341103+0000","last_unstale":"2023-11-27T15:17:33.773451+0000","last_undegraded":"2023-11-27T15:17:33.773451+0000","last_fullsized":"2023-11-27T15:17:33.773451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:51:15.698428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.776500+0000","last_change":"2023-11-27T15:16:09.345938+0000","last_active":"2023-11-27T15:17:33.776500+0000","last_peered":"2023-11-27T15:17:33.776500+0000","last_clean":"2023-11-27T15:17:33.776500+0000","last_became_active":"2023-11-27T15:16:09.345624+0000","last_became_peered":"2023-11-27T15:16:09.345624+0000","last_unstale":"2023-11-27T15:17:33.776500+0000","last_undegraded":"2023-11-27T15:17:33.776500+0000","last_fullsized":"2023-11-27T15:17:33.776500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:29:31.241865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.765022+0000","last_change":"2023-11-27T15:16:09.343896+0000","last_active":"2023-11-27T15:17:33.765022+0000","last_peered":"2023-11-27T15:17:33.765022+0000","last_clean":"2023-11-27T15:17:33.765022+0000","last_became_active":"2023-11-27T15:16:09.343603+0000","last_became_peered":"2023-11-27T15:16:09.343603+0000","last_unstale":"2023-11-27T15:17:33.765022+0000","last_undegraded":"2023-11-27T15:17:33.765022+0000","last_fullsized":"2023-11-27T15:17:33.765022+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:20:29.881258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":249380,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63670,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":533,"num_read_kb":336,"num_write":163,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1020,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":196389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449118,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6096,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739056,"statfs":{"total":95995035648,"available":95988793344,"internally_reserved":0,"allocated":925696,"data_stored":444516,"data_compressed":5787,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54000000000000004,"15min":0.53600000000000003},"min":{"1min":0.39100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51300000000000001,"15min":0.51600000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.68000000000000005,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.48199999999999998}]},{"osd":1,"last update":"Mon Nov 27 15:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.63100000000000001,"15min":0.625},"min":{"1min":0.51900000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.80500000000000005,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81399999999999995,"5min":1.262,"15min":1.262},"last":0.58799999999999997}]},{"osd":2,"last update":"Mon Nov 27 15:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.46000000000000002,"15min":0.45700000000000002},"min":{"1min":0.32800000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.58099999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.48799999999999999,"15min":0.48699999999999999},"min":{"1min":0.373,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.66400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.504}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":4,"last update":"Mon Nov 27 15:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.64900000000000002,"15min":0.64400000000000002},"min":{"1min":0.57099999999999995,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83299999999999996,"5min":1.155,"15min":1.155},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82199999999999995,"5min":1.048,"15min":1.048},"last":0.69199999999999995}]},{"osd":5,"last update":"Mon Nov 27 15:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52700000000000002,"15min":0.52600000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.63800000000000001,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58999999999999997,"15min":0.58499999999999996},"min":{"1min":0.434,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82999999999999996,"5min":1.02,"15min":1.02},"last":0.755}]},{"osd":6,"last update":"Mon Nov 27 15:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.61899999999999999}]},{"osd":7,"last update":"Mon Nov 27 15:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54200000000000004,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.91900000000000004,"5min":1.232,"15min":1.232},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.71899999999999997,"15min":0.71799999999999997},"min":{"1min":0.51500000000000001,"5min":0.503,"15min":0.503},"max":{"1min":0.85699999999999998,"5min":1.302,"15min":1.302},"last":0.78500000000000003}]},{"osd":8,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.73399999999999999}]},{"osd":9,"last update":"Mon Nov 27 15:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72199999999999998,"15min":0.72499999999999998},"min":{"1min":0.51600000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.97399999999999998,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.75600000000000001,"15min":0.752},"min":{"1min":0.628,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.96199999999999997,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.876}]},{"osd":10,"last update":"Mon Nov 27 15:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.76300000000000001,"15min":0.76100000000000001},"min":{"1min":0.59799999999999998,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.88600000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.628,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.92000000000000004,"5min":1.706,"15min":1.706},"last":0.77100000000000002}]}]},{"osd":10,"up_from":71,"seq":304942678050,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6556,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738596,"statfs":{"total":95995035648,"available":95988322304,"internally_reserved":0,"allocated":1069056,"data_stored":1095206,"data_compressed":41881,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.496},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.433}]},{"osd":1,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.44500000000000001,"15min":0.442},"min":{"1min":0.248,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.318},{"interface":"front","average":{"1min":0.434,"5min":0.45800000000000002,"15min":0.46200000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.58499999999999996,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.36499999999999999}]},{"osd":2,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.61499999999999999,"15min":0.62},"min":{"1min":0.40100000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84199999999999997,"5min":0.995,"15min":0.995},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76600000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.65700000000000003}]},{"osd":3,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.58799999999999997,"15min":0.60299999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72999999999999998,"5min":1.071,"15min":1.071},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63600000000000001,"15min":0.64500000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.80100000000000005,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.38600000000000001}]},{"osd":4,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53100000000000003,"15min":0.53700000000000003},"min":{"1min":0.34499999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.71599999999999997,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.53100000000000003,"15min":0.53800000000000003},"min":{"1min":0.35599999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71199999999999997,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.34499999999999997}]},{"osd":5,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.629,"15min":0.627},"min":{"1min":0.47499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77700000000000002,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.66500000000000004,"15min":0.65900000000000003},"min":{"1min":0.46800000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.61899999999999999}]},{"osd":6,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.71199999999999997,"15min":0.71799999999999997},"min":{"1min":0.45000000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.876,"5min":1.304,"15min":1.304},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.67000000000000004,"15min":0.68200000000000005},"min":{"1min":0.42499999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82399999999999995,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.52500000000000002}]},{"osd":7,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.53300000000000003}]},{"osd":8,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.70699999999999996,"15min":0.71299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85799999999999998,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.72999999999999998,"15min":0.73399999999999999},"min":{"1min":0.504,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.98899999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.55900000000000005}]},{"osd":9,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.73999999999999999,"15min":0.752},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.85299999999999998,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.76900000000000002,"15min":0.78300000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.85299999999999998,"5min":1.109,"15min":1.109},"last":0.48399999999999999}]},{"osd":11,"last update":"Mon Nov 27 15:17:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71699999999999997,"15min":0.71499999999999997},"min":{"1min":0.63,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89000000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.72199999999999998,"15min":0.71899999999999997},"min":{"1min":0.57999999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.63500000000000001}]}]},{"osd":9,"up_from":66,"seq":283467841575,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6232,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93738920,"statfs":{"total":95995035648,"available":95988654080,"internally_reserved":0,"allocated":933888,"data_stored":444696,"data_compressed":5787,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.437,"15min":0.443},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.65300000000000002,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44500000000000001,"15min":0.44800000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.39400000000000002}]},{"osd":1,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.61099999999999999,"15min":0.61499999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.501},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.628,"15min":0.63800000000000001},"min":{"1min":0.42499999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77900000000000003,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.51200000000000001}]},{"osd":2,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64100000000000001,"15min":0.64200000000000002},"min":{"1min":0.42899999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.441},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.46300000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.81999999999999995,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.54900000000000004}]},{"osd":3,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.52900000000000003,"15min":0.53500000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.59499999999999997,"15min":0.61199999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65500000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.32500000000000001}]},{"osd":4,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.65200000000000002,"15min":0.66600000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79300000000000004,"5min":1.236,"15min":1.236},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.52700000000000002}]},{"osd":5,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.70899999999999996,"15min":0.72699999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.82499999999999996,"5min":1.274,"15min":1.274},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.68500000000000005,"15min":0.68700000000000006},"min":{"1min":0.439,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.47899999999999998}]},{"osd":6,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.69499999999999995,"15min":0.70899999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.71199999999999997,"15min":0.73399999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.77100000000000002,"5min":1.351,"15min":1.351},"last":0.61399999999999999}]},{"osd":7,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.75800000000000001,"15min":0.76200000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.77900000000000003,"15min":0.79500000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.92000000000000004,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.58399999999999996}]},{"osd":8,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79600000000000004,"15min":0.79900000000000004},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.78500000000000003,"15min":0.80300000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.86699999999999999,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.68899999999999995}]},{"osd":10,"last update":"Mon Nov 27 15:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.74399999999999999,"15min":0.753},"min":{"1min":0.51200000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.95399999999999996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.746,"15min":0.75800000000000001},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.88800000000000001,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.70599999999999996}]}]},{"osd":8,"up_from":58,"seq":249108103211,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6420,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738732,"statfs":{"total":95995035648,"available":95988461568,"internally_reserved":0,"allocated":929792,"data_stored":444594,"data_compressed":5787,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.34899999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51500000000000001,"15min":0.5},"min":{"1min":0.40899999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.52400000000000002}]},{"osd":1,"last update":"Mon Nov 27 15:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.57199999999999995,"15min":0.55600000000000005},"min":{"1min":0.48399999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.81100000000000005,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.59099999999999997,"15min":0.56899999999999995},"min":{"1min":0.52300000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.92000000000000004,"5min":1.004,"15min":1.004},"last":0.67400000000000004}]},{"osd":2,"last update":"Mon Nov 27 15:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46200000000000002,"15min":0.44900000000000001},"min":{"1min":0.33100000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.68000000000000005,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.46800000000000003,"15min":0.44800000000000001},"min":{"1min":0.32900000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.38400000000000001}]},{"osd":3,"last update":"Mon Nov 27 15:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55200000000000005,"15min":0.53600000000000003},"min":{"1min":0.41599999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.57899999999999996,"15min":0.56100000000000005},"min":{"1min":0.44,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.56200000000000006}]},{"osd":4,"last update":"Mon Nov 27 15:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.60699999999999998,"15min":0.58699999999999997},"min":{"1min":0.53800000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79800000000000004,"5min":1.014,"15min":1.014},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.64800000000000002,"15min":0.63300000000000001},"min":{"1min":0.54000000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.84999999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.622}]},{"osd":5,"last update":"Mon Nov 27 15:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.40000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77500000000000002,"5min":0.873,"15min":0.873},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56599999999999995,"15min":0.54900000000000004},"min":{"1min":0.376,"5min":0.315,"15min":0.315},"max":{"1min":0.86599999999999999,"5min":1.056,"15min":1.056},"last":0.57999999999999996}]},{"osd":6,"last update":"Mon Nov 27 15:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66500000000000004,"15min":0.65500000000000003},"min":{"1min":0.46100000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.66500000000000004,"15min":0.65000000000000002},"min":{"1min":0.51300000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.59799999999999998}]},{"osd":7,"last update":"Mon Nov 27 15:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.69099999999999995,"15min":0.67500000000000004},"min":{"1min":0.53800000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.70799999999999996,"15min":0.68799999999999994},"min":{"1min":0.63700000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.98599999999999999,"5min":1.115,"15min":1.115},"last":0.72899999999999998}]},{"osd":9,"last update":"Mon Nov 27 15:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.72899999999999998,"15min":0.71699999999999997},"min":{"1min":0.52600000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.74399999999999999,"15min":0.73099999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.75700000000000001}]},{"osd":10,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.68400000000000005,"15min":0.66800000000000004},"min":{"1min":0.58599999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1,"5min":1,"15min":1},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.72399999999999998,"15min":0.70699999999999996},"min":{"1min":0.60599999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.80300000000000005}]},{"osd":11,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.78000000000000003}]}]},{"osd":7,"up_from":50,"seq":214748364847,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6536,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738616,"statfs":{"total":95995035648,"available":95988342784,"internally_reserved":0,"allocated":983040,"data_stored":1037445,"data_compressed":41881,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.35599999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.76300000000000001,"5min":1.052,"15min":1.052},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.51800000000000002,"15min":0.503},"min":{"1min":0.315,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.38700000000000001}]},{"osd":1,"last update":"Mon Nov 27 15:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.40500000000000003,"15min":0.38500000000000001},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.497},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.42799999999999999,"15min":0.40500000000000003},"min":{"1min":0.27900000000000003,"5min":0.254,"15min":0.254},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.59899999999999998}]},{"osd":2,"last update":"Mon Nov 27 15:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58299999999999996,"15min":0.56599999999999995},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56399999999999995,"15min":0.54400000000000004},"min":{"1min":0.38,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77500000000000002,"5min":1.077,"15min":1.077},"last":0.58199999999999996}]},{"osd":3,"last update":"Mon Nov 27 15:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54800000000000004,"15min":0.52700000000000002},"min":{"1min":0.36199999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71399999999999997,"5min":1.026,"15min":1.026},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59299999999999997,"15min":0.57999999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79100000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.628}]},{"osd":4,"last update":"Mon Nov 27 15:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.505,"15min":0.47899999999999998},"min":{"1min":0.35499999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.871,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.498,"15min":0.47699999999999998},"min":{"1min":0.318,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.73799999999999999}]},{"osd":5,"last update":"Mon Nov 27 15:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63500000000000001,"15min":0.60099999999999998},"min":{"1min":0.45600000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.96399999999999997,"5min":2.3719999999999999,"15min":2.3719999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65300000000000002,"15min":0.61599999999999999},"min":{"1min":0.47199999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76000000000000001,"5min":3.1059999999999999,"15min":3.1059999999999999},"last":0.70299999999999996}]},{"osd":6,"last update":"Mon Nov 27 15:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67200000000000004,"15min":0.65600000000000003},"min":{"1min":0.495,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.63,"15min":0.61199999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.77500000000000002}]},{"osd":8,"last update":"Mon Nov 27 15:17:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67800000000000005,"15min":0.65800000000000003},"min":{"1min":0.46500000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.97299999999999998,"5min":1.157,"15min":1.157},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.72799999999999998,"15min":0.69999999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.89100000000000001,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":0.80700000000000005}]},{"osd":9,"last update":"Mon Nov 27 15:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.752,"15min":0.75},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":0.754},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.73999999999999999,"15min":0.72999999999999998},"min":{"1min":0.50900000000000001,"5min":0.497,"15min":0.497},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":0.86399999999999999}]},{"osd":10,"last update":"Mon Nov 27 15:17:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.755,"15min":0.76500000000000001},"min":{"1min":0.52600000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.83099999999999996,"5min":1.214,"15min":1.214},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.77100000000000002,"15min":0.78900000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93400000000000005,"5min":1.095,"15min":1.095},"last":0.79500000000000004}]},{"osd":11,"last update":"Mon Nov 27 15:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.84199999999999997}]}]},{"osd":6,"up_from":45,"seq":193273528372,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6348,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738804,"statfs":{"total":95995035648,"available":95988535296,"internally_reserved":0,"allocated":921600,"data_stored":444091,"data_compressed":5787,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.438,"15min":0.44500000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.86099999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.503,"15min":0.53200000000000003},"min":{"1min":0.29899999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.83699999999999997,"5min":2.508,"15min":2.508},"last":0.36899999999999999}]},{"osd":1,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59599999999999997,"15min":0.58699999999999997},"min":{"1min":0.45000000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.61499999999999999,"15min":0.63800000000000001},"min":{"1min":0.41099999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73799999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":0.48499999999999999}]},{"osd":2,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.65300000000000002,"15min":0.67500000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91000000000000003,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.497},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.64600000000000002,"15min":0.65000000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.58399999999999996}]},{"osd":3,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52900000000000003,"15min":0.53100000000000003},"min":{"1min":0.377,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89600000000000002,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.433,"5min":0.54500000000000004,"15min":0.58699999999999997},"min":{"1min":0.33000000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.879,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":0.39000000000000001}]},{"osd":4,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.64300000000000002,"15min":0.65800000000000003},"min":{"1min":0.42699999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95299999999999996,"5min":1.51,"15min":1.51},"last":0.437},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65500000000000003,"15min":0.64300000000000002},"min":{"1min":0.495,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.012,"5min":1.054,"15min":1.054},"last":0.54800000000000004}]},{"osd":5,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.69299999999999995,"15min":0.73099999999999998},"min":{"1min":0.47399999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77700000000000002,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.70199999999999996,"15min":0.71899999999999997},"min":{"1min":0.51600000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.93300000000000005,"5min":1.599,"15min":1.599},"last":0.53000000000000003}]},{"osd":7,"last update":"Mon Nov 27 15:17:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.71199999999999997,"15min":0.71999999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88700000000000001,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73799999999999999,"15min":0.73499999999999999},"min":{"1min":0.56200000000000006,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.96099999999999997,"5min":1.206,"15min":1.206},"last":0.56899999999999995}]},{"osd":8,"last update":"Mon Nov 27 15:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.81000000000000005,"15min":0.81599999999999995},"min":{"1min":0.58699999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":4.0449999999999999,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.74299999999999999,"15min":0.79300000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.89200000000000002,"5min":2.766,"15min":2.766},"last":0.66400000000000003}]},{"osd":9,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.71599999999999997,"15min":0.74099999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.77800000000000002,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.75,"15min":0.77000000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90000000000000002,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.59999999999999998}]},{"osd":10,"last update":"Mon Nov 27 15:17:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.68700000000000006,"15min":0.68500000000000005},"min":{"1min":0.51200000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.95499999999999996,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.79300000000000004,"15min":0.79900000000000004},"min":{"1min":0.64100000000000001,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.93700000000000006,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.63300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7436,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737716,"statfs":{"total":95995035648,"available":95987421184,"internally_reserved":0,"allocated":987136,"data_stored":508351,"data_compressed":5787,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[0,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51200000000000001,"15min":0.47299999999999998},"min":{"1min":0.32700000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.52000000000000002,"15min":0.46999999999999997},"min":{"1min":0.33900000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.86799999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.41899999999999998}]},{"osd":2,"last update":"Mon Nov 27 15:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.56699999999999995,"15min":0.54700000000000004},"min":{"1min":0.40200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80900000000000005,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.622},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.55800000000000005,"15min":0.53200000000000003},"min":{"1min":0.41699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.66700000000000004}]},{"osd":3,"last update":"Mon Nov 27 15:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56999999999999995,"15min":0.53400000000000003},"min":{"1min":0.38100000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.96099999999999997,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56399999999999995,"15min":0.55700000000000005},"min":{"1min":0.32200000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.78300000000000003,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.55200000000000005}]},{"osd":4,"last update":"Mon Nov 27 15:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.45100000000000001,"15min":0.43099999999999999},"min":{"1min":0.307,"5min":0.251,"15min":0.251},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.46600000000000003,"15min":0.44400000000000001},"min":{"1min":0.33000000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.63600000000000001,"5min":0.754,"15min":0.754},"last":0.629}]},{"osd":5,"last update":"Mon Nov 27 15:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.59799999999999998,"15min":0.58899999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.63,"15min":0.623},"min":{"1min":0.53100000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.72699999999999998}]},{"osd":6,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.60599999999999998,"15min":0.58299999999999996},"min":{"1min":0.40699999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.63600000000000001,"15min":0.626},"min":{"1min":0.434,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95499999999999996,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.54200000000000004}]},{"osd":7,"last update":"Mon Nov 27 15:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54100000000000004,"15min":0.52800000000000002},"min":{"1min":0.38300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.90900000000000003,"5min":1.02,"15min":1.02},"last":0.499},{"interface":"front","average":{"1min":0.621,"5min":0.59199999999999997,"15min":0.57599999999999996},"min":{"1min":0.33900000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0549999999999999,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.58099999999999996}]},{"osd":8,"last update":"Mon Nov 27 15:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.67800000000000005,"15min":0.65400000000000003},"min":{"1min":0.52900000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91800000000000004,"5min":1.177,"15min":1.177},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.65300000000000002,"15min":0.63},"min":{"1min":0.53400000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.875,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.68200000000000005}]},{"osd":9,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72899999999999998,"15min":0.72099999999999997},"min":{"1min":0.57899999999999996,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.72899999999999998,"15min":0.72499999999999998},"min":{"1min":0.53400000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.60399999999999998}]},{"osd":10,"last update":"Mon Nov 27 15:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71699999999999997,"15min":0.71799999999999997},"min":{"1min":0.48299999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0149999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.71899999999999997,"15min":0.71299999999999997},"min":{"1min":0.57299999999999995,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.93799999999999994,"5min":0.995,"15min":0.995},"last":0.69399999999999995}]},{"osd":11,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.751}]}]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7328,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737824,"statfs":{"total":95995035648,"available":95987531776,"internally_reserved":0,"allocated":942080,"data_stored":445356,"data_compressed":5787,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 27 15:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59299999999999997,"15min":0.54900000000000004},"min":{"1min":0.45400000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79000000000000004,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.60999999999999999,"15min":0.56100000000000005},"min":{"1min":0.432,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.70099999999999996,"5min":1.55,"15min":1.55},"last":0.38900000000000001}]},{"osd":2,"last update":"Mon Nov 27 15:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58299999999999996,"15min":0.54500000000000004},"min":{"1min":0.42599999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54800000000000004,"15min":0.52100000000000002},"min":{"1min":0.35999999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.41099999999999998}]},{"osd":3,"last update":"Mon Nov 27 15:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.438},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.372}]},{"osd":4,"last update":"Mon Nov 27 15:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.56999999999999995,"15min":0.54800000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68300000000000005,"5min":0.999,"15min":0.999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.58599999999999997,"15min":0.55800000000000005},"min":{"1min":0.40500000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.67700000000000005,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.497}]},{"osd":5,"last update":"Mon Nov 27 15:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.61499999999999999},"min":{"1min":0.47699999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85499999999999998,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61799999999999999,"15min":0.60799999999999998},"min":{"1min":0.41899999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71899999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.65000000000000002}]},{"osd":6,"last update":"Mon Nov 27 15:16:43 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.308,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.63900000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.754,"5min":1.032,"15min":1.032},"last":0.61799999999999999}]},{"osd":7,"last update":"Mon Nov 27 15:16:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67800000000000005,"15min":0.67200000000000004},"min":{"1min":0.434,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.46100000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.31}]},{"osd":8,"last update":"Mon Nov 27 15:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.66000000000000003,"15min":0.69499999999999995},"min":{"1min":0.441,"5min":0.434,"15min":0.434},"max":{"1min":0.73699999999999999,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.70899999999999996,"15min":0.73899999999999999},"min":{"1min":0.45500000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.78200000000000003,"5min":1.004,"15min":1.004},"last":0.55300000000000005}]},{"osd":9,"last update":"Mon Nov 27 15:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.73499999999999999,"15min":0.76100000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88800000000000001,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.72299999999999998,"15min":0.74299999999999999},"min":{"1min":0.438,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.46899999999999997}]},{"osd":10,"last update":"Mon Nov 27 15:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.74399999999999999,"15min":0.75800000000000001},"min":{"1min":0.53000000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.81399999999999995,"5min":1.145,"15min":1.145},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.77100000000000002,"15min":0.78700000000000003},"min":{"1min":0.503,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.95399999999999996,"5min":1.248,"15min":1.248},"last":0.57499999999999996}]},{"osd":11,"last update":"Mon Nov 27 15:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65000000000000002,"15min":0.64900000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72699999999999998,"15min":0.73199999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.82399999999999995,"5min":0.879,"15min":0.879},"last":0.66300000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7508,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":6528,"kb_avail":93737644,"statfs":{"total":95995035648,"available":95987347456,"internally_reserved":0,"allocated":995328,"data_stored":503348,"data_compressed":5787,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":6684672},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.61199999999999999,"15min":0.51100000000000001},"min":{"1min":0.35199999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":8.5069999999999997,"5min":8.5069999999999997,"15min":8.5069999999999997},"last":0.442},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.50900000000000001,"15min":0.48399999999999999},"min":{"1min":0.36199999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.47699999999999998}]},{"osd":1,"last update":"Mon Nov 27 15:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.56899999999999995,"15min":0.52200000000000002},"min":{"1min":0.45900000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.53900000000000003,"15min":0.498},"min":{"1min":0.48099999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.57999999999999996}]},{"osd":3,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.59899999999999998,"15min":0.54600000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.2170000000000001,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.59599999999999997,"15min":0.54600000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":3.278,"5min":3.278,"15min":3.278},"last":0.50900000000000001}]},{"osd":4,"last update":"Mon Nov 27 15:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.63600000000000001}]},{"osd":5,"last update":"Mon Nov 27 15:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.47099999999999997,"15min":0.41299999999999998},"min":{"1min":0.26600000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.86799999999999999,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.48099999999999998,"15min":0.42699999999999999},"min":{"1min":0.33100000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.61299999999999999}]},{"osd":6,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.58599999999999997,"15min":0.56499999999999995},"min":{"1min":0.44900000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.70399999999999996,"15min":0.64300000000000002},"min":{"1min":0.41599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":4.5800000000000001,"5min":4.5800000000000001,"15min":4.5800000000000001},"last":0.754}]},{"osd":7,"last update":"Mon Nov 27 15:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.39800000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.44},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69199999999999995,"15min":0.69599999999999995},"min":{"1min":0.46300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93300000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.51100000000000001}]},{"osd":8,"last update":"Mon Nov 27 15:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.65900000000000003,"15min":0.66100000000000003},"min":{"1min":0.35099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.94799999999999995,"5min":0.997,"15min":0.997},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.59999999999999998,"15min":0.58199999999999996},"min":{"1min":0.41999999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.66800000000000004}]},{"osd":9,"last update":"Mon Nov 27 15:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.72199999999999998,"15min":0.70199999999999996},"min":{"1min":0.49399999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":3.633,"5min":3.633,"15min":3.633},"last":0.71699999999999997},{"interface":"front","average":{"1min":1.036,"5min":0.79500000000000004,"15min":0.755},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":7.6369999999999996,"5min":7.6369999999999996,"15min":7.6369999999999996},"last":0.67900000000000005}]},{"osd":10,"last update":"Mon Nov 27 15:17:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.67700000000000005,"15min":0.66200000000000003},"min":{"1min":0.59299999999999997,"5min":0.496,"15min":0.496},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.70299999999999996,"15min":0.67900000000000005},"min":{"1min":0.67300000000000004,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.76300000000000001}]},{"osd":11,"last update":"Mon Nov 27 15:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.78300000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182464,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6684,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738468,"statfs":{"total":95995035648,"available":95988191232,"internally_reserved":0,"allocated":937984,"data_stored":444408,"data_compressed":5787,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.36599999999999999,"15min":0.33500000000000002},"min":{"1min":0.254,"5min":0.22,"15min":0.22},"max":{"1min":0.60499999999999998,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.38500000000000001,"15min":0.35699999999999998},"min":{"1min":0.27300000000000002,"5min":0.214,"15min":0.214},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.35899999999999999}]},{"osd":1,"last update":"Mon Nov 27 15:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53800000000000003,"15min":0.505},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78500000000000003,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53800000000000003,"15min":0.52400000000000002},"min":{"1min":0.376,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82599999999999996,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.499}]},{"osd":2,"last update":"Mon Nov 27 15:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58699999999999997,"15min":0.54500000000000004},"min":{"1min":0.40200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80100000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58099999999999996,"15min":0.54400000000000004},"min":{"1min":0.41699999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.81100000000000005,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.47999999999999998}]},{"osd":4,"last update":"Mon Nov 27 15:17:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55300000000000005,"15min":0.52700000000000002},"min":{"1min":0.41799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78900000000000003,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.52800000000000002},"min":{"1min":0.40300000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79700000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.56999999999999995}]},{"osd":5,"last update":"Mon Nov 27 15:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.624,"15min":0.58199999999999996},"min":{"1min":0.44900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.64300000000000002,"15min":0.59599999999999997},"min":{"1min":0.47299999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":0.51100000000000001}]},{"osd":6,"last update":"Mon Nov 27 15:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56999999999999995,"15min":0.55000000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.91100000000000003,"5min":1.089,"15min":1.089},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55900000000000005,"15min":0.54300000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.42899999999999999}]},{"osd":7,"last update":"Mon Nov 27 15:17:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.45900000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.80100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72599999999999998,"15min":0.70899999999999996},"min":{"1min":0.57299999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.83299999999999996,"5min":1.034,"15min":1.034},"last":0.67300000000000004}]},{"osd":8,"last update":"Mon Nov 27 15:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.628}]},{"osd":9,"last update":"Mon Nov 27 15:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.80600000000000005,"15min":0.82599999999999996},"min":{"1min":0.49299999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.083,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.625},{"interface":"front","average":{"1min":0.63,"5min":0.73799999999999999,"15min":0.75600000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95399999999999996,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.56200000000000006}]},{"osd":10,"last update":"Mon Nov 27 15:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.71799999999999997,"15min":0.73199999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.77200000000000002,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.84399999999999997,"15min":0.86199999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.86899999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.65200000000000002}]},{"osd":11,"last update":"Mon Nov 27 15:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.64000000000000001}]}]},{"osd":4,"up_from":33,"seq":141733920829,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6552,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738600,"statfs":{"total":95995035648,"available":95988326400,"internally_reserved":0,"allocated":933888,"data_stored":446428,"data_compressed":5787,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.48099999999999998,"15min":0.46800000000000003},"min":{"1min":0.35299999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.52200000000000002,"15min":0.50700000000000001},"min":{"1min":0.40300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.432}]},{"osd":1,"last update":"Mon Nov 27 15:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.39400000000000002,"15min":0.35599999999999998},"min":{"1min":0.29999999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.77200000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.39900000000000002,"15min":0.35699999999999998},"min":{"1min":0.318,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.90500000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.55600000000000005}]},{"osd":2,"last update":"Mon Nov 27 15:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.52800000000000002,"15min":0.496},"min":{"1min":0.36599999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.56299999999999994,"15min":0.54100000000000004},"min":{"1min":0.42599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.65000000000000002}]},{"osd":3,"last update":"Mon Nov 27 15:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.56299999999999994,"15min":0.52200000000000002},"min":{"1min":0.48799999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.872,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.71899999999999997}]},{"osd":5,"last update":"Mon Nov 27 15:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.58399999999999996,"15min":0.54300000000000004},"min":{"1min":0.47599999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.60299999999999998,"15min":0.56799999999999995},"min":{"1min":0.56100000000000005,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.74399999999999999}]},{"osd":6,"last update":"Mon Nov 27 15:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.63500000000000001,"15min":0.59999999999999998},"min":{"1min":0.46500000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":5.1040000000000001,"5min":5.1040000000000001,"15min":5.1040000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.67400000000000004,"15min":0.63800000000000001},"min":{"1min":0.46700000000000003,"5min":0.441,"15min":0.441},"max":{"1min":5.1139999999999999,"5min":5.1139999999999999,"15min":5.1139999999999999},"last":0.76700000000000002}]},{"osd":7,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.56000000000000005,"15min":0.53300000000000003},"min":{"1min":0.376,"5min":0.316,"15min":0.316},"max":{"1min":0.90300000000000002,"5min":1.018,"15min":1.018},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58799999999999997,"15min":0.55500000000000005},"min":{"1min":0.39400000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.873,"5min":1.115,"15min":1.115},"last":0.81599999999999995}]},{"osd":8,"last update":"Mon Nov 27 15:17:25 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.68799999999999994,"15min":0.66600000000000004},"min":{"1min":0.51900000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94299999999999995,"5min":1.054,"15min":1.054},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.71999999999999997,"15min":0.70099999999999996},"min":{"1min":0.63100000000000001,"5min":0.432,"15min":0.432},"max":{"1min":1.022,"5min":1.179,"15min":1.179},"last":0.84999999999999998}]},{"osd":9,"last update":"Mon Nov 27 15:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.78100000000000003,"15min":0.75900000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":4.8780000000000001,"5min":4.8780000000000001,"15min":4.8780000000000001},"last":0.878},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.747,"15min":0.72299999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":4.9169999999999998,"5min":4.9169999999999998,"15min":4.9169999999999998},"last":0.86099999999999999}]},{"osd":10,"last update":"Mon Nov 27 15:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.75900000000000001,"15min":0.76000000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95899999999999996,"5min":1.103,"15min":1.103},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.91200000000000003}]},{"osd":11,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.92500000000000004}]}]},{"osd":5,"up_from":38,"seq":163208757304,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6676,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738476,"statfs":{"total":95995035648,"available":95988199424,"internally_reserved":0,"allocated":995328,"data_stored":1036328,"data_compressed":41881,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.50900000000000001,"15min":0.497},"min":{"1min":0.37,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.56000000000000005,"15min":0.498},"min":{"1min":0.39900000000000002,"5min":0.312,"15min":0.312},"max":{"1min":7.673,"5min":7.673,"15min":7.673},"last":0.44}]},{"osd":1,"last update":"Mon Nov 27 15:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.56899999999999995,"15min":0.53500000000000003},"min":{"1min":0.44700000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.57099999999999995,"15min":0.52500000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.60599999999999998}]},{"osd":2,"last update":"Mon Nov 27 15:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.41899999999999998,"15min":0.39600000000000002},"min":{"1min":0.29499999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.60499999999999998,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.628},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.44500000000000001,"15min":0.41299999999999998},"min":{"1min":0.28399999999999997,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.70599999999999996,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.54500000000000004}]},{"osd":3,"last update":"Mon Nov 27 15:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.623,"15min":0.57099999999999995},"min":{"1min":0.433,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":7.875,"5min":7.875,"15min":7.875},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.621,"15min":0.56200000000000006},"min":{"1min":0.46500000000000002,"5min":0.372,"15min":0.372},"max":{"1min":7.6059999999999999,"5min":7.6059999999999999,"15min":7.6059999999999999},"last":0.51700000000000002}]},{"osd":4,"last update":"Mon Nov 27 15:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.60299999999999998,"15min":0.56399999999999995},"min":{"1min":0.50600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.58199999999999996},"min":{"1min":0.42699999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.64700000000000002}]},{"osd":6,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":0.68500000000000005,"15min":0.61899999999999999},"min":{"1min":0.50800000000000001,"5min":0.436,"15min":0.436},"max":{"1min":8.0280000000000005,"5min":8.0280000000000005,"15min":8.0280000000000005},"last":0.69899999999999995},{"interface":"front","average":{"1min":1.1319999999999999,"5min":0.73899999999999999,"15min":0.67100000000000004},"min":{"1min":0.51800000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":8.0389999999999997,"5min":8.0389999999999997,"15min":8.0389999999999997},"last":0.68799999999999994}]},{"osd":7,"last update":"Mon Nov 27 15:17:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.67800000000000005,"15min":0.65900000000000003},"min":{"1min":0.498,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67000000000000004,"15min":0.66000000000000003},"min":{"1min":0.56100000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90400000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.73799999999999999}]},{"osd":8,"last update":"Mon Nov 27 15:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.66400000000000003,"15min":0.64400000000000002},"min":{"1min":0.38600000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.752},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.60899999999999999,"15min":0.59699999999999998},"min":{"1min":0.39700000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80900000000000005,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.71399999999999997}]},{"osd":9,"last update":"Mon Nov 27 15:16:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":0.747,"15min":0.69199999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":7.8070000000000004,"5min":7.8070000000000004,"15min":7.8070000000000004},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.194,"5min":0.81899999999999995,"15min":0.75700000000000001},"min":{"1min":0.56599999999999995,"5min":0.503,"15min":0.503},"max":{"1min":7.7690000000000001,"5min":7.7690000000000001,"15min":7.7690000000000001},"last":0.79200000000000004}]},{"osd":10,"last update":"Mon Nov 27 15:17:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.76000000000000001,"15min":0.747},"min":{"1min":0.63900000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.75,"5min":0.69599999999999995,"15min":0.68700000000000006},"min":{"1min":0.57099999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.77900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34455,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34455,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":59470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-27T15:17:42.848 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-11-27T15:17:42.848 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-27T15:17:43.009 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.005+0000 7f22e68d3700 1 -- 172.21.15.52:0/1774488998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22e0100710 msgr2=0x7f22e0100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:43.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.005+0000 7f22e68d3700 1 --2- 172.21.15.52:0/1774488998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22e0100710 0x7f22e0100b20 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009990 tx=0x7f22d0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.005+0000 7f22e68d3700 1 -- 172.21.15.52:0/1774488998 shutdown_connections 2023-11-27T15:17:43.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.005+0000 7f22e68d3700 1 --2- 172.21.15.52:0/1774488998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22e00ffc80 0x7f22e0107530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.005+0000 7f22e68d3700 1 --2- 172.21.15.52:0/1774488998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22e0100710 0x7f22e0100b20 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.005+0000 7f22e68d3700 1 --2- 172.21.15.52:0/1774488998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22e00ff2d0 0x7f22e00ff740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.005+0000 7f22e68d3700 1 -- 172.21.15.52:0/1774488998 >> 172.21.15.52:0/1774488998 conn(0x7f22e00f9e10 msgr2=0x7f22e00fc260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:43.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.005+0000 7f22e68d3700 1 -- 172.21.15.52:0/1774488998 shutdown_connections 2023-11-27T15:17:43.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.005+0000 7f22e68d3700 1 -- 172.21.15.52:0/1774488998 wait complete. 2023-11-27T15:17:43.013 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.006+0000 7f22e68d3700 1 Processor -- start 2023-11-27T15:17:43.013 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.006+0000 7f22e68d3700 1 -- start start 2023-11-27T15:17:43.013 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.006+0000 7f22e68d3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22e00ff2d0 0x7f22e0194dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:43.013 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.006+0000 7f22e68d3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22e00ffc80 0x7f22e0195310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:43.013 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.006+0000 7f22e68d3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22e0100710 0x7f22e019c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:43.013 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.006+0000 7f22e68d3700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e019c950 con 0x7f22e00ff2d0 2023-11-27T15:17:43.013 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.006+0000 7f22e68d3700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e019ca90 con 0x7f22e00ffc80 2023-11-27T15:17:43.013 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.006+0000 7f22e68d3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e019cc20 con 0x7f22e0100710 2023-11-27T15:17:43.013 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.007+0000 7f22dffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22e00ff2d0 0x7f22e0194dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:43.013 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.007+0000 7f22dffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22e00ff2d0 0x7f22e0194dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51500/0 (socket says 172.21.15.52:51500) 2023-11-27T15:17:43.013 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.007+0000 7f22dffff700 1 -- 172.21.15.52:0/1618500727 learned_addr learned my addr 172.21.15.52:0/1618500727 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:43.014 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.007+0000 7f22e4e70700 1 --2- 172.21.15.52:0/1618500727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22e0100710 0x7f22e019c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:43.014 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.007+0000 7f22df7fe700 1 --2- 172.21.15.52:0/1618500727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22e00ffc80 0x7f22e0195310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:43.014 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.007+0000 7f22dffff700 1 -- 172.21.15.52:0/1618500727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22e0100710 msgr2=0x7f22e019c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:43.014 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.007+0000 7f22dffff700 1 --2- 172.21.15.52:0/1618500727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22e0100710 0x7f22e019c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.014 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.007+0000 7f22dffff700 1 -- 172.21.15.52:0/1618500727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22e00ffc80 msgr2=0x7f22e0195310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:43.014 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.007+0000 7f22dffff700 1 --2- 172.21.15.52:0/1618500727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22e00ffc80 0x7f22e0195310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.014 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.007+0000 7f22dffff700 1 -- 172.21.15.52:0/1618500727 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d00095f0 con 0x7f22e00ff2d0 2023-11-27T15:17:43.014 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.007+0000 7f22df7fe700 1 --2- 172.21.15.52:0/1618500727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22e00ffc80 0x7f22e0195310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:17:43.014 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.007+0000 7f22dffff700 1 --2- 172.21.15.52:0/1618500727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22e00ff2d0 0x7f22e0194dd0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f22d0000c00 tx=0x7f22d001a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:43.014 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.007+0000 7f22dd7fa700 1 -- 172.21.15.52:0/1618500727 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d001f070 con 0x7f22e00ff2d0 2023-11-27T15:17:43.015 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.007+0000 7f22dd7fa700 1 -- 172.21.15.52:0/1618500727 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22d000f460 con 0x7f22e00ff2d0 2023-11-27T15:17:43.015 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.007+0000 7f22dd7fa700 1 -- 172.21.15.52:0/1618500727 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d0025680 con 0x7f22e00ff2d0 2023-11-27T15:17:43.015 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.007+0000 7f22e68d3700 1 -- 172.21.15.52:0/1618500727 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e010aa30 con 0x7f22e00ff2d0 2023-11-27T15:17:43.015 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.007+0000 7f22e68d3700 1 -- 172.21.15.52:0/1618500727 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e010a8a0 con 0x7f22e00ff2d0 2023-11-27T15:17:43.015 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.008+0000 7f22e68d3700 1 -- 172.21.15.52:0/1618500727 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e00fecf0 con 0x7f22e00ff2d0 2023-11-27T15:17:43.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.010+0000 7f22dd7fa700 1 -- 172.21.15.52:0/1618500727 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f22d0023070 con 0x7f22e00ff2d0 2023-11-27T15:17:43.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.010+0000 7f22dd7fa700 1 --2- 172.21.15.52:0/1618500727 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f22c80a1150 0x7f22c80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:43.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.011+0000 7f22df7fe700 1 --2- 172.21.15.52:0/1618500727 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f22c80a1150 0x7f22c80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:43.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.011+0000 7f22dd7fa700 1 -- 172.21.15.52:0/1618500727 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f22d00d01b0 con 0x7f22e00ff2d0 2023-11-27T15:17:43.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.011+0000 7f22df7fe700 1 --2- 172.21.15.52:0/1618500727 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f22c80a1150 0x7f22c80a3600 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f22d4005fd0 tx=0x7f22d4005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:43.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.014+0000 7f22dd7fa700 1 -- 172.21.15.52:0/1618500727 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f22d0099910 con 0x7f22e00ff2d0 2023-11-27T15:17:43.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:42 smithi102 ceph-mon[100296]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 6 op/s 2023-11-27T15:17:43.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:42 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3372519384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:17:43.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:42 smithi102 ceph-mon[100296]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:17:43.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:42 smithi052 ceph-mon[99056]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 6 op/s 2023-11-27T15:17:43.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:42 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3372519384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:17:43.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:42 smithi052 ceph-mon[99056]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:17:43.213 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.209+0000 7f22e68d3700 1 -- 172.21.15.52:0/1618500727 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f22e00fe520 con 0x7f22c80a1150 2023-11-27T15:17:43.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.221+0000 7f22dd7fa700 1 -- 172.21.15.52:0/1618500727 <== mgr.14152 v2:172.21.15.52:6800/2640036585 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344013 (secure 0 0 0) 0x7f22e00fe520 con 0x7f22c80a1150 2023-11-27T15:17:43.230 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:17:43.232 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.225+0000 7f22e68d3700 1 -- 172.21.15.52:0/1618500727 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f22c80a1150 msgr2=0x7f22c80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:43.232 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.225+0000 7f22e68d3700 1 --2- 172.21.15.52:0/1618500727 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f22c80a1150 0x7f22c80a3600 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f22d4005fd0 tx=0x7f22d4005ee0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.232 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.225+0000 7f22e68d3700 1 -- 172.21.15.52:0/1618500727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22e00ff2d0 msgr2=0x7f22e0194dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:43.232 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.225+0000 7f22e68d3700 1 --2- 172.21.15.52:0/1618500727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22e00ff2d0 0x7f22e0194dd0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f22d0000c00 tx=0x7f22d001a8c0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.233 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.225+0000 7f22e68d3700 1 -- 172.21.15.52:0/1618500727 shutdown_connections 2023-11-27T15:17:43.233 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.225+0000 7f22e68d3700 1 --2- 172.21.15.52:0/1618500727 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f22c80a1150 0x7f22c80a3600 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.233 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.225+0000 7f22e68d3700 1 --2- 172.21.15.52:0/1618500727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22e0100710 0x7f22e019c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.233 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.225+0000 7f22e68d3700 1 --2- 172.21.15.52:0/1618500727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22e00ff2d0 0x7f22e0194dd0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.233 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.225+0000 7f22e68d3700 1 --2- 172.21.15.52:0/1618500727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22e00ffc80 0x7f22e0195310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.233 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.225+0000 7f22e68d3700 1 -- 172.21.15.52:0/1618500727 >> 172.21.15.52:0/1618500727 conn(0x7f22e00f9e10 msgr2=0x7f22e00fc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:43.233 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.225+0000 7f22e68d3700 1 -- 172.21.15.52:0/1618500727 shutdown_connections 2023-11-27T15:17:43.233 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.225+0000 7f22e68d3700 1 -- 172.21.15.52:0/1618500727 wait complete. 2023-11-27T15:17:43.233 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2023-11-27T15:17:43.241 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2023-11-27T15:17:41.748543+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654284,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":669,"num_read_kb":455,"num_write":346,"num_write_kb":3053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181756,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":782,"ondisk_log_size":782,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":80372,"kb_used_data":11284,"kb_used_omap":0,"kb_used_meta":68992,"kb_avail":1124861452,"statfs":{"total":1151940427776,"available":1151858126848,"internally_reserved":0,"allocated":11554816,"data_stored":7294767,"data_compressed":177726,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":0,"internal_metadata":70647808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":849,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":34,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005611"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.765734+0000","last_change":"2023-11-27T15:16:09.343007+0000","last_active":"2023-11-27T15:17:33.765734+0000","last_peered":"2023-11-27T15:17:33.765734+0000","last_clean":"2023-11-27T15:17:33.765734+0000","last_became_active":"2023-11-27T15:16:09.342125+0000","last_became_peered":"2023-11-27T15:16:09.342125+0000","last_unstale":"2023-11-27T15:17:33.765734+0000","last_undegraded":"2023-11-27T15:17:33.765734+0000","last_fullsized":"2023-11-27T15:17:33.765734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:24:26.323974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.764972+0000","last_change":"2023-11-27T15:16:09.338942+0000","last_active":"2023-11-27T15:17:33.764972+0000","last_peered":"2023-11-27T15:17:33.764972+0000","last_clean":"2023-11-27T15:17:33.764972+0000","last_became_active":"2023-11-27T15:16:09.338636+0000","last_became_peered":"2023-11-27T15:16:09.338636+0000","last_unstale":"2023-11-27T15:17:33.764972+0000","last_undegraded":"2023-11-27T15:17:33.764972+0000","last_fullsized":"2023-11-27T15:17:33.764972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:40:36.907284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.765902+0000","last_change":"2023-11-27T15:16:09.347845+0000","last_active":"2023-11-27T15:17:33.765902+0000","last_peered":"2023-11-27T15:17:33.765902+0000","last_clean":"2023-11-27T15:17:33.765902+0000","last_became_active":"2023-11-27T15:16:09.347533+0000","last_became_peered":"2023-11-27T15:16:09.347533+0000","last_unstale":"2023-11-27T15:17:33.765902+0000","last_undegraded":"2023-11-27T15:17:33.765902+0000","last_fullsized":"2023-11-27T15:17:33.765902+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:30:09.473655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.762843+0000","last_change":"2023-11-27T15:17:04.811819+0000","last_active":"2023-11-27T15:17:33.762843+0000","last_peered":"2023-11-27T15:17:33.762843+0000","last_clean":"2023-11-27T15:17:33.762843+0000","last_became_active":"2023-11-27T15:17:04.811443+0000","last_became_peered":"2023-11-27T15:17:04.811443+0000","last_unstale":"2023-11-27T15:17:33.762843+0000","last_undegraded":"2023-11-27T15:17:33.762843+0000","last_fullsized":"2023-11-27T15:17:33.762843+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:03:57.997950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.772322+0000","last_change":"2023-11-27T15:16:09.349097+0000","last_active":"2023-11-27T15:17:33.772322+0000","last_peered":"2023-11-27T15:17:33.772322+0000","last_clean":"2023-11-27T15:17:33.772322+0000","last_became_active":"2023-11-27T15:16:09.348793+0000","last_became_peered":"2023-11-27T15:16:09.348793+0000","last_unstale":"2023-11-27T15:17:33.772322+0000","last_undegraded":"2023-11-27T15:17:33.772322+0000","last_fullsized":"2023-11-27T15:17:33.772322+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:26:40.373634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.765958+0000","last_change":"2023-11-27T15:16:09.338246+0000","last_active":"2023-11-27T15:17:33.765958+0000","last_peered":"2023-11-27T15:17:33.765958+0000","last_clean":"2023-11-27T15:17:33.765958+0000","last_became_active":"2023-11-27T15:16:09.338003+0000","last_became_peered":"2023-11-27T15:16:09.338003+0000","last_unstale":"2023-11-27T15:17:33.765958+0000","last_undegraded":"2023-11-27T15:17:33.765958+0000","last_fullsized":"2023-11-27T15:17:33.765958+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:50:11.348331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.773179+0000","last_change":"2023-11-27T15:16:09.342131+0000","last_active":"2023-11-27T15:17:33.773179+0000","last_peered":"2023-11-27T15:17:33.773179+0000","last_clean":"2023-11-27T15:17:33.773179+0000","last_became_active":"2023-11-27T15:16:09.341834+0000","last_became_peered":"2023-11-27T15:16:09.341834+0000","last_unstale":"2023-11-27T15:17:33.773179+0000","last_undegraded":"2023-11-27T15:17:33.773179+0000","last_fullsized":"2023-11-27T15:17:33.773179+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:22:39.182216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.776357+0000","last_change":"2023-11-27T15:16:09.337308+0000","last_active":"2023-11-27T15:17:33.776357+0000","last_peered":"2023-11-27T15:17:33.776357+0000","last_clean":"2023-11-27T15:17:33.776357+0000","last_became_active":"2023-11-27T15:16:09.336958+0000","last_became_peered":"2023-11-27T15:16:09.336958+0000","last_unstale":"2023-11-27T15:17:33.776357+0000","last_undegraded":"2023-11-27T15:17:33.776357+0000","last_fullsized":"2023-11-27T15:17:33.776357+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:00:00.299148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.771230+0000","last_change":"2023-11-27T15:17:04.810530+0000","last_active":"2023-11-27T15:17:33.771230+0000","last_peered":"2023-11-27T15:17:33.771230+0000","last_clean":"2023-11-27T15:17:33.771230+0000","last_became_active":"2023-11-27T15:17:04.810197+0000","last_became_peered":"2023-11-27T15:17:04.810197+0000","last_unstale":"2023-11-27T15:17:33.771230+0000","last_undegraded":"2023-11-27T15:17:33.771230+0000","last_fullsized":"2023-11-27T15:17:33.771230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:58:40.385968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,11],"acting":[1,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.771264+0000","last_change":"2023-11-27T15:16:09.336871+0000","last_active":"2023-11-27T15:17:33.771264+0000","last_peered":"2023-11-27T15:17:33.771264+0000","last_clean":"2023-11-27T15:17:33.771264+0000","last_became_active":"2023-11-27T15:16:09.336496+0000","last_became_peered":"2023-11-27T15:16:09.336496+0000","last_unstale":"2023-11-27T15:17:33.771264+0000","last_undegraded":"2023-11-27T15:17:33.771264+0000","last_fullsized":"2023-11-27T15:17:33.771264+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:12:27.254009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.776130+0000","last_change":"2023-11-27T15:16:09.343120+0000","last_active":"2023-11-27T15:17:33.776130+0000","last_peered":"2023-11-27T15:17:33.776130+0000","last_clean":"2023-11-27T15:17:33.776130+0000","last_became_active":"2023-11-27T15:16:09.342797+0000","last_became_peered":"2023-11-27T15:16:09.342797+0000","last_unstale":"2023-11-27T15:17:33.776130+0000","last_undegraded":"2023-11-27T15:17:33.776130+0000","last_fullsized":"2023-11-27T15:17:33.776130+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:22:16.706218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.759194+0000","last_change":"2023-11-27T15:16:09.336521+0000","last_active":"2023-11-27T15:17:33.759194+0000","last_peered":"2023-11-27T15:17:33.759194+0000","last_clean":"2023-11-27T15:17:33.759194+0000","last_became_active":"2023-11-27T15:16:09.336183+0000","last_became_peered":"2023-11-27T15:16:09.336183+0000","last_unstale":"2023-11-27T15:17:33.759194+0000","last_undegraded":"2023-11-27T15:17:33.759194+0000","last_fullsized":"2023-11-27T15:17:33.759194+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:54:55.143503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.775471+0000","last_change":"2023-11-27T15:16:09.346839+0000","last_active":"2023-11-27T15:17:33.775471+0000","last_peered":"2023-11-27T15:17:33.775471+0000","last_clean":"2023-11-27T15:17:33.775471+0000","last_became_active":"2023-11-27T15:16:09.345230+0000","last_became_peered":"2023-11-27T15:16:09.345230+0000","last_unstale":"2023-11-27T15:17:33.775471+0000","last_undegraded":"2023-11-27T15:17:33.775471+0000","last_fullsized":"2023-11-27T15:17:33.775471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:35:36.915491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.758451+0000","last_change":"2023-11-27T15:16:09.342380+0000","last_active":"2023-11-27T15:17:33.758451+0000","last_peered":"2023-11-27T15:17:33.758451+0000","last_clean":"2023-11-27T15:17:33.758451+0000","last_became_active":"2023-11-27T15:16:09.341967+0000","last_became_peered":"2023-11-27T15:16:09.341967+0000","last_unstale":"2023-11-27T15:17:33.758451+0000","last_undegraded":"2023-11-27T15:17:33.758451+0000","last_fullsized":"2023-11-27T15:17:33.758451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:12:37.584284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.754779+0000","last_change":"2023-11-27T15:16:09.349738+0000","last_active":"2023-11-27T15:17:33.754779+0000","last_peered":"2023-11-27T15:17:33.754779+0000","last_clean":"2023-11-27T15:17:33.754779+0000","last_became_active":"2023-11-27T15:16:09.349349+0000","last_became_peered":"2023-11-27T15:16:09.349349+0000","last_unstale":"2023-11-27T15:17:33.754779+0000","last_undegraded":"2023-11-27T15:17:33.754779+0000","last_fullsized":"2023-11-27T15:17:33.754779+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:59:30.065309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.776017+0000","last_change":"2023-11-27T15:16:09.339171+0000","last_active":"2023-11-27T15:17:33.776017+0000","last_peered":"2023-11-27T15:17:33.776017+0000","last_clean":"2023-11-27T15:17:33.776017+0000","last_became_active":"2023-11-27T15:16:09.338887+0000","last_became_peered":"2023-11-27T15:16:09.338887+0000","last_unstale":"2023-11-27T15:17:33.776017+0000","last_undegraded":"2023-11-27T15:17:33.776017+0000","last_fullsized":"2023-11-27T15:17:33.776017+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:08:50.297424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.768341+0000","last_change":"2023-11-27T15:16:09.343202+0000","last_active":"2023-11-27T15:17:33.768341+0000","last_peered":"2023-11-27T15:17:33.768341+0000","last_clean":"2023-11-27T15:17:33.768341+0000","last_became_active":"2023-11-27T15:16:09.342539+0000","last_became_peered":"2023-11-27T15:16:09.342539+0000","last_unstale":"2023-11-27T15:17:33.768341+0000","last_undegraded":"2023-11-27T15:17:33.768341+0000","last_fullsized":"2023-11-27T15:17:33.768341+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:53:20.106298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.772964+0000","last_change":"2023-11-27T15:16:09.339915+0000","last_active":"2023-11-27T15:17:33.772964+0000","last_peered":"2023-11-27T15:17:33.772964+0000","last_clean":"2023-11-27T15:17:33.772964+0000","last_became_active":"2023-11-27T15:16:09.339637+0000","last_became_peered":"2023-11-27T15:16:09.339637+0000","last_unstale":"2023-11-27T15:17:33.772964+0000","last_undegraded":"2023-11-27T15:17:33.772964+0000","last_fullsized":"2023-11-27T15:17:33.772964+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:04:48.736761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.773767+0000","last_change":"2023-11-27T15:16:09.347852+0000","last_active":"2023-11-27T15:17:33.773767+0000","last_peered":"2023-11-27T15:17:33.773767+0000","last_clean":"2023-11-27T15:17:33.773767+0000","last_became_active":"2023-11-27T15:16:09.347281+0000","last_became_peered":"2023-11-27T15:16:09.347281+0000","last_unstale":"2023-11-27T15:17:33.773767+0000","last_undegraded":"2023-11-27T15:17:33.773767+0000","last_fullsized":"2023-11-27T15:17:33.773767+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:29:49.527742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.806625+0000","last_change":"2023-11-27T15:16:08.912206+0000","last_active":"2023-11-27T15:17:04.806625+0000","last_peered":"2023-11-27T15:17:04.806625+0000","last_clean":"2023-11-27T15:17:04.806625+0000","last_became_active":"2023-11-27T15:16:08.911846+0000","last_became_peered":"2023-11-27T15:16:08.911846+0000","last_unstale":"2023-11-27T15:17:04.806625+0000","last_undegraded":"2023-11-27T15:17:04.806625+0000","last_fullsized":"2023-11-27T15:17:04.806625+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:38:19.552326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.770773+0000","last_change":"2023-11-27T15:16:09.336485+0000","last_active":"2023-11-27T15:17:33.770773+0000","last_peered":"2023-11-27T15:17:33.770773+0000","last_clean":"2023-11-27T15:17:33.770773+0000","last_became_active":"2023-11-27T15:16:09.336236+0000","last_became_peered":"2023-11-27T15:16:09.336236+0000","last_unstale":"2023-11-27T15:17:33.770773+0000","last_undegraded":"2023-11-27T15:17:33.770773+0000","last_fullsized":"2023-11-27T15:17:33.770773+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:56:02.954043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.808190+0000","last_change":"2023-11-27T15:16:08.921198+0000","last_active":"2023-11-27T15:17:04.808190+0000","last_peered":"2023-11-27T15:17:04.808190+0000","last_clean":"2023-11-27T15:17:04.808190+0000","last_became_active":"2023-11-27T15:16:08.920804+0000","last_became_peered":"2023-11-27T15:16:08.920804+0000","last_unstale":"2023-11-27T15:17:04.808190+0000","last_undegraded":"2023-11-27T15:17:04.808190+0000","last_fullsized":"2023-11-27T15:17:04.808190+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:29:23.298656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.767388+0000","last_change":"2023-11-27T15:16:09.340642+0000","last_active":"2023-11-27T15:17:33.767388+0000","last_peered":"2023-11-27T15:17:33.767388+0000","last_clean":"2023-11-27T15:17:33.767388+0000","last_became_active":"2023-11-27T15:16:09.340328+0000","last_became_peered":"2023-11-27T15:16:09.340328+0000","last_unstale":"2023-11-27T15:17:33.767388+0000","last_undegraded":"2023-11-27T15:17:33.767388+0000","last_fullsized":"2023-11-27T15:17:33.767388+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:46:04.794567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.810213+0000","last_change":"2023-11-27T15:16:08.915317+0000","last_active":"2023-11-27T15:17:04.810213+0000","last_peered":"2023-11-27T15:17:04.810213+0000","last_clean":"2023-11-27T15:17:04.810213+0000","last_became_active":"2023-11-27T15:16:08.914949+0000","last_became_peered":"2023-11-27T15:16:08.914949+0000","last_unstale":"2023-11-27T15:17:04.810213+0000","last_undegraded":"2023-11-27T15:17:04.810213+0000","last_fullsized":"2023-11-27T15:17:04.810213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:06:23.361651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.773082+0000","last_change":"2023-11-27T15:16:09.348483+0000","last_active":"2023-11-27T15:17:33.773082+0000","last_peered":"2023-11-27T15:17:33.773082+0000","last_clean":"2023-11-27T15:17:33.773082+0000","last_became_active":"2023-11-27T15:16:09.348185+0000","last_became_peered":"2023-11-27T15:16:09.348185+0000","last_unstale":"2023-11-27T15:17:33.773082+0000","last_undegraded":"2023-11-27T15:17:33.773082+0000","last_fullsized":"2023-11-27T15:17:33.773082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:48:03.154051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.809047+0000","last_change":"2023-11-27T15:16:08.911018+0000","last_active":"2023-11-27T15:17:04.809047+0000","last_peered":"2023-11-27T15:17:04.809047+0000","last_clean":"2023-11-27T15:17:04.809047+0000","last_became_active":"2023-11-27T15:16:08.910684+0000","last_became_peered":"2023-11-27T15:16:08.910684+0000","last_unstale":"2023-11-27T15:17:04.809047+0000","last_undegraded":"2023-11-27T15:17:04.809047+0000","last_fullsized":"2023-11-27T15:17:04.809047+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:08:38.910735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.774391+0000","last_change":"2023-11-27T15:16:09.341414+0000","last_active":"2023-11-27T15:17:33.774391+0000","last_peered":"2023-11-27T15:17:33.774391+0000","last_clean":"2023-11-27T15:17:33.774391+0000","last_became_active":"2023-11-27T15:16:09.341068+0000","last_became_peered":"2023-11-27T15:16:09.341068+0000","last_unstale":"2023-11-27T15:17:33.774391+0000","last_undegraded":"2023-11-27T15:17:33.774391+0000","last_fullsized":"2023-11-27T15:17:33.774391+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:24:59.603494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'7","reported_seq":125,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:36.892827+0000","last_change":"2023-11-27T15:17:04.849470+0000","last_active":"2023-11-27T15:17:36.892827+0000","last_peered":"2023-11-27T15:17:36.892827+0000","last_clean":"2023-11-27T15:17:36.892827+0000","last_became_active":"2023-11-27T15:17:04.845827+0000","last_became_peered":"2023-11-27T15:17:04.845827+0000","last_unstale":"2023-11-27T15:17:36.892827+0000","last_undegraded":"2023-11-27T15:17:36.892827+0000","last_fullsized":"2023-11-27T15:17:36.892827+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:59:52.295722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1105,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":964,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.758486+0000","last_change":"2023-11-27T15:16:09.337330+0000","last_active":"2023-11-27T15:17:33.758486+0000","last_peered":"2023-11-27T15:17:33.758486+0000","last_clean":"2023-11-27T15:17:33.758486+0000","last_became_active":"2023-11-27T15:16:09.336991+0000","last_became_peered":"2023-11-27T15:16:09.336991+0000","last_unstale":"2023-11-27T15:17:33.758486+0000","last_undegraded":"2023-11-27T15:17:33.758486+0000","last_fullsized":"2023-11-27T15:17:33.758486+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:35:01.499613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.809306+0000","last_change":"2023-11-27T15:16:08.912208+0000","last_active":"2023-11-27T15:17:04.809306+0000","last_peered":"2023-11-27T15:17:04.809306+0000","last_clean":"2023-11-27T15:17:04.809306+0000","last_became_active":"2023-11-27T15:16:08.911879+0000","last_became_peered":"2023-11-27T15:16:08.911879+0000","last_unstale":"2023-11-27T15:17:04.809306+0000","last_undegraded":"2023-11-27T15:17:04.809306+0000","last_fullsized":"2023-11-27T15:17:04.809306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:34:55.140701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.772328+0000","last_change":"2023-11-27T15:16:09.339660+0000","last_active":"2023-11-27T15:17:33.772328+0000","last_peered":"2023-11-27T15:17:33.772328+0000","last_clean":"2023-11-27T15:17:33.772328+0000","last_became_active":"2023-11-27T15:16:09.339323+0000","last_became_peered":"2023-11-27T15:16:09.339323+0000","last_unstale":"2023-11-27T15:17:33.772328+0000","last_undegraded":"2023-11-27T15:17:33.772328+0000","last_fullsized":"2023-11-27T15:17:33.772328+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:31:05.598344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.803818+0000","last_change":"2023-11-27T15:16:08.911103+0000","last_active":"2023-11-27T15:17:04.803818+0000","last_peered":"2023-11-27T15:17:04.803818+0000","last_clean":"2023-11-27T15:17:04.803818+0000","last_became_active":"2023-11-27T15:16:08.910765+0000","last_became_peered":"2023-11-27T15:16:08.910765+0000","last_unstale":"2023-11-27T15:17:04.803818+0000","last_undegraded":"2023-11-27T15:17:04.803818+0000","last_fullsized":"2023-11-27T15:17:04.803818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:08:20.782772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.774114+0000","last_change":"2023-11-27T15:16:09.351741+0000","last_active":"2023-11-27T15:17:33.774114+0000","last_peered":"2023-11-27T15:17:33.774114+0000","last_clean":"2023-11-27T15:17:33.774114+0000","last_became_active":"2023-11-27T15:16:09.351280+0000","last_became_peered":"2023-11-27T15:16:09.351280+0000","last_unstale":"2023-11-27T15:17:33.774114+0000","last_undegraded":"2023-11-27T15:17:33.774114+0000","last_fullsized":"2023-11-27T15:17:33.774114+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:29:29.349396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.802775+0000","last_change":"2023-11-27T15:16:08.915630+0000","last_active":"2023-11-27T15:17:04.802775+0000","last_peered":"2023-11-27T15:17:04.802775+0000","last_clean":"2023-11-27T15:17:04.802775+0000","last_became_active":"2023-11-27T15:16:08.915294+0000","last_became_peered":"2023-11-27T15:16:08.915294+0000","last_unstale":"2023-11-27T15:17:04.802775+0000","last_undegraded":"2023-11-27T15:17:04.802775+0000","last_fullsized":"2023-11-27T15:17:04.802775+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:58:19.686545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.777372+0000","last_change":"2023-11-27T15:16:09.338436+0000","last_active":"2023-11-27T15:17:33.777372+0000","last_peered":"2023-11-27T15:17:33.777372+0000","last_clean":"2023-11-27T15:17:33.777372+0000","last_became_active":"2023-11-27T15:16:09.338148+0000","last_became_peered":"2023-11-27T15:16:09.338148+0000","last_unstale":"2023-11-27T15:17:33.777372+0000","last_undegraded":"2023-11-27T15:17:33.777372+0000","last_fullsized":"2023-11-27T15:17:33.777372+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:49:40.837798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":106,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:36.892579+0000","last_change":"2023-11-27T15:16:08.913072+0000","last_active":"2023-11-27T15:17:36.892579+0000","last_peered":"2023-11-27T15:17:36.892579+0000","last_clean":"2023-11-27T15:17:36.892579+0000","last_became_active":"2023-11-27T15:16:08.912757+0000","last_became_peered":"2023-11-27T15:16:08.912757+0000","last_unstale":"2023-11-27T15:17:36.892579+0000","last_undegraded":"2023-11-27T15:17:36.892579+0000","last_fullsized":"2023-11-27T15:17:36.892579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:48:31.534178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.772079+0000","last_change":"2023-11-27T15:16:09.340380+0000","last_active":"2023-11-27T15:17:33.772079+0000","last_peered":"2023-11-27T15:17:33.772079+0000","last_clean":"2023-11-27T15:17:33.772079+0000","last_became_active":"2023-11-27T15:16:09.339581+0000","last_became_peered":"2023-11-27T15:16:09.339581+0000","last_unstale":"2023-11-27T15:17:33.772079+0000","last_undegraded":"2023-11-27T15:17:33.772079+0000","last_fullsized":"2023-11-27T15:17:33.772079+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:21:44.281165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.807944+0000","last_change":"2023-11-27T15:16:08.917924+0000","last_active":"2023-11-27T15:17:04.807944+0000","last_peered":"2023-11-27T15:17:04.807944+0000","last_clean":"2023-11-27T15:17:04.807944+0000","last_became_active":"2023-11-27T15:16:08.917618+0000","last_became_peered":"2023-11-27T15:16:08.917618+0000","last_unstale":"2023-11-27T15:17:04.807944+0000","last_undegraded":"2023-11-27T15:17:04.807944+0000","last_fullsized":"2023-11-27T15:17:04.807944+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:26:02.008425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.767487+0000","last_change":"2023-11-27T15:16:09.346020+0000","last_active":"2023-11-27T15:17:33.767487+0000","last_peered":"2023-11-27T15:17:33.767487+0000","last_clean":"2023-11-27T15:17:33.767487+0000","last_became_active":"2023-11-27T15:16:09.345822+0000","last_became_peered":"2023-11-27T15:16:09.345822+0000","last_unstale":"2023-11-27T15:17:33.767487+0000","last_undegraded":"2023-11-27T15:17:33.767487+0000","last_fullsized":"2023-11-27T15:17:33.767487+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:09:13.031060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.803585+0000","last_change":"2023-11-27T15:16:08.921810+0000","last_active":"2023-11-27T15:17:04.803585+0000","last_peered":"2023-11-27T15:17:04.803585+0000","last_clean":"2023-11-27T15:17:04.803585+0000","last_became_active":"2023-11-27T15:16:08.921518+0000","last_became_peered":"2023-11-27T15:16:08.921518+0000","last_unstale":"2023-11-27T15:17:04.803585+0000","last_undegraded":"2023-11-27T15:17:04.803585+0000","last_fullsized":"2023-11-27T15:17:04.803585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:57:19.901063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.772314+0000","last_change":"2023-11-27T15:16:09.351083+0000","last_active":"2023-11-27T15:17:33.772314+0000","last_peered":"2023-11-27T15:17:33.772314+0000","last_clean":"2023-11-27T15:17:33.772314+0000","last_became_active":"2023-11-27T15:16:09.350540+0000","last_became_peered":"2023-11-27T15:16:09.350540+0000","last_unstale":"2023-11-27T15:17:33.772314+0000","last_undegraded":"2023-11-27T15:17:33.772314+0000","last_fullsized":"2023-11-27T15:17:33.772314+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:08:16.021485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'31","reported_seq":226,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:37.894182+0000","last_change":"2023-11-27T15:16:08.913023+0000","last_active":"2023-11-27T15:17:37.894182+0000","last_peered":"2023-11-27T15:17:37.894182+0000","last_clean":"2023-11-27T15:17:37.894182+0000","last_became_active":"2023-11-27T15:16:08.912693+0000","last_became_peered":"2023-11-27T15:16:08.912693+0000","last_unstale":"2023-11-27T15:17:37.894182+0000","last_undegraded":"2023-11-27T15:17:37.894182+0000","last_fullsized":"2023-11-27T15:17:37.894182+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:33:54.579228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58776,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":177,"num_read_kb":145,"num_write":32,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.775109+0000","last_change":"2023-11-27T15:16:09.343446+0000","last_active":"2023-11-27T15:17:33.775109+0000","last_peered":"2023-11-27T15:17:33.775109+0000","last_clean":"2023-11-27T15:17:33.775109+0000","last_became_active":"2023-11-27T15:16:09.341980+0000","last_became_peered":"2023-11-27T15:16:09.341980+0000","last_unstale":"2023-11-27T15:17:33.775109+0000","last_undegraded":"2023-11-27T15:17:33.775109+0000","last_fullsized":"2023-11-27T15:17:33.775109+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:48:41.501519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.822854+0000","last_change":"2023-11-27T15:17:04.822416+0000","last_active":"2023-11-27T15:17:04.822854+0000","last_peered":"2023-11-27T15:17:04.822854+0000","last_clean":"2023-11-27T15:17:04.822854+0000","last_became_active":"2023-11-27T15:17:04.810377+0000","last_became_peered":"2023-11-27T15:17:04.810377+0000","last_unstale":"2023-11-27T15:17:04.822854+0000","last_undegraded":"2023-11-27T15:17:04.822854+0000","last_fullsized":"2023-11-27T15:17:04.822854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:03:53.866628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.775297+0000","last_change":"2023-11-27T15:17:04.809248+0000","last_active":"2023-11-27T15:17:33.775297+0000","last_peered":"2023-11-27T15:17:33.775297+0000","last_clean":"2023-11-27T15:17:33.775297+0000","last_became_active":"2023-11-27T15:17:04.808813+0000","last_became_peered":"2023-11-27T15:17:04.808813+0000","last_unstale":"2023-11-27T15:17:33.775297+0000","last_undegraded":"2023-11-27T15:17:33.775297+0000","last_fullsized":"2023-11-27T15:17:33.775297+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:58:27.173973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.803065+0000","last_change":"2023-11-27T15:16:08.915597+0000","last_active":"2023-11-27T15:17:04.803065+0000","last_peered":"2023-11-27T15:17:04.803065+0000","last_clean":"2023-11-27T15:17:04.803065+0000","last_became_active":"2023-11-27T15:16:08.914727+0000","last_became_peered":"2023-11-27T15:16:08.914727+0000","last_unstale":"2023-11-27T15:17:04.803065+0000","last_undegraded":"2023-11-27T15:17:04.803065+0000","last_fullsized":"2023-11-27T15:17:04.803065+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:24:30.347721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.777426+0000","last_change":"2023-11-27T15:16:09.352643+0000","last_active":"2023-11-27T15:17:33.777426+0000","last_peered":"2023-11-27T15:17:33.777426+0000","last_clean":"2023-11-27T15:17:33.777426+0000","last_became_active":"2023-11-27T15:16:09.352336+0000","last_became_peered":"2023-11-27T15:16:09.352336+0000","last_unstale":"2023-11-27T15:17:33.777426+0000","last_undegraded":"2023-11-27T15:17:33.777426+0000","last_fullsized":"2023-11-27T15:17:33.777426+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:26:11.095933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'6","reported_seq":116,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:39.898232+0000","last_change":"2023-11-27T15:16:08.914303+0000","last_active":"2023-11-27T15:17:39.898232+0000","last_peered":"2023-11-27T15:17:39.898232+0000","last_clean":"2023-11-27T15:17:39.898232+0000","last_became_active":"2023-11-27T15:16:08.913601+0000","last_became_peered":"2023-11-27T15:16:08.913601+0000","last_unstale":"2023-11-27T15:17:39.898232+0000","last_undegraded":"2023-11-27T15:17:39.898232+0000","last_fullsized":"2023-11-27T15:17:39.898232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:32:26.143895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1389,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.775199+0000","last_change":"2023-11-27T15:16:09.341342+0000","last_active":"2023-11-27T15:17:33.775199+0000","last_peered":"2023-11-27T15:17:33.775199+0000","last_clean":"2023-11-27T15:17:33.775199+0000","last_became_active":"2023-11-27T15:16:09.341100+0000","last_became_peered":"2023-11-27T15:16:09.341100+0000","last_unstale":"2023-11-27T15:17:33.775199+0000","last_undegraded":"2023-11-27T15:17:33.775199+0000","last_fullsized":"2023-11-27T15:17:33.775199+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:11:05.729312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.808459+0000","last_change":"2023-11-27T15:16:08.921771+0000","last_active":"2023-11-27T15:17:04.808459+0000","last_peered":"2023-11-27T15:17:04.808459+0000","last_clean":"2023-11-27T15:17:04.808459+0000","last_became_active":"2023-11-27T15:16:08.921396+0000","last_became_peered":"2023-11-27T15:16:08.921396+0000","last_unstale":"2023-11-27T15:17:04.808459+0000","last_undegraded":"2023-11-27T15:17:04.808459+0000","last_fullsized":"2023-11-27T15:17:04.808459+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:05:46.765380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.771089+0000","last_change":"2023-11-27T15:16:09.341258+0000","last_active":"2023-11-27T15:17:33.771089+0000","last_peered":"2023-11-27T15:17:33.771089+0000","last_clean":"2023-11-27T15:17:33.771089+0000","last_became_active":"2023-11-27T15:16:09.340921+0000","last_became_peered":"2023-11-27T15:16:09.340921+0000","last_unstale":"2023-11-27T15:17:33.771089+0000","last_undegraded":"2023-11-27T15:17:33.771089+0000","last_fullsized":"2023-11-27T15:17:33.771089+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:12:25.330319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.809915+0000","last_change":"2023-11-27T15:16:08.911219+0000","last_active":"2023-11-27T15:17:04.809915+0000","last_peered":"2023-11-27T15:17:04.809915+0000","last_clean":"2023-11-27T15:17:04.809915+0000","last_became_active":"2023-11-27T15:16:08.910788+0000","last_became_peered":"2023-11-27T15:16:08.910788+0000","last_unstale":"2023-11-27T15:17:04.809915+0000","last_undegraded":"2023-11-27T15:17:04.809915+0000","last_fullsized":"2023-11-27T15:17:04.809915+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:39:39.465171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.765541+0000","last_change":"2023-11-27T15:16:09.343793+0000","last_active":"2023-11-27T15:17:33.765541+0000","last_peered":"2023-11-27T15:17:33.765541+0000","last_clean":"2023-11-27T15:17:33.765541+0000","last_became_active":"2023-11-27T15:16:09.343480+0000","last_became_peered":"2023-11-27T15:16:09.343480+0000","last_unstale":"2023-11-27T15:17:33.765541+0000","last_undegraded":"2023-11-27T15:17:33.765541+0000","last_fullsized":"2023-11-27T15:17:33.765541+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:37:38.699371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.806936+0000","last_change":"2023-11-27T15:16:08.920651+0000","last_active":"2023-11-27T15:17:04.806936+0000","last_peered":"2023-11-27T15:17:04.806936+0000","last_clean":"2023-11-27T15:17:04.806936+0000","last_became_active":"2023-11-27T15:16:08.920294+0000","last_became_peered":"2023-11-27T15:16:08.920294+0000","last_unstale":"2023-11-27T15:17:04.806936+0000","last_undegraded":"2023-11-27T15:17:04.806936+0000","last_fullsized":"2023-11-27T15:17:04.806936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:41:12.965663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.773421+0000","last_change":"2023-11-27T15:16:09.348279+0000","last_active":"2023-11-27T15:17:33.773421+0000","last_peered":"2023-11-27T15:17:33.773421+0000","last_clean":"2023-11-27T15:17:33.773421+0000","last_became_active":"2023-11-27T15:16:09.347760+0000","last_became_peered":"2023-11-27T15:16:09.347760+0000","last_unstale":"2023-11-27T15:17:33.773421+0000","last_undegraded":"2023-11-27T15:17:33.773421+0000","last_fullsized":"2023-11-27T15:17:33.773421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:19:36.809531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.807809+0000","last_change":"2023-11-27T15:16:08.915622+0000","last_active":"2023-11-27T15:17:04.807809+0000","last_peered":"2023-11-27T15:17:04.807809+0000","last_clean":"2023-11-27T15:17:04.807809+0000","last_became_active":"2023-11-27T15:16:08.915293+0000","last_became_peered":"2023-11-27T15:16:08.915293+0000","last_unstale":"2023-11-27T15:17:04.807809+0000","last_undegraded":"2023-11-27T15:17:04.807809+0000","last_fullsized":"2023-11-27T15:17:04.807809+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:31:34.850798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.769972+0000","last_change":"2023-11-27T15:16:09.337096+0000","last_active":"2023-11-27T15:17:33.769972+0000","last_peered":"2023-11-27T15:17:33.769972+0000","last_clean":"2023-11-27T15:17:33.769972+0000","last_became_active":"2023-11-27T15:16:09.336359+0000","last_became_peered":"2023-11-27T15:16:09.336359+0000","last_unstale":"2023-11-27T15:17:33.769972+0000","last_undegraded":"2023-11-27T15:17:33.769972+0000","last_fullsized":"2023-11-27T15:17:33.769972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:23:45.942895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.774368+0000","last_change":"2023-11-27T15:16:09.340478+0000","last_active":"2023-11-27T15:17:33.774368+0000","last_peered":"2023-11-27T15:17:33.774368+0000","last_clean":"2023-11-27T15:17:33.774368+0000","last_became_active":"2023-11-27T15:16:09.340214+0000","last_became_peered":"2023-11-27T15:16:09.340214+0000","last_unstale":"2023-11-27T15:17:33.774368+0000","last_undegraded":"2023-11-27T15:17:33.774368+0000","last_fullsized":"2023-11-27T15:17:33.774368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:51:37.081922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":108,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:37.893754+0000","last_change":"2023-11-27T15:16:08.920387+0000","last_active":"2023-11-27T15:17:37.893754+0000","last_peered":"2023-11-27T15:17:37.893754+0000","last_clean":"2023-11-27T15:17:37.893754+0000","last_became_active":"2023-11-27T15:16:08.920004+0000","last_became_peered":"2023-11-27T15:16:08.920004+0000","last_unstale":"2023-11-27T15:17:37.893754+0000","last_undegraded":"2023-11-27T15:17:37.893754+0000","last_fullsized":"2023-11-27T15:17:37.893754+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:31:37.559127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":85,"num_read_kb":85,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.770288+0000","last_change":"2023-11-27T15:17:04.808609+0000","last_active":"2023-11-27T15:17:33.770288+0000","last_peered":"2023-11-27T15:17:33.770288+0000","last_clean":"2023-11-27T15:17:33.770288+0000","last_became_active":"2023-11-27T15:17:04.808268+0000","last_became_peered":"2023-11-27T15:17:04.808268+0000","last_unstale":"2023-11-27T15:17:33.770288+0000","last_undegraded":"2023-11-27T15:17:33.770288+0000","last_fullsized":"2023-11-27T15:17:33.770288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:31:40.207340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.845316+0000","last_change":"2023-11-27T15:16:08.909694+0000","last_active":"2023-11-27T15:17:04.845316+0000","last_peered":"2023-11-27T15:17:04.845316+0000","last_clean":"2023-11-27T15:17:04.845316+0000","last_became_active":"2023-11-27T15:16:08.909324+0000","last_became_peered":"2023-11-27T15:16:08.909324+0000","last_unstale":"2023-11-27T15:17:04.845316+0000","last_undegraded":"2023-11-27T15:17:04.845316+0000","last_fullsized":"2023-11-27T15:17:04.845316+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:50:22.630279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":344,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.807076+0000","last_change":"2023-11-27T15:15:10.678805+0000","last_active":"2023-11-27T15:17:04.807076+0000","last_peered":"2023-11-27T15:17:04.807076+0000","last_clean":"2023-11-27T15:17:04.807076+0000","last_became_active":"2023-11-27T15:15:10.678267+0000","last_became_peered":"2023-11-27T15:15:10.678267+0000","last_unstale":"2023-11-27T15:17:04.807076+0000","last_undegraded":"2023-11-27T15:17:04.807076+0000","last_fullsized":"2023-11-27T15:17:04.807076+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:12:14.179315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:12:14.179315+0000","last_clean_scrub_stamp":"2023-11-27T15:12:14.179315+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:46:50.658179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.770453+0000","last_change":"2023-11-27T15:16:09.346622+0000","last_active":"2023-11-27T15:17:33.770453+0000","last_peered":"2023-11-27T15:17:33.770453+0000","last_clean":"2023-11-27T15:17:33.770453+0000","last_became_active":"2023-11-27T15:16:09.346287+0000","last_became_peered":"2023-11-27T15:16:09.346287+0000","last_unstale":"2023-11-27T15:17:33.770453+0000","last_undegraded":"2023-11-27T15:17:33.770453+0000","last_fullsized":"2023-11-27T15:17:33.770453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:45:31.217367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.814144+0000","last_change":"2023-11-27T15:17:04.814144+0000","last_active":"2023-11-27T15:17:04.814144+0000","last_peered":"2023-11-27T15:17:04.814144+0000","last_clean":"2023-11-27T15:17:04.814144+0000","last_became_active":"2023-11-27T15:17:04.810509+0000","last_became_peered":"2023-11-27T15:17:04.810509+0000","last_unstale":"2023-11-27T15:17:04.814144+0000","last_undegraded":"2023-11-27T15:17:04.814144+0000","last_fullsized":"2023-11-27T15:17:04.814144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:54:31.956866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.815900+0000","last_change":"2023-11-27T15:17:04.815501+0000","last_active":"2023-11-27T15:17:04.815900+0000","last_peered":"2023-11-27T15:17:04.815900+0000","last_clean":"2023-11-27T15:17:04.815900+0000","last_became_active":"2023-11-27T15:17:04.810004+0000","last_became_peered":"2023-11-27T15:17:04.810004+0000","last_unstale":"2023-11-27T15:17:04.815900+0000","last_undegraded":"2023-11-27T15:17:04.815900+0000","last_fullsized":"2023-11-27T15:17:04.815900+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:15:11.028146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.767894+0000","last_change":"2023-11-27T15:16:09.345119+0000","last_active":"2023-11-27T15:17:33.767894+0000","last_peered":"2023-11-27T15:17:33.767894+0000","last_clean":"2023-11-27T15:17:33.767894+0000","last_became_active":"2023-11-27T15:16:09.344807+0000","last_became_peered":"2023-11-27T15:16:09.344807+0000","last_unstale":"2023-11-27T15:17:33.767894+0000","last_undegraded":"2023-11-27T15:17:33.767894+0000","last_fullsized":"2023-11-27T15:17:33.767894+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:16:43.558443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'7","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:34.645800+0000","last_change":"2023-11-27T15:16:08.911564+0000","last_active":"2023-11-27T15:17:34.645800+0000","last_peered":"2023-11-27T15:17:34.645800+0000","last_clean":"2023-11-27T15:17:34.645800+0000","last_became_active":"2023-11-27T15:16:08.911057+0000","last_became_peered":"2023-11-27T15:16:08.911057+0000","last_unstale":"2023-11-27T15:17:34.645800+0000","last_undegraded":"2023-11-27T15:17:34.645800+0000","last_fullsized":"2023-11-27T15:17:34.645800+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:45:19.804625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.777021+0000","last_change":"2023-11-27T15:16:09.339085+0000","last_active":"2023-11-27T15:17:33.777021+0000","last_peered":"2023-11-27T15:17:33.777021+0000","last_clean":"2023-11-27T15:17:33.777021+0000","last_became_active":"2023-11-27T15:16:09.338777+0000","last_became_peered":"2023-11-27T15:16:09.338777+0000","last_unstale":"2023-11-27T15:17:33.777021+0000","last_undegraded":"2023-11-27T15:17:33.777021+0000","last_fullsized":"2023-11-27T15:17:33.777021+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:08:45.733215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.845177+0000","last_change":"2023-11-27T15:16:08.913939+0000","last_active":"2023-11-27T15:17:04.845177+0000","last_peered":"2023-11-27T15:17:04.845177+0000","last_clean":"2023-11-27T15:17:04.845177+0000","last_became_active":"2023-11-27T15:16:08.913629+0000","last_became_peered":"2023-11-27T15:16:08.913629+0000","last_unstale":"2023-11-27T15:17:04.845177+0000","last_undegraded":"2023-11-27T15:17:04.845177+0000","last_fullsized":"2023-11-27T15:17:04.845177+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:52:20.803089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.766880+0000","last_change":"2023-11-27T15:16:09.350712+0000","last_active":"2023-11-27T15:17:33.766880+0000","last_peered":"2023-11-27T15:17:33.766880+0000","last_clean":"2023-11-27T15:17:33.766880+0000","last_became_active":"2023-11-27T15:16:09.350359+0000","last_became_peered":"2023-11-27T15:16:09.350359+0000","last_unstale":"2023-11-27T15:17:33.766880+0000","last_undegraded":"2023-11-27T15:17:33.766880+0000","last_fullsized":"2023-11-27T15:17:33.766880+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:17:11.861223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.805284+0000","last_change":"2023-11-27T15:16:08.919501+0000","last_active":"2023-11-27T15:17:04.805284+0000","last_peered":"2023-11-27T15:17:04.805284+0000","last_clean":"2023-11-27T15:17:04.805284+0000","last_became_active":"2023-11-27T15:16:08.919258+0000","last_became_peered":"2023-11-27T15:16:08.919258+0000","last_unstale":"2023-11-27T15:17:04.805284+0000","last_undegraded":"2023-11-27T15:17:04.805284+0000","last_fullsized":"2023-11-27T15:17:04.805284+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:04:56.583314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.734625+0000","last_change":"2023-11-27T15:16:09.344066+0000","last_active":"2023-11-27T15:17:33.734625+0000","last_peered":"2023-11-27T15:17:33.734625+0000","last_clean":"2023-11-27T15:17:33.734625+0000","last_became_active":"2023-11-27T15:16:09.343757+0000","last_became_peered":"2023-11-27T15:16:09.343757+0000","last_unstale":"2023-11-27T15:17:33.734625+0000","last_undegraded":"2023-11-27T15:17:33.734625+0000","last_fullsized":"2023-11-27T15:17:33.734625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:13:29.747291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.808214+0000","last_change":"2023-11-27T15:16:08.906205+0000","last_active":"2023-11-27T15:17:04.808214+0000","last_peered":"2023-11-27T15:17:04.808214+0000","last_clean":"2023-11-27T15:17:04.808214+0000","last_became_active":"2023-11-27T15:16:08.905885+0000","last_became_peered":"2023-11-27T15:16:08.905885+0000","last_unstale":"2023-11-27T15:17:04.808214+0000","last_undegraded":"2023-11-27T15:17:04.808214+0000","last_fullsized":"2023-11-27T15:17:04.808214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:10:14.001695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.761768+0000","last_change":"2023-11-27T15:16:09.349642+0000","last_active":"2023-11-27T15:17:33.761768+0000","last_peered":"2023-11-27T15:17:33.761768+0000","last_clean":"2023-11-27T15:17:33.761768+0000","last_became_active":"2023-11-27T15:16:09.349086+0000","last_became_peered":"2023-11-27T15:16:09.349086+0000","last_unstale":"2023-11-27T15:17:33.761768+0000","last_undegraded":"2023-11-27T15:17:33.761768+0000","last_fullsized":"2023-11-27T15:17:33.761768+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:36:33.604216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.806096+0000","last_change":"2023-11-27T15:16:08.917835+0000","last_active":"2023-11-27T15:17:04.806096+0000","last_peered":"2023-11-27T15:17:04.806096+0000","last_clean":"2023-11-27T15:17:04.806096+0000","last_became_active":"2023-11-27T15:16:08.917532+0000","last_became_peered":"2023-11-27T15:16:08.917532+0000","last_unstale":"2023-11-27T15:17:04.806096+0000","last_undegraded":"2023-11-27T15:17:04.806096+0000","last_fullsized":"2023-11-27T15:17:04.806096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:29:51.697143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.774919+0000","last_change":"2023-11-27T15:16:09.338459+0000","last_active":"2023-11-27T15:17:33.774919+0000","last_peered":"2023-11-27T15:17:33.774919+0000","last_clean":"2023-11-27T15:17:33.774919+0000","last_became_active":"2023-11-27T15:16:09.338148+0000","last_became_peered":"2023-11-27T15:16:09.338148+0000","last_unstale":"2023-11-27T15:17:33.774919+0000","last_undegraded":"2023-11-27T15:17:33.774919+0000","last_fullsized":"2023-11-27T15:17:33.774919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:19:16.710021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.809804+0000","last_change":"2023-11-27T15:16:08.915401+0000","last_active":"2023-11-27T15:17:04.809804+0000","last_peered":"2023-11-27T15:17:04.809804+0000","last_clean":"2023-11-27T15:17:04.809804+0000","last_became_active":"2023-11-27T15:16:08.915012+0000","last_became_peered":"2023-11-27T15:16:08.915012+0000","last_unstale":"2023-11-27T15:17:04.809804+0000","last_undegraded":"2023-11-27T15:17:04.809804+0000","last_fullsized":"2023-11-27T15:17:04.809804+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:03:26.905621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.765523+0000","last_change":"2023-11-27T15:16:09.352309+0000","last_active":"2023-11-27T15:17:33.765523+0000","last_peered":"2023-11-27T15:17:33.765523+0000","last_clean":"2023-11-27T15:17:33.765523+0000","last_became_active":"2023-11-27T15:16:09.352039+0000","last_became_peered":"2023-11-27T15:16:09.352039+0000","last_unstale":"2023-11-27T15:17:33.765523+0000","last_undegraded":"2023-11-27T15:17:33.765523+0000","last_fullsized":"2023-11-27T15:17:33.765523+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:20:09.773389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.805829+0000","last_change":"2023-11-27T15:16:08.920446+0000","last_active":"2023-11-27T15:17:04.805829+0000","last_peered":"2023-11-27T15:17:04.805829+0000","last_clean":"2023-11-27T15:17:04.805829+0000","last_became_active":"2023-11-27T15:16:08.920232+0000","last_became_peered":"2023-11-27T15:16:08.920232+0000","last_unstale":"2023-11-27T15:17:04.805829+0000","last_undegraded":"2023-11-27T15:17:04.805829+0000","last_fullsized":"2023-11-27T15:17:04.805829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:08:07.892335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.775022+0000","last_change":"2023-11-27T15:16:09.344328+0000","last_active":"2023-11-27T15:17:33.775022+0000","last_peered":"2023-11-27T15:17:33.775022+0000","last_clean":"2023-11-27T15:17:33.775022+0000","last_became_active":"2023-11-27T15:16:09.344047+0000","last_became_peered":"2023-11-27T15:16:09.344047+0000","last_unstale":"2023-11-27T15:17:33.775022+0000","last_undegraded":"2023-11-27T15:17:33.775022+0000","last_fullsized":"2023-11-27T15:17:33.775022+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:00:56.015107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.803420+0000","last_change":"2023-11-27T15:16:08.909865+0000","last_active":"2023-11-27T15:17:04.803420+0000","last_peered":"2023-11-27T15:17:04.803420+0000","last_clean":"2023-11-27T15:17:04.803420+0000","last_became_active":"2023-11-27T15:16:08.909559+0000","last_became_peered":"2023-11-27T15:16:08.909559+0000","last_unstale":"2023-11-27T15:17:04.803420+0000","last_undegraded":"2023-11-27T15:17:04.803420+0000","last_fullsized":"2023-11-27T15:17:04.803420+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:59:09.511504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.775967+0000","last_change":"2023-11-27T15:16:09.342525+0000","last_active":"2023-11-27T15:17:33.775967+0000","last_peered":"2023-11-27T15:17:33.775967+0000","last_clean":"2023-11-27T15:17:33.775967+0000","last_became_active":"2023-11-27T15:16:09.340987+0000","last_became_peered":"2023-11-27T15:16:09.340987+0000","last_unstale":"2023-11-27T15:17:33.775967+0000","last_undegraded":"2023-11-27T15:17:33.775967+0000","last_fullsized":"2023-11-27T15:17:33.775967+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:37:38.155819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:04.804148+0000","last_change":"2023-11-27T15:16:08.914088+0000","last_active":"2023-11-27T15:17:04.804148+0000","last_peered":"2023-11-27T15:17:04.804148+0000","last_clean":"2023-11-27T15:17:04.804148+0000","last_became_active":"2023-11-27T15:16:08.913750+0000","last_became_peered":"2023-11-27T15:16:08.913750+0000","last_unstale":"2023-11-27T15:17:04.804148+0000","last_undegraded":"2023-11-27T15:17:04.804148+0000","last_fullsized":"2023-11-27T15:17:04.804148+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:07.892386+0000","last_clean_scrub_stamp":"2023-11-27T15:16:07.892386+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:57:55.114032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.774914+0000","last_change":"2023-11-27T15:16:09.344403+0000","last_active":"2023-11-27T15:17:33.774914+0000","last_peered":"2023-11-27T15:17:33.774914+0000","last_clean":"2023-11-27T15:17:33.774914+0000","last_became_active":"2023-11-27T15:16:09.344120+0000","last_became_peered":"2023-11-27T15:16:09.344120+0000","last_unstale":"2023-11-27T15:17:33.774914+0000","last_undegraded":"2023-11-27T15:17:33.774914+0000","last_fullsized":"2023-11-27T15:17:33.774914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:47:27.200812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.762743+0000","last_change":"2023-11-27T15:16:09.349780+0000","last_active":"2023-11-27T15:17:33.762743+0000","last_peered":"2023-11-27T15:17:33.762743+0000","last_clean":"2023-11-27T15:17:33.762743+0000","last_became_active":"2023-11-27T15:16:09.349493+0000","last_became_peered":"2023-11-27T15:16:09.349493+0000","last_unstale":"2023-11-27T15:17:33.762743+0000","last_undegraded":"2023-11-27T15:17:33.762743+0000","last_fullsized":"2023-11-27T15:17:33.762743+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:14:43.605196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.773008+0000","last_change":"2023-11-27T15:17:04.809959+0000","last_active":"2023-11-27T15:17:33.773008+0000","last_peered":"2023-11-27T15:17:33.773008+0000","last_clean":"2023-11-27T15:17:33.773008+0000","last_became_active":"2023-11-27T15:17:04.809529+0000","last_became_peered":"2023-11-27T15:17:04.809529+0000","last_unstale":"2023-11-27T15:17:33.773008+0000","last_undegraded":"2023-11-27T15:17:33.773008+0000","last_fullsized":"2023-11-27T15:17:33.773008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:19:17.711075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.770374+0000","last_change":"2023-11-27T15:16:09.337989+0000","last_active":"2023-11-27T15:17:33.770374+0000","last_peered":"2023-11-27T15:17:33.770374+0000","last_clean":"2023-11-27T15:17:33.770374+0000","last_became_active":"2023-11-27T15:16:09.336712+0000","last_became_peered":"2023-11-27T15:16:09.336712+0000","last_unstale":"2023-11-27T15:17:33.770374+0000","last_undegraded":"2023-11-27T15:17:33.770374+0000","last_fullsized":"2023-11-27T15:17:33.770374+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:25:41.941448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.768395+0000","last_change":"2023-11-27T15:17:04.812727+0000","last_active":"2023-11-27T15:17:33.768395+0000","last_peered":"2023-11-27T15:17:33.768395+0000","last_clean":"2023-11-27T15:17:33.768395+0000","last_became_active":"2023-11-27T15:17:04.811852+0000","last_became_peered":"2023-11-27T15:17:04.811852+0000","last_unstale":"2023-11-27T15:17:33.768395+0000","last_undegraded":"2023-11-27T15:17:33.768395+0000","last_fullsized":"2023-11-27T15:17:33.768395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:03:44.659440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.771893+0000","last_change":"2023-11-27T15:16:09.344317+0000","last_active":"2023-11-27T15:17:33.771893+0000","last_peered":"2023-11-27T15:17:33.771893+0000","last_clean":"2023-11-27T15:17:33.771893+0000","last_became_active":"2023-11-27T15:16:09.343962+0000","last_became_peered":"2023-11-27T15:16:09.343962+0000","last_unstale":"2023-11-27T15:17:33.771893+0000","last_undegraded":"2023-11-27T15:17:33.771893+0000","last_fullsized":"2023-11-27T15:17:33.771893+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:15:39.862196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.766088+0000","last_change":"2023-11-27T15:16:09.344690+0000","last_active":"2023-11-27T15:17:33.766088+0000","last_peered":"2023-11-27T15:17:33.766088+0000","last_clean":"2023-11-27T15:17:33.766088+0000","last_became_active":"2023-11-27T15:16:09.344259+0000","last_became_peered":"2023-11-27T15:16:09.344259+0000","last_unstale":"2023-11-27T15:17:33.766088+0000","last_undegraded":"2023-11-27T15:17:33.766088+0000","last_fullsized":"2023-11-27T15:17:33.766088+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:48:11.448071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.773804+0000","last_change":"2023-11-27T15:16:09.340459+0000","last_active":"2023-11-27T15:17:33.773804+0000","last_peered":"2023-11-27T15:17:33.773804+0000","last_clean":"2023-11-27T15:17:33.773804+0000","last_became_active":"2023-11-27T15:16:09.340140+0000","last_became_peered":"2023-11-27T15:16:09.340140+0000","last_unstale":"2023-11-27T15:17:33.773804+0000","last_undegraded":"2023-11-27T15:17:33.773804+0000","last_fullsized":"2023-11-27T15:17:33.773804+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:50:28.936856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.770718+0000","last_change":"2023-11-27T15:16:09.336625+0000","last_active":"2023-11-27T15:17:33.770718+0000","last_peered":"2023-11-27T15:17:33.770718+0000","last_clean":"2023-11-27T15:17:33.770718+0000","last_became_active":"2023-11-27T15:16:09.336287+0000","last_became_peered":"2023-11-27T15:16:09.336287+0000","last_unstale":"2023-11-27T15:17:33.770718+0000","last_undegraded":"2023-11-27T15:17:33.770718+0000","last_fullsized":"2023-11-27T15:17:33.770718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:09:58.745009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.772059+0000","last_change":"2023-11-27T15:16:09.335480+0000","last_active":"2023-11-27T15:17:33.772059+0000","last_peered":"2023-11-27T15:17:33.772059+0000","last_clean":"2023-11-27T15:17:33.772059+0000","last_became_active":"2023-11-27T15:16:09.335163+0000","last_became_peered":"2023-11-27T15:16:09.335163+0000","last_unstale":"2023-11-27T15:17:33.772059+0000","last_undegraded":"2023-11-27T15:17:33.772059+0000","last_fullsized":"2023-11-27T15:17:33.772059+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:57:09.686634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.775133+0000","last_change":"2023-11-27T15:16:09.336879+0000","last_active":"2023-11-27T15:17:33.775133+0000","last_peered":"2023-11-27T15:17:33.775133+0000","last_clean":"2023-11-27T15:17:33.775133+0000","last_became_active":"2023-11-27T15:16:09.336580+0000","last_became_peered":"2023-11-27T15:16:09.336580+0000","last_unstale":"2023-11-27T15:17:33.775133+0000","last_undegraded":"2023-11-27T15:17:33.775133+0000","last_fullsized":"2023-11-27T15:17:33.775133+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:18:10.805656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.773451+0000","last_change":"2023-11-27T15:16:09.341400+0000","last_active":"2023-11-27T15:17:33.773451+0000","last_peered":"2023-11-27T15:17:33.773451+0000","last_clean":"2023-11-27T15:17:33.773451+0000","last_became_active":"2023-11-27T15:16:09.341103+0000","last_became_peered":"2023-11-27T15:16:09.341103+0000","last_unstale":"2023-11-27T15:17:33.773451+0000","last_undegraded":"2023-11-27T15:17:33.773451+0000","last_fullsized":"2023-11-27T15:17:33.773451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:51:15.698428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.776500+0000","last_change":"2023-11-27T15:16:09.345938+0000","last_active":"2023-11-27T15:17:33.776500+0000","last_peered":"2023-11-27T15:17:33.776500+0000","last_clean":"2023-11-27T15:17:33.776500+0000","last_became_active":"2023-11-27T15:16:09.345624+0000","last_became_peered":"2023-11-27T15:16:09.345624+0000","last_unstale":"2023-11-27T15:17:33.776500+0000","last_undegraded":"2023-11-27T15:17:33.776500+0000","last_fullsized":"2023-11-27T15:17:33.776500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:29:31.241865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-27T15:17:33.765022+0000","last_change":"2023-11-27T15:16:09.343896+0000","last_active":"2023-11-27T15:17:33.765022+0000","last_peered":"2023-11-27T15:17:33.765022+0000","last_clean":"2023-11-27T15:17:33.765022+0000","last_became_active":"2023-11-27T15:16:09.343603+0000","last_became_peered":"2023-11-27T15:16:09.343603+0000","last_unstale":"2023-11-27T15:17:33.765022+0000","last_undegraded":"2023-11-27T15:17:33.765022+0000","last_fullsized":"2023-11-27T15:17:33.765022+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:08.894095+0000","last_clean_scrub_stamp":"2023-11-27T15:16:08.894095+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:20:29.881258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":249380,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63670,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":533,"num_read_kb":336,"num_write":163,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1020,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":196389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449118,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6096,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739056,"statfs":{"total":95995035648,"available":95988793344,"internally_reserved":0,"allocated":925696,"data_stored":444516,"data_compressed":5787,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54000000000000004,"15min":0.53600000000000003},"min":{"1min":0.39100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51300000000000001,"15min":0.51600000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.68000000000000005,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.48199999999999998}]},{"osd":1,"last update":"Mon Nov 27 15:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.63100000000000001,"15min":0.625},"min":{"1min":0.51900000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.80500000000000005,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81399999999999995,"5min":1.262,"15min":1.262},"last":0.58799999999999997}]},{"osd":2,"last update":"Mon Nov 27 15:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.46000000000000002,"15min":0.45700000000000002},"min":{"1min":0.32800000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.58099999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.48799999999999999,"15min":0.48699999999999999},"min":{"1min":0.373,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.66400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.504}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":4,"last update":"Mon Nov 27 15:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.64900000000000002,"15min":0.64400000000000002},"min":{"1min":0.57099999999999995,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83299999999999996,"5min":1.155,"15min":1.155},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82199999999999995,"5min":1.048,"15min":1.048},"last":0.69199999999999995}]},{"osd":5,"last update":"Mon Nov 27 15:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52700000000000002,"15min":0.52600000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.63800000000000001,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58999999999999997,"15min":0.58499999999999996},"min":{"1min":0.434,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82999999999999996,"5min":1.02,"15min":1.02},"last":0.755}]},{"osd":6,"last update":"Mon Nov 27 15:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.61899999999999999}]},{"osd":7,"last update":"Mon Nov 27 15:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54200000000000004,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.91900000000000004,"5min":1.232,"15min":1.232},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.71899999999999997,"15min":0.71799999999999997},"min":{"1min":0.51500000000000001,"5min":0.503,"15min":0.503},"max":{"1min":0.85699999999999998,"5min":1.302,"15min":1.302},"last":0.78500000000000003}]},{"osd":8,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.73399999999999999}]},{"osd":9,"last update":"Mon Nov 27 15:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72199999999999998,"15min":0.72499999999999998},"min":{"1min":0.51600000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.97399999999999998,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.75600000000000001,"15min":0.752},"min":{"1min":0.628,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.96199999999999997,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.876}]},{"osd":10,"last update":"Mon Nov 27 15:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.76300000000000001,"15min":0.76100000000000001},"min":{"1min":0.59799999999999998,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.88600000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.628,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.92000000000000004,"5min":1.706,"15min":1.706},"last":0.77100000000000002}]}]},{"osd":10,"up_from":71,"seq":304942678050,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6556,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738596,"statfs":{"total":95995035648,"available":95988322304,"internally_reserved":0,"allocated":1069056,"data_stored":1095206,"data_compressed":41881,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.496},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.433}]},{"osd":1,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.44500000000000001,"15min":0.442},"min":{"1min":0.248,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.318},{"interface":"front","average":{"1min":0.434,"5min":0.45800000000000002,"15min":0.46200000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.58499999999999996,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.36499999999999999}]},{"osd":2,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.61499999999999999,"15min":0.62},"min":{"1min":0.40100000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84199999999999997,"5min":0.995,"15min":0.995},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76600000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.65700000000000003}]},{"osd":3,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.58799999999999997,"15min":0.60299999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72999999999999998,"5min":1.071,"15min":1.071},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63600000000000001,"15min":0.64500000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.80100000000000005,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.38600000000000001}]},{"osd":4,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53100000000000003,"15min":0.53700000000000003},"min":{"1min":0.34499999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.71599999999999997,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.53100000000000003,"15min":0.53800000000000003},"min":{"1min":0.35599999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71199999999999997,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.34499999999999997}]},{"osd":5,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.629,"15min":0.627},"min":{"1min":0.47499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77700000000000002,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.66500000000000004,"15min":0.65900000000000003},"min":{"1min":0.46800000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.61899999999999999}]},{"osd":6,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.71199999999999997,"15min":0.71799999999999997},"min":{"1min":0.45000000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.876,"5min":1.304,"15min":1.304},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.67000000000000004,"15min":0.68200000000000005},"min":{"1min":0.42499999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82399999999999995,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.52500000000000002}]},{"osd":7,"last update":"Mon Nov 27 15:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.53300000000000003}]},{"osd":8,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.70699999999999996,"15min":0.71299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85799999999999998,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.72999999999999998,"15min":0.73399999999999999},"min":{"1min":0.504,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.98899999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.55900000000000005}]},{"osd":9,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.73999999999999999,"15min":0.752},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.85299999999999998,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.76900000000000002,"15min":0.78300000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.85299999999999998,"5min":1.109,"15min":1.109},"last":0.48399999999999999}]},{"osd":11,"last update":"Mon Nov 27 15:17:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71699999999999997,"15min":0.71499999999999997},"min":{"1min":0.63,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89000000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.72199999999999998,"15min":0.71899999999999997},"min":{"1min":0.57999999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.63500000000000001}]}]},{"osd":9,"up_from":66,"seq":283467841575,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6232,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93738920,"statfs":{"total":95995035648,"available":95988654080,"internally_reserved":0,"allocated":933888,"data_stored":444696,"data_compressed":5787,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.437,"15min":0.443},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.65300000000000002,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44500000000000001,"15min":0.44800000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.39400000000000002}]},{"osd":1,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.61099999999999999,"15min":0.61499999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.501},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.628,"15min":0.63800000000000001},"min":{"1min":0.42499999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77900000000000003,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.51200000000000001}]},{"osd":2,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64100000000000001,"15min":0.64200000000000002},"min":{"1min":0.42899999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.441},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.46300000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.81999999999999995,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.54900000000000004}]},{"osd":3,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.52900000000000003,"15min":0.53500000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.59499999999999997,"15min":0.61199999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65500000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.32500000000000001}]},{"osd":4,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.65200000000000002,"15min":0.66600000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79300000000000004,"5min":1.236,"15min":1.236},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.52700000000000002}]},{"osd":5,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.70899999999999996,"15min":0.72699999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.82499999999999996,"5min":1.274,"15min":1.274},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.68500000000000005,"15min":0.68700000000000006},"min":{"1min":0.439,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.47899999999999998}]},{"osd":6,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.69499999999999995,"15min":0.70899999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.71199999999999997,"15min":0.73399999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.77100000000000002,"5min":1.351,"15min":1.351},"last":0.61399999999999999}]},{"osd":7,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.75800000000000001,"15min":0.76200000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.77900000000000003,"15min":0.79500000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.92000000000000004,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.58399999999999996}]},{"osd":8,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79600000000000004,"15min":0.79900000000000004},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.78500000000000003,"15min":0.80300000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.86699999999999999,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.68899999999999995}]},{"osd":10,"last update":"Mon Nov 27 15:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.74399999999999999,"15min":0.753},"min":{"1min":0.51200000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.95399999999999996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.746,"15min":0.75800000000000001},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.88800000000000001,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.70599999999999996}]}]},{"osd":8,"up_from":58,"seq":249108103211,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6420,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738732,"statfs":{"total":95995035648,"available":95988461568,"internally_reserved":0,"allocated":929792,"data_stored":444594,"data_compressed":5787,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.34899999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51500000000000001,"15min":0.5},"min":{"1min":0.40899999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.52400000000000002}]},{"osd":1,"last update":"Mon Nov 27 15:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.57199999999999995,"15min":0.55600000000000005},"min":{"1min":0.48399999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.81100000000000005,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.59099999999999997,"15min":0.56899999999999995},"min":{"1min":0.52300000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.92000000000000004,"5min":1.004,"15min":1.004},"last":0.67400000000000004}]},{"osd":2,"last update":"Mon Nov 27 15:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46200000000000002,"15min":0.44900000000000001},"min":{"1min":0.33100000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.68000000000000005,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.46800000000000003,"15min":0.44800000000000001},"min":{"1min":0.32900000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.38400000000000001}]},{"osd":3,"last update":"Mon Nov 27 15:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55200000000000005,"15min":0.53600000000000003},"min":{"1min":0.41599999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.57899999999999996,"15min":0.56100000000000005},"min":{"1min":0.44,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.56200000000000006}]},{"osd":4,"last update":"Mon Nov 27 15:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.60699999999999998,"15min":0.58699999999999997},"min":{"1min":0.53800000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79800000000000004,"5min":1.014,"15min":1.014},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.64800000000000002,"15min":0.63300000000000001},"min":{"1min":0.54000000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.84999999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.622}]},{"osd":5,"last update":"Mon Nov 27 15:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.40000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77500000000000002,"5min":0.873,"15min":0.873},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56599999999999995,"15min":0.54900000000000004},"min":{"1min":0.376,"5min":0.315,"15min":0.315},"max":{"1min":0.86599999999999999,"5min":1.056,"15min":1.056},"last":0.57999999999999996}]},{"osd":6,"last update":"Mon Nov 27 15:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66500000000000004,"15min":0.65500000000000003},"min":{"1min":0.46100000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.66500000000000004,"15min":0.65000000000000002},"min":{"1min":0.51300000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.59799999999999998}]},{"osd":7,"last update":"Mon Nov 27 15:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.69099999999999995,"15min":0.67500000000000004},"min":{"1min":0.53800000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.70799999999999996,"15min":0.68799999999999994},"min":{"1min":0.63700000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.98599999999999999,"5min":1.115,"15min":1.115},"last":0.72899999999999998}]},{"osd":9,"last update":"Mon Nov 27 15:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.72899999999999998,"15min":0.71699999999999997},"min":{"1min":0.52600000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.74399999999999999,"15min":0.73099999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.75700000000000001}]},{"osd":10,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.68400000000000005,"15min":0.66800000000000004},"min":{"1min":0.58599999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1,"5min":1,"15min":1},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.72399999999999998,"15min":0.70699999999999996},"min":{"1min":0.60599999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.80300000000000005}]},{"osd":11,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.78000000000000003}]}]},{"osd":7,"up_from":50,"seq":214748364847,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6536,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738616,"statfs":{"total":95995035648,"available":95988342784,"internally_reserved":0,"allocated":983040,"data_stored":1037445,"data_compressed":41881,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.35599999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.76300000000000001,"5min":1.052,"15min":1.052},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.51800000000000002,"15min":0.503},"min":{"1min":0.315,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.38700000000000001}]},{"osd":1,"last update":"Mon Nov 27 15:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.40500000000000003,"15min":0.38500000000000001},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.497},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.42799999999999999,"15min":0.40500000000000003},"min":{"1min":0.27900000000000003,"5min":0.254,"15min":0.254},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.59899999999999998}]},{"osd":2,"last update":"Mon Nov 27 15:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58299999999999996,"15min":0.56599999999999995},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56399999999999995,"15min":0.54400000000000004},"min":{"1min":0.38,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77500000000000002,"5min":1.077,"15min":1.077},"last":0.58199999999999996}]},{"osd":3,"last update":"Mon Nov 27 15:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54800000000000004,"15min":0.52700000000000002},"min":{"1min":0.36199999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71399999999999997,"5min":1.026,"15min":1.026},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59299999999999997,"15min":0.57999999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79100000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.628}]},{"osd":4,"last update":"Mon Nov 27 15:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.505,"15min":0.47899999999999998},"min":{"1min":0.35499999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.871,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.498,"15min":0.47699999999999998},"min":{"1min":0.318,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.73799999999999999}]},{"osd":5,"last update":"Mon Nov 27 15:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63500000000000001,"15min":0.60099999999999998},"min":{"1min":0.45600000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.96399999999999997,"5min":2.3719999999999999,"15min":2.3719999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65300000000000002,"15min":0.61599999999999999},"min":{"1min":0.47199999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76000000000000001,"5min":3.1059999999999999,"15min":3.1059999999999999},"last":0.70299999999999996}]},{"osd":6,"last update":"Mon Nov 27 15:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67200000000000004,"15min":0.65600000000000003},"min":{"1min":0.495,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.63,"15min":0.61199999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.77500000000000002}]},{"osd":8,"last update":"Mon Nov 27 15:17:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67800000000000005,"15min":0.65800000000000003},"min":{"1min":0.46500000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.97299999999999998,"5min":1.157,"15min":1.157},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.72799999999999998,"15min":0.69999999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.89100000000000001,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":0.80700000000000005}]},{"osd":9,"last update":"Mon Nov 27 15:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.752,"15min":0.75},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":0.754},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.73999999999999999,"15min":0.72999999999999998},"min":{"1min":0.50900000000000001,"5min":0.497,"15min":0.497},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":0.86399999999999999}]},{"osd":10,"last update":"Mon Nov 27 15:17:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.755,"15min":0.76500000000000001},"min":{"1min":0.52600000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.83099999999999996,"5min":1.214,"15min":1.214},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.77100000000000002,"15min":0.78900000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93400000000000005,"5min":1.095,"15min":1.095},"last":0.79500000000000004}]},{"osd":11,"last update":"Mon Nov 27 15:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.84199999999999997}]}]},{"osd":6,"up_from":45,"seq":193273528372,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6348,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738804,"statfs":{"total":95995035648,"available":95988535296,"internally_reserved":0,"allocated":921600,"data_stored":444091,"data_compressed":5787,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.438,"15min":0.44500000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.86099999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.503,"15min":0.53200000000000003},"min":{"1min":0.29899999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.83699999999999997,"5min":2.508,"15min":2.508},"last":0.36899999999999999}]},{"osd":1,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59599999999999997,"15min":0.58699999999999997},"min":{"1min":0.45000000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.61499999999999999,"15min":0.63800000000000001},"min":{"1min":0.41099999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73799999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":0.48499999999999999}]},{"osd":2,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.65300000000000002,"15min":0.67500000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91000000000000003,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.497},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.64600000000000002,"15min":0.65000000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.58399999999999996}]},{"osd":3,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52900000000000003,"15min":0.53100000000000003},"min":{"1min":0.377,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89600000000000002,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.433,"5min":0.54500000000000004,"15min":0.58699999999999997},"min":{"1min":0.33000000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.879,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":0.39000000000000001}]},{"osd":4,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.64300000000000002,"15min":0.65800000000000003},"min":{"1min":0.42699999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95299999999999996,"5min":1.51,"15min":1.51},"last":0.437},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65500000000000003,"15min":0.64300000000000002},"min":{"1min":0.495,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.012,"5min":1.054,"15min":1.054},"last":0.54800000000000004}]},{"osd":5,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.69299999999999995,"15min":0.73099999999999998},"min":{"1min":0.47399999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77700000000000002,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.70199999999999996,"15min":0.71899999999999997},"min":{"1min":0.51600000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.93300000000000005,"5min":1.599,"15min":1.599},"last":0.53000000000000003}]},{"osd":7,"last update":"Mon Nov 27 15:17:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.71199999999999997,"15min":0.71999999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88700000000000001,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73799999999999999,"15min":0.73499999999999999},"min":{"1min":0.56200000000000006,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.96099999999999997,"5min":1.206,"15min":1.206},"last":0.56899999999999995}]},{"osd":8,"last update":"Mon Nov 27 15:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.81000000000000005,"15min":0.81599999999999995},"min":{"1min":0.58699999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":4.0449999999999999,"5min":4.0449999999999999,"15min":4.0449999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.74299999999999999,"15min":0.79300000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.89200000000000002,"5min":2.766,"15min":2.766},"last":0.66400000000000003}]},{"osd":9,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.71599999999999997,"15min":0.74099999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.77800000000000002,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.75,"15min":0.77000000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90000000000000002,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.59999999999999998}]},{"osd":10,"last update":"Mon Nov 27 15:17:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.68700000000000006,"15min":0.68500000000000005},"min":{"1min":0.51200000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.95499999999999996,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.79300000000000004,"15min":0.79900000000000004},"min":{"1min":0.64100000000000001,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.93700000000000006,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.63300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7436,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737716,"statfs":{"total":95995035648,"available":95987421184,"internally_reserved":0,"allocated":987136,"data_stored":508351,"data_compressed":5787,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[0,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51200000000000001,"15min":0.47299999999999998},"min":{"1min":0.32700000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.52000000000000002,"15min":0.46999999999999997},"min":{"1min":0.33900000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.86799999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.41899999999999998}]},{"osd":2,"last update":"Mon Nov 27 15:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.56699999999999995,"15min":0.54700000000000004},"min":{"1min":0.40200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80900000000000005,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.622},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.55800000000000005,"15min":0.53200000000000003},"min":{"1min":0.41699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.66700000000000004}]},{"osd":3,"last update":"Mon Nov 27 15:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56999999999999995,"15min":0.53400000000000003},"min":{"1min":0.38100000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.96099999999999997,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56399999999999995,"15min":0.55700000000000005},"min":{"1min":0.32200000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.78300000000000003,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.55200000000000005}]},{"osd":4,"last update":"Mon Nov 27 15:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.45100000000000001,"15min":0.43099999999999999},"min":{"1min":0.307,"5min":0.251,"15min":0.251},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.46600000000000003,"15min":0.44400000000000001},"min":{"1min":0.33000000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.63600000000000001,"5min":0.754,"15min":0.754},"last":0.629}]},{"osd":5,"last update":"Mon Nov 27 15:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.59799999999999998,"15min":0.58899999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.63,"15min":0.623},"min":{"1min":0.53100000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.72699999999999998}]},{"osd":6,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.60599999999999998,"15min":0.58299999999999996},"min":{"1min":0.40699999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.63600000000000001,"15min":0.626},"min":{"1min":0.434,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95499999999999996,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.54200000000000004}]},{"osd":7,"last update":"Mon Nov 27 15:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54100000000000004,"15min":0.52800000000000002},"min":{"1min":0.38300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.90900000000000003,"5min":1.02,"15min":1.02},"last":0.499},{"interface":"front","average":{"1min":0.621,"5min":0.59199999999999997,"15min":0.57599999999999996},"min":{"1min":0.33900000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0549999999999999,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.58099999999999996}]},{"osd":8,"last update":"Mon Nov 27 15:17:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.67800000000000005,"15min":0.65400000000000003},"min":{"1min":0.52900000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91800000000000004,"5min":1.177,"15min":1.177},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.65300000000000002,"15min":0.63},"min":{"1min":0.53400000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.875,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.68200000000000005}]},{"osd":9,"last update":"Mon Nov 27 15:16:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72899999999999998,"15min":0.72099999999999997},"min":{"1min":0.57899999999999996,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.72899999999999998,"15min":0.72499999999999998},"min":{"1min":0.53400000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.60399999999999998}]},{"osd":10,"last update":"Mon Nov 27 15:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71699999999999997,"15min":0.71799999999999997},"min":{"1min":0.48299999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0149999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.71899999999999997,"15min":0.71299999999999997},"min":{"1min":0.57299999999999995,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.93799999999999994,"5min":0.995,"15min":0.995},"last":0.69399999999999995}]},{"osd":11,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.751}]}]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7328,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737824,"statfs":{"total":95995035648,"available":95987531776,"internally_reserved":0,"allocated":942080,"data_stored":445356,"data_compressed":5787,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 27 15:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59299999999999997,"15min":0.54900000000000004},"min":{"1min":0.45400000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79000000000000004,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.60999999999999999,"15min":0.56100000000000005},"min":{"1min":0.432,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.70099999999999996,"5min":1.55,"15min":1.55},"last":0.38900000000000001}]},{"osd":2,"last update":"Mon Nov 27 15:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58299999999999996,"15min":0.54500000000000004},"min":{"1min":0.42599999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54800000000000004,"15min":0.52100000000000002},"min":{"1min":0.35999999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.41099999999999998}]},{"osd":3,"last update":"Mon Nov 27 15:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.438},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.372}]},{"osd":4,"last update":"Mon Nov 27 15:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.56999999999999995,"15min":0.54800000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68300000000000005,"5min":0.999,"15min":0.999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.58599999999999997,"15min":0.55800000000000005},"min":{"1min":0.40500000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.67700000000000005,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.497}]},{"osd":5,"last update":"Mon Nov 27 15:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.61499999999999999},"min":{"1min":0.47699999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85499999999999998,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61799999999999999,"15min":0.60799999999999998},"min":{"1min":0.41899999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71899999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.65000000000000002}]},{"osd":6,"last update":"Mon Nov 27 15:16:43 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.308,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.63900000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.754,"5min":1.032,"15min":1.032},"last":0.61799999999999999}]},{"osd":7,"last update":"Mon Nov 27 15:16:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67800000000000005,"15min":0.67200000000000004},"min":{"1min":0.434,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.46100000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.31}]},{"osd":8,"last update":"Mon Nov 27 15:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.66000000000000003,"15min":0.69499999999999995},"min":{"1min":0.441,"5min":0.434,"15min":0.434},"max":{"1min":0.73699999999999999,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.70899999999999996,"15min":0.73899999999999999},"min":{"1min":0.45500000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.78200000000000003,"5min":1.004,"15min":1.004},"last":0.55300000000000005}]},{"osd":9,"last update":"Mon Nov 27 15:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.73499999999999999,"15min":0.76100000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88800000000000001,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.72299999999999998,"15min":0.74299999999999999},"min":{"1min":0.438,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.46899999999999997}]},{"osd":10,"last update":"Mon Nov 27 15:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.74399999999999999,"15min":0.75800000000000001},"min":{"1min":0.53000000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.81399999999999995,"5min":1.145,"15min":1.145},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.77100000000000002,"15min":0.78700000000000003},"min":{"1min":0.503,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.95399999999999996,"5min":1.248,"15min":1.248},"last":0.57499999999999996}]},{"osd":11,"last update":"Mon Nov 27 15:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65000000000000002,"15min":0.64900000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72699999999999998,"15min":0.73199999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.82399999999999995,"5min":0.879,"15min":0.879},"last":0.66300000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7508,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":6528,"kb_avail":93737644,"statfs":{"total":95995035648,"available":95987347456,"internally_reserved":0,"allocated":995328,"data_stored":503348,"data_compressed":5787,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":6684672},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.61199999999999999,"15min":0.51100000000000001},"min":{"1min":0.35199999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":8.5069999999999997,"5min":8.5069999999999997,"15min":8.5069999999999997},"last":0.442},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.50900000000000001,"15min":0.48399999999999999},"min":{"1min":0.36199999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.47699999999999998}]},{"osd":1,"last update":"Mon Nov 27 15:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.56899999999999995,"15min":0.52200000000000002},"min":{"1min":0.45900000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.53900000000000003,"15min":0.498},"min":{"1min":0.48099999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.57999999999999996}]},{"osd":3,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.59899999999999998,"15min":0.54600000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.2170000000000001,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.59599999999999997,"15min":0.54600000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":3.278,"5min":3.278,"15min":3.278},"last":0.50900000000000001}]},{"osd":4,"last update":"Mon Nov 27 15:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.63600000000000001}]},{"osd":5,"last update":"Mon Nov 27 15:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.47099999999999997,"15min":0.41299999999999998},"min":{"1min":0.26600000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.86799999999999999,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.48099999999999998,"15min":0.42699999999999999},"min":{"1min":0.33100000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.61299999999999999}]},{"osd":6,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.58599999999999997,"15min":0.56499999999999995},"min":{"1min":0.44900000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.70399999999999996,"15min":0.64300000000000002},"min":{"1min":0.41599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":4.5800000000000001,"5min":4.5800000000000001,"15min":4.5800000000000001},"last":0.754}]},{"osd":7,"last update":"Mon Nov 27 15:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.39800000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.44},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69199999999999995,"15min":0.69599999999999995},"min":{"1min":0.46300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93300000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.51100000000000001}]},{"osd":8,"last update":"Mon Nov 27 15:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.65900000000000003,"15min":0.66100000000000003},"min":{"1min":0.35099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.94799999999999995,"5min":0.997,"15min":0.997},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.59999999999999998,"15min":0.58199999999999996},"min":{"1min":0.41999999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.66800000000000004}]},{"osd":9,"last update":"Mon Nov 27 15:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.72199999999999998,"15min":0.70199999999999996},"min":{"1min":0.49399999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":3.633,"5min":3.633,"15min":3.633},"last":0.71699999999999997},{"interface":"front","average":{"1min":1.036,"5min":0.79500000000000004,"15min":0.755},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":7.6369999999999996,"5min":7.6369999999999996,"15min":7.6369999999999996},"last":0.67900000000000005}]},{"osd":10,"last update":"Mon Nov 27 15:17:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.67700000000000005,"15min":0.66200000000000003},"min":{"1min":0.59299999999999997,"5min":0.496,"15min":0.496},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.70299999999999996,"15min":0.67900000000000005},"min":{"1min":0.67300000000000004,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.76300000000000001}]},{"osd":11,"last update":"Mon Nov 27 15:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.78300000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182464,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6684,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738468,"statfs":{"total":95995035648,"available":95988191232,"internally_reserved":0,"allocated":937984,"data_stored":444408,"data_compressed":5787,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.36599999999999999,"15min":0.33500000000000002},"min":{"1min":0.254,"5min":0.22,"15min":0.22},"max":{"1min":0.60499999999999998,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.38500000000000001,"15min":0.35699999999999998},"min":{"1min":0.27300000000000002,"5min":0.214,"15min":0.214},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.35899999999999999}]},{"osd":1,"last update":"Mon Nov 27 15:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53800000000000003,"15min":0.505},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78500000000000003,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53800000000000003,"15min":0.52400000000000002},"min":{"1min":0.376,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.82599999999999996,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.499}]},{"osd":2,"last update":"Mon Nov 27 15:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58699999999999997,"15min":0.54500000000000004},"min":{"1min":0.40200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80100000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58099999999999996,"15min":0.54400000000000004},"min":{"1min":0.41699999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.81100000000000005,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.47999999999999998}]},{"osd":4,"last update":"Mon Nov 27 15:17:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55300000000000005,"15min":0.52700000000000002},"min":{"1min":0.41799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78900000000000003,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.52800000000000002},"min":{"1min":0.40300000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79700000000000004,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.56999999999999995}]},{"osd":5,"last update":"Mon Nov 27 15:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.624,"15min":0.58199999999999996},"min":{"1min":0.44900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.64300000000000002,"15min":0.59599999999999997},"min":{"1min":0.47299999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":0.51100000000000001}]},{"osd":6,"last update":"Mon Nov 27 15:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56999999999999995,"15min":0.55000000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.91100000000000003,"5min":1.089,"15min":1.089},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55900000000000005,"15min":0.54300000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.42899999999999999}]},{"osd":7,"last update":"Mon Nov 27 15:17:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.45900000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.80100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72599999999999998,"15min":0.70899999999999996},"min":{"1min":0.57299999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.83299999999999996,"5min":1.034,"15min":1.034},"last":0.67300000000000004}]},{"osd":8,"last update":"Mon Nov 27 15:15:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.628}]},{"osd":9,"last update":"Mon Nov 27 15:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.80600000000000005,"15min":0.82599999999999996},"min":{"1min":0.49299999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.083,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.625},{"interface":"front","average":{"1min":0.63,"5min":0.73799999999999999,"15min":0.75600000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95399999999999996,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.56200000000000006}]},{"osd":10,"last update":"Mon Nov 27 15:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.71799999999999997,"15min":0.73199999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.77200000000000002,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.84399999999999997,"15min":0.86199999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.86899999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.65200000000000002}]},{"osd":11,"last update":"Mon Nov 27 15:17:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.64000000000000001}]}]},{"osd":4,"up_from":33,"seq":141733920829,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6552,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738600,"statfs":{"total":95995035648,"available":95988326400,"internally_reserved":0,"allocated":933888,"data_stored":446428,"data_compressed":5787,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.48099999999999998,"15min":0.46800000000000003},"min":{"1min":0.35299999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.52200000000000002,"15min":0.50700000000000001},"min":{"1min":0.40300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.432}]},{"osd":1,"last update":"Mon Nov 27 15:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.39400000000000002,"15min":0.35599999999999998},"min":{"1min":0.29999999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.77200000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.39900000000000002,"15min":0.35699999999999998},"min":{"1min":0.318,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.90500000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.55600000000000005}]},{"osd":2,"last update":"Mon Nov 27 15:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.52800000000000002,"15min":0.496},"min":{"1min":0.36599999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.56299999999999994,"15min":0.54100000000000004},"min":{"1min":0.42599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.65000000000000002}]},{"osd":3,"last update":"Mon Nov 27 15:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.56299999999999994,"15min":0.52200000000000002},"min":{"1min":0.48799999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.872,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.71899999999999997}]},{"osd":5,"last update":"Mon Nov 27 15:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.58399999999999996,"15min":0.54300000000000004},"min":{"1min":0.47599999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.60299999999999998,"15min":0.56799999999999995},"min":{"1min":0.56100000000000005,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.74399999999999999}]},{"osd":6,"last update":"Mon Nov 27 15:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.63500000000000001,"15min":0.59999999999999998},"min":{"1min":0.46500000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":5.1040000000000001,"5min":5.1040000000000001,"15min":5.1040000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.67400000000000004,"15min":0.63800000000000001},"min":{"1min":0.46700000000000003,"5min":0.441,"15min":0.441},"max":{"1min":5.1139999999999999,"5min":5.1139999999999999,"15min":5.1139999999999999},"last":0.76700000000000002}]},{"osd":7,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.56000000000000005,"15min":0.53300000000000003},"min":{"1min":0.376,"5min":0.316,"15min":0.316},"max":{"1min":0.90300000000000002,"5min":1.018,"15min":1.018},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58799999999999997,"15min":0.55500000000000005},"min":{"1min":0.39400000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.873,"5min":1.115,"15min":1.115},"last":0.81599999999999995}]},{"osd":8,"last update":"Mon Nov 27 15:17:25 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.68799999999999994,"15min":0.66600000000000004},"min":{"1min":0.51900000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.94299999999999995,"5min":1.054,"15min":1.054},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.71999999999999997,"15min":0.70099999999999996},"min":{"1min":0.63100000000000001,"5min":0.432,"15min":0.432},"max":{"1min":1.022,"5min":1.179,"15min":1.179},"last":0.84999999999999998}]},{"osd":9,"last update":"Mon Nov 27 15:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.78100000000000003,"15min":0.75900000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":4.8780000000000001,"5min":4.8780000000000001,"15min":4.8780000000000001},"last":0.878},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.747,"15min":0.72299999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":4.9169999999999998,"5min":4.9169999999999998,"15min":4.9169999999999998},"last":0.86099999999999999}]},{"osd":10,"last update":"Mon Nov 27 15:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.75900000000000001,"15min":0.76000000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95899999999999996,"5min":1.103,"15min":1.103},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.91200000000000003}]},{"osd":11,"last update":"Mon Nov 27 15:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.92500000000000004}]}]},{"osd":5,"up_from":38,"seq":163208757304,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6676,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738476,"statfs":{"total":95995035648,"available":95988199424,"internally_reserved":0,"allocated":995328,"data_stored":1036328,"data_compressed":41881,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.50900000000000001,"15min":0.497},"min":{"1min":0.37,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.56000000000000005,"15min":0.498},"min":{"1min":0.39900000000000002,"5min":0.312,"15min":0.312},"max":{"1min":7.673,"5min":7.673,"15min":7.673},"last":0.44}]},{"osd":1,"last update":"Mon Nov 27 15:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.56899999999999995,"15min":0.53500000000000003},"min":{"1min":0.44700000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.57099999999999995,"15min":0.52500000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.60599999999999998}]},{"osd":2,"last update":"Mon Nov 27 15:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.41899999999999998,"15min":0.39600000000000002},"min":{"1min":0.29499999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.60499999999999998,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.628},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.44500000000000001,"15min":0.41299999999999998},"min":{"1min":0.28399999999999997,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.70599999999999996,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.54500000000000004}]},{"osd":3,"last update":"Mon Nov 27 15:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.623,"15min":0.57099999999999995},"min":{"1min":0.433,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":7.875,"5min":7.875,"15min":7.875},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.621,"15min":0.56200000000000006},"min":{"1min":0.46500000000000002,"5min":0.372,"15min":0.372},"max":{"1min":7.6059999999999999,"5min":7.6059999999999999,"15min":7.6059999999999999},"last":0.51700000000000002}]},{"osd":4,"last update":"Mon Nov 27 15:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.60299999999999998,"15min":0.56399999999999995},"min":{"1min":0.50600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.58199999999999996},"min":{"1min":0.42699999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.64700000000000002}]},{"osd":6,"last update":"Mon Nov 27 15:16:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":0.68500000000000005,"15min":0.61899999999999999},"min":{"1min":0.50800000000000001,"5min":0.436,"15min":0.436},"max":{"1min":8.0280000000000005,"5min":8.0280000000000005,"15min":8.0280000000000005},"last":0.69899999999999995},{"interface":"front","average":{"1min":1.1319999999999999,"5min":0.73899999999999999,"15min":0.67100000000000004},"min":{"1min":0.51800000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":8.0389999999999997,"5min":8.0389999999999997,"15min":8.0389999999999997},"last":0.68799999999999994}]},{"osd":7,"last update":"Mon Nov 27 15:17:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.67800000000000005,"15min":0.65900000000000003},"min":{"1min":0.498,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67000000000000004,"15min":0.66000000000000003},"min":{"1min":0.56100000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90400000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.73799999999999999}]},{"osd":8,"last update":"Mon Nov 27 15:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.66400000000000003,"15min":0.64400000000000002},"min":{"1min":0.38600000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.752},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.60899999999999999,"15min":0.59699999999999998},"min":{"1min":0.39700000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80900000000000005,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.71399999999999997}]},{"osd":9,"last update":"Mon Nov 27 15:16:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":0.747,"15min":0.69199999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":7.8070000000000004,"5min":7.8070000000000004,"15min":7.8070000000000004},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.194,"5min":0.81899999999999995,"15min":0.75700000000000001},"min":{"1min":0.56599999999999995,"5min":0.503,"15min":0.503},"max":{"1min":7.7690000000000001,"5min":7.7690000000000001,"15min":7.7690000000000001},"last":0.79200000000000004}]},{"osd":10,"last update":"Mon Nov 27 15:17:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.76000000000000001,"15min":0.747},"min":{"1min":0.63900000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.75,"5min":0.69599999999999995,"15min":0.68700000000000006},"min":{"1min":0.57099999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.77900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34455,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34455,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36094,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":59470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-27T15:17:43.247 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:17:43.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:42 smithi150 ceph-mon[100559]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 6 op/s 2023-11-27T15:17:43.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:42 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3372519384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:17:43.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:42 smithi150 ceph-mon[100559]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:17:43.408 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.403+0000 7fab91231700 1 -- 172.21.15.52:0/184460540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab8c0ff170 msgr2=0x7fab8c0ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:43.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.403+0000 7fab91231700 1 --2- 172.21.15.52:0/184460540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab8c0ff170 0x7fab8c0ff580 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fab7c009990 tx=0x7fab7c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.404+0000 7fab91231700 1 -- 172.21.15.52:0/184460540 shutdown_connections 2023-11-27T15:17:43.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.404+0000 7fab91231700 1 --2- 172.21.15.52:0/184460540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab8c100db0 0x7fab8c107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.404+0000 7fab91231700 1 --2- 172.21.15.52:0/184460540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab8c0ff170 0x7fab8c0ff580 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.404+0000 7fab91231700 1 --2- 172.21.15.52:0/184460540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab8c100400 0x7fab8c100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.404+0000 7fab91231700 1 -- 172.21.15.52:0/184460540 >> 172.21.15.52:0/184460540 conn(0x7fab8c0fa750 msgr2=0x7fab8c0fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:43.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.404+0000 7fab91231700 1 -- 172.21.15.52:0/184460540 shutdown_connections 2023-11-27T15:17:43.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.404+0000 7fab91231700 1 -- 172.21.15.52:0/184460540 wait complete. 2023-11-27T15:17:43.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.404+0000 7fab91231700 1 Processor -- start 2023-11-27T15:17:43.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.405+0000 7fab91231700 1 -- start start 2023-11-27T15:17:43.411 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.405+0000 7fab91231700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab8c0ff170 0x7fab8c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:43.411 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.405+0000 7fab91231700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab8c100400 0x7fab8c19ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:43.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.405+0000 7fab91231700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab8c100db0 0x7fab8c19d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:43.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.405+0000 7fab91231700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c19da40 con 0x7fab8c100400 2023-11-27T15:17:43.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.405+0000 7fab91231700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c19dbb0 con 0x7fab8c100db0 2023-11-27T15:17:43.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.405+0000 7fab91231700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c19dd70 con 0x7fab8c0ff170 2023-11-27T15:17:43.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.405+0000 7fab8a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab8c100400 0x7fab8c19ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:43.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.405+0000 7fab8a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab8c100400 0x7fab8c19ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51516/0 (socket says 172.21.15.52:51516) 2023-11-27T15:17:43.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.405+0000 7fab8a59c700 1 -- 172.21.15.52:0/960593652 learned_addr learned my addr 172.21.15.52:0/960593652 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:43.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.405+0000 7fab8ad9d700 1 --2- 172.21.15.52:0/960593652 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab8c0ff170 0x7fab8c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:43.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.405+0000 7fab8b59e700 1 --2- 172.21.15.52:0/960593652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab8c100db0 0x7fab8c19d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:43.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.405+0000 7fab8a59c700 1 -- 172.21.15.52:0/960593652 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab8c0ff170 msgr2=0x7fab8c068960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:43.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.405+0000 7fab8a59c700 1 --2- 172.21.15.52:0/960593652 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab8c0ff170 0x7fab8c068960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.405+0000 7fab8a59c700 1 -- 172.21.15.52:0/960593652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab8c100db0 msgr2=0x7fab8c19d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:43.414 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.405+0000 7fab8a59c700 1 --2- 172.21.15.52:0/960593652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab8c100db0 0x7fab8c19d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.414 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.405+0000 7fab8a59c700 1 -- 172.21.15.52:0/960593652 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab7c0095f0 con 0x7fab8c100400 2023-11-27T15:17:43.414 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.405+0000 7fab8b59e700 1 --2- 172.21.15.52:0/960593652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab8c100db0 0x7fab8c19d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:17:43.414 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.405+0000 7fab8a59c700 1 --2- 172.21.15.52:0/960593652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab8c100400 0x7fab8c19ac20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fab8000d980 tx=0x7fab8000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:43.414 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.406+0000 7fab73fff700 1 -- 172.21.15.52:0/960593652 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab80004220 con 0x7fab8c100400 2023-11-27T15:17:43.414 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.406+0000 7fab73fff700 1 -- 172.21.15.52:0/960593652 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab80007640 con 0x7fab8c100400 2023-11-27T15:17:43.414 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.406+0000 7fab91231700 1 -- 172.21.15.52:0/960593652 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab8c10ab30 con 0x7fab8c100400 2023-11-27T15:17:43.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.406+0000 7fab73fff700 1 -- 172.21.15.52:0/960593652 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab80005610 con 0x7fab8c100400 2023-11-27T15:17:43.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.406+0000 7fab91231700 1 -- 172.21.15.52:0/960593652 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab8c10a9a0 con 0x7fab8c100400 2023-11-27T15:17:43.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.407+0000 7fab91231700 1 -- 172.21.15.52:0/960593652 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab8c106190 con 0x7fab8c100400 2023-11-27T15:17:43.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.408+0000 7fab73fff700 1 -- 172.21.15.52:0/960593652 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fab80004750 con 0x7fab8c100400 2023-11-27T15:17:43.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.408+0000 7fab73fff700 1 --2- 172.21.15.52:0/960593652 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fab740a1150 0x7fab740a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:43.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.408+0000 7fab8ad9d700 1 --2- 172.21.15.52:0/960593652 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fab740a1150 0x7fab740a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:43.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.408+0000 7fab73fff700 1 -- 172.21.15.52:0/960593652 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fab800cccc0 con 0x7fab8c100400 2023-11-27T15:17:43.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.409+0000 7fab8ad9d700 1 --2- 172.21.15.52:0/960593652 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fab740a1150 0x7fab740a3600 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00b5c0 tx=0x7fab7c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:43.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.412+0000 7fab73fff700 1 -- 172.21.15.52:0/960593652 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fab800963f0 con 0x7fab8c100400 2023-11-27T15:17:43.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.615+0000 7fab91231700 1 -- 172.21.15.52:0/960593652 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab8c061700 con 0x7fab8c100400 2023-11-27T15:17:43.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.617+0000 7fab73fff700 1 -- 172.21.15.52:0/960593652 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22657 (secure 0 0 0) 0x7fab80016090 con 0x7fab8c100400 2023-11-27T15:17:43.621 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:17:43.621 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":85,"fsid":"c6c7df88-8d36-11ee-95a2-87774f69a715","created":"2023-11-27T15:09:03.333345+0000","modified":"2023-11-27T15:17:04.797892+0000","last_up_change":"2023-11-27T15:15:27.656134+0000","last_in_change":"2023-11-27T15:15:10.671153+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:12:14.177523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:16:07.842251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:16:08.728520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58d69ede-c293-49f5-abbe-8187793f2b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6803","nonce":608216415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6805","nonce":608216415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6809","nonce":608216415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6807","nonce":608216415}]},"public_addr":"172.21.15.52:6803/608216415","cluster_addr":"172.21.15.52:6805/608216415","heartbeat_back_addr":"172.21.15.52:6809/608216415","heartbeat_front_addr":"172.21.15.52:6807/608216415","state":["exists","up"]},{"osd":1,"uuid":"d6a46b91-adc8-45ef-80f5-566ae400911f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6801","nonce":2538384087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6803","nonce":2538384087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6807","nonce":2538384087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6805","nonce":2538384087}]},"public_addr":"172.21.15.102:6801/2538384087","cluster_addr":"172.21.15.102:6803/2538384087","heartbeat_back_addr":"172.21.15.102:6807/2538384087","heartbeat_front_addr":"172.21.15.102:6805/2538384087","state":["exists","up"]},{"osd":2,"uuid":"846ba125-11f8-46b2-8941-bd5e1947cc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6801","nonce":4258038221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6803","nonce":4258038221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6807","nonce":4258038221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6805","nonce":4258038221}]},"public_addr":"172.21.15.150:6801/4258038221","cluster_addr":"172.21.15.150:6803/4258038221","heartbeat_back_addr":"172.21.15.150:6807/4258038221","heartbeat_front_addr":"172.21.15.150:6805/4258038221","state":["exists","up"]},{"osd":3,"uuid":"950fcd67-1057-4288-a389-714f1ec94261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6811","nonce":2622100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6813","nonce":2622100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6817","nonce":2622100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6815","nonce":2622100752}]},"public_addr":"172.21.15.52:6811/2622100752","cluster_addr":"172.21.15.52:6813/2622100752","heartbeat_back_addr":"172.21.15.52:6817/2622100752","heartbeat_front_addr":"172.21.15.52:6815/2622100752","state":["exists","up"]},{"osd":4,"uuid":"1c96a676-cea9-4deb-bad1-f7fd240c3d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6809","nonce":1546794763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6811","nonce":1546794763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6815","nonce":1546794763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6813","nonce":1546794763}]},"public_addr":"172.21.15.102:6809/1546794763","cluster_addr":"172.21.15.102:6811/1546794763","heartbeat_back_addr":"172.21.15.102:6815/1546794763","heartbeat_front_addr":"172.21.15.102:6813/1546794763","state":["exists","up"]},{"osd":5,"uuid":"38a75af1-fd4d-433e-b579-0609109c851c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6809","nonce":461511375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6811","nonce":461511375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6815","nonce":461511375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6813","nonce":461511375}]},"public_addr":"172.21.15.150:6809/461511375","cluster_addr":"172.21.15.150:6811/461511375","heartbeat_back_addr":"172.21.15.150:6815/461511375","heartbeat_front_addr":"172.21.15.150:6813/461511375","state":["exists","up"]},{"osd":6,"uuid":"2f0ede4a-d175-4b4a-b876-c0f442048901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6819","nonce":1002457464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6821","nonce":1002457464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6825","nonce":1002457464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6823","nonce":1002457464}]},"public_addr":"172.21.15.52:6819/1002457464","cluster_addr":"172.21.15.52:6821/1002457464","heartbeat_back_addr":"172.21.15.52:6825/1002457464","heartbeat_front_addr":"172.21.15.52:6823/1002457464","state":["exists","up"]},{"osd":7,"uuid":"6b635192-abcf-4bbf-bf78-a019b351bca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6817","nonce":3249840064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6819","nonce":3249840064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6823","nonce":3249840064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6821","nonce":3249840064}]},"public_addr":"172.21.15.102:6817/3249840064","cluster_addr":"172.21.15.102:6819/3249840064","heartbeat_back_addr":"172.21.15.102:6823/3249840064","heartbeat_front_addr":"172.21.15.102:6821/3249840064","state":["exists","up"]},{"osd":8,"uuid":"c7ca8007-7170-47bf-956c-3f00edf27e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6817","nonce":3672583574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6819","nonce":3672583574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6823","nonce":3672583574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6821","nonce":3672583574}]},"public_addr":"172.21.15.150:6817/3672583574","cluster_addr":"172.21.15.150:6819/3672583574","heartbeat_back_addr":"172.21.15.150:6823/3672583574","heartbeat_front_addr":"172.21.15.150:6821/3672583574","state":["exists","up"]},{"osd":9,"uuid":"0e252c2c-01a4-48ba-832d-067ed940fefb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6827","nonce":4287934320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6829","nonce":4287934320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6833","nonce":4287934320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6831","nonce":4287934320}]},"public_addr":"172.21.15.52:6827/4287934320","cluster_addr":"172.21.15.52:6829/4287934320","heartbeat_back_addr":"172.21.15.52:6833/4287934320","heartbeat_front_addr":"172.21.15.52:6831/4287934320","state":["exists","up"]},{"osd":10,"uuid":"bb540142-7563-44a4-b7a3-cb662714c128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6825","nonce":3056034201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6827","nonce":3056034201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6831","nonce":3056034201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6829","nonce":3056034201}]},"public_addr":"172.21.15.102:6825/3056034201","cluster_addr":"172.21.15.102:6827/3056034201","heartbeat_back_addr":"172.21.15.102:6831/3056034201","heartbeat_front_addr":"172.21.15.102:6829/3056034201","state":["exists","up"]},{"osd":11,"uuid":"3156528e-e0f8-4d44-9946-c8b6339c2bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6825","nonce":3207559739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6827","nonce":3207559739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6831","nonce":3207559739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6829","nonce":3207559739}]},"public_addr":"172.21.15.150:6825/3207559739","cluster_addr":"172.21.15.150:6827/3207559739","heartbeat_back_addr":"172.21.15.150:6831/3207559739","heartbeat_front_addr":"172.21.15.150:6829/3207559739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:26.826099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:48.331371+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:10.459738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:32.356472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:54.043170+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:15.648582+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:37.166463+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:59.240819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:20.710097+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:43.364563+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:05.279433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:26.344573+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":5,"to":7}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/1392282110":"2023-11-28T15:10:03.644870+0000","172.21.15.52:0/1479993479":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6800/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/961562652":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/1290665390":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2410529742":"2023-11-28T15:09:37.118690+0000","172.21.15.52:6800/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2130037044":"2023-11-28T15:10:03.644870+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:17:43.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.619+0000 7fab91231700 1 -- 172.21.15.52:0/960593652 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fab740a1150 msgr2=0x7fab740a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:43.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.619+0000 7fab91231700 1 --2- 172.21.15.52:0/960593652 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fab740a1150 0x7fab740a3600 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00b5c0 tx=0x7fab7c005fd0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.619+0000 7fab91231700 1 -- 172.21.15.52:0/960593652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab8c100400 msgr2=0x7fab8c19ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:43.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.619+0000 7fab91231700 1 --2- 172.21.15.52:0/960593652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab8c100400 0x7fab8c19ac20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fab8000d980 tx=0x7fab8000dc90 comp rx=0 tx=0).stop 2023-11-27T15:17:43.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.619+0000 7fab91231700 1 -- 172.21.15.52:0/960593652 shutdown_connections 2023-11-27T15:17:43.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.619+0000 7fab91231700 1 --2- 172.21.15.52:0/960593652 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fab740a1150 0x7fab740a3600 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.619+0000 7fab91231700 1 --2- 172.21.15.52:0/960593652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab8c100db0 0x7fab8c19d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.619+0000 7fab91231700 1 --2- 172.21.15.52:0/960593652 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab8c0ff170 0x7fab8c068960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.619+0000 7fab91231700 1 --2- 172.21.15.52:0/960593652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab8c100400 0x7fab8c19ac20 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.619+0000 7fab91231700 1 -- 172.21.15.52:0/960593652 >> 172.21.15.52:0/960593652 conn(0x7fab8c0fa750 msgr2=0x7fab8c0fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:43.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.619+0000 7fab91231700 1 -- 172.21.15.52:0/960593652 shutdown_connections 2023-11-27T15:17:43.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.619+0000 7fab91231700 1 -- 172.21.15.52:0/960593652 wait complete. 2023-11-27T15:17:43.636 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:12:14.177523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-27T15:16:07.842251+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-27T15:16:08.728520+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-27T15:17:43.636 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:17:43.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.762+0000 7fd65160f700 1 -- 172.21.15.52:0/1779141752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd64c1036e0 msgr2=0x7fd64c105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:43.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.762+0000 7fd65160f700 1 --2- 172.21.15.52:0/1779141752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd64c1036e0 0x7fd64c105ac0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd63c009990 tx=0x7fd63c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.762+0000 7fd65160f700 1 -- 172.21.15.52:0/1779141752 shutdown_connections 2023-11-27T15:17:43.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.762+0000 7fd65160f700 1 --2- 172.21.15.52:0/1779141752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd64c100dc0 0x7fd64c1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.762+0000 7fd65160f700 1 --2- 172.21.15.52:0/1779141752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd64c106000 0x7fd64c1083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.762+0000 7fd65160f700 1 --2- 172.21.15.52:0/1779141752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd64c1036e0 0x7fd64c105ac0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.762+0000 7fd65160f700 1 -- 172.21.15.52:0/1779141752 >> 172.21.15.52:0/1779141752 conn(0x7fd64c0fa7c0 msgr2=0x7fd64c0fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:43.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.762+0000 7fd65160f700 1 -- 172.21.15.52:0/1779141752 shutdown_connections 2023-11-27T15:17:43.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.762+0000 7fd65160f700 1 -- 172.21.15.52:0/1779141752 wait complete. 2023-11-27T15:17:43.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.762+0000 7fd65160f700 1 Processor -- start 2023-11-27T15:17:43.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.763+0000 7fd65160f700 1 -- start start 2023-11-27T15:17:43.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.763+0000 7fd65160f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd64c100dc0 0x7fd64c196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:43.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.763+0000 7fd65160f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd64c1036e0 0x7fd64c196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:43.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.763+0000 7fd65160f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd64c106000 0x7fd64c19d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:43.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.763+0000 7fd65160f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c19dba0 con 0x7fd64c100dc0 2023-11-27T15:17:43.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.763+0000 7fd65160f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c19dce0 con 0x7fd64c1036e0 2023-11-27T15:17:43.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.763+0000 7fd65160f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c19de70 con 0x7fd64c106000 2023-11-27T15:17:43.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.763+0000 7fd64affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd64c100dc0 0x7fd64c196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:43.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.763+0000 7fd64affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd64c100dc0 0x7fd64c196020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51528/0 (socket says 172.21.15.52:51528) 2023-11-27T15:17:43.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.763+0000 7fd64affd700 1 -- 172.21.15.52:0/3186393162 learned_addr learned my addr 172.21.15.52:0/3186393162 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:43.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.763+0000 7fd64a7fc700 1 --2- 172.21.15.52:0/3186393162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd64c1036e0 0x7fd64c196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:43.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.763+0000 7fd64b7fe700 1 --2- 172.21.15.52:0/3186393162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd64c106000 0x7fd64c19d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:43.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.763+0000 7fd64affd700 1 -- 172.21.15.52:0/3186393162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd64c106000 msgr2=0x7fd64c19d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:43.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.763+0000 7fd64affd700 1 --2- 172.21.15.52:0/3186393162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd64c106000 0x7fd64c19d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.763+0000 7fd64affd700 1 -- 172.21.15.52:0/3186393162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd64c1036e0 msgr2=0x7fd64c196560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:43.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.763+0000 7fd64affd700 1 --2- 172.21.15.52:0/3186393162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd64c1036e0 0x7fd64c196560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.763+0000 7fd64affd700 1 -- 172.21.15.52:0/3186393162 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c0095f0 con 0x7fd64c100dc0 2023-11-27T15:17:43.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.763+0000 7fd64affd700 1 --2- 172.21.15.52:0/3186393162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd64c100dc0 0x7fd64c196020 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd64000e9a0 tx=0x7fd64000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:43.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.764+0000 7fd633fff700 1 -- 172.21.15.52:0/3186393162 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd640009790 con 0x7fd64c100dc0 2023-11-27T15:17:43.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.764+0000 7fd633fff700 1 -- 172.21.15.52:0/3186393162 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd64000cbd0 con 0x7fd64c100dc0 2023-11-27T15:17:43.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.764+0000 7fd65160f700 1 -- 172.21.15.52:0/3186393162 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd64c10bc20 con 0x7fd64c100dc0 2023-11-27T15:17:43.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.764+0000 7fd633fff700 1 -- 172.21.15.52:0/3186393162 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd640018680 con 0x7fd64c100dc0 2023-11-27T15:17:43.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.764+0000 7fd65160f700 1 -- 172.21.15.52:0/3186393162 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd64c10ba90 con 0x7fd64c100dc0 2023-11-27T15:17:43.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.765+0000 7fd65160f700 1 -- 172.21.15.52:0/3186393162 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd64c0fc330 con 0x7fd64c100dc0 2023-11-27T15:17:43.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.766+0000 7fd633fff700 1 -- 172.21.15.52:0/3186393162 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd64000cd40 con 0x7fd64c100dc0 2023-11-27T15:17:43.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.767+0000 7fd633fff700 1 --2- 172.21.15.52:0/3186393162 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd6340a1250 0x7fd6340a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:43.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.767+0000 7fd64a7fc700 1 --2- 172.21.15.52:0/3186393162 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd6340a1250 0x7fd6340a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:43.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.767+0000 7fd633fff700 1 -- 172.21.15.52:0/3186393162 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd640014070 con 0x7fd64c100dc0 2023-11-27T15:17:43.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.768+0000 7fd64a7fc700 1 --2- 172.21.15.52:0/3186393162 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd6340a1250 0x7fd6340a3700 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00b2e0 tx=0x7fd63c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:43.775 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.771+0000 7fd633fff700 1 -- 172.21.15.52:0/3186393162 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd6400964f0 con 0x7fd64c100dc0 2023-11-27T15:17:43.976 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.972+0000 7fd65160f700 1 -- 172.21.15.52:0/3186393162 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd64c061700 con 0x7fd64c100dc0 2023-11-27T15:17:43.977 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.973+0000 7fd633fff700 1 -- 172.21.15.52:0/3186393162 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd6400260a0 con 0x7fd64c100dc0 2023-11-27T15:17:43.977 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2023-11-27T15:17:43.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.975+0000 7fd65160f700 1 -- 172.21.15.52:0/3186393162 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd6340a1250 msgr2=0x7fd6340a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:43.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.975+0000 7fd65160f700 1 --2- 172.21.15.52:0/3186393162 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd6340a1250 0x7fd6340a3700 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00b2e0 tx=0x7fd63c005fd0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.975+0000 7fd65160f700 1 -- 172.21.15.52:0/3186393162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd64c100dc0 msgr2=0x7fd64c196020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:43.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.975+0000 7fd65160f700 1 --2- 172.21.15.52:0/3186393162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd64c100dc0 0x7fd64c196020 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd64000e9a0 tx=0x7fd64000ecb0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.980 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.975+0000 7fd65160f700 1 -- 172.21.15.52:0/3186393162 shutdown_connections 2023-11-27T15:17:43.980 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.975+0000 7fd65160f700 1 --2- 172.21.15.52:0/3186393162 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fd6340a1250 0x7fd6340a3700 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.980 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.975+0000 7fd65160f700 1 --2- 172.21.15.52:0/3186393162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd64c106000 0x7fd64c19d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.980 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.975+0000 7fd65160f700 1 --2- 172.21.15.52:0/3186393162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd64c100dc0 0x7fd64c196020 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.980 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.975+0000 7fd65160f700 1 --2- 172.21.15.52:0/3186393162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd64c1036e0 0x7fd64c196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:43.980 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.975+0000 7fd65160f700 1 -- 172.21.15.52:0/3186393162 >> 172.21.15.52:0/3186393162 conn(0x7fd64c0fa7c0 msgr2=0x7fd64c101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:43.980 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.975+0000 7fd65160f700 1 -- 172.21.15.52:0/3186393162 shutdown_connections 2023-11-27T15:17:43.980 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:43.975+0000 7fd65160f700 1 -- 172.21.15.52:0/3186393162 wait complete. 2023-11-27T15:17:43.990 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:43 smithi052 ceph-mon[99056]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:17:43.990 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:43 smithi052 ceph-mon[99056]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:17:43.990 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:43 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/960593652' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:17:43.991 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-27T15:17:44.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:43 smithi102 ceph-mon[100296]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:17:44.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:43 smithi102 ceph-mon[100296]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:17:44.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:43 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/960593652' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:17:44.121 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.117+0000 7f93c0abd700 1 -- 172.21.15.52:0/1307811705 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93bc0ff1c0 msgr2=0x7f93bc0ff5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:44.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.117+0000 7f93c0abd700 1 --2- 172.21.15.52:0/1307811705 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93bc0ff1c0 0x7f93bc0ff5d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009990 tx=0x7f93b0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:44.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.117+0000 7f93c0abd700 1 -- 172.21.15.52:0/1307811705 shutdown_connections 2023-11-27T15:17:44.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.117+0000 7f93c0abd700 1 --2- 172.21.15.52:0/1307811705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93bc100d40 0x7f93bc107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:44.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.117+0000 7f93c0abd700 1 --2- 172.21.15.52:0/1307811705 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93bc0ff1c0 0x7f93bc0ff5d0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:44.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.117+0000 7f93c0abd700 1 --2- 172.21.15.52:0/1307811705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93bc1003b0 0x7f93bc100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:44.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.117+0000 7f93c0abd700 1 -- 172.21.15.52:0/1307811705 >> 172.21.15.52:0/1307811705 conn(0x7f93bc0fa7c0 msgr2=0x7f93bc0fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:44.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.117+0000 7f93c0abd700 1 -- 172.21.15.52:0/1307811705 shutdown_connections 2023-11-27T15:17:44.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.117+0000 7f93c0abd700 1 -- 172.21.15.52:0/1307811705 wait complete. 2023-11-27T15:17:44.126 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.118+0000 7f93c0abd700 1 Processor -- start 2023-11-27T15:17:44.126 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.118+0000 7f93c0abd700 1 -- start start 2023-11-27T15:17:44.126 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.118+0000 7f93c0abd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93bc0ff1c0 0x7f93bc1960a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:44.126 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.118+0000 7f93c0abd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93bc1003b0 0x7f93bc1965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:44.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.118+0000 7f93c0abd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93bc100d40 0x7f93bc19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:44.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.118+0000 7f93c0abd700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc19dc20 con 0x7f93bc0ff1c0 2023-11-27T15:17:44.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.118+0000 7f93c0abd700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc19dd60 con 0x7f93bc100d40 2023-11-27T15:17:44.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.118+0000 7f93c0abd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc19def0 con 0x7f93bc1003b0 2023-11-27T15:17:44.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.118+0000 7f93ba59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93bc0ff1c0 0x7f93bc1960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:44.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.118+0000 7f93ba59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93bc0ff1c0 0x7f93bc1960a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51542/0 (socket says 172.21.15.52:51542) 2023-11-27T15:17:44.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.118+0000 7f93ba59c700 1 -- 172.21.15.52:0/4233453545 learned_addr learned my addr 172.21.15.52:0/4233453545 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:44.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.118+0000 7f93b9d9b700 1 --2- 172.21.15.52:0/4233453545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93bc1003b0 0x7f93bc1965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:44.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.118+0000 7f93bad9d700 1 --2- 172.21.15.52:0/4233453545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93bc100d40 0x7f93bc19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:44.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.119+0000 7f93ba59c700 1 -- 172.21.15.52:0/4233453545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93bc1003b0 msgr2=0x7f93bc1965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:44.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.119+0000 7f93ba59c700 1 --2- 172.21.15.52:0/4233453545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93bc1003b0 0x7f93bc1965e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:44.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.119+0000 7f93ba59c700 1 -- 172.21.15.52:0/4233453545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93bc100d40 msgr2=0x7f93bc19d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:44.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.119+0000 7f93ba59c700 1 --2- 172.21.15.52:0/4233453545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93bc100d40 0x7f93bc19d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:44.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.119+0000 7f93ba59c700 1 -- 172.21.15.52:0/4233453545 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b00095f0 con 0x7f93bc0ff1c0 2023-11-27T15:17:44.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.119+0000 7f93bad9d700 1 --2- 172.21.15.52:0/4233453545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93bc100d40 0x7f93bc19d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:17:44.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.119+0000 7f93ba59c700 1 --2- 172.21.15.52:0/4233453545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93bc0ff1c0 0x7f93bc1960a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f93b0006010 tx=0x7f93b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:44.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.119+0000 7f93ab7fe700 1 -- 172.21.15.52:0/4233453545 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b001d070 con 0x7f93bc0ff1c0 2023-11-27T15:17:44.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.119+0000 7f93ab7fe700 1 -- 172.21.15.52:0/4233453545 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93b0003fe0 con 0x7f93bc0ff1c0 2023-11-27T15:17:44.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.119+0000 7f93c0abd700 1 -- 172.21.15.52:0/4233453545 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93bc10ab20 con 0x7f93bc0ff1c0 2023-11-27T15:17:44.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.119+0000 7f93ab7fe700 1 -- 172.21.15.52:0/4233453545 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0022c10 con 0x7f93bc0ff1c0 2023-11-27T15:17:44.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.119+0000 7f93c0abd700 1 -- 172.21.15.52:0/4233453545 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93bc10a990 con 0x7f93bc0ff1c0 2023-11-27T15:17:44.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.120+0000 7f93c0abd700 1 -- 172.21.15.52:0/4233453545 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93bc19e4f0 con 0x7f93bc0ff1c0 2023-11-27T15:17:44.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.121+0000 7f93ab7fe700 1 -- 172.21.15.52:0/4233453545 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f93b0022470 con 0x7f93bc0ff1c0 2023-11-27T15:17:44.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.122+0000 7f93ab7fe700 1 --2- 172.21.15.52:0/4233453545 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f93a40a11d0 0x7f93a40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:44.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.122+0000 7f93b9d9b700 1 --2- 172.21.15.52:0/4233453545 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f93a40a11d0 0x7f93a40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:44.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.122+0000 7f93ab7fe700 1 -- 172.21.15.52:0/4233453545 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f93b00cded0 con 0x7f93bc0ff1c0 2023-11-27T15:17:44.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.122+0000 7f93b9d9b700 1 --2- 172.21.15.52:0/4233453545 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f93a40a11d0 0x7f93a40a3680 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f93ac005950 tx=0x7f93ac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:44.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.126+0000 7f93ab7fe700 1 -- 172.21.15.52:0/4233453545 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f93b0097630 con 0x7f93bc0ff1c0 2023-11-27T15:17:44.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:43 smithi150 ceph-mon[100559]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:17:44.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:43 smithi150 ceph-mon[100559]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:17:44.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:43 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/960593652' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:17:44.328 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.324+0000 7f93c0abd700 1 -- 172.21.15.52:0/4233453545 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f93bc19ebd0 con 0x7f93bc0ff1c0 2023-11-27T15:17:44.329 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.324+0000 7f93ab7fe700 1 -- 172.21.15.52:0/4233453545 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f93b009ac50 con 0x7f93bc0ff1c0 2023-11-27T15:17:44.329 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2023-11-27T15:17:44.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.326+0000 7f93c0abd700 1 -- 172.21.15.52:0/4233453545 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f93a40a11d0 msgr2=0x7f93a40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:44.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.326+0000 7f93c0abd700 1 --2- 172.21.15.52:0/4233453545 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f93a40a11d0 0x7f93a40a3680 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f93ac005950 tx=0x7f93ac00b410 comp rx=0 tx=0).stop 2023-11-27T15:17:44.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.326+0000 7f93c0abd700 1 -- 172.21.15.52:0/4233453545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93bc0ff1c0 msgr2=0x7f93bc1960a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:44.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.326+0000 7f93c0abd700 1 --2- 172.21.15.52:0/4233453545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93bc0ff1c0 0x7f93bc1960a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f93b0006010 tx=0x7f93b0004730 comp rx=0 tx=0).stop 2023-11-27T15:17:44.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.326+0000 7f93c0abd700 1 -- 172.21.15.52:0/4233453545 shutdown_connections 2023-11-27T15:17:44.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.326+0000 7f93c0abd700 1 --2- 172.21.15.52:0/4233453545 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f93a40a11d0 0x7f93a40a3680 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:44.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.326+0000 7f93c0abd700 1 --2- 172.21.15.52:0/4233453545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93bc100d40 0x7f93bc19d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:44.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.326+0000 7f93c0abd700 1 --2- 172.21.15.52:0/4233453545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93bc0ff1c0 0x7f93bc1960a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:44.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.326+0000 7f93c0abd700 1 --2- 172.21.15.52:0/4233453545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93bc1003b0 0x7f93bc1965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:44.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.326+0000 7f93c0abd700 1 -- 172.21.15.52:0/4233453545 >> 172.21.15.52:0/4233453545 conn(0x7f93bc0fa7c0 msgr2=0x7f93bc067e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:44.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.327+0000 7f93c0abd700 1 -- 172.21.15.52:0/4233453545 shutdown_connections 2023-11-27T15:17:44.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.327+0000 7f93c0abd700 1 -- 172.21.15.52:0/4233453545 wait complete. 2023-11-27T15:17:44.343 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-27T15:17:44.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.467+0000 7f8e03b84700 1 -- 172.21.15.52:0/2328243568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8dfc1014c0 msgr2=0x7f8dfc101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.467+0000 7f8e03b84700 1 --2- 172.21.15.52:0/2328243568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8dfc1014c0 0x7f8dfc101910 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8df0009990 tx=0x7f8df0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.467+0000 7f8e03b84700 1 -- 172.21.15.52:0/2328243568 shutdown_connections 2023-11-27T15:17:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.467+0000 7f8e03b84700 1 --2- 172.21.15.52:0/2328243568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dfc101e50 0x7f8dfc108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.467+0000 7f8e03b84700 1 --2- 172.21.15.52:0/2328243568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dfc1002d0 0x7f8dfc1006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.467+0000 7f8e03b84700 1 --2- 172.21.15.52:0/2328243568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8dfc1014c0 0x7f8dfc101910 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.467+0000 7f8e03b84700 1 -- 172.21.15.52:0/2328243568 >> 172.21.15.52:0/2328243568 conn(0x7f8dfc0fb8d0 msgr2=0x7f8dfc0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.467+0000 7f8e03b84700 1 -- 172.21.15.52:0/2328243568 shutdown_connections 2023-11-27T15:17:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.467+0000 7f8e03b84700 1 -- 172.21.15.52:0/2328243568 wait complete. 2023-11-27T15:17:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.468+0000 7f8e03b84700 1 Processor -- start 2023-11-27T15:17:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.468+0000 7f8e03b84700 1 -- start start 2023-11-27T15:17:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.468+0000 7f8e03b84700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8dfc1002d0 0x7f8dfc0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.468+0000 7f8e03b84700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dfc1014c0 0x7f8dfc072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.468+0000 7f8e03b84700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dfc101e50 0x7f8dfc073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.468+0000 7f8e03b84700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc073bc0 con 0x7f8dfc1002d0 2023-11-27T15:17:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.468+0000 7f8e03b84700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc073d30 con 0x7f8dfc1014c0 2023-11-27T15:17:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.468+0000 7f8e03b84700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc1a0020 con 0x7f8dfc101e50 2023-11-27T15:17:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.468+0000 7f8e01920700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8dfc1002d0 0x7f8dfc0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.468+0000 7f8e01920700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8dfc1002d0 0x7f8dfc0748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51556/0 (socket says 172.21.15.52:51556) 2023-11-27T15:17:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.468+0000 7f8e01920700 1 -- 172.21.15.52:0/616908979 learned_addr learned my addr 172.21.15.52:0/616908979 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.468+0000 7f8e0111f700 1 --2- 172.21.15.52:0/616908979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dfc1014c0 0x7f8dfc072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:44.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.468+0000 7f8e02121700 1 --2- 172.21.15.52:0/616908979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dfc101e50 0x7f8dfc073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:44.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.469+0000 7f8e01920700 1 -- 172.21.15.52:0/616908979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dfc101e50 msgr2=0x7f8dfc073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:44.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.469+0000 7f8e01920700 1 --2- 172.21.15.52:0/616908979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dfc101e50 0x7f8dfc073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:44.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.469+0000 7f8e01920700 1 -- 172.21.15.52:0/616908979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dfc1014c0 msgr2=0x7f8dfc072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:44.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.469+0000 7f8e01920700 1 --2- 172.21.15.52:0/616908979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dfc1014c0 0x7f8dfc072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:44.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.469+0000 7f8e01920700 1 -- 172.21.15.52:0/616908979 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df00095f0 con 0x7f8dfc1002d0 2023-11-27T15:17:44.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.469+0000 7f8e02121700 1 --2- 172.21.15.52:0/616908979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dfc101e50 0x7f8dfc073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:17:44.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.469+0000 7f8e01920700 1 --2- 172.21.15.52:0/616908979 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8dfc1002d0 0x7f8dfc0748a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8df400d980 tx=0x7f8df400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:44.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.469+0000 7f8dfaffd700 1 -- 172.21.15.52:0/616908979 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df4004020 con 0x7f8dfc1002d0 2023-11-27T15:17:44.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.469+0000 7f8dfaffd700 1 -- 172.21.15.52:0/616908979 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8df4009730 con 0x7f8dfc1002d0 2023-11-27T15:17:44.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.469+0000 7f8e03b84700 1 -- 172.21.15.52:0/616908979 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dfc10bc40 con 0x7f8dfc1002d0 2023-11-27T15:17:44.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.469+0000 7f8dfaffd700 1 -- 172.21.15.52:0/616908979 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df4003a80 con 0x7f8dfc1002d0 2023-11-27T15:17:44.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.469+0000 7f8e03b84700 1 -- 172.21.15.52:0/616908979 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dfc10bab0 con 0x7f8dfc1002d0 2023-11-27T15:17:44.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.470+0000 7f8e03b84700 1 -- 172.21.15.52:0/616908979 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dfc1a0770 con 0x7f8dfc1002d0 2023-11-27T15:17:44.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.471+0000 7f8dfaffd700 1 -- 172.21.15.52:0/616908979 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8df40098a0 con 0x7f8dfc1002d0 2023-11-27T15:17:44.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.472+0000 7f8dfaffd700 1 --2- 172.21.15.52:0/616908979 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f8de40a1150 0x7f8de40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:44.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.472+0000 7f8e0111f700 1 --2- 172.21.15.52:0/616908979 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f8de40a1150 0x7f8de40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:44.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.472+0000 7f8dfaffd700 1 -- 172.21.15.52:0/616908979 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8df40cc9d0 con 0x7f8dfc1002d0 2023-11-27T15:17:44.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.472+0000 7f8e0111f700 1 --2- 172.21.15.52:0/616908979 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f8de40a1150 0x7f8de40a3600 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8df000b5c0 tx=0x7f8df0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:44.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.475+0000 7f8dfaffd700 1 -- 172.21.15.52:0/616908979 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8df40960a0 con 0x7f8dfc1002d0 2023-11-27T15:17:44.678 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.673+0000 7f8e03b84700 1 -- 172.21.15.52:0/616908979 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8dfc107120 con 0x7f8dfc1002d0 2023-11-27T15:17:44.679 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.673+0000 7f8dfaffd700 1 -- 172.21.15.52:0/616908979 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8df40996c0 con 0x7f8dfc1002d0 2023-11-27T15:17:44.679 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2023-11-27T15:17:44.680 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.675+0000 7f8e03b84700 1 -- 172.21.15.52:0/616908979 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f8de40a1150 msgr2=0x7f8de40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:44.680 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.675+0000 7f8e03b84700 1 --2- 172.21.15.52:0/616908979 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f8de40a1150 0x7f8de40a3600 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8df000b5c0 tx=0x7f8df0005960 comp rx=0 tx=0).stop 2023-11-27T15:17:44.680 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.675+0000 7f8e03b84700 1 -- 172.21.15.52:0/616908979 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8dfc1002d0 msgr2=0x7f8dfc0748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:44.680 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.675+0000 7f8e03b84700 1 --2- 172.21.15.52:0/616908979 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8dfc1002d0 0x7f8dfc0748a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8df400d980 tx=0x7f8df400dc90 comp rx=0 tx=0).stop 2023-11-27T15:17:44.680 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.675+0000 7f8e03b84700 1 -- 172.21.15.52:0/616908979 shutdown_connections 2023-11-27T15:17:44.680 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.675+0000 7f8e03b84700 1 --2- 172.21.15.52:0/616908979 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f8de40a1150 0x7f8de40a3600 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:44.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.675+0000 7f8e03b84700 1 --2- 172.21.15.52:0/616908979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8dfc101e50 0x7f8dfc073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:44.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.675+0000 7f8e03b84700 1 --2- 172.21.15.52:0/616908979 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8dfc1002d0 0x7f8dfc0748a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:44.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.675+0000 7f8e03b84700 1 --2- 172.21.15.52:0/616908979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dfc1014c0 0x7f8dfc072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:44.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.675+0000 7f8e03b84700 1 -- 172.21.15.52:0/616908979 >> 172.21.15.52:0/616908979 conn(0x7f8dfc0fb8d0 msgr2=0x7f8dfc0fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:44.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.675+0000 7f8e03b84700 1 -- 172.21.15.52:0/616908979 shutdown_connections 2023-11-27T15:17:44.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.675+0000 7f8e03b84700 1 -- 172.21.15.52:0/616908979 wait complete. 2023-11-27T15:17:44.692 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:17:44.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.814+0000 7f6bca0b7700 1 -- 172.21.15.52:0/3504371628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6bc4100d90 msgr2=0x7f6bc4103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:44.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.814+0000 7f6bca0b7700 1 --2- 172.21.15.52:0/3504371628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6bc4100d90 0x7f6bc4103170 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4009990 tx=0x7f6bb4009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:44.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.814+0000 7f6bca0b7700 1 -- 172.21.15.52:0/3504371628 shutdown_connections 2023-11-27T15:17:44.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.814+0000 7f6bca0b7700 1 --2- 172.21.15.52:0/3504371628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bc4105fd0 0x7f6bc41083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:44.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.814+0000 7f6bca0b7700 1 --2- 172.21.15.52:0/3504371628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6bc4100d90 0x7f6bc4103170 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:44.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.814+0000 7f6bca0b7700 1 --2- 172.21.15.52:0/3504371628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bc41036b0 0x7f6bc4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:44.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.814+0000 7f6bca0b7700 1 -- 172.21.15.52:0/3504371628 >> 172.21.15.52:0/3504371628 conn(0x7f6bc40fa750 msgr2=0x7f6bc40fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:44.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.814+0000 7f6bca0b7700 1 -- 172.21.15.52:0/3504371628 shutdown_connections 2023-11-27T15:17:44.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.814+0000 7f6bca0b7700 1 -- 172.21.15.52:0/3504371628 wait complete. 2023-11-27T15:17:44.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.815+0000 7f6bca0b7700 1 Processor -- start 2023-11-27T15:17:44.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.815+0000 7f6bca0b7700 1 -- start start 2023-11-27T15:17:44.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.815+0000 7f6bca0b7700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bc4100d90 0x7f6bc4195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:44.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.816+0000 7f6bca0b7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6bc41036b0 0x7f6bc4196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:44.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.816+0000 7f6bca0b7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bc4105fd0 0x7f6bc419d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:44.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.816+0000 7f6bca0b7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc419db70 con 0x7f6bc41036b0 2023-11-27T15:17:44.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.816+0000 7f6bca0b7700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc419dcb0 con 0x7f6bc4100d90 2023-11-27T15:17:44.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.816+0000 7f6bca0b7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc419de40 con 0x7f6bc4105fd0 2023-11-27T15:17:44.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.816+0000 7f6bc2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6bc41036b0 0x7f6bc4196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:44.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.816+0000 7f6bc2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6bc41036b0 0x7f6bc4196530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51576/0 (socket says 172.21.15.52:51576) 2023-11-27T15:17:44.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.816+0000 7f6bc37fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bc4100d90 0x7f6bc4195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:44.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.816+0000 7f6bc2ffd700 1 -- 172.21.15.52:0/1620092975 learned_addr learned my addr 172.21.15.52:0/1620092975 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:44.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.816+0000 7f6bc3fff700 1 --2- 172.21.15.52:0/1620092975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bc4105fd0 0x7f6bc419d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:44.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.816+0000 7f6bc2ffd700 1 -- 172.21.15.52:0/1620092975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bc4105fd0 msgr2=0x7f6bc419d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:44.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.816+0000 7f6bc2ffd700 1 --2- 172.21.15.52:0/1620092975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bc4105fd0 0x7f6bc419d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:44.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.816+0000 7f6bc2ffd700 1 -- 172.21.15.52:0/1620092975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bc4100d90 msgr2=0x7f6bc4195ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:44.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.816+0000 7f6bc2ffd700 1 --2- 172.21.15.52:0/1620092975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bc4100d90 0x7f6bc4195ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:44.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.816+0000 7f6bc2ffd700 1 -- 172.21.15.52:0/1620092975 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb40095f0 con 0x7f6bc41036b0 2023-11-27T15:17:44.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.816+0000 7f6bc37fe700 1 --2- 172.21.15.52:0/1620092975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bc4100d90 0x7f6bc4195ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:17:44.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.816+0000 7f6bc2ffd700 1 --2- 172.21.15.52:0/1620092975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6bc41036b0 0x7f6bc4196530 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6bb800e9a0 tx=0x7f6bb800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:44.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.818+0000 7f6bc0ff9700 1 -- 172.21.15.52:0/1620092975 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8009790 con 0x7f6bc41036b0 2023-11-27T15:17:44.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.818+0000 7f6bca0b7700 1 -- 172.21.15.52:0/1620092975 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bc410bc40 con 0x7f6bc41036b0 2023-11-27T15:17:44.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.818+0000 7f6bc0ff9700 1 -- 172.21.15.52:0/1620092975 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bb8010430 con 0x7f6bc41036b0 2023-11-27T15:17:44.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.818+0000 7f6bc0ff9700 1 -- 172.21.15.52:0/1620092975 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8018680 con 0x7f6bc41036b0 2023-11-27T15:17:44.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.818+0000 7f6bca0b7700 1 -- 172.21.15.52:0/1620092975 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bc410bab0 con 0x7f6bc41036b0 2023-11-27T15:17:44.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.819+0000 7f6bca0b7700 1 -- 172.21.15.52:0/1620092975 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bc40fc2c0 con 0x7f6bc41036b0 2023-11-27T15:17:44.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.820+0000 7f6bc0ff9700 1 -- 172.21.15.52:0/1620092975 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6bb80105a0 con 0x7f6bc41036b0 2023-11-27T15:17:44.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.820+0000 7f6bc0ff9700 1 --2- 172.21.15.52:0/1620092975 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6bac0a1150 0x7f6bac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:44.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.821+0000 7f6bc0ff9700 1 -- 172.21.15.52:0/1620092975 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6bb8014070 con 0x7f6bc41036b0 2023-11-27T15:17:44.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.821+0000 7f6bc37fe700 1 --2- 172.21.15.52:0/1620092975 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6bac0a1150 0x7f6bac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:44.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.821+0000 7f6bc37fe700 1 --2- 172.21.15.52:0/1620092975 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6bac0a1150 0x7f6bac0a3600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6bb400b2e0 tx=0x7f6bb4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:44.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:44.825+0000 7f6bc0ff9700 1 -- 172.21.15.52:0/1620092975 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6bb8051730 con 0x7f6bc41036b0 2023-11-27T15:17:45.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.073+0000 7f6bca0b7700 1 -- 172.21.15.52:0/1620092975 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6bc402cfe0 con 0x7f6bc41036b0 2023-11-27T15:17:45.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:44 smithi102 ceph-mon[100296]: pgmap v305: 97 pgs: 97 active+clean; 645 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2023-11-27T15:17:45.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:44 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3186393162' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:17:45.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:44 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/4233453545' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:17:45.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:44 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/616908979' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:17:45.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.074+0000 7f6bc0ff9700 1 -- 172.21.15.52:0/1620092975 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10804 (secure 0 0 0) 0x7f6bb8096220 con 0x7f6bc41036b0 2023-11-27T15:17:45.079 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:17:45.079 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2328881376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2328881376},{"type":"v1","addr":"172.21.15.52:6839","nonce":2328881376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2799551570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2799551570},{"type":"v1","addr":"172.21.15.52:6841","nonce":2799551570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:16:09.319630+0000","modified":"2023-11-27T15:16:14.742405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24494,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/215725838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":215725838},{"type":"v1","addr":"172.21.15.150:6839","nonce":215725838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3571830559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3571830559},{"type":"v1","addr":"172.21.15.102:6839","nonce":3571830559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:17:45.081 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.076+0000 7f6bca0b7700 1 -- 172.21.15.52:0/1620092975 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6bac0a1150 msgr2=0x7f6bac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:45.081 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.076+0000 7f6bca0b7700 1 --2- 172.21.15.52:0/1620092975 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6bac0a1150 0x7f6bac0a3600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6bb400b2e0 tx=0x7f6bb4005fb0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.081 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.076+0000 7f6bca0b7700 1 -- 172.21.15.52:0/1620092975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6bc41036b0 msgr2=0x7f6bc4196530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:45.081 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.076+0000 7f6bca0b7700 1 --2- 172.21.15.52:0/1620092975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6bc41036b0 0x7f6bc4196530 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6bb800e9a0 tx=0x7f6bb800ecb0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.081 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.076+0000 7f6bca0b7700 1 -- 172.21.15.52:0/1620092975 shutdown_connections 2023-11-27T15:17:45.081 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.076+0000 7f6bca0b7700 1 --2- 172.21.15.52:0/1620092975 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f6bac0a1150 0x7f6bac0a3600 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.081 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.076+0000 7f6bca0b7700 1 --2- 172.21.15.52:0/1620092975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bc4100d90 0x7f6bc4195ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.076+0000 7f6bca0b7700 1 --2- 172.21.15.52:0/1620092975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bc4105fd0 0x7f6bc419d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.076+0000 7f6bca0b7700 1 --2- 172.21.15.52:0/1620092975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6bc41036b0 0x7f6bc4196530 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.076+0000 7f6bca0b7700 1 -- 172.21.15.52:0/1620092975 >> 172.21.15.52:0/1620092975 conn(0x7f6bc40fa750 msgr2=0x7f6bc41019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:45.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.077+0000 7f6bca0b7700 1 -- 172.21.15.52:0/1620092975 shutdown_connections 2023-11-27T15:17:45.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.077+0000 7f6bca0b7700 1 -- 172.21.15.52:0/1620092975 wait complete. 2023-11-27T15:17:45.082 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 23 2023-11-27T15:17:45.092 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:44 smithi052 ceph-mon[99056]: pgmap v305: 97 pgs: 97 active+clean; 645 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2023-11-27T15:17:45.092 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:44 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3186393162' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:17:45.092 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:44 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/4233453545' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:17:45.092 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:44 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/616908979' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:17:45.093 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:17:45.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.215+0000 7fcbdab27700 1 -- 172.21.15.52:0/3403226913 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbd40ff170 msgr2=0x7fcbd40ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:45.222 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.215+0000 7fcbdab27700 1 --2- 172.21.15.52:0/3403226913 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbd40ff170 0x7fcbd40ff580 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8009990 tx=0x7fcbc8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.222 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.215+0000 7fcbdab27700 1 -- 172.21.15.52:0/3403226913 shutdown_connections 2023-11-27T15:17:45.222 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.215+0000 7fcbdab27700 1 --2- 172.21.15.52:0/3403226913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbd4100db0 0x7fcbd4107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.222 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.215+0000 7fcbdab27700 1 --2- 172.21.15.52:0/3403226913 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbd40ff170 0x7fcbd40ff580 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.215+0000 7fcbdab27700 1 --2- 172.21.15.52:0/3403226913 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbd4100400 0x7fcbd4100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.215+0000 7fcbdab27700 1 -- 172.21.15.52:0/3403226913 >> 172.21.15.52:0/3403226913 conn(0x7fcbd40fa750 msgr2=0x7fcbd40fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:45.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.215+0000 7fcbdab27700 1 -- 172.21.15.52:0/3403226913 shutdown_connections 2023-11-27T15:17:45.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.215+0000 7fcbdab27700 1 -- 172.21.15.52:0/3403226913 wait complete. 2023-11-27T15:17:45.223 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.215+0000 7fcbdab27700 1 Processor -- start 2023-11-27T15:17:45.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.216+0000 7fcbdab27700 1 -- start start 2023-11-27T15:17:45.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.216+0000 7fcbdab27700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbd40ff170 0x7fcbd4068940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.216+0000 7fcbdab27700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbd4100400 0x7fcbd419aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.216+0000 7fcbdab27700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbd4100db0 0x7fcbd419d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.224 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.216+0000 7fcbdab27700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd419da80 con 0x7fcbd4100400 2023-11-27T15:17:45.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.216+0000 7fcbdab27700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd419dbf0 con 0x7fcbd4100db0 2023-11-27T15:17:45.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.216+0000 7fcbdab27700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd419ddb0 con 0x7fcbd40ff170 2023-11-27T15:17:45.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.216+0000 7fcbd3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbd4100400 0x7fcbd419aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.216+0000 7fcbd3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbd4100400 0x7fcbd419aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51602/0 (socket says 172.21.15.52:51602) 2023-11-27T15:17:45.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.216+0000 7fcbd3fff700 1 -- 172.21.15.52:0/3792656876 learned_addr learned my addr 172.21.15.52:0/3792656876 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:45.226 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.216+0000 7fcbd90c4700 1 --2- 172.21.15.52:0/3792656876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbd4100db0 0x7fcbd419d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.226 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.216+0000 7fcbd88c3700 1 --2- 172.21.15.52:0/3792656876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbd40ff170 0x7fcbd4068940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.226 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.216+0000 7fcbd3fff700 1 -- 172.21.15.52:0/3792656876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbd40ff170 msgr2=0x7fcbd4068940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:45.226 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.216+0000 7fcbd3fff700 1 --2- 172.21.15.52:0/3792656876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbd40ff170 0x7fcbd4068940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.226 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.216+0000 7fcbd3fff700 1 -- 172.21.15.52:0/3792656876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbd4100db0 msgr2=0x7fcbd419d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:45.226 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.216+0000 7fcbd3fff700 1 --2- 172.21.15.52:0/3792656876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbd4100db0 0x7fcbd419d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.226 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.216+0000 7fcbd3fff700 1 -- 172.21.15.52:0/3792656876 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc80095f0 con 0x7fcbd4100400 2023-11-27T15:17:45.226 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.216+0000 7fcbd88c3700 1 --2- 172.21.15.52:0/3792656876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbd40ff170 0x7fcbd4068940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:17:45.226 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.216+0000 7fcbd90c4700 1 --2- 172.21.15.52:0/3792656876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbd4100db0 0x7fcbd419d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:17:45.226 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.216+0000 7fcbd3fff700 1 --2- 172.21.15.52:0/3792656876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbd4100400 0x7fcbd419aca0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcbc000d950 tx=0x7fcbc000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:45.227 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.216+0000 7fcbd1ffb700 1 -- 172.21.15.52:0/3792656876 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0009b40 con 0x7fcbd4100400 2023-11-27T15:17:45.227 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.216+0000 7fcbdab27700 1 -- 172.21.15.52:0/3792656876 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd410ab10 con 0x7fcbd4100400 2023-11-27T15:17:45.227 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.217+0000 7fcbd1ffb700 1 -- 172.21.15.52:0/3792656876 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbc0009ca0 con 0x7fcbd4100400 2023-11-27T15:17:45.227 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.217+0000 7fcbd1ffb700 1 -- 172.21.15.52:0/3792656876 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc000f620 con 0x7fcbd4100400 2023-11-27T15:17:45.227 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.217+0000 7fcbdab27700 1 -- 172.21.15.52:0/3792656876 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd410a980 con 0x7fcbd4100400 2023-11-27T15:17:45.228 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.218+0000 7fcbdab27700 1 -- 172.21.15.52:0/3792656876 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd419e4f0 con 0x7fcbd4100400 2023-11-27T15:17:45.228 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.219+0000 7fcbd1ffb700 1 -- 172.21.15.52:0/3792656876 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fcbc0010460 con 0x7fcbd4100400 2023-11-27T15:17:45.228 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.219+0000 7fcbd1ffb700 1 --2- 172.21.15.52:0/3792656876 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fcbc40a1150 0x7fcbc40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.228 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.219+0000 7fcbd1ffb700 1 -- 172.21.15.52:0/3792656876 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fcbc00cccb0 con 0x7fcbd4100400 2023-11-27T15:17:45.228 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.219+0000 7fcbd88c3700 1 --2- 172.21.15.52:0/3792656876 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fcbc40a1150 0x7fcbc40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.229 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.220+0000 7fcbd88c3700 1 --2- 172.21.15.52:0/3792656876 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fcbc40a1150 0x7fcbc40a3600 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcbc800b5c0 tx=0x7fcbc8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:45.229 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.224+0000 7fcbd1ffb700 1 -- 172.21.15.52:0/3792656876 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcbc00963e0 con 0x7fcbd4100400 2023-11-27T15:17:45.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:44 smithi150 ceph-mon[100559]: pgmap v305: 97 pgs: 97 active+clean; 645 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2023-11-27T15:17:45.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:44 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3186393162' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:17:45.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:44 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/4233453545' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:17:45.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:44 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/616908979' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:17:45.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.426+0000 7fcbdab27700 1 -- 172.21.15.52:0/3792656876 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcbd4106140 con 0x7fcbd4100400 2023-11-27T15:17:45.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.428+0000 7fcbd1ffb700 1 -- 172.21.15.52:0/3792656876 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22657 (secure 0 0 0) 0x7fcbc0020090 con 0x7fcbd4100400 2023-11-27T15:17:45.432 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:17:45.433 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":85,"fsid":"c6c7df88-8d36-11ee-95a2-87774f69a715","created":"2023-11-27T15:09:03.333345+0000","modified":"2023-11-27T15:17:04.797892+0000","last_up_change":"2023-11-27T15:15:27.656134+0000","last_in_change":"2023-11-27T15:15:10.671153+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:12:14.177523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:16:07.842251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:16:08.728520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58d69ede-c293-49f5-abbe-8187793f2b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6803","nonce":608216415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6805","nonce":608216415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6809","nonce":608216415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6807","nonce":608216415}]},"public_addr":"172.21.15.52:6803/608216415","cluster_addr":"172.21.15.52:6805/608216415","heartbeat_back_addr":"172.21.15.52:6809/608216415","heartbeat_front_addr":"172.21.15.52:6807/608216415","state":["exists","up"]},{"osd":1,"uuid":"d6a46b91-adc8-45ef-80f5-566ae400911f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6801","nonce":2538384087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6803","nonce":2538384087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6807","nonce":2538384087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6805","nonce":2538384087}]},"public_addr":"172.21.15.102:6801/2538384087","cluster_addr":"172.21.15.102:6803/2538384087","heartbeat_back_addr":"172.21.15.102:6807/2538384087","heartbeat_front_addr":"172.21.15.102:6805/2538384087","state":["exists","up"]},{"osd":2,"uuid":"846ba125-11f8-46b2-8941-bd5e1947cc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6801","nonce":4258038221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6803","nonce":4258038221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6807","nonce":4258038221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6805","nonce":4258038221}]},"public_addr":"172.21.15.150:6801/4258038221","cluster_addr":"172.21.15.150:6803/4258038221","heartbeat_back_addr":"172.21.15.150:6807/4258038221","heartbeat_front_addr":"172.21.15.150:6805/4258038221","state":["exists","up"]},{"osd":3,"uuid":"950fcd67-1057-4288-a389-714f1ec94261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6811","nonce":2622100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6813","nonce":2622100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6817","nonce":2622100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6815","nonce":2622100752}]},"public_addr":"172.21.15.52:6811/2622100752","cluster_addr":"172.21.15.52:6813/2622100752","heartbeat_back_addr":"172.21.15.52:6817/2622100752","heartbeat_front_addr":"172.21.15.52:6815/2622100752","state":["exists","up"]},{"osd":4,"uuid":"1c96a676-cea9-4deb-bad1-f7fd240c3d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6809","nonce":1546794763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6811","nonce":1546794763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6815","nonce":1546794763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6813","nonce":1546794763}]},"public_addr":"172.21.15.102:6809/1546794763","cluster_addr":"172.21.15.102:6811/1546794763","heartbeat_back_addr":"172.21.15.102:6815/1546794763","heartbeat_front_addr":"172.21.15.102:6813/1546794763","state":["exists","up"]},{"osd":5,"uuid":"38a75af1-fd4d-433e-b579-0609109c851c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6809","nonce":461511375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6811","nonce":461511375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6815","nonce":461511375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6813","nonce":461511375}]},"public_addr":"172.21.15.150:6809/461511375","cluster_addr":"172.21.15.150:6811/461511375","heartbeat_back_addr":"172.21.15.150:6815/461511375","heartbeat_front_addr":"172.21.15.150:6813/461511375","state":["exists","up"]},{"osd":6,"uuid":"2f0ede4a-d175-4b4a-b876-c0f442048901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6819","nonce":1002457464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6821","nonce":1002457464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6825","nonce":1002457464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6823","nonce":1002457464}]},"public_addr":"172.21.15.52:6819/1002457464","cluster_addr":"172.21.15.52:6821/1002457464","heartbeat_back_addr":"172.21.15.52:6825/1002457464","heartbeat_front_addr":"172.21.15.52:6823/1002457464","state":["exists","up"]},{"osd":7,"uuid":"6b635192-abcf-4bbf-bf78-a019b351bca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6817","nonce":3249840064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6819","nonce":3249840064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6823","nonce":3249840064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6821","nonce":3249840064}]},"public_addr":"172.21.15.102:6817/3249840064","cluster_addr":"172.21.15.102:6819/3249840064","heartbeat_back_addr":"172.21.15.102:6823/3249840064","heartbeat_front_addr":"172.21.15.102:6821/3249840064","state":["exists","up"]},{"osd":8,"uuid":"c7ca8007-7170-47bf-956c-3f00edf27e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6817","nonce":3672583574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6819","nonce":3672583574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6823","nonce":3672583574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6821","nonce":3672583574}]},"public_addr":"172.21.15.150:6817/3672583574","cluster_addr":"172.21.15.150:6819/3672583574","heartbeat_back_addr":"172.21.15.150:6823/3672583574","heartbeat_front_addr":"172.21.15.150:6821/3672583574","state":["exists","up"]},{"osd":9,"uuid":"0e252c2c-01a4-48ba-832d-067ed940fefb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6827","nonce":4287934320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6829","nonce":4287934320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6833","nonce":4287934320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6831","nonce":4287934320}]},"public_addr":"172.21.15.52:6827/4287934320","cluster_addr":"172.21.15.52:6829/4287934320","heartbeat_back_addr":"172.21.15.52:6833/4287934320","heartbeat_front_addr":"172.21.15.52:6831/4287934320","state":["exists","up"]},{"osd":10,"uuid":"bb540142-7563-44a4-b7a3-cb662714c128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6825","nonce":3056034201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6827","nonce":3056034201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6831","nonce":3056034201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6829","nonce":3056034201}]},"public_addr":"172.21.15.102:6825/3056034201","cluster_addr":"172.21.15.102:6827/3056034201","heartbeat_back_addr":"172.21.15.102:6831/3056034201","heartbeat_front_addr":"172.21.15.102:6829/3056034201","state":["exists","up"]},{"osd":11,"uuid":"3156528e-e0f8-4d44-9946-c8b6339c2bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6825","nonce":3207559739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6827","nonce":3207559739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6831","nonce":3207559739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6829","nonce":3207559739}]},"public_addr":"172.21.15.150:6825/3207559739","cluster_addr":"172.21.15.150:6827/3207559739","heartbeat_back_addr":"172.21.15.150:6831/3207559739","heartbeat_front_addr":"172.21.15.150:6829/3207559739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:26.826099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:48.331371+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:10.459738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:32.356472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:54.043170+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:15.648582+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:37.166463+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:59.240819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:20.710097+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:43.364563+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:05.279433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:26.344573+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":5,"to":7}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/1392282110":"2023-11-28T15:10:03.644870+0000","172.21.15.52:0/1479993479":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6800/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/961562652":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/1290665390":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2410529742":"2023-11-28T15:09:37.118690+0000","172.21.15.52:6800/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2130037044":"2023-11-28T15:10:03.644870+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:17:45.434 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.430+0000 7fcbdab27700 1 -- 172.21.15.52:0/3792656876 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fcbc40a1150 msgr2=0x7fcbc40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:45.434 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.430+0000 7fcbdab27700 1 --2- 172.21.15.52:0/3792656876 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fcbc40a1150 0x7fcbc40a3600 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcbc800b5c0 tx=0x7fcbc8005fb0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.435 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.430+0000 7fcbdab27700 1 -- 172.21.15.52:0/3792656876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbd4100400 msgr2=0x7fcbd419aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:45.435 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.430+0000 7fcbdab27700 1 --2- 172.21.15.52:0/3792656876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbd4100400 0x7fcbd419aca0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcbc000d950 tx=0x7fcbc000dd10 comp rx=0 tx=0).stop 2023-11-27T15:17:45.435 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.430+0000 7fcbdab27700 1 -- 172.21.15.52:0/3792656876 shutdown_connections 2023-11-27T15:17:45.435 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.430+0000 7fcbdab27700 1 --2- 172.21.15.52:0/3792656876 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fcbc40a1150 0x7fcbc40a3600 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.435 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.430+0000 7fcbdab27700 1 --2- 172.21.15.52:0/3792656876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbd4100db0 0x7fcbd419d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.435 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.430+0000 7fcbdab27700 1 --2- 172.21.15.52:0/3792656876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbd40ff170 0x7fcbd4068940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.436 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.430+0000 7fcbdab27700 1 --2- 172.21.15.52:0/3792656876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbd4100400 0x7fcbd419aca0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.436 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.430+0000 7fcbdab27700 1 -- 172.21.15.52:0/3792656876 >> 172.21.15.52:0/3792656876 conn(0x7fcbd40fa750 msgr2=0x7fcbd40fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:45.436 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.430+0000 7fcbdab27700 1 -- 172.21.15.52:0/3792656876 shutdown_connections 2023-11-27T15:17:45.436 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.430+0000 7fcbdab27700 1 -- 172.21.15.52:0/3792656876 wait complete. 2023-11-27T15:17:45.447 DEBUG:tasks.fwd_scrub:Yielding 2023-11-27T15:17:45.447 INFO:teuthology.run_tasks:Running task exec... 2023-11-27T15:17:45.458 INFO:teuthology.task.exec:Executing custom commands... 2023-11-27T15:17:45.458 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi052.front.sepia.ceph.com 2023-11-27T15:17:45.458 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2023-11-27T15:17:45.459 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-11-27T15:17:45.459 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-27T15:17:45.459 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-27T15:17:45.588 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.582+0000 7f7d5334a700 1 -- 172.21.15.52:0/2385230760 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d4c0feb30 msgr2=0x7f7d4c0fef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:45.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.582+0000 7f7d5334a700 1 --2- 172.21.15.52:0/2385230760 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d4c0feb30 0x7f7d4c0fef40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009990 tx=0x7f7d48009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.582+0000 7f7d5334a700 1 -- 172.21.15.52:0/2385230760 shutdown_connections 2023-11-27T15:17:45.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.582+0000 7f7d5334a700 1 --2- 172.21.15.52:0/2385230760 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d4c0ffe30 0x7f7d4c1002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.582+0000 7f7d5334a700 1 --2- 172.21.15.52:0/2385230760 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d4c0feb30 0x7f7d4c0fef40 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.582+0000 7f7d5334a700 1 --2- 172.21.15.52:0/2385230760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d4c0ff480 0x7f7d4c0ff8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.582+0000 7f7d5334a700 1 -- 172.21.15.52:0/2385230760 >> 172.21.15.52:0/2385230760 conn(0x7f7d4c0fa720 msgr2=0x7f7d4c0fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:45.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.582+0000 7f7d5334a700 1 -- 172.21.15.52:0/2385230760 shutdown_connections 2023-11-27T15:17:45.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.582+0000 7f7d5334a700 1 -- 172.21.15.52:0/2385230760 wait complete. 2023-11-27T15:17:45.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.582+0000 7f7d5334a700 1 Processor -- start 2023-11-27T15:17:45.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.582+0000 7f7d5334a700 1 -- start start 2023-11-27T15:17:45.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.583+0000 7f7d5334a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d4c0feb30 0x7f7d4c198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.583+0000 7f7d5334a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d4c0ff480 0x7f7d4c1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.583+0000 7f7d5334a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d4c0ffe30 0x7f7d4c19f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.583+0000 7f7d5334a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4c19fce0 con 0x7f7d4c0feb30 2023-11-27T15:17:45.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.583+0000 7f7d5334a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4c19fe20 con 0x7f7d4c0ff480 2023-11-27T15:17:45.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.583+0000 7f7d5334a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4c19ffb0 con 0x7f7d4c0ffe30 2023-11-27T15:17:45.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.583+0000 7f7d510e6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d4c0feb30 0x7f7d4c198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.583+0000 7f7d510e6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d4c0feb30 0x7f7d4c198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51622/0 (socket says 172.21.15.52:51622) 2023-11-27T15:17:45.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.583+0000 7f7d510e6700 1 -- 172.21.15.52:0/1401207324 learned_addr learned my addr 172.21.15.52:0/1401207324 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:45.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.583+0000 7f7d518e7700 1 --2- 172.21.15.52:0/1401207324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d4c0ffe30 0x7f7d4c19f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.583+0000 7f7d508e5700 1 --2- 172.21.15.52:0/1401207324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d4c0ff480 0x7f7d4c1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.583+0000 7f7d510e6700 1 -- 172.21.15.52:0/1401207324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d4c0ffe30 msgr2=0x7f7d4c19f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:45.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.583+0000 7f7d510e6700 1 --2- 172.21.15.52:0/1401207324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d4c0ffe30 0x7f7d4c19f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.583+0000 7f7d510e6700 1 -- 172.21.15.52:0/1401207324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d4c0ff480 msgr2=0x7f7d4c1986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:45.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.583+0000 7f7d510e6700 1 --2- 172.21.15.52:0/1401207324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d4c0ff480 0x7f7d4c1986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.583+0000 7f7d510e6700 1 -- 172.21.15.52:0/1401207324 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d480095f0 con 0x7f7d4c0feb30 2023-11-27T15:17:45.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.583+0000 7f7d518e7700 1 --2- 172.21.15.52:0/1401207324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d4c0ffe30 0x7f7d4c19f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:17:45.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.583+0000 7f7d510e6700 1 --2- 172.21.15.52:0/1401207324 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d4c0feb30 0x7f7d4c198160 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7d48004760 tx=0x7f7d48009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:45.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.583+0000 7f7d427fc700 1 -- 172.21.15.52:0/1401207324 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4801d070 con 0x7f7d4c0feb30 2023-11-27T15:17:45.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.583+0000 7f7d5334a700 1 -- 172.21.15.52:0/1401207324 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d4c10dd90 con 0x7f7d4c0feb30 2023-11-27T15:17:45.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.583+0000 7f7d427fc700 1 -- 172.21.15.52:0/1401207324 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d48004200 con 0x7f7d4c0feb30 2023-11-27T15:17:45.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.583+0000 7f7d427fc700 1 -- 172.21.15.52:0/1401207324 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d480049f0 con 0x7f7d4c0feb30 2023-11-27T15:17:45.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.584+0000 7f7d5334a700 1 -- 172.21.15.52:0/1401207324 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d4c10dc00 con 0x7f7d4c0feb30 2023-11-27T15:17:45.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.584+0000 7f7d5334a700 1 -- 172.21.15.52:0/1401207324 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d4c109910 con 0x7f7d4c0feb30 2023-11-27T15:17:45.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.587+0000 7f7d427fc700 1 -- 172.21.15.52:0/1401207324 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7d48003d40 con 0x7f7d4c0feb30 2023-11-27T15:17:45.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.587+0000 7f7d427fc700 1 --2- 172.21.15.52:0/1401207324 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7d380a1150 0x7f7d380a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.588+0000 7f7d508e5700 1 --2- 172.21.15.52:0/1401207324 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7d380a1150 0x7f7d380a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.588+0000 7f7d427fc700 1 -- 172.21.15.52:0/1401207324 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7d480cdca0 con 0x7f7d4c0feb30 2023-11-27T15:17:45.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.588+0000 7f7d508e5700 1 --2- 172.21.15.52:0/1401207324 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7d380a1150 0x7f7d380a3600 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c005950 tx=0x7f7d3c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:45.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.590+0000 7f7d427fc700 1 -- 172.21.15.52:0/1401207324 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7d48097460 con 0x7f7d4c0feb30 2023-11-27T15:17:45.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.591+0000 7f7dd56ab700 1 -- 172.21.15.52:0/2915889621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dd01003b0 msgr2=0x7f7dd0100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:45.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.591+0000 7f7dd56ab700 1 --2- 172.21.15.52:0/2915889621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dd01003b0 0x7f7dd0100800 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4009990 tx=0x7f7dc4009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.591+0000 7f7dd56ab700 1 -- 172.21.15.52:0/2915889621 shutdown_connections 2023-11-27T15:17:45.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.591+0000 7f7dd56ab700 1 --2- 172.21.15.52:0/2915889621 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dd0100d40 0x7f7dd0107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.591+0000 7f7dd56ab700 1 --2- 172.21.15.52:0/2915889621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7dd00ff1c0 0x7f7dd00ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.591+0000 7f7dd56ab700 1 --2- 172.21.15.52:0/2915889621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dd01003b0 0x7f7dd0100800 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.591+0000 7f7dd56ab700 1 -- 172.21.15.52:0/2915889621 >> 172.21.15.52:0/2915889621 conn(0x7f7dd00fa7a0 msgr2=0x7f7dd00fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:45.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.591+0000 7f7dd56ab700 1 -- 172.21.15.52:0/2915889621 shutdown_connections 2023-11-27T15:17:45.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.591+0000 7f7dd56ab700 1 -- 172.21.15.52:0/2915889621 wait complete. 2023-11-27T15:17:45.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.592+0000 7f7dd56ab700 1 Processor -- start 2023-11-27T15:17:45.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.592+0000 7f7dd56ab700 1 -- start start 2023-11-27T15:17:45.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.593+0000 7f7dd56ab700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dd00ff1c0 0x7f7dd0069c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.593+0000 7f7dd56ab700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7dd01003b0 0x7f7dd019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.593+0000 7f7dd56ab700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dd0100d40 0x7f7dd019d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.593+0000 7f7dd56ab700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd019db90 con 0x7f7dd00ff1c0 2023-11-27T15:17:45.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.593+0000 7f7dd56ab700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd019dd00 con 0x7f7dd0100d40 2023-11-27T15:17:45.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.593+0000 7f7dd56ab700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd019dec0 con 0x7f7dd01003b0 2023-11-27T15:17:45.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.593+0000 7f7dceffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dd00ff1c0 0x7f7dd0069c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.593+0000 7f7dceffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dd00ff1c0 0x7f7dd0069c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51636/0 (socket says 172.21.15.52:51636) 2023-11-27T15:17:45.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.593+0000 7f7dceffd700 1 -- 172.21.15.52:0/535152460 learned_addr learned my addr 172.21.15.52:0/535152460 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:45.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.593+0000 7f7dce7fc700 1 --2- 172.21.15.52:0/535152460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7dd01003b0 0x7f7dd019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.593+0000 7f7dcf7fe700 1 --2- 172.21.15.52:0/535152460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dd0100d40 0x7f7dd019d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.593+0000 7f7dceffd700 1 -- 172.21.15.52:0/535152460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7dd01003b0 msgr2=0x7f7dd019ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:45.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.593+0000 7f7dceffd700 1 --2- 172.21.15.52:0/535152460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7dd01003b0 0x7f7dd019ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.593+0000 7f7dceffd700 1 -- 172.21.15.52:0/535152460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dd0100d40 msgr2=0x7f7dd019d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:45.601 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.593+0000 7f7dceffd700 1 --2- 172.21.15.52:0/535152460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dd0100d40 0x7f7dd019d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.601 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.593+0000 7f7dceffd700 1 -- 172.21.15.52:0/535152460 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc40095f0 con 0x7f7dd00ff1c0 2023-11-27T15:17:45.601 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.593+0000 7f7dceffd700 1 --2- 172.21.15.52:0/535152460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dd00ff1c0 0x7f7dd0069c90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7dc000e970 tx=0x7f7dc000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:45.601 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.594+0000 7f7db7fff700 1 -- 172.21.15.52:0/535152460 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0009790 con 0x7f7dd00ff1c0 2023-11-27T15:17:45.601 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.594+0000 7f7dd56ab700 1 -- 172.21.15.52:0/535152460 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dd010ab70 con 0x7f7dd00ff1c0 2023-11-27T15:17:45.601 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.594+0000 7f7db7fff700 1 -- 172.21.15.52:0/535152460 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dc000eec0 con 0x7f7dd00ff1c0 2023-11-27T15:17:45.601 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.594+0000 7f7db7fff700 1 -- 172.21.15.52:0/535152460 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0003cf0 con 0x7f7dd00ff1c0 2023-11-27T15:17:45.601 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.594+0000 7f7dd56ab700 1 -- 172.21.15.52:0/535152460 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd010a9e0 con 0x7f7dd00ff1c0 2023-11-27T15:17:45.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.596+0000 7f7db7fff700 1 -- 172.21.15.52:0/535152460 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7dc000ccd0 con 0x7f7dd00ff1c0 2023-11-27T15:17:45.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.596+0000 7f7db7fff700 1 --2- 172.21.15.52:0/535152460 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7db80a1150 0x7f7db80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.597+0000 7f7db7fff700 1 -- 172.21.15.52:0/535152460 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7dc0014070 con 0x7f7dd00ff1c0 2023-11-27T15:17:45.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.597+0000 7f7dce7fc700 1 --2- 172.21.15.52:0/535152460 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7db80a1150 0x7f7db80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.597+0000 7f7dce7fc700 1 --2- 172.21.15.52:0/535152460 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7db80a1150 0x7f7db80a3600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0069a00 tx=0x7f7dc4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:45.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.597+0000 7f7de1837b80 1 Processor -- start 2023-11-27T15:17:45.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.597+0000 7f7de1837b80 1 -- start start 2023-11-27T15:17:45.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.598+0000 7f7de1837b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d4ec710c80 0x55d4ec711090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.598+0000 7f7de1837b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4ec7115d0 0x55d4ec711bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.598+0000 7f7de1837b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4ec712190 0x55d4ec712600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.598+0000 7f7de1837b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ec6160e0 con 0x55d4ec712190 2023-11-27T15:17:45.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.598+0000 7f7de1837b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ec668920 con 0x55d4ec7115d0 2023-11-27T15:17:45.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.598+0000 7f7de1837b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ec661db0 con 0x55d4ec710c80 2023-11-27T15:17:45.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.598+0000 7f7dcf7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4ec712190 0x55d4ec712600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.598+0000 7f7dcf7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4ec712190 0x55d4ec712600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51648/0 (socket says 172.21.15.52:51648) 2023-11-27T15:17:45.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.598+0000 7f7dcf7fe700 1 -- 172.21.15.52:0/1670921118 learned_addr learned my addr 172.21.15.52:0/1670921118 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:45.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.598+0000 7f7dce7fc700 1 --2- 172.21.15.52:0/1670921118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4ec7115d0 0x55d4ec711bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.598+0000 7f7dceffd700 1 --2- 172.21.15.52:0/1670921118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d4ec710c80 0x55d4ec711090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.598+0000 7f7dcf7fe700 1 -- 172.21.15.52:0/1670921118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d4ec710c80 msgr2=0x55d4ec711090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:45.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.598+0000 7f7dcf7fe700 1 --2- 172.21.15.52:0/1670921118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d4ec710c80 0x55d4ec711090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.598+0000 7f7dcf7fe700 1 -- 172.21.15.52:0/1670921118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4ec7115d0 msgr2=0x55d4ec711bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:45.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.598+0000 7f7dcf7fe700 1 --2- 172.21.15.52:0/1670921118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4ec7115d0 0x55d4ec711bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.598+0000 7f7dcf7fe700 1 -- 172.21.15.52:0/1670921118 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4ec4e36b0 con 0x55d4ec712190 2023-11-27T15:17:45.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.598+0000 7f7dceffd700 1 --2- 172.21.15.52:0/1670921118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d4ec710c80 0x55d4ec711090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:17:45.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.598+0000 7f7dcf7fe700 1 --2- 172.21.15.52:0/1670921118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4ec712190 0x55d4ec712600 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8009440 tx=0x7f7dc8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90983267adce78d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:45.612 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.599+0000 7f7db57fa700 1 -- 172.21.15.52:0/1670921118 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc800e070 con 0x55d4ec712190 2023-11-27T15:17:45.612 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.599+0000 7f7db57fa700 1 -- 172.21.15.52:0/1670921118 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dc800f040 con 0x55d4ec712190 2023-11-27T15:17:45.612 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.599+0000 7f7db57fa700 1 -- 172.21.15.52:0/1670921118 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8015ac0 con 0x55d4ec712190 2023-11-27T15:17:45.612 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.599+0000 7f7de1837b80 1 -- 172.21.15.52:0/1670921118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4ec712190 msgr2=0x55d4ec712600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:45.612 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.599+0000 7f7de1837b80 1 --2- 172.21.15.52:0/1670921118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4ec712190 0x55d4ec712600 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8009440 tx=0x7f7dc8009920 comp rx=0 tx=0).stop 2023-11-27T15:17:45.612 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.599+0000 7f7de1837b80 1 -- 172.21.15.52:0/1670921118 shutdown_connections 2023-11-27T15:17:45.612 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.599+0000 7f7de1837b80 1 --2- 172.21.15.52:0/1670921118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4ec712190 0x55d4ec712600 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.612 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.599+0000 7f7de1837b80 1 --2- 172.21.15.52:0/1670921118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d4ec710c80 0x55d4ec711090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.599+0000 7f7de1837b80 1 --2- 172.21.15.52:0/1670921118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4ec7115d0 0x55d4ec711bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.599+0000 7f7de1837b80 1 -- 172.21.15.52:0/1670921118 >> 172.21.15.52:0/1670921118 conn(0x55d4ec7063e0 msgr2=0x55d4ec70b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:45.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.599+0000 7f7de1837b80 1 -- 172.21.15.52:0/1670921118 shutdown_connections 2023-11-27T15:17:45.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.599+0000 7f7de1837b80 1 -- 172.21.15.52:0/1670921118 wait complete. 2023-11-27T15:17:45.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.600+0000 7f7de1837b80 1 -- start start 2023-11-27T15:17:45.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.600+0000 7f7de1837b80 1 Processor -- start 2023-11-27T15:17:45.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.600+0000 7f7de1837b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4ec710c80 0x55d4ec71a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.600+0000 7f7de1837b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d4ec7115d0 0x55d4ec71ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.600+0000 7f7de1837b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4ec712190 0x55d4ec721bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.600+0000 7f7de1837b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ec603960 con 0x55d4ec710c80 2023-11-27T15:17:45.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.600+0000 7f7de1837b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ec603600 con 0x55d4ec712190 2023-11-27T15:17:45.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.600+0000 7f7de1837b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ec4e59a0 con 0x55d4ec7115d0 2023-11-27T15:17:45.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.600+0000 7f7de1837b80 10 client.0 _ll_register_callbacks cb 0x55d4ec621800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:17:45.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.600+0000 7f7dceffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4ec710c80 0x55d4ec71a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.600+0000 7f7dceffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4ec710c80 0x55d4ec71a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51650/0 (socket says 172.21.15.52:51650) 2023-11-27T15:17:45.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.600+0000 7f7dceffd700 1 -- 172.21.15.52:0/3043449457 learned_addr learned my addr 172.21.15.52:0/3043449457 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:45.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.600+0000 7f7dcf7fe700 1 --2- 172.21.15.52:0/3043449457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4ec712190 0x55d4ec721bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.600+0000 7f7dce7fc700 1 --2- 172.21.15.52:0/3043449457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d4ec7115d0 0x55d4ec71ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.601+0000 7f7dceffd700 1 -- 172.21.15.52:0/3043449457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d4ec7115d0 msgr2=0x55d4ec71ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:45.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.601+0000 7f7dceffd700 1 --2- 172.21.15.52:0/3043449457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d4ec7115d0 0x55d4ec71ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.601+0000 7f7dceffd700 1 -- 172.21.15.52:0/3043449457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4ec712190 msgr2=0x55d4ec721bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:45.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.601+0000 7f7dceffd700 1 --2- 172.21.15.52:0/3043449457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4ec712190 0x55d4ec721bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.601+0000 7f7dceffd700 1 -- 172.21.15.52:0/3043449457 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc80090f0 con 0x55d4ec710c80 2023-11-27T15:17:45.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.601+0000 7f7dceffd700 1 --2- 172.21.15.52:0/3043449457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4ec710c80 0x55d4ec71a630 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0099f00 tx=0x7f7dc0016400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4a492ed0ece56 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:45.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.601+0000 7f7db2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:17:45.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.601+0000 7f7db2ff5700 1 -- 172.21.15.52:0/3043449457 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0099660 con 0x55d4ec710c80 2023-11-27T15:17:45.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.601+0000 7f7db2ff5700 1 -- 172.21.15.52:0/3043449457 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7dc00a23c0 con 0x55d4ec710c80 2023-11-27T15:17:45.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.601+0000 7f7de1837b80 1 -- 172.21.15.52:0/3043449457 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d4ec703ba0 con 0x55d4ec710c80 2023-11-27T15:17:45.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.601+0000 7f7db2ff5700 1 -- 172.21.15.52:0/3043449457 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7dc00a1f30 con 0x55d4ec710c80 2023-11-27T15:17:45.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.601+0000 7f7de1837b80 10 client.15345 fetch_fsmap learned FSMap version 23 2023-11-27T15:17:45.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.601+0000 7f7de1837b80 1 -- 172.21.15.52:0/3043449457 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d4ec628210 con 0x55d4ec710c80 2023-11-27T15:17:45.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.601+0000 7f7db2ff5700 1 -- 172.21.15.52:0/3043449457 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d4ec703ba0 con 0x55d4ec710c80 2023-11-27T15:17:45.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.601+0000 7f7db2ff5700 1 -- 172.21.15.52:0/3043449457 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7dc0094c60 con 0x55d4ec710c80 2023-11-27T15:17:45.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.601+0000 7f7de1837b80 10 client.15345 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:17:45.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.601+0000 7f7de1837b80 10 client.15345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:17:45.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.602+0000 7f7de1837b80 20 client.15345 populate_metadata read hostname 'smithi052' 2023-11-27T15:17:45.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.602+0000 7f7de1837b80 1 --2- 172.21.15.52:0/3043449457 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d4ec726360 0x55d4ec728770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.602+0000 7f7de1837b80 4 client.15345 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:17:45.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.602+0000 7f7de1837b80 1 -- 172.21.15.52:0/3043449457 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d4ec624120 con 0x55d4ec726360 2023-11-27T15:17:45.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.602+0000 7f7dce7fc700 1 --2- 172.21.15.52:0/3043449457 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d4ec726360 0x55d4ec728770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.602+0000 7f7db2ff5700 10 client.15345 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:17:45.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.602+0000 7f7dce7fc700 1 --2- 172.21.15.52:0/3043449457 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d4ec726360 0x55d4ec728770 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91de58b171aad8fa server_cookie=837b35e58e345740 in_seq=0 out_seq=0 2023-11-27T15:17:45.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.603+0000 7f7db2ff5700 1 -- 172.21.15.52:0/3043449457 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d4ec624120 con 0x55d4ec726360 2023-11-27T15:17:45.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.603+0000 7f7db2ff5700 10 client.15345 handle_command_reply: tid=0 2023-11-27T15:17:45.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.603+0000 7f7de1837b80 1 client.15345 shutdown 2023-11-27T15:17:45.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.603+0000 7f7de1837b80 1 -- 172.21.15.52:0/3043449457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4ec710c80 msgr2=0x55d4ec71a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:45.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.603+0000 7f7de1837b80 1 --2- 172.21.15.52:0/3043449457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4ec710c80 0x55d4ec71a630 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0099f00 tx=0x7f7dc0016400 comp rx=0 tx=0).stop 2023-11-27T15:17:45.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.603+0000 7f7de1837b80 1 -- 172.21.15.52:0/3043449457 shutdown_connections 2023-11-27T15:17:45.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.603+0000 7f7de1837b80 1 --2- 172.21.15.52:0/3043449457 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d4ec726360 0x55d4ec728770 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.603+0000 7f7de1837b80 1 --2- 172.21.15.52:0/3043449457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4ec712190 0x55d4ec721bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.603+0000 7f7de1837b80 1 --2- 172.21.15.52:0/3043449457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4ec710c80 0x55d4ec71a630 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.603+0000 7f7de1837b80 1 --2- 172.21.15.52:0/3043449457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d4ec7115d0 0x55d4ec71ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.603+0000 7f7db2ff5700 0 client.15345 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:17:45.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.603+0000 7f7de1837b80 1 -- 172.21.15.52:0/3043449457 >> 172.21.15.52:0/3043449457 conn(0x55d4ec7063e0 msgr2=0x55d4ec70ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:45.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.603+0000 7f7de1837b80 1 -- 172.21.15.52:0/3043449457 shutdown_connections 2023-11-27T15:17:45.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.603+0000 7f7de1837b80 1 -- 172.21.15.52:0/3043449457 wait complete. 2023-11-27T15:17:45.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.603+0000 7f7de1837b80 20 client.15345 trim_cache size 0 max 16384 2023-11-27T15:17:45.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.621+0000 7f7de1837b80 1 Processor -- start 2023-11-27T15:17:45.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.621+0000 7f7de1837b80 1 -- start start 2023-11-27T15:17:45.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.621+0000 7f7de1837b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4ec710c80 0x55d4ec71a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.621+0000 7f7de1837b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4ec7115d0 0x55d4ec71a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.621+0000 7f7de1837b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d4ec712190 0x55d4ec70a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.621+0000 7f7de1837b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ec70cb60 con 0x55d4ec7115d0 2023-11-27T15:17:45.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.621+0000 7f7de1837b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ec721f30 con 0x55d4ec710c80 2023-11-27T15:17:45.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.621+0000 7f7de1837b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ec70c1e0 con 0x55d4ec712190 2023-11-27T15:17:45.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.621+0000 7f7dce7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4ec7115d0 0x55d4ec71a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.621+0000 7f7dce7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4ec7115d0 0x55d4ec71a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51660/0 (socket says 172.21.15.52:51660) 2023-11-27T15:17:45.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.621+0000 7f7dce7fc700 1 -- 172.21.15.52:0/457098507 learned_addr learned my addr 172.21.15.52:0/457098507 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:45.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.621+0000 7f7dceffd700 1 --2- 172.21.15.52:0/457098507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4ec710c80 0x55d4ec71a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.621+0000 7f7dcf7fe700 1 --2- 172.21.15.52:0/457098507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d4ec712190 0x55d4ec70a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.621+0000 7f7dce7fc700 1 -- 172.21.15.52:0/457098507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d4ec712190 msgr2=0x55d4ec70a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:45.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.621+0000 7f7dce7fc700 1 --2- 172.21.15.52:0/457098507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d4ec712190 0x55d4ec70a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.621+0000 7f7dce7fc700 1 -- 172.21.15.52:0/457098507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4ec710c80 msgr2=0x55d4ec71a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:45.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.621+0000 7f7dce7fc700 1 --2- 172.21.15.52:0/457098507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4ec710c80 0x55d4ec71a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.621+0000 7f7dce7fc700 1 -- 172.21.15.52:0/457098507 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4ec707b80 con 0x55d4ec7115d0 2023-11-27T15:17:45.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.621+0000 7f7dceffd700 1 --2- 172.21.15.52:0/457098507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4ec710c80 0x55d4ec71a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:17:45.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.621+0000 7f7dce7fc700 1 --2- 172.21.15.52:0/457098507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4ec7115d0 0x55d4ec71a920 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4015000 tx=0x7f7dc4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a9c7576e18d279 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:45.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.622+0000 7f7db5ffb700 1 -- 172.21.15.52:0/457098507 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4035ae0 con 0x55d4ec7115d0 2023-11-27T15:17:45.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.622+0000 7f7db5ffb700 1 -- 172.21.15.52:0/457098507 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dc401e040 con 0x55d4ec7115d0 2023-11-27T15:17:45.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.622+0000 7f7db5ffb700 1 -- 172.21.15.52:0/457098507 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc400fcd0 con 0x55d4ec7115d0 2023-11-27T15:17:45.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.622+0000 7f7de1837b80 1 -- 172.21.15.52:0/457098507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4ec7115d0 msgr2=0x55d4ec71a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:45.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.622+0000 7f7de1837b80 1 --2- 172.21.15.52:0/457098507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4ec7115d0 0x55d4ec71a920 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4015000 tx=0x7f7dc4020000 comp rx=0 tx=0).stop 2023-11-27T15:17:45.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.622+0000 7f7de1837b80 1 -- 172.21.15.52:0/457098507 shutdown_connections 2023-11-27T15:17:45.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.622+0000 7f7de1837b80 1 --2- 172.21.15.52:0/457098507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4ec710c80 0x55d4ec71a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.622+0000 7f7de1837b80 1 --2- 172.21.15.52:0/457098507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d4ec712190 0x55d4ec70a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.622+0000 7f7de1837b80 1 --2- 172.21.15.52:0/457098507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4ec7115d0 0x55d4ec71a920 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.622+0000 7f7de1837b80 1 -- 172.21.15.52:0/457098507 >> 172.21.15.52:0/457098507 conn(0x55d4ec7063e0 msgr2=0x55d4ec715000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:45.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.622+0000 7f7de1837b80 1 -- 172.21.15.52:0/457098507 shutdown_connections 2023-11-27T15:17:45.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.622+0000 7f7de1837b80 1 -- 172.21.15.52:0/457098507 wait complete. 2023-11-27T15:17:45.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.623+0000 7f7de1837b80 1 -- start start 2023-11-27T15:17:45.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.623+0000 7f7de1837b80 1 Processor -- start 2023-11-27T15:17:45.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.623+0000 7f7de1837b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d4ec710c80 0x55d4ec776b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.623+0000 7f7de1837b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4ec7115d0 0x55d4ec777060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.623+0000 7f7de1837b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4ec712190 0x55d4ec77e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.623+0000 7f7de1837b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ec77e5b0 con 0x55d4ec7115d0 2023-11-27T15:17:45.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.623+0000 7f7de1837b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ec77e6f0 con 0x55d4ec712190 2023-11-27T15:17:45.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.623+0000 7f7de1837b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ec77e880 con 0x55d4ec710c80 2023-11-27T15:17:45.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.623+0000 7f7de1837b80 10 client.0 _ll_register_callbacks cb 0x7f7dc0099ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:17:45.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.623+0000 7f7dce7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4ec7115d0 0x55d4ec777060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.623+0000 7f7dce7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4ec7115d0 0x55d4ec777060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51666/0 (socket says 172.21.15.52:51666) 2023-11-27T15:17:45.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.623+0000 7f7dce7fc700 1 -- 172.21.15.52:0/2157602514 learned_addr learned my addr 172.21.15.52:0/2157602514 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:45.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.623+0000 7f7dceffd700 1 --2- 172.21.15.52:0/2157602514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d4ec710c80 0x55d4ec776b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.623+0000 7f7dcf7fe700 1 --2- 172.21.15.52:0/2157602514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4ec712190 0x55d4ec77e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.623+0000 7f7dce7fc700 1 -- 172.21.15.52:0/2157602514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d4ec710c80 msgr2=0x55d4ec776b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:45.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.623+0000 7f7dce7fc700 1 --2- 172.21.15.52:0/2157602514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d4ec710c80 0x55d4ec776b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.623+0000 7f7dce7fc700 1 -- 172.21.15.52:0/2157602514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4ec712190 msgr2=0x55d4ec77e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:45.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.624+0000 7f7dce7fc700 1 --2- 172.21.15.52:0/2157602514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4ec712190 0x55d4ec77e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.624+0000 7f7dce7fc700 1 -- 172.21.15.52:0/2157602514 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd0107f30 con 0x55d4ec7115d0 2023-11-27T15:17:45.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.624+0000 7f7dce7fc700 1 --2- 172.21.15.52:0/2157602514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4ec7115d0 0x55d4ec777060 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4009990 tx=0x7f7dc4035ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19427be4529fa31 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:45.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.624+0000 7f7db2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:17:45.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.624+0000 7f7db2ff5700 1 -- 172.21.15.52:0/2157602514 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc400fe60 con 0x55d4ec7115d0 2023-11-27T15:17:45.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.624+0000 7f7db2ff5700 1 -- 172.21.15.52:0/2157602514 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7dc40350f0 con 0x55d4ec7115d0 2023-11-27T15:17:45.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.624+0000 7f7de1837b80 1 -- 172.21.15.52:0/2157602514 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7dc800f040 con 0x55d4ec7115d0 2023-11-27T15:17:45.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.624+0000 7f7db2ff5700 1 -- 172.21.15.52:0/2157602514 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7dc40352b0 con 0x55d4ec7115d0 2023-11-27T15:17:45.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.624+0000 7f7de1837b80 10 client.15351 fetch_fsmap learned FSMap version 23 2023-11-27T15:17:45.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.624+0000 7f7de1837b80 1 -- 172.21.15.52:0/2157602514 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d4ec48fe60 con 0x55d4ec7115d0 2023-11-27T15:17:45.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.624+0000 7f7db2ff5700 1 -- 172.21.15.52:0/2157602514 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dc800f040 con 0x55d4ec7115d0 2023-11-27T15:17:45.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.624+0000 7f7db2ff5700 1 -- 172.21.15.52:0/2157602514 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7dc4024410 con 0x55d4ec7115d0 2023-11-27T15:17:45.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.624+0000 7f7de1837b80 10 client.15351 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:17:45.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.624+0000 7f7de1837b80 10 client.15351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:17:45.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.624+0000 7f7de1837b80 20 client.15351 populate_metadata read hostname 'smithi052' 2023-11-27T15:17:45.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.624+0000 7f7de1837b80 1 --2- 172.21.15.52:0/2157602514 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d4ec77fd70 0x55d4ec782150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.624+0000 7f7de1837b80 4 client.15351 mds_command: new command op to 24536 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-27T15:17:45.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.624+0000 7f7de1837b80 1 -- 172.21.15.52:0/2157602514 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d4ec6306e0 con 0x55d4ec77fd70 2023-11-27T15:17:45.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.625+0000 7f7dceffd700 1 --2- 172.21.15.52:0/2157602514 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d4ec77fd70 0x55d4ec782150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.625+0000 7f7db2ff5700 10 client.15351 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:17:45.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.625+0000 7f7dceffd700 1 --2- 172.21.15.52:0/2157602514 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d4ec77fd70 0x55d4ec782150 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe6eb8e1b4a0ee73 server_cookie=8891529fd74bd83b in_seq=0 out_seq=0 2023-11-27T15:17:45.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.626+0000 7f7db2ff5700 1 -- 172.21.15.52:0/2157602514 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d4ec6306e0 con 0x55d4ec77fd70 2023-11-27T15:17:45.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.626+0000 7f7db2ff5700 10 client.15351 handle_command_reply: tid=0 2023-11-27T15:17:45.638 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.626+0000 7f7de1837b80 1 client.15351 shutdown 2023-11-27T15:17:45.638 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.627+0000 7f7de1837b80 1 -- 172.21.15.52:0/2157602514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4ec7115d0 msgr2=0x55d4ec777060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:45.638 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.627+0000 7f7de1837b80 1 --2- 172.21.15.52:0/2157602514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4ec7115d0 0x55d4ec777060 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4009990 tx=0x7f7dc4035ae0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.638 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.627+0000 7f7de1837b80 1 -- 172.21.15.52:0/2157602514 shutdown_connections 2023-11-27T15:17:45.638 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.627+0000 7f7de1837b80 1 --2- 172.21.15.52:0/2157602514 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d4ec77fd70 0x55d4ec782150 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.638 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.627+0000 7f7de1837b80 1 --2- 172.21.15.52:0/2157602514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d4ec712190 0x55d4ec77e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.638 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.627+0000 7f7de1837b80 1 --2- 172.21.15.52:0/2157602514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d4ec710c80 0x55d4ec776b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.638 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.627+0000 7f7de1837b80 1 --2- 172.21.15.52:0/2157602514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4ec7115d0 0x55d4ec777060 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.638 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.627+0000 7f7de1837b80 1 -- 172.21.15.52:0/2157602514 >> 172.21.15.52:0/2157602514 conn(0x55d4ec7063e0 msgr2=0x55d4ec71a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:45.638 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.627+0000 7f7db2ff5700 0 client.15351 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:17:45.638 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.627+0000 7f7de1837b80 1 -- 172.21.15.52:0/2157602514 shutdown_connections 2023-11-27T15:17:45.639 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.627+0000 7f7de1837b80 1 -- 172.21.15.52:0/2157602514 wait complete. 2023-11-27T15:17:45.639 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.627+0000 7f7de1837b80 20 client.15351 trim_cache size 0 max 16384 2023-11-27T15:17:45.639 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:17:45.639 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2023-11-27T15:17:45.639 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "ccdf23fc-1f9f-4bf6-b299-7bc6c9a580fb", 2023-11-27T15:17:45.639 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2023-11-27T15:17:45.639 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:17:45.640 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.627+0000 7f7dd56ab700 1 -- 172.21.15.52:0/535152460 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7db80a1150 msgr2=0x7f7db80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:45.640 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.627+0000 7f7dd56ab700 1 --2- 172.21.15.52:0/535152460 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7db80a1150 0x7f7db80a3600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0069a00 tx=0x7f7dc4005fb0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.640 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.627+0000 7f7dd56ab700 1 -- 172.21.15.52:0/535152460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dd00ff1c0 msgr2=0x7f7dd0069c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:45.640 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.627+0000 7f7dd56ab700 1 --2- 172.21.15.52:0/535152460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dd00ff1c0 0x7f7dd0069c90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7dc000e970 tx=0x7f7dc000ec80 comp rx=0 tx=0).stop 2023-11-27T15:17:45.640 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.627+0000 7f7dd56ab700 1 -- 172.21.15.52:0/535152460 shutdown_connections 2023-11-27T15:17:45.640 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.627+0000 7f7dd56ab700 1 --2- 172.21.15.52:0/535152460 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7db80a1150 0x7f7db80a3600 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.641 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.627+0000 7f7dd56ab700 1 --2- 172.21.15.52:0/535152460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7dd0100d40 0x7f7dd019d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.641 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.627+0000 7f7dd56ab700 1 --2- 172.21.15.52:0/535152460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dd00ff1c0 0x7f7dd0069c90 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.641 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.627+0000 7f7dd56ab700 1 --2- 172.21.15.52:0/535152460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7dd01003b0 0x7f7dd019ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.641 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.628+0000 7f7dd56ab700 1 -- 172.21.15.52:0/535152460 >> 172.21.15.52:0/535152460 conn(0x7f7dd00fa7a0 msgr2=0x7f7dd00fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:45.641 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.628+0000 7f7dd56ab700 1 -- 172.21.15.52:0/535152460 shutdown_connections 2023-11-27T15:17:45.641 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.628+0000 7f7dd56ab700 1 -- 172.21.15.52:0/535152460 wait complete. 2023-11-27T15:17:45.644 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:17:45.775 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.768+0000 7f0baebe9700 1 -- 172.21.15.52:0/777696696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ba8103720 msgr2=0x7f0ba8105b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:45.775 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.768+0000 7f0baebe9700 1 --2- 172.21.15.52:0/777696696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ba8103720 0x7f0ba8105b00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0b94009990 tx=0x7f0b94009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.775 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.768+0000 7f0baebe9700 1 -- 172.21.15.52:0/777696696 shutdown_connections 2023-11-27T15:17:45.775 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.768+0000 7f0baebe9700 1 --2- 172.21.15.52:0/777696696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ba8106040 0x7f0ba8108420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.775 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.768+0000 7f0baebe9700 1 --2- 172.21.15.52:0/777696696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ba8100e00 0x7f0ba81031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.775 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.768+0000 7f0baebe9700 1 --2- 172.21.15.52:0/777696696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ba8103720 0x7f0ba8105b00 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.775 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.768+0000 7f0baebe9700 1 -- 172.21.15.52:0/777696696 >> 172.21.15.52:0/777696696 conn(0x7f0ba80fa7c0 msgr2=0x7f0ba80fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:45.775 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.768+0000 7f0baebe9700 1 -- 172.21.15.52:0/777696696 shutdown_connections 2023-11-27T15:17:45.775 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.768+0000 7f0baebe9700 1 -- 172.21.15.52:0/777696696 wait complete. 2023-11-27T15:17:45.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.769+0000 7f0baebe9700 1 Processor -- start 2023-11-27T15:17:45.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.769+0000 7f0baebe9700 1 -- start start 2023-11-27T15:17:45.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.769+0000 7f0baebe9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ba8100e00 0x7f0ba8196070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.769+0000 7f0baebe9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ba8103720 0x7f0ba81965b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.769+0000 7f0baebe9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ba8106040 0x7f0ba819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.769+0000 7f0baebe9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba819dbf0 con 0x7f0ba8100e00 2023-11-27T15:17:45.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.769+0000 7f0baebe9700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba819dd30 con 0x7f0ba8106040 2023-11-27T15:17:45.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.769+0000 7f0baebe9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba819dec0 con 0x7f0ba8103720 2023-11-27T15:17:45.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.769+0000 7f0bac985700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ba8100e00 0x7f0ba8196070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.769+0000 7f0bac985700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ba8100e00 0x7f0ba8196070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51678/0 (socket says 172.21.15.52:51678) 2023-11-27T15:17:45.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.769+0000 7f0bac985700 1 -- 172.21.15.52:0/2891624334 learned_addr learned my addr 172.21.15.52:0/2891624334 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:45.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.769+0000 7f0ba7fff700 1 --2- 172.21.15.52:0/2891624334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ba8103720 0x7f0ba81965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.769+0000 7f0bad186700 1 --2- 172.21.15.52:0/2891624334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ba8106040 0x7f0ba819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.769+0000 7f0bac985700 1 -- 172.21.15.52:0/2891624334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ba8103720 msgr2=0x7f0ba81965b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:45.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.769+0000 7f0bac985700 1 --2- 172.21.15.52:0/2891624334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ba8103720 0x7f0ba81965b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.769+0000 7f0bac985700 1 -- 172.21.15.52:0/2891624334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ba8106040 msgr2=0x7f0ba819d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:45.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.769+0000 7f0bac985700 1 --2- 172.21.15.52:0/2891624334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ba8106040 0x7f0ba819d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.769+0000 7f0bac985700 1 -- 172.21.15.52:0/2891624334 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b940095f0 con 0x7f0ba8100e00 2023-11-27T15:17:45.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.769+0000 7f0bad186700 1 --2- 172.21.15.52:0/2891624334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ba8106040 0x7f0ba819d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:17:45.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.770+0000 7f0bac985700 1 --2- 172.21.15.52:0/2891624334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ba8100e00 0x7f0ba8196070 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c00e9a0 tx=0x7f0b9c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:45.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.770+0000 7f0ba5ffb700 1 -- 172.21.15.52:0/2891624334 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c009790 con 0x7f0ba8100e00 2023-11-27T15:17:45.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.770+0000 7f0baebe9700 1 -- 172.21.15.52:0/2891624334 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ba810bca0 con 0x7f0ba8100e00 2023-11-27T15:17:45.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.770+0000 7f0ba5ffb700 1 -- 172.21.15.52:0/2891624334 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b9c00cbd0 con 0x7f0ba8100e00 2023-11-27T15:17:45.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.770+0000 7f0ba5ffb700 1 -- 172.21.15.52:0/2891624334 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c018680 con 0x7f0ba8100e00 2023-11-27T15:17:45.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.770+0000 7f0baebe9700 1 -- 172.21.15.52:0/2891624334 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba810bb10 con 0x7f0ba8100e00 2023-11-27T15:17:45.787 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.772+0000 7f0ba5ffb700 1 -- 172.21.15.52:0/2891624334 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0b9c00cd40 con 0x7f0ba8100e00 2023-11-27T15:17:45.787 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.773+0000 7f0ba5ffb700 1 --2- 172.21.15.52:0/2891624334 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0b980a1150 0x7f0b980a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.787 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.773+0000 7f0ba7fff700 1 --2- 172.21.15.52:0/2891624334 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0b980a1150 0x7f0b980a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.787 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.773+0000 7f0ba5ffb700 1 -- 172.21.15.52:0/2891624334 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0b9c014070 con 0x7f0ba8100e00 2023-11-27T15:17:45.787 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.774+0000 7f0ba7fff700 1 --2- 172.21.15.52:0/2891624334 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0b980a1150 0x7f0b980a3600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0b9400b2e0 tx=0x7f0b94005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:45.787 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.774+0000 7f0bbad75b80 1 Processor -- start 2023-11-27T15:17:45.787 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.774+0000 7f0bbad75b80 1 -- start start 2023-11-27T15:17:45.787 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.774+0000 7f0bbad75b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d28b7e7c00 0x55d28b7e8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.787 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.774+0000 7f0bbad75b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28b7e8550 0x55d28b7e8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.787 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.774+0000 7f0bbad75b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d28b7e9110 0x55d28b7e9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.774+0000 7f0bbad75b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d28b6ed060 con 0x55d28b7e9110 2023-11-27T15:17:45.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.774+0000 7f0bbad75b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d28b73f8a0 con 0x55d28b7e8550 2023-11-27T15:17:45.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.774+0000 7f0bbad75b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d28b738d30 con 0x55d28b7e7c00 2023-11-27T15:17:45.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.774+0000 7f0bad186700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d28b7e9110 0x55d28b7e9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.774+0000 7f0bad186700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d28b7e9110 0x55d28b7e9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51690/0 (socket says 172.21.15.52:51690) 2023-11-27T15:17:45.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.774+0000 7f0bad186700 1 -- 172.21.15.52:0/3984922769 learned_addr learned my addr 172.21.15.52:0/3984922769 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:45.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.774+0000 7f0ba7fff700 1 --2- 172.21.15.52:0/3984922769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28b7e8550 0x55d28b7e8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.774+0000 7f0bac985700 1 --2- 172.21.15.52:0/3984922769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d28b7e7c00 0x55d28b7e8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.775+0000 7f0bad186700 1 -- 172.21.15.52:0/3984922769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d28b7e7c00 msgr2=0x55d28b7e8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:45.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.775+0000 7f0bad186700 1 --2- 172.21.15.52:0/3984922769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d28b7e7c00 0x55d28b7e8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.775+0000 7f0bad186700 1 -- 172.21.15.52:0/3984922769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28b7e8550 msgr2=0x55d28b7e8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:45.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.775+0000 7f0bad186700 1 --2- 172.21.15.52:0/3984922769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28b7e8550 0x55d28b7e8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.789 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.775+0000 7f0bad186700 1 -- 172.21.15.52:0/3984922769 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d28b5ba630 con 0x55d28b7e9110 2023-11-27T15:17:45.789 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.775+0000 7f0bad186700 1 --2- 172.21.15.52:0/3984922769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d28b7e9110 0x55d28b7e9580 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0009400 tx=0x7f0ba0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4b046926f35bc6 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:45.789 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.775+0000 7f0b92ffd700 1 -- 172.21.15.52:0/3984922769 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0014610 con 0x55d28b7e9110 2023-11-27T15:17:45.789 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.775+0000 7f0b92ffd700 1 -- 172.21.15.52:0/3984922769 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ba000f050 con 0x55d28b7e9110 2023-11-27T15:17:45.789 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.775+0000 7f0b92ffd700 1 -- 172.21.15.52:0/3984922769 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0013a40 con 0x55d28b7e9110 2023-11-27T15:17:45.789 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.776+0000 7f0bbad75b80 1 -- 172.21.15.52:0/3984922769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d28b7e9110 msgr2=0x55d28b7e9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:45.789 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.776+0000 7f0bbad75b80 1 --2- 172.21.15.52:0/3984922769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d28b7e9110 0x55d28b7e9580 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0009400 tx=0x7f0ba0009920 comp rx=0 tx=0).stop 2023-11-27T15:17:45.789 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.776+0000 7f0bbad75b80 1 -- 172.21.15.52:0/3984922769 shutdown_connections 2023-11-27T15:17:45.789 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.776+0000 7f0bbad75b80 1 --2- 172.21.15.52:0/3984922769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d28b7e9110 0x55d28b7e9580 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.789 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.776+0000 7f0bbad75b80 1 --2- 172.21.15.52:0/3984922769 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d28b7e7c00 0x55d28b7e8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.789 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.776+0000 7f0bbad75b80 1 --2- 172.21.15.52:0/3984922769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28b7e8550 0x55d28b7e8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.789 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.776+0000 7f0bbad75b80 1 -- 172.21.15.52:0/3984922769 >> 172.21.15.52:0/3984922769 conn(0x55d28b7dd360 msgr2=0x55d28b7e2670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:45.789 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.776+0000 7f0bbad75b80 1 -- 172.21.15.52:0/3984922769 shutdown_connections 2023-11-27T15:17:45.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.776+0000 7f0bbad75b80 1 -- 172.21.15.52:0/3984922769 wait complete. 2023-11-27T15:17:45.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.776+0000 7f0bbad75b80 1 -- start start 2023-11-27T15:17:45.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.776+0000 7f0bbad75b80 1 Processor -- start 2023-11-27T15:17:45.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.777+0000 7f0bbad75b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d28b7e7c00 0x55d28b7f1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.777+0000 7f0bbad75b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d28b7e8550 0x55d28b7f1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.777+0000 7f0bbad75b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28b7e9110 0x55d28b7f8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.777+0000 7f0bbad75b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d28b6da8e0 con 0x55d28b7e7c00 2023-11-27T15:17:45.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.777+0000 7f0bbad75b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d28b6da580 con 0x55d28b7e9110 2023-11-27T15:17:45.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.777+0000 7f0bbad75b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d28b5bc920 con 0x55d28b7e8550 2023-11-27T15:17:45.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.777+0000 7f0bbad75b80 10 client.0 _ll_register_callbacks cb 0x55d28b6f8780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:17:45.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.777+0000 7f0bac985700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d28b7e7c00 0x55d28b7f1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.777+0000 7f0bac985700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d28b7e7c00 0x55d28b7f1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51692/0 (socket says 172.21.15.52:51692) 2023-11-27T15:17:45.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.777+0000 7f0bac985700 1 -- 172.21.15.52:0/3753213842 learned_addr learned my addr 172.21.15.52:0/3753213842 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:45.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.777+0000 7f0bad186700 1 --2- 172.21.15.52:0/3753213842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28b7e9110 0x55d28b7f8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.777+0000 7f0ba7fff700 1 --2- 172.21.15.52:0/3753213842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d28b7e8550 0x55d28b7f1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.777+0000 7f0bac985700 1 -- 172.21.15.52:0/3753213842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d28b7e8550 msgr2=0x55d28b7f1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:45.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.777+0000 7f0bac985700 1 --2- 172.21.15.52:0/3753213842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d28b7e8550 0x55d28b7f1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.777+0000 7f0bac985700 1 -- 172.21.15.52:0/3753213842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28b7e9110 msgr2=0x55d28b7f8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:45.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.777+0000 7f0bac985700 1 --2- 172.21.15.52:0/3753213842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28b7e9110 0x55d28b7f8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.777+0000 7f0bac985700 1 -- 172.21.15.52:0/3753213842 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba00090b0 con 0x55d28b7e7c00 2023-11-27T15:17:45.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.777+0000 7f0bad186700 1 --2- 172.21.15.52:0/3753213842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28b7e9110 0x55d28b7f8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:17:45.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.777+0000 7f0bac985700 1 --2- 172.21.15.52:0/3753213842 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d28b7e7c00 0x55d28b7f1590 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c0a1e90 tx=0x7f0b9c09a4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc5c9815149e420 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:45.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.777+0000 7f0b87fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:17:45.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.777+0000 7f0b87fff700 1 -- 172.21.15.52:0/3753213842 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c0a2130 con 0x55d28b7e7c00 2023-11-27T15:17:45.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.777+0000 7f0b87fff700 1 -- 172.21.15.52:0/3753213842 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0b9c09dfa0 con 0x55d28b7e7c00 2023-11-27T15:17:45.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.777+0000 7f0bbad75b80 1 -- 172.21.15.52:0/3753213842 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d28b7dab20 con 0x55d28b7e7c00 2023-11-27T15:17:45.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.778+0000 7f0b87fff700 1 -- 172.21.15.52:0/3753213842 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b9c0a23c0 con 0x55d28b7e7c00 2023-11-27T15:17:45.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.778+0000 7f0bbad75b80 10 client.15363 fetch_fsmap learned FSMap version 23 2023-11-27T15:17:45.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.778+0000 7f0bbad75b80 1 -- 172.21.15.52:0/3753213842 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d28b6ff190 con 0x55d28b7e7c00 2023-11-27T15:17:45.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.778+0000 7f0b87fff700 1 -- 172.21.15.52:0/3753213842 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b9c0b8030 con 0x55d28b7e7c00 2023-11-27T15:17:45.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.778+0000 7f0b87fff700 1 -- 172.21.15.52:0/3753213842 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0b9c095d60 con 0x55d28b7e7c00 2023-11-27T15:17:45.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.778+0000 7f0bbad75b80 10 client.15363 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:17:45.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.778+0000 7f0bbad75b80 10 client.15363 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:17:45.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.778+0000 7f0bbad75b80 20 client.15363 populate_metadata read hostname 'smithi052' 2023-11-27T15:17:45.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.778+0000 7f0bbad75b80 1 --2- 172.21.15.52:0/3753213842 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d28b7fd3f0 0x55d28b7ff800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.778+0000 7f0bbad75b80 4 client.15363 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:17:45.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.778+0000 7f0bbad75b80 1 -- 172.21.15.52:0/3753213842 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d28b6fb0a0 con 0x55d28b7fd3f0 2023-11-27T15:17:45.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.778+0000 7f0ba7fff700 1 --2- 172.21.15.52:0/3753213842 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d28b7fd3f0 0x55d28b7ff800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.779+0000 7f0b87fff700 10 client.15363 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:17:45.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.779+0000 7f0ba7fff700 1 --2- 172.21.15.52:0/3753213842 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d28b7fd3f0 0x55d28b7ff800 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=111f3f4b706c418a server_cookie=406a8652a1885405 in_seq=0 out_seq=0 2023-11-27T15:17:45.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.779+0000 7f0b87fff700 1 -- 172.21.15.52:0/3753213842 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d28b6fb0a0 con 0x55d28b7fd3f0 2023-11-27T15:17:45.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.779+0000 7f0b87fff700 10 client.15363 handle_command_reply: tid=0 2023-11-27T15:17:45.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.779+0000 7f0bbad75b80 1 client.15363 shutdown 2023-11-27T15:17:45.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.779+0000 7f0bbad75b80 1 -- 172.21.15.52:0/3753213842 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d28b7e7c00 msgr2=0x55d28b7f1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:45.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.779+0000 7f0bbad75b80 1 --2- 172.21.15.52:0/3753213842 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d28b7e7c00 0x55d28b7f1590 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c0a1e90 tx=0x7f0b9c09a4b0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.779+0000 7f0bbad75b80 1 -- 172.21.15.52:0/3753213842 shutdown_connections 2023-11-27T15:17:45.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.779+0000 7f0bbad75b80 1 --2- 172.21.15.52:0/3753213842 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d28b7fd3f0 0x55d28b7ff800 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.779+0000 7f0bbad75b80 1 --2- 172.21.15.52:0/3753213842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28b7e9110 0x55d28b7f8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.779+0000 7f0b87fff700 0 client.15363 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:17:45.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.779+0000 7f0bbad75b80 1 --2- 172.21.15.52:0/3753213842 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d28b7e7c00 0x55d28b7f1590 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.779+0000 7f0bbad75b80 1 --2- 172.21.15.52:0/3753213842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d28b7e8550 0x55d28b7f1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.779+0000 7f0bbad75b80 1 -- 172.21.15.52:0/3753213842 >> 172.21.15.52:0/3753213842 conn(0x55d28b7dd360 msgr2=0x55d28b7e1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:45.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.780+0000 7f0bbad75b80 1 -- 172.21.15.52:0/3753213842 shutdown_connections 2023-11-27T15:17:45.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.780+0000 7f0bbad75b80 1 -- 172.21.15.52:0/3753213842 wait complete. 2023-11-27T15:17:45.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.780+0000 7f0bbad75b80 20 client.15363 trim_cache size 0 max 16384 2023-11-27T15:17:45.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.796+0000 7f0bbad75b80 1 Processor -- start 2023-11-27T15:17:45.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.797+0000 7f0bbad75b80 1 -- start start 2023-11-27T15:17:45.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.797+0000 7f0bbad75b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28b7e7c00 0x55d28b7ebf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.797+0000 7f0bbad75b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d28b7e8550 0x55d28b7f8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.797+0000 7f0bbad75b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d28b7e9110 0x55d28b7ec4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.797+0000 7f0bbad75b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d28b7ef1f0 con 0x55d28b7e8550 2023-11-27T15:17:45.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.797+0000 7f0bbad75b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d28b7e40f0 con 0x55d28b7e7c00 2023-11-27T15:17:45.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.797+0000 7f0bbad75b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d28b7efcd0 con 0x55d28b7e9110 2023-11-27T15:17:45.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.797+0000 7f0ba7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d28b7e8550 0x55d28b7f8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.797+0000 7f0ba7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d28b7e8550 0x55d28b7f8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51702/0 (socket says 172.21.15.52:51702) 2023-11-27T15:17:45.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.797+0000 7f0ba7fff700 1 -- 172.21.15.52:0/2853677731 learned_addr learned my addr 172.21.15.52:0/2853677731 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:45.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.797+0000 7f0bac985700 1 --2- 172.21.15.52:0/2853677731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28b7e7c00 0x55d28b7ebf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.797+0000 7f0ba7fff700 1 -- 172.21.15.52:0/2853677731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d28b7e9110 msgr2=0x55d28b7ec4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:45.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.797+0000 7f0ba7fff700 1 --2- 172.21.15.52:0/2853677731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d28b7e9110 0x55d28b7ec4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.797+0000 7f0ba7fff700 1 -- 172.21.15.52:0/2853677731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28b7e7c00 msgr2=0x55d28b7ebf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:45.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.797+0000 7f0ba7fff700 1 --2- 172.21.15.52:0/2853677731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28b7e7c00 0x55d28b7ebf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.797+0000 7f0ba7fff700 1 -- 172.21.15.52:0/2853677731 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d28b6feee0 con 0x55d28b7e8550 2023-11-27T15:17:45.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.797+0000 7f0ba7fff700 1 --2- 172.21.15.52:0/2853677731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d28b7e8550 0x55d28b7f8ad0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0b94009990 tx=0x7f0b94023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd7ebeeea27db13 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:45.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.797+0000 7f0b937fe700 1 -- 172.21.15.52:0/2853677731 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b94032970 con 0x55d28b7e8550 2023-11-27T15:17:45.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.798+0000 7f0b937fe700 1 -- 172.21.15.52:0/2853677731 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b9401e040 con 0x55d28b7e8550 2023-11-27T15:17:45.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.798+0000 7f0b937fe700 1 -- 172.21.15.52:0/2853677731 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b94004e60 con 0x55d28b7e8550 2023-11-27T15:17:45.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.798+0000 7f0bbad75b80 1 -- 172.21.15.52:0/2853677731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d28b7e8550 msgr2=0x55d28b7f8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:45.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.798+0000 7f0bbad75b80 1 --2- 172.21.15.52:0/2853677731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d28b7e8550 0x55d28b7f8ad0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0b94009990 tx=0x7f0b94023000 comp rx=0 tx=0).stop 2023-11-27T15:17:45.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.798+0000 7f0bbad75b80 1 -- 172.21.15.52:0/2853677731 shutdown_connections 2023-11-27T15:17:45.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.798+0000 7f0bbad75b80 1 --2- 172.21.15.52:0/2853677731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28b7e7c00 0x55d28b7ebf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.798+0000 7f0bbad75b80 1 --2- 172.21.15.52:0/2853677731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d28b7e9110 0x55d28b7ec4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.798+0000 7f0bbad75b80 1 --2- 172.21.15.52:0/2853677731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d28b7e8550 0x55d28b7f8ad0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.798+0000 7f0bbad75b80 1 -- 172.21.15.52:0/2853677731 >> 172.21.15.52:0/2853677731 conn(0x55d28b7dd360 msgr2=0x55d28b804170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:45.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.798+0000 7f0bbad75b80 1 -- 172.21.15.52:0/2853677731 shutdown_connections 2023-11-27T15:17:45.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.798+0000 7f0bbad75b80 1 -- 172.21.15.52:0/2853677731 wait complete. 2023-11-27T15:17:45.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.798+0000 7f0bbad75b80 1 -- start start 2023-11-27T15:17:45.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.799+0000 7f0bbad75b80 1 Processor -- start 2023-11-27T15:17:45.808 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.799+0000 7f0bbad75b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d28b7e7c00 0x55d28b7f4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.808 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.799+0000 7f0bbad75b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d28b7e8550 0x55d28b7f4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.799+0000 7f0bbad75b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28b7e9110 0x55d28b7f7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.799+0000 7f0bbad75b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d28b7f84f0 con 0x55d28b7e8550 2023-11-27T15:17:45.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.799+0000 7f0bbad75b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d28b7f8660 con 0x55d28b7e9110 2023-11-27T15:17:45.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.799+0000 7f0bbad75b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d28b7f8820 con 0x55d28b7e7c00 2023-11-27T15:17:45.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.799+0000 7f0bbad75b80 10 client.0 _ll_register_callbacks cb 0x7f0b9c099dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:17:45.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.799+0000 7f0ba7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d28b7e8550 0x55d28b7f4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.799+0000 7f0ba7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d28b7e8550 0x55d28b7f4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51708/0 (socket says 172.21.15.52:51708) 2023-11-27T15:17:45.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.799+0000 7f0ba7fff700 1 -- 172.21.15.52:0/2327462329 learned_addr learned my addr 172.21.15.52:0/2327462329 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:45.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.799+0000 7f0bad186700 1 --2- 172.21.15.52:0/2327462329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28b7e9110 0x55d28b7f7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.799+0000 7f0bac985700 1 --2- 172.21.15.52:0/2327462329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d28b7e7c00 0x55d28b7f4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.799+0000 7f0ba7fff700 1 -- 172.21.15.52:0/2327462329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d28b7e7c00 msgr2=0x55d28b7f4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:45.810 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.799+0000 7f0ba7fff700 1 --2- 172.21.15.52:0/2327462329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d28b7e7c00 0x55d28b7f4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.810 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.799+0000 7f0ba7fff700 1 -- 172.21.15.52:0/2327462329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28b7e9110 msgr2=0x55d28b7f7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:17:45.810 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.799+0000 7f0ba7fff700 1 --2- 172.21.15.52:0/2327462329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28b7e9110 0x55d28b7f7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.810 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.799+0000 7f0ba7fff700 1 -- 172.21.15.52:0/2327462329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba8108ff0 con 0x55d28b7e8550 2023-11-27T15:17:45.810 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.800+0000 7f0ba7fff700 1 --2- 172.21.15.52:0/2327462329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d28b7e8550 0x55d28b7f4f90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0b94000c00 tx=0x7f0b94005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d636ed5a07754b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:45.810 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.800+0000 7f0b87fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:17:45.810 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.800+0000 7f0b87fff700 1 -- 172.21.15.52:0/2327462329 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b94004500 con 0x55d28b7e8550 2023-11-27T15:17:45.810 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.800+0000 7f0b87fff700 1 -- 172.21.15.52:0/2327462329 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0b94004be0 con 0x55d28b7e8550 2023-11-27T15:17:45.810 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.800+0000 7f0bbad75b80 1 -- 172.21.15.52:0/2327462329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ba000f050 con 0x55d28b7e8550 2023-11-27T15:17:45.810 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.800+0000 7f0b87fff700 1 -- 172.21.15.52:0/2327462329 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b94004dc0 con 0x55d28b7e8550 2023-11-27T15:17:45.810 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.800+0000 7f0bbad75b80 10 client.15369 fetch_fsmap learned FSMap version 23 2023-11-27T15:17:45.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.800+0000 7f0bbad75b80 1 -- 172.21.15.52:0/2327462329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d28b54fc60 con 0x55d28b7e8550 2023-11-27T15:17:45.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.800+0000 7f0b87fff700 1 -- 172.21.15.52:0/2327462329 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ba000f050 con 0x55d28b7e8550 2023-11-27T15:17:45.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.800+0000 7f0b87fff700 1 -- 172.21.15.52:0/2327462329 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0b9400fa30 con 0x55d28b7e8550 2023-11-27T15:17:45.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.800+0000 7f0bbad75b80 10 client.15369 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:17:45.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.800+0000 7f0bbad75b80 10 client.15369 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:17:45.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.800+0000 7f0bbad75b80 20 client.15369 populate_metadata read hostname 'smithi052' 2023-11-27T15:17:45.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.800+0000 7f0bbad75b80 1 --2- 172.21.15.52:0/2327462329 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d28b851ef0 0x55d28b854300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:45.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.800+0000 7f0bbad75b80 4 client.15369 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:17:45.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.800+0000 7f0bbad75b80 1 -- 172.21.15.52:0/2327462329 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d28b707660 con 0x55d28b851ef0 2023-11-27T15:17:45.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.801+0000 7f0bac985700 1 --2- 172.21.15.52:0/2327462329 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d28b851ef0 0x55d28b854300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:45.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.801+0000 7f0b87fff700 10 client.15369 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:17:45.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.801+0000 7f0bac985700 1 --2- 172.21.15.52:0/2327462329 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d28b851ef0 0x55d28b854300 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7aefdc604756d05 server_cookie=93a077a3c99de461 in_seq=0 out_seq=0 2023-11-27T15:17:45.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.801+0000 7f0b87fff700 1 -- 172.21.15.52:0/2327462329 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d28b707660 con 0x55d28b851ef0 2023-11-27T15:17:45.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.801+0000 7f0b87fff700 10 client.15369 handle_command_reply: tid=0 2023-11-27T15:17:45.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.801+0000 7f0bbad75b80 1 client.15369 shutdown 2023-11-27T15:17:45.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.801+0000 7f0bbad75b80 1 -- 172.21.15.52:0/2327462329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d28b7e8550 msgr2=0x55d28b7f4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:45.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.801+0000 7f0bbad75b80 1 --2- 172.21.15.52:0/2327462329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d28b7e8550 0x55d28b7f4f90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0b94000c00 tx=0x7f0b94005e80 comp rx=0 tx=0).stop 2023-11-27T15:17:45.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.801+0000 7f0bbad75b80 1 -- 172.21.15.52:0/2327462329 shutdown_connections 2023-11-27T15:17:45.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.801+0000 7f0bbad75b80 1 --2- 172.21.15.52:0/2327462329 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d28b851ef0 0x55d28b854300 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.801+0000 7f0bbad75b80 1 --2- 172.21.15.52:0/2327462329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28b7e9110 0x55d28b7f7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.801+0000 7f0bbad75b80 1 --2- 172.21.15.52:0/2327462329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d28b7e7c00 0x55d28b7f4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.801+0000 7f0b87fff700 0 client.15369 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:17:45.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.801+0000 7f0bbad75b80 1 --2- 172.21.15.52:0/2327462329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d28b7e8550 0x55d28b7f4f90 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.801+0000 7f0bbad75b80 1 -- 172.21.15.52:0/2327462329 >> 172.21.15.52:0/2327462329 conn(0x55d28b7dd360 msgr2=0x55d28b7ebf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:45.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.802+0000 7f0bbad75b80 1 -- 172.21.15.52:0/2327462329 shutdown_connections 2023-11-27T15:17:45.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.802+0000 7f0bbad75b80 1 -- 172.21.15.52:0/2327462329 wait complete. 2023-11-27T15:17:45.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.802+0000 7f0bbad75b80 20 client.15369 trim_cache size 0 max 16384 2023-11-27T15:17:45.813 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:17:45.813 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-27T15:17:45.813 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2023-11-27T15:17:45.813 INFO:teuthology.orchestra.run.smithi052.stdout: "ccdf23fc-1f9f-4bf6-b299-7bc6c9a580fb": { 2023-11-27T15:17:45.814 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2023-11-27T15:17:45.814 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "ccdf23fc-1f9f-4bf6-b299-7bc6c9a580fb", 2023-11-27T15:17:45.814 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2023-11-27T15:17:45.814 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:17:45.814 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:17:45.814 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:17:45.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.802+0000 7f0baebe9700 1 -- 172.21.15.52:0/2891624334 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0b980a1150 msgr2=0x7f0b980a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:45.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.802+0000 7f0baebe9700 1 --2- 172.21.15.52:0/2891624334 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0b980a1150 0x7f0b980a3600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0b9400b2e0 tx=0x7f0b94005fd0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.802+0000 7f0baebe9700 1 -- 172.21.15.52:0/2891624334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ba8100e00 msgr2=0x7f0ba8196070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:45.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.802+0000 7f0baebe9700 1 --2- 172.21.15.52:0/2891624334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ba8100e00 0x7f0ba8196070 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c00e9a0 tx=0x7f0b9c00ecb0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.802+0000 7f0baebe9700 1 -- 172.21.15.52:0/2891624334 shutdown_connections 2023-11-27T15:17:45.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.802+0000 7f0baebe9700 1 --2- 172.21.15.52:0/2891624334 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f0b980a1150 0x7f0b980a3600 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.802+0000 7f0baebe9700 1 --2- 172.21.15.52:0/2891624334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ba8106040 0x7f0ba819d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.802+0000 7f0baebe9700 1 --2- 172.21.15.52:0/2891624334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ba8100e00 0x7f0ba8196070 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.802+0000 7f0baebe9700 1 --2- 172.21.15.52:0/2891624334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ba8103720 0x7f0ba81965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:45.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.802+0000 7f0baebe9700 1 -- 172.21.15.52:0/2891624334 >> 172.21.15.52:0/2891624334 conn(0x7f0ba80fa7c0 msgr2=0x7f0ba8102290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:45.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.802+0000 7f0baebe9700 1 -- 172.21.15.52:0/2891624334 shutdown_connections 2023-11-27T15:17:45.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.802+0000 7f0baebe9700 1 -- 172.21.15.52:0/2891624334 wait complete. 2023-11-27T15:17:45.819 INFO:tasks.cephfs.filesystem:scrub status for tag:ccdf23fc-1f9f-4bf6-b299-7bc6c9a580fb - {'path': '/', 'tag': 'ccdf23fc-1f9f-4bf6-b299-7bc6c9a580fb', 'options': 'recursive,force'} 2023-11-27T15:17:45.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:45.859+0000 7f7d5334a700 1 -- 172.21.15.52:0/1401207324 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f7d4c061700 con 0x7f7d4c0feb30 2023-11-27T15:17:46.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:45 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1620092975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:17:46.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:45 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3792656876' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:17:46.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:45 smithi102 ceph-mon[100296]: scrub queued for path: / 2023-11-27T15:17:46.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:45 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:17:46.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:45 smithi102 ceph-mon[100296]: scrub summary: active paths [/] 2023-11-27T15:17:46.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:45 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:17:46.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:45 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1620092975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:17:46.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:45 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3792656876' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:17:46.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:45 smithi052 ceph-mon[99056]: scrub queued for path: / 2023-11-27T15:17:46.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:45 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:17:46.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:45 smithi052 ceph-mon[99056]: scrub summary: active paths [/] 2023-11-27T15:17:46.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:45 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:17:46.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:45 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1620092975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:17:46.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:45 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3792656876' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:17:46.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:45 smithi150 ceph-mon[100559]: scrub queued for path: / 2023-11-27T15:17:46.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:45 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:17:46.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:45 smithi150 ceph-mon[100559]: scrub summary: active paths [/] 2023-11-27T15:17:46.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:45 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:17:46.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:46.838+0000 7f7d427fc700 1 -- 172.21.15.52:0/1401207324 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 19) v1 ==== 135732+0+0 (secure 0 0 0) 0x7f7d48097830 con 0x7f7d4c0feb30 2023-11-27T15:17:46.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:46.839+0000 7f7d427fc700 1 -- 172.21.15.52:0/1401207324 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f7d48059070 con 0x7f7d4c0feb30 2023-11-27T15:17:46.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:46.842+0000 7f7d5334a700 1 -- 172.21.15.52:0/1401207324 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7d380a1150 msgr2=0x7f7d380a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:46.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:46.842+0000 7f7d5334a700 1 --2- 172.21.15.52:0/1401207324 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7d380a1150 0x7f7d380a3600 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c005950 tx=0x7f7d3c0058e0 comp rx=0 tx=0).stop 2023-11-27T15:17:46.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:46.842+0000 7f7d5334a700 1 -- 172.21.15.52:0/1401207324 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d4c0feb30 msgr2=0x7f7d4c198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:46.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:46.842+0000 7f7d5334a700 1 --2- 172.21.15.52:0/1401207324 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d4c0feb30 0x7f7d4c198160 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7d48004760 tx=0x7f7d48009f90 comp rx=0 tx=0).stop 2023-11-27T15:17:46.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:46.842+0000 7f7d5334a700 1 -- 172.21.15.52:0/1401207324 shutdown_connections 2023-11-27T15:17:46.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:46.842+0000 7f7d5334a700 1 --2- 172.21.15.52:0/1401207324 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f7d380a1150 0x7f7d380a3600 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:46.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:46.842+0000 7f7d5334a700 1 --2- 172.21.15.52:0/1401207324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d4c0ffe30 0x7f7d4c19f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:46.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:46.842+0000 7f7d5334a700 1 --2- 172.21.15.52:0/1401207324 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d4c0feb30 0x7f7d4c198160 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:46.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:46.842+0000 7f7d5334a700 1 --2- 172.21.15.52:0/1401207324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d4c0ff480 0x7f7d4c1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:46.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:46.842+0000 7f7d5334a700 1 -- 172.21.15.52:0/1401207324 >> 172.21.15.52:0/1401207324 conn(0x7f7d4c0fa720 msgr2=0x7f7d4c0fc410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:46.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:46.842+0000 7f7d5334a700 1 -- 172.21.15.52:0/1401207324 shutdown_connections 2023-11-27T15:17:46.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:46.842+0000 7f7d5334a700 1 -- 172.21.15.52:0/1401207324 wait complete. 2023-11-27T15:17:46.864 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2023-11-27T15:17:47.009 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.004+0000 7f53b262b700 1 -- 172.21.15.52:0/1821405620 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53ac105f00 msgr2=0x7f53ac108360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:47.009 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.004+0000 7f53b262b700 1 --2- 172.21.15.52:0/1821405620 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53ac105f00 0x7f53ac108360 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f53a400b0a0 tx=0x7f53a400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:17:47.009 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.004+0000 7f53b262b700 1 -- 172.21.15.52:0/1821405620 shutdown_connections 2023-11-27T15:17:47.009 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.004+0000 7f53b262b700 1 --2- 172.21.15.52:0/1821405620 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53ac105f00 0x7f53ac108360 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:47.009 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.004+0000 7f53b262b700 1 --2- 172.21.15.52:0/1821405620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53ac100cc0 0x7f53ac1030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:47.009 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.004+0000 7f53b262b700 1 --2- 172.21.15.52:0/1821405620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ac1035e0 0x7f53ac1059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:47.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.004+0000 7f53b262b700 1 -- 172.21.15.52:0/1821405620 >> 172.21.15.52:0/1821405620 conn(0x7f53ac0fa720 msgr2=0x7f53ac0fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:47.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.004+0000 7f53b262b700 1 -- 172.21.15.52:0/1821405620 shutdown_connections 2023-11-27T15:17:47.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.005+0000 7f53b262b700 1 -- 172.21.15.52:0/1821405620 wait complete. 2023-11-27T15:17:47.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.005+0000 7f53b262b700 1 Processor -- start 2023-11-27T15:17:47.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.005+0000 7f53b262b700 1 -- start start 2023-11-27T15:17:47.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.005+0000 7f53b262b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53ac100cc0 0x7f53ac196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:47.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.005+0000 7f53b262b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ac1035e0 0x7f53ac196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:47.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.005+0000 7f53b262b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53ac105f00 0x7f53ac19d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:47.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.005+0000 7f53b262b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac19dcb0 con 0x7f53ac100cc0 2023-11-27T15:17:47.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.005+0000 7f53b262b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac19ddf0 con 0x7f53ac1035e0 2023-11-27T15:17:47.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.005+0000 7f53b262b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac19df80 con 0x7f53ac105f00 2023-11-27T15:17:47.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.006+0000 7f53abfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53ac100cc0 0x7f53ac196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:47.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.006+0000 7f53ab7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ac1035e0 0x7f53ac196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:47.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.006+0000 7f53ab7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ac1035e0 0x7f53ac196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:41410/0 (socket says 172.21.15.52:41410) 2023-11-27T15:17:47.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.006+0000 7f53abfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53ac100cc0 0x7f53ac196130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51750/0 (socket says 172.21.15.52:51750) 2023-11-27T15:17:47.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.006+0000 7f53ab7fe700 1 -- 172.21.15.52:0/1700573056 learned_addr learned my addr 172.21.15.52:0/1700573056 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:47.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.006+0000 7f53b0bc8700 1 --2- 172.21.15.52:0/1700573056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53ac105f00 0x7f53ac19d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:47.013 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.006+0000 7f53abfff700 1 -- 172.21.15.52:0/1700573056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53ac105f00 msgr2=0x7f53ac19d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:47.013 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.006+0000 7f53abfff700 1 --2- 172.21.15.52:0/1700573056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53ac105f00 0x7f53ac19d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:47.013 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.006+0000 7f53abfff700 1 -- 172.21.15.52:0/1700573056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ac1035e0 msgr2=0x7f53ac196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:47.013 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.006+0000 7f53abfff700 1 --2- 172.21.15.52:0/1700573056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ac1035e0 0x7f53ac196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:47.013 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.006+0000 7f53abfff700 1 -- 172.21.15.52:0/1700573056 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f539c009520 con 0x7f53ac100cc0 2023-11-27T15:17:47.013 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.006+0000 7f53ab7fe700 1 --2- 172.21.15.52:0/1700573056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ac1035e0 0x7f53ac196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:17:47.013 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.006+0000 7f53abfff700 1 --2- 172.21.15.52:0/1700573056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53ac100cc0 0x7f53ac196130 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f539c00e9f0 tx=0x7f539c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:47.013 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.006+0000 7f53a97fa700 1 -- 172.21.15.52:0/1700573056 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539c00cb70 con 0x7f53ac100cc0 2023-11-27T15:17:47.013 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.006+0000 7f53a97fa700 1 -- 172.21.15.52:0/1700573056 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f539c00ee90 con 0x7f53ac100cc0 2023-11-27T15:17:47.013 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.006+0000 7f53b262b700 1 -- 172.21.15.52:0/1700573056 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a4009d00 con 0x7f53ac100cc0 2023-11-27T15:17:47.013 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.006+0000 7f53a97fa700 1 -- 172.21.15.52:0/1700573056 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539c003cf0 con 0x7f53ac100cc0 2023-11-27T15:17:47.013 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.006+0000 7f53b262b700 1 -- 172.21.15.52:0/1700573056 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ac10bbf0 con 0x7f53ac100cc0 2023-11-27T15:17:47.014 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.007+0000 7f53b262b700 1 -- 172.21.15.52:0/1700573056 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53ac10bd80 con 0x7f53ac100cc0 2023-11-27T15:17:47.014 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.009+0000 7f53a97fa700 1 -- 172.21.15.52:0/1700573056 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) v1 ==== 135732+0+0 (secure 0 0 0) 0x7f539c00ccd0 con 0x7f53ac100cc0 2023-11-27T15:17:47.014 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.010+0000 7f53a97fa700 1 --2- 172.21.15.52:0/1700573056 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f53940a11a0 0x7f53940a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:47.014 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.010+0000 7f53ab7fe700 1 -- 172.21.15.52:0/1700573056 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f53940a11a0 msgr2=0x7f53940a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2640036585 2023-11-27T15:17:47.015 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.010+0000 7f53a97fa700 1 -- 172.21.15.52:0/1700573056 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f539c014070 con 0x7f53ac100cc0 2023-11-27T15:17:47.015 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.010+0000 7f53ab7fe700 1 --2- 172.21.15.52:0/1700573056 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f53940a11a0 0x7f53940a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-27T15:17:47.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.014+0000 7f53a97fa700 1 -- 172.21.15.52:0/1700573056 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f539c0964c0 con 0x7f53ac100cc0 2023-11-27T15:17:47.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:46 smithi102 ceph-mon[100296]: pgmap v306: 97 pgs: 97 active+clean; 645 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2023-11-27T15:17:47.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:46 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1401207324' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-11-27T15:17:47.078 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:17:46 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: ignoring --setuser ceph since I am not root 2023-11-27T15:17:47.078 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:17:46 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: ignoring --setgroup ceph since I am not root 2023-11-27T15:17:47.078 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:17:47 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:17:47.006+0000 7f3211749200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-27T15:17:47.087 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:46 smithi150 ceph-mon[100559]: pgmap v306: 97 pgs: 97 active+clean; 645 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2023-11-27T15:17:47.087 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:46 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1401207324' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-11-27T15:17:47.087 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:17:46 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: ignoring --setuser ceph since I am not root 2023-11-27T15:17:47.087 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:17:46 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: ignoring --setgroup ceph since I am not root 2023-11-27T15:17:47.087 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:17:47 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:17:47.003+0000 7f3c3c690200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-27T15:17:47.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:46 smithi052 ceph-mon[99056]: pgmap v306: 97 pgs: 97 active+clean; 645 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2023-11-27T15:17:47.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:46 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1401207324' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-11-27T15:17:47.117 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:46 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: ignoring --setuser ceph since I am not root 2023-11-27T15:17:47.117 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:46 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: ignoring --setgroup ceph since I am not root 2023-11-27T15:17:47.117 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:46 smithi052 ceph-mgr[99327]: -- 172.21.15.52:0/2771314254 <== mon.0 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5643f4881520 con 0x5643f4856800 2023-11-27T15:17:47.117 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:47 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:47.045+0000 7f4048652200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-27T15:17:47.215 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.211+0000 7f53ab7fe700 1 -- 172.21.15.52:0/1700573056 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f53940a11a0 msgr2=0x7f53940a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2640036585 2023-11-27T15:17:47.215 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.211+0000 7f53ab7fe700 1 --2- 172.21.15.52:0/1700573056 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f53940a11a0 0x7f53940a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-27T15:17:47.218 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.214+0000 7f53b262b700 1 -- 172.21.15.52:0/1700573056 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f53ac19ed40 con 0x7f53ac100cc0 2023-11-27T15:17:47.222 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.218+0000 7f53a97fa700 1 -- 172.21.15.52:0/1700573056 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f539c099ae0 con 0x7f53ac100cc0 2023-11-27T15:17:47.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.221+0000 7f53b262b700 1 -- 172.21.15.52:0/1700573056 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f53940a11a0 msgr2=0x7f53940a3650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-27T15:17:47.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.221+0000 7f53b262b700 1 --2- 172.21.15.52:0/1700573056 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f53940a11a0 0x7f53940a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:47.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.221+0000 7f53b262b700 1 -- 172.21.15.52:0/1700573056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53ac100cc0 msgr2=0x7f53ac196130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:47.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.221+0000 7f53b262b700 1 --2- 172.21.15.52:0/1700573056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53ac100cc0 0x7f53ac196130 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f539c00e9f0 tx=0x7f539c00ed00 comp rx=0 tx=0).stop 2023-11-27T15:17:47.225 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.221+0000 7f53b262b700 1 -- 172.21.15.52:0/1700573056 shutdown_connections 2023-11-27T15:17:47.226 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.221+0000 7f53b262b700 1 --2- 172.21.15.52:0/1700573056 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f53940a11a0 0x7f53940a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:47.226 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.221+0000 7f53b262b700 1 --2- 172.21.15.52:0/1700573056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53ac105f00 0x7f53ac19d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:47.226 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.221+0000 7f53b262b700 1 --2- 172.21.15.52:0/1700573056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53ac100cc0 0x7f53ac196130 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:47.226 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.221+0000 7f53b262b700 1 --2- 172.21.15.52:0/1700573056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ac1035e0 0x7f53ac196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:47.226 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.221+0000 7f53b262b700 1 -- 172.21.15.52:0/1700573056 >> 172.21.15.52:0/1700573056 conn(0x7f53ac0fa720 msgr2=0x7f53ac102100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:47.226 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.221+0000 7f53b262b700 1 -- 172.21.15.52:0/1700573056 shutdown_connections 2023-11-27T15:17:47.226 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.221+0000 7f53b262b700 1 -- 172.21.15.52:0/1700573056 wait complete. 2023-11-27T15:17:47.239 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2023-11-27T15:17:47.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.364+0000 7fe388683700 1 -- 172.21.15.52:0/2486559316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe380100cc0 msgr2=0x7fe380107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:47.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.364+0000 7fe388683700 1 --2- 172.21.15.52:0/2486559316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe380100cc0 0x7fe380107580 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe37800b0a0 tx=0x7fe37800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:17:47.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.364+0000 7fe388683700 1 -- 172.21.15.52:0/2486559316 shutdown_connections 2023-11-27T15:17:47.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.364+0000 7fe388683700 1 --2- 172.21.15.52:0/2486559316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3800ff140 0x7fe3800ff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:47.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.364+0000 7fe388683700 1 --2- 172.21.15.52:0/2486559316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe380100cc0 0x7fe380107580 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:47.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.364+0000 7fe388683700 1 --2- 172.21.15.52:0/2486559316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe380100330 0x7fe380100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:47.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.364+0000 7fe388683700 1 -- 172.21.15.52:0/2486559316 >> 172.21.15.52:0/2486559316 conn(0x7fe3800fa720 msgr2=0x7fe3800fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:47.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.364+0000 7fe388683700 1 -- 172.21.15.52:0/2486559316 shutdown_connections 2023-11-27T15:17:47.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.364+0000 7fe388683700 1 -- 172.21.15.52:0/2486559316 wait complete. 2023-11-27T15:17:47.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.364+0000 7fe388683700 1 Processor -- start 2023-11-27T15:17:47.371 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.365+0000 7fe388683700 1 -- start start 2023-11-27T15:17:47.371 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.365+0000 7fe388683700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3800ff140 0x7fe3801030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:47.371 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.365+0000 7fe388683700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe380100330 0x7fe380101710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:47.371 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.365+0000 7fe388683700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe380100cc0 0x7fe380101c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:47.371 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.365+0000 7fe388683700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3801023f0 con 0x7fe3800ff140 2023-11-27T15:17:47.371 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.365+0000 7fe388683700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe380102560 con 0x7fe380100330 2023-11-27T15:17:47.371 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.365+0000 7fe388683700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3801a0170 con 0x7fe380100cc0 2023-11-27T15:17:47.371 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.365+0000 7fe38641f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3800ff140 0x7fe3801030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:47.371 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.365+0000 7fe38641f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3800ff140 0x7fe3801030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51790/0 (socket says 172.21.15.52:51790) 2023-11-27T15:17:47.371 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.365+0000 7fe38641f700 1 -- 172.21.15.52:0/3461809780 learned_addr learned my addr 172.21.15.52:0/3461809780 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:47.371 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.365+0000 7fe386c20700 1 --2- 172.21.15.52:0/3461809780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe380100cc0 0x7fe380101c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:47.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.365+0000 7fe385c1e700 1 --2- 172.21.15.52:0/3461809780 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe380100330 0x7fe380101710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:47.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.365+0000 7fe38641f700 1 -- 172.21.15.52:0/3461809780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe380100cc0 msgr2=0x7fe380101c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:47.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.365+0000 7fe38641f700 1 --2- 172.21.15.52:0/3461809780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe380100cc0 0x7fe380101c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:47.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.365+0000 7fe38641f700 1 -- 172.21.15.52:0/3461809780 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe380100330 msgr2=0x7fe380101710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:47.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.365+0000 7fe38641f700 1 --2- 172.21.15.52:0/3461809780 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe380100330 0x7fe380101710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:47.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.365+0000 7fe38641f700 1 -- 172.21.15.52:0/3461809780 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe378009d00 con 0x7fe3800ff140 2023-11-27T15:17:47.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.365+0000 7fe386c20700 1 --2- 172.21.15.52:0/3461809780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe380100cc0 0x7fe380101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:17:47.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.365+0000 7fe38641f700 1 --2- 172.21.15.52:0/3461809780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3800ff140 0x7fe3801030d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe37c00d980 tx=0x7fe37c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:47.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.365+0000 7fe3777fe700 1 -- 172.21.15.52:0/3461809780 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c00f890 con 0x7fe3800ff140 2023-11-27T15:17:47.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.366+0000 7fe388683700 1 -- 172.21.15.52:0/3461809780 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe38010aab0 con 0x7fe3800ff140 2023-11-27T15:17:47.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.366+0000 7fe388683700 1 -- 172.21.15.52:0/3461809780 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe38010a920 con 0x7fe3800ff140 2023-11-27T15:17:47.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.366+0000 7fe3777fe700 1 -- 172.21.15.52:0/3461809780 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe37c007620 con 0x7fe3800ff140 2023-11-27T15:17:47.373 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.366+0000 7fe3777fe700 1 -- 172.21.15.52:0/3461809780 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c019360 con 0x7fe3800ff140 2023-11-27T15:17:47.373 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.366+0000 7fe388683700 1 -- 172.21.15.52:0/3461809780 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe364005320 con 0x7fe3800ff140 2023-11-27T15:17:47.374 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.369+0000 7fe3777fe700 1 -- 172.21.15.52:0/3461809780 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) v1 ==== 135732+0+0 (secure 0 0 0) 0x7fe37c010560 con 0x7fe3800ff140 2023-11-27T15:17:47.374 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.369+0000 7fe3777fe700 1 --2- 172.21.15.52:0/3461809780 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fe36c0a11a0 0x7fe36c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:47.374 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.369+0000 7fe385c1e700 1 -- 172.21.15.52:0/3461809780 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fe36c0a11a0 msgr2=0x7fe36c0a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2640036585 2023-11-27T15:17:47.374 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.369+0000 7fe3777fe700 1 -- 172.21.15.52:0/3461809780 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe37c0cc840 con 0x7fe3800ff140 2023-11-27T15:17:47.374 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.369+0000 7fe385c1e700 1 --2- 172.21.15.52:0/3461809780 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fe36c0a11a0 0x7fe36c0a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-27T15:17:47.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.373+0000 7fe3777fe700 1 -- 172.21.15.52:0/3461809780 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe37c095f70 con 0x7fe3800ff140 2023-11-27T15:17:47.439 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:17:47 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:17:47.082+0000 7f3c3c690200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-27T15:17:47.439 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:17:47 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:17:47.172+0000 7f3c3c690200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-27T15:17:47.439 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:17:47 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:17:47.265+0000 7f3c3c690200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-27T15:17:47.440 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:17:47 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:17:47.086+0000 7f3211749200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-27T15:17:47.440 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:17:47 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:17:47.174+0000 7f3211749200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-27T15:17:47.440 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:17:47 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:17:47.266+0000 7f3211749200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-27T15:17:47.441 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:17:47 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:17:47.436+0000 7f3211749200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-27T15:17:47.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.570+0000 7fe385c1e700 1 -- 172.21.15.52:0/3461809780 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fe36c0a11a0 msgr2=0x7fe36c0a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2640036585 2023-11-27T15:17:47.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.570+0000 7fe385c1e700 1 --2- 172.21.15.52:0/3461809780 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fe36c0a11a0 0x7fe36c0a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-27T15:17:47.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.577+0000 7fe388683700 1 -- 172.21.15.52:0/3461809780 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fe364005cc0 con 0x7fe3800ff140 2023-11-27T15:17:47.583 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:47 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:47.143+0000 7f4048652200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-27T15:17:47.583 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:47 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:47.253+0000 7f4048652200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-27T15:17:47.583 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:47 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:47.371+0000 7f4048652200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-27T15:17:47.585 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.581+0000 7fe3777fe700 1 -- 172.21.15.52:0/3461809780 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7fe37c0200c0 con 0x7fe3800ff140 2023-11-27T15:17:47.588 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.584+0000 7fe388683700 1 -- 172.21.15.52:0/3461809780 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fe36c0a11a0 msgr2=0x7fe36c0a3650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-27T15:17:47.588 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.584+0000 7fe388683700 1 --2- 172.21.15.52:0/3461809780 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fe36c0a11a0 0x7fe36c0a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:47.588 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.584+0000 7fe388683700 1 -- 172.21.15.52:0/3461809780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3800ff140 msgr2=0x7fe3801030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:47.588 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.584+0000 7fe388683700 1 --2- 172.21.15.52:0/3461809780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3800ff140 0x7fe3801030d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe37c00d980 tx=0x7fe37c00dc90 comp rx=0 tx=0).stop 2023-11-27T15:17:47.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.584+0000 7fe388683700 1 -- 172.21.15.52:0/3461809780 shutdown_connections 2023-11-27T15:17:47.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.584+0000 7fe388683700 1 --2- 172.21.15.52:0/3461809780 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7fe36c0a11a0 0x7fe36c0a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:47.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.584+0000 7fe388683700 1 --2- 172.21.15.52:0/3461809780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe380100cc0 0x7fe380101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:47.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.584+0000 7fe388683700 1 --2- 172.21.15.52:0/3461809780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3800ff140 0x7fe3801030d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:47.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.584+0000 7fe388683700 1 --2- 172.21.15.52:0/3461809780 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe380100330 0x7fe380101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:47.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.584+0000 7fe388683700 1 -- 172.21.15.52:0/3461809780 >> 172.21.15.52:0/3461809780 conn(0x7fe3800fa720 msgr2=0x7fe380068100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:47.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.584+0000 7fe388683700 1 -- 172.21.15.52:0/3461809780 shutdown_connections 2023-11-27T15:17:47.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.584+0000 7fe388683700 1 -- 172.21.15.52:0/3461809780 wait complete. 2023-11-27T15:17:47.601 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M' 2023-11-27T15:17:47.731 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.727+0000 7f4c9fc8a700 1 -- 172.21.15.52:0/854971693 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c98100750 msgr2=0x7f4c98100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:47.732 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.727+0000 7f4c9fc8a700 1 --2- 172.21.15.52:0/854971693 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c98100750 0x7f4c98100b60 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4c94009990 tx=0x7f4c94009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:47.732 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.727+0000 7f4c9fc8a700 1 -- 172.21.15.52:0/854971693 shutdown_connections 2023-11-27T15:17:47.732 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.727+0000 7f4c9fc8a700 1 --2- 172.21.15.52:0/854971693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c980ffd50 0x7f4c98107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:47.732 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.727+0000 7f4c9fc8a700 1 --2- 172.21.15.52:0/854971693 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c98100750 0x7f4c98100b60 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:47.732 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.727+0000 7f4c9fc8a700 1 --2- 172.21.15.52:0/854971693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c980ff3a0 0x7f4c980ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:47.732 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.727+0000 7f4c9fc8a700 1 -- 172.21.15.52:0/854971693 >> 172.21.15.52:0/854971693 conn(0x7f4c980f9ea0 msgr2=0x7f4c980fc310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:47.732 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.727+0000 7f4c9fc8a700 1 -- 172.21.15.52:0/854971693 shutdown_connections 2023-11-27T15:17:47.732 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.727+0000 7f4c9fc8a700 1 -- 172.21.15.52:0/854971693 wait complete. 2023-11-27T15:17:47.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.728+0000 7f4c9fc8a700 1 Processor -- start 2023-11-27T15:17:47.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.728+0000 7f4c9fc8a700 1 -- start start 2023-11-27T15:17:47.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.728+0000 7f4c9fc8a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c980ff3a0 0x7f4c98195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:47.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.728+0000 7f4c9fc8a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c980ffd50 0x7f4c981964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:47.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.728+0000 7f4c9fc8a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c98100750 0x7f4c9819d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:47.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.728+0000 7f4c9fc8a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9819db30 con 0x7f4c980ff3a0 2023-11-27T15:17:47.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.728+0000 7f4c9fc8a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9819dc70 con 0x7f4c980ffd50 2023-11-27T15:17:47.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.728+0000 7f4c9fc8a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9819de00 con 0x7f4c98100750 2023-11-27T15:17:47.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.728+0000 7f4c9da26700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c980ff3a0 0x7f4c98195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:47.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.728+0000 7f4c9da26700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c980ff3a0 0x7f4c98195fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51838/0 (socket says 172.21.15.52:51838) 2023-11-27T15:17:47.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.728+0000 7f4c9da26700 1 -- 172.21.15.52:0/2245596585 learned_addr learned my addr 172.21.15.52:0/2245596585 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:47.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.728+0000 7f4c9e227700 1 --2- 172.21.15.52:0/2245596585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c98100750 0x7f4c9819d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:47.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.728+0000 7f4c9d225700 1 --2- 172.21.15.52:0/2245596585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c980ffd50 0x7f4c981964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:47.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.728+0000 7f4c9da26700 1 -- 172.21.15.52:0/2245596585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c98100750 msgr2=0x7f4c9819d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:47.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.728+0000 7f4c9da26700 1 --2- 172.21.15.52:0/2245596585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c98100750 0x7f4c9819d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:47.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.728+0000 7f4c9da26700 1 -- 172.21.15.52:0/2245596585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c980ffd50 msgr2=0x7f4c981964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:47.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.728+0000 7f4c9da26700 1 --2- 172.21.15.52:0/2245596585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4c980ffd50 0x7f4c981964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:47.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.728+0000 7f4c9da26700 1 -- 172.21.15.52:0/2245596585 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c940095f0 con 0x7f4c980ff3a0 2023-11-27T15:17:47.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.729+0000 7f4c9da26700 1 --2- 172.21.15.52:0/2245596585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c980ff3a0 0x7f4c98195fb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4c94004730 tx=0x7f4c94005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:47.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.729+0000 7f4c8affd700 1 -- 172.21.15.52:0/2245596585 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9401d070 con 0x7f4c980ff3a0 2023-11-27T15:17:47.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.729+0000 7f4c8affd700 1 -- 172.21.15.52:0/2245596585 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c94004830 con 0x7f4c980ff3a0 2023-11-27T15:17:47.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.729+0000 7f4c9fc8a700 1 -- 172.21.15.52:0/2245596585 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c9810aac0 con 0x7f4c980ff3a0 2023-11-27T15:17:47.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.729+0000 7f4c8affd700 1 -- 172.21.15.52:0/2245596585 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c94005420 con 0x7f4c980ff3a0 2023-11-27T15:17:47.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.729+0000 7f4c9fc8a700 1 -- 172.21.15.52:0/2245596585 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9810a930 con 0x7f4c980ff3a0 2023-11-27T15:17:47.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.730+0000 7f4c9fc8a700 1 -- 172.21.15.52:0/2245596585 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c9819e400 con 0x7f4c980ff3a0 2023-11-27T15:17:47.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.731+0000 7f4c8affd700 1 -- 172.21.15.52:0/2245596585 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) v1 ==== 135732+0+0 (secure 0 0 0) 0x7f4c940049a0 con 0x7f4c980ff3a0 2023-11-27T15:17:47.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.731+0000 7f4c8affd700 1 --2- 172.21.15.52:0/2245596585 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4c840a1220 0x7f4c840a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:47.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.732+0000 7f4c9d225700 1 -- 172.21.15.52:0/2245596585 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4c840a1220 msgr2=0x7f4c840a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2640036585 2023-11-27T15:17:47.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.732+0000 7f4c8affd700 1 -- 172.21.15.52:0/2245596585 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4c940ce0d0 con 0x7f4c980ff3a0 2023-11-27T15:17:47.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.732+0000 7f4c9d225700 1 --2- 172.21.15.52:0/2245596585 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4c840a1220 0x7f4c840a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-27T15:17:47.739 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.735+0000 7f4c8affd700 1 -- 172.21.15.52:0/2245596585 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4c94097830 con 0x7f4c980ff3a0 2023-11-27T15:17:47.745 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:17:47 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:17:47.434+0000 7f3c3c690200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-27T15:17:47.867 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:47 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:47.579+0000 7f4048652200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-27T15:17:47.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:47 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1401207324' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-11-27T15:17:47.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:47 smithi052 ceph-mon[99056]: mgrmap e19: x(active, since 7m), standbys: z, y 2023-11-27T15:17:47.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:47 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1700573056' entity='client.admin' 2023-11-27T15:17:47.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:47 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3461809780' entity='client.admin' 2023-11-27T15:17:47.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.933+0000 7f4c9d225700 1 -- 172.21.15.52:0/2245596585 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4c840a1220 msgr2=0x7f4c840a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2640036585 2023-11-27T15:17:47.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.933+0000 7f4c9d225700 1 --2- 172.21.15.52:0/2245596585 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4c840a1220 0x7f4c840a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-27T15:17:47.966 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:47.962+0000 7f4c9fc8a700 1 -- 172.21.15.52:0/2245596585 --> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c980fe9c0 con 0x7f4c840a1220 2023-11-27T15:17:48.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:47 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1401207324' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-11-27T15:17:48.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:47 smithi150 ceph-mon[100559]: mgrmap e19: x(active, since 7m), standbys: z, y 2023-11-27T15:17:48.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:47 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1700573056' entity='client.admin' 2023-11-27T15:17:48.029 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:47 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3461809780' entity='client.admin' 2023-11-27T15:17:48.029 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:17:47 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:17:47.741+0000 7f3c3c690200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-27T15:17:48.029 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:17:47 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:17:47.829+0000 7f3c3c690200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-27T15:17:48.029 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:17:47 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:17:47.959+0000 7f3c3c690200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-27T15:17:48.055 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:47 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1401207324' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-11-27T15:17:48.055 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:47 smithi102 ceph-mon[100296]: mgrmap e19: x(active, since 7m), standbys: z, y 2023-11-27T15:17:48.055 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:47 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1700573056' entity='client.admin' 2023-11-27T15:17:48.055 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:47 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3461809780' entity='client.admin' 2023-11-27T15:17:48.056 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:17:47 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:17:47.742+0000 7f3211749200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-27T15:17:48.056 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:17:47 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:17:47.830+0000 7f3211749200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-27T15:17:48.056 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:17:47 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:17:47.959+0000 7f3211749200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-27T15:17:48.250 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:47 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:47.986+0000 7f4048652200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-27T15:17:48.250 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:48 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:48.094+0000 7f4048652200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-27T15:17:48.327 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:17:48 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:17:48.051+0000 7f3211749200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-27T15:17:48.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:48.334+0000 7f4c9d225700 1 -- 172.21.15.52:0/2245596585 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4c840a1220 msgr2=0x7f4c840a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2640036585 2023-11-27T15:17:48.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:48.334+0000 7f4c9d225700 1 --2- 172.21.15.52:0/2245596585 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4c840a1220 0x7f4c840a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-27T15:17:48.477 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:17:48 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:17:48.054+0000 7f3c3c690200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-27T15:17:48.617 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:48 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:48.246+0000 7f4048652200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-27T15:17:48.617 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:48 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:48.362+0000 7f4048652200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-27T15:17:48.778 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:17:48 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:17:48.473+0000 7f3c3c690200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-27T15:17:48.827 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:17:48 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:17:48.469+0000 7f3211749200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-27T15:17:49.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:49.135+0000 7f4c9d225700 1 -- 172.21.15.52:0/2245596585 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4c840a1220 msgr2=0x7f4c840a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2640036585 2023-11-27T15:17:49.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:49.135+0000 7f4c9d225700 1 --2- 172.21.15.52:0/2245596585 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4c840a1220 0x7f4c840a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-27T15:17:49.328 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:17:49 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:17:49.029+0000 7f3211749200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-27T15:17:49.328 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:17:49 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:17:49.197+0000 7f3211749200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-27T15:17:49.366 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:48 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:48.874+0000 7f4048652200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-27T15:17:49.528 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:17:49 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:17:49.036+0000 7f3c3c690200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-27T15:17:49.528 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:17:49 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:17:49.205+0000 7f3c3c690200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-27T15:17:49.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:49 smithi052 ceph-mon[99056]: scrub summary: idle 2023-11-27T15:17:49.867 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:49 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:49.561+0000 7f4048652200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-27T15:17:49.867 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:49 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:49.755+0000 7f4048652200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-27T15:17:50.028 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:17:49 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:17:49.968+0000 7f3c3c690200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:17:50.028 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:49 smithi150 ceph-mon[100559]: scrub summary: idle 2023-11-27T15:17:50.049 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:17:49 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:17:49.956+0000 7f3211749200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:17:50.049 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:49 smithi102 ceph-mon[100296]: scrub summary: idle 2023-11-27T15:17:50.327 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:17:50 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:17:50.045+0000 7f3211749200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-27T15:17:50.328 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:17:50 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:17:50.215+0000 7f3211749200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:17:50.508 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:17:50 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:17:50.058+0000 7f3c3c690200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-27T15:17:50.509 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:17:50 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:17:50.228+0000 7f3c3c690200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:17:50.740 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:50.736+0000 7f4c9d225700 1 -- 172.21.15.52:0/2245596585 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4c840a1220 msgr2=0x7f4c840a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2640036585 2023-11-27T15:17:50.741 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:50.736+0000 7f4c9d225700 1 --2- 172.21.15.52:0/2245596585 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4c840a1220 0x7f4c840a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-11-27T15:17:50.778 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:17:50 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:17:50.504+0000 7f3c3c690200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-27T15:17:50.778 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:17:50 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:17:50.730+0000 7f3c3c690200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-27T15:17:50.827 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:17:50 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:17:50.487+0000 7f3211749200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-27T15:17:50.828 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:17:50 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:17:50.709+0000 7f3211749200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-27T15:17:51.058 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:50 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:50.722+0000 7f4048652200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:17:51.058 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:50 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:50.833+0000 7f4048652200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-27T15:17:51.117 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:17:50 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:17:50.854+0000 7f3211749200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-27T15:17:51.142 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:17:50 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:17:50.877+0000 7f3c3c690200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-27T15:17:51.366 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:51 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:51.054+0000 7f4048652200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:17:51.528 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:17:51 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:17:51.138+0000 7f3c3c690200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-27T15:17:51.577 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:17:51 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:17:51.113+0000 7f3211749200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-27T15:17:51.665 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:51 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:51.406+0000 7f4048652200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-27T15:17:52.077 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:17:51 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:17:51.804+0000 7f3211749200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-27T15:17:52.077 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:17:51 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:17:51.933+0000 7f3211749200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-27T15:17:52.117 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:51 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:51.660+0000 7f4048652200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-27T15:17:52.117 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:51 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:51.844+0000 7f4048652200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-27T15:17:52.162 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:17:51 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:17:51.838+0000 7f3c3c690200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-27T15:17:52.162 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:17:51 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:17:51.968+0000 7f3c3c690200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-27T15:17:52.449 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:17:52 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:17:52.120+0000 7f3211749200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-27T15:17:52.449 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:17:52 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:17:52.233+0000 7f3211749200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-27T15:17:52.449 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:17:52 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:17:52.445+0000 7f3211749200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-27T15:17:52.490 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:17:52 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:17:52.157+0000 7f3c3c690200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-27T15:17:52.490 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:17:52 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:17:52.271+0000 7f3c3c690200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-27T15:17:52.616 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:52 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:52.179+0000 7f4048652200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-27T15:17:52.778 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:17:52 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:17:52.486+0000 7f3c3c690200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-27T15:17:53.366 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:53.362+0000 7f4c8affd700 1 -- 172.21.15.52:0/2245596585 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 20) v1 ==== 135732+0+0 (secure 0 0 0) 0x7f4c94004ca0 con 0x7f4c980ff3a0 2023-11-27T15:17:53.366 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:53 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:53.013+0000 7f4048652200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-27T15:17:53.366 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:53 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:53.174+0000 7f4048652200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-27T15:17:53.577 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:17:53 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y[101756]: 2023-11-27T15:17:53.288+0000 7f3211749200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-27T15:17:53.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:53 smithi102 ceph-mon[100296]: Standby manager daemon y restarted 2023-11-27T15:17:53.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:53 smithi102 ceph-mon[100296]: Standby manager daemon y started 2023-11-27T15:17:53.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:53 smithi102 ceph-mon[100296]: from='mgr.? 172.21.15.102:0/1273116035' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-11-27T15:17:53.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:53 smithi102 ceph-mon[100296]: from='mgr.? 172.21.15.102:0/1273116035' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-27T15:17:53.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:53 smithi102 ceph-mon[100296]: from='mgr.? 172.21.15.102:0/1273116035' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-11-27T15:17:53.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:53 smithi102 ceph-mon[100296]: from='mgr.? 172.21.15.102:0/1273116035' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-27T15:17:53.778 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:17:53 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z[102663]: 2023-11-27T15:17:53.339+0000 7f3c3c690200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-27T15:17:53.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:53 smithi150 ceph-mon[100559]: Standby manager daemon y restarted 2023-11-27T15:17:53.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:53 smithi150 ceph-mon[100559]: Standby manager daemon y started 2023-11-27T15:17:53.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:53 smithi150 ceph-mon[100559]: from='mgr.? 172.21.15.102:0/1273116035' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-11-27T15:17:53.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:53 smithi150 ceph-mon[100559]: from='mgr.? 172.21.15.102:0/1273116035' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-27T15:17:53.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:53 smithi150 ceph-mon[100559]: from='mgr.? 172.21.15.102:0/1273116035' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-11-27T15:17:53.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:53 smithi150 ceph-mon[100559]: from='mgr.? 172.21.15.102:0/1273116035' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-27T15:17:53.851 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:53 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:53.427+0000 7f4048652200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-27T15:17:53.851 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:53 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:53.568+0000 7f4048652200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-27T15:17:53.851 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:53 smithi052 ceph-mon[99056]: Standby manager daemon y restarted 2023-11-27T15:17:53.851 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:53 smithi052 ceph-mon[99056]: Standby manager daemon y started 2023-11-27T15:17:53.851 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:53 smithi052 ceph-mon[99056]: from='mgr.? 172.21.15.102:0/1273116035' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-11-27T15:17:53.851 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:53 smithi052 ceph-mon[99056]: from='mgr.? 172.21.15.102:0/1273116035' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-27T15:17:53.851 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:53 smithi052 ceph-mon[99056]: from='mgr.? 172.21.15.102:0/1273116035' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-11-27T15:17:53.851 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:53 smithi052 ceph-mon[99056]: from='mgr.? 172.21.15.102:0/1273116035' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-27T15:17:53.944 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:53.940+0000 7f4c9d225700 1 -- 172.21.15.52:0/2245596585 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4c840a1220 msgr2=0x7f4c840a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2640036585 2023-11-27T15:17:53.944 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:53.940+0000 7f4c9d225700 1 --2- 172.21.15.52:0/2245596585 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4c840a1220 0x7f4c840a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-11-27T15:17:54.116 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:53 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:53.846+0000 7f4048652200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-27T15:17:54.371 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:54.366+0000 7f4c8affd700 1 -- 172.21.15.52:0/2245596585 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mgrmap(e 21) v1 ==== 135732+0+0 (secure 0 0 0) 0x7f4c94097c00 con 0x7f4c980ff3a0 2023-11-27T15:17:54.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:54 smithi150 ceph-mon[100559]: Standby manager daemon z restarted 2023-11-27T15:17:54.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:54 smithi150 ceph-mon[100559]: Standby manager daemon z started 2023-11-27T15:17:54.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:54 smithi150 ceph-mon[100559]: mgrmap e20: x(active, since 7m), standbys: y, z 2023-11-27T15:17:54.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:54 smithi150 ceph-mon[100559]: from='mgr.? 172.21.15.150:0/168037393' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-11-27T15:17:54.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:54 smithi150 ceph-mon[100559]: from='mgr.? 172.21.15.150:0/168037393' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-27T15:17:54.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:54 smithi150 ceph-mon[100559]: from='mgr.? 172.21.15.150:0/168037393' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-11-27T15:17:54.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:54 smithi150 ceph-mon[100559]: from='mgr.? 172.21.15.150:0/168037393' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-27T15:17:54.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:54 smithi102 ceph-mon[100296]: Standby manager daemon z restarted 2023-11-27T15:17:54.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:54 smithi102 ceph-mon[100296]: Standby manager daemon z started 2023-11-27T15:17:54.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:54 smithi102 ceph-mon[100296]: mgrmap e20: x(active, since 7m), standbys: y, z 2023-11-27T15:17:54.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:54 smithi102 ceph-mon[100296]: from='mgr.? 172.21.15.150:0/168037393' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-11-27T15:17:54.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:54 smithi102 ceph-mon[100296]: from='mgr.? 172.21.15.150:0/168037393' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-27T15:17:54.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:54 smithi102 ceph-mon[100296]: from='mgr.? 172.21.15.150:0/168037393' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-11-27T15:17:54.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:54 smithi102 ceph-mon[100296]: from='mgr.? 172.21.15.150:0/168037393' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-27T15:17:54.866 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:54 smithi052 ceph-mon[99056]: Standby manager daemon z restarted 2023-11-27T15:17:54.866 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:54 smithi052 ceph-mon[99056]: Standby manager daemon z started 2023-11-27T15:17:54.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:54 smithi052 ceph-mon[99056]: mgrmap e20: x(active, since 7m), standbys: y, z 2023-11-27T15:17:54.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:54 smithi052 ceph-mon[99056]: from='mgr.? 172.21.15.150:0/168037393' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-11-27T15:17:54.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:54 smithi052 ceph-mon[99056]: from='mgr.? 172.21.15.150:0/168037393' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-27T15:17:54.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:54 smithi052 ceph-mon[99056]: from='mgr.? 172.21.15.150:0/168037393' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-11-27T15:17:54.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:54 smithi052 ceph-mon[99056]: from='mgr.? 172.21.15.150:0/168037393' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-27T15:17:54.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:54.923+0000 7f4c8affd700 1 -- 172.21.15.52:0/2245596585 <== mon.0 v2:172.21.15.52:3300/0 9 ==== mgrmap(e 22) v1 ==== 135400+0+0 (secure 0 0 0) 0x7f4c940cbb80 con 0x7f4c980ff3a0 2023-11-27T15:17:54.928 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:54.924+0000 7f4c8affd700 1 -- 172.21.15.52:0/2245596585 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4c840a1220 msgr2=0x7f4c840a36d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-27T15:17:54.928 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:54.924+0000 7f4c8affd700 1 --2- 172.21.15.52:0/2245596585 >> [v2:172.21.15.52:6800/2640036585,v1:172.21.15.52:6801/2640036585] conn(0x7f4c840a1220 0x7f4c840a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:55.155 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:54 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:54.900+0000 7f4048652200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-27T15:17:55.156 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:55.149+0000 7f3fa4aea700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-27T15:17:55.156 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:55.149+0000 7f3fa4aea700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-27T15:17:55.156 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:55.149+0000 7f3fa4aea700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-27T15:17:55.156 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:55.149+0000 7f3fa4aea700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-27T15:17:55.156 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:55.149+0000 7f3fa4aea700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-27T15:17:55.156 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:55.150+0000 7f3fa72ef700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-27T15:17:55.156 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:55.150+0000 7f3fa72ef700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-27T15:17:55.156 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:55.150+0000 7f3fa72ef700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-27T15:17:55.156 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:55.150+0000 7f3fa72ef700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-27T15:17:55.156 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:17:55.150+0000 7f3fa72ef700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-27T15:17:55.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: mgrmap e21: x(active, since 7m), standbys: y, z 2023-11-27T15:17:55.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: Active manager daemon x restarted 2023-11-27T15:17:55.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: Activating manager daemon x 2023-11-27T15:17:55.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: osdmap e86: 12 total, 12 up, 12 in 2023-11-27T15:17:55.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: mgrmap e22: x(active, starting, since 0.0128822s), standbys: y, z 2023-11-27T15:17:55.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:17:55.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:17:55.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:17:55.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-11-27T15:17:55.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-11-27T15:17:55.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-27T15:17:55.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-27T15:17:55.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-27T15:17:55.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-27T15:17:55.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-27T15:17:55.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-27T15:17:55.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-27T15:17:55.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-27T15:17:55.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-27T15:17:55.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-27T15:17:55.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-11-27T15:17:55.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-11-27T15:17:55.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-11-27T15:17:55.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:17:55.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:17:55.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:17:55.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:17:55.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:17:55.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:17:55.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:17:55.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:17:55.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:17:55.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:17:55.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:17:55.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:17:55.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-11-27T15:17:55.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-11-27T15:17:55.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-11-27T15:17:55.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: Manager daemon x is now available 2023-11-27T15:17:55.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:55.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:55.621 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-27T15:17:55.621 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:55 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-27T15:17:55.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: mgrmap e21: x(active, since 7m), standbys: y, z 2023-11-27T15:17:55.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: Active manager daemon x restarted 2023-11-27T15:17:55.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: Activating manager daemon x 2023-11-27T15:17:55.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: osdmap e86: 12 total, 12 up, 12 in 2023-11-27T15:17:55.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: mgrmap e22: x(active, starting, since 0.0128822s), standbys: y, z 2023-11-27T15:17:55.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:17:55.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:17:55.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:17:55.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-11-27T15:17:55.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-11-27T15:17:55.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-27T15:17:55.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-27T15:17:55.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-27T15:17:55.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-27T15:17:55.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-27T15:17:55.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-27T15:17:55.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-27T15:17:55.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-27T15:17:55.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-27T15:17:55.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-27T15:17:55.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-11-27T15:17:55.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-11-27T15:17:55.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-11-27T15:17:55.782 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:17:55.782 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:17:55.782 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:17:55.782 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:17:55.782 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:17:55.782 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:17:55.782 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:17:55.782 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:17:55.782 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:17:55.783 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:17:55.783 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:17:55.783 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:17:55.783 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-11-27T15:17:55.783 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-11-27T15:17:55.783 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-11-27T15:17:55.783 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: Manager daemon x is now available 2023-11-27T15:17:55.783 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:55.783 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:55.784 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-27T15:17:55.784 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:55 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-27T15:17:55.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: mgrmap e21: x(active, since 7m), standbys: y, z 2023-11-27T15:17:55.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: Active manager daemon x restarted 2023-11-27T15:17:55.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: Activating manager daemon x 2023-11-27T15:17:55.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: osdmap e86: 12 total, 12 up, 12 in 2023-11-27T15:17:55.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: mgrmap e22: x(active, starting, since 0.0128822s), standbys: y, z 2023-11-27T15:17:55.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:17:55.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:17:55.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:17:55.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-11-27T15:17:55.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-11-27T15:17:55.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-27T15:17:55.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-27T15:17:55.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-27T15:17:55.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-27T15:17:55.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-27T15:17:55.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-27T15:17:55.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-27T15:17:55.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-27T15:17:55.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-27T15:17:55.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-27T15:17:55.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-11-27T15:17:55.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-11-27T15:17:55.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-11-27T15:17:55.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:17:55.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:17:55.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:17:55.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:17:55.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:17:55.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:17:55.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:17:55.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:17:55.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:17:55.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:17:55.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:17:55.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:17:55.832 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-11-27T15:17:55.832 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-11-27T15:17:55.832 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-11-27T15:17:55.832 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: Manager daemon x is now available 2023-11-27T15:17:55.832 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:55.832 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:55.832 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-27T15:17:55.832 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:55 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-27T15:17:55.944 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:55.940+0000 7f4c8affd700 1 -- 172.21.15.52:0/2245596585 <== mon.0 v2:172.21.15.52:3300/0 10 ==== mgrmap(e 23) v1 ==== 135527+0+0 (secure 0 0 0) 0x7f4c940984d0 con 0x7f4c980ff3a0 2023-11-27T15:17:55.945 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:55.941+0000 7f4c8affd700 1 --2- 172.21.15.52:0/2245596585 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f4c840663d0 0x7f4c840667a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:55.945 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:55.941+0000 7f4c8affd700 1 -- 172.21.15.52:0/2245596585 --> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c980fe9c0 con 0x7f4c840663d0 2023-11-27T15:17:55.948 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:55.944+0000 7f4c9d225700 1 --2- 172.21.15.52:0/2245596585 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f4c840663d0 0x7f4c840667a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:55.948 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:55.945+0000 7f4c9d225700 1 --2- 172.21.15.52:0/2245596585 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f4c840663d0 0x7f4c840667a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009880 tx=0x7f4c8c009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:56.015 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.010+0000 7f4c8affd700 1 -- 172.21.15.52:0/2245596585 <== mgr.15387 v2:172.21.15.52:6800/1148545855 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f4c980fe9c0 con 0x7f4c840663d0 2023-11-27T15:17:56.015 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule set for path / 2023-11-27T15:17:56.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.012+0000 7f4c9fc8a700 1 -- 172.21.15.52:0/2245596585 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f4c840663d0 msgr2=0x7f4c840667a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:56.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.012+0000 7f4c9fc8a700 1 --2- 172.21.15.52:0/2245596585 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f4c840663d0 0x7f4c840667a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009880 tx=0x7f4c8c009260 comp rx=0 tx=0).stop 2023-11-27T15:17:56.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.012+0000 7f4c9fc8a700 1 -- 172.21.15.52:0/2245596585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c980ff3a0 msgr2=0x7f4c98195fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:56.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.012+0000 7f4c9fc8a700 1 --2- 172.21.15.52:0/2245596585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c980ff3a0 0x7f4c98195fb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4c94004730 tx=0x7f4c94005f10 comp rx=0 tx=0).stop 2023-11-27T15:17:56.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.012+0000 7f4c9e227700 1 -- 172.21.15.52:0/2245596585 reap_dead start 2023-11-27T15:17:56.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.012+0000 7f4c9fc8a700 1 -- 172.21.15.52:0/2245596585 shutdown_connections 2023-11-27T15:17:56.017 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.012+0000 7f4c9fc8a700 1 -- 172.21.15.52:0/2245596585 >> 172.21.15.52:0/2245596585 conn(0x7f4c980f9ea0 msgr2=0x7f4c980fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:56.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.013+0000 7f4c9fc8a700 1 -- 172.21.15.52:0/2245596585 shutdown_connections 2023-11-27T15:17:56.018 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.013+0000 7f4c9fc8a700 1 -- 172.21.15.52:0/2245596585 wait complete. 2023-11-27T15:17:56.029 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h' 2023-11-27T15:17:56.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.163+0000 7f852d20a700 1 -- 172.21.15.52:0/2354142870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8528100d20 msgr2=0x7f8528103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:56.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.163+0000 7f852d20a700 1 --2- 172.21.15.52:0/2354142870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8528100d20 0x7f8528103100 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8518009990 tx=0x7f8518009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:56.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.164+0000 7f852d20a700 1 -- 172.21.15.52:0/2354142870 shutdown_connections 2023-11-27T15:17:56.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.164+0000 7f852d20a700 1 --2- 172.21.15.52:0/2354142870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8528105f60 0x7f8528108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:56.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.164+0000 7f852d20a700 1 --2- 172.21.15.52:0/2354142870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8528100d20 0x7f8528103100 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:56.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.164+0000 7f852d20a700 1 --2- 172.21.15.52:0/2354142870 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8528103640 0x7f8528105a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:56.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.164+0000 7f852d20a700 1 -- 172.21.15.52:0/2354142870 >> 172.21.15.52:0/2354142870 conn(0x7f85280fa720 msgr2=0x7f85280fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:56.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.164+0000 7f852d20a700 1 -- 172.21.15.52:0/2354142870 shutdown_connections 2023-11-27T15:17:56.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.164+0000 7f852d20a700 1 -- 172.21.15.52:0/2354142870 wait complete. 2023-11-27T15:17:56.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.165+0000 7f852d20a700 1 Processor -- start 2023-11-27T15:17:56.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.165+0000 7f852d20a700 1 -- start start 2023-11-27T15:17:56.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.165+0000 7f852d20a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8528100d20 0x7f8528195f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:56.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.165+0000 7f852d20a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8528103640 0x7f8528196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:56.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.165+0000 7f852d20a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8528105f60 0x7f852819d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:56.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.165+0000 7f852d20a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852819dad0 con 0x7f8528103640 2023-11-27T15:17:56.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.165+0000 7f852d20a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852819dc10 con 0x7f8528100d20 2023-11-27T15:17:56.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.165+0000 7f852d20a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852819dda0 con 0x7f8528105f60 2023-11-27T15:17:56.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.165+0000 7f852659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8528103640 0x7f8528196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:56.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.165+0000 7f852659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8528103640 0x7f8528196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52094/0 (socket says 172.21.15.52:52094) 2023-11-27T15:17:56.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.165+0000 7f852659c700 1 -- 172.21.15.52:0/2579161613 learned_addr learned my addr 172.21.15.52:0/2579161613 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:56.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.165+0000 7f852759e700 1 --2- 172.21.15.52:0/2579161613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8528105f60 0x7f852819d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:56.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.165+0000 7f8526d9d700 1 --2- 172.21.15.52:0/2579161613 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8528100d20 0x7f8528195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:56.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.165+0000 7f852659c700 1 -- 172.21.15.52:0/2579161613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8528105f60 msgr2=0x7f852819d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:56.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.165+0000 7f852659c700 1 --2- 172.21.15.52:0/2579161613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8528105f60 0x7f852819d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:56.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.165+0000 7f852659c700 1 -- 172.21.15.52:0/2579161613 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8528100d20 msgr2=0x7f8528195f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:56.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.165+0000 7f852659c700 1 --2- 172.21.15.52:0/2579161613 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8528100d20 0x7f8528195f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:56.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.165+0000 7f852659c700 1 -- 172.21.15.52:0/2579161613 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85180095f0 con 0x7f8528103640 2023-11-27T15:17:56.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.165+0000 7f8526d9d700 1 --2- 172.21.15.52:0/2579161613 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8528100d20 0x7f8528195f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:17:56.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.166+0000 7f852659c700 1 --2- 172.21.15.52:0/2579161613 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8528103640 0x7f8528196490 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f851c00d980 tx=0x7f851c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:56.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.166+0000 7f850ffff700 1 -- 172.21.15.52:0/2579161613 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c004220 con 0x7f8528103640 2023-11-27T15:17:56.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.166+0000 7f852d20a700 1 -- 172.21.15.52:0/2579161613 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f852810bba0 con 0x7f8528103640 2023-11-27T15:17:56.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.166+0000 7f850ffff700 1 -- 172.21.15.52:0/2579161613 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f851c007640 con 0x7f8528103640 2023-11-27T15:17:56.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.166+0000 7f850ffff700 1 -- 172.21.15.52:0/2579161613 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c005610 con 0x7f8528103640 2023-11-27T15:17:56.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.166+0000 7f852d20a700 1 -- 172.21.15.52:0/2579161613 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f852810ba10 con 0x7f8528103640 2023-11-27T15:17:56.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.167+0000 7f852d20a700 1 -- 172.21.15.52:0/2579161613 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8528067c10 con 0x7f8528103640 2023-11-27T15:17:56.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.168+0000 7f850ffff700 1 -- 172.21.15.52:0/2579161613 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 23) v1 ==== 135527+0+0 (secure 0 0 0) 0x7f851c004750 con 0x7f8528103640 2023-11-27T15:17:56.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.169+0000 7f850ffff700 1 --2- 172.21.15.52:0/2579161613 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f85100a0f50 0x7f85100a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:56.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.169+0000 7f8526d9d700 1 --2- 172.21.15.52:0/2579161613 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f85100a0f50 0x7f85100a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:56.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.169+0000 7f850ffff700 1 -- 172.21.15.52:0/2579161613 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f851c0cc540 con 0x7f8528103640 2023-11-27T15:17:56.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.169+0000 7f8526d9d700 1 --2- 172.21.15.52:0/2579161613 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f85100a0f50 0x7f85100a3400 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8518006010 tx=0x7f851800b540 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:56.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.173+0000 7f850ffff700 1 -- 172.21.15.52:0/2579161613 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f851c095e60 con 0x7f8528103640 2023-11-27T15:17:56.419 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.414+0000 7f852d20a700 1 -- 172.21.15.52:0/2579161613 --> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f8528102890 con 0x7f85100a0f50 2023-11-27T15:17:56.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.474+0000 7f850ffff700 1 -- 172.21.15.52:0/2579161613 <== mgr.15387 v2:172.21.15.52:6800/1148545855 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f8528102890 con 0x7f85100a0f50 2023-11-27T15:17:56.478 INFO:teuthology.orchestra.run.smithi052.stdout:Retention added to path / 2023-11-27T15:17:56.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.476+0000 7f852d20a700 1 -- 172.21.15.52:0/2579161613 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f85100a0f50 msgr2=0x7f85100a3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:56.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.476+0000 7f852d20a700 1 --2- 172.21.15.52:0/2579161613 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f85100a0f50 0x7f85100a3400 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8518006010 tx=0x7f851800b540 comp rx=0 tx=0).stop 2023-11-27T15:17:56.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.476+0000 7f852d20a700 1 -- 172.21.15.52:0/2579161613 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8528103640 msgr2=0x7f8528196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:56.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.476+0000 7f852d20a700 1 --2- 172.21.15.52:0/2579161613 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8528103640 0x7f8528196490 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f851c00d980 tx=0x7f851c00dc90 comp rx=0 tx=0).stop 2023-11-27T15:17:56.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.476+0000 7f852d20a700 1 -- 172.21.15.52:0/2579161613 shutdown_connections 2023-11-27T15:17:56.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.476+0000 7f852d20a700 1 --2- 172.21.15.52:0/2579161613 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f85100a0f50 0x7f85100a3400 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:56.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.476+0000 7f852d20a700 1 --2- 172.21.15.52:0/2579161613 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8528100d20 0x7f8528195f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:56.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.476+0000 7f852d20a700 1 --2- 172.21.15.52:0/2579161613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8528105f60 0x7f852819d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:56.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.476+0000 7f852d20a700 1 --2- 172.21.15.52:0/2579161613 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8528103640 0x7f8528196490 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:56.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.476+0000 7f852d20a700 1 -- 172.21.15.52:0/2579161613 >> 172.21.15.52:0/2579161613 conn(0x7f85280fa720 msgr2=0x7f8528102180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:56.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.476+0000 7f852d20a700 1 -- 172.21.15.52:0/2579161613 shutdown_connections 2023-11-27T15:17:56.482 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.476+0000 7f852d20a700 1 -- 172.21.15.52:0/2579161613 wait complete. 2023-11-27T15:17:56.493 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-11-27T15:17:56.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.621+0000 7fd3fc052700 1 -- 172.21.15.52:0/3811997686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3f4105f20 msgr2=0x7fd3f4108380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:56.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.621+0000 7fd3fc052700 1 --2- 172.21.15.52:0/3811997686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3f4105f20 0x7fd3f4108380 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec00b0a0 tx=0x7fd3ec00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:17:56.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.621+0000 7fd3fc052700 1 -- 172.21.15.52:0/3811997686 shutdown_connections 2023-11-27T15:17:56.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.621+0000 7fd3fc052700 1 --2- 172.21.15.52:0/3811997686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3f4100ce0 0x7fd3f41030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:56.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.621+0000 7fd3fc052700 1 --2- 172.21.15.52:0/3811997686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3f4105f20 0x7fd3f4108380 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:56.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.621+0000 7fd3fc052700 1 --2- 172.21.15.52:0/3811997686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3f4103600 0x7fd3f41059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:56.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.621+0000 7fd3fc052700 1 -- 172.21.15.52:0/3811997686 >> 172.21.15.52:0/3811997686 conn(0x7fd3f40fa780 msgr2=0x7fd3f40fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:56.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.621+0000 7fd3fc052700 1 -- 172.21.15.52:0/3811997686 shutdown_connections 2023-11-27T15:17:56.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.621+0000 7fd3fc052700 1 -- 172.21.15.52:0/3811997686 wait complete. 2023-11-27T15:17:56.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.621+0000 7fd3fc052700 1 Processor -- start 2023-11-27T15:17:56.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.622+0000 7fd3fc052700 1 -- start start 2023-11-27T15:17:56.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.622+0000 7fd3fc052700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3f4100ce0 0x7fd3f4100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:56.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.622+0000 7fd3fc052700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3f4103600 0x7fd3f40fec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:56.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.622+0000 7fd3fc052700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3f4105f20 0x7fd3f40ff1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:56.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.622+0000 7fd3fc052700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f40ff970 con 0x7fd3f4105f20 2023-11-27T15:17:56.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.622+0000 7fd3fc052700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f40ffae0 con 0x7fd3f4103600 2023-11-27T15:17:56.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.622+0000 7fd3fc052700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f41a0110 con 0x7fd3f4100ce0 2023-11-27T15:17:56.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.622+0000 7fd3fa5ef700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3f4105f20 0x7fd3f40ff1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:56.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.622+0000 7fd3f9dee700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3f4100ce0 0x7fd3f4100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:56.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.622+0000 7fd3f9dee700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3f4100ce0 0x7fd3f4100650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:38748/0 (socket says 172.21.15.52:38748) 2023-11-27T15:17:56.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.622+0000 7fd3fa5ef700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3f4105f20 0x7fd3f40ff1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52108/0 (socket says 172.21.15.52:52108) 2023-11-27T15:17:56.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.622+0000 7fd3f9dee700 1 -- 172.21.15.52:0/961028678 learned_addr learned my addr 172.21.15.52:0/961028678 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:56.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.622+0000 7fd3f95ed700 1 --2- 172.21.15.52:0/961028678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3f4103600 0x7fd3f40fec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:56.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.622+0000 7fd3fa5ef700 1 -- 172.21.15.52:0/961028678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3f4100ce0 msgr2=0x7fd3f4100650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:56.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.622+0000 7fd3fa5ef700 1 --2- 172.21.15.52:0/961028678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3f4100ce0 0x7fd3f4100650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:56.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.622+0000 7fd3fa5ef700 1 -- 172.21.15.52:0/961028678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3f4103600 msgr2=0x7fd3f40fec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:56.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.622+0000 7fd3fa5ef700 1 --2- 172.21.15.52:0/961028678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3f4103600 0x7fd3f40fec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:56.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.622+0000 7fd3fa5ef700 1 -- 172.21.15.52:0/961028678 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f0009520 con 0x7fd3f4105f20 2023-11-27T15:17:56.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.622+0000 7fd3fa5ef700 1 --2- 172.21.15.52:0/961028678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3f4105f20 0x7fd3f40ff1d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec009fd0 tx=0x7fd3ec007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:56.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.623+0000 7fd3eaffd700 1 -- 172.21.15.52:0/961028678 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec00e040 con 0x7fd3f4105f20 2023-11-27T15:17:56.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.623+0000 7fd3fc052700 1 -- 172.21.15.52:0/961028678 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3ec009d00 con 0x7fd3f4105f20 2023-11-27T15:17:56.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.623+0000 7fd3eaffd700 1 -- 172.21.15.52:0/961028678 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3ec004c50 con 0x7fd3f4105f20 2023-11-27T15:17:56.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.623+0000 7fd3eaffd700 1 -- 172.21.15.52:0/961028678 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec01b6c0 con 0x7fd3f4105f20 2023-11-27T15:17:56.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.623+0000 7fd3fc052700 1 -- 172.21.15.52:0/961028678 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f410bc10 con 0x7fd3f4105f20 2023-11-27T15:17:56.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.624+0000 7fd3fc052700 1 -- 172.21.15.52:0/961028678 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3f410bda0 con 0x7fd3f4105f20 2023-11-27T15:17:56.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.625+0000 7fd3eaffd700 1 -- 172.21.15.52:0/961028678 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 23) v1 ==== 135527+0+0 (secure 0 0 0) 0x7fd3ec01d470 con 0x7fd3f4105f20 2023-11-27T15:17:56.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.626+0000 7fd3eaffd700 1 --2- 172.21.15.52:0/961028678 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd3e00a0f50 0x7fd3e00a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:56.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.626+0000 7fd3f9dee700 1 --2- 172.21.15.52:0/961028678 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd3e00a0f50 0x7fd3e00a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:56.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.626+0000 7fd3eaffd700 1 -- 172.21.15.52:0/961028678 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fd3ec0cd890 con 0x7fd3f4105f20 2023-11-27T15:17:56.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.626+0000 7fd3f9dee700 1 --2- 172.21.15.52:0/961028678 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd3e00a0f50 0x7fd3e00a3400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0009ef0 tx=0x7fd3f0009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:56.638 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.634+0000 7fd3eaffd700 1 -- 172.21.15.52:0/961028678 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd3ec097240 con 0x7fd3f4105f20 2023-11-27T15:17:56.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.872+0000 7fd3fc052700 1 -- 172.21.15.52:0/961028678 --> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3f4060f30 con 0x7fd3e00a0f50 2023-11-27T15:17:56.886 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.881+0000 7fd3eaffd700 1 -- 172.21.15.52:0/961028678 <== mgr.15387 v2:172.21.15.52:6800/1148545855 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7fd3f4060f30 con 0x7fd3e00a0f50 2023-11-27T15:17:56.886 INFO:teuthology.orchestra.run.smithi052.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-11-27T00:00:00", "created": "2023-11-27T15:17:55", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2023-11-27T15:17:56.888 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.884+0000 7fd3fc052700 1 -- 172.21.15.52:0/961028678 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd3e00a0f50 msgr2=0x7fd3e00a3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:56.888 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.884+0000 7fd3fc052700 1 --2- 172.21.15.52:0/961028678 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd3e00a0f50 0x7fd3e00a3400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0009ef0 tx=0x7fd3f0009260 comp rx=0 tx=0).stop 2023-11-27T15:17:56.888 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.884+0000 7fd3fc052700 1 -- 172.21.15.52:0/961028678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3f4105f20 msgr2=0x7fd3f40ff1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:56.888 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.884+0000 7fd3fc052700 1 --2- 172.21.15.52:0/961028678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3f4105f20 0x7fd3f40ff1d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec009fd0 tx=0x7fd3ec007930 comp rx=0 tx=0).stop 2023-11-27T15:17:56.889 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.884+0000 7fd3fc052700 1 -- 172.21.15.52:0/961028678 shutdown_connections 2023-11-27T15:17:56.889 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.884+0000 7fd3fc052700 1 --2- 172.21.15.52:0/961028678 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd3e00a0f50 0x7fd3e00a3400 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:56.889 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.884+0000 7fd3fc052700 1 --2- 172.21.15.52:0/961028678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3f4105f20 0x7fd3f40ff1d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:56.889 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.884+0000 7fd3fc052700 1 --2- 172.21.15.52:0/961028678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3f4100ce0 0x7fd3f4100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:56.889 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.884+0000 7fd3fc052700 1 --2- 172.21.15.52:0/961028678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3f4103600 0x7fd3f40fec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:56.889 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.884+0000 7fd3fc052700 1 -- 172.21.15.52:0/961028678 >> 172.21.15.52:0/961028678 conn(0x7fd3f40fa780 msgr2=0x7fd3f4101940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:56.889 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.884+0000 7fd3fc052700 1 -- 172.21.15.52:0/961028678 shutdown_connections 2023-11-27T15:17:56.889 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:56.884+0000 7fd3fc052700 1 -- 172.21.15.52:0/961028678 wait complete. 2023-11-27T15:17:56.901 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-11-27T15:17:57.036 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.031+0000 7fef89eec700 1 -- 172.21.15.52:0/659006772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef840fe9d0 msgr2=0x7fef840fee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:57.036 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.031+0000 7fef89eec700 1 --2- 172.21.15.52:0/659006772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef840fe9d0 0x7fef840fee20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00b0a0 tx=0x7fef7c00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:17:57.036 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.031+0000 7fef89eec700 1 -- 172.21.15.52:0/659006772 shutdown_connections 2023-11-27T15:17:57.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.031+0000 7fef89eec700 1 --2- 172.21.15.52:0/659006772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef840fe9d0 0x7fef840fee20 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:57.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.031+0000 7fef89eec700 1 --2- 172.21.15.52:0/659006772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef84101940 0x7fef84101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:57.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.031+0000 7fef89eec700 1 --2- 172.21.15.52:0/659006772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef84102290 0x7fef840fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:57.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.031+0000 7fef89eec700 1 -- 172.21.15.52:0/659006772 >> 172.21.15.52:0/659006772 conn(0x7fef840f9f80 msgr2=0x7fef840fc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:57.038 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.031+0000 7fef89eec700 1 -- 172.21.15.52:0/659006772 shutdown_connections 2023-11-27T15:17:57.038 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.031+0000 7fef89eec700 1 -- 172.21.15.52:0/659006772 wait complete. 2023-11-27T15:17:57.038 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.032+0000 7fef89eec700 1 Processor -- start 2023-11-27T15:17:57.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.032+0000 7fef89eec700 1 -- start start 2023-11-27T15:17:57.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.032+0000 7fef89eec700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef840fe9d0 0x7fef84101040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:57.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.032+0000 7fef89eec700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef84101940 0x7fef840ff680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:57.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.032+0000 7fef89eec700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef84102290 0x7fef840ffbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:57.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.032+0000 7fef89eec700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef84100360 con 0x7fef84102290 2023-11-27T15:17:57.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.032+0000 7fef89eec700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef841004d0 con 0x7fef840fe9d0 2023-11-27T15:17:57.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.032+0000 7fef89eec700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef841a0140 con 0x7fef84101940 2023-11-27T15:17:57.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.032+0000 7fef837fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef840fe9d0 0x7fef84101040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:57.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.032+0000 7fef83fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef84102290 0x7fef840ffbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:57.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.032+0000 7fef837fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef840fe9d0 0x7fef84101040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:41778/0 (socket says 172.21.15.52:41778) 2023-11-27T15:17:57.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.032+0000 7fef837fe700 1 -- 172.21.15.52:0/1060667226 learned_addr learned my addr 172.21.15.52:0/1060667226 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:17:57.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.033+0000 7fef82ffd700 1 --2- 172.21.15.52:0/1060667226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef84101940 0x7fef840ff680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:57.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.033+0000 7fef837fe700 1 -- 172.21.15.52:0/1060667226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef84101940 msgr2=0x7fef840ff680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:57.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.033+0000 7fef837fe700 1 --2- 172.21.15.52:0/1060667226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef84101940 0x7fef840ff680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:57.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.033+0000 7fef837fe700 1 -- 172.21.15.52:0/1060667226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef84102290 msgr2=0x7fef840ffbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:57.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.033+0000 7fef837fe700 1 --2- 172.21.15.52:0/1060667226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef84102290 0x7fef840ffbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:57.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.033+0000 7fef837fe700 1 -- 172.21.15.52:0/1060667226 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7c009d00 con 0x7fef840fe9d0 2023-11-27T15:17:57.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.033+0000 7fef837fe700 1 --2- 172.21.15.52:0/1060667226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef840fe9d0 0x7fef84101040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fef7400c920 tx=0x7fef7400cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:57.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.033+0000 7fef80ff9700 1 -- 172.21.15.52:0/1060667226 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef74004a30 con 0x7fef840fe9d0 2023-11-27T15:17:57.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.033+0000 7fef89eec700 1 -- 172.21.15.52:0/1060667226 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef8410bc80 con 0x7fef840fe9d0 2023-11-27T15:17:57.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.033+0000 7fef89eec700 1 -- 172.21.15.52:0/1060667226 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8410baf0 con 0x7fef840fe9d0 2023-11-27T15:17:57.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.034+0000 7fef83fff700 1 --2- 172.21.15.52:0/1060667226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef84102290 0x7fef840ffbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:17:57.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.035+0000 7fef89eec700 1 -- 172.21.15.52:0/1060667226 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef841070b0 con 0x7fef840fe9d0 2023-11-27T15:17:57.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.035+0000 7fef80ff9700 1 -- 172.21.15.52:0/1060667226 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef740054e0 con 0x7fef840fe9d0 2023-11-27T15:17:57.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.035+0000 7fef80ff9700 1 -- 172.21.15.52:0/1060667226 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef74007c70 con 0x7fef840fe9d0 2023-11-27T15:17:57.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.037+0000 7fef80ff9700 1 -- 172.21.15.52:0/1060667226 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 23) v1 ==== 135527+0+0 (secure 0 0 0) 0x7fef74004c20 con 0x7fef840fe9d0 2023-11-27T15:17:57.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.037+0000 7fef80ff9700 1 --2- 172.21.15.52:0/1060667226 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fef6c0a0f50 0x7fef6c0a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:57.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.037+0000 7fef80ff9700 1 -- 172.21.15.52:0/1060667226 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fef740cc7a0 con 0x7fef840fe9d0 2023-11-27T15:17:57.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.037+0000 7fef82ffd700 1 --2- 172.21.15.52:0/1060667226 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fef6c0a0f50 0x7fef6c0a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:57.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.038+0000 7fef82ffd700 1 --2- 172.21.15.52:0/1060667226 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fef6c0a0f50 0x7fef6c0a3400 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fef78009c60 tx=0x7fef78009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:57.044 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.040+0000 7fef80ff9700 1 -- 172.21.15.52:0/1060667226 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fef740960c0 con 0x7fef840fe9d0 2023-11-27T15:17:57.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:56 smithi150 ceph-mon[100559]: mgrmap e23: x(active, since 1.0268s), standbys: y, z 2023-11-27T15:17:57.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:56 smithi150 ceph-mon[100559]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:57.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:56 smithi150 ceph-mon[100559]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:17:57.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:56 smithi150 ceph-mon[100559]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:57.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:56 smithi150 ceph-mon[100559]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:57.285 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.281+0000 7fef89eec700 1 -- 172.21.15.52:0/1060667226 --> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fef841a0420 con 0x7fef6c0a0f50 2023-11-27T15:17:57.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.288+0000 7fef80ff9700 1 -- 172.21.15.52:0/1060667226 <== mgr.15387 v2:172.21.15.52:6800/1148545855 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fef841a0420 con 0x7fef6c0a0f50 2023-11-27T15:17:57.292 INFO:teuthology.orchestra.run.smithi052.stdout:/ 1M 6M3h 2023-11-27T15:17:57.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.290+0000 7fef89eec700 1 -- 172.21.15.52:0/1060667226 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fef6c0a0f50 msgr2=0x7fef6c0a3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:57.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.290+0000 7fef89eec700 1 --2- 172.21.15.52:0/1060667226 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fef6c0a0f50 0x7fef6c0a3400 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fef78009c60 tx=0x7fef78009260 comp rx=0 tx=0).stop 2023-11-27T15:17:57.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.290+0000 7fef89eec700 1 -- 172.21.15.52:0/1060667226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef840fe9d0 msgr2=0x7fef84101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:57.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.290+0000 7fef89eec700 1 --2- 172.21.15.52:0/1060667226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef840fe9d0 0x7fef84101040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fef7400c920 tx=0x7fef7400cc30 comp rx=0 tx=0).stop 2023-11-27T15:17:57.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.290+0000 7fef89eec700 1 -- 172.21.15.52:0/1060667226 shutdown_connections 2023-11-27T15:17:57.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.290+0000 7fef89eec700 1 --2- 172.21.15.52:0/1060667226 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fef6c0a0f50 0x7fef6c0a3400 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:57.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.290+0000 7fef89eec700 1 --2- 172.21.15.52:0/1060667226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef840fe9d0 0x7fef84101040 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:57.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.290+0000 7fef89eec700 1 --2- 172.21.15.52:0/1060667226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef84102290 0x7fef840ffbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:57.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.290+0000 7fef89eec700 1 --2- 172.21.15.52:0/1060667226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef84101940 0x7fef840ff680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:57.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.290+0000 7fef89eec700 1 -- 172.21.15.52:0/1060667226 >> 172.21.15.52:0/1060667226 conn(0x7fef840f9f80 msgr2=0x7fef840fbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:57.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.290+0000 7fef89eec700 1 -- 172.21.15.52:0/1060667226 shutdown_connections 2023-11-27T15:17:57.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:17:57.290+0000 7fef89eec700 1 -- 172.21.15.52:0/1060667226 wait complete. 2023-11-27T15:17:57.305 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:56 smithi052 ceph-mon[99056]: mgrmap e23: x(active, since 1.0268s), standbys: y, z 2023-11-27T15:17:57.305 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:56 smithi052 ceph-mon[99056]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:57.305 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:56 smithi052 ceph-mon[99056]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:17:57.306 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:56 smithi052 ceph-mon[99056]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:57.306 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:56 smithi052 ceph-mon[99056]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:57.307 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2023-11-27T15:17:57.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:56 smithi102 ceph-mon[100296]: mgrmap e23: x(active, since 1.0268s), standbys: y, z 2023-11-27T15:17:57.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:56 smithi102 ceph-mon[100296]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:57.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:56 smithi102 ceph-mon[100296]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:17:57.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:56 smithi102 ceph-mon[100296]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:57.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:56 smithi102 ceph-mon[100296]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:57.337 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2023-11-27T15:17:57.347 INFO:teuthology.run_tasks:Running task workunit... 2023-11-27T15:17:57.361 INFO:tasks.workunit:Pulling workunits from ref 5e4a743ad7fd1422ebd6c4dc950a21a303279207 2023-11-27T15:17:57.361 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-11-27T15:17:57.362 INFO:tasks.workunit:timeout=3h 2023-11-27T15:17:57.362 INFO:tasks.workunit:cleanup=True 2023-11-27T15:17:57.362 DEBUG:teuthology.orchestra.run.smithi052:> stat -- /home/ubuntu/cephtest/mnt.0 2023-11-27T15:17:57.395 INFO:teuthology.orchestra.run.smithi052.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-11-27T15:17:57.395 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-11-27T15:17:57.395 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 7ch/124d Inode: 1099511628283 Links: 2 2023-11-27T15:17:57.395 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:17:57.395 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:cephfs_t:s0 2023-11-27T15:17:57.396 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2023-11-27 15:16:15.968186162 +0000 2023-11-27T15:17:57.396 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2023-11-27 15:16:15.968186162 +0000 2023-11-27T15:17:57.396 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2023-11-27 15:17:38.473334021 +0000 2023-11-27T15:17:57.396 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2023-11-27 15:16:15.968186162 +0000 2023-11-27T15:17:57.396 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-11-27T15:17:57.396 DEBUG:teuthology.orchestra.run.smithi052:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-11-27T15:17:57.491 DEBUG:teuthology.orchestra.run.smithi052:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/chrisphoffman/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 5e4a743ad7fd1422ebd6c4dc950a21a303279207 2023-11-27T15:17:57.509 INFO:tasks.workunit.client.0.smithi052.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-11-27T15:17:58.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:57 smithi150 ceph-mon[100559]: [27/Nov/2023:15:17:56] ENGINE Bus STARTING 2023-11-27T15:17:58.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:57 smithi150 ceph-mon[100559]: [27/Nov/2023:15:17:57] ENGINE Serving on http://172.21.15.52:8765 2023-11-27T15:17:58.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:57 smithi150 ceph-mon[100559]: [27/Nov/2023:15:17:57] ENGINE Serving on https://172.21.15.52:7150 2023-11-27T15:17:58.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:57 smithi150 ceph-mon[100559]: [27/Nov/2023:15:17:57] ENGINE Bus STARTED 2023-11-27T15:17:58.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:57 smithi150 ceph-mon[100559]: from='client.25265 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:58.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:57 smithi102 ceph-mon[100296]: [27/Nov/2023:15:17:56] ENGINE Bus STARTING 2023-11-27T15:17:58.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:57 smithi102 ceph-mon[100296]: [27/Nov/2023:15:17:57] ENGINE Serving on http://172.21.15.52:8765 2023-11-27T15:17:58.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:57 smithi102 ceph-mon[100296]: [27/Nov/2023:15:17:57] ENGINE Serving on https://172.21.15.52:7150 2023-11-27T15:17:58.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:57 smithi102 ceph-mon[100296]: [27/Nov/2023:15:17:57] ENGINE Bus STARTED 2023-11-27T15:17:58.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:57 smithi102 ceph-mon[100296]: from='client.25265 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:58.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:57 smithi052 ceph-mon[99056]: [27/Nov/2023:15:17:56] ENGINE Bus STARTING 2023-11-27T15:17:58.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:57 smithi052 ceph-mon[99056]: [27/Nov/2023:15:17:57] ENGINE Serving on http://172.21.15.52:8765 2023-11-27T15:17:58.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:57 smithi052 ceph-mon[99056]: [27/Nov/2023:15:17:57] ENGINE Serving on https://172.21.15.52:7150 2023-11-27T15:17:58.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:57 smithi052 ceph-mon[99056]: [27/Nov/2023:15:17:57] ENGINE Bus STARTED 2023-11-27T15:17:58.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:57 smithi052 ceph-mon[99056]: from='client.25265 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:59.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:58 smithi150 ceph-mon[100559]: mgrmap e24: x(active, since 3s), standbys: y, z 2023-11-27T15:17:59.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:58 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:17:59.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:58 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:17:59.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:58 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:17:59.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:58 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:17:59.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:58 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:17:59.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:17:58 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:17:59.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:58 smithi102 ceph-mon[100296]: mgrmap e24: x(active, since 3s), standbys: y, z 2023-11-27T15:17:59.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:58 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:17:59.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:58 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:17:59.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:58 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:17:59.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:58 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:17:59.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:58 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:17:59.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:17:58 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:17:59.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:58 smithi052 ceph-mon[99056]: mgrmap e24: x(active, since 3s), standbys: y, z 2023-11-27T15:17:59.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:58 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:17:59.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:58 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:17:59.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:58 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:17:59.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:58 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:17:59.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:58 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:17:59.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:17:58 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:00.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:18:00.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:00.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:00.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: Adjusting osd_memory_target on smithi052 to 197.2M 2023-11-27T15:18:00.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: Unable to set osd_memory_target on smithi052 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-11-27T15:18:00.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:00.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:00.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:00.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:00.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: Adjusting osd_memory_target on smithi102 to 196.2M 2023-11-27T15:18:00.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: Unable to set osd_memory_target on smithi102 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-11-27T15:18:00.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: Adjusting osd_memory_target on smithi150 to 196.2M 2023-11-27T15:18:00.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: Unable to set osd_memory_target on smithi150 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-11-27T15:18:00.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:00.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:00.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:18:00.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: Updating smithi052:/etc/ceph/ceph.conf 2023-11-27T15:18:00.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: Updating smithi102:/etc/ceph/ceph.conf 2023-11-27T15:18:00.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: Updating smithi150:/etc/ceph/ceph.conf 2023-11-27T15:18:00.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: Updating smithi102:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:18:00.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: Updating smithi150:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:18:00.781 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:00 smithi150 ceph-mon[100559]: Updating smithi052:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:18:00.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:18:00.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:00.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:00.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: Adjusting osd_memory_target on smithi052 to 197.2M 2023-11-27T15:18:00.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: Unable to set osd_memory_target on smithi052 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-11-27T15:18:00.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:00.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:00.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:00.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:00.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: Adjusting osd_memory_target on smithi102 to 196.2M 2023-11-27T15:18:00.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: Unable to set osd_memory_target on smithi102 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-11-27T15:18:00.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: Adjusting osd_memory_target on smithi150 to 196.2M 2023-11-27T15:18:00.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: Unable to set osd_memory_target on smithi150 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-11-27T15:18:00.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:00.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:00.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:18:00.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: Updating smithi052:/etc/ceph/ceph.conf 2023-11-27T15:18:00.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: Updating smithi102:/etc/ceph/ceph.conf 2023-11-27T15:18:00.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: Updating smithi150:/etc/ceph/ceph.conf 2023-11-27T15:18:00.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: Updating smithi102:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:18:00.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: Updating smithi150:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:18:00.830 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:00 smithi102 ceph-mon[100296]: Updating smithi052:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:18:00.866 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:18:00.486+0000 7f3f907c2700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-27T15:18:00.867 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:18:00.486+0000 7f3f907c2700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-27T15:18:00.867 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:18:00.486+0000 7f3f907c2700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-27T15:18:00.867 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:18:00.486+0000 7f3f907c2700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-27T15:18:00.867 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:18:00.486+0000 7f3f907c2700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-27T15:18:00.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:18:00.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:00.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:00.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: Adjusting osd_memory_target on smithi052 to 197.2M 2023-11-27T15:18:00.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: Unable to set osd_memory_target on smithi052 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-11-27T15:18:00.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:00.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:00.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:00.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:00.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: Adjusting osd_memory_target on smithi102 to 196.2M 2023-11-27T15:18:00.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:00.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: Unable to set osd_memory_target on smithi102 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-11-27T15:18:00.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: Adjusting osd_memory_target on smithi150 to 196.2M 2023-11-27T15:18:00.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: Unable to set osd_memory_target on smithi150 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-11-27T15:18:00.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:00.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:00.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:18:00.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: Updating smithi052:/etc/ceph/ceph.conf 2023-11-27T15:18:00.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: Updating smithi102:/etc/ceph/ceph.conf 2023-11-27T15:18:00.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: Updating smithi150:/etc/ceph/ceph.conf 2023-11-27T15:18:00.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: Updating smithi102:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:18:00.870 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: Updating smithi150:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:18:00.870 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:00 smithi052 ceph-mon[99056]: Updating smithi052:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:18:01.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:01 smithi150 ceph-mon[100559]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:18:01.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:01 smithi150 ceph-mon[100559]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:18:01.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:01 smithi150 ceph-mon[100559]: Updating smithi052:/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:18:01.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:01 smithi150 ceph-mon[100559]: Updating smithi102:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-11-27T15:18:01.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:01 smithi150 ceph-mon[100559]: Updating smithi150:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-11-27T15:18:01.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:01 smithi150 ceph-mon[100559]: Updating smithi052:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-11-27T15:18:01.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:01 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:01.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:01 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:01.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:01 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:01.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:01 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:01.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:01 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:01.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:01 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:01.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:01 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:01.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:01 smithi150 ceph-mon[100559]: Service mds.all has no related daemon already placed. 2023-11-27T15:18:01.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:01 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:18:01.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:01 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:18:01.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:01 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:01.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:01 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:01.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:01 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:01.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:01 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:18:01.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:01 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:01.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:01 smithi150 ceph-mon[100559]: Service mds.all has no related daemon already placed. 2023-11-27T15:18:01.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:01 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:18:01.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:01 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:18:01.827 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:01 smithi102 ceph-mon[100296]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:18:01.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:01 smithi102 ceph-mon[100296]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:18:01.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:01 smithi102 ceph-mon[100296]: Updating smithi052:/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:18:01.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:01 smithi102 ceph-mon[100296]: Updating smithi102:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-11-27T15:18:01.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:01 smithi102 ceph-mon[100296]: Updating smithi150:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-11-27T15:18:01.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:01 smithi102 ceph-mon[100296]: Updating smithi052:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-11-27T15:18:01.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:01 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:01.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:01 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:01.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:01 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:01.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:01 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:01.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:01 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:01.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:01 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:01.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:01 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:01.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:01 smithi102 ceph-mon[100296]: Service mds.all has no related daemon already placed. 2023-11-27T15:18:01.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:01 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:18:01.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:01 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:18:01.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:01 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:01.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:01 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:01.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:01 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:01.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:01 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:18:01.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:01 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:01.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:01 smithi102 ceph-mon[100296]: Service mds.all has no related daemon already placed. 2023-11-27T15:18:01.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:01 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:18:01.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:01 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:18:01.866 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:01 smithi052 ceph-mon[99056]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:18:01.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:01 smithi052 ceph-mon[99056]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:18:01.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:01 smithi052 ceph-mon[99056]: Updating smithi052:/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:18:01.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:01 smithi052 ceph-mon[99056]: Updating smithi102:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-11-27T15:18:01.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:01 smithi052 ceph-mon[99056]: Updating smithi150:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-11-27T15:18:01.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:01 smithi052 ceph-mon[99056]: Updating smithi052:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-11-27T15:18:01.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:01 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:01.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:01 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:01.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:01 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:01.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:01 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:01.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:01 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:01.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:01 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:01.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:01 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:01.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:01 smithi052 ceph-mon[99056]: Service mds.all has no related daemon already placed. 2023-11-27T15:18:01.868 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:01 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:18:01.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:01 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:18:01.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:01 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:01.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:01 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:01.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:01 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:01.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:01 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:18:01.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:01 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:18:01.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:01 smithi052 ceph-mon[99056]: Service mds.all has no related daemon already placed. 2023-11-27T15:18:01.869 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:01 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:18:01.870 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:01 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:18:03.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:02 smithi052 ceph-mon[99056]: pgmap v6: 97 pgs: 97 active+clean; 1.0 MiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s rd, 205 B/s wr, 1 op/s 2023-11-27T15:18:03.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:02 smithi052 ceph-mon[99056]: mgrmap e25: x(active, since 6s), standbys: y, z 2023-11-27T15:18:03.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:02 smithi150 ceph-mon[100559]: pgmap v6: 97 pgs: 97 active+clean; 1.0 MiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s rd, 205 B/s wr, 1 op/s 2023-11-27T15:18:03.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:02 smithi150 ceph-mon[100559]: mgrmap e25: x(active, since 6s), standbys: y, z 2023-11-27T15:18:03.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:02 smithi102 ceph-mon[100296]: pgmap v6: 97 pgs: 97 active+clean; 1.0 MiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s rd, 205 B/s wr, 1 op/s 2023-11-27T15:18:03.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:02 smithi102 ceph-mon[100296]: mgrmap e25: x(active, since 6s), standbys: y, z 2023-11-27T15:18:05.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:04 smithi052 ceph-mon[99056]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 170 KiB/s wr, 72 op/s 2023-11-27T15:18:05.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:04 smithi150 ceph-mon[100559]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 170 KiB/s wr, 72 op/s 2023-11-27T15:18:05.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:04 smithi102 ceph-mon[100296]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 170 KiB/s wr, 72 op/s 2023-11-27T15:18:07.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:06 smithi052 ceph-mon[99056]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 132 KiB/s wr, 56 op/s 2023-11-27T15:18:07.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:06 smithi150 ceph-mon[100559]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 132 KiB/s wr, 56 op/s 2023-11-27T15:18:07.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:06 smithi102 ceph-mon[100296]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 132 KiB/s wr, 56 op/s 2023-11-27T15:18:09.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:08 smithi052 ceph-mon[99056]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 109 KiB/s wr, 48 op/s 2023-11-27T15:18:09.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:08 smithi150 ceph-mon[100559]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 109 KiB/s wr, 48 op/s 2023-11-27T15:18:09.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:08 smithi102 ceph-mon[100296]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 109 KiB/s wr, 48 op/s 2023-11-27T15:18:11.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:10 smithi052 ceph-mon[99056]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 100 KiB/s wr, 44 op/s 2023-11-27T15:18:11.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:10 smithi150 ceph-mon[100559]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 100 KiB/s wr, 44 op/s 2023-11-27T15:18:11.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:10 smithi102 ceph-mon[100296]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 100 KiB/s wr, 44 op/s 2023-11-27T15:18:13.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:12 smithi052 ceph-mon[99056]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 100 KiB/s wr, 45 op/s 2023-11-27T15:18:13.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:12 smithi150 ceph-mon[100559]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 100 KiB/s wr, 45 op/s 2023-11-27T15:18:13.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:12 smithi102 ceph-mon[100296]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 100 KiB/s wr, 45 op/s 2023-11-27T15:18:15.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:14 smithi052 ceph-mon[99056]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 100 KiB/s wr, 46 op/s 2023-11-27T15:18:15.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:14 smithi150 ceph-mon[100559]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 100 KiB/s wr, 46 op/s 2023-11-27T15:18:15.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:14 smithi102 ceph-mon[100296]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 100 KiB/s wr, 46 op/s 2023-11-27T15:18:15.820 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:18:15.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.978+0000 7fa722ac7700 1 -- 172.21.15.52:0/1732066585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa71c0ffa00 msgr2=0x7fa71c10c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:15.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.978+0000 7fa722ac7700 1 --2- 172.21.15.52:0/1732066585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa71c0ffa00 0x7fa71c10c760 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa708009990 tx=0x7fa708009ca0 comp rx=0 tx=0).stop 2023-11-27T15:18:15.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.978+0000 7fa722ac7700 1 -- 172.21.15.52:0/1732066585 shutdown_connections 2023-11-27T15:18:15.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.978+0000 7fa722ac7700 1 --2- 172.21.15.52:0/1732066585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa71c0ff0f0 0x7fa71c0ff4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:15.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.978+0000 7fa722ac7700 1 --2- 172.21.15.52:0/1732066585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa71c10cca0 0x7fa71c10f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:15.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.978+0000 7fa722ac7700 1 --2- 172.21.15.52:0/1732066585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa71c0ffa00 0x7fa71c10c760 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:15.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.978+0000 7fa722ac7700 1 -- 172.21.15.52:0/1732066585 >> 172.21.15.52:0/1732066585 conn(0x7fa71c0770b0 msgr2=0x7fa71c0774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:15.985 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.978+0000 7fa722ac7700 1 -- 172.21.15.52:0/1732066585 shutdown_connections 2023-11-27T15:18:15.985 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.978+0000 7fa722ac7700 1 -- 172.21.15.52:0/1732066585 wait complete. 2023-11-27T15:18:15.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.979+0000 7fa722ac7700 1 Processor -- start 2023-11-27T15:18:15.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.979+0000 7fa722ac7700 1 -- start start 2023-11-27T15:18:15.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.980+0000 7fa722ac7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa71c0ff0f0 0x7fa71c198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:15.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.980+0000 7fa722ac7700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa71c0ffa00 0x7fa71c198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:15.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.980+0000 7fa722ac7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa71c10cca0 0x7fa71c19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:15.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.980+0000 7fa722ac7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c19d630 con 0x7fa71c10cca0 2023-11-27T15:18:15.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.980+0000 7fa722ac7700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c19d7a0 con 0x7fa71c0ffa00 2023-11-27T15:18:15.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.980+0000 7fa722ac7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c19d960 con 0x7fa71c0ff0f0 2023-11-27T15:18:15.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.980+0000 7fa721064700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa71c10cca0 0x7fa71c19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:15.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.980+0000 7fa71bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa71c0ffa00 0x7fa71c198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:15.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.980+0000 7fa721064700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa71c10cca0 0x7fa71c19cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52164/0 (socket says 172.21.15.52:52164) 2023-11-27T15:18:15.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.980+0000 7fa721064700 1 -- 172.21.15.52:0/2808429202 learned_addr learned my addr 172.21.15.52:0/2808429202 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:15.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.980+0000 7fa720863700 1 --2- 172.21.15.52:0/2808429202 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa71c0ff0f0 0x7fa71c198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:15.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.980+0000 7fa721064700 1 -- 172.21.15.52:0/2808429202 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa71c0ff0f0 msgr2=0x7fa71c198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:15.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.980+0000 7fa721064700 1 --2- 172.21.15.52:0/2808429202 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa71c0ff0f0 0x7fa71c198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:15.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.980+0000 7fa721064700 1 -- 172.21.15.52:0/2808429202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa71c0ffa00 msgr2=0x7fa71c198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:15.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.980+0000 7fa721064700 1 --2- 172.21.15.52:0/2808429202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa71c0ffa00 0x7fa71c198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:15.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.980+0000 7fa721064700 1 -- 172.21.15.52:0/2808429202 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7080095f0 con 0x7fa71c10cca0 2023-11-27T15:18:15.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.980+0000 7fa721064700 1 --2- 172.21.15.52:0/2808429202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa71c10cca0 0x7fa71c19cff0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa71400ef90 tx=0x7fa71400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:15.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.980+0000 7fa719ffb700 1 -- 172.21.15.52:0/2808429202 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa714018070 con 0x7fa71c10cca0 2023-11-27T15:18:15.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.981+0000 7fa722ac7700 1 -- 172.21.15.52:0/2808429202 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa71c1125a0 con 0x7fa71c10cca0 2023-11-27T15:18:15.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.981+0000 7fa719ffb700 1 -- 172.21.15.52:0/2808429202 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa71400f040 con 0x7fa71c10cca0 2023-11-27T15:18:15.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.981+0000 7fa719ffb700 1 -- 172.21.15.52:0/2808429202 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71400cda0 con 0x7fa71c10cca0 2023-11-27T15:18:15.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.981+0000 7fa722ac7700 1 -- 172.21.15.52:0/2808429202 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa71c112410 con 0x7fa71c10cca0 2023-11-27T15:18:15.989 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.984+0000 7fa719ffb700 1 -- 172.21.15.52:0/2808429202 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7fa714003a00 con 0x7fa71c10cca0 2023-11-27T15:18:15.991 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.984+0000 7fa72ec53b80 1 Processor -- start 2023-11-27T15:18:15.991 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.984+0000 7fa72ec53b80 1 -- start start 2023-11-27T15:18:15.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.985+0000 7fa719ffb700 1 --2- 172.21.15.52:0/2808429202 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fa70c0a13e0 0x7fa70c0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:15.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.985+0000 7fa72ec53b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f0a87b60 0x5654f0a87f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:15.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.985+0000 7fa72ec53b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5654f0a88470 0x5654f0a8c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:15.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.985+0000 7fa72ec53b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654f0a8c8b0 0x5654f0a8cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:15.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.985+0000 7fa72ec53b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654f098d060 con 0x5654f0a87b60 2023-11-27T15:18:15.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.985+0000 7fa72ec53b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654f09df8a0 con 0x5654f0a88470 2023-11-27T15:18:15.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.985+0000 7fa72ec53b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654f09d8d30 con 0x5654f0a8c8b0 2023-11-27T15:18:15.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.985+0000 7fa719ffb700 1 -- 172.21.15.52:0/2808429202 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fa7140ccac0 con 0x7fa71c10cca0 2023-11-27T15:18:15.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.985+0000 7fa71bfff700 1 --2- 172.21.15.52:0/2808429202 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fa70c0a13e0 0x7fa70c0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:15.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.985+0000 7fa720863700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f0a87b60 0x5654f0a87f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:15.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.985+0000 7fa720863700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f0a87b60 0x5654f0a87f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52168/0 (socket says 172.21.15.52:52168) 2023-11-27T15:18:15.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.985+0000 7fa720863700 1 -- 172.21.15.52:0/553561677 learned_addr learned my addr 172.21.15.52:0/553561677 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:15.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.985+0000 7fa71bfff700 1 --2- 172.21.15.52:0/553561677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5654f0a88470 0x5654f0a8c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:15.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.985+0000 7fa721064700 1 --2- 172.21.15.52:0/553561677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654f0a8c8b0 0x5654f0a8cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:15.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.985+0000 7fa720863700 1 -- 172.21.15.52:0/553561677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654f0a8c8b0 msgr2=0x5654f0a8cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:15.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.985+0000 7fa720863700 1 --2- 172.21.15.52:0/553561677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654f0a8c8b0 0x5654f0a8cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:15.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.985+0000 7fa720863700 1 -- 172.21.15.52:0/553561677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5654f0a88470 msgr2=0x5654f0a8c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:15.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.985+0000 7fa720863700 1 --2- 172.21.15.52:0/553561677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5654f0a88470 0x5654f0a8c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:15.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.985+0000 7fa720863700 1 -- 172.21.15.52:0/553561677 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654f085a630 con 0x5654f0a87b60 2023-11-27T15:18:15.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.985+0000 7fa71bfff700 1 --2- 172.21.15.52:0/2808429202 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fa70c0a13e0 0x7fa70c0a3890 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa71c199c60 tx=0x7fa70800bbb0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:15.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.985+0000 7fa720863700 1 --2- 172.21.15.52:0/553561677 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f0a87b60 0x5654f0a87f30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa71000bd70 tx=0x7fa7100106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a3307f1902485 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:15.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.986+0000 7fa706ffd700 1 -- 172.21.15.52:0/553561677 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa710010e40 con 0x5654f0a87b60 2023-11-27T15:18:15.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.986+0000 7fa706ffd700 1 -- 172.21.15.52:0/553561677 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa710015070 con 0x5654f0a87b60 2023-11-27T15:18:15.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.986+0000 7fa706ffd700 1 -- 172.21.15.52:0/553561677 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa710011680 con 0x5654f0a87b60 2023-11-27T15:18:15.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.986+0000 7fa72ec53b80 1 -- 172.21.15.52:0/553561677 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f0a87b60 msgr2=0x5654f0a87f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:15.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.986+0000 7fa72ec53b80 1 --2- 172.21.15.52:0/553561677 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f0a87b60 0x5654f0a87f30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa71000bd70 tx=0x7fa7100106b0 comp rx=0 tx=0).stop 2023-11-27T15:18:15.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.986+0000 7fa72ec53b80 1 -- 172.21.15.52:0/553561677 shutdown_connections 2023-11-27T15:18:15.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.986+0000 7fa72ec53b80 1 --2- 172.21.15.52:0/553561677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654f0a8c8b0 0x5654f0a8cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:15.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.986+0000 7fa72ec53b80 1 --2- 172.21.15.52:0/553561677 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f0a87b60 0x5654f0a87f30 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:15.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.986+0000 7fa72ec53b80 1 --2- 172.21.15.52:0/553561677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5654f0a88470 0x5654f0a8c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:15.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.986+0000 7fa72ec53b80 1 -- 172.21.15.52:0/553561677 >> 172.21.15.52:0/553561677 conn(0x5654f0a7d360 msgr2=0x5654f0a82b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:15.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.987+0000 7fa72ec53b80 1 -- 172.21.15.52:0/553561677 shutdown_connections 2023-11-27T15:18:15.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.987+0000 7fa72ec53b80 1 -- 172.21.15.52:0/553561677 wait complete. 2023-11-27T15:18:15.998 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.987+0000 7fa72ec53b80 1 -- start start 2023-11-27T15:18:15.998 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.987+0000 7fa72ec53b80 1 Processor -- start 2023-11-27T15:18:15.998 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.988+0000 7fa72ec53b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f0a87b60 0x5654f0a94070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:15.998 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.988+0000 7fa72ec53b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654f0a88470 0x5654f0a912f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:15.998 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.988+0000 7fa72ec53b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5654f0a8c8b0 0x5654f0a91990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:15.998 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.988+0000 7fa72ec53b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654f097a8e0 con 0x5654f0a87b60 2023-11-27T15:18:15.998 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.988+0000 7fa72ec53b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654f097a580 con 0x5654f0a8c8b0 2023-11-27T15:18:15.998 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.988+0000 7fa72ec53b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654f085c920 con 0x5654f0a88470 2023-11-27T15:18:15.998 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.988+0000 7fa72ec53b80 10 client.0 _ll_register_callbacks cb 0x5654f0998780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:18:15.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.988+0000 7fa720863700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f0a87b60 0x5654f0a94070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:15.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.988+0000 7fa720863700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f0a87b60 0x5654f0a94070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52174/0 (socket says 172.21.15.52:52174) 2023-11-27T15:18:15.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.988+0000 7fa720863700 1 -- 172.21.15.52:0/1655228816 learned_addr learned my addr 172.21.15.52:0/1655228816 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:15.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.988+0000 7fa721064700 1 --2- 172.21.15.52:0/1655228816 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5654f0a8c8b0 0x5654f0a91990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:15.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.988+0000 7fa71bfff700 1 --2- 172.21.15.52:0/1655228816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654f0a88470 0x5654f0a912f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:15.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.988+0000 7fa720863700 1 -- 172.21.15.52:0/1655228816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654f0a88470 msgr2=0x5654f0a912f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:15.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.988+0000 7fa720863700 1 --2- 172.21.15.52:0/1655228816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654f0a88470 0x5654f0a912f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:15.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.988+0000 7fa720863700 1 -- 172.21.15.52:0/1655228816 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5654f0a8c8b0 msgr2=0x5654f0a91990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:15.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.988+0000 7fa720863700 1 --2- 172.21.15.52:0/1655228816 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5654f0a8c8b0 0x5654f0a91990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:15.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.988+0000 7fa720863700 1 -- 172.21.15.52:0/1655228816 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa71000b9a0 con 0x5654f0a87b60 2023-11-27T15:18:15.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.988+0000 7fa71bfff700 1 --2- 172.21.15.52:0/1655228816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654f0a88470 0x5654f0a912f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:18:15.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.988+0000 7fa720863700 1 --2- 172.21.15.52:0/1655228816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f0a87b60 0x5654f0a94070 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa71001cdc0 tx=0x7fa710010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be0e3dba39cd670 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:15.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.988+0000 7fa6fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:18:16.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.988+0000 7fa6fbfff700 1 -- 172.21.15.52:0/1655228816 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71001ca10 con 0x5654f0a87b60 2023-11-27T15:18:16.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.989+0000 7fa72ec53b80 1 -- 172.21.15.52:0/1655228816 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5654f0a7ab20 con 0x5654f0a87b60 2023-11-27T15:18:16.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.989+0000 7fa6fbfff700 1 -- 172.21.15.52:0/1655228816 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fa710017070 con 0x5654f0a87b60 2023-11-27T15:18:16.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.989+0000 7fa6fbfff700 1 -- 172.21.15.52:0/1655228816 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa71001cb70 con 0x5654f0a87b60 2023-11-27T15:18:16.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.989+0000 7fa72ec53b80 10 client.15483 fetch_fsmap learned FSMap version 23 2023-11-27T15:18:16.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.989+0000 7fa72ec53b80 1 -- 172.21.15.52:0/1655228816 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5654f099f190 con 0x5654f0a87b60 2023-11-27T15:18:16.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.989+0000 7fa6fbfff700 1 -- 172.21.15.52:0/1655228816 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5654f0a7ab20 con 0x5654f0a87b60 2023-11-27T15:18:16.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.989+0000 7fa6fbfff700 1 -- 172.21.15.52:0/1655228816 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa710027440 con 0x5654f0a87b60 2023-11-27T15:18:16.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.989+0000 7fa72ec53b80 10 client.15483 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:18:16.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.989+0000 7fa72ec53b80 10 client.15483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:18:16.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.989+0000 7fa72ec53b80 20 client.15483 populate_metadata read hostname 'smithi052' 2023-11-27T15:18:16.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.989+0000 7fa72ec53b80 1 --2- 172.21.15.52:0/1655228816 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5654f0aa1890 0x5654f0aa3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.989+0000 7fa72ec53b80 4 client.15483 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:18:16.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.989+0000 7fa72ec53b80 1 -- 172.21.15.52:0/1655228816 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5654f099b0a0 con 0x5654f0aa1890 2023-11-27T15:18:16.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.989+0000 7fa71bfff700 1 --2- 172.21.15.52:0/1655228816 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5654f0aa1890 0x5654f0aa3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.990+0000 7fa6fbfff700 10 client.15483 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:18:16.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.990+0000 7fa71bfff700 1 --2- 172.21.15.52:0/1655228816 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5654f0aa1890 0x5654f0aa3ca0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9363656277c719e server_cookie=bb821fdcc9ee1a69 in_seq=0 out_seq=0 2023-11-27T15:18:16.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.991+0000 7fa6fbfff700 1 -- 172.21.15.52:0/1655228816 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5654f099b0a0 con 0x5654f0aa1890 2023-11-27T15:18:16.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.991+0000 7fa6fbfff700 10 client.15483 handle_command_reply: tid=0 2023-11-27T15:18:16.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.991+0000 7fa72ec53b80 1 client.15483 shutdown 2023-11-27T15:18:16.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.991+0000 7fa72ec53b80 1 -- 172.21.15.52:0/1655228816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f0a87b60 msgr2=0x5654f0a94070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:16.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.991+0000 7fa72ec53b80 1 --2- 172.21.15.52:0/1655228816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f0a87b60 0x5654f0a94070 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa71001cdc0 tx=0x7fa710010ef0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.991+0000 7fa72ec53b80 1 -- 172.21.15.52:0/1655228816 shutdown_connections 2023-11-27T15:18:16.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.991+0000 7fa72ec53b80 1 --2- 172.21.15.52:0/1655228816 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5654f0aa1890 0x5654f0aa3ca0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.991+0000 7fa72ec53b80 1 --2- 172.21.15.52:0/1655228816 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5654f0a8c8b0 0x5654f0a91990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.991+0000 7fa72ec53b80 1 --2- 172.21.15.52:0/1655228816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f0a87b60 0x5654f0a94070 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.991+0000 7fa6fbfff700 0 client.15483 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:18:16.002 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.991+0000 7fa72ec53b80 1 --2- 172.21.15.52:0/1655228816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654f0a88470 0x5654f0a912f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.002 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.991+0000 7fa72ec53b80 1 -- 172.21.15.52:0/1655228816 >> 172.21.15.52:0/1655228816 conn(0x5654f0a7d360 msgr2=0x5654f0a81de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:16.002 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.991+0000 7fa72ec53b80 1 -- 172.21.15.52:0/1655228816 shutdown_connections 2023-11-27T15:18:16.002 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.991+0000 7fa72ec53b80 1 -- 172.21.15.52:0/1655228816 wait complete. 2023-11-27T15:18:16.002 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:15.991+0000 7fa72ec53b80 20 client.15483 trim_cache size 0 max 16384 2023-11-27T15:18:16.015 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.008+0000 7fa72ec53b80 1 Processor -- start 2023-11-27T15:18:16.015 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.008+0000 7fa72ec53b80 1 -- start start 2023-11-27T15:18:16.015 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.008+0000 7fa72ec53b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f0a87b60 0x5654f0a82560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.015 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.009+0000 7fa72ec53b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654f0a88470 0x5654f0a82aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.015 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.009+0000 7fa72ec53b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5654f0a8c8b0 0x5654f0a8e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.015 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.009+0000 7fa72ec53b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654f0a83950 con 0x5654f0a87b60 2023-11-27T15:18:16.015 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.009+0000 7fa72ec53b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654f0a8f850 con 0x5654f0a8c8b0 2023-11-27T15:18:16.015 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.009+0000 7fa72ec53b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654f0a92640 con 0x5654f0a88470 2023-11-27T15:18:16.015 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.009+0000 7fa720863700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f0a87b60 0x5654f0a82560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.015 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.009+0000 7fa720863700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f0a87b60 0x5654f0a82560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52182/0 (socket says 172.21.15.52:52182) 2023-11-27T15:18:16.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.009+0000 7fa720863700 1 -- 172.21.15.52:0/2286163950 learned_addr learned my addr 172.21.15.52:0/2286163950 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:16.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.009+0000 7fa721064700 1 --2- 172.21.15.52:0/2286163950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5654f0a8c8b0 0x5654f0a8e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.009+0000 7fa71bfff700 1 --2- 172.21.15.52:0/2286163950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654f0a88470 0x5654f0a82aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.009+0000 7fa720863700 1 -- 172.21.15.52:0/2286163950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654f0a88470 msgr2=0x5654f0a82aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:16.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.009+0000 7fa720863700 1 --2- 172.21.15.52:0/2286163950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654f0a88470 0x5654f0a82aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.009+0000 7fa720863700 1 -- 172.21.15.52:0/2286163950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5654f0a8c8b0 msgr2=0x5654f0a8e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:16.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.009+0000 7fa720863700 1 --2- 172.21.15.52:0/2286163950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5654f0a8c8b0 0x5654f0a8e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.009+0000 7fa720863700 1 -- 172.21.15.52:0/2286163950 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654f099eee0 con 0x5654f0a87b60 2023-11-27T15:18:16.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.009+0000 7fa721064700 1 --2- 172.21.15.52:0/2286163950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5654f0a8c8b0 0x5654f0a8e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:18:16.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.009+0000 7fa720863700 1 --2- 172.21.15.52:0/2286163950 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f0a87b60 0x5654f0a82560 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa710005790 tx=0x7fa71001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0f2ed66b5f2337 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:16.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.009+0000 7fa7077fe700 1 -- 172.21.15.52:0/2286163950 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa710027ac0 con 0x5654f0a87b60 2023-11-27T15:18:16.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.009+0000 7fa7077fe700 1 -- 172.21.15.52:0/2286163950 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa710011440 con 0x5654f0a87b60 2023-11-27T15:18:16.016 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.010+0000 7fa7077fe700 1 -- 172.21.15.52:0/2286163950 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa710030690 con 0x5654f0a87b60 2023-11-27T15:18:16.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.010+0000 7fa72ec53b80 1 -- 172.21.15.52:0/2286163950 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f0a87b60 msgr2=0x5654f0a82560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:16.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.010+0000 7fa72ec53b80 1 --2- 172.21.15.52:0/2286163950 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f0a87b60 0x5654f0a82560 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa710005790 tx=0x7fa71001c480 comp rx=0 tx=0).stop 2023-11-27T15:18:16.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.010+0000 7fa72ec53b80 1 -- 172.21.15.52:0/2286163950 shutdown_connections 2023-11-27T15:18:16.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.010+0000 7fa72ec53b80 1 --2- 172.21.15.52:0/2286163950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5654f0a8c8b0 0x5654f0a8e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.010+0000 7fa72ec53b80 1 --2- 172.21.15.52:0/2286163950 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f0a87b60 0x5654f0a82560 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.010+0000 7fa72ec53b80 1 --2- 172.21.15.52:0/2286163950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654f0a88470 0x5654f0a82aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.010+0000 7fa72ec53b80 1 -- 172.21.15.52:0/2286163950 >> 172.21.15.52:0/2286163950 conn(0x5654f0a7d360 msgr2=0x5654f0a8b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:16.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.010+0000 7fa72ec53b80 1 -- 172.21.15.52:0/2286163950 shutdown_connections 2023-11-27T15:18:16.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.010+0000 7fa72ec53b80 1 -- 172.21.15.52:0/2286163950 wait complete. 2023-11-27T15:18:16.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.011+0000 7fa72ec53b80 1 -- start start 2023-11-27T15:18:16.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.011+0000 7fa72ec53b80 1 Processor -- start 2023-11-27T15:18:16.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.011+0000 7fa72ec53b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5654f0a87b60 0x5654f0a94b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.011+0000 7fa72ec53b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f0a88470 0x5654f0a950b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.011+0000 7fa72ec53b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654f0a8c8b0 0x5654f0af4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.011+0000 7fa72ec53b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654f0af4c40 con 0x5654f0a88470 2023-11-27T15:18:16.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.011+0000 7fa72ec53b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654f0af4d80 con 0x5654f0a87b60 2023-11-27T15:18:16.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.011+0000 7fa72ec53b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654f0af4f10 con 0x5654f0a8c8b0 2023-11-27T15:18:16.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.011+0000 7fa72ec53b80 10 client.0 _ll_register_callbacks cb 0x7fa71000bbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:18:16.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.011+0000 7fa71bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f0a88470 0x5654f0a950b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.011+0000 7fa71bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f0a88470 0x5654f0a950b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52190/0 (socket says 172.21.15.52:52190) 2023-11-27T15:18:16.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.011+0000 7fa71bfff700 1 -- 172.21.15.52:0/3262476896 learned_addr learned my addr 172.21.15.52:0/3262476896 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:16.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.011+0000 7fa720863700 1 --2- 172.21.15.52:0/3262476896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5654f0a87b60 0x5654f0a94b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.012+0000 7fa721064700 1 --2- 172.21.15.52:0/3262476896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654f0a8c8b0 0x5654f0af4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.012+0000 7fa71bfff700 1 -- 172.21.15.52:0/3262476896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654f0a8c8b0 msgr2=0x5654f0af4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:16.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.012+0000 7fa71bfff700 1 --2- 172.21.15.52:0/3262476896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654f0a8c8b0 0x5654f0af4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.012+0000 7fa71bfff700 1 -- 172.21.15.52:0/3262476896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5654f0a87b60 msgr2=0x5654f0a94b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:16.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.012+0000 7fa71bfff700 1 --2- 172.21.15.52:0/3262476896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5654f0a87b60 0x5654f0a94b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.012+0000 7fa71bfff700 1 -- 172.21.15.52:0/3262476896 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5654f099f190 con 0x5654f0a88470 2023-11-27T15:18:16.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.012+0000 7fa720863700 1 --2- 172.21.15.52:0/3262476896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5654f0a87b60 0x5654f0a94b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:18:16.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.012+0000 7fa721064700 1 --2- 172.21.15.52:0/3262476896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654f0a8c8b0 0x5654f0af4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:18:16.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.012+0000 7fa71bfff700 1 --2- 172.21.15.52:0/3262476896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f0a88470 0x5654f0a950b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa708015000 tx=0x7fa708009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19144c0322c41ee4 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:16.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.012+0000 7fa6fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:18:16.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.012+0000 7fa6fbfff700 1 -- 172.21.15.52:0/3262476896 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70803a070 con 0x5654f0a88470 2023-11-27T15:18:16.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.012+0000 7fa6fbfff700 1 -- 172.21.15.52:0/3262476896 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fa708035a00 con 0x5654f0a88470 2023-11-27T15:18:16.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.012+0000 7fa72ec53b80 1 -- 172.21.15.52:0/3262476896 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa710015070 con 0x5654f0a88470 2023-11-27T15:18:16.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.012+0000 7fa6fbfff700 1 -- 172.21.15.52:0/3262476896 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa70801d070 con 0x5654f0a88470 2023-11-27T15:18:16.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.012+0000 7fa72ec53b80 10 client.15489 fetch_fsmap learned FSMap version 23 2023-11-27T15:18:16.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.012+0000 7fa72ec53b80 1 -- 172.21.15.52:0/3262476896 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5654f07efc60 con 0x5654f0a88470 2023-11-27T15:18:16.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.013+0000 7fa6fbfff700 1 -- 172.21.15.52:0/3262476896 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa70804b020 con 0x5654f0a88470 2023-11-27T15:18:16.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.013+0000 7fa6fbfff700 1 -- 172.21.15.52:0/3262476896 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa70800fbf0 con 0x5654f0a88470 2023-11-27T15:18:16.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.013+0000 7fa72ec53b80 10 client.15489 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:18:16.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.013+0000 7fa72ec53b80 10 client.15489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:18:16.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.013+0000 7fa72ec53b80 20 client.15489 populate_metadata read hostname 'smithi052' 2023-11-27T15:18:16.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.013+0000 7fa72ec53b80 1 --2- 172.21.15.52:0/3262476896 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5654f0af6680 0x5654f0af8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.013+0000 7fa72ec53b80 4 client.15489 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:18:16.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.013+0000 7fa72ec53b80 1 -- 172.21.15.52:0/3262476896 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5654f09a7660 con 0x5654f0af6680 2023-11-27T15:18:16.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.013+0000 7fa720863700 1 --2- 172.21.15.52:0/3262476896 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5654f0af6680 0x5654f0af8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.014+0000 7fa6fbfff700 10 client.15489 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:18:16.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.014+0000 7fa720863700 1 --2- 172.21.15.52:0/3262476896 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5654f0af6680 0x5654f0af8a90 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c4d3ead35684bee server_cookie=563ddade94e27ee0 in_seq=0 out_seq=0 2023-11-27T15:18:16.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.014+0000 7fa6fbfff700 1 -- 172.21.15.52:0/3262476896 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5654f09a7660 con 0x5654f0af6680 2023-11-27T15:18:16.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.014+0000 7fa6fbfff700 10 client.15489 handle_command_reply: tid=0 2023-11-27T15:18:16.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.014+0000 7fa72ec53b80 1 client.15489 shutdown 2023-11-27T15:18:16.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.014+0000 7fa72ec53b80 1 -- 172.21.15.52:0/3262476896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f0a88470 msgr2=0x5654f0a950b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:16.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.014+0000 7fa72ec53b80 1 --2- 172.21.15.52:0/3262476896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f0a88470 0x5654f0a950b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa708015000 tx=0x7fa708009ca0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.014+0000 7fa72ec53b80 1 -- 172.21.15.52:0/3262476896 shutdown_connections 2023-11-27T15:18:16.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.014+0000 7fa72ec53b80 1 --2- 172.21.15.52:0/3262476896 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5654f0af6680 0x5654f0af8a90 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.014+0000 7fa72ec53b80 1 --2- 172.21.15.52:0/3262476896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5654f0a87b60 0x5654f0a94b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.014+0000 7fa6fbfff700 0 client.15489 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:18:16.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.014+0000 7fa72ec53b80 1 --2- 172.21.15.52:0/3262476896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5654f0a8c8b0 0x5654f0af4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.014+0000 7fa72ec53b80 1 --2- 172.21.15.52:0/3262476896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654f0a88470 0x5654f0a950b0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.014+0000 7fa72ec53b80 1 -- 172.21.15.52:0/3262476896 >> 172.21.15.52:0/3262476896 conn(0x5654f0a7d360 msgr2=0x5654f0a86960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:16.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.014+0000 7fa72ec53b80 1 -- 172.21.15.52:0/3262476896 shutdown_connections 2023-11-27T15:18:16.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.014+0000 7fa72ec53b80 1 -- 172.21.15.52:0/3262476896 wait complete. 2023-11-27T15:18:16.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.014+0000 7fa72ec53b80 20 client.15489 trim_cache size 0 max 16384 2023-11-27T15:18:16.026 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:18:16.026 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2023-11-27T15:18:16.027 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2023-11-27T15:18:16.027 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:18:16.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.015+0000 7fa722ac7700 1 -- 172.21.15.52:0/2808429202 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fa70c0a13e0 msgr2=0x7fa70c0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:16.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.015+0000 7fa722ac7700 1 --2- 172.21.15.52:0/2808429202 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fa70c0a13e0 0x7fa70c0a3890 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa71c199c60 tx=0x7fa70800bbb0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.015+0000 7fa722ac7700 1 -- 172.21.15.52:0/2808429202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa71c10cca0 msgr2=0x7fa71c19cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:16.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.015+0000 7fa722ac7700 1 --2- 172.21.15.52:0/2808429202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa71c10cca0 0x7fa71c19cff0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa71400ef90 tx=0x7fa71400c3a0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.028 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.015+0000 7fa722ac7700 1 -- 172.21.15.52:0/2808429202 shutdown_connections 2023-11-27T15:18:16.028 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.015+0000 7fa722ac7700 1 --2- 172.21.15.52:0/2808429202 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fa70c0a13e0 0x7fa70c0a3890 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.028 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.015+0000 7fa722ac7700 1 --2- 172.21.15.52:0/2808429202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa71c10cca0 0x7fa71c19cff0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.028 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.015+0000 7fa722ac7700 1 --2- 172.21.15.52:0/2808429202 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa71c0ff0f0 0x7fa71c198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.028 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.015+0000 7fa722ac7700 1 --2- 172.21.15.52:0/2808429202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa71c0ffa00 0x7fa71c198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.028 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.015+0000 7fa722ac7700 1 -- 172.21.15.52:0/2808429202 >> 172.21.15.52:0/2808429202 conn(0x7fa71c0770b0 msgr2=0x7fa71c10dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:16.028 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.015+0000 7fa722ac7700 1 -- 172.21.15.52:0/2808429202 shutdown_connections 2023-11-27T15:18:16.028 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.015+0000 7fa722ac7700 1 -- 172.21.15.52:0/2808429202 wait complete. 2023-11-27T15:18:16.033 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-27T15:18:16.033 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:18:16.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.156+0000 7f01ccc89700 1 -- 172.21.15.52:0/710387780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01c80ff970 msgr2=0x7f01c810c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:16.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.156+0000 7f01ccc89700 1 --2- 172.21.15.52:0/710387780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01c80ff970 0x7f01c810c6d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f01bc009990 tx=0x7f01bc009ca0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.157+0000 7f01ccc89700 1 -- 172.21.15.52:0/710387780 shutdown_connections 2023-11-27T15:18:16.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.157+0000 7f01ccc89700 1 --2- 172.21.15.52:0/710387780 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01c810cc10 0x7f01c810eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.157+0000 7f01ccc89700 1 --2- 172.21.15.52:0/710387780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01c80ff060 0x7f01c80ff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.157+0000 7f01ccc89700 1 --2- 172.21.15.52:0/710387780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01c80ff970 0x7f01c810c6d0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.157+0000 7f01ccc89700 1 -- 172.21.15.52:0/710387780 >> 172.21.15.52:0/710387780 conn(0x7f01c8076310 msgr2=0x7f01c8076710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:16.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.157+0000 7f01ccc89700 1 -- 172.21.15.52:0/710387780 shutdown_connections 2023-11-27T15:18:16.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.157+0000 7f01ccc89700 1 -- 172.21.15.52:0/710387780 wait complete. 2023-11-27T15:18:16.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.157+0000 7f01ccc89700 1 Processor -- start 2023-11-27T15:18:16.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.158+0000 7f01ccc89700 1 -- start start 2023-11-27T15:18:16.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.158+0000 7f01ccc89700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01c80ff060 0x7f01c81985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.158+0000 7f01ccc89700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01c80ff970 0x7f01c8198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.158+0000 7f01ccc89700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01c810cc10 0x7f01c819cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.158+0000 7f01ccc89700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c819d5d0 con 0x7f01c80ff060 2023-11-27T15:18:16.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.158+0000 7f01ccc89700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c819d740 con 0x7f01c80ff970 2023-11-27T15:18:16.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.158+0000 7f01ccc89700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c819d900 con 0x7f01c810cc10 2023-11-27T15:18:16.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.158+0000 7f01c659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01c80ff060 0x7f01c81985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.158+0000 7f01c659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01c80ff060 0x7f01c81985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52202/0 (socket says 172.21.15.52:52202) 2023-11-27T15:18:16.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.158+0000 7f01c659c700 1 -- 172.21.15.52:0/3987195199 learned_addr learned my addr 172.21.15.52:0/3987195199 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:16.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.158+0000 7f01c6d9d700 1 --2- 172.21.15.52:0/3987195199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01c810cc10 0x7f01c819cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.158+0000 7f01c5d9b700 1 --2- 172.21.15.52:0/3987195199 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01c80ff970 0x7f01c8198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.158+0000 7f01c659c700 1 -- 172.21.15.52:0/3987195199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01c810cc10 msgr2=0x7f01c819cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:16.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.158+0000 7f01c659c700 1 --2- 172.21.15.52:0/3987195199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01c810cc10 0x7f01c819cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.158+0000 7f01c659c700 1 -- 172.21.15.52:0/3987195199 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01c80ff970 msgr2=0x7f01c8198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:16.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.158+0000 7f01c659c700 1 --2- 172.21.15.52:0/3987195199 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01c80ff970 0x7f01c8198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.158+0000 7f01c659c700 1 -- 172.21.15.52:0/3987195199 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01bc0095f0 con 0x7f01c80ff060 2023-11-27T15:18:16.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.158+0000 7f01c659c700 1 --2- 172.21.15.52:0/3987195199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01c80ff060 0x7f01c81985f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f01b800e9a0 tx=0x7f01b800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:16.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.159+0000 7f01b77fe700 1 -- 172.21.15.52:0/3987195199 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b8009790 con 0x7f01c80ff060 2023-11-27T15:18:16.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.159+0000 7f01b77fe700 1 -- 172.21.15.52:0/3987195199 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01b800cbd0 con 0x7f01c80ff060 2023-11-27T15:18:16.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.159+0000 7f01ccc89700 1 -- 172.21.15.52:0/3987195199 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01c8112530 con 0x7f01c80ff060 2023-11-27T15:18:16.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.159+0000 7f01b77fe700 1 -- 172.21.15.52:0/3987195199 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b8018680 con 0x7f01c80ff060 2023-11-27T15:18:16.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.159+0000 7f01ccc89700 1 -- 172.21.15.52:0/3987195199 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c81123a0 con 0x7f01c80ff060 2023-11-27T15:18:16.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.160+0000 7f01ccc89700 1 -- 172.21.15.52:0/3987195199 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01c81126c0 con 0x7f01c80ff060 2023-11-27T15:18:16.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.161+0000 7f01b77fe700 1 -- 172.21.15.52:0/3987195199 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f01b800cd40 con 0x7f01c80ff060 2023-11-27T15:18:16.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.162+0000 7f01b77fe700 1 --2- 172.21.15.52:0/3987195199 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f01b00a13e0 0x7f01b00a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.162+0000 7f01b77fe700 1 -- 172.21.15.52:0/3987195199 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f01b8014070 con 0x7f01c80ff060 2023-11-27T15:18:16.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.162+0000 7f01c5d9b700 1 --2- 172.21.15.52:0/3987195199 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f01b00a13e0 0x7f01b00a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.162+0000 7f01c5d9b700 1 --2- 172.21.15.52:0/3987195199 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f01b00a13e0 0x7f01b00a3890 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f01bc006010 tx=0x7f01bc00b260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:16.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.165+0000 7f01b77fe700 1 -- 172.21.15.52:0/3987195199 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f01b80966c0 con 0x7f01c80ff060 2023-11-27T15:18:16.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.413+0000 7f01ccc89700 1 -- 172.21.15.52:0/3987195199 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01c8067a60 con 0x7f01c80ff060 2023-11-27T15:18:16.419 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.414+0000 7f01b77fe700 1 -- 172.21.15.52:0/3987195199 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10804 (secure 0 0 0) 0x7f01b8099ce0 con 0x7f01c80ff060 2023-11-27T15:18:16.419 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:18:16.419 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2328881376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2328881376},{"type":"v1","addr":"172.21.15.52:6839","nonce":2328881376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2799551570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2799551570},{"type":"v1","addr":"172.21.15.52:6841","nonce":2799551570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:16:09.319630+0000","modified":"2023-11-27T15:16:14.742405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24494,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/215725838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":215725838},{"type":"v1","addr":"172.21.15.150:6839","nonce":215725838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3571830559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3571830559},{"type":"v1","addr":"172.21.15.102:6839","nonce":3571830559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:18:16.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.416+0000 7f01ccc89700 1 -- 172.21.15.52:0/3987195199 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f01b00a13e0 msgr2=0x7f01b00a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:16.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.416+0000 7f01ccc89700 1 --2- 172.21.15.52:0/3987195199 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f01b00a13e0 0x7f01b00a3890 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f01bc006010 tx=0x7f01bc00b260 comp rx=0 tx=0).stop 2023-11-27T15:18:16.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.416+0000 7f01ccc89700 1 -- 172.21.15.52:0/3987195199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01c80ff060 msgr2=0x7f01c81985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:16.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.416+0000 7f01ccc89700 1 --2- 172.21.15.52:0/3987195199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01c80ff060 0x7f01c81985f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f01b800e9a0 tx=0x7f01b800ecb0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.416+0000 7f01ccc89700 1 -- 172.21.15.52:0/3987195199 shutdown_connections 2023-11-27T15:18:16.422 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.416+0000 7f01ccc89700 1 --2- 172.21.15.52:0/3987195199 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f01b00a13e0 0x7f01b00a3890 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.422 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.416+0000 7f01ccc89700 1 --2- 172.21.15.52:0/3987195199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01c810cc10 0x7f01c819cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.422 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.416+0000 7f01ccc89700 1 --2- 172.21.15.52:0/3987195199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01c80ff060 0x7f01c81985f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.422 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.416+0000 7f01ccc89700 1 --2- 172.21.15.52:0/3987195199 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01c80ff970 0x7f01c8198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.422 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.416+0000 7f01ccc89700 1 -- 172.21.15.52:0/3987195199 >> 172.21.15.52:0/3987195199 conn(0x7f01c8076310 msgr2=0x7f01c80fcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:16.422 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.416+0000 7f01ccc89700 1 -- 172.21.15.52:0/3987195199 shutdown_connections 2023-11-27T15:18:16.422 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.416+0000 7f01ccc89700 1 -- 172.21.15.52:0/3987195199 wait complete. 2023-11-27T15:18:16.422 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 23 2023-11-27T15:18:16.436 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-27T15:18:16.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.628+0000 7f97bf7c7700 1 -- 172.21.15.52:0/2805244788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97b80ffdf0 msgr2=0x7f97b8103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:16.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.628+0000 7f97bf7c7700 1 --2- 172.21.15.52:0/2805244788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97b80ffdf0 0x7f97b8103db0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f97a80099c0 tx=0x7f97a8009cd0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.628+0000 7f97bf7c7700 1 -- 172.21.15.52:0/2805244788 shutdown_connections 2023-11-27T15:18:16.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.628+0000 7f97bf7c7700 1 --2- 172.21.15.52:0/2805244788 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97b8104490 0x7f97b810c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.628+0000 7f97bf7c7700 1 --2- 172.21.15.52:0/2805244788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97b80ff450 0x7f97b80ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.628+0000 7f97bf7c7700 1 --2- 172.21.15.52:0/2805244788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97b80ffdf0 0x7f97b8103db0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.628+0000 7f97bf7c7700 1 -- 172.21.15.52:0/2805244788 >> 172.21.15.52:0/2805244788 conn(0x7f97b80770b0 msgr2=0x7f97b80774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:16.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.628+0000 7f97bf7c7700 1 -- 172.21.15.52:0/2805244788 shutdown_connections 2023-11-27T15:18:16.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.628+0000 7f97bf7c7700 1 -- 172.21.15.52:0/2805244788 wait complete. 2023-11-27T15:18:16.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.629+0000 7f97bf7c7700 1 Processor -- start 2023-11-27T15:18:16.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.629+0000 7f97bf7c7700 1 -- start start 2023-11-27T15:18:16.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.629+0000 7f97bf7c7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97b80ff450 0x7f97b8194260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.629+0000 7f97bf7c7700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97b80ffdf0 0x7f97b81947a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.629+0000 7f97bf7c7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97b8104490 0x7f97b8198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.629+0000 7f97bf7c7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b8199240 con 0x7f97b8104490 2023-11-27T15:18:16.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.629+0000 7f97bf7c7700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b81993b0 con 0x7f97b80ffdf0 2023-11-27T15:18:16.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.629+0000 7f97bf7c7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b8199570 con 0x7f97b80ff450 2023-11-27T15:18:16.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.629+0000 7f97bdd64700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97b8104490 0x7f97b8198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.629+0000 7f97bdd64700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97b8104490 0x7f97b8198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52218/0 (socket says 172.21.15.52:52218) 2023-11-27T15:18:16.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.629+0000 7f97bdd64700 1 -- 172.21.15.52:0/1681463648 learned_addr learned my addr 172.21.15.52:0/1681463648 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:16.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.629+0000 7f97bd563700 1 --2- 172.21.15.52:0/1681463648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97b80ff450 0x7f97b8194260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.629+0000 7f97bcd62700 1 --2- 172.21.15.52:0/1681463648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97b80ffdf0 0x7f97b81947a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.630+0000 7f97bdd64700 1 -- 172.21.15.52:0/1681463648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97b80ff450 msgr2=0x7f97b8194260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:16.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.630+0000 7f97bdd64700 1 --2- 172.21.15.52:0/1681463648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97b80ff450 0x7f97b8194260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.630+0000 7f97bdd64700 1 -- 172.21.15.52:0/1681463648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97b80ffdf0 msgr2=0x7f97b81947a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:16.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.630+0000 7f97bdd64700 1 --2- 172.21.15.52:0/1681463648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97b80ffdf0 0x7f97b81947a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.630+0000 7f97bdd64700 1 -- 172.21.15.52:0/1681463648 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a80095f0 con 0x7f97b8104490 2023-11-27T15:18:16.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.630+0000 7f97bdd64700 1 --2- 172.21.15.52:0/1681463648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97b8104490 0x7f97b8198c00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f97b000ef30 tx=0x7f97b000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:16.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.630+0000 7f97ae7fc700 1 -- 172.21.15.52:0/1681463648 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b0018070 con 0x7f97b8104490 2023-11-27T15:18:16.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.630+0000 7f97bf7c7700 1 -- 172.21.15.52:0/1681463648 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b810e170 con 0x7f97b8104490 2023-11-27T15:18:16.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.630+0000 7f97ae7fc700 1 -- 172.21.15.52:0/1681463648 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97b000f040 con 0x7f97b8104490 2023-11-27T15:18:16.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.630+0000 7f97ae7fc700 1 -- 172.21.15.52:0/1681463648 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b000cd00 con 0x7f97b8104490 2023-11-27T15:18:16.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.630+0000 7f97bf7c7700 1 -- 172.21.15.52:0/1681463648 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b810dfe0 con 0x7f97b8104490 2023-11-27T15:18:16.638 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.633+0000 7f97ae7fc700 1 -- 172.21.15.52:0/1681463648 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f97b0003a00 con 0x7f97b8104490 2023-11-27T15:18:16.639 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.634+0000 7f97ae7fc700 1 --2- 172.21.15.52:0/1681463648 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f97a40a13e0 0x7f97a40a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.639 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.634+0000 7f97bd563700 1 --2- 172.21.15.52:0/1681463648 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f97a40a13e0 0x7f97a40a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.640 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.634+0000 7f97ae7fc700 1 -- 172.21.15.52:0/1681463648 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f97b00cca80 con 0x7f97b8104490 2023-11-27T15:18:16.640 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.635+0000 7f97bd563700 1 --2- 172.21.15.52:0/1681463648 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f97a40a13e0 0x7f97a40a3890 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f97b400b3c0 tx=0x7f97b400d040 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:16.641 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.636+0000 7f97cb953b80 1 Processor -- start 2023-11-27T15:18:16.641 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.636+0000 7f97cb953b80 1 -- start start 2023-11-27T15:18:16.642 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.636+0000 7f97cb953b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626c3a3fb60 0x5626c3a3ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.642 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.636+0000 7f97cb953b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626c3a40470 0x5626c3a44370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.642 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.636+0000 7f97cb953b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626c3a448b0 0x5626c3a44d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.642 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.636+0000 7f97cb953b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c3945060 con 0x5626c3a40470 2023-11-27T15:18:16.642 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.636+0000 7f97cb953b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c39978a0 con 0x5626c3a448b0 2023-11-27T15:18:16.642 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.636+0000 7f97cb953b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c3990d30 con 0x5626c3a3fb60 2023-11-27T15:18:16.642 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.636+0000 7f97bd563700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626c3a3fb60 0x5626c3a3ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.642 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.636+0000 7f97bd563700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626c3a3fb60 0x5626c3a3ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:38868/0 (socket says 172.21.15.52:38868) 2023-11-27T15:18:16.643 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.637+0000 7f97bd563700 1 -- 172.21.15.52:0/4108861638 learned_addr learned my addr 172.21.15.52:0/4108861638 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:16.643 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.637+0000 7f97bdd64700 1 --2- 172.21.15.52:0/4108861638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626c3a448b0 0x5626c3a44d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.643 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.637+0000 7f97bcd62700 1 --2- 172.21.15.52:0/4108861638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626c3a40470 0x5626c3a44370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.643 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.637+0000 7f97bd563700 1 -- 172.21.15.52:0/4108861638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626c3a448b0 msgr2=0x5626c3a44d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:16.643 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.637+0000 7f97bd563700 1 --2- 172.21.15.52:0/4108861638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626c3a448b0 0x5626c3a44d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.643 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.637+0000 7f97bd563700 1 -- 172.21.15.52:0/4108861638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626c3a40470 msgr2=0x5626c3a44370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:16.643 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.637+0000 7f97bd563700 1 --2- 172.21.15.52:0/4108861638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626c3a40470 0x5626c3a44370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.643 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.637+0000 7f97bd563700 1 -- 172.21.15.52:0/4108861638 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626c3812630 con 0x5626c3a3fb60 2023-11-27T15:18:16.643 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.637+0000 7f97bdd64700 1 --2- 172.21.15.52:0/4108861638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626c3a448b0 0x5626c3a44d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:18:16.643 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.637+0000 7f97bd563700 1 --2- 172.21.15.52:0/4108861638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626c3a3fb60 0x5626c3a3ff30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f97b4003b00 tx=0x7f97b4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7e0a3405a7ca012 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:16.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.637+0000 7f97a37fe700 1 -- 172.21.15.52:0/4108861638 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b4010040 con 0x5626c3a3fb60 2023-11-27T15:18:16.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.637+0000 7f97a37fe700 1 -- 172.21.15.52:0/4108861638 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97b4018dd0 con 0x5626c3a3fb60 2023-11-27T15:18:16.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.637+0000 7f97a37fe700 1 -- 172.21.15.52:0/4108861638 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b4005020 con 0x5626c3a3fb60 2023-11-27T15:18:16.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.638+0000 7f97cb953b80 1 -- 172.21.15.52:0/4108861638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626c3a3fb60 msgr2=0x5626c3a3ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:16.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.638+0000 7f97cb953b80 1 --2- 172.21.15.52:0/4108861638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626c3a3fb60 0x5626c3a3ff30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f97b4003b00 tx=0x7f97b4014000 comp rx=0 tx=0).stop 2023-11-27T15:18:16.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.638+0000 7f97cb953b80 1 -- 172.21.15.52:0/4108861638 shutdown_connections 2023-11-27T15:18:16.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.638+0000 7f97cb953b80 1 --2- 172.21.15.52:0/4108861638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626c3a448b0 0x5626c3a44d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.638+0000 7f97cb953b80 1 --2- 172.21.15.52:0/4108861638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626c3a3fb60 0x5626c3a3ff30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.638+0000 7f97cb953b80 1 --2- 172.21.15.52:0/4108861638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626c3a40470 0x5626c3a44370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.638+0000 7f97cb953b80 1 -- 172.21.15.52:0/4108861638 >> 172.21.15.52:0/4108861638 conn(0x5626c3a35360 msgr2=0x5626c3a3ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:16.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.638+0000 7f97cb953b80 1 -- 172.21.15.52:0/4108861638 shutdown_connections 2023-11-27T15:18:16.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.639+0000 7f97cb953b80 1 -- 172.21.15.52:0/4108861638 wait complete. 2023-11-27T15:18:16.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.639+0000 7f97cb953b80 1 -- start start 2023-11-27T15:18:16.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.639+0000 7f97cb953b80 1 Processor -- start 2023-11-27T15:18:16.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.640+0000 7f97cb953b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626c3a3fb60 0x5626c3a4c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.640+0000 7f97cb953b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626c3a40470 0x5626c3a49350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.640+0000 7f97cb953b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626c3a448b0 0x5626c3a499d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.640+0000 7f97cb953b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c39328e0 con 0x5626c3a448b0 2023-11-27T15:18:16.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.640+0000 7f97cb953b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c3932580 con 0x5626c3a40470 2023-11-27T15:18:16.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.640+0000 7f97cb953b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c3814920 con 0x5626c3a3fb60 2023-11-27T15:18:16.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.640+0000 7f97cb953b80 10 client.0 _ll_register_callbacks cb 0x5626c3950780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:18:16.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.640+0000 7f97bdd64700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626c3a448b0 0x5626c3a499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.640+0000 7f97bdd64700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626c3a448b0 0x5626c3a499d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52232/0 (socket says 172.21.15.52:52232) 2023-11-27T15:18:16.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.640+0000 7f97bdd64700 1 -- 172.21.15.52:0/3369371820 learned_addr learned my addr 172.21.15.52:0/3369371820 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:16.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.640+0000 7f97bd563700 1 --2- 172.21.15.52:0/3369371820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626c3a3fb60 0x5626c3a4c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.640+0000 7f97bcd62700 1 --2- 172.21.15.52:0/3369371820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626c3a40470 0x5626c3a49350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.640+0000 7f97bd563700 1 -- 172.21.15.52:0/3369371820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626c3a40470 msgr2=0x5626c3a49350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:16.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.640+0000 7f97bd563700 1 --2- 172.21.15.52:0/3369371820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626c3a40470 0x5626c3a49350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.640+0000 7f97bd563700 1 -- 172.21.15.52:0/3369371820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626c3a448b0 msgr2=0x5626c3a499d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:16.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.640+0000 7f97bd563700 1 --2- 172.21.15.52:0/3369371820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626c3a448b0 0x5626c3a499d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.640+0000 7f97bd563700 1 -- 172.21.15.52:0/3369371820 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b4003870 con 0x5626c3a3fb60 2023-11-27T15:18:16.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.640+0000 7f97bcd62700 1 --2- 172.21.15.52:0/3369371820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626c3a40470 0x5626c3a49350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:18:16.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.640+0000 7f97bdd64700 1 --2- 172.21.15.52:0/3369371820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626c3a448b0 0x5626c3a499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:18:16.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.641+0000 7f97bd563700 1 --2- 172.21.15.52:0/3369371820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626c3a3fb60 0x5626c3a4c0d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f97b4004b40 tx=0x7f97b4004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f6e70827981abe4 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:16.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.641+0000 7f97a0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2023-11-27T15:18:16.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.641+0000 7f97a0ff9700 1 -- 172.21.15.52:0/3369371820 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b4010040 con 0x5626c3a3fb60 2023-11-27T15:18:16.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.641+0000 7f97a0ff9700 1 -- 172.21.15.52:0/3369371820 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f97b4018c10 con 0x5626c3a3fb60 2023-11-27T15:18:16.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.641+0000 7f97cb953b80 1 -- 172.21.15.52:0/3369371820 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626c3a32b20 con 0x5626c3a3fb60 2023-11-27T15:18:16.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.641+0000 7f97a0ff9700 1 -- 172.21.15.52:0/3369371820 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97b40054f0 con 0x5626c3a3fb60 2023-11-27T15:18:16.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.641+0000 7f97cb953b80 10 client.25225 fetch_fsmap learned FSMap version 23 2023-11-27T15:18:16.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.641+0000 7f97cb953b80 1 -- 172.21.15.52:0/3369371820 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5626c3957190 con 0x5626c3a3fb60 2023-11-27T15:18:16.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.641+0000 7f97a0ff9700 1 -- 172.21.15.52:0/3369371820 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5626c3a32b20 con 0x5626c3a3fb60 2023-11-27T15:18:16.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.641+0000 7f97a0ff9700 1 -- 172.21.15.52:0/3369371820 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f97b402c820 con 0x5626c3a3fb60 2023-11-27T15:18:16.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.642+0000 7f97cb953b80 10 client.25225 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:18:16.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.642+0000 7f97cb953b80 10 client.25225 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:18:16.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.642+0000 7f97cb953b80 20 client.25225 populate_metadata read hostname 'smithi052' 2023-11-27T15:18:16.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.642+0000 7f97cb953b80 1 --2- 172.21.15.52:0/3369371820 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5626c3a59850 0x5626c3a5bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.642+0000 7f97cb953b80 4 client.25225 mds_command: new command op to 24472 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:18:16.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.642+0000 7f97cb953b80 1 -- 172.21.15.52:0/3369371820 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626c39530a0 con 0x5626c3a59850 2023-11-27T15:18:16.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.642+0000 7f97bcd62700 1 --2- 172.21.15.52:0/3369371820 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5626c3a59850 0x5626c3a5bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.642+0000 7f97a0ff9700 10 client.25225 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:18:16.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.642+0000 7f97bcd62700 1 --2- 172.21.15.52:0/3369371820 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5626c3a59850 0x5626c3a5bc30 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5a2dfe2e9a5ee9b0 server_cookie=ed13f188bc607046 in_seq=0 out_seq=0 2023-11-27T15:18:16.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.643+0000 7f97a0ff9700 1 -- 172.21.15.52:0/3369371820 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5626c39530a0 con 0x5626c3a59850 2023-11-27T15:18:16.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.643+0000 7f97a0ff9700 10 client.25225 handle_command_reply: tid=0 2023-11-27T15:18:16.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.643+0000 7f97cb953b80 1 client.25225 shutdown 2023-11-27T15:18:16.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.643+0000 7f97cb953b80 1 -- 172.21.15.52:0/3369371820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626c3a3fb60 msgr2=0x5626c3a4c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:16.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.643+0000 7f97cb953b80 1 --2- 172.21.15.52:0/3369371820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626c3a3fb60 0x5626c3a4c0d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f97b4004b40 tx=0x7f97b4004040 comp rx=0 tx=0).stop 2023-11-27T15:18:16.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.643+0000 7f97cb953b80 1 -- 172.21.15.52:0/3369371820 shutdown_connections 2023-11-27T15:18:16.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.643+0000 7f97cb953b80 1 --2- 172.21.15.52:0/3369371820 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5626c3a59850 0x5626c3a5bc30 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.643+0000 7f97cb953b80 1 --2- 172.21.15.52:0/3369371820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626c3a448b0 0x5626c3a499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.643+0000 7f97cb953b80 1 --2- 172.21.15.52:0/3369371820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626c3a3fb60 0x5626c3a4c0d0 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f97b4004b40 tx=0x7f97b4004040 comp rx=0 tx=0).stop 2023-11-27T15:18:16.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.643+0000 7f97cb953b80 1 --2- 172.21.15.52:0/3369371820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626c3a40470 0x5626c3a49350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.643+0000 7f97cb953b80 1 -- 172.21.15.52:0/3369371820 >> 172.21.15.52:0/3369371820 conn(0x5626c3a35360 msgr2=0x5626c3a39de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:16.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.644+0000 7f97a0ff9700 0 client.25225 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:18:16.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.644+0000 7f97cb953b80 1 -- 172.21.15.52:0/3369371820 shutdown_connections 2023-11-27T15:18:16.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.644+0000 7f97cb953b80 1 -- 172.21.15.52:0/3369371820 wait complete. 2023-11-27T15:18:16.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.644+0000 7f97cb953b80 20 client.25225 trim_cache size 0 max 16384 2023-11-27T15:18:16.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.669+0000 7f97cb953b80 1 Processor -- start 2023-11-27T15:18:16.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.669+0000 7f97cb953b80 1 -- start start 2023-11-27T15:18:16.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.669+0000 7f97cb953b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626c3a40470 0x5626c3a431a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.669+0000 7f97cb953b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626c3a448b0 0x5626c3a4bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.669+0000 7f97cb953b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626c3a4c1b0 0x5626c3a43a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.669+0000 7f97cb953b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c3a59670 con 0x5626c3a40470 2023-11-27T15:18:16.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.669+0000 7f97cb953b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c3a40bc0 con 0x5626c3a448b0 2023-11-27T15:18:16.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.669+0000 7f97cb953b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c3a4a930 con 0x5626c3a4c1b0 2023-11-27T15:18:16.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.670+0000 7f97bd563700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626c3a40470 0x5626c3a431a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.670+0000 7f97bd563700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626c3a40470 0x5626c3a431a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52238/0 (socket says 172.21.15.52:52238) 2023-11-27T15:18:16.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.670+0000 7f97bd563700 1 -- 172.21.15.52:0/3211289254 learned_addr learned my addr 172.21.15.52:0/3211289254 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:16.677 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.670+0000 7f97bcd62700 1 --2- 172.21.15.52:0/3211289254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626c3a448b0 0x5626c3a4bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.677 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.670+0000 7f97bdd64700 1 --2- 172.21.15.52:0/3211289254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626c3a4c1b0 0x5626c3a43a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.677 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.670+0000 7f97bd563700 1 -- 172.21.15.52:0/3211289254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626c3a4c1b0 msgr2=0x5626c3a43a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:16.677 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.670+0000 7f97bd563700 1 --2- 172.21.15.52:0/3211289254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626c3a4c1b0 0x5626c3a43a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.677 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.670+0000 7f97bd563700 1 -- 172.21.15.52:0/3211289254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626c3a448b0 msgr2=0x5626c3a4bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:16.677 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.670+0000 7f97bd563700 1 --2- 172.21.15.52:0/3211289254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626c3a448b0 0x5626c3a4bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.678 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.670+0000 7f97bd563700 1 -- 172.21.15.52:0/3211289254 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626c3956ee0 con 0x5626c3a40470 2023-11-27T15:18:16.678 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.670+0000 7f97bd563700 1 --2- 172.21.15.52:0/3211289254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626c3a40470 0x5626c3a431a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f97b400bf50 tx=0x7f97b4005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec535508f4cad4a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:16.678 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.671+0000 7f97a3fff700 1 -- 172.21.15.52:0/3211289254 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b4010040 con 0x5626c3a40470 2023-11-27T15:18:16.678 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.671+0000 7f97a3fff700 1 -- 172.21.15.52:0/3211289254 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97b4018c10 con 0x5626c3a40470 2023-11-27T15:18:16.678 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.671+0000 7f97a3fff700 1 -- 172.21.15.52:0/3211289254 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b402cee0 con 0x5626c3a40470 2023-11-27T15:18:16.678 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.672+0000 7f97cb953b80 1 -- 172.21.15.52:0/3211289254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626c3a40470 msgr2=0x5626c3a431a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:16.679 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.672+0000 7f97cb953b80 1 --2- 172.21.15.52:0/3211289254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626c3a40470 0x5626c3a431a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f97b400bf50 tx=0x7f97b4005b40 comp rx=0 tx=0).stop 2023-11-27T15:18:16.679 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.672+0000 7f97cb953b80 1 -- 172.21.15.52:0/3211289254 shutdown_connections 2023-11-27T15:18:16.679 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.672+0000 7f97cb953b80 1 --2- 172.21.15.52:0/3211289254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626c3a4c1b0 0x5626c3a43a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.679 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.672+0000 7f97cb953b80 1 --2- 172.21.15.52:0/3211289254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626c3a40470 0x5626c3a431a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.679 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.672+0000 7f97cb953b80 1 --2- 172.21.15.52:0/3211289254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626c3a448b0 0x5626c3a4bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.679 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.672+0000 7f97cb953b80 1 -- 172.21.15.52:0/3211289254 >> 172.21.15.52:0/3211289254 conn(0x5626c3a35360 msgr2=0x5626c3a55b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:16.679 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.672+0000 7f97cb953b80 1 -- 172.21.15.52:0/3211289254 shutdown_connections 2023-11-27T15:18:16.679 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.672+0000 7f97cb953b80 1 -- 172.21.15.52:0/3211289254 wait complete. 2023-11-27T15:18:16.683 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.672+0000 7f97cb953b80 1 -- start start 2023-11-27T15:18:16.683 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.672+0000 7f97cb953b80 1 Processor -- start 2023-11-27T15:18:16.683 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.673+0000 7f97cb953b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626c3a40470 0x5626c3a50960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.683 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.673+0000 7f97cb953b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626c3a448b0 0x5626c3a4c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.673+0000 7f97cb953b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626c3a4c1b0 0x5626c3a4ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.673+0000 7f97cb953b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c3a4d3a0 con 0x5626c3a40470 2023-11-27T15:18:16.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.673+0000 7f97cb953b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c3a4d510 con 0x5626c3a448b0 2023-11-27T15:18:16.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.673+0000 7f97cb953b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626c3a4d6d0 con 0x5626c3a4c1b0 2023-11-27T15:18:16.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.673+0000 7f97cb953b80 10 client.0 _ll_register_callbacks cb 0x7f97b0009370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:18:16.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.673+0000 7f97bd563700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626c3a40470 0x5626c3a50960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.673+0000 7f97bd563700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626c3a40470 0x5626c3a50960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52244/0 (socket says 172.21.15.52:52244) 2023-11-27T15:18:16.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.673+0000 7f97bd563700 1 -- 172.21.15.52:0/2450022519 learned_addr learned my addr 172.21.15.52:0/2450022519 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:16.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.673+0000 7f97bdd64700 1 --2- 172.21.15.52:0/2450022519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626c3a4c1b0 0x5626c3a4ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.673+0000 7f97bcd62700 1 --2- 172.21.15.52:0/2450022519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626c3a448b0 0x5626c3a4c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.673+0000 7f97bdd64700 1 -- 172.21.15.52:0/2450022519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626c3a448b0 msgr2=0x5626c3a4c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:16.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.673+0000 7f97bdd64700 1 --2- 172.21.15.52:0/2450022519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626c3a448b0 0x5626c3a4c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.673+0000 7f97bdd64700 1 -- 172.21.15.52:0/2450022519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626c3a40470 msgr2=0x5626c3a50960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:16.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.673+0000 7f97bdd64700 1 --2- 172.21.15.52:0/2450022519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626c3a40470 0x5626c3a50960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.673+0000 7f97bdd64700 1 -- 172.21.15.52:0/2450022519 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626c3957190 con 0x5626c3a4c1b0 2023-11-27T15:18:16.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.673+0000 7f97bd563700 1 --2- 172.21.15.52:0/2450022519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626c3a40470 0x5626c3a50960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:18:16.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.674+0000 7f97bdd64700 1 --2- 172.21.15.52:0/2450022519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626c3a4c1b0 0x5626c3a4ce60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f97b0096400 tx=0x7f97b00955b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d886d179943935b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:16.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.674+0000 7f97a0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2023-11-27T15:18:16.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.674+0000 7f97a0ff9700 1 -- 172.21.15.52:0/2450022519 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b000ce60 con 0x5626c3a4c1b0 2023-11-27T15:18:16.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.674+0000 7f97a0ff9700 1 -- 172.21.15.52:0/2450022519 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f97b0084040 con 0x5626c3a4c1b0 2023-11-27T15:18:16.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.674+0000 7f97cb953b80 1 -- 172.21.15.52:0/2450022519 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f97b4018dd0 con 0x5626c3a4c1b0 2023-11-27T15:18:16.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.674+0000 7f97a0ff9700 1 -- 172.21.15.52:0/2450022519 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97b00c5030 con 0x5626c3a4c1b0 2023-11-27T15:18:16.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.674+0000 7f97cb953b80 10 client.25231 fetch_fsmap learned FSMap version 23 2023-11-27T15:18:16.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.674+0000 7f97cb953b80 1 -- 172.21.15.52:0/2450022519 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5626c37a7c60 con 0x5626c3a4c1b0 2023-11-27T15:18:16.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.674+0000 7f97a0ff9700 1 -- 172.21.15.52:0/2450022519 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97b4018dd0 con 0x5626c3a4c1b0 2023-11-27T15:18:16.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.675+0000 7f97a0ff9700 1 -- 172.21.15.52:0/2450022519 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f97b00ca330 con 0x5626c3a4c1b0 2023-11-27T15:18:16.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.675+0000 7f97cb953b80 10 client.25231 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:18:16.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.675+0000 7f97cb953b80 10 client.25231 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:18:16.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.675+0000 7f97cb953b80 20 client.25231 populate_metadata read hostname 'smithi052' 2023-11-27T15:18:16.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.675+0000 7f97cb953b80 1 --2- 172.21.15.52:0/2450022519 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5626c3ab6250 0x5626c3ab8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.675+0000 7f97cb953b80 4 client.25231 mds_command: new command op to 24472 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:18:16.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.675+0000 7f97cb953b80 1 -- 172.21.15.52:0/2450022519 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5626c395f660 con 0x5626c3ab6250 2023-11-27T15:18:16.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.675+0000 7f97bd563700 1 --2- 172.21.15.52:0/2450022519 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5626c3ab6250 0x5626c3ab8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.675+0000 7f97a0ff9700 10 client.25231 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:18:16.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.675+0000 7f97bd563700 1 --2- 172.21.15.52:0/2450022519 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5626c3ab6250 0x5626c3ab8660 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f3f67621a3bbc17 server_cookie=bbd8512c7e140a3c in_seq=0 out_seq=0 2023-11-27T15:18:16.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.675+0000 7f97a0ff9700 1 -- 172.21.15.52:0/2450022519 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5626c395f660 con 0x5626c3ab6250 2023-11-27T15:18:16.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.676+0000 7f97a0ff9700 10 client.25231 handle_command_reply: tid=0 2023-11-27T15:18:16.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.676+0000 7f97cb953b80 1 client.25231 shutdown 2023-11-27T15:18:16.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.676+0000 7f97cb953b80 1 -- 172.21.15.52:0/2450022519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626c3a4c1b0 msgr2=0x5626c3a4ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:16.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.676+0000 7f97cb953b80 1 --2- 172.21.15.52:0/2450022519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626c3a4c1b0 0x5626c3a4ce60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f97b0096400 tx=0x7f97b00955b0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.676+0000 7f97cb953b80 1 -- 172.21.15.52:0/2450022519 shutdown_connections 2023-11-27T15:18:16.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.676+0000 7f97cb953b80 1 --2- 172.21.15.52:0/2450022519 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5626c3ab6250 0x5626c3ab8660 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.676+0000 7f97cb953b80 1 --2- 172.21.15.52:0/2450022519 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626c3a4c1b0 0x5626c3a4ce60 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.676+0000 7f97cb953b80 1 --2- 172.21.15.52:0/2450022519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626c3a40470 0x5626c3a50960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.676+0000 7f97cb953b80 1 --2- 172.21.15.52:0/2450022519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626c3a448b0 0x5626c3a4c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.676+0000 7f97cb953b80 1 -- 172.21.15.52:0/2450022519 >> 172.21.15.52:0/2450022519 conn(0x5626c3a35360 msgr2=0x5626c3a43a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:16.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.676+0000 7f97a0ff9700 0 client.25231 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:18:16.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.676+0000 7f97cb953b80 1 -- 172.21.15.52:0/2450022519 shutdown_connections 2023-11-27T15:18:16.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.676+0000 7f97cb953b80 1 -- 172.21.15.52:0/2450022519 wait complete. 2023-11-27T15:18:16.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.676+0000 7f97cb953b80 20 client.25231 trim_cache size 0 max 16384 2023-11-27T15:18:16.688 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:18:16.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.677+0000 7f97bf7c7700 1 -- 172.21.15.52:0/1681463648 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f97a40a13e0 msgr2=0x7f97a40a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:16.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.677+0000 7f97bf7c7700 1 --2- 172.21.15.52:0/1681463648 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f97a40a13e0 0x7f97a40a3890 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f97b400b3c0 tx=0x7f97b400d040 comp rx=0 tx=0).stop 2023-11-27T15:18:16.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.677+0000 7f97bf7c7700 1 -- 172.21.15.52:0/1681463648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97b8104490 msgr2=0x7f97b8198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:16.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.677+0000 7f97bf7c7700 1 --2- 172.21.15.52:0/1681463648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97b8104490 0x7f97b8198c00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f97b000ef30 tx=0x7f97b000c3a0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.677+0000 7f97bf7c7700 1 -- 172.21.15.52:0/1681463648 shutdown_connections 2023-11-27T15:18:16.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.677+0000 7f97bf7c7700 1 --2- 172.21.15.52:0/1681463648 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f97a40a13e0 0x7f97a40a3890 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.677+0000 7f97bf7c7700 1 --2- 172.21.15.52:0/1681463648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97b8104490 0x7f97b8198c00 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.677+0000 7f97bf7c7700 1 --2- 172.21.15.52:0/1681463648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97b80ff450 0x7f97b8194260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.677+0000 7f97bf7c7700 1 --2- 172.21.15.52:0/1681463648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97b80ffdf0 0x7f97b81947a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.677+0000 7f97bf7c7700 1 -- 172.21.15.52:0/1681463648 >> 172.21.15.52:0/1681463648 conn(0x7f97b80770b0 msgr2=0x7f97b80fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:16.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.677+0000 7f97bf7c7700 1 -- 172.21.15.52:0/1681463648 shutdown_connections 2023-11-27T15:18:16.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.677+0000 7f97bf7c7700 1 -- 172.21.15.52:0/1681463648 wait complete. 2023-11-27T15:18:16.698 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-27T15:18:16.835 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.830+0000 7f11c4fb7700 1 -- 172.21.15.52:0/4056006768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11c0100340 msgr2=0x7f11c010c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:16.836 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.830+0000 7f11c4fb7700 1 --2- 172.21.15.52:0/4056006768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11c0100340 0x7f11c010c010 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f11b800b0d0 tx=0x7f11b800b3e0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.836 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.830+0000 7f11c4fb7700 1 -- 172.21.15.52:0/4056006768 shutdown_connections 2023-11-27T15:18:16.836 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.830+0000 7f11c4fb7700 1 --2- 172.21.15.52:0/4056006768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11c0104180 0x7f11c0104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.836 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.830+0000 7f11c4fb7700 1 --2- 172.21.15.52:0/4056006768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11c0100340 0x7f11c010c010 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.836 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.830+0000 7f11c4fb7700 1 --2- 172.21.15.52:0/4056006768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11c00ff990 0x7f11c00ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.836 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.830+0000 7f11c4fb7700 1 -- 172.21.15.52:0/4056006768 >> 172.21.15.52:0/4056006768 conn(0x7f11c0075920 msgr2=0x7f11c0075d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:16.836 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.830+0000 7f11c4fb7700 1 -- 172.21.15.52:0/4056006768 shutdown_connections 2023-11-27T15:18:16.836 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.830+0000 7f11c4fb7700 1 -- 172.21.15.52:0/4056006768 wait complete. 2023-11-27T15:18:16.838 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.831+0000 7f11c4fb7700 1 Processor -- start 2023-11-27T15:18:16.838 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.831+0000 7f11c4fb7700 1 -- start start 2023-11-27T15:18:16.839 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.831+0000 7f11c4fb7700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11c00ff990 0x7f11c01965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.839 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.831+0000 7f11c4fb7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11c0100340 0x7f11c0196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.839 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.832+0000 7f11c4fb7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11c0104180 0x7f11c019af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.839 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.832+0000 7f11c4fb7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c019b580 con 0x7f11c0104180 2023-11-27T15:18:16.839 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.832+0000 7f11c4fb7700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c019b6f0 con 0x7f11c00ff990 2023-11-27T15:18:16.839 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.832+0000 7f11c4fb7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c019b8b0 con 0x7f11c0100340 2023-11-27T15:18:16.839 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.832+0000 7f11bed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11c0104180 0x7f11c019af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.839 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.832+0000 7f11bed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11c0104180 0x7f11c019af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52258/0 (socket says 172.21.15.52:52258) 2023-11-27T15:18:16.839 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.832+0000 7f11bed9d700 1 -- 172.21.15.52:0/3061872635 learned_addr learned my addr 172.21.15.52:0/3061872635 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:16.839 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.832+0000 7f11be59c700 1 --2- 172.21.15.52:0/3061872635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11c00ff990 0x7f11c01965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.839 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.832+0000 7f11be59c700 1 -- 172.21.15.52:0/3061872635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11c0100340 msgr2=0x7f11c0196ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:16.839 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.832+0000 7f11be59c700 1 --2- 172.21.15.52:0/3061872635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11c0100340 0x7f11c0196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.839 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.832+0000 7f11be59c700 1 -- 172.21.15.52:0/3061872635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11c0104180 msgr2=0x7f11c019af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:16.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.832+0000 7f11be59c700 1 --2- 172.21.15.52:0/3061872635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11c0104180 0x7f11c019af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.832+0000 7f11be59c700 1 -- 172.21.15.52:0/3061872635 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b8009d00 con 0x7f11c00ff990 2023-11-27T15:18:16.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.832+0000 7f11bed9d700 1 --2- 172.21.15.52:0/3061872635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11c0104180 0x7f11c019af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:18:16.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.832+0000 7f11be59c700 1 --2- 172.21.15.52:0/3061872635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11c00ff990 0x7f11c01965a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f11b0009c50 tx=0x7f11b0009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:16.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.833+0000 7f11af7fe700 1 -- 172.21.15.52:0/3061872635 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b0015070 con 0x7f11c00ff990 2023-11-27T15:18:16.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.833+0000 7f11c4fb7700 1 -- 172.21.15.52:0/3061872635 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c010f260 con 0x7f11c00ff990 2023-11-27T15:18:16.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.833+0000 7f11af7fe700 1 -- 172.21.15.52:0/3061872635 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11b0019410 con 0x7f11c00ff990 2023-11-27T15:18:16.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.833+0000 7f11af7fe700 1 -- 172.21.15.52:0/3061872635 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b0018610 con 0x7f11c00ff990 2023-11-27T15:18:16.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.833+0000 7f11c4fb7700 1 -- 172.21.15.52:0/3061872635 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c010f0d0 con 0x7f11c00ff990 2023-11-27T15:18:16.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.836+0000 7f11d1143b80 1 Processor -- start 2023-11-27T15:18:16.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.836+0000 7f11d1143b80 1 -- start start 2023-11-27T15:18:16.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.836+0000 7f11af7fe700 1 -- 172.21.15.52:0/3061872635 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f11b0019880 con 0x7f11c00ff990 2023-11-27T15:18:16.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.837+0000 7f11d1143b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651b9fbeb60 0x5651b9fbef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.837+0000 7f11d1143b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651b9fbf470 0x5651b9fc3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.837+0000 7f11d1143b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651b9fc38b0 0x5651b9fc3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.837+0000 7f11d1143b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651b9ec4060 con 0x5651b9fbf470 2023-11-27T15:18:16.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.837+0000 7f11d1143b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651b9f168a0 con 0x5651b9fc38b0 2023-11-27T15:18:16.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.837+0000 7f11d1143b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651b9f0fd30 con 0x5651b9fbeb60 2023-11-27T15:18:16.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.837+0000 7f11bed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651b9fbeb60 0x5651b9fbef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.837+0000 7f11bed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651b9fbeb60 0x5651b9fbef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:38906/0 (socket says 172.21.15.52:38906) 2023-11-27T15:18:16.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.837+0000 7f11bed9d700 1 -- 172.21.15.52:0/4048156570 learned_addr learned my addr 172.21.15.52:0/4048156570 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:16.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.837+0000 7f11be59c700 1 --2- 172.21.15.52:0/4048156570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651b9fbf470 0x5651b9fc3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.837+0000 7f11bdd9b700 1 --2- 172.21.15.52:0/4048156570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651b9fc38b0 0x5651b9fc3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.837+0000 7f11bed9d700 1 -- 172.21.15.52:0/4048156570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651b9fc38b0 msgr2=0x5651b9fc3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:16.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.837+0000 7f11bed9d700 1 --2- 172.21.15.52:0/4048156570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651b9fc38b0 0x5651b9fc3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.837+0000 7f11bed9d700 1 -- 172.21.15.52:0/4048156570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651b9fbf470 msgr2=0x5651b9fc3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:16.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.837+0000 7f11bed9d700 1 --2- 172.21.15.52:0/4048156570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651b9fbf470 0x5651b9fc3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.837+0000 7f11bed9d700 1 -- 172.21.15.52:0/4048156570 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651b9d91630 con 0x5651b9fbeb60 2023-11-27T15:18:16.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.837+0000 7f11af7fe700 1 --2- 172.21.15.52:0/3061872635 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f11a80a1460 0x7f11a80a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.837+0000 7f11be59c700 1 --2- 172.21.15.52:0/4048156570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651b9fbf470 0x5651b9fc3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:18:16.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.837+0000 7f11be59c700 1 --2- 172.21.15.52:0/3061872635 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f11a80a1460 0x7f11a80a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.838+0000 7f11bed9d700 1 --2- 172.21.15.52:0/4048156570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651b9fbeb60 0x5651b9fbef30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f11b8005b40 tx=0x7f11b8005b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff1aff960e9bbcd1 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:16.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.838+0000 7f11af7fe700 1 -- 172.21.15.52:0/3061872635 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f11b00d0730 con 0x7f11c00ff990 2023-11-27T15:18:16.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.838+0000 7f11acff9700 1 -- 172.21.15.52:0/4048156570 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b800e040 con 0x5651b9fbeb60 2023-11-27T15:18:16.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.838+0000 7f11acff9700 1 -- 172.21.15.52:0/4048156570 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11b800bca0 con 0x5651b9fbeb60 2023-11-27T15:18:16.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.838+0000 7f11acff9700 1 -- 172.21.15.52:0/4048156570 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b8005330 con 0x5651b9fbeb60 2023-11-27T15:18:16.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.838+0000 7f11be59c700 1 --2- 172.21.15.52:0/3061872635 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f11a80a1460 0x7f11a80a3910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f11b00cdd00 tx=0x7f11b0099890 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:16.849 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.839+0000 7f11d1143b80 1 -- 172.21.15.52:0/4048156570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651b9fbeb60 msgr2=0x5651b9fbef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:16.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.839+0000 7f11d1143b80 1 --2- 172.21.15.52:0/4048156570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651b9fbeb60 0x5651b9fbef30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f11b8005b40 tx=0x7f11b8005b70 comp rx=0 tx=0).stop 2023-11-27T15:18:16.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.839+0000 7f11d1143b80 1 -- 172.21.15.52:0/4048156570 shutdown_connections 2023-11-27T15:18:16.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.839+0000 7f11d1143b80 1 --2- 172.21.15.52:0/4048156570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651b9fc38b0 0x5651b9fc3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.839+0000 7f11d1143b80 1 --2- 172.21.15.52:0/4048156570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651b9fbeb60 0x5651b9fbef30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.839+0000 7f11d1143b80 1 --2- 172.21.15.52:0/4048156570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651b9fbf470 0x5651b9fc3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.839+0000 7f11d1143b80 1 -- 172.21.15.52:0/4048156570 >> 172.21.15.52:0/4048156570 conn(0x5651b9fb4360 msgr2=0x5651b9fb9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:16.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.839+0000 7f11d1143b80 1 -- 172.21.15.52:0/4048156570 shutdown_connections 2023-11-27T15:18:16.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.839+0000 7f11d1143b80 1 -- 172.21.15.52:0/4048156570 wait complete. 2023-11-27T15:18:16.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.839+0000 7f11d1143b80 1 -- start start 2023-11-27T15:18:16.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.839+0000 7f11d1143b80 1 Processor -- start 2023-11-27T15:18:16.851 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.840+0000 7f11d1143b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651b9fbeb60 0x5651b9fcb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.851 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.840+0000 7f11d1143b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651b9fbf470 0x5651b9fc8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.851 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.840+0000 7f11d1143b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651b9fc38b0 0x5651b9fc89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.851 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.840+0000 7f11d1143b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651b9eb18e0 con 0x5651b9fbf470 2023-11-27T15:18:16.851 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.840+0000 7f11d1143b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651b9eb1580 con 0x5651b9fbeb60 2023-11-27T15:18:16.851 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.840+0000 7f11d1143b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651b9d93920 con 0x5651b9fc38b0 2023-11-27T15:18:16.851 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.840+0000 7f11d1143b80 10 client.0 _ll_register_callbacks cb 0x5651b9ecf780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:18:16.851 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.840+0000 7f11bdd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651b9fbf470 0x5651b9fc8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.851 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.840+0000 7f11bdd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651b9fbf470 0x5651b9fc8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52274/0 (socket says 172.21.15.52:52274) 2023-11-27T15:18:16.851 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.840+0000 7f11bdd9b700 1 -- 172.21.15.52:0/2274468741 learned_addr learned my addr 172.21.15.52:0/2274468741 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:16.851 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.840+0000 7f11bed9d700 1 --2- 172.21.15.52:0/2274468741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651b9fbeb60 0x5651b9fcb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.851 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.840+0000 7f11bed9d700 1 --2- 172.21.15.52:0/2274468741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651b9fc38b0 0x5651b9fc89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.852 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.841+0000 7f11bdd9b700 1 -- 172.21.15.52:0/2274468741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651b9fc38b0 msgr2=0x5651b9fc89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:16.852 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.841+0000 7f11bdd9b700 1 --2- 172.21.15.52:0/2274468741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651b9fc38b0 0x5651b9fc89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.852 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.841+0000 7f11bdd9b700 1 -- 172.21.15.52:0/2274468741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651b9fbeb60 msgr2=0x5651b9fcb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:16.852 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.841+0000 7f11bdd9b700 1 --2- 172.21.15.52:0/2274468741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651b9fbeb60 0x5651b9fcb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.852 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.841+0000 7f11bdd9b700 1 -- 172.21.15.52:0/2274468741 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c010c8a0 con 0x5651b9fbf470 2023-11-27T15:18:16.852 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.841+0000 7f11bed9d700 1 --2- 172.21.15.52:0/2274468741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651b9fc38b0 0x5651b9fc89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:18:16.852 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.841+0000 7f11bdd9b700 1 --2- 172.21.15.52:0/2274468741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651b9fbf470 0x5651b9fc8290 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f11b400e5d0 tx=0x7f11b400e8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb919e9fbc80ffb server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:16.852 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.841+0000 7f11a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:18:16.852 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.841+0000 7f11a5ffb700 1 -- 172.21.15.52:0/2274468741 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b4004220 con 0x5651b9fbf470 2023-11-27T15:18:16.852 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.841+0000 7f11a5ffb700 1 -- 172.21.15.52:0/2274468741 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f11b4004380 con 0x5651b9fbf470 2023-11-27T15:18:16.852 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.841+0000 7f11d1143b80 1 -- 172.21.15.52:0/2274468741 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5651b9fb1b20 con 0x5651b9fbf470 2023-11-27T15:18:16.852 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.841+0000 7f11a5ffb700 1 -- 172.21.15.52:0/2274468741 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11b4004540 con 0x5651b9fbf470 2023-11-27T15:18:16.852 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.841+0000 7f11d1143b80 10 client.15519 fetch_fsmap learned FSMap version 23 2023-11-27T15:18:16.852 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.841+0000 7f11d1143b80 1 -- 172.21.15.52:0/2274468741 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5651b9ed6190 con 0x5651b9fbf470 2023-11-27T15:18:16.853 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.842+0000 7f11a5ffb700 1 -- 172.21.15.52:0/2274468741 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5651b9fb1b20 con 0x5651b9fbf470 2023-11-27T15:18:16.853 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.842+0000 7f11a5ffb700 1 -- 172.21.15.52:0/2274468741 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f11b4003ac0 con 0x5651b9fbf470 2023-11-27T15:18:16.853 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.842+0000 7f11d1143b80 10 client.15519 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:18:16.853 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.842+0000 7f11d1143b80 10 client.15519 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:18:16.853 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.842+0000 7f11d1143b80 20 client.15519 populate_metadata read hostname 'smithi052' 2023-11-27T15:18:16.853 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.842+0000 7f11d1143b80 1 --2- 172.21.15.52:0/2274468741 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x5651b9fe9760 0x5651b9febb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.853 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.842+0000 7f11d1143b80 4 client.15519 mds_command: new command op to 24494 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:18:16.853 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.842+0000 7f11d1143b80 1 -- 172.21.15.52:0/2274468741 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5651b9ed20a0 con 0x5651b9fe9760 2023-11-27T15:18:16.853 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.842+0000 7f11bed9d700 1 --2- 172.21.15.52:0/2274468741 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x5651b9fe9760 0x5651b9febb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.853 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.843+0000 7f11a5ffb700 10 client.15519 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:18:16.853 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.843+0000 7f11bed9d700 1 --2- 172.21.15.52:0/2274468741 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x5651b9fe9760 0x5651b9febb70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cb80b6b730b7d721 server_cookie=53ea0c6b0603b49a in_seq=0 out_seq=0 2023-11-27T15:18:16.853 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.843+0000 7f11a5ffb700 1 -- 172.21.15.52:0/2274468741 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5651b9ed20a0 con 0x5651b9fe9760 2023-11-27T15:18:16.853 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.843+0000 7f11a5ffb700 10 client.15519 handle_command_reply: tid=0 2023-11-27T15:18:16.854 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.843+0000 7f11d1143b80 1 client.15519 shutdown 2023-11-27T15:18:16.854 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.844+0000 7f11d1143b80 1 -- 172.21.15.52:0/2274468741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651b9fbf470 msgr2=0x5651b9fc8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:16.854 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.844+0000 7f11d1143b80 1 --2- 172.21.15.52:0/2274468741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651b9fbf470 0x5651b9fc8290 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f11b400e5d0 tx=0x7f11b400e8e0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.854 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.844+0000 7f11d1143b80 1 -- 172.21.15.52:0/2274468741 shutdown_connections 2023-11-27T15:18:16.854 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.844+0000 7f11d1143b80 1 --2- 172.21.15.52:0/2274468741 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x5651b9fe9760 0x5651b9febb70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.854 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.844+0000 7f11d1143b80 1 --2- 172.21.15.52:0/2274468741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651b9fbeb60 0x5651b9fcb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.854 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.844+0000 7f11a5ffb700 0 client.15519 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:18:16.854 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.844+0000 7f11d1143b80 1 --2- 172.21.15.52:0/2274468741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651b9fc38b0 0x5651b9fc89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.854 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.844+0000 7f11d1143b80 1 --2- 172.21.15.52:0/2274468741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651b9fbf470 0x5651b9fc8290 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.854 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.844+0000 7f11d1143b80 1 -- 172.21.15.52:0/2274468741 >> 172.21.15.52:0/2274468741 conn(0x5651b9fb4360 msgr2=0x5651b9fc4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:16.854 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.844+0000 7f11d1143b80 1 -- 172.21.15.52:0/2274468741 shutdown_connections 2023-11-27T15:18:16.855 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.844+0000 7f11d1143b80 1 -- 172.21.15.52:0/2274468741 wait complete. 2023-11-27T15:18:16.855 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.844+0000 7f11d1143b80 20 client.15519 trim_cache size 0 max 16384 2023-11-27T15:18:16.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.858+0000 7f11d1143b80 1 Processor -- start 2023-11-27T15:18:16.864 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.858+0000 7f11d1143b80 1 -- start start 2023-11-27T15:18:16.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.858+0000 7f11d1143b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651b9fbeb60 0x5651b9fcabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.858+0000 7f11d1143b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651b9fbf470 0x5651b9fc5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.858+0000 7f11d1143b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651b9fc38b0 0x5651b9fc61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.858+0000 7f11d1143b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651b9fb19a0 con 0x5651b9fbf470 2023-11-27T15:18:16.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.858+0000 7f11d1143b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651b9fbf930 con 0x5651b9fc38b0 2023-11-27T15:18:16.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.858+0000 7f11d1143b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651b9fc2460 con 0x5651b9fbeb60 2023-11-27T15:18:16.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.859+0000 7f11bdd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651b9fbf470 0x5651b9fc5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.859+0000 7f11bdd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651b9fbf470 0x5651b9fc5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52282/0 (socket says 172.21.15.52:52282) 2023-11-27T15:18:16.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.859+0000 7f11bed9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651b9fc38b0 0x5651b9fc61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.859+0000 7f11bdd9b700 1 -- 172.21.15.52:0/617105348 learned_addr learned my addr 172.21.15.52:0/617105348 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:16.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.859+0000 7f11bed9d700 1 --2- 172.21.15.52:0/617105348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651b9fbeb60 0x5651b9fcabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.866 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.859+0000 7f11bdd9b700 1 -- 172.21.15.52:0/617105348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651b9fbeb60 msgr2=0x5651b9fcabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:16.866 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.859+0000 7f11bdd9b700 1 --2- 172.21.15.52:0/617105348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651b9fbeb60 0x5651b9fcabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.866 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.859+0000 7f11bdd9b700 1 -- 172.21.15.52:0/617105348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651b9fc38b0 msgr2=0x5651b9fc61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:16.866 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.859+0000 7f11bdd9b700 1 --2- 172.21.15.52:0/617105348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651b9fc38b0 0x5651b9fc61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.866 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.859+0000 7f11bdd9b700 1 -- 172.21.15.52:0/617105348 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651b9ed5ee0 con 0x5651b9fbf470 2023-11-27T15:18:16.866 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.859+0000 7f11bed9d700 1 --2- 172.21.15.52:0/617105348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651b9fc38b0 0x5651b9fc61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:18:16.866 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.859+0000 7f11bdd9b700 1 --2- 172.21.15.52:0/617105348 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651b9fbf470 0x5651b9fc5c90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f11b4001eb0 tx=0x7f11b4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d2eb64076f208 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:16.866 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.860+0000 7f11ad7fa700 1 -- 172.21.15.52:0/617105348 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b400ece0 con 0x5651b9fbf470 2023-11-27T15:18:16.866 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.860+0000 7f11ad7fa700 1 -- 172.21.15.52:0/617105348 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11b4004460 con 0x5651b9fbf470 2023-11-27T15:18:16.866 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.860+0000 7f11ad7fa700 1 -- 172.21.15.52:0/617105348 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b4010960 con 0x5651b9fbf470 2023-11-27T15:18:16.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.860+0000 7f11d1143b80 1 -- 172.21.15.52:0/617105348 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651b9fbf470 msgr2=0x5651b9fc5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:16.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.860+0000 7f11d1143b80 1 --2- 172.21.15.52:0/617105348 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651b9fbf470 0x5651b9fc5c90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f11b4001eb0 tx=0x7f11b4005e80 comp rx=0 tx=0).stop 2023-11-27T15:18:16.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.860+0000 7f11d1143b80 1 -- 172.21.15.52:0/617105348 shutdown_connections 2023-11-27T15:18:16.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.860+0000 7f11d1143b80 1 --2- 172.21.15.52:0/617105348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651b9fc38b0 0x5651b9fc61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.860+0000 7f11d1143b80 1 --2- 172.21.15.52:0/617105348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651b9fbeb60 0x5651b9fcabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.860+0000 7f11d1143b80 1 --2- 172.21.15.52:0/617105348 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651b9fbf470 0x5651b9fc5c90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.860+0000 7f11d1143b80 1 -- 172.21.15.52:0/617105348 >> 172.21.15.52:0/617105348 conn(0x5651b9fb4360 msgr2=0x5651b9fe5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:16.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.861+0000 7f11d1143b80 1 -- 172.21.15.52:0/617105348 shutdown_connections 2023-11-27T15:18:16.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.862+0000 7f11d1143b80 1 -- 172.21.15.52:0/617105348 wait complete. 2023-11-27T15:18:16.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.862+0000 7f11d1143b80 1 -- start start 2023-11-27T15:18:16.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.862+0000 7f11d1143b80 1 Processor -- start 2023-11-27T15:18:16.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.862+0000 7f11d1143b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651b9fbeb60 0x5651b9fe02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.862+0000 7f11d1143b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651b9fbf470 0x5651b9fe0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.862+0000 7f11d1143b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651b9fc38b0 0x5651b9fde940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.862+0000 7f11d1143b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651b9fdeef0 con 0x5651b9fbf470 2023-11-27T15:18:16.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.862+0000 7f11d1143b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651b9fdf060 con 0x5651b9fc38b0 2023-11-27T15:18:16.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.862+0000 7f11d1143b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651b9fdf220 con 0x5651b9fbeb60 2023-11-27T15:18:16.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.862+0000 7f11d1143b80 10 client.0 _ll_register_callbacks cb 0x7f11b400caf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:18:16.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.862+0000 7f11bed9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651b9fc38b0 0x5651b9fde940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.863+0000 7f11bed9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651b9fc38b0 0x5651b9fde940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:41944/0 (socket says 172.21.15.52:41944) 2023-11-27T15:18:16.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.863+0000 7f11bed9d700 1 -- 172.21.15.52:0/3849595215 learned_addr learned my addr 172.21.15.52:0/3849595215 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:16.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.863+0000 7f11bed9d700 1 --2- 172.21.15.52:0/3849595215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651b9fbeb60 0x5651b9fe02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.863+0000 7f11bed9d700 1 -- 172.21.15.52:0/3849595215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651b9fbeb60 msgr2=0x5651b9fe02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:16.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.863+0000 7f11bed9d700 1 --2- 172.21.15.52:0/3849595215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651b9fbeb60 0x5651b9fe02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.863+0000 7f11bed9d700 1 -- 172.21.15.52:0/3849595215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651b9fbf470 msgr2=0x5651b9fe0800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-27T15:18:16.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.863+0000 7f11bed9d700 1 --2- 172.21.15.52:0/3849595215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651b9fbf470 0x5651b9fe0800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.863+0000 7f11bed9d700 1 -- 172.21.15.52:0/3849595215 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651b9d91630 con 0x5651b9fc38b0 2023-11-27T15:18:16.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.863+0000 7f11bed9d700 1 --2- 172.21.15.52:0/3849595215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651b9fc38b0 0x5651b9fde940 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f11b8003c90 tx=0x7f11b400cba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=799865a3a2121261 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:16.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.863+0000 7f11a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2023-11-27T15:18:16.872 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.863+0000 7f11a5ffb700 1 -- 172.21.15.52:0/3849595215 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b803c070 con 0x5651b9fc38b0 2023-11-27T15:18:16.872 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.863+0000 7f11a5ffb700 1 -- 172.21.15.52:0/3849595215 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f11b8007560 con 0x5651b9fc38b0 2023-11-27T15:18:16.872 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.863+0000 7f11d1143b80 1 -- 172.21.15.52:0/3849595215 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11b800bca0 con 0x5651b9fc38b0 2023-11-27T15:18:16.872 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.863+0000 7f11a5ffb700 1 -- 172.21.15.52:0/3849595215 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11b8007720 con 0x5651b9fc38b0 2023-11-27T15:18:16.872 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.864+0000 7f11d1143b80 10 client.25331 fetch_fsmap learned FSMap version 23 2023-11-27T15:18:16.872 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.864+0000 7f11d1143b80 1 -- 172.21.15.52:0/3849595215 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5651b9ed6190 con 0x5651b9fc38b0 2023-11-27T15:18:16.872 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.864+0000 7f11a5ffb700 1 -- 172.21.15.52:0/3849595215 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11b800bca0 con 0x5651b9fc38b0 2023-11-27T15:18:16.872 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.864+0000 7f11a5ffb700 1 -- 172.21.15.52:0/3849595215 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f11b803f070 con 0x5651b9fc38b0 2023-11-27T15:18:16.872 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.864+0000 7f11d1143b80 10 client.25331 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:18:16.872 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.864+0000 7f11d1143b80 10 client.25331 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:18:16.872 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.864+0000 7f11d1143b80 20 client.25331 populate_metadata read hostname 'smithi052' 2023-11-27T15:18:16.872 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.864+0000 7f11d1143b80 1 --2- 172.21.15.52:0/3849595215 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x5651ba03e470 0x5651ba040850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:16.872 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.864+0000 7f11d1143b80 4 client.25331 mds_command: new command op to 24494 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:18:16.872 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.864+0000 7f11d1143b80 1 -- 172.21.15.52:0/3849595215 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5651b9ede660 con 0x5651ba03e470 2023-11-27T15:18:16.873 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.864+0000 7f11bdd9b700 1 --2- 172.21.15.52:0/3849595215 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x5651ba03e470 0x5651ba040850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:16.874 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.865+0000 7f11a5ffb700 10 client.25331 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:18:16.874 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.865+0000 7f11bdd9b700 1 --2- 172.21.15.52:0/3849595215 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x5651ba03e470 0x5651ba040850 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5b89ddf3026ee5bc server_cookie=2310b7a4b0ceea51 in_seq=0 out_seq=0 2023-11-27T15:18:16.874 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.865+0000 7f11a5ffb700 1 -- 172.21.15.52:0/3849595215 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5651b9ede660 con 0x5651ba03e470 2023-11-27T15:18:16.874 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.865+0000 7f11a5ffb700 10 client.25331 handle_command_reply: tid=0 2023-11-27T15:18:16.874 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.865+0000 7f11d1143b80 1 client.25331 shutdown 2023-11-27T15:18:16.874 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.865+0000 7f11d1143b80 1 -- 172.21.15.52:0/3849595215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651b9fc38b0 msgr2=0x5651b9fde940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:16.874 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.865+0000 7f11d1143b80 1 --2- 172.21.15.52:0/3849595215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651b9fc38b0 0x5651b9fde940 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f11b8003c90 tx=0x7f11b400cba0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.874 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.865+0000 7f11d1143b80 1 -- 172.21.15.52:0/3849595215 shutdown_connections 2023-11-27T15:18:16.874 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.865+0000 7f11d1143b80 1 --2- 172.21.15.52:0/3849595215 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x5651ba03e470 0x5651ba040850 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.875 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.865+0000 7f11d1143b80 1 --2- 172.21.15.52:0/3849595215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651b9fc38b0 0x5651b9fde940 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.875 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.865+0000 7f11d1143b80 1 --2- 172.21.15.52:0/3849595215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5651b9fbeb60 0x5651b9fe02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.875 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.865+0000 7f11d1143b80 1 --2- 172.21.15.52:0/3849595215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651b9fbf470 0x5651b9fe0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.875 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.865+0000 7f11d1143b80 1 -- 172.21.15.52:0/3849595215 >> 172.21.15.52:0/3849595215 conn(0x5651b9fb4360 msgr2=0x5651b9fc5c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:16.875 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.865+0000 7f11a5ffb700 0 client.25331 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:18:16.875 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.865+0000 7f11d1143b80 1 -- 172.21.15.52:0/3849595215 shutdown_connections 2023-11-27T15:18:16.875 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.865+0000 7f11d1143b80 1 -- 172.21.15.52:0/3849595215 wait complete. 2023-11-27T15:18:16.875 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.866+0000 7f11d1143b80 20 client.25331 trim_cache size 0 max 16384 2023-11-27T15:18:16.875 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:18:16.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.866+0000 7f11c4fb7700 1 -- 172.21.15.52:0/3061872635 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f11a80a1460 msgr2=0x7f11a80a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:16.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.866+0000 7f11c4fb7700 1 --2- 172.21.15.52:0/3061872635 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f11a80a1460 0x7f11a80a3910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f11b00cdd00 tx=0x7f11b0099890 comp rx=0 tx=0).stop 2023-11-27T15:18:16.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.866+0000 7f11c4fb7700 1 -- 172.21.15.52:0/3061872635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11c00ff990 msgr2=0x7f11c01965a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:16.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.866+0000 7f11c4fb7700 1 --2- 172.21.15.52:0/3061872635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11c00ff990 0x7f11c01965a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f11b0009c50 tx=0x7f11b0009f60 comp rx=0 tx=0).stop 2023-11-27T15:18:16.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.866+0000 7f11c4fb7700 1 -- 172.21.15.52:0/3061872635 shutdown_connections 2023-11-27T15:18:16.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.866+0000 7f11c4fb7700 1 --2- 172.21.15.52:0/3061872635 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f11a80a1460 0x7f11a80a3910 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.866+0000 7f11c4fb7700 1 --2- 172.21.15.52:0/3061872635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11c00ff990 0x7f11c01965a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.866+0000 7f11c4fb7700 1 --2- 172.21.15.52:0/3061872635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11c0104180 0x7f11c019af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.866+0000 7f11c4fb7700 1 --2- 172.21.15.52:0/3061872635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11c0100340 0x7f11c0196ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:16.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.866+0000 7f11c4fb7700 1 -- 172.21.15.52:0/3061872635 >> 172.21.15.52:0/3061872635 conn(0x7f11c0075920 msgr2=0x7f11c00fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:16.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.866+0000 7f11c4fb7700 1 -- 172.21.15.52:0/3061872635 shutdown_connections 2023-11-27T15:18:16.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:16.866+0000 7f11c4fb7700 1 -- 172.21.15.52:0/3061872635 wait complete. 2023-11-27T15:18:16.884 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-27T15:18:17.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.015+0000 7f90bf771700 1 -- 172.21.15.52:0/283939402 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90b81055f0 msgr2=0x7f90b810d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:17.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.015+0000 7f90bf771700 1 --2- 172.21.15.52:0/283939402 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90b81055f0 0x7f90b810d1c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f90b000b0d0 tx=0x7f90b000b3e0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.015+0000 7f90bf771700 1 -- 172.21.15.52:0/283939402 shutdown_connections 2023-11-27T15:18:17.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.015+0000 7f90bf771700 1 --2- 172.21.15.52:0/283939402 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90b81005b0 0x7f90b8100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.015+0000 7f90bf771700 1 --2- 172.21.15.52:0/283939402 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90b81055f0 0x7f90b810d1c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.015+0000 7f90bf771700 1 --2- 172.21.15.52:0/283939402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b8100f50 0x7f90b8104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.015+0000 7f90bf771700 1 -- 172.21.15.52:0/283939402 >> 172.21.15.52:0/283939402 conn(0x7f90b8077170 msgr2=0x7f90b8077570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:17.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.015+0000 7f90bf771700 1 -- 172.21.15.52:0/283939402 shutdown_connections 2023-11-27T15:18:17.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.015+0000 7f90bf771700 1 -- 172.21.15.52:0/283939402 wait complete. 2023-11-27T15:18:17.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.016+0000 7f90bf771700 1 Processor -- start 2023-11-27T15:18:17.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.016+0000 7f90bf771700 1 -- start start 2023-11-27T15:18:17.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.016+0000 7f90bf771700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b81005b0 0x7f90b8196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:17.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.016+0000 7f90bf771700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90b8100f50 0x7f90b8196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:17.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.016+0000 7f90bf771700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90b81055f0 0x7f90b819aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:17.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.016+0000 7f90bf771700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b819b500 con 0x7f90b81055f0 2023-11-27T15:18:17.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.016+0000 7f90bf771700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b819b670 con 0x7f90b8100f50 2023-11-27T15:18:17.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.016+0000 7f90bf771700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b819b830 con 0x7f90b81005b0 2023-11-27T15:18:17.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.016+0000 7f90bdd0e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90b81055f0 0x7f90b819aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:17.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.016+0000 7f90bd50d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b81005b0 0x7f90b8196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:17.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.017+0000 7f90bdd0e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90b81055f0 0x7f90b819aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52296/0 (socket says 172.21.15.52:52296) 2023-11-27T15:18:17.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.017+0000 7f90bd50d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b81005b0 0x7f90b8196520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:38938/0 (socket says 172.21.15.52:38938) 2023-11-27T15:18:17.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.017+0000 7f90bdd0e700 1 -- 172.21.15.52:0/3240994948 learned_addr learned my addr 172.21.15.52:0/3240994948 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:17.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.017+0000 7f90bcd0c700 1 --2- 172.21.15.52:0/3240994948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90b8100f50 0x7f90b8196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:17.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.017+0000 7f90bd50d700 1 -- 172.21.15.52:0/3240994948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90b8100f50 msgr2=0x7f90b8196a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:17.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.017+0000 7f90bd50d700 1 --2- 172.21.15.52:0/3240994948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90b8100f50 0x7f90b8196a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.017+0000 7f90bd50d700 1 -- 172.21.15.52:0/3240994948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90b81055f0 msgr2=0x7f90b819aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:17.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.017+0000 7f90bd50d700 1 --2- 172.21.15.52:0/3240994948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90b81055f0 0x7f90b819aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.017+0000 7f90bd50d700 1 -- 172.21.15.52:0/3240994948 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b0009d00 con 0x7f90b81005b0 2023-11-27T15:18:17.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.017+0000 7f90bcd0c700 1 --2- 172.21.15.52:0/3240994948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90b8100f50 0x7f90b8196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:18:17.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.017+0000 7f90bd50d700 1 --2- 172.21.15.52:0/3240994948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b81005b0 0x7f90b8196520 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f90b400e9f0 tx=0x7f90b400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:17.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.017+0000 7f90ae7fc700 1 -- 172.21.15.52:0/3240994948 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b400eee0 con 0x7f90b81005b0 2023-11-27T15:18:17.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.017+0000 7f90bf771700 1 -- 172.21.15.52:0/3240994948 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b8110410 con 0x7f90b81005b0 2023-11-27T15:18:17.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.017+0000 7f90ae7fc700 1 -- 172.21.15.52:0/3240994948 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90b400cbd0 con 0x7f90b81005b0 2023-11-27T15:18:17.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.017+0000 7f90ae7fc700 1 -- 172.21.15.52:0/3240994948 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b4018680 con 0x7f90b81005b0 2023-11-27T15:18:17.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.017+0000 7f90bf771700 1 -- 172.21.15.52:0/3240994948 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b8110280 con 0x7f90b81005b0 2023-11-27T15:18:17.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.020+0000 7f90ae7fc700 1 -- 172.21.15.52:0/3240994948 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f90b400cd40 con 0x7f90b81005b0 2023-11-27T15:18:17.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.021+0000 7f90ae7fc700 1 --2- 172.21.15.52:0/3240994948 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f90a40a13e0 0x7f90a40a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:17.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.021+0000 7f90bcd0c700 1 --2- 172.21.15.52:0/3240994948 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f90a40a13e0 0x7f90a40a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:17.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.021+0000 7f90ae7fc700 1 -- 172.21.15.52:0/3240994948 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f90b4014070 con 0x7f90b81005b0 2023-11-27T15:18:17.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.022+0000 7f90cb8fdb80 1 Processor -- start 2023-11-27T15:18:17.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.022+0000 7f90cb8fdb80 1 -- start start 2023-11-27T15:18:17.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.022+0000 7f90bcd0c700 1 --2- 172.21.15.52:0/3240994948 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f90a40a13e0 0x7f90a40a3890 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f90b8197b30 tx=0x7f90a8009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:17.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.022+0000 7f90cb8fdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556628dcdb60 0x556628dcdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:17.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.022+0000 7f90cb8fdb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556628dce470 0x556628dd2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:17.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.022+0000 7f90cb8fdb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556628dd28b0 0x556628dd2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:17.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.022+0000 7f90cb8fdb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x556628cd3060 con 0x556628dce470 2023-11-27T15:18:17.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.022+0000 7f90cb8fdb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556628d258a0 con 0x556628dd28b0 2023-11-27T15:18:17.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.022+0000 7f90cb8fdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556628d1ed30 con 0x556628dcdb60 2023-11-27T15:18:17.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.023+0000 7f90bcd0c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556628dce470 0x556628dd2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:17.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.023+0000 7f90bcd0c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556628dce470 0x556628dd2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52304/0 (socket says 172.21.15.52:52304) 2023-11-27T15:18:17.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.023+0000 7f90bcd0c700 1 -- 172.21.15.52:0/2945086350 learned_addr learned my addr 172.21.15.52:0/2945086350 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:17.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.023+0000 7f90bdd0e700 1 --2- 172.21.15.52:0/2945086350 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556628dd28b0 0x556628dd2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:17.030 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.023+0000 7f90bd50d700 1 --2- 172.21.15.52:0/2945086350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556628dcdb60 0x556628dcdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:17.030 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.023+0000 7f90bdd0e700 1 -- 172.21.15.52:0/2945086350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556628dcdb60 msgr2=0x556628dcdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:17.030 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.023+0000 7f90bdd0e700 1 --2- 172.21.15.52:0/2945086350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556628dcdb60 0x556628dcdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.030 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.023+0000 7f90bdd0e700 1 -- 172.21.15.52:0/2945086350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556628dce470 msgr2=0x556628dd2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:17.030 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.023+0000 7f90bdd0e700 1 --2- 172.21.15.52:0/2945086350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556628dce470 0x556628dd2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.030 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.023+0000 7f90bdd0e700 1 -- 172.21.15.52:0/2945086350 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556628ba0630 con 0x556628dd28b0 2023-11-27T15:18:17.030 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.023+0000 7f90bcd0c700 1 --2- 172.21.15.52:0/2945086350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556628dce470 0x556628dd2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:18:17.030 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.023+0000 7f90bdd0e700 1 --2- 172.21.15.52:0/2945086350 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556628dd28b0 0x556628dd2d00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009c60 tx=0x7f90b0009c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de3c6c5628eb08f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:17.030 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.023+0000 7f90a37fe700 1 -- 172.21.15.52:0/2945086350 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b000e040 con 0x556628dd28b0 2023-11-27T15:18:17.031 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.024+0000 7f90a37fe700 1 -- 172.21.15.52:0/2945086350 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90b00074e0 con 0x556628dd28b0 2023-11-27T15:18:17.031 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.024+0000 7f90a37fe700 1 -- 172.21.15.52:0/2945086350 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b0005260 con 0x556628dd28b0 2023-11-27T15:18:17.034 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.024+0000 7f90cb8fdb80 1 -- 172.21.15.52:0/2945086350 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556628dd28b0 msgr2=0x556628dd2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:17.035 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.024+0000 7f90cb8fdb80 1 --2- 172.21.15.52:0/2945086350 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556628dd28b0 0x556628dd2d00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009c60 tx=0x7f90b0009c90 comp rx=0 tx=0).stop 2023-11-27T15:18:17.035 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.024+0000 7f90cb8fdb80 1 -- 172.21.15.52:0/2945086350 shutdown_connections 2023-11-27T15:18:17.035 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.024+0000 7f90cb8fdb80 1 --2- 172.21.15.52:0/2945086350 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556628dd28b0 0x556628dd2d00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.035 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.024+0000 7f90cb8fdb80 1 --2- 172.21.15.52:0/2945086350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556628dcdb60 0x556628dcdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.035 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.024+0000 7f90cb8fdb80 1 --2- 172.21.15.52:0/2945086350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556628dce470 0x556628dd2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.035 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.024+0000 7f90cb8fdb80 1 -- 172.21.15.52:0/2945086350 >> 172.21.15.52:0/2945086350 conn(0x556628dc3360 msgr2=0x556628dc8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:17.035 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.024+0000 7f90cb8fdb80 1 -- 172.21.15.52:0/2945086350 shutdown_connections 2023-11-27T15:18:17.035 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.024+0000 7f90cb8fdb80 1 -- 172.21.15.52:0/2945086350 wait complete. 2023-11-27T15:18:17.035 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.025+0000 7f90cb8fdb80 1 -- start start 2023-11-27T15:18:17.035 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.025+0000 7f90cb8fdb80 1 Processor -- start 2023-11-27T15:18:17.035 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.026+0000 7f90cb8fdb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556628dcdb60 0x556628dd9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:17.035 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.026+0000 7f90cb8fdb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556628dce470 0x556628dd7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:17.036 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.026+0000 7f90cb8fdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556628dd28b0 0x556628dd7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:17.036 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.026+0000 7f90cb8fdb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x556628cc08e0 con 0x556628dce470 2023-11-27T15:18:17.036 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.026+0000 7f90cb8fdb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556628cc0580 con 0x556628dcdb60 2023-11-27T15:18:17.036 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.026+0000 7f90cb8fdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556628ba2920 con 0x556628dd28b0 2023-11-27T15:18:17.036 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.026+0000 7f90cb8fdb80 10 client.0 _ll_register_callbacks cb 0x556628cde780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:18:17.036 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.026+0000 7f90bcd0c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556628dce470 0x556628dd7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:17.036 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.026+0000 7f90bcd0c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556628dce470 0x556628dd7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52312/0 (socket says 172.21.15.52:52312) 2023-11-27T15:18:17.036 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.026+0000 7f90bcd0c700 1 -- 172.21.15.52:0/2254566985 learned_addr learned my addr 172.21.15.52:0/2254566985 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:17.036 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.026+0000 7f90bd50d700 1 --2- 172.21.15.52:0/2254566985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556628dcdb60 0x556628dd9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:17.036 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.026+0000 7f90bdd0e700 1 --2- 172.21.15.52:0/2254566985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556628dd28b0 0x556628dd7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:17.036 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.026+0000 7f90bd50d700 1 -- 172.21.15.52:0/2254566985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556628dd28b0 msgr2=0x556628dd7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:17.036 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.026+0000 7f90bd50d700 1 --2- 172.21.15.52:0/2254566985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556628dd28b0 0x556628dd7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.036 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.026+0000 7f90bd50d700 1 -- 172.21.15.52:0/2254566985 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556628dce470 msgr2=0x556628dd7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:17.036 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.026+0000 7f90bd50d700 1 --2- 172.21.15.52:0/2254566985 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556628dce470 0x556628dd7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.026+0000 7f90bd50d700 1 -- 172.21.15.52:0/2254566985 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b810da00 con 0x556628dcdb60 2023-11-27T15:18:17.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.026+0000 7f90bdd0e700 1 --2- 172.21.15.52:0/2254566985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556628dd28b0 0x556628dd7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:18:17.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.026+0000 7f90bcd0c700 1 --2- 172.21.15.52:0/2254566985 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556628dce470 0x556628dd7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:18:17.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.027+0000 7f90bd50d700 1 --2- 172.21.15.52:0/2254566985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556628dcdb60 0x556628dd9f80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f90b40ca760 tx=0x7f90b409a550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e4860d30a7a016b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:17.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.027+0000 7f90a0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2023-11-27T15:18:17.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.027+0000 7f90a0ff9700 1 -- 172.21.15.52:0/2254566985 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b409e040 con 0x556628dcdb60 2023-11-27T15:18:17.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.027+0000 7f90cb8fdb80 1 -- 172.21.15.52:0/2254566985 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556628dc0b20 con 0x556628dcdb60 2023-11-27T15:18:17.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.027+0000 7f90a0ff9700 1 -- 172.21.15.52:0/2254566985 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f90b40a2460 con 0x556628dcdb60 2023-11-27T15:18:17.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.027+0000 7f90a0ff9700 1 -- 172.21.15.52:0/2254566985 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90b40b8310 con 0x556628dcdb60 2023-11-27T15:18:17.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.027+0000 7f90cb8fdb80 10 client.25343 fetch_fsmap learned FSMap version 23 2023-11-27T15:18:17.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.027+0000 7f90cb8fdb80 1 -- 172.21.15.52:0/2254566985 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x556628ce5190 con 0x556628dcdb60 2023-11-27T15:18:17.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.027+0000 7f90a0ff9700 1 -- 172.21.15.52:0/2254566985 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556628dc0b20 con 0x556628dcdb60 2023-11-27T15:18:17.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.027+0000 7f90a0ff9700 1 -- 172.21.15.52:0/2254566985 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f90b4099090 con 0x556628dcdb60 2023-11-27T15:18:17.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.027+0000 7f90cb8fdb80 10 client.25343 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:18:17.038 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.027+0000 7f90cb8fdb80 10 client.25343 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:18:17.038 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.027+0000 7f90cb8fdb80 20 client.25343 populate_metadata read hostname 'smithi052' 2023-11-27T15:18:17.038 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.027+0000 7f90cb8fdb80 1 --2- 172.21.15.52:0/2254566985 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x556628e00fd0 0x556628e033e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:17.038 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.027+0000 7f90cb8fdb80 4 client.25343 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:18:17.038 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.027+0000 7f90cb8fdb80 1 -- 172.21.15.52:0/2254566985 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556628ce10a0 con 0x556628e00fd0 2023-11-27T15:18:17.038 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.028+0000 7f90bcd0c700 1 --2- 172.21.15.52:0/2254566985 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x556628e00fd0 0x556628e033e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:17.038 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.028+0000 7f90bcd0c700 1 --2- 172.21.15.52:0/2254566985 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x556628e00fd0 0x556628e033e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fe3ad9d39aa3794 server_cookie=77ca7d16be8cb0c5 in_seq=0 out_seq=0 2023-11-27T15:18:17.038 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.028+0000 7f90a0ff9700 10 client.25343 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:18:17.038 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.029+0000 7f90a0ff9700 1 -- 172.21.15.52:0/2254566985 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556628ce10a0 con 0x556628e00fd0 2023-11-27T15:18:17.038 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.029+0000 7f90a0ff9700 10 client.25343 handle_command_reply: tid=0 2023-11-27T15:18:17.038 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.029+0000 7f90cb8fdb80 1 client.25343 shutdown 2023-11-27T15:18:17.038 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.029+0000 7f90cb8fdb80 1 -- 172.21.15.52:0/2254566985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556628dcdb60 msgr2=0x556628dd9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:17.038 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.029+0000 7f90cb8fdb80 1 --2- 172.21.15.52:0/2254566985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556628dcdb60 0x556628dd9f80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f90b40ca760 tx=0x7f90b409a550 comp rx=0 tx=0).stop 2023-11-27T15:18:17.039 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.029+0000 7f90cb8fdb80 1 -- 172.21.15.52:0/2254566985 shutdown_connections 2023-11-27T15:18:17.039 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.029+0000 7f90cb8fdb80 1 --2- 172.21.15.52:0/2254566985 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x556628e00fd0 0x556628e033e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.039 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.029+0000 7f90cb8fdb80 1 --2- 172.21.15.52:0/2254566985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556628dcdb60 0x556628dd9f80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.039 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.029+0000 7f90cb8fdb80 1 --2- 172.21.15.52:0/2254566985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556628dd28b0 0x556628dd7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.039 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.029+0000 7f90cb8fdb80 1 --2- 172.21.15.52:0/2254566985 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556628dce470 0x556628dd7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.039 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.029+0000 7f90cb8fdb80 1 -- 172.21.15.52:0/2254566985 >> 172.21.15.52:0/2254566985 conn(0x556628dc3360 msgr2=0x556628dd3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:17.039 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.029+0000 7f90a0ff9700 0 client.25343 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:18:17.039 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.029+0000 7f90cb8fdb80 1 -- 172.21.15.52:0/2254566985 shutdown_connections 2023-11-27T15:18:17.039 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.029+0000 7f90cb8fdb80 1 -- 172.21.15.52:0/2254566985 wait complete. 2023-11-27T15:18:17.039 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.029+0000 7f90cb8fdb80 20 client.25343 trim_cache size 0 max 16384 2023-11-27T15:18:17.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.043+0000 7f90cb8fdb80 1 Processor -- start 2023-11-27T15:18:17.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.043+0000 7f90cb8fdb80 1 -- start start 2023-11-27T15:18:17.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.043+0000 7f90cb8fdb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556628dcdb60 0x556628dc8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:17.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.043+0000 7f90cb8fdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556628dce470 0x556628def980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:17.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.043+0000 7f90cb8fdb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556628dd28b0 0x556628dcd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:17.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.043+0000 7f90cb8fdb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x556628dcebc0 con 0x556628dcdb60 2023-11-27T15:18:17.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.043+0000 7f90cb8fdb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556628dc2ee0 con 0x556628dd28b0 2023-11-27T15:18:17.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.043+0000 7f90cb8fdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556628df5140 con 0x556628dce470 2023-11-27T15:18:17.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.044+0000 7f90bdd0e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556628dd28b0 0x556628dcd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:17.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.044+0000 7f90bcd0c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556628dce470 0x556628def980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:17.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.044+0000 7f90bdd0e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556628dd28b0 0x556628dcd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:41974/0 (socket says 172.21.15.52:41974) 2023-11-27T15:18:17.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.044+0000 7f90bd50d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556628dcdb60 0x556628dc8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:17.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.044+0000 7f90bdd0e700 1 -- 172.21.15.52:0/1240338706 learned_addr learned my addr 172.21.15.52:0/1240338706 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:17.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.044+0000 7f90bdd0e700 1 -- 172.21.15.52:0/1240338706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556628dce470 msgr2=0x556628def980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:17.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.044+0000 7f90bdd0e700 1 --2- 172.21.15.52:0/1240338706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556628dce470 0x556628def980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.044+0000 7f90bdd0e700 1 -- 172.21.15.52:0/1240338706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556628dcdb60 msgr2=0x556628dc8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:17.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.044+0000 7f90bdd0e700 1 --2- 172.21.15.52:0/1240338706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556628dcdb60 0x556628dc8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.044+0000 7f90bdd0e700 1 -- 172.21.15.52:0/1240338706 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556628ce4ee0 con 0x556628dd28b0 2023-11-27T15:18:17.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.044+0000 7f90bd50d700 1 --2- 172.21.15.52:0/1240338706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556628dcdb60 0x556628dc8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:18:17.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.044+0000 7f90bcd0c700 1 --2- 172.21.15.52:0/1240338706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556628dce470 0x556628def980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:18:17.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.045+0000 7f90bdd0e700 1 --2- 172.21.15.52:0/1240338706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556628dd28b0 0x556628dcd070 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f90b0003c10 tx=0x7f90b0009550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b08679ffd7500643 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:17.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.045+0000 7f90a3fff700 1 -- 172.21.15.52:0/1240338706 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b000e040 con 0x556628dd28b0 2023-11-27T15:18:17.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.045+0000 7f90a3fff700 1 -- 172.21.15.52:0/1240338706 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90b0005070 con 0x556628dd28b0 2023-11-27T15:18:17.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.045+0000 7f90a3fff700 1 -- 172.21.15.52:0/1240338706 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b002d6b0 con 0x556628dd28b0 2023-11-27T15:18:17.054 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.046+0000 7f90cb8fdb80 1 -- 172.21.15.52:0/1240338706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556628dd28b0 msgr2=0x556628dcd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:17.054 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.046+0000 7f90cb8fdb80 1 --2- 172.21.15.52:0/1240338706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556628dd28b0 0x556628dcd070 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f90b0003c10 tx=0x7f90b0009550 comp rx=0 tx=0).stop 2023-11-27T15:18:17.054 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.046+0000 7f90cb8fdb80 1 -- 172.21.15.52:0/1240338706 shutdown_connections 2023-11-27T15:18:17.054 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.046+0000 7f90cb8fdb80 1 --2- 172.21.15.52:0/1240338706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556628dd28b0 0x556628dcd070 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.054 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.046+0000 7f90cb8fdb80 1 --2- 172.21.15.52:0/1240338706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556628dcdb60 0x556628dc8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.054 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.046+0000 7f90cb8fdb80 1 --2- 172.21.15.52:0/1240338706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556628dce470 0x556628def980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.054 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.046+0000 7f90cb8fdb80 1 -- 172.21.15.52:0/1240338706 >> 172.21.15.52:0/1240338706 conn(0x556628dc3360 msgr2=0x556628dcc960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:17.054 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.046+0000 7f90cb8fdb80 1 -- 172.21.15.52:0/1240338706 shutdown_connections 2023-11-27T15:18:17.054 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.046+0000 7f90cb8fdb80 1 -- 172.21.15.52:0/1240338706 wait complete. 2023-11-27T15:18:17.054 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.046+0000 7f90cb8fdb80 1 -- start start 2023-11-27T15:18:17.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.046+0000 7f90cb8fdb80 1 Processor -- start 2023-11-27T15:18:17.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.047+0000 7f90cb8fdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556628dcdb60 0x556628deb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:17.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.047+0000 7f90cb8fdb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556628dce470 0x556628debea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:17.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.047+0000 7f90cb8fdb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556628dd28b0 0x556628e53e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:17.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.047+0000 7f90cb8fdb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x556628dec7b0 con 0x556628dd28b0 2023-11-27T15:18:17.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.047+0000 7f90cb8fdb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556628e543c0 con 0x556628dce470 2023-11-27T15:18:17.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.047+0000 7f90cb8fdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556628df5e50 con 0x556628dcdb60 2023-11-27T15:18:17.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.047+0000 7f90cb8fdb80 10 client.0 _ll_register_callbacks cb 0x7f90b4099970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:18:17.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.047+0000 7f90bdd0e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556628dd28b0 0x556628e53e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:17.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.047+0000 7f90bd50d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556628dcdb60 0x556628deb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:17.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.047+0000 7f90bdd0e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556628dd28b0 0x556628e53e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52326/0 (socket says 172.21.15.52:52326) 2023-11-27T15:18:17.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.047+0000 7f90bd50d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556628dcdb60 0x556628deb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:38966/0 (socket says 172.21.15.52:38966) 2023-11-27T15:18:17.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.047+0000 7f90bdd0e700 1 -- 172.21.15.52:0/2671514249 learned_addr learned my addr 172.21.15.52:0/2671514249 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:17.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.047+0000 7f90bcd0c700 1 --2- 172.21.15.52:0/2671514249 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556628dce470 0x556628debea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:17.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.047+0000 7f90bdd0e700 1 -- 172.21.15.52:0/2671514249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556628dcdb60 msgr2=0x556628deb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:17.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.047+0000 7f90bdd0e700 1 --2- 172.21.15.52:0/2671514249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556628dcdb60 0x556628deb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.047+0000 7f90bdd0e700 1 -- 172.21.15.52:0/2671514249 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556628dce470 msgr2=0x556628debea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:17.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.047+0000 7f90bdd0e700 1 --2- 172.21.15.52:0/2671514249 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556628dce470 0x556628debea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.047+0000 7f90bdd0e700 1 -- 172.21.15.52:0/2671514249 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556628ce5190 con 0x556628dd28b0 2023-11-27T15:18:17.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.048+0000 7f90bdd0e700 1 --2- 172.21.15.52:0/2671514249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556628dd28b0 0x556628e53e80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f90b0006230 tx=0x7f90b00055e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15216c50b61719be server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:17.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.048+0000 7f90a0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:18:17.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.048+0000 7f90a0ff9700 1 -- 172.21.15.52:0/2671514249 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b0003a80 con 0x556628dd28b0 2023-11-27T15:18:17.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.048+0000 7f90a0ff9700 1 -- 172.21.15.52:0/2671514249 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f90b000baa0 con 0x556628dd28b0 2023-11-27T15:18:17.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.048+0000 7f90cb8fdb80 1 -- 172.21.15.52:0/2671514249 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f90b00074e0 con 0x556628dd28b0 2023-11-27T15:18:17.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.048+0000 7f90a0ff9700 1 -- 172.21.15.52:0/2671514249 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90b000bc80 con 0x556628dd28b0 2023-11-27T15:18:17.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.048+0000 7f90cb8fdb80 10 client.15540 fetch_fsmap learned FSMap version 23 2023-11-27T15:18:17.057 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.048+0000 7f90cb8fdb80 1 -- 172.21.15.52:0/2671514249 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f90a8029950 con 0x556628dd28b0 2023-11-27T15:18:17.057 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.048+0000 7f90a0ff9700 1 -- 172.21.15.52:0/2671514249 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90b00074e0 con 0x556628dd28b0 2023-11-27T15:18:17.057 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.048+0000 7f90a0ff9700 1 -- 172.21.15.52:0/2671514249 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f90b000b100 con 0x556628dd28b0 2023-11-27T15:18:17.057 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.049+0000 7f90cb8fdb80 10 client.15540 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:18:17.057 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.049+0000 7f90cb8fdb80 10 client.15540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:18:17.057 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.049+0000 7f90cb8fdb80 20 client.15540 populate_metadata read hostname 'smithi052' 2023-11-27T15:18:17.057 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.049+0000 7f90cb8fdb80 1 --2- 172.21.15.52:0/2671514249 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x556628df7060 0x556628df9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:17.057 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.049+0000 7f90cb8fdb80 4 client.15540 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:18:17.057 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.049+0000 7f90cb8fdb80 1 -- 172.21.15.52:0/2671514249 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556628ced660 con 0x556628df7060 2023-11-27T15:18:17.057 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.049+0000 7f90bd50d700 1 --2- 172.21.15.52:0/2671514249 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x556628df7060 0x556628df9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:17.058 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.049+0000 7f90a0ff9700 10 client.15540 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:18:17.058 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.049+0000 7f90bd50d700 1 --2- 172.21.15.52:0/2671514249 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x556628df7060 0x556628df9440 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fba32f133e466a64 server_cookie=ae6f6fadf25f126a in_seq=0 out_seq=0 2023-11-27T15:18:17.058 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.049+0000 7f90a0ff9700 1 -- 172.21.15.52:0/2671514249 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556628ced660 con 0x556628df7060 2023-11-27T15:18:17.058 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.049+0000 7f90a0ff9700 10 client.15540 handle_command_reply: tid=0 2023-11-27T15:18:17.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.050+0000 7f90cb8fdb80 1 client.15540 shutdown 2023-11-27T15:18:17.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.050+0000 7f90cb8fdb80 1 -- 172.21.15.52:0/2671514249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556628dd28b0 msgr2=0x556628e53e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:17.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.050+0000 7f90cb8fdb80 1 --2- 172.21.15.52:0/2671514249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556628dd28b0 0x556628e53e80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f90b0006230 tx=0x7f90b00055e0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.050+0000 7f90cb8fdb80 1 -- 172.21.15.52:0/2671514249 shutdown_connections 2023-11-27T15:18:17.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.050+0000 7f90cb8fdb80 1 --2- 172.21.15.52:0/2671514249 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x556628df7060 0x556628df9440 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.050+0000 7f90cb8fdb80 1 --2- 172.21.15.52:0/2671514249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556628dd28b0 0x556628e53e80 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.050+0000 7f90cb8fdb80 1 --2- 172.21.15.52:0/2671514249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556628dcdb60 0x556628deb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.050+0000 7f90a0ff9700 0 client.15540 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:18:17.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.050+0000 7f90cb8fdb80 1 --2- 172.21.15.52:0/2671514249 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556628dce470 0x556628debea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.050+0000 7f90cb8fdb80 1 -- 172.21.15.52:0/2671514249 >> 172.21.15.52:0/2671514249 conn(0x556628dc3360 msgr2=0x556628dcd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:17.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.050+0000 7f90cb8fdb80 1 -- 172.21.15.52:0/2671514249 shutdown_connections 2023-11-27T15:18:17.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.050+0000 7f90cb8fdb80 1 -- 172.21.15.52:0/2671514249 wait complete. 2023-11-27T15:18:17.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.050+0000 7f90cb8fdb80 20 client.15540 trim_cache size 0 max 16384 2023-11-27T15:18:17.060 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:18:17.060 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.051+0000 7f90bf771700 1 -- 172.21.15.52:0/3240994948 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f90a40a13e0 msgr2=0x7f90a40a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:17.060 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.051+0000 7f90bf771700 1 --2- 172.21.15.52:0/3240994948 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f90a40a13e0 0x7f90a40a3890 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f90b8197b30 tx=0x7f90a8009260 comp rx=0 tx=0).stop 2023-11-27T15:18:17.061 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.051+0000 7f90bf771700 1 -- 172.21.15.52:0/3240994948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b81005b0 msgr2=0x7f90b8196520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:17.061 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.051+0000 7f90bf771700 1 --2- 172.21.15.52:0/3240994948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b81005b0 0x7f90b8196520 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f90b400e9f0 tx=0x7f90b400ed00 comp rx=0 tx=0).stop 2023-11-27T15:18:17.061 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.051+0000 7f90bf771700 1 -- 172.21.15.52:0/3240994948 shutdown_connections 2023-11-27T15:18:17.061 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.051+0000 7f90bf771700 1 --2- 172.21.15.52:0/3240994948 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f90a40a13e0 0x7f90a40a3890 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.061 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.051+0000 7f90bf771700 1 --2- 172.21.15.52:0/3240994948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90b81055f0 0x7f90b819aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.061 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.051+0000 7f90bf771700 1 --2- 172.21.15.52:0/3240994948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b81005b0 0x7f90b8196520 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.061 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.051+0000 7f90bf771700 1 --2- 172.21.15.52:0/3240994948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90b8100f50 0x7f90b8196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.061 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.051+0000 7f90bf771700 1 -- 172.21.15.52:0/3240994948 >> 172.21.15.52:0/3240994948 conn(0x7f90b8077170 msgr2=0x7f90b80fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:17.061 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.051+0000 7f90bf771700 1 -- 172.21.15.52:0/3240994948 shutdown_connections 2023-11-27T15:18:17.061 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:17.051+0000 7f90bf771700 1 -- 172.21.15.52:0/3240994948 wait complete. 2023-11-27T15:18:17.067 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:16 smithi052 ceph-mon[99056]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2023-11-27T15:18:17.067 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:16 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3987195199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:18:17.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:16 smithi150 ceph-mon[100559]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2023-11-27T15:18:17.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:16 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3987195199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:18:17.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:16 smithi102 ceph-mon[100296]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2023-11-27T15:18:17.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:16 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3987195199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:18:18.069 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-27T15:18:18.069 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-27T15:18:18.268 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.260+0000 7f0fceb46700 1 -- 172.21.15.52:0/1689425721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0fc80ffdf0 msgr2=0x7f0fc8103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:18.268 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.260+0000 7f0fceb46700 1 --2- 172.21.15.52:0/1689425721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0fc80ffdf0 0x7f0fc8103db0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0fb40099c0 tx=0x7f0fb4009cd0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.268 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.260+0000 7f0fceb46700 1 -- 172.21.15.52:0/1689425721 shutdown_connections 2023-11-27T15:18:18.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.260+0000 7f0fceb46700 1 --2- 172.21.15.52:0/1689425721 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fc8104490 0x7f0fc810c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.260+0000 7f0fceb46700 1 --2- 172.21.15.52:0/1689425721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fc80ff450 0x7f0fc80ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.260+0000 7f0fceb46700 1 --2- 172.21.15.52:0/1689425721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0fc80ffdf0 0x7f0fc8103db0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.260+0000 7f0fceb46700 1 -- 172.21.15.52:0/1689425721 >> 172.21.15.52:0/1689425721 conn(0x7f0fc80770b0 msgr2=0x7f0fc80774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:18.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.260+0000 7f0fceb46700 1 -- 172.21.15.52:0/1689425721 shutdown_connections 2023-11-27T15:18:18.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.260+0000 7f0fceb46700 1 -- 172.21.15.52:0/1689425721 wait complete. 2023-11-27T15:18:18.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.261+0000 7f0fceb46700 1 Processor -- start 2023-11-27T15:18:18.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.261+0000 7f0fceb46700 1 -- start start 2023-11-27T15:18:18.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.261+0000 7f0fceb46700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fc80ff450 0x7f0fc810cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.261+0000 7f0fceb46700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fc80ffdf0 0x7f0fc810d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.261+0000 7f0fceb46700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0fc8104490 0x7f0fc81a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.261+0000 7f0fceb46700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc810d930 con 0x7f0fc8104490 2023-11-27T15:18:18.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.261+0000 7f0fceb46700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc81a4ac0 con 0x7f0fc80ff450 2023-11-27T15:18:18.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.261+0000 7f0fceb46700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc81a4c00 con 0x7f0fc80ffdf0 2023-11-27T15:18:18.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.261+0000 7f0fcc8e2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fc80ff450 0x7f0fc810cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.261+0000 7f0fcc8e2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fc80ff450 0x7f0fc810cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:41994/0 (socket says 172.21.15.52:41994) 2023-11-27T15:18:18.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.261+0000 7f0fcc8e2700 1 -- 172.21.15.52:0/3093092453 learned_addr learned my addr 172.21.15.52:0/3093092453 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:18.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.261+0000 7f0fcd0e3700 1 --2- 172.21.15.52:0/3093092453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0fc8104490 0x7f0fc81a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.262+0000 7f0fc7fff700 1 --2- 172.21.15.52:0/3093092453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fc80ffdf0 0x7f0fc810d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.262+0000 7f0fcc8e2700 1 -- 172.21.15.52:0/3093092453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fc80ffdf0 msgr2=0x7f0fc810d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:18.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.262+0000 7f0fcc8e2700 1 --2- 172.21.15.52:0/3093092453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fc80ffdf0 0x7f0fc810d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.262+0000 7f0fcc8e2700 1 -- 172.21.15.52:0/3093092453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0fc8104490 msgr2=0x7f0fc81a4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:18.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.262+0000 7f0fcc8e2700 1 --2- 172.21.15.52:0/3093092453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0fc8104490 0x7f0fc81a4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.262+0000 7f0fcc8e2700 1 -- 172.21.15.52:0/3093092453 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb40095f0 con 0x7f0fc80ff450 2023-11-27T15:18:18.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.262+0000 7f0fcd0e3700 1 --2- 172.21.15.52:0/3093092453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0fc8104490 0x7f0fc81a4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:18:18.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.262+0000 7f0fcc8e2700 1 --2- 172.21.15.52:0/3093092453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fc80ff450 0x7f0fc810cbc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00d950 tx=0x7f0fbc00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:18.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.262+0000 7f0fc5ffb700 1 -- 172.21.15.52:0/3093092453 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc00f890 con 0x7f0fc80ff450 2023-11-27T15:18:18.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.262+0000 7f0fceb46700 1 -- 172.21.15.52:0/3093092453 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc8110320 con 0x7f0fc80ff450 2023-11-27T15:18:18.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.262+0000 7f0fc5ffb700 1 -- 172.21.15.52:0/3093092453 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fbc010460 con 0x7f0fc80ff450 2023-11-27T15:18:18.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.262+0000 7f0fc5ffb700 1 -- 172.21.15.52:0/3093092453 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc00e610 con 0x7f0fc80ff450 2023-11-27T15:18:18.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.262+0000 7f0fceb46700 1 -- 172.21.15.52:0/3093092453 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc8110190 con 0x7f0fc80ff450 2023-11-27T15:18:18.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.265+0000 7f0fc5ffb700 1 -- 172.21.15.52:0/3093092453 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f0fbc0108d0 con 0x7f0fc80ff450 2023-11-27T15:18:18.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.266+0000 7f0fc5ffb700 1 --2- 172.21.15.52:0/3093092453 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f0fb80a13e0 0x7f0fb80a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.266+0000 7f0fc7fff700 1 --2- 172.21.15.52:0/3093092453 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f0fb80a13e0 0x7f0fb80a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.266+0000 7f0fc5ffb700 1 -- 172.21.15.52:0/3093092453 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f0fbc0cd0f0 con 0x7f0fc80ff450 2023-11-27T15:18:18.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.267+0000 7f0fc7fff700 1 --2- 172.21.15.52:0/3093092453 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f0fb80a13e0 0x7f0fb80a3890 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4009f50 tx=0x7f0fb400b540 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:18.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.267+0000 7f0fdacd2b80 1 Processor -- start 2023-11-27T15:18:18.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.267+0000 7f0fdacd2b80 1 -- start start 2023-11-27T15:18:18.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.268+0000 7f0fdacd2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb786a2be0 0x55fb786a2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.268+0000 7f0fdacd2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb786a34f0 0x55fb786a73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.268+0000 7f0fdacd2b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb786a7930 0x55fb786a7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.268+0000 7f0fdacd2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb785a80e0 con 0x55fb786a34f0 2023-11-27T15:18:18.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.268+0000 7f0fdacd2b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb785fa920 con 0x55fb786a7930 2023-11-27T15:18:18.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.268+0000 7f0fdacd2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb785f3db0 con 0x55fb786a2be0 2023-11-27T15:18:18.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.268+0000 7f0fcd0e3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb786a7930 0x55fb786a7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.268+0000 7f0fcc8e2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb786a2be0 0x55fb786a2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.268+0000 7f0fcd0e3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb786a7930 0x55fb786a7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:42004/0 (socket says 172.21.15.52:42004) 2023-11-27T15:18:18.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.268+0000 7f0fcd0e3700 1 -- 172.21.15.52:0/843443428 learned_addr learned my addr 172.21.15.52:0/843443428 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:18.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.268+0000 7f0fc7fff700 1 --2- 172.21.15.52:0/843443428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb786a34f0 0x55fb786a73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.268+0000 7f0fcd0e3700 1 -- 172.21.15.52:0/843443428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb786a2be0 msgr2=0x55fb786a2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:18.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.268+0000 7f0fcd0e3700 1 --2- 172.21.15.52:0/843443428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb786a2be0 0x55fb786a2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.268+0000 7f0fcd0e3700 1 -- 172.21.15.52:0/843443428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb786a34f0 msgr2=0x55fb786a73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:18.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.268+0000 7f0fcd0e3700 1 --2- 172.21.15.52:0/843443428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb786a34f0 0x55fb786a73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.268+0000 7f0fcd0e3700 1 -- 172.21.15.52:0/843443428 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb784756b0 con 0x55fb786a7930 2023-11-27T15:18:18.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.268+0000 7f0fcc8e2700 1 --2- 172.21.15.52:0/843443428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb786a2be0 0x55fb786a2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:18:18.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.268+0000 7f0fc7fff700 1 --2- 172.21.15.52:0/843443428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb786a34f0 0x55fb786a73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:18:18.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.269+0000 7f0fcd0e3700 1 --2- 172.21.15.52:0/843443428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb786a7930 0x55fb786a7d80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000d710 tx=0x7f0fc000da20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=611636d56e7c8466 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:18.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.269+0000 7f0fb2ffd700 1 -- 172.21.15.52:0/843443428 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0004bb0 con 0x55fb786a7930 2023-11-27T15:18:18.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.269+0000 7f0fb2ffd700 1 -- 172.21.15.52:0/843443428 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fc0004d10 con 0x55fb786a7930 2023-11-27T15:18:18.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.269+0000 7f0fb2ffd700 1 -- 172.21.15.52:0/843443428 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc00052e0 con 0x55fb786a7930 2023-11-27T15:18:18.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.270+0000 7f0fdacd2b80 1 -- 172.21.15.52:0/843443428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb786a7930 msgr2=0x55fb786a7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:18.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.270+0000 7f0fdacd2b80 1 --2- 172.21.15.52:0/843443428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb786a7930 0x55fb786a7d80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000d710 tx=0x7f0fc000da20 comp rx=0 tx=0).stop 2023-11-27T15:18:18.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.270+0000 7f0fdacd2b80 1 -- 172.21.15.52:0/843443428 shutdown_connections 2023-11-27T15:18:18.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.270+0000 7f0fdacd2b80 1 --2- 172.21.15.52:0/843443428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb786a7930 0x55fb786a7d80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.270+0000 7f0fdacd2b80 1 --2- 172.21.15.52:0/843443428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb786a2be0 0x55fb786a2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.270+0000 7f0fdacd2b80 1 --2- 172.21.15.52:0/843443428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb786a34f0 0x55fb786a73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.270+0000 7f0fdacd2b80 1 -- 172.21.15.52:0/843443428 >> 172.21.15.52:0/843443428 conn(0x55fb786983e0 msgr2=0x55fb7869dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:18.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.270+0000 7f0fdacd2b80 1 -- 172.21.15.52:0/843443428 shutdown_connections 2023-11-27T15:18:18.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.270+0000 7f0fdacd2b80 1 -- 172.21.15.52:0/843443428 wait complete. 2023-11-27T15:18:18.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.271+0000 7f0fdacd2b80 1 -- start start 2023-11-27T15:18:18.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.271+0000 7f0fdacd2b80 1 Processor -- start 2023-11-27T15:18:18.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.271+0000 7f0fdacd2b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb786a2be0 0x55fb786af140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.271+0000 7f0fdacd2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb786a34f0 0x55fb786ac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.271+0000 7f0fdacd2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb786a7930 0x55fb786aca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.271+0000 7f0fdacd2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb78595960 con 0x55fb786a34f0 2023-11-27T15:18:18.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.271+0000 7f0fdacd2b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb78595600 con 0x55fb786a2be0 2023-11-27T15:18:18.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.271+0000 7f0fdacd2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb784779a0 con 0x55fb786a7930 2023-11-27T15:18:18.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.271+0000 7f0fdacd2b80 10 client.0 _ll_register_callbacks cb 0x55fb785b3800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:18:18.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.271+0000 7f0fcc8e2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb786a2be0 0x55fb786af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.271+0000 7f0fcc8e2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb786a2be0 0x55fb786af140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:42008/0 (socket says 172.21.15.52:42008) 2023-11-27T15:18:18.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.271+0000 7f0fcc8e2700 1 -- 172.21.15.52:0/3984795004 learned_addr learned my addr 172.21.15.52:0/3984795004 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:18.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.272+0000 7f0fcd0e3700 1 --2- 172.21.15.52:0/3984795004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb786a7930 0x55fb786aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.272+0000 7f0fcc8e2700 1 -- 172.21.15.52:0/3984795004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb786a7930 msgr2=0x55fb786aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:18.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.272+0000 7f0fcc8e2700 1 --2- 172.21.15.52:0/3984795004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb786a7930 0x55fb786aca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.272+0000 7f0fcc8e2700 1 -- 172.21.15.52:0/3984795004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb786a34f0 msgr2=0x55fb786ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:18.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.272+0000 7f0fcc8e2700 1 --2- 172.21.15.52:0/3984795004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb786a34f0 0x55fb786ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.272+0000 7f0fcc8e2700 1 -- 172.21.15.52:0/3984795004 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc000d340 con 0x55fb786a2be0 2023-11-27T15:18:18.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.272+0000 7f0fcd0e3700 1 --2- 172.21.15.52:0/3984795004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb786a7930 0x55fb786aca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:18:18.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.272+0000 7f0fcc8e2700 1 --2- 172.21.15.52:0/3984795004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb786a2be0 0x55fb786af140 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc096470 tx=0x7f0fbc09e050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ba83e41967e231a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:18.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.272+0000 7f0fa7fff700 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2023-11-27T15:18:18.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.272+0000 7f0fa7fff700 1 -- 172.21.15.52:0/3984795004 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc096800 con 0x55fb786a2be0 2023-11-27T15:18:18.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.272+0000 7f0fa7fff700 1 -- 172.21.15.52:0/3984795004 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f0fbc099460 con 0x55fb786a2be0 2023-11-27T15:18:18.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.272+0000 7f0fdacd2b80 1 -- 172.21.15.52:0/3984795004 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb78695ba0 con 0x55fb786a2be0 2023-11-27T15:18:18.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.272+0000 7f0fa7fff700 1 -- 172.21.15.52:0/3984795004 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fbc099620 con 0x55fb786a2be0 2023-11-27T15:18:18.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.273+0000 7f0fdacd2b80 10 client.25358 fetch_fsmap learned FSMap version 23 2023-11-27T15:18:18.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.273+0000 7f0fdacd2b80 1 -- 172.21.15.52:0/3984795004 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55fb785ba210 con 0x55fb786a2be0 2023-11-27T15:18:18.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.273+0000 7f0fa7fff700 1 -- 172.21.15.52:0/3984795004 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fbc0b8030 con 0x55fb786a2be0 2023-11-27T15:18:18.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.273+0000 7f0fa7fff700 1 -- 172.21.15.52:0/3984795004 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0fbc0ca9a0 con 0x55fb786a2be0 2023-11-27T15:18:18.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.273+0000 7f0fdacd2b80 10 client.25358 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:18:18.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.273+0000 7f0fdacd2b80 10 client.25358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:18:18.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.273+0000 7f0fdacd2b80 20 client.25358 populate_metadata read hostname 'smithi052' 2023-11-27T15:18:18.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.273+0000 7f0fdacd2b80 1 --2- 172.21.15.52:0/3984795004 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55fb786bc900 0x55fb786bed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.273+0000 7f0fdacd2b80 4 client.25358 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:18:18.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.273+0000 7f0fdacd2b80 1 -- 172.21.15.52:0/3984795004 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb785b6120 con 0x55fb786bc900 2023-11-27T15:18:18.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.273+0000 7f0fc7fff700 1 --2- 172.21.15.52:0/3984795004 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55fb786bc900 0x55fb786bed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.274+0000 7f0fc7fff700 1 --2- 172.21.15.52:0/3984795004 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55fb786bc900 0x55fb786bed10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8446d9f37e04ee1d server_cookie=178d28870a8b25d5 in_seq=0 out_seq=0 2023-11-27T15:18:18.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.274+0000 7f0fa7fff700 10 client.25358 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:18:18.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.274+0000 7f0fa7fff700 1 -- 172.21.15.52:0/3984795004 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fb785b6120 con 0x55fb786bc900 2023-11-27T15:18:18.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.274+0000 7f0fa7fff700 10 client.25358 handle_command_reply: tid=0 2023-11-27T15:18:18.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.274+0000 7f0fdacd2b80 1 client.25358 shutdown 2023-11-27T15:18:18.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.275+0000 7f0fdacd2b80 1 -- 172.21.15.52:0/3984795004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb786a2be0 msgr2=0x55fb786af140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:18.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.275+0000 7f0fdacd2b80 1 --2- 172.21.15.52:0/3984795004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb786a2be0 0x55fb786af140 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc096470 tx=0x7f0fbc09e050 comp rx=0 tx=0).stop 2023-11-27T15:18:18.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.275+0000 7f0fdacd2b80 1 -- 172.21.15.52:0/3984795004 shutdown_connections 2023-11-27T15:18:18.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.275+0000 7f0fdacd2b80 1 --2- 172.21.15.52:0/3984795004 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55fb786bc900 0x55fb786bed10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.275+0000 7f0fdacd2b80 1 --2- 172.21.15.52:0/3984795004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb786a2be0 0x55fb786af140 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.275+0000 7f0fdacd2b80 1 --2- 172.21.15.52:0/3984795004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb786a7930 0x55fb786aca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.275+0000 7f0fdacd2b80 1 --2- 172.21.15.52:0/3984795004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb786a34f0 0x55fb786ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.275+0000 7f0fdacd2b80 1 -- 172.21.15.52:0/3984795004 >> 172.21.15.52:0/3984795004 conn(0x55fb786983e0 msgr2=0x55fb7869ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:18.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.275+0000 7f0fa7fff700 0 client.25358 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:18:18.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.275+0000 7f0fdacd2b80 1 -- 172.21.15.52:0/3984795004 shutdown_connections 2023-11-27T15:18:18.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.275+0000 7f0fdacd2b80 1 -- 172.21.15.52:0/3984795004 wait complete. 2023-11-27T15:18:18.285 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.275+0000 7f0fdacd2b80 20 client.25358 trim_cache size 0 max 16384 2023-11-27T15:18:18.314 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.309+0000 7f0fdacd2b80 1 Processor -- start 2023-11-27T15:18:18.314 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.309+0000 7f0fdacd2b80 1 -- start start 2023-11-27T15:18:18.314 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.309+0000 7f0fdacd2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb786a2be0 0x55fb786a9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.314 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.309+0000 7f0fdacd2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb786a34f0 0x55fb786b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.309+0000 7f0fdacd2b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb786a7930 0x55fb786aec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.309+0000 7f0fdacd2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb786aa350 con 0x55fb786a34f0 2023-11-27T15:18:18.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.309+0000 7f0fdacd2b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb786a3980 con 0x55fb786a7930 2023-11-27T15:18:18.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.309+0000 7f0fdacd2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb786b3f60 con 0x55fb786a2be0 2023-11-27T15:18:18.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.309+0000 7f0fc7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb786a34f0 0x55fb786b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.310+0000 7f0fc7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb786a34f0 0x55fb786b3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52362/0 (socket says 172.21.15.52:52362) 2023-11-27T15:18:18.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.310+0000 7f0fc7fff700 1 -- 172.21.15.52:0/2791952530 learned_addr learned my addr 172.21.15.52:0/2791952530 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:18.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.310+0000 7f0fcc8e2700 1 --2- 172.21.15.52:0/2791952530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb786a2be0 0x55fb786a9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.310+0000 7f0fcd0e3700 1 --2- 172.21.15.52:0/2791952530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb786a7930 0x55fb786aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.310+0000 7f0fc7fff700 1 -- 172.21.15.52:0/2791952530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb786a2be0 msgr2=0x55fb786a9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:18.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.310+0000 7f0fc7fff700 1 --2- 172.21.15.52:0/2791952530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb786a2be0 0x55fb786a9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.310+0000 7f0fc7fff700 1 -- 172.21.15.52:0/2791952530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb786a7930 msgr2=0x55fb786aec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:18.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.310+0000 7f0fc7fff700 1 --2- 172.21.15.52:0/2791952530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb786a7930 0x55fb786aec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.310+0000 7f0fc7fff700 1 -- 172.21.15.52:0/2791952530 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb786a7fd0 con 0x55fb786a34f0 2023-11-27T15:18:18.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.310+0000 7f0fc7fff700 1 --2- 172.21.15.52:0/2791952530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb786a34f0 0x55fb786b3a20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4000c00 tx=0x7f0fb4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b73408ce9925d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:18.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.310+0000 7f0fb37fe700 1 -- 172.21.15.52:0/2791952530 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4033ea0 con 0x55fb786a34f0 2023-11-27T15:18:18.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.310+0000 7f0fb37fe700 1 -- 172.21.15.52:0/2791952530 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fb402a040 con 0x55fb786a34f0 2023-11-27T15:18:18.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.310+0000 7f0fb37fe700 1 -- 172.21.15.52:0/2791952530 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb40038c0 con 0x55fb786a34f0 2023-11-27T15:18:18.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.311+0000 7f0fdacd2b80 1 -- 172.21.15.52:0/2791952530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb786a34f0 msgr2=0x55fb786b3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:18.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.311+0000 7f0fdacd2b80 1 --2- 172.21.15.52:0/2791952530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb786a34f0 0x55fb786b3a20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4000c00 tx=0x7f0fb4020000 comp rx=0 tx=0).stop 2023-11-27T15:18:18.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.311+0000 7f0fdacd2b80 1 -- 172.21.15.52:0/2791952530 shutdown_connections 2023-11-27T15:18:18.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.311+0000 7f0fdacd2b80 1 --2- 172.21.15.52:0/2791952530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb786a7930 0x55fb786aec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.311+0000 7f0fdacd2b80 1 --2- 172.21.15.52:0/2791952530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb786a2be0 0x55fb786a9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.311+0000 7f0fdacd2b80 1 --2- 172.21.15.52:0/2791952530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb786a34f0 0x55fb786b3a20 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.312+0000 7f0fdacd2b80 1 -- 172.21.15.52:0/2791952530 >> 172.21.15.52:0/2791952530 conn(0x55fb786983e0 msgr2=0x55fb786a59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:18.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.312+0000 7f0fdacd2b80 1 -- 172.21.15.52:0/2791952530 shutdown_connections 2023-11-27T15:18:18.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.312+0000 7f0fdacd2b80 1 -- 172.21.15.52:0/2791952530 wait complete. 2023-11-27T15:18:18.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.312+0000 7f0fdacd2b80 1 -- start start 2023-11-27T15:18:18.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.312+0000 7f0fdacd2b80 1 Processor -- start 2023-11-27T15:18:18.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.313+0000 7f0fdacd2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb786a2be0 0x55fb786ac190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.313+0000 7f0fdacd2b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb786a34f0 0x55fb786aa990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.313+0000 7f0fdacd2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb786a7930 0x55fb786aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.313+0000 7f0fdacd2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb786ab5b0 con 0x55fb786a7930 2023-11-27T15:18:18.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.313+0000 7f0fdacd2b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb786ab720 con 0x55fb786a34f0 2023-11-27T15:18:18.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.313+0000 7f0fdacd2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb787107e0 con 0x55fb786a2be0 2023-11-27T15:18:18.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.313+0000 7f0fdacd2b80 10 client.0 _ll_register_callbacks cb 0x7f0fbc099980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:18:18.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.313+0000 7f0fcc8e2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb786a2be0 0x55fb786ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.313+0000 7f0fcc8e2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb786a2be0 0x55fb786ac190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:39008/0 (socket says 172.21.15.52:39008) 2023-11-27T15:18:18.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.313+0000 7f0fc7fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb786a34f0 0x55fb786aa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.313+0000 7f0fcc8e2700 1 -- 172.21.15.52:0/1491751499 learned_addr learned my addr 172.21.15.52:0/1491751499 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:18.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.313+0000 7f0fcc8e2700 1 -- 172.21.15.52:0/1491751499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb786a34f0 msgr2=0x55fb786aa990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:18.322 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.313+0000 7f0fcc8e2700 1 --2- 172.21.15.52:0/1491751499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb786a34f0 0x55fb786aa990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.322 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.313+0000 7f0fcc8e2700 1 -- 172.21.15.52:0/1491751499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb786a7930 msgr2=0x55fb786aaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:18.322 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.313+0000 7f0fcc8e2700 1 --2- 172.21.15.52:0/1491751499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb786a7930 0x55fb786aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.322 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.313+0000 7f0fcc8e2700 1 -- 172.21.15.52:0/1491751499 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc810c8a0 con 0x55fb786a2be0 2023-11-27T15:18:18.322 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.314+0000 7f0fcc8e2700 1 --2- 172.21.15.52:0/1491751499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb786a2be0 0x55fb786ac190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc0cad20 tx=0x7f0fbc09a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6faebd31b354bf9c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:18.322 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.314+0000 7f0fa7fff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2023-11-27T15:18:18.322 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.314+0000 7f0fa7fff700 1 -- 172.21.15.52:0/1491751499 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc0b83b0 con 0x55fb786a2be0 2023-11-27T15:18:18.322 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.314+0000 7f0fa7fff700 1 -- 172.21.15.52:0/1491751499 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f0fbc096800 con 0x55fb786a2be0 2023-11-27T15:18:18.322 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.314+0000 7f0fdacd2b80 1 -- 172.21.15.52:0/1491751499 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0fc0004d10 con 0x55fb786a2be0 2023-11-27T15:18:18.322 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.314+0000 7f0fa7fff700 1 -- 172.21.15.52:0/1491751499 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fbc0a23c0 con 0x55fb786a2be0 2023-11-27T15:18:18.322 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.314+0000 7f0fdacd2b80 10 client.25279 fetch_fsmap learned FSMap version 23 2023-11-27T15:18:18.322 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.314+0000 7f0fdacd2b80 1 -- 172.21.15.52:0/1491751499 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55fb78421e60 con 0x55fb786a2be0 2023-11-27T15:18:18.323 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.314+0000 7f0fa7fff700 1 -- 172.21.15.52:0/1491751499 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fbc063080 con 0x55fb786a2be0 2023-11-27T15:18:18.323 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.314+0000 7f0fa7fff700 1 -- 172.21.15.52:0/1491751499 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0fbc098fd0 con 0x55fb786a2be0 2023-11-27T15:18:18.323 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.314+0000 7f0fdacd2b80 10 client.25279 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:18:18.323 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.314+0000 7f0fdacd2b80 10 client.25279 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:18:18.323 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.314+0000 7f0fdacd2b80 20 client.25279 populate_metadata read hostname 'smithi052' 2023-11-27T15:18:18.323 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.314+0000 7f0fdacd2b80 1 --2- 172.21.15.52:0/1491751499 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55fb787119e0 0x55fb78713df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.323 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.314+0000 7f0fdacd2b80 4 client.25279 mds_command: new command op to 24536 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-27T15:18:18.323 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.314+0000 7f0fdacd2b80 1 -- 172.21.15.52:0/1491751499 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55fb785c26e0 con 0x55fb787119e0 2023-11-27T15:18:18.323 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.319+0000 7f0fc7fff700 1 --2- 172.21.15.52:0/1491751499 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55fb787119e0 0x55fb78713df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.324 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.319+0000 7f0fa7fff700 10 client.25279 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:18:18.324 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.319+0000 7f0fc7fff700 1 --2- 172.21.15.52:0/1491751499 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55fb787119e0 0x55fb78713df0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cf52e60b59c1eee server_cookie=288a8edd9f295169 in_seq=0 out_seq=0 2023-11-27T15:18:18.325 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.320+0000 7f0fa7fff700 1 -- 172.21.15.52:0/1491751499 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fb785c26e0 con 0x55fb787119e0 2023-11-27T15:18:18.325 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.320+0000 7f0fa7fff700 10 client.25279 handle_command_reply: tid=0 2023-11-27T15:18:18.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.320+0000 7f0fdacd2b80 1 client.25279 shutdown 2023-11-27T15:18:18.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.321+0000 7f0fdacd2b80 1 -- 172.21.15.52:0/1491751499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb786a2be0 msgr2=0x55fb786ac190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:18.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.321+0000 7f0fdacd2b80 1 --2- 172.21.15.52:0/1491751499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb786a2be0 0x55fb786ac190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc0cad20 tx=0x7f0fbc09a0f0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.329 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.324+0000 7f0fdacd2b80 1 -- 172.21.15.52:0/1491751499 shutdown_connections 2023-11-27T15:18:18.329 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.324+0000 7f0fdacd2b80 1 --2- 172.21.15.52:0/1491751499 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55fb787119e0 0x55fb78713df0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.329 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.324+0000 7f0fdacd2b80 1 --2- 172.21.15.52:0/1491751499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fb786a7930 0x55fb786aaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.329 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.324+0000 7f0fa7fff700 0 client.25279 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:18:18.329 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.324+0000 7f0fdacd2b80 1 --2- 172.21.15.52:0/1491751499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fb786a2be0 0x55fb786ac190 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.330 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.324+0000 7f0fdacd2b80 1 --2- 172.21.15.52:0/1491751499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fb786a34f0 0x55fb786aa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.330 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.324+0000 7f0fdacd2b80 1 -- 172.21.15.52:0/1491751499 >> 172.21.15.52:0/1491751499 conn(0x55fb786983e0 msgr2=0x55fb786a68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:18.330 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.324+0000 7f0fdacd2b80 1 -- 172.21.15.52:0/1491751499 shutdown_connections 2023-11-27T15:18:18.330 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.324+0000 7f0fdacd2b80 1 -- 172.21.15.52:0/1491751499 wait complete. 2023-11-27T15:18:18.330 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.324+0000 7f0fdacd2b80 20 client.25279 trim_cache size 0 max 16384 2023-11-27T15:18:18.330 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:18:18.330 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2023-11-27T15:18:18.330 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "d506e2e1-2357-4652-8afc-7708159185ae", 2023-11-27T15:18:18.330 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2023-11-27T15:18:18.331 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:18:18.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.325+0000 7f0fceb46700 1 -- 172.21.15.52:0/3093092453 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f0fb80a13e0 msgr2=0x7f0fb80a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:18.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.325+0000 7f0fceb46700 1 --2- 172.21.15.52:0/3093092453 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f0fb80a13e0 0x7f0fb80a3890 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4009f50 tx=0x7f0fb400b540 comp rx=0 tx=0).stop 2023-11-27T15:18:18.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.325+0000 7f0fceb46700 1 -- 172.21.15.52:0/3093092453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fc80ff450 msgr2=0x7f0fc810cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:18.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.325+0000 7f0fceb46700 1 --2- 172.21.15.52:0/3093092453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fc80ff450 0x7f0fc810cbc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00d950 tx=0x7f0fbc00dc60 comp rx=0 tx=0).stop 2023-11-27T15:18:18.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.325+0000 7f0fceb46700 1 -- 172.21.15.52:0/3093092453 shutdown_connections 2023-11-27T15:18:18.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.325+0000 7f0fceb46700 1 --2- 172.21.15.52:0/3093092453 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f0fb80a13e0 0x7f0fb80a3890 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.325+0000 7f0fceb46700 1 --2- 172.21.15.52:0/3093092453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0fc80ff450 0x7f0fc810cbc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.325+0000 7f0fceb46700 1 --2- 172.21.15.52:0/3093092453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0fc8104490 0x7f0fc81a4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.325+0000 7f0fceb46700 1 --2- 172.21.15.52:0/3093092453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fc80ffdf0 0x7f0fc810d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.325+0000 7f0fceb46700 1 -- 172.21.15.52:0/3093092453 >> 172.21.15.52:0/3093092453 conn(0x7f0fc80770b0 msgr2=0x7f0fc80fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:18.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.325+0000 7f0fceb46700 1 -- 172.21.15.52:0/3093092453 shutdown_connections 2023-11-27T15:18:18.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.325+0000 7f0fceb46700 1 -- 172.21.15.52:0/3093092453 wait complete. 2023-11-27T15:18:18.347 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:18:18.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.475+0000 7f3e78d01700 1 -- 172.21.15.52:0/269852277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e74101140 msgr2=0x7f3e74101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:18.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.475+0000 7f3e78d01700 1 --2- 172.21.15.52:0/269852277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e74101140 0x7f3e74101510 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3e640098f0 tx=0x7f3e64009c00 comp rx=0 tx=0).stop 2023-11-27T15:18:18.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.475+0000 7f3e78d01700 1 -- 172.21.15.52:0/269852277 shutdown_connections 2023-11-27T15:18:18.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.475+0000 7f3e78d01700 1 --2- 172.21.15.52:0/269852277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e740ff310 0x7f3e7410f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.475+0000 7f3e78d01700 1 --2- 172.21.15.52:0/269852277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e74101140 0x7f3e74101510 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.475+0000 7f3e78d01700 1 --2- 172.21.15.52:0/269852277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e740fe880 0x7f3e740fecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.475+0000 7f3e78d01700 1 -- 172.21.15.52:0/269852277 >> 172.21.15.52:0/269852277 conn(0x7f3e74075920 msgr2=0x7f3e74075d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:18.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.475+0000 7f3e78d01700 1 -- 172.21.15.52:0/269852277 shutdown_connections 2023-11-27T15:18:18.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.475+0000 7f3e78d01700 1 -- 172.21.15.52:0/269852277 wait complete. 2023-11-27T15:18:18.483 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.476+0000 7f3e78d01700 1 Processor -- start 2023-11-27T15:18:18.483 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.476+0000 7f3e78d01700 1 -- start start 2023-11-27T15:18:18.483 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.477+0000 7f3e78d01700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e740fe880 0x7f3e741985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.483 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.477+0000 7f3e78d01700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e740ff310 0x7f3e74198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.483 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.477+0000 7f3e78d01700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e74101140 0x7f3e7419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.483 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.477+0000 7f3e78d01700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7419d580 con 0x7f3e740ff310 2023-11-27T15:18:18.483 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.477+0000 7f3e78d01700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7419d6f0 con 0x7f3e740fe880 2023-11-27T15:18:18.483 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.477+0000 7f3e78d01700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7419d8b0 con 0x7f3e74101140 2023-11-27T15:18:18.483 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.477+0000 7f3e71d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e740ff310 0x7f3e74198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.484 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.477+0000 7f3e71d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e740ff310 0x7f3e74198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52382/0 (socket says 172.21.15.52:52382) 2023-11-27T15:18:18.484 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.477+0000 7f3e71d9b700 1 -- 172.21.15.52:0/802232294 learned_addr learned my addr 172.21.15.52:0/802232294 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:18.484 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.477+0000 7f3e7259c700 1 --2- 172.21.15.52:0/802232294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e740fe880 0x7f3e741985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.484 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.477+0000 7f3e72d9d700 1 --2- 172.21.15.52:0/802232294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e74101140 0x7f3e7419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.484 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.477+0000 7f3e71d9b700 1 -- 172.21.15.52:0/802232294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e74101140 msgr2=0x7f3e7419cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:18.484 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.477+0000 7f3e71d9b700 1 --2- 172.21.15.52:0/802232294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e74101140 0x7f3e7419cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.484 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.477+0000 7f3e71d9b700 1 -- 172.21.15.52:0/802232294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e740fe880 msgr2=0x7f3e741985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:18.484 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.477+0000 7f3e71d9b700 1 --2- 172.21.15.52:0/802232294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e740fe880 0x7f3e741985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.484 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.477+0000 7f3e71d9b700 1 -- 172.21.15.52:0/802232294 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e680095f0 con 0x7f3e740ff310 2023-11-27T15:18:18.484 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.477+0000 7f3e72d9d700 1 --2- 172.21.15.52:0/802232294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e74101140 0x7f3e7419cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:18:18.485 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.477+0000 7f3e71d9b700 1 --2- 172.21.15.52:0/802232294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e740ff310 0x7f3e74198ae0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3e6800ec10 tx=0x7f3e6800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:18.485 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.478+0000 7f3e637fe700 1 -- 172.21.15.52:0/802232294 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6800cd40 con 0x7f3e740ff310 2023-11-27T15:18:18.485 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.478+0000 7f3e78d01700 1 -- 172.21.15.52:0/802232294 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e64009520 con 0x7f3e740ff310 2023-11-27T15:18:18.485 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.478+0000 7f3e637fe700 1 -- 172.21.15.52:0/802232294 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e680044e0 con 0x7f3e740ff310 2023-11-27T15:18:18.485 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.478+0000 7f3e637fe700 1 -- 172.21.15.52:0/802232294 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e68010690 con 0x7f3e740ff310 2023-11-27T15:18:18.485 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.478+0000 7f3e78d01700 1 -- 172.21.15.52:0/802232294 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e741124f0 con 0x7f3e740ff310 2023-11-27T15:18:18.488 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.481+0000 7f3e637fe700 1 -- 172.21.15.52:0/802232294 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f3e68003710 con 0x7f3e740ff310 2023-11-27T15:18:18.488 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.481+0000 7f3e637fe700 1 --2- 172.21.15.52:0/802232294 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f3e5c0a13e0 0x7f3e5c0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.488 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.482+0000 7f3e637fe700 1 -- 172.21.15.52:0/802232294 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f3e68014070 con 0x7f3e740ff310 2023-11-27T15:18:18.488 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.482+0000 7f3e7259c700 1 --2- 172.21.15.52:0/802232294 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f3e5c0a13e0 0x7f3e5c0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.488 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.482+0000 7f3e7259c700 1 --2- 172.21.15.52:0/802232294 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f3e5c0a13e0 0x7f3e5c0a3890 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3e64009fd0 tx=0x7f3e64019040 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:18.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.482+0000 7f3e84e8db80 1 Processor -- start 2023-11-27T15:18:18.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.483+0000 7f3e84e8db80 1 -- start start 2023-11-27T15:18:18.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.483+0000 7f3e84e8db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d46029f370 0x55d46029f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.483+0000 7f3e84e8db80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d46029fc80 0x55d4602a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.483+0000 7f3e84e8db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4602a40c0 0x55d4602a4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.483+0000 7f3e84e8db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46019c060 con 0x55d4602a40c0 2023-11-27T15:18:18.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.483+0000 7f3e84e8db80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4601ee8a0 con 0x55d46029fc80 2023-11-27T15:18:18.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.483+0000 7f3e84e8db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4601e7d30 con 0x55d46029f370 2023-11-27T15:18:18.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.483+0000 7f3e72d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4602a40c0 0x55d4602a4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.483+0000 7f3e7259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d46029f370 0x55d46029f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.483+0000 7f3e72d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4602a40c0 0x55d4602a4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52390/0 (socket says 172.21.15.52:52390) 2023-11-27T15:18:18.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.483+0000 7f3e72d9d700 1 -- 172.21.15.52:0/535849461 learned_addr learned my addr 172.21.15.52:0/535849461 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:18.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.484+0000 7f3e71d9b700 1 --2- 172.21.15.52:0/535849461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d46029fc80 0x55d4602a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.484+0000 7f3e7259c700 1 -- 172.21.15.52:0/535849461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d46029fc80 msgr2=0x55d4602a3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:18.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.484+0000 7f3e7259c700 1 --2- 172.21.15.52:0/535849461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d46029fc80 0x55d4602a3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.484+0000 7f3e7259c700 1 -- 172.21.15.52:0/535849461 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4602a40c0 msgr2=0x55d4602a4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:18.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.484+0000 7f3e7259c700 1 --2- 172.21.15.52:0/535849461 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4602a40c0 0x55d4602a4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.484+0000 7f3e7259c700 1 -- 172.21.15.52:0/535849461 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d460069630 con 0x55d46029f370 2023-11-27T15:18:18.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.484+0000 7f3e72d9d700 1 --2- 172.21.15.52:0/535849461 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4602a40c0 0x55d4602a4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:18:18.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.484+0000 7f3e71d9b700 1 --2- 172.21.15.52:0/535849461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d46029fc80 0x55d4602a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:18:18.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.484+0000 7f3e7259c700 1 --2- 172.21.15.52:0/535849461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d46029f370 0x55d46029f740 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3e64009810 tx=0x7f3e6401e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff4b0441aee0d915 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:18.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.484+0000 7f3e60ff9700 1 -- 172.21.15.52:0/535849461 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6400b560 con 0x55d46029f370 2023-11-27T15:18:18.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.484+0000 7f3e60ff9700 1 -- 172.21.15.52:0/535849461 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e6401d040 con 0x55d46029f370 2023-11-27T15:18:18.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.484+0000 7f3e60ff9700 1 -- 172.21.15.52:0/535849461 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6402f6e0 con 0x55d46029f370 2023-11-27T15:18:18.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.485+0000 7f3e84e8db80 1 -- 172.21.15.52:0/535849461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d46029f370 msgr2=0x55d46029f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:18.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.485+0000 7f3e84e8db80 1 --2- 172.21.15.52:0/535849461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d46029f370 0x55d46029f740 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3e64009810 tx=0x7f3e6401e000 comp rx=0 tx=0).stop 2023-11-27T15:18:18.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.485+0000 7f3e84e8db80 1 -- 172.21.15.52:0/535849461 shutdown_connections 2023-11-27T15:18:18.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.485+0000 7f3e84e8db80 1 --2- 172.21.15.52:0/535849461 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4602a40c0 0x55d4602a4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.485+0000 7f3e84e8db80 1 --2- 172.21.15.52:0/535849461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d46029f370 0x55d46029f740 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.485+0000 7f3e84e8db80 1 --2- 172.21.15.52:0/535849461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d46029fc80 0x55d4602a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.485+0000 7f3e84e8db80 1 -- 172.21.15.52:0/535849461 >> 172.21.15.52:0/535849461 conn(0x55d46028c360 msgr2=0x55d460291b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:18.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.485+0000 7f3e84e8db80 1 -- 172.21.15.52:0/535849461 shutdown_connections 2023-11-27T15:18:18.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.485+0000 7f3e84e8db80 1 -- 172.21.15.52:0/535849461 wait complete. 2023-11-27T15:18:18.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.486+0000 7f3e84e8db80 1 -- start start 2023-11-27T15:18:18.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.486+0000 7f3e84e8db80 1 Processor -- start 2023-11-27T15:18:18.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.486+0000 7f3e84e8db80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d46029f370 0x55d4602ab820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.486+0000 7f3e84e8db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d46029fc80 0x55d4602a8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.486+0000 7f3e84e8db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d4602a40c0 0x55d4602a9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.486+0000 7f3e84e8db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4601898e0 con 0x55d46029fc80 2023-11-27T15:18:18.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.486+0000 7f3e84e8db80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d460189580 con 0x55d46029f370 2023-11-27T15:18:18.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.486+0000 7f3e84e8db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46006b920 con 0x55d4602a40c0 2023-11-27T15:18:18.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.486+0000 7f3e84e8db80 10 client.0 _ll_register_callbacks cb 0x55d4601a7780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:18:18.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.487+0000 7f3e71d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d46029fc80 0x55d4602a8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.487+0000 7f3e71d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d46029fc80 0x55d4602a8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52394/0 (socket says 172.21.15.52:52394) 2023-11-27T15:18:18.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.487+0000 7f3e71d9b700 1 -- 172.21.15.52:0/628912192 learned_addr learned my addr 172.21.15.52:0/628912192 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:18.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.487+0000 7f3e7259c700 1 --2- 172.21.15.52:0/628912192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d46029f370 0x55d4602ab820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.487+0000 7f3e71d9b700 1 -- 172.21.15.52:0/628912192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d4602a40c0 msgr2=0x55d4602a9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:18.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.487+0000 7f3e72d9d700 1 --2- 172.21.15.52:0/628912192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d4602a40c0 0x55d4602a9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.487+0000 7f3e71d9b700 1 --2- 172.21.15.52:0/628912192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d4602a40c0 0x55d4602a9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.487+0000 7f3e71d9b700 1 -- 172.21.15.52:0/628912192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d46029f370 msgr2=0x55d4602ab820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:18.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.487+0000 7f3e71d9b700 1 --2- 172.21.15.52:0/628912192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d46029f370 0x55d4602ab820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.487+0000 7f3e71d9b700 1 -- 172.21.15.52:0/628912192 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e7410fb60 con 0x55d46029fc80 2023-11-27T15:18:18.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.487+0000 7f3e72d9d700 1 --2- 172.21.15.52:0/628912192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d4602a40c0 0x55d4602a9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:18:18.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.487+0000 7f3e71d9b700 1 --2- 172.21.15.52:0/628912192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d46029fc80 0x55d4602a8aa0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3e6809a360 tx=0x7f3e68099db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2b3cd584a1973b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:18.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.487+0000 7f3e59ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:18:18.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.487+0000 7f3e59ffb700 1 -- 172.21.15.52:0/628912192 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6809a3d0 con 0x55d46029fc80 2023-11-27T15:18:18.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.488+0000 7f3e59ffb700 1 -- 172.21.15.52:0/628912192 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f3e6809e400 con 0x55d46029fc80 2023-11-27T15:18:18.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.488+0000 7f3e84e8db80 1 -- 172.21.15.52:0/628912192 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d460289b20 con 0x55d46029fc80 2023-11-27T15:18:18.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.488+0000 7f3e59ffb700 1 -- 172.21.15.52:0/628912192 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e680a2820 con 0x55d46029fc80 2023-11-27T15:18:18.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.488+0000 7f3e84e8db80 10 client.15561 fetch_fsmap learned FSMap version 23 2023-11-27T15:18:18.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.488+0000 7f3e84e8db80 1 -- 172.21.15.52:0/628912192 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d4601ae190 con 0x55d46029fc80 2023-11-27T15:18:18.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.488+0000 7f3e59ffb700 1 -- 172.21.15.52:0/628912192 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e680c8070 con 0x55d46029fc80 2023-11-27T15:18:18.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.488+0000 7f3e59ffb700 1 -- 172.21.15.52:0/628912192 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3e680a2390 con 0x55d46029fc80 2023-11-27T15:18:18.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.488+0000 7f3e84e8db80 10 client.15561 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:18:18.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.488+0000 7f3e84e8db80 10 client.15561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:18:18.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.488+0000 7f3e84e8db80 20 client.15561 populate_metadata read hostname 'smithi052' 2023-11-27T15:18:18.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.488+0000 7f3e84e8db80 1 --2- 172.21.15.52:0/628912192 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d4602b8fc0 0x55d4602bb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.488+0000 7f3e84e8db80 4 client.15561 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:18:18.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.488+0000 7f3e84e8db80 1 -- 172.21.15.52:0/628912192 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d4601aa0a0 con 0x55d4602b8fc0 2023-11-27T15:18:18.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.488+0000 7f3e7259c700 1 --2- 172.21.15.52:0/628912192 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d4602b8fc0 0x55d4602bb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.489+0000 7f3e7259c700 1 --2- 172.21.15.52:0/628912192 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d4602b8fc0 0x55d4602bb3d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7d34fa544d98017 server_cookie=e5203091b42c3bb8 in_seq=0 out_seq=0 2023-11-27T15:18:18.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.489+0000 7f3e59ffb700 10 client.15561 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:18:18.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.489+0000 7f3e59ffb700 1 -- 172.21.15.52:0/628912192 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d4601aa0a0 con 0x55d4602b8fc0 2023-11-27T15:18:18.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.489+0000 7f3e59ffb700 10 client.15561 handle_command_reply: tid=0 2023-11-27T15:18:18.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.489+0000 7f3e84e8db80 1 client.15561 shutdown 2023-11-27T15:18:18.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.490+0000 7f3e84e8db80 1 -- 172.21.15.52:0/628912192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d46029fc80 msgr2=0x55d4602a8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:18.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.490+0000 7f3e84e8db80 1 --2- 172.21.15.52:0/628912192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d46029fc80 0x55d4602a8aa0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3e6809a360 tx=0x7f3e68099db0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.490+0000 7f3e84e8db80 1 -- 172.21.15.52:0/628912192 shutdown_connections 2023-11-27T15:18:18.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.490+0000 7f3e84e8db80 1 --2- 172.21.15.52:0/628912192 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d4602b8fc0 0x55d4602bb3d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.490+0000 7f3e84e8db80 1 --2- 172.21.15.52:0/628912192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d46029f370 0x55d4602ab820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.490+0000 7f3e84e8db80 1 --2- 172.21.15.52:0/628912192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d4602a40c0 0x55d4602a9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.490+0000 7f3e84e8db80 1 --2- 172.21.15.52:0/628912192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d46029fc80 0x55d4602a8aa0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.490+0000 7f3e84e8db80 1 -- 172.21.15.52:0/628912192 >> 172.21.15.52:0/628912192 conn(0x55d46028c360 msgr2=0x55d4602a5570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:18.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.490+0000 7f3e59ffb700 0 client.15561 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:18:18.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.490+0000 7f3e84e8db80 1 -- 172.21.15.52:0/628912192 shutdown_connections 2023-11-27T15:18:18.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.490+0000 7f3e84e8db80 1 -- 172.21.15.52:0/628912192 wait complete. 2023-11-27T15:18:18.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.490+0000 7f3e84e8db80 20 client.15561 trim_cache size 0 max 16384 2023-11-27T15:18:18.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.507+0000 7f3e84e8db80 1 Processor -- start 2023-11-27T15:18:18.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.507+0000 7f3e84e8db80 1 -- start start 2023-11-27T15:18:18.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.507+0000 7f3e84e8db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d46029f370 0x55d4602a8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.507+0000 7f3e84e8db80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d46029fc80 0x55d4602b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.507+0000 7f3e84e8db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4602a40c0 0x55d4602bafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.507+0000 7f3e84e8db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d460293240 con 0x55d4602a40c0 2023-11-27T15:18:18.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.507+0000 7f3e84e8db80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4602a03d0 con 0x55d46029fc80 2023-11-27T15:18:18.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.507+0000 7f3e84e8db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d460293440 con 0x55d46029f370 2023-11-27T15:18:18.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.507+0000 7f3e7259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d46029f370 0x55d4602a8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.508+0000 7f3e7259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d46029f370 0x55d4602a8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:39044/0 (socket says 172.21.15.52:39044) 2023-11-27T15:18:18.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.508+0000 7f3e7259c700 1 -- 172.21.15.52:0/950982144 learned_addr learned my addr 172.21.15.52:0/950982144 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:18.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.508+0000 7f3e71d9b700 1 --2- 172.21.15.52:0/950982144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d46029fc80 0x55d4602b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.508+0000 7f3e7259c700 1 -- 172.21.15.52:0/950982144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d46029fc80 msgr2=0x55d4602b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:18.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.508+0000 7f3e7259c700 1 --2- 172.21.15.52:0/950982144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d46029fc80 0x55d4602b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.508+0000 7f3e72d9d700 1 --2- 172.21.15.52:0/950982144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4602a40c0 0x55d4602bafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.508+0000 7f3e7259c700 1 -- 172.21.15.52:0/950982144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4602a40c0 msgr2=0x55d4602bafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:18.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.508+0000 7f3e7259c700 1 --2- 172.21.15.52:0/950982144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4602a40c0 0x55d4602bafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.508+0000 7f3e7259c700 1 -- 172.21.15.52:0/950982144 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4602a4760 con 0x55d46029f370 2023-11-27T15:18:18.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.508+0000 7f3e72d9d700 1 --2- 172.21.15.52:0/950982144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4602a40c0 0x55d4602bafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:18:18.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.508+0000 7f3e7259c700 1 --2- 172.21.15.52:0/950982144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d46029f370 0x55d4602a8260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3e64009c40 tx=0x7f3e6400fa50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81e06795dfe1c27a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:18.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.508+0000 7f3e617fa700 1 -- 172.21.15.52:0/950982144 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e64034040 con 0x55d46029f370 2023-11-27T15:18:18.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.508+0000 7f3e617fa700 1 -- 172.21.15.52:0/950982144 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e64038da0 con 0x55d46029f370 2023-11-27T15:18:18.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.508+0000 7f3e617fa700 1 -- 172.21.15.52:0/950982144 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e64038780 con 0x55d46029f370 2023-11-27T15:18:18.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.509+0000 7f3e84e8db80 1 -- 172.21.15.52:0/950982144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d46029f370 msgr2=0x55d4602a8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:18.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.509+0000 7f3e84e8db80 1 --2- 172.21.15.52:0/950982144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d46029f370 0x55d4602a8260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3e64009c40 tx=0x7f3e6400fa50 comp rx=0 tx=0).stop 2023-11-27T15:18:18.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.509+0000 7f3e84e8db80 1 -- 172.21.15.52:0/950982144 shutdown_connections 2023-11-27T15:18:18.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.509+0000 7f3e84e8db80 1 --2- 172.21.15.52:0/950982144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4602a40c0 0x55d4602bafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.509+0000 7f3e84e8db80 1 --2- 172.21.15.52:0/950982144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d46029f370 0x55d4602a8260 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.509+0000 7f3e84e8db80 1 --2- 172.21.15.52:0/950982144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d46029fc80 0x55d4602b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.509+0000 7f3e84e8db80 1 -- 172.21.15.52:0/950982144 >> 172.21.15.52:0/950982144 conn(0x55d46028c360 msgr2=0x55d460290550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:18.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.509+0000 7f3e84e8db80 1 -- 172.21.15.52:0/950982144 shutdown_connections 2023-11-27T15:18:18.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.509+0000 7f3e84e8db80 1 -- 172.21.15.52:0/950982144 wait complete. 2023-11-27T15:18:18.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.510+0000 7f3e84e8db80 1 -- start start 2023-11-27T15:18:18.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.510+0000 7f3e84e8db80 1 Processor -- start 2023-11-27T15:18:18.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.510+0000 7f3e84e8db80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d46029f370 0x55d4602b0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.510+0000 7f3e84e8db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d46029fc80 0x55d4602ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.510+0000 7f3e84e8db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4602a40c0 0x55d4602ae710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.510+0000 7f3e84e8db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4602aee20 con 0x55d4602a40c0 2023-11-27T15:18:18.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.510+0000 7f3e84e8db80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4602aef90 con 0x55d46029f370 2023-11-27T15:18:18.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.510+0000 7f3e84e8db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46030be80 con 0x55d46029fc80 2023-11-27T15:18:18.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.510+0000 7f3e84e8db80 10 client.0 _ll_register_callbacks cb 0x7f3e6809a270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:18:18.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.510+0000 7f3e7259c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d46029f370 0x55d4602b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.510+0000 7f3e7259c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d46029f370 0x55d4602b0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:42064/0 (socket says 172.21.15.52:42064) 2023-11-27T15:18:18.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.510+0000 7f3e7259c700 1 -- 172.21.15.52:0/3785563269 learned_addr learned my addr 172.21.15.52:0/3785563269 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:18.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.510+0000 7f3e72d9d700 1 --2- 172.21.15.52:0/3785563269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4602a40c0 0x55d4602ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.511+0000 7f3e71d9b700 1 --2- 172.21.15.52:0/3785563269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d46029fc80 0x55d4602ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.511+0000 7f3e7259c700 1 -- 172.21.15.52:0/3785563269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d46029fc80 msgr2=0x55d4602ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:18.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.511+0000 7f3e7259c700 1 --2- 172.21.15.52:0/3785563269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d46029fc80 0x55d4602ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.511+0000 7f3e7259c700 1 -- 172.21.15.52:0/3785563269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4602a40c0 msgr2=0x55d4602ae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:18.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.511+0000 7f3e7259c700 1 --2- 172.21.15.52:0/3785563269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4602a40c0 0x55d4602ae710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.511+0000 7f3e7259c700 1 -- 172.21.15.52:0/3785563269 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d460069630 con 0x55d46029f370 2023-11-27T15:18:18.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.511+0000 7f3e7259c700 1 --2- 172.21.15.52:0/3785563269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d46029f370 0x55d4602b0380 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3e640057e0 tx=0x7f3e640050d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=958f44af0b91cc8a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:18.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.511+0000 7f3e59ffb700 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2023-11-27T15:18:18.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.511+0000 7f3e59ffb700 1 -- 172.21.15.52:0/3785563269 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e64034070 con 0x55d46029f370 2023-11-27T15:18:18.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.511+0000 7f3e84e8db80 1 -- 172.21.15.52:0/3785563269 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e6401d040 con 0x55d46029f370 2023-11-27T15:18:18.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.511+0000 7f3e59ffb700 1 -- 172.21.15.52:0/3785563269 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f3e6400b760 con 0x55d46029f370 2023-11-27T15:18:18.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.511+0000 7f3e59ffb700 1 -- 172.21.15.52:0/3785563269 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e6400b920 con 0x55d46029f370 2023-11-27T15:18:18.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.511+0000 7f3e84e8db80 10 client.25379 fetch_fsmap learned FSMap version 23 2023-11-27T15:18:18.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.511+0000 7f3e84e8db80 1 -- 172.21.15.52:0/3785563269 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d460015de0 con 0x55d46029f370 2023-11-27T15:18:18.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.512+0000 7f3e59ffb700 1 -- 172.21.15.52:0/3785563269 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e6401d040 con 0x55d46029f370 2023-11-27T15:18:18.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.512+0000 7f3e59ffb700 1 -- 172.21.15.52:0/3785563269 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3e6403f020 con 0x55d46029f370 2023-11-27T15:18:18.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.512+0000 7f3e84e8db80 10 client.25379 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:18:18.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.512+0000 7f3e84e8db80 10 client.25379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:18:18.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.512+0000 7f3e84e8db80 20 client.25379 populate_metadata read hostname 'smithi052' 2023-11-27T15:18:18.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.512+0000 7f3e84e8db80 1 --2- 172.21.15.52:0/3785563269 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d46030d0a0 0x55d46030f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.512+0000 7f3e84e8db80 4 client.25379 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:18:18.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.512+0000 7f3e84e8db80 1 -- 172.21.15.52:0/3785563269 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d4601b6660 con 0x55d46030d0a0 2023-11-27T15:18:18.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.512+0000 7f3e71d9b700 1 --2- 172.21.15.52:0/3785563269 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d46030d0a0 0x55d46030f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.512+0000 7f3e71d9b700 1 --2- 172.21.15.52:0/3785563269 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d46030d0a0 0x55d46030f4b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed868d18831d1f0 server_cookie=e20091d79ecf28c0 in_seq=0 out_seq=0 2023-11-27T15:18:18.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.512+0000 7f3e59ffb700 10 client.25379 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:18:18.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.513+0000 7f3e59ffb700 1 -- 172.21.15.52:0/3785563269 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d4601b6660 con 0x55d46030d0a0 2023-11-27T15:18:18.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.513+0000 7f3e59ffb700 10 client.25379 handle_command_reply: tid=0 2023-11-27T15:18:18.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.513+0000 7f3e84e8db80 1 client.25379 shutdown 2023-11-27T15:18:18.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.513+0000 7f3e84e8db80 1 -- 172.21.15.52:0/3785563269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d46029f370 msgr2=0x55d4602b0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:18.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.513+0000 7f3e84e8db80 1 --2- 172.21.15.52:0/3785563269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d46029f370 0x55d4602b0380 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3e640057e0 tx=0x7f3e640050d0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.513+0000 7f3e84e8db80 1 -- 172.21.15.52:0/3785563269 shutdown_connections 2023-11-27T15:18:18.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.513+0000 7f3e84e8db80 1 --2- 172.21.15.52:0/3785563269 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d46030d0a0 0x55d46030f4b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.513+0000 7f3e84e8db80 1 --2- 172.21.15.52:0/3785563269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d46029f370 0x55d4602b0380 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.513+0000 7f3e59ffb700 0 client.25379 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:18:18.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.513+0000 7f3e84e8db80 1 --2- 172.21.15.52:0/3785563269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4602a40c0 0x55d4602ae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.513+0000 7f3e84e8db80 1 --2- 172.21.15.52:0/3785563269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d46029fc80 0x55d4602ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.513+0000 7f3e84e8db80 1 -- 172.21.15.52:0/3785563269 >> 172.21.15.52:0/3785563269 conn(0x55d46028c360 msgr2=0x55d4602bafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:18.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.513+0000 7f3e84e8db80 1 -- 172.21.15.52:0/3785563269 shutdown_connections 2023-11-27T15:18:18.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.513+0000 7f3e84e8db80 1 -- 172.21.15.52:0/3785563269 wait complete. 2023-11-27T15:18:18.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.513+0000 7f3e84e8db80 20 client.25379 trim_cache size 0 max 16384 2023-11-27T15:18:18.524 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:18:18.524 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-27T15:18:18.524 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2023-11-27T15:18:18.524 INFO:teuthology.orchestra.run.smithi052.stdout: "d506e2e1-2357-4652-8afc-7708159185ae": { 2023-11-27T15:18:18.524 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2023-11-27T15:18:18.524 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "d506e2e1-2357-4652-8afc-7708159185ae", 2023-11-27T15:18:18.524 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2023-11-27T15:18:18.525 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:18:18.525 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:18:18.525 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:18:18.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.514+0000 7f3e78d01700 1 -- 172.21.15.52:0/802232294 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f3e5c0a13e0 msgr2=0x7f3e5c0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:18.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.514+0000 7f3e78d01700 1 --2- 172.21.15.52:0/802232294 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f3e5c0a13e0 0x7f3e5c0a3890 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3e64009fd0 tx=0x7f3e64019040 comp rx=0 tx=0).stop 2023-11-27T15:18:18.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.514+0000 7f3e78d01700 1 -- 172.21.15.52:0/802232294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e740ff310 msgr2=0x7f3e74198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:18.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.514+0000 7f3e78d01700 1 --2- 172.21.15.52:0/802232294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e740ff310 0x7f3e74198ae0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3e6800ec10 tx=0x7f3e6800ef20 comp rx=0 tx=0).stop 2023-11-27T15:18:18.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.514+0000 7f3e78d01700 1 -- 172.21.15.52:0/802232294 shutdown_connections 2023-11-27T15:18:18.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.514+0000 7f3e78d01700 1 --2- 172.21.15.52:0/802232294 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f3e5c0a13e0 0x7f3e5c0a3890 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.514+0000 7f3e78d01700 1 --2- 172.21.15.52:0/802232294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e740fe880 0x7f3e741985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.514+0000 7f3e78d01700 1 --2- 172.21.15.52:0/802232294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e74101140 0x7f3e7419cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.514+0000 7f3e78d01700 1 --2- 172.21.15.52:0/802232294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e740ff310 0x7f3e74198ae0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.514+0000 7f3e78d01700 1 -- 172.21.15.52:0/802232294 >> 172.21.15.52:0/802232294 conn(0x7f3e74075920 msgr2=0x7f3e740fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:18.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.514+0000 7f3e78d01700 1 -- 172.21.15.52:0/802232294 shutdown_connections 2023-11-27T15:18:18.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:18.514+0000 7f3e78d01700 1 -- 172.21.15.52:0/802232294 wait complete. 2023-11-27T15:18:18.531 INFO:tasks.cephfs.filesystem:scrub status for tag:d506e2e1-2357-4652-8afc-7708159185ae - {'path': '/', 'tag': 'd506e2e1-2357-4652-8afc-7708159185ae', 'options': 'recursive,force'} 2023-11-27T15:18:19.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:18 smithi052 ceph-mon[99056]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-11-27T15:18:19.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:18 smithi052 ceph-mon[99056]: scrub queued for path: / 2023-11-27T15:18:19.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:18 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:18:19.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:18 smithi052 ceph-mon[99056]: scrub summary: active paths [/] 2023-11-27T15:18:19.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:18 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:18:19.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:18 smithi150 ceph-mon[100559]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-11-27T15:18:19.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:18 smithi150 ceph-mon[100559]: scrub queued for path: / 2023-11-27T15:18:19.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:18 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:18:19.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:18 smithi150 ceph-mon[100559]: scrub summary: active paths [/] 2023-11-27T15:18:19.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:18 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:18:19.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:18 smithi102 ceph-mon[100296]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 767 B/s wr, 7 op/s 2023-11-27T15:18:19.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:18 smithi102 ceph-mon[100296]: scrub queued for path: / 2023-11-27T15:18:19.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:18 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:18:19.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:18 smithi102 ceph-mon[100296]: scrub summary: active paths [/] 2023-11-27T15:18:19.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:18 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:18:20.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:19 smithi052 ceph-mon[99056]: scrub summary: idle 2023-11-27T15:18:20.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:19 smithi150 ceph-mon[100559]: scrub summary: idle 2023-11-27T15:18:20.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:19 smithi102 ceph-mon[100296]: scrub summary: idle 2023-11-27T15:18:21.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:20 smithi052 ceph-mon[99056]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 4 op/s 2023-11-27T15:18:21.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:20 smithi150 ceph-mon[100559]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 4 op/s 2023-11-27T15:18:21.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:20 smithi102 ceph-mon[100296]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 4 op/s 2023-11-27T15:18:23.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:22 smithi150 ceph-mon[100559]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-11-27T15:18:23.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:22 smithi102 ceph-mon[100296]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-11-27T15:18:23.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:22 smithi052 ceph-mon[99056]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-11-27T15:18:25.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:24 smithi150 ceph-mon[100559]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2023-11-27T15:18:25.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:24 smithi102 ceph-mon[100296]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2023-11-27T15:18:25.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:24 smithi052 ceph-mon[99056]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2023-11-27T15:18:27.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:26 smithi150 ceph-mon[100559]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:18:27.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:26 smithi102 ceph-mon[100296]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:18:27.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:26 smithi052 ceph-mon[99056]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:18:29.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:28 smithi150 ceph-mon[100559]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:18:29.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:28 smithi102 ceph-mon[100296]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:18:29.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:28 smithi052 ceph-mon[99056]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:18:31.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:30 smithi150 ceph-mon[100559]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:18:31.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:30 smithi102 ceph-mon[100296]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:18:31.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:30 smithi052 ceph-mon[99056]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:18:33.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:32 smithi150 ceph-mon[100559]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2023-11-27T15:18:33.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:32 smithi102 ceph-mon[100296]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2023-11-27T15:18:33.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:32 smithi052 ceph-mon[99056]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2023-11-27T15:18:35.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:34 smithi150 ceph-mon[100559]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:18:35.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:34 smithi102 ceph-mon[100296]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:18:35.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:34 smithi052 ceph-mon[99056]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:18:37.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:36 smithi150 ceph-mon[100559]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:18:37.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:36 smithi102 ceph-mon[100296]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:18:37.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:36 smithi052 ceph-mon[99056]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:18:39.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:38 smithi150 ceph-mon[100559]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 7 op/s 2023-11-27T15:18:39.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:38 smithi102 ceph-mon[100296]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 7 op/s 2023-11-27T15:18:39.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:38 smithi052 ceph-mon[99056]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 7 op/s 2023-11-27T15:18:39.741 INFO:tasks.workunit.client.0.smithi052.stderr:Note: switching to '5e4a743ad7fd1422ebd6c4dc950a21a303279207'. 2023-11-27T15:18:39.741 INFO:tasks.workunit.client.0.smithi052.stderr: 2023-11-27T15:18:39.741 INFO:tasks.workunit.client.0.smithi052.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-11-27T15:18:39.741 INFO:tasks.workunit.client.0.smithi052.stderr:changes and commit them, and you can discard any commits you make in this 2023-11-27T15:18:39.741 INFO:tasks.workunit.client.0.smithi052.stderr:state without impacting any branches by switching back to a branch. 2023-11-27T15:18:39.742 INFO:tasks.workunit.client.0.smithi052.stderr: 2023-11-27T15:18:39.742 INFO:tasks.workunit.client.0.smithi052.stderr:If you want to create a new branch to retain commits you create, you may 2023-11-27T15:18:39.742 INFO:tasks.workunit.client.0.smithi052.stderr:do so (now or later) by using -c with the switch command. Example: 2023-11-27T15:18:39.742 INFO:tasks.workunit.client.0.smithi052.stderr: 2023-11-27T15:18:39.742 INFO:tasks.workunit.client.0.smithi052.stderr: git switch -c 2023-11-27T15:18:39.742 INFO:tasks.workunit.client.0.smithi052.stderr: 2023-11-27T15:18:39.742 INFO:tasks.workunit.client.0.smithi052.stderr:Or undo this operation with: 2023-11-27T15:18:39.742 INFO:tasks.workunit.client.0.smithi052.stderr: 2023-11-27T15:18:39.742 INFO:tasks.workunit.client.0.smithi052.stderr: git switch - 2023-11-27T15:18:39.742 INFO:tasks.workunit.client.0.smithi052.stderr: 2023-11-27T15:18:39.742 INFO:tasks.workunit.client.0.smithi052.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-11-27T15:18:39.742 INFO:tasks.workunit.client.0.smithi052.stderr: 2023-11-27T15:18:39.742 INFO:tasks.workunit.client.0.smithi052.stderr:HEAD is now at 5e4a743ad7f doc/cephfs: Update docs to match remove functionality and respective vxattrs 2023-11-27T15:18:39.744 DEBUG:teuthology.orchestra.run.smithi052:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-11-27T15:18:39.773 INFO:tasks.workunit.client.0.smithi052.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-11-27T15:18:39.775 INFO:tasks.workunit.client.0.smithi052.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-11-27T15:18:39.776 INFO:tasks.workunit.client.0.smithi052.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-11-27T15:18:40.323 INFO:tasks.workunit.client.0.smithi052.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-11-27T15:18:40.382 INFO:tasks.workunit.client.0.smithi052.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-11-27T15:18:40.420 INFO:tasks.workunit.client.0.smithi052.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-11-27T15:18:40.422 INFO:tasks.workunit.client.0.smithi052.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-11-27T15:18:40.422 INFO:tasks.workunit.client.0.smithi052.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-11-27T15:18:40.459 INFO:tasks.workunit.client.0.smithi052.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-11-27T15:18:40.463 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2023-11-27T15:18:40.463 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-11-27T15:18:40.520 INFO:tasks.workunit:Running workunits matching fs/misc on client.0... 2023-11-27T15:18:40.521 INFO:tasks.workunit:Running workunit fs/misc/acl.sh... 2023-11-27T15:18:40.521 DEBUG:teuthology.orchestra.run.smithi052:workunit test fs/misc/acl.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/acl.sh 2023-11-27T15:18:40.582 INFO:tasks.workunit.client.0.smithi052.stderr:+ set -e 2023-11-27T15:18:40.583 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir -p testdir 2023-11-27T15:18:40.585 INFO:tasks.workunit.client.0.smithi052.stderr:+ cd testdir 2023-11-27T15:18:40.585 INFO:tasks.workunit.client.0.smithi052.stderr:+ set +e 2023-11-27T15:18:40.585 INFO:tasks.workunit.client.0.smithi052.stderr:+ setfacl -d -m u:nobody:rw . 2023-11-27T15:18:40.596 INFO:tasks.workunit.client.0.smithi052.stderr:+ test 0 '!=' 0 2023-11-27T15:18:40.596 INFO:tasks.workunit.client.0.smithi052.stderr:+ set -e 2023-11-27T15:18:40.596 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=0 2023-11-27T15:18:40.596 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 0 -lt 100 ']' 2023-11-27T15:18:40.597 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 0 + 1 2023-11-27T15:18:40.597 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=1 2023-11-27T15:18:40.597 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:18:40.599 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:40.599 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:40.603 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:18:40.604 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:18:40.604 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:18:41.230 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:40 smithi052 ceph-mon[99056]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2023-11-27T15:18:41.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:40 smithi150 ceph-mon[100559]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2023-11-27T15:18:41.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:40 smithi102 ceph-mon[100296]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2023-11-27T15:18:41.385 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:41.385 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:41.486 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:18:41.487 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:18:41.499 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:41.499 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:41.499 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 1 -lt 100 ']' 2023-11-27T15:18:41.499 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 1 + 1 2023-11-27T15:18:41.528 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=2 2023-11-27T15:18:41.528 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:18:41.551 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:41.551 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:41.552 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:18:41.552 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:18:41.553 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:18:42.018 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:42.018 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:42.128 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:18:42.128 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:18:42.141 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:42.142 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:42.142 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -lt 100 ']' 2023-11-27T15:18:42.142 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 2 + 1 2023-11-27T15:18:42.161 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=3 2023-11-27T15:18:42.161 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:18:42.182 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:42.182 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:42.183 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:18:42.183 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:18:42.183 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:18:42.568 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:42.568 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:42.677 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:18:42.677 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:18:42.682 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:42.682 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:42.682 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 3 -lt 100 ']' 2023-11-27T15:18:42.682 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 3 + 1 2023-11-27T15:18:42.702 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=4 2023-11-27T15:18:42.702 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:18:42.723 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:42.723 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:42.724 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:18:42.724 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:18:42.724 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:18:43.166 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:42 smithi052 ceph-mon[99056]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2023-11-27T15:18:43.242 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:43.243 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:43.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:42 smithi150 ceph-mon[100559]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2023-11-27T15:18:43.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:42 smithi102 ceph-mon[100296]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2023-11-27T15:18:43.344 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:18:43.344 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:18:43.374 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:43.374 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:43.374 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 4 -lt 100 ']' 2023-11-27T15:18:43.374 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 4 + 1 2023-11-27T15:18:43.415 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=5 2023-11-27T15:18:43.415 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:18:43.439 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:43.439 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:43.440 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:18:43.440 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:18:43.440 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:18:43.826 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:43.826 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:43.936 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:18:43.936 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:18:43.948 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:43.948 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:43.949 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 5 -lt 100 ']' 2023-11-27T15:18:43.949 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 5 + 1 2023-11-27T15:18:43.969 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=6 2023-11-27T15:18:43.969 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:18:44.001 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:44.001 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:44.002 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:18:44.002 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:18:44.002 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:18:44.392 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:44.392 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:44.470 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:18:44.471 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:18:44.483 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:44.483 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:44.483 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 6 -lt 100 ']' 2023-11-27T15:18:44.483 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 6 + 1 2023-11-27T15:18:44.502 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=7 2023-11-27T15:18:44.502 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:18:44.534 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:44.535 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:44.536 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:18:44.536 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:18:44.536 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:18:44.900 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:44.900 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:45.019 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:18:45.019 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:18:45.040 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:45.040 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:45.040 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 7 -lt 100 ']' 2023-11-27T15:18:45.040 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 7 + 1 2023-11-27T15:18:45.073 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=8 2023-11-27T15:18:45.073 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:18:45.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:44 smithi052 ceph-mon[99056]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2023-11-27T15:18:45.140 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:45.140 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:45.141 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:18:45.141 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:18:45.141 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:18:45.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:44 smithi150 ceph-mon[100559]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2023-11-27T15:18:45.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:44 smithi102 ceph-mon[100296]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2023-11-27T15:18:45.509 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:45.509 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:45.576 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:18:45.577 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:18:45.590 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:45.591 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:45.591 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 8 -lt 100 ']' 2023-11-27T15:18:45.591 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 8 + 1 2023-11-27T15:18:45.619 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=9 2023-11-27T15:18:45.619 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:18:45.639 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:45.639 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:45.640 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:18:45.640 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:18:45.640 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:18:46.034 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:46.034 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:46.144 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:18:46.144 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:18:46.149 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:46.149 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:46.149 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 9 -lt 100 ']' 2023-11-27T15:18:46.149 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 9 + 1 2023-11-27T15:18:46.169 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=10 2023-11-27T15:18:46.169 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:18:46.189 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:46.189 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:46.190 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:18:46.190 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:18:46.191 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:18:46.667 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:46.667 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:46.886 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:18:46.886 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:18:46.900 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:46.901 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:46.901 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 10 -lt 100 ']' 2023-11-27T15:18:46.901 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 10 + 1 2023-11-27T15:18:46.918 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=11 2023-11-27T15:18:46.919 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:18:46.940 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:46.940 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:46.940 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:18:46.941 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:18:46.941 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:18:47.219 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:46 smithi052 ceph-mon[99056]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.3 KiB/s wr, 5 op/s 2023-11-27T15:18:47.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:46 smithi150 ceph-mon[100559]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.3 KiB/s wr, 5 op/s 2023-11-27T15:18:47.326 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:47.326 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:47.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:46 smithi102 ceph-mon[100296]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.3 KiB/s wr, 5 op/s 2023-11-27T15:18:47.485 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:18:47.485 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:18:47.506 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:47.507 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:47.507 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 11 -lt 100 ']' 2023-11-27T15:18:47.507 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 11 + 1 2023-11-27T15:18:47.548 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=12 2023-11-27T15:18:47.548 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:18:47.598 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:47.598 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:47.599 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:18:47.599 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:18:47.600 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:18:48.367 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:48.367 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:48.502 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:18:48.502 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:18:48.516 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:48.516 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:48.516 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 12 -lt 100 ']' 2023-11-27T15:18:48.516 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 12 + 1 2023-11-27T15:18:48.531 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:18:48.535 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=13 2023-11-27T15:18:48.535 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:18:48.563 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:48.563 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:48.564 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:18:48.564 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:18:48.565 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:18:48.967 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:48.967 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:49.042 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:18:49.042 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:18:49.048 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:49.048 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:49.048 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 13 -lt 100 ']' 2023-11-27T15:18:49.048 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 13 + 1 2023-11-27T15:18:49.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:48 smithi052 ceph-mon[99056]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 KiB/s wr, 8 op/s 2023-11-27T15:18:49.131 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=14 2023-11-27T15:18:49.132 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:18:49.171 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:49.171 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:49.172 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:18:49.172 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:18:49.172 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:18:49.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:48 smithi150 ceph-mon[100559]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 KiB/s wr, 8 op/s 2023-11-27T15:18:49.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:48 smithi102 ceph-mon[100296]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 KiB/s wr, 8 op/s 2023-11-27T15:18:49.725 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:49.725 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:49.876 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:18:49.877 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:18:49.898 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:49.898 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:49.898 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 14 -lt 100 ']' 2023-11-27T15:18:49.898 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 14 + 1 2023-11-27T15:18:49.923 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=15 2023-11-27T15:18:49.923 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:18:49.964 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:49.964 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:49.965 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:18:49.965 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:18:49.965 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:18:50.817 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:50.817 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:51.018 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:18:51.018 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:18:51.032 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:51.032 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:51.032 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 15 -lt 100 ']' 2023-11-27T15:18:51.033 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 15 + 1 2023-11-27T15:18:51.073 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=16 2023-11-27T15:18:51.073 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:18:51.123 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:51.123 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:51.124 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:18:51.124 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:18:51.125 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:18:51.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:50 smithi150 ceph-mon[100559]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.0 KiB/s wr, 5 op/s 2023-11-27T15:18:51.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:50 smithi102 ceph-mon[100296]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.0 KiB/s wr, 5 op/s 2023-11-27T15:18:51.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:50 smithi052 ceph-mon[99056]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.0 KiB/s wr, 5 op/s 2023-11-27T15:18:51.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.468+0000 7fd682cda700 1 -- 172.21.15.52:0/2815486229 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd67c0ffdf0 msgr2=0x7fd67c103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:51.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.468+0000 7fd682cda700 1 --2- 172.21.15.52:0/2815486229 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd67c0ffdf0 0x7fd67c103db0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd66800b0d0 tx=0x7fd66800b3e0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.468+0000 7fd682cda700 1 -- 172.21.15.52:0/2815486229 shutdown_connections 2023-11-27T15:18:51.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.468+0000 7fd682cda700 1 --2- 172.21.15.52:0/2815486229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd67c0ff450 0x7fd67c0ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.468+0000 7fd682cda700 1 --2- 172.21.15.52:0/2815486229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd67c104490 0x7fd67c10c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.468+0000 7fd682cda700 1 --2- 172.21.15.52:0/2815486229 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd67c0ffdf0 0x7fd67c103db0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.468+0000 7fd682cda700 1 -- 172.21.15.52:0/2815486229 >> 172.21.15.52:0/2815486229 conn(0x7fd67c0770b0 msgr2=0x7fd67c0774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:51.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.468+0000 7fd682cda700 1 -- 172.21.15.52:0/2815486229 shutdown_connections 2023-11-27T15:18:51.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.468+0000 7fd682cda700 1 -- 172.21.15.52:0/2815486229 wait complete. 2023-11-27T15:18:51.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.469+0000 7fd682cda700 1 Processor -- start 2023-11-27T15:18:51.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.469+0000 7fd682cda700 1 -- start start 2023-11-27T15:18:51.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.469+0000 7fd682cda700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd67c0ff450 0x7fd67c195530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:51.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.469+0000 7fd682cda700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd67c0ffdf0 0x7fd67c195a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:51.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.469+0000 7fd682cda700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd67c104490 0x7fd67c199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:51.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.469+0000 7fd682cda700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67c19a510 con 0x7fd67c0ffdf0 2023-11-27T15:18:51.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.469+0000 7fd682cda700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67c19a680 con 0x7fd67c0ff450 2023-11-27T15:18:51.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.469+0000 7fd682cda700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67c19a840 con 0x7fd67c104490 2023-11-27T15:18:51.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.469+0000 7fd67bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd67c0ffdf0 0x7fd67c195a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:51.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.470+0000 7fd67bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd67c0ffdf0 0x7fd67c195a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52422/0 (socket says 172.21.15.52:52422) 2023-11-27T15:18:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.470+0000 7fd67bfff700 1 -- 172.21.15.52:0/283382453 learned_addr learned my addr 172.21.15.52:0/283382453 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.470+0000 7fd680a76700 1 --2- 172.21.15.52:0/283382453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd67c0ff450 0x7fd67c195530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.470+0000 7fd681277700 1 --2- 172.21.15.52:0/283382453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd67c104490 0x7fd67c199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.470+0000 7fd67bfff700 1 -- 172.21.15.52:0/283382453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd67c104490 msgr2=0x7fd67c199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.470+0000 7fd67bfff700 1 --2- 172.21.15.52:0/283382453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd67c104490 0x7fd67c199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.470+0000 7fd67bfff700 1 -- 172.21.15.52:0/283382453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd67c0ff450 msgr2=0x7fd67c195530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.470+0000 7fd67bfff700 1 --2- 172.21.15.52:0/283382453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd67c0ff450 0x7fd67c195530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.470+0000 7fd67bfff700 1 -- 172.21.15.52:0/283382453 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd668009d00 con 0x7fd67c0ffdf0 2023-11-27T15:18:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.470+0000 7fd680a76700 1 --2- 172.21.15.52:0/283382453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd67c0ff450 0x7fd67c195530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:18:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.470+0000 7fd67bfff700 1 --2- 172.21.15.52:0/283382453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd67c0ffdf0 0x7fd67c195a70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd66800b770 tx=0x7fd668007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:51.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.470+0000 7fd679ffb700 1 -- 172.21.15.52:0/283382453 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66800e040 con 0x7fd67c0ffdf0 2023-11-27T15:18:51.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.470+0000 7fd679ffb700 1 -- 172.21.15.52:0/283382453 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd668005070 con 0x7fd67c0ffdf0 2023-11-27T15:18:51.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.470+0000 7fd682cda700 1 -- 172.21.15.52:0/283382453 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd67c10f260 con 0x7fd67c0ffdf0 2023-11-27T15:18:51.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.471+0000 7fd679ffb700 1 -- 172.21.15.52:0/283382453 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66801d6d0 con 0x7fd67c0ffdf0 2023-11-27T15:18:51.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.471+0000 7fd682cda700 1 -- 172.21.15.52:0/283382453 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd67c10f0d0 con 0x7fd67c0ffdf0 2023-11-27T15:18:51.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.474+0000 7fd679ffb700 1 -- 172.21.15.52:0/283382453 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7fd6680054e0 con 0x7fd67c0ffdf0 2023-11-27T15:18:51.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.474+0000 7fd679ffb700 1 --2- 172.21.15.52:0/283382453 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd6600a1460 0x7fd6600a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:51.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.474+0000 7fd679ffb700 1 -- 172.21.15.52:0/283382453 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fd6680cdcb0 con 0x7fd67c0ffdf0 2023-11-27T15:18:51.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.474+0000 7fd680a76700 1 --2- 172.21.15.52:0/283382453 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd6600a1460 0x7fd6600a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:51.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.475+0000 7fd680a76700 1 --2- 172.21.15.52:0/283382453 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd6600a1460 0x7fd6600a3910 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd66c00e530 tx=0x7fd66c010040 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:51.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.525+0000 7fd68ee66b80 1 Processor -- start 2023-11-27T15:18:51.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.525+0000 7fd68ee66b80 1 -- start start 2023-11-27T15:18:51.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.525+0000 7fd68ee66b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559de97a2b60 0x559de97a2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:51.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.525+0000 7fd68ee66b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559de97a3470 0x559de97a7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:51.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.525+0000 7fd68ee66b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559de97a78b0 0x559de97a7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:51.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.525+0000 7fd68ee66b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de96a8060 con 0x559de97a3470 2023-11-27T15:18:51.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.525+0000 7fd68ee66b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de96fa8a0 con 0x559de97a2b60 2023-11-27T15:18:51.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.525+0000 7fd68ee66b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de96f3d30 con 0x559de97a78b0 2023-11-27T15:18:51.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.525+0000 7fd67bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559de97a3470 0x559de97a7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:51.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.525+0000 7fd67bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559de97a3470 0x559de97a7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52430/0 (socket says 172.21.15.52:52430) 2023-11-27T15:18:51.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.525+0000 7fd67bfff700 1 -- 172.21.15.52:0/3344546686 learned_addr learned my addr 172.21.15.52:0/3344546686 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:51.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.525+0000 7fd680a76700 1 --2- 172.21.15.52:0/3344546686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559de97a2b60 0x559de97a2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:51.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.526+0000 7fd681277700 1 --2- 172.21.15.52:0/3344546686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559de97a78b0 0x559de97a7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:51.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.526+0000 7fd67bfff700 1 -- 172.21.15.52:0/3344546686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559de97a78b0 msgr2=0x559de97a7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:51.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.526+0000 7fd67bfff700 1 --2- 172.21.15.52:0/3344546686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559de97a78b0 0x559de97a7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.526+0000 7fd67bfff700 1 -- 172.21.15.52:0/3344546686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559de97a2b60 msgr2=0x559de97a2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:51.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.526+0000 7fd67bfff700 1 --2- 172.21.15.52:0/3344546686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559de97a2b60 0x559de97a2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.526+0000 7fd67bfff700 1 -- 172.21.15.52:0/3344546686 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559de9575630 con 0x559de97a3470 2023-11-27T15:18:51.533 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.526+0000 7fd67bfff700 1 --2- 172.21.15.52:0/3344546686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559de97a3470 0x559de97a7370 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd668009fd0 tx=0x7fd668009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aee7061cc700633 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:51.533 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.526+0000 7fd672ffd700 1 -- 172.21.15.52:0/3344546686 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66809a5b0 con 0x559de97a3470 2023-11-27T15:18:51.533 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.526+0000 7fd672ffd700 1 -- 172.21.15.52:0/3344546686 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd66809ac50 con 0x559de97a3470 2023-11-27T15:18:51.533 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.526+0000 7fd672ffd700 1 -- 172.21.15.52:0/3344546686 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66809b0d0 con 0x559de97a3470 2023-11-27T15:18:51.539 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.527+0000 7fd68ee66b80 1 -- 172.21.15.52:0/3344546686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559de97a3470 msgr2=0x559de97a7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:51.539 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.527+0000 7fd68ee66b80 1 --2- 172.21.15.52:0/3344546686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559de97a3470 0x559de97a7370 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd668009fd0 tx=0x7fd668009c70 comp rx=0 tx=0).stop 2023-11-27T15:18:51.539 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.527+0000 7fd68ee66b80 1 -- 172.21.15.52:0/3344546686 shutdown_connections 2023-11-27T15:18:51.539 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.527+0000 7fd68ee66b80 1 --2- 172.21.15.52:0/3344546686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559de97a2b60 0x559de97a2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.539 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.527+0000 7fd68ee66b80 1 --2- 172.21.15.52:0/3344546686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559de97a78b0 0x559de97a7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.539 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.527+0000 7fd68ee66b80 1 --2- 172.21.15.52:0/3344546686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559de97a3470 0x559de97a7370 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.539 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.527+0000 7fd68ee66b80 1 -- 172.21.15.52:0/3344546686 >> 172.21.15.52:0/3344546686 conn(0x559de9798360 msgr2=0x559de979db40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.527+0000 7fd68ee66b80 1 -- 172.21.15.52:0/3344546686 shutdown_connections 2023-11-27T15:18:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.527+0000 7fd68ee66b80 1 -- 172.21.15.52:0/3344546686 wait complete. 2023-11-27T15:18:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.528+0000 7fd68ee66b80 1 -- start start 2023-11-27T15:18:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.528+0000 7fd68ee66b80 1 Processor -- start 2023-11-27T15:18:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.528+0000 7fd68ee66b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559de97a2b60 0x559de97aef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.528+0000 7fd68ee66b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559de97a3470 0x559de97ac160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.528+0000 7fd68ee66b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559de97a78b0 0x559de97ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.528+0000 7fd68ee66b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de96958e0 con 0x559de97a78b0 2023-11-27T15:18:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.528+0000 7fd68ee66b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de9695580 con 0x559de97a3470 2023-11-27T15:18:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.528+0000 7fd68ee66b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de9577920 con 0x559de97a2b60 2023-11-27T15:18:51.541 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.528+0000 7fd68ee66b80 10 client.0 _ll_register_callbacks cb 0x559de96b3780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:18:51.541 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.529+0000 7fd681277700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559de97a78b0 0x559de97ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:51.541 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.529+0000 7fd67bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559de97a3470 0x559de97ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:51.541 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.529+0000 7fd681277700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559de97a78b0 0x559de97ac6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52438/0 (socket says 172.21.15.52:52438) 2023-11-27T15:18:51.541 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.529+0000 7fd67bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559de97a3470 0x559de97ac160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:42094/0 (socket says 172.21.15.52:42094) 2023-11-27T15:18:51.541 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.529+0000 7fd681277700 1 -- 172.21.15.52:0/1217355055 learned_addr learned my addr 172.21.15.52:0/1217355055 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:51.541 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.529+0000 7fd680a76700 1 --2- 172.21.15.52:0/1217355055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559de97a2b60 0x559de97aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:51.541 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.529+0000 7fd67bfff700 1 -- 172.21.15.52:0/1217355055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559de97a2b60 msgr2=0x559de97aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:51.541 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.529+0000 7fd67bfff700 1 --2- 172.21.15.52:0/1217355055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559de97a2b60 0x559de97aef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.541 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.529+0000 7fd67bfff700 1 -- 172.21.15.52:0/1217355055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559de97a78b0 msgr2=0x559de97ac6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:51.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.529+0000 7fd67bfff700 1 --2- 172.21.15.52:0/1217355055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559de97a78b0 0x559de97ac6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.529+0000 7fd67bfff700 1 -- 172.21.15.52:0/1217355055 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd67c10f0d0 con 0x559de97a3470 2023-11-27T15:18:51.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.529+0000 7fd681277700 1 --2- 172.21.15.52:0/1217355055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559de97a78b0 0x559de97ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:18:51.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.529+0000 7fd67bfff700 1 --2- 172.21.15.52:0/1217355055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559de97a3470 0x559de97ac160 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd66809ee40 tx=0x7fd668005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc3dfd49a28ae3ac server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:51.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.529+0000 7fd65ffff700 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2023-11-27T15:18:51.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.529+0000 7fd65ffff700 1 -- 172.21.15.52:0/1217355055 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66809ec80 con 0x559de97a3470 2023-11-27T15:18:51.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.530+0000 7fd65ffff700 1 -- 172.21.15.52:0/1217355055 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fd6680a3010 con 0x559de97a3470 2023-11-27T15:18:51.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.530+0000 7fd68ee66b80 1 -- 172.21.15.52:0/1217355055 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559de9795b20 con 0x559de97a3470 2023-11-27T15:18:51.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.530+0000 7fd65ffff700 1 -- 172.21.15.52:0/1217355055 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd66809a350 con 0x559de97a3470 2023-11-27T15:18:51.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.530+0000 7fd68ee66b80 10 client.25391 fetch_fsmap learned FSMap version 23 2023-11-27T15:18:51.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.530+0000 7fd68ee66b80 1 -- 172.21.15.52:0/1217355055 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x559de96ba190 con 0x559de97a3470 2023-11-27T15:18:51.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.530+0000 7fd65ffff700 1 -- 172.21.15.52:0/1217355055 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd66807d080 con 0x559de97a3470 2023-11-27T15:18:51.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.530+0000 7fd65ffff700 1 -- 172.21.15.52:0/1217355055 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd6680cb560 con 0x559de97a3470 2023-11-27T15:18:51.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.530+0000 7fd68ee66b80 10 client.25391 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:18:51.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.530+0000 7fd68ee66b80 10 client.25391 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:18:51.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.530+0000 7fd68ee66b80 20 client.25391 populate_metadata read hostname 'smithi052' 2023-11-27T15:18:51.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.530+0000 7fd68ee66b80 1 --2- 172.21.15.52:0/1217355055 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x559de97bc0c0 0x559de97be4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:51.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.530+0000 7fd68ee66b80 4 client.25391 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:18:51.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.530+0000 7fd68ee66b80 1 -- 172.21.15.52:0/1217355055 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559de96b60a0 con 0x559de97bc0c0 2023-11-27T15:18:51.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.530+0000 7fd680a76700 1 --2- 172.21.15.52:0/1217355055 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x559de97bc0c0 0x559de97be4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:51.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.531+0000 7fd65ffff700 10 client.25391 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:18:51.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.531+0000 7fd680a76700 1 --2- 172.21.15.52:0/1217355055 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x559de97bc0c0 0x559de97be4d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6d1895051d94f9c server_cookie=43bdbbc5abefe56 in_seq=0 out_seq=0 2023-11-27T15:18:51.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.532+0000 7fd65ffff700 1 -- 172.21.15.52:0/1217355055 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559de96b60a0 con 0x559de97bc0c0 2023-11-27T15:18:51.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.532+0000 7fd65ffff700 10 client.25391 handle_command_reply: tid=0 2023-11-27T15:18:51.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.532+0000 7fd68ee66b80 1 client.25391 shutdown 2023-11-27T15:18:51.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.532+0000 7fd68ee66b80 1 -- 172.21.15.52:0/1217355055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559de97a3470 msgr2=0x559de97ac160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:51.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.532+0000 7fd68ee66b80 1 --2- 172.21.15.52:0/1217355055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559de97a3470 0x559de97ac160 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd66809ee40 tx=0x7fd668005800 comp rx=0 tx=0).stop 2023-11-27T15:18:51.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.532+0000 7fd68ee66b80 1 -- 172.21.15.52:0/1217355055 shutdown_connections 2023-11-27T15:18:51.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.532+0000 7fd68ee66b80 1 --2- 172.21.15.52:0/1217355055 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x559de97bc0c0 0x559de97be4d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.532+0000 7fd68ee66b80 1 --2- 172.21.15.52:0/1217355055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559de97a78b0 0x559de97ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.532+0000 7fd68ee66b80 1 --2- 172.21.15.52:0/1217355055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559de97a2b60 0x559de97aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.532+0000 7fd68ee66b80 1 --2- 172.21.15.52:0/1217355055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559de97a3470 0x559de97ac160 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.532+0000 7fd65ffff700 0 client.25391 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:18:51.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.532+0000 7fd68ee66b80 1 -- 172.21.15.52:0/1217355055 >> 172.21.15.52:0/1217355055 conn(0x559de9798360 msgr2=0x559de979cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:51.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.532+0000 7fd68ee66b80 1 -- 172.21.15.52:0/1217355055 shutdown_connections 2023-11-27T15:18:51.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.532+0000 7fd68ee66b80 1 -- 172.21.15.52:0/1217355055 wait complete. 2023-11-27T15:18:51.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.532+0000 7fd68ee66b80 20 client.25391 trim_cache size 0 max 16384 2023-11-27T15:18:51.557 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.551+0000 7fd68ee66b80 1 Processor -- start 2023-11-27T15:18:51.558 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.551+0000 7fd68ee66b80 1 -- start start 2023-11-27T15:18:51.558 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.551+0000 7fd68ee66b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559de97a2b60 0x559de97b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:51.558 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.551+0000 7fd68ee66b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559de97a3470 0x559de97a9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:51.558 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.551+0000 7fd68ee66b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559de97a78b0 0x559de97b7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:51.558 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.551+0000 7fd68ee66b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de97a3bf0 con 0x559de97a3470 2023-11-27T15:18:51.558 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.551+0000 7fd68ee66b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de97aca10 con 0x559de97a78b0 2023-11-27T15:18:51.558 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.551+0000 7fd68ee66b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de979f1c0 con 0x559de97a2b60 2023-11-27T15:18:51.558 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.551+0000 7fd67bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559de97a3470 0x559de97a9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:51.559 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.551+0000 7fd67bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559de97a3470 0x559de97a9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52444/0 (socket says 172.21.15.52:52444) 2023-11-27T15:18:51.559 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.551+0000 7fd67bfff700 1 -- 172.21.15.52:0/2775002918 learned_addr learned my addr 172.21.15.52:0/2775002918 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:51.559 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.551+0000 7fd681277700 1 --2- 172.21.15.52:0/2775002918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559de97a78b0 0x559de97b7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:51.559 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.551+0000 7fd680a76700 1 --2- 172.21.15.52:0/2775002918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559de97a2b60 0x559de97b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:51.559 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.552+0000 7fd67bfff700 1 -- 172.21.15.52:0/2775002918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559de97a2b60 msgr2=0x559de97b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:51.559 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.552+0000 7fd67bfff700 1 --2- 172.21.15.52:0/2775002918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559de97a2b60 0x559de97b37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.559 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.552+0000 7fd67bfff700 1 -- 172.21.15.52:0/2775002918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559de97a78b0 msgr2=0x559de97b7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:51.559 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.552+0000 7fd67bfff700 1 --2- 172.21.15.52:0/2775002918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559de97a78b0 0x559de97b7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.559 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.552+0000 7fd67bfff700 1 -- 172.21.15.52:0/2775002918 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559de96b9ee0 con 0x559de97a3470 2023-11-27T15:18:51.560 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.552+0000 7fd681277700 1 --2- 172.21.15.52:0/2775002918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559de97a78b0 0x559de97b7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:18:51.560 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.552+0000 7fd680a76700 1 --2- 172.21.15.52:0/2775002918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559de97a2b60 0x559de97b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:18:51.560 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.552+0000 7fd67bfff700 1 --2- 172.21.15.52:0/2775002918 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559de97a3470 0x559de97a9a80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd668098fa0 tx=0x7fd6680cb390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df98d2c9d6198e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:51.560 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.552+0000 7fd6737fe700 1 -- 172.21.15.52:0/2775002918 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66809aa20 con 0x559de97a3470 2023-11-27T15:18:51.560 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.552+0000 7fd6737fe700 1 -- 172.21.15.52:0/2775002918 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559de9795b20 con 0x559de97a3470 2023-11-27T15:18:51.560 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.552+0000 7fd6737fe700 1 -- 172.21.15.52:0/2775002918 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66809a350 con 0x559de97a3470 2023-11-27T15:18:51.560 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.553+0000 7fd68ee66b80 1 -- 172.21.15.52:0/2775002918 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559de97a3470 msgr2=0x559de97a9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:51.560 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.553+0000 7fd68ee66b80 1 --2- 172.21.15.52:0/2775002918 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559de97a3470 0x559de97a9a80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd668098fa0 tx=0x7fd6680cb390 comp rx=0 tx=0).stop 2023-11-27T15:18:51.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.553+0000 7fd68ee66b80 1 -- 172.21.15.52:0/2775002918 shutdown_connections 2023-11-27T15:18:51.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.553+0000 7fd68ee66b80 1 --2- 172.21.15.52:0/2775002918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559de97a78b0 0x559de97b7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.553+0000 7fd68ee66b80 1 --2- 172.21.15.52:0/2775002918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559de97a2b60 0x559de97b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.553+0000 7fd68ee66b80 1 --2- 172.21.15.52:0/2775002918 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559de97a3470 0x559de97a9a80 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.553+0000 7fd68ee66b80 1 -- 172.21.15.52:0/2775002918 >> 172.21.15.52:0/2775002918 conn(0x559de9798360 msgr2=0x559de97a21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:51.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.553+0000 7fd68ee66b80 1 -- 172.21.15.52:0/2775002918 shutdown_connections 2023-11-27T15:18:51.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.553+0000 7fd68ee66b80 1 -- 172.21.15.52:0/2775002918 wait complete. 2023-11-27T15:18:51.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.553+0000 7fd68ee66b80 1 -- start start 2023-11-27T15:18:51.562 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.553+0000 7fd68ee66b80 1 Processor -- start 2023-11-27T15:18:51.565 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.554+0000 7fd68ee66b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559de97a2b60 0x559de97af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:51.565 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.554+0000 7fd68ee66b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559de97a3470 0x559de97afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:51.566 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.554+0000 7fd68ee66b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559de97a78b0 0x559de97b0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:51.566 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.554+0000 7fd68ee66b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de97b3df0 con 0x559de97a3470 2023-11-27T15:18:51.566 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.554+0000 7fd68ee66b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de980ef30 con 0x559de97a2b60 2023-11-27T15:18:51.566 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.554+0000 7fd68ee66b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559de980f0a0 con 0x559de97a78b0 2023-11-27T15:18:51.566 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.554+0000 7fd68ee66b80 10 client.0 _ll_register_callbacks cb 0x7fd66c009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:18:51.566 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.554+0000 7fd67bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559de97a3470 0x559de97afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:51.566 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.554+0000 7fd67bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559de97a3470 0x559de97afcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52450/0 (socket says 172.21.15.52:52450) 2023-11-27T15:18:51.566 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.554+0000 7fd67bfff700 1 -- 172.21.15.52:0/3561193866 learned_addr learned my addr 172.21.15.52:0/3561193866 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:51.567 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.554+0000 7fd680a76700 1 --2- 172.21.15.52:0/3561193866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559de97a2b60 0x559de97af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:51.567 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.554+0000 7fd681277700 1 --2- 172.21.15.52:0/3561193866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559de97a78b0 0x559de97b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:51.567 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.555+0000 7fd67bfff700 1 -- 172.21.15.52:0/3561193866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559de97a78b0 msgr2=0x559de97b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:51.567 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.555+0000 7fd67bfff700 1 --2- 172.21.15.52:0/3561193866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559de97a78b0 0x559de97b0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.567 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.555+0000 7fd67bfff700 1 -- 172.21.15.52:0/3561193866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559de97a2b60 msgr2=0x559de97af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:51.567 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.555+0000 7fd67bfff700 1 --2- 172.21.15.52:0/3561193866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559de97a2b60 0x559de97af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.567 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.555+0000 7fd67bfff700 1 -- 172.21.15.52:0/3561193866 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd66c0041d0 con 0x559de97a3470 2023-11-27T15:18:51.567 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.555+0000 7fd67bfff700 1 --2- 172.21.15.52:0/3561193866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559de97a3470 0x559de97afcd0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd6680cba00 tx=0x7fd668009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc941c924d895e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:51.567 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.555+0000 7fd65ffff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:18:51.568 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.555+0000 7fd65ffff700 1 -- 172.21.15.52:0/3561193866 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66809aa20 con 0x559de97a3470 2023-11-27T15:18:51.568 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.555+0000 7fd65ffff700 1 -- 172.21.15.52:0/3561193866 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fd66809ec80 con 0x559de97a3470 2023-11-27T15:18:51.568 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.555+0000 7fd68ee66b80 1 -- 172.21.15.52:0/3561193866 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd66809ac50 con 0x559de97a3470 2023-11-27T15:18:51.568 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.555+0000 7fd65ffff700 1 -- 172.21.15.52:0/3561193866 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd66809a5b0 con 0x559de97a3470 2023-11-27T15:18:51.568 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.555+0000 7fd68ee66b80 10 client.15582 fetch_fsmap learned FSMap version 23 2023-11-27T15:18:51.568 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.555+0000 7fd68ee66b80 1 -- 172.21.15.52:0/3561193866 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fd674009e40 con 0x559de97a3470 2023-11-27T15:18:51.568 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.555+0000 7fd65ffff700 1 -- 172.21.15.52:0/3561193866 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd66809ac50 con 0x559de97a3470 2023-11-27T15:18:51.568 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.556+0000 7fd65ffff700 1 -- 172.21.15.52:0/3561193866 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd668096c20 con 0x559de97a3470 2023-11-27T15:18:51.568 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.556+0000 7fd68ee66b80 10 client.15582 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:18:51.568 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.556+0000 7fd68ee66b80 10 client.15582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:18:51.568 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.556+0000 7fd68ee66b80 20 client.15582 populate_metadata read hostname 'smithi052' 2023-11-27T15:18:51.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.556+0000 7fd68ee66b80 1 --2- 172.21.15.52:0/3561193866 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x559de9810490 0x559de9812870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:51.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.556+0000 7fd68ee66b80 4 client.15582 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:18:51.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.556+0000 7fd68ee66b80 1 -- 172.21.15.52:0/3561193866 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559de96c2660 con 0x559de9810490 2023-11-27T15:18:51.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.556+0000 7fd680a76700 1 --2- 172.21.15.52:0/3561193866 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x559de9810490 0x559de9812870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:51.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.556+0000 7fd65ffff700 10 client.15582 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:18:51.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.556+0000 7fd680a76700 1 --2- 172.21.15.52:0/3561193866 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x559de9810490 0x559de9812870 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3d056d516abe4cc server_cookie=155260e0d1637b90 in_seq=0 out_seq=0 2023-11-27T15:18:51.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.557+0000 7fd65ffff700 1 -- 172.21.15.52:0/3561193866 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559de96c2660 con 0x559de9810490 2023-11-27T15:18:51.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.557+0000 7fd65ffff700 10 client.15582 handle_command_reply: tid=0 2023-11-27T15:18:51.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.557+0000 7fd68ee66b80 1 client.15582 shutdown 2023-11-27T15:18:51.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.557+0000 7fd68ee66b80 1 -- 172.21.15.52:0/3561193866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559de97a3470 msgr2=0x559de97afcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:51.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.557+0000 7fd68ee66b80 1 --2- 172.21.15.52:0/3561193866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559de97a3470 0x559de97afcd0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd6680cba00 tx=0x7fd668009eb0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.557+0000 7fd68ee66b80 1 -- 172.21.15.52:0/3561193866 shutdown_connections 2023-11-27T15:18:51.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.557+0000 7fd68ee66b80 1 --2- 172.21.15.52:0/3561193866 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x559de9810490 0x559de9812870 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.557+0000 7fd68ee66b80 1 --2- 172.21.15.52:0/3561193866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559de97a2b60 0x559de97af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.557+0000 7fd68ee66b80 1 --2- 172.21.15.52:0/3561193866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559de97a78b0 0x559de97b0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.557+0000 7fd65ffff700 0 client.15582 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:18:51.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.557+0000 7fd68ee66b80 1 --2- 172.21.15.52:0/3561193866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559de97a3470 0x559de97afcd0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.557+0000 7fd68ee66b80 1 -- 172.21.15.52:0/3561193866 >> 172.21.15.52:0/3561193866 conn(0x559de9798360 msgr2=0x559de97a66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:51.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.557+0000 7fd68ee66b80 1 -- 172.21.15.52:0/3561193866 shutdown_connections 2023-11-27T15:18:51.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.557+0000 7fd68ee66b80 1 -- 172.21.15.52:0/3561193866 wait complete. 2023-11-27T15:18:51.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.557+0000 7fd68ee66b80 20 client.15582 trim_cache size 0 max 16384 2023-11-27T15:18:51.571 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:18:51.571 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2023-11-27T15:18:51.571 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2023-11-27T15:18:51.571 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:18:51.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.558+0000 7fd682cda700 1 -- 172.21.15.52:0/283382453 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd6600a1460 msgr2=0x7fd6600a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:51.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.558+0000 7fd682cda700 1 --2- 172.21.15.52:0/283382453 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd6600a1460 0x7fd6600a3910 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd66c00e530 tx=0x7fd66c010040 comp rx=0 tx=0).stop 2023-11-27T15:18:51.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.558+0000 7fd682cda700 1 -- 172.21.15.52:0/283382453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd67c0ffdf0 msgr2=0x7fd67c195a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:51.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.558+0000 7fd682cda700 1 --2- 172.21.15.52:0/283382453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd67c0ffdf0 0x7fd67c195a70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd66800b770 tx=0x7fd668007870 comp rx=0 tx=0).stop 2023-11-27T15:18:51.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.558+0000 7fd682cda700 1 -- 172.21.15.52:0/283382453 shutdown_connections 2023-11-27T15:18:51.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.558+0000 7fd682cda700 1 --2- 172.21.15.52:0/283382453 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd6600a1460 0x7fd6600a3910 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.558+0000 7fd682cda700 1 --2- 172.21.15.52:0/283382453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd67c0ff450 0x7fd67c195530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.558+0000 7fd682cda700 1 --2- 172.21.15.52:0/283382453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd67c104490 0x7fd67c199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.558+0000 7fd682cda700 1 --2- 172.21.15.52:0/283382453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd67c0ffdf0 0x7fd67c195a70 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:51.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.558+0000 7fd682cda700 1 -- 172.21.15.52:0/283382453 >> 172.21.15.52:0/283382453 conn(0x7fd67c0770b0 msgr2=0x7fd67c0fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:51.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.558+0000 7fd682cda700 1 -- 172.21.15.52:0/283382453 shutdown_connections 2023-11-27T15:18:51.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:51.558+0000 7fd682cda700 1 -- 172.21.15.52:0/283382453 wait complete. 2023-11-27T15:18:51.653 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-27T15:18:51.653 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:18:51.653 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:51.654 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:51.810 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:18:51.810 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:18:51.823 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:51.823 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:51.823 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 16 -lt 100 ']' 2023-11-27T15:18:51.823 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 16 + 1 2023-11-27T15:18:51.872 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=17 2023-11-27T15:18:51.873 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:18:51.908 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:51.908 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:51.909 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:18:51.909 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:18:51.909 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:18:52.350 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:52.350 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:52.445 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:18:52.445 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:18:52.465 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:52.465 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:52.465 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 17 -lt 100 ']' 2023-11-27T15:18:52.465 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 17 + 1 2023-11-27T15:18:52.514 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=18 2023-11-27T15:18:52.514 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:18:52.556 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:52.556 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:52.557 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:18:52.557 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:18:52.557 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:18:53.267 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:53.267 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:53.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:52 smithi150 ceph-mon[100559]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 8 op/s 2023-11-27T15:18:53.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:52 smithi102 ceph-mon[100296]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 8 op/s 2023-11-27T15:18:53.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:52 smithi052 ceph-mon[99056]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 8 op/s 2023-11-27T15:18:53.393 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:18:53.393 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:18:53.422 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:53.423 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:53.423 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 18 -lt 100 ']' 2023-11-27T15:18:53.423 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 18 + 1 2023-11-27T15:18:53.456 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=19 2023-11-27T15:18:53.456 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:18:53.535 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:53.535 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:53.536 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:18:53.536 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:18:53.537 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:18:54.308 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:54.308 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:54.451 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:18:54.451 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:18:54.464 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:54.464 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:54.464 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 19 -lt 100 ']' 2023-11-27T15:18:54.465 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 19 + 1 2023-11-27T15:18:54.506 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=20 2023-11-27T15:18:54.506 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:18:54.573 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:54.573 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:54.574 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:18:54.574 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:18:54.574 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:18:54.867 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.859+0000 7ff5cc76f700 1 -- 172.21.15.52:0/2932522637 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5c41055a0 msgr2=0x7ff5c410d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:54.867 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.859+0000 7ff5cc76f700 1 --2- 172.21.15.52:0/2932522637 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5c41055a0 0x7ff5c410d170 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00b0d0 tx=0x7ff5bc00b3e0 comp rx=0 tx=0).stop 2023-11-27T15:18:54.867 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.859+0000 7ff5cc76f700 1 -- 172.21.15.52:0/2932522637 shutdown_connections 2023-11-27T15:18:54.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.859+0000 7ff5cc76f700 1 --2- 172.21.15.52:0/2932522637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5c4100560 0x7ff5c4100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:54.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.859+0000 7ff5cc76f700 1 --2- 172.21.15.52:0/2932522637 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5c41055a0 0x7ff5c410d170 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:54.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.859+0000 7ff5cc76f700 1 --2- 172.21.15.52:0/2932522637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5c4100f00 0x7ff5c4104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:54.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.859+0000 7ff5cc76f700 1 -- 172.21.15.52:0/2932522637 >> 172.21.15.52:0/2932522637 conn(0x7ff5c4077080 msgr2=0x7ff5c4077480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:54.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.859+0000 7ff5cc76f700 1 -- 172.21.15.52:0/2932522637 shutdown_connections 2023-11-27T15:18:54.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.859+0000 7ff5cc76f700 1 -- 172.21.15.52:0/2932522637 wait complete. 2023-11-27T15:18:54.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.860+0000 7ff5cc76f700 1 Processor -- start 2023-11-27T15:18:54.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.860+0000 7ff5cc76f700 1 -- start start 2023-11-27T15:18:54.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.860+0000 7ff5cc76f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5c4100560 0x7ff5c4072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:54.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.860+0000 7ff5cc76f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5c4100f00 0x7ff5c4073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:54.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.860+0000 7ff5cc76f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5c41055a0 0x7ff5c41a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:54.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.860+0000 7ff5cc76f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c4073b60 con 0x7ff5c4100560 2023-11-27T15:18:54.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.860+0000 7ff5cc76f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c41a4b40 con 0x7ff5c41055a0 2023-11-27T15:18:54.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.860+0000 7ff5cc76f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c41a4c80 con 0x7ff5c4100f00 2023-11-27T15:18:54.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.861+0000 7ff5ca50b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5c4100560 0x7ff5c4072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:54.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.861+0000 7ff5ca50b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5c4100560 0x7ff5c4072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52462/0 (socket says 172.21.15.52:52462) 2023-11-27T15:18:54.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.861+0000 7ff5ca50b700 1 -- 172.21.15.52:0/3667467213 learned_addr learned my addr 172.21.15.52:0/3667467213 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:54.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.861+0000 7ff5c9d0a700 1 --2- 172.21.15.52:0/3667467213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5c4100f00 0x7ff5c4073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:54.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.861+0000 7ff5cad0c700 1 --2- 172.21.15.52:0/3667467213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5c41055a0 0x7ff5c41a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:54.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.861+0000 7ff5ca50b700 1 -- 172.21.15.52:0/3667467213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5c4100f00 msgr2=0x7ff5c4073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:54.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.861+0000 7ff5ca50b700 1 --2- 172.21.15.52:0/3667467213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5c4100f00 0x7ff5c4073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:54.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.861+0000 7ff5ca50b700 1 -- 172.21.15.52:0/3667467213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5c41055a0 msgr2=0x7ff5c41a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:54.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.861+0000 7ff5ca50b700 1 --2- 172.21.15.52:0/3667467213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5c41055a0 0x7ff5c41a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:54.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.861+0000 7ff5ca50b700 1 -- 172.21.15.52:0/3667467213 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5bc009d00 con 0x7ff5c4100560 2023-11-27T15:18:54.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.861+0000 7ff5cad0c700 1 --2- 172.21.15.52:0/3667467213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5c41055a0 0x7ff5c41a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:18:54.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.861+0000 7ff5ca50b700 1 --2- 172.21.15.52:0/3667467213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5c4100560 0x7ff5c4072df0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000d980 tx=0x7ff5c000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:54.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.861+0000 7ff5bb7fe700 1 -- 172.21.15.52:0/3667467213 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0004220 con 0x7ff5c4100560 2023-11-27T15:18:54.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.861+0000 7ff5cc76f700 1 -- 172.21.15.52:0/3667467213 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c4110370 con 0x7ff5c4100560 2023-11-27T15:18:54.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.861+0000 7ff5bb7fe700 1 -- 172.21.15.52:0/3667467213 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5c0007640 con 0x7ff5c4100560 2023-11-27T15:18:54.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.861+0000 7ff5bb7fe700 1 -- 172.21.15.52:0/3667467213 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0005610 con 0x7ff5c4100560 2023-11-27T15:18:54.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.862+0000 7ff5cc76f700 1 -- 172.21.15.52:0/3667467213 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c41101e0 con 0x7ff5c4100560 2023-11-27T15:18:54.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.863+0000 7ff5cc76f700 1 -- 172.21.15.52:0/3667467213 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c4110500 con 0x7ff5c4100560 2023-11-27T15:18:54.872 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.864+0000 7ff5bb7fe700 1 -- 172.21.15.52:0/3667467213 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7ff5c0004750 con 0x7ff5c4100560 2023-11-27T15:18:54.873 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.865+0000 7ff5bb7fe700 1 --2- 172.21.15.52:0/3667467213 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ff5b00a13e0 0x7ff5b00a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:54.873 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.865+0000 7ff5bb7fe700 1 -- 172.21.15.52:0/3667467213 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff5c00cce20 con 0x7ff5c4100560 2023-11-27T15:18:54.873 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.865+0000 7ff5c9d0a700 1 --2- 172.21.15.52:0/3667467213 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ff5b00a13e0 0x7ff5b00a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:54.873 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.865+0000 7ff5c9d0a700 1 --2- 172.21.15.52:0/3667467213 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ff5b00a13e0 0x7ff5b00a3890 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40745e0 tx=0x7ff5b4009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:54.874 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:54.870+0000 7ff5bb7fe700 1 -- 172.21.15.52:0/3667467213 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff5c0095fe0 con 0x7ff5c4100560 2023-11-27T15:18:55.049 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:55.050 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:55.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:54 smithi102 ceph-mon[100296]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 6 op/s 2023-11-27T15:18:55.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:54 smithi052 ceph-mon[99056]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 6 op/s 2023-11-27T15:18:55.137 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:55.133+0000 7ff5cc76f700 1 -- 172.21.15.52:0/3667467213 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5c4067a60 con 0x7ff5c4100560 2023-11-27T15:18:55.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:55.134+0000 7ff5bb7fe700 1 -- 172.21.15.52:0/3667467213 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10804 (secure 0 0 0) 0x7ff5c0016020 con 0x7ff5c4100560 2023-11-27T15:18:55.138 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:18:55.139 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2328881376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2328881376},{"type":"v1","addr":"172.21.15.52:6839","nonce":2328881376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2799551570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2799551570},{"type":"v1","addr":"172.21.15.52:6841","nonce":2799551570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:16:09.319630+0000","modified":"2023-11-27T15:16:14.742405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24494,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/215725838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":215725838},{"type":"v1","addr":"172.21.15.150:6839","nonce":215725838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3571830559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3571830559},{"type":"v1","addr":"172.21.15.102:6839","nonce":3571830559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:18:55.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:55.136+0000 7ff5cc76f700 1 -- 172.21.15.52:0/3667467213 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ff5b00a13e0 msgr2=0x7ff5b00a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:55.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:55.136+0000 7ff5cc76f700 1 --2- 172.21.15.52:0/3667467213 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ff5b00a13e0 0x7ff5b00a3890 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40745e0 tx=0x7ff5b4009260 comp rx=0 tx=0).stop 2023-11-27T15:18:55.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:55.136+0000 7ff5cc76f700 1 -- 172.21.15.52:0/3667467213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5c4100560 msgr2=0x7ff5c4072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:55.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:55.136+0000 7ff5cc76f700 1 --2- 172.21.15.52:0/3667467213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5c4100560 0x7ff5c4072df0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000d980 tx=0x7ff5c000dc90 comp rx=0 tx=0).stop 2023-11-27T15:18:55.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:55.136+0000 7ff5cc76f700 1 -- 172.21.15.52:0/3667467213 shutdown_connections 2023-11-27T15:18:55.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:55.136+0000 7ff5cc76f700 1 --2- 172.21.15.52:0/3667467213 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ff5b00a13e0 0x7ff5b00a3890 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:55.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:55.136+0000 7ff5cc76f700 1 --2- 172.21.15.52:0/3667467213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5c41055a0 0x7ff5c41a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:55.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:55.136+0000 7ff5cc76f700 1 --2- 172.21.15.52:0/3667467213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5c4100560 0x7ff5c4072df0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:55.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:55.136+0000 7ff5cc76f700 1 --2- 172.21.15.52:0/3667467213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5c4100f00 0x7ff5c4073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:55.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:55.136+0000 7ff5cc76f700 1 -- 172.21.15.52:0/3667467213 >> 172.21.15.52:0/3667467213 conn(0x7ff5c4077080 msgr2=0x7ff5c40fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:55.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:55.136+0000 7ff5cc76f700 1 -- 172.21.15.52:0/3667467213 shutdown_connections 2023-11-27T15:18:55.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:55.136+0000 7ff5cc76f700 1 -- 172.21.15.52:0/3667467213 wait complete. 2023-11-27T15:18:55.142 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 23 2023-11-27T15:18:55.143 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:18:55.143 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:18:55.164 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:55.164 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:55.164 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 20 -lt 100 ']' 2023-11-27T15:18:55.164 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 20 + 1 2023-11-27T15:18:55.247 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=21 2023-11-27T15:18:55.247 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:18:55.257 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-27T15:18:55.276 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:55.276 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:55.277 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:18:55.277 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:18:55.277 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:18:55.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:54 smithi150 ceph-mon[100559]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 6 op/s 2023-11-27T15:18:55.824 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:55.824 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:55.900 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:18:55.901 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:18:55.908 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:55.908 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:55.908 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 21 -lt 100 ']' 2023-11-27T15:18:55.908 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 21 + 1 2023-11-27T15:18:55.947 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=22 2023-11-27T15:18:55.947 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:18:55.995 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:55.995 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:55.996 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:18:55.996 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:18:55.997 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:18:56.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:55 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3667467213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:18:56.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:55 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3667467213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:18:56.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:55 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3667467213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:18:56.582 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:56.583 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:56.693 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:18:56.693 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:18:56.713 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:56.713 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:56.714 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 22 -lt 100 ']' 2023-11-27T15:18:56.714 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 22 + 1 2023-11-27T15:18:56.747 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=23 2023-11-27T15:18:56.747 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:18:56.810 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:56.810 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:56.811 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:18:56.811 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:18:56.811 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:18:57.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:56 smithi102 ceph-mon[100296]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 6 op/s 2023-11-27T15:18:57.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:56 smithi052 ceph-mon[99056]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 6 op/s 2023-11-27T15:18:57.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:56 smithi150 ceph-mon[100559]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 6 op/s 2023-11-27T15:18:57.491 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:57.491 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:57.684 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:18:57.684 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:18:57.706 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:57.707 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:57.707 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 23 -lt 100 ']' 2023-11-27T15:18:57.707 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 23 + 1 2023-11-27T15:18:57.755 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=24 2023-11-27T15:18:57.755 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:18:57.815 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:57.816 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:57.817 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:18:57.817 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:18:57.817 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:18:58.355 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.350+0000 7f152c3fb700 1 -- 172.21.15.52:0/1459454784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1524101ca0 msgr2=0x7f152410a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:58.355 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.350+0000 7f152c3fb700 1 --2- 172.21.15.52:0/1459454784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1524101ca0 0x7f152410a370 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1518009990 tx=0x7f1518009ca0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.355 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.350+0000 7f152c3fb700 1 -- 172.21.15.52:0/1459454784 shutdown_connections 2023-11-27T15:18:58.355 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.350+0000 7f152c3fb700 1 --2- 172.21.15.52:0/1459454784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f152410a8b0 0x7f152410ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.356 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.350+0000 7f152c3fb700 1 --2- 172.21.15.52:0/1459454784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1524101390 0x7f1524101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.356 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.350+0000 7f152c3fb700 1 --2- 172.21.15.52:0/1459454784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1524101ca0 0x7f152410a370 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.356 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.350+0000 7f152c3fb700 1 -- 172.21.15.52:0/1459454784 >> 172.21.15.52:0/1459454784 conn(0x7f15240770b0 msgr2=0x7f15240774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:58.356 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.350+0000 7f152c3fb700 1 -- 172.21.15.52:0/1459454784 shutdown_connections 2023-11-27T15:18:58.356 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.350+0000 7f152c3fb700 1 -- 172.21.15.52:0/1459454784 wait complete. 2023-11-27T15:18:58.359 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.351+0000 7f152c3fb700 1 Processor -- start 2023-11-27T15:18:58.359 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.351+0000 7f152c3fb700 1 -- start start 2023-11-27T15:18:58.359 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.352+0000 7f152c3fb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1524101390 0x7f1524196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:58.359 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.352+0000 7f152c3fb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1524101ca0 0x7f1524196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:58.359 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.352+0000 7f152c3fb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f152410a8b0 0x7f152419adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:58.359 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.352+0000 7f152c3fb700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152419b3f0 con 0x7f1524101ca0 2023-11-27T15:18:58.359 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.352+0000 7f152c3fb700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152419b560 con 0x7f152410a8b0 2023-11-27T15:18:58.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.352+0000 7f152c3fb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152419b720 con 0x7f1524101390 2023-11-27T15:18:58.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.352+0000 7f1529996700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1524101ca0 0x7f1524196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:58.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.352+0000 7f1529996700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1524101ca0 0x7f1524196950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52476/0 (socket says 172.21.15.52:52476) 2023-11-27T15:18:58.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.352+0000 7f1529996700 1 -- 172.21.15.52:0/2583531768 learned_addr learned my addr 172.21.15.52:0/2583531768 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:58.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.352+0000 7f152a998700 1 --2- 172.21.15.52:0/2583531768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f152410a8b0 0x7f152419adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:58.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.352+0000 7f152a197700 1 --2- 172.21.15.52:0/2583531768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1524101390 0x7f1524196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:58.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.352+0000 7f1529996700 1 -- 172.21.15.52:0/2583531768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1524101390 msgr2=0x7f1524196410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:58.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.352+0000 7f1529996700 1 --2- 172.21.15.52:0/2583531768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1524101390 0x7f1524196410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.361 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.352+0000 7f1529996700 1 -- 172.21.15.52:0/2583531768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f152410a8b0 msgr2=0x7f152419adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:58.361 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.352+0000 7f1529996700 1 --2- 172.21.15.52:0/2583531768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f152410a8b0 0x7f152419adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.361 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.352+0000 7f1529996700 1 -- 172.21.15.52:0/2583531768 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15180095f0 con 0x7f1524101ca0 2023-11-27T15:18:58.361 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.353+0000 7f1529996700 1 --2- 172.21.15.52:0/2583531768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1524101ca0 0x7f1524196950 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f15180098e0 tx=0x7f151800f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:58.361 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.353+0000 7f15237fe700 1 -- 172.21.15.52:0/2583531768 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151801d070 con 0x7f1524101ca0 2023-11-27T15:18:58.361 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.353+0000 7f15237fe700 1 -- 172.21.15.52:0/2583531768 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1518005050 con 0x7f1524101ca0 2023-11-27T15:18:58.361 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.353+0000 7f15237fe700 1 -- 172.21.15.52:0/2583531768 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1518004b30 con 0x7f1524101ca0 2023-11-27T15:18:58.361 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.353+0000 7f152c3fb700 1 -- 172.21.15.52:0/2583531768 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1524110340 con 0x7f1524101ca0 2023-11-27T15:18:58.362 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.353+0000 7f152c3fb700 1 -- 172.21.15.52:0/2583531768 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15241101b0 con 0x7f1524101ca0 2023-11-27T15:18:58.363 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.355+0000 7f15237fe700 1 -- 172.21.15.52:0/2583531768 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f15180051c0 con 0x7f1524101ca0 2023-11-27T15:18:58.363 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.356+0000 7f15237fe700 1 --2- 172.21.15.52:0/2583531768 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f150c0a13e0 0x7f150c0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:58.363 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.356+0000 7f15237fe700 1 -- 172.21.15.52:0/2583531768 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f15180cdd50 con 0x7f1524101ca0 2023-11-27T15:18:58.363 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.356+0000 7f152a197700 1 --2- 172.21.15.52:0/2583531768 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f150c0a13e0 0x7f150c0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:58.363 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.357+0000 7f152a197700 1 --2- 172.21.15.52:0/2583531768 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f150c0a13e0 0x7f150c0a3890 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1514009550 tx=0x7f15140091b0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:58.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.444+0000 7f1538587b80 1 Processor -- start 2023-11-27T15:18:58.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.444+0000 7f1538587b80 1 -- start start 2023-11-27T15:18:58.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.444+0000 7f1538587b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55acd1cd5b60 0x55acd1cd5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:58.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.444+0000 7f1538587b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55acd1cd6470 0x55acd1cda370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:58.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.444+0000 7f1538587b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acd1cda8b0 0x55acd1cdad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:58.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.444+0000 7f1538587b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acd1bdb060 con 0x55acd1cd6470 2023-11-27T15:18:58.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.444+0000 7f1538587b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acd1c2d8a0 con 0x55acd1cda8b0 2023-11-27T15:18:58.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.444+0000 7f1538587b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acd1c26d30 con 0x55acd1cd5b60 2023-11-27T15:18:58.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.445+0000 7f1529996700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55acd1cd6470 0x55acd1cda370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:58.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.445+0000 7f1529996700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55acd1cd6470 0x55acd1cda370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52486/0 (socket says 172.21.15.52:52486) 2023-11-27T15:18:58.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.445+0000 7f1529996700 1 -- 172.21.15.52:0/3974608295 learned_addr learned my addr 172.21.15.52:0/3974608295 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:58.454 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.445+0000 7f152a197700 1 --2- 172.21.15.52:0/3974608295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55acd1cd5b60 0x55acd1cd5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:58.454 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.445+0000 7f152a998700 1 --2- 172.21.15.52:0/3974608295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acd1cda8b0 0x55acd1cdad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:58.454 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.445+0000 7f1529996700 1 -- 172.21.15.52:0/3974608295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55acd1cd5b60 msgr2=0x55acd1cd5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:58.454 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.445+0000 7f1529996700 1 --2- 172.21.15.52:0/3974608295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55acd1cd5b60 0x55acd1cd5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.445+0000 7f1529996700 1 -- 172.21.15.52:0/3974608295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acd1cda8b0 msgr2=0x55acd1cdad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:58.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.445+0000 7f1529996700 1 --2- 172.21.15.52:0/3974608295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acd1cda8b0 0x55acd1cdad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.445+0000 7f1529996700 1 -- 172.21.15.52:0/3974608295 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acd1aa8630 con 0x55acd1cd6470 2023-11-27T15:18:58.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.445+0000 7f152a197700 1 --2- 172.21.15.52:0/3974608295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55acd1cd5b60 0x55acd1cd5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:18:58.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.445+0000 7f1529996700 1 --2- 172.21.15.52:0/3974608295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55acd1cd6470 0x55acd1cda370 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f151809ab90 tx=0x7f151809ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2afdf0305f8ba87 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:58.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.445+0000 7f1520dac700 1 -- 172.21.15.52:0/3974608295 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15180a2fc0 con 0x55acd1cd6470 2023-11-27T15:18:58.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.446+0000 7f1520dac700 1 -- 172.21.15.52:0/3974608295 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f151809ee90 con 0x55acd1cd6470 2023-11-27T15:18:58.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.446+0000 7f1520dac700 1 -- 172.21.15.52:0/3974608295 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151809a3c0 con 0x55acd1cd6470 2023-11-27T15:18:58.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.446+0000 7f1538587b80 1 -- 172.21.15.52:0/3974608295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55acd1cd6470 msgr2=0x55acd1cda370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:58.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.446+0000 7f1538587b80 1 --2- 172.21.15.52:0/3974608295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55acd1cd6470 0x55acd1cda370 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f151809ab90 tx=0x7f151809ac00 comp rx=0 tx=0).stop 2023-11-27T15:18:58.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.446+0000 7f1538587b80 1 -- 172.21.15.52:0/3974608295 shutdown_connections 2023-11-27T15:18:58.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.446+0000 7f1538587b80 1 --2- 172.21.15.52:0/3974608295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acd1cda8b0 0x55acd1cdad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.446+0000 7f1538587b80 1 --2- 172.21.15.52:0/3974608295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55acd1cd5b60 0x55acd1cd5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.446+0000 7f1538587b80 1 --2- 172.21.15.52:0/3974608295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55acd1cd6470 0x55acd1cda370 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.446+0000 7f1538587b80 1 -- 172.21.15.52:0/3974608295 >> 172.21.15.52:0/3974608295 conn(0x55acd1ccb360 msgr2=0x55acd1cd0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:58.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.447+0000 7f1538587b80 1 -- 172.21.15.52:0/3974608295 shutdown_connections 2023-11-27T15:18:58.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.447+0000 7f1538587b80 1 -- 172.21.15.52:0/3974608295 wait complete. 2023-11-27T15:18:58.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.447+0000 7f1538587b80 1 -- start start 2023-11-27T15:18:58.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.447+0000 7f1538587b80 1 Processor -- start 2023-11-27T15:18:58.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.448+0000 7f1538587b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55acd1cd5b60 0x55acd1ce1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:58.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.448+0000 7f1538587b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55acd1cd6470 0x55acd1cdf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:58.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.448+0000 7f1538587b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acd1cda8b0 0x55acd1cdf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:58.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.448+0000 7f1538587b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acd1bc88e0 con 0x55acd1cd5b60 2023-11-27T15:18:58.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.448+0000 7f1538587b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acd1bc8580 con 0x55acd1cda8b0 2023-11-27T15:18:58.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.448+0000 7f1538587b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acd1aaa920 con 0x55acd1cd6470 2023-11-27T15:18:58.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.448+0000 7f1538587b80 10 client.0 _ll_register_callbacks cb 0x55acd1be6780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:18:58.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.448+0000 7f152a197700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55acd1cd5b60 0x55acd1ce1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:58.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.448+0000 7f152a197700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55acd1cd5b60 0x55acd1ce1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52490/0 (socket says 172.21.15.52:52490) 2023-11-27T15:18:58.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.448+0000 7f152a197700 1 -- 172.21.15.52:0/2310071153 learned_addr learned my addr 172.21.15.52:0/2310071153 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:58.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.448+0000 7f152a998700 1 --2- 172.21.15.52:0/2310071153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acd1cda8b0 0x55acd1cdf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:58.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.448+0000 7f1529996700 1 --2- 172.21.15.52:0/2310071153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55acd1cd6470 0x55acd1cdf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:58.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.448+0000 7f152a197700 1 -- 172.21.15.52:0/2310071153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55acd1cd6470 msgr2=0x55acd1cdf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:58.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.448+0000 7f152a197700 1 --2- 172.21.15.52:0/2310071153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55acd1cd6470 0x55acd1cdf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.448+0000 7f152a197700 1 -- 172.21.15.52:0/2310071153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acd1cda8b0 msgr2=0x55acd1cdf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:58.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.448+0000 7f152a197700 1 --2- 172.21.15.52:0/2310071153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acd1cda8b0 0x55acd1cdf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.448+0000 7f152a197700 1 -- 172.21.15.52:0/2310071153 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15241101b0 con 0x55acd1cd5b60 2023-11-27T15:18:58.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.449+0000 7f152a197700 1 --2- 172.21.15.52:0/2310071153 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55acd1cd5b60 0x55acd1ce1f10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1514007ab0 tx=0x7f1514004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5043fd4afe45f19c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:58.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.449+0000 7f1509ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:18:58.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.449+0000 7f1509ffb700 1 -- 172.21.15.52:0/2310071153 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1514015ad0 con 0x55acd1cd5b60 2023-11-27T15:18:58.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.449+0000 7f1509ffb700 1 -- 172.21.15.52:0/2310071153 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f1514015c30 con 0x55acd1cd5b60 2023-11-27T15:18:58.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.449+0000 7f1538587b80 1 -- 172.21.15.52:0/2310071153 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55acd1cc8b20 con 0x55acd1cd5b60 2023-11-27T15:18:58.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.449+0000 7f1509ffb700 1 -- 172.21.15.52:0/2310071153 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1514015df0 con 0x55acd1cd5b60 2023-11-27T15:18:58.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.449+0000 7f1538587b80 10 client.15600 fetch_fsmap learned FSMap version 23 2023-11-27T15:18:58.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.449+0000 7f1538587b80 1 -- 172.21.15.52:0/2310071153 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55acd1bed190 con 0x55acd1cd5b60 2023-11-27T15:18:58.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.449+0000 7f1509ffb700 1 -- 172.21.15.52:0/2310071153 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55acd1cc8b20 con 0x55acd1cd5b60 2023-11-27T15:18:58.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.450+0000 7f1509ffb700 1 -- 172.21.15.52:0/2310071153 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1514016410 con 0x55acd1cd5b60 2023-11-27T15:18:58.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.450+0000 7f1538587b80 10 client.15600 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:18:58.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.450+0000 7f1538587b80 10 client.15600 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:18:58.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.450+0000 7f1538587b80 20 client.15600 populate_metadata read hostname 'smithi052' 2023-11-27T15:18:58.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.450+0000 7f1538587b80 1 --2- 172.21.15.52:0/2310071153 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55acd1cef490 0x55acd1cf18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:58.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.450+0000 7f1538587b80 4 client.15600 mds_command: new command op to 24472 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:18:58.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.450+0000 7f1538587b80 1 -- 172.21.15.52:0/2310071153 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55acd1be90a0 con 0x55acd1cef490 2023-11-27T15:18:58.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.450+0000 7f1529996700 1 --2- 172.21.15.52:0/2310071153 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55acd1cef490 0x55acd1cf18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:58.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.450+0000 7f1529996700 1 --2- 172.21.15.52:0/2310071153 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55acd1cef490 0x55acd1cf18a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=37291820f93a986b server_cookie=5884b1ec7d68d542 in_seq=0 out_seq=0 2023-11-27T15:18:58.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.451+0000 7f1509ffb700 10 client.15600 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:18:58.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.451+0000 7f1509ffb700 1 -- 172.21.15.52:0/2310071153 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55acd1be90a0 con 0x55acd1cef490 2023-11-27T15:18:58.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.451+0000 7f1509ffb700 10 client.15600 handle_command_reply: tid=0 2023-11-27T15:18:58.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.451+0000 7f1538587b80 1 client.15600 shutdown 2023-11-27T15:18:58.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.452+0000 7f1538587b80 1 -- 172.21.15.52:0/2310071153 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55acd1cd5b60 msgr2=0x55acd1ce1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:58.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.452+0000 7f1538587b80 1 --2- 172.21.15.52:0/2310071153 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55acd1cd5b60 0x55acd1ce1f10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1514007ab0 tx=0x7f1514004b40 comp rx=0 tx=0).stop 2023-11-27T15:18:58.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.452+0000 7f1538587b80 1 -- 172.21.15.52:0/2310071153 shutdown_connections 2023-11-27T15:18:58.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.452+0000 7f1538587b80 1 --2- 172.21.15.52:0/2310071153 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55acd1cef490 0x55acd1cf18a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.452+0000 7f1538587b80 1 --2- 172.21.15.52:0/2310071153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acd1cda8b0 0x55acd1cdf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.452+0000 7f1538587b80 1 --2- 172.21.15.52:0/2310071153 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55acd1cd5b60 0x55acd1ce1f10 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.452+0000 7f1538587b80 1 --2- 172.21.15.52:0/2310071153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55acd1cd6470 0x55acd1cdf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.452+0000 7f1509ffb700 0 client.15600 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:18:58.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.452+0000 7f1538587b80 1 -- 172.21.15.52:0/2310071153 >> 172.21.15.52:0/2310071153 conn(0x55acd1ccb360 msgr2=0x55acd1ccfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:58.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.452+0000 7f1538587b80 1 -- 172.21.15.52:0/2310071153 shutdown_connections 2023-11-27T15:18:58.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.452+0000 7f1538587b80 1 -- 172.21.15.52:0/2310071153 wait complete. 2023-11-27T15:18:58.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.452+0000 7f1538587b80 20 client.15600 trim_cache size 0 max 16384 2023-11-27T15:18:58.483 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.466+0000 7f1538587b80 1 Processor -- start 2023-11-27T15:18:58.483 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.466+0000 7f1538587b80 1 -- start start 2023-11-27T15:18:58.484 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.467+0000 7f1538587b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55acd1cd5b60 0x55acd1ce67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:58.484 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.467+0000 7f1538587b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55acd1cd6470 0x55acd1cdcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:58.484 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.467+0000 7f1538587b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acd1cda8b0 0x55acd1ce1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:58.484 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.467+0000 7f1538587b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acd1cdba40 con 0x55acd1cd5b60 2023-11-27T15:18:58.484 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.467+0000 7f1538587b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acd1cecc50 con 0x55acd1cda8b0 2023-11-27T15:18:58.485 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.467+0000 7f1538587b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acd1cd17b0 con 0x55acd1cd6470 2023-11-27T15:18:58.485 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.467+0000 7f152a197700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55acd1cd5b60 0x55acd1ce67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:58.485 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.467+0000 7f152a197700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55acd1cd5b60 0x55acd1ce67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52498/0 (socket says 172.21.15.52:52498) 2023-11-27T15:18:58.485 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.467+0000 7f152a197700 1 -- 172.21.15.52:0/3889206231 learned_addr learned my addr 172.21.15.52:0/3889206231 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:58.486 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.467+0000 7f1529996700 1 --2- 172.21.15.52:0/3889206231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55acd1cd6470 0x55acd1cdcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:58.486 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.467+0000 7f152a998700 1 --2- 172.21.15.52:0/3889206231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acd1cda8b0 0x55acd1ce1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:58.486 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.467+0000 7f152a197700 1 -- 172.21.15.52:0/3889206231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55acd1cd6470 msgr2=0x55acd1cdcc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:58.486 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.467+0000 7f152a197700 1 --2- 172.21.15.52:0/3889206231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55acd1cd6470 0x55acd1cdcc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.487 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.467+0000 7f152a197700 1 -- 172.21.15.52:0/3889206231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acd1cda8b0 msgr2=0x55acd1ce1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:58.487 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.467+0000 7f152a197700 1 --2- 172.21.15.52:0/3889206231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acd1cda8b0 0x55acd1ce1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.487 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.467+0000 7f152a197700 1 -- 172.21.15.52:0/3889206231 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acd1cccb00 con 0x55acd1cd5b60 2023-11-27T15:18:58.487 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.467+0000 7f152a197700 1 --2- 172.21.15.52:0/3889206231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55acd1cd5b60 0x55acd1ce67b0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1514007a80 tx=0x7f1514003cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc273758f4ad248 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:58.488 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.468+0000 7f15215ad700 1 -- 172.21.15.52:0/3889206231 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15140163f0 con 0x55acd1cd5b60 2023-11-27T15:18:58.488 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.468+0000 7f15215ad700 1 -- 172.21.15.52:0/3889206231 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f151401a070 con 0x55acd1cd5b60 2023-11-27T15:18:58.488 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.468+0000 7f15215ad700 1 -- 172.21.15.52:0/3889206231 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151402b8e0 con 0x55acd1cd5b60 2023-11-27T15:18:58.488 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.468+0000 7f1538587b80 1 -- 172.21.15.52:0/3889206231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55acd1cd5b60 msgr2=0x55acd1ce67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:58.488 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.468+0000 7f1538587b80 1 --2- 172.21.15.52:0/3889206231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55acd1cd5b60 0x55acd1ce67b0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1514007a80 tx=0x7f1514003cf0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.468+0000 7f1538587b80 1 -- 172.21.15.52:0/3889206231 shutdown_connections 2023-11-27T15:18:58.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.468+0000 7f1538587b80 1 --2- 172.21.15.52:0/3889206231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acd1cda8b0 0x55acd1ce1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.468+0000 7f1538587b80 1 --2- 172.21.15.52:0/3889206231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55acd1cd5b60 0x55acd1ce67b0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.468+0000 7f1538587b80 1 --2- 172.21.15.52:0/3889206231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55acd1cd6470 0x55acd1cdcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.468+0000 7f1538587b80 1 -- 172.21.15.52:0/3889206231 >> 172.21.15.52:0/3889206231 conn(0x55acd1ccb360 msgr2=0x55acd1cdb330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:58.490 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.468+0000 7f1538587b80 1 -- 172.21.15.52:0/3889206231 shutdown_connections 2023-11-27T15:18:58.490 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.469+0000 7f1538587b80 1 -- 172.21.15.52:0/3889206231 wait complete. 2023-11-27T15:18:58.490 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.469+0000 7f1538587b80 1 -- start start 2023-11-27T15:18:58.490 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.469+0000 7f1538587b80 1 Processor -- start 2023-11-27T15:18:58.491 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.469+0000 7f1538587b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55acd1cd5b60 0x55acd1ce47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:58.491 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.469+0000 7f1538587b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55acd1cd6470 0x55acd1ce4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:58.491 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.469+0000 7f1538587b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acd1cda8b0 0x55acd1d42350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:58.491 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.469+0000 7f1538587b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acd1ce54f0 con 0x55acd1cd6470 2023-11-27T15:18:58.491 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.469+0000 7f1538587b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acd1ce5660 con 0x55acd1cda8b0 2023-11-27T15:18:58.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.469+0000 7f1538587b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acd1d42910 con 0x55acd1cd5b60 2023-11-27T15:18:58.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.469+0000 7f1538587b80 10 client.0 _ll_register_callbacks cb 0x7f1514004890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:18:58.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.469+0000 7f1529996700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55acd1cd6470 0x55acd1ce4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:58.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.470+0000 7f1529996700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55acd1cd6470 0x55acd1ce4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52506/0 (socket says 172.21.15.52:52506) 2023-11-27T15:18:58.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.470+0000 7f1529996700 1 -- 172.21.15.52:0/408928046 learned_addr learned my addr 172.21.15.52:0/408928046 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:18:58.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.470+0000 7f152a998700 1 --2- 172.21.15.52:0/408928046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acd1cda8b0 0x55acd1d42350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:58.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.470+0000 7f152a197700 1 --2- 172.21.15.52:0/408928046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55acd1cd5b60 0x55acd1ce47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:58.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.470+0000 7f1529996700 1 -- 172.21.15.52:0/408928046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55acd1cd5b60 msgr2=0x55acd1ce47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:58.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.470+0000 7f1529996700 1 --2- 172.21.15.52:0/408928046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55acd1cd5b60 0x55acd1ce47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.470+0000 7f1529996700 1 -- 172.21.15.52:0/408928046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acd1cda8b0 msgr2=0x55acd1d42350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:18:58.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.470+0000 7f1529996700 1 --2- 172.21.15.52:0/408928046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acd1cda8b0 0x55acd1d42350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.470+0000 7f1529996700 1 -- 172.21.15.52:0/408928046 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55acd1bed190 con 0x55acd1cd6470 2023-11-27T15:18:58.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.470+0000 7f1529996700 1 --2- 172.21.15.52:0/408928046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55acd1cd6470 0x55acd1ce4ce0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1518097670 tx=0x7f151800b5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74edf77b926f7c62 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:58.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.470+0000 7f1509ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:18:58.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.470+0000 7f1509ffb700 1 -- 172.21.15.52:0/408928046 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151809ac70 con 0x55acd1cd6470 2023-11-27T15:18:58.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.470+0000 7f1509ffb700 1 -- 172.21.15.52:0/408928046 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f1518005640 con 0x55acd1cd6470 2023-11-27T15:18:58.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.470+0000 7f1538587b80 1 -- 172.21.15.52:0/408928046 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f151809ee90 con 0x55acd1cd6470 2023-11-27T15:18:58.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.471+0000 7f1509ffb700 1 -- 172.21.15.52:0/408928046 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15180b3240 con 0x55acd1cd6470 2023-11-27T15:18:58.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.471+0000 7f1538587b80 10 client.15606 fetch_fsmap learned FSMap version 23 2023-11-27T15:18:58.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.471+0000 7f1538587b80 1 -- 172.21.15.52:0/408928046 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55acd1a54de0 con 0x55acd1cd6470 2023-11-27T15:18:58.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.471+0000 7f1509ffb700 1 -- 172.21.15.52:0/408928046 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f151809ee90 con 0x55acd1cd6470 2023-11-27T15:18:58.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.471+0000 7f1509ffb700 1 -- 172.21.15.52:0/408928046 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f15180a2b30 con 0x55acd1cd6470 2023-11-27T15:18:58.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.471+0000 7f1538587b80 10 client.15606 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:18:58.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.471+0000 7f1538587b80 10 client.15606 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:18:58.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.471+0000 7f1538587b80 20 client.15606 populate_metadata read hostname 'smithi052' 2023-11-27T15:18:58.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.471+0000 7f1538587b80 1 --2- 172.21.15.52:0/408928046 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55acd1d43d50 0x55acd1d46160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:58.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.471+0000 7f1538587b80 4 client.15606 mds_command: new command op to 24472 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:18:58.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.471+0000 7f1538587b80 1 -- 172.21.15.52:0/408928046 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55acd1bf5660 con 0x55acd1d43d50 2023-11-27T15:18:58.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.471+0000 7f152a197700 1 --2- 172.21.15.52:0/408928046 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55acd1d43d50 0x55acd1d46160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:58.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.472+0000 7f152a197700 1 --2- 172.21.15.52:0/408928046 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55acd1d43d50 0x55acd1d46160 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f03e8dbd74a0b9d8 server_cookie=54671bda290ff830 in_seq=0 out_seq=0 2023-11-27T15:18:58.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.472+0000 7f1509ffb700 10 client.15606 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:18:58.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.472+0000 7f1509ffb700 1 -- 172.21.15.52:0/408928046 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55acd1bf5660 con 0x55acd1d43d50 2023-11-27T15:18:58.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.472+0000 7f1509ffb700 10 client.15606 handle_command_reply: tid=0 2023-11-27T15:18:58.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.472+0000 7f1538587b80 1 client.15606 shutdown 2023-11-27T15:18:58.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.472+0000 7f1538587b80 1 -- 172.21.15.52:0/408928046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55acd1cd6470 msgr2=0x55acd1ce4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:58.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.472+0000 7f1538587b80 1 --2- 172.21.15.52:0/408928046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55acd1cd6470 0x55acd1ce4ce0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1518097670 tx=0x7f151800b5f0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.472+0000 7f1538587b80 1 -- 172.21.15.52:0/408928046 shutdown_connections 2023-11-27T15:18:58.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.472+0000 7f1538587b80 1 --2- 172.21.15.52:0/408928046 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55acd1d43d50 0x55acd1d46160 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.472+0000 7f1538587b80 1 --2- 172.21.15.52:0/408928046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55acd1cda8b0 0x55acd1d42350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.472+0000 7f1538587b80 1 --2- 172.21.15.52:0/408928046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55acd1cd5b60 0x55acd1ce47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.472+0000 7f1509ffb700 0 client.15606 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:18:58.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.472+0000 7f1538587b80 1 --2- 172.21.15.52:0/408928046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55acd1cd6470 0x55acd1ce4ce0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.472+0000 7f1538587b80 1 -- 172.21.15.52:0/408928046 >> 172.21.15.52:0/408928046 conn(0x55acd1ccb360 msgr2=0x55acd1cd4960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:58.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.472+0000 7f1538587b80 1 -- 172.21.15.52:0/408928046 shutdown_connections 2023-11-27T15:18:58.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.472+0000 7f1538587b80 1 -- 172.21.15.52:0/408928046 wait complete. 2023-11-27T15:18:58.501 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.472+0000 7f1538587b80 20 client.15606 trim_cache size 0 max 16384 2023-11-27T15:18:58.501 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:18:58.502 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.473+0000 7f152c3fb700 1 -- 172.21.15.52:0/2583531768 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f150c0a13e0 msgr2=0x7f150c0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:58.502 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.473+0000 7f152c3fb700 1 --2- 172.21.15.52:0/2583531768 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f150c0a13e0 0x7f150c0a3890 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1514009550 tx=0x7f15140091b0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.502 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.473+0000 7f152c3fb700 1 -- 172.21.15.52:0/2583531768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1524101ca0 msgr2=0x7f1524196950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:58.502 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.473+0000 7f152c3fb700 1 --2- 172.21.15.52:0/2583531768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1524101ca0 0x7f1524196950 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f15180098e0 tx=0x7f151800f670 comp rx=0 tx=0).stop 2023-11-27T15:18:58.503 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.473+0000 7f152c3fb700 1 -- 172.21.15.52:0/2583531768 shutdown_connections 2023-11-27T15:18:58.503 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.473+0000 7f152c3fb700 1 --2- 172.21.15.52:0/2583531768 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f150c0a13e0 0x7f150c0a3890 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.503 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.473+0000 7f152c3fb700 1 --2- 172.21.15.52:0/2583531768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f152410a8b0 0x7f152419adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.503 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.473+0000 7f152c3fb700 1 --2- 172.21.15.52:0/2583531768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1524101390 0x7f1524196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.503 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.473+0000 7f152c3fb700 1 --2- 172.21.15.52:0/2583531768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1524101ca0 0x7f1524196950 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:58.504 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.473+0000 7f152c3fb700 1 -- 172.21.15.52:0/2583531768 >> 172.21.15.52:0/2583531768 conn(0x7f15240770b0 msgr2=0x7f15240fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:58.504 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.473+0000 7f152c3fb700 1 -- 172.21.15.52:0/2583531768 shutdown_connections 2023-11-27T15:18:58.504 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:18:58.473+0000 7f152c3fb700 1 -- 172.21.15.52:0/2583531768 wait complete. 2023-11-27T15:18:58.564 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-27T15:18:58.740 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:58.741 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:58.851 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:18:58.851 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:18:58.866 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:18:58 smithi052 ceph-mon[99056]: pgmap v34: 97 pgs: 97 active+clean; 1.5 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 9 op/s 2023-11-27T15:18:58.873 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:58.873 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:58.873 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 24 -lt 100 ']' 2023-11-27T15:18:58.873 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 24 + 1 2023-11-27T15:18:58.922 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=25 2023-11-27T15:18:58.922 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:18:58.933 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:58.933 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:58.934 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:18:58.934 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:18:58.935 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:18:59.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:18:58 smithi102 ceph-mon[100296]: pgmap v34: 97 pgs: 97 active+clean; 1.5 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 9 op/s 2023-11-27T15:18:59.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:18:58 smithi150 ceph-mon[100559]: pgmap v34: 97 pgs: 97 active+clean; 1.5 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 9 op/s 2023-11-27T15:18:59.290 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:59.291 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:59.327 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:18:59.327 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:18:59.332 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:59.332 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:59.332 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 25 -lt 100 ']' 2023-11-27T15:18:59.332 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 25 + 1 2023-11-27T15:18:59.367 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=26 2023-11-27T15:18:59.367 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:18:59.407 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:59.407 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:59.408 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:18:59.409 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:18:59.409 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:18:59.749 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:18:59.749 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:18:59.925 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:18:59.925 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:18:59.938 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:59.938 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:18:59.939 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 26 -lt 100 ']' 2023-11-27T15:18:59.939 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 26 + 1 2023-11-27T15:18:59.971 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=27 2023-11-27T15:18:59.972 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:00.019 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:00.019 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:00.020 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:00.021 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:00.021 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:00.815 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:00.816 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:00.984 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:00.984 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:01.014 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:01.015 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:01.015 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 27 -lt 100 ']' 2023-11-27T15:19:01.015 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 27 + 1 2023-11-27T15:19:01.041 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=28 2023-11-27T15:19:01.042 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:01.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:00 smithi102 ceph-mon[100296]: pgmap v35: 97 pgs: 97 active+clean; 1.5 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 6 op/s 2023-11-27T15:19:01.092 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:01.093 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:01.093 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:01.094 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:01.094 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:01.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:00 smithi052 ceph-mon[99056]: pgmap v35: 97 pgs: 97 active+clean; 1.5 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 6 op/s 2023-11-27T15:19:01.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:00 smithi150 ceph-mon[100559]: pgmap v35: 97 pgs: 97 active+clean; 1.5 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 6 op/s 2023-11-27T15:19:01.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.457+0000 7fdecf231700 1 -- 172.21.15.52:0/2406671684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdec80ff450 msgr2=0x7fdec80ff820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:01.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.457+0000 7fdecf231700 1 --2- 172.21.15.52:0/2406671684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdec80ff450 0x7fdec80ff820 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fdebc0099c0 tx=0x7fdebc009cd0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.457+0000 7fdecf231700 1 -- 172.21.15.52:0/2406671684 shutdown_connections 2023-11-27T15:19:01.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.457+0000 7fdecf231700 1 --2- 172.21.15.52:0/2406671684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdec8104490 0x7fdec810c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.457+0000 7fdecf231700 1 --2- 172.21.15.52:0/2406671684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdec80ff450 0x7fdec80ff820 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.457+0000 7fdecf231700 1 --2- 172.21.15.52:0/2406671684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdec80ffdf0 0x7fdec8103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.457+0000 7fdecf231700 1 -- 172.21.15.52:0/2406671684 >> 172.21.15.52:0/2406671684 conn(0x7fdec8077080 msgr2=0x7fdec8077480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:01.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.457+0000 7fdecf231700 1 -- 172.21.15.52:0/2406671684 shutdown_connections 2023-11-27T15:19:01.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.457+0000 7fdecf231700 1 -- 172.21.15.52:0/2406671684 wait complete. 2023-11-27T15:19:01.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.458+0000 7fdecf231700 1 Processor -- start 2023-11-27T15:19:01.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.458+0000 7fdecf231700 1 -- start start 2023-11-27T15:19:01.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.458+0000 7fdecf231700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdec80ff450 0x7fdec8100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:01.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.458+0000 7fdecf231700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdec80ffdf0 0x7fdec8101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:01.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.458+0000 7fdecf231700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdec8104490 0x7fdec81a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:01.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.458+0000 7fdecf231700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec8101bc0 con 0x7fdec8104490 2023-11-27T15:19:01.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.458+0000 7fdecf231700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec81a4ba0 con 0x7fdec80ff450 2023-11-27T15:19:01.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.458+0000 7fdecf231700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec81a4ce0 con 0x7fdec80ffdf0 2023-11-27T15:19:01.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.458+0000 7fdecd7ce700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdec8104490 0x7fdec81a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:01.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.459+0000 7fdecd7ce700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdec8104490 0x7fdec81a4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52522/0 (socket says 172.21.15.52:52522) 2023-11-27T15:19:01.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.459+0000 7fdecd7ce700 1 -- 172.21.15.52:0/3544271378 learned_addr learned my addr 172.21.15.52:0/3544271378 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:01.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.459+0000 7fdec7fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdec80ffdf0 0x7fdec8101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:01.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.459+0000 7fdeccfcd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdec80ff450 0x7fdec8100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:01.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.459+0000 7fdecd7ce700 1 -- 172.21.15.52:0/3544271378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdec80ffdf0 msgr2=0x7fdec8101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:01.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.459+0000 7fdecd7ce700 1 --2- 172.21.15.52:0/3544271378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdec80ffdf0 0x7fdec8101390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.459+0000 7fdecd7ce700 1 -- 172.21.15.52:0/3544271378 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdec80ff450 msgr2=0x7fdec8100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:01.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.459+0000 7fdecd7ce700 1 --2- 172.21.15.52:0/3544271378 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdec80ff450 0x7fdec8100e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.459+0000 7fdecd7ce700 1 -- 172.21.15.52:0/3544271378 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdebc0095f0 con 0x7fdec8104490 2023-11-27T15:19:01.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.459+0000 7fdec7fff700 1 --2- 172.21.15.52:0/3544271378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdec80ffdf0 0x7fdec8101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:01.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.459+0000 7fdecd7ce700 1 --2- 172.21.15.52:0/3544271378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdec8104490 0x7fdec81a4660 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fdec000ef30 tx=0x7fdec000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:01.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.459+0000 7fdec5ffb700 1 -- 172.21.15.52:0/3544271378 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec0018070 con 0x7fdec8104490 2023-11-27T15:19:01.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.459+0000 7fdec5ffb700 1 -- 172.21.15.52:0/3544271378 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdec000f040 con 0x7fdec8104490 2023-11-27T15:19:01.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.459+0000 7fdecf231700 1 -- 172.21.15.52:0/3544271378 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec810f250 con 0x7fdec8104490 2023-11-27T15:19:01.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.459+0000 7fdec5ffb700 1 -- 172.21.15.52:0/3544271378 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec000cd00 con 0x7fdec8104490 2023-11-27T15:19:01.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.460+0000 7fdecf231700 1 -- 172.21.15.52:0/3544271378 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec810f0c0 con 0x7fdec8104490 2023-11-27T15:19:01.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.461+0000 7fdec5ffb700 1 -- 172.21.15.52:0/3544271378 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7fdec0003a00 con 0x7fdec8104490 2023-11-27T15:19:01.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.462+0000 7fdec5ffb700 1 --2- 172.21.15.52:0/3544271378 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fdeb80a1460 0x7fdeb80a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:01.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.462+0000 7fdeccfcd700 1 --2- 172.21.15.52:0/3544271378 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fdeb80a1460 0x7fdeb80a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:01.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.462+0000 7fdec5ffb700 1 -- 172.21.15.52:0/3544271378 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fdec00cc9e0 con 0x7fdec8104490 2023-11-27T15:19:01.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.462+0000 7fdeccfcd700 1 --2- 172.21.15.52:0/3544271378 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fdeb80a1460 0x7fdeb80a3910 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdebc009990 tx=0x7fdebc005fb0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:01.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.536+0000 7fdedb3bdb80 1 Processor -- start 2023-11-27T15:19:01.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.536+0000 7fdedb3bdb80 1 -- start start 2023-11-27T15:19:01.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.536+0000 7fdedb3bdb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04363b60 0x560d04363f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:01.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.536+0000 7fdedb3bdb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d04364470 0x560d04368370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:01.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.536+0000 7fdedb3bdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d043688b0 0x560d04368d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:01.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.536+0000 7fdedb3bdb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d04269060 con 0x560d04363b60 2023-11-27T15:19:01.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.536+0000 7fdedb3bdb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d042bb8a0 con 0x560d04364470 2023-11-27T15:19:01.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.536+0000 7fdedb3bdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d042b4d30 con 0x560d043688b0 2023-11-27T15:19:01.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.536+0000 7fdeccfcd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04363b60 0x560d04363f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:01.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.536+0000 7fdeccfcd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04363b60 0x560d04363f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52526/0 (socket says 172.21.15.52:52526) 2023-11-27T15:19:01.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.536+0000 7fdeccfcd700 1 -- 172.21.15.52:0/2977146796 learned_addr learned my addr 172.21.15.52:0/2977146796 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:01.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.536+0000 7fdec7fff700 1 --2- 172.21.15.52:0/2977146796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d04364470 0x560d04368370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:01.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.536+0000 7fdecd7ce700 1 --2- 172.21.15.52:0/2977146796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d043688b0 0x560d04368d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:01.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.537+0000 7fdec7fff700 1 -- 172.21.15.52:0/2977146796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d043688b0 msgr2=0x560d04368d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:01.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.537+0000 7fdec7fff700 1 --2- 172.21.15.52:0/2977146796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d043688b0 0x560d04368d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.537+0000 7fdec7fff700 1 -- 172.21.15.52:0/2977146796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04363b60 msgr2=0x560d04363f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:01.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.537+0000 7fdec7fff700 1 --2- 172.21.15.52:0/2977146796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04363b60 0x560d04363f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.537+0000 7fdec7fff700 1 -- 172.21.15.52:0/2977146796 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d04136630 con 0x560d04364470 2023-11-27T15:19:01.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.537+0000 7fdeccfcd700 1 --2- 172.21.15.52:0/2977146796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04363b60 0x560d04363f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:01.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.537+0000 7fdec7fff700 1 --2- 172.21.15.52:0/2977146796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d04364470 0x560d04368370 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4005950 tx=0x7fdeb4010730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c1b1db0000a108b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:01.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.537+0000 7fdeb2ffd700 1 -- 172.21.15.52:0/2977146796 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb401c480 con 0x560d04364470 2023-11-27T15:19:01.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.537+0000 7fdeb2ffd700 1 -- 172.21.15.52:0/2977146796 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdeb4015070 con 0x560d04364470 2023-11-27T15:19:01.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.537+0000 7fdeb2ffd700 1 -- 172.21.15.52:0/2977146796 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4011910 con 0x560d04364470 2023-11-27T15:19:01.549 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.538+0000 7fdedb3bdb80 1 -- 172.21.15.52:0/2977146796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d04364470 msgr2=0x560d04368370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:01.550 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.538+0000 7fdedb3bdb80 1 --2- 172.21.15.52:0/2977146796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d04364470 0x560d04368370 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4005950 tx=0x7fdeb4010730 comp rx=0 tx=0).stop 2023-11-27T15:19:01.550 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.538+0000 7fdedb3bdb80 1 -- 172.21.15.52:0/2977146796 shutdown_connections 2023-11-27T15:19:01.550 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.538+0000 7fdedb3bdb80 1 --2- 172.21.15.52:0/2977146796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d043688b0 0x560d04368d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.550 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.538+0000 7fdedb3bdb80 1 --2- 172.21.15.52:0/2977146796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04363b60 0x560d04363f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.550 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.538+0000 7fdedb3bdb80 1 --2- 172.21.15.52:0/2977146796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d04364470 0x560d04368370 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.550 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.538+0000 7fdedb3bdb80 1 -- 172.21.15.52:0/2977146796 >> 172.21.15.52:0/2977146796 conn(0x560d04359360 msgr2=0x560d0435eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:01.550 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.538+0000 7fdedb3bdb80 1 -- 172.21.15.52:0/2977146796 shutdown_connections 2023-11-27T15:19:01.550 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.538+0000 7fdedb3bdb80 1 -- 172.21.15.52:0/2977146796 wait complete. 2023-11-27T15:19:01.551 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.539+0000 7fdedb3bdb80 1 -- start start 2023-11-27T15:19:01.551 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.539+0000 7fdedb3bdb80 1 Processor -- start 2023-11-27T15:19:01.551 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.539+0000 7fdedb3bdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d04363b60 0x560d0436ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:01.551 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.539+0000 7fdedb3bdb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d04364470 0x560d0436d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:01.551 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.539+0000 7fdedb3bdb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d043688b0 0x560d0436d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:01.551 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.539+0000 7fdedb3bdb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d042568e0 con 0x560d043688b0 2023-11-27T15:19:01.551 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.539+0000 7fdedb3bdb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d04256580 con 0x560d04364470 2023-11-27T15:19:01.551 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.539+0000 7fdedb3bdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d04138920 con 0x560d04363b60 2023-11-27T15:19:01.551 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.540+0000 7fdedb3bdb80 10 client.0 _ll_register_callbacks cb 0x560d04274780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:19:01.552 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.540+0000 7fdecd7ce700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d043688b0 0x560d0436d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:01.552 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.540+0000 7fdeccfcd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d04363b60 0x560d0436ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:01.552 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.540+0000 7fdeccfcd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d04363b60 0x560d0436ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:39176/0 (socket says 172.21.15.52:39176) 2023-11-27T15:19:01.552 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.540+0000 7fdecd7ce700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d043688b0 0x560d0436d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52536/0 (socket says 172.21.15.52:52536) 2023-11-27T15:19:01.552 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.540+0000 7fdeccfcd700 1 -- 172.21.15.52:0/4179490645 learned_addr learned my addr 172.21.15.52:0/4179490645 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:01.552 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.540+0000 7fdec7fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d04364470 0x560d0436d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:01.552 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.540+0000 7fdec7fff700 1 -- 172.21.15.52:0/4179490645 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d04363b60 msgr2=0x560d0436ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:01.552 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.540+0000 7fdec7fff700 1 --2- 172.21.15.52:0/4179490645 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d04363b60 0x560d0436ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.540+0000 7fdec7fff700 1 -- 172.21.15.52:0/4179490645 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d043688b0 msgr2=0x560d0436d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:01.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.540+0000 7fdec7fff700 1 --2- 172.21.15.52:0/4179490645 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d043688b0 0x560d0436d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.540+0000 7fdec7fff700 1 -- 172.21.15.52:0/4179490645 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb400bcd0 con 0x560d04364470 2023-11-27T15:19:01.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.540+0000 7fdec7fff700 1 --2- 172.21.15.52:0/4179490645 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d04364470 0x560d0436d1f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4000c00 tx=0x7fdeb40050c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f7dd43e7dfcbe1d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:01.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.540+0000 7fdea7fff700 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2023-11-27T15:19:01.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.540+0000 7fdea7fff700 1 -- 172.21.15.52:0/4179490645 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4020070 con 0x560d04364470 2023-11-27T15:19:01.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.540+0000 7fdea7fff700 1 -- 172.21.15.52:0/4179490645 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fdeb4017070 con 0x560d04364470 2023-11-27T15:19:01.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.540+0000 7fdedb3bdb80 1 -- 172.21.15.52:0/4179490645 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560d04356b20 con 0x560d04364470 2023-11-27T15:19:01.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.541+0000 7fdea7fff700 1 -- 172.21.15.52:0/4179490645 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdeb4005580 con 0x560d04364470 2023-11-27T15:19:01.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.541+0000 7fdedb3bdb80 10 client.25433 fetch_fsmap learned FSMap version 23 2023-11-27T15:19:01.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.541+0000 7fdedb3bdb80 1 -- 172.21.15.52:0/4179490645 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560d0427b190 con 0x560d04364470 2023-11-27T15:19:01.554 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.541+0000 7fdea7fff700 1 -- 172.21.15.52:0/4179490645 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560d04356b20 con 0x560d04364470 2023-11-27T15:19:01.554 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.541+0000 7fdea7fff700 1 -- 172.21.15.52:0/4179490645 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdeb401c7e0 con 0x560d04364470 2023-11-27T15:19:01.554 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.541+0000 7fdedb3bdb80 10 client.25433 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:19:01.554 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.541+0000 7fdedb3bdb80 10 client.25433 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:19:01.554 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.541+0000 7fdedb3bdb80 20 client.25433 populate_metadata read hostname 'smithi052' 2023-11-27T15:19:01.554 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.541+0000 7fdedb3bdb80 1 --2- 172.21.15.52:0/4179490645 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x560d0437d6f0 0x560d0437fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:01.554 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.541+0000 7fdedb3bdb80 4 client.25433 mds_command: new command op to 24494 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:19:01.554 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.541+0000 7fdedb3bdb80 1 -- 172.21.15.52:0/4179490645 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560d042770a0 con 0x560d0437d6f0 2023-11-27T15:19:01.554 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.541+0000 7fdeccfcd700 1 --2- 172.21.15.52:0/4179490645 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x560d0437d6f0 0x560d0437fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:01.555 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.542+0000 7fdea7fff700 10 client.25433 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:19:01.555 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.542+0000 7fdeccfcd700 1 --2- 172.21.15.52:0/4179490645 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x560d0437d6f0 0x560d0437fb00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=55ca42814c80dc88 server_cookie=e65c383f47743da3 in_seq=0 out_seq=0 2023-11-27T15:19:01.555 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.542+0000 7fdea7fff700 1 -- 172.21.15.52:0/4179490645 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560d042770a0 con 0x560d0437d6f0 2023-11-27T15:19:01.555 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.542+0000 7fdea7fff700 10 client.25433 handle_command_reply: tid=0 2023-11-27T15:19:01.555 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.542+0000 7fdedb3bdb80 1 client.25433 shutdown 2023-11-27T15:19:01.555 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.542+0000 7fdedb3bdb80 1 -- 172.21.15.52:0/4179490645 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d04364470 msgr2=0x560d0436d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:01.555 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.542+0000 7fdedb3bdb80 1 --2- 172.21.15.52:0/4179490645 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d04364470 0x560d0436d1f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4000c00 tx=0x7fdeb40050c0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.555 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.542+0000 7fdedb3bdb80 1 -- 172.21.15.52:0/4179490645 shutdown_connections 2023-11-27T15:19:01.555 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.542+0000 7fdedb3bdb80 1 --2- 172.21.15.52:0/4179490645 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x560d0437d6f0 0x560d0437fb00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.556 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.542+0000 7fdedb3bdb80 1 --2- 172.21.15.52:0/4179490645 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d043688b0 0x560d0436d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.556 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.542+0000 7fdedb3bdb80 1 --2- 172.21.15.52:0/4179490645 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d04363b60 0x560d0436ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.556 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.542+0000 7fdedb3bdb80 1 --2- 172.21.15.52:0/4179490645 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d04364470 0x560d0436d1f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.556 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.542+0000 7fdea7fff700 0 client.25433 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:19:01.556 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.542+0000 7fdedb3bdb80 1 -- 172.21.15.52:0/4179490645 >> 172.21.15.52:0/4179490645 conn(0x560d04359360 msgr2=0x560d04369d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:01.556 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.543+0000 7fdedb3bdb80 1 -- 172.21.15.52:0/4179490645 shutdown_connections 2023-11-27T15:19:01.556 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.543+0000 7fdedb3bdb80 1 -- 172.21.15.52:0/4179490645 wait complete. 2023-11-27T15:19:01.556 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.543+0000 7fdedb3bdb80 20 client.25433 trim_cache size 0 max 16384 2023-11-27T15:19:01.568 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.557+0000 7fdedb3bdb80 1 Processor -- start 2023-11-27T15:19:01.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.557+0000 7fdedb3bdb80 1 -- start start 2023-11-27T15:19:01.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.557+0000 7fdedb3bdb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d04363b60 0x560d043749a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:01.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.557+0000 7fdedb3bdb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04364470 0x560d04379b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:01.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.557+0000 7fdedb3bdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d043688b0 0x560d0436fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:01.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.557+0000 7fdedb3bdb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d04358ee0 con 0x560d04364470 2023-11-27T15:19:01.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.557+0000 7fdedb3bdb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0436b770 con 0x560d04363b60 2023-11-27T15:19:01.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.557+0000 7fdedb3bdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d04364840 con 0x560d043688b0 2023-11-27T15:19:01.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.558+0000 7fdec7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04364470 0x560d04379b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:01.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.558+0000 7fdec7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04364470 0x560d04379b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52542/0 (socket says 172.21.15.52:52542) 2023-11-27T15:19:01.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.558+0000 7fdec7fff700 1 -- 172.21.15.52:0/1279011882 learned_addr learned my addr 172.21.15.52:0/1279011882 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:01.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.558+0000 7fdeccfcd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d04363b60 0x560d043749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:01.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.558+0000 7fdecd7ce700 1 --2- 172.21.15.52:0/1279011882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d043688b0 0x560d0436fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:01.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.558+0000 7fdec7fff700 1 -- 172.21.15.52:0/1279011882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d043688b0 msgr2=0x560d0436fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:01.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.558+0000 7fdec7fff700 1 --2- 172.21.15.52:0/1279011882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d043688b0 0x560d0436fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.558+0000 7fdec7fff700 1 -- 172.21.15.52:0/1279011882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d04363b60 msgr2=0x560d043749a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:01.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.558+0000 7fdec7fff700 1 --2- 172.21.15.52:0/1279011882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d04363b60 0x560d043749a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.558+0000 7fdec7fff700 1 -- 172.21.15.52:0/1279011882 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d0436cd60 con 0x560d04364470 2023-11-27T15:19:01.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.558+0000 7fdecd7ce700 1 --2- 172.21.15.52:0/1279011882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d043688b0 0x560d0436fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:19:01.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.558+0000 7fdec7fff700 1 --2- 172.21.15.52:0/1279011882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04364470 0x560d04379b20 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40051b0 tx=0x7fdeb40051e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7862ea9f6d4600 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:01.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.558+0000 7fdeb37fe700 1 -- 172.21.15.52:0/1279011882 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4020070 con 0x560d04364470 2023-11-27T15:19:01.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.558+0000 7fdeb37fe700 1 -- 172.21.15.52:0/1279011882 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdeb401ccc0 con 0x560d04364470 2023-11-27T15:19:01.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.558+0000 7fdeb37fe700 1 -- 172.21.15.52:0/1279011882 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb40237d0 con 0x560d04364470 2023-11-27T15:19:01.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.559+0000 7fdedb3bdb80 1 -- 172.21.15.52:0/1279011882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04364470 msgr2=0x560d04379b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:01.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.559+0000 7fdedb3bdb80 1 --2- 172.21.15.52:0/1279011882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04364470 0x560d04379b20 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40051b0 tx=0x7fdeb40051e0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.559+0000 7fdedb3bdb80 1 -- 172.21.15.52:0/1279011882 shutdown_connections 2023-11-27T15:19:01.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.559+0000 7fdedb3bdb80 1 --2- 172.21.15.52:0/1279011882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d04363b60 0x560d043749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.559+0000 7fdedb3bdb80 1 --2- 172.21.15.52:0/1279011882 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d043688b0 0x560d0436fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.559+0000 7fdedb3bdb80 1 --2- 172.21.15.52:0/1279011882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04364470 0x560d04379b20 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.559+0000 7fdedb3bdb80 1 -- 172.21.15.52:0/1279011882 >> 172.21.15.52:0/1279011882 conn(0x560d04359360 msgr2=0x560d0435d550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:01.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.559+0000 7fdedb3bdb80 1 -- 172.21.15.52:0/1279011882 shutdown_connections 2023-11-27T15:19:01.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.559+0000 7fdedb3bdb80 1 -- 172.21.15.52:0/1279011882 wait complete. 2023-11-27T15:19:01.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.559+0000 7fdedb3bdb80 1 -- start start 2023-11-27T15:19:01.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.560+0000 7fdedb3bdb80 1 Processor -- start 2023-11-27T15:19:01.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.560+0000 7fdedb3bdb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04363b60 0x560d04372990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:01.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.560+0000 7fdedb3bdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d04364470 0x560d04372ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:01.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.560+0000 7fdedb3bdb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d043688b0 0x560d043d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:01.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.560+0000 7fdedb3bdb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d043737c0 con 0x560d04363b60 2023-11-27T15:19:01.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.560+0000 7fdedb3bdb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d043d0af0 con 0x560d043688b0 2023-11-27T15:19:01.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.560+0000 7fdedb3bdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d043d0c30 con 0x560d04364470 2023-11-27T15:19:01.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.560+0000 7fdedb3bdb80 10 client.0 _ll_register_callbacks cb 0x7fdebc0037f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:19:01.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.560+0000 7fdeccfcd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04363b60 0x560d04372990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:01.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.560+0000 7fdec7fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d04364470 0x560d04372ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:01.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.560+0000 7fdecd7ce700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d043688b0 0x560d043d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:01.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.560+0000 7fdeccfcd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04363b60 0x560d04372990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52546/0 (socket says 172.21.15.52:52546) 2023-11-27T15:19:01.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.560+0000 7fdec7fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d04364470 0x560d04372ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:39192/0 (socket says 172.21.15.52:39192) 2023-11-27T15:19:01.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.560+0000 7fdecd7ce700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d043688b0 0x560d043d05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:42208/0 (socket says 172.21.15.52:42208) 2023-11-27T15:19:01.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.560+0000 7fdeccfcd700 1 -- 172.21.15.52:0/3556778138 learned_addr learned my addr 172.21.15.52:0/3556778138 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:01.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.560+0000 7fdecd7ce700 1 -- 172.21.15.52:0/3556778138 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d04364470 msgr2=0x560d04372ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:01.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.560+0000 7fdecd7ce700 1 --2- 172.21.15.52:0/3556778138 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d04364470 0x560d04372ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.560+0000 7fdecd7ce700 1 -- 172.21.15.52:0/3556778138 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04363b60 msgr2=0x560d04372990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:01.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.560+0000 7fdecd7ce700 1 --2- 172.21.15.52:0/3556778138 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04363b60 0x560d04372990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.561+0000 7fdecd7ce700 1 -- 172.21.15.52:0/3556778138 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560d0427b190 con 0x560d043688b0 2023-11-27T15:19:01.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.561+0000 7fdeccfcd700 1 --2- 172.21.15.52:0/3556778138 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04363b60 0x560d04372990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:01.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.561+0000 7fdecd7ce700 1 --2- 172.21.15.52:0/3556778138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d043688b0 0x560d043d05b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdec00c9fb0 tx=0x7fdec0099270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c51468cd698a8ce6 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:01.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.561+0000 7fdea7fff700 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2023-11-27T15:19:01.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.561+0000 7fdea7fff700 1 -- 172.21.15.52:0/3556778138 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec000ce60 con 0x560d043688b0 2023-11-27T15:19:01.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.561+0000 7fdea7fff700 1 -- 172.21.15.52:0/3556778138 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fdec00998f0 con 0x560d043688b0 2023-11-27T15:19:01.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.561+0000 7fdedb3bdb80 1 -- 172.21.15.52:0/3556778138 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdeb4015070 con 0x560d043688b0 2023-11-27T15:19:01.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.561+0000 7fdea7fff700 1 -- 172.21.15.52:0/3556778138 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdec009d920 con 0x560d043688b0 2023-11-27T15:19:01.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.561+0000 7fdedb3bdb80 10 client.25439 fetch_fsmap learned FSMap version 23 2023-11-27T15:19:01.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.561+0000 7fdedb3bdb80 1 -- 172.21.15.52:0/3556778138 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560d040e2de0 con 0x560d043688b0 2023-11-27T15:19:01.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.561+0000 7fdea7fff700 1 -- 172.21.15.52:0/3556778138 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdeb4015070 con 0x560d043688b0 2023-11-27T15:19:01.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.562+0000 7fdea7fff700 1 -- 172.21.15.52:0/3556778138 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdec00b9140 con 0x560d043688b0 2023-11-27T15:19:01.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.562+0000 7fdedb3bdb80 10 client.25439 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:19:01.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.562+0000 7fdedb3bdb80 10 client.25439 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:19:01.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.562+0000 7fdedb3bdb80 20 client.25439 populate_metadata read hostname 'smithi052' 2023-11-27T15:19:01.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.562+0000 7fdedb3bdb80 1 --2- 172.21.15.52:0/3556778138 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x560d043d2250 0x560d043d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:01.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.562+0000 7fdedb3bdb80 4 client.25439 mds_command: new command op to 24494 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:19:01.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.562+0000 7fdedb3bdb80 1 -- 172.21.15.52:0/3556778138 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560d04283660 con 0x560d043d2250 2023-11-27T15:19:01.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.562+0000 7fdeccfcd700 1 --2- 172.21.15.52:0/3556778138 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x560d043d2250 0x560d043d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:01.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.562+0000 7fdea7fff700 10 client.25439 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:19:01.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.562+0000 7fdeccfcd700 1 --2- 172.21.15.52:0/3556778138 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x560d043d2250 0x560d043d4660 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c758ff5baa68ffeb server_cookie=d100f12965b1595f in_seq=0 out_seq=0 2023-11-27T15:19:01.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.563+0000 7fdea7fff700 1 -- 172.21.15.52:0/3556778138 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560d04283660 con 0x560d043d2250 2023-11-27T15:19:01.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.563+0000 7fdea7fff700 10 client.25439 handle_command_reply: tid=0 2023-11-27T15:19:01.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.563+0000 7fdedb3bdb80 1 client.25439 shutdown 2023-11-27T15:19:01.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.563+0000 7fdedb3bdb80 1 -- 172.21.15.52:0/3556778138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d043688b0 msgr2=0x560d043d05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:01.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.563+0000 7fdedb3bdb80 1 --2- 172.21.15.52:0/3556778138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d043688b0 0x560d043d05b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdec00c9fb0 tx=0x7fdec0099270 comp rx=0 tx=0).stop 2023-11-27T15:19:01.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.563+0000 7fdedb3bdb80 1 -- 172.21.15.52:0/3556778138 shutdown_connections 2023-11-27T15:19:01.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.563+0000 7fdedb3bdb80 1 --2- 172.21.15.52:0/3556778138 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x560d043d2250 0x560d043d4660 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.563+0000 7fdedb3bdb80 1 --2- 172.21.15.52:0/3556778138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d043688b0 0x560d043d05b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.563+0000 7fdea7fff700 0 client.25439 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:19:01.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.563+0000 7fdedb3bdb80 1 --2- 172.21.15.52:0/3556778138 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04363b60 0x560d04372990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.563+0000 7fdedb3bdb80 1 --2- 172.21.15.52:0/3556778138 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d04364470 0x560d04372ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.563+0000 7fdedb3bdb80 1 -- 172.21.15.52:0/3556778138 >> 172.21.15.52:0/3556778138 conn(0x560d04359360 msgr2=0x560d04379b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:01.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.563+0000 7fdedb3bdb80 1 -- 172.21.15.52:0/3556778138 shutdown_connections 2023-11-27T15:19:01.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.563+0000 7fdedb3bdb80 1 -- 172.21.15.52:0/3556778138 wait complete. 2023-11-27T15:19:01.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.563+0000 7fdedb3bdb80 20 client.25439 trim_cache size 0 max 16384 2023-11-27T15:19:01.578 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:19:01.579 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.563+0000 7fdecf231700 1 -- 172.21.15.52:0/3544271378 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fdeb80a1460 msgr2=0x7fdeb80a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:01.579 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.563+0000 7fdecf231700 1 --2- 172.21.15.52:0/3544271378 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fdeb80a1460 0x7fdeb80a3910 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdebc009990 tx=0x7fdebc005fb0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.579 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.564+0000 7fdecf231700 1 -- 172.21.15.52:0/3544271378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdec8104490 msgr2=0x7fdec81a4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:01.579 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.564+0000 7fdecf231700 1 --2- 172.21.15.52:0/3544271378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdec8104490 0x7fdec81a4660 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fdec000ef30 tx=0x7fdec000c3a0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.579 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.564+0000 7fdecf231700 1 -- 172.21.15.52:0/3544271378 shutdown_connections 2023-11-27T15:19:01.579 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.564+0000 7fdecf231700 1 --2- 172.21.15.52:0/3544271378 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fdeb80a1460 0x7fdeb80a3910 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.579 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.564+0000 7fdecf231700 1 --2- 172.21.15.52:0/3544271378 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdec80ff450 0x7fdec8100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.580 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.564+0000 7fdecf231700 1 --2- 172.21.15.52:0/3544271378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdec8104490 0x7fdec81a4660 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.580 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.564+0000 7fdecf231700 1 --2- 172.21.15.52:0/3544271378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdec80ffdf0 0x7fdec8101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.580 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.564+0000 7fdecf231700 1 -- 172.21.15.52:0/3544271378 >> 172.21.15.52:0/3544271378 conn(0x7fdec8077080 msgr2=0x7fdec8069160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:01.580 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.564+0000 7fdecf231700 1 -- 172.21.15.52:0/3544271378 shutdown_connections 2023-11-27T15:19:01.580 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:01.564+0000 7fdecf231700 1 -- 172.21.15.52:0/3544271378 wait complete. 2023-11-27T15:19:01.621 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-27T15:19:01.873 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:01.873 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:01.964 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:01.964 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:02.004 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:02.004 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:02.004 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 28 -lt 100 ']' 2023-11-27T15:19:02.004 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 28 + 1 2023-11-27T15:19:02.071 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=29 2023-11-27T15:19:02.071 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:02.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:01 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:02.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:01 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:02.158 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:02.158 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:02.159 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:02.159 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:02.160 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:02.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:01 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:02.816 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:02.817 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:03.066 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:03.067 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:03.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:02 smithi102 ceph-mon[100296]: pgmap v36: 97 pgs: 97 active+clean; 1.6 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-11-27T15:19:03.096 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:03.096 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:03.096 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 29 -lt 100 ']' 2023-11-27T15:19:03.096 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 29 + 1 2023-11-27T15:19:03.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:02 smithi052 ceph-mon[99056]: pgmap v36: 97 pgs: 97 active+clean; 1.6 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-11-27T15:19:03.150 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=30 2023-11-27T15:19:03.150 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:03.212 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:03.212 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:03.213 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:03.213 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:03.213 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:03.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:02 smithi150 ceph-mon[100559]: pgmap v36: 97 pgs: 97 active+clean; 1.6 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 35 KiB/s wr, 11 op/s 2023-11-27T15:19:03.957 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:03.957 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:03.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.982+0000 7fc4f648b700 1 -- 172.21.15.52:0/332121396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4f0101ca0 msgr2=0x7fc4f010a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:03.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.982+0000 7fc4f648b700 1 --2- 172.21.15.52:0/332121396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4f0101ca0 0x7fc4f010a370 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4009990 tx=0x7fc4e4009ca0 comp rx=0 tx=0).stop 2023-11-27T15:19:03.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.982+0000 7fc4f648b700 1 -- 172.21.15.52:0/332121396 shutdown_connections 2023-11-27T15:19:03.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.982+0000 7fc4f648b700 1 --2- 172.21.15.52:0/332121396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4f0101390 0x7fc4f0101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:03.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.982+0000 7fc4f648b700 1 --2- 172.21.15.52:0/332121396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4f010a8b0 0x7fc4f010ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:03.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.982+0000 7fc4f648b700 1 --2- 172.21.15.52:0/332121396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4f0101ca0 0x7fc4f010a370 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:03.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.982+0000 7fc4f648b700 1 -- 172.21.15.52:0/332121396 >> 172.21.15.52:0/332121396 conn(0x7fc4f00770b0 msgr2=0x7fc4f00774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:03.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.982+0000 7fc4f648b700 1 -- 172.21.15.52:0/332121396 shutdown_connections 2023-11-27T15:19:03.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.982+0000 7fc4f648b700 1 -- 172.21.15.52:0/332121396 wait complete. 2023-11-27T15:19:03.989 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.983+0000 7fc4f648b700 1 Processor -- start 2023-11-27T15:19:03.990 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.983+0000 7fc4f648b700 1 -- start start 2023-11-27T15:19:03.990 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.984+0000 7fc4f648b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4f0101390 0x7fc4f01963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:03.990 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.984+0000 7fc4f648b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4f0101ca0 0x7fc4f01968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:03.990 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.984+0000 7fc4f648b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4f010a8b0 0x7fc4f019ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:03.990 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.984+0000 7fc4f648b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f019b390 con 0x7fc4f0101390 2023-11-27T15:19:03.990 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.984+0000 7fc4f648b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f019b500 con 0x7fc4f010a8b0 2023-11-27T15:19:03.990 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.984+0000 7fc4f648b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f019b6c0 con 0x7fc4f0101ca0 2023-11-27T15:19:03.990 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.984+0000 7fc4effff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4f0101390 0x7fc4f01963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:03.990 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.984+0000 7fc4effff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4f0101390 0x7fc4f01963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52560/0 (socket says 172.21.15.52:52560) 2023-11-27T15:19:03.991 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.984+0000 7fc4effff700 1 -- 172.21.15.52:0/3981369647 learned_addr learned my addr 172.21.15.52:0/3981369647 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:03.991 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.984+0000 7fc4ef7fe700 1 --2- 172.21.15.52:0/3981369647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4f0101ca0 0x7fc4f01968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:03.991 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.984+0000 7fc4f4a28700 1 --2- 172.21.15.52:0/3981369647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4f010a8b0 0x7fc4f019ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:03.991 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.984+0000 7fc4effff700 1 -- 172.21.15.52:0/3981369647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4f0101ca0 msgr2=0x7fc4f01968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:03.991 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.984+0000 7fc4effff700 1 --2- 172.21.15.52:0/3981369647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4f0101ca0 0x7fc4f01968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:03.991 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.984+0000 7fc4effff700 1 -- 172.21.15.52:0/3981369647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4f010a8b0 msgr2=0x7fc4f019ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:03.991 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.984+0000 7fc4effff700 1 --2- 172.21.15.52:0/3981369647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4f010a8b0 0x7fc4f019ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:03.991 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.984+0000 7fc4effff700 1 -- 172.21.15.52:0/3981369647 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e40095f0 con 0x7fc4f0101390 2023-11-27T15:19:03.991 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.984+0000 7fc4effff700 1 --2- 172.21.15.52:0/3981369647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4f0101390 0x7fc4f01963b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000e9a0 tx=0x7fc4e000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:03.991 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.985+0000 7fc4ed7fa700 1 -- 172.21.15.52:0/3981369647 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0009790 con 0x7fc4f0101390 2023-11-27T15:19:03.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.985+0000 7fc4ed7fa700 1 -- 172.21.15.52:0/3981369647 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4e000cbd0 con 0x7fc4f0101390 2023-11-27T15:19:03.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.985+0000 7fc4f648b700 1 -- 172.21.15.52:0/3981369647 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f0110340 con 0x7fc4f0101390 2023-11-27T15:19:03.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.985+0000 7fc4ed7fa700 1 -- 172.21.15.52:0/3981369647 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0018680 con 0x7fc4f0101390 2023-11-27T15:19:03.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.985+0000 7fc4f648b700 1 -- 172.21.15.52:0/3981369647 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f01101b0 con 0x7fc4f0101390 2023-11-27T15:19:03.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.987+0000 7fc4ed7fa700 1 -- 172.21.15.52:0/3981369647 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7fc4e000cd40 con 0x7fc4f0101390 2023-11-27T15:19:03.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.988+0000 7fc4ed7fa700 1 --2- 172.21.15.52:0/3981369647 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fc4d80a1460 0x7fc4d80a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:03.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.988+0000 7fc4ef7fe700 1 --2- 172.21.15.52:0/3981369647 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fc4d80a1460 0x7fc4d80a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:03.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.988+0000 7fc4ed7fa700 1 -- 172.21.15.52:0/3981369647 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fc4e0014070 con 0x7fc4f0101390 2023-11-27T15:19:03.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:03.988+0000 7fc4ef7fe700 1 --2- 172.21.15.52:0/3981369647 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fc4d80a1460 0x7fc4d80a3910 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc4f01979c0 tx=0x7fc4e4005fd0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:04.148 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.143+0000 7fc502617b80 1 Processor -- start 2023-11-27T15:19:04.148 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.143+0000 7fc502617b80 1 -- start start 2023-11-27T15:19:04.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.143+0000 7fc502617b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5593af7c0b60 0x5593af7c0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:04.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.143+0000 7fc502617b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593af7c1470 0x5593af7c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:04.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.143+0000 7fc502617b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593af7c58b0 0x5593af7c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:04.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.143+0000 7fc502617b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593af6c6060 con 0x5593af7c58b0 2023-11-27T15:19:04.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.143+0000 7fc502617b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593af7188a0 con 0x5593af7c0b60 2023-11-27T15:19:04.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.143+0000 7fc502617b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593af711d30 con 0x5593af7c1470 2023-11-27T15:19:04.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.143+0000 7fc4f4a28700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593af7c58b0 0x5593af7c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:04.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.143+0000 7fc4f4a28700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593af7c58b0 0x5593af7c5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52572/0 (socket says 172.21.15.52:52572) 2023-11-27T15:19:04.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.143+0000 7fc4f4a28700 1 -- 172.21.15.52:0/3197859252 learned_addr learned my addr 172.21.15.52:0/3197859252 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:04.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.143+0000 7fc4effff700 1 --2- 172.21.15.52:0/3197859252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5593af7c0b60 0x5593af7c0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:04.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.143+0000 7fc4ef7fe700 1 --2- 172.21.15.52:0/3197859252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593af7c1470 0x5593af7c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:04.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.144+0000 7fc4f4a28700 1 -- 172.21.15.52:0/3197859252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593af7c1470 msgr2=0x5593af7c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:04.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.144+0000 7fc4f4a28700 1 --2- 172.21.15.52:0/3197859252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593af7c1470 0x5593af7c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:04.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.144+0000 7fc4f4a28700 1 -- 172.21.15.52:0/3197859252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5593af7c0b60 msgr2=0x5593af7c0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:04.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.144+0000 7fc4f4a28700 1 --2- 172.21.15.52:0/3197859252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5593af7c0b60 0x5593af7c0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:04.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.144+0000 7fc4f4a28700 1 -- 172.21.15.52:0/3197859252 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593af593630 con 0x5593af7c58b0 2023-11-27T15:19:04.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.144+0000 7fc4effff700 1 --2- 172.21.15.52:0/3197859252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5593af7c0b60 0x5593af7c0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:04.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.144+0000 7fc4f4a28700 1 --2- 172.21.15.52:0/3197859252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593af7c58b0 0x5593af7c5d00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009cf0 tx=0x7fc4e8007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e0352c6e07793b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:04.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.144+0000 7fc4d67fc700 1 -- 172.21.15.52:0/3197859252 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8018070 con 0x5593af7c58b0 2023-11-27T15:19:04.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.144+0000 7fc4d67fc700 1 -- 172.21.15.52:0/3197859252 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4e800f050 con 0x5593af7c58b0 2023-11-27T15:19:04.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.144+0000 7fc4d67fc700 1 -- 172.21.15.52:0/3197859252 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e80059e0 con 0x5593af7c58b0 2023-11-27T15:19:04.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.145+0000 7fc502617b80 1 -- 172.21.15.52:0/3197859252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593af7c58b0 msgr2=0x5593af7c5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:04.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.145+0000 7fc502617b80 1 --2- 172.21.15.52:0/3197859252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593af7c58b0 0x5593af7c5d00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009cf0 tx=0x7fc4e8007500 comp rx=0 tx=0).stop 2023-11-27T15:19:04.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.145+0000 7fc502617b80 1 -- 172.21.15.52:0/3197859252 shutdown_connections 2023-11-27T15:19:04.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.145+0000 7fc502617b80 1 --2- 172.21.15.52:0/3197859252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5593af7c0b60 0x5593af7c0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:04.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.145+0000 7fc502617b80 1 --2- 172.21.15.52:0/3197859252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593af7c58b0 0x5593af7c5d00 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:04.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.145+0000 7fc502617b80 1 --2- 172.21.15.52:0/3197859252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593af7c1470 0x5593af7c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:04.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.145+0000 7fc502617b80 1 -- 172.21.15.52:0/3197859252 >> 172.21.15.52:0/3197859252 conn(0x5593af7b6360 msgr2=0x5593af7bbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:04.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.145+0000 7fc502617b80 1 -- 172.21.15.52:0/3197859252 shutdown_connections 2023-11-27T15:19:04.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.145+0000 7fc502617b80 1 -- 172.21.15.52:0/3197859252 wait complete. 2023-11-27T15:19:04.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.145+0000 7fc502617b80 1 -- start start 2023-11-27T15:19:04.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.146+0000 7fc502617b80 1 Processor -- start 2023-11-27T15:19:04.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.146+0000 7fc502617b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593af7c0b60 0x5593af7cd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:04.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.146+0000 7fc502617b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5593af7c1470 0x5593af7ca320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:04.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.146+0000 7fc502617b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593af7c58b0 0x5593af7ca980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:04.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.146+0000 7fc502617b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593af6b38e0 con 0x5593af7c0b60 2023-11-27T15:19:04.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.146+0000 7fc502617b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593af6b3580 con 0x5593af7c1470 2023-11-27T15:19:04.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.146+0000 7fc502617b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593af595920 con 0x5593af7c58b0 2023-11-27T15:19:04.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.146+0000 7fc502617b80 10 client.0 _ll_register_callbacks cb 0x5593af6d1780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:19:04.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.146+0000 7fc4effff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593af7c0b60 0x5593af7cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:04.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.146+0000 7fc4effff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593af7c0b60 0x5593af7cd030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52574/0 (socket says 172.21.15.52:52574) 2023-11-27T15:19:04.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.146+0000 7fc4effff700 1 -- 172.21.15.52:0/2475169688 learned_addr learned my addr 172.21.15.52:0/2475169688 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:04.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.146+0000 7fc4ef7fe700 1 --2- 172.21.15.52:0/2475169688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5593af7c1470 0x5593af7ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:04.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.146+0000 7fc4f4a28700 1 --2- 172.21.15.52:0/2475169688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593af7c58b0 0x5593af7ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:04.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.147+0000 7fc4effff700 1 -- 172.21.15.52:0/2475169688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593af7c58b0 msgr2=0x5593af7ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:04.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.147+0000 7fc4effff700 1 --2- 172.21.15.52:0/2475169688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593af7c58b0 0x5593af7ca980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:04.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.147+0000 7fc4effff700 1 -- 172.21.15.52:0/2475169688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5593af7c1470 msgr2=0x5593af7ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:04.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.147+0000 7fc4effff700 1 --2- 172.21.15.52:0/2475169688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5593af7c1470 0x5593af7ca320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:04.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.147+0000 7fc4effff700 1 -- 172.21.15.52:0/2475169688 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e8009920 con 0x5593af7c0b60 2023-11-27T15:19:04.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.147+0000 7fc4f4a28700 1 --2- 172.21.15.52:0/2475169688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593af7c58b0 0x5593af7ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:04.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.147+0000 7fc4ef7fe700 1 --2- 172.21.15.52:0/2475169688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5593af7c1470 0x5593af7ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:04.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.147+0000 7fc4effff700 1 --2- 172.21.15.52:0/2475169688 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593af7c0b60 0x5593af7cd030 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0099d20 tx=0x7fc4e009a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44fe642e1ecdccaa server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:04.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.147+0000 7fc4cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:19:04.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.147+0000 7fc4cf7fe700 1 -- 172.21.15.52:0/2475169688 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e009e050 con 0x5593af7c0b60 2023-11-27T15:19:04.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.147+0000 7fc4cf7fe700 1 -- 172.21.15.52:0/2475169688 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fc4e00a2470 con 0x5593af7c0b60 2023-11-27T15:19:04.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.147+0000 7fc502617b80 1 -- 172.21.15.52:0/2475169688 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5593af7b3b20 con 0x5593af7c0b60 2023-11-27T15:19:04.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.147+0000 7fc4cf7fe700 1 -- 172.21.15.52:0/2475169688 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4e00b81a0 con 0x5593af7c0b60 2023-11-27T15:19:04.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.147+0000 7fc502617b80 10 client.15636 fetch_fsmap learned FSMap version 23 2023-11-27T15:19:04.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.147+0000 7fc502617b80 1 -- 172.21.15.52:0/2475169688 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5593af6d8190 con 0x5593af7c0b60 2023-11-27T15:19:04.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.148+0000 7fc4cf7fe700 1 -- 172.21.15.52:0/2475169688 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5593af7b3b20 con 0x5593af7c0b60 2023-11-27T15:19:04.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.148+0000 7fc4cf7fe700 1 -- 172.21.15.52:0/2475169688 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc4e00add20 con 0x5593af7c0b60 2023-11-27T15:19:04.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.148+0000 7fc502617b80 10 client.15636 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:19:04.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.148+0000 7fc502617b80 10 client.15636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:19:04.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.148+0000 7fc502617b80 20 client.15636 populate_metadata read hostname 'smithi052' 2023-11-27T15:19:04.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.148+0000 7fc502617b80 1 --2- 172.21.15.52:0/2475169688 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5593af7dec80 0x5593af7e1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:04.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.148+0000 7fc502617b80 4 client.15636 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:19:04.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.148+0000 7fc502617b80 1 -- 172.21.15.52:0/2475169688 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5593af6d40a0 con 0x5593af7dec80 2023-11-27T15:19:04.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.148+0000 7fc4ef7fe700 1 --2- 172.21.15.52:0/2475169688 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5593af7dec80 0x5593af7e1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:04.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.148+0000 7fc4cf7fe700 10 client.15636 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:04.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.148+0000 7fc4ef7fe700 1 --2- 172.21.15.52:0/2475169688 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5593af7dec80 0x5593af7e1060 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28ef98a3f05eb8b2 server_cookie=4d3a825dc3a0dd57 in_seq=0 out_seq=0 2023-11-27T15:19:04.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.149+0000 7fc4cf7fe700 1 -- 172.21.15.52:0/2475169688 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5593af6d40a0 con 0x5593af7dec80 2023-11-27T15:19:04.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.149+0000 7fc4cf7fe700 10 client.15636 handle_command_reply: tid=0 2023-11-27T15:19:04.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.149+0000 7fc502617b80 1 client.15636 shutdown 2023-11-27T15:19:04.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.149+0000 7fc502617b80 1 -- 172.21.15.52:0/2475169688 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593af7c0b60 msgr2=0x5593af7cd030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:04.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.149+0000 7fc502617b80 1 --2- 172.21.15.52:0/2475169688 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593af7c0b60 0x5593af7cd030 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0099d20 tx=0x7fc4e009a560 comp rx=0 tx=0).stop 2023-11-27T15:19:04.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.149+0000 7fc502617b80 1 -- 172.21.15.52:0/2475169688 shutdown_connections 2023-11-27T15:19:04.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.149+0000 7fc502617b80 1 --2- 172.21.15.52:0/2475169688 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5593af7dec80 0x5593af7e1060 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:04.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.149+0000 7fc502617b80 1 --2- 172.21.15.52:0/2475169688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593af7c58b0 0x5593af7ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:04.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.149+0000 7fc502617b80 1 --2- 172.21.15.52:0/2475169688 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593af7c0b60 0x5593af7cd030 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:04.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.149+0000 7fc4cf7fe700 0 client.15636 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:04.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.149+0000 7fc502617b80 1 --2- 172.21.15.52:0/2475169688 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5593af7c1470 0x5593af7ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:04.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.149+0000 7fc502617b80 1 -- 172.21.15.52:0/2475169688 >> 172.21.15.52:0/2475169688 conn(0x5593af7b6360 msgr2=0x5593af7c6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:04.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.150+0000 7fc502617b80 1 -- 172.21.15.52:0/2475169688 shutdown_connections 2023-11-27T15:19:04.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.150+0000 7fc502617b80 1 -- 172.21.15.52:0/2475169688 wait complete. 2023-11-27T15:19:04.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.150+0000 7fc502617b80 20 client.15636 trim_cache size 0 max 16384 2023-11-27T15:19:04.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.164+0000 7fc502617b80 1 Processor -- start 2023-11-27T15:19:04.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.164+0000 7fc502617b80 1 -- start start 2023-11-27T15:19:04.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.164+0000 7fc502617b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5593af7c0b60 0x5593af7d5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:04.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.164+0000 7fc502617b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593af7c1470 0x5593af7ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:04.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.164+0000 7fc502617b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593af7c58b0 0x5593af7bb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:04.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.164+0000 7fc502617b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593af7c8880 con 0x5593af7c58b0 2023-11-27T15:19:04.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.164+0000 7fc502617b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593af7c4450 con 0x5593af7c0b60 2023-11-27T15:19:04.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.164+0000 7fc502617b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593af7d6370 con 0x5593af7c1470 2023-11-27T15:19:04.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.164+0000 7fc4effff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5593af7c0b60 0x5593af7d5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:04.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.164+0000 7fc4f4a28700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593af7c58b0 0x5593af7bb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:04.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.164+0000 7fc4ef7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593af7c1470 0x5593af7ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:04.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.164+0000 7fc4effff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5593af7c0b60 0x5593af7d5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:42240/0 (socket says 172.21.15.52:42240) 2023-11-27T15:19:04.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.164+0000 7fc4effff700 1 -- 172.21.15.52:0/3264601617 learned_addr learned my addr 172.21.15.52:0/3264601617 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:04.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.164+0000 7fc4effff700 1 -- 172.21.15.52:0/3264601617 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593af7c1470 msgr2=0x5593af7ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:04.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.164+0000 7fc4effff700 1 --2- 172.21.15.52:0/3264601617 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593af7c1470 0x5593af7ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:04.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.164+0000 7fc4effff700 1 -- 172.21.15.52:0/3264601617 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593af7c58b0 msgr2=0x5593af7bb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:04.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.164+0000 7fc4effff700 1 --2- 172.21.15.52:0/3264601617 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593af7c58b0 0x5593af7bb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:04.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.164+0000 7fc4effff700 1 -- 172.21.15.52:0/3264601617 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593af7bd240 con 0x5593af7c0b60 2023-11-27T15:19:04.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.164+0000 7fc4f4a28700 1 --2- 172.21.15.52:0/3264601617 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593af7c58b0 0x5593af7bb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:04.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.165+0000 7fc4effff700 1 --2- 172.21.15.52:0/3264601617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5593af7c0b60 0x5593af7d5e30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc4e00cd270 tx=0x7fc4e0005b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3b3776c080032dc server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:04.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.165+0000 7fc4d6ffd700 1 -- 172.21.15.52:0/3264601617 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e00a2470 con 0x5593af7c0b60 2023-11-27T15:19:04.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.165+0000 7fc4d6ffd700 1 -- 172.21.15.52:0/3264601617 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4e0099170 con 0x5593af7c0b60 2023-11-27T15:19:04.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.165+0000 7fc4d6ffd700 1 -- 172.21.15.52:0/3264601617 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e00caa50 con 0x5593af7c0b60 2023-11-27T15:19:04.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.165+0000 7fc502617b80 1 -- 172.21.15.52:0/3264601617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5593af7c0b60 msgr2=0x5593af7d5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:04.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.165+0000 7fc502617b80 1 --2- 172.21.15.52:0/3264601617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5593af7c0b60 0x5593af7d5e30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc4e00cd270 tx=0x7fc4e0005b40 comp rx=0 tx=0).stop 2023-11-27T15:19:04.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.165+0000 7fc502617b80 1 -- 172.21.15.52:0/3264601617 shutdown_connections 2023-11-27T15:19:04.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.165+0000 7fc502617b80 1 --2- 172.21.15.52:0/3264601617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5593af7c0b60 0x5593af7d5e30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:04.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.165+0000 7fc502617b80 1 --2- 172.21.15.52:0/3264601617 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593af7c58b0 0x5593af7bb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:04.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.165+0000 7fc502617b80 1 --2- 172.21.15.52:0/3264601617 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593af7c1470 0x5593af7ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:04.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.165+0000 7fc502617b80 1 -- 172.21.15.52:0/3264601617 >> 172.21.15.52:0/3264601617 conn(0x5593af7b6360 msgr2=0x5593af7c38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:04.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.165+0000 7fc502617b80 1 -- 172.21.15.52:0/3264601617 shutdown_connections 2023-11-27T15:19:04.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.165+0000 7fc502617b80 1 -- 172.21.15.52:0/3264601617 wait complete. 2023-11-27T15:19:04.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.166+0000 7fc502617b80 1 -- start start 2023-11-27T15:19:04.176 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.166+0000 7fc502617b80 1 Processor -- start 2023-11-27T15:19:04.176 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.166+0000 7fc502617b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5593af7c0b60 0x5593af7d3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:04.176 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.166+0000 7fc502617b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593af7c1470 0x5593af7d4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:04.176 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.166+0000 7fc502617b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593af7c58b0 0x5593af7d48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:04.176 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.166+0000 7fc502617b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593af831b70 con 0x5593af7c1470 2023-11-27T15:19:04.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.166+0000 7fc502617b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593af831ce0 con 0x5593af7c0b60 2023-11-27T15:19:04.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.166+0000 7fc502617b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593af831ea0 con 0x5593af7c58b0 2023-11-27T15:19:04.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.166+0000 7fc502617b80 10 client.0 _ll_register_callbacks cb 0x7fc4e0099c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:19:04.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.167+0000 7fc4ef7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593af7c1470 0x5593af7d4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:04.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.167+0000 7fc4ef7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593af7c1470 0x5593af7d4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52590/0 (socket says 172.21.15.52:52590) 2023-11-27T15:19:04.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.167+0000 7fc4ef7fe700 1 -- 172.21.15.52:0/730258097 learned_addr learned my addr 172.21.15.52:0/730258097 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:04.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.167+0000 7fc4f4a28700 1 --2- 172.21.15.52:0/730258097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593af7c58b0 0x5593af7d48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:04.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.167+0000 7fc4effff700 1 --2- 172.21.15.52:0/730258097 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5593af7c0b60 0x5593af7d3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:04.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.167+0000 7fc4ef7fe700 1 -- 172.21.15.52:0/730258097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593af7c58b0 msgr2=0x5593af7d48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:04.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.167+0000 7fc4ef7fe700 1 --2- 172.21.15.52:0/730258097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593af7c58b0 0x5593af7d48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:04.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.167+0000 7fc4ef7fe700 1 -- 172.21.15.52:0/730258097 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5593af7c0b60 msgr2=0x5593af7d3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:04.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.167+0000 7fc4ef7fe700 1 --2- 172.21.15.52:0/730258097 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5593af7c0b60 0x5593af7d3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:04.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.167+0000 7fc4ef7fe700 1 -- 172.21.15.52:0/730258097 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5593af593630 con 0x5593af7c1470 2023-11-27T15:19:04.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.167+0000 7fc4effff700 1 --2- 172.21.15.52:0/730258097 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5593af7c0b60 0x5593af7d3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:19:04.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.167+0000 7fc4ef7fe700 1 --2- 172.21.15.52:0/730258097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593af7c1470 0x5593af7d4360 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc4e40098e0 tx=0x7fc4e4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a228feb25a3168fa server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:04.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.167+0000 7fc4cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:19:04.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.167+0000 7fc4cf7fe700 1 -- 172.21.15.52:0/730258097 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e402b040 con 0x5593af7c1470 2023-11-27T15:19:04.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.167+0000 7fc4cf7fe700 1 -- 172.21.15.52:0/730258097 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fc4e4035cc0 con 0x5593af7c1470 2023-11-27T15:19:04.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.167+0000 7fc502617b80 1 -- 172.21.15.52:0/730258097 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc4e800f050 con 0x5593af7c1470 2023-11-27T15:19:04.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.168+0000 7fc4cf7fe700 1 -- 172.21.15.52:0/730258097 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4e4035e80 con 0x5593af7c1470 2023-11-27T15:19:04.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.168+0000 7fc502617b80 10 client.15642 fetch_fsmap learned FSMap version 23 2023-11-27T15:19:04.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.168+0000 7fc502617b80 1 -- 172.21.15.52:0/730258097 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5593af6d8190 con 0x5593af7c1470 2023-11-27T15:19:04.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.168+0000 7fc4cf7fe700 1 -- 172.21.15.52:0/730258097 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4e800f050 con 0x5593af7c1470 2023-11-27T15:19:04.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.168+0000 7fc4cf7fe700 1 -- 172.21.15.52:0/730258097 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc4e400f870 con 0x5593af7c1470 2023-11-27T15:19:04.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.168+0000 7fc502617b80 10 client.15642 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:19:04.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.168+0000 7fc502617b80 10 client.15642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:19:04.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.168+0000 7fc502617b80 20 client.15642 populate_metadata read hostname 'smithi052' 2023-11-27T15:19:04.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.168+0000 7fc502617b80 1 --2- 172.21.15.52:0/730258097 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5593af833540 0x5593af835950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:04.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.168+0000 7fc502617b80 4 client.15642 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:19:04.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.168+0000 7fc502617b80 1 -- 172.21.15.52:0/730258097 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5593af6e0660 con 0x5593af833540 2023-11-27T15:19:04.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.168+0000 7fc4effff700 1 --2- 172.21.15.52:0/730258097 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5593af833540 0x5593af835950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:04.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.169+0000 7fc4cf7fe700 10 client.15642 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:04.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.169+0000 7fc4effff700 1 --2- 172.21.15.52:0/730258097 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5593af833540 0x5593af835950 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cef00092a1c452c server_cookie=86550d5bda6e0448 in_seq=0 out_seq=0 2023-11-27T15:19:04.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.169+0000 7fc4cf7fe700 1 -- 172.21.15.52:0/730258097 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5593af6e0660 con 0x5593af833540 2023-11-27T15:19:04.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.169+0000 7fc4cf7fe700 10 client.15642 handle_command_reply: tid=0 2023-11-27T15:19:04.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.169+0000 7fc502617b80 1 client.15642 shutdown 2023-11-27T15:19:04.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.169+0000 7fc502617b80 1 -- 172.21.15.52:0/730258097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593af7c1470 msgr2=0x5593af7d4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:04.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.169+0000 7fc502617b80 1 --2- 172.21.15.52:0/730258097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593af7c1470 0x5593af7d4360 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc4e40098e0 tx=0x7fc4e4009ca0 comp rx=0 tx=0).stop 2023-11-27T15:19:04.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.169+0000 7fc502617b80 1 -- 172.21.15.52:0/730258097 shutdown_connections 2023-11-27T15:19:04.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.169+0000 7fc502617b80 1 --2- 172.21.15.52:0/730258097 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5593af833540 0x5593af835950 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:04.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.169+0000 7fc502617b80 1 --2- 172.21.15.52:0/730258097 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5593af7c0b60 0x5593af7d3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:04.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.169+0000 7fc502617b80 1 --2- 172.21.15.52:0/730258097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5593af7c58b0 0x5593af7d48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:04.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.169+0000 7fc4cf7fe700 0 client.15642 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:04.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.169+0000 7fc502617b80 1 --2- 172.21.15.52:0/730258097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593af7c1470 0x5593af7d4360 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:04.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.169+0000 7fc502617b80 1 -- 172.21.15.52:0/730258097 >> 172.21.15.52:0/730258097 conn(0x5593af7b6360 msgr2=0x5593af7c4860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:04.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.169+0000 7fc502617b80 1 -- 172.21.15.52:0/730258097 shutdown_connections 2023-11-27T15:19:04.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.169+0000 7fc502617b80 1 -- 172.21.15.52:0/730258097 wait complete. 2023-11-27T15:19:04.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.169+0000 7fc502617b80 20 client.15642 trim_cache size 0 max 16384 2023-11-27T15:19:04.181 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:19:04.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.170+0000 7fc4f648b700 1 -- 172.21.15.52:0/3981369647 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fc4d80a1460 msgr2=0x7fc4d80a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:04.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.170+0000 7fc4f648b700 1 --2- 172.21.15.52:0/3981369647 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fc4d80a1460 0x7fc4d80a3910 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc4f01979c0 tx=0x7fc4e4005fd0 comp rx=0 tx=0).stop 2023-11-27T15:19:04.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.170+0000 7fc4f648b700 1 -- 172.21.15.52:0/3981369647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4f0101390 msgr2=0x7fc4f01963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:04.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.170+0000 7fc4f648b700 1 --2- 172.21.15.52:0/3981369647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4f0101390 0x7fc4f01963b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000e9a0 tx=0x7fc4e000ecb0 comp rx=0 tx=0).stop 2023-11-27T15:19:04.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.170+0000 7fc4f648b700 1 -- 172.21.15.52:0/3981369647 shutdown_connections 2023-11-27T15:19:04.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.170+0000 7fc4f648b700 1 --2- 172.21.15.52:0/3981369647 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fc4d80a1460 0x7fc4d80a3910 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:04.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.170+0000 7fc4f648b700 1 --2- 172.21.15.52:0/3981369647 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4f010a8b0 0x7fc4f019ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:04.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.170+0000 7fc4f648b700 1 --2- 172.21.15.52:0/3981369647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4f0101390 0x7fc4f01963b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:04.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.170+0000 7fc4f648b700 1 --2- 172.21.15.52:0/3981369647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4f0101ca0 0x7fc4f01968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:04.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.170+0000 7fc4f648b700 1 -- 172.21.15.52:0/3981369647 >> 172.21.15.52:0/3981369647 conn(0x7fc4f00770b0 msgr2=0x7fc4f00fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:04.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.170+0000 7fc4f648b700 1 -- 172.21.15.52:0/3981369647 shutdown_connections 2023-11-27T15:19:04.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:04.170+0000 7fc4f648b700 1 -- 172.21.15.52:0/3981369647 wait complete. 2023-11-27T15:19:04.183 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:04.184 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:04.204 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:04.204 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:04.204 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 30 -lt 100 ']' 2023-11-27T15:19:04.204 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 30 + 1 2023-11-27T15:19:04.254 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=31 2023-11-27T15:19:04.254 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:04.295 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:04.295 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:04.296 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:04.296 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:04.296 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:05.023 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:05.023 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:05.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:04 smithi102 ceph-mon[100296]: pgmap v37: 97 pgs: 97 active+clean; 1.6 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 KiB/s wr, 19 op/s 2023-11-27T15:19:05.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:04 smithi052 ceph-mon[99056]: pgmap v37: 97 pgs: 97 active+clean; 1.6 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 KiB/s wr, 19 op/s 2023-11-27T15:19:05.213 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-27T15:19:05.213 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-27T15:19:05.258 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:05.258 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:05.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:04 smithi150 ceph-mon[100559]: pgmap v37: 97 pgs: 97 active+clean; 1.6 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 KiB/s wr, 19 op/s 2023-11-27T15:19:05.288 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:05.288 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:05.288 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 31 -lt 100 ']' 2023-11-27T15:19:05.288 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 31 + 1 2023-11-27T15:19:05.317 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=32 2023-11-27T15:19:05.317 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:05.346 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:05.346 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:05.348 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:05.348 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:05.348 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:06.081 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:06.081 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:06.241 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:06.241 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:06.279 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:06.279 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:06.280 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 32 -lt 100 ']' 2023-11-27T15:19:06.280 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 32 + 1 2023-11-27T15:19:06.321 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=33 2023-11-27T15:19:06.321 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:06.359 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:06.359 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:06.360 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:06.360 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:06.360 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:06.856 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:06.856 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:06.974 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:06.975 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:06.996 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:06.996 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:06.996 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 33 -lt 100 ']' 2023-11-27T15:19:06.996 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 33 + 1 2023-11-27T15:19:07.045 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=34 2023-11-27T15:19:07.046 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:07.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:06 smithi102 ceph-mon[100296]: pgmap v38: 97 pgs: 97 active+clean; 1.6 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 KiB/s wr, 19 op/s 2023-11-27T15:19:07.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:06 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:07.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:06 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:07.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:06 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:19:07.086 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:07.086 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:07.087 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:07.087 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:07.087 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:07.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:06 smithi052 ceph-mon[99056]: pgmap v38: 97 pgs: 97 active+clean; 1.6 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 KiB/s wr, 19 op/s 2023-11-27T15:19:07.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:06 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:07.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:06 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:07.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:06 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:19:07.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:06 smithi150 ceph-mon[100559]: pgmap v38: 97 pgs: 97 active+clean; 1.6 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 KiB/s wr, 19 op/s 2023-11-27T15:19:07.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:06 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:07.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:06 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:07.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:06 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:19:08.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:07 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:19:08.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:07 smithi102 ceph-mon[100296]: Service mds.all has no related daemon already placed. 2023-11-27T15:19:08.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:07 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:19:08.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:07 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:19:08.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:07 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:19:08.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:07 smithi052 ceph-mon[99056]: Service mds.all has no related daemon already placed. 2023-11-27T15:19:08.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:07 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:19:08.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:07 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:19:08.131 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:08.131 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:08.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:07 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:19:08.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:07 smithi150 ceph-mon[100559]: Service mds.all has no related daemon already placed. 2023-11-27T15:19:08.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:07 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:19:08.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:07 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:19:08.333 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:08.333 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:08.347 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:08.347 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:08.347 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 34 -lt 100 ']' 2023-11-27T15:19:08.347 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 34 + 1 2023-11-27T15:19:08.387 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=35 2023-11-27T15:19:08.387 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:08.431 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:08.431 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:08.432 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:08.433 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:08.433 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:08.836 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.831+0000 7fb089129700 1 -- 172.21.15.52:0/3713452239 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb084106410 msgr2=0x7fb0841067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:08.836 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.831+0000 7fb089129700 1 --2- 172.21.15.52:0/3713452239 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb084106410 0x7fb0841067e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb074009990 tx=0x7fb074009ca0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.836 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.831+0000 7fb089129700 1 -- 172.21.15.52:0/3713452239 shutdown_connections 2023-11-27T15:19:08.837 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.831+0000 7fb089129700 1 --2- 172.21.15.52:0/3713452239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb084101370 0x7fb08410d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.837 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.831+0000 7fb089129700 1 --2- 172.21.15.52:0/3713452239 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb084106410 0x7fb0841067e0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.837 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.831+0000 7fb089129700 1 --2- 172.21.15.52:0/3713452239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0841009e0 0x7fb084100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.837 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.831+0000 7fb089129700 1 -- 172.21.15.52:0/3713452239 >> 172.21.15.52:0/3713452239 conn(0x7fb0840775c0 msgr2=0x7fb0840779c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:08.837 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.832+0000 7fb089129700 1 -- 172.21.15.52:0/3713452239 shutdown_connections 2023-11-27T15:19:08.837 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.832+0000 7fb089129700 1 -- 172.21.15.52:0/3713452239 wait complete. 2023-11-27T15:19:08.839 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.833+0000 7fb089129700 1 Processor -- start 2023-11-27T15:19:08.839 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.833+0000 7fb089129700 1 -- start start 2023-11-27T15:19:08.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.833+0000 7fb089129700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0841009e0 0x7fb084196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:08.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.833+0000 7fb089129700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb084101370 0x7fb084196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:08.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.833+0000 7fb089129700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb084106410 0x7fb08419ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:08.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.833+0000 7fb089129700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08419b4a0 con 0x7fb084106410 2023-11-27T15:19:08.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.833+0000 7fb089129700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08419b610 con 0x7fb0841009e0 2023-11-27T15:19:08.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.833+0000 7fb089129700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08419b7d0 con 0x7fb084101370 2023-11-27T15:19:08.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.833+0000 7fb08359e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb084106410 0x7fb08419ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:08.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.833+0000 7fb08359e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb084106410 0x7fb08419ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52606/0 (socket says 172.21.15.52:52606) 2023-11-27T15:19:08.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.833+0000 7fb08359e700 1 -- 172.21.15.52:0/464340247 learned_addr learned my addr 172.21.15.52:0/464340247 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:08.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.833+0000 7fb082d9d700 1 --2- 172.21.15.52:0/464340247 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0841009e0 0x7fb084196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:08.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.834+0000 7fb08259c700 1 --2- 172.21.15.52:0/464340247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb084101370 0x7fb084196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:08.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.834+0000 7fb08359e700 1 -- 172.21.15.52:0/464340247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb084101370 msgr2=0x7fb084196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:08.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.834+0000 7fb08359e700 1 --2- 172.21.15.52:0/464340247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb084101370 0x7fb084196a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.834+0000 7fb08359e700 1 -- 172.21.15.52:0/464340247 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0841009e0 msgr2=0x7fb084196510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:08.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.834+0000 7fb08359e700 1 --2- 172.21.15.52:0/464340247 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0841009e0 0x7fb084196510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.834+0000 7fb08359e700 1 -- 172.21.15.52:0/464340247 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0740095f0 con 0x7fb084106410 2023-11-27T15:19:08.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.834+0000 7fb08359e700 1 --2- 172.21.15.52:0/464340247 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb084106410 0x7fb08419ae60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb07c007ee0 tx=0x7fb07c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:08.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.834+0000 7fb07bfff700 1 -- 172.21.15.52:0/464340247 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c00ddd0 con 0x7fb084106410 2023-11-27T15:19:08.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.834+0000 7fb07bfff700 1 -- 172.21.15.52:0/464340247 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb07c00f040 con 0x7fb084106410 2023-11-27T15:19:08.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.834+0000 7fb089129700 1 -- 172.21.15.52:0/464340247 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb084110460 con 0x7fb084106410 2023-11-27T15:19:08.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.834+0000 7fb07bfff700 1 -- 172.21.15.52:0/464340247 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c0048e0 con 0x7fb084106410 2023-11-27T15:19:08.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.834+0000 7fb089129700 1 -- 172.21.15.52:0/464340247 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0841102d0 con 0x7fb084106410 2023-11-27T15:19:08.842 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.837+0000 7fb07bfff700 1 -- 172.21.15.52:0/464340247 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7fb07c003a00 con 0x7fb084106410 2023-11-27T15:19:08.842 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.837+0000 7fb07bfff700 1 --2- 172.21.15.52:0/464340247 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb0700a13e0 0x7fb0700a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:08.842 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.838+0000 7fb07bfff700 1 -- 172.21.15.52:0/464340247 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb07c0cc8a0 con 0x7fb084106410 2023-11-27T15:19:08.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.838+0000 7fb082d9d700 1 --2- 172.21.15.52:0/464340247 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb0700a13e0 0x7fb0700a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:08.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.838+0000 7fb082d9d700 1 --2- 172.21.15.52:0/464340247 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb0700a13e0 0x7fb0700a3890 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb07400b5c0 tx=0x7fb074005fb0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:08.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.896+0000 7fb0952b5b80 1 Processor -- start 2023-11-27T15:19:08.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.896+0000 7fb0952b5b80 1 -- start start 2023-11-27T15:19:08.906 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.897+0000 7fb0952b5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55907d0dbbe0 0x55907d0dbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:08.906 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.897+0000 7fb0952b5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55907d0dc4f0 0x55907d0e03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:08.906 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.897+0000 7fb0952b5b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55907d0e0930 0x55907d0e0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:08.906 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.897+0000 7fb0952b5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907cfe10e0 con 0x55907d0dc4f0 2023-11-27T15:19:08.906 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.897+0000 7fb0952b5b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907d033920 con 0x55907d0e0930 2023-11-27T15:19:08.907 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.897+0000 7fb0952b5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907d02cdb0 con 0x55907d0dbbe0 2023-11-27T15:19:08.907 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.897+0000 7fb08259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55907d0dc4f0 0x55907d0e03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:08.907 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.897+0000 7fb08259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55907d0dc4f0 0x55907d0e03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52612/0 (socket says 172.21.15.52:52612) 2023-11-27T15:19:08.907 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.897+0000 7fb08259c700 1 -- 172.21.15.52:0/3150250472 learned_addr learned my addr 172.21.15.52:0/3150250472 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:08.907 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.897+0000 7fb082d9d700 1 --2- 172.21.15.52:0/3150250472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55907d0dbbe0 0x55907d0dbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:08.907 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.897+0000 7fb08359e700 1 --2- 172.21.15.52:0/3150250472 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55907d0e0930 0x55907d0e0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:08.907 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.897+0000 7fb08259c700 1 -- 172.21.15.52:0/3150250472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55907d0dbbe0 msgr2=0x55907d0dbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:08.907 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.897+0000 7fb08259c700 1 --2- 172.21.15.52:0/3150250472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55907d0dbbe0 0x55907d0dbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.907 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.897+0000 7fb08259c700 1 -- 172.21.15.52:0/3150250472 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55907d0e0930 msgr2=0x55907d0e0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:08.907 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.897+0000 7fb08259c700 1 --2- 172.21.15.52:0/3150250472 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55907d0e0930 0x55907d0e0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.908 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.897+0000 7fb08259c700 1 -- 172.21.15.52:0/3150250472 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55907ceae6b0 con 0x55907d0dc4f0 2023-11-27T15:19:08.908 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.897+0000 7fb08259c700 1 --2- 172.21.15.52:0/3150250472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55907d0dc4f0 0x55907d0e03f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb06c00dc40 tx=0x7fb06c00df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5657bc55dabdd206 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:08.908 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.898+0000 7fb0797fa700 1 -- 172.21.15.52:0/3150250472 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06c00fb60 con 0x55907d0dc4f0 2023-11-27T15:19:08.908 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.898+0000 7fb0797fa700 1 -- 172.21.15.52:0/3150250472 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb06c004cf0 con 0x55907d0dc4f0 2023-11-27T15:19:08.908 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.898+0000 7fb0797fa700 1 -- 172.21.15.52:0/3150250472 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06c0056a0 con 0x55907d0dc4f0 2023-11-27T15:19:08.908 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.898+0000 7fb0952b5b80 1 -- 172.21.15.52:0/3150250472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55907d0dc4f0 msgr2=0x55907d0e03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:08.908 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.898+0000 7fb0952b5b80 1 --2- 172.21.15.52:0/3150250472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55907d0dc4f0 0x55907d0e03f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb06c00dc40 tx=0x7fb06c00df50 comp rx=0 tx=0).stop 2023-11-27T15:19:08.908 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.899+0000 7fb0952b5b80 1 -- 172.21.15.52:0/3150250472 shutdown_connections 2023-11-27T15:19:08.908 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.899+0000 7fb0952b5b80 1 --2- 172.21.15.52:0/3150250472 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55907d0e0930 0x55907d0e0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.908 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.899+0000 7fb0952b5b80 1 --2- 172.21.15.52:0/3150250472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55907d0dbbe0 0x55907d0dbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.909 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.899+0000 7fb0952b5b80 1 --2- 172.21.15.52:0/3150250472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55907d0dc4f0 0x55907d0e03f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.909 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.899+0000 7fb0952b5b80 1 -- 172.21.15.52:0/3150250472 >> 172.21.15.52:0/3150250472 conn(0x55907d0d13e0 msgr2=0x55907d0d6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:08.909 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.899+0000 7fb0952b5b80 1 -- 172.21.15.52:0/3150250472 shutdown_connections 2023-11-27T15:19:08.909 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.899+0000 7fb0952b5b80 1 -- 172.21.15.52:0/3150250472 wait complete. 2023-11-27T15:19:08.909 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.899+0000 7fb0952b5b80 1 -- start start 2023-11-27T15:19:08.909 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.899+0000 7fb0952b5b80 1 Processor -- start 2023-11-27T15:19:08.909 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.899+0000 7fb0952b5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55907d0dbbe0 0x55907d0e80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:08.909 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.899+0000 7fb0952b5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55907d0dc4f0 0x55907d0e5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:08.909 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.900+0000 7fb0952b5b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55907d0e0930 0x55907d0e59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:08.909 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.900+0000 7fb0952b5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907cfce960 con 0x55907d0dbbe0 2023-11-27T15:19:08.909 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.900+0000 7fb0952b5b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907cfce600 con 0x55907d0e0930 2023-11-27T15:19:08.909 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.900+0000 7fb0952b5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907ceb09a0 con 0x55907d0dc4f0 2023-11-27T15:19:08.910 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.900+0000 7fb0952b5b80 10 client.0 _ll_register_callbacks cb 0x55907cfec800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:19:08.910 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.900+0000 7fb082d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55907d0dbbe0 0x55907d0e80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:08.910 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.900+0000 7fb082d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55907d0dbbe0 0x55907d0e80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52616/0 (socket says 172.21.15.52:52616) 2023-11-27T15:19:08.910 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.900+0000 7fb082d9d700 1 -- 172.21.15.52:0/670317684 learned_addr learned my addr 172.21.15.52:0/670317684 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:08.910 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.900+0000 7fb08359e700 1 --2- 172.21.15.52:0/670317684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55907d0e0930 0x55907d0e59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:08.910 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.900+0000 7fb08259c700 1 --2- 172.21.15.52:0/670317684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55907d0dc4f0 0x55907d0e5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:08.910 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.900+0000 7fb082d9d700 1 -- 172.21.15.52:0/670317684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55907d0dc4f0 msgr2=0x55907d0e5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:08.910 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.900+0000 7fb082d9d700 1 --2- 172.21.15.52:0/670317684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55907d0dc4f0 0x55907d0e5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.910 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.900+0000 7fb082d9d700 1 -- 172.21.15.52:0/670317684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55907d0e0930 msgr2=0x55907d0e59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:08.911 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.900+0000 7fb082d9d700 1 --2- 172.21.15.52:0/670317684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55907d0e0930 0x55907d0e59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.911 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.900+0000 7fb082d9d700 1 -- 172.21.15.52:0/670317684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb06c00d8a0 con 0x55907d0dbbe0 2023-11-27T15:19:08.911 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.900+0000 7fb08259c700 1 --2- 172.21.15.52:0/670317684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55907d0dc4f0 0x55907d0e5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:19:08.911 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.900+0000 7fb082d9d700 1 --2- 172.21.15.52:0/670317684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55907d0dbbe0 0x55907d0e80a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb074003ca0 tx=0x7fb074009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dffd7e937725d6d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:08.911 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.900+0000 7fb06a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:19:08.911 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.901+0000 7fb06a7fc700 1 -- 172.21.15.52:0/670317684 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07402a050 con 0x55907d0dbbe0 2023-11-27T15:19:08.911 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.901+0000 7fb06a7fc700 1 -- 172.21.15.52:0/670317684 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb074023660 con 0x55907d0dbbe0 2023-11-27T15:19:08.911 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.901+0000 7fb0952b5b80 1 -- 172.21.15.52:0/670317684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55907d0ceba0 con 0x55907d0dbbe0 2023-11-27T15:19:08.911 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.901+0000 7fb06a7fc700 1 -- 172.21.15.52:0/670317684 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb074023820 con 0x55907d0dbbe0 2023-11-27T15:19:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.901+0000 7fb0952b5b80 10 client.15654 fetch_fsmap learned FSMap version 23 2023-11-27T15:19:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.901+0000 7fb0952b5b80 1 -- 172.21.15.52:0/670317684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55907cff3210 con 0x55907d0dbbe0 2023-11-27T15:19:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.901+0000 7fb06a7fc700 1 -- 172.21.15.52:0/670317684 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55907d0ceba0 con 0x55907d0dbbe0 2023-11-27T15:19:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.901+0000 7fb06a7fc700 1 -- 172.21.15.52:0/670317684 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb07400fa00 con 0x55907d0dbbe0 2023-11-27T15:19:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.901+0000 7fb0952b5b80 10 client.15654 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:19:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.901+0000 7fb0952b5b80 10 client.15654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:19:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.901+0000 7fb0952b5b80 20 client.15654 populate_metadata read hostname 'smithi052' 2023-11-27T15:19:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.901+0000 7fb0952b5b80 1 --2- 172.21.15.52:0/670317684 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55907d0f5930 0x55907d0f7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.901+0000 7fb0952b5b80 4 client.15654 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:19:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.901+0000 7fb0952b5b80 1 -- 172.21.15.52:0/670317684 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55907cfef120 con 0x55907d0f5930 2023-11-27T15:19:08.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.902+0000 7fb08259c700 1 --2- 172.21.15.52:0/670317684 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55907d0f5930 0x55907d0f7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:08.913 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.902+0000 7fb08259c700 1 --2- 172.21.15.52:0/670317684 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55907d0f5930 0x55907d0f7d40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=870f79dd75d7e89c server_cookie=d77db6879ec18732 in_seq=0 out_seq=0 2023-11-27T15:19:08.913 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.902+0000 7fb06a7fc700 10 client.15654 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:08.914 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.903+0000 7fb06a7fc700 1 -- 172.21.15.52:0/670317684 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55907cfef120 con 0x55907d0f5930 2023-11-27T15:19:08.914 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.903+0000 7fb06a7fc700 10 client.15654 handle_command_reply: tid=0 2023-11-27T15:19:08.914 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.903+0000 7fb0952b5b80 1 client.15654 shutdown 2023-11-27T15:19:08.914 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.903+0000 7fb0952b5b80 1 -- 172.21.15.52:0/670317684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55907d0dbbe0 msgr2=0x55907d0e80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:08.914 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.903+0000 7fb0952b5b80 1 --2- 172.21.15.52:0/670317684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55907d0dbbe0 0x55907d0e80a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb074003ca0 tx=0x7fb074009ca0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.914 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.903+0000 7fb0952b5b80 1 -- 172.21.15.52:0/670317684 shutdown_connections 2023-11-27T15:19:08.914 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.903+0000 7fb0952b5b80 1 --2- 172.21.15.52:0/670317684 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55907d0f5930 0x55907d0f7d40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.914 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.903+0000 7fb0952b5b80 1 --2- 172.21.15.52:0/670317684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55907d0e0930 0x55907d0e59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.914 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.903+0000 7fb06a7fc700 0 client.15654 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:08.915 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.903+0000 7fb0952b5b80 1 --2- 172.21.15.52:0/670317684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55907d0dbbe0 0x55907d0e80a0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.915 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.903+0000 7fb0952b5b80 1 --2- 172.21.15.52:0/670317684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55907d0dc4f0 0x55907d0e5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.915 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.903+0000 7fb0952b5b80 1 -- 172.21.15.52:0/670317684 >> 172.21.15.52:0/670317684 conn(0x55907d0d13e0 msgr2=0x55907d0d5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:08.915 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.903+0000 7fb0952b5b80 1 -- 172.21.15.52:0/670317684 shutdown_connections 2023-11-27T15:19:08.915 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.904+0000 7fb0952b5b80 1 -- 172.21.15.52:0/670317684 wait complete. 2023-11-27T15:19:08.915 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.904+0000 7fb0952b5b80 20 client.15654 trim_cache size 0 max 16384 2023-11-27T15:19:08.928 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.922+0000 7fb0952b5b80 1 Processor -- start 2023-11-27T15:19:08.928 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.922+0000 7fb0952b5b80 1 -- start start 2023-11-27T15:19:08.928 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.923+0000 7fb0952b5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55907d0dbbe0 0x55907d0e7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:08.928 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.923+0000 7fb0952b5b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55907d0dc4f0 0x55907d0d65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:08.928 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.923+0000 7fb0952b5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55907d0e0930 0x55907d0d6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:08.929 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.923+0000 7fb0952b5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907d0e6950 con 0x55907d0e0930 2023-11-27T15:19:08.929 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.923+0000 7fb0952b5b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907d0e62b0 con 0x55907d0dc4f0 2023-11-27T15:19:08.929 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.923+0000 7fb0952b5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907d0d8270 con 0x55907d0dbbe0 2023-11-27T15:19:08.929 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.923+0000 7fb08359e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55907d0e0930 0x55907d0d6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:08.929 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.923+0000 7fb08359e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55907d0e0930 0x55907d0d6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52626/0 (socket says 172.21.15.52:52626) 2023-11-27T15:19:08.929 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.923+0000 7fb08359e700 1 -- 172.21.15.52:0/2176204010 learned_addr learned my addr 172.21.15.52:0/2176204010 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:08.929 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.923+0000 7fb08259c700 1 --2- 172.21.15.52:0/2176204010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55907d0dc4f0 0x55907d0d65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:08.929 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.923+0000 7fb082d9d700 1 --2- 172.21.15.52:0/2176204010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55907d0dbbe0 0x55907d0e7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:08.930 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.924+0000 7fb08259c700 1 -- 172.21.15.52:0/2176204010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55907d0dbbe0 msgr2=0x55907d0e7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:08.930 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.924+0000 7fb08259c700 1 --2- 172.21.15.52:0/2176204010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55907d0dbbe0 0x55907d0e7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.930 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.924+0000 7fb08259c700 1 -- 172.21.15.52:0/2176204010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55907d0e0930 msgr2=0x55907d0d6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:08.930 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.924+0000 7fb08259c700 1 --2- 172.21.15.52:0/2176204010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55907d0e0930 0x55907d0d6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.930 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.924+0000 7fb08259c700 1 -- 172.21.15.52:0/2176204010 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55907d0d2b80 con 0x55907d0dc4f0 2023-11-27T15:19:08.930 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.924+0000 7fb08359e700 1 --2- 172.21.15.52:0/2176204010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55907d0e0930 0x55907d0d6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:08.930 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.924+0000 7fb08259c700 1 --2- 172.21.15.52:0/2176204010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55907d0dc4f0 0x55907d0d65e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb06c005e90 tx=0x7fb06c00ffb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b9700455ecb5b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:08.930 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.924+0000 7fb079ffb700 1 -- 172.21.15.52:0/2176204010 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06c020040 con 0x55907d0dc4f0 2023-11-27T15:19:08.930 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.924+0000 7fb079ffb700 1 -- 172.21.15.52:0/2176204010 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb06c0055a0 con 0x55907d0dc4f0 2023-11-27T15:19:08.930 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.924+0000 7fb079ffb700 1 -- 172.21.15.52:0/2176204010 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06c024780 con 0x55907d0dc4f0 2023-11-27T15:19:08.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.925+0000 7fb0952b5b80 1 -- 172.21.15.52:0/2176204010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55907d0dc4f0 msgr2=0x55907d0d65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:08.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.925+0000 7fb0952b5b80 1 --2- 172.21.15.52:0/2176204010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55907d0dc4f0 0x55907d0d65e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb06c005e90 tx=0x7fb06c00ffb0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.925+0000 7fb0952b5b80 1 -- 172.21.15.52:0/2176204010 shutdown_connections 2023-11-27T15:19:08.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.925+0000 7fb0952b5b80 1 --2- 172.21.15.52:0/2176204010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55907d0e0930 0x55907d0d6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.925+0000 7fb0952b5b80 1 --2- 172.21.15.52:0/2176204010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55907d0dbbe0 0x55907d0e7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.925+0000 7fb0952b5b80 1 --2- 172.21.15.52:0/2176204010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55907d0dc4f0 0x55907d0d65e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.925+0000 7fb0952b5b80 1 -- 172.21.15.52:0/2176204010 >> 172.21.15.52:0/2176204010 conn(0x55907d0d13e0 msgr2=0x55907d0e2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:08.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.925+0000 7fb0952b5b80 1 -- 172.21.15.52:0/2176204010 shutdown_connections 2023-11-27T15:19:08.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.925+0000 7fb0952b5b80 1 -- 172.21.15.52:0/2176204010 wait complete. 2023-11-27T15:19:08.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.926+0000 7fb0952b5b80 1 -- start start 2023-11-27T15:19:08.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.926+0000 7fb0952b5b80 1 Processor -- start 2023-11-27T15:19:08.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.926+0000 7fb0952b5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55907d0dbbe0 0x55907d147ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:08.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.926+0000 7fb0952b5b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55907d0dc4f0 0x55907d1481e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:08.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.926+0000 7fb0952b5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55907d0e0930 0x55907d142040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:08.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.926+0000 7fb0952b5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907d142620 con 0x55907d0dbbe0 2023-11-27T15:19:08.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.926+0000 7fb0952b5b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907d142790 con 0x55907d0dc4f0 2023-11-27T15:19:08.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.926+0000 7fb0952b5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907d142950 con 0x55907d0e0930 2023-11-27T15:19:08.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.926+0000 7fb0952b5b80 10 client.0 _ll_register_callbacks cb 0x7fb074003890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:19:08.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.926+0000 7fb082d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55907d0dbbe0 0x55907d147ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:08.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.926+0000 7fb082d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55907d0dbbe0 0x55907d147ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52630/0 (socket says 172.21.15.52:52630) 2023-11-27T15:19:08.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.926+0000 7fb082d9d700 1 -- 172.21.15.52:0/3445726373 learned_addr learned my addr 172.21.15.52:0/3445726373 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:08.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.926+0000 7fb08259c700 1 --2- 172.21.15.52:0/3445726373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55907d0dc4f0 0x55907d1481e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:08.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.926+0000 7fb08359e700 1 --2- 172.21.15.52:0/3445726373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55907d0e0930 0x55907d142040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:08.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.927+0000 7fb082d9d700 1 -- 172.21.15.52:0/3445726373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55907d0e0930 msgr2=0x55907d142040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:08.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.927+0000 7fb082d9d700 1 --2- 172.21.15.52:0/3445726373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55907d0e0930 0x55907d142040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.927+0000 7fb082d9d700 1 -- 172.21.15.52:0/3445726373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55907d0dc4f0 msgr2=0x55907d1481e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:08.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.927+0000 7fb082d9d700 1 --2- 172.21.15.52:0/3445726373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55907d0dc4f0 0x55907d1481e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.927+0000 7fb082d9d700 1 -- 172.21.15.52:0/3445726373 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0841102d0 con 0x55907d0dbbe0 2023-11-27T15:19:08.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.927+0000 7fb08259c700 1 --2- 172.21.15.52:0/3445726373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55907d0dc4f0 0x55907d1481e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:08.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.927+0000 7fb082d9d700 1 --2- 172.21.15.52:0/3445726373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55907d0dbbe0 0x55907d147ca0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb074003c70 tx=0x7fb07400f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dedd207f93905a18 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:08.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.927+0000 7fb06a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:19:08.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.927+0000 7fb06a7fc700 1 -- 172.21.15.52:0/3445726373 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07402a040 con 0x55907d0dbbe0 2023-11-27T15:19:08.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.927+0000 7fb06a7fc700 1 -- 172.21.15.52:0/3445726373 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb07400be80 con 0x55907d0dbbe0 2023-11-27T15:19:08.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.927+0000 7fb0952b5b80 1 -- 172.21.15.52:0/3445726373 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb06c004cf0 con 0x55907d0dbbe0 2023-11-27T15:19:08.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.927+0000 7fb06a7fc700 1 -- 172.21.15.52:0/3445726373 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb07400b5f0 con 0x55907d0dbbe0 2023-11-27T15:19:08.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.927+0000 7fb0952b5b80 10 client.15660 fetch_fsmap learned FSMap version 23 2023-11-27T15:19:08.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.927+0000 7fb0952b5b80 1 -- 172.21.15.52:0/3445726373 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55907ceae6b0 con 0x55907d0dbbe0 2023-11-27T15:19:08.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.927+0000 7fb06a7fc700 1 -- 172.21.15.52:0/3445726373 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb06c004cf0 con 0x55907d0dbbe0 2023-11-27T15:19:08.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.928+0000 7fb06a7fc700 1 -- 172.21.15.52:0/3445726373 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb074023580 con 0x55907d0dbbe0 2023-11-27T15:19:08.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.928+0000 7fb0952b5b80 10 client.15660 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:19:08.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.928+0000 7fb0952b5b80 10 client.15660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:19:08.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.928+0000 7fb0952b5b80 20 client.15660 populate_metadata read hostname 'smithi052' 2023-11-27T15:19:08.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.928+0000 7fb0952b5b80 1 --2- 172.21.15.52:0/3445726373 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55907d14f7f0 0x55907d151c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:08.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.928+0000 7fb0952b5b80 4 client.15660 mds_command: new command op to 24536 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-27T15:19:08.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.928+0000 7fb0952b5b80 1 -- 172.21.15.52:0/3445726373 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55907cffb6e0 con 0x55907d14f7f0 2023-11-27T15:19:08.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.928+0000 7fb08259c700 1 --2- 172.21.15.52:0/3445726373 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55907d14f7f0 0x55907d151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:08.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.928+0000 7fb06a7fc700 10 client.15660 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:08.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.928+0000 7fb08259c700 1 --2- 172.21.15.52:0/3445726373 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55907d14f7f0 0x55907d151c00 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89d7453000a3e8cf server_cookie=717924b8cad1051e in_seq=0 out_seq=0 2023-11-27T15:19:08.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.930+0000 7fb06a7fc700 1 -- 172.21.15.52:0/3445726373 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55907cffb6e0 con 0x55907d14f7f0 2023-11-27T15:19:08.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.930+0000 7fb06a7fc700 10 client.15660 handle_command_reply: tid=0 2023-11-27T15:19:08.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.930+0000 7fb0952b5b80 1 client.15660 shutdown 2023-11-27T15:19:08.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.930+0000 7fb0952b5b80 1 -- 172.21.15.52:0/3445726373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55907d0dbbe0 msgr2=0x55907d147ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:08.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.930+0000 7fb0952b5b80 1 --2- 172.21.15.52:0/3445726373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55907d0dbbe0 0x55907d147ca0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb074003c70 tx=0x7fb07400f710 comp rx=0 tx=0).stop 2023-11-27T15:19:08.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.930+0000 7fb0952b5b80 1 -- 172.21.15.52:0/3445726373 shutdown_connections 2023-11-27T15:19:08.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.930+0000 7fb0952b5b80 1 --2- 172.21.15.52:0/3445726373 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55907d14f7f0 0x55907d151c00 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.930+0000 7fb0952b5b80 1 --2- 172.21.15.52:0/3445726373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55907d0e0930 0x55907d142040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.930+0000 7fb06a7fc700 0 client.15660 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:08.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.930+0000 7fb0952b5b80 1 --2- 172.21.15.52:0/3445726373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55907d0dbbe0 0x55907d147ca0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.930+0000 7fb0952b5b80 1 --2- 172.21.15.52:0/3445726373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55907d0dc4f0 0x55907d1481e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.930+0000 7fb0952b5b80 1 -- 172.21.15.52:0/3445726373 >> 172.21.15.52:0/3445726373 conn(0x55907d0d13e0 msgr2=0x55907d0e2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:08.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.930+0000 7fb0952b5b80 1 -- 172.21.15.52:0/3445726373 shutdown_connections 2023-11-27T15:19:08.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.930+0000 7fb0952b5b80 1 -- 172.21.15.52:0/3445726373 wait complete. 2023-11-27T15:19:08.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.930+0000 7fb0952b5b80 20 client.15660 trim_cache size 0 max 16384 2023-11-27T15:19:08.941 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:19:08.941 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2023-11-27T15:19:08.941 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "a73e324e-3a1d-41e1-bf7f-0efa1eaada82", 2023-11-27T15:19:08.941 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2023-11-27T15:19:08.941 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:19:08.942 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.931+0000 7fb089129700 1 -- 172.21.15.52:0/464340247 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb0700a13e0 msgr2=0x7fb0700a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:08.942 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.931+0000 7fb089129700 1 --2- 172.21.15.52:0/464340247 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb0700a13e0 0x7fb0700a3890 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb07400b5c0 tx=0x7fb074005fb0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.942 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.931+0000 7fb089129700 1 -- 172.21.15.52:0/464340247 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb084106410 msgr2=0x7fb08419ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:08.943 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.931+0000 7fb089129700 1 --2- 172.21.15.52:0/464340247 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb084106410 0x7fb08419ae60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb07c007ee0 tx=0x7fb07c00d3b0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.943 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.931+0000 7fb089129700 1 -- 172.21.15.52:0/464340247 shutdown_connections 2023-11-27T15:19:08.943 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.931+0000 7fb089129700 1 --2- 172.21.15.52:0/464340247 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb0700a13e0 0x7fb0700a3890 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.943 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.931+0000 7fb089129700 1 --2- 172.21.15.52:0/464340247 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0841009e0 0x7fb084196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.943 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.931+0000 7fb089129700 1 --2- 172.21.15.52:0/464340247 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb084106410 0x7fb08419ae60 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.943 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.931+0000 7fb089129700 1 --2- 172.21.15.52:0/464340247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb084101370 0x7fb084196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:08.943 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.931+0000 7fb089129700 1 -- 172.21.15.52:0/464340247 >> 172.21.15.52:0/464340247 conn(0x7fb0840775c0 msgr2=0x7fb0840ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:08.943 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.931+0000 7fb089129700 1 -- 172.21.15.52:0/464340247 shutdown_connections 2023-11-27T15:19:08.943 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:08.931+0000 7fb089129700 1 -- 172.21.15.52:0/464340247 wait complete. 2023-11-27T15:19:09.056 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:09.056 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:09.069 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:08 smithi052 ceph-mon[99056]: pgmap v39: 97 pgs: 97 active+clean; 1.7 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-27T15:19:09.070 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:19:09.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:08 smithi102 ceph-mon[100296]: pgmap v39: 97 pgs: 97 active+clean; 1.7 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-27T15:19:09.199 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:09.199 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:09.212 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:09.212 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:09.212 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 35 -lt 100 ']' 2023-11-27T15:19:09.212 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 35 + 1 2023-11-27T15:19:09.262 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=36 2023-11-27T15:19:09.262 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:09.272 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:09.272 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:09.273 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:09.273 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:09.274 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:09.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:08 smithi150 ceph-mon[100559]: pgmap v39: 97 pgs: 97 active+clean; 1.7 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-27T15:19:09.781 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:09.781 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:09.849 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:09.849 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:09.862 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:09.862 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:09.862 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 36 -lt 100 ']' 2023-11-27T15:19:09.862 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 36 + 1 2023-11-27T15:19:09.866 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:09 smithi052 ceph-mon[99056]: scrub queued for path: / 2023-11-27T15:19:09.866 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:09 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:19:09.866 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:09 smithi052 ceph-mon[99056]: scrub summary: active paths [/] 2023-11-27T15:19:09.866 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:09 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:19:09.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:09 smithi052 ceph-mon[99056]: scrub summary: idle 2023-11-27T15:19:09.912 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=37 2023-11-27T15:19:09.912 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:09.945 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:09.945 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:09.946 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:09.946 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:09.946 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:10.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:09 smithi102 ceph-mon[100296]: scrub queued for path: / 2023-11-27T15:19:10.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:09 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:19:10.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:09 smithi102 ceph-mon[100296]: scrub summary: active paths [/] 2023-11-27T15:19:10.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:09 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:19:10.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:09 smithi102 ceph-mon[100296]: scrub summary: idle 2023-11-27T15:19:10.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:09 smithi150 ceph-mon[100559]: scrub queued for path: / 2023-11-27T15:19:10.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:09 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:19:10.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:09 smithi150 ceph-mon[100559]: scrub summary: active paths [/] 2023-11-27T15:19:10.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:09 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:19:10.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:09 smithi150 ceph-mon[100559]: scrub summary: idle 2023-11-27T15:19:10.472 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:10.472 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:10.640 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:10.640 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:10.661 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:10.662 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:10.662 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 37 -lt 100 ']' 2023-11-27T15:19:10.662 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 37 + 1 2023-11-27T15:19:10.707 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=38 2023-11-27T15:19:10.707 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:10.746 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:10.746 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:10.747 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:10.747 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:10.747 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:11.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:10 smithi052 ceph-mon[99056]: pgmap v40: 97 pgs: 97 active+clean; 1.7 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 38 KiB/s wr, 18 op/s 2023-11-27T15:19:11.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:10 smithi150 ceph-mon[100559]: pgmap v40: 97 pgs: 97 active+clean; 1.7 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 38 KiB/s wr, 18 op/s 2023-11-27T15:19:11.315 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:11.315 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:11.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:10 smithi102 ceph-mon[100296]: pgmap v40: 97 pgs: 97 active+clean; 1.7 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 38 KiB/s wr, 18 op/s 2023-11-27T15:19:11.499 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:11.499 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:11.530 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:11.530 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:11.530 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 38 -lt 100 ']' 2023-11-27T15:19:11.530 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 38 + 1 2023-11-27T15:19:11.653 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=39 2023-11-27T15:19:11.653 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:11.735 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:11.735 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:11.736 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:11.736 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:11.736 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:11.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.773+0000 7fd0d7726700 1 -- 172.21.15.52:0/2609831381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0d0069a80 msgr2=0x7fd0d0104f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:11.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.773+0000 7fd0d7726700 1 --2- 172.21.15.52:0/2609831381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0d0069a80 0x7fd0d0104f20 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd0c40099c0 tx=0x7fd0c4009cd0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.773+0000 7fd0d7726700 1 -- 172.21.15.52:0/2609831381 shutdown_connections 2023-11-27T15:19:11.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.773+0000 7fd0d7726700 1 --2- 172.21.15.52:0/2609831381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0d0105600 0x7fd0d010d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.773+0000 7fd0d7726700 1 --2- 172.21.15.52:0/2609831381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0d00690e0 0x7fd0d00694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.773+0000 7fd0d7726700 1 --2- 172.21.15.52:0/2609831381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0d0069a80 0x7fd0d0104f20 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.773+0000 7fd0d7726700 1 -- 172.21.15.52:0/2609831381 >> 172.21.15.52:0/2609831381 conn(0x7fd0d0075470 msgr2=0x7fd0d0075870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:11.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.773+0000 7fd0d7726700 1 -- 172.21.15.52:0/2609831381 shutdown_connections 2023-11-27T15:19:11.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.773+0000 7fd0d7726700 1 -- 172.21.15.52:0/2609831381 wait complete. 2023-11-27T15:19:11.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.774+0000 7fd0d7726700 1 Processor -- start 2023-11-27T15:19:11.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.774+0000 7fd0d7726700 1 -- start start 2023-11-27T15:19:11.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.774+0000 7fd0d7726700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0d00690e0 0x7fd0d01963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:11.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.774+0000 7fd0d7726700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0d0069a80 0x7fd0d0196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:11.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.774+0000 7fd0d7726700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0d0105600 0x7fd0d019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:11.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.774+0000 7fd0d7726700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d019b3b0 con 0x7fd0d0069a80 2023-11-27T15:19:11.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.774+0000 7fd0d7726700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d019b520 con 0x7fd0d00690e0 2023-11-27T15:19:11.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.774+0000 7fd0d7726700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d019b6e0 con 0x7fd0d0105600 2023-11-27T15:19:11.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.774+0000 7fd0d4cc1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0d0069a80 0x7fd0d0196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:11.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.774+0000 7fd0d4cc1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0d0069a80 0x7fd0d0196910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52646/0 (socket says 172.21.15.52:52646) 2023-11-27T15:19:11.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.774+0000 7fd0d4cc1700 1 -- 172.21.15.52:0/3379601563 learned_addr learned my addr 172.21.15.52:0/3379601563 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:11.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.774+0000 7fd0d5cc3700 1 --2- 172.21.15.52:0/3379601563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0d0105600 0x7fd0d019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:11.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.774+0000 7fd0d54c2700 1 --2- 172.21.15.52:0/3379601563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0d00690e0 0x7fd0d01963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:11.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.774+0000 7fd0d4cc1700 1 -- 172.21.15.52:0/3379601563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0d0105600 msgr2=0x7fd0d019ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:11.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.774+0000 7fd0d4cc1700 1 --2- 172.21.15.52:0/3379601563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0d0105600 0x7fd0d019ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.774+0000 7fd0d4cc1700 1 -- 172.21.15.52:0/3379601563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0d00690e0 msgr2=0x7fd0d01963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:11.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.774+0000 7fd0d4cc1700 1 --2- 172.21.15.52:0/3379601563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0d00690e0 0x7fd0d01963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.774+0000 7fd0d4cc1700 1 -- 172.21.15.52:0/3379601563 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c40095f0 con 0x7fd0d0069a80 2023-11-27T15:19:11.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.774+0000 7fd0d54c2700 1 --2- 172.21.15.52:0/3379601563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0d00690e0 0x7fd0d01963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:19:11.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.775+0000 7fd0d4cc1700 1 --2- 172.21.15.52:0/3379601563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0d0069a80 0x7fd0d0196910 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4009990 tx=0x7fd0c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:11.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.775+0000 7fd0ce7fc700 1 -- 172.21.15.52:0/3379601563 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c401d070 con 0x7fd0d0069a80 2023-11-27T15:19:11.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.775+0000 7fd0ce7fc700 1 -- 172.21.15.52:0/3379601563 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0c4003f90 con 0x7fd0d0069a80 2023-11-27T15:19:11.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.775+0000 7fd0d7726700 1 -- 172.21.15.52:0/3379601563 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0d0110380 con 0x7fd0d0069a80 2023-11-27T15:19:11.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.775+0000 7fd0ce7fc700 1 -- 172.21.15.52:0/3379601563 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4022b70 con 0x7fd0d0069a80 2023-11-27T15:19:11.783 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.775+0000 7fd0d7726700 1 -- 172.21.15.52:0/3379601563 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d01101f0 con 0x7fd0d0069a80 2023-11-27T15:19:11.783 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.777+0000 7fd0ce7fc700 1 -- 172.21.15.52:0/3379601563 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7fd0c4003ad0 con 0x7fd0d0069a80 2023-11-27T15:19:11.784 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.778+0000 7fd0ce7fc700 1 --2- 172.21.15.52:0/3379601563 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd0b80a13e0 0x7fd0b80a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:11.784 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.778+0000 7fd0d54c2700 1 --2- 172.21.15.52:0/3379601563 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd0b80a13e0 0x7fd0b80a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:11.784 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.778+0000 7fd0ce7fc700 1 -- 172.21.15.52:0/3379601563 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fd0c40ce420 con 0x7fd0d0069a80 2023-11-27T15:19:11.784 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.778+0000 7fd0d54c2700 1 --2- 172.21.15.52:0/3379601563 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd0b80a13e0 0x7fd0b80a3890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0009dd0 tx=0x7fd0c0009040 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:11.890 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.885+0000 7fd0e38b2b80 1 Processor -- start 2023-11-27T15:19:11.890 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.885+0000 7fd0e38b2b80 1 -- start start 2023-11-27T15:19:11.890 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.885+0000 7fd0e38b2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f998b9b60 0x560f998b9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:11.890 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.885+0000 7fd0e38b2b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560f998ba470 0x560f998be370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:11.891 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.885+0000 7fd0e38b2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560f998be8b0 0x560f998bed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:11.891 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.885+0000 7fd0e38b2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f997bf060 con 0x560f998be8b0 2023-11-27T15:19:11.891 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.885+0000 7fd0e38b2b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f998118a0 con 0x560f998ba470 2023-11-27T15:19:11.891 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.885+0000 7fd0e38b2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f9980ad30 con 0x560f998b9b60 2023-11-27T15:19:11.891 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.885+0000 7fd0d5cc3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560f998be8b0 0x560f998bed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:11.891 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.885+0000 7fd0d5cc3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560f998be8b0 0x560f998bed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52656/0 (socket says 172.21.15.52:52656) 2023-11-27T15:19:11.891 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.885+0000 7fd0d5cc3700 1 -- 172.21.15.52:0/3219190397 learned_addr learned my addr 172.21.15.52:0/3219190397 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:11.891 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.885+0000 7fd0d54c2700 1 --2- 172.21.15.52:0/3219190397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f998b9b60 0x560f998b9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:11.891 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.885+0000 7fd0d4cc1700 1 --2- 172.21.15.52:0/3219190397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560f998ba470 0x560f998be370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:11.891 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.886+0000 7fd0d5cc3700 1 -- 172.21.15.52:0/3219190397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f998b9b60 msgr2=0x560f998b9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:11.891 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.886+0000 7fd0d5cc3700 1 --2- 172.21.15.52:0/3219190397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f998b9b60 0x560f998b9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.891 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.886+0000 7fd0d5cc3700 1 -- 172.21.15.52:0/3219190397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560f998ba470 msgr2=0x560f998be370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:11.892 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.886+0000 7fd0d5cc3700 1 --2- 172.21.15.52:0/3219190397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560f998ba470 0x560f998be370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.892 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.886+0000 7fd0d5cc3700 1 -- 172.21.15.52:0/3219190397 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f9968c630 con 0x560f998be8b0 2023-11-27T15:19:11.892 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.886+0000 7fd0d54c2700 1 --2- 172.21.15.52:0/3219190397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f998b9b60 0x560f998b9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:19:11.892 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.886+0000 7fd0d5cc3700 1 --2- 172.21.15.52:0/3219190397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560f998be8b0 0x560f998bed00 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd0c80046a0 tx=0x7fd0c800b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65211fd52edee28c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:11.892 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.886+0000 7fd0b77fe700 1 -- 172.21.15.52:0/3219190397 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8004a30 con 0x560f998be8b0 2023-11-27T15:19:11.892 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.886+0000 7fd0b77fe700 1 -- 172.21.15.52:0/3219190397 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0c80054e0 con 0x560f998be8b0 2023-11-27T15:19:11.892 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.886+0000 7fd0b77fe700 1 -- 172.21.15.52:0/3219190397 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8003b20 con 0x560f998be8b0 2023-11-27T15:19:11.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.887+0000 7fd0e38b2b80 1 -- 172.21.15.52:0/3219190397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560f998be8b0 msgr2=0x560f998bed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:11.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.887+0000 7fd0e38b2b80 1 --2- 172.21.15.52:0/3219190397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560f998be8b0 0x560f998bed00 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd0c80046a0 tx=0x7fd0c800b970 comp rx=0 tx=0).stop 2023-11-27T15:19:11.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.887+0000 7fd0e38b2b80 1 -- 172.21.15.52:0/3219190397 shutdown_connections 2023-11-27T15:19:11.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.887+0000 7fd0e38b2b80 1 --2- 172.21.15.52:0/3219190397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560f998be8b0 0x560f998bed00 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.887+0000 7fd0e38b2b80 1 --2- 172.21.15.52:0/3219190397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f998b9b60 0x560f998b9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.887+0000 7fd0e38b2b80 1 --2- 172.21.15.52:0/3219190397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560f998ba470 0x560f998be370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.887+0000 7fd0e38b2b80 1 -- 172.21.15.52:0/3219190397 >> 172.21.15.52:0/3219190397 conn(0x560f998af360 msgr2=0x560f998b4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:11.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.887+0000 7fd0e38b2b80 1 -- 172.21.15.52:0/3219190397 shutdown_connections 2023-11-27T15:19:11.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.887+0000 7fd0e38b2b80 1 -- 172.21.15.52:0/3219190397 wait complete. 2023-11-27T15:19:11.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.887+0000 7fd0e38b2b80 1 -- start start 2023-11-27T15:19:11.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.887+0000 7fd0e38b2b80 1 Processor -- start 2023-11-27T15:19:11.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.888+0000 7fd0e38b2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f998b9b60 0x560f998c6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:11.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.888+0000 7fd0e38b2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560f998ba470 0x560f998c32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:11.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.888+0000 7fd0e38b2b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560f998be8b0 0x560f998c3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:11.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.888+0000 7fd0e38b2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f997ac8e0 con 0x560f998ba470 2023-11-27T15:19:11.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.888+0000 7fd0e38b2b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f997ac580 con 0x560f998be8b0 2023-11-27T15:19:11.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.888+0000 7fd0e38b2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f9968e920 con 0x560f998b9b60 2023-11-27T15:19:11.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.888+0000 7fd0e38b2b80 10 client.0 _ll_register_callbacks cb 0x560f997ca780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:19:11.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.888+0000 7fd0d4cc1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560f998ba470 0x560f998c32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:11.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.888+0000 7fd0d4cc1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560f998ba470 0x560f998c32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52660/0 (socket says 172.21.15.52:52660) 2023-11-27T15:19:11.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.888+0000 7fd0d4cc1700 1 -- 172.21.15.52:0/3336904870 learned_addr learned my addr 172.21.15.52:0/3336904870 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:11.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.888+0000 7fd0d54c2700 1 --2- 172.21.15.52:0/3336904870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f998b9b60 0x560f998c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:11.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.888+0000 7fd0d5cc3700 1 --2- 172.21.15.52:0/3336904870 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560f998be8b0 0x560f998c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:11.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.888+0000 7fd0d4cc1700 1 -- 172.21.15.52:0/3336904870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f998b9b60 msgr2=0x560f998c6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:11.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.888+0000 7fd0d4cc1700 1 --2- 172.21.15.52:0/3336904870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f998b9b60 0x560f998c6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.888+0000 7fd0d4cc1700 1 -- 172.21.15.52:0/3336904870 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560f998be8b0 msgr2=0x560f998c3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:11.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.888+0000 7fd0d4cc1700 1 --2- 172.21.15.52:0/3336904870 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560f998be8b0 0x560f998c3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.888+0000 7fd0d4cc1700 1 -- 172.21.15.52:0/3336904870 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0c80042d0 con 0x560f998ba470 2023-11-27T15:19:11.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.889+0000 7fd0d4cc1700 1 --2- 172.21.15.52:0/3336904870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560f998ba470 0x560f998c32d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd0c409b2c0 tx=0x7fd0c4005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630ac93ea61ad567 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:11.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.889+0000 7fd0b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:19:11.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.889+0000 7fd0b4ff9700 1 -- 172.21.15.52:0/3336904870 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c409ac90 con 0x560f998ba470 2023-11-27T15:19:11.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.889+0000 7fd0b4ff9700 1 -- 172.21.15.52:0/3336904870 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fd0c409f120 con 0x560f998ba470 2023-11-27T15:19:11.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.889+0000 7fd0e38b2b80 1 -- 172.21.15.52:0/3336904870 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f998acb20 con 0x560f998ba470 2023-11-27T15:19:11.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.889+0000 7fd0b4ff9700 1 -- 172.21.15.52:0/3336904870 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0c409b330 con 0x560f998ba470 2023-11-27T15:19:11.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.889+0000 7fd0e38b2b80 10 client.15672 fetch_fsmap learned FSMap version 23 2023-11-27T15:19:11.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.889+0000 7fd0e38b2b80 1 -- 172.21.15.52:0/3336904870 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560f997d1190 con 0x560f998ba470 2023-11-27T15:19:11.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.889+0000 7fd0b4ff9700 1 -- 172.21.15.52:0/3336904870 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0c40b7080 con 0x560f998ba470 2023-11-27T15:19:11.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.889+0000 7fd0b4ff9700 1 -- 172.21.15.52:0/3336904870 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd0c40a32f0 con 0x560f998ba470 2023-11-27T15:19:11.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.889+0000 7fd0e38b2b80 10 client.15672 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:19:11.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.889+0000 7fd0e38b2b80 10 client.15672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:19:11.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.890+0000 7fd0e38b2b80 20 client.15672 populate_metadata read hostname 'smithi052' 2023-11-27T15:19:11.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.890+0000 7fd0e38b2b80 1 --2- 172.21.15.52:0/3336904870 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560f998d3550 0x560f998d5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:11.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.890+0000 7fd0e38b2b80 4 client.15672 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:19:11.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.890+0000 7fd0e38b2b80 1 -- 172.21.15.52:0/3336904870 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f997cd0a0 con 0x560f998d3550 2023-11-27T15:19:11.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.890+0000 7fd0d54c2700 1 --2- 172.21.15.52:0/3336904870 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560f998d3550 0x560f998d5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:11.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.890+0000 7fd0d54c2700 1 --2- 172.21.15.52:0/3336904870 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560f998d3550 0x560f998d5960 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a952540a12d023c3 server_cookie=3c244ce8f5da3757 in_seq=0 out_seq=0 2023-11-27T15:19:11.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.890+0000 7fd0b4ff9700 10 client.15672 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:11.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.891+0000 7fd0b4ff9700 1 -- 172.21.15.52:0/3336904870 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560f997cd0a0 con 0x560f998d3550 2023-11-27T15:19:11.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.891+0000 7fd0b4ff9700 10 client.15672 handle_command_reply: tid=0 2023-11-27T15:19:11.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.891+0000 7fd0e38b2b80 1 client.15672 shutdown 2023-11-27T15:19:11.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.891+0000 7fd0e38b2b80 1 -- 172.21.15.52:0/3336904870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560f998ba470 msgr2=0x560f998c32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:11.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.891+0000 7fd0e38b2b80 1 --2- 172.21.15.52:0/3336904870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560f998ba470 0x560f998c32d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd0c409b2c0 tx=0x7fd0c4005a60 comp rx=0 tx=0).stop 2023-11-27T15:19:11.901 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.891+0000 7fd0e38b2b80 1 -- 172.21.15.52:0/3336904870 shutdown_connections 2023-11-27T15:19:11.901 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.891+0000 7fd0e38b2b80 1 --2- 172.21.15.52:0/3336904870 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560f998d3550 0x560f998d5960 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.901 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.891+0000 7fd0e38b2b80 1 --2- 172.21.15.52:0/3336904870 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560f998be8b0 0x560f998c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.901 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.891+0000 7fd0e38b2b80 1 --2- 172.21.15.52:0/3336904870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f998b9b60 0x560f998c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.901 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.891+0000 7fd0e38b2b80 1 --2- 172.21.15.52:0/3336904870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560f998ba470 0x560f998c32d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.901 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.891+0000 7fd0b4ff9700 0 client.15672 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:11.901 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.891+0000 7fd0e38b2b80 1 -- 172.21.15.52:0/3336904870 >> 172.21.15.52:0/3336904870 conn(0x560f998af360 msgr2=0x560f998b3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:11.901 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.891+0000 7fd0e38b2b80 1 -- 172.21.15.52:0/3336904870 shutdown_connections 2023-11-27T15:19:11.901 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.892+0000 7fd0e38b2b80 1 -- 172.21.15.52:0/3336904870 wait complete. 2023-11-27T15:19:11.901 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.892+0000 7fd0e38b2b80 20 client.15672 trim_cache size 0 max 16384 2023-11-27T15:19:11.914 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.909+0000 7fd0e38b2b80 1 Processor -- start 2023-11-27T15:19:11.915 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.909+0000 7fd0e38b2b80 1 -- start start 2023-11-27T15:19:11.915 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.909+0000 7fd0e38b2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560f998b9b60 0x560f998bc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:11.915 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.909+0000 7fd0e38b2b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560f998ba470 0x560f998bce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:11.915 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.909+0000 7fd0e38b2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f998be8b0 0x560f998ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:11.915 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.909+0000 7fd0e38b2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f998c1830 con 0x560f998b9b60 2023-11-27T15:19:11.915 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.909+0000 7fd0e38b2b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f998bd400 con 0x560f998ba470 2023-11-27T15:19:11.915 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.909+0000 7fd0e38b2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f998caee0 con 0x560f998be8b0 2023-11-27T15:19:11.915 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.910+0000 7fd0d54c2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560f998b9b60 0x560f998bc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:11.915 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.910+0000 7fd0d54c2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560f998b9b60 0x560f998bc8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52666/0 (socket says 172.21.15.52:52666) 2023-11-27T15:19:11.915 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.910+0000 7fd0d54c2700 1 -- 172.21.15.52:0/1926064204 learned_addr learned my addr 172.21.15.52:0/1926064204 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:11.915 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.910+0000 7fd0d5cc3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f998be8b0 0x560f998ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:11.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.910+0000 7fd0d4cc1700 1 --2- 172.21.15.52:0/1926064204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560f998ba470 0x560f998bce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:11.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.910+0000 7fd0d54c2700 1 -- 172.21.15.52:0/1926064204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f998be8b0 msgr2=0x560f998ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:11.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.910+0000 7fd0d54c2700 1 --2- 172.21.15.52:0/1926064204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f998be8b0 0x560f998ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.910+0000 7fd0d54c2700 1 -- 172.21.15.52:0/1926064204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560f998ba470 msgr2=0x560f998bce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:11.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.910+0000 7fd0d54c2700 1 --2- 172.21.15.52:0/1926064204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560f998ba470 0x560f998bce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.910+0000 7fd0d54c2700 1 -- 172.21.15.52:0/1926064204 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f997d0ee0 con 0x560f998b9b60 2023-11-27T15:19:11.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.910+0000 7fd0d5cc3700 1 --2- 172.21.15.52:0/1926064204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f998be8b0 0x560f998ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:11.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.910+0000 7fd0d54c2700 1 --2- 172.21.15.52:0/1926064204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560f998b9b60 0x560f998bc8f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x560f998d6240 tx=0x7fd0c0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c1c2edecc226a6b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:11.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.910+0000 7fd0b7fff700 1 -- 172.21.15.52:0/1926064204 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c0016ca0 con 0x560f998b9b60 2023-11-27T15:19:11.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.910+0000 7fd0b7fff700 1 -- 172.21.15.52:0/1926064204 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0c0016e00 con 0x560f998b9b60 2023-11-27T15:19:11.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.910+0000 7fd0b7fff700 1 -- 172.21.15.52:0/1926064204 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c00048f0 con 0x560f998b9b60 2023-11-27T15:19:11.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.911+0000 7fd0e38b2b80 1 -- 172.21.15.52:0/1926064204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560f998b9b60 msgr2=0x560f998bc8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:11.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.911+0000 7fd0e38b2b80 1 --2- 172.21.15.52:0/1926064204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560f998b9b60 0x560f998bc8f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x560f998d6240 tx=0x7fd0c0015000 comp rx=0 tx=0).stop 2023-11-27T15:19:11.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.911+0000 7fd0e38b2b80 1 -- 172.21.15.52:0/1926064204 shutdown_connections 2023-11-27T15:19:11.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.911+0000 7fd0e38b2b80 1 --2- 172.21.15.52:0/1926064204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f998be8b0 0x560f998ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.911+0000 7fd0e38b2b80 1 --2- 172.21.15.52:0/1926064204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560f998b9b60 0x560f998bc8f0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.911+0000 7fd0e38b2b80 1 --2- 172.21.15.52:0/1926064204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560f998ba470 0x560f998bce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.911+0000 7fd0e38b2b80 1 -- 172.21.15.52:0/1926064204 >> 172.21.15.52:0/1926064204 conn(0x560f998af360 msgr2=0x560f998c0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:11.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.911+0000 7fd0e38b2b80 1 -- 172.21.15.52:0/1926064204 shutdown_connections 2023-11-27T15:19:11.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.911+0000 7fd0e38b2b80 1 -- 172.21.15.52:0/1926064204 wait complete. 2023-11-27T15:19:11.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.912+0000 7fd0e38b2b80 1 -- start start 2023-11-27T15:19:11.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.912+0000 7fd0e38b2b80 1 Processor -- start 2023-11-27T15:19:11.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.912+0000 7fd0e38b2b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560f998b9b60 0x560f9991f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:11.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.912+0000 7fd0e38b2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f998ba470 0x560f9991fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:11.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.912+0000 7fd0e38b2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560f998be8b0 0x560f998c6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:11.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.912+0000 7fd0e38b2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f998c6f90 con 0x560f998be8b0 2023-11-27T15:19:11.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.912+0000 7fd0e38b2b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f998c7100 con 0x560f998b9b60 2023-11-27T15:19:11.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.912+0000 7fd0e38b2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f998c72c0 con 0x560f998ba470 2023-11-27T15:19:11.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.912+0000 7fd0e38b2b80 10 client.0 _ll_register_callbacks cb 0x7fd0c409b1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:19:11.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.912+0000 7fd0d5cc3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560f998be8b0 0x560f998c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:11.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.912+0000 7fd0d4cc1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f998ba470 0x560f9991fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:11.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.912+0000 7fd0d5cc3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560f998be8b0 0x560f998c6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52676/0 (socket says 172.21.15.52:52676) 2023-11-27T15:19:11.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.912+0000 7fd0d5cc3700 1 -- 172.21.15.52:0/3889508547 learned_addr learned my addr 172.21.15.52:0/3889508547 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:11.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.912+0000 7fd0d54c2700 1 --2- 172.21.15.52:0/3889508547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560f998b9b60 0x560f9991f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:11.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.913+0000 7fd0d4cc1700 1 -- 172.21.15.52:0/3889508547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560f998b9b60 msgr2=0x560f9991f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:11.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.913+0000 7fd0d4cc1700 1 --2- 172.21.15.52:0/3889508547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560f998b9b60 0x560f9991f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.913+0000 7fd0d4cc1700 1 -- 172.21.15.52:0/3889508547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560f998be8b0 msgr2=0x560f998c6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:11.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.913+0000 7fd0d4cc1700 1 --2- 172.21.15.52:0/3889508547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560f998be8b0 0x560f998c6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.913+0000 7fd0d4cc1700 1 -- 172.21.15.52:0/3889508547 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f9968c630 con 0x560f998ba470 2023-11-27T15:19:11.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.913+0000 7fd0d5cc3700 1 --2- 172.21.15.52:0/3889508547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560f998be8b0 0x560f998c6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:11.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.913+0000 7fd0d54c2700 1 --2- 172.21.15.52:0/3889508547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560f998b9b60 0x560f9991f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:11.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.913+0000 7fd0d4cc1700 1 --2- 172.21.15.52:0/3889508547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f998ba470 0x560f9991fc40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd0c409b030 tx=0x7fd0c4097700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84f694621d375e42 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:11.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.913+0000 7fd0b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2023-11-27T15:19:11.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.913+0000 7fd0b4ff9700 1 -- 172.21.15.52:0/3889508547 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c40cbcd0 con 0x560f998ba470 2023-11-27T15:19:11.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.913+0000 7fd0b4ff9700 1 -- 172.21.15.52:0/3889508547 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fd0c409f120 con 0x560f998ba470 2023-11-27T15:19:11.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.913+0000 7fd0e38b2b80 1 -- 172.21.15.52:0/3889508547 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd0c80054e0 con 0x560f998ba470 2023-11-27T15:19:11.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.914+0000 7fd0b4ff9700 1 -- 172.21.15.52:0/3889508547 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0c409ac90 con 0x560f998ba470 2023-11-27T15:19:11.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.914+0000 7fd0e38b2b80 10 client.25408 fetch_fsmap learned FSMap version 23 2023-11-27T15:19:11.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.914+0000 7fd0e38b2b80 1 -- 172.21.15.52:0/3889508547 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fd0c001e040 con 0x560f998ba470 2023-11-27T15:19:11.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.914+0000 7fd0b4ff9700 1 -- 172.21.15.52:0/3889508547 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0c80054e0 con 0x560f998ba470 2023-11-27T15:19:11.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.914+0000 7fd0b4ff9700 1 -- 172.21.15.52:0/3889508547 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd0c4097120 con 0x560f998ba470 2023-11-27T15:19:11.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.914+0000 7fd0e38b2b80 10 client.25408 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:19:11.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.914+0000 7fd0e38b2b80 10 client.25408 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:19:11.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.914+0000 7fd0e38b2b80 20 client.25408 populate_metadata read hostname 'smithi052' 2023-11-27T15:19:11.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.914+0000 7fd0e38b2b80 1 --2- 172.21.15.52:0/3889508547 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560f99928170 0x560f9992a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:11.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.914+0000 7fd0e38b2b80 4 client.25408 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:19:11.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.914+0000 7fd0e38b2b80 1 -- 172.21.15.52:0/3889508547 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560f997d9660 con 0x560f99928170 2023-11-27T15:19:11.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.914+0000 7fd0d54c2700 1 --2- 172.21.15.52:0/3889508547 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560f99928170 0x560f9992a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:11.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.915+0000 7fd0b4ff9700 10 client.25408 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:11.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.915+0000 7fd0d54c2700 1 --2- 172.21.15.52:0/3889508547 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560f99928170 0x560f9992a580 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c58c1501c947180 server_cookie=729f12608866ffc in_seq=0 out_seq=0 2023-11-27T15:19:11.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.915+0000 7fd0b4ff9700 1 -- 172.21.15.52:0/3889508547 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560f997d9660 con 0x560f99928170 2023-11-27T15:19:11.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.915+0000 7fd0b4ff9700 10 client.25408 handle_command_reply: tid=0 2023-11-27T15:19:11.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.915+0000 7fd0e38b2b80 1 client.25408 shutdown 2023-11-27T15:19:11.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.915+0000 7fd0e38b2b80 1 -- 172.21.15.52:0/3889508547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f998ba470 msgr2=0x560f9991fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:11.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.915+0000 7fd0e38b2b80 1 --2- 172.21.15.52:0/3889508547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f998ba470 0x560f9991fc40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd0c409b030 tx=0x7fd0c4097700 comp rx=0 tx=0).stop 2023-11-27T15:19:11.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.915+0000 7fd0e38b2b80 1 -- 172.21.15.52:0/3889508547 shutdown_connections 2023-11-27T15:19:11.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.915+0000 7fd0e38b2b80 1 --2- 172.21.15.52:0/3889508547 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560f99928170 0x560f9992a580 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.915+0000 7fd0e38b2b80 1 --2- 172.21.15.52:0/3889508547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560f998b9b60 0x560f9991f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.915+0000 7fd0b4ff9700 0 client.25408 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:11.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.915+0000 7fd0e38b2b80 1 --2- 172.21.15.52:0/3889508547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560f998be8b0 0x560f998c6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.915+0000 7fd0e38b2b80 1 --2- 172.21.15.52:0/3889508547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560f998ba470 0x560f9991fc40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.915+0000 7fd0e38b2b80 1 -- 172.21.15.52:0/3889508547 >> 172.21.15.52:0/3889508547 conn(0x560f998af360 msgr2=0x560f998c0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:11.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.915+0000 7fd0e38b2b80 1 -- 172.21.15.52:0/3889508547 shutdown_connections 2023-11-27T15:19:11.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.915+0000 7fd0e38b2b80 1 -- 172.21.15.52:0/3889508547 wait complete. 2023-11-27T15:19:11.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.915+0000 7fd0e38b2b80 20 client.25408 trim_cache size 0 max 16384 2023-11-27T15:19:11.926 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:19:11.926 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2023-11-27T15:19:11.926 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2023-11-27T15:19:11.926 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:19:11.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.916+0000 7fd0d7726700 1 -- 172.21.15.52:0/3379601563 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd0b80a13e0 msgr2=0x7fd0b80a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:11.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.916+0000 7fd0d7726700 1 --2- 172.21.15.52:0/3379601563 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd0b80a13e0 0x7fd0b80a3890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0009dd0 tx=0x7fd0c0009040 comp rx=0 tx=0).stop 2023-11-27T15:19:11.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.916+0000 7fd0d7726700 1 -- 172.21.15.52:0/3379601563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0d0069a80 msgr2=0x7fd0d0196910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:11.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.916+0000 7fd0d7726700 1 --2- 172.21.15.52:0/3379601563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0d0069a80 0x7fd0d0196910 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4009990 tx=0x7fd0c4004730 comp rx=0 tx=0).stop 2023-11-27T15:19:11.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.916+0000 7fd0d7726700 1 -- 172.21.15.52:0/3379601563 shutdown_connections 2023-11-27T15:19:11.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.916+0000 7fd0d7726700 1 --2- 172.21.15.52:0/3379601563 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd0b80a13e0 0x7fd0b80a3890 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.916+0000 7fd0d7726700 1 --2- 172.21.15.52:0/3379601563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0d00690e0 0x7fd0d01963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.916+0000 7fd0d7726700 1 --2- 172.21.15.52:0/3379601563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0d0105600 0x7fd0d019ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.916+0000 7fd0d7726700 1 --2- 172.21.15.52:0/3379601563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0d0069a80 0x7fd0d0196910 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:11.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.916+0000 7fd0d7726700 1 -- 172.21.15.52:0/3379601563 >> 172.21.15.52:0/3379601563 conn(0x7fd0d0075470 msgr2=0x7fd0d00fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:11.928 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.916+0000 7fd0d7726700 1 -- 172.21.15.52:0/3379601563 shutdown_connections 2023-11-27T15:19:11.928 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:11.916+0000 7fd0d7726700 1 -- 172.21.15.52:0/3379601563 wait complete. 2023-11-27T15:19:12.003 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-27T15:19:12.003 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:19:12.647 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:12.647 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:12.899 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:12.899 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:12.922 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:12.922 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:12.922 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 39 -lt 100 ']' 2023-11-27T15:19:12.922 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 39 + 1 2023-11-27T15:19:12.973 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=40 2023-11-27T15:19:12.974 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:12.989 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:12.989 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:12.990 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:12.990 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:12.990 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:13.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:12 smithi052 ceph-mon[99056]: pgmap v41: 97 pgs: 97 active+clean; 1.8 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 47 KiB/s wr, 21 op/s 2023-11-27T15:19:13.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:12 smithi150 ceph-mon[100559]: pgmap v41: 97 pgs: 97 active+clean; 1.8 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 47 KiB/s wr, 21 op/s 2023-11-27T15:19:13.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:12 smithi102 ceph-mon[100296]: pgmap v41: 97 pgs: 97 active+clean; 1.8 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 47 KiB/s wr, 21 op/s 2023-11-27T15:19:15.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:14 smithi052 ceph-mon[99056]: pgmap v42: 97 pgs: 97 active+clean; 1.8 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 18 op/s 2023-11-27T15:19:15.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:14 smithi150 ceph-mon[100559]: pgmap v42: 97 pgs: 97 active+clean; 1.8 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 18 op/s 2023-11-27T15:19:15.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:14 smithi102 ceph-mon[100296]: pgmap v42: 97 pgs: 97 active+clean; 1.8 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 18 op/s 2023-11-27T15:19:15.405 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:15.405 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:15.550 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:15.550 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:15.579 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:15.579 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:15.579 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 40 -lt 100 ']' 2023-11-27T15:19:15.579 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 40 + 1 2023-11-27T15:19:15.619 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=41 2023-11-27T15:19:15.620 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:15.699 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:15.699 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:15.700 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:15.700 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:15.700 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:16.338 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:16.339 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:16.457 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:16.457 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:16.496 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:16.496 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:16.496 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 41 -lt 100 ']' 2023-11-27T15:19:16.496 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 41 + 1 2023-11-27T15:19:16.578 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=42 2023-11-27T15:19:16.578 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:16.610 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:16.610 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:16.612 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:16.612 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:16.612 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:17.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:16 smithi052 ceph-mon[99056]: pgmap v43: 97 pgs: 97 active+clean; 1.8 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 6 op/s 2023-11-27T15:19:17.230 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:17.230 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:17.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:16 smithi150 ceph-mon[100559]: pgmap v43: 97 pgs: 97 active+clean; 1.8 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 6 op/s 2023-11-27T15:19:17.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:16 smithi102 ceph-mon[100296]: pgmap v43: 97 pgs: 97 active+clean; 1.8 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 6 op/s 2023-11-27T15:19:17.448 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:17.449 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:17.469 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:17.469 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:17.469 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 42 -lt 100 ']' 2023-11-27T15:19:17.470 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 42 + 1 2023-11-27T15:19:17.502 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=43 2023-11-27T15:19:17.502 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:17.562 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:17.562 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:17.563 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:17.563 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:17.563 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:17.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.772+0000 7ff23d835700 1 -- 172.21.15.52:0/2482727749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff238106380 msgr2=0x7ff238106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:17.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.772+0000 7ff23d835700 1 --2- 172.21.15.52:0/2482727749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff238106380 0x7ff238106750 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff2280099c0 tx=0x7ff228009cd0 comp rx=0 tx=0).stop 2023-11-27T15:19:17.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.772+0000 7ff23d835700 1 -- 172.21.15.52:0/2482727749 shutdown_connections 2023-11-27T15:19:17.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.772+0000 7ff23d835700 1 --2- 172.21.15.52:0/2482727749 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff238069770 0x7ff23810d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:17.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.772+0000 7ff23d835700 1 --2- 172.21.15.52:0/2482727749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff238106380 0x7ff238106750 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:17.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.772+0000 7ff23d835700 1 --2- 172.21.15.52:0/2482727749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff238068dc0 0x7ff238069230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:17.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.772+0000 7ff23d835700 1 -- 172.21.15.52:0/2482727749 >> 172.21.15.52:0/2482727749 conn(0x7ff2380763a0 msgr2=0x7ff2380767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:17.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.772+0000 7ff23d835700 1 -- 172.21.15.52:0/2482727749 shutdown_connections 2023-11-27T15:19:17.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.772+0000 7ff23d835700 1 -- 172.21.15.52:0/2482727749 wait complete. 2023-11-27T15:19:17.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.773+0000 7ff23d835700 1 Processor -- start 2023-11-27T15:19:17.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.773+0000 7ff23d835700 1 -- start start 2023-11-27T15:19:17.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.773+0000 7ff23d835700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff238068dc0 0x7ff2380ffb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:17.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.773+0000 7ff23d835700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff238069770 0x7ff238100090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:17.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.773+0000 7ff23d835700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff238106380 0x7ff2381a4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:17.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.773+0000 7ff23d835700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2381008c0 con 0x7ff238106380 2023-11-27T15:19:17.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.773+0000 7ff23d835700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2381a4a70 con 0x7ff238068dc0 2023-11-27T15:19:17.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.773+0000 7ff23d835700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2381a4bb0 con 0x7ff238069770 2023-11-27T15:19:17.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.773+0000 7ff2377fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff238106380 0x7ff2381a4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:17.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.773+0000 7ff236ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff238068dc0 0x7ff2380ffb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:17.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.773+0000 7ff2377fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff238106380 0x7ff2381a4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52690/0 (socket says 172.21.15.52:52690) 2023-11-27T15:19:17.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.773+0000 7ff2377fe700 1 -- 172.21.15.52:0/2945520257 learned_addr learned my addr 172.21.15.52:0/2945520257 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:17.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.773+0000 7ff2367fc700 1 --2- 172.21.15.52:0/2945520257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff238069770 0x7ff238100090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:17.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.774+0000 7ff236ffd700 1 -- 172.21.15.52:0/2945520257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff238069770 msgr2=0x7ff238100090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:17.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.774+0000 7ff236ffd700 1 --2- 172.21.15.52:0/2945520257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff238069770 0x7ff238100090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:17.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.774+0000 7ff236ffd700 1 -- 172.21.15.52:0/2945520257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff238106380 msgr2=0x7ff2381a4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:17.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.774+0000 7ff236ffd700 1 --2- 172.21.15.52:0/2945520257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff238106380 0x7ff2381a4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:17.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.774+0000 7ff236ffd700 1 -- 172.21.15.52:0/2945520257 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2280095f0 con 0x7ff238068dc0 2023-11-27T15:19:17.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.774+0000 7ff2377fe700 1 --2- 172.21.15.52:0/2945520257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff238106380 0x7ff2381a4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:17.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.774+0000 7ff236ffd700 1 --2- 172.21.15.52:0/2945520257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff238068dc0 0x7ff2380ffb50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff228009f50 tx=0x7ff228005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:17.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.774+0000 7ff23c833700 1 -- 172.21.15.52:0/2945520257 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22801d070 con 0x7ff238068dc0 2023-11-27T15:19:17.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.774+0000 7ff23d835700 1 -- 172.21.15.52:0/2945520257 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2381103a0 con 0x7ff238068dc0 2023-11-27T15:19:17.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.774+0000 7ff23c833700 1 -- 172.21.15.52:0/2945520257 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff228003f90 con 0x7ff238068dc0 2023-11-27T15:19:17.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.774+0000 7ff23c833700 1 -- 172.21.15.52:0/2945520257 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff228022b60 con 0x7ff238068dc0 2023-11-27T15:19:17.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.774+0000 7ff23d835700 1 -- 172.21.15.52:0/2945520257 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff238110210 con 0x7ff238068dc0 2023-11-27T15:19:17.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.775+0000 7ff23d835700 1 -- 172.21.15.52:0/2945520257 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff238110530 con 0x7ff238068dc0 2023-11-27T15:19:17.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.777+0000 7ff23c833700 1 -- 172.21.15.52:0/2945520257 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7ff228003ad0 con 0x7ff238068dc0 2023-11-27T15:19:17.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.777+0000 7ff23c833700 1 --2- 172.21.15.52:0/2945520257 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ff2240a1460 0x7ff2240a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:17.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.778+0000 7ff2367fc700 1 --2- 172.21.15.52:0/2945520257 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ff2240a1460 0x7ff2240a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:17.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.778+0000 7ff23c833700 1 -- 172.21.15.52:0/2945520257 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff2280ceb10 con 0x7ff238068dc0 2023-11-27T15:19:17.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.778+0000 7ff2367fc700 1 --2- 172.21.15.52:0/2945520257 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ff2240a1460 0x7ff2240a3910 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff238101340 tx=0x7ff22000a380 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:17.785 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:17.781+0000 7ff23c833700 1 -- 172.21.15.52:0/2945520257 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff228097d00 con 0x7ff238068dc0 2023-11-27T15:19:18.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:18.036+0000 7ff23d835700 1 -- 172.21.15.52:0/2945520257 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2380671d0 con 0x7ff238068dc0 2023-11-27T15:19:18.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:18.037+0000 7ff23c833700 1 -- 172.21.15.52:0/2945520257 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10804 (secure 0 0 0) 0x7ff22809b320 con 0x7ff238068dc0 2023-11-27T15:19:18.041 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:19:18.041 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2328881376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2328881376},{"type":"v1","addr":"172.21.15.52:6839","nonce":2328881376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2799551570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2799551570},{"type":"v1","addr":"172.21.15.52:6841","nonce":2799551570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:16:09.319630+0000","modified":"2023-11-27T15:16:14.742405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24494,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/215725838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":215725838},{"type":"v1","addr":"172.21.15.150:6839","nonce":215725838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3571830559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3571830559},{"type":"v1","addr":"172.21.15.102:6839","nonce":3571830559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:19:18.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:18.039+0000 7ff23d835700 1 -- 172.21.15.52:0/2945520257 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ff2240a1460 msgr2=0x7ff2240a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:18.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:18.039+0000 7ff23d835700 1 --2- 172.21.15.52:0/2945520257 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ff2240a1460 0x7ff2240a3910 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff238101340 tx=0x7ff22000a380 comp rx=0 tx=0).stop 2023-11-27T15:19:18.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:18.039+0000 7ff23d835700 1 -- 172.21.15.52:0/2945520257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff238068dc0 msgr2=0x7ff2380ffb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:18.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:18.039+0000 7ff23d835700 1 --2- 172.21.15.52:0/2945520257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff238068dc0 0x7ff2380ffb50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff228009f50 tx=0x7ff228005740 comp rx=0 tx=0).stop 2023-11-27T15:19:18.044 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:18.039+0000 7ff23d835700 1 -- 172.21.15.52:0/2945520257 shutdown_connections 2023-11-27T15:19:18.044 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:18.039+0000 7ff23d835700 1 --2- 172.21.15.52:0/2945520257 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ff2240a1460 0x7ff2240a3910 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:18.044 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:18.039+0000 7ff23d835700 1 --2- 172.21.15.52:0/2945520257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff238068dc0 0x7ff2380ffb50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:18.044 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:18.039+0000 7ff23d835700 1 --2- 172.21.15.52:0/2945520257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff238106380 0x7ff2381a4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:18.044 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:18.039+0000 7ff23d835700 1 --2- 172.21.15.52:0/2945520257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff238069770 0x7ff238100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:18.044 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:18.039+0000 7ff23d835700 1 -- 172.21.15.52:0/2945520257 >> 172.21.15.52:0/2945520257 conn(0x7ff2380763a0 msgr2=0x7ff2380fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:18.044 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:18.039+0000 7ff23d835700 1 -- 172.21.15.52:0/2945520257 shutdown_connections 2023-11-27T15:19:18.044 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:18.039+0000 7ff23d835700 1 -- 172.21.15.52:0/2945520257 wait complete. 2023-11-27T15:19:18.044 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 23 2023-11-27T15:19:18.105 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:18.105 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:18.161 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-27T15:19:18.298 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:18.298 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:18.337 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:18.337 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:18.337 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 43 -lt 100 ']' 2023-11-27T15:19:18.337 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 43 + 1 2023-11-27T15:19:18.377 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=44 2023-11-27T15:19:18.378 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:18.383 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:18.383 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:18.384 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:18.384 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:18.384 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:18.872 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:18.872 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:18.963 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:18.963 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:18.993 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:18.993 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:18.993 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 44 -lt 100 ']' 2023-11-27T15:19:18.994 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 44 + 1 2023-11-27T15:19:19.061 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=45 2023-11-27T15:19:19.061 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:19.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:18 smithi052 ceph-mon[99056]: pgmap v44: 97 pgs: 97 active+clean; 1.9 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-11-27T15:19:19.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:18 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2945520257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:19:19.125 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:19.125 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:19.126 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:19.127 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:19.127 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:19.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:18 smithi150 ceph-mon[100559]: pgmap v44: 97 pgs: 97 active+clean; 1.9 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-11-27T15:19:19.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:18 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2945520257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:19:19.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:18 smithi102 ceph-mon[100296]: pgmap v44: 97 pgs: 97 active+clean; 1.9 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 10 op/s 2023-11-27T15:19:19.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:18 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2945520257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:19:20.088 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:20.088 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:20.215 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:20.215 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:20.236 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:20.236 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:20.236 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 45 -lt 100 ']' 2023-11-27T15:19:20.236 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 45 + 1 2023-11-27T15:19:20.286 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=46 2023-11-27T15:19:20.286 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:20.337 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:20.337 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:20.338 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:20.338 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:20.338 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:21.029 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:21.030 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:21.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:20 smithi052 ceph-mon[99056]: pgmap v45: 97 pgs: 97 active+clean; 1.9 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 7 op/s 2023-11-27T15:19:21.256 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:21.256 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:21.278 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:21.278 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:21.278 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 46 -lt 100 ']' 2023-11-27T15:19:21.278 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 46 + 1 2023-11-27T15:19:21.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:20 smithi150 ceph-mon[100559]: pgmap v45: 97 pgs: 97 active+clean; 1.9 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 7 op/s 2023-11-27T15:19:21.319 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=47 2023-11-27T15:19:21.319 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:21.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:20 smithi102 ceph-mon[100296]: pgmap v45: 97 pgs: 97 active+clean; 1.9 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 7 op/s 2023-11-27T15:19:21.411 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:21.412 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:21.413 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:21.413 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:21.413 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:21.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.680+0000 7fe6e5d18700 1 -- 172.21.15.52:0/3879958964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6e0105620 msgr2=0x7fe6e010d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:21.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.680+0000 7fe6e5d18700 1 --2- 172.21.15.52:0/3879958964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6e0105620 0x7fe6e010d1a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe6d800b0d0 tx=0x7fe6d800b3e0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.680+0000 7fe6e5d18700 1 -- 172.21.15.52:0/3879958964 shutdown_connections 2023-11-27T15:19:21.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.680+0000 7fe6e5d18700 1 --2- 172.21.15.52:0/3879958964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6e0105620 0x7fe6e010d1a0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.680+0000 7fe6e5d18700 1 --2- 172.21.15.52:0/3879958964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6e00693a0 0x7fe6e0069770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.680+0000 7fe6e5d18700 1 --2- 172.21.15.52:0/3879958964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6e0069d40 0x7fe6e0104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.680+0000 7fe6e5d18700 1 -- 172.21.15.52:0/3879958964 >> 172.21.15.52:0/3879958964 conn(0x7fe6e00763a0 msgr2=0x7fe6e00767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:21.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.680+0000 7fe6e5d18700 1 -- 172.21.15.52:0/3879958964 shutdown_connections 2023-11-27T15:19:21.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.680+0000 7fe6e5d18700 1 -- 172.21.15.52:0/3879958964 wait complete. 2023-11-27T15:19:21.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.681+0000 7fe6e5d18700 1 Processor -- start 2023-11-27T15:19:21.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.681+0000 7fe6e5d18700 1 -- start start 2023-11-27T15:19:21.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.681+0000 7fe6e5d18700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6e00693a0 0x7fe6e0196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:21.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.681+0000 7fe6e5d18700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6e0069d40 0x7fe6e0196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:21.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.681+0000 7fe6e5d18700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6e0105620 0x7fe6e019af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:21.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.681+0000 7fe6e5d18700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e019b550 con 0x7fe6e00693a0 2023-11-27T15:19:21.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.681+0000 7fe6e5d18700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e019b6c0 con 0x7fe6e0105620 2023-11-27T15:19:21.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.681+0000 7fe6e5d18700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e019b880 con 0x7fe6e0069d40 2023-11-27T15:19:21.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.681+0000 7fe6df7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6e00693a0 0x7fe6e0196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:21.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.682+0000 7fe6df7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6e00693a0 0x7fe6e0196570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52700/0 (socket says 172.21.15.52:52700) 2023-11-27T15:19:21.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.682+0000 7fe6df7fe700 1 -- 172.21.15.52:0/3833406030 learned_addr learned my addr 172.21.15.52:0/3833406030 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:21.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.682+0000 7fe6dffff700 1 --2- 172.21.15.52:0/3833406030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6e0105620 0x7fe6e019af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:21.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.682+0000 7fe6deffd700 1 --2- 172.21.15.52:0/3833406030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6e0069d40 0x7fe6e0196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:21.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.682+0000 7fe6df7fe700 1 -- 172.21.15.52:0/3833406030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6e0069d40 msgr2=0x7fe6e0196ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:21.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.682+0000 7fe6df7fe700 1 --2- 172.21.15.52:0/3833406030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6e0069d40 0x7fe6e0196ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.682+0000 7fe6df7fe700 1 -- 172.21.15.52:0/3833406030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6e0105620 msgr2=0x7fe6e019af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:21.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.682+0000 7fe6df7fe700 1 --2- 172.21.15.52:0/3833406030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6e0105620 0x7fe6e019af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.682+0000 7fe6df7fe700 1 -- 172.21.15.52:0/3833406030 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d8009d00 con 0x7fe6e00693a0 2023-11-27T15:19:21.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.682+0000 7fe6dffff700 1 --2- 172.21.15.52:0/3833406030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6e0105620 0x7fe6e019af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:21.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.682+0000 7fe6df7fe700 1 --2- 172.21.15.52:0/3833406030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6e00693a0 0x7fe6e0196570 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe6d000e9a0 tx=0x7fe6d000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:21.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.682+0000 7fe6dcff9700 1 -- 172.21.15.52:0/3833406030 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0009790 con 0x7fe6e00693a0 2023-11-27T15:19:21.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.682+0000 7fe6dcff9700 1 -- 172.21.15.52:0/3833406030 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6d0010430 con 0x7fe6e00693a0 2023-11-27T15:19:21.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.683+0000 7fe6e5d18700 1 -- 172.21.15.52:0/3833406030 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e0110380 con 0x7fe6e00693a0 2023-11-27T15:19:21.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.683+0000 7fe6dcff9700 1 -- 172.21.15.52:0/3833406030 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0018680 con 0x7fe6e00693a0 2023-11-27T15:19:21.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.683+0000 7fe6e5d18700 1 -- 172.21.15.52:0/3833406030 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e01101f0 con 0x7fe6e00693a0 2023-11-27T15:19:21.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.685+0000 7fe6dcff9700 1 -- 172.21.15.52:0/3833406030 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7fe6d00105a0 con 0x7fe6e00693a0 2023-11-27T15:19:21.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.686+0000 7fe6dcff9700 1 --2- 172.21.15.52:0/3833406030 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe6c80a1460 0x7fe6c80a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:21.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.686+0000 7fe6deffd700 1 --2- 172.21.15.52:0/3833406030 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe6c80a1460 0x7fe6c80a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:21.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.686+0000 7fe6dcff9700 1 -- 172.21.15.52:0/3833406030 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fe6d0014070 con 0x7fe6e00693a0 2023-11-27T15:19:21.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.686+0000 7fe6deffd700 1 --2- 172.21.15.52:0/3833406030 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe6c80a1460 0x7fe6c80a3910 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0197b80 tx=0x7fe6d4009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:21.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.756+0000 7fe6f1ea4b80 1 Processor -- start 2023-11-27T15:19:21.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.756+0000 7fe6f1ea4b80 1 -- start start 2023-11-27T15:19:21.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.757+0000 7fe6f1ea4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604b4794b60 0x5604b4794f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:21.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.757+0000 7fe6f1ea4b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604b4795470 0x5604b4799370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:21.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.757+0000 7fe6f1ea4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604b47998b0 0x5604b4799d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:21.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.757+0000 7fe6f1ea4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604b469a060 con 0x5604b47998b0 2023-11-27T15:19:21.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.757+0000 7fe6f1ea4b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604b46ec8a0 con 0x5604b4795470 2023-11-27T15:19:21.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.757+0000 7fe6f1ea4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604b46e5d30 con 0x5604b4794b60 2023-11-27T15:19:21.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.757+0000 7fe6df7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604b4794b60 0x5604b4794f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:21.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.757+0000 7fe6df7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604b4794b60 0x5604b4794f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:39352/0 (socket says 172.21.15.52:39352) 2023-11-27T15:19:21.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.757+0000 7fe6df7fe700 1 -- 172.21.15.52:0/360050996 learned_addr learned my addr 172.21.15.52:0/360050996 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:21.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.757+0000 7fe6dffff700 1 --2- 172.21.15.52:0/360050996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604b47998b0 0x5604b4799d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:21.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.757+0000 7fe6deffd700 1 --2- 172.21.15.52:0/360050996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604b4795470 0x5604b4799370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:21.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.757+0000 7fe6df7fe700 1 -- 172.21.15.52:0/360050996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604b4795470 msgr2=0x5604b4799370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:21.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.757+0000 7fe6df7fe700 1 --2- 172.21.15.52:0/360050996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604b4795470 0x5604b4799370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.757+0000 7fe6df7fe700 1 -- 172.21.15.52:0/360050996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604b47998b0 msgr2=0x5604b4799d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:21.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.757+0000 7fe6df7fe700 1 --2- 172.21.15.52:0/360050996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604b47998b0 0x5604b4799d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.757+0000 7fe6df7fe700 1 -- 172.21.15.52:0/360050996 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604b4567630 con 0x5604b4794b60 2023-11-27T15:19:21.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.758+0000 7fe6df7fe700 1 --2- 172.21.15.52:0/360050996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604b4794b60 0x5604b4794f30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0099f50 tx=0x7fe6d00999f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85e7744325683943 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:21.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.758+0000 7fe6c5ffb700 1 -- 172.21.15.52:0/360050996 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d00a2410 con 0x5604b4794b60 2023-11-27T15:19:21.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.758+0000 7fe6c5ffb700 1 -- 172.21.15.52:0/360050996 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6d0010880 con 0x5604b4794b60 2023-11-27T15:19:21.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.758+0000 7fe6c5ffb700 1 -- 172.21.15.52:0/360050996 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0094e10 con 0x5604b4794b60 2023-11-27T15:19:21.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.759+0000 7fe6f1ea4b80 1 -- 172.21.15.52:0/360050996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604b4794b60 msgr2=0x5604b4794f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:21.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.759+0000 7fe6f1ea4b80 1 --2- 172.21.15.52:0/360050996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604b4794b60 0x5604b4794f30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0099f50 tx=0x7fe6d00999f0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.759+0000 7fe6f1ea4b80 1 -- 172.21.15.52:0/360050996 shutdown_connections 2023-11-27T15:19:21.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.759+0000 7fe6f1ea4b80 1 --2- 172.21.15.52:0/360050996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604b47998b0 0x5604b4799d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.759+0000 7fe6f1ea4b80 1 --2- 172.21.15.52:0/360050996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604b4794b60 0x5604b4794f30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.759+0000 7fe6f1ea4b80 1 --2- 172.21.15.52:0/360050996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604b4795470 0x5604b4799370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.759+0000 7fe6f1ea4b80 1 -- 172.21.15.52:0/360050996 >> 172.21.15.52:0/360050996 conn(0x5604b478a360 msgr2=0x5604b478fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:21.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.759+0000 7fe6f1ea4b80 1 -- 172.21.15.52:0/360050996 shutdown_connections 2023-11-27T15:19:21.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.759+0000 7fe6f1ea4b80 1 -- 172.21.15.52:0/360050996 wait complete. 2023-11-27T15:19:21.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.759+0000 7fe6f1ea4b80 1 -- start start 2023-11-27T15:19:21.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.760+0000 7fe6f1ea4b80 1 Processor -- start 2023-11-27T15:19:21.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.760+0000 7fe6f1ea4b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604b4794b60 0x5604b47a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:21.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.760+0000 7fe6f1ea4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604b4795470 0x5604b479e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:21.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.760+0000 7fe6f1ea4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604b47998b0 0x5604b479ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:21.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.760+0000 7fe6f1ea4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604b46878e0 con 0x5604b4795470 2023-11-27T15:19:21.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.760+0000 7fe6f1ea4b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604b4687580 con 0x5604b4794b60 2023-11-27T15:19:21.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.760+0000 7fe6f1ea4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604b4569920 con 0x5604b47998b0 2023-11-27T15:19:21.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.760+0000 7fe6f1ea4b80 10 client.0 _ll_register_callbacks cb 0x5604b46a5780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:19:21.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.760+0000 7fe6deffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604b4795470 0x5604b479e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:21.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.760+0000 7fe6df7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604b4794b60 0x5604b47a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:21.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.760+0000 7fe6df7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604b4794b60 0x5604b47a10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:42372/0 (socket says 172.21.15.52:42372) 2023-11-27T15:19:21.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.760+0000 7fe6deffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604b4795470 0x5604b479e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52716/0 (socket says 172.21.15.52:52716) 2023-11-27T15:19:21.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.760+0000 7fe6df7fe700 1 -- 172.21.15.52:0/35118728 learned_addr learned my addr 172.21.15.52:0/35118728 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:21.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.760+0000 7fe6dffff700 1 --2- 172.21.15.52:0/35118728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604b47998b0 0x5604b479ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:21.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.761+0000 7fe6df7fe700 1 -- 172.21.15.52:0/35118728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604b47998b0 msgr2=0x5604b479ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:21.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.761+0000 7fe6df7fe700 1 --2- 172.21.15.52:0/35118728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604b47998b0 0x5604b479ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.761+0000 7fe6df7fe700 1 -- 172.21.15.52:0/35118728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604b4795470 msgr2=0x5604b479e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:21.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.761+0000 7fe6df7fe700 1 --2- 172.21.15.52:0/35118728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604b4795470 0x5604b479e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.761+0000 7fe6df7fe700 1 -- 172.21.15.52:0/35118728 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e01101f0 con 0x5604b4794b60 2023-11-27T15:19:21.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.761+0000 7fe6deffd700 1 --2- 172.21.15.52:0/35118728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604b4795470 0x5604b479e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:21.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.761+0000 7fe6df7fe700 1 --2- 172.21.15.52:0/35118728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604b4794b60 0x5604b47a10a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0009230 tx=0x7fe6d0099760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bb3751ab6cdbd68 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:21.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.761+0000 7fe6beffd700 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2023-11-27T15:19:21.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.761+0000 7fe6beffd700 1 -- 172.21.15.52:0/35118728 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d00b8070 con 0x5604b4794b60 2023-11-27T15:19:21.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.761+0000 7fe6f1ea4b80 1 -- 172.21.15.52:0/35118728 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604b4787b20 con 0x5604b4794b60 2023-11-27T15:19:21.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.761+0000 7fe6beffd700 1 -- 172.21.15.52:0/35118728 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fe6d00a2410 con 0x5604b4794b60 2023-11-27T15:19:21.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.761+0000 7fe6beffd700 1 -- 172.21.15.52:0/35118728 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6d00a2190 con 0x5604b4794b60 2023-11-27T15:19:21.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.761+0000 7fe6f1ea4b80 10 client.25511 fetch_fsmap learned FSMap version 23 2023-11-27T15:19:21.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.761+0000 7fe6f1ea4b80 1 -- 172.21.15.52:0/35118728 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5604b46ac190 con 0x5604b4794b60 2023-11-27T15:19:21.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.762+0000 7fe6beffd700 1 -- 172.21.15.52:0/35118728 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6d007c020 con 0x5604b4794b60 2023-11-27T15:19:21.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.762+0000 7fe6beffd700 1 -- 172.21.15.52:0/35118728 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe6d00caa80 con 0x5604b4794b60 2023-11-27T15:19:21.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.762+0000 7fe6f1ea4b80 10 client.25511 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:19:21.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.762+0000 7fe6f1ea4b80 10 client.25511 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:19:21.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.762+0000 7fe6f1ea4b80 20 client.25511 populate_metadata read hostname 'smithi052' 2023-11-27T15:19:21.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.762+0000 7fe6f1ea4b80 1 --2- 172.21.15.52:0/35118728 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5604b47ae460 0x5604b47b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:21.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.762+0000 7fe6f1ea4b80 4 client.25511 mds_command: new command op to 24472 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:19:21.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.762+0000 7fe6f1ea4b80 1 -- 172.21.15.52:0/35118728 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5604b46a80a0 con 0x5604b47ae460 2023-11-27T15:19:21.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.762+0000 7fe6deffd700 1 --2- 172.21.15.52:0/35118728 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5604b47ae460 0x5604b47b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:21.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.763+0000 7fe6deffd700 1 --2- 172.21.15.52:0/35118728 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5604b47ae460 0x5604b47b0870 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7ca97b11c2f99152 server_cookie=bc28dfbf4e527a0d in_seq=0 out_seq=0 2023-11-27T15:19:21.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.763+0000 7fe6beffd700 10 client.25511 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:19:21.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.763+0000 7fe6beffd700 1 -- 172.21.15.52:0/35118728 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5604b46a80a0 con 0x5604b47ae460 2023-11-27T15:19:21.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.763+0000 7fe6beffd700 10 client.25511 handle_command_reply: tid=0 2023-11-27T15:19:21.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.763+0000 7fe6f1ea4b80 1 client.25511 shutdown 2023-11-27T15:19:21.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.764+0000 7fe6f1ea4b80 1 -- 172.21.15.52:0/35118728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604b4794b60 msgr2=0x5604b47a10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:21.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.764+0000 7fe6f1ea4b80 1 --2- 172.21.15.52:0/35118728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604b4794b60 0x5604b47a10a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0009230 tx=0x7fe6d0099760 comp rx=0 tx=0).stop 2023-11-27T15:19:21.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.764+0000 7fe6f1ea4b80 1 -- 172.21.15.52:0/35118728 shutdown_connections 2023-11-27T15:19:21.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.764+0000 7fe6f1ea4b80 1 --2- 172.21.15.52:0/35118728 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5604b47ae460 0x5604b47b0870 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.764+0000 7fe6f1ea4b80 1 --2- 172.21.15.52:0/35118728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604b4794b60 0x5604b47a10a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.764+0000 7fe6f1ea4b80 1 --2- 172.21.15.52:0/35118728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604b47998b0 0x5604b479ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.764+0000 7fe6f1ea4b80 1 --2- 172.21.15.52:0/35118728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604b4795470 0x5604b479e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.764+0000 7fe6beffd700 0 client.25511 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:19:21.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.764+0000 7fe6f1ea4b80 1 -- 172.21.15.52:0/35118728 >> 172.21.15.52:0/35118728 conn(0x5604b478a360 msgr2=0x5604b479ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:21.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.764+0000 7fe6f1ea4b80 1 -- 172.21.15.52:0/35118728 shutdown_connections 2023-11-27T15:19:21.775 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.764+0000 7fe6f1ea4b80 1 -- 172.21.15.52:0/35118728 wait complete. 2023-11-27T15:19:21.775 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.764+0000 7fe6f1ea4b80 20 client.25511 trim_cache size 0 max 16384 2023-11-27T15:19:21.784 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.778+0000 7fe6f1ea4b80 1 Processor -- start 2023-11-27T15:19:21.784 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.778+0000 7fe6f1ea4b80 1 -- start start 2023-11-27T15:19:21.784 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.778+0000 7fe6f1ea4b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604b4794b60 0x5604b47a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:21.784 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.778+0000 7fe6f1ea4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604b4795470 0x5604b479dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:21.785 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.778+0000 7fe6f1ea4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604b47998b0 0x5604b479e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:21.785 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.778+0000 7fe6f1ea4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604b479c6a0 con 0x5604b47998b0 2023-11-27T15:19:21.785 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.778+0000 7fe6f1ea4b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604b479c8a0 con 0x5604b4794b60 2023-11-27T15:19:21.785 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.778+0000 7fe6f1ea4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604b47984e0 con 0x5604b4795470 2023-11-27T15:19:21.785 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.778+0000 7fe6dffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604b47998b0 0x5604b479e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:21.785 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.778+0000 7fe6dffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604b47998b0 0x5604b479e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52726/0 (socket says 172.21.15.52:52726) 2023-11-27T15:19:21.785 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.778+0000 7fe6dffff700 1 -- 172.21.15.52:0/589074299 learned_addr learned my addr 172.21.15.52:0/589074299 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:21.785 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.779+0000 7fe6df7fe700 1 --2- 172.21.15.52:0/589074299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604b4794b60 0x5604b47a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:21.785 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.779+0000 7fe6deffd700 1 --2- 172.21.15.52:0/589074299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604b4795470 0x5604b479dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:21.785 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.779+0000 7fe6dffff700 1 -- 172.21.15.52:0/589074299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604b4795470 msgr2=0x5604b479dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:21.785 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.779+0000 7fe6dffff700 1 --2- 172.21.15.52:0/589074299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604b4795470 0x5604b479dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.786 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.779+0000 7fe6dffff700 1 -- 172.21.15.52:0/589074299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604b4794b60 msgr2=0x5604b47a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:21.786 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.779+0000 7fe6dffff700 1 --2- 172.21.15.52:0/589074299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604b4794b60 0x5604b47a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.786 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.779+0000 7fe6dffff700 1 -- 172.21.15.52:0/589074299 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604b46abee0 con 0x5604b47998b0 2023-11-27T15:19:21.786 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.779+0000 7fe6dffff700 1 --2- 172.21.15.52:0/589074299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604b47998b0 0x5604b479e100 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8009460 tx=0x7fe6d800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70de75e076a1b06a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:21.786 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.779+0000 7fe6c67fc700 1 -- 172.21.15.52:0/589074299 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8021070 con 0x5604b47998b0 2023-11-27T15:19:21.786 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.779+0000 7fe6c67fc700 1 -- 172.21.15.52:0/589074299 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6d80074e0 con 0x5604b47998b0 2023-11-27T15:19:21.786 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.779+0000 7fe6c67fc700 1 -- 172.21.15.52:0/589074299 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d800e040 con 0x5604b47998b0 2023-11-27T15:19:21.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.780+0000 7fe6f1ea4b80 1 -- 172.21.15.52:0/589074299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604b47998b0 msgr2=0x5604b479e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:21.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.780+0000 7fe6f1ea4b80 1 --2- 172.21.15.52:0/589074299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604b47998b0 0x5604b479e100 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8009460 tx=0x7fe6d800b040 comp rx=0 tx=0).stop 2023-11-27T15:19:21.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.780+0000 7fe6f1ea4b80 1 -- 172.21.15.52:0/589074299 shutdown_connections 2023-11-27T15:19:21.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.780+0000 7fe6f1ea4b80 1 --2- 172.21.15.52:0/589074299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604b4794b60 0x5604b47a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.780+0000 7fe6f1ea4b80 1 --2- 172.21.15.52:0/589074299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604b47998b0 0x5604b479e100 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.780+0000 7fe6f1ea4b80 1 --2- 172.21.15.52:0/589074299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604b4795470 0x5604b479dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.780+0000 7fe6f1ea4b80 1 -- 172.21.15.52:0/589074299 >> 172.21.15.52:0/589074299 conn(0x5604b478a360 msgr2=0x5604b47979d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:21.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.780+0000 7fe6f1ea4b80 1 -- 172.21.15.52:0/589074299 shutdown_connections 2023-11-27T15:19:21.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.780+0000 7fe6f1ea4b80 1 -- 172.21.15.52:0/589074299 wait complete. 2023-11-27T15:19:21.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.780+0000 7fe6f1ea4b80 1 -- start start 2023-11-27T15:19:21.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.781+0000 7fe6f1ea4b80 1 Processor -- start 2023-11-27T15:19:21.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.781+0000 7fe6f1ea4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604b4794b60 0x5604b47fb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:21.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.781+0000 7fe6f1ea4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604b4795470 0x5604b47fba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:21.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.781+0000 7fe6f1ea4b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604b47998b0 0x5604b47a3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:21.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.781+0000 7fe6f1ea4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604b47a3ed0 con 0x5604b4795470 2023-11-27T15:19:21.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.781+0000 7fe6f1ea4b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604b47a4040 con 0x5604b47998b0 2023-11-27T15:19:21.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.781+0000 7fe6f1ea4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604b47a4200 con 0x5604b4794b60 2023-11-27T15:19:21.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.781+0000 7fe6f1ea4b80 10 client.0 _ll_register_callbacks cb 0x7fe6d4005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:19:21.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.781+0000 7fe6deffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604b4795470 0x5604b47fba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:21.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.781+0000 7fe6deffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604b4795470 0x5604b47fba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52730/0 (socket says 172.21.15.52:52730) 2023-11-27T15:19:21.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.781+0000 7fe6deffd700 1 -- 172.21.15.52:0/3585952274 learned_addr learned my addr 172.21.15.52:0/3585952274 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:21.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.781+0000 7fe6df7fe700 1 --2- 172.21.15.52:0/3585952274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604b4794b60 0x5604b47fb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:21.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.781+0000 7fe6dffff700 1 --2- 172.21.15.52:0/3585952274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604b47998b0 0x5604b47a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:21.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.781+0000 7fe6deffd700 1 -- 172.21.15.52:0/3585952274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604b4794b60 msgr2=0x5604b47fb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:21.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.781+0000 7fe6deffd700 1 --2- 172.21.15.52:0/3585952274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604b4794b60 0x5604b47fb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.781+0000 7fe6deffd700 1 -- 172.21.15.52:0/3585952274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604b47998b0 msgr2=0x5604b47a3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:21.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.782+0000 7fe6deffd700 1 --2- 172.21.15.52:0/3585952274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604b47998b0 0x5604b47a3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.782+0000 7fe6deffd700 1 -- 172.21.15.52:0/3585952274 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e010d9b0 con 0x5604b4795470 2023-11-27T15:19:21.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.782+0000 7fe6deffd700 1 --2- 172.21.15.52:0/3585952274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604b4795470 0x5604b47fba60 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe6e006a110 tx=0x7fe6d40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=767346773a6346ca server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:21.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.782+0000 7fe6beffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:19:21.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.782+0000 7fe6beffd700 1 -- 172.21.15.52:0/3585952274 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4024040 con 0x5604b4795470 2023-11-27T15:19:21.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.782+0000 7fe6beffd700 1 -- 172.21.15.52:0/3585952274 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fe6d401f9f0 con 0x5604b4795470 2023-11-27T15:19:21.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.782+0000 7fe6f1ea4b80 1 -- 172.21.15.52:0/3585952274 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6d0010880 con 0x5604b4795470 2023-11-27T15:19:21.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.782+0000 7fe6beffd700 1 -- 172.21.15.52:0/3585952274 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6d401fbb0 con 0x5604b4795470 2023-11-27T15:19:21.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.782+0000 7fe6f1ea4b80 10 client.15702 fetch_fsmap learned FSMap version 23 2023-11-27T15:19:21.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.782+0000 7fe6f1ea4b80 1 -- 172.21.15.52:0/3585952274 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5604b44fcc60 con 0x5604b4795470 2023-11-27T15:19:21.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.783+0000 7fe6beffd700 1 -- 172.21.15.52:0/3585952274 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6d0010880 con 0x5604b4795470 2023-11-27T15:19:21.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.783+0000 7fe6beffd700 1 -- 172.21.15.52:0/3585952274 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe6d4003970 con 0x5604b4795470 2023-11-27T15:19:21.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.783+0000 7fe6f1ea4b80 10 client.15702 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:19:21.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.783+0000 7fe6f1ea4b80 10 client.15702 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:19:21.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.783+0000 7fe6f1ea4b80 20 client.15702 populate_metadata read hostname 'smithi052' 2023-11-27T15:19:21.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.783+0000 7fe6f1ea4b80 1 --2- 172.21.15.52:0/3585952274 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5604b4803340 0x5604b4805750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:21.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.783+0000 7fe6f1ea4b80 4 client.15702 mds_command: new command op to 24472 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:19:21.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.783+0000 7fe6f1ea4b80 1 -- 172.21.15.52:0/3585952274 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5604b46b4660 con 0x5604b4803340 2023-11-27T15:19:21.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.783+0000 7fe6df7fe700 1 --2- 172.21.15.52:0/3585952274 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5604b4803340 0x5604b4805750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:21.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.783+0000 7fe6beffd700 10 client.15702 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:19:21.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.783+0000 7fe6df7fe700 1 --2- 172.21.15.52:0/3585952274 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5604b4803340 0x5604b4805750 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a32c12aab84d5959 server_cookie=7ea2f109fc7560a in_seq=0 out_seq=0 2023-11-27T15:19:21.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.784+0000 7fe6beffd700 1 -- 172.21.15.52:0/3585952274 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5604b46b4660 con 0x5604b4803340 2023-11-27T15:19:21.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.784+0000 7fe6beffd700 10 client.15702 handle_command_reply: tid=0 2023-11-27T15:19:21.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.784+0000 7fe6f1ea4b80 1 client.15702 shutdown 2023-11-27T15:19:21.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.784+0000 7fe6f1ea4b80 1 -- 172.21.15.52:0/3585952274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604b4795470 msgr2=0x5604b47fba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:21.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.784+0000 7fe6f1ea4b80 1 --2- 172.21.15.52:0/3585952274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604b4795470 0x5604b47fba60 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe6e006a110 tx=0x7fe6d40047c0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.784+0000 7fe6f1ea4b80 1 -- 172.21.15.52:0/3585952274 shutdown_connections 2023-11-27T15:19:21.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.784+0000 7fe6f1ea4b80 1 --2- 172.21.15.52:0/3585952274 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5604b4803340 0x5604b4805750 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.784+0000 7fe6f1ea4b80 1 --2- 172.21.15.52:0/3585952274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604b47998b0 0x5604b47a3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.784+0000 7fe6f1ea4b80 1 --2- 172.21.15.52:0/3585952274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5604b4794b60 0x5604b47fb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.784+0000 7fe6f1ea4b80 1 --2- 172.21.15.52:0/3585952274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604b4795470 0x5604b47fba60 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.784+0000 7fe6beffd700 0 client.15702 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:19:21.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.784+0000 7fe6f1ea4b80 1 -- 172.21.15.52:0/3585952274 >> 172.21.15.52:0/3585952274 conn(0x5604b478a360 msgr2=0x5604b47979d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:21.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.784+0000 7fe6f1ea4b80 1 -- 172.21.15.52:0/3585952274 shutdown_connections 2023-11-27T15:19:21.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.784+0000 7fe6f1ea4b80 1 -- 172.21.15.52:0/3585952274 wait complete. 2023-11-27T15:19:21.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.784+0000 7fe6f1ea4b80 20 client.15702 trim_cache size 0 max 16384 2023-11-27T15:19:21.795 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:19:21.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.785+0000 7fe6e5d18700 1 -- 172.21.15.52:0/3833406030 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe6c80a1460 msgr2=0x7fe6c80a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:21.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.785+0000 7fe6e5d18700 1 --2- 172.21.15.52:0/3833406030 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe6c80a1460 0x7fe6c80a3910 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0197b80 tx=0x7fe6d4009260 comp rx=0 tx=0).stop 2023-11-27T15:19:21.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.785+0000 7fe6e5d18700 1 -- 172.21.15.52:0/3833406030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6e00693a0 msgr2=0x7fe6e0196570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:21.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.785+0000 7fe6e5d18700 1 --2- 172.21.15.52:0/3833406030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6e00693a0 0x7fe6e0196570 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe6d000e9a0 tx=0x7fe6d000ecb0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.785+0000 7fe6e5d18700 1 -- 172.21.15.52:0/3833406030 shutdown_connections 2023-11-27T15:19:21.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.785+0000 7fe6e5d18700 1 --2- 172.21.15.52:0/3833406030 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe6c80a1460 0x7fe6c80a3910 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.785+0000 7fe6e5d18700 1 --2- 172.21.15.52:0/3833406030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6e0105620 0x7fe6e019af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.785+0000 7fe6e5d18700 1 --2- 172.21.15.52:0/3833406030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6e00693a0 0x7fe6e0196570 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.785+0000 7fe6e5d18700 1 --2- 172.21.15.52:0/3833406030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6e0069d40 0x7fe6e0196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:21.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.785+0000 7fe6e5d18700 1 -- 172.21.15.52:0/3833406030 >> 172.21.15.52:0/3833406030 conn(0x7fe6e00763a0 msgr2=0x7fe6e00fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:21.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.785+0000 7fe6e5d18700 1 -- 172.21.15.52:0/3833406030 shutdown_connections 2023-11-27T15:19:21.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:21.785+0000 7fe6e5d18700 1 -- 172.21.15.52:0/3833406030 wait complete. 2023-11-27T15:19:21.869 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-27T15:19:22.371 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:22.371 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:22.414 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:22.414 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:22.430 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:22.430 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:22.430 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 47 -lt 100 ']' 2023-11-27T15:19:22.430 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 47 + 1 2023-11-27T15:19:22.452 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=48 2023-11-27T15:19:22.452 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:22.471 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:22.471 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:22.472 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:22.472 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:22.472 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:22.879 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:22.879 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:22.924 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:22.925 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:22.944 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:22.944 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:22.944 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 48 -lt 100 ']' 2023-11-27T15:19:22.944 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 48 + 1 2023-11-27T15:19:23.010 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=49 2023-11-27T15:19:23.010 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:23.052 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:23.052 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:23.053 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:23.054 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:23.054 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:23.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:22 smithi052 ceph-mon[99056]: pgmap v46: 97 pgs: 97 active+clean; 2.0 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 10 op/s 2023-11-27T15:19:23.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:22 smithi150 ceph-mon[100559]: pgmap v46: 97 pgs: 97 active+clean; 2.0 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 10 op/s 2023-11-27T15:19:23.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:22 smithi102 ceph-mon[100296]: pgmap v46: 97 pgs: 97 active+clean; 2.0 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 10 op/s 2023-11-27T15:19:23.479 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:23.479 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:23.597 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:23.597 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:23.612 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:23.613 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:23.613 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 49 -lt 100 ']' 2023-11-27T15:19:23.613 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 49 + 1 2023-11-27T15:19:23.660 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=50 2023-11-27T15:19:23.660 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:23.689 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:23.689 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:23.689 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:23.690 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:23.690 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:24.388 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:24.388 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:24.597 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:24.597 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:24.610 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:24.611 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:24.611 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 50 -lt 100 ']' 2023-11-27T15:19:24.611 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 50 + 1 2023-11-27T15:19:24.635 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=51 2023-11-27T15:19:24.635 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:24.681 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:24.681 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:24.683 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:24.683 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:24.683 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:25.114 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:24 smithi052 ceph-mon[99056]: pgmap v47: 97 pgs: 97 active+clean; 2.0 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 7 op/s 2023-11-27T15:19:25.184 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.179+0000 7febd3789700 1 -- 172.21.15.52:0/4253421412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7febcc104590 msgr2=0x7febcc10c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:25.184 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.179+0000 7febd3789700 1 --2- 172.21.15.52:0/4253421412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7febcc104590 0x7febcc10c110 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7febc400b0d0 tx=0x7febc400b3e0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.184 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.179+0000 7febd3789700 1 -- 172.21.15.52:0/4253421412 shutdown_connections 2023-11-27T15:19:25.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.179+0000 7febd3789700 1 --2- 172.21.15.52:0/4253421412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febcc0ffc30 0x7febcc100000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.179+0000 7febd3789700 1 --2- 172.21.15.52:0/4253421412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7febcc104590 0x7febcc10c110 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.179+0000 7febd3789700 1 --2- 172.21.15.52:0/4253421412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febcc1005d0 0x7febcc103e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.179+0000 7febd3789700 1 -- 172.21.15.52:0/4253421412 >> 172.21.15.52:0/4253421412 conn(0x7febcc075470 msgr2=0x7febcc075870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:25.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.179+0000 7febd3789700 1 -- 172.21.15.52:0/4253421412 shutdown_connections 2023-11-27T15:19:25.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.179+0000 7febd3789700 1 -- 172.21.15.52:0/4253421412 wait complete. 2023-11-27T15:19:25.219 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.212+0000 7febd3789700 1 Processor -- start 2023-11-27T15:19:25.219 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.212+0000 7febd3789700 1 -- start start 2023-11-27T15:19:25.219 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.213+0000 7febd3789700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7febcc0ffc30 0x7febcc100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:25.219 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.213+0000 7febd3789700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febcc1005d0 0x7febcc101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:25.219 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.213+0000 7febd3789700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febcc104590 0x7febcc1a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:25.219 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.213+0000 7febd3789700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc101c70 con 0x7febcc0ffc30 2023-11-27T15:19:25.219 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.213+0000 7febd3789700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc1a4ce0 con 0x7febcc1005d0 2023-11-27T15:19:25.219 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.213+0000 7febd3789700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc1a4e20 con 0x7febcc104590 2023-11-27T15:19:25.219 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.213+0000 7febd1525700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7febcc0ffc30 0x7febcc100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:25.219 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.213+0000 7febd1525700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7febcc0ffc30 0x7febcc100f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52742/0 (socket says 172.21.15.52:52742) 2023-11-27T15:19:25.220 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.213+0000 7febd1525700 1 -- 172.21.15.52:0/239301279 learned_addr learned my addr 172.21.15.52:0/239301279 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:25.220 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.213+0000 7febd0d24700 1 --2- 172.21.15.52:0/239301279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febcc1005d0 0x7febcc101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:25.220 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.213+0000 7febd1d26700 1 --2- 172.21.15.52:0/239301279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febcc104590 0x7febcc1a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:25.220 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.213+0000 7febd1525700 1 -- 172.21.15.52:0/239301279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febcc104590 msgr2=0x7febcc1a47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:25.220 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.213+0000 7febd1525700 1 --2- 172.21.15.52:0/239301279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febcc104590 0x7febcc1a47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.220 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.213+0000 7febd1525700 1 -- 172.21.15.52:0/239301279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febcc1005d0 msgr2=0x7febcc101440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:25.220 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.213+0000 7febd1525700 1 --2- 172.21.15.52:0/239301279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febcc1005d0 0x7febcc101440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.220 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.213+0000 7febd1525700 1 -- 172.21.15.52:0/239301279 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc4009d00 con 0x7febcc0ffc30 2023-11-27T15:19:25.220 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.213+0000 7febd1525700 1 --2- 172.21.15.52:0/239301279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7febcc0ffc30 0x7febcc100f00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7febbc00e9a0 tx=0x7febbc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:25.220 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.214+0000 7febca7fc700 1 -- 172.21.15.52:0/239301279 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc009790 con 0x7febcc0ffc30 2023-11-27T15:19:25.220 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.214+0000 7febca7fc700 1 -- 172.21.15.52:0/239301279 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febbc010430 con 0x7febcc0ffc30 2023-11-27T15:19:25.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.214+0000 7febd3789700 1 -- 172.21.15.52:0/239301279 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febcc10f320 con 0x7febcc0ffc30 2023-11-27T15:19:25.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.214+0000 7febca7fc700 1 -- 172.21.15.52:0/239301279 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc018680 con 0x7febcc0ffc30 2023-11-27T15:19:25.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.214+0000 7febd3789700 1 -- 172.21.15.52:0/239301279 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febcc10f190 con 0x7febcc0ffc30 2023-11-27T15:19:25.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.216+0000 7febca7fc700 1 -- 172.21.15.52:0/239301279 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7febbc0105a0 con 0x7febcc0ffc30 2023-11-27T15:19:25.221 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.216+0000 7febca7fc700 1 --2- 172.21.15.52:0/239301279 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7febb40a13e0 0x7febb40a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:25.222 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.217+0000 7febca7fc700 1 -- 172.21.15.52:0/239301279 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7febbc014070 con 0x7febcc0ffc30 2023-11-27T15:19:25.222 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.217+0000 7febd0d24700 1 --2- 172.21.15.52:0/239301279 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7febb40a13e0 0x7febb40a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:25.222 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.217+0000 7febd0d24700 1 --2- 172.21.15.52:0/239301279 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7febb40a13e0 0x7febb40a3890 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7febcc1026f0 tx=0x7febc0009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:25.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:24 smithi150 ceph-mon[100559]: pgmap v47: 97 pgs: 97 active+clean; 2.0 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 7 op/s 2023-11-27T15:19:25.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:24 smithi102 ceph-mon[100296]: pgmap v47: 97 pgs: 97 active+clean; 2.0 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 7 op/s 2023-11-27T15:19:25.329 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:25.329 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:25.389 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.381+0000 7febdf915b80 1 Processor -- start 2023-11-27T15:19:25.389 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.381+0000 7febdf915b80 1 -- start start 2023-11-27T15:19:25.389 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.381+0000 7febdf915b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fd0712b60 0x555fd0712f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:25.389 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.381+0000 7febdf915b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fd0713470 0x555fd0717370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:25.389 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.381+0000 7febdf915b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555fd07178b0 0x555fd0717d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:25.389 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.381+0000 7febdf915b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd0618060 con 0x555fd07178b0 2023-11-27T15:19:25.389 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.381+0000 7febdf915b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd066a8a0 con 0x555fd0712b60 2023-11-27T15:19:25.389 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.381+0000 7febdf915b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd0663d30 con 0x555fd0713470 2023-11-27T15:19:25.389 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.381+0000 7febd1d26700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555fd07178b0 0x555fd0717d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:25.389 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.381+0000 7febd1d26700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555fd07178b0 0x555fd0717d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52754/0 (socket says 172.21.15.52:52754) 2023-11-27T15:19:25.390 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.381+0000 7febd1d26700 1 -- 172.21.15.52:0/1658252890 learned_addr learned my addr 172.21.15.52:0/1658252890 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:25.390 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.382+0000 7febd0d24700 1 --2- 172.21.15.52:0/1658252890 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fd0713470 0x555fd0717370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:25.390 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.382+0000 7febd1525700 1 --2- 172.21.15.52:0/1658252890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fd0712b60 0x555fd0712f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:25.390 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.382+0000 7febd1d26700 1 -- 172.21.15.52:0/1658252890 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fd0713470 msgr2=0x555fd0717370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:25.390 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.382+0000 7febd1d26700 1 --2- 172.21.15.52:0/1658252890 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fd0713470 0x555fd0717370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.390 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.382+0000 7febd1d26700 1 -- 172.21.15.52:0/1658252890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fd0712b60 msgr2=0x555fd0712f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:25.390 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.382+0000 7febd1d26700 1 --2- 172.21.15.52:0/1658252890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fd0712b60 0x555fd0712f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.390 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.382+0000 7febd1d26700 1 -- 172.21.15.52:0/1658252890 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fd04e5630 con 0x555fd07178b0 2023-11-27T15:19:25.390 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.382+0000 7febd1525700 1 --2- 172.21.15.52:0/1658252890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fd0712b60 0x555fd0712f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:19:25.391 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.382+0000 7febd1d26700 1 --2- 172.21.15.52:0/1658252890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555fd07178b0 0x555fd0717d00 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7febc4009c60 tx=0x7febc4009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45671b2952b522a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:25.391 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.382+0000 7febb37fe700 1 -- 172.21.15.52:0/1658252890 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc400e040 con 0x555fd07178b0 2023-11-27T15:19:25.391 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.382+0000 7febb37fe700 1 -- 172.21.15.52:0/1658252890 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febc4009360 con 0x555fd07178b0 2023-11-27T15:19:25.391 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.382+0000 7febb37fe700 1 -- 172.21.15.52:0/1658252890 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc4003dc0 con 0x555fd07178b0 2023-11-27T15:19:25.391 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.383+0000 7febdf915b80 1 -- 172.21.15.52:0/1658252890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555fd07178b0 msgr2=0x555fd0717d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:25.391 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.383+0000 7febdf915b80 1 --2- 172.21.15.52:0/1658252890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555fd07178b0 0x555fd0717d00 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7febc4009c60 tx=0x7febc4009c90 comp rx=0 tx=0).stop 2023-11-27T15:19:25.391 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.383+0000 7febdf915b80 1 -- 172.21.15.52:0/1658252890 shutdown_connections 2023-11-27T15:19:25.391 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.383+0000 7febdf915b80 1 --2- 172.21.15.52:0/1658252890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fd0712b60 0x555fd0712f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.391 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.383+0000 7febdf915b80 1 --2- 172.21.15.52:0/1658252890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555fd07178b0 0x555fd0717d00 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.391 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.383+0000 7febdf915b80 1 --2- 172.21.15.52:0/1658252890 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fd0713470 0x555fd0717370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.391 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.383+0000 7febdf915b80 1 -- 172.21.15.52:0/1658252890 >> 172.21.15.52:0/1658252890 conn(0x555fd0708360 msgr2=0x555fd070db40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:25.392 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.383+0000 7febdf915b80 1 -- 172.21.15.52:0/1658252890 shutdown_connections 2023-11-27T15:19:25.392 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.383+0000 7febdf915b80 1 -- 172.21.15.52:0/1658252890 wait complete. 2023-11-27T15:19:25.392 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.384+0000 7febdf915b80 1 -- start start 2023-11-27T15:19:25.392 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.384+0000 7febdf915b80 1 Processor -- start 2023-11-27T15:19:25.396 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.384+0000 7febdf915b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555fd0712b60 0x555fd071efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:25.396 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.384+0000 7febdf915b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fd0713470 0x555fd071c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:25.396 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.384+0000 7febdf915b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fd07178b0 0x555fd071c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:25.396 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.384+0000 7febdf915b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd06058e0 con 0x555fd0712b60 2023-11-27T15:19:25.396 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.384+0000 7febdf915b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd0605580 con 0x555fd07178b0 2023-11-27T15:19:25.396 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.384+0000 7febdf915b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd04e7920 con 0x555fd0713470 2023-11-27T15:19:25.396 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.384+0000 7febdf915b80 10 client.0 _ll_register_callbacks cb 0x555fd0623780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:19:25.397 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.384+0000 7febd1525700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555fd0712b60 0x555fd071efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:25.397 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.385+0000 7febd1525700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555fd0712b60 0x555fd071efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52756/0 (socket says 172.21.15.52:52756) 2023-11-27T15:19:25.397 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.385+0000 7febd1525700 1 -- 172.21.15.52:0/3545109992 learned_addr learned my addr 172.21.15.52:0/3545109992 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:25.397 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.385+0000 7febd1d26700 1 --2- 172.21.15.52:0/3545109992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fd07178b0 0x555fd071c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:25.397 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.385+0000 7febd0d24700 1 --2- 172.21.15.52:0/3545109992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fd0713470 0x555fd071c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:25.397 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.385+0000 7febd1d26700 1 -- 172.21.15.52:0/3545109992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fd0713470 msgr2=0x555fd071c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:25.397 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.385+0000 7febd1d26700 1 --2- 172.21.15.52:0/3545109992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fd0713470 0x555fd071c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.397 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.385+0000 7febd1d26700 1 -- 172.21.15.52:0/3545109992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555fd0712b60 msgr2=0x555fd071efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:25.397 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.385+0000 7febd1d26700 1 --2- 172.21.15.52:0/3545109992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555fd0712b60 0x555fd071efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.398 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.385+0000 7febd1d26700 1 -- 172.21.15.52:0/3545109992 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febcc10c950 con 0x555fd07178b0 2023-11-27T15:19:25.398 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.385+0000 7febd0d24700 1 --2- 172.21.15.52:0/3545109992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fd0713470 0x555fd071c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:19:25.398 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.385+0000 7febd1525700 1 --2- 172.21.15.52:0/3545109992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555fd0712b60 0x555fd071efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:25.398 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.385+0000 7febd1d26700 1 --2- 172.21.15.52:0/3545109992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fd07178b0 0x555fd071c870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7febc4005ad0 tx=0x7febc400bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ad5edc95a868e94 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:25.398 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.385+0000 7febb0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2023-11-27T15:19:25.398 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.385+0000 7febb0ff9700 1 -- 172.21.15.52:0/3545109992 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc400e040 con 0x555fd07178b0 2023-11-27T15:19:25.398 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.385+0000 7febdf915b80 1 -- 172.21.15.52:0/3545109992 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555fd0705b20 con 0x555fd07178b0 2023-11-27T15:19:25.398 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.385+0000 7febb0ff9700 1 -- 172.21.15.52:0/3545109992 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7febc4003e10 con 0x555fd07178b0 2023-11-27T15:19:25.398 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.386+0000 7febb0ff9700 1 -- 172.21.15.52:0/3545109992 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7febc4005070 con 0x555fd07178b0 2023-11-27T15:19:25.399 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.386+0000 7febdf915b80 10 client.25529 fetch_fsmap learned FSMap version 23 2023-11-27T15:19:25.399 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.386+0000 7febdf915b80 1 -- 172.21.15.52:0/3545109992 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x555fd062a190 con 0x555fd07178b0 2023-11-27T15:19:25.399 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.386+0000 7febb0ff9700 1 -- 172.21.15.52:0/3545109992 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555fd0705b20 con 0x555fd07178b0 2023-11-27T15:19:25.399 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.386+0000 7febb0ff9700 1 -- 172.21.15.52:0/3545109992 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7febc400b100 con 0x555fd07178b0 2023-11-27T15:19:25.399 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.386+0000 7febdf915b80 10 client.25529 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:19:25.399 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.386+0000 7febdf915b80 10 client.25529 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:19:25.399 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.386+0000 7febdf915b80 20 client.25529 populate_metadata read hostname 'smithi052' 2023-11-27T15:19:25.399 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.386+0000 7febdf915b80 1 --2- 172.21.15.52:0/3545109992 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x555fd072c310 0x555fd072e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:25.399 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.386+0000 7febdf915b80 4 client.25529 mds_command: new command op to 24494 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:19:25.399 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.386+0000 7febdf915b80 1 -- 172.21.15.52:0/3545109992 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555fd06260a0 con 0x555fd072c310 2023-11-27T15:19:25.399 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.386+0000 7febd1525700 1 --2- 172.21.15.52:0/3545109992 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x555fd072c310 0x555fd072e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:25.400 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.387+0000 7febb0ff9700 10 client.25529 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:19:25.400 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.387+0000 7febd1525700 1 --2- 172.21.15.52:0/3545109992 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x555fd072c310 0x555fd072e720 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=641d26ef5b5b3d05 server_cookie=4fff9f676e776313 in_seq=0 out_seq=0 2023-11-27T15:19:25.400 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.387+0000 7febb0ff9700 1 -- 172.21.15.52:0/3545109992 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555fd06260a0 con 0x555fd072c310 2023-11-27T15:19:25.400 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.387+0000 7febb0ff9700 10 client.25529 handle_command_reply: tid=0 2023-11-27T15:19:25.400 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.387+0000 7febdf915b80 1 client.25529 shutdown 2023-11-27T15:19:25.400 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.388+0000 7febdf915b80 1 -- 172.21.15.52:0/3545109992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fd07178b0 msgr2=0x555fd071c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:25.400 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.388+0000 7febdf915b80 1 --2- 172.21.15.52:0/3545109992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fd07178b0 0x555fd071c870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7febc4005ad0 tx=0x7febc400bcd0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.400 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.388+0000 7febdf915b80 1 -- 172.21.15.52:0/3545109992 shutdown_connections 2023-11-27T15:19:25.400 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.388+0000 7febdf915b80 1 --2- 172.21.15.52:0/3545109992 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x555fd072c310 0x555fd072e720 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.401 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.388+0000 7febdf915b80 1 --2- 172.21.15.52:0/3545109992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fd07178b0 0x555fd071c870 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.401 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.388+0000 7febb0ff9700 0 client.25529 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:19:25.401 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.388+0000 7febdf915b80 1 --2- 172.21.15.52:0/3545109992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555fd0712b60 0x555fd071efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.401 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.388+0000 7febdf915b80 1 --2- 172.21.15.52:0/3545109992 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fd0713470 0x555fd071c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.401 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.388+0000 7febdf915b80 1 -- 172.21.15.52:0/3545109992 >> 172.21.15.52:0/3545109992 conn(0x555fd0708360 msgr2=0x555fd0718d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:25.401 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.388+0000 7febdf915b80 1 -- 172.21.15.52:0/3545109992 shutdown_connections 2023-11-27T15:19:25.401 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.388+0000 7febdf915b80 1 -- 172.21.15.52:0/3545109992 wait complete. 2023-11-27T15:19:25.401 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.388+0000 7febdf915b80 20 client.25529 trim_cache size 0 max 16384 2023-11-27T15:19:25.410 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.402+0000 7febdf915b80 1 Processor -- start 2023-11-27T15:19:25.410 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.402+0000 7febdf915b80 1 -- start start 2023-11-27T15:19:25.410 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.402+0000 7febdf915b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fd0712b60 0x555fd071eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:25.410 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.402+0000 7febdf915b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fd0713470 0x555fd070d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:25.410 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.402+0000 7febdf915b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555fd07178b0 0x555fd070daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:25.410 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.402+0000 7febdf915b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd0707f10 con 0x555fd07178b0 2023-11-27T15:19:25.410 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.402+0000 7febdf915b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd0716400 con 0x555fd0712b60 2023-11-27T15:19:25.410 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.402+0000 7febdf915b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd0705960 con 0x555fd0713470 2023-11-27T15:19:25.411 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.403+0000 7febd1d26700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555fd07178b0 0x555fd070daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:25.411 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.403+0000 7febd1525700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fd0712b60 0x555fd071eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:25.411 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.403+0000 7febd1d26700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555fd07178b0 0x555fd070daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52768/0 (socket says 172.21.15.52:52768) 2023-11-27T15:19:25.411 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.403+0000 7febd1d26700 1 -- 172.21.15.52:0/3124966025 learned_addr learned my addr 172.21.15.52:0/3124966025 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:25.411 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.403+0000 7febd1525700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fd0712b60 0x555fd071eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:42422/0 (socket says 172.21.15.52:42422) 2023-11-27T15:19:25.411 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.403+0000 7febd0d24700 1 --2- 172.21.15.52:0/3124966025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fd0713470 0x555fd070d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:25.411 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.403+0000 7febd1525700 1 -- 172.21.15.52:0/3124966025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fd0713470 msgr2=0x555fd070d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:25.411 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.403+0000 7febd1525700 1 --2- 172.21.15.52:0/3124966025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fd0713470 0x555fd070d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.411 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.403+0000 7febd1525700 1 -- 172.21.15.52:0/3124966025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555fd07178b0 msgr2=0x555fd070daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:25.411 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.403+0000 7febd1525700 1 --2- 172.21.15.52:0/3124966025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555fd07178b0 0x555fd070daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.403+0000 7febd1525700 1 -- 172.21.15.52:0/3124966025 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fd0629ee0 con 0x555fd0712b60 2023-11-27T15:19:25.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.403+0000 7febd1d26700 1 --2- 172.21.15.52:0/3124966025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555fd07178b0 0x555fd070daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:25.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.403+0000 7febd1525700 1 --2- 172.21.15.52:0/3124966025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fd0712b60 0x555fd071eb40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7febbc096510 tx=0x7febbc010c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56495b3d7ae63762 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:25.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.403+0000 7febb3fff700 1 -- 172.21.15.52:0/3124966025 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc099b50 con 0x555fd0712b60 2023-11-27T15:19:25.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.403+0000 7febb3fff700 1 -- 172.21.15.52:0/3124966025 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febbc0a2460 con 0x555fd0712b60 2023-11-27T15:19:25.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.403+0000 7febb3fff700 1 -- 172.21.15.52:0/3124966025 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc0caa40 con 0x555fd0712b60 2023-11-27T15:19:25.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.404+0000 7febdf915b80 1 -- 172.21.15.52:0/3124966025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fd0712b60 msgr2=0x555fd071eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:25.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.404+0000 7febdf915b80 1 --2- 172.21.15.52:0/3124966025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fd0712b60 0x555fd071eb40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7febbc096510 tx=0x7febbc010c80 comp rx=0 tx=0).stop 2023-11-27T15:19:25.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.404+0000 7febdf915b80 1 -- 172.21.15.52:0/3124966025 shutdown_connections 2023-11-27T15:19:25.412 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.404+0000 7febdf915b80 1 --2- 172.21.15.52:0/3124966025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fd0712b60 0x555fd071eb40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.404+0000 7febdf915b80 1 --2- 172.21.15.52:0/3124966025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555fd07178b0 0x555fd070daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.404+0000 7febdf915b80 1 --2- 172.21.15.52:0/3124966025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fd0713470 0x555fd070d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.404+0000 7febdf915b80 1 -- 172.21.15.52:0/3124966025 >> 172.21.15.52:0/3124966025 conn(0x555fd0708360 msgr2=0x555fd0728b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:25.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.404+0000 7febdf915b80 1 -- 172.21.15.52:0/3124966025 shutdown_connections 2023-11-27T15:19:25.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.404+0000 7febdf915b80 1 -- 172.21.15.52:0/3124966025 wait complete. 2023-11-27T15:19:25.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.405+0000 7febdf915b80 1 -- start start 2023-11-27T15:19:25.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.405+0000 7febdf915b80 1 Processor -- start 2023-11-27T15:19:25.417 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.405+0000 7febdf915b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fd0712b60 0x555fd0774310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:25.417 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.406+0000 7febdf915b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fd0713470 0x555fd0774850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:25.417 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.406+0000 7febdf915b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555fd07178b0 0x555fd077f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:25.417 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.406+0000 7febdf915b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd0774fc0 con 0x555fd07178b0 2023-11-27T15:19:25.417 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.406+0000 7febdf915b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd077f6e0 con 0x555fd0712b60 2023-11-27T15:19:25.417 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.406+0000 7febdf915b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd077f8a0 con 0x555fd0713470 2023-11-27T15:19:25.417 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.406+0000 7febdf915b80 10 client.0 _ll_register_callbacks cb 0x7febc4009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:19:25.417 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.406+0000 7febd1d26700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555fd07178b0 0x555fd077f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:25.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.406+0000 7febd0d24700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fd0713470 0x555fd0774850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:25.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.406+0000 7febd1d26700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555fd07178b0 0x555fd077f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52774/0 (socket says 172.21.15.52:52774) 2023-11-27T15:19:25.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.406+0000 7febd1d26700 1 -- 172.21.15.52:0/1363706516 learned_addr learned my addr 172.21.15.52:0/1363706516 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:25.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.406+0000 7febd1525700 1 --2- 172.21.15.52:0/1363706516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fd0712b60 0x555fd0774310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:25.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.406+0000 7febd1d26700 1 -- 172.21.15.52:0/1363706516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fd0713470 msgr2=0x555fd0774850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:25.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.406+0000 7febd1d26700 1 --2- 172.21.15.52:0/1363706516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fd0713470 0x555fd0774850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.406+0000 7febd1d26700 1 -- 172.21.15.52:0/1363706516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fd0712b60 msgr2=0x555fd0774310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:25.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.406+0000 7febd1d26700 1 --2- 172.21.15.52:0/1363706516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fd0712b60 0x555fd0774310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.406+0000 7febd1d26700 1 -- 172.21.15.52:0/1363706516 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555fd062a190 con 0x555fd07178b0 2023-11-27T15:19:25.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.406+0000 7febd1d26700 1 --2- 172.21.15.52:0/1363706516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555fd07178b0 0x555fd077f1a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7febc40258c0 tx=0x7febc4006230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45dc65baf9c56750 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:25.419 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.406+0000 7febb0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:19:25.419 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.406+0000 7febb0ff9700 1 -- 172.21.15.52:0/1363706516 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc400e070 con 0x555fd07178b0 2023-11-27T15:19:25.419 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.407+0000 7febb0ff9700 1 -- 172.21.15.52:0/1363706516 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7febc4025b30 con 0x555fd07178b0 2023-11-27T15:19:25.419 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.407+0000 7febdf915b80 1 -- 172.21.15.52:0/1363706516 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febc4009360 con 0x555fd07178b0 2023-11-27T15:19:25.419 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.407+0000 7febb0ff9700 1 -- 172.21.15.52:0/1363706516 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7febc4009b80 con 0x555fd07178b0 2023-11-27T15:19:25.419 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.407+0000 7febdf915b80 10 client.15720 fetch_fsmap learned FSMap version 23 2023-11-27T15:19:25.419 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.407+0000 7febdf915b80 1 -- 172.21.15.52:0/1363706516 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7febcc10f190 con 0x555fd07178b0 2023-11-27T15:19:25.419 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.407+0000 7febb0ff9700 1 -- 172.21.15.52:0/1363706516 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febc4009360 con 0x555fd07178b0 2023-11-27T15:19:25.419 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.407+0000 7febb0ff9700 1 -- 172.21.15.52:0/1363706516 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7febc402e810 con 0x555fd07178b0 2023-11-27T15:19:25.419 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.407+0000 7febdf915b80 10 client.15720 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:19:25.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.407+0000 7febdf915b80 10 client.15720 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:19:25.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.407+0000 7febdf915b80 20 client.15720 populate_metadata read hostname 'smithi052' 2023-11-27T15:19:25.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.407+0000 7febdf915b80 1 --2- 172.21.15.52:0/1363706516 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x555fd0780af0 0x555fd0782ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:25.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.407+0000 7febdf915b80 4 client.15720 mds_command: new command op to 24494 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:19:25.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.407+0000 7febdf915b80 1 -- 172.21.15.52:0/1363706516 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555fd0632660 con 0x555fd0780af0 2023-11-27T15:19:25.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.407+0000 7febd1525700 1 --2- 172.21.15.52:0/1363706516 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x555fd0780af0 0x555fd0782ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:25.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.408+0000 7febd1525700 1 --2- 172.21.15.52:0/1363706516 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x555fd0780af0 0x555fd0782ed0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=965077d6a09ae11 server_cookie=8fa605757acb135b in_seq=0 out_seq=0 2023-11-27T15:19:25.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.408+0000 7febb0ff9700 10 client.15720 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:19:25.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.408+0000 7febb0ff9700 1 -- 172.21.15.52:0/1363706516 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555fd0632660 con 0x555fd0780af0 2023-11-27T15:19:25.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.408+0000 7febb0ff9700 10 client.15720 handle_command_reply: tid=0 2023-11-27T15:19:25.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.408+0000 7febdf915b80 1 client.15720 shutdown 2023-11-27T15:19:25.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.408+0000 7febdf915b80 1 -- 172.21.15.52:0/1363706516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555fd07178b0 msgr2=0x555fd077f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:25.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.408+0000 7febdf915b80 1 --2- 172.21.15.52:0/1363706516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555fd07178b0 0x555fd077f1a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7febc40258c0 tx=0x7febc4006230 comp rx=0 tx=0).stop 2023-11-27T15:19:25.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.408+0000 7febdf915b80 1 -- 172.21.15.52:0/1363706516 shutdown_connections 2023-11-27T15:19:25.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.408+0000 7febdf915b80 1 --2- 172.21.15.52:0/1363706516 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x555fd0780af0 0x555fd0782ed0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.408+0000 7febdf915b80 1 --2- 172.21.15.52:0/1363706516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fd0712b60 0x555fd0774310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.408+0000 7febdf915b80 1 --2- 172.21.15.52:0/1363706516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555fd07178b0 0x555fd077f1a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.408+0000 7febb0ff9700 0 client.15720 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:19:25.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.408+0000 7febdf915b80 1 --2- 172.21.15.52:0/1363706516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555fd0713470 0x555fd0774850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.408+0000 7febdf915b80 1 -- 172.21.15.52:0/1363706516 >> 172.21.15.52:0/1363706516 conn(0x555fd0708360 msgr2=0x555fd0716810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:25.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.408+0000 7febdf915b80 1 -- 172.21.15.52:0/1363706516 shutdown_connections 2023-11-27T15:19:25.422 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.408+0000 7febdf915b80 1 -- 172.21.15.52:0/1363706516 wait complete. 2023-11-27T15:19:25.422 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.409+0000 7febdf915b80 20 client.15720 trim_cache size 0 max 16384 2023-11-27T15:19:25.422 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:19:25.423 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.409+0000 7febd3789700 1 -- 172.21.15.52:0/239301279 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7febb40a13e0 msgr2=0x7febb40a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:25.423 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.409+0000 7febd3789700 1 --2- 172.21.15.52:0/239301279 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7febb40a13e0 0x7febb40a3890 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7febcc1026f0 tx=0x7febc0009260 comp rx=0 tx=0).stop 2023-11-27T15:19:25.423 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.409+0000 7febd3789700 1 -- 172.21.15.52:0/239301279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7febcc0ffc30 msgr2=0x7febcc100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:25.423 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.409+0000 7febd3789700 1 --2- 172.21.15.52:0/239301279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7febcc0ffc30 0x7febcc100f00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7febbc00e9a0 tx=0x7febbc00ecb0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.423 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.409+0000 7febd3789700 1 -- 172.21.15.52:0/239301279 shutdown_connections 2023-11-27T15:19:25.423 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.409+0000 7febd3789700 1 --2- 172.21.15.52:0/239301279 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7febb40a13e0 0x7febb40a3890 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.423 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.409+0000 7febd3789700 1 --2- 172.21.15.52:0/239301279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febcc104590 0x7febcc1a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.424 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.409+0000 7febd3789700 1 --2- 172.21.15.52:0/239301279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7febcc0ffc30 0x7febcc100f00 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.424 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.409+0000 7febd3789700 1 --2- 172.21.15.52:0/239301279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febcc1005d0 0x7febcc101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:25.424 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.409+0000 7febd3789700 1 -- 172.21.15.52:0/239301279 >> 172.21.15.52:0/239301279 conn(0x7febcc075470 msgr2=0x7febcc068f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:25.424 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.409+0000 7febd3789700 1 -- 172.21.15.52:0/239301279 shutdown_connections 2023-11-27T15:19:25.424 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:25.409+0000 7febd3789700 1 -- 172.21.15.52:0/239301279 wait complete. 2023-11-27T15:19:25.485 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-27T15:19:25.505 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:25.505 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:25.518 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:25.518 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:25.518 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 51 -lt 100 ']' 2023-11-27T15:19:25.519 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 51 + 1 2023-11-27T15:19:25.560 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=52 2023-11-27T15:19:25.560 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:25.580 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:25.580 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:25.581 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:25.581 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:25.582 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:26.279 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:26.279 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:26.357 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:26.357 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:26.411 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:26.411 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:26.411 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 52 -lt 100 ']' 2023-11-27T15:19:26.411 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 52 + 1 2023-11-27T15:19:26.476 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=53 2023-11-27T15:19:26.477 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:26.535 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:26.535 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:26.536 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:26.536 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:26.536 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:27.107 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:26 smithi052 ceph-mon[99056]: pgmap v48: 97 pgs: 97 active+clean; 2.0 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 7 op/s 2023-11-27T15:19:27.154 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:27.154 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:27.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:26 smithi150 ceph-mon[100559]: pgmap v48: 97 pgs: 97 active+clean; 2.0 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 7 op/s 2023-11-27T15:19:27.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:26 smithi102 ceph-mon[100296]: pgmap v48: 97 pgs: 97 active+clean; 2.0 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 7 op/s 2023-11-27T15:19:27.347 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:27.347 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:27.376 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:27.376 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:27.377 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 53 -lt 100 ']' 2023-11-27T15:19:27.377 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 53 + 1 2023-11-27T15:19:27.410 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=54 2023-11-27T15:19:27.410 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:27.455 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:27.455 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:27.456 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:27.457 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:27.457 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:28.062 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:28.062 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:28.180 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:28.180 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:28.209 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:28.210 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:28.210 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 54 -lt 100 ']' 2023-11-27T15:19:28.210 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 54 + 1 2023-11-27T15:19:28.243 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=55 2023-11-27T15:19:28.243 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:28.289 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:28.289 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:28.290 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:28.291 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:28.291 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:28.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.661+0000 7fe05560e700 1 -- 172.21.15.52:0/869780883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe050101400 msgr2=0x7fe0501017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:28.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.661+0000 7fe05560e700 1 --2- 172.21.15.52:0/869780883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe050101400 0x7fe0501017d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe040009990 tx=0x7fe040009ca0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.661+0000 7fe05560e700 1 -- 172.21.15.52:0/869780883 shutdown_connections 2023-11-27T15:19:28.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.661+0000 7fe05560e700 1 --2- 172.21.15.52:0/869780883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe05010a920 0x7fe05010cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.661+0000 7fe05560e700 1 --2- 172.21.15.52:0/869780883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe050101400 0x7fe0501017d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.661+0000 7fe05560e700 1 --2- 172.21.15.52:0/869780883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe050101d10 0x7fe05010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.661+0000 7fe05560e700 1 -- 172.21.15.52:0/869780883 >> 172.21.15.52:0/869780883 conn(0x7fe050076200 msgr2=0x7fe050076600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:28.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.661+0000 7fe05560e700 1 -- 172.21.15.52:0/869780883 shutdown_connections 2023-11-27T15:19:28.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.662+0000 7fe05560e700 1 -- 172.21.15.52:0/869780883 wait complete. 2023-11-27T15:19:28.669 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.662+0000 7fe05560e700 1 Processor -- start 2023-11-27T15:19:28.669 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.662+0000 7fe05560e700 1 -- start start 2023-11-27T15:19:28.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.663+0000 7fe05560e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe050101400 0x7fe050196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:28.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.663+0000 7fe05560e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe050101d10 0x7fe0501969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:28.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.663+0000 7fe05560e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe05010a920 0x7fe05019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:28.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.663+0000 7fe05560e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05019b470 con 0x7fe05010a920 2023-11-27T15:19:28.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.663+0000 7fe05560e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05019b5e0 con 0x7fe050101400 2023-11-27T15:19:28.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.663+0000 7fe05560e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05019b7a0 con 0x7fe050101d10 2023-11-27T15:19:28.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.663+0000 7fe04f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe05010a920 0x7fe05019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:28.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.663+0000 7fe04f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe05010a920 0x7fe05019ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52788/0 (socket says 172.21.15.52:52788) 2023-11-27T15:19:28.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.663+0000 7fe04f7fe700 1 -- 172.21.15.52:0/1631703157 learned_addr learned my addr 172.21.15.52:0/1631703157 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:28.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.663+0000 7fe04effd700 1 --2- 172.21.15.52:0/1631703157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe050101400 0x7fe050196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:28.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.663+0000 7fe04e7fc700 1 --2- 172.21.15.52:0/1631703157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe050101d10 0x7fe0501969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:28.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.663+0000 7fe04f7fe700 1 -- 172.21.15.52:0/1631703157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe050101d10 msgr2=0x7fe0501969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:28.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.663+0000 7fe04f7fe700 1 --2- 172.21.15.52:0/1631703157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe050101d10 0x7fe0501969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.663+0000 7fe04f7fe700 1 -- 172.21.15.52:0/1631703157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe050101400 msgr2=0x7fe050196490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:28.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.663+0000 7fe04f7fe700 1 --2- 172.21.15.52:0/1631703157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe050101400 0x7fe050196490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.663+0000 7fe04f7fe700 1 -- 172.21.15.52:0/1631703157 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0400095f0 con 0x7fe05010a920 2023-11-27T15:19:28.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.663+0000 7fe04f7fe700 1 --2- 172.21.15.52:0/1631703157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe05010a920 0x7fe05019ae30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe04800ef90 tx=0x7fe04800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:28.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.664+0000 7fe037fff700 1 -- 172.21.15.52:0/1631703157 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe048018070 con 0x7fe05010a920 2023-11-27T15:19:28.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.664+0000 7fe037fff700 1 -- 172.21.15.52:0/1631703157 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe04800f040 con 0x7fe05010a920 2023-11-27T15:19:28.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.664+0000 7fe05560e700 1 -- 172.21.15.52:0/1631703157 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0501103a0 con 0x7fe05010a920 2023-11-27T15:19:28.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.664+0000 7fe037fff700 1 -- 172.21.15.52:0/1631703157 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe048003e00 con 0x7fe05010a920 2023-11-27T15:19:28.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.664+0000 7fe05560e700 1 -- 172.21.15.52:0/1631703157 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe050110210 con 0x7fe05010a920 2023-11-27T15:19:28.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.666+0000 7fe037fff700 1 -- 172.21.15.52:0/1631703157 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe04800ca50 con 0x7fe05010a920 2023-11-27T15:19:28.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.667+0000 7fe037fff700 1 --2- 172.21.15.52:0/1631703157 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe0380a12c0 0x7fe0380a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:28.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.667+0000 7fe04effd700 1 --2- 172.21.15.52:0/1631703157 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe0380a12c0 0x7fe0380a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:28.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.667+0000 7fe037fff700 1 -- 172.21.15.52:0/1631703157 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fe0480cc930 con 0x7fe05010a920 2023-11-27T15:19:28.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.667+0000 7fe04effd700 1 --2- 172.21.15.52:0/1631703157 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe0380a12c0 0x7fe0380a3770 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe04000b2e0 tx=0x7fe040005fd0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:28.741 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.735+0000 7fe06179ab80 1 Processor -- start 2023-11-27T15:19:28.741 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.735+0000 7fe06179ab80 1 -- start start 2023-11-27T15:19:28.742 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.735+0000 7fe06179ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb67e69b60 0x55bb67e69f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:28.742 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.735+0000 7fe06179ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb67e6a470 0x55bb67e6e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:28.742 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.735+0000 7fe06179ab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb67e6e8b0 0x55bb67e6ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:28.742 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.735+0000 7fe06179ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb67d6f060 con 0x55bb67e6a470 2023-11-27T15:19:28.742 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.735+0000 7fe06179ab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb67dc18a0 con 0x55bb67e6e8b0 2023-11-27T15:19:28.742 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.735+0000 7fe06179ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb67dbad30 con 0x55bb67e69b60 2023-11-27T15:19:28.742 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.735+0000 7fe04e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb67e6a470 0x55bb67e6e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:28.742 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.736+0000 7fe04e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb67e6a470 0x55bb67e6e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52794/0 (socket says 172.21.15.52:52794) 2023-11-27T15:19:28.742 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.736+0000 7fe04e7fc700 1 -- 172.21.15.52:0/2783136798 learned_addr learned my addr 172.21.15.52:0/2783136798 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:28.742 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.736+0000 7fe04f7fe700 1 --2- 172.21.15.52:0/2783136798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb67e6e8b0 0x55bb67e6ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:28.743 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.736+0000 7fe04effd700 1 --2- 172.21.15.52:0/2783136798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb67e69b60 0x55bb67e69f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:28.743 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.736+0000 7fe04e7fc700 1 -- 172.21.15.52:0/2783136798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb67e69b60 msgr2=0x55bb67e69f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:28.743 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.736+0000 7fe04e7fc700 1 --2- 172.21.15.52:0/2783136798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb67e69b60 0x55bb67e69f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.743 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.736+0000 7fe04e7fc700 1 -- 172.21.15.52:0/2783136798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb67e6e8b0 msgr2=0x55bb67e6ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:28.743 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.736+0000 7fe04e7fc700 1 --2- 172.21.15.52:0/2783136798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb67e6e8b0 0x55bb67e6ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.743 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.736+0000 7fe04e7fc700 1 -- 172.21.15.52:0/2783136798 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb67c3c630 con 0x55bb67e6a470 2023-11-27T15:19:28.743 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.736+0000 7fe04e7fc700 1 --2- 172.21.15.52:0/2783136798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb67e6a470 0x55bb67e6e370 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe044009cb0 tx=0x7fe0440113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2925d3a6438d98 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:28.743 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.736+0000 7fe0357fa700 1 -- 172.21.15.52:0/2783136798 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0440097b0 con 0x55bb67e6a470 2023-11-27T15:19:28.743 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.736+0000 7fe0357fa700 1 -- 172.21.15.52:0/2783136798 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe044011c60 con 0x55bb67e6a470 2023-11-27T15:19:28.743 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.736+0000 7fe0357fa700 1 -- 172.21.15.52:0/2783136798 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04400f730 con 0x55bb67e6a470 2023-11-27T15:19:28.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.737+0000 7fe06179ab80 1 -- 172.21.15.52:0/2783136798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb67e6a470 msgr2=0x55bb67e6e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:28.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.737+0000 7fe06179ab80 1 --2- 172.21.15.52:0/2783136798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb67e6a470 0x55bb67e6e370 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe044009cb0 tx=0x7fe0440113e0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.737+0000 7fe06179ab80 1 -- 172.21.15.52:0/2783136798 shutdown_connections 2023-11-27T15:19:28.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.737+0000 7fe06179ab80 1 --2- 172.21.15.52:0/2783136798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb67e6e8b0 0x55bb67e6ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.737+0000 7fe06179ab80 1 --2- 172.21.15.52:0/2783136798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb67e69b60 0x55bb67e69f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.737+0000 7fe06179ab80 1 --2- 172.21.15.52:0/2783136798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb67e6a470 0x55bb67e6e370 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.737+0000 7fe06179ab80 1 -- 172.21.15.52:0/2783136798 >> 172.21.15.52:0/2783136798 conn(0x55bb67e5f360 msgr2=0x55bb67e64b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:28.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.737+0000 7fe06179ab80 1 -- 172.21.15.52:0/2783136798 shutdown_connections 2023-11-27T15:19:28.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.737+0000 7fe06179ab80 1 -- 172.21.15.52:0/2783136798 wait complete. 2023-11-27T15:19:28.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.738+0000 7fe06179ab80 1 -- start start 2023-11-27T15:19:28.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.738+0000 7fe06179ab80 1 Processor -- start 2023-11-27T15:19:28.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.738+0000 7fe06179ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb67e69b60 0x55bb67e76050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:28.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.738+0000 7fe06179ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb67e6a470 0x55bb67e73260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:28.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.738+0000 7fe06179ab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb67e6e8b0 0x55bb67e73870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:28.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.738+0000 7fe06179ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb67d5c8e0 con 0x55bb67e6a470 2023-11-27T15:19:28.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.738+0000 7fe06179ab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb67d5c580 con 0x55bb67e6e8b0 2023-11-27T15:19:28.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.738+0000 7fe06179ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb67c3e920 con 0x55bb67e69b60 2023-11-27T15:19:28.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.738+0000 7fe06179ab80 10 client.0 _ll_register_callbacks cb 0x55bb67d7a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:19:28.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.738+0000 7fe04e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb67e6a470 0x55bb67e73260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:28.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.738+0000 7fe04e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb67e6a470 0x55bb67e73260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52800/0 (socket says 172.21.15.52:52800) 2023-11-27T15:19:28.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.738+0000 7fe04e7fc700 1 -- 172.21.15.52:0/3123369945 learned_addr learned my addr 172.21.15.52:0/3123369945 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:28.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.738+0000 7fe04effd700 1 --2- 172.21.15.52:0/3123369945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb67e69b60 0x55bb67e76050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:28.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.738+0000 7fe04f7fe700 1 --2- 172.21.15.52:0/3123369945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb67e6e8b0 0x55bb67e73870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:28.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.739+0000 7fe04e7fc700 1 -- 172.21.15.52:0/3123369945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb67e69b60 msgr2=0x55bb67e76050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:28.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.739+0000 7fe04e7fc700 1 --2- 172.21.15.52:0/3123369945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb67e69b60 0x55bb67e76050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.739+0000 7fe04e7fc700 1 -- 172.21.15.52:0/3123369945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb67e6e8b0 msgr2=0x55bb67e73870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:28.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.739+0000 7fe04e7fc700 1 --2- 172.21.15.52:0/3123369945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb67e6e8b0 0x55bb67e73870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.739+0000 7fe04e7fc700 1 -- 172.21.15.52:0/3123369945 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe044009910 con 0x55bb67e6a470 2023-11-27T15:19:28.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.739+0000 7fe04effd700 1 --2- 172.21.15.52:0/3123369945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb67e69b60 0x55bb67e76050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:28.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.739+0000 7fe04f7fe700 1 --2- 172.21.15.52:0/3123369945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb67e6e8b0 0x55bb67e73870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:28.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.739+0000 7fe04e7fc700 1 --2- 172.21.15.52:0/3123369945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb67e6a470 0x55bb67e73260 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe044012e80 tx=0x7fe044012ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1896f8b8dceaeb5a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:28.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.739+0000 7fe032ff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:19:28.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.739+0000 7fe032ff5700 1 -- 172.21.15.52:0/3123369945 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe044004520 con 0x55bb67e6a470 2023-11-27T15:19:28.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.739+0000 7fe032ff5700 1 -- 172.21.15.52:0/3123369945 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fe044012660 con 0x55bb67e6a470 2023-11-27T15:19:28.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.739+0000 7fe06179ab80 1 -- 172.21.15.52:0/3123369945 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb67e5cb20 con 0x55bb67e6a470 2023-11-27T15:19:28.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.739+0000 7fe032ff5700 1 -- 172.21.15.52:0/3123369945 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe044012820 con 0x55bb67e6a470 2023-11-27T15:19:28.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.739+0000 7fe06179ab80 10 client.15732 fetch_fsmap learned FSMap version 23 2023-11-27T15:19:28.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.739+0000 7fe06179ab80 1 -- 172.21.15.52:0/3123369945 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55bb67d81190 con 0x55bb67e6a470 2023-11-27T15:19:28.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.739+0000 7fe032ff5700 1 -- 172.21.15.52:0/3123369945 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bb67e5cb20 con 0x55bb67e6a470 2023-11-27T15:19:28.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.740+0000 7fe032ff5700 1 -- 172.21.15.52:0/3123369945 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe044022040 con 0x55bb67e6a470 2023-11-27T15:19:28.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.740+0000 7fe06179ab80 10 client.15732 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:19:28.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.740+0000 7fe06179ab80 10 client.15732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:19:28.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.740+0000 7fe06179ab80 20 client.15732 populate_metadata read hostname 'smithi052' 2023-11-27T15:19:28.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.740+0000 7fe06179ab80 1 --2- 172.21.15.52:0/3123369945 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55bb67e83710 0x55bb67e85b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:28.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.740+0000 7fe06179ab80 4 client.15732 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:19:28.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.740+0000 7fe06179ab80 1 -- 172.21.15.52:0/3123369945 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb67d7d0a0 con 0x55bb67e83710 2023-11-27T15:19:28.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.740+0000 7fe04effd700 1 --2- 172.21.15.52:0/3123369945 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55bb67e83710 0x55bb67e85b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:28.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.740+0000 7fe032ff5700 10 client.15732 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:28.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.740+0000 7fe04effd700 1 --2- 172.21.15.52:0/3123369945 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55bb67e83710 0x55bb67e85b20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e1feab7aa5a4af server_cookie=84de8b78c9b54e2a in_seq=0 out_seq=0 2023-11-27T15:19:28.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.741+0000 7fe032ff5700 1 -- 172.21.15.52:0/3123369945 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bb67d7d0a0 con 0x55bb67e83710 2023-11-27T15:19:28.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.741+0000 7fe032ff5700 10 client.15732 handle_command_reply: tid=0 2023-11-27T15:19:28.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.741+0000 7fe06179ab80 1 client.15732 shutdown 2023-11-27T15:19:28.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.741+0000 7fe06179ab80 1 -- 172.21.15.52:0/3123369945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb67e6a470 msgr2=0x55bb67e73260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:28.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.741+0000 7fe06179ab80 1 --2- 172.21.15.52:0/3123369945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb67e6a470 0x55bb67e73260 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe044012e80 tx=0x7fe044012ca0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.741+0000 7fe06179ab80 1 -- 172.21.15.52:0/3123369945 shutdown_connections 2023-11-27T15:19:28.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.741+0000 7fe06179ab80 1 --2- 172.21.15.52:0/3123369945 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55bb67e83710 0x55bb67e85b20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.741+0000 7fe06179ab80 1 --2- 172.21.15.52:0/3123369945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb67e6e8b0 0x55bb67e73870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.741+0000 7fe06179ab80 1 --2- 172.21.15.52:0/3123369945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb67e69b60 0x55bb67e76050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.741+0000 7fe032ff5700 0 client.15732 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:28.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.741+0000 7fe06179ab80 1 --2- 172.21.15.52:0/3123369945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb67e6a470 0x55bb67e73260 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.741+0000 7fe06179ab80 1 -- 172.21.15.52:0/3123369945 >> 172.21.15.52:0/3123369945 conn(0x55bb67e5f360 msgr2=0x55bb67e6fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:28.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.741+0000 7fe06179ab80 1 -- 172.21.15.52:0/3123369945 shutdown_connections 2023-11-27T15:19:28.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.741+0000 7fe06179ab80 1 -- 172.21.15.52:0/3123369945 wait complete. 2023-11-27T15:19:28.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.742+0000 7fe06179ab80 20 client.15732 trim_cache size 0 max 16384 2023-11-27T15:19:28.761 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.755+0000 7fe06179ab80 1 Processor -- start 2023-11-27T15:19:28.761 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.755+0000 7fe06179ab80 1 -- start start 2023-11-27T15:19:28.761 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.755+0000 7fe06179ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb67e69b60 0x55bb67e7fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:28.761 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.755+0000 7fe06179ab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb67e6a470 0x55bb67e75b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:28.761 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.755+0000 7fe06179ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb67e6e8b0 0x55bb67e6c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:28.761 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.755+0000 7fe06179ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb67e5eee0 con 0x55bb67e69b60 2023-11-27T15:19:28.762 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.755+0000 7fe06179ab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb67e66280 con 0x55bb67e6a470 2023-11-27T15:19:28.762 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.755+0000 7fe06179ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb67e71800 con 0x55bb67e6e8b0 2023-11-27T15:19:28.762 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.755+0000 7fe04effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb67e69b60 0x55bb67e7fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:28.762 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.755+0000 7fe04effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb67e69b60 0x55bb67e7fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52806/0 (socket says 172.21.15.52:52806) 2023-11-27T15:19:28.762 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.755+0000 7fe04effd700 1 -- 172.21.15.52:0/626911422 learned_addr learned my addr 172.21.15.52:0/626911422 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:28.762 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.755+0000 7fe04e7fc700 1 --2- 172.21.15.52:0/626911422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb67e6a470 0x55bb67e75b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:28.762 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.755+0000 7fe04f7fe700 1 --2- 172.21.15.52:0/626911422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb67e6e8b0 0x55bb67e6c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:28.762 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.755+0000 7fe04effd700 1 -- 172.21.15.52:0/626911422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb67e6e8b0 msgr2=0x55bb67e6c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:28.762 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.755+0000 7fe04effd700 1 --2- 172.21.15.52:0/626911422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb67e6e8b0 0x55bb67e6c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.762 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.755+0000 7fe04effd700 1 -- 172.21.15.52:0/626911422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb67e6a470 msgr2=0x55bb67e75b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:28.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.755+0000 7fe04effd700 1 --2- 172.21.15.52:0/626911422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb67e6a470 0x55bb67e75b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.756+0000 7fe04effd700 1 -- 172.21.15.52:0/626911422 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb67d80ee0 con 0x55bb67e69b60 2023-11-27T15:19:28.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.756+0000 7fe04f7fe700 1 --2- 172.21.15.52:0/626911422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb67e6e8b0 0x55bb67e6c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:19:28.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.756+0000 7fe04e7fc700 1 --2- 172.21.15.52:0/626911422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb67e6a470 0x55bb67e75b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:19:28.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.756+0000 7fe04effd700 1 --2- 172.21.15.52:0/626911422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb67e69b60 0x55bb67e7fb80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe0400098e0 tx=0x7fe040023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ffba4d6c422f3b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:28.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.756+0000 7fe035ffb700 1 -- 172.21.15.52:0/626911422 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe040034d10 con 0x55bb67e69b60 2023-11-27T15:19:28.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.756+0000 7fe035ffb700 1 -- 172.21.15.52:0/626911422 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe04002c040 con 0x55bb67e69b60 2023-11-27T15:19:28.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.756+0000 7fe035ffb700 1 -- 172.21.15.52:0/626911422 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe040004bb0 con 0x55bb67e69b60 2023-11-27T15:19:28.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.757+0000 7fe06179ab80 1 -- 172.21.15.52:0/626911422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb67e69b60 msgr2=0x55bb67e7fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:28.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.757+0000 7fe06179ab80 1 --2- 172.21.15.52:0/626911422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb67e69b60 0x55bb67e7fb80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe0400098e0 tx=0x7fe040023000 comp rx=0 tx=0).stop 2023-11-27T15:19:28.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.757+0000 7fe06179ab80 1 -- 172.21.15.52:0/626911422 shutdown_connections 2023-11-27T15:19:28.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.757+0000 7fe06179ab80 1 --2- 172.21.15.52:0/626911422 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb67e6e8b0 0x55bb67e6c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.757+0000 7fe06179ab80 1 --2- 172.21.15.52:0/626911422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb67e69b60 0x55bb67e7fb80 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.757+0000 7fe06179ab80 1 --2- 172.21.15.52:0/626911422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb67e6a470 0x55bb67e75b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.757+0000 7fe06179ab80 1 -- 172.21.15.52:0/626911422 >> 172.21.15.52:0/626911422 conn(0x55bb67e5f360 msgr2=0x55bb67e70c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:28.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.757+0000 7fe06179ab80 1 -- 172.21.15.52:0/626911422 shutdown_connections 2023-11-27T15:19:28.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.757+0000 7fe06179ab80 1 -- 172.21.15.52:0/626911422 wait complete. 2023-11-27T15:19:28.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.757+0000 7fe06179ab80 1 -- start start 2023-11-27T15:19:28.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.757+0000 7fe06179ab80 1 Processor -- start 2023-11-27T15:19:28.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.757+0000 7fe06179ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb67e69b60 0x55bb67e78990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:28.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.757+0000 7fe06179ab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb67e6a470 0x55bb67e78ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:28.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.757+0000 7fe06179ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb67e6e8b0 0x55bb67e79430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:28.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.757+0000 7fe06179ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb67ed67b0 con 0x55bb67e69b60 2023-11-27T15:19:28.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.757+0000 7fe06179ab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb67ed6920 con 0x55bb67e6a470 2023-11-27T15:19:28.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.757+0000 7fe06179ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb67ed6ae0 con 0x55bb67e6e8b0 2023-11-27T15:19:28.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.757+0000 7fe06179ab80 10 client.0 _ll_register_callbacks cb 0x7fe048009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:19:28.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.757+0000 7fe04effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb67e69b60 0x55bb67e78990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:28.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.758+0000 7fe04effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb67e69b60 0x55bb67e78990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52812/0 (socket says 172.21.15.52:52812) 2023-11-27T15:19:28.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.758+0000 7fe04effd700 1 -- 172.21.15.52:0/3166302528 learned_addr learned my addr 172.21.15.52:0/3166302528 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:28.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.758+0000 7fe04e7fc700 1 --2- 172.21.15.52:0/3166302528 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb67e6a470 0x55bb67e78ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:28.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.758+0000 7fe04f7fe700 1 --2- 172.21.15.52:0/3166302528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb67e6e8b0 0x55bb67e79430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:28.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.758+0000 7fe04effd700 1 -- 172.21.15.52:0/3166302528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb67e6e8b0 msgr2=0x55bb67e79430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:28.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.758+0000 7fe04effd700 1 --2- 172.21.15.52:0/3166302528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb67e6e8b0 0x55bb67e79430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.758+0000 7fe04effd700 1 -- 172.21.15.52:0/3166302528 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb67e6a470 msgr2=0x55bb67e78ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:28.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.758+0000 7fe04effd700 1 --2- 172.21.15.52:0/3166302528 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb67e6a470 0x55bb67e78ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.758+0000 7fe04effd700 1 -- 172.21.15.52:0/3166302528 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe05010da00 con 0x55bb67e69b60 2023-11-27T15:19:28.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.758+0000 7fe04effd700 1 --2- 172.21.15.52:0/3166302528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb67e69b60 0x55bb67e78990 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe04000c010 tx=0x7fe040004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa780303d04f62d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:28.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.758+0000 7fe032ff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:19:28.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.758+0000 7fe032ff5700 1 -- 172.21.15.52:0/3166302528 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe040034050 con 0x55bb67e69b60 2023-11-27T15:19:28.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.758+0000 7fe032ff5700 1 -- 172.21.15.52:0/3166302528 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fe0400341b0 con 0x55bb67e69b60 2023-11-27T15:19:28.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.758+0000 7fe06179ab80 1 -- 172.21.15.52:0/3166302528 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe044011c60 con 0x55bb67e69b60 2023-11-27T15:19:28.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.758+0000 7fe032ff5700 1 -- 172.21.15.52:0/3166302528 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe040034370 con 0x55bb67e69b60 2023-11-27T15:19:28.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.758+0000 7fe06179ab80 10 client.15738 fetch_fsmap learned FSMap version 23 2023-11-27T15:19:28.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.758+0000 7fe06179ab80 1 -- 172.21.15.52:0/3166302528 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55bb67bd1c60 con 0x55bb67e69b60 2023-11-27T15:19:28.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.759+0000 7fe032ff5700 1 -- 172.21.15.52:0/3166302528 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe040051080 con 0x55bb67e69b60 2023-11-27T15:19:28.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.759+0000 7fe032ff5700 1 -- 172.21.15.52:0/3166302528 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe04004b030 con 0x55bb67e69b60 2023-11-27T15:19:28.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.759+0000 7fe06179ab80 10 client.15738 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:19:28.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.759+0000 7fe06179ab80 10 client.15738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:19:28.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.759+0000 7fe06179ab80 20 client.15738 populate_metadata read hostname 'smithi052' 2023-11-27T15:19:28.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.759+0000 7fe06179ab80 1 --2- 172.21.15.52:0/3166302528 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55bb67ed7d70 0x55bb67eda180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:28.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.759+0000 7fe06179ab80 4 client.15738 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:19:28.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.759+0000 7fe06179ab80 1 -- 172.21.15.52:0/3166302528 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bb67d89660 con 0x55bb67ed7d70 2023-11-27T15:19:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.759+0000 7fe04e7fc700 1 --2- 172.21.15.52:0/3166302528 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55bb67ed7d70 0x55bb67eda180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.760+0000 7fe032ff5700 10 client.15738 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.760+0000 7fe04e7fc700 1 --2- 172.21.15.52:0/3166302528 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55bb67ed7d70 0x55bb67eda180 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=697b9f6cc0524f4e server_cookie=c0651e4b5bbe7884 in_seq=0 out_seq=0 2023-11-27T15:19:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.760+0000 7fe032ff5700 1 -- 172.21.15.52:0/3166302528 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bb67d89660 con 0x55bb67ed7d70 2023-11-27T15:19:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.760+0000 7fe032ff5700 10 client.15738 handle_command_reply: tid=0 2023-11-27T15:19:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.760+0000 7fe06179ab80 1 client.15738 shutdown 2023-11-27T15:19:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.760+0000 7fe06179ab80 1 -- 172.21.15.52:0/3166302528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb67e69b60 msgr2=0x55bb67e78990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.760+0000 7fe06179ab80 1 --2- 172.21.15.52:0/3166302528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb67e69b60 0x55bb67e78990 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe04000c010 tx=0x7fe040004590 comp rx=0 tx=0).stop 2023-11-27T15:19:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.760+0000 7fe06179ab80 1 -- 172.21.15.52:0/3166302528 shutdown_connections 2023-11-27T15:19:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.760+0000 7fe06179ab80 1 --2- 172.21.15.52:0/3166302528 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55bb67ed7d70 0x55bb67eda180 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.760+0000 7fe06179ab80 1 --2- 172.21.15.52:0/3166302528 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bb67e6e8b0 0x55bb67e79430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.760+0000 7fe06179ab80 1 --2- 172.21.15.52:0/3166302528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb67e69b60 0x55bb67e78990 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.760+0000 7fe032ff5700 0 client.15738 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:28.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.760+0000 7fe06179ab80 1 --2- 172.21.15.52:0/3166302528 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb67e6a470 0x55bb67e78ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.760+0000 7fe06179ab80 1 -- 172.21.15.52:0/3166302528 >> 172.21.15.52:0/3166302528 conn(0x55bb67e5f360 msgr2=0x55bb67e63c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:28.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.760+0000 7fe06179ab80 1 -- 172.21.15.52:0/3166302528 shutdown_connections 2023-11-27T15:19:28.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.760+0000 7fe06179ab80 1 -- 172.21.15.52:0/3166302528 wait complete. 2023-11-27T15:19:28.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.760+0000 7fe06179ab80 20 client.15738 trim_cache size 0 max 16384 2023-11-27T15:19:28.772 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:19:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.761+0000 7fe05560e700 1 -- 172.21.15.52:0/1631703157 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe0380a12c0 msgr2=0x7fe0380a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.761+0000 7fe05560e700 1 --2- 172.21.15.52:0/1631703157 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe0380a12c0 0x7fe0380a3770 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe04000b2e0 tx=0x7fe040005fd0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.761+0000 7fe05560e700 1 -- 172.21.15.52:0/1631703157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe05010a920 msgr2=0x7fe05019ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.761+0000 7fe05560e700 1 --2- 172.21.15.52:0/1631703157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe05010a920 0x7fe05019ae30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe04800ef90 tx=0x7fe04800c3a0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.761+0000 7fe05560e700 1 -- 172.21.15.52:0/1631703157 shutdown_connections 2023-11-27T15:19:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.761+0000 7fe05560e700 1 --2- 172.21.15.52:0/1631703157 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe0380a12c0 0x7fe0380a3770 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.761+0000 7fe05560e700 1 --2- 172.21.15.52:0/1631703157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe050101400 0x7fe050196490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.761+0000 7fe05560e700 1 --2- 172.21.15.52:0/1631703157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe05010a920 0x7fe05019ae30 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.761+0000 7fe05560e700 1 --2- 172.21.15.52:0/1631703157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe050101d10 0x7fe0501969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.761+0000 7fe05560e700 1 -- 172.21.15.52:0/1631703157 >> 172.21.15.52:0/1631703157 conn(0x7fe050076200 msgr2=0x7fe0500fbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.761+0000 7fe05560e700 1 -- 172.21.15.52:0/1631703157 shutdown_connections 2023-11-27T15:19:28.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:28.761+0000 7fe05560e700 1 -- 172.21.15.52:0/1631703157 wait complete. 2023-11-27T15:19:28.870 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:28.870 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:28.972 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:28.972 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:28.985 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:28.985 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:28.985 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 55 -lt 100 ']' 2023-11-27T15:19:28.986 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 55 + 1 2023-11-27T15:19:29.034 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=56 2023-11-27T15:19:29.034 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:29.108 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:29.108 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:29.109 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:29.109 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:29.110 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:29.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:28 smithi052 ceph-mon[99056]: pgmap v49: 97 pgs: 97 active+clean; 2.1 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 10 op/s 2023-11-27T15:19:29.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:28 smithi052 ceph-mon[99056]: mgrmap e26: x(active, since 92s), standbys: y, z 2023-11-27T15:19:29.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:28 smithi150 ceph-mon[100559]: pgmap v49: 97 pgs: 97 active+clean; 2.1 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 10 op/s 2023-11-27T15:19:29.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:28 smithi150 ceph-mon[100559]: mgrmap e26: x(active, since 92s), standbys: y, z 2023-11-27T15:19:29.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:28 smithi102 ceph-mon[100296]: pgmap v49: 97 pgs: 97 active+clean; 2.1 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 10 op/s 2023-11-27T15:19:29.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:28 smithi102 ceph-mon[100296]: mgrmap e26: x(active, since 92s), standbys: y, z 2023-11-27T15:19:29.578 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:29.578 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:29.671 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:29.671 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:29.684 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:29.684 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:29.684 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 56 -lt 100 ']' 2023-11-27T15:19:29.684 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 56 + 1 2023-11-27T15:19:29.696 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=57 2023-11-27T15:19:29.697 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:29.731 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:29.731 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:29.732 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:29.732 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:29.732 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:29.884 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-27T15:19:29.885 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-27T15:19:30.111 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:30.112 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:30.163 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:30.163 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:30.166 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:30.166 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:30.167 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 57 -lt 100 ']' 2023-11-27T15:19:30.167 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 57 + 1 2023-11-27T15:19:30.209 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=58 2023-11-27T15:19:30.209 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:30.270 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:30.270 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:30.271 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:30.271 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:30.272 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:30.795 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:30.795 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:30.866 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:30 smithi052 ceph-mon[99056]: pgmap v50: 97 pgs: 97 active+clean; 2.1 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-11-27T15:19:30.955 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:30.955 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:30.992 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:30.993 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:30.993 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 58 -lt 100 ']' 2023-11-27T15:19:30.993 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 58 + 1 2023-11-27T15:19:31.068 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=59 2023-11-27T15:19:31.068 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:31.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:30 smithi150 ceph-mon[100559]: pgmap v50: 97 pgs: 97 active+clean; 2.1 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-11-27T15:19:31.317 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:31.317 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:31.318 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:31.318 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:31.319 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:31.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:30 smithi102 ceph-mon[100296]: pgmap v50: 97 pgs: 97 active+clean; 2.1 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-11-27T15:19:32.053 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:32.054 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:32.205 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:32.205 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:32.234 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:32.234 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:32.235 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 59 -lt 100 ']' 2023-11-27T15:19:32.235 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 59 + 1 2023-11-27T15:19:32.276 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=60 2023-11-27T15:19:32.276 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:32.357 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:32.357 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:32.358 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:32.358 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:32.359 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:32.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.594+0000 7ff6d1208700 1 -- 172.21.15.52:0/1542202643 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6cc1005b0 msgr2=0x7ff6cc100980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:32.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.594+0000 7ff6d1208700 1 --2- 172.21.15.52:0/1542202643 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6cc1005b0 0x7ff6cc100980 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc0099c0 tx=0x7ff6bc009cd0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.595+0000 7ff6d1208700 1 -- 172.21.15.52:0/1542202643 shutdown_connections 2023-11-27T15:19:32.601 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.595+0000 7ff6d1208700 1 --2- 172.21.15.52:0/1542202643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6cc1055f0 0x7ff6cc10d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.601 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.595+0000 7ff6d1208700 1 --2- 172.21.15.52:0/1542202643 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6cc1005b0 0x7ff6cc100980 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.601 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.595+0000 7ff6d1208700 1 --2- 172.21.15.52:0/1542202643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6cc100f50 0x7ff6cc104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.601 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.595+0000 7ff6d1208700 1 -- 172.21.15.52:0/1542202643 >> 172.21.15.52:0/1542202643 conn(0x7ff6cc077140 msgr2=0x7ff6cc077540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:32.601 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.595+0000 7ff6d1208700 1 -- 172.21.15.52:0/1542202643 shutdown_connections 2023-11-27T15:19:32.602 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.595+0000 7ff6d1208700 1 -- 172.21.15.52:0/1542202643 wait complete. 2023-11-27T15:19:32.604 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.596+0000 7ff6d1208700 1 Processor -- start 2023-11-27T15:19:32.604 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.596+0000 7ff6d1208700 1 -- start start 2023-11-27T15:19:32.605 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.596+0000 7ff6d1208700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6cc1005b0 0x7ff6cc072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:32.605 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.596+0000 7ff6d1208700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6cc100f50 0x7ff6cc073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:32.605 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.596+0000 7ff6d1208700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6cc1055f0 0x7ff6cc1a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:32.605 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.596+0000 7ff6d1208700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc073b60 con 0x7ff6cc1055f0 2023-11-27T15:19:32.605 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.596+0000 7ff6d1208700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc1a4b30 con 0x7ff6cc1005b0 2023-11-27T15:19:32.605 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.596+0000 7ff6d1208700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc1a4c70 con 0x7ff6cc100f50 2023-11-27T15:19:32.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.597+0000 7ff6cb59e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6cc1055f0 0x7ff6cc1a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:32.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.597+0000 7ff6cb59e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6cc1055f0 0x7ff6cc1a45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52830/0 (socket says 172.21.15.52:52830) 2023-11-27T15:19:32.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.597+0000 7ff6cb59e700 1 -- 172.21.15.52:0/3098467062 learned_addr learned my addr 172.21.15.52:0/3098467062 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:32.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.597+0000 7ff6cad9d700 1 --2- 172.21.15.52:0/3098467062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6cc1005b0 0x7ff6cc072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:32.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.597+0000 7ff6ca59c700 1 --2- 172.21.15.52:0/3098467062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6cc100f50 0x7ff6cc073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:32.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.597+0000 7ff6cb59e700 1 -- 172.21.15.52:0/3098467062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6cc100f50 msgr2=0x7ff6cc073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:32.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.597+0000 7ff6cb59e700 1 --2- 172.21.15.52:0/3098467062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6cc100f50 0x7ff6cc073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.607 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.597+0000 7ff6cb59e700 1 -- 172.21.15.52:0/3098467062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6cc1005b0 msgr2=0x7ff6cc072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:32.607 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.597+0000 7ff6cb59e700 1 --2- 172.21.15.52:0/3098467062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6cc1005b0 0x7ff6cc072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.607 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.597+0000 7ff6cb59e700 1 -- 172.21.15.52:0/3098467062 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6bc0095f0 con 0x7ff6cc1055f0 2023-11-27T15:19:32.607 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.597+0000 7ff6ca59c700 1 --2- 172.21.15.52:0/3098467062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6cc100f50 0x7ff6cc073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:19:32.607 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.597+0000 7ff6cad9d700 1 --2- 172.21.15.52:0/3098467062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6cc1005b0 0x7ff6cc072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:19:32.607 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.597+0000 7ff6cb59e700 1 --2- 172.21.15.52:0/3098467062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6cc1055f0 0x7ff6cc1a45f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff6c400efc0 tx=0x7ff6c400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:32.607 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.597+0000 7ff6b3fff700 1 -- 172.21.15.52:0/3098467062 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4018070 con 0x7ff6cc1055f0 2023-11-27T15:19:32.608 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.598+0000 7ff6b3fff700 1 -- 172.21.15.52:0/3098467062 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6c400f040 con 0x7ff6cc1055f0 2023-11-27T15:19:32.608 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.598+0000 7ff6d1208700 1 -- 172.21.15.52:0/3098467062 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6cc1103b0 con 0x7ff6cc1055f0 2023-11-27T15:19:32.608 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.598+0000 7ff6b3fff700 1 -- 172.21.15.52:0/3098467062 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c400cd80 con 0x7ff6cc1055f0 2023-11-27T15:19:32.608 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.598+0000 7ff6d1208700 1 -- 172.21.15.52:0/3098467062 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6cc110220 con 0x7ff6cc1055f0 2023-11-27T15:19:32.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.600+0000 7ff6b3fff700 1 -- 172.21.15.52:0/3098467062 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7ff6c4003a00 con 0x7ff6cc1055f0 2023-11-27T15:19:32.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.601+0000 7ff6b3fff700 1 --2- 172.21.15.52:0/3098467062 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ff6b40a12c0 0x7ff6b40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:32.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.601+0000 7ff6b3fff700 1 -- 172.21.15.52:0/3098467062 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff6c40cc850 con 0x7ff6cc1055f0 2023-11-27T15:19:32.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.601+0000 7ff6cad9d700 1 --2- 172.21.15.52:0/3098467062 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ff6b40a12c0 0x7ff6b40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:32.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.602+0000 7ff6cad9d700 1 --2- 172.21.15.52:0/3098467062 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ff6b40a12c0 0x7ff6b40a3770 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc009990 tx=0x7ff6bc01a040 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:32.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.676+0000 7ff6dd394b80 1 Processor -- start 2023-11-27T15:19:32.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.676+0000 7ff6dd394b80 1 -- start start 2023-11-27T15:19:32.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.677+0000 7ff6dd394b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560649de7be0 0x560649de7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:32.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.677+0000 7ff6dd394b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560649de84f0 0x560649dec3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:32.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.677+0000 7ff6dd394b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560649dec930 0x560649decd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:32.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.677+0000 7ff6dd394b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x560649ced0e0 con 0x560649dec930 2023-11-27T15:19:32.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.677+0000 7ff6dd394b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560649d3f920 con 0x560649de7be0 2023-11-27T15:19:32.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.677+0000 7ff6dd394b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560649d38db0 con 0x560649de84f0 2023-11-27T15:19:32.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.677+0000 7ff6cb59e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560649dec930 0x560649decd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:32.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.677+0000 7ff6cb59e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560649dec930 0x560649decd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52838/0 (socket says 172.21.15.52:52838) 2023-11-27T15:19:32.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.677+0000 7ff6cb59e700 1 -- 172.21.15.52:0/2588925993 learned_addr learned my addr 172.21.15.52:0/2588925993 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:32.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.677+0000 7ff6cad9d700 1 --2- 172.21.15.52:0/2588925993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560649de7be0 0x560649de7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:32.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.677+0000 7ff6ca59c700 1 --2- 172.21.15.52:0/2588925993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560649de84f0 0x560649dec3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:32.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.677+0000 7ff6cb59e700 1 -- 172.21.15.52:0/2588925993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560649de84f0 msgr2=0x560649dec3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:32.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.677+0000 7ff6cb59e700 1 --2- 172.21.15.52:0/2588925993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560649de84f0 0x560649dec3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.677+0000 7ff6cb59e700 1 -- 172.21.15.52:0/2588925993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560649de7be0 msgr2=0x560649de7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:32.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.677+0000 7ff6cb59e700 1 --2- 172.21.15.52:0/2588925993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560649de7be0 0x560649de7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.677+0000 7ff6cb59e700 1 -- 172.21.15.52:0/2588925993 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560649bba6b0 con 0x560649dec930 2023-11-27T15:19:32.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.678+0000 7ff6cb59e700 1 --2- 172.21.15.52:0/2588925993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560649dec930 0x560649decd80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40090a0 tx=0x7ff6c400cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4c123c0555bee3 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:32.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.678+0000 7ff6b17fa700 1 -- 172.21.15.52:0/2588925993 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4099910 con 0x560649dec930 2023-11-27T15:19:32.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.678+0000 7ff6b17fa700 1 -- 172.21.15.52:0/2588925993 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6c409d940 con 0x560649dec930 2023-11-27T15:19:32.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.678+0000 7ff6b17fa700 1 -- 172.21.15.52:0/2588925993 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4099d70 con 0x560649dec930 2023-11-27T15:19:32.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.679+0000 7ff6dd394b80 1 -- 172.21.15.52:0/2588925993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560649dec930 msgr2=0x560649decd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:32.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.679+0000 7ff6dd394b80 1 --2- 172.21.15.52:0/2588925993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560649dec930 0x560649decd80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40090a0 tx=0x7ff6c400cfd0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.679+0000 7ff6dd394b80 1 -- 172.21.15.52:0/2588925993 shutdown_connections 2023-11-27T15:19:32.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.679+0000 7ff6dd394b80 1 --2- 172.21.15.52:0/2588925993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560649de7be0 0x560649de7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.679+0000 7ff6dd394b80 1 --2- 172.21.15.52:0/2588925993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560649dec930 0x560649decd80 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.679+0000 7ff6dd394b80 1 --2- 172.21.15.52:0/2588925993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560649de84f0 0x560649dec3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.679+0000 7ff6dd394b80 1 -- 172.21.15.52:0/2588925993 >> 172.21.15.52:0/2588925993 conn(0x560649ddd3e0 msgr2=0x560649de2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:32.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.679+0000 7ff6dd394b80 1 -- 172.21.15.52:0/2588925993 shutdown_connections 2023-11-27T15:19:32.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.679+0000 7ff6dd394b80 1 -- 172.21.15.52:0/2588925993 wait complete. 2023-11-27T15:19:32.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.679+0000 7ff6dd394b80 1 -- start start 2023-11-27T15:19:32.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.680+0000 7ff6dd394b80 1 Processor -- start 2023-11-27T15:19:32.694 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.680+0000 7ff6dd394b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560649de7be0 0x560649df40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:32.694 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.680+0000 7ff6dd394b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560649de84f0 0x560649df1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:32.694 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.680+0000 7ff6dd394b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560649dec930 0x560649df19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:32.694 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.680+0000 7ff6dd394b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x560649cda960 con 0x560649de84f0 2023-11-27T15:19:32.695 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.680+0000 7ff6dd394b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560649cda600 con 0x560649de7be0 2023-11-27T15:19:32.695 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.680+0000 7ff6dd394b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560649bbc9a0 con 0x560649dec930 2023-11-27T15:19:32.695 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.680+0000 7ff6dd394b80 10 client.0 _ll_register_callbacks cb 0x560649cf8800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:19:32.695 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.680+0000 7ff6ca59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560649de84f0 0x560649df1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:32.695 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.680+0000 7ff6cad9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560649de7be0 0x560649df40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:32.695 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.680+0000 7ff6ca59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560649de84f0 0x560649df1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52842/0 (socket says 172.21.15.52:52842) 2023-11-27T15:19:32.695 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.680+0000 7ff6ca59c700 1 -- 172.21.15.52:0/1649702943 learned_addr learned my addr 172.21.15.52:0/1649702943 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:32.695 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.680+0000 7ff6cb59e700 1 --2- 172.21.15.52:0/1649702943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560649dec930 0x560649df19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:32.696 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.681+0000 7ff6ca59c700 1 -- 172.21.15.52:0/1649702943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560649dec930 msgr2=0x560649df19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:32.696 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.681+0000 7ff6ca59c700 1 --2- 172.21.15.52:0/1649702943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560649dec930 0x560649df19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.696 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.681+0000 7ff6ca59c700 1 -- 172.21.15.52:0/1649702943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560649de7be0 msgr2=0x560649df40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:32.696 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.681+0000 7ff6ca59c700 1 --2- 172.21.15.52:0/1649702943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560649de7be0 0x560649df40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.696 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.681+0000 7ff6ca59c700 1 -- 172.21.15.52:0/1649702943 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6cc110220 con 0x560649de84f0 2023-11-27T15:19:32.696 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.681+0000 7ff6cad9d700 1 --2- 172.21.15.52:0/1649702943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560649de7be0 0x560649df40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:32.696 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.681+0000 7ff6ca59c700 1 --2- 172.21.15.52:0/1649702943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560649de84f0 0x560649df1320 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0012790 tx=0x7ff6c0012aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e8c652fcc78f35 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:32.696 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.681+0000 7ff6aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:19:32.697 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.681+0000 7ff6aeff5700 1 -- 172.21.15.52:0/1649702943 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0017070 con 0x560649de84f0 2023-11-27T15:19:32.697 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.681+0000 7ff6aeff5700 1 -- 172.21.15.52:0/1649702943 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff6c0004260 con 0x560649de84f0 2023-11-27T15:19:32.697 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.681+0000 7ff6dd394b80 1 -- 172.21.15.52:0/1649702943 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560649ddaba0 con 0x560649de84f0 2023-11-27T15:19:32.697 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.681+0000 7ff6aeff5700 1 -- 172.21.15.52:0/1649702943 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6c0004420 con 0x560649de84f0 2023-11-27T15:19:32.697 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.681+0000 7ff6dd394b80 10 client.15750 fetch_fsmap learned FSMap version 23 2023-11-27T15:19:32.697 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.681+0000 7ff6dd394b80 1 -- 172.21.15.52:0/1649702943 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560649cff210 con 0x560649de84f0 2023-11-27T15:19:32.697 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.682+0000 7ff6aeff5700 1 -- 172.21.15.52:0/1649702943 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560649ddaba0 con 0x560649de84f0 2023-11-27T15:19:32.697 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.682+0000 7ff6aeff5700 1 -- 172.21.15.52:0/1649702943 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff6c0011850 con 0x560649de84f0 2023-11-27T15:19:32.698 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.682+0000 7ff6dd394b80 10 client.15750 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:19:32.698 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.682+0000 7ff6dd394b80 10 client.15750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:19:32.698 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.682+0000 7ff6dd394b80 20 client.15750 populate_metadata read hostname 'smithi052' 2023-11-27T15:19:32.698 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.682+0000 7ff6dd394b80 1 --2- 172.21.15.52:0/1649702943 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560649e01730 0x560649e03b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:32.698 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.682+0000 7ff6dd394b80 4 client.15750 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:19:32.698 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.682+0000 7ff6dd394b80 1 -- 172.21.15.52:0/1649702943 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560649cfb120 con 0x560649e01730 2023-11-27T15:19:32.698 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.682+0000 7ff6cad9d700 1 --2- 172.21.15.52:0/1649702943 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560649e01730 0x560649e03b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:32.698 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.683+0000 7ff6cad9d700 1 --2- 172.21.15.52:0/1649702943 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560649e01730 0x560649e03b40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc367abb8890df12 server_cookie=2cbf0b92b226010b in_seq=0 out_seq=0 2023-11-27T15:19:32.699 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.683+0000 7ff6aeff5700 10 client.15750 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:32.699 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.683+0000 7ff6aeff5700 1 -- 172.21.15.52:0/1649702943 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560649cfb120 con 0x560649e01730 2023-11-27T15:19:32.699 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.683+0000 7ff6aeff5700 10 client.15750 handle_command_reply: tid=0 2023-11-27T15:19:32.699 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.683+0000 7ff6dd394b80 1 client.15750 shutdown 2023-11-27T15:19:32.699 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.684+0000 7ff6dd394b80 1 -- 172.21.15.52:0/1649702943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560649de84f0 msgr2=0x560649df1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:32.699 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.684+0000 7ff6dd394b80 1 --2- 172.21.15.52:0/1649702943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560649de84f0 0x560649df1320 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0012790 tx=0x7ff6c0012aa0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.699 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.684+0000 7ff6dd394b80 1 -- 172.21.15.52:0/1649702943 shutdown_connections 2023-11-27T15:19:32.699 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.684+0000 7ff6dd394b80 1 --2- 172.21.15.52:0/1649702943 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560649e01730 0x560649e03b40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.684+0000 7ff6dd394b80 1 --2- 172.21.15.52:0/1649702943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560649de7be0 0x560649df40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.684+0000 7ff6dd394b80 1 --2- 172.21.15.52:0/1649702943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560649dec930 0x560649df19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.684+0000 7ff6dd394b80 1 --2- 172.21.15.52:0/1649702943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560649de84f0 0x560649df1320 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.684+0000 7ff6aeff5700 0 client.15750 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:32.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.684+0000 7ff6dd394b80 1 -- 172.21.15.52:0/1649702943 >> 172.21.15.52:0/1649702943 conn(0x560649ddd3e0 msgr2=0x560649dedde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:32.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.684+0000 7ff6dd394b80 1 -- 172.21.15.52:0/1649702943 shutdown_connections 2023-11-27T15:19:32.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.684+0000 7ff6dd394b80 1 -- 172.21.15.52:0/1649702943 wait complete. 2023-11-27T15:19:32.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.684+0000 7ff6dd394b80 20 client.15750 trim_cache size 0 max 16384 2023-11-27T15:19:32.715 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.702+0000 7ff6dd394b80 1 Processor -- start 2023-11-27T15:19:32.715 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.702+0000 7ff6dd394b80 1 -- start start 2023-11-27T15:19:32.716 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.702+0000 7ff6dd394b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560649de7be0 0x560649deee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:32.716 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.702+0000 7ff6dd394b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560649de84f0 0x560649df8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:32.716 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.702+0000 7ff6dd394b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560649dec930 0x560649de70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:32.716 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.702+0000 7ff6dd394b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x560649df22c0 con 0x560649de7be0 2023-11-27T15:19:32.716 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.702+0000 7ff6dd394b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560649de8c70 con 0x560649de84f0 2023-11-27T15:19:32.716 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.702+0000 7ff6dd394b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560649deb530 con 0x560649dec930 2023-11-27T15:19:32.716 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.703+0000 7ff6cad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560649de7be0 0x560649deee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:32.716 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.703+0000 7ff6cad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560649de7be0 0x560649deee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52848/0 (socket says 172.21.15.52:52848) 2023-11-27T15:19:32.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.703+0000 7ff6cad9d700 1 -- 172.21.15.52:0/3183370232 learned_addr learned my addr 172.21.15.52:0/3183370232 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:32.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.703+0000 7ff6cb59e700 1 --2- 172.21.15.52:0/3183370232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560649dec930 0x560649de70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:32.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.703+0000 7ff6ca59c700 1 --2- 172.21.15.52:0/3183370232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560649de84f0 0x560649df8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:32.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.703+0000 7ff6cad9d700 1 -- 172.21.15.52:0/3183370232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560649dec930 msgr2=0x560649de70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:32.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.703+0000 7ff6cad9d700 1 --2- 172.21.15.52:0/3183370232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560649dec930 0x560649de70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.703+0000 7ff6cad9d700 1 -- 172.21.15.52:0/3183370232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560649de84f0 msgr2=0x560649df8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:32.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.703+0000 7ff6cad9d700 1 --2- 172.21.15.52:0/3183370232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560649de84f0 0x560649df8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.703+0000 7ff6cad9d700 1 -- 172.21.15.52:0/3183370232 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560649dfe200 con 0x560649de7be0 2023-11-27T15:19:32.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.703+0000 7ff6cb59e700 1 --2- 172.21.15.52:0/3183370232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560649dec930 0x560649de70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:19:32.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.703+0000 7ff6ca59c700 1 --2- 172.21.15.52:0/3183370232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560649de84f0 0x560649df8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:19:32.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.703+0000 7ff6cad9d700 1 --2- 172.21.15.52:0/3183370232 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560649de7be0 0x560649deee80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc015000 tx=0x7ff6bc022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd2892cbb0fc961 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:32.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.703+0000 7ff6b1ffb700 1 -- 172.21.15.52:0/3183370232 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc02c040 con 0x560649de7be0 2023-11-27T15:19:32.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.703+0000 7ff6b1ffb700 1 -- 172.21.15.52:0/3183370232 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6bc02d040 con 0x560649de7be0 2023-11-27T15:19:32.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.703+0000 7ff6b1ffb700 1 -- 172.21.15.52:0/3183370232 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc01d040 con 0x560649de7be0 2023-11-27T15:19:32.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.704+0000 7ff6dd394b80 1 -- 172.21.15.52:0/3183370232 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560649de7be0 msgr2=0x560649deee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:32.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.704+0000 7ff6dd394b80 1 --2- 172.21.15.52:0/3183370232 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560649de7be0 0x560649deee80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc015000 tx=0x7ff6bc022000 comp rx=0 tx=0).stop 2023-11-27T15:19:32.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.704+0000 7ff6dd394b80 1 -- 172.21.15.52:0/3183370232 shutdown_connections 2023-11-27T15:19:32.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.704+0000 7ff6dd394b80 1 --2- 172.21.15.52:0/3183370232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560649dec930 0x560649de70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.704+0000 7ff6dd394b80 1 --2- 172.21.15.52:0/3183370232 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560649de7be0 0x560649deee80 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.704+0000 7ff6dd394b80 1 --2- 172.21.15.52:0/3183370232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560649de84f0 0x560649df8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.704+0000 7ff6dd394b80 1 -- 172.21.15.52:0/3183370232 >> 172.21.15.52:0/3183370232 conn(0x560649ddd3e0 msgr2=0x560649de69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:32.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.704+0000 7ff6dd394b80 1 -- 172.21.15.52:0/3183370232 shutdown_connections 2023-11-27T15:19:32.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.704+0000 7ff6dd394b80 1 -- 172.21.15.52:0/3183370232 wait complete. 2023-11-27T15:19:32.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.705+0000 7ff6dd394b80 1 -- start start 2023-11-27T15:19:32.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.705+0000 7ff6dd394b80 1 Processor -- start 2023-11-27T15:19:32.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.705+0000 7ff6dd394b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560649de7be0 0x560649e4fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:32.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.705+0000 7ff6dd394b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560649de84f0 0x560649e4ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:32.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.705+0000 7ff6dd394b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560649dec930 0x560649df0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:32.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.705+0000 7ff6dd394b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x560649df0ef0 con 0x560649de7be0 2023-11-27T15:19:32.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.705+0000 7ff6dd394b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560649df1060 con 0x560649de84f0 2023-11-27T15:19:32.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.705+0000 7ff6dd394b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560649df1220 con 0x560649dec930 2023-11-27T15:19:32.721 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.705+0000 7ff6dd394b80 10 client.0 _ll_register_callbacks cb 0x560649cf8800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:19:32.721 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.705+0000 7ff6cad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560649de7be0 0x560649e4fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:32.721 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.705+0000 7ff6ca59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560649de84f0 0x560649e4ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:32.721 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.705+0000 7ff6cad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560649de7be0 0x560649e4fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52854/0 (socket says 172.21.15.52:52854) 2023-11-27T15:19:32.721 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.705+0000 7ff6cad9d700 1 -- 172.21.15.52:0/3327787291 learned_addr learned my addr 172.21.15.52:0/3327787291 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:32.721 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.705+0000 7ff6cb59e700 1 --2- 172.21.15.52:0/3327787291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560649dec930 0x560649df0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:32.721 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.706+0000 7ff6ca59c700 1 -- 172.21.15.52:0/3327787291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560649dec930 msgr2=0x560649df0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:32.721 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.706+0000 7ff6ca59c700 1 --2- 172.21.15.52:0/3327787291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560649dec930 0x560649df0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.721 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.706+0000 7ff6ca59c700 1 -- 172.21.15.52:0/3327787291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560649de7be0 msgr2=0x560649e4fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:32.722 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.706+0000 7ff6ca59c700 1 --2- 172.21.15.52:0/3327787291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560649de7be0 0x560649e4fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.722 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.706+0000 7ff6ca59c700 1 -- 172.21.15.52:0/3327787291 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6cc10da00 con 0x560649de84f0 2023-11-27T15:19:32.722 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.706+0000 7ff6cad9d700 1 --2- 172.21.15.52:0/3327787291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560649de7be0 0x560649e4fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:32.722 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.706+0000 7ff6ca59c700 1 --2- 172.21.15.52:0/3327787291 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560649de84f0 0x560649e4ff60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0011c60 tx=0x7ff6c0011730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e22754f57fefea3 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:32.722 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.706+0000 7ff6aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2023-11-27T15:19:32.722 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.706+0000 7ff6aeff5700 1 -- 172.21.15.52:0/3327787291 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0012ea0 con 0x560649de84f0 2023-11-27T15:19:32.722 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.706+0000 7ff6aeff5700 1 -- 172.21.15.52:0/3327787291 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff6c0003680 con 0x560649de84f0 2023-11-27T15:19:32.722 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.706+0000 7ff6dd394b80 1 -- 172.21.15.52:0/3327787291 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff6c409d940 con 0x560649de84f0 2023-11-27T15:19:32.723 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.706+0000 7ff6aeff5700 1 -- 172.21.15.52:0/3327787291 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6c002a070 con 0x560649de84f0 2023-11-27T15:19:32.723 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.706+0000 7ff6dd394b80 10 client.25571 fetch_fsmap learned FSMap version 23 2023-11-27T15:19:32.723 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.706+0000 7ff6dd394b80 1 -- 172.21.15.52:0/3327787291 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560649cfef60 con 0x560649de84f0 2023-11-27T15:19:32.723 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.707+0000 7ff6aeff5700 1 -- 172.21.15.52:0/3327787291 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6c409d940 con 0x560649de84f0 2023-11-27T15:19:32.723 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.707+0000 7ff6aeff5700 1 -- 172.21.15.52:0/3327787291 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff6c0025670 con 0x560649de84f0 2023-11-27T15:19:32.723 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.707+0000 7ff6dd394b80 10 client.25571 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:19:32.723 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.707+0000 7ff6dd394b80 10 client.25571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:19:32.723 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.707+0000 7ff6dd394b80 20 client.25571 populate_metadata read hostname 'smithi052' 2023-11-27T15:19:32.723 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.707+0000 7ff6dd394b80 1 --2- 172.21.15.52:0/3327787291 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560649e577e0 0x560649e59bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:32.724 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.707+0000 7ff6dd394b80 4 client.25571 mds_command: new command op to 24536 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-27T15:19:32.724 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.707+0000 7ff6dd394b80 1 -- 172.21.15.52:0/3327787291 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560649d076e0 con 0x560649e577e0 2023-11-27T15:19:32.724 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.707+0000 7ff6cad9d700 1 --2- 172.21.15.52:0/3327787291 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560649e577e0 0x560649e59bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:32.724 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.707+0000 7ff6aeff5700 10 client.25571 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:32.724 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.707+0000 7ff6cad9d700 1 --2- 172.21.15.52:0/3327787291 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560649e577e0 0x560649e59bc0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15ee078824a91d75 server_cookie=41f7e8436d84b6a1 in_seq=0 out_seq=0 2023-11-27T15:19:32.724 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.709+0000 7ff6aeff5700 1 -- 172.21.15.52:0/3327787291 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560649d076e0 con 0x560649e577e0 2023-11-27T15:19:32.724 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.709+0000 7ff6aeff5700 10 client.25571 handle_command_reply: tid=0 2023-11-27T15:19:32.724 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.709+0000 7ff6dd394b80 1 client.25571 shutdown 2023-11-27T15:19:32.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.709+0000 7ff6dd394b80 1 -- 172.21.15.52:0/3327787291 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560649de84f0 msgr2=0x560649e4ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:32.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.709+0000 7ff6dd394b80 1 --2- 172.21.15.52:0/3327787291 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560649de84f0 0x560649e4ff60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0011c60 tx=0x7ff6c0011730 comp rx=0 tx=0).stop 2023-11-27T15:19:32.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.709+0000 7ff6dd394b80 1 -- 172.21.15.52:0/3327787291 shutdown_connections 2023-11-27T15:19:32.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.709+0000 7ff6dd394b80 1 --2- 172.21.15.52:0/3327787291 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560649e577e0 0x560649e59bc0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.709+0000 7ff6dd394b80 1 --2- 172.21.15.52:0/3327787291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560649dec930 0x560649df0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.709+0000 7ff6dd394b80 1 --2- 172.21.15.52:0/3327787291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560649de7be0 0x560649e4fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.709+0000 7ff6dd394b80 1 --2- 172.21.15.52:0/3327787291 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560649de84f0 0x560649e4ff60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.709+0000 7ff6aeff5700 0 client.25571 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:32.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.709+0000 7ff6dd394b80 1 -- 172.21.15.52:0/3327787291 >> 172.21.15.52:0/3327787291 conn(0x560649ddd3e0 msgr2=0x560649de7270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:32.726 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.710+0000 7ff6dd394b80 1 -- 172.21.15.52:0/3327787291 shutdown_connections 2023-11-27T15:19:32.726 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.710+0000 7ff6dd394b80 1 -- 172.21.15.52:0/3327787291 wait complete. 2023-11-27T15:19:32.726 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.710+0000 7ff6dd394b80 20 client.25571 trim_cache size 0 max 16384 2023-11-27T15:19:32.726 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:19:32.726 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2023-11-27T15:19:32.727 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "8cb20f72-eb2c-477d-a493-a7ded872a482", 2023-11-27T15:19:32.727 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2023-11-27T15:19:32.727 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:19:32.728 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.710+0000 7ff6d1208700 1 -- 172.21.15.52:0/3098467062 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ff6b40a12c0 msgr2=0x7ff6b40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:32.728 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.710+0000 7ff6d1208700 1 --2- 172.21.15.52:0/3098467062 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ff6b40a12c0 0x7ff6b40a3770 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc009990 tx=0x7ff6bc01a040 comp rx=0 tx=0).stop 2023-11-27T15:19:32.728 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.710+0000 7ff6d1208700 1 -- 172.21.15.52:0/3098467062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6cc1055f0 msgr2=0x7ff6cc1a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:32.728 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.710+0000 7ff6d1208700 1 --2- 172.21.15.52:0/3098467062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6cc1055f0 0x7ff6cc1a45f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff6c400efc0 tx=0x7ff6c400c3a0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.728 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.710+0000 7ff6d1208700 1 -- 172.21.15.52:0/3098467062 shutdown_connections 2023-11-27T15:19:32.728 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.710+0000 7ff6d1208700 1 --2- 172.21.15.52:0/3098467062 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ff6b40a12c0 0x7ff6b40a3770 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.729 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.710+0000 7ff6d1208700 1 --2- 172.21.15.52:0/3098467062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6cc1005b0 0x7ff6cc072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.729 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.710+0000 7ff6d1208700 1 --2- 172.21.15.52:0/3098467062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6cc1055f0 0x7ff6cc1a45f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.729 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.710+0000 7ff6d1208700 1 --2- 172.21.15.52:0/3098467062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6cc100f50 0x7ff6cc073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:32.729 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.710+0000 7ff6d1208700 1 -- 172.21.15.52:0/3098467062 >> 172.21.15.52:0/3098467062 conn(0x7ff6cc077140 msgr2=0x7ff6cc0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:32.729 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.710+0000 7ff6d1208700 1 -- 172.21.15.52:0/3098467062 shutdown_connections 2023-11-27T15:19:32.729 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:32.710+0000 7ff6d1208700 1 -- 172.21.15.52:0/3098467062 wait complete. 2023-11-27T15:19:32.777 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:19:33.053 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:33.053 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:33.114 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:33.115 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:33.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:32 smithi052 ceph-mon[99056]: pgmap v51: 97 pgs: 97 active+clean; 2.2 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-11-27T15:19:33.128 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:33.128 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:33.128 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 60 -lt 100 ']' 2023-11-27T15:19:33.128 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 60 + 1 2023-11-27T15:19:33.151 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=61 2023-11-27T15:19:33.151 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:33.169 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:33.169 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:33.170 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:33.170 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:33.170 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:33.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:32 smithi150 ceph-mon[100559]: pgmap v51: 97 pgs: 97 active+clean; 2.2 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-11-27T15:19:33.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:32 smithi102 ceph-mon[100296]: pgmap v51: 97 pgs: 97 active+clean; 2.2 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-11-27T15:19:33.354 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:33.354 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:33.388 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:33.388 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:33.394 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:33.394 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:33.394 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 61 -lt 100 ']' 2023-11-27T15:19:33.394 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 61 + 1 2023-11-27T15:19:33.425 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=62 2023-11-27T15:19:33.425 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:33.441 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:33.441 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:33.441 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:33.442 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:33.442 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:33.928 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:33.928 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:34.079 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:34.079 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:34.092 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:34.093 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:34.093 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 62 -lt 100 ']' 2023-11-27T15:19:34.093 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 62 + 1 2023-11-27T15:19:34.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:33 smithi052 ceph-mon[99056]: scrub queued for path: / 2023-11-27T15:19:34.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:33 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:19:34.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:33 smithi052 ceph-mon[99056]: scrub summary: active paths [/] 2023-11-27T15:19:34.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:33 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:19:34.117 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=63 2023-11-27T15:19:34.118 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:34.158 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:34.158 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:34.159 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:34.160 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:34.160 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:34.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:33 smithi150 ceph-mon[100559]: scrub queued for path: / 2023-11-27T15:19:34.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:33 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:19:34.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:33 smithi150 ceph-mon[100559]: scrub summary: active paths [/] 2023-11-27T15:19:34.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:33 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:19:34.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:33 smithi102 ceph-mon[100296]: scrub queued for path: / 2023-11-27T15:19:34.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:33 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:19:34.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:33 smithi102 ceph-mon[100296]: scrub summary: active paths [/] 2023-11-27T15:19:34.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:33 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:19:34.911 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:34.912 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:35.087 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:35.088 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:35.101 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:35.102 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:35.102 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 63 -lt 100 ']' 2023-11-27T15:19:35.102 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 63 + 1 2023-11-27T15:19:35.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:34 smithi052 ceph-mon[99056]: pgmap v52: 97 pgs: 97 active+clean; 2.2 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-11-27T15:19:35.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:34 smithi052 ceph-mon[99056]: scrub summary: idle 2023-11-27T15:19:35.142 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=64 2023-11-27T15:19:35.142 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:35.177 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:35.177 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:35.178 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:35.179 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:35.179 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:35.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:34 smithi150 ceph-mon[100559]: pgmap v52: 97 pgs: 97 active+clean; 2.2 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-11-27T15:19:35.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:34 smithi150 ceph-mon[100559]: scrub summary: idle 2023-11-27T15:19:35.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:34 smithi102 ceph-mon[100296]: pgmap v52: 97 pgs: 97 active+clean; 2.2 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-11-27T15:19:35.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:34 smithi102 ceph-mon[100296]: scrub summary: idle 2023-11-27T15:19:35.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.519+0000 7fb70dfe7700 1 -- 172.21.15.52:0/1518177259 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb708104490 msgr2=0x7fb70810c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:35.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.519+0000 7fb70dfe7700 1 --2- 172.21.15.52:0/1518177259 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb708104490 0x7fb70810c060 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb70000b0d0 tx=0x7fb70000b3e0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.520+0000 7fb70dfe7700 1 -- 172.21.15.52:0/1518177259 shutdown_connections 2023-11-27T15:19:35.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.520+0000 7fb70dfe7700 1 --2- 172.21.15.52:0/1518177259 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb708104490 0x7fb70810c060 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.520+0000 7fb70dfe7700 1 --2- 172.21.15.52:0/1518177259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7080ff450 0x7fb7080ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.520+0000 7fb70dfe7700 1 --2- 172.21.15.52:0/1518177259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb7080ffdf0 0x7fb708103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.520+0000 7fb70dfe7700 1 -- 172.21.15.52:0/1518177259 >> 172.21.15.52:0/1518177259 conn(0x7fb7080770b0 msgr2=0x7fb7080774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:35.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.520+0000 7fb70dfe7700 1 -- 172.21.15.52:0/1518177259 shutdown_connections 2023-11-27T15:19:35.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.520+0000 7fb70dfe7700 1 -- 172.21.15.52:0/1518177259 wait complete. 2023-11-27T15:19:35.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.520+0000 7fb70dfe7700 1 Processor -- start 2023-11-27T15:19:35.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.521+0000 7fb70dfe7700 1 -- start start 2023-11-27T15:19:35.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.521+0000 7fb70dfe7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7080ff450 0x7fb708100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:35.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.521+0000 7fb70dfe7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7080ffdf0 0x7fb708101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:35.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.521+0000 7fb70dfe7700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb708104490 0x7fb7081a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:35.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.521+0000 7fb70dfe7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb708101bc0 con 0x7fb7080ff450 2023-11-27T15:19:35.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.521+0000 7fb70dfe7700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7081a4c60 con 0x7fb708104490 2023-11-27T15:19:35.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.521+0000 7fb70dfe7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7081a4da0 con 0x7fb7080ffdf0 2023-11-27T15:19:35.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.521+0000 7fb7077fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7080ff450 0x7fb708100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:35.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.521+0000 7fb7077fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7080ff450 0x7fb708100e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52868/0 (socket says 172.21.15.52:52868) 2023-11-27T15:19:35.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.521+0000 7fb7077fe700 1 -- 172.21.15.52:0/2438294837 learned_addr learned my addr 172.21.15.52:0/2438294837 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:35.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.521+0000 7fb707fff700 1 --2- 172.21.15.52:0/2438294837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb708104490 0x7fb7081a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:35.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.521+0000 7fb706ffd700 1 --2- 172.21.15.52:0/2438294837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7080ffdf0 0x7fb708101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:35.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.522+0000 7fb7077fe700 1 -- 172.21.15.52:0/2438294837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7080ffdf0 msgr2=0x7fb708101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:35.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.522+0000 7fb7077fe700 1 --2- 172.21.15.52:0/2438294837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7080ffdf0 0x7fb708101390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.522+0000 7fb7077fe700 1 -- 172.21.15.52:0/2438294837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb708104490 msgr2=0x7fb7081a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:35.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.522+0000 7fb7077fe700 1 --2- 172.21.15.52:0/2438294837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb708104490 0x7fb7081a4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.522+0000 7fb7077fe700 1 -- 172.21.15.52:0/2438294837 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb700009d00 con 0x7fb7080ff450 2023-11-27T15:19:35.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.522+0000 7fb7077fe700 1 --2- 172.21.15.52:0/2438294837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7080ff450 0x7fb708100e50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb6f800e9a0 tx=0x7fb6f800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:35.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.522+0000 7fb704ff9700 1 -- 172.21.15.52:0/2438294837 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8009790 con 0x7fb7080ff450 2023-11-27T15:19:35.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.522+0000 7fb704ff9700 1 -- 172.21.15.52:0/2438294837 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6f8010430 con 0x7fb7080ff450 2023-11-27T15:19:35.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.522+0000 7fb70dfe7700 1 -- 172.21.15.52:0/2438294837 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb70810f2b0 con 0x7fb7080ff450 2023-11-27T15:19:35.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.522+0000 7fb704ff9700 1 -- 172.21.15.52:0/2438294837 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8018680 con 0x7fb7080ff450 2023-11-27T15:19:35.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.522+0000 7fb70dfe7700 1 -- 172.21.15.52:0/2438294837 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb70810f120 con 0x7fb7080ff450 2023-11-27T15:19:35.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.525+0000 7fb704ff9700 1 -- 172.21.15.52:0/2438294837 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb6f80105a0 con 0x7fb7080ff450 2023-11-27T15:19:35.533 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.526+0000 7fb704ff9700 1 --2- 172.21.15.52:0/2438294837 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb6f00a12c0 0x7fb6f00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:35.533 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.526+0000 7fb706ffd700 1 --2- 172.21.15.52:0/2438294837 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb6f00a12c0 0x7fb6f00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:35.533 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.526+0000 7fb704ff9700 1 -- 172.21.15.52:0/2438294837 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb6f8014070 con 0x7fb7080ff450 2023-11-27T15:19:35.533 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.526+0000 7fb706ffd700 1 --2- 172.21.15.52:0/2438294837 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb6f00a12c0 0x7fb6f00a3770 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb708102640 tx=0x7fb6fc009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:35.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.585+0000 7fb71a173b80 1 Processor -- start 2023-11-27T15:19:35.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.585+0000 7fb71a173b80 1 -- start start 2023-11-27T15:19:35.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.585+0000 7fb71a173b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eebd9e6b60 0x55eebd9e6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:35.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.585+0000 7fb71a173b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eebd9e7470 0x55eebd9eb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:35.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.585+0000 7fb71a173b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eebd9eb8b0 0x55eebd9ebd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:35.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.585+0000 7fb71a173b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eebd8ec060 con 0x55eebd9e7470 2023-11-27T15:19:35.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.586+0000 7fb71a173b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eebd93e8a0 con 0x55eebd9e6b60 2023-11-27T15:19:35.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.586+0000 7fb71a173b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eebd937d30 con 0x55eebd9eb8b0 2023-11-27T15:19:35.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.586+0000 7fb706ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eebd9e7470 0x55eebd9eb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:35.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.586+0000 7fb706ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eebd9e7470 0x55eebd9eb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52878/0 (socket says 172.21.15.52:52878) 2023-11-27T15:19:35.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.586+0000 7fb706ffd700 1 -- 172.21.15.52:0/2539702264 learned_addr learned my addr 172.21.15.52:0/2539702264 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:35.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.586+0000 7fb707fff700 1 --2- 172.21.15.52:0/2539702264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eebd9eb8b0 0x55eebd9ebd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:35.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.586+0000 7fb7077fe700 1 --2- 172.21.15.52:0/2539702264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eebd9e6b60 0x55eebd9e6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:35.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.586+0000 7fb706ffd700 1 -- 172.21.15.52:0/2539702264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eebd9eb8b0 msgr2=0x55eebd9ebd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:35.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.586+0000 7fb706ffd700 1 --2- 172.21.15.52:0/2539702264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eebd9eb8b0 0x55eebd9ebd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.586+0000 7fb706ffd700 1 -- 172.21.15.52:0/2539702264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eebd9e6b60 msgr2=0x55eebd9e6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:35.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.586+0000 7fb706ffd700 1 --2- 172.21.15.52:0/2539702264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eebd9e6b60 0x55eebd9e6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.586+0000 7fb706ffd700 1 -- 172.21.15.52:0/2539702264 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eebd7b9630 con 0x55eebd9e7470 2023-11-27T15:19:35.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.586+0000 7fb707fff700 1 --2- 172.21.15.52:0/2539702264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eebd9eb8b0 0x55eebd9ebd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:35.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.586+0000 7fb706ffd700 1 --2- 172.21.15.52:0/2539702264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eebd9e7470 0x55eebd9eb370 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc011cd0 tx=0x7fb6fc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74d14006d86d2db server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:35.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.587+0000 7fb6edffb700 1 -- 172.21.15.52:0/2539702264 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc004030 con 0x55eebd9e7470 2023-11-27T15:19:35.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.587+0000 7fb6edffb700 1 -- 172.21.15.52:0/2539702264 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6fc00e840 con 0x55eebd9e7470 2023-11-27T15:19:35.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.587+0000 7fb6edffb700 1 -- 172.21.15.52:0/2539702264 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc0039f0 con 0x55eebd9e7470 2023-11-27T15:19:35.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.587+0000 7fb71a173b80 1 -- 172.21.15.52:0/2539702264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eebd9e7470 msgr2=0x55eebd9eb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:35.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.587+0000 7fb71a173b80 1 --2- 172.21.15.52:0/2539702264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eebd9e7470 0x55eebd9eb370 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc011cd0 tx=0x7fb6fc014000 comp rx=0 tx=0).stop 2023-11-27T15:19:35.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.588+0000 7fb71a173b80 1 -- 172.21.15.52:0/2539702264 shutdown_connections 2023-11-27T15:19:35.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.588+0000 7fb71a173b80 1 --2- 172.21.15.52:0/2539702264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eebd9e6b60 0x55eebd9e6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.601 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.588+0000 7fb71a173b80 1 --2- 172.21.15.52:0/2539702264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eebd9eb8b0 0x55eebd9ebd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.601 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.588+0000 7fb71a173b80 1 --2- 172.21.15.52:0/2539702264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eebd9e7470 0x55eebd9eb370 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.601 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.588+0000 7fb71a173b80 1 -- 172.21.15.52:0/2539702264 >> 172.21.15.52:0/2539702264 conn(0x55eebd9dc360 msgr2=0x55eebd9e1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:35.601 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.588+0000 7fb71a173b80 1 -- 172.21.15.52:0/2539702264 shutdown_connections 2023-11-27T15:19:35.601 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.588+0000 7fb71a173b80 1 -- 172.21.15.52:0/2539702264 wait complete. 2023-11-27T15:19:35.601 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.588+0000 7fb71a173b80 1 -- start start 2023-11-27T15:19:35.601 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.588+0000 7fb71a173b80 1 Processor -- start 2023-11-27T15:19:35.601 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.589+0000 7fb71a173b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eebd9e6b60 0x55eebd9f3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:35.602 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.589+0000 7fb71a173b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eebd9e7470 0x55eebd9f02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:35.602 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.589+0000 7fb71a173b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eebd9eb8b0 0x55eebd9f0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:35.602 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.589+0000 7fb71a173b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eebd8d98e0 con 0x55eebd9e7470 2023-11-27T15:19:35.602 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.589+0000 7fb71a173b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eebd8d9580 con 0x55eebd9e6b60 2023-11-27T15:19:35.602 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.589+0000 7fb71a173b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eebd7bb920 con 0x55eebd9eb8b0 2023-11-27T15:19:35.602 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.589+0000 7fb71a173b80 10 client.0 _ll_register_callbacks cb 0x55eebd8f7780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:19:35.602 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.589+0000 7fb706ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eebd9e7470 0x55eebd9f02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:35.602 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.589+0000 7fb706ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eebd9e7470 0x55eebd9f02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52884/0 (socket says 172.21.15.52:52884) 2023-11-27T15:19:35.603 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.589+0000 7fb706ffd700 1 -- 172.21.15.52:0/2533897745 learned_addr learned my addr 172.21.15.52:0/2533897745 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:35.603 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.589+0000 7fb707fff700 1 --2- 172.21.15.52:0/2533897745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eebd9eb8b0 0x55eebd9f0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:35.603 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.589+0000 7fb7077fe700 1 --2- 172.21.15.52:0/2533897745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eebd9e6b60 0x55eebd9f3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:35.603 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.589+0000 7fb706ffd700 1 -- 172.21.15.52:0/2533897745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eebd9eb8b0 msgr2=0x55eebd9f0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:35.603 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.589+0000 7fb706ffd700 1 --2- 172.21.15.52:0/2533897745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eebd9eb8b0 0x55eebd9f0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.603 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.589+0000 7fb706ffd700 1 -- 172.21.15.52:0/2533897745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eebd9e6b60 msgr2=0x55eebd9f3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:35.603 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.589+0000 7fb706ffd700 1 --2- 172.21.15.52:0/2533897745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eebd9e6b60 0x55eebd9f3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.603 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.589+0000 7fb706ffd700 1 -- 172.21.15.52:0/2533897745 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6fc017070 con 0x55eebd9e7470 2023-11-27T15:19:35.603 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.589+0000 7fb707fff700 1 --2- 172.21.15.52:0/2533897745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eebd9eb8b0 0x55eebd9f0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:19:35.604 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.590+0000 7fb706ffd700 1 --2- 172.21.15.52:0/2533897745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eebd9e7470 0x55eebd9f02e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc0099d0 tx=0x7fb6fc0052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=758125cc8dae5363 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:35.604 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.590+0000 7fb6e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:19:35.604 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.590+0000 7fb6e6ffd700 1 -- 172.21.15.52:0/2533897745 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc02b070 con 0x55eebd9e7470 2023-11-27T15:19:35.604 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.590+0000 7fb6e6ffd700 1 -- 172.21.15.52:0/2533897745 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb6fc004710 con 0x55eebd9e7470 2023-11-27T15:19:35.604 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.590+0000 7fb71a173b80 1 -- 172.21.15.52:0/2533897745 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eebd9d9b20 con 0x55eebd9e7470 2023-11-27T15:19:35.604 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.590+0000 7fb6e6ffd700 1 -- 172.21.15.52:0/2533897745 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6fc0048d0 con 0x55eebd9e7470 2023-11-27T15:19:35.604 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.590+0000 7fb71a173b80 10 client.15768 fetch_fsmap learned FSMap version 23 2023-11-27T15:19:35.604 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.590+0000 7fb71a173b80 1 -- 172.21.15.52:0/2533897745 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55eebd8fe190 con 0x55eebd9e7470 2023-11-27T15:19:35.604 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.590+0000 7fb6e6ffd700 1 -- 172.21.15.52:0/2533897745 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6fc03c080 con 0x55eebd9e7470 2023-11-27T15:19:35.605 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.590+0000 7fb6e6ffd700 1 -- 172.21.15.52:0/2533897745 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb6fc036030 con 0x55eebd9e7470 2023-11-27T15:19:35.605 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.590+0000 7fb71a173b80 10 client.15768 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:19:35.605 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.590+0000 7fb71a173b80 10 client.15768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:19:35.605 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.591+0000 7fb71a173b80 20 client.15768 populate_metadata read hostname 'smithi052' 2023-11-27T15:19:35.605 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.591+0000 7fb71a173b80 1 --2- 172.21.15.52:0/2533897745 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55eebda00810 0x55eebda02c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:35.605 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.591+0000 7fb71a173b80 4 client.15768 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:19:35.605 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.591+0000 7fb71a173b80 1 -- 172.21.15.52:0/2533897745 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eebd8fa0a0 con 0x55eebda00810 2023-11-27T15:19:35.605 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.591+0000 7fb7077fe700 1 --2- 172.21.15.52:0/2533897745 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55eebda00810 0x55eebda02c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:35.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.591+0000 7fb6e6ffd700 10 client.15768 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:35.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.591+0000 7fb7077fe700 1 --2- 172.21.15.52:0/2533897745 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55eebda00810 0x55eebda02c20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad367b8772f8ca88 server_cookie=165dd6337bbc9dfb in_seq=0 out_seq=0 2023-11-27T15:19:35.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.592+0000 7fb6e6ffd700 1 -- 172.21.15.52:0/2533897745 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55eebd8fa0a0 con 0x55eebda00810 2023-11-27T15:19:35.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.592+0000 7fb6e6ffd700 10 client.15768 handle_command_reply: tid=0 2023-11-27T15:19:35.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.592+0000 7fb71a173b80 1 client.15768 shutdown 2023-11-27T15:19:35.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.592+0000 7fb71a173b80 1 -- 172.21.15.52:0/2533897745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eebd9e7470 msgr2=0x55eebd9f02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:35.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.592+0000 7fb71a173b80 1 --2- 172.21.15.52:0/2533897745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eebd9e7470 0x55eebd9f02e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc0099d0 tx=0x7fb6fc0052e0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.592+0000 7fb71a173b80 1 -- 172.21.15.52:0/2533897745 shutdown_connections 2023-11-27T15:19:35.607 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.592+0000 7fb71a173b80 1 --2- 172.21.15.52:0/2533897745 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55eebda00810 0x55eebda02c20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.607 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.592+0000 7fb71a173b80 1 --2- 172.21.15.52:0/2533897745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eebd9e6b60 0x55eebd9f3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.607 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.592+0000 7fb71a173b80 1 --2- 172.21.15.52:0/2533897745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eebd9eb8b0 0x55eebd9f0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.607 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.592+0000 7fb71a173b80 1 --2- 172.21.15.52:0/2533897745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eebd9e7470 0x55eebd9f02e0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.607 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.592+0000 7fb71a173b80 1 -- 172.21.15.52:0/2533897745 >> 172.21.15.52:0/2533897745 conn(0x55eebd9dc360 msgr2=0x55eebd9ecd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:35.607 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.592+0000 7fb6e6ffd700 0 client.15768 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:35.607 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.592+0000 7fb71a173b80 1 -- 172.21.15.52:0/2533897745 shutdown_connections 2023-11-27T15:19:35.607 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.592+0000 7fb71a173b80 1 -- 172.21.15.52:0/2533897745 wait complete. 2023-11-27T15:19:35.607 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.592+0000 7fb71a173b80 20 client.15768 trim_cache size 0 max 16384 2023-11-27T15:19:35.622 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.610+0000 7fb71a173b80 1 Processor -- start 2023-11-27T15:19:35.622 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.611+0000 7fb71a173b80 1 -- start start 2023-11-27T15:19:35.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.611+0000 7fb71a173b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eebd9e6b60 0x55eebd9fc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:35.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.611+0000 7fb71a173b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eebd9e7470 0x55eebda02820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:35.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.611+0000 7fb71a173b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eebd9eb8b0 0x55eebd9e0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:35.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.611+0000 7fb71a173b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eebd9ea230 con 0x55eebd9e7470 2023-11-27T15:19:35.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.611+0000 7fb71a173b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eebd9e7bc0 con 0x55eebd9eb8b0 2023-11-27T15:19:35.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.611+0000 7fb71a173b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eebd9f3730 con 0x55eebd9e6b60 2023-11-27T15:19:35.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.611+0000 7fb706ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eebd9e7470 0x55eebda02820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:35.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.611+0000 7fb706ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eebd9e7470 0x55eebda02820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52892/0 (socket says 172.21.15.52:52892) 2023-11-27T15:19:35.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.611+0000 7fb706ffd700 1 -- 172.21.15.52:0/1521136978 learned_addr learned my addr 172.21.15.52:0/1521136978 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:35.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.611+0000 7fb7077fe700 1 --2- 172.21.15.52:0/1521136978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eebd9e6b60 0x55eebd9fc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:35.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.611+0000 7fb707fff700 1 --2- 172.21.15.52:0/1521136978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eebd9eb8b0 0x55eebd9e0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:35.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.611+0000 7fb706ffd700 1 -- 172.21.15.52:0/1521136978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eebd9e6b60 msgr2=0x55eebd9fc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:35.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.611+0000 7fb706ffd700 1 --2- 172.21.15.52:0/1521136978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eebd9e6b60 0x55eebd9fc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.611+0000 7fb706ffd700 1 -- 172.21.15.52:0/1521136978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eebd9eb8b0 msgr2=0x55eebd9e0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:35.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.611+0000 7fb706ffd700 1 --2- 172.21.15.52:0/1521136978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eebd9eb8b0 0x55eebd9e0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.611+0000 7fb706ffd700 1 -- 172.21.15.52:0/1521136978 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eebd9ebf50 con 0x55eebd9e7470 2023-11-27T15:19:35.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.611+0000 7fb707fff700 1 --2- 172.21.15.52:0/1521136978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eebd9eb8b0 0x55eebd9e0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:19:35.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.612+0000 7fb706ffd700 1 --2- 172.21.15.52:0/1521136978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eebd9e7470 0x55eebda02820 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc016040 tx=0x7fb6fc005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c67d1ff367a8c46 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:35.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.612+0000 7fb6ee7fc700 1 -- 172.21.15.52:0/1521136978 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc02b070 con 0x55eebd9e7470 2023-11-27T15:19:35.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.612+0000 7fb6ee7fc700 1 -- 172.21.15.52:0/1521136978 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55eebd9d9b20 con 0x55eebd9e7470 2023-11-27T15:19:35.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.612+0000 7fb6ee7fc700 1 -- 172.21.15.52:0/1521136978 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc003b20 con 0x55eebd9e7470 2023-11-27T15:19:35.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.612+0000 7fb71a173b80 1 -- 172.21.15.52:0/1521136978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eebd9e7470 msgr2=0x55eebda02820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:35.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.612+0000 7fb71a173b80 1 --2- 172.21.15.52:0/1521136978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eebd9e7470 0x55eebda02820 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc016040 tx=0x7fb6fc005b40 comp rx=0 tx=0).stop 2023-11-27T15:19:35.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.612+0000 7fb71a173b80 1 -- 172.21.15.52:0/1521136978 shutdown_connections 2023-11-27T15:19:35.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.612+0000 7fb71a173b80 1 --2- 172.21.15.52:0/1521136978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eebd9eb8b0 0x55eebd9e0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.612+0000 7fb71a173b80 1 --2- 172.21.15.52:0/1521136978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eebd9e6b60 0x55eebd9fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.612+0000 7fb71a173b80 1 --2- 172.21.15.52:0/1521136978 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eebd9e7470 0x55eebda02820 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.612+0000 7fb71a173b80 1 -- 172.21.15.52:0/1521136978 >> 172.21.15.52:0/1521136978 conn(0x55eebd9dc360 msgr2=0x55eebd9fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:35.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.613+0000 7fb71a173b80 1 -- 172.21.15.52:0/1521136978 shutdown_connections 2023-11-27T15:19:35.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.613+0000 7fb71a173b80 1 -- 172.21.15.52:0/1521136978 wait complete. 2023-11-27T15:19:35.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.613+0000 7fb71a173b80 1 -- start start 2023-11-27T15:19:35.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.613+0000 7fb71a173b80 1 Processor -- start 2023-11-27T15:19:35.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.613+0000 7fb71a173b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eebd9e6b60 0x55eebd9f5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:35.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.613+0000 7fb71a173b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eebd9e7470 0x55eebd9f5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:35.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.613+0000 7fb71a173b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eebd9eb8b0 0x55eebd9f6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:35.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.614+0000 7fb71a173b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eebda536d0 con 0x55eebd9e7470 2023-11-27T15:19:35.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.614+0000 7fb71a173b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eebda53810 con 0x55eebd9eb8b0 2023-11-27T15:19:35.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.614+0000 7fb71a173b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eebda53950 con 0x55eebd9e6b60 2023-11-27T15:19:35.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.614+0000 7fb71a173b80 10 client.0 _ll_register_callbacks cb 0x7fb6fc005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:19:35.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.614+0000 7fb706ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eebd9e7470 0x55eebd9f5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:35.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.614+0000 7fb706ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eebd9e7470 0x55eebd9f5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52898/0 (socket says 172.21.15.52:52898) 2023-11-27T15:19:35.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.614+0000 7fb706ffd700 1 -- 172.21.15.52:0/2309042959 learned_addr learned my addr 172.21.15.52:0/2309042959 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:35.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.614+0000 7fb707fff700 1 --2- 172.21.15.52:0/2309042959 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eebd9eb8b0 0x55eebd9f6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:35.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.614+0000 7fb7077fe700 1 --2- 172.21.15.52:0/2309042959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eebd9e6b60 0x55eebd9f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:35.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.614+0000 7fb706ffd700 1 -- 172.21.15.52:0/2309042959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eebd9e6b60 msgr2=0x55eebd9f5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:35.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.614+0000 7fb706ffd700 1 --2- 172.21.15.52:0/2309042959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eebd9e6b60 0x55eebd9f5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.614+0000 7fb706ffd700 1 -- 172.21.15.52:0/2309042959 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eebd9eb8b0 msgr2=0x55eebd9f6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:35.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.614+0000 7fb706ffd700 1 --2- 172.21.15.52:0/2309042959 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eebd9eb8b0 0x55eebd9f6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.614+0000 7fb706ffd700 1 -- 172.21.15.52:0/2309042959 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eebd8fe190 con 0x55eebd9e7470 2023-11-27T15:19:35.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.614+0000 7fb706ffd700 1 --2- 172.21.15.52:0/2309042959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eebd9e7470 0x55eebd9f5ed0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc011b10 tx=0x7fb6fc009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d8a3b83a68db4e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:35.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.614+0000 7fb6e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:19:35.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.614+0000 7fb6e6ffd700 1 -- 172.21.15.52:0/2309042959 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc02b040 con 0x55eebd9e7470 2023-11-27T15:19:35.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.614+0000 7fb6e6ffd700 1 -- 172.21.15.52:0/2309042959 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb6fc003720 con 0x55eebd9e7470 2023-11-27T15:19:35.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.615+0000 7fb71a173b80 1 -- 172.21.15.52:0/2309042959 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb6fc00e840 con 0x55eebd9e7470 2023-11-27T15:19:35.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.615+0000 7fb6e6ffd700 1 -- 172.21.15.52:0/2309042959 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6fc0038e0 con 0x55eebd9e7470 2023-11-27T15:19:35.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.615+0000 7fb71a173b80 10 client.15774 fetch_fsmap learned FSMap version 23 2023-11-27T15:19:35.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.615+0000 7fb71a173b80 1 -- 172.21.15.52:0/2309042959 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55eebd765de0 con 0x55eebd9e7470 2023-11-27T15:19:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.615+0000 7fb6e6ffd700 1 -- 172.21.15.52:0/2309042959 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6fc00e840 con 0x55eebd9e7470 2023-11-27T15:19:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.615+0000 7fb6e6ffd700 1 -- 172.21.15.52:0/2309042959 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb6fc041070 con 0x55eebd9e7470 2023-11-27T15:19:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.615+0000 7fb71a173b80 10 client.15774 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:19:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.615+0000 7fb71a173b80 10 client.15774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:19:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.615+0000 7fb71a173b80 20 client.15774 populate_metadata read hostname 'smithi052' 2023-11-27T15:19:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.615+0000 7fb71a173b80 1 --2- 172.21.15.52:0/2309042959 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55eebda54b00 0x55eebda56ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.615+0000 7fb71a173b80 4 client.15774 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:19:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.615+0000 7fb71a173b80 1 -- 172.21.15.52:0/2309042959 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55eebd906660 con 0x55eebda54b00 2023-11-27T15:19:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.615+0000 7fb7077fe700 1 --2- 172.21.15.52:0/2309042959 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55eebda54b00 0x55eebda56ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:35.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.616+0000 7fb7077fe700 1 --2- 172.21.15.52:0/2309042959 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55eebda54b00 0x55eebda56ee0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d315f16bbee5763b server_cookie=b187fb11e6cb8c7f in_seq=0 out_seq=0 2023-11-27T15:19:35.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.616+0000 7fb6e6ffd700 10 client.15774 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:35.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.616+0000 7fb6e6ffd700 1 -- 172.21.15.52:0/2309042959 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55eebd906660 con 0x55eebda54b00 2023-11-27T15:19:35.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.616+0000 7fb6e6ffd700 10 client.15774 handle_command_reply: tid=0 2023-11-27T15:19:35.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.616+0000 7fb71a173b80 1 client.15774 shutdown 2023-11-27T15:19:35.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.616+0000 7fb71a173b80 1 -- 172.21.15.52:0/2309042959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eebd9e7470 msgr2=0x55eebd9f5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:35.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.616+0000 7fb71a173b80 1 --2- 172.21.15.52:0/2309042959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eebd9e7470 0x55eebd9f5ed0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc011b10 tx=0x7fb6fc009950 comp rx=0 tx=0).stop 2023-11-27T15:19:35.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.616+0000 7fb71a173b80 1 -- 172.21.15.52:0/2309042959 shutdown_connections 2023-11-27T15:19:35.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.616+0000 7fb71a173b80 1 --2- 172.21.15.52:0/2309042959 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55eebda54b00 0x55eebda56ee0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.616+0000 7fb71a173b80 1 --2- 172.21.15.52:0/2309042959 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eebd9eb8b0 0x55eebd9f6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.616+0000 7fb71a173b80 1 --2- 172.21.15.52:0/2309042959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55eebd9e6b60 0x55eebd9f5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.616+0000 7fb6e6ffd700 0 client.15774 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:35.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.616+0000 7fb71a173b80 1 --2- 172.21.15.52:0/2309042959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eebd9e7470 0x55eebd9f5ed0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.616+0000 7fb71a173b80 1 -- 172.21.15.52:0/2309042959 >> 172.21.15.52:0/2309042959 conn(0x55eebd9dc360 msgr2=0x55eebda02820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:35.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.616+0000 7fb71a173b80 1 -- 172.21.15.52:0/2309042959 shutdown_connections 2023-11-27T15:19:35.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.616+0000 7fb71a173b80 1 -- 172.21.15.52:0/2309042959 wait complete. 2023-11-27T15:19:35.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.617+0000 7fb71a173b80 20 client.15774 trim_cache size 0 max 16384 2023-11-27T15:19:35.632 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:19:35.632 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2023-11-27T15:19:35.632 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2023-11-27T15:19:35.632 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:19:35.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.617+0000 7fb70dfe7700 1 -- 172.21.15.52:0/2438294837 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb6f00a12c0 msgr2=0x7fb6f00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:35.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.617+0000 7fb70dfe7700 1 --2- 172.21.15.52:0/2438294837 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb6f00a12c0 0x7fb6f00a3770 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb708102640 tx=0x7fb6fc009260 comp rx=0 tx=0).stop 2023-11-27T15:19:35.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.617+0000 7fb70dfe7700 1 -- 172.21.15.52:0/2438294837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7080ff450 msgr2=0x7fb708100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:35.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.617+0000 7fb70dfe7700 1 --2- 172.21.15.52:0/2438294837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7080ff450 0x7fb708100e50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb6f800e9a0 tx=0x7fb6f800ecb0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.617+0000 7fb70dfe7700 1 -- 172.21.15.52:0/2438294837 shutdown_connections 2023-11-27T15:19:35.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.617+0000 7fb70dfe7700 1 --2- 172.21.15.52:0/2438294837 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb6f00a12c0 0x7fb6f00a3770 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.617+0000 7fb70dfe7700 1 --2- 172.21.15.52:0/2438294837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb708104490 0x7fb7081a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.617+0000 7fb70dfe7700 1 --2- 172.21.15.52:0/2438294837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7080ff450 0x7fb708100e50 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.617+0000 7fb70dfe7700 1 --2- 172.21.15.52:0/2438294837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7080ffdf0 0x7fb708101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:35.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.617+0000 7fb70dfe7700 1 -- 172.21.15.52:0/2438294837 >> 172.21.15.52:0/2438294837 conn(0x7fb7080770b0 msgr2=0x7fb708067df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:35.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.617+0000 7fb70dfe7700 1 -- 172.21.15.52:0/2438294837 shutdown_connections 2023-11-27T15:19:35.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:35.617+0000 7fb70dfe7700 1 -- 172.21.15.52:0/2438294837 wait complete. 2023-11-27T15:19:35.684 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-27T15:19:35.684 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:19:35.919 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:35.919 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:36.010 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:36.010 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:36.050 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:36.050 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:36.050 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 64 -lt 100 ']' 2023-11-27T15:19:36.050 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 64 + 1 2023-11-27T15:19:36.083 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=65 2023-11-27T15:19:36.084 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:36.095 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:36.095 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:36.096 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:36.096 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:36.096 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:36.652 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:36.652 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:36.742 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:36.742 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:36.761 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:36.762 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:36.762 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 65 -lt 100 ']' 2023-11-27T15:19:36.762 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 65 + 1 2023-11-27T15:19:36.808 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=66 2023-11-27T15:19:36.808 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:36.893 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:36.894 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:36.895 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:36.895 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:36.895 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:37.237 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:36 smithi052 ceph-mon[99056]: pgmap v53: 97 pgs: 97 active+clean; 2.2 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-11-27T15:19:37.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:36 smithi150 ceph-mon[100559]: pgmap v53: 97 pgs: 97 active+clean; 2.2 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-11-27T15:19:37.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:36 smithi102 ceph-mon[100296]: pgmap v53: 97 pgs: 97 active+clean; 2.2 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-11-27T15:19:37.385 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:37.386 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:37.554 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:37.554 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:37.566 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:37.567 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:37.567 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 66 -lt 100 ']' 2023-11-27T15:19:37.567 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 66 + 1 2023-11-27T15:19:37.595 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=67 2023-11-27T15:19:37.595 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:37.641 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:37.641 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:37.642 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:37.643 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:37.643 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:38.486 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:38.486 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:38.645 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:38.646 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:38.667 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:38.668 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:38.668 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 67 -lt 100 ']' 2023-11-27T15:19:38.668 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 67 + 1 2023-11-27T15:19:38.716 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=68 2023-11-27T15:19:38.717 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:38.773 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:38.774 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:38.775 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:38.775 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:38.775 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:39.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.060+0000 7f994030c700 1 -- 172.21.15.52:0/1219903019 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99380feff0 msgr2=0x7f993810c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:39.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.060+0000 7f994030c700 1 --2- 172.21.15.52:0/1219903019 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99380feff0 0x7f993810c470 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9928009990 tx=0x7f9928009ca0 comp rx=0 tx=0).stop 2023-11-27T15:19:39.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.061+0000 7f994030c700 1 -- 172.21.15.52:0/1219903019 shutdown_connections 2023-11-27T15:19:39.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.061+0000 7f994030c700 1 --2- 172.21.15.52:0/1219903019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99380fe6e0 0x7f99380feab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:39.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.061+0000 7f994030c700 1 --2- 172.21.15.52:0/1219903019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f993810cae0 0x7f993810eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:39.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.061+0000 7f994030c700 1 --2- 172.21.15.52:0/1219903019 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99380feff0 0x7f993810c470 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:39.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.061+0000 7f994030c700 1 -- 172.21.15.52:0/1219903019 >> 172.21.15.52:0/1219903019 conn(0x7f99380742a0 msgr2=0x7f99380746a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:39.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.061+0000 7f994030c700 1 -- 172.21.15.52:0/1219903019 shutdown_connections 2023-11-27T15:19:39.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.061+0000 7f994030c700 1 -- 172.21.15.52:0/1219903019 wait complete. 2023-11-27T15:19:39.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.061+0000 7f994030c700 1 Processor -- start 2023-11-27T15:19:39.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.061+0000 7f994030c700 1 -- start start 2023-11-27T15:19:39.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.062+0000 7f994030c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99380fe6e0 0x7f9938198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:39.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.062+0000 7f994030c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99380feff0 0x7f9938198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:39.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.062+0000 7f994030c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f993810cae0 0x7f993819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:39.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.062+0000 7f994030c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993819d4f0 con 0x7f99380fe6e0 2023-11-27T15:19:39.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.062+0000 7f994030c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993819d660 con 0x7f99380feff0 2023-11-27T15:19:39.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.062+0000 7f994030c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993819d820 con 0x7f993810cae0 2023-11-27T15:19:39.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.062+0000 7f993e0a8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99380fe6e0 0x7f9938198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:39.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.062+0000 7f993e0a8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99380fe6e0 0x7f9938198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52910/0 (socket says 172.21.15.52:52910) 2023-11-27T15:19:39.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.062+0000 7f993e0a8700 1 -- 172.21.15.52:0/4117455444 learned_addr learned my addr 172.21.15.52:0/4117455444 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:39.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.062+0000 7f993d8a7700 1 --2- 172.21.15.52:0/4117455444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99380feff0 0x7f9938198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:39.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.062+0000 7f993e8a9700 1 --2- 172.21.15.52:0/4117455444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f993810cae0 0x7f993819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:39.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.062+0000 7f993e0a8700 1 -- 172.21.15.52:0/4117455444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f993810cae0 msgr2=0x7f993819ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:39.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.062+0000 7f993e0a8700 1 --2- 172.21.15.52:0/4117455444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f993810cae0 0x7f993819ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:39.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.062+0000 7f993e0a8700 1 -- 172.21.15.52:0/4117455444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99380feff0 msgr2=0x7f9938198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:39.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.062+0000 7f993e0a8700 1 --2- 172.21.15.52:0/4117455444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99380feff0 0x7f9938198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:39.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.062+0000 7f993e0a8700 1 -- 172.21.15.52:0/4117455444 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99280095f0 con 0x7f99380fe6e0 2023-11-27T15:19:39.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.063+0000 7f993e0a8700 1 --2- 172.21.15.52:0/4117455444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99380fe6e0 0x7f9938198510 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f993400e9a0 tx=0x7f993400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:39.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.063+0000 7f992f7fe700 1 -- 172.21.15.52:0/4117455444 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9934009790 con 0x7f99380fe6e0 2023-11-27T15:19:39.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.063+0000 7f992f7fe700 1 -- 172.21.15.52:0/4117455444 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9934010430 con 0x7f99380fe6e0 2023-11-27T15:19:39.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.063+0000 7f994030c700 1 -- 172.21.15.52:0/4117455444 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99381124f0 con 0x7f99380fe6e0 2023-11-27T15:19:39.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.063+0000 7f992f7fe700 1 -- 172.21.15.52:0/4117455444 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9934018680 con 0x7f99380fe6e0 2023-11-27T15:19:39.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.063+0000 7f994030c700 1 -- 172.21.15.52:0/4117455444 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9938112360 con 0x7f99380fe6e0 2023-11-27T15:19:39.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.064+0000 7f994030c700 1 -- 172.21.15.52:0/4117455444 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9938112680 con 0x7f99380fe6e0 2023-11-27T15:19:39.074 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.065+0000 7f992f7fe700 1 -- 172.21.15.52:0/4117455444 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f99340105a0 con 0x7f99380fe6e0 2023-11-27T15:19:39.074 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.066+0000 7f992f7fe700 1 --2- 172.21.15.52:0/4117455444 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f99240a12c0 0x7f99240a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:39.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.066+0000 7f992f7fe700 1 -- 172.21.15.52:0/4117455444 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f9934014070 con 0x7f99380fe6e0 2023-11-27T15:19:39.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.066+0000 7f993d8a7700 1 --2- 172.21.15.52:0/4117455444 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f99240a12c0 0x7f99240a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:39.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.067+0000 7f993d8a7700 1 --2- 172.21.15.52:0/4117455444 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f99240a12c0 0x7f99240a3770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9938199b20 tx=0x7f9928005fd0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:39.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.071+0000 7f992f7fe700 1 -- 172.21.15.52:0/4117455444 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9934051730 con 0x7f99380fe6e0 2023-11-27T15:19:39.179 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:38 smithi052 ceph-mon[99056]: pgmap v54: 97 pgs: 97 active+clean; 2.3 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-11-27T15:19:39.277 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:39.277 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:39.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:38 smithi150 ceph-mon[100559]: pgmap v54: 97 pgs: 97 active+clean; 2.3 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-11-27T15:19:39.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:38 smithi102 ceph-mon[100296]: pgmap v54: 97 pgs: 97 active+clean; 2.3 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 11 op/s 2023-11-27T15:19:39.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.330+0000 7f994030c700 1 -- 172.21.15.52:0/4117455444 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9938067eb0 con 0x7f99380fe6e0 2023-11-27T15:19:39.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.331+0000 7f992f7fe700 1 -- 172.21.15.52:0/4117455444 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10804 (secure 0 0 0) 0x7f99340962e0 con 0x7f99380fe6e0 2023-11-27T15:19:39.336 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:19:39.336 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2328881376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2328881376},{"type":"v1","addr":"172.21.15.52:6839","nonce":2328881376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2799551570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2799551570},{"type":"v1","addr":"172.21.15.52:6841","nonce":2799551570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:16:09.319630+0000","modified":"2023-11-27T15:16:14.742405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24494,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/215725838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":215725838},{"type":"v1","addr":"172.21.15.150:6839","nonce":215725838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3571830559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3571830559},{"type":"v1","addr":"172.21.15.102:6839","nonce":3571830559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:19:39.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.333+0000 7f994030c700 1 -- 172.21.15.52:0/4117455444 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f99240a12c0 msgr2=0x7f99240a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:39.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.333+0000 7f994030c700 1 --2- 172.21.15.52:0/4117455444 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f99240a12c0 0x7f99240a3770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9938199b20 tx=0x7f9928005fd0 comp rx=0 tx=0).stop 2023-11-27T15:19:39.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.333+0000 7f994030c700 1 -- 172.21.15.52:0/4117455444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99380fe6e0 msgr2=0x7f9938198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:39.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.333+0000 7f994030c700 1 --2- 172.21.15.52:0/4117455444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99380fe6e0 0x7f9938198510 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f993400e9a0 tx=0x7f993400ecb0 comp rx=0 tx=0).stop 2023-11-27T15:19:39.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.333+0000 7f994030c700 1 -- 172.21.15.52:0/4117455444 shutdown_connections 2023-11-27T15:19:39.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.333+0000 7f994030c700 1 --2- 172.21.15.52:0/4117455444 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f99240a12c0 0x7f99240a3770 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:39.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.333+0000 7f994030c700 1 --2- 172.21.15.52:0/4117455444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f993810cae0 0x7f993819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:39.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.333+0000 7f994030c700 1 --2- 172.21.15.52:0/4117455444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99380fe6e0 0x7f9938198510 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:39.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.333+0000 7f994030c700 1 --2- 172.21.15.52:0/4117455444 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99380feff0 0x7f9938198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:39.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.333+0000 7f994030c700 1 -- 172.21.15.52:0/4117455444 >> 172.21.15.52:0/4117455444 conn(0x7f99380742a0 msgr2=0x7f99380ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:39.340 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.333+0000 7f994030c700 1 -- 172.21.15.52:0/4117455444 shutdown_connections 2023-11-27T15:19:39.340 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:39.333+0000 7f994030c700 1 -- 172.21.15.52:0/4117455444 wait complete. 2023-11-27T15:19:39.340 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 23 2023-11-27T15:19:39.475 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-27T15:19:39.487 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:39.487 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:39.500 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:39.500 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:39.500 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 68 -lt 100 ']' 2023-11-27T15:19:39.500 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 68 + 1 2023-11-27T15:19:39.541 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=69 2023-11-27T15:19:39.541 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:39.560 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:39.560 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:39.561 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:39.561 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:39.561 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:39.935 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:39.935 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:39.978 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:39.978 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:40.006 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:40.006 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:40.006 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 69 -lt 100 ']' 2023-11-27T15:19:40.006 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 69 + 1 2023-11-27T15:19:40.066 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=70 2023-11-27T15:19:40.066 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:40.114 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:40.114 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:40.115 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:40.115 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:40.115 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:40.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:39 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/4117455444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:19:40.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:39 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/4117455444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:19:40.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:39 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/4117455444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:19:40.543 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:40.543 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:40.778 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:40.778 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:40.791 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:40.791 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:40.792 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 70 -lt 100 ']' 2023-11-27T15:19:40.792 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 70 + 1 2023-11-27T15:19:40.820 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=71 2023-11-27T15:19:40.820 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:40.865 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:40.865 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:40.866 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:40.866 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:40.866 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:41.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:40 smithi150 ceph-mon[100559]: pgmap v55: 97 pgs: 97 active+clean; 2.3 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-11-27T15:19:41.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:40 smithi102 ceph-mon[100296]: pgmap v55: 97 pgs: 97 active+clean; 2.3 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-11-27T15:19:41.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:40 smithi052 ceph-mon[99056]: pgmap v55: 97 pgs: 97 active+clean; 2.3 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-11-27T15:19:41.818 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:41.818 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:42.078 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:42.078 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:42.121 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:42.121 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:42.121 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 71 -lt 100 ']' 2023-11-27T15:19:42.121 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 71 + 1 2023-11-27T15:19:42.166 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=72 2023-11-27T15:19:42.166 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:42.224 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:42.224 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:42.225 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:42.226 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:42.226 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:42.490 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.485+0000 7f61cd07f700 1 -- 172.21.15.52:0/2817396091 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61c8101390 msgr2=0x7f61c8101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:42.490 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.485+0000 7f61cd07f700 1 --2- 172.21.15.52:0/2817396091 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61c8101390 0x7f61c8101760 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009990 tx=0x7f61b8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.490 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.485+0000 7f61cd07f700 1 -- 172.21.15.52:0/2817396091 shutdown_connections 2023-11-27T15:19:42.490 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.485+0000 7f61cd07f700 1 --2- 172.21.15.52:0/2817396091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c810a8b0 0x7f61c810ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.490 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.485+0000 7f61cd07f700 1 --2- 172.21.15.52:0/2817396091 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61c8101390 0x7f61c8101760 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.491 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.485+0000 7f61cd07f700 1 --2- 172.21.15.52:0/2817396091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61c8101ca0 0x7f61c810a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.491 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.485+0000 7f61cd07f700 1 -- 172.21.15.52:0/2817396091 >> 172.21.15.52:0/2817396091 conn(0x7f61c80770b0 msgr2=0x7f61c80774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:42.491 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.485+0000 7f61cd07f700 1 -- 172.21.15.52:0/2817396091 shutdown_connections 2023-11-27T15:19:42.491 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.485+0000 7f61cd07f700 1 -- 172.21.15.52:0/2817396091 wait complete. 2023-11-27T15:19:42.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.486+0000 7f61cd07f700 1 Processor -- start 2023-11-27T15:19:42.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.486+0000 7f61cd07f700 1 -- start start 2023-11-27T15:19:42.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.486+0000 7f61cd07f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c8101390 0x7f61c80ff1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:42.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.486+0000 7f61cd07f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61c8101ca0 0x7f61c80ff700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:42.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.486+0000 7f61cd07f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61c810a8b0 0x7f61c81a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:42.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.486+0000 7f61cd07f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c80fff30 con 0x7f61c8101ca0 2023-11-27T15:19:42.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.486+0000 7f61cd07f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c81a4b30 con 0x7f61c810a8b0 2023-11-27T15:19:42.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.486+0000 7f61cd07f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c81a4c70 con 0x7f61c8101390 2023-11-27T15:19:42.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.487+0000 7f61c659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61c8101ca0 0x7f61c80ff700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:42.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.487+0000 7f61c659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61c8101ca0 0x7f61c80ff700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52926/0 (socket says 172.21.15.52:52926) 2023-11-27T15:19:42.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.487+0000 7f61c659c700 1 -- 172.21.15.52:0/1166128146 learned_addr learned my addr 172.21.15.52:0/1166128146 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:42.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.487+0000 7f61c759e700 1 --2- 172.21.15.52:0/1166128146 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61c810a8b0 0x7f61c81a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:42.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.487+0000 7f61c6d9d700 1 --2- 172.21.15.52:0/1166128146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c8101390 0x7f61c80ff1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:42.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.487+0000 7f61c659c700 1 -- 172.21.15.52:0/1166128146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c8101390 msgr2=0x7f61c80ff1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:42.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.487+0000 7f61c659c700 1 --2- 172.21.15.52:0/1166128146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c8101390 0x7f61c80ff1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.487+0000 7f61c659c700 1 -- 172.21.15.52:0/1166128146 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61c810a8b0 msgr2=0x7f61c81a45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:42.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.487+0000 7f61c659c700 1 --2- 172.21.15.52:0/1166128146 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61c810a8b0 0x7f61c81a45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.487+0000 7f61c659c700 1 -- 172.21.15.52:0/1166128146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b80095f0 con 0x7f61c8101ca0 2023-11-27T15:19:42.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.487+0000 7f61c659c700 1 --2- 172.21.15.52:0/1166128146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61c8101ca0 0x7f61c80ff700 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f61b000e9a0 tx=0x7f61b000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:42.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.487+0000 7f61bffff700 1 -- 172.21.15.52:0/1166128146 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b0009790 con 0x7f61c8101ca0 2023-11-27T15:19:42.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.488+0000 7f61cd07f700 1 -- 172.21.15.52:0/1166128146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c8110350 con 0x7f61c8101ca0 2023-11-27T15:19:42.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.488+0000 7f61bffff700 1 -- 172.21.15.52:0/1166128146 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61b0010430 con 0x7f61c8101ca0 2023-11-27T15:19:42.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.488+0000 7f61bffff700 1 -- 172.21.15.52:0/1166128146 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b0018680 con 0x7f61c8101ca0 2023-11-27T15:19:42.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.488+0000 7f61cd07f700 1 -- 172.21.15.52:0/1166128146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c81101c0 con 0x7f61c8101ca0 2023-11-27T15:19:42.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.490+0000 7f61bffff700 1 -- 172.21.15.52:0/1166128146 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f61b00105a0 con 0x7f61c8101ca0 2023-11-27T15:19:42.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.491+0000 7f61bffff700 1 --2- 172.21.15.52:0/1166128146 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f61b40a12c0 0x7f61b40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:42.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.491+0000 7f61bffff700 1 -- 172.21.15.52:0/1166128146 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f61b0014070 con 0x7f61c8101ca0 2023-11-27T15:19:42.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.491+0000 7f61c6d9d700 1 --2- 172.21.15.52:0/1166128146 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f61b40a12c0 0x7f61b40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:42.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.491+0000 7f61c6d9d700 1 --2- 172.21.15.52:0/1166128146 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f61b40a12c0 0x7f61b40a3770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f61b800b2e0 tx=0x7f61b8005fd0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:42.558 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.554+0000 7f61d920bb80 1 Processor -- start 2023-11-27T15:19:42.559 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.554+0000 7f61d920bb80 1 -- start start 2023-11-27T15:19:42.560 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.554+0000 7f61d920bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac5b75eb60 0x55ac5b75ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:42.560 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.554+0000 7f61d920bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac5b75f470 0x55ac5b763370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:42.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.554+0000 7f61d920bb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac5b7638b0 0x55ac5b763d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:42.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.554+0000 7f61d920bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac5b664060 con 0x55ac5b75f470 2023-11-27T15:19:42.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.554+0000 7f61d920bb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac5b6b68a0 con 0x55ac5b7638b0 2023-11-27T15:19:42.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.554+0000 7f61d920bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac5b6afd30 con 0x55ac5b75eb60 2023-11-27T15:19:42.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.555+0000 7f61c659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac5b75f470 0x55ac5b763370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:42.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.555+0000 7f61c659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac5b75f470 0x55ac5b763370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52932/0 (socket says 172.21.15.52:52932) 2023-11-27T15:19:42.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.555+0000 7f61c659c700 1 -- 172.21.15.52:0/392757961 learned_addr learned my addr 172.21.15.52:0/392757961 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:42.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.555+0000 7f61c6d9d700 1 --2- 172.21.15.52:0/392757961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac5b75eb60 0x55ac5b75ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:42.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.555+0000 7f61c759e700 1 --2- 172.21.15.52:0/392757961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac5b7638b0 0x55ac5b763d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:42.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.555+0000 7f61c659c700 1 -- 172.21.15.52:0/392757961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac5b75eb60 msgr2=0x55ac5b75ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:42.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.555+0000 7f61c659c700 1 --2- 172.21.15.52:0/392757961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac5b75eb60 0x55ac5b75ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.555+0000 7f61c659c700 1 -- 172.21.15.52:0/392757961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac5b7638b0 msgr2=0x55ac5b763d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:42.561 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.555+0000 7f61c659c700 1 --2- 172.21.15.52:0/392757961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac5b7638b0 0x55ac5b763d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.562 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.555+0000 7f61c659c700 1 -- 172.21.15.52:0/392757961 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac5b531630 con 0x55ac5b75f470 2023-11-27T15:19:42.562 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.555+0000 7f61c659c700 1 --2- 172.21.15.52:0/392757961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac5b75f470 0x55ac5b763370 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f61b00a1ec0 tx=0x7f61b0099fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b294bb3ebad22195 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:42.562 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.555+0000 7f61bd7fa700 1 -- 172.21.15.52:0/392757961 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b0010880 con 0x55ac5b75f470 2023-11-27T15:19:42.562 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.555+0000 7f61bd7fa700 1 -- 172.21.15.52:0/392757961 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61b00109e0 con 0x55ac5b75f470 2023-11-27T15:19:42.562 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.555+0000 7f61bd7fa700 1 -- 172.21.15.52:0/392757961 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b00a1f60 con 0x55ac5b75f470 2023-11-27T15:19:42.566 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.556+0000 7f61d920bb80 1 -- 172.21.15.52:0/392757961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac5b75f470 msgr2=0x55ac5b763370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:42.566 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.556+0000 7f61d920bb80 1 --2- 172.21.15.52:0/392757961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac5b75f470 0x55ac5b763370 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f61b00a1ec0 tx=0x7f61b0099fa0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.566 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.556+0000 7f61d920bb80 1 -- 172.21.15.52:0/392757961 shutdown_connections 2023-11-27T15:19:42.566 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.556+0000 7f61d920bb80 1 --2- 172.21.15.52:0/392757961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac5b7638b0 0x55ac5b763d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.567 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.556+0000 7f61d920bb80 1 --2- 172.21.15.52:0/392757961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac5b75eb60 0x55ac5b75ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.567 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.556+0000 7f61d920bb80 1 --2- 172.21.15.52:0/392757961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac5b75f470 0x55ac5b763370 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.567 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.556+0000 7f61d920bb80 1 -- 172.21.15.52:0/392757961 >> 172.21.15.52:0/392757961 conn(0x55ac5b754360 msgr2=0x55ac5b759b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:42.567 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.556+0000 7f61d920bb80 1 -- 172.21.15.52:0/392757961 shutdown_connections 2023-11-27T15:19:42.567 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.557+0000 7f61d920bb80 1 -- 172.21.15.52:0/392757961 wait complete. 2023-11-27T15:19:42.567 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.557+0000 7f61d920bb80 1 -- start start 2023-11-27T15:19:42.567 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.557+0000 7f61d920bb80 1 Processor -- start 2023-11-27T15:19:42.567 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.557+0000 7f61d920bb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac5b75eb60 0x55ac5b76b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:42.567 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.557+0000 7f61d920bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac5b75f470 0x55ac5b768380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:42.567 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.557+0000 7f61d920bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac5b7638b0 0x55ac5b768ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:42.567 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.557+0000 7f61d920bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac5b6518e0 con 0x55ac5b75f470 2023-11-27T15:19:42.567 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.557+0000 7f61d920bb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac5b651580 con 0x55ac5b75eb60 2023-11-27T15:19:42.568 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.557+0000 7f61d920bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac5b533920 con 0x55ac5b7638b0 2023-11-27T15:19:42.568 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.557+0000 7f61d920bb80 10 client.0 _ll_register_callbacks cb 0x55ac5b66f780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:19:42.568 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.557+0000 7f61c659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac5b75f470 0x55ac5b768380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:42.568 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.558+0000 7f61c659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac5b75f470 0x55ac5b768380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52940/0 (socket says 172.21.15.52:52940) 2023-11-27T15:19:42.568 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.558+0000 7f61c659c700 1 -- 172.21.15.52:0/4006391448 learned_addr learned my addr 172.21.15.52:0/4006391448 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:42.568 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.558+0000 7f61c759e700 1 --2- 172.21.15.52:0/4006391448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac5b7638b0 0x55ac5b768ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:42.568 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.558+0000 7f61c6d9d700 1 --2- 172.21.15.52:0/4006391448 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac5b75eb60 0x55ac5b76b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:42.568 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.558+0000 7f61c659c700 1 -- 172.21.15.52:0/4006391448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac5b7638b0 msgr2=0x55ac5b768ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:42.568 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.558+0000 7f61c659c700 1 --2- 172.21.15.52:0/4006391448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac5b7638b0 0x55ac5b768ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.568 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.558+0000 7f61c659c700 1 -- 172.21.15.52:0/4006391448 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac5b75eb60 msgr2=0x55ac5b76b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:42.568 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.558+0000 7f61c659c700 1 --2- 172.21.15.52:0/4006391448 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac5b75eb60 0x55ac5b76b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.568 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.558+0000 7f61c659c700 1 -- 172.21.15.52:0/4006391448 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c81101c0 con 0x55ac5b75f470 2023-11-27T15:19:42.568 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.558+0000 7f61c659c700 1 --2- 172.21.15.52:0/4006391448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac5b75f470 0x55ac5b768380 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f61b00ca6d0 tx=0x7f61b0099740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a557383b6cffac3 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:42.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.558+0000 7f61ae7fc700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:19:42.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.558+0000 7f61ae7fc700 1 -- 172.21.15.52:0/4006391448 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b0010880 con 0x55ac5b75f470 2023-11-27T15:19:42.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.558+0000 7f61ae7fc700 1 -- 172.21.15.52:0/4006391448 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f61b00a2170 con 0x55ac5b75f470 2023-11-27T15:19:42.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.558+0000 7f61d920bb80 1 -- 172.21.15.52:0/4006391448 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac5b751b20 con 0x55ac5b75f470 2023-11-27T15:19:42.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.558+0000 7f61ae7fc700 1 -- 172.21.15.52:0/4006391448 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61b0094bb0 con 0x55ac5b75f470 2023-11-27T15:19:42.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.559+0000 7f61d920bb80 10 client.15792 fetch_fsmap learned FSMap version 23 2023-11-27T15:19:42.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.559+0000 7f61d920bb80 1 -- 172.21.15.52:0/4006391448 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ac5b676190 con 0x55ac5b75f470 2023-11-27T15:19:42.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.559+0000 7f61ae7fc700 1 -- 172.21.15.52:0/4006391448 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ac5b751b20 con 0x55ac5b75f470 2023-11-27T15:19:42.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.559+0000 7f61ae7fc700 1 -- 172.21.15.52:0/4006391448 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f61b00b7f90 con 0x55ac5b75f470 2023-11-27T15:19:42.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.559+0000 7f61d920bb80 10 client.15792 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:19:42.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.559+0000 7f61d920bb80 10 client.15792 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:19:42.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.559+0000 7f61d920bb80 20 client.15792 populate_metadata read hostname 'smithi052' 2023-11-27T15:19:42.569 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.559+0000 7f61d920bb80 1 --2- 172.21.15.52:0/4006391448 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55ac5b7784a0 0x55ac5b77a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:42.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.559+0000 7f61d920bb80 4 client.15792 mds_command: new command op to 24472 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:19:42.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.559+0000 7f61d920bb80 1 -- 172.21.15.52:0/4006391448 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac5b6720a0 con 0x55ac5b7784a0 2023-11-27T15:19:42.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.559+0000 7f61c6d9d700 1 --2- 172.21.15.52:0/4006391448 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55ac5b7784a0 0x55ac5b77a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:42.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.560+0000 7f61c6d9d700 1 --2- 172.21.15.52:0/4006391448 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55ac5b7784a0 0x55ac5b77a880 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=803287fb6a330e98 server_cookie=d843502e75806743 in_seq=0 out_seq=0 2023-11-27T15:19:42.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.560+0000 7f61ae7fc700 10 client.15792 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:19:42.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.560+0000 7f61ae7fc700 1 -- 172.21.15.52:0/4006391448 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ac5b6720a0 con 0x55ac5b7784a0 2023-11-27T15:19:42.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.561+0000 7f61ae7fc700 10 client.15792 handle_command_reply: tid=0 2023-11-27T15:19:42.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.561+0000 7f61d920bb80 1 client.15792 shutdown 2023-11-27T15:19:42.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.561+0000 7f61d920bb80 1 -- 172.21.15.52:0/4006391448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac5b75f470 msgr2=0x55ac5b768380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:42.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.561+0000 7f61d920bb80 1 --2- 172.21.15.52:0/4006391448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac5b75f470 0x55ac5b768380 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f61b00ca6d0 tx=0x7f61b0099740 comp rx=0 tx=0).stop 2023-11-27T15:19:42.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.561+0000 7f61d920bb80 1 -- 172.21.15.52:0/4006391448 shutdown_connections 2023-11-27T15:19:42.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.561+0000 7f61d920bb80 1 --2- 172.21.15.52:0/4006391448 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55ac5b7784a0 0x55ac5b77a880 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.561+0000 7f61d920bb80 1 --2- 172.21.15.52:0/4006391448 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac5b75eb60 0x55ac5b76b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.561+0000 7f61d920bb80 1 --2- 172.21.15.52:0/4006391448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac5b7638b0 0x55ac5b768ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.561+0000 7f61d920bb80 1 --2- 172.21.15.52:0/4006391448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac5b75f470 0x55ac5b768380 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.561+0000 7f61d920bb80 1 -- 172.21.15.52:0/4006391448 >> 172.21.15.52:0/4006391448 conn(0x55ac5b754360 msgr2=0x55ac5b764d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:42.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.561+0000 7f61ae7fc700 0 client.15792 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:19:42.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.561+0000 7f61d920bb80 1 -- 172.21.15.52:0/4006391448 shutdown_connections 2023-11-27T15:19:42.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.561+0000 7f61d920bb80 1 -- 172.21.15.52:0/4006391448 wait complete. 2023-11-27T15:19:42.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.561+0000 7f61d920bb80 20 client.15792 trim_cache size 0 max 16384 2023-11-27T15:19:42.579 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.575+0000 7f61d920bb80 1 Processor -- start 2023-11-27T15:19:42.579 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.575+0000 7f61d920bb80 1 -- start start 2023-11-27T15:19:42.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.575+0000 7f61d920bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac5b75eb60 0x55ac5b76ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:42.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.575+0000 7f61d920bb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac5b75f470 0x55ac5b7619d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:42.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.575+0000 7f61d920bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac5b7638b0 0x55ac5b761f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:42.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.575+0000 7f61d920bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac5b75b1f0 con 0x55ac5b7638b0 2023-11-27T15:19:42.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.575+0000 7f61d920bb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac5b766900 con 0x55ac5b75f470 2023-11-27T15:19:42.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.575+0000 7f61d920bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac5b7624b0 con 0x55ac5b75eb60 2023-11-27T15:19:42.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.575+0000 7f61c759e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac5b7638b0 0x55ac5b761f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:42.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.575+0000 7f61c759e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac5b7638b0 0x55ac5b761f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52950/0 (socket says 172.21.15.52:52950) 2023-11-27T15:19:42.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.575+0000 7f61c759e700 1 -- 172.21.15.52:0/1275191188 learned_addr learned my addr 172.21.15.52:0/1275191188 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:42.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.575+0000 7f61c6d9d700 1 --2- 172.21.15.52:0/1275191188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac5b75eb60 0x55ac5b76ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:42.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.575+0000 7f61c659c700 1 --2- 172.21.15.52:0/1275191188 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac5b75f470 0x55ac5b7619d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:42.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.576+0000 7f61c759e700 1 -- 172.21.15.52:0/1275191188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac5b75eb60 msgr2=0x55ac5b76ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:42.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.576+0000 7f61c759e700 1 --2- 172.21.15.52:0/1275191188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac5b75eb60 0x55ac5b76ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.576+0000 7f61c759e700 1 -- 172.21.15.52:0/1275191188 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac5b75f470 msgr2=0x55ac5b7619d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:42.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.576+0000 7f61c759e700 1 --2- 172.21.15.52:0/1275191188 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac5b75f470 0x55ac5b7619d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.576+0000 7f61c759e700 1 -- 172.21.15.52:0/1275191188 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac5b763f50 con 0x55ac5b7638b0 2023-11-27T15:19:42.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.576+0000 7f61c759e700 1 --2- 172.21.15.52:0/1275191188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac5b7638b0 0x55ac5b761f10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f61c0011840 tx=0x7f61c0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea8a177c75fc7d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:42.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.576+0000 7f61bdffb700 1 -- 172.21.15.52:0/1275191188 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c000e040 con 0x55ac5b7638b0 2023-11-27T15:19:42.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.576+0000 7f61bdffb700 1 -- 172.21.15.52:0/1275191188 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61c000f040 con 0x55ac5b7638b0 2023-11-27T15:19:42.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.576+0000 7f61bdffb700 1 -- 172.21.15.52:0/1275191188 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c0007500 con 0x55ac5b7638b0 2023-11-27T15:19:42.587 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.577+0000 7f61d920bb80 1 -- 172.21.15.52:0/1275191188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac5b7638b0 msgr2=0x55ac5b761f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:42.587 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.577+0000 7f61d920bb80 1 --2- 172.21.15.52:0/1275191188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac5b7638b0 0x55ac5b761f10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f61c0011840 tx=0x7f61c0009920 comp rx=0 tx=0).stop 2023-11-27T15:19:42.587 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.577+0000 7f61d920bb80 1 -- 172.21.15.52:0/1275191188 shutdown_connections 2023-11-27T15:19:42.587 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.577+0000 7f61d920bb80 1 --2- 172.21.15.52:0/1275191188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac5b7638b0 0x55ac5b761f10 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.587 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.577+0000 7f61d920bb80 1 --2- 172.21.15.52:0/1275191188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac5b75eb60 0x55ac5b76ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.587 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.577+0000 7f61d920bb80 1 --2- 172.21.15.52:0/1275191188 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac5b75f470 0x55ac5b7619d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.587 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.577+0000 7f61d920bb80 1 -- 172.21.15.52:0/1275191188 >> 172.21.15.52:0/1275191188 conn(0x55ac5b754360 msgr2=0x55ac5b774b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:42.588 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.577+0000 7f61d920bb80 1 -- 172.21.15.52:0/1275191188 shutdown_connections 2023-11-27T15:19:42.588 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.577+0000 7f61d920bb80 1 -- 172.21.15.52:0/1275191188 wait complete. 2023-11-27T15:19:42.588 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.577+0000 7f61d920bb80 1 -- start start 2023-11-27T15:19:42.588 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.577+0000 7f61d920bb80 1 Processor -- start 2023-11-27T15:19:42.588 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.578+0000 7f61d920bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac5b75eb60 0x55ac5b76d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:42.588 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.578+0000 7f61d920bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac5b75f470 0x55ac5b76ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:42.588 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.578+0000 7f61d920bb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac5b7638b0 0x55ac5b76e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:42.588 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.578+0000 7f61d920bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac5b7cb330 con 0x55ac5b75f470 2023-11-27T15:19:42.588 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.578+0000 7f61d920bb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac5b7cb4a0 con 0x55ac5b7638b0 2023-11-27T15:19:42.588 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.578+0000 7f61d920bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac5b7cb660 con 0x55ac5b75eb60 2023-11-27T15:19:42.588 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.578+0000 7f61d920bb80 10 client.0 _ll_register_callbacks cb 0x55ac5b66f780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:19:42.588 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.578+0000 7f61c659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac5b75f470 0x55ac5b76ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:42.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.578+0000 7f61c659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac5b75f470 0x55ac5b76ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52954/0 (socket says 172.21.15.52:52954) 2023-11-27T15:19:42.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.578+0000 7f61c659c700 1 -- 172.21.15.52:0/1165358860 learned_addr learned my addr 172.21.15.52:0/1165358860 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:42.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.578+0000 7f61c759e700 1 --2- 172.21.15.52:0/1165358860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac5b7638b0 0x55ac5b76e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:42.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.578+0000 7f61c6d9d700 1 --2- 172.21.15.52:0/1165358860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac5b75eb60 0x55ac5b76d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:42.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.578+0000 7f61c659c700 1 -- 172.21.15.52:0/1165358860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac5b75eb60 msgr2=0x55ac5b76d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:42.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.578+0000 7f61c659c700 1 --2- 172.21.15.52:0/1165358860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac5b75eb60 0x55ac5b76d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.578+0000 7f61c659c700 1 -- 172.21.15.52:0/1165358860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac5b7638b0 msgr2=0x55ac5b76e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:42.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.578+0000 7f61c659c700 1 --2- 172.21.15.52:0/1165358860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac5b7638b0 0x55ac5b76e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.578+0000 7f61c659c700 1 -- 172.21.15.52:0/1165358860 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c00114f0 con 0x55ac5b75f470 2023-11-27T15:19:42.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.579+0000 7f61c659c700 1 --2- 172.21.15.52:0/1165358860 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac5b75f470 0x55ac5b76ded0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f61b00964a0 tx=0x7f61b00996a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a17f979afac5b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:42.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.579+0000 7f61ae7fc700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:19:42.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.579+0000 7f61ae7fc700 1 -- 172.21.15.52:0/1165358860 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b0010880 con 0x55ac5b75f470 2023-11-27T15:19:42.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.579+0000 7f61ae7fc700 1 -- 172.21.15.52:0/1165358860 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f61b0094dc0 con 0x55ac5b75f470 2023-11-27T15:19:42.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.579+0000 7f61d920bb80 1 -- 172.21.15.52:0/1165358860 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f61b00109e0 con 0x55ac5b75f470 2023-11-27T15:19:42.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.579+0000 7f61ae7fc700 1 -- 172.21.15.52:0/1165358860 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61b009a010 con 0x55ac5b75f470 2023-11-27T15:19:42.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.579+0000 7f61d920bb80 10 client.15798 fetch_fsmap learned FSMap version 23 2023-11-27T15:19:42.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.579+0000 7f61d920bb80 1 -- 172.21.15.52:0/1165358860 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ac5b4ddde0 con 0x55ac5b75f470 2023-11-27T15:19:42.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.579+0000 7f61ae7fc700 1 -- 172.21.15.52:0/1165358860 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61b0063080 con 0x55ac5b75f470 2023-11-27T15:19:42.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.579+0000 7f61ae7fc700 1 -- 172.21.15.52:0/1165358860 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f61b0098f00 con 0x55ac5b75f470 2023-11-27T15:19:42.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.579+0000 7f61d920bb80 10 client.15798 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:19:42.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.579+0000 7f61d920bb80 10 client.15798 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:19:42.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.579+0000 7f61d920bb80 20 client.15798 populate_metadata read hostname 'smithi052' 2023-11-27T15:19:42.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.579+0000 7f61d920bb80 1 --2- 172.21.15.52:0/1165358860 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55ac5b7cc850 0x55ac5b7cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:42.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.580+0000 7f61d920bb80 4 client.15798 mds_command: new command op to 24472 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:19:42.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.580+0000 7f61d920bb80 1 -- 172.21.15.52:0/1165358860 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ac5b67e660 con 0x55ac5b7cc850 2023-11-27T15:19:42.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.580+0000 7f61c6d9d700 1 --2- 172.21.15.52:0/1165358860 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55ac5b7cc850 0x55ac5b7cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:42.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.580+0000 7f61ae7fc700 10 client.15798 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:19:42.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.580+0000 7f61c6d9d700 1 --2- 172.21.15.52:0/1165358860 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55ac5b7cc850 0x55ac5b7cec30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dc2dd41098c77241 server_cookie=ab96cd0ec4886ee3 in_seq=0 out_seq=0 2023-11-27T15:19:42.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.580+0000 7f61ae7fc700 1 -- 172.21.15.52:0/1165358860 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ac5b67e660 con 0x55ac5b7cc850 2023-11-27T15:19:42.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.580+0000 7f61ae7fc700 10 client.15798 handle_command_reply: tid=0 2023-11-27T15:19:42.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.580+0000 7f61d920bb80 1 client.15798 shutdown 2023-11-27T15:19:42.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.581+0000 7f61d920bb80 1 -- 172.21.15.52:0/1165358860 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac5b75f470 msgr2=0x55ac5b76ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:42.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.581+0000 7f61d920bb80 1 --2- 172.21.15.52:0/1165358860 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac5b75f470 0x55ac5b76ded0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f61b00964a0 tx=0x7f61b00996a0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.581+0000 7f61d920bb80 1 -- 172.21.15.52:0/1165358860 shutdown_connections 2023-11-27T15:19:42.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.581+0000 7f61d920bb80 1 --2- 172.21.15.52:0/1165358860 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55ac5b7cc850 0x55ac5b7cec30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.581+0000 7f61d920bb80 1 --2- 172.21.15.52:0/1165358860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac5b7638b0 0x55ac5b76e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.581+0000 7f61d920bb80 1 --2- 172.21.15.52:0/1165358860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ac5b75eb60 0x55ac5b76d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.581+0000 7f61d920bb80 1 --2- 172.21.15.52:0/1165358860 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac5b75f470 0x55ac5b76ded0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.581+0000 7f61ae7fc700 0 client.15798 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:19:42.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.581+0000 7f61d920bb80 1 -- 172.21.15.52:0/1165358860 >> 172.21.15.52:0/1165358860 conn(0x55ac5b754360 msgr2=0x55ac5b7619d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:42.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.581+0000 7f61d920bb80 1 -- 172.21.15.52:0/1165358860 shutdown_connections 2023-11-27T15:19:42.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.581+0000 7f61d920bb80 1 -- 172.21.15.52:0/1165358860 wait complete. 2023-11-27T15:19:42.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.581+0000 7f61d920bb80 20 client.15798 trim_cache size 0 max 16384 2023-11-27T15:19:42.592 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:19:42.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.581+0000 7f61cd07f700 1 -- 172.21.15.52:0/1166128146 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f61b40a12c0 msgr2=0x7f61b40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:42.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.581+0000 7f61cd07f700 1 --2- 172.21.15.52:0/1166128146 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f61b40a12c0 0x7f61b40a3770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f61b800b2e0 tx=0x7f61b8005fd0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.581+0000 7f61cd07f700 1 -- 172.21.15.52:0/1166128146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61c8101ca0 msgr2=0x7f61c80ff700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:42.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.581+0000 7f61cd07f700 1 --2- 172.21.15.52:0/1166128146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61c8101ca0 0x7f61c80ff700 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f61b000e9a0 tx=0x7f61b000ecb0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.582+0000 7f61cd07f700 1 -- 172.21.15.52:0/1166128146 shutdown_connections 2023-11-27T15:19:42.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.582+0000 7f61cd07f700 1 --2- 172.21.15.52:0/1166128146 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f61b40a12c0 0x7f61b40a3770 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.582+0000 7f61cd07f700 1 --2- 172.21.15.52:0/1166128146 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61c810a8b0 0x7f61c81a45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.582+0000 7f61cd07f700 1 --2- 172.21.15.52:0/1166128146 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c8101390 0x7f61c80ff1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.582+0000 7f61cd07f700 1 --2- 172.21.15.52:0/1166128146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61c8101ca0 0x7f61c80ff700 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:42.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.582+0000 7f61cd07f700 1 -- 172.21.15.52:0/1166128146 >> 172.21.15.52:0/1166128146 conn(0x7f61c80770b0 msgr2=0x7f61c80fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:42.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.582+0000 7f61cd07f700 1 -- 172.21.15.52:0/1166128146 shutdown_connections 2023-11-27T15:19:42.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:42.582+0000 7f61cd07f700 1 -- 172.21.15.52:0/1166128146 wait complete. 2023-11-27T15:19:42.641 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-27T15:19:42.885 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:42.885 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:42.923 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:42.923 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:42.949 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:42.949 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:42.949 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 72 -lt 100 ']' 2023-11-27T15:19:42.949 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 72 + 1 2023-11-27T15:19:42.974 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=73 2023-11-27T15:19:42.974 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:42.985 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:42.985 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:42.986 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:42.986 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:42.986 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:43.245 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:42 smithi052 ceph-mon[99056]: pgmap v56: 97 pgs: 97 active+clean; 2.4 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-11-27T15:19:43.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:42 smithi150 ceph-mon[100559]: pgmap v56: 97 pgs: 97 active+clean; 2.4 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-11-27T15:19:43.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:42 smithi102 ceph-mon[100296]: pgmap v56: 97 pgs: 97 active+clean; 2.4 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 11 op/s 2023-11-27T15:19:43.385 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:43.385 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:43.458 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:43.458 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:43.517 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:43.517 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:43.517 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 73 -lt 100 ']' 2023-11-27T15:19:43.517 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 73 + 1 2023-11-27T15:19:43.549 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=74 2023-11-27T15:19:43.549 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:43.593 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:43.593 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:43.594 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:43.594 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:43.594 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:44.184 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:44.185 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:44.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:43 smithi150 ceph-mon[100559]: pgmap v57: 97 pgs: 97 active+clean; 2.4 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-11-27T15:19:44.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:43 smithi102 ceph-mon[100296]: pgmap v57: 97 pgs: 97 active+clean; 2.4 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-11-27T15:19:44.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:43 smithi052 ceph-mon[99056]: pgmap v57: 97 pgs: 97 active+clean; 2.4 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-11-27T15:19:44.428 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:44.428 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:44.449 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:44.449 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:44.449 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 74 -lt 100 ']' 2023-11-27T15:19:44.449 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 74 + 1 2023-11-27T15:19:44.491 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=75 2023-11-27T15:19:44.491 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:44.553 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:44.553 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:44.554 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:44.555 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:44.555 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:45.143 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:45.143 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:45.328 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:45.328 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:45.357 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:45.357 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:45.357 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 75 -lt 100 ']' 2023-11-27T15:19:45.357 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 75 + 1 2023-11-27T15:19:45.474 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=76 2023-11-27T15:19:45.474 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:45.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.526+0000 7fdb28afe700 1 -- 172.21.15.52:0/829597926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb241055d0 msgr2=0x7fdb2410d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:45.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.526+0000 7fdb28afe700 1 --2- 172.21.15.52:0/829597926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb241055d0 0x7fdb2410d1e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c00b0d0 tx=0x7fdb1c00b3e0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.526+0000 7fdb28afe700 1 -- 172.21.15.52:0/829597926 shutdown_connections 2023-11-27T15:19:45.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.526+0000 7fdb28afe700 1 --2- 172.21.15.52:0/829597926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb241055d0 0x7fdb2410d1e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.526+0000 7fdb28afe700 1 --2- 172.21.15.52:0/829597926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb240feaf0 0x7fdb240feec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.526+0000 7fdb28afe700 1 --2- 172.21.15.52:0/829597926 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb240ff490 0x7fdb24104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.526+0000 7fdb28afe700 1 -- 172.21.15.52:0/829597926 >> 172.21.15.52:0/829597926 conn(0x7fdb24075920 msgr2=0x7fdb24075d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:45.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.526+0000 7fdb28afe700 1 -- 172.21.15.52:0/829597926 shutdown_connections 2023-11-27T15:19:45.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.526+0000 7fdb28afe700 1 -- 172.21.15.52:0/829597926 wait complete. 2023-11-27T15:19:45.534 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.527+0000 7fdb28afe700 1 Processor -- start 2023-11-27T15:19:45.534 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.527+0000 7fdb28afe700 1 -- start start 2023-11-27T15:19:45.534 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.528+0000 7fdb28afe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb240feaf0 0x7fdb24196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:45.534 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.528+0000 7fdb28afe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb240ff490 0x7fdb24196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:45.534 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.528+0000 7fdb28afe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb241055d0 0x7fdb2419aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:45.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.528+0000 7fdb28afe700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2419b4f0 con 0x7fdb241055d0 2023-11-27T15:19:45.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.528+0000 7fdb28afe700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2419b660 con 0x7fdb240feaf0 2023-11-27T15:19:45.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.528+0000 7fdb28afe700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2419b820 con 0x7fdb240ff490 2023-11-27T15:19:45.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.528+0000 7fdb22d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb241055d0 0x7fdb2419aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:45.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.528+0000 7fdb22d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb241055d0 0x7fdb2419aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52970/0 (socket says 172.21.15.52:52970) 2023-11-27T15:19:45.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.528+0000 7fdb22d9d700 1 -- 172.21.15.52:0/2150236229 learned_addr learned my addr 172.21.15.52:0/2150236229 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:45.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.528+0000 7fdb21d9b700 1 --2- 172.21.15.52:0/2150236229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb240ff490 0x7fdb24196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:45.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.528+0000 7fdb2259c700 1 --2- 172.21.15.52:0/2150236229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb240feaf0 0x7fdb24196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:45.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.528+0000 7fdb22d9d700 1 -- 172.21.15.52:0/2150236229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb240ff490 msgr2=0x7fdb24196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:45.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.528+0000 7fdb22d9d700 1 --2- 172.21.15.52:0/2150236229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb240ff490 0x7fdb24196a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.528+0000 7fdb22d9d700 1 -- 172.21.15.52:0/2150236229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb240feaf0 msgr2=0x7fdb24196510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:45.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.528+0000 7fdb22d9d700 1 --2- 172.21.15.52:0/2150236229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb240feaf0 0x7fdb24196510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.529+0000 7fdb22d9d700 1 -- 172.21.15.52:0/2150236229 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb1c009d00 con 0x7fdb241055d0 2023-11-27T15:19:45.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.529+0000 7fdb21d9b700 1 --2- 172.21.15.52:0/2150236229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb240ff490 0x7fdb24196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:45.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.529+0000 7fdb22d9d700 1 --2- 172.21.15.52:0/2150236229 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb241055d0 0x7fdb2419aeb0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c007870 tx=0x7fdb1c0078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:45.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.529+0000 7fdb1b7fe700 1 -- 172.21.15.52:0/2150236229 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c00e040 con 0x7fdb241055d0 2023-11-27T15:19:45.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.529+0000 7fdb28afe700 1 -- 172.21.15.52:0/2150236229 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb241103f0 con 0x7fdb241055d0 2023-11-27T15:19:45.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.529+0000 7fdb1b7fe700 1 -- 172.21.15.52:0/2150236229 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb1c005070 con 0x7fdb241055d0 2023-11-27T15:19:45.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.529+0000 7fdb1b7fe700 1 -- 172.21.15.52:0/2150236229 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c01b6c0 con 0x7fdb241055d0 2023-11-27T15:19:45.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.529+0000 7fdb28afe700 1 -- 172.21.15.52:0/2150236229 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb24110260 con 0x7fdb241055d0 2023-11-27T15:19:45.537 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.532+0000 7fdb1b7fe700 1 -- 172.21.15.52:0/2150236229 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fdb1c0054e0 con 0x7fdb241055d0 2023-11-27T15:19:45.537 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:45.537 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:45.538 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.532+0000 7fdb1b7fe700 1 --2- 172.21.15.52:0/2150236229 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fdb100a12c0 0x7fdb100a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:45.538 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.532+0000 7fdb1b7fe700 1 -- 172.21.15.52:0/2150236229 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fdb1c0cddf0 con 0x7fdb241055d0 2023-11-27T15:19:45.538 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.532+0000 7fdb2259c700 1 --2- 172.21.15.52:0/2150236229 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fdb100a12c0 0x7fdb100a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:45.538 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.533+0000 7fdb2259c700 1 --2- 172.21.15.52:0/2150236229 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fdb100a12c0 0x7fdb100a3770 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009c60 tx=0x7fdb14009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:45.538 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:45.539 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:45.539 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:45.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.612+0000 7fdb34c8ab80 1 Processor -- start 2023-11-27T15:19:45.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.612+0000 7fdb34c8ab80 1 -- start start 2023-11-27T15:19:45.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.612+0000 7fdb34c8ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa8781ab60 0x55aa8781af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:45.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.612+0000 7fdb34c8ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa8781b470 0x55aa8781f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:45.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.612+0000 7fdb34c8ab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa8781f8b0 0x55aa8781fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:45.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.612+0000 7fdb34c8ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa87720060 con 0x55aa8781ab60 2023-11-27T15:19:45.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.613+0000 7fdb34c8ab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa877728a0 con 0x55aa8781f8b0 2023-11-27T15:19:45.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.613+0000 7fdb34c8ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa8776bd30 con 0x55aa8781b470 2023-11-27T15:19:45.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.613+0000 7fdb2259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa8781ab60 0x55aa8781af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:45.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.613+0000 7fdb2259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa8781ab60 0x55aa8781af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52974/0 (socket says 172.21.15.52:52974) 2023-11-27T15:19:45.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.613+0000 7fdb22d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa8781f8b0 0x55aa8781fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:45.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.613+0000 7fdb2259c700 1 -- 172.21.15.52:0/1417758212 learned_addr learned my addr 172.21.15.52:0/1417758212 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:45.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.613+0000 7fdb21d9b700 1 --2- 172.21.15.52:0/1417758212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa8781b470 0x55aa8781f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:45.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.613+0000 7fdb2259c700 1 -- 172.21.15.52:0/1417758212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa8781b470 msgr2=0x55aa8781f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:45.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.613+0000 7fdb2259c700 1 --2- 172.21.15.52:0/1417758212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa8781b470 0x55aa8781f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.613+0000 7fdb2259c700 1 -- 172.21.15.52:0/1417758212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa8781f8b0 msgr2=0x55aa8781fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:45.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.613+0000 7fdb2259c700 1 --2- 172.21.15.52:0/1417758212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa8781f8b0 0x55aa8781fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.613+0000 7fdb2259c700 1 -- 172.21.15.52:0/1417758212 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa875ed630 con 0x55aa8781ab60 2023-11-27T15:19:45.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.613+0000 7fdb22d9d700 1 --2- 172.21.15.52:0/1417758212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa8781f8b0 0x55aa8781fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:45.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.613+0000 7fdb2259c700 1 --2- 172.21.15.52:0/1417758212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa8781ab60 0x55aa8781af30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fdb1400e4e0 tx=0x7fdb14014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd96081d13bd145 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:45.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.614+0000 7fdb18ff9700 1 -- 172.21.15.52:0/1417758212 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1400ee00 con 0x55aa8781ab60 2023-11-27T15:19:45.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.614+0000 7fdb18ff9700 1 -- 172.21.15.52:0/1417758212 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb14021070 con 0x55aa8781ab60 2023-11-27T15:19:45.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.614+0000 7fdb18ff9700 1 -- 172.21.15.52:0/1417758212 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1401d810 con 0x55aa8781ab60 2023-11-27T15:19:45.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.615+0000 7fdb34c8ab80 1 -- 172.21.15.52:0/1417758212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa8781ab60 msgr2=0x55aa8781af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:45.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.615+0000 7fdb34c8ab80 1 --2- 172.21.15.52:0/1417758212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa8781ab60 0x55aa8781af30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fdb1400e4e0 tx=0x7fdb14014000 comp rx=0 tx=0).stop 2023-11-27T15:19:45.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.615+0000 7fdb34c8ab80 1 -- 172.21.15.52:0/1417758212 shutdown_connections 2023-11-27T15:19:45.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.615+0000 7fdb34c8ab80 1 --2- 172.21.15.52:0/1417758212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa8781f8b0 0x55aa8781fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.615+0000 7fdb34c8ab80 1 --2- 172.21.15.52:0/1417758212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa8781ab60 0x55aa8781af30 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.615+0000 7fdb34c8ab80 1 --2- 172.21.15.52:0/1417758212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa8781b470 0x55aa8781f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.615+0000 7fdb34c8ab80 1 -- 172.21.15.52:0/1417758212 >> 172.21.15.52:0/1417758212 conn(0x55aa87810360 msgr2=0x55aa87815b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:45.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.615+0000 7fdb34c8ab80 1 -- 172.21.15.52:0/1417758212 shutdown_connections 2023-11-27T15:19:45.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.615+0000 7fdb34c8ab80 1 -- 172.21.15.52:0/1417758212 wait complete. 2023-11-27T15:19:45.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.615+0000 7fdb34c8ab80 1 -- start start 2023-11-27T15:19:45.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.616+0000 7fdb34c8ab80 1 Processor -- start 2023-11-27T15:19:45.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.616+0000 7fdb34c8ab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa8781ab60 0x55aa878270c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:45.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.616+0000 7fdb34c8ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa8781b470 0x55aa87824340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:45.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.616+0000 7fdb34c8ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa8781f8b0 0x55aa87824a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:45.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.616+0000 7fdb34c8ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa8770d8e0 con 0x55aa8781b470 2023-11-27T15:19:45.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.616+0000 7fdb34c8ab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa8770d580 con 0x55aa8781ab60 2023-11-27T15:19:45.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.616+0000 7fdb34c8ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa875ef920 con 0x55aa8781f8b0 2023-11-27T15:19:45.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.616+0000 7fdb21d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa8781b470 0x55aa87824340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:45.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.616+0000 7fdb34c8ab80 10 client.0 _ll_register_callbacks cb 0x55aa8772b780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:19:45.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.616+0000 7fdb21d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa8781b470 0x55aa87824340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52982/0 (socket says 172.21.15.52:52982) 2023-11-27T15:19:45.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.616+0000 7fdb2259c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa8781ab60 0x55aa878270c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:45.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.616+0000 7fdb21d9b700 1 -- 172.21.15.52:0/2231841734 learned_addr learned my addr 172.21.15.52:0/2231841734 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:45.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.617+0000 7fdb22d9d700 1 --2- 172.21.15.52:0/2231841734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa8781f8b0 0x55aa87824a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:45.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.617+0000 7fdb21d9b700 1 -- 172.21.15.52:0/2231841734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa8781f8b0 msgr2=0x55aa87824a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:45.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.617+0000 7fdb21d9b700 1 --2- 172.21.15.52:0/2231841734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa8781f8b0 0x55aa87824a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.617+0000 7fdb21d9b700 1 -- 172.21.15.52:0/2231841734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa8781ab60 msgr2=0x55aa878270c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:45.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.617+0000 7fdb21d9b700 1 --2- 172.21.15.52:0/2231841734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa8781ab60 0x55aa878270c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.617+0000 7fdb21d9b700 1 -- 172.21.15.52:0/2231841734 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb14017070 con 0x55aa8781b470 2023-11-27T15:19:45.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.617+0000 7fdb2259c700 1 --2- 172.21.15.52:0/2231841734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa8781ab60 0x55aa878270c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:45.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.617+0000 7fdb21d9b700 1 --2- 172.21.15.52:0/2231841734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa8781b470 0x55aa87824340 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c012d20 tx=0x7fdb0c00f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3235effe1d93af1 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:45.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.617+0000 7fdb09ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:19:45.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.617+0000 7fdb09ffb700 1 -- 172.21.15.52:0/2231841734 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c017070 con 0x55aa8781b470 2023-11-27T15:19:45.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.617+0000 7fdb09ffb700 1 -- 172.21.15.52:0/2231841734 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fdb0c00fbd0 con 0x55aa8781b470 2023-11-27T15:19:45.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.617+0000 7fdb34c8ab80 1 -- 172.21.15.52:0/2231841734 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa8780db20 con 0x55aa8781b470 2023-11-27T15:19:45.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.617+0000 7fdb09ffb700 1 -- 172.21.15.52:0/2231841734 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb0c00fd90 con 0x55aa8781b470 2023-11-27T15:19:45.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.617+0000 7fdb34c8ab80 10 client.15810 fetch_fsmap learned FSMap version 23 2023-11-27T15:19:45.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.617+0000 7fdb34c8ab80 1 -- 172.21.15.52:0/2231841734 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55aa87732190 con 0x55aa8781b470 2023-11-27T15:19:45.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.618+0000 7fdb09ffb700 1 -- 172.21.15.52:0/2231841734 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aa8780db20 con 0x55aa8781b470 2023-11-27T15:19:45.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.618+0000 7fdb09ffb700 1 -- 172.21.15.52:0/2231841734 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdb0c003c20 con 0x55aa8781b470 2023-11-27T15:19:45.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.618+0000 7fdb34c8ab80 10 client.15810 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:19:45.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.618+0000 7fdb34c8ab80 10 client.15810 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:19:45.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.618+0000 7fdb34c8ab80 20 client.15810 populate_metadata read hostname 'smithi052' 2023-11-27T15:19:45.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.618+0000 7fdb34c8ab80 1 --2- 172.21.15.52:0/2231841734 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55aa87834880 0x55aa87836c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:45.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.618+0000 7fdb34c8ab80 4 client.15810 mds_command: new command op to 24494 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:19:45.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.618+0000 7fdb34c8ab80 1 -- 172.21.15.52:0/2231841734 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa8772e0a0 con 0x55aa87834880 2023-11-27T15:19:45.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.618+0000 7fdb2259c700 1 --2- 172.21.15.52:0/2231841734 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55aa87834880 0x55aa87836c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:45.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.619+0000 7fdb09ffb700 10 client.15810 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:19:45.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.619+0000 7fdb2259c700 1 --2- 172.21.15.52:0/2231841734 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55aa87834880 0x55aa87836c90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1def7ced34280583 server_cookie=45719500b215a179 in_seq=0 out_seq=0 2023-11-27T15:19:45.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.619+0000 7fdb09ffb700 1 -- 172.21.15.52:0/2231841734 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55aa8772e0a0 con 0x55aa87834880 2023-11-27T15:19:45.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.619+0000 7fdb09ffb700 10 client.15810 handle_command_reply: tid=0 2023-11-27T15:19:45.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.619+0000 7fdb34c8ab80 1 client.15810 shutdown 2023-11-27T15:19:45.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.620+0000 7fdb34c8ab80 1 -- 172.21.15.52:0/2231841734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa8781b470 msgr2=0x55aa87824340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:45.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.620+0000 7fdb34c8ab80 1 --2- 172.21.15.52:0/2231841734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa8781b470 0x55aa87824340 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c012d20 tx=0x7fdb0c00f6a0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.620+0000 7fdb34c8ab80 1 -- 172.21.15.52:0/2231841734 shutdown_connections 2023-11-27T15:19:45.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.620+0000 7fdb34c8ab80 1 --2- 172.21.15.52:0/2231841734 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55aa87834880 0x55aa87836c90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.620+0000 7fdb34c8ab80 1 --2- 172.21.15.52:0/2231841734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa8781ab60 0x55aa878270c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.620+0000 7fdb34c8ab80 1 --2- 172.21.15.52:0/2231841734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa8781f8b0 0x55aa87824a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.620+0000 7fdb09ffb700 0 client.15810 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:19:45.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.620+0000 7fdb34c8ab80 1 --2- 172.21.15.52:0/2231841734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa8781b470 0x55aa87824340 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.620+0000 7fdb34c8ab80 1 -- 172.21.15.52:0/2231841734 >> 172.21.15.52:0/2231841734 conn(0x55aa87810360 msgr2=0x55aa87820d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:45.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.620+0000 7fdb34c8ab80 1 -- 172.21.15.52:0/2231841734 shutdown_connections 2023-11-27T15:19:45.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.620+0000 7fdb34c8ab80 1 -- 172.21.15.52:0/2231841734 wait complete. 2023-11-27T15:19:45.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.620+0000 7fdb34c8ab80 20 client.15810 trim_cache size 0 max 16384 2023-11-27T15:19:45.640 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.634+0000 7fdb34c8ab80 1 Processor -- start 2023-11-27T15:19:45.640 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.634+0000 7fdb34c8ab80 1 -- start start 2023-11-27T15:19:45.640 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.634+0000 7fdb34c8ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa8781ab60 0x55aa8781d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:45.640 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.634+0000 7fdb34c8ab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa8781b470 0x55aa87826bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:45.640 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.634+0000 7fdb34c8ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa8781f8b0 0x55aa8781da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:45.640 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.634+0000 7fdb34c8ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa8781e4c0 con 0x55aa8781ab60 2023-11-27T15:19:45.641 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.634+0000 7fdb34c8ab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa87815e20 con 0x55aa8781b470 2023-11-27T15:19:45.641 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.634+0000 7fdb34c8ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa87815fe0 con 0x55aa8781f8b0 2023-11-27T15:19:45.641 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.634+0000 7fdb2259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa8781ab60 0x55aa8781d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:45.641 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.634+0000 7fdb2259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa8781ab60 0x55aa8781d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52988/0 (socket says 172.21.15.52:52988) 2023-11-27T15:19:45.641 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.634+0000 7fdb2259c700 1 -- 172.21.15.52:0/2370662638 learned_addr learned my addr 172.21.15.52:0/2370662638 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:45.641 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.634+0000 7fdb21d9b700 1 --2- 172.21.15.52:0/2370662638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa8781b470 0x55aa87826bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:45.641 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.635+0000 7fdb22d9d700 1 --2- 172.21.15.52:0/2370662638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa8781f8b0 0x55aa8781da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:45.641 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.635+0000 7fdb2259c700 1 -- 172.21.15.52:0/2370662638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa8781f8b0 msgr2=0x55aa8781da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:45.641 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.635+0000 7fdb2259c700 1 --2- 172.21.15.52:0/2370662638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa8781f8b0 0x55aa8781da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.641 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.635+0000 7fdb2259c700 1 -- 172.21.15.52:0/2370662638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa8781b470 msgr2=0x55aa87826bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:45.641 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.635+0000 7fdb2259c700 1 --2- 172.21.15.52:0/2370662638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa8781b470 0x55aa87826bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.641 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.635+0000 7fdb2259c700 1 -- 172.21.15.52:0/2370662638 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa87731ee0 con 0x55aa8781ab60 2023-11-27T15:19:45.641 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.635+0000 7fdb21d9b700 1 --2- 172.21.15.52:0/2370662638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa8781b470 0x55aa87826bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:45.641 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.635+0000 7fdb2259c700 1 --2- 172.21.15.52:0/2370662638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa8781ab60 0x55aa8781d1a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009ec0 tx=0x7fdb14009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9e43a6005ed840 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:45.642 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.635+0000 7fdb197fa700 1 -- 172.21.15.52:0/2370662638 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1400ed80 con 0x55aa8781ab60 2023-11-27T15:19:45.642 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.635+0000 7fdb197fa700 1 -- 172.21.15.52:0/2370662638 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb14033170 con 0x55aa8781ab60 2023-11-27T15:19:45.642 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.635+0000 7fdb197fa700 1 -- 172.21.15.52:0/2370662638 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb14003680 con 0x55aa8781ab60 2023-11-27T15:19:45.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.636+0000 7fdb34c8ab80 1 -- 172.21.15.52:0/2370662638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa8781ab60 msgr2=0x55aa8781d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:45.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.636+0000 7fdb34c8ab80 1 --2- 172.21.15.52:0/2370662638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa8781ab60 0x55aa8781d1a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009ec0 tx=0x7fdb14009e50 comp rx=0 tx=0).stop 2023-11-27T15:19:45.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.636+0000 7fdb34c8ab80 1 -- 172.21.15.52:0/2370662638 shutdown_connections 2023-11-27T15:19:45.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.636+0000 7fdb34c8ab80 1 --2- 172.21.15.52:0/2370662638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa8781f8b0 0x55aa8781da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.636+0000 7fdb34c8ab80 1 --2- 172.21.15.52:0/2370662638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa8781ab60 0x55aa8781d1a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.636+0000 7fdb34c8ab80 1 --2- 172.21.15.52:0/2370662638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa8781b470 0x55aa87826bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.636+0000 7fdb34c8ab80 1 -- 172.21.15.52:0/2370662638 >> 172.21.15.52:0/2370662638 conn(0x55aa87810360 msgr2=0x55aa87830150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:45.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.636+0000 7fdb34c8ab80 1 -- 172.21.15.52:0/2370662638 shutdown_connections 2023-11-27T15:19:45.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.636+0000 7fdb34c8ab80 1 -- 172.21.15.52:0/2370662638 wait complete. 2023-11-27T15:19:45.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.636+0000 7fdb34c8ab80 1 -- start start 2023-11-27T15:19:45.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.637+0000 7fdb34c8ab80 1 Processor -- start 2023-11-27T15:19:45.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.637+0000 7fdb34c8ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa8781ab60 0x55aa87829990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:45.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.637+0000 7fdb34c8ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa8781b470 0x55aa87829ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:45.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.637+0000 7fdb34c8ab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa8781f8b0 0x55aa8782a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:45.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.637+0000 7fdb34c8ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa8788bb50 con 0x55aa8781b470 2023-11-27T15:19:45.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.637+0000 7fdb34c8ab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa8788bc90 con 0x55aa8781f8b0 2023-11-27T15:19:45.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.637+0000 7fdb34c8ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa8788bdd0 con 0x55aa8781ab60 2023-11-27T15:19:45.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.637+0000 7fdb34c8ab80 10 client.0 _ll_register_callbacks cb 0x7fdb0c0127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:19:45.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.637+0000 7fdb21d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa8781b470 0x55aa87829ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:45.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.637+0000 7fdb2259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa8781ab60 0x55aa87829990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:45.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.637+0000 7fdb2259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa8781ab60 0x55aa87829990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:39638/0 (socket says 172.21.15.52:39638) 2023-11-27T15:19:45.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.637+0000 7fdb2259c700 1 -- 172.21.15.52:0/2904335125 learned_addr learned my addr 172.21.15.52:0/2904335125 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:45.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.637+0000 7fdb22d9d700 1 --2- 172.21.15.52:0/2904335125 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa8781f8b0 0x55aa8782a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:45.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.637+0000 7fdb2259c700 1 -- 172.21.15.52:0/2904335125 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa8781f8b0 msgr2=0x55aa8782a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:45.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.637+0000 7fdb2259c700 1 --2- 172.21.15.52:0/2904335125 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa8781f8b0 0x55aa8782a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.637+0000 7fdb2259c700 1 -- 172.21.15.52:0/2904335125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa8781b470 msgr2=0x55aa87829ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:45.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.637+0000 7fdb2259c700 1 --2- 172.21.15.52:0/2904335125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa8781b470 0x55aa87829ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.637+0000 7fdb2259c700 1 -- 172.21.15.52:0/2904335125 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa87732190 con 0x55aa8781ab60 2023-11-27T15:19:45.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.638+0000 7fdb2259c700 1 --2- 172.21.15.52:0/2904335125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa8781ab60 0x55aa87829990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdb1400e4e0 tx=0x7fdb1400ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40a0cd8bf7ab15b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:45.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.638+0000 7fdb09ffb700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2023-11-27T15:19:45.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.638+0000 7fdb09ffb700 1 -- 172.21.15.52:0/2904335125 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb140054c0 con 0x55aa8781ab60 2023-11-27T15:19:45.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.638+0000 7fdb34c8ab80 1 -- 172.21.15.52:0/2904335125 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb14021070 con 0x55aa8781ab60 2023-11-27T15:19:45.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.638+0000 7fdb09ffb700 1 -- 172.21.15.52:0/2904335125 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fdb14005620 con 0x55aa8781ab60 2023-11-27T15:19:45.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.638+0000 7fdb09ffb700 1 -- 172.21.15.52:0/2904335125 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb140332e0 con 0x55aa8781ab60 2023-11-27T15:19:45.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.638+0000 7fdb34c8ab80 10 client.25540 fetch_fsmap learned FSMap version 23 2023-11-27T15:19:45.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.638+0000 7fdb34c8ab80 1 -- 172.21.15.52:0/2904335125 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fdb1402a040 con 0x55aa8781ab60 2023-11-27T15:19:45.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.638+0000 7fdb09ffb700 1 -- 172.21.15.52:0/2904335125 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb14021070 con 0x55aa8781ab60 2023-11-27T15:19:45.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.639+0000 7fdb09ffb700 1 -- 172.21.15.52:0/2904335125 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdb1403a070 con 0x55aa8781ab60 2023-11-27T15:19:45.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.639+0000 7fdb34c8ab80 10 client.25540 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:19:45.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.639+0000 7fdb34c8ab80 10 client.25540 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:19:45.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.639+0000 7fdb34c8ab80 20 client.25540 populate_metadata read hostname 'smithi052' 2023-11-27T15:19:45.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.639+0000 7fdb34c8ab80 1 --2- 172.21.15.52:0/2904335125 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55aa8788cf60 0x55aa8788f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:45.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.639+0000 7fdb34c8ab80 4 client.25540 mds_command: new command op to 24494 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:19:45.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.639+0000 7fdb34c8ab80 1 -- 172.21.15.52:0/2904335125 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55aa8773a660 con 0x55aa8788cf60 2023-11-27T15:19:45.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.639+0000 7fdb21d9b700 1 --2- 172.21.15.52:0/2904335125 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55aa8788cf60 0x55aa8788f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:45.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.639+0000 7fdb09ffb700 10 client.25540 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:19:45.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.639+0000 7fdb21d9b700 1 --2- 172.21.15.52:0/2904335125 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55aa8788cf60 0x55aa8788f340 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=232fb52d95814f0b server_cookie=96a753052bdc362b in_seq=0 out_seq=0 2023-11-27T15:19:45.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.640+0000 7fdb09ffb700 1 -- 172.21.15.52:0/2904335125 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55aa8773a660 con 0x55aa8788cf60 2023-11-27T15:19:45.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.640+0000 7fdb09ffb700 10 client.25540 handle_command_reply: tid=0 2023-11-27T15:19:45.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.640+0000 7fdb34c8ab80 1 client.25540 shutdown 2023-11-27T15:19:45.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.640+0000 7fdb34c8ab80 1 -- 172.21.15.52:0/2904335125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa8781ab60 msgr2=0x55aa87829990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:45.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.640+0000 7fdb34c8ab80 1 --2- 172.21.15.52:0/2904335125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa8781ab60 0x55aa87829990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdb1400e4e0 tx=0x7fdb1400ee00 comp rx=0 tx=0).stop 2023-11-27T15:19:45.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.640+0000 7fdb34c8ab80 1 -- 172.21.15.52:0/2904335125 shutdown_connections 2023-11-27T15:19:45.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.640+0000 7fdb34c8ab80 1 --2- 172.21.15.52:0/2904335125 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55aa8788cf60 0x55aa8788f340 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.640+0000 7fdb34c8ab80 1 --2- 172.21.15.52:0/2904335125 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aa8781f8b0 0x55aa8782a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.640+0000 7fdb34c8ab80 1 --2- 172.21.15.52:0/2904335125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55aa8781ab60 0x55aa87829990 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.640+0000 7fdb34c8ab80 1 --2- 172.21.15.52:0/2904335125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa8781b470 0x55aa87829ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.640+0000 7fdb09ffb700 0 client.25540 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:19:45.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.640+0000 7fdb34c8ab80 1 -- 172.21.15.52:0/2904335125 >> 172.21.15.52:0/2904335125 conn(0x55aa87810360 msgr2=0x55aa8781da10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:45.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.640+0000 7fdb34c8ab80 1 -- 172.21.15.52:0/2904335125 shutdown_connections 2023-11-27T15:19:45.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.640+0000 7fdb34c8ab80 1 -- 172.21.15.52:0/2904335125 wait complete. 2023-11-27T15:19:45.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.640+0000 7fdb34c8ab80 20 client.25540 trim_cache size 0 max 16384 2023-11-27T15:19:45.651 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:19:45.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.641+0000 7fdb28afe700 1 -- 172.21.15.52:0/2150236229 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fdb100a12c0 msgr2=0x7fdb100a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:45.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.641+0000 7fdb28afe700 1 --2- 172.21.15.52:0/2150236229 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fdb100a12c0 0x7fdb100a3770 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009c60 tx=0x7fdb14009260 comp rx=0 tx=0).stop 2023-11-27T15:19:45.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.641+0000 7fdb28afe700 1 -- 172.21.15.52:0/2150236229 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb241055d0 msgr2=0x7fdb2419aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:45.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.641+0000 7fdb28afe700 1 --2- 172.21.15.52:0/2150236229 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb241055d0 0x7fdb2419aeb0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c007870 tx=0x7fdb1c0078a0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.641+0000 7fdb28afe700 1 -- 172.21.15.52:0/2150236229 shutdown_connections 2023-11-27T15:19:45.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.641+0000 7fdb28afe700 1 --2- 172.21.15.52:0/2150236229 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fdb100a12c0 0x7fdb100a3770 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.641+0000 7fdb28afe700 1 --2- 172.21.15.52:0/2150236229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb240feaf0 0x7fdb24196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.641+0000 7fdb28afe700 1 --2- 172.21.15.52:0/2150236229 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb241055d0 0x7fdb2419aeb0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.641+0000 7fdb28afe700 1 --2- 172.21.15.52:0/2150236229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb240ff490 0x7fdb24196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.641+0000 7fdb28afe700 1 -- 172.21.15.52:0/2150236229 >> 172.21.15.52:0/2150236229 conn(0x7fdb24075920 msgr2=0x7fdb240fc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:45.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.641+0000 7fdb28afe700 1 -- 172.21.15.52:0/2150236229 shutdown_connections 2023-11-27T15:19:45.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:45.641+0000 7fdb28afe700 1 -- 172.21.15.52:0/2150236229 wait complete. 2023-11-27T15:19:45.708 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-27T15:19:46.043 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:46.043 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:46.103 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:46.103 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:46.107 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:46.107 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:46.108 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 76 -lt 100 ']' 2023-11-27T15:19:46.108 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 76 + 1 2023-11-27T15:19:46.117 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=77 2023-11-27T15:19:46.117 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:46.131 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:46.132 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:46.133 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:46.133 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:46.133 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:46.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:46 smithi052 ceph-mon[99056]: pgmap v58: 97 pgs: 97 active+clean; 2.4 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-11-27T15:19:46.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:46 smithi150 ceph-mon[100559]: pgmap v58: 97 pgs: 97 active+clean; 2.4 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-11-27T15:19:46.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:46 smithi102 ceph-mon[100296]: pgmap v58: 97 pgs: 97 active+clean; 2.4 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-11-27T15:19:46.659 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:46.660 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:46.828 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:46.828 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:46.868 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:46.868 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:46.868 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 77 -lt 100 ']' 2023-11-27T15:19:46.868 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 77 + 1 2023-11-27T15:19:46.899 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=78 2023-11-27T15:19:46.899 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:46.947 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:46.948 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:46.949 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:46.949 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:46.949 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:47.793 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:47.793 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:47.994 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:47.994 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:48.049 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:48.050 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:48.050 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 78 -lt 100 ']' 2023-11-27T15:19:48.050 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 78 + 1 2023-11-27T15:19:48.099 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=79 2023-11-27T15:19:48.099 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:48.124 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:48.124 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:48.125 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:48.125 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:48.125 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:48.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:48 smithi052 ceph-mon[99056]: pgmap v59: 97 pgs: 97 active+clean; 2.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 10 op/s 2023-11-27T15:19:48.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:48 smithi150 ceph-mon[100559]: pgmap v59: 97 pgs: 97 active+clean; 2.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 10 op/s 2023-11-27T15:19:48.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:48 smithi102 ceph-mon[100296]: pgmap v59: 97 pgs: 97 active+clean; 2.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 10 op/s 2023-11-27T15:19:48.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.618+0000 7f6d7a2e8700 1 -- 172.21.15.52:0/3244218205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d740ffdf0 msgr2=0x7f6d74103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:48.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.618+0000 7f6d7a2e8700 1 --2- 172.21.15.52:0/3244218205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d740ffdf0 0x7f6d74103db0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6d680099c0 tx=0x7f6d68009cd0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.618+0000 7f6d7a2e8700 1 -- 172.21.15.52:0/3244218205 shutdown_connections 2023-11-27T15:19:48.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.618+0000 7f6d7a2e8700 1 --2- 172.21.15.52:0/3244218205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d740ff450 0x7f6d740ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.618+0000 7f6d7a2e8700 1 --2- 172.21.15.52:0/3244218205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d74104490 0x7f6d7410c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.618+0000 7f6d7a2e8700 1 --2- 172.21.15.52:0/3244218205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d740ffdf0 0x7f6d74103db0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.618+0000 7f6d7a2e8700 1 -- 172.21.15.52:0/3244218205 >> 172.21.15.52:0/3244218205 conn(0x7f6d740770b0 msgr2=0x7f6d740774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:48.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.618+0000 7f6d7a2e8700 1 -- 172.21.15.52:0/3244218205 shutdown_connections 2023-11-27T15:19:48.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.618+0000 7f6d7a2e8700 1 -- 172.21.15.52:0/3244218205 wait complete. 2023-11-27T15:19:48.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.619+0000 7f6d7a2e8700 1 Processor -- start 2023-11-27T15:19:48.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.619+0000 7f6d7a2e8700 1 -- start start 2023-11-27T15:19:48.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.619+0000 7f6d7a2e8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d740ff450 0x7f6d74100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:48.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.619+0000 7f6d7a2e8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d740ffdf0 0x7f6d74101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:48.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.619+0000 7f6d7a2e8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d74104490 0x7f6d741a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:48.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.619+0000 7f6d7a2e8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d74101bc0 con 0x7f6d740ff450 2023-11-27T15:19:48.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.619+0000 7f6d7a2e8700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d741a4b40 con 0x7f6d74104490 2023-11-27T15:19:48.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.619+0000 7f6d7a2e8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d741a4c80 con 0x7f6d740ffdf0 2023-11-27T15:19:48.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.620+0000 7f6d73fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d740ff450 0x7f6d74100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:48.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.620+0000 7f6d73fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d740ff450 0x7f6d74100e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53008/0 (socket says 172.21.15.52:53008) 2023-11-27T15:19:48.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.620+0000 7f6d73fff700 1 -- 172.21.15.52:0/89796526 learned_addr learned my addr 172.21.15.52:0/89796526 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:48.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.620+0000 7f6d78885700 1 --2- 172.21.15.52:0/89796526 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d74104490 0x7f6d741a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:48.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.620+0000 7f6d737fe700 1 --2- 172.21.15.52:0/89796526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d740ffdf0 0x7f6d74101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:48.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.620+0000 7f6d73fff700 1 -- 172.21.15.52:0/89796526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d740ffdf0 msgr2=0x7f6d74101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:48.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.620+0000 7f6d73fff700 1 --2- 172.21.15.52:0/89796526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d740ffdf0 0x7f6d74101390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.620+0000 7f6d73fff700 1 -- 172.21.15.52:0/89796526 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d74104490 msgr2=0x7f6d741a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:48.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.620+0000 7f6d73fff700 1 --2- 172.21.15.52:0/89796526 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d74104490 0x7f6d741a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.620+0000 7f6d73fff700 1 -- 172.21.15.52:0/89796526 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d680095f0 con 0x7f6d740ff450 2023-11-27T15:19:48.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.620+0000 7f6d737fe700 1 --2- 172.21.15.52:0/89796526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d740ffdf0 0x7f6d74101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:19:48.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.620+0000 7f6d73fff700 1 --2- 172.21.15.52:0/89796526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d740ff450 0x7f6d74100e50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400e9a0 tx=0x7f6d6400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:48.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.620+0000 7f6d717fa700 1 -- 172.21.15.52:0/89796526 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64009790 con 0x7f6d740ff450 2023-11-27T15:19:48.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.621+0000 7f6d7a2e8700 1 -- 172.21.15.52:0/89796526 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d7410f210 con 0x7f6d740ff450 2023-11-27T15:19:48.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.621+0000 7f6d717fa700 1 -- 172.21.15.52:0/89796526 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d6400cbd0 con 0x7f6d740ff450 2023-11-27T15:19:48.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.621+0000 7f6d717fa700 1 -- 172.21.15.52:0/89796526 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64018680 con 0x7f6d740ff450 2023-11-27T15:19:48.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.621+0000 7f6d7a2e8700 1 -- 172.21.15.52:0/89796526 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d7410f080 con 0x7f6d740ff450 2023-11-27T15:19:48.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.623+0000 7f6d717fa700 1 -- 172.21.15.52:0/89796526 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f6d6400cd40 con 0x7f6d740ff450 2023-11-27T15:19:48.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.624+0000 7f6d717fa700 1 --2- 172.21.15.52:0/89796526 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f6d5c0a12c0 0x7f6d5c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:48.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.624+0000 7f6d737fe700 1 --2- 172.21.15.52:0/89796526 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f6d5c0a12c0 0x7f6d5c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:48.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.624+0000 7f6d717fa700 1 -- 172.21.15.52:0/89796526 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f6d64014070 con 0x7f6d740ff450 2023-11-27T15:19:48.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.625+0000 7f6d737fe700 1 --2- 172.21.15.52:0/89796526 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f6d5c0a12c0 0x7f6d5c0a3770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6d680098e0 tx=0x7f6d6801a040 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:48.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.694+0000 7f6d86474b80 1 Processor -- start 2023-11-27T15:19:48.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.694+0000 7f6d86474b80 1 -- start start 2023-11-27T15:19:48.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.694+0000 7f6d86474b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd3cd20b60 0x55cd3cd20f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:48.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.694+0000 7f6d86474b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd3cd21470 0x55cd3cd25370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:48.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.694+0000 7f6d86474b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd3cd258b0 0x55cd3cd25d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:48.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.694+0000 7f6d86474b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd3cc26060 con 0x55cd3cd21470 2023-11-27T15:19:48.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.694+0000 7f6d86474b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd3cc788a0 con 0x55cd3cd20b60 2023-11-27T15:19:48.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.694+0000 7f6d86474b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd3cc71d30 con 0x55cd3cd258b0 2023-11-27T15:19:48.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.694+0000 7f6d737fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd3cd21470 0x55cd3cd25370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:48.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.694+0000 7f6d737fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd3cd21470 0x55cd3cd25370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53016/0 (socket says 172.21.15.52:53016) 2023-11-27T15:19:48.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.694+0000 7f6d737fe700 1 -- 172.21.15.52:0/3580906593 learned_addr learned my addr 172.21.15.52:0/3580906593 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:48.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.694+0000 7f6d73fff700 1 --2- 172.21.15.52:0/3580906593 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd3cd20b60 0x55cd3cd20f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:48.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.694+0000 7f6d78885700 1 --2- 172.21.15.52:0/3580906593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd3cd258b0 0x55cd3cd25d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:48.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.694+0000 7f6d737fe700 1 -- 172.21.15.52:0/3580906593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd3cd258b0 msgr2=0x55cd3cd25d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:48.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.694+0000 7f6d737fe700 1 --2- 172.21.15.52:0/3580906593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd3cd258b0 0x55cd3cd25d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.694+0000 7f6d737fe700 1 -- 172.21.15.52:0/3580906593 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd3cd20b60 msgr2=0x55cd3cd20f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:48.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.694+0000 7f6d737fe700 1 --2- 172.21.15.52:0/3580906593 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd3cd20b60 0x55cd3cd20f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.694+0000 7f6d737fe700 1 -- 172.21.15.52:0/3580906593 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd3caf3630 con 0x55cd3cd21470 2023-11-27T15:19:48.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.694+0000 7f6d78885700 1 --2- 172.21.15.52:0/3580906593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd3cd258b0 0x55cd3cd25d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:19:48.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.694+0000 7f6d737fe700 1 --2- 172.21.15.52:0/3580906593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd3cd21470 0x55cd3cd25370 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6d680099c0 tx=0x7f6d6801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf440d73bcb1f46 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:48.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.694+0000 7f6d5a7fc700 1 -- 172.21.15.52:0/3580906593 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6802a070 con 0x55cd3cd21470 2023-11-27T15:19:48.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.695+0000 7f6d5a7fc700 1 -- 172.21.15.52:0/3580906593 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d6800be50 con 0x55cd3cd21470 2023-11-27T15:19:48.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.695+0000 7f6d5a7fc700 1 -- 172.21.15.52:0/3580906593 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6801d040 con 0x55cd3cd21470 2023-11-27T15:19:48.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.695+0000 7f6d86474b80 1 -- 172.21.15.52:0/3580906593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd3cd21470 msgr2=0x55cd3cd25370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:48.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.695+0000 7f6d86474b80 1 --2- 172.21.15.52:0/3580906593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd3cd21470 0x55cd3cd25370 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6d680099c0 tx=0x7f6d6801e000 comp rx=0 tx=0).stop 2023-11-27T15:19:48.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.695+0000 7f6d86474b80 1 -- 172.21.15.52:0/3580906593 shutdown_connections 2023-11-27T15:19:48.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.695+0000 7f6d86474b80 1 --2- 172.21.15.52:0/3580906593 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd3cd20b60 0x55cd3cd20f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.695+0000 7f6d86474b80 1 --2- 172.21.15.52:0/3580906593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd3cd258b0 0x55cd3cd25d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.695+0000 7f6d86474b80 1 --2- 172.21.15.52:0/3580906593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd3cd21470 0x55cd3cd25370 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.695+0000 7f6d86474b80 1 -- 172.21.15.52:0/3580906593 >> 172.21.15.52:0/3580906593 conn(0x55cd3cd16360 msgr2=0x55cd3cd1bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:48.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.696+0000 7f6d86474b80 1 -- 172.21.15.52:0/3580906593 shutdown_connections 2023-11-27T15:19:48.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.696+0000 7f6d86474b80 1 -- 172.21.15.52:0/3580906593 wait complete. 2023-11-27T15:19:48.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.715+0000 7f6d86474b80 1 -- start start 2023-11-27T15:19:48.724 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.715+0000 7f6d86474b80 1 Processor -- start 2023-11-27T15:19:48.724 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.715+0000 7f6d86474b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd3cd20b60 0x55cd3cd2cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:48.724 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.715+0000 7f6d86474b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd3cd21470 0x55cd3cd2a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:48.724 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.715+0000 7f6d86474b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd3cd258b0 0x55cd3cd2a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:48.724 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.715+0000 7f6d86474b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd3cc138e0 con 0x55cd3cd21470 2023-11-27T15:19:48.724 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.715+0000 7f6d86474b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd3cc13580 con 0x55cd3cd20b60 2023-11-27T15:19:48.724 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.715+0000 7f6d86474b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd3caf5920 con 0x55cd3cd258b0 2023-11-27T15:19:48.724 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.715+0000 7f6d86474b80 10 client.0 _ll_register_callbacks cb 0x55cd3cc31780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:19:48.724 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.715+0000 7f6d737fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd3cd21470 0x55cd3cd2a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:48.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.715+0000 7f6d737fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd3cd21470 0x55cd3cd2a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53024/0 (socket says 172.21.15.52:53024) 2023-11-27T15:19:48.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.715+0000 7f6d737fe700 1 -- 172.21.15.52:0/1856281775 learned_addr learned my addr 172.21.15.52:0/1856281775 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:48.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.715+0000 7f6d73fff700 1 --2- 172.21.15.52:0/1856281775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd3cd20b60 0x55cd3cd2cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:48.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.715+0000 7f6d78885700 1 --2- 172.21.15.52:0/1856281775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd3cd258b0 0x55cd3cd2a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:48.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.715+0000 7f6d737fe700 1 -- 172.21.15.52:0/1856281775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd3cd258b0 msgr2=0x55cd3cd2a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:48.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.715+0000 7f6d737fe700 1 --2- 172.21.15.52:0/1856281775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd3cd258b0 0x55cd3cd2a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.715+0000 7f6d737fe700 1 -- 172.21.15.52:0/1856281775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd3cd20b60 msgr2=0x55cd3cd2cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:48.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.715+0000 7f6d737fe700 1 --2- 172.21.15.52:0/1856281775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd3cd20b60 0x55cd3cd2cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.715+0000 7f6d737fe700 1 -- 172.21.15.52:0/1856281775 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d7410c8a0 con 0x55cd3cd21470 2023-11-27T15:19:48.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.715+0000 7f6d73fff700 1 --2- 172.21.15.52:0/1856281775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd3cd20b60 0x55cd3cd2cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:48.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.715+0000 7f6d737fe700 1 --2- 172.21.15.52:0/1856281775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd3cd21470 0x55cd3cd2a2d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6d680058e0 tx=0x7f6d6800fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac4dad7b8e5909d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:48.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.716+0000 7f6d537fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:19:48.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.716+0000 7f6d537fe700 1 -- 172.21.15.52:0/1856281775 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6802a040 con 0x55cd3cd21470 2023-11-27T15:19:48.725 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.716+0000 7f6d537fe700 1 -- 172.21.15.52:0/1856281775 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f6d68005160 con 0x55cd3cd21470 2023-11-27T15:19:48.726 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.716+0000 7f6d86474b80 1 -- 172.21.15.52:0/1856281775 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd3cd13b20 con 0x55cd3cd21470 2023-11-27T15:19:48.726 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.716+0000 7f6d537fe700 1 -- 172.21.15.52:0/1856281775 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d68004c80 con 0x55cd3cd21470 2023-11-27T15:19:48.726 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.716+0000 7f6d86474b80 10 client.15828 fetch_fsmap learned FSMap version 23 2023-11-27T15:19:48.726 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.716+0000 7f6d86474b80 1 -- 172.21.15.52:0/1856281775 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55cd3cc38190 con 0x55cd3cd21470 2023-11-27T15:19:48.726 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.716+0000 7f6d537fe700 1 -- 172.21.15.52:0/1856281775 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cd3cd13b20 con 0x55cd3cd21470 2023-11-27T15:19:48.726 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.716+0000 7f6d537fe700 1 -- 172.21.15.52:0/1856281775 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6d68023b90 con 0x55cd3cd21470 2023-11-27T15:19:48.726 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.716+0000 7f6d86474b80 10 client.15828 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:19:48.726 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.716+0000 7f6d86474b80 10 client.15828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:19:48.726 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.716+0000 7f6d86474b80 20 client.15828 populate_metadata read hostname 'smithi052' 2023-11-27T15:19:48.726 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.716+0000 7f6d86474b80 1 --2- 172.21.15.52:0/1856281775 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55cd3cd3a250 0x55cd3cd3c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:48.726 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.716+0000 7f6d86474b80 4 client.15828 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:19:48.726 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.716+0000 7f6d86474b80 1 -- 172.21.15.52:0/1856281775 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd3cc340a0 con 0x55cd3cd3a250 2023-11-27T15:19:48.726 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.717+0000 7f6d73fff700 1 --2- 172.21.15.52:0/1856281775 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55cd3cd3a250 0x55cd3cd3c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:48.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.717+0000 7f6d73fff700 1 --2- 172.21.15.52:0/1856281775 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55cd3cd3a250 0x55cd3cd3c630 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d51e0b587a80fc2d server_cookie=1bd55a1b38c87ca7 in_seq=0 out_seq=0 2023-11-27T15:19:48.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.717+0000 7f6d537fe700 10 client.15828 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:48.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.718+0000 7f6d537fe700 1 -- 172.21.15.52:0/1856281775 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cd3cc340a0 con 0x55cd3cd3a250 2023-11-27T15:19:48.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.718+0000 7f6d537fe700 10 client.15828 handle_command_reply: tid=0 2023-11-27T15:19:48.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.718+0000 7f6d86474b80 1 client.15828 shutdown 2023-11-27T15:19:48.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.718+0000 7f6d86474b80 1 -- 172.21.15.52:0/1856281775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd3cd21470 msgr2=0x55cd3cd2a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:48.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.718+0000 7f6d86474b80 1 --2- 172.21.15.52:0/1856281775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd3cd21470 0x55cd3cd2a2d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6d680058e0 tx=0x7f6d6800fb60 comp rx=0 tx=0).stop 2023-11-27T15:19:48.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.718+0000 7f6d86474b80 1 -- 172.21.15.52:0/1856281775 shutdown_connections 2023-11-27T15:19:48.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.718+0000 7f6d86474b80 1 --2- 172.21.15.52:0/1856281775 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55cd3cd3a250 0x55cd3cd3c630 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.718+0000 7f6d86474b80 1 --2- 172.21.15.52:0/1856281775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd3cd20b60 0x55cd3cd2cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.718+0000 7f6d86474b80 1 --2- 172.21.15.52:0/1856281775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd3cd258b0 0x55cd3cd2a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.718+0000 7f6d537fe700 0 client.15828 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:48.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.718+0000 7f6d86474b80 1 --2- 172.21.15.52:0/1856281775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd3cd21470 0x55cd3cd2a2d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.718+0000 7f6d86474b80 1 -- 172.21.15.52:0/1856281775 >> 172.21.15.52:0/1856281775 conn(0x55cd3cd16360 msgr2=0x55cd3cd26d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:48.728 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.718+0000 7f6d86474b80 1 -- 172.21.15.52:0/1856281775 shutdown_connections 2023-11-27T15:19:48.728 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.718+0000 7f6d86474b80 1 -- 172.21.15.52:0/1856281775 wait complete. 2023-11-27T15:19:48.728 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.718+0000 7f6d86474b80 20 client.15828 trim_cache size 0 max 16384 2023-11-27T15:19:48.744 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.732+0000 7f6d86474b80 1 Processor -- start 2023-11-27T15:19:48.744 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.732+0000 7f6d86474b80 1 -- start start 2023-11-27T15:19:48.744 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.732+0000 7f6d86474b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd3cd20b60 0x55cd3cd318e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:48.744 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.732+0000 7f6d86474b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd3cd21470 0x55cd3cd2cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:48.744 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.732+0000 7f6d86474b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd3cd258b0 0x55cd3cd1b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:48.744 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.732+0000 7f6d86474b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd3cd24200 con 0x55cd3cd20b60 2023-11-27T15:19:48.744 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.732+0000 7f6d86474b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd3cd21950 con 0x55cd3cd21470 2023-11-27T15:19:48.745 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.732+0000 7f6d86474b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd3cd1d1c0 con 0x55cd3cd258b0 2023-11-27T15:19:48.745 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.732+0000 7f6d73fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd3cd20b60 0x55cd3cd318e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:48.745 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.732+0000 7f6d73fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd3cd20b60 0x55cd3cd318e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53030/0 (socket says 172.21.15.52:53030) 2023-11-27T15:19:48.745 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.732+0000 7f6d73fff700 1 -- 172.21.15.52:0/3042613276 learned_addr learned my addr 172.21.15.52:0/3042613276 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:48.745 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.732+0000 7f6d737fe700 1 --2- 172.21.15.52:0/3042613276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd3cd21470 0x55cd3cd2cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:48.745 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.733+0000 7f6d73fff700 1 -- 172.21.15.52:0/3042613276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd3cd258b0 msgr2=0x55cd3cd1b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:48.746 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.733+0000 7f6d73fff700 1 --2- 172.21.15.52:0/3042613276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd3cd258b0 0x55cd3cd1b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.746 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.733+0000 7f6d73fff700 1 -- 172.21.15.52:0/3042613276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd3cd21470 msgr2=0x55cd3cd2cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:48.746 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.733+0000 7f6d73fff700 1 --2- 172.21.15.52:0/3042613276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd3cd21470 0x55cd3cd2cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.746 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.733+0000 7f6d73fff700 1 -- 172.21.15.52:0/3042613276 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd3cd25f50 con 0x55cd3cd20b60 2023-11-27T15:19:48.746 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.733+0000 7f6d73fff700 1 --2- 172.21.15.52:0/3042613276 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd3cd20b60 0x55cd3cd318e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6d64099fd0 tx=0x7f6d6401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2017c2f56fd4a13 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:48.746 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.733+0000 7f6d5affd700 1 -- 172.21.15.52:0/3042613276 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d640a2490 con 0x55cd3cd20b60 2023-11-27T15:19:48.746 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.733+0000 7f6d5affd700 1 -- 172.21.15.52:0/3042613276 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d640983e0 con 0x55cd3cd20b60 2023-11-27T15:19:48.746 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.733+0000 7f6d5affd700 1 -- 172.21.15.52:0/3042613276 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64098fa0 con 0x55cd3cd20b60 2023-11-27T15:19:48.746 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.734+0000 7f6d86474b80 1 -- 172.21.15.52:0/3042613276 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd3cd20b60 msgr2=0x55cd3cd318e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:48.746 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.734+0000 7f6d86474b80 1 --2- 172.21.15.52:0/3042613276 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd3cd20b60 0x55cd3cd318e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6d64099fd0 tx=0x7f6d6401f000 comp rx=0 tx=0).stop 2023-11-27T15:19:48.746 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.734+0000 7f6d86474b80 1 -- 172.21.15.52:0/3042613276 shutdown_connections 2023-11-27T15:19:48.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.734+0000 7f6d86474b80 1 --2- 172.21.15.52:0/3042613276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd3cd258b0 0x55cd3cd1b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.734+0000 7f6d86474b80 1 --2- 172.21.15.52:0/3042613276 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd3cd20b60 0x55cd3cd318e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.734+0000 7f6d86474b80 1 --2- 172.21.15.52:0/3042613276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd3cd21470 0x55cd3cd2cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.734+0000 7f6d86474b80 1 -- 172.21.15.52:0/3042613276 >> 172.21.15.52:0/3042613276 conn(0x55cd3cd16360 msgr2=0x55cd3cd36b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:48.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.734+0000 7f6d86474b80 1 -- 172.21.15.52:0/3042613276 shutdown_connections 2023-11-27T15:19:48.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.734+0000 7f6d86474b80 1 -- 172.21.15.52:0/3042613276 wait complete. 2023-11-27T15:19:48.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.734+0000 7f6d86474b80 1 -- start start 2023-11-27T15:19:48.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.734+0000 7f6d86474b80 1 Processor -- start 2023-11-27T15:19:48.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.735+0000 7f6d86474b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd3cd20b60 0x55cd3cd2f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:48.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.735+0000 7f6d86474b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd3cd21470 0x55cd3cd2fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:48.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.735+0000 7f6d86474b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd3cd258b0 0x55cd3cd30350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:48.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.735+0000 7f6d86474b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd3cd8d0a0 con 0x55cd3cd20b60 2023-11-27T15:19:48.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.735+0000 7f6d86474b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd3cd8d1e0 con 0x55cd3cd258b0 2023-11-27T15:19:48.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.735+0000 7f6d86474b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd3cd8d3a0 con 0x55cd3cd21470 2023-11-27T15:19:48.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.735+0000 7f6d86474b80 10 client.0 _ll_register_callbacks cb 0x7f6d64010aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:19:48.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.735+0000 7f6d73fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd3cd20b60 0x55cd3cd2f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:48.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.735+0000 7f6d73fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd3cd20b60 0x55cd3cd2f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53036/0 (socket says 172.21.15.52:53036) 2023-11-27T15:19:48.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.735+0000 7f6d73fff700 1 -- 172.21.15.52:0/2990984575 learned_addr learned my addr 172.21.15.52:0/2990984575 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:48.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.735+0000 7f6d737fe700 1 --2- 172.21.15.52:0/2990984575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd3cd21470 0x55cd3cd2fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:48.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.735+0000 7f6d78885700 1 --2- 172.21.15.52:0/2990984575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd3cd258b0 0x55cd3cd30350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:48.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.735+0000 7f6d73fff700 1 -- 172.21.15.52:0/2990984575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd3cd21470 msgr2=0x55cd3cd2fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:48.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.735+0000 7f6d73fff700 1 --2- 172.21.15.52:0/2990984575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd3cd21470 0x55cd3cd2fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.735+0000 7f6d73fff700 1 -- 172.21.15.52:0/2990984575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd3cd258b0 msgr2=0x55cd3cd30350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:48.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.735+0000 7f6d73fff700 1 --2- 172.21.15.52:0/2990984575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd3cd258b0 0x55cd3cd30350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.735+0000 7f6d73fff700 1 -- 172.21.15.52:0/2990984575 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d7410f080 con 0x55cd3cd20b60 2023-11-27T15:19:48.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.735+0000 7f6d78885700 1 --2- 172.21.15.52:0/2990984575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd3cd258b0 0x55cd3cd30350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:19:48.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.735+0000 7f6d73fff700 1 --2- 172.21.15.52:0/2990984575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd3cd20b60 0x55cd3cd2f8d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6d64096540 tx=0x7f6d6400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a00774821426ae server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:48.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.735+0000 7f6d537fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:19:48.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.736+0000 7f6d537fe700 1 -- 172.21.15.52:0/2990984575 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64099530 con 0x55cd3cd20b60 2023-11-27T15:19:48.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.736+0000 7f6d537fe700 1 -- 172.21.15.52:0/2990984575 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f6d64099690 con 0x55cd3cd20b60 2023-11-27T15:19:48.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.736+0000 7f6d86474b80 1 -- 172.21.15.52:0/2990984575 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d6800be50 con 0x55cd3cd20b60 2023-11-27T15:19:48.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.736+0000 7f6d537fe700 1 -- 172.21.15.52:0/2990984575 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d640ca310 con 0x55cd3cd20b60 2023-11-27T15:19:48.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.736+0000 7f6d86474b80 10 client.15834 fetch_fsmap learned FSMap version 23 2023-11-27T15:19:48.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.736+0000 7f6d86474b80 1 -- 172.21.15.52:0/2990984575 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55cd3ca9fde0 con 0x55cd3cd20b60 2023-11-27T15:19:48.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.736+0000 7f6d537fe700 1 -- 172.21.15.52:0/2990984575 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d64063020 con 0x55cd3cd20b60 2023-11-27T15:19:48.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.736+0000 7f6d537fe700 1 -- 172.21.15.52:0/2990984575 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6d64099160 con 0x55cd3cd20b60 2023-11-27T15:19:48.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.736+0000 7f6d86474b80 10 client.15834 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:19:48.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.736+0000 7f6d86474b80 10 client.15834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:19:48.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.736+0000 7f6d86474b80 20 client.15834 populate_metadata read hostname 'smithi052' 2023-11-27T15:19:48.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.736+0000 7f6d86474b80 1 --2- 172.21.15.52:0/2990984575 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55cd3cd8e870 0x55cd3cd90c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:48.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.736+0000 7f6d86474b80 4 client.15834 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:19:48.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.736+0000 7f6d86474b80 1 -- 172.21.15.52:0/2990984575 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cd3cc40660 con 0x55cd3cd8e870 2023-11-27T15:19:48.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.737+0000 7f6d737fe700 1 --2- 172.21.15.52:0/2990984575 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55cd3cd8e870 0x55cd3cd90c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:48.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.737+0000 7f6d737fe700 1 --2- 172.21.15.52:0/2990984575 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55cd3cd8e870 0x55cd3cd90c80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=facf44140fc3bd2f server_cookie=af7025e899d3afda in_seq=0 out_seq=0 2023-11-27T15:19:48.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.737+0000 7f6d537fe700 10 client.15834 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:48.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.737+0000 7f6d537fe700 1 -- 172.21.15.52:0/2990984575 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cd3cc40660 con 0x55cd3cd8e870 2023-11-27T15:19:48.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.737+0000 7f6d537fe700 10 client.15834 handle_command_reply: tid=0 2023-11-27T15:19:48.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.737+0000 7f6d86474b80 1 client.15834 shutdown 2023-11-27T15:19:48.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.737+0000 7f6d86474b80 1 -- 172.21.15.52:0/2990984575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd3cd20b60 msgr2=0x55cd3cd2f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:48.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.737+0000 7f6d86474b80 1 --2- 172.21.15.52:0/2990984575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd3cd20b60 0x55cd3cd2f8d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6d64096540 tx=0x7f6d6400ef60 comp rx=0 tx=0).stop 2023-11-27T15:19:48.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.737+0000 7f6d86474b80 1 -- 172.21.15.52:0/2990984575 shutdown_connections 2023-11-27T15:19:48.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.737+0000 7f6d86474b80 1 --2- 172.21.15.52:0/2990984575 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55cd3cd8e870 0x55cd3cd90c80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.737+0000 7f6d86474b80 1 --2- 172.21.15.52:0/2990984575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd3cd258b0 0x55cd3cd30350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.737+0000 7f6d86474b80 1 --2- 172.21.15.52:0/2990984575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd3cd20b60 0x55cd3cd2f8d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.737+0000 7f6d537fe700 0 client.15834 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:48.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.737+0000 7f6d86474b80 1 --2- 172.21.15.52:0/2990984575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cd3cd21470 0x55cd3cd2fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.737+0000 7f6d86474b80 1 -- 172.21.15.52:0/2990984575 >> 172.21.15.52:0/2990984575 conn(0x55cd3cd16360 msgr2=0x55cd3cd1f960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:48.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.738+0000 7f6d86474b80 1 -- 172.21.15.52:0/2990984575 shutdown_connections 2023-11-27T15:19:48.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.738+0000 7f6d86474b80 1 -- 172.21.15.52:0/2990984575 wait complete. 2023-11-27T15:19:48.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.738+0000 7f6d86474b80 20 client.15834 trim_cache size 0 max 16384 2023-11-27T15:19:48.751 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:19:48.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.738+0000 7f6d7a2e8700 1 -- 172.21.15.52:0/89796526 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f6d5c0a12c0 msgr2=0x7f6d5c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:48.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.738+0000 7f6d7a2e8700 1 --2- 172.21.15.52:0/89796526 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f6d5c0a12c0 0x7f6d5c0a3770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6d680098e0 tx=0x7f6d6801a040 comp rx=0 tx=0).stop 2023-11-27T15:19:48.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.738+0000 7f6d7a2e8700 1 -- 172.21.15.52:0/89796526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d740ff450 msgr2=0x7f6d74100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:48.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.738+0000 7f6d7a2e8700 1 --2- 172.21.15.52:0/89796526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d740ff450 0x7f6d74100e50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400e9a0 tx=0x7f6d6400ecb0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.738+0000 7f6d7a2e8700 1 -- 172.21.15.52:0/89796526 shutdown_connections 2023-11-27T15:19:48.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.738+0000 7f6d7a2e8700 1 --2- 172.21.15.52:0/89796526 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f6d5c0a12c0 0x7f6d5c0a3770 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.738+0000 7f6d7a2e8700 1 --2- 172.21.15.52:0/89796526 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6d74104490 0x7f6d741a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.738+0000 7f6d7a2e8700 1 --2- 172.21.15.52:0/89796526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d740ff450 0x7f6d74100e50 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.738+0000 7f6d7a2e8700 1 --2- 172.21.15.52:0/89796526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d740ffdf0 0x7f6d74101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.738+0000 7f6d7a2e8700 1 -- 172.21.15.52:0/89796526 >> 172.21.15.52:0/89796526 conn(0x7f6d740770b0 msgr2=0x7f6d74067f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:48.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.738+0000 7f6d7a2e8700 1 -- 172.21.15.52:0/89796526 shutdown_connections 2023-11-27T15:19:48.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:48.738+0000 7f6d7a2e8700 1 -- 172.21.15.52:0/89796526 wait complete. 2023-11-27T15:19:48.817 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:48.817 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:48.944 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:48.944 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:48.948 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:48.948 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:48.949 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 79 -lt 100 ']' 2023-11-27T15:19:48.949 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 79 + 1 2023-11-27T15:19:48.960 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=80 2023-11-27T15:19:48.960 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:48.991 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:48.991 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:48.992 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:48.992 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:48.992 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:49.360 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:49.397 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:49.428 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:49.428 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:49.441 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:49.442 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:49.442 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 80 -lt 100 ']' 2023-11-27T15:19:49.442 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 80 + 1 2023-11-27T15:19:49.472 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=81 2023-11-27T15:19:49.472 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:49.501 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:49.501 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:49.502 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:49.502 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:49.502 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:49.833 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-27T15:19:49.833 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-27T15:19:49.925 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:49.926 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:50.036 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:50.036 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:50.056 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:50.056 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:50.056 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 81 -lt 100 ']' 2023-11-27T15:19:50.057 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 81 + 1 2023-11-27T15:19:50.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:49 smithi102 ceph-mon[100296]: pgmap v60: 97 pgs: 97 active+clean; 2.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 6 op/s 2023-11-27T15:19:50.082 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=82 2023-11-27T15:19:50.082 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:50.092 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:50.092 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:50.093 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:50.093 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:50.094 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:50.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:49 smithi052 ceph-mon[99056]: pgmap v60: 97 pgs: 97 active+clean; 2.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 6 op/s 2023-11-27T15:19:50.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:49 smithi150 ceph-mon[100559]: pgmap v60: 97 pgs: 97 active+clean; 2.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 6 op/s 2023-11-27T15:19:50.501 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:50.501 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:50.547 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:50.548 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:50.553 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:50.553 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:50.553 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 82 -lt 100 ']' 2023-11-27T15:19:50.553 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 82 + 1 2023-11-27T15:19:50.623 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=83 2023-11-27T15:19:50.623 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:50.717 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:50.718 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:50.719 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:50.719 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:50.720 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:51.184 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:51.184 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:51.302 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:51.302 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:51.315 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:51.315 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:51.315 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 83 -lt 100 ']' 2023-11-27T15:19:51.315 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 83 + 1 2023-11-27T15:19:51.356 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=84 2023-11-27T15:19:51.356 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:51.427 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:51.427 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:51.428 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:51.428 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:51.428 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:52.193 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:52.193 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:52.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:52 smithi052 ceph-mon[99056]: pgmap v61: 97 pgs: 97 active+clean; 2.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 10 op/s 2023-11-27T15:19:52.452 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:52.452 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:52.492 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:52.492 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:52.492 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 84 -lt 100 ']' 2023-11-27T15:19:52.492 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 84 + 1 2023-11-27T15:19:52.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:52 smithi150 ceph-mon[100559]: pgmap v61: 97 pgs: 97 active+clean; 2.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 10 op/s 2023-11-27T15:19:52.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:52 smithi102 ceph-mon[100296]: pgmap v61: 97 pgs: 97 active+clean; 2.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 10 op/s 2023-11-27T15:19:52.735 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=85 2023-11-27T15:19:52.735 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:52.835 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:52.836 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:52.837 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:52.837 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:52.837 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:53.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.168+0000 7fac2b153700 1 -- 172.21.15.52:0/880237015 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac24100c50 msgr2=0x7fac24101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:53.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.168+0000 7fac2b153700 1 --2- 172.21.15.52:0/880237015 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac24100c50 0x7fac24101020 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fac140099c0 tx=0x7fac14009cd0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.168+0000 7fac2b153700 1 -- 172.21.15.52:0/880237015 shutdown_connections 2023-11-27T15:19:53.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.168+0000 7fac2b153700 1 --2- 172.21.15.52:0/880237015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac24105630 0x7fac2410d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.168+0000 7fac2b153700 1 --2- 172.21.15.52:0/880237015 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac24100c50 0x7fac24101020 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.168+0000 7fac2b153700 1 --2- 172.21.15.52:0/880237015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac241015f0 0x7fac24104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.175 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.168+0000 7fac2b153700 1 -- 172.21.15.52:0/880237015 >> 172.21.15.52:0/880237015 conn(0x7fac240775c0 msgr2=0x7fac240779c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:53.175 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.168+0000 7fac2b153700 1 -- 172.21.15.52:0/880237015 shutdown_connections 2023-11-27T15:19:53.175 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.168+0000 7fac2b153700 1 -- 172.21.15.52:0/880237015 wait complete. 2023-11-27T15:19:53.175 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.169+0000 7fac2b153700 1 Processor -- start 2023-11-27T15:19:53.176 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.169+0000 7fac2b153700 1 -- start start 2023-11-27T15:19:53.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.169+0000 7fac2b153700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac24100c50 0x7fac24196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:53.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.169+0000 7fac2b153700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac241015f0 0x7fac24196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:53.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.169+0000 7fac2b153700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac24105630 0x7fac2419adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:53.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.169+0000 7fac2b153700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2419b3f0 con 0x7fac24105630 2023-11-27T15:19:53.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.169+0000 7fac2b153700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2419b560 con 0x7fac24100c50 2023-11-27T15:19:53.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.169+0000 7fac2b153700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2419b720 con 0x7fac241015f0 2023-11-27T15:19:53.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.170+0000 7fac28eef700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac24100c50 0x7fac24196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:53.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.170+0000 7fac296f0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac24105630 0x7fac2419adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:53.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.170+0000 7fac28eef700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac24100c50 0x7fac24196410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:42708/0 (socket says 172.21.15.52:42708) 2023-11-27T15:19:53.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.170+0000 7fac28eef700 1 -- 172.21.15.52:0/1752043126 learned_addr learned my addr 172.21.15.52:0/1752043126 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:53.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.170+0000 7fac296f0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac24105630 0x7fac2419adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53054/0 (socket says 172.21.15.52:53054) 2023-11-27T15:19:53.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.170+0000 7fac23fff700 1 --2- 172.21.15.52:0/1752043126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac241015f0 0x7fac24196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:53.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.170+0000 7fac296f0700 1 -- 172.21.15.52:0/1752043126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac241015f0 msgr2=0x7fac24196950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:53.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.170+0000 7fac296f0700 1 --2- 172.21.15.52:0/1752043126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac241015f0 0x7fac24196950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.170+0000 7fac296f0700 1 -- 172.21.15.52:0/1752043126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac24100c50 msgr2=0x7fac24196410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:53.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.170+0000 7fac296f0700 1 --2- 172.21.15.52:0/1752043126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac24100c50 0x7fac24196410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.170+0000 7fac296f0700 1 -- 172.21.15.52:0/1752043126 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac140095f0 con 0x7fac24105630 2023-11-27T15:19:53.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.170+0000 7fac28eef700 1 --2- 172.21.15.52:0/1752043126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac24100c50 0x7fac24196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:53.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.170+0000 7fac296f0700 1 --2- 172.21.15.52:0/1752043126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac24105630 0x7fac2419adb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fac1c00efc0 tx=0x7fac1c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:53.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.170+0000 7fac21ffb700 1 -- 172.21.15.52:0/1752043126 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c018070 con 0x7fac24105630 2023-11-27T15:19:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.171+0000 7fac21ffb700 1 -- 172.21.15.52:0/1752043126 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac1c00f040 con 0x7fac24105630 2023-11-27T15:19:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.171+0000 7fac2b153700 1 -- 172.21.15.52:0/1752043126 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac241103a0 con 0x7fac24105630 2023-11-27T15:19:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.171+0000 7fac21ffb700 1 -- 172.21.15.52:0/1752043126 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c00cd40 con 0x7fac24105630 2023-11-27T15:19:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.171+0000 7fac2b153700 1 -- 172.21.15.52:0/1752043126 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac24110210 con 0x7fac24105630 2023-11-27T15:19:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.173+0000 7fac21ffb700 1 -- 172.21.15.52:0/1752043126 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fac1c003a00 con 0x7fac24105630 2023-11-27T15:19:53.184 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.174+0000 7fac21ffb700 1 --2- 172.21.15.52:0/1752043126 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fac100a12c0 0x7fac100a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:53.184 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.174+0000 7fac21ffb700 1 -- 172.21.15.52:0/1752043126 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fac1c0cc890 con 0x7fac24105630 2023-11-27T15:19:53.184 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.174+0000 7fac28eef700 1 --2- 172.21.15.52:0/1752043126 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fac100a12c0 0x7fac100a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:53.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.175+0000 7fac28eef700 1 --2- 172.21.15.52:0/1752043126 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fac100a12c0 0x7fac100a3770 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fac14004800 tx=0x7fac1401a040 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:53.249 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.245+0000 7fac372dfb80 1 Processor -- start 2023-11-27T15:19:53.249 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.245+0000 7fac372dfb80 1 -- start start 2023-11-27T15:19:53.251 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.246+0000 7fac372dfb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d03e2be0 0x55d0d03e2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:53.251 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.246+0000 7fac372dfb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0d03e34f0 0x55d0d03e73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:53.251 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.246+0000 7fac372dfb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d0d03e7930 0x55d0d03e7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:53.251 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.246+0000 7fac372dfb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d02e80e0 con 0x55d0d03e2be0 2023-11-27T15:19:53.251 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.246+0000 7fac372dfb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d033a920 con 0x55d0d03e34f0 2023-11-27T15:19:53.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.246+0000 7fac372dfb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d0333db0 con 0x55d0d03e7930 2023-11-27T15:19:53.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.246+0000 7fac28eef700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d03e2be0 0x55d0d03e2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:53.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.246+0000 7fac28eef700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d03e2be0 0x55d0d03e2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53058/0 (socket says 172.21.15.52:53058) 2023-11-27T15:19:53.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.246+0000 7fac28eef700 1 -- 172.21.15.52:0/3220209749 learned_addr learned my addr 172.21.15.52:0/3220209749 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:53.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.246+0000 7fac296f0700 1 --2- 172.21.15.52:0/3220209749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d0d03e7930 0x55d0d03e7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:53.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.246+0000 7fac23fff700 1 --2- 172.21.15.52:0/3220209749 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0d03e34f0 0x55d0d03e73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:53.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.246+0000 7fac28eef700 1 -- 172.21.15.52:0/3220209749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d0d03e7930 msgr2=0x55d0d03e7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:53.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.246+0000 7fac28eef700 1 --2- 172.21.15.52:0/3220209749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d0d03e7930 0x55d0d03e7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.246+0000 7fac28eef700 1 -- 172.21.15.52:0/3220209749 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0d03e34f0 msgr2=0x55d0d03e73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:53.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.246+0000 7fac28eef700 1 --2- 172.21.15.52:0/3220209749 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0d03e34f0 0x55d0d03e73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.246+0000 7fac28eef700 1 -- 172.21.15.52:0/3220209749 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0d01b56b0 con 0x55d0d03e2be0 2023-11-27T15:19:53.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.246+0000 7fac296f0700 1 --2- 172.21.15.52:0/3220209749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d0d03e7930 0x55d0d03e7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:53.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.247+0000 7fac28eef700 1 --2- 172.21.15.52:0/3220209749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d03e2be0 0x55d0d03e2fb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fac140098e0 tx=0x7fac1401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8904985ef54f008d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:53.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.247+0000 7fac1affd700 1 -- 172.21.15.52:0/3220209749 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1402a070 con 0x55d0d03e2be0 2023-11-27T15:19:53.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.247+0000 7fac1affd700 1 -- 172.21.15.52:0/3220209749 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac1401e040 con 0x55d0d03e2be0 2023-11-27T15:19:53.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.247+0000 7fac1affd700 1 -- 172.21.15.52:0/3220209749 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1402fc20 con 0x55d0d03e2be0 2023-11-27T15:19:53.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.248+0000 7fac372dfb80 1 -- 172.21.15.52:0/3220209749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d03e2be0 msgr2=0x55d0d03e2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:53.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.248+0000 7fac372dfb80 1 --2- 172.21.15.52:0/3220209749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d03e2be0 0x55d0d03e2fb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fac140098e0 tx=0x7fac1401f000 comp rx=0 tx=0).stop 2023-11-27T15:19:53.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.248+0000 7fac372dfb80 1 -- 172.21.15.52:0/3220209749 shutdown_connections 2023-11-27T15:19:53.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.248+0000 7fac372dfb80 1 --2- 172.21.15.52:0/3220209749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d0d03e7930 0x55d0d03e7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.259 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.248+0000 7fac372dfb80 1 --2- 172.21.15.52:0/3220209749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d03e2be0 0x55d0d03e2fb0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.259 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.248+0000 7fac372dfb80 1 --2- 172.21.15.52:0/3220209749 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0d03e34f0 0x55d0d03e73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.259 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.248+0000 7fac372dfb80 1 -- 172.21.15.52:0/3220209749 >> 172.21.15.52:0/3220209749 conn(0x55d0d03d83e0 msgr2=0x55d0d03ddbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:53.259 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.248+0000 7fac372dfb80 1 -- 172.21.15.52:0/3220209749 shutdown_connections 2023-11-27T15:19:53.259 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.248+0000 7fac372dfb80 1 -- 172.21.15.52:0/3220209749 wait complete. 2023-11-27T15:19:53.259 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.248+0000 7fac372dfb80 1 -- start start 2023-11-27T15:19:53.259 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.248+0000 7fac372dfb80 1 Processor -- start 2023-11-27T15:19:53.260 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.249+0000 7fac372dfb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d03e2be0 0x55d0d03ef0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:53.260 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.249+0000 7fac372dfb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d0d03e34f0 0x55d0d03ec370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:53.260 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.249+0000 7fac372dfb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0d03e7930 0x55d0d03eca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:53.260 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.249+0000 7fac372dfb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d02d5960 con 0x55d0d03e2be0 2023-11-27T15:19:53.260 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.249+0000 7fac372dfb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d02d5600 con 0x55d0d03e7930 2023-11-27T15:19:53.260 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.249+0000 7fac372dfb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d01b79a0 con 0x55d0d03e34f0 2023-11-27T15:19:53.260 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.249+0000 7fac372dfb80 10 client.0 _ll_register_callbacks cb 0x55d0d02f3800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:19:53.261 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.249+0000 7fac28eef700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d03e2be0 0x55d0d03ef0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:53.261 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.249+0000 7fac28eef700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d03e2be0 0x55d0d03ef0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53064/0 (socket says 172.21.15.52:53064) 2023-11-27T15:19:53.261 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.249+0000 7fac28eef700 1 -- 172.21.15.52:0/1277005974 learned_addr learned my addr 172.21.15.52:0/1277005974 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:53.261 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.250+0000 7fac296f0700 1 --2- 172.21.15.52:0/1277005974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0d03e7930 0x55d0d03eca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:53.261 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.250+0000 7fac23fff700 1 --2- 172.21.15.52:0/1277005974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d0d03e34f0 0x55d0d03ec370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:53.261 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.250+0000 7fac28eef700 1 -- 172.21.15.52:0/1277005974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d0d03e34f0 msgr2=0x55d0d03ec370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:53.261 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.250+0000 7fac28eef700 1 --2- 172.21.15.52:0/1277005974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d0d03e34f0 0x55d0d03ec370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.250+0000 7fac28eef700 1 -- 172.21.15.52:0/1277005974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0d03e7930 msgr2=0x55d0d03eca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:53.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.250+0000 7fac28eef700 1 --2- 172.21.15.52:0/1277005974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0d03e7930 0x55d0d03eca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.250+0000 7fac28eef700 1 -- 172.21.15.52:0/1277005974 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac2410d9d0 con 0x55d0d03e2be0 2023-11-27T15:19:53.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.250+0000 7fac28eef700 1 --2- 172.21.15.52:0/1277005974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d03e2be0 0x55d0d03ef0f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fac14009d10 tx=0x7fac1402fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=716f37c7b3b2f2c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:53.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.250+0000 7fac03fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:19:53.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.250+0000 7fac03fff700 1 -- 172.21.15.52:0/1277005974 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1402a050 con 0x55d0d03e2be0 2023-11-27T15:19:53.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.250+0000 7fac03fff700 1 -- 172.21.15.52:0/1277005974 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fac1400fa40 con 0x55d0d03e2be0 2023-11-27T15:19:53.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.250+0000 7fac372dfb80 1 -- 172.21.15.52:0/1277005974 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0d03d5ba0 con 0x55d0d03e2be0 2023-11-27T15:19:53.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.251+0000 7fac03fff700 1 -- 172.21.15.52:0/1277005974 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac1400fc00 con 0x55d0d03e2be0 2023-11-27T15:19:53.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.251+0000 7fac372dfb80 10 client.15846 fetch_fsmap learned FSMap version 23 2023-11-27T15:19:53.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.251+0000 7fac372dfb80 1 -- 172.21.15.52:0/1277005974 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d0d02fa210 con 0x55d0d03e2be0 2023-11-27T15:19:53.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.251+0000 7fac03fff700 1 -- 172.21.15.52:0/1277005974 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac14047020 con 0x55d0d03e2be0 2023-11-27T15:19:53.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.251+0000 7fac03fff700 1 -- 172.21.15.52:0/1277005974 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fac140173f0 con 0x55d0d03e2be0 2023-11-27T15:19:53.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.251+0000 7fac372dfb80 10 client.15846 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:19:53.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.251+0000 7fac372dfb80 10 client.15846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:19:53.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.251+0000 7fac372dfb80 20 client.15846 populate_metadata read hostname 'smithi052' 2023-11-27T15:19:53.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.251+0000 7fac372dfb80 1 --2- 172.21.15.52:0/1277005974 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d0d0400860 0x55d0d0402c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:53.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.251+0000 7fac372dfb80 4 client.15846 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:19:53.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.251+0000 7fac372dfb80 1 -- 172.21.15.52:0/1277005974 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0d02f6120 con 0x55d0d0400860 2023-11-27T15:19:53.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.252+0000 7fac23fff700 1 --2- 172.21.15.52:0/1277005974 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d0d0400860 0x55d0d0402c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:53.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.252+0000 7fac23fff700 1 --2- 172.21.15.52:0/1277005974 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d0d0400860 0x55d0d0402c70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef4f7fdb5b6c0f8f server_cookie=a02a67247f0bede2 in_seq=0 out_seq=0 2023-11-27T15:19:53.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.252+0000 7fac03fff700 10 client.15846 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:53.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.253+0000 7fac03fff700 1 -- 172.21.15.52:0/1277005974 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d0d02f6120 con 0x55d0d0400860 2023-11-27T15:19:53.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.253+0000 7fac03fff700 10 client.15846 handle_command_reply: tid=0 2023-11-27T15:19:53.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.253+0000 7fac372dfb80 1 client.15846 shutdown 2023-11-27T15:19:53.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.253+0000 7fac372dfb80 1 -- 172.21.15.52:0/1277005974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d03e2be0 msgr2=0x55d0d03ef0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:53.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.253+0000 7fac372dfb80 1 --2- 172.21.15.52:0/1277005974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d03e2be0 0x55d0d03ef0f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fac14009d10 tx=0x7fac1402fab0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.253+0000 7fac372dfb80 1 -- 172.21.15.52:0/1277005974 shutdown_connections 2023-11-27T15:19:53.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.253+0000 7fac372dfb80 1 --2- 172.21.15.52:0/1277005974 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d0d0400860 0x55d0d0402c70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.253+0000 7fac372dfb80 1 --2- 172.21.15.52:0/1277005974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0d03e7930 0x55d0d03eca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.253+0000 7fac372dfb80 1 --2- 172.21.15.52:0/1277005974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d03e2be0 0x55d0d03ef0f0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.253+0000 7fac03fff700 0 client.15846 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:53.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.253+0000 7fac372dfb80 1 --2- 172.21.15.52:0/1277005974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d0d03e34f0 0x55d0d03ec370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.253+0000 7fac372dfb80 1 -- 172.21.15.52:0/1277005974 >> 172.21.15.52:0/1277005974 conn(0x55d0d03d83e0 msgr2=0x55d0d03dce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:53.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.253+0000 7fac372dfb80 1 -- 172.21.15.52:0/1277005974 shutdown_connections 2023-11-27T15:19:53.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.253+0000 7fac372dfb80 1 -- 172.21.15.52:0/1277005974 wait complete. 2023-11-27T15:19:53.267 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.253+0000 7fac372dfb80 20 client.15846 trim_cache size 0 max 16384 2023-11-27T15:19:53.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.272+0000 7fac372dfb80 1 Processor -- start 2023-11-27T15:19:53.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.272+0000 7fac372dfb80 1 -- start start 2023-11-27T15:19:53.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.272+0000 7fac372dfb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0d03e2be0 0x55d0d03e20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:53.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.272+0000 7fac372dfb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d03e34f0 0x55d0d03eec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:53.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.273+0000 7fac372dfb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d0d03e7930 0x55d0d03dd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:53.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.273+0000 7fac372dfb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d03d7f60 con 0x55d0d03e34f0 2023-11-27T15:19:53.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.273+0000 7fac372dfb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d03d81f0 con 0x55d0d03e2be0 2023-11-27T15:19:53.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.273+0000 7fac372dfb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d03df240 con 0x55d0d03e7930 2023-11-27T15:19:53.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.273+0000 7fac23fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d03e34f0 0x55d0d03eec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:53.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.273+0000 7fac23fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d03e34f0 0x55d0d03eec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53074/0 (socket says 172.21.15.52:53074) 2023-11-27T15:19:53.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.273+0000 7fac23fff700 1 -- 172.21.15.52:0/348829217 learned_addr learned my addr 172.21.15.52:0/348829217 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:53.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.273+0000 7fac28eef700 1 --2- 172.21.15.52:0/348829217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0d03e2be0 0x55d0d03e20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:53.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.273+0000 7fac296f0700 1 --2- 172.21.15.52:0/348829217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d0d03e7930 0x55d0d03dd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:53.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.273+0000 7fac23fff700 1 -- 172.21.15.52:0/348829217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d0d03e7930 msgr2=0x55d0d03dd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:53.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.273+0000 7fac23fff700 1 --2- 172.21.15.52:0/348829217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d0d03e7930 0x55d0d03dd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.273+0000 7fac23fff700 1 -- 172.21.15.52:0/348829217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0d03e2be0 msgr2=0x55d0d03e20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:53.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.273+0000 7fac23fff700 1 --2- 172.21.15.52:0/348829217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0d03e2be0 0x55d0d03e20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.273+0000 7fac23fff700 1 -- 172.21.15.52:0/348829217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0d03e7fd0 con 0x55d0d03e34f0 2023-11-27T15:19:53.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.274+0000 7fac23fff700 1 --2- 172.21.15.52:0/348829217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d03e34f0 0x55d0d03eec90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fac0c0097c0 tx=0x7fac0c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78adb681469cc9f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:53.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.274+0000 7fac1b7fe700 1 -- 172.21.15.52:0/348829217 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0c017070 con 0x55d0d03e34f0 2023-11-27T15:19:53.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.274+0000 7fac1b7fe700 1 -- 172.21.15.52:0/348829217 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac0c018040 con 0x55d0d03e34f0 2023-11-27T15:19:53.285 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.274+0000 7fac1b7fe700 1 -- 172.21.15.52:0/348829217 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0c010690 con 0x55d0d03e34f0 2023-11-27T15:19:53.285 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.275+0000 7fac372dfb80 1 -- 172.21.15.52:0/348829217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d03e34f0 msgr2=0x55d0d03eec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:53.285 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.275+0000 7fac372dfb80 1 --2- 172.21.15.52:0/348829217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d03e34f0 0x55d0d03eec90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fac0c0097c0 tx=0x7fac0c009260 comp rx=0 tx=0).stop 2023-11-27T15:19:53.285 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.275+0000 7fac372dfb80 1 -- 172.21.15.52:0/348829217 shutdown_connections 2023-11-27T15:19:53.285 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.275+0000 7fac372dfb80 1 --2- 172.21.15.52:0/348829217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0d03e2be0 0x55d0d03e20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.286 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.275+0000 7fac372dfb80 1 --2- 172.21.15.52:0/348829217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d0d03e7930 0x55d0d03dd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.286 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.275+0000 7fac372dfb80 1 --2- 172.21.15.52:0/348829217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d03e34f0 0x55d0d03eec90 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.286 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.275+0000 7fac372dfb80 1 -- 172.21.15.52:0/348829217 >> 172.21.15.52:0/348829217 conn(0x55d0d03d83e0 msgr2=0x55d0d03e19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:53.286 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.275+0000 7fac372dfb80 1 -- 172.21.15.52:0/348829217 shutdown_connections 2023-11-27T15:19:53.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.275+0000 7fac372dfb80 1 -- 172.21.15.52:0/348829217 wait complete. 2023-11-27T15:19:53.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.275+0000 7fac372dfb80 1 -- start start 2023-11-27T15:19:53.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.275+0000 7fac372dfb80 1 Processor -- start 2023-11-27T15:19:53.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.276+0000 7fac372dfb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0d03e2be0 0x55d0d03f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:53.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.276+0000 7fac372dfb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d03e34f0 0x55d0d03f4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:53.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.276+0000 7fac372dfb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d0d03e7930 0x55d0d03f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:53.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.276+0000 7fac372dfb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d03f8490 con 0x55d0d03e34f0 2023-11-27T15:19:53.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.276+0000 7fac372dfb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d0454710 con 0x55d0d03e2be0 2023-11-27T15:19:53.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.276+0000 7fac372dfb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d04548d0 con 0x55d0d03e7930 2023-11-27T15:19:53.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.276+0000 7fac372dfb80 10 client.0 _ll_register_callbacks cb 0x7fac0c009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:19:53.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.276+0000 7fac23fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d03e34f0 0x55d0d03f4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:53.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.276+0000 7fac28eef700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0d03e2be0 0x55d0d03f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:53.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.276+0000 7fac23fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d03e34f0 0x55d0d03f4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53080/0 (socket says 172.21.15.52:53080) 2023-11-27T15:19:53.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.276+0000 7fac23fff700 1 -- 172.21.15.52:0/2023075032 learned_addr learned my addr 172.21.15.52:0/2023075032 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:53.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.276+0000 7fac296f0700 1 --2- 172.21.15.52:0/2023075032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d0d03e7930 0x55d0d03f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:53.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.276+0000 7fac23fff700 1 -- 172.21.15.52:0/2023075032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d0d03e7930 msgr2=0x55d0d03f4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:53.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.276+0000 7fac23fff700 1 --2- 172.21.15.52:0/2023075032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d0d03e7930 0x55d0d03f4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.276+0000 7fac23fff700 1 -- 172.21.15.52:0/2023075032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0d03e2be0 msgr2=0x55d0d03f43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:53.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.276+0000 7fac23fff700 1 --2- 172.21.15.52:0/2023075032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0d03e2be0 0x55d0d03f43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.276+0000 7fac23fff700 1 -- 172.21.15.52:0/2023075032 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac0c004040 con 0x55d0d03e34f0 2023-11-27T15:19:53.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.276+0000 7fac28eef700 1 --2- 172.21.15.52:0/2023075032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0d03e2be0 0x55d0d03f43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:53.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.277+0000 7fac23fff700 1 --2- 172.21.15.52:0/2023075032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d03e34f0 0x55d0d03f4930 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fac0c005830 tx=0x7fac0c01e4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c661c92f6473dd server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:53.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.277+0000 7fac03fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:19:53.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.277+0000 7fac03fff700 1 -- 172.21.15.52:0/2023075032 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0c017070 con 0x55d0d03e34f0 2023-11-27T15:19:53.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.277+0000 7fac03fff700 1 -- 172.21.15.52:0/2023075032 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fac0c003cc0 con 0x55d0d03e34f0 2023-11-27T15:19:53.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.277+0000 7fac372dfb80 1 -- 172.21.15.52:0/2023075032 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac1401e040 con 0x55d0d03e34f0 2023-11-27T15:19:53.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.277+0000 7fac03fff700 1 -- 172.21.15.52:0/2023075032 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac0c003e80 con 0x55d0d03e34f0 2023-11-27T15:19:53.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.277+0000 7fac372dfb80 10 client.15852 fetch_fsmap learned FSMap version 23 2023-11-27T15:19:53.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.277+0000 7fac372dfb80 1 -- 172.21.15.52:0/2023075032 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d0d0161e60 con 0x55d0d03e34f0 2023-11-27T15:19:53.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.277+0000 7fac03fff700 1 -- 172.21.15.52:0/2023075032 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac1401e040 con 0x55d0d03e34f0 2023-11-27T15:19:53.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.277+0000 7fac03fff700 1 -- 172.21.15.52:0/2023075032 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fac0c025070 con 0x55d0d03e34f0 2023-11-27T15:19:53.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.277+0000 7fac372dfb80 10 client.15852 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:19:53.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.277+0000 7fac372dfb80 10 client.15852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:19:53.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.277+0000 7fac372dfb80 20 client.15852 populate_metadata read hostname 'smithi052' 2023-11-27T15:19:53.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.278+0000 7fac372dfb80 1 --2- 172.21.15.52:0/2023075032 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d0d0455fc0 0x55d0d04583d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:53.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.278+0000 7fac372dfb80 4 client.15852 mds_command: new command op to 24536 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-27T15:19:53.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.278+0000 7fac372dfb80 1 -- 172.21.15.52:0/2023075032 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d0d03026e0 con 0x55d0d0455fc0 2023-11-27T15:19:53.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.278+0000 7fac28eef700 1 --2- 172.21.15.52:0/2023075032 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d0d0455fc0 0x55d0d04583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:53.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.278+0000 7fac28eef700 1 --2- 172.21.15.52:0/2023075032 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d0d0455fc0 0x55d0d04583d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3925ec5eecebae61 server_cookie=1942b6f7f0ea2740 in_seq=0 out_seq=0 2023-11-27T15:19:53.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.278+0000 7fac03fff700 10 client.15852 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:53.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.279+0000 7fac03fff700 1 -- 172.21.15.52:0/2023075032 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d0d03026e0 con 0x55d0d0455fc0 2023-11-27T15:19:53.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.279+0000 7fac03fff700 10 client.15852 handle_command_reply: tid=0 2023-11-27T15:19:53.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.280+0000 7fac372dfb80 1 client.15852 shutdown 2023-11-27T15:19:53.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.280+0000 7fac372dfb80 1 -- 172.21.15.52:0/2023075032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d03e34f0 msgr2=0x55d0d03f4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:53.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.280+0000 7fac372dfb80 1 --2- 172.21.15.52:0/2023075032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d03e34f0 0x55d0d03f4930 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fac0c005830 tx=0x7fac0c01e4a0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.280+0000 7fac372dfb80 1 -- 172.21.15.52:0/2023075032 shutdown_connections 2023-11-27T15:19:53.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.280+0000 7fac372dfb80 1 --2- 172.21.15.52:0/2023075032 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d0d0455fc0 0x55d0d04583d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.280+0000 7fac372dfb80 1 --2- 172.21.15.52:0/2023075032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d0d03e2be0 0x55d0d03f43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.280+0000 7fac372dfb80 1 --2- 172.21.15.52:0/2023075032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d0d03e7930 0x55d0d03f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.280+0000 7fac03fff700 0 client.15852 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:53.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.280+0000 7fac372dfb80 1 --2- 172.21.15.52:0/2023075032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d0d03e34f0 0x55d0d03f4930 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.280+0000 7fac372dfb80 1 -- 172.21.15.52:0/2023075032 >> 172.21.15.52:0/2023075032 conn(0x55d0d03d83e0 msgr2=0x55d0d03edc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:53.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.280+0000 7fac372dfb80 1 -- 172.21.15.52:0/2023075032 shutdown_connections 2023-11-27T15:19:53.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.280+0000 7fac372dfb80 1 -- 172.21.15.52:0/2023075032 wait complete. 2023-11-27T15:19:53.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.280+0000 7fac372dfb80 20 client.15852 trim_cache size 0 max 16384 2023-11-27T15:19:53.295 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:19:53.295 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2023-11-27T15:19:53.295 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "f68bc523-ab20-4362-9b59-b57fdee9d395", 2023-11-27T15:19:53.295 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2023-11-27T15:19:53.296 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:19:53.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.281+0000 7fac2b153700 1 -- 172.21.15.52:0/1752043126 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fac100a12c0 msgr2=0x7fac100a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:53.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.281+0000 7fac2b153700 1 --2- 172.21.15.52:0/1752043126 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fac100a12c0 0x7fac100a3770 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fac14004800 tx=0x7fac1401a040 comp rx=0 tx=0).stop 2023-11-27T15:19:53.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.281+0000 7fac2b153700 1 -- 172.21.15.52:0/1752043126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac24105630 msgr2=0x7fac2419adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:53.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.281+0000 7fac2b153700 1 --2- 172.21.15.52:0/1752043126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac24105630 0x7fac2419adb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fac1c00efc0 tx=0x7fac1c00c3a0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.281+0000 7fac2b153700 1 -- 172.21.15.52:0/1752043126 shutdown_connections 2023-11-27T15:19:53.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.281+0000 7fac2b153700 1 --2- 172.21.15.52:0/1752043126 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fac100a12c0 0x7fac100a3770 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.298 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.281+0000 7fac2b153700 1 --2- 172.21.15.52:0/1752043126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac24100c50 0x7fac24196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.298 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.281+0000 7fac2b153700 1 --2- 172.21.15.52:0/1752043126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac24105630 0x7fac2419adb0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.298 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.281+0000 7fac2b153700 1 --2- 172.21.15.52:0/1752043126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac241015f0 0x7fac24196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.298 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.281+0000 7fac2b153700 1 -- 172.21.15.52:0/1752043126 >> 172.21.15.52:0/1752043126 conn(0x7fac240775c0 msgr2=0x7fac240fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:53.298 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.281+0000 7fac2b153700 1 -- 172.21.15.52:0/1752043126 shutdown_connections 2023-11-27T15:19:53.298 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:53.281+0000 7fac2b153700 1 -- 172.21.15.52:0/1752043126 wait complete. 2023-11-27T15:19:53.332 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:19:53.475 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:53.475 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:53.552 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:53.552 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:53.574 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:53.574 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:53.574 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 85 -lt 100 ']' 2023-11-27T15:19:53.574 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 85 + 1 2023-11-27T15:19:53.585 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=86 2023-11-27T15:19:53.585 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:53.600 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:53.600 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:53.601 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:53.601 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:53.602 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:54.033 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:54.034 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:54.078 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:54.078 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:54.081 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:54.081 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:54.081 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 86 -lt 100 ']' 2023-11-27T15:19:54.081 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 86 + 1 2023-11-27T15:19:54.131 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=87 2023-11-27T15:19:54.131 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:54.179 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:54.179 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:54.180 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:54.181 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:54.181 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:54.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:54 smithi052 ceph-mon[99056]: pgmap v62: 97 pgs: 97 active+clean; 2.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-11-27T15:19:54.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:54 smithi052 ceph-mon[99056]: scrub queued for path: / 2023-11-27T15:19:54.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:54 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:19:54.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:54 smithi052 ceph-mon[99056]: scrub summary: active paths [/] 2023-11-27T15:19:54.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:54 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:19:54.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:54 smithi150 ceph-mon[100559]: pgmap v62: 97 pgs: 97 active+clean; 2.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-11-27T15:19:54.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:54 smithi150 ceph-mon[100559]: scrub queued for path: / 2023-11-27T15:19:54.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:54 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:19:54.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:54 smithi150 ceph-mon[100559]: scrub summary: active paths [/] 2023-11-27T15:19:54.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:54 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:19:54.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:54 smithi102 ceph-mon[100296]: pgmap v62: 97 pgs: 97 active+clean; 2.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-11-27T15:19:54.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:54 smithi102 ceph-mon[100296]: scrub queued for path: / 2023-11-27T15:19:54.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:54 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:19:54.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:54 smithi102 ceph-mon[100296]: scrub summary: active paths [/] 2023-11-27T15:19:54.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:54 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:19:54.666 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:54.667 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:54.810 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:54.810 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:54.823 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:54.824 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:54.824 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 87 -lt 100 ']' 2023-11-27T15:19:54.824 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 87 + 1 2023-11-27T15:19:54.864 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=88 2023-11-27T15:19:54.864 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:54.888 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:54.889 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:54.890 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:54.890 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:54.890 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:55.485 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:55 smithi052 ceph-mon[99056]: scrub summary: idle 2023-11-27T15:19:55.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:55 smithi150 ceph-mon[100559]: scrub summary: idle 2023-11-27T15:19:55.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:55 smithi102 ceph-mon[100296]: scrub summary: idle 2023-11-27T15:19:55.666 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:55.667 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:55.826 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:55.826 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:55.865 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:55.866 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:55.866 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 88 -lt 100 ']' 2023-11-27T15:19:55.866 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 88 + 1 2023-11-27T15:19:55.898 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=89 2023-11-27T15:19:55.898 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:55.941 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:55.941 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:55.942 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:55.942 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:55.942 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:56.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.268+0000 7fa4e29e7700 1 -- 172.21.15.52:0/2627779414 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4dc1063f0 msgr2=0x7fa4dc1067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:56.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.268+0000 7fa4e29e7700 1 --2- 172.21.15.52:0/2627779414 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4dc1063f0 0x7fa4dc1067c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc00b0a0 tx=0x7fa4cc00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.268+0000 7fa4e29e7700 1 -- 172.21.15.52:0/2627779414 shutdown_connections 2023-11-27T15:19:56.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.268+0000 7fa4e29e7700 1 --2- 172.21.15.52:0/2627779414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4dc101400 0x7fa4dc10d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.268+0000 7fa4e29e7700 1 --2- 172.21.15.52:0/2627779414 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4dc1063f0 0x7fa4dc1067c0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.268+0000 7fa4e29e7700 1 --2- 172.21.15.52:0/2627779414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4dc100a50 0x7fa4dc100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.268+0000 7fa4e29e7700 1 -- 172.21.15.52:0/2627779414 >> 172.21.15.52:0/2627779414 conn(0x7fa4dc077590 msgr2=0x7fa4dc077990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:56.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.268+0000 7fa4e29e7700 1 -- 172.21.15.52:0/2627779414 shutdown_connections 2023-11-27T15:19:56.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.268+0000 7fa4e29e7700 1 -- 172.21.15.52:0/2627779414 wait complete. 2023-11-27T15:19:56.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.269+0000 7fa4e29e7700 1 Processor -- start 2023-11-27T15:19:56.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.269+0000 7fa4e29e7700 1 -- start start 2023-11-27T15:19:56.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.269+0000 7fa4e29e7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4dc100a50 0x7fa4dc1965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:56.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.269+0000 7fa4e29e7700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4dc101400 0x7fa4dc196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:56.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.269+0000 7fa4e29e7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4dc1063f0 0x7fa4dc19af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:56.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.269+0000 7fa4e29e7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4dc19b5d0 con 0x7fa4dc100a50 2023-11-27T15:19:56.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.269+0000 7fa4e29e7700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4dc19b740 con 0x7fa4dc101400 2023-11-27T15:19:56.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.269+0000 7fa4e29e7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4dc19b900 con 0x7fa4dc1063f0 2023-11-27T15:19:56.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.270+0000 7fa4dbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4dc100a50 0x7fa4dc1965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:56.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.270+0000 7fa4dbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4dc100a50 0x7fa4dc1965f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53092/0 (socket says 172.21.15.52:53092) 2023-11-27T15:19:56.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.270+0000 7fa4dbfff700 1 -- 172.21.15.52:0/417623506 learned_addr learned my addr 172.21.15.52:0/417623506 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:56.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.270+0000 7fa4d37fe700 1 --2- 172.21.15.52:0/417623506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4dc101400 0x7fa4dc196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:56.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.270+0000 7fa4e0f84700 1 --2- 172.21.15.52:0/417623506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4dc1063f0 0x7fa4dc19af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:56.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.270+0000 7fa4dbfff700 1 -- 172.21.15.52:0/417623506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4dc1063f0 msgr2=0x7fa4dc19af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:56.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.270+0000 7fa4dbfff700 1 --2- 172.21.15.52:0/417623506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4dc1063f0 0x7fa4dc19af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.270+0000 7fa4dbfff700 1 -- 172.21.15.52:0/417623506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4dc101400 msgr2=0x7fa4dc196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:56.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.270+0000 7fa4dbfff700 1 --2- 172.21.15.52:0/417623506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4dc101400 0x7fa4dc196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.270+0000 7fa4dbfff700 1 -- 172.21.15.52:0/417623506 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d4009520 con 0x7fa4dc100a50 2023-11-27T15:19:56.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.270+0000 7fa4d37fe700 1 --2- 172.21.15.52:0/417623506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4dc101400 0x7fa4dc196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:56.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.270+0000 7fa4dbfff700 1 --2- 172.21.15.52:0/417623506 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4dc100a50 0x7fa4dc1965f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc0062a0 tx=0x7fa4cc007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:56.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.270+0000 7fa4d9ffb700 1 -- 172.21.15.52:0/417623506 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc00e040 con 0x7fa4dc100a50 2023-11-27T15:19:56.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.271+0000 7fa4e29e7700 1 -- 172.21.15.52:0/417623506 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4cc009d00 con 0x7fa4dc100a50 2023-11-27T15:19:56.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.271+0000 7fa4d9ffb700 1 -- 172.21.15.52:0/417623506 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4cc00be20 con 0x7fa4dc100a50 2023-11-27T15:19:56.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.271+0000 7fa4d9ffb700 1 -- 172.21.15.52:0/417623506 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc01b6c0 con 0x7fa4dc100a50 2023-11-27T15:19:56.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.271+0000 7fa4e29e7700 1 -- 172.21.15.52:0/417623506 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4dc110380 con 0x7fa4dc100a50 2023-11-27T15:19:56.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.274+0000 7fa4d9ffb700 1 -- 172.21.15.52:0/417623506 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fa4cc01d470 con 0x7fa4dc100a50 2023-11-27T15:19:56.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.274+0000 7fa4d9ffb700 1 --2- 172.21.15.52:0/417623506 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fa4c80a12c0 0x7fa4c80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:56.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.274+0000 7fa4d9ffb700 1 -- 172.21.15.52:0/417623506 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fa4cc0cdd90 con 0x7fa4dc100a50 2023-11-27T15:19:56.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.274+0000 7fa4d37fe700 1 --2- 172.21.15.52:0/417623506 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fa4c80a12c0 0x7fa4c80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:56.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.275+0000 7fa4d37fe700 1 --2- 172.21.15.52:0/417623506 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fa4c80a12c0 0x7fa4c80a3770 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc197c00 tx=0x7fa4c4009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:56.341 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.336+0000 7fa4eeb73b80 1 Processor -- start 2023-11-27T15:19:56.341 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.336+0000 7fa4eeb73b80 1 -- start start 2023-11-27T15:19:56.341 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.337+0000 7fa4eeb73b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c976fbb60 0x563c976fbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:56.341 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.337+0000 7fa4eeb73b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c976fc470 0x563c97700370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:56.341 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.337+0000 7fa4eeb73b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c977008b0 0x563c97700d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:56.341 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.337+0000 7fa4eeb73b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c97601060 con 0x563c976fbb60 2023-11-27T15:19:56.342 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.337+0000 7fa4eeb73b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c976538a0 con 0x563c976fc470 2023-11-27T15:19:56.342 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.337+0000 7fa4eeb73b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c9764cd30 con 0x563c977008b0 2023-11-27T15:19:56.342 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.337+0000 7fa4dbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c976fbb60 0x563c976fbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:56.342 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.337+0000 7fa4dbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c976fbb60 0x563c976fbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53100/0 (socket says 172.21.15.52:53100) 2023-11-27T15:19:56.342 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.337+0000 7fa4dbfff700 1 -- 172.21.15.52:0/436871737 learned_addr learned my addr 172.21.15.52:0/436871737 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:56.342 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.337+0000 7fa4e0f84700 1 --2- 172.21.15.52:0/436871737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c977008b0 0x563c97700d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:56.342 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.337+0000 7fa4d37fe700 1 --2- 172.21.15.52:0/436871737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c976fc470 0x563c97700370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:56.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.337+0000 7fa4e0f84700 1 -- 172.21.15.52:0/436871737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c976fc470 msgr2=0x563c97700370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:56.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.337+0000 7fa4e0f84700 1 --2- 172.21.15.52:0/436871737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c976fc470 0x563c97700370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.338+0000 7fa4e0f84700 1 -- 172.21.15.52:0/436871737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c976fbb60 msgr2=0x563c976fbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:56.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.338+0000 7fa4e0f84700 1 --2- 172.21.15.52:0/436871737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c976fbb60 0x563c976fbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.338+0000 7fa4e0f84700 1 -- 172.21.15.52:0/436871737 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c974ce630 con 0x563c977008b0 2023-11-27T15:19:56.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.338+0000 7fa4dbfff700 1 --2- 172.21.15.52:0/436871737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c976fbb60 0x563c976fbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:56.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.338+0000 7fa4e0f84700 1 --2- 172.21.15.52:0/436871737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c977008b0 0x563c97700d00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4009cf0 tx=0x7fa4d40103d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74bd263b093efe0b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:56.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.338+0000 7fa4d27fc700 1 -- 172.21.15.52:0/436871737 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4010c50 con 0x563c977008b0 2023-11-27T15:19:56.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.338+0000 7fa4d27fc700 1 -- 172.21.15.52:0/436871737 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4d4015070 con 0x563c977008b0 2023-11-27T15:19:56.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.338+0000 7fa4d27fc700 1 -- 172.21.15.52:0/436871737 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4005680 con 0x563c977008b0 2023-11-27T15:19:56.345 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.339+0000 7fa4eeb73b80 1 -- 172.21.15.52:0/436871737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c977008b0 msgr2=0x563c97700d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:56.345 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.339+0000 7fa4eeb73b80 1 --2- 172.21.15.52:0/436871737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c977008b0 0x563c97700d00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4009cf0 tx=0x7fa4d40103d0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.345 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.339+0000 7fa4eeb73b80 1 -- 172.21.15.52:0/436871737 shutdown_connections 2023-11-27T15:19:56.345 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.339+0000 7fa4eeb73b80 1 --2- 172.21.15.52:0/436871737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c977008b0 0x563c97700d00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.345 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.339+0000 7fa4eeb73b80 1 --2- 172.21.15.52:0/436871737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c976fbb60 0x563c976fbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.345 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.339+0000 7fa4eeb73b80 1 --2- 172.21.15.52:0/436871737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c976fc470 0x563c97700370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.345 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.339+0000 7fa4eeb73b80 1 -- 172.21.15.52:0/436871737 >> 172.21.15.52:0/436871737 conn(0x563c976f1360 msgr2=0x563c976f6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:56.345 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.339+0000 7fa4eeb73b80 1 -- 172.21.15.52:0/436871737 shutdown_connections 2023-11-27T15:19:56.345 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.339+0000 7fa4eeb73b80 1 -- 172.21.15.52:0/436871737 wait complete. 2023-11-27T15:19:56.345 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.340+0000 7fa4eeb73b80 1 -- start start 2023-11-27T15:19:56.345 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.340+0000 7fa4eeb73b80 1 Processor -- start 2023-11-27T15:19:56.349 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.340+0000 7fa4eeb73b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c976fbb60 0x563c97705340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:56.349 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.340+0000 7fa4eeb73b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c976fc470 0x563c97705880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:56.349 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.340+0000 7fa4eeb73b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c977008b0 0x563c97705dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:56.349 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.340+0000 7fa4eeb73b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c975ee8e0 con 0x563c976fbb60 2023-11-27T15:19:56.349 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.340+0000 7fa4eeb73b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c975ee580 con 0x563c977008b0 2023-11-27T15:19:56.349 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.340+0000 7fa4eeb73b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c974d0920 con 0x563c976fc470 2023-11-27T15:19:56.349 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.340+0000 7fa4eeb73b80 10 client.0 _ll_register_callbacks cb 0x563c9760c780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:19:56.349 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.341+0000 7fa4dbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c976fbb60 0x563c97705340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.341+0000 7fa4dbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c976fbb60 0x563c97705340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53106/0 (socket says 172.21.15.52:53106) 2023-11-27T15:19:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.341+0000 7fa4dbfff700 1 -- 172.21.15.52:0/3803020957 learned_addr learned my addr 172.21.15.52:0/3803020957 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.341+0000 7fa4e0f84700 1 --2- 172.21.15.52:0/3803020957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c977008b0 0x563c97705dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.341+0000 7fa4d37fe700 1 --2- 172.21.15.52:0/3803020957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c976fc470 0x563c97705880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.341+0000 7fa4dbfff700 1 -- 172.21.15.52:0/3803020957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c976fc470 msgr2=0x563c97705880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.341+0000 7fa4dbfff700 1 --2- 172.21.15.52:0/3803020957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c976fc470 0x563c97705880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.341+0000 7fa4dbfff700 1 -- 172.21.15.52:0/3803020957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c977008b0 msgr2=0x563c97705dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.341+0000 7fa4dbfff700 1 --2- 172.21.15.52:0/3803020957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c977008b0 0x563c97705dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.341+0000 7fa4dbfff700 1 -- 172.21.15.52:0/3803020957 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4d40099a0 con 0x563c976fbb60 2023-11-27T15:19:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.341+0000 7fa4dbfff700 1 --2- 172.21.15.52:0/3803020957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c976fbb60 0x563c97705340 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc003f50 tx=0x7fa4cc09aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e3b3a1babe8a54 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.341+0000 7fa4bb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:19:56.350 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.341+0000 7fa4bb7fe700 1 -- 172.21.15.52:0/3803020957 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc09ef20 con 0x563c976fbb60 2023-11-27T15:19:56.351 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.341+0000 7fa4bb7fe700 1 -- 172.21.15.52:0/3803020957 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fa4cc0a32b0 con 0x563c976fbb60 2023-11-27T15:19:56.351 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.342+0000 7fa4eeb73b80 1 -- 172.21.15.52:0/3803020957 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c976eeb20 con 0x563c976fbb60 2023-11-27T15:19:56.351 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.342+0000 7fa4bb7fe700 1 -- 172.21.15.52:0/3803020957 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4cc0b9170 con 0x563c976fbb60 2023-11-27T15:19:56.351 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.342+0000 7fa4eeb73b80 10 client.15864 fetch_fsmap learned FSMap version 23 2023-11-27T15:19:56.351 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.342+0000 7fa4eeb73b80 1 -- 172.21.15.52:0/3803020957 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x563c97613190 con 0x563c976fbb60 2023-11-27T15:19:56.351 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.342+0000 7fa4bb7fe700 1 -- 172.21.15.52:0/3803020957 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4cc0c9080 con 0x563c976fbb60 2023-11-27T15:19:56.351 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.342+0000 7fa4bb7fe700 1 -- 172.21.15.52:0/3803020957 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa4cc0caf80 con 0x563c976fbb60 2023-11-27T15:19:56.351 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.342+0000 7fa4eeb73b80 10 client.15864 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:19:56.351 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.342+0000 7fa4eeb73b80 10 client.15864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:19:56.351 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.342+0000 7fa4eeb73b80 20 client.15864 populate_metadata read hostname 'smithi052' 2023-11-27T15:19:56.351 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.342+0000 7fa4eeb73b80 1 --2- 172.21.15.52:0/3803020957 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x563c97715670 0x563c97717a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:56.351 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.342+0000 7fa4eeb73b80 4 client.15864 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:19:56.351 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.342+0000 7fa4eeb73b80 1 -- 172.21.15.52:0/3803020957 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c9760f0a0 con 0x563c97715670 2023-11-27T15:19:56.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.343+0000 7fa4d37fe700 1 --2- 172.21.15.52:0/3803020957 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x563c97715670 0x563c97717a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:56.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.343+0000 7fa4d37fe700 1 --2- 172.21.15.52:0/3803020957 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x563c97715670 0x563c97717a80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a01554a1307d14e server_cookie=8b35d758cac4e36b in_seq=0 out_seq=0 2023-11-27T15:19:56.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.343+0000 7fa4bb7fe700 10 client.15864 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:56.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.344+0000 7fa4bb7fe700 1 -- 172.21.15.52:0/3803020957 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563c9760f0a0 con 0x563c97715670 2023-11-27T15:19:56.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.344+0000 7fa4bb7fe700 10 client.15864 handle_command_reply: tid=0 2023-11-27T15:19:56.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.344+0000 7fa4eeb73b80 1 client.15864 shutdown 2023-11-27T15:19:56.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.344+0000 7fa4eeb73b80 1 -- 172.21.15.52:0/3803020957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c976fbb60 msgr2=0x563c97705340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:56.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.344+0000 7fa4eeb73b80 1 --2- 172.21.15.52:0/3803020957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c976fbb60 0x563c97705340 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc003f50 tx=0x7fa4cc09aab0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.344+0000 7fa4eeb73b80 1 -- 172.21.15.52:0/3803020957 shutdown_connections 2023-11-27T15:19:56.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.344+0000 7fa4eeb73b80 1 --2- 172.21.15.52:0/3803020957 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x563c97715670 0x563c97717a80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.344+0000 7fa4eeb73b80 1 --2- 172.21.15.52:0/3803020957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c977008b0 0x563c97705dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.344+0000 7fa4eeb73b80 1 --2- 172.21.15.52:0/3803020957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c976fbb60 0x563c97705340 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.353 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.344+0000 7fa4eeb73b80 1 --2- 172.21.15.52:0/3803020957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c976fc470 0x563c97705880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.353 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.344+0000 7fa4bb7fe700 0 client.15864 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:56.353 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.344+0000 7fa4eeb73b80 1 -- 172.21.15.52:0/3803020957 >> 172.21.15.52:0/3803020957 conn(0x563c976f1360 msgr2=0x563c97701d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:56.353 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.344+0000 7fa4eeb73b80 1 -- 172.21.15.52:0/3803020957 shutdown_connections 2023-11-27T15:19:56.353 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.344+0000 7fa4eeb73b80 1 -- 172.21.15.52:0/3803020957 wait complete. 2023-11-27T15:19:56.353 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.344+0000 7fa4eeb73b80 20 client.15864 trim_cache size 0 max 16384 2023-11-27T15:19:56.368 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.363+0000 7fa4eeb73b80 1 Processor -- start 2023-11-27T15:19:56.368 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.363+0000 7fa4eeb73b80 1 -- start start 2023-11-27T15:19:56.368 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.363+0000 7fa4eeb73b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c976fbb60 0x563c9770c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:56.368 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.363+0000 7fa4eeb73b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c976fc470 0x563c976fff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:56.368 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.363+0000 7fa4eeb73b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c977008b0 0x563c97705a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:56.368 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.363+0000 7fa4eeb73b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c976fcbc0 con 0x563c976fbb60 2023-11-27T15:19:56.368 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.363+0000 7fa4eeb73b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c976ff200 con 0x563c977008b0 2023-11-27T15:19:56.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.363+0000 7fa4eeb73b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c976f8110 con 0x563c976fc470 2023-11-27T15:19:56.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.363+0000 7fa4dbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c976fbb60 0x563c9770c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:56.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.363+0000 7fa4dbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c976fbb60 0x563c9770c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53114/0 (socket says 172.21.15.52:53114) 2023-11-27T15:19:56.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.363+0000 7fa4dbfff700 1 -- 172.21.15.52:0/2613036759 learned_addr learned my addr 172.21.15.52:0/2613036759 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:56.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.363+0000 7fa4e0f84700 1 --2- 172.21.15.52:0/2613036759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c977008b0 0x563c97705a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:56.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.363+0000 7fa4d37fe700 1 --2- 172.21.15.52:0/2613036759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c976fc470 0x563c976fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:56.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.364+0000 7fa4dbfff700 1 -- 172.21.15.52:0/2613036759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c976fc470 msgr2=0x563c976fff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:56.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.364+0000 7fa4dbfff700 1 --2- 172.21.15.52:0/2613036759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c976fc470 0x563c976fff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.364+0000 7fa4dbfff700 1 -- 172.21.15.52:0/2613036759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c977008b0 msgr2=0x563c97705a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:56.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.364+0000 7fa4dbfff700 1 --2- 172.21.15.52:0/2613036759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c977008b0 0x563c97705a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.364+0000 7fa4dbfff700 1 -- 172.21.15.52:0/2613036759 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c97700f50 con 0x563c976fbb60 2023-11-27T15:19:56.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.364+0000 7fa4dbfff700 1 --2- 172.21.15.52:0/2613036759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c976fbb60 0x563c9770c9a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc09ae80 tx=0x7fa4cc09b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd3ae45b1610224 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:56.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.364+0000 7fa4d2ffd700 1 -- 172.21.15.52:0/2613036759 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc09a850 con 0x563c976fbb60 2023-11-27T15:19:56.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.364+0000 7fa4d2ffd700 1 -- 172.21.15.52:0/2613036759 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563c976eeb20 con 0x563c976fbb60 2023-11-27T15:19:56.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.364+0000 7fa4d2ffd700 1 -- 172.21.15.52:0/2613036759 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc0b9250 con 0x563c976fbb60 2023-11-27T15:19:56.374 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.365+0000 7fa4eeb73b80 1 -- 172.21.15.52:0/2613036759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c976fbb60 msgr2=0x563c9770c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:56.374 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.365+0000 7fa4eeb73b80 1 --2- 172.21.15.52:0/2613036759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c976fbb60 0x563c9770c9a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc09ae80 tx=0x7fa4cc09b070 comp rx=0 tx=0).stop 2023-11-27T15:19:56.374 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.365+0000 7fa4eeb73b80 1 -- 172.21.15.52:0/2613036759 shutdown_connections 2023-11-27T15:19:56.374 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.365+0000 7fa4eeb73b80 1 --2- 172.21.15.52:0/2613036759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c977008b0 0x563c97705a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.374 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.365+0000 7fa4eeb73b80 1 --2- 172.21.15.52:0/2613036759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c976fbb60 0x563c9770c9a0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.365+0000 7fa4eeb73b80 1 --2- 172.21.15.52:0/2613036759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c976fc470 0x563c976fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.365+0000 7fa4eeb73b80 1 -- 172.21.15.52:0/2613036759 >> 172.21.15.52:0/2613036759 conn(0x563c976f1360 msgr2=0x563c976ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:56.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.365+0000 7fa4eeb73b80 1 -- 172.21.15.52:0/2613036759 shutdown_connections 2023-11-27T15:19:56.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.365+0000 7fa4eeb73b80 1 -- 172.21.15.52:0/2613036759 wait complete. 2023-11-27T15:19:56.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.365+0000 7fa4eeb73b80 1 -- start start 2023-11-27T15:19:56.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.366+0000 7fa4eeb73b80 1 Processor -- start 2023-11-27T15:19:56.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.366+0000 7fa4eeb73b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c976fbb60 0x563c9770ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:56.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.366+0000 7fa4eeb73b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c976fc470 0x563c9770b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:56.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.366+0000 7fa4eeb73b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c977008b0 0x563c97768530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:56.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.366+0000 7fa4eeb73b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c97768a70 con 0x563c976fc470 2023-11-27T15:19:56.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.366+0000 7fa4eeb73b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c97768bb0 con 0x563c976fbb60 2023-11-27T15:19:56.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.366+0000 7fa4eeb73b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c97768cf0 con 0x563c977008b0 2023-11-27T15:19:56.375 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.366+0000 7fa4eeb73b80 10 client.0 _ll_register_callbacks cb 0x7fa4cc097320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:19:56.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.366+0000 7fa4d37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c976fc470 0x563c9770b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:56.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.366+0000 7fa4d37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c976fc470 0x563c9770b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53122/0 (socket says 172.21.15.52:53122) 2023-11-27T15:19:56.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.366+0000 7fa4d37fe700 1 -- 172.21.15.52:0/4133571938 learned_addr learned my addr 172.21.15.52:0/4133571938 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:56.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.366+0000 7fa4dbfff700 1 --2- 172.21.15.52:0/4133571938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c976fbb60 0x563c9770ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:56.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.366+0000 7fa4e0f84700 1 --2- 172.21.15.52:0/4133571938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c977008b0 0x563c97768530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:56.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.367+0000 7fa4d37fe700 1 -- 172.21.15.52:0/4133571938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c977008b0 msgr2=0x563c97768530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:56.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.367+0000 7fa4d37fe700 1 --2- 172.21.15.52:0/4133571938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c977008b0 0x563c97768530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.367+0000 7fa4d37fe700 1 -- 172.21.15.52:0/4133571938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c976fbb60 msgr2=0x563c9770ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:19:56.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.367+0000 7fa4d37fe700 1 --2- 172.21.15.52:0/4133571938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c976fbb60 0x563c9770ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.367+0000 7fa4d37fe700 1 -- 172.21.15.52:0/4133571938 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c97613190 con 0x563c976fc470 2023-11-27T15:19:56.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.367+0000 7fa4dbfff700 1 --2- 172.21.15.52:0/4133571938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c976fbb60 0x563c9770ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:56.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.367+0000 7fa4e0f84700 1 --2- 172.21.15.52:0/4133571938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c977008b0 0x563c97768530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:19:56.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.367+0000 7fa4d37fe700 1 --2- 172.21.15.52:0/4133571938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c976fc470 0x563c9770b0c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x563c97718390 tx=0x7fa4c4013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db9be6ff4248f61c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:56.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.367+0000 7fa4bb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:19:56.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.367+0000 7fa4bb7fe700 1 -- 172.21.15.52:0/4133571938 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4024040 con 0x563c976fc470 2023-11-27T15:19:56.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.367+0000 7fa4eeb73b80 1 -- 172.21.15.52:0/4133571938 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4d4015070 con 0x563c976fc470 2023-11-27T15:19:56.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.367+0000 7fa4bb7fe700 1 -- 172.21.15.52:0/4133571938 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fa4c4003cb0 con 0x563c976fc470 2023-11-27T15:19:56.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.367+0000 7fa4bb7fe700 1 -- 172.21.15.52:0/4133571938 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4c4028780 con 0x563c976fc470 2023-11-27T15:19:56.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.367+0000 7fa4eeb73b80 10 client.15870 fetch_fsmap learned FSMap version 23 2023-11-27T15:19:56.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.367+0000 7fa4eeb73b80 1 -- 172.21.15.52:0/4133571938 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x563c9747ade0 con 0x563c976fc470 2023-11-27T15:19:56.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.368+0000 7fa4bb7fe700 1 -- 172.21.15.52:0/4133571938 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4d4015070 con 0x563c976fc470 2023-11-27T15:19:56.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.368+0000 7fa4bb7fe700 1 -- 172.21.15.52:0/4133571938 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa4c403a980 con 0x563c976fc470 2023-11-27T15:19:56.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.368+0000 7fa4eeb73b80 10 client.15870 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:19:56.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.368+0000 7fa4eeb73b80 10 client.15870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:19:56.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.368+0000 7fa4eeb73b80 20 client.15870 populate_metadata read hostname 'smithi052' 2023-11-27T15:19:56.378 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.368+0000 7fa4eeb73b80 1 --2- 172.21.15.52:0/4133571938 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x563c9776a3c0 0x563c9776c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:56.378 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.368+0000 7fa4eeb73b80 4 client.15870 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:19:56.378 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.368+0000 7fa4eeb73b80 1 -- 172.21.15.52:0/4133571938 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563c9761b660 con 0x563c9776a3c0 2023-11-27T15:19:56.378 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.368+0000 7fa4dbfff700 1 --2- 172.21.15.52:0/4133571938 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x563c9776a3c0 0x563c9776c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:56.378 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.369+0000 7fa4bb7fe700 10 client.15870 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:56.378 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.369+0000 7fa4dbfff700 1 --2- 172.21.15.52:0/4133571938 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x563c9776a3c0 0x563c9776c7d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a717a64c66f1f033 server_cookie=89498173cc7fde40 in_seq=0 out_seq=0 2023-11-27T15:19:56.378 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.369+0000 7fa4bb7fe700 1 -- 172.21.15.52:0/4133571938 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563c9761b660 con 0x563c9776a3c0 2023-11-27T15:19:56.378 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.369+0000 7fa4bb7fe700 10 client.15870 handle_command_reply: tid=0 2023-11-27T15:19:56.378 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.369+0000 7fa4eeb73b80 1 client.15870 shutdown 2023-11-27T15:19:56.378 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.369+0000 7fa4eeb73b80 1 -- 172.21.15.52:0/4133571938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c976fc470 msgr2=0x563c9770b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:56.378 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.369+0000 7fa4eeb73b80 1 --2- 172.21.15.52:0/4133571938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c976fc470 0x563c9770b0c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x563c97718390 tx=0x7fa4c4013040 comp rx=0 tx=0).stop 2023-11-27T15:19:56.378 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.369+0000 7fa4eeb73b80 1 -- 172.21.15.52:0/4133571938 shutdown_connections 2023-11-27T15:19:56.378 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.369+0000 7fa4eeb73b80 1 --2- 172.21.15.52:0/4133571938 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x563c9776a3c0 0x563c9776c7d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.379 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.369+0000 7fa4eeb73b80 1 --2- 172.21.15.52:0/4133571938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c976fbb60 0x563c9770ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.379 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.369+0000 7fa4eeb73b80 1 --2- 172.21.15.52:0/4133571938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563c977008b0 0x563c97768530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.379 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.369+0000 7fa4eeb73b80 1 --2- 172.21.15.52:0/4133571938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c976fc470 0x563c9770b0c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.379 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.369+0000 7fa4bb7fe700 0 client.15870 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:19:56.379 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.369+0000 7fa4eeb73b80 1 -- 172.21.15.52:0/4133571938 >> 172.21.15.52:0/4133571938 conn(0x563c976f1360 msgr2=0x563c977013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:56.379 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.369+0000 7fa4eeb73b80 1 -- 172.21.15.52:0/4133571938 shutdown_connections 2023-11-27T15:19:56.379 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.369+0000 7fa4eeb73b80 1 -- 172.21.15.52:0/4133571938 wait complete. 2023-11-27T15:19:56.379 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.369+0000 7fa4eeb73b80 20 client.15870 trim_cache size 0 max 16384 2023-11-27T15:19:56.379 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:19:56.379 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2023-11-27T15:19:56.379 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2023-11-27T15:19:56.380 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:19:56.380 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.370+0000 7fa4e29e7700 1 -- 172.21.15.52:0/417623506 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fa4c80a12c0 msgr2=0x7fa4c80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:56.380 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.370+0000 7fa4e29e7700 1 --2- 172.21.15.52:0/417623506 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fa4c80a12c0 0x7fa4c80a3770 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc197c00 tx=0x7fa4c4009260 comp rx=0 tx=0).stop 2023-11-27T15:19:56.381 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.370+0000 7fa4e29e7700 1 -- 172.21.15.52:0/417623506 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4dc100a50 msgr2=0x7fa4dc1965f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:56.381 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.370+0000 7fa4e29e7700 1 --2- 172.21.15.52:0/417623506 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4dc100a50 0x7fa4dc1965f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc0062a0 tx=0x7fa4cc007870 comp rx=0 tx=0).stop 2023-11-27T15:19:56.381 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.370+0000 7fa4e29e7700 1 -- 172.21.15.52:0/417623506 shutdown_connections 2023-11-27T15:19:56.381 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.370+0000 7fa4e29e7700 1 --2- 172.21.15.52:0/417623506 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fa4c80a12c0 0x7fa4c80a3770 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.381 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.370+0000 7fa4e29e7700 1 --2- 172.21.15.52:0/417623506 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4dc1063f0 0x7fa4dc19af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.381 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.370+0000 7fa4e29e7700 1 --2- 172.21.15.52:0/417623506 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4dc100a50 0x7fa4dc1965f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.381 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.370+0000 7fa4e29e7700 1 --2- 172.21.15.52:0/417623506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4dc101400 0x7fa4dc196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.381 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.370+0000 7fa4e29e7700 1 -- 172.21.15.52:0/417623506 >> 172.21.15.52:0/417623506 conn(0x7fa4dc077590 msgr2=0x7fa4dc0ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:56.381 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.370+0000 7fa4e29e7700 1 -- 172.21.15.52:0/417623506 shutdown_connections 2023-11-27T15:19:56.381 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:56.370+0000 7fa4e29e7700 1 -- 172.21.15.52:0/417623506 wait complete. 2023-11-27T15:19:56.401 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:56 smithi052 ceph-mon[99056]: pgmap v63: 97 pgs: 97 active+clean; 2.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-11-27T15:19:56.414 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-27T15:19:56.415 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:19:56.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:56 smithi150 ceph-mon[100559]: pgmap v63: 97 pgs: 97 active+clean; 2.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-11-27T15:19:56.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:56 smithi102 ceph-mon[100296]: pgmap v63: 97 pgs: 97 active+clean; 2.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-11-27T15:19:56.592 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:56.592 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:56.676 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:56.676 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:56.698 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:56.698 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:56.698 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 89 -lt 100 ']' 2023-11-27T15:19:56.698 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 89 + 1 2023-11-27T15:19:56.723 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=90 2023-11-27T15:19:56.723 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:56.734 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:56.734 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:56.735 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:56.735 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:56.735 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:57.383 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:57.383 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:57.474 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:57.474 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:57.500 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:57.500 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:57.501 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 90 -lt 100 ']' 2023-11-27T15:19:57.501 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 90 + 1 2023-11-27T15:19:57.548 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=91 2023-11-27T15:19:57.548 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:57.592 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:57.592 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:57.593 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:57.593 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:57.593 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:58.125 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:58.125 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:58.351 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:58.351 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:58.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:58 smithi052 ceph-mon[99056]: pgmap v64: 97 pgs: 97 active+clean; 2.7 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-11-27T15:19:58.406 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:58.406 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:58.406 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 91 -lt 100 ']' 2023-11-27T15:19:58.406 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 91 + 1 2023-11-27T15:19:58.439 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=92 2023-11-27T15:19:58.439 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:58.484 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:58.484 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:58.485 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:58.485 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:58.485 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:58.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:58 smithi150 ceph-mon[100559]: pgmap v64: 97 pgs: 97 active+clean; 2.7 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-11-27T15:19:58.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:58 smithi102 ceph-mon[100296]: pgmap v64: 97 pgs: 97 active+clean; 2.7 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-11-27T15:19:59.066 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:59.067 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:59.226 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:19:59.226 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:19:59.265 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:59.265 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:19:59.266 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 92 -lt 100 ']' 2023-11-27T15:19:59.266 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 92 + 1 2023-11-27T15:19:59.298 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=93 2023-11-27T15:19:59.298 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:19:59.384 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:59.384 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:59.385 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:19:59.385 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:19:59.386 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:19:59.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.509+0000 7f28eeaba700 1 -- 172.21.15.52:0/3914957185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28e80ff420 msgr2=0x7f28e80ff7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:59.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.509+0000 7f28eeaba700 1 --2- 172.21.15.52:0/3914957185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28e80ff420 0x7f28e80ff7f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f28d80099c0 tx=0x7f28d8009cd0 comp rx=0 tx=0).stop 2023-11-27T15:19:59.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.509+0000 7f28eeaba700 1 -- 172.21.15.52:0/3914957185 shutdown_connections 2023-11-27T15:19:59.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.509+0000 7f28eeaba700 1 --2- 172.21.15.52:0/3914957185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28e8104460 0x7f28e810c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:59.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.509+0000 7f28eeaba700 1 --2- 172.21.15.52:0/3914957185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28e80ff420 0x7f28e80ff7f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:59.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.509+0000 7f28eeaba700 1 --2- 172.21.15.52:0/3914957185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28e80ffdc0 0x7f28e8103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:59.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.509+0000 7f28eeaba700 1 -- 172.21.15.52:0/3914957185 >> 172.21.15.52:0/3914957185 conn(0x7f28e8076310 msgr2=0x7f28e8076710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:59.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.509+0000 7f28eeaba700 1 -- 172.21.15.52:0/3914957185 shutdown_connections 2023-11-27T15:19:59.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.509+0000 7f28eeaba700 1 -- 172.21.15.52:0/3914957185 wait complete. 2023-11-27T15:19:59.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.510+0000 7f28eeaba700 1 Processor -- start 2023-11-27T15:19:59.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.510+0000 7f28eeaba700 1 -- start start 2023-11-27T15:19:59.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.510+0000 7f28eeaba700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28e80ff420 0x7f28e8196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:59.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.510+0000 7f28eeaba700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28e80ffdc0 0x7f28e8196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:59.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.510+0000 7f28eeaba700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28e8104460 0x7f28e819adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:59.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.510+0000 7f28eeaba700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e819b400 con 0x7f28e80ffdc0 2023-11-27T15:19:59.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.510+0000 7f28eeaba700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e819b570 con 0x7f28e8104460 2023-11-27T15:19:59.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.510+0000 7f28eeaba700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e819b730 con 0x7f28e80ff420 2023-11-27T15:19:59.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.510+0000 7f28e7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28e80ffdc0 0x7f28e8196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:59.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.510+0000 7f28e7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28e80ffdc0 0x7f28e8196960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53136/0 (socket says 172.21.15.52:53136) 2023-11-27T15:19:59.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.510+0000 7f28e7fff700 1 -- 172.21.15.52:0/1755794390 learned_addr learned my addr 172.21.15.52:0/1755794390 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:19:59.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.511+0000 7f28ec856700 1 --2- 172.21.15.52:0/1755794390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28e80ff420 0x7f28e8196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:59.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.511+0000 7f28ed057700 1 --2- 172.21.15.52:0/1755794390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28e8104460 0x7f28e819adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:59.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.511+0000 7f28e7fff700 1 -- 172.21.15.52:0/1755794390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28e80ff420 msgr2=0x7f28e8196420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:59.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.511+0000 7f28e7fff700 1 --2- 172.21.15.52:0/1755794390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28e80ff420 0x7f28e8196420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:59.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.511+0000 7f28e7fff700 1 -- 172.21.15.52:0/1755794390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28e8104460 msgr2=0x7f28e819adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:59.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.511+0000 7f28e7fff700 1 --2- 172.21.15.52:0/1755794390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28e8104460 0x7f28e819adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:59.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.511+0000 7f28e7fff700 1 -- 172.21.15.52:0/1755794390 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d80095f0 con 0x7f28e80ffdc0 2023-11-27T15:19:59.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.511+0000 7f28ed057700 1 --2- 172.21.15.52:0/1755794390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28e8104460 0x7f28e819adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:19:59.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.511+0000 7f28e7fff700 1 --2- 172.21.15.52:0/1755794390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28e80ffdc0 0x7f28e8196960 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f28d000d950 tx=0x7f28d000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:59.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.511+0000 7f28e5ffb700 1 -- 172.21.15.52:0/1755794390 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d0004220 con 0x7f28e80ffdc0 2023-11-27T15:19:59.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.511+0000 7f28e5ffb700 1 -- 172.21.15.52:0/1755794390 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28d0004cf0 con 0x7f28e80ffdc0 2023-11-27T15:19:59.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.511+0000 7f28eeaba700 1 -- 172.21.15.52:0/1755794390 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28e810f220 con 0x7f28e80ffdc0 2023-11-27T15:19:59.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.511+0000 7f28e5ffb700 1 -- 172.21.15.52:0/1755794390 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d0005620 con 0x7f28e80ffdc0 2023-11-27T15:19:59.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.512+0000 7f28eeaba700 1 -- 172.21.15.52:0/1755794390 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e810f090 con 0x7f28e80ffdc0 2023-11-27T15:19:59.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.513+0000 7f28eeaba700 1 -- 172.21.15.52:0/1755794390 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28e810f3b0 con 0x7f28e80ffdc0 2023-11-27T15:19:59.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.514+0000 7f28e5ffb700 1 -- 172.21.15.52:0/1755794390 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f28d0004750 con 0x7f28e80ffdc0 2023-11-27T15:19:59.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.514+0000 7f28e5ffb700 1 --2- 172.21.15.52:0/1755794390 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f28d40a12c0 0x7f28d40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:59.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.515+0000 7f28ec856700 1 --2- 172.21.15.52:0/1755794390 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f28d40a12c0 0x7f28d40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:59.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.515+0000 7f28e5ffb700 1 -- 172.21.15.52:0/1755794390 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f28d00ccbc0 con 0x7f28e80ffdc0 2023-11-27T15:19:59.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.515+0000 7f28ec856700 1 --2- 172.21.15.52:0/1755794390 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f28d40a12c0 0x7f28d40a3770 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f28d8009f50 tx=0x7f28d8005fb0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:59.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.523+0000 7f28e5ffb700 1 -- 172.21.15.52:0/1755794390 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f28d0095f70 con 0x7f28e80ffdc0 2023-11-27T15:19:59.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.773+0000 7f28eeaba700 1 -- 172.21.15.52:0/1755794390 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28e8108690 con 0x7f28e80ffdc0 2023-11-27T15:19:59.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.773+0000 7f28e5ffb700 1 -- 172.21.15.52:0/1755794390 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10804 (secure 0 0 0) 0x7f28d0016020 con 0x7f28e80ffdc0 2023-11-27T15:19:59.778 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:19:59.778 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2328881376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2328881376},{"type":"v1","addr":"172.21.15.52:6839","nonce":2328881376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2799551570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2799551570},{"type":"v1","addr":"172.21.15.52:6841","nonce":2799551570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:16:09.319630+0000","modified":"2023-11-27T15:16:14.742405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24494,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/215725838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":215725838},{"type":"v1","addr":"172.21.15.150:6839","nonce":215725838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3571830559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3571830559},{"type":"v1","addr":"172.21.15.102:6839","nonce":3571830559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:19:59.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.776+0000 7f28eeaba700 1 -- 172.21.15.52:0/1755794390 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f28d40a12c0 msgr2=0x7f28d40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:59.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.776+0000 7f28eeaba700 1 --2- 172.21.15.52:0/1755794390 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f28d40a12c0 0x7f28d40a3770 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f28d8009f50 tx=0x7f28d8005fb0 comp rx=0 tx=0).stop 2023-11-27T15:19:59.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.776+0000 7f28eeaba700 1 -- 172.21.15.52:0/1755794390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28e80ffdc0 msgr2=0x7f28e8196960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:59.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.776+0000 7f28eeaba700 1 --2- 172.21.15.52:0/1755794390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28e80ffdc0 0x7f28e8196960 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f28d000d950 tx=0x7f28d000dc60 comp rx=0 tx=0).stop 2023-11-27T15:19:59.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.776+0000 7f28eeaba700 1 -- 172.21.15.52:0/1755794390 shutdown_connections 2023-11-27T15:19:59.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.776+0000 7f28eeaba700 1 --2- 172.21.15.52:0/1755794390 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f28d40a12c0 0x7f28d40a3770 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:59.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.776+0000 7f28eeaba700 1 --2- 172.21.15.52:0/1755794390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28e8104460 0x7f28e819adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:59.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.776+0000 7f28eeaba700 1 --2- 172.21.15.52:0/1755794390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f28e80ff420 0x7f28e8196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:59.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.776+0000 7f28eeaba700 1 --2- 172.21.15.52:0/1755794390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28e80ffdc0 0x7f28e8196960 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:59.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.776+0000 7f28eeaba700 1 -- 172.21.15.52:0/1755794390 >> 172.21.15.52:0/1755794390 conn(0x7f28e8076310 msgr2=0x7f28e8103140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:59.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.776+0000 7f28eeaba700 1 -- 172.21.15.52:0/1755794390 shutdown_connections 2023-11-27T15:19:59.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:19:59.776+0000 7f28eeaba700 1 -- 172.21.15.52:0/1755794390 wait complete. 2023-11-27T15:19:59.782 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 23 2023-11-27T15:19:59.874 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:19:59.874 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:19:59.880 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-27T15:20:00.001 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:20:00.001 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:20:00.013 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:20:00.014 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:20:00.014 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 93 -lt 100 ']' 2023-11-27T15:20:00.014 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 93 + 1 2023-11-27T15:20:00.042 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=94 2023-11-27T15:20:00.042 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:20:00.049 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:20:00.049 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:20:00.050 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:20:00.050 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:20:00.050 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:20:00.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:59 smithi102 ceph-mon[100296]: pgmap v65: 97 pgs: 97 active+clean; 2.7 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 7 op/s 2023-11-27T15:20:00.078 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:19:59 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1755794390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:00.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:59 smithi052 ceph-mon[99056]: pgmap v65: 97 pgs: 97 active+clean; 2.7 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 7 op/s 2023-11-27T15:20:00.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:19:59 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1755794390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:00.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:59 smithi150 ceph-mon[100559]: pgmap v65: 97 pgs: 97 active+clean; 2.7 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 7 op/s 2023-11-27T15:20:00.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:19:59 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1755794390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:00.592 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:20:00.592 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:20:00.664 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:20:00.664 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:20:00.698 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:20:00.698 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:20:00.698 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 94 -lt 100 ']' 2023-11-27T15:20:00.698 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 94 + 1 2023-11-27T15:20:00.805 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=95 2023-11-27T15:20:00.805 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:20:00.816 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:20:00.816 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:20:00.818 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:20:00.818 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:20:00.818 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:20:01.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:00 smithi102 ceph-mon[100296]: overall HEALTH_OK 2023-11-27T15:20:01.092 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:00 smithi052 ceph-mon[99056]: overall HEALTH_OK 2023-11-27T15:20:01.232 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:20:01.233 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:20:01.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:00 smithi150 ceph-mon[100559]: overall HEALTH_OK 2023-11-27T15:20:01.384 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:20:01.384 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:20:01.406 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:20:01.406 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:20:01.406 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 95 -lt 100 ']' 2023-11-27T15:20:01.406 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 95 + 1 2023-11-27T15:20:01.455 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=96 2023-11-27T15:20:01.455 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:20:01.542 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:20:01.543 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:20:01.543 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:20:01.544 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:20:01.544 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:20:02.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:01 smithi102 ceph-mon[100296]: pgmap v66: 97 pgs: 97 active+clean; 2.8 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-11-27T15:20:02.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:01 smithi052 ceph-mon[99056]: pgmap v66: 97 pgs: 97 active+clean; 2.8 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-11-27T15:20:02.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:01 smithi150 ceph-mon[100559]: pgmap v66: 97 pgs: 97 active+clean; 2.8 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-11-27T15:20:02.457 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:20:02.458 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:20:02.659 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:20:02.659 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:20:02.698 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:20:02.698 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:20:02.698 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 96 -lt 100 ']' 2023-11-27T15:20:02.698 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 96 + 1 2023-11-27T15:20:02.730 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=97 2023-11-27T15:20:02.730 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:20:02.758 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:20:02.758 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:20:02.760 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:20:02.760 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:20:02.760 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:20:02.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.991+0000 7f9a894a2700 1 -- 172.21.15.52:0/1112270793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a84101390 msgr2=0x7f9a84101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:02.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.991+0000 7f9a894a2700 1 --2- 172.21.15.52:0/1112270793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a84101390 0x7f9a84101760 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9a74009990 tx=0x7f9a74009ca0 comp rx=0 tx=0).stop 2023-11-27T15:20:02.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.992+0000 7f9a894a2700 1 -- 172.21.15.52:0/1112270793 shutdown_connections 2023-11-27T15:20:02.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.992+0000 7f9a894a2700 1 --2- 172.21.15.52:0/1112270793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a8410a8b0 0x7f9a8410ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:02.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.992+0000 7f9a894a2700 1 --2- 172.21.15.52:0/1112270793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a84101390 0x7f9a84101760 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:02.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.992+0000 7f9a894a2700 1 --2- 172.21.15.52:0/1112270793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a84101ca0 0x7f9a8410a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:02.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.992+0000 7f9a894a2700 1 -- 172.21.15.52:0/1112270793 >> 172.21.15.52:0/1112270793 conn(0x7f9a84077080 msgr2=0x7f9a84077480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:02.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.992+0000 7f9a894a2700 1 -- 172.21.15.52:0/1112270793 shutdown_connections 2023-11-27T15:20:02.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.992+0000 7f9a894a2700 1 -- 172.21.15.52:0/1112270793 wait complete. 2023-11-27T15:20:02.998 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.992+0000 7f9a894a2700 1 Processor -- start 2023-11-27T15:20:02.998 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.993+0000 7f9a894a2700 1 -- start start 2023-11-27T15:20:02.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.993+0000 7f9a894a2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a84101390 0x7f9a84196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:02.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.993+0000 7f9a894a2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a84101ca0 0x7f9a84196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:02.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.993+0000 7f9a894a2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a8410a8b0 0x7f9a8419adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:02.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.993+0000 7f9a894a2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8419b3f0 con 0x7f9a84101ca0 2023-11-27T15:20:02.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.993+0000 7f9a894a2700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8419b560 con 0x7f9a84101390 2023-11-27T15:20:02.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.993+0000 7f9a894a2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8419b720 con 0x7f9a8410a8b0 2023-11-27T15:20:02.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.993+0000 7f9a827fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a84101ca0 0x7f9a84196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:02.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.993+0000 7f9a827fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a84101ca0 0x7f9a84196950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53150/0 (socket says 172.21.15.52:53150) 2023-11-27T15:20:02.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.993+0000 7f9a827fc700 1 -- 172.21.15.52:0/2485250156 learned_addr learned my addr 172.21.15.52:0/2485250156 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:02.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.994+0000 7f9a82ffd700 1 --2- 172.21.15.52:0/2485250156 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a84101390 0x7f9a84196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:02.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.994+0000 7f9a837fe700 1 --2- 172.21.15.52:0/2485250156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a8410a8b0 0x7f9a8419adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:03.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.994+0000 7f9a827fc700 1 -- 172.21.15.52:0/2485250156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a8410a8b0 msgr2=0x7f9a8419adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:03.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.994+0000 7f9a827fc700 1 --2- 172.21.15.52:0/2485250156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a8410a8b0 0x7f9a8419adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.994+0000 7f9a827fc700 1 -- 172.21.15.52:0/2485250156 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a84101390 msgr2=0x7f9a84196410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:03.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.994+0000 7f9a827fc700 1 --2- 172.21.15.52:0/2485250156 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a84101390 0x7f9a84196410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.994+0000 7f9a827fc700 1 -- 172.21.15.52:0/2485250156 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a740095f0 con 0x7f9a84101ca0 2023-11-27T15:20:03.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.994+0000 7f9a827fc700 1 --2- 172.21.15.52:0/2485250156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a84101ca0 0x7f9a84196950 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800e9a0 tx=0x7f9a7800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:03.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.994+0000 7f9a6bfff700 1 -- 172.21.15.52:0/2485250156 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a78009790 con 0x7f9a84101ca0 2023-11-27T15:20:03.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.994+0000 7f9a6bfff700 1 -- 172.21.15.52:0/2485250156 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a78010430 con 0x7f9a84101ca0 2023-11-27T15:20:03.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.994+0000 7f9a6bfff700 1 -- 172.21.15.52:0/2485250156 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a78018680 con 0x7f9a84101ca0 2023-11-27T15:20:03.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.994+0000 7f9a894a2700 1 -- 172.21.15.52:0/2485250156 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a84110330 con 0x7f9a84101ca0 2023-11-27T15:20:03.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.995+0000 7f9a894a2700 1 -- 172.21.15.52:0/2485250156 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a841101a0 con 0x7f9a84101ca0 2023-11-27T15:20:03.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.997+0000 7f9a6bfff700 1 -- 172.21.15.52:0/2485250156 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9a780105a0 con 0x7f9a84101ca0 2023-11-27T15:20:03.002 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.998+0000 7f9a6bfff700 1 --2- 172.21.15.52:0/2485250156 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f9a6c0a1340 0x7f9a6c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:03.002 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.998+0000 7f9a82ffd700 1 --2- 172.21.15.52:0/2485250156 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f9a6c0a1340 0x7f9a6c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:03.002 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.998+0000 7f9a6bfff700 1 -- 172.21.15.52:0/2485250156 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f9a78014070 con 0x7f9a84101ca0 2023-11-27T15:20:03.003 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:02.999+0000 7f9a82ffd700 1 --2- 172.21.15.52:0/2485250156 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f9a6c0a1340 0x7f9a6c0a37f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400b2e0 tx=0x7f9a74005fd0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:03.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.127+0000 7f9a9562eb80 1 Processor -- start 2023-11-27T15:20:03.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.127+0000 7f9a9562eb80 1 -- start start 2023-11-27T15:20:03.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.128+0000 7f9a9562eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bbab67eb60 0x55bbab67ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:03.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.128+0000 7f9a9562eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbab67f470 0x55bbab683370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:03.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.128+0000 7f9a9562eb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bbab6838b0 0x55bbab683d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:03.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.128+0000 7f9a9562eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbab584060 con 0x55bbab67eb60 2023-11-27T15:20:03.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.128+0000 7f9a9562eb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbab5d68a0 con 0x55bbab6838b0 2023-11-27T15:20:03.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.128+0000 7f9a9562eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbab5cfd30 con 0x55bbab67f470 2023-11-27T15:20:03.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.128+0000 7f9a82ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bbab67eb60 0x55bbab67ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:03.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.128+0000 7f9a82ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bbab67eb60 0x55bbab67ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53156/0 (socket says 172.21.15.52:53156) 2023-11-27T15:20:03.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.128+0000 7f9a82ffd700 1 -- 172.21.15.52:0/3372757638 learned_addr learned my addr 172.21.15.52:0/3372757638 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:03.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.128+0000 7f9a827fc700 1 --2- 172.21.15.52:0/3372757638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbab67f470 0x55bbab683370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:03.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.128+0000 7f9a837fe700 1 --2- 172.21.15.52:0/3372757638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bbab6838b0 0x55bbab683d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:03.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.128+0000 7f9a827fc700 1 -- 172.21.15.52:0/3372757638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bbab6838b0 msgr2=0x55bbab683d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:03.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.128+0000 7f9a827fc700 1 --2- 172.21.15.52:0/3372757638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bbab6838b0 0x55bbab683d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.128+0000 7f9a827fc700 1 -- 172.21.15.52:0/3372757638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bbab67eb60 msgr2=0x55bbab67ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:03.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.128+0000 7f9a827fc700 1 --2- 172.21.15.52:0/3372757638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bbab67eb60 0x55bbab67ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.129+0000 7f9a827fc700 1 -- 172.21.15.52:0/3372757638 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbab451630 con 0x55bbab67f470 2023-11-27T15:20:03.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.129+0000 7f9a827fc700 1 --2- 172.21.15.52:0/3372757638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbab67f470 0x55bbab683370 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9a780ca5e0 tx=0x7f9a780998c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53481935f7e436 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:03.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.129+0000 7f9a697fa700 1 -- 172.21.15.52:0/3372757638 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a780a23b0 con 0x55bbab67f470 2023-11-27T15:20:03.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.129+0000 7f9a697fa700 1 -- 172.21.15.52:0/3372757638 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a78010880 con 0x55bbab67f470 2023-11-27T15:20:03.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.129+0000 7f9a697fa700 1 -- 172.21.15.52:0/3372757638 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a780a1f20 con 0x55bbab67f470 2023-11-27T15:20:03.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.130+0000 7f9a9562eb80 1 -- 172.21.15.52:0/3372757638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbab67f470 msgr2=0x55bbab683370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:03.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.130+0000 7f9a9562eb80 1 --2- 172.21.15.52:0/3372757638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbab67f470 0x55bbab683370 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9a780ca5e0 tx=0x7f9a780998c0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.130+0000 7f9a9562eb80 1 -- 172.21.15.52:0/3372757638 shutdown_connections 2023-11-27T15:20:03.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.130+0000 7f9a9562eb80 1 --2- 172.21.15.52:0/3372757638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bbab6838b0 0x55bbab683d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.130+0000 7f9a9562eb80 1 --2- 172.21.15.52:0/3372757638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bbab67eb60 0x55bbab67ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.130+0000 7f9a9562eb80 1 --2- 172.21.15.52:0/3372757638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbab67f470 0x55bbab683370 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.130+0000 7f9a9562eb80 1 -- 172.21.15.52:0/3372757638 >> 172.21.15.52:0/3372757638 conn(0x55bbab674360 msgr2=0x55bbab679b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:03.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.130+0000 7f9a9562eb80 1 -- 172.21.15.52:0/3372757638 shutdown_connections 2023-11-27T15:20:03.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.130+0000 7f9a9562eb80 1 -- 172.21.15.52:0/3372757638 wait complete. 2023-11-27T15:20:03.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.130+0000 7f9a9562eb80 1 -- start start 2023-11-27T15:20:03.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.131+0000 7f9a9562eb80 1 Processor -- start 2023-11-27T15:20:03.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.131+0000 7f9a9562eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbab67eb60 0x55bbab68b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:03.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.131+0000 7f9a9562eb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bbab67f470 0x55bbab6883a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:03.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.131+0000 7f9a9562eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bbab6838b0 0x55bbab688ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:03.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.131+0000 7f9a9562eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbab5718e0 con 0x55bbab6838b0 2023-11-27T15:20:03.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.131+0000 7f9a9562eb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbab571580 con 0x55bbab67f470 2023-11-27T15:20:03.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.131+0000 7f9a9562eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbab453920 con 0x55bbab67eb60 2023-11-27T15:20:03.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.131+0000 7f9a9562eb80 10 client.0 _ll_register_callbacks cb 0x55bbab58f780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:20:03.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.131+0000 7f9a837fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bbab6838b0 0x55bbab688ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:03.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.131+0000 7f9a827fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bbab67f470 0x55bbab6883a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:03.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.131+0000 7f9a837fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bbab6838b0 0x55bbab688ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53166/0 (socket says 172.21.15.52:53166) 2023-11-27T15:20:03.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.131+0000 7f9a82ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbab67eb60 0x55bbab68b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:03.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.131+0000 7f9a837fe700 1 -- 172.21.15.52:0/3081833817 learned_addr learned my addr 172.21.15.52:0/3081833817 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:03.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.132+0000 7f9a837fe700 1 -- 172.21.15.52:0/3081833817 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbab67eb60 msgr2=0x55bbab68b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:03.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.132+0000 7f9a837fe700 1 --2- 172.21.15.52:0/3081833817 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbab67eb60 0x55bbab68b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.132+0000 7f9a837fe700 1 -- 172.21.15.52:0/3081833817 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bbab67f470 msgr2=0x55bbab6883a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:03.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.132+0000 7f9a837fe700 1 --2- 172.21.15.52:0/3081833817 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bbab67f470 0x55bbab6883a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.132+0000 7f9a837fe700 1 -- 172.21.15.52:0/3081833817 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a841101a0 con 0x55bbab6838b0 2023-11-27T15:20:03.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.132+0000 7f9a827fc700 1 --2- 172.21.15.52:0/3081833817 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bbab67f470 0x55bbab6883a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:03.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.132+0000 7f9a837fe700 1 --2- 172.21.15.52:0/3081833817 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bbab6838b0 0x55bbab688ae0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c011ef0 tx=0x7f9a7c014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a64e2d2413ba513 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:03.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.132+0000 7f9a66ff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:20:03.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.132+0000 7f9a66ff5700 1 -- 172.21.15.52:0/3081833817 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c00e040 con 0x55bbab6838b0 2023-11-27T15:20:03.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.132+0000 7f9a66ff5700 1 -- 172.21.15.52:0/3081833817 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f9a7c0093a0 con 0x55bbab6838b0 2023-11-27T15:20:03.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.132+0000 7f9a9562eb80 1 -- 172.21.15.52:0/3081833817 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bbab671b20 con 0x55bbab6838b0 2023-11-27T15:20:03.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.132+0000 7f9a66ff5700 1 -- 172.21.15.52:0/3081833817 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a7c014ce0 con 0x55bbab6838b0 2023-11-27T15:20:03.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.132+0000 7f9a9562eb80 10 client.15888 fetch_fsmap learned FSMap version 23 2023-11-27T15:20:03.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.132+0000 7f9a9562eb80 1 -- 172.21.15.52:0/3081833817 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55bbab596190 con 0x55bbab6838b0 2023-11-27T15:20:03.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.133+0000 7f9a66ff5700 1 -- 172.21.15.52:0/3081833817 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bbab671b20 con 0x55bbab6838b0 2023-11-27T15:20:03.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.133+0000 7f9a66ff5700 1 -- 172.21.15.52:0/3081833817 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9a7c01c020 con 0x55bbab6838b0 2023-11-27T15:20:03.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.133+0000 7f9a9562eb80 10 client.15888 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:20:03.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.133+0000 7f9a9562eb80 10 client.15888 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:20:03.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.133+0000 7f9a9562eb80 20 client.15888 populate_metadata read hostname 'smithi052' 2023-11-27T15:20:03.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.133+0000 7f9a9562eb80 1 --2- 172.21.15.52:0/3081833817 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55bbab6989d0 0x55bbab69ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:03.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.133+0000 7f9a9562eb80 4 client.15888 mds_command: new command op to 24472 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:20:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.133+0000 7f9a9562eb80 1 -- 172.21.15.52:0/3081833817 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bbab5920a0 con 0x55bbab6989d0 2023-11-27T15:20:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.133+0000 7f9a82ffd700 1 --2- 172.21.15.52:0/3081833817 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55bbab6989d0 0x55bbab69ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.134+0000 7f9a66ff5700 10 client.15888 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:20:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.134+0000 7f9a82ffd700 1 --2- 172.21.15.52:0/3081833817 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55bbab6989d0 0x55bbab69ade0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f233a8f372673b76 server_cookie=8e7d15c7b24fbfde in_seq=0 out_seq=0 2023-11-27T15:20:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.134+0000 7f9a66ff5700 1 -- 172.21.15.52:0/3081833817 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bbab5920a0 con 0x55bbab6989d0 2023-11-27T15:20:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.134+0000 7f9a66ff5700 10 client.15888 handle_command_reply: tid=0 2023-11-27T15:20:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.135+0000 7f9a9562eb80 1 client.15888 shutdown 2023-11-27T15:20:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.135+0000 7f9a9562eb80 1 -- 172.21.15.52:0/3081833817 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bbab6838b0 msgr2=0x55bbab688ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.135+0000 7f9a9562eb80 1 --2- 172.21.15.52:0/3081833817 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bbab6838b0 0x55bbab688ae0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c011ef0 tx=0x7f9a7c014600 comp rx=0 tx=0).stop 2023-11-27T15:20:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.135+0000 7f9a9562eb80 1 -- 172.21.15.52:0/3081833817 shutdown_connections 2023-11-27T15:20:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.135+0000 7f9a9562eb80 1 --2- 172.21.15.52:0/3081833817 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55bbab6989d0 0x55bbab69ade0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.135+0000 7f9a9562eb80 1 --2- 172.21.15.52:0/3081833817 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bbab6838b0 0x55bbab688ae0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.135+0000 7f9a9562eb80 1 --2- 172.21.15.52:0/3081833817 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbab67eb60 0x55bbab68b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.135+0000 7f9a9562eb80 1 --2- 172.21.15.52:0/3081833817 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bbab67f470 0x55bbab6883a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.135+0000 7f9a66ff5700 0 client.15888 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:20:03.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.135+0000 7f9a9562eb80 1 -- 172.21.15.52:0/3081833817 >> 172.21.15.52:0/3081833817 conn(0x55bbab674360 msgr2=0x55bbab684d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:03.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.135+0000 7f9a9562eb80 1 -- 172.21.15.52:0/3081833817 shutdown_connections 2023-11-27T15:20:03.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.135+0000 7f9a9562eb80 1 -- 172.21.15.52:0/3081833817 wait complete. 2023-11-27T15:20:03.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.135+0000 7f9a9562eb80 20 client.15888 trim_cache size 0 max 16384 2023-11-27T15:20:03.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.150+0000 7f9a9562eb80 1 Processor -- start 2023-11-27T15:20:03.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.150+0000 7f9a9562eb80 1 -- start start 2023-11-27T15:20:03.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.150+0000 7f9a9562eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbab67eb60 0x55bbab68f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:03.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.150+0000 7f9a9562eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bbab67f470 0x55bbab681970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:03.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.150+0000 7f9a9562eb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bbab6838b0 0x55bbab681eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:03.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.150+0000 7f9a9562eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbab67fbc0 con 0x55bbab67f470 2023-11-27T15:20:03.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.150+0000 7f9a9562eb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbab6890c0 con 0x55bbab6838b0 2023-11-27T15:20:03.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.150+0000 7f9a9562eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbab688e40 con 0x55bbab67eb60 2023-11-27T15:20:03.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.150+0000 7f9a827fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bbab67f470 0x55bbab681970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:03.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.150+0000 7f9a837fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bbab6838b0 0x55bbab681eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:03.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.150+0000 7f9a827fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bbab67f470 0x55bbab681970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53172/0 (socket says 172.21.15.52:53172) 2023-11-27T15:20:03.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.150+0000 7f9a827fc700 1 -- 172.21.15.52:0/1481293103 learned_addr learned my addr 172.21.15.52:0/1481293103 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:03.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.151+0000 7f9a82ffd700 1 --2- 172.21.15.52:0/1481293103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbab67eb60 0x55bbab68f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:03.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.151+0000 7f9a827fc700 1 -- 172.21.15.52:0/1481293103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbab67eb60 msgr2=0x55bbab68f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:03.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.151+0000 7f9a827fc700 1 --2- 172.21.15.52:0/1481293103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbab67eb60 0x55bbab68f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.151+0000 7f9a827fc700 1 -- 172.21.15.52:0/1481293103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bbab6838b0 msgr2=0x55bbab681eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:03.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.151+0000 7f9a827fc700 1 --2- 172.21.15.52:0/1481293103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bbab6838b0 0x55bbab681eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.151+0000 7f9a827fc700 1 -- 172.21.15.52:0/1481293103 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbab683f50 con 0x55bbab67f470 2023-11-27T15:20:03.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.151+0000 7f9a837fe700 1 --2- 172.21.15.52:0/1481293103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bbab6838b0 0x55bbab681eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:03.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.151+0000 7f9a827fc700 1 --2- 172.21.15.52:0/1481293103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bbab67f470 0x55bbab681970 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9a780969d0 tx=0x7f9a78010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61759fd45ccb5b7e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:03.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.151+0000 7f9a69ffb700 1 -- 172.21.15.52:0/1481293103 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a780109f0 con 0x55bbab67f470 2023-11-27T15:20:03.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.151+0000 7f9a69ffb700 1 -- 172.21.15.52:0/1481293103 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a780a23b0 con 0x55bbab67f470 2023-11-27T15:20:03.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.151+0000 7f9a69ffb700 1 -- 172.21.15.52:0/1481293103 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a78099120 con 0x55bbab67f470 2023-11-27T15:20:03.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.152+0000 7f9a9562eb80 1 -- 172.21.15.52:0/1481293103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bbab67f470 msgr2=0x55bbab681970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:03.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.152+0000 7f9a9562eb80 1 --2- 172.21.15.52:0/1481293103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bbab67f470 0x55bbab681970 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9a780969d0 tx=0x7f9a78010c80 comp rx=0 tx=0).stop 2023-11-27T15:20:03.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.152+0000 7f9a9562eb80 1 -- 172.21.15.52:0/1481293103 shutdown_connections 2023-11-27T15:20:03.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.152+0000 7f9a9562eb80 1 --2- 172.21.15.52:0/1481293103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bbab6838b0 0x55bbab681eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.152+0000 7f9a9562eb80 1 --2- 172.21.15.52:0/1481293103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbab67eb60 0x55bbab68f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.152+0000 7f9a9562eb80 1 --2- 172.21.15.52:0/1481293103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bbab67f470 0x55bbab681970 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.152+0000 7f9a9562eb80 1 -- 172.21.15.52:0/1481293103 >> 172.21.15.52:0/1481293103 conn(0x55bbab674360 msgr2=0x55bbab68ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:03.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.153+0000 7f9a9562eb80 1 -- 172.21.15.52:0/1481293103 shutdown_connections 2023-11-27T15:20:03.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.153+0000 7f9a9562eb80 1 -- 172.21.15.52:0/1481293103 wait complete. 2023-11-27T15:20:03.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.153+0000 7f9a9562eb80 1 -- start start 2023-11-27T15:20:03.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.153+0000 7f9a9562eb80 1 Processor -- start 2023-11-27T15:20:03.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.153+0000 7f9a9562eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bbab67eb60 0x55bbab68f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:03.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.153+0000 7f9a9562eb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bbab67f470 0x55bbab679370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:03.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.153+0000 7f9a9562eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbab6838b0 0x55bbab6798b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:03.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.153+0000 7f9a9562eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbab679fb0 con 0x55bbab67eb60 2023-11-27T15:20:03.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.153+0000 7f9a9562eb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbab686b50 con 0x55bbab67f470 2023-11-27T15:20:03.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.153+0000 7f9a9562eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbab686d10 con 0x55bbab6838b0 2023-11-27T15:20:03.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.153+0000 7f9a9562eb80 10 client.0 _ll_register_callbacks cb 0x7f9a7c011820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:20:03.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.154+0000 7f9a82ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bbab67eb60 0x55bbab68f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:03.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.154+0000 7f9a82ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bbab67eb60 0x55bbab68f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53176/0 (socket says 172.21.15.52:53176) 2023-11-27T15:20:03.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.154+0000 7f9a82ffd700 1 -- 172.21.15.52:0/1770103162 learned_addr learned my addr 172.21.15.52:0/1770103162 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:03.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.154+0000 7f9a827fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bbab67f470 0x55bbab679370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:03.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.154+0000 7f9a837fe700 1 --2- 172.21.15.52:0/1770103162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbab6838b0 0x55bbab6798b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:03.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.154+0000 7f9a82ffd700 1 -- 172.21.15.52:0/1770103162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbab6838b0 msgr2=0x55bbab6798b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:03.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.154+0000 7f9a82ffd700 1 --2- 172.21.15.52:0/1770103162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbab6838b0 0x55bbab6798b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.154+0000 7f9a82ffd700 1 -- 172.21.15.52:0/1770103162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bbab67f470 msgr2=0x55bbab679370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:03.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.154+0000 7f9a82ffd700 1 --2- 172.21.15.52:0/1770103162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bbab67f470 0x55bbab679370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.154+0000 7f9a82ffd700 1 -- 172.21.15.52:0/1770103162 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bbab596190 con 0x55bbab67eb60 2023-11-27T15:20:03.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.154+0000 7f9a827fc700 1 --2- 172.21.15.52:0/1770103162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bbab67f470 0x55bbab679370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:03.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.154+0000 7f9a82ffd700 1 --2- 172.21.15.52:0/1770103162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bbab67eb60 0x55bbab68f9b0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400bce0 tx=0x7f9a74009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6ccd315874fbda server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:03.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.154+0000 7f9a66ff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:20:03.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.154+0000 7f9a66ff5700 1 -- 172.21.15.52:0/1770103162 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7402d040 con 0x55bbab67eb60 2023-11-27T15:20:03.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.154+0000 7f9a66ff5700 1 -- 172.21.15.52:0/1770103162 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f9a7402b070 con 0x55bbab67eb60 2023-11-27T15:20:03.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.154+0000 7f9a9562eb80 1 -- 172.21.15.52:0/1770103162 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a78010880 con 0x55bbab67eb60 2023-11-27T15:20:03.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.155+0000 7f9a66ff5700 1 -- 172.21.15.52:0/1770103162 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a74005710 con 0x55bbab67eb60 2023-11-27T15:20:03.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.155+0000 7f9a9562eb80 10 client.15894 fetch_fsmap learned FSMap version 23 2023-11-27T15:20:03.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.155+0000 7f9a9562eb80 1 -- 172.21.15.52:0/1770103162 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55bbab451630 con 0x55bbab67eb60 2023-11-27T15:20:03.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.155+0000 7f9a66ff5700 1 -- 172.21.15.52:0/1770103162 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a78010880 con 0x55bbab67eb60 2023-11-27T15:20:03.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.155+0000 7f9a66ff5700 1 -- 172.21.15.52:0/1770103162 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9a7400f460 con 0x55bbab67eb60 2023-11-27T15:20:03.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.155+0000 7f9a9562eb80 10 client.15894 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:20:03.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.155+0000 7f9a9562eb80 10 client.15894 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:20:03.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.155+0000 7f9a9562eb80 20 client.15894 populate_metadata read hostname 'smithi052' 2023-11-27T15:20:03.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.155+0000 7f9a9562eb80 1 --2- 172.21.15.52:0/1770103162 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55bbab6ece80 0x55bbab6ef290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:03.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.155+0000 7f9a9562eb80 4 client.15894 mds_command: new command op to 24472 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:20:03.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.155+0000 7f9a9562eb80 1 -- 172.21.15.52:0/1770103162 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bbab59e660 con 0x55bbab6ece80 2023-11-27T15:20:03.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.156+0000 7f9a827fc700 1 --2- 172.21.15.52:0/1770103162 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55bbab6ece80 0x55bbab6ef290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:03.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.156+0000 7f9a827fc700 1 --2- 172.21.15.52:0/1770103162 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55bbab6ece80 0x55bbab6ef290 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a15d0cbe426d9ba1 server_cookie=a9ddf9cf3101de44 in_seq=0 out_seq=0 2023-11-27T15:20:03.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.156+0000 7f9a66ff5700 10 client.15894 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:20:03.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.156+0000 7f9a66ff5700 1 -- 172.21.15.52:0/1770103162 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bbab59e660 con 0x55bbab6ece80 2023-11-27T15:20:03.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.156+0000 7f9a66ff5700 10 client.15894 handle_command_reply: tid=0 2023-11-27T15:20:03.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.156+0000 7f9a9562eb80 1 client.15894 shutdown 2023-11-27T15:20:03.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.156+0000 7f9a9562eb80 1 -- 172.21.15.52:0/1770103162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bbab67eb60 msgr2=0x55bbab68f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:03.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.156+0000 7f9a9562eb80 1 --2- 172.21.15.52:0/1770103162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bbab67eb60 0x55bbab68f9b0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400bce0 tx=0x7f9a74009ca0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.156+0000 7f9a9562eb80 1 -- 172.21.15.52:0/1770103162 shutdown_connections 2023-11-27T15:20:03.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.156+0000 7f9a9562eb80 1 --2- 172.21.15.52:0/1770103162 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55bbab6ece80 0x55bbab6ef290 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.156+0000 7f9a9562eb80 1 --2- 172.21.15.52:0/1770103162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bbab6838b0 0x55bbab6798b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.156+0000 7f9a66ff5700 0 client.15894 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:20:03.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.156+0000 7f9a9562eb80 1 --2- 172.21.15.52:0/1770103162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bbab67eb60 0x55bbab68f9b0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.156+0000 7f9a9562eb80 1 --2- 172.21.15.52:0/1770103162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bbab67f470 0x55bbab679370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.156+0000 7f9a9562eb80 1 -- 172.21.15.52:0/1770103162 >> 172.21.15.52:0/1770103162 conn(0x55bbab674360 msgr2=0x55bbab681970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:03.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.157+0000 7f9a9562eb80 1 -- 172.21.15.52:0/1770103162 shutdown_connections 2023-11-27T15:20:03.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.157+0000 7f9a9562eb80 1 -- 172.21.15.52:0/1770103162 wait complete. 2023-11-27T15:20:03.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.157+0000 7f9a9562eb80 20 client.15894 trim_cache size 0 max 16384 2023-11-27T15:20:03.166 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:20:03.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.157+0000 7f9a894a2700 1 -- 172.21.15.52:0/2485250156 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f9a6c0a1340 msgr2=0x7f9a6c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:03.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.157+0000 7f9a894a2700 1 --2- 172.21.15.52:0/2485250156 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f9a6c0a1340 0x7f9a6c0a37f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400b2e0 tx=0x7f9a74005fd0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.157+0000 7f9a894a2700 1 -- 172.21.15.52:0/2485250156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a84101ca0 msgr2=0x7f9a84196950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:03.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.157+0000 7f9a894a2700 1 --2- 172.21.15.52:0/2485250156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a84101ca0 0x7f9a84196950 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800e9a0 tx=0x7f9a7800ecb0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.157+0000 7f9a894a2700 1 -- 172.21.15.52:0/2485250156 shutdown_connections 2023-11-27T15:20:03.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.157+0000 7f9a894a2700 1 --2- 172.21.15.52:0/2485250156 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f9a6c0a1340 0x7f9a6c0a37f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.157+0000 7f9a894a2700 1 --2- 172.21.15.52:0/2485250156 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9a84101390 0x7f9a84196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.157+0000 7f9a894a2700 1 --2- 172.21.15.52:0/2485250156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a8410a8b0 0x7f9a8419adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.157+0000 7f9a894a2700 1 --2- 172.21.15.52:0/2485250156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a84101ca0 0x7f9a84196950 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.157+0000 7f9a894a2700 1 -- 172.21.15.52:0/2485250156 >> 172.21.15.52:0/2485250156 conn(0x7f9a84077080 msgr2=0x7f9a84102290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:03.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.158+0000 7f9a894a2700 1 -- 172.21.15.52:0/2485250156 shutdown_connections 2023-11-27T15:20:03.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:03.158+0000 7f9a894a2700 1 -- 172.21.15.52:0/2485250156 wait complete. 2023-11-27T15:20:03.206 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-27T15:20:03.599 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:20:03.599 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:20:03.642 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:20:03.642 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:20:03.655 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:20:03.655 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:20:03.655 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 97 -lt 100 ']' 2023-11-27T15:20:03.655 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 97 + 1 2023-11-27T15:20:03.685 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=98 2023-11-27T15:20:03.685 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:20:03.690 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:20:03.691 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:20:03.692 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:20:03.692 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:20:03.692 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:20:04.191 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:20:04.191 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:20:04.350 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:20:04.350 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:20:04.356 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:20:04.357 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:20:04.357 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 98 -lt 100 ']' 2023-11-27T15:20:04.357 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 98 + 1 2023-11-27T15:20:04.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:04 smithi052 ceph-mon[99056]: pgmap v67: 97 pgs: 97 active+clean; 2.8 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 43 KiB/s wr, 19 op/s 2023-11-27T15:20:04.396 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=99 2023-11-27T15:20:04.396 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:20:04.425 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:20:04.425 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:20:04.426 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:20:04.426 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:20:04.426 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:20:04.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:04 smithi150 ceph-mon[100559]: pgmap v67: 97 pgs: 97 active+clean; 2.8 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 43 KiB/s wr, 19 op/s 2023-11-27T15:20:04.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:04 smithi102 ceph-mon[100296]: pgmap v67: 97 pgs: 97 active+clean; 2.8 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 43 KiB/s wr, 19 op/s 2023-11-27T15:20:05.057 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:20:05.057 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:20:05.234 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:20:05.234 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:20:05.254 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:20:05.254 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:20:05.254 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 99 -lt 100 ']' 2023-11-27T15:20:05.254 INFO:tasks.workunit.client.0.smithi052.stderr:++ expr 99 + 1 2023-11-27T15:20:05.313 INFO:tasks.workunit.client.0.smithi052.stderr:+ c=100 2023-11-27T15:20:05.313 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:20:05.337 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:20:05.337 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:20:05.339 INFO:tasks.workunit.client.0.smithi052.stderr:+ c1=2 2023-11-27T15:20:05.339 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo 3 2023-11-27T15:20:05.339 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:20:05.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.791+0000 7f5eb188a700 1 -- 172.21.15.52:0/4078508362 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5eac101ca0 msgr2=0x7f5eac10a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:05.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.791+0000 7f5eb188a700 1 --2- 172.21.15.52:0/4078508362 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5eac101ca0 0x7f5eac10a370 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5e94009990 tx=0x7f5e94009ca0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.791+0000 7f5eb188a700 1 -- 172.21.15.52:0/4078508362 shutdown_connections 2023-11-27T15:20:05.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.791+0000 7f5eb188a700 1 --2- 172.21.15.52:0/4078508362 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5eac10a8b0 0x7f5eac10ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.791+0000 7f5eb188a700 1 --2- 172.21.15.52:0/4078508362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5eac101390 0x7f5eac101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.791+0000 7f5eb188a700 1 --2- 172.21.15.52:0/4078508362 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5eac101ca0 0x7f5eac10a370 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.791+0000 7f5eb188a700 1 -- 172.21.15.52:0/4078508362 >> 172.21.15.52:0/4078508362 conn(0x7f5eac0770b0 msgr2=0x7f5eac0774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:05.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.792+0000 7f5eb188a700 1 -- 172.21.15.52:0/4078508362 shutdown_connections 2023-11-27T15:20:05.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.792+0000 7f5eb188a700 1 -- 172.21.15.52:0/4078508362 wait complete. 2023-11-27T15:20:05.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.792+0000 7f5eb188a700 1 Processor -- start 2023-11-27T15:20:05.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.793+0000 7f5eb188a700 1 -- start start 2023-11-27T15:20:05.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.793+0000 7f5eb188a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5eac101390 0x7f5eac0ff1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:05.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.793+0000 7f5eb188a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5eac101ca0 0x7f5eac0ff700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:05.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.793+0000 7f5eb188a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5eac10a8b0 0x7f5eac1a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:05.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.793+0000 7f5eb188a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac0fff30 con 0x7f5eac101390 2023-11-27T15:20:05.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.793+0000 7f5eb188a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac1a4b20 con 0x7f5eac101ca0 2023-11-27T15:20:05.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.793+0000 7f5eb188a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac1a4c60 con 0x7f5eac10a8b0 2023-11-27T15:20:05.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.793+0000 7f5eaaffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5eac101390 0x7f5eac0ff1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:05.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.793+0000 7f5eaaffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5eac101390 0x7f5eac0ff1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53190/0 (socket says 172.21.15.52:53190) 2023-11-27T15:20:05.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.793+0000 7f5eaaffd700 1 -- 172.21.15.52:0/1834943305 learned_addr learned my addr 172.21.15.52:0/1834943305 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:05.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.793+0000 7f5eaa7fc700 1 --2- 172.21.15.52:0/1834943305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5eac101ca0 0x7f5eac0ff700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:05.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.793+0000 7f5eab7fe700 1 --2- 172.21.15.52:0/1834943305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5eac10a8b0 0x7f5eac1a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:05.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.794+0000 7f5eaaffd700 1 -- 172.21.15.52:0/1834943305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5eac10a8b0 msgr2=0x7f5eac1a45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:05.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.794+0000 7f5eaaffd700 1 --2- 172.21.15.52:0/1834943305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5eac10a8b0 0x7f5eac1a45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.794+0000 7f5eaaffd700 1 -- 172.21.15.52:0/1834943305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5eac101ca0 msgr2=0x7f5eac0ff700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:05.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.794+0000 7f5eaaffd700 1 --2- 172.21.15.52:0/1834943305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5eac101ca0 0x7f5eac0ff700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.794+0000 7f5eaaffd700 1 -- 172.21.15.52:0/1834943305 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e940095f0 con 0x7f5eac101390 2023-11-27T15:20:05.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.794+0000 7f5eab7fe700 1 --2- 172.21.15.52:0/1834943305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5eac10a8b0 0x7f5eac1a45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:20:05.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.794+0000 7f5eaaffd700 1 --2- 172.21.15.52:0/1834943305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5eac101390 0x7f5eac0ff1c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00d980 tx=0x7f5e9c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:05.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.794+0000 7f5eb0888700 1 -- 172.21.15.52:0/1834943305 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c004220 con 0x7f5eac101390 2023-11-27T15:20:05.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.794+0000 7f5eb0888700 1 -- 172.21.15.52:0/1834943305 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e9c007640 con 0x7f5eac101390 2023-11-27T15:20:05.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.794+0000 7f5eb188a700 1 -- 172.21.15.52:0/1834943305 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eac110340 con 0x7f5eac101390 2023-11-27T15:20:05.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.794+0000 7f5eb0888700 1 -- 172.21.15.52:0/1834943305 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c005610 con 0x7f5eac101390 2023-11-27T15:20:05.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.794+0000 7f5eb188a700 1 -- 172.21.15.52:0/1834943305 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eac1101b0 con 0x7f5eac101390 2023-11-27T15:20:05.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.797+0000 7f5eb0888700 1 -- 172.21.15.52:0/1834943305 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f5e9c004750 con 0x7f5eac101390 2023-11-27T15:20:05.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.797+0000 7f5eb0888700 1 --2- 172.21.15.52:0/1834943305 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f5e980a12c0 0x7f5e980a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:05.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.798+0000 7f5eb0888700 1 -- 172.21.15.52:0/1834943305 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5e9c0ccbb0 con 0x7f5eac101390 2023-11-27T15:20:05.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.798+0000 7f5eaa7fc700 1 --2- 172.21.15.52:0/1834943305 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f5e980a12c0 0x7f5e980a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:05.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.798+0000 7f5eaa7fc700 1 --2- 172.21.15.52:0/1834943305 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f5e980a12c0 0x7f5e980a3770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5e9400b5c0 tx=0x7f5e94005960 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:05.875 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.869+0000 7f5ebda16b80 1 Processor -- start 2023-11-27T15:20:05.875 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.869+0000 7f5ebda16b80 1 -- start start 2023-11-27T15:20:05.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.869+0000 7f5ebda16b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0651e6b60 0x55f0651e6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:05.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.869+0000 7f5ebda16b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0651e7470 0x55f0651eb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:05.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.869+0000 7f5ebda16b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0651eb8b0 0x55f0651ebd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:05.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.869+0000 7f5ebda16b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0650ec060 con 0x55f0651e6b60 2023-11-27T15:20:05.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.869+0000 7f5ebda16b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f06513e8a0 con 0x55f0651eb8b0 2023-11-27T15:20:05.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.869+0000 7f5ebda16b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f065137d30 con 0x55f0651e7470 2023-11-27T15:20:05.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.869+0000 7f5eaaffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0651e6b60 0x55f0651e6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:05.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.869+0000 7f5eaaffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0651e6b60 0x55f0651e6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53198/0 (socket says 172.21.15.52:53198) 2023-11-27T15:20:05.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.869+0000 7f5eaaffd700 1 -- 172.21.15.52:0/2336796461 learned_addr learned my addr 172.21.15.52:0/2336796461 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:05.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.869+0000 7f5eab7fe700 1 --2- 172.21.15.52:0/2336796461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0651eb8b0 0x55f0651ebd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:05.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.870+0000 7f5eaa7fc700 1 --2- 172.21.15.52:0/2336796461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0651e7470 0x55f0651eb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:05.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.870+0000 7f5eaaffd700 1 -- 172.21.15.52:0/2336796461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0651e7470 msgr2=0x55f0651eb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:05.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.870+0000 7f5eaaffd700 1 --2- 172.21.15.52:0/2336796461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0651e7470 0x55f0651eb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.870+0000 7f5eaaffd700 1 -- 172.21.15.52:0/2336796461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0651eb8b0 msgr2=0x55f0651ebd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:05.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.870+0000 7f5eaaffd700 1 --2- 172.21.15.52:0/2336796461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0651eb8b0 0x55f0651ebd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.870+0000 7f5eaaffd700 1 -- 172.21.15.52:0/2336796461 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f064fb9630 con 0x55f0651e6b60 2023-11-27T15:20:05.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.870+0000 7f5eab7fe700 1 --2- 172.21.15.52:0/2336796461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0651eb8b0 0x55f0651ebd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:05.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.870+0000 7f5eaaffd700 1 --2- 172.21.15.52:0/2336796461 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0651e6b60 0x55f0651e6f30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c096180 tx=0x7f5e9c00f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5357d7b78ac89da1 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:05.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.870+0000 7f5ea1ffb700 1 -- 172.21.15.52:0/2336796461 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c09dcb0 con 0x55f0651e6b60 2023-11-27T15:20:05.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.870+0000 7f5ea1ffb700 1 -- 172.21.15.52:0/2336796461 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e9c0a20d0 con 0x55f0651e6b60 2023-11-27T15:20:05.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.870+0000 7f5ea1ffb700 1 -- 172.21.15.52:0/2336796461 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c09a140 con 0x55f0651e6b60 2023-11-27T15:20:05.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.871+0000 7f5ebda16b80 1 -- 172.21.15.52:0/2336796461 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0651e6b60 msgr2=0x55f0651e6f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:05.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.871+0000 7f5ebda16b80 1 --2- 172.21.15.52:0/2336796461 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0651e6b60 0x55f0651e6f30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c096180 tx=0x7f5e9c00f440 comp rx=0 tx=0).stop 2023-11-27T15:20:05.878 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.871+0000 7f5ebda16b80 1 -- 172.21.15.52:0/2336796461 shutdown_connections 2023-11-27T15:20:05.878 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.871+0000 7f5ebda16b80 1 --2- 172.21.15.52:0/2336796461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0651eb8b0 0x55f0651ebd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.878 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.871+0000 7f5ebda16b80 1 --2- 172.21.15.52:0/2336796461 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0651e6b60 0x55f0651e6f30 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.878 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.871+0000 7f5ebda16b80 1 --2- 172.21.15.52:0/2336796461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0651e7470 0x55f0651eb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.878 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.871+0000 7f5ebda16b80 1 -- 172.21.15.52:0/2336796461 >> 172.21.15.52:0/2336796461 conn(0x55f0651dc360 msgr2=0x55f0651e1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:05.878 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.871+0000 7f5ebda16b80 1 -- 172.21.15.52:0/2336796461 shutdown_connections 2023-11-27T15:20:05.878 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.872+0000 7f5ebda16b80 1 -- 172.21.15.52:0/2336796461 wait complete. 2023-11-27T15:20:05.881 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.872+0000 7f5ebda16b80 1 -- start start 2023-11-27T15:20:05.882 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.872+0000 7f5ebda16b80 1 Processor -- start 2023-11-27T15:20:05.882 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.873+0000 7f5ebda16b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0651e6b60 0x55f0651f30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:05.882 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.873+0000 7f5ebda16b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0651e7470 0x55f0651f03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:05.882 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.873+0000 7f5ebda16b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0651eb8b0 0x55f0651f0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:05.882 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.873+0000 7f5ebda16b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0650d98e0 con 0x55f0651e6b60 2023-11-27T15:20:05.882 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.873+0000 7f5ebda16b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0650d9580 con 0x55f0651eb8b0 2023-11-27T15:20:05.882 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.873+0000 7f5ebda16b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f064fbb920 con 0x55f0651e7470 2023-11-27T15:20:05.882 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.873+0000 7f5ebda16b80 10 client.0 _ll_register_callbacks cb 0x55f0650f7780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:20:05.882 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.873+0000 7f5eaaffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0651e6b60 0x55f0651f30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:05.882 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.873+0000 7f5eaaffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0651e6b60 0x55f0651f30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53204/0 (socket says 172.21.15.52:53204) 2023-11-27T15:20:05.882 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.873+0000 7f5eaaffd700 1 -- 172.21.15.52:0/3985663854 learned_addr learned my addr 172.21.15.52:0/3985663854 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:05.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.873+0000 7f5eaa7fc700 1 --2- 172.21.15.52:0/3985663854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0651e7470 0x55f0651f03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:05.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.873+0000 7f5eab7fe700 1 --2- 172.21.15.52:0/3985663854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0651eb8b0 0x55f0651f0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:05.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.874+0000 7f5eaaffd700 1 -- 172.21.15.52:0/3985663854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0651e7470 msgr2=0x55f0651f03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:05.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.874+0000 7f5eaaffd700 1 --2- 172.21.15.52:0/3985663854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0651e7470 0x55f0651f03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.874+0000 7f5eaaffd700 1 -- 172.21.15.52:0/3985663854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0651eb8b0 msgr2=0x55f0651f0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:05.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.874+0000 7f5eaaffd700 1 --2- 172.21.15.52:0/3985663854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0651eb8b0 0x55f0651f0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.874+0000 7f5eaaffd700 1 -- 172.21.15.52:0/3985663854 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eac1101b0 con 0x55f0651e6b60 2023-11-27T15:20:05.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.874+0000 7f5eab7fe700 1 --2- 172.21.15.52:0/3985663854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0651eb8b0 0x55f0651f0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:20:05.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.874+0000 7f5eaaffd700 1 --2- 172.21.15.52:0/3985663854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0651e6b60 0x55f0651f30f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c0ca3b0 tx=0x7f5e9c0050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ed2da9eaef9572 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:05.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.874+0000 7f5e8affd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:20:05.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.874+0000 7f5e8affd700 1 -- 172.21.15.52:0/3985663854 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c099370 con 0x55f0651e6b60 2023-11-27T15:20:05.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.874+0000 7f5e8affd700 1 -- 172.21.15.52:0/3985663854 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5e9c0b2420 con 0x55f0651e6b60 2023-11-27T15:20:05.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.874+0000 7f5ebda16b80 1 -- 172.21.15.52:0/3985663854 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f0651d9b20 con 0x55f0651e6b60 2023-11-27T15:20:05.884 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.875+0000 7f5e8affd700 1 -- 172.21.15.52:0/3985663854 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e9c099ee0 con 0x55f0651e6b60 2023-11-27T15:20:05.884 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.875+0000 7f5ebda16b80 10 client.15906 fetch_fsmap learned FSMap version 23 2023-11-27T15:20:05.884 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.875+0000 7f5ebda16b80 1 -- 172.21.15.52:0/3985663854 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55f0650fe190 con 0x55f0651e6b60 2023-11-27T15:20:05.884 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.875+0000 7f5e8affd700 1 -- 172.21.15.52:0/3985663854 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f0651d9b20 con 0x55f0651e6b60 2023-11-27T15:20:05.884 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.875+0000 7f5e8affd700 1 -- 172.21.15.52:0/3985663854 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5e9c0b9050 con 0x55f0651e6b60 2023-11-27T15:20:05.884 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.875+0000 7f5ebda16b80 10 client.15906 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:20:05.884 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.875+0000 7f5ebda16b80 10 client.15906 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:20:05.884 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.875+0000 7f5ebda16b80 20 client.15906 populate_metadata read hostname 'smithi052' 2023-11-27T15:20:05.884 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.875+0000 7f5ebda16b80 1 --2- 172.21.15.52:0/3985663854 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55f065200480 0x55f065202860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:05.884 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.875+0000 7f5ebda16b80 4 client.15906 mds_command: new command op to 24494 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:20:05.884 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.875+0000 7f5ebda16b80 1 -- 172.21.15.52:0/3985663854 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f0650fa0a0 con 0x55f065200480 2023-11-27T15:20:05.884 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.876+0000 7f5eaa7fc700 1 --2- 172.21.15.52:0/3985663854 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55f065200480 0x55f065202860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:05.884 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.876+0000 7f5eaa7fc700 1 --2- 172.21.15.52:0/3985663854 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55f065200480 0x55f065202860 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d73ab54cc1c23c85 server_cookie=ce41b90f40b79ca in_seq=0 out_seq=0 2023-11-27T15:20:05.885 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.876+0000 7f5e8affd700 10 client.15906 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:20:05.885 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.877+0000 7f5e8affd700 1 -- 172.21.15.52:0/3985663854 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f0650fa0a0 con 0x55f065200480 2023-11-27T15:20:05.885 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.877+0000 7f5e8affd700 10 client.15906 handle_command_reply: tid=0 2023-11-27T15:20:05.885 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.877+0000 7f5ebda16b80 1 client.15906 shutdown 2023-11-27T15:20:05.885 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.877+0000 7f5ebda16b80 1 -- 172.21.15.52:0/3985663854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0651e6b60 msgr2=0x55f0651f30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:05.885 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.877+0000 7f5ebda16b80 1 --2- 172.21.15.52:0/3985663854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0651e6b60 0x55f0651f30f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c0ca3b0 tx=0x7f5e9c0050b0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.885 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.877+0000 7f5ebda16b80 1 -- 172.21.15.52:0/3985663854 shutdown_connections 2023-11-27T15:20:05.885 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.877+0000 7f5ebda16b80 1 --2- 172.21.15.52:0/3985663854 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55f065200480 0x55f065202860 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.885 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.877+0000 7f5ebda16b80 1 --2- 172.21.15.52:0/3985663854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0651eb8b0 0x55f0651f0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.885 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.877+0000 7f5ebda16b80 1 --2- 172.21.15.52:0/3985663854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0651e6b60 0x55f0651f30f0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.885 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.877+0000 7f5e8affd700 0 client.15906 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:20:05.885 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.877+0000 7f5ebda16b80 1 --2- 172.21.15.52:0/3985663854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0651e7470 0x55f0651f03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.886 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.877+0000 7f5ebda16b80 1 -- 172.21.15.52:0/3985663854 >> 172.21.15.52:0/3985663854 conn(0x55f0651dc360 msgr2=0x55f0651ecd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:05.886 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.877+0000 7f5ebda16b80 1 -- 172.21.15.52:0/3985663854 shutdown_connections 2023-11-27T15:20:05.886 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.877+0000 7f5ebda16b80 1 -- 172.21.15.52:0/3985663854 wait complete. 2023-11-27T15:20:05.886 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.877+0000 7f5ebda16b80 20 client.15906 trim_cache size 0 max 16384 2023-11-27T15:20:05.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.891+0000 7f5ebda16b80 1 Processor -- start 2023-11-27T15:20:05.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.891+0000 7f5ebda16b80 1 -- start start 2023-11-27T15:20:05.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.892+0000 7f5ebda16b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0651e6b60 0x55f0651e91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:05.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.892+0000 7f5ebda16b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0651e7470 0x55f0651f79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:05.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.892+0000 7f5ebda16b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0651eb8b0 0x55f0651f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:05.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.892+0000 7f5ebda16b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0651f3790 con 0x55f0651e6b60 2023-11-27T15:20:05.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.892+0000 7f5ebda16b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0651e7bc0 con 0x55f0651e7470 2023-11-27T15:20:05.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.892+0000 7f5ebda16b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0651f1930 con 0x55f0651eb8b0 2023-11-27T15:20:05.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.892+0000 7f5eaaffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0651e6b60 0x55f0651e91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:05.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.892+0000 7f5eaa7fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0651e7470 0x55f0651f79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:05.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.892+0000 7f5eaaffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0651e6b60 0x55f0651e91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53212/0 (socket says 172.21.15.52:53212) 2023-11-27T15:20:05.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.892+0000 7f5eaaffd700 1 -- 172.21.15.52:0/3193268379 learned_addr learned my addr 172.21.15.52:0/3193268379 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:05.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.892+0000 7f5eab7fe700 1 --2- 172.21.15.52:0/3193268379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0651eb8b0 0x55f0651f2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:05.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.892+0000 7f5eaa7fc700 1 -- 172.21.15.52:0/3193268379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0651eb8b0 msgr2=0x55f0651f2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:05.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.892+0000 7f5eaa7fc700 1 --2- 172.21.15.52:0/3193268379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0651eb8b0 0x55f0651f2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.892+0000 7f5eaa7fc700 1 -- 172.21.15.52:0/3193268379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0651e6b60 msgr2=0x55f0651e91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:05.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.892+0000 7f5eaa7fc700 1 --2- 172.21.15.52:0/3193268379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0651e6b60 0x55f0651e91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.892+0000 7f5eaa7fc700 1 -- 172.21.15.52:0/3193268379 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0650fdee0 con 0x55f0651e7470 2023-11-27T15:20:05.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.892+0000 7f5eaaffd700 1 --2- 172.21.15.52:0/3193268379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0651e6b60 0x55f0651e91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:05.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.893+0000 7f5eaa7fc700 1 --2- 172.21.15.52:0/3193268379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0651e7470 0x55f0651f79d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5e940059d0 tx=0x7f5e94005000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=773230ada36cdecf server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:05.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.893+0000 7f5ea27fc700 1 -- 172.21.15.52:0/3193268379 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9402c040 con 0x55f0651e7470 2023-11-27T15:20:05.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.893+0000 7f5ea27fc700 1 -- 172.21.15.52:0/3193268379 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e9402d040 con 0x55f0651e7470 2023-11-27T15:20:05.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.893+0000 7f5ea27fc700 1 -- 172.21.15.52:0/3193268379 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e94023a00 con 0x55f0651e7470 2023-11-27T15:20:05.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.894+0000 7f5ebda16b80 1 -- 172.21.15.52:0/3193268379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0651e7470 msgr2=0x55f0651f79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:05.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.894+0000 7f5ebda16b80 1 --2- 172.21.15.52:0/3193268379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0651e7470 0x55f0651f79d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5e940059d0 tx=0x7f5e94005000 comp rx=0 tx=0).stop 2023-11-27T15:20:05.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.894+0000 7f5ebda16b80 1 -- 172.21.15.52:0/3193268379 shutdown_connections 2023-11-27T15:20:05.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.894+0000 7f5ebda16b80 1 --2- 172.21.15.52:0/3193268379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0651eb8b0 0x55f0651f2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.894+0000 7f5ebda16b80 1 --2- 172.21.15.52:0/3193268379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0651e6b60 0x55f0651e91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.894+0000 7f5ebda16b80 1 --2- 172.21.15.52:0/3193268379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0651e7470 0x55f0651f79d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.894+0000 7f5ebda16b80 1 -- 172.21.15.52:0/3193268379 >> 172.21.15.52:0/3193268379 conn(0x55f0651dc360 msgr2=0x55f0651fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:05.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.894+0000 7f5ebda16b80 1 -- 172.21.15.52:0/3193268379 shutdown_connections 2023-11-27T15:20:05.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.894+0000 7f5ebda16b80 1 -- 172.21.15.52:0/3193268379 wait complete. 2023-11-27T15:20:05.917 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.913+0000 7f5ebda16b80 1 -- start start 2023-11-27T15:20:05.917 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.914+0000 7f5ebda16b80 1 Processor -- start 2023-11-27T15:20:05.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.914+0000 7f5ebda16b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0651e6b60 0x55f0651efe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:05.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.914+0000 7f5ebda16b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0651e7470 0x55f0651f03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:05.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.914+0000 7f5ebda16b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0651eb8b0 0x55f0651eecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:05.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.914+0000 7f5ebda16b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0651ef3c0 con 0x55f0651e7470 2023-11-27T15:20:05.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.914+0000 7f5ebda16b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0651ef530 con 0x55f0651eb8b0 2023-11-27T15:20:05.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.914+0000 7f5ebda16b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0651ef6f0 con 0x55f0651e6b60 2023-11-27T15:20:05.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.914+0000 7f5ebda16b80 10 client.0 _ll_register_callbacks cb 0x7f5ea40091d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:20:05.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.914+0000 7f5eaa7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0651e7470 0x55f0651f03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:05.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.914+0000 7f5eaa7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0651e7470 0x55f0651f03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53218/0 (socket says 172.21.15.52:53218) 2023-11-27T15:20:05.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.914+0000 7f5eaa7fc700 1 -- 172.21.15.52:0/745359768 learned_addr learned my addr 172.21.15.52:0/745359768 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:05.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.914+0000 7f5eab7fe700 1 --2- 172.21.15.52:0/745359768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0651eb8b0 0x55f0651eecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:05.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.914+0000 7f5eaaffd700 1 --2- 172.21.15.52:0/745359768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0651e6b60 0x55f0651efe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:05.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.915+0000 7f5eaa7fc700 1 -- 172.21.15.52:0/745359768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0651e6b60 msgr2=0x55f0651efe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:05.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.915+0000 7f5eaa7fc700 1 --2- 172.21.15.52:0/745359768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0651e6b60 0x55f0651efe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.915+0000 7f5eaa7fc700 1 -- 172.21.15.52:0/745359768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0651eb8b0 msgr2=0x55f0651eecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:05.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.915+0000 7f5eaa7fc700 1 --2- 172.21.15.52:0/745359768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0651eb8b0 0x55f0651eecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.915+0000 7f5eaa7fc700 1 -- 172.21.15.52:0/745359768 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f0650fe190 con 0x55f0651e7470 2023-11-27T15:20:05.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.915+0000 7f5eaa7fc700 1 --2- 172.21.15.52:0/745359768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0651e7470 0x55f0651f03d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5e94005f50 tx=0x7f5e940334b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e8b39b672f37e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:05.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.915+0000 7f5e8affd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:20:05.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.915+0000 7f5e8affd700 1 -- 172.21.15.52:0/745359768 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9402c040 con 0x55f0651e7470 2023-11-27T15:20:05.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.915+0000 7f5e8affd700 1 -- 172.21.15.52:0/745359768 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5e94033de0 con 0x55f0651e7470 2023-11-27T15:20:05.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.915+0000 7f5ebda16b80 1 -- 172.21.15.52:0/745359768 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e9c0a20d0 con 0x55f0651e7470 2023-11-27T15:20:05.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.915+0000 7f5e8affd700 1 -- 172.21.15.52:0/745359768 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e94005070 con 0x55f0651e7470 2023-11-27T15:20:05.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.915+0000 7f5ebda16b80 10 client.15912 fetch_fsmap learned FSMap version 23 2023-11-27T15:20:05.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.915+0000 7f5ebda16b80 1 -- 172.21.15.52:0/745359768 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f5eac10d990 con 0x55f0651e7470 2023-11-27T15:20:05.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.915+0000 7f5e8affd700 1 -- 172.21.15.52:0/745359768 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e9c0a20d0 con 0x55f0651e7470 2023-11-27T15:20:05.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.916+0000 7f5e8affd700 1 -- 172.21.15.52:0/745359768 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5e9400f460 con 0x55f0651e7470 2023-11-27T15:20:05.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.916+0000 7f5ebda16b80 10 client.15912 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:20:05.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.916+0000 7f5ebda16b80 10 client.15912 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:20:05.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.916+0000 7f5ebda16b80 20 client.15912 populate_metadata read hostname 'smithi052' 2023-11-27T15:20:05.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.916+0000 7f5ebda16b80 1 --2- 172.21.15.52:0/745359768 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55f065254e60 0x55f065257270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:05.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.916+0000 7f5ebda16b80 4 client.15912 mds_command: new command op to 24494 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:20:05.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.916+0000 7f5ebda16b80 1 -- 172.21.15.52:0/745359768 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f065106660 con 0x55f065254e60 2023-11-27T15:20:05.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.916+0000 7f5eaaffd700 1 --2- 172.21.15.52:0/745359768 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55f065254e60 0x55f065257270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:05.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.916+0000 7f5e8affd700 10 client.15912 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:20:05.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.916+0000 7f5eaaffd700 1 --2- 172.21.15.52:0/745359768 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55f065254e60 0x55f065257270 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8f509ad72c8b25b2 server_cookie=f42b2d58acb4385 in_seq=0 out_seq=0 2023-11-27T15:20:05.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.917+0000 7f5e8affd700 1 -- 172.21.15.52:0/745359768 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f065106660 con 0x55f065254e60 2023-11-27T15:20:05.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.917+0000 7f5e8affd700 10 client.15912 handle_command_reply: tid=0 2023-11-27T15:20:05.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.917+0000 7f5ebda16b80 1 client.15912 shutdown 2023-11-27T15:20:05.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.917+0000 7f5ebda16b80 1 -- 172.21.15.52:0/745359768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0651e7470 msgr2=0x55f0651f03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:05.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.917+0000 7f5ebda16b80 1 --2- 172.21.15.52:0/745359768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0651e7470 0x55f0651f03d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5e94005f50 tx=0x7f5e940334b0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.917+0000 7f5ebda16b80 1 -- 172.21.15.52:0/745359768 shutdown_connections 2023-11-27T15:20:05.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.917+0000 7f5ebda16b80 1 --2- 172.21.15.52:0/745359768 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55f065254e60 0x55f065257270 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.917+0000 7f5ebda16b80 1 --2- 172.21.15.52:0/745359768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0651eb8b0 0x55f0651eecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.917+0000 7f5ebda16b80 1 --2- 172.21.15.52:0/745359768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f0651e6b60 0x55f0651efe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.917+0000 7f5e8affd700 0 client.15912 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:20:05.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.917+0000 7f5ebda16b80 1 --2- 172.21.15.52:0/745359768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0651e7470 0x55f0651f03d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.917+0000 7f5ebda16b80 1 -- 172.21.15.52:0/745359768 >> 172.21.15.52:0/745359768 conn(0x55f0651dc360 msgr2=0x55f0651e9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:05.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.917+0000 7f5ebda16b80 1 -- 172.21.15.52:0/745359768 shutdown_connections 2023-11-27T15:20:05.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.917+0000 7f5ebda16b80 1 -- 172.21.15.52:0/745359768 wait complete. 2023-11-27T15:20:05.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.917+0000 7f5ebda16b80 20 client.15912 trim_cache size 0 max 16384 2023-11-27T15:20:05.925 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:20:05.933 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.929+0000 7f5eb188a700 1 -- 172.21.15.52:0/1834943305 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f5e980a12c0 msgr2=0x7f5e980a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:05.933 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.929+0000 7f5eb188a700 1 --2- 172.21.15.52:0/1834943305 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f5e980a12c0 0x7f5e980a3770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5e9400b5c0 tx=0x7f5e94005960 comp rx=0 tx=0).stop 2023-11-27T15:20:05.933 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.929+0000 7f5eb188a700 1 -- 172.21.15.52:0/1834943305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5eac101390 msgr2=0x7f5eac0ff1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:05.933 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.929+0000 7f5eb188a700 1 --2- 172.21.15.52:0/1834943305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5eac101390 0x7f5eac0ff1c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00d980 tx=0x7f5e9c00dc90 comp rx=0 tx=0).stop 2023-11-27T15:20:05.933 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.929+0000 7f5eb188a700 1 -- 172.21.15.52:0/1834943305 shutdown_connections 2023-11-27T15:20:05.933 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.929+0000 7f5eb188a700 1 --2- 172.21.15.52:0/1834943305 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f5e980a12c0 0x7f5e980a3770 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.933 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.929+0000 7f5eb188a700 1 --2- 172.21.15.52:0/1834943305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5eac10a8b0 0x7f5eac1a45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.933 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.929+0000 7f5eb188a700 1 --2- 172.21.15.52:0/1834943305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5eac101390 0x7f5eac0ff1c0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.933 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.929+0000 7f5eb188a700 1 --2- 172.21.15.52:0/1834943305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5eac101ca0 0x7f5eac0ff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.933 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.929+0000 7f5eb188a700 1 -- 172.21.15.52:0/1834943305 >> 172.21.15.52:0/1834943305 conn(0x7f5eac0770b0 msgr2=0x7f5eac0fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:05.933 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.929+0000 7f5eb188a700 1 -- 172.21.15.52:0/1834943305 shutdown_connections 2023-11-27T15:20:05.934 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:05.929+0000 7f5eb188a700 1 -- 172.21.15.52:0/1834943305 wait complete. 2023-11-27T15:20:06.049 INFO:tasks.workunit.client.0.smithi052.stderr:++ getfacl d1 2023-11-27T15:20:06.049 INFO:tasks.workunit.client.0.smithi052.stderr:++ grep -c nobody:rw 2023-11-27T15:20:06.063 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-27T15:20:06.250 INFO:tasks.workunit.client.0.smithi052.stderr:+ c2=2 2023-11-27T15:20:06.250 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:20:06.265 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:20:06.265 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:20:06.265 INFO:tasks.workunit.client.0.smithi052.stderr:+ '[' 100 -lt 100 ']' 2023-11-27T15:20:06.274 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir d1 2023-11-27T15:20:06.299 INFO:tasks.workunit.client.0.smithi052.stderr:+ setfattr -n system.posix_acl_access -v 0x02000000 d1 2023-11-27T15:20:06.304 INFO:tasks.workunit.client.0.smithi052.stderr:+ setfattr -n system.posix_acl_default -v 0x02000000 . 2023-11-27T15:20:06.307 INFO:tasks.workunit.client.0.smithi052.stderr:+ expect_failure getfattr -n system.posix_acl_access d1 2023-11-27T15:20:06.307 INFO:tasks.workunit.client.0.smithi052.stderr:+ getfattr -n system.posix_acl_access d1 2023-11-27T15:20:06.330 INFO:tasks.workunit.client.0.smithi052.stderr:d1: system.posix_acl_access: No such attribute 2023-11-27T15:20:06.330 INFO:tasks.workunit.client.0.smithi052.stderr:+ return 0 2023-11-27T15:20:06.331 INFO:tasks.workunit.client.0.smithi052.stderr:+ expect_failure getfattr -n system.posix_acl_default . 2023-11-27T15:20:06.331 INFO:tasks.workunit.client.0.smithi052.stderr:+ getfattr -n system.posix_acl_default . 2023-11-27T15:20:06.331 INFO:tasks.workunit.client.0.smithi052.stderr:.: system.posix_acl_default: No such attribute 2023-11-27T15:20:06.331 INFO:tasks.workunit.client.0.smithi052.stderr:+ return 0 2023-11-27T15:20:06.331 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir d1 2023-11-27T15:20:06.334 INFO:tasks.workunit.client.0.smithi052.stderr:+ cd .. 2023-11-27T15:20:06.334 INFO:tasks.workunit.client.0.smithi052.stderr:+ rmdir testdir 2023-11-27T15:20:06.345 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo OK 2023-11-27T15:20:06.346 INFO:tasks.workunit.client.0.smithi052.stdout:OK 2023-11-27T15:20:06.346 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-27T15:20:06.346 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-27T15:20:06.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:06 smithi052 ceph-mon[99056]: pgmap v68: 97 pgs: 97 active+clean; 2.8 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 43 KiB/s wr, 19 op/s 2023-11-27T15:20:06.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:06 smithi150 ceph-mon[100559]: pgmap v68: 97 pgs: 97 active+clean; 2.8 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 43 KiB/s wr, 19 op/s 2023-11-27T15:20:06.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:06 smithi102 ceph-mon[100296]: pgmap v68: 97 pgs: 97 active+clean; 2.8 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 43 KiB/s wr, 19 op/s 2023-11-27T15:20:06.702 INFO:tasks.workunit:Running workunit fs/misc/chmod.sh... 2023-11-27T15:20:06.702 DEBUG:teuthology.orchestra.run.smithi052:workunit test fs/misc/chmod.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/chmod.sh 2023-11-27T15:20:06.808 INFO:tasks.workunit.client.0.smithi052.stderr:+ set -e 2023-11-27T15:20:06.809 INFO:tasks.workunit.client.0.smithi052.stderr:+ file=test_chmod.153357 2023-11-27T15:20:06.809 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo foo 2023-11-27T15:20:06.812 INFO:tasks.workunit.client.0.smithi052.stderr:+ test 0 '!=' 0 2023-11-27T15:20:06.812 INFO:tasks.workunit.client.0.smithi052.stderr:+ chmod 400 test_chmod.153357 2023-11-27T15:20:06.814 INFO:tasks.workunit.client.0.smithi052.stderr:+ test 0 '!=' 0 2023-11-27T15:20:06.814 INFO:tasks.workunit.client.0.smithi052.stderr:+ check_perms test_chmod.153357 -r-------- 2023-11-27T15:20:06.814 INFO:tasks.workunit.client.0.smithi052.stderr:+ file=test_chmod.153357 2023-11-27T15:20:06.815 INFO:tasks.workunit.client.0.smithi052.stderr:++ ls -la test_chmod.153357 2023-11-27T15:20:06.829 INFO:tasks.workunit.client.0.smithi052.stderr:+ r='-r--------. 1 ubuntu ubuntu 4 Nov 27 15:20 test_chmod.153357' 2023-11-27T15:20:06.829 INFO:tasks.workunit.client.0.smithi052.stderr:+ test 0 '!=' 0 2023-11-27T15:20:06.829 INFO:tasks.workunit.client.0.smithi052.stderr:+ perms=-r-------- 2023-11-27T15:20:06.829 INFO:tasks.workunit.client.0.smithi052.stderr:++ echo -r--------. 1 ubuntu ubuntu 4 Nov 27 15:20 test_chmod.153357 2023-11-27T15:20:06.830 INFO:tasks.workunit.client.0.smithi052.stderr:++ awk '{print $1}' 2023-11-27T15:20:06.897 INFO:tasks.workunit.client.0.smithi052.stderr:+ test -r-------- '!=' -r--------. 2023-11-27T15:20:06.898 INFO:tasks.workunit.client.0.smithi052.stderr:++ echo -r--------. 1 ubuntu ubuntu 4 Nov 27 15:20 test_chmod.153357 2023-11-27T15:20:06.898 INFO:tasks.workunit.client.0.smithi052.stderr:++ awk '{print $1}' 2023-11-27T15:20:06.900 INFO:tasks.workunit.client.0.smithi052.stderr:+ test -r--------. '!=' -r--------. 2023-11-27T15:20:06.900 INFO:tasks.workunit.client.0.smithi052.stderr:+ set +e 2023-11-27T15:20:06.900 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo bar 2023-11-27T15:20:06.900 INFO:tasks.workunit.client.0.smithi052.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/chmod.sh: line 40: test_chmod.153357: Permission denied 2023-11-27T15:20:06.900 INFO:tasks.workunit.client.0.smithi052.stderr:+ test 1 = 0 2023-11-27T15:20:06.900 INFO:tasks.workunit.client.0.smithi052.stderr:+ set -e 2023-11-27T15:20:06.900 INFO:tasks.workunit.client.0.smithi052.stderr:+ chmod 600 test_chmod.153357 2023-11-27T15:20:06.901 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo bar 2023-11-27T15:20:06.901 INFO:tasks.workunit.client.0.smithi052.stderr:+ test 0 '!=' 0 2023-11-27T15:20:06.901 INFO:tasks.workunit.client.0.smithi052.stderr:+ check_perms test_chmod.153357 -rw------- 2023-11-27T15:20:06.901 INFO:tasks.workunit.client.0.smithi052.stderr:+ file=test_chmod.153357 2023-11-27T15:20:06.901 INFO:tasks.workunit.client.0.smithi052.stderr:++ ls -la test_chmod.153357 2023-11-27T15:20:06.902 INFO:tasks.workunit.client.0.smithi052.stderr:+ r='-rw-------. 1 ubuntu ubuntu 8 Nov 27 15:20 test_chmod.153357' 2023-11-27T15:20:06.902 INFO:tasks.workunit.client.0.smithi052.stderr:+ test 0 '!=' 0 2023-11-27T15:20:06.902 INFO:tasks.workunit.client.0.smithi052.stderr:+ perms=-rw------- 2023-11-27T15:20:06.903 INFO:tasks.workunit.client.0.smithi052.stderr:++ echo -rw-------. 1 ubuntu ubuntu 8 Nov 27 15:20 test_chmod.153357 2023-11-27T15:20:06.903 INFO:tasks.workunit.client.0.smithi052.stderr:++ awk '{print $1}' 2023-11-27T15:20:06.904 INFO:tasks.workunit.client.0.smithi052.stderr:+ test -rw------- '!=' -rw-------. 2023-11-27T15:20:06.904 INFO:tasks.workunit.client.0.smithi052.stderr:++ echo -rw-------. 1 ubuntu ubuntu 8 Nov 27 15:20 test_chmod.153357 2023-11-27T15:20:06.904 INFO:tasks.workunit.client.0.smithi052.stderr:++ awk '{print $1}' 2023-11-27T15:20:06.907 INFO:tasks.workunit.client.0.smithi052.stderr:+ test -rw-------. '!=' -rw-------. 2023-11-27T15:20:06.907 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo foo 2023-11-27T15:20:06.907 INFO:tasks.workunit.client.0.smithi052.stderr:+ test 0 '!=' 0 2023-11-27T15:20:06.907 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-27T15:20:06.907 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-27T15:20:06.954 INFO:tasks.workunit:Running workunit fs/misc/dac_override.sh... 2023-11-27T15:20:06.954 DEBUG:teuthology.orchestra.run.smithi052:workunit test fs/misc/dac_override.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/dac_override.sh 2023-11-27T15:20:07.080 INFO:tasks.workunit.client.0.smithi052.stderr:+ set -e 2023-11-27T15:20:07.080 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir -p testdir 2023-11-27T15:20:07.082 INFO:tasks.workunit.client.0.smithi052.stderr:+ file=test_chmod.153416 2023-11-27T15:20:07.082 INFO:tasks.workunit.client.0.smithi052.stderr:+ echo foo 2023-11-27T15:20:07.083 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo chmod 600 testdir 2023-11-27T15:20:07.101 INFO:tasks.workunit.client.0.smithi052.stderr:+ expect_failure cat testdir/test_chmod.153416 2023-11-27T15:20:07.101 INFO:tasks.workunit.client.0.smithi052.stderr:+ cat testdir/test_chmod.153416 2023-11-27T15:20:07.105 INFO:tasks.workunit.client.0.smithi052.stderr:cat: testdir/test_chmod.153416: Permission denied 2023-11-27T15:20:07.105 INFO:tasks.workunit.client.0.smithi052.stderr:+ return 0 2023-11-27T15:20:07.105 INFO:tasks.workunit.client.0.smithi052.stderr:+ sudo cat testdir/test_chmod.153416 2023-11-27T15:20:07.116 INFO:tasks.workunit.client.0.smithi052.stdout:foo 2023-11-27T15:20:07.117 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-27T15:20:07.118 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-27T15:20:07.166 INFO:tasks.workunit:Running workunit fs/misc/direct_io.py... 2023-11-27T15:20:07.166 DEBUG:teuthology.orchestra.run.smithi052:workunit test fs/misc/direct_io.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/direct_io.py 2023-11-27T15:20:07.610 INFO:tasks.workunit.client.0.smithi052.stdout:ok 2023-11-27T15:20:07.615 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-27T15:20:07.615 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-27T15:20:07.656 INFO:tasks.workunit:Running workunit fs/misc/dirfrag.sh... 2023-11-27T15:20:07.657 DEBUG:teuthology.orchestra.run.smithi052:workunit test fs/misc/dirfrag.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/dirfrag.sh 2023-11-27T15:20:07.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.657+0000 7f93eec2e700 1 -- 172.21.15.52:0/888900217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93e81055d0 msgr2=0x7f93e810d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:07.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.657+0000 7f93eec2e700 1 --2- 172.21.15.52:0/888900217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93e81055d0 0x7f93e810d1e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f93e000b0d0 tx=0x7f93e000b3e0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.657+0000 7f93eec2e700 1 -- 172.21.15.52:0/888900217 shutdown_connections 2023-11-27T15:20:07.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.657+0000 7f93eec2e700 1 --2- 172.21.15.52:0/888900217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93e80feaf0 0x7f93e80feec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.657+0000 7f93eec2e700 1 --2- 172.21.15.52:0/888900217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93e81055d0 0x7f93e810d1e0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.657+0000 7f93eec2e700 1 --2- 172.21.15.52:0/888900217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93e80ff490 0x7f93e8104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.657+0000 7f93eec2e700 1 -- 172.21.15.52:0/888900217 >> 172.21.15.52:0/888900217 conn(0x7f93e8075920 msgr2=0x7f93e8075d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:07.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.657+0000 7f93eec2e700 1 -- 172.21.15.52:0/888900217 shutdown_connections 2023-11-27T15:20:07.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.657+0000 7f93eec2e700 1 -- 172.21.15.52:0/888900217 wait complete. 2023-11-27T15:20:07.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.658+0000 7f93eec2e700 1 Processor -- start 2023-11-27T15:20:07.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.658+0000 7f93eec2e700 1 -- start start 2023-11-27T15:20:07.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.658+0000 7f93eec2e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93e80feaf0 0x7f93e8196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:07.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.658+0000 7f93eec2e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93e80ff490 0x7f93e8196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:07.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.658+0000 7f93eec2e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93e81055d0 0x7f93e819aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:07.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.658+0000 7f93eec2e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e819b4f0 con 0x7f93e81055d0 2023-11-27T15:20:07.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.658+0000 7f93eec2e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e819b660 con 0x7f93e80feaf0 2023-11-27T15:20:07.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.658+0000 7f93eec2e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e819b820 con 0x7f93e80ff490 2023-11-27T15:20:07.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.658+0000 7f93ec9ca700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93e80feaf0 0x7f93e8196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:07.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.658+0000 7f93ed1cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93e81055d0 0x7f93e819aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:07.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.658+0000 7f93e7fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93e80ff490 0x7f93e8196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:07.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.658+0000 7f93ec9ca700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93e80feaf0 0x7f93e8196510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:42934/0 (socket says 172.21.15.52:42934) 2023-11-27T15:20:07.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.658+0000 7f93ed1cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93e81055d0 0x7f93e819aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53280/0 (socket says 172.21.15.52:53280) 2023-11-27T15:20:07.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.658+0000 7f93ec9ca700 1 -- 172.21.15.52:0/1141258967 learned_addr learned my addr 172.21.15.52:0/1141258967 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:07.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.658+0000 7f93e7fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93e80ff490 0x7f93e8196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:39922/0 (socket says 172.21.15.52:39922) 2023-11-27T15:20:07.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.658+0000 7f93ed1cb700 1 -- 172.21.15.52:0/1141258967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93e80ff490 msgr2=0x7f93e8196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:07.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.658+0000 7f93ed1cb700 1 --2- 172.21.15.52:0/1141258967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93e80ff490 0x7f93e8196a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.658+0000 7f93ed1cb700 1 -- 172.21.15.52:0/1141258967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93e80feaf0 msgr2=0x7f93e8196510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:07.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.658+0000 7f93ed1cb700 1 --2- 172.21.15.52:0/1141258967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93e80feaf0 0x7f93e8196510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.658+0000 7f93ed1cb700 1 -- 172.21.15.52:0/1141258967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e0009d00 con 0x7f93e81055d0 2023-11-27T15:20:07.666 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.658+0000 7f93e7fff700 1 --2- 172.21.15.52:0/1141258967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93e80ff490 0x7f93e8196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:07.666 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.658+0000 7f93ec9ca700 1 --2- 172.21.15.52:0/1141258967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93e80feaf0 0x7f93e8196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:07.666 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.659+0000 7f93ed1cb700 1 --2- 172.21.15.52:0/1141258967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93e81055d0 0x7f93e819aeb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f93e000b770 tx=0x7f93e0007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:07.666 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.659+0000 7f93e5ffb700 1 -- 172.21.15.52:0/1141258967 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e000e040 con 0x7f93e81055d0 2023-11-27T15:20:07.666 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.659+0000 7f93e5ffb700 1 -- 172.21.15.52:0/1141258967 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93e001d930 con 0x7f93e81055d0 2023-11-27T15:20:07.666 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.659+0000 7f93e5ffb700 1 -- 172.21.15.52:0/1141258967 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e001b680 con 0x7f93e81055d0 2023-11-27T15:20:07.666 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.659+0000 7f93eec2e700 1 -- 172.21.15.52:0/1141258967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e81103f0 con 0x7f93e81055d0 2023-11-27T15:20:07.666 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.659+0000 7f93eec2e700 1 -- 172.21.15.52:0/1141258967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e8110260 con 0x7f93e81055d0 2023-11-27T15:20:07.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.662+0000 7f93e5ffb700 1 -- 172.21.15.52:0/1141258967 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f93e001d470 con 0x7f93e81055d0 2023-11-27T15:20:07.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.662+0000 7f93e5ffb700 1 --2- 172.21.15.52:0/1141258967 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f93d80a12c0 0x7f93d80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:07.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.663+0000 7f93e5ffb700 1 -- 172.21.15.52:0/1141258967 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f93e00ce0b0 con 0x7f93e81055d0 2023-11-27T15:20:07.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.663+0000 7f93ec9ca700 1 --2- 172.21.15.52:0/1141258967 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f93d80a12c0 0x7f93d80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:07.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.663+0000 7f93ec9ca700 1 --2- 172.21.15.52:0/1141258967 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f93d80a12c0 0x7f93d80a3770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f93dc011550 tx=0x7f93dc009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:07.708 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.703+0000 7f93fadbab80 1 Processor -- start 2023-11-27T15:20:07.708 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.703+0000 7f93fadbab80 1 -- start start 2023-11-27T15:20:07.708 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.703+0000 7f93fadbab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b2369ab60 0x564b2369af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:07.708 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.703+0000 7f93fadbab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b2369b470 0x564b2369f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:07.708 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.703+0000 7f93fadbab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b2369f8b0 0x564b2369fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:07.708 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.703+0000 7f93fadbab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b235a0060 con 0x564b2369f8b0 2023-11-27T15:20:07.708 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.703+0000 7f93fadbab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b235f28a0 con 0x564b2369ab60 2023-11-27T15:20:07.708 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.703+0000 7f93fadbab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b235ebd30 con 0x564b2369b470 2023-11-27T15:20:07.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.703+0000 7f93ed1cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b2369f8b0 0x564b2369fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:07.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.703+0000 7f93ec9ca700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b2369ab60 0x564b2369af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:07.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.703+0000 7f93ed1cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b2369f8b0 0x564b2369fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53288/0 (socket says 172.21.15.52:53288) 2023-11-27T15:20:07.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.703+0000 7f93ed1cb700 1 -- 172.21.15.52:0/3414543047 learned_addr learned my addr 172.21.15.52:0/3414543047 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:07.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.703+0000 7f93e7fff700 1 --2- 172.21.15.52:0/3414543047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b2369b470 0x564b2369f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:07.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.703+0000 7f93ed1cb700 1 -- 172.21.15.52:0/3414543047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b2369b470 msgr2=0x564b2369f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:07.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.703+0000 7f93ed1cb700 1 --2- 172.21.15.52:0/3414543047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b2369b470 0x564b2369f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.703+0000 7f93ed1cb700 1 -- 172.21.15.52:0/3414543047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b2369ab60 msgr2=0x564b2369af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:07.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.703+0000 7f93ed1cb700 1 --2- 172.21.15.52:0/3414543047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b2369ab60 0x564b2369af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.704+0000 7f93ed1cb700 1 -- 172.21.15.52:0/3414543047 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b2346d630 con 0x564b2369f8b0 2023-11-27T15:20:07.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.704+0000 7f93ed1cb700 1 --2- 172.21.15.52:0/3414543047 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b2369f8b0 0x564b2369fd00 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f93e0097710 tx=0x7f93e00095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96f8681ac1539a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:07.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.704+0000 7f93d2ffd700 1 -- 172.21.15.52:0/3414543047 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e009b250 con 0x564b2369f8b0 2023-11-27T15:20:07.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.704+0000 7f93d2ffd700 1 -- 172.21.15.52:0/3414543047 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93e009f240 con 0x564b2369f8b0 2023-11-27T15:20:07.710 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.704+0000 7f93d2ffd700 1 -- 172.21.15.52:0/3414543047 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e009b6d0 con 0x564b2369f8b0 2023-11-27T15:20:07.714 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.704+0000 7f93fadbab80 1 -- 172.21.15.52:0/3414543047 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b2369f8b0 msgr2=0x564b2369fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:07.714 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.704+0000 7f93fadbab80 1 --2- 172.21.15.52:0/3414543047 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b2369f8b0 0x564b2369fd00 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f93e0097710 tx=0x7f93e00095a0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.714 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.705+0000 7f93fadbab80 1 -- 172.21.15.52:0/3414543047 shutdown_connections 2023-11-27T15:20:07.714 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.705+0000 7f93fadbab80 1 --2- 172.21.15.52:0/3414543047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b2369ab60 0x564b2369af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.714 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.705+0000 7f93fadbab80 1 --2- 172.21.15.52:0/3414543047 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b2369f8b0 0x564b2369fd00 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.714 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.705+0000 7f93fadbab80 1 --2- 172.21.15.52:0/3414543047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b2369b470 0x564b2369f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.714 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.705+0000 7f93fadbab80 1 -- 172.21.15.52:0/3414543047 >> 172.21.15.52:0/3414543047 conn(0x564b23690360 msgr2=0x564b23695b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:07.714 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.705+0000 7f93fadbab80 1 -- 172.21.15.52:0/3414543047 shutdown_connections 2023-11-27T15:20:07.714 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.705+0000 7f93fadbab80 1 -- 172.21.15.52:0/3414543047 wait complete. 2023-11-27T15:20:07.714 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.705+0000 7f93fadbab80 1 -- start start 2023-11-27T15:20:07.715 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.705+0000 7f93fadbab80 1 Processor -- start 2023-11-27T15:20:07.715 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.706+0000 7f93fadbab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b2369ab60 0x564b236a70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:07.715 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.706+0000 7f93fadbab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b2369b470 0x564b236a4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:07.715 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.706+0000 7f93fadbab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b2369f8b0 0x564b236a4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:07.715 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.706+0000 7f93fadbab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b2358d8e0 con 0x564b2369b470 2023-11-27T15:20:07.715 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.706+0000 7f93fadbab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b2358d580 con 0x564b2369ab60 2023-11-27T15:20:07.715 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.706+0000 7f93fadbab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b2346f920 con 0x564b2369f8b0 2023-11-27T15:20:07.715 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.706+0000 7f93fadbab80 10 client.0 _ll_register_callbacks cb 0x564b235ab780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:20:07.715 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.706+0000 7f93e7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b2369b470 0x564b236a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:07.715 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.706+0000 7f93e7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b2369b470 0x564b236a4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53292/0 (socket says 172.21.15.52:53292) 2023-11-27T15:20:07.715 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.706+0000 7f93e7fff700 1 -- 172.21.15.52:0/2335518035 learned_addr learned my addr 172.21.15.52:0/2335518035 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:07.715 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.706+0000 7f93ed1cb700 1 --2- 172.21.15.52:0/2335518035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b2369f8b0 0x564b236a4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:07.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.706+0000 7f93e7fff700 1 -- 172.21.15.52:0/2335518035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b2369f8b0 msgr2=0x564b236a4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:07.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.706+0000 7f93e7fff700 1 --2- 172.21.15.52:0/2335518035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b2369f8b0 0x564b236a4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.706+0000 7f93e7fff700 1 -- 172.21.15.52:0/2335518035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b2369ab60 msgr2=0x564b236a70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:07.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.706+0000 7f93e7fff700 1 --2- 172.21.15.52:0/2335518035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b2369ab60 0x564b236a70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.706+0000 7f93e7fff700 1 -- 172.21.15.52:0/2335518035 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e8110260 con 0x564b2369b470 2023-11-27T15:20:07.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.706+0000 7f93e7fff700 1 --2- 172.21.15.52:0/2335518035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b2369b470 0x564b236a4300 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f93d4012d20 tx=0x7f93d400f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6016341c23170ee6 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:07.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.706+0000 7f93c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:20:07.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.706+0000 7f93c7fff700 1 -- 172.21.15.52:0/2335518035 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d4017070 con 0x564b2369b470 2023-11-27T15:20:07.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.706+0000 7f93c7fff700 1 -- 172.21.15.52:0/2335518035 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f93d400fbd0 con 0x564b2369b470 2023-11-27T15:20:07.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.706+0000 7f93fadbab80 1 -- 172.21.15.52:0/2335518035 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564b2368db20 con 0x564b2369b470 2023-11-27T15:20:07.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.707+0000 7f93c7fff700 1 -- 172.21.15.52:0/2335518035 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93d400fd90 con 0x564b2369b470 2023-11-27T15:20:07.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.707+0000 7f93fadbab80 10 client.15942 fetch_fsmap learned FSMap version 23 2023-11-27T15:20:07.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.707+0000 7f93fadbab80 1 -- 172.21.15.52:0/2335518035 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x564b235b2190 con 0x564b2369b470 2023-11-27T15:20:07.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.707+0000 7f93c7fff700 1 -- 172.21.15.52:0/2335518035 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564b2368db20 con 0x564b2369b470 2023-11-27T15:20:07.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.707+0000 7f93c7fff700 1 -- 172.21.15.52:0/2335518035 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f93d4003c20 con 0x564b2369b470 2023-11-27T15:20:07.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.707+0000 7f93fadbab80 10 client.15942 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:20:07.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.707+0000 7f93fadbab80 10 client.15942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:20:07.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.707+0000 7f93fadbab80 20 client.15942 populate_metadata read hostname 'smithi052' 2023-11-27T15:20:07.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.707+0000 7f93fadbab80 1 --2- 172.21.15.52:0/2335518035 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x564b236b4720 0x564b236b6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:07.718 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.707+0000 7f93fadbab80 4 client.15942 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:20:07.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.707+0000 7f93fadbab80 1 -- 172.21.15.52:0/2335518035 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b235ae0a0 con 0x564b236b4720 2023-11-27T15:20:07.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.707+0000 7f93ec9ca700 1 --2- 172.21.15.52:0/2335518035 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x564b236b4720 0x564b236b6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:07.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.708+0000 7f93c7fff700 10 client.15942 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:20:07.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.708+0000 7f93ec9ca700 1 --2- 172.21.15.52:0/2335518035 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x564b236b4720 0x564b236b6b30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11dd4396638be0fb server_cookie=9617d36f14175096 in_seq=0 out_seq=0 2023-11-27T15:20:07.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.708+0000 7f93c7fff700 1 -- 172.21.15.52:0/2335518035 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564b235ae0a0 con 0x564b236b4720 2023-11-27T15:20:07.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.708+0000 7f93c7fff700 10 client.15942 handle_command_reply: tid=0 2023-11-27T15:20:07.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.708+0000 7f93fadbab80 1 client.15942 shutdown 2023-11-27T15:20:07.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.708+0000 7f93fadbab80 1 -- 172.21.15.52:0/2335518035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b2369b470 msgr2=0x564b236a4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:07.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.708+0000 7f93fadbab80 1 --2- 172.21.15.52:0/2335518035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b2369b470 0x564b236a4300 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f93d4012d20 tx=0x7f93d400f6a0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.708+0000 7f93fadbab80 1 -- 172.21.15.52:0/2335518035 shutdown_connections 2023-11-27T15:20:07.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.708+0000 7f93fadbab80 1 --2- 172.21.15.52:0/2335518035 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x564b236b4720 0x564b236b6b30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.719 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.708+0000 7f93c7fff700 0 client.15942 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:20:07.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.708+0000 7f93fadbab80 1 --2- 172.21.15.52:0/2335518035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b2369ab60 0x564b236a70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.708+0000 7f93fadbab80 1 --2- 172.21.15.52:0/2335518035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b2369f8b0 0x564b236a4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.708+0000 7f93fadbab80 1 --2- 172.21.15.52:0/2335518035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b2369b470 0x564b236a4300 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.708+0000 7f93fadbab80 1 -- 172.21.15.52:0/2335518035 >> 172.21.15.52:0/2335518035 conn(0x564b23690360 msgr2=0x564b236a0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:07.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.709+0000 7f93fadbab80 1 -- 172.21.15.52:0/2335518035 shutdown_connections 2023-11-27T15:20:07.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.709+0000 7f93fadbab80 1 -- 172.21.15.52:0/2335518035 wait complete. 2023-11-27T15:20:07.720 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.709+0000 7f93fadbab80 20 client.15942 trim_cache size 0 max 16384 2023-11-27T15:20:07.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.722+0000 7f93fadbab80 1 Processor -- start 2023-11-27T15:20:07.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.722+0000 7f93fadbab80 1 -- start start 2023-11-27T15:20:07.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.722+0000 7f93fadbab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b2369ab60 0x564b236ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:07.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.722+0000 7f93fadbab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b2369b470 0x564b236a6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:07.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.722+0000 7f93fadbab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b2369f8b0 0x564b236a3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:07.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.722+0000 7f93fadbab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b2369e480 con 0x564b2369f8b0 2023-11-27T15:20:07.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.722+0000 7f93fadbab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b2369b900 con 0x564b2369b470 2023-11-27T15:20:07.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.722+0000 7f93fadbab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b236abcf0 con 0x564b2369ab60 2023-11-27T15:20:07.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.722+0000 7f93ed1cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b2369f8b0 0x564b236a3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:07.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.722+0000 7f93ed1cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b2369f8b0 0x564b236a3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53302/0 (socket says 172.21.15.52:53302) 2023-11-27T15:20:07.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.722+0000 7f93ed1cb700 1 -- 172.21.15.52:0/3758120824 learned_addr learned my addr 172.21.15.52:0/3758120824 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:07.727 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.722+0000 7f93e7fff700 1 --2- 172.21.15.52:0/3758120824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b2369b470 0x564b236a6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:07.728 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.722+0000 7f93ed1cb700 1 -- 172.21.15.52:0/3758120824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b2369ab60 msgr2=0x564b236ab7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:07.728 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.722+0000 7f93ed1cb700 1 --2- 172.21.15.52:0/3758120824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b2369ab60 0x564b236ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.728 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.722+0000 7f93ed1cb700 1 -- 172.21.15.52:0/3758120824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b2369b470 msgr2=0x564b236a6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:07.728 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.722+0000 7f93ed1cb700 1 --2- 172.21.15.52:0/3758120824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b2369b470 0x564b236a6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.728 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.722+0000 7f93ed1cb700 1 -- 172.21.15.52:0/3758120824 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b235b1ee0 con 0x564b2369f8b0 2023-11-27T15:20:07.728 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.722+0000 7f93ed1cb700 1 --2- 172.21.15.52:0/3758120824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b2369f8b0 0x564b236a3ba0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f93e0097c80 tx=0x7f93e000b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee2f269b7a6681e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:07.728 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.723+0000 7f93d37fe700 1 -- 172.21.15.52:0/3758120824 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e009b210 con 0x564b2369f8b0 2023-11-27T15:20:07.728 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.723+0000 7f93d37fe700 1 -- 172.21.15.52:0/3758120824 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93e00a35d0 con 0x564b2369f8b0 2023-11-27T15:20:07.728 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.723+0000 7f93d37fe700 1 -- 172.21.15.52:0/3758120824 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e009a700 con 0x564b2369f8b0 2023-11-27T15:20:07.728 INFO:tasks.workunit.client.0.smithi052.stdout:creating folder hierarchy 2023-11-27T15:20:07.732 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.723+0000 7f93fadbab80 1 -- 172.21.15.52:0/3758120824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b2369f8b0 msgr2=0x564b236a3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:07.732 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.723+0000 7f93fadbab80 1 --2- 172.21.15.52:0/3758120824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b2369f8b0 0x564b236a3ba0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f93e0097c80 tx=0x7f93e000b3e0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.732 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.723+0000 7f93fadbab80 1 -- 172.21.15.52:0/3758120824 shutdown_connections 2023-11-27T15:20:07.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.723+0000 7f93fadbab80 1 --2- 172.21.15.52:0/3758120824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b2369f8b0 0x564b236a3ba0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.723+0000 7f93fadbab80 1 --2- 172.21.15.52:0/3758120824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b2369ab60 0x564b236ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.723+0000 7f93fadbab80 1 --2- 172.21.15.52:0/3758120824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b2369b470 0x564b236a6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.723+0000 7f93fadbab80 1 -- 172.21.15.52:0/3758120824 >> 172.21.15.52:0/3758120824 conn(0x564b23690360 msgr2=0x564b2369d930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:07.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.723+0000 7f93fadbab80 1 -- 172.21.15.52:0/3758120824 shutdown_connections 2023-11-27T15:20:07.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.723+0000 7f93fadbab80 1 -- 172.21.15.52:0/3758120824 wait complete. 2023-11-27T15:20:07.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.724+0000 7f93fadbab80 1 -- start start 2023-11-27T15:20:07.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.724+0000 7f93fadbab80 1 Processor -- start 2023-11-27T15:20:07.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.724+0000 7f93fadbab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b2369ab60 0x564b23694550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:07.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.724+0000 7f93fadbab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b2369b470 0x564b23694a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:07.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.724+0000 7f93fadbab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b2369f8b0 0x564b23694fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:07.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.724+0000 7f93fadbab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b23695510 con 0x564b2369ab60 2023-11-27T15:20:07.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.724+0000 7f93fadbab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b23695680 con 0x564b2369f8b0 2023-11-27T15:20:07.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.724+0000 7f93fadbab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b23695840 con 0x564b2369b470 2023-11-27T15:20:07.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.724+0000 7f93fadbab80 10 client.0 _ll_register_callbacks cb 0x7f93d40127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:20:07.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.724+0000 7f93ec9ca700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b2369ab60 0x564b23694550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:07.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.724+0000 7f93ed1cb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b2369f8b0 0x564b23694fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:07.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.724+0000 7f93ec9ca700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b2369ab60 0x564b23694550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53304/0 (socket says 172.21.15.52:53304) 2023-11-27T15:20:07.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.724+0000 7f93ec9ca700 1 -- 172.21.15.52:0/1144075843 learned_addr learned my addr 172.21.15.52:0/1144075843 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:07.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.724+0000 7f93e7fff700 1 --2- 172.21.15.52:0/1144075843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b2369b470 0x564b23694a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:07.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.725+0000 7f93ed1cb700 1 -- 172.21.15.52:0/1144075843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b2369b470 msgr2=0x564b23694a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:07.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.725+0000 7f93ed1cb700 1 --2- 172.21.15.52:0/1144075843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b2369b470 0x564b23694a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.725+0000 7f93ed1cb700 1 -- 172.21.15.52:0/1144075843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b2369ab60 msgr2=0x564b23694550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:07.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.725+0000 7f93ed1cb700 1 --2- 172.21.15.52:0/1144075843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b2369ab60 0x564b23694550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.725+0000 7f93ed1cb700 1 -- 172.21.15.52:0/1144075843 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b2346d630 con 0x564b2369f8b0 2023-11-27T15:20:07.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.725+0000 7f93ec9ca700 1 --2- 172.21.15.52:0/1144075843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b2369ab60 0x564b23694550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:07.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.725+0000 7f93ed1cb700 1 --2- 172.21.15.52:0/1144075843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b2369f8b0 0x564b23694fd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f93e00cbe00 tx=0x7f93e00995b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73a8598a947d806a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:07.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.725+0000 7f93c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2023-11-27T15:20:07.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.725+0000 7f93c7fff700 1 -- 172.21.15.52:0/1144075843 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e009b210 con 0x564b2369f8b0 2023-11-27T15:20:07.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.725+0000 7f93fadbab80 1 -- 172.21.15.52:0/1144075843 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93e009f240 con 0x564b2369f8b0 2023-11-27T15:20:07.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.725+0000 7f93c7fff700 1 -- 172.21.15.52:0/1144075843 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f93e009a700 con 0x564b2369f8b0 2023-11-27T15:20:07.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.725+0000 7f93c7fff700 1 -- 172.21.15.52:0/1144075843 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93e009a8c0 con 0x564b2369f8b0 2023-11-27T15:20:07.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.725+0000 7f93fadbab80 10 client.25757 fetch_fsmap learned FSMap version 23 2023-11-27T15:20:07.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.725+0000 7f93fadbab80 1 -- 172.21.15.52:0/1144075843 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x564b23402c60 con 0x564b2369f8b0 2023-11-27T15:20:07.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.725+0000 7f93c7fff700 1 -- 172.21.15.52:0/1144075843 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93e009f240 con 0x564b2369f8b0 2023-11-27T15:20:07.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.725+0000 7f93c7fff700 1 -- 172.21.15.52:0/1144075843 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f93e00b9450 con 0x564b2369f8b0 2023-11-27T15:20:07.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.726+0000 7f93fadbab80 10 client.25757 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:20:07.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.726+0000 7f93fadbab80 10 client.25757 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:20:07.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.726+0000 7f93fadbab80 20 client.25757 populate_metadata read hostname 'smithi052' 2023-11-27T15:20:07.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.726+0000 7f93fadbab80 1 --2- 172.21.15.52:0/1144075843 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x564b236b4720 0x564b2370ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:07.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.726+0000 7f93fadbab80 4 client.25757 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:20:07.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.726+0000 7f93fadbab80 1 -- 172.21.15.52:0/1144075843 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564b235ba660 con 0x564b236b4720 2023-11-27T15:20:07.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.726+0000 7f93ec9ca700 1 --2- 172.21.15.52:0/1144075843 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x564b236b4720 0x564b2370ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:07.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.726+0000 7f93c7fff700 10 client.25757 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:20:07.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.726+0000 7f93ec9ca700 1 --2- 172.21.15.52:0/1144075843 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x564b236b4720 0x564b2370ad80 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ddebfa55e76dfb2 server_cookie=c485e95dd523a799 in_seq=0 out_seq=0 2023-11-27T15:20:07.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.726+0000 7f93c7fff700 1 -- 172.21.15.52:0/1144075843 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564b235ba660 con 0x564b236b4720 2023-11-27T15:20:07.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.726+0000 7f93c7fff700 10 client.25757 handle_command_reply: tid=0 2023-11-27T15:20:07.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.726+0000 7f93fadbab80 1 client.25757 shutdown 2023-11-27T15:20:07.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.726+0000 7f93fadbab80 1 -- 172.21.15.52:0/1144075843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b2369f8b0 msgr2=0x564b23694fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:07.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.726+0000 7f93fadbab80 1 --2- 172.21.15.52:0/1144075843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b2369f8b0 0x564b23694fd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f93e00cbe00 tx=0x7f93e00995b0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.726+0000 7f93fadbab80 1 -- 172.21.15.52:0/1144075843 shutdown_connections 2023-11-27T15:20:07.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.726+0000 7f93fadbab80 1 --2- 172.21.15.52:0/1144075843 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x564b236b4720 0x564b2370ad80 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.726+0000 7f93fadbab80 1 --2- 172.21.15.52:0/1144075843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b2369f8b0 0x564b23694fd0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.726+0000 7f93fadbab80 1 --2- 172.21.15.52:0/1144075843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b2369ab60 0x564b23694550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.726+0000 7f93fadbab80 1 --2- 172.21.15.52:0/1144075843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b2369b470 0x564b23694a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.726+0000 7f93c7fff700 0 client.25757 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:20:07.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.726+0000 7f93fadbab80 1 -- 172.21.15.52:0/1144075843 >> 172.21.15.52:0/1144075843 conn(0x564b23690360 msgr2=0x564b236a3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:07.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.727+0000 7f93fadbab80 1 -- 172.21.15.52:0/1144075843 shutdown_connections 2023-11-27T15:20:07.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.727+0000 7f93fadbab80 1 -- 172.21.15.52:0/1144075843 wait complete. 2023-11-27T15:20:07.738 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.727+0000 7f93fadbab80 20 client.25757 trim_cache size 0 max 16384 2023-11-27T15:20:07.738 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:20:07.739 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.727+0000 7f93eec2e700 1 -- 172.21.15.52:0/1141258967 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f93d80a12c0 msgr2=0x7f93d80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:07.739 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.727+0000 7f93eec2e700 1 --2- 172.21.15.52:0/1141258967 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f93d80a12c0 0x7f93d80a3770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f93dc011550 tx=0x7f93dc009260 comp rx=0 tx=0).stop 2023-11-27T15:20:07.739 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.727+0000 7f93eec2e700 1 -- 172.21.15.52:0/1141258967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93e81055d0 msgr2=0x7f93e819aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:07.739 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.727+0000 7f93eec2e700 1 --2- 172.21.15.52:0/1141258967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93e81055d0 0x7f93e819aeb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f93e000b770 tx=0x7f93e0007870 comp rx=0 tx=0).stop 2023-11-27T15:20:07.739 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.727+0000 7f93eec2e700 1 -- 172.21.15.52:0/1141258967 shutdown_connections 2023-11-27T15:20:07.739 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.727+0000 7f93eec2e700 1 --2- 172.21.15.52:0/1141258967 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f93d80a12c0 0x7f93d80a3770 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.739 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.727+0000 7f93eec2e700 1 --2- 172.21.15.52:0/1141258967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93e80feaf0 0x7f93e8196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.739 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.727+0000 7f93eec2e700 1 --2- 172.21.15.52:0/1141258967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93e81055d0 0x7f93e819aeb0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.739 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.727+0000 7f93eec2e700 1 --2- 172.21.15.52:0/1141258967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93e80ff490 0x7f93e8196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.740 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.727+0000 7f93eec2e700 1 -- 172.21.15.52:0/1141258967 >> 172.21.15.52:0/1141258967 conn(0x7f93e8075920 msgr2=0x7f93e80fc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:07.740 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.728+0000 7f93eec2e700 1 -- 172.21.15.52:0/1141258967 shutdown_connections 2023-11-27T15:20:07.740 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:07.728+0000 7f93eec2e700 1 -- 172.21.15.52:0/1141258967 wait complete. 2023-11-27T15:20:08.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:08 smithi150 ceph-mon[100559]: pgmap v69: 97 pgs: 97 active+clean; 2.9 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 50 KiB/s wr, 22 op/s 2023-11-27T15:20:08.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:08 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:20:08.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:08 smithi102 ceph-mon[100296]: pgmap v69: 97 pgs: 97 active+clean; 2.9 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 50 KiB/s wr, 22 op/s 2023-11-27T15:20:08.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:08 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:20:08.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:08 smithi052 ceph-mon[99056]: pgmap v69: 97 pgs: 97 active+clean; 2.9 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 50 KiB/s wr, 22 op/s 2023-11-27T15:20:08.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:08 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:20:08.746 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-27T15:20:08.746 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-27T15:20:08.885 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.879+0000 7f2c3d7c8700 1 -- 172.21.15.52:0/3323291108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c38104490 msgr2=0x7f2c3810c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:08.885 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.879+0000 7f2c3d7c8700 1 --2- 172.21.15.52:0/3323291108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c38104490 0x7f2c3810c060 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2c3000b0d0 tx=0x7f2c3000b3e0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.886 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.879+0000 7f2c3d7c8700 1 -- 172.21.15.52:0/3323291108 shutdown_connections 2023-11-27T15:20:08.886 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.879+0000 7f2c3d7c8700 1 --2- 172.21.15.52:0/3323291108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c38104490 0x7f2c3810c060 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.886 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.879+0000 7f2c3d7c8700 1 --2- 172.21.15.52:0/3323291108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c380ff450 0x7f2c380ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.886 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.879+0000 7f2c3d7c8700 1 --2- 172.21.15.52:0/3323291108 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c380ffdf0 0x7f2c38103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.886 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.879+0000 7f2c3d7c8700 1 -- 172.21.15.52:0/3323291108 >> 172.21.15.52:0/3323291108 conn(0x7f2c380770b0 msgr2=0x7f2c380774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:08.886 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.879+0000 7f2c3d7c8700 1 -- 172.21.15.52:0/3323291108 shutdown_connections 2023-11-27T15:20:08.886 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.879+0000 7f2c3d7c8700 1 -- 172.21.15.52:0/3323291108 wait complete. 2023-11-27T15:20:08.886 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.880+0000 7f2c3d7c8700 1 Processor -- start 2023-11-27T15:20:08.886 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.880+0000 7f2c3d7c8700 1 -- start start 2023-11-27T15:20:08.886 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.880+0000 7f2c3d7c8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c380ff450 0x7f2c38196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:08.886 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.880+0000 7f2c3d7c8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c380ffdf0 0x7f2c38196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:08.886 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.880+0000 7f2c3d7c8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c38104490 0x7f2c3819aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:08.887 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.880+0000 7f2c3d7c8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3819b510 con 0x7f2c38104490 2023-11-27T15:20:08.887 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.880+0000 7f2c3d7c8700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3819b680 con 0x7f2c380ffdf0 2023-11-27T15:20:08.887 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.880+0000 7f2c3d7c8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3819b840 con 0x7f2c380ff450 2023-11-27T15:20:08.887 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.880+0000 7f2c377fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c38104490 0x7f2c3819aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:08.887 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.880+0000 7f2c377fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c38104490 0x7f2c3819aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53322/0 (socket says 172.21.15.52:53322) 2023-11-27T15:20:08.887 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.880+0000 7f2c377fe700 1 -- 172.21.15.52:0/1926332186 learned_addr learned my addr 172.21.15.52:0/1926332186 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:08.887 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.880+0000 7f2c367fc700 1 --2- 172.21.15.52:0/1926332186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c380ffdf0 0x7f2c38196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:08.887 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.880+0000 7f2c36ffd700 1 --2- 172.21.15.52:0/1926332186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c380ff450 0x7f2c38196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:08.887 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.880+0000 7f2c377fe700 1 -- 172.21.15.52:0/1926332186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c380ff450 msgr2=0x7f2c38196530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:08.887 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.880+0000 7f2c377fe700 1 --2- 172.21.15.52:0/1926332186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c380ff450 0x7f2c38196530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.887 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.880+0000 7f2c377fe700 1 -- 172.21.15.52:0/1926332186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c380ffdf0 msgr2=0x7f2c38196a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:08.887 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.880+0000 7f2c377fe700 1 --2- 172.21.15.52:0/1926332186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c380ffdf0 0x7f2c38196a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.888 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.880+0000 7f2c377fe700 1 -- 172.21.15.52:0/1926332186 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c30009d00 con 0x7f2c38104490 2023-11-27T15:20:08.888 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.880+0000 7f2c367fc700 1 --2- 172.21.15.52:0/1926332186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c380ffdf0 0x7f2c38196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:20:08.888 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.881+0000 7f2c377fe700 1 --2- 172.21.15.52:0/1926332186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c38104490 0x7f2c3819aed0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2c30007870 tx=0x7f2c30007950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:08.888 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.881+0000 7f2c1ffff700 1 -- 172.21.15.52:0/1926332186 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3000e040 con 0x7f2c38104490 2023-11-27T15:20:08.888 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.881+0000 7f2c3d7c8700 1 -- 172.21.15.52:0/1926332186 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c3810f260 con 0x7f2c38104490 2023-11-27T15:20:08.888 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.881+0000 7f2c1ffff700 1 -- 172.21.15.52:0/1926332186 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c30003e10 con 0x7f2c38104490 2023-11-27T15:20:08.888 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.881+0000 7f2c1ffff700 1 -- 172.21.15.52:0/1926332186 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c30012af0 con 0x7f2c38104490 2023-11-27T15:20:08.888 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.881+0000 7f2c3d7c8700 1 -- 172.21.15.52:0/1926332186 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c3810f0d0 con 0x7f2c38104490 2023-11-27T15:20:08.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.883+0000 7f2c1ffff700 1 -- 172.21.15.52:0/1926332186 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f2c300056c0 con 0x7f2c38104490 2023-11-27T15:20:08.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.883+0000 7f2c1ffff700 1 --2- 172.21.15.52:0/1926332186 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f2c200a12c0 0x7f2c200a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:08.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.884+0000 7f2c1ffff700 1 -- 172.21.15.52:0/1926332186 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2c300ce4e0 con 0x7f2c38104490 2023-11-27T15:20:08.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.884+0000 7f2c36ffd700 1 --2- 172.21.15.52:0/1926332186 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f2c200a12c0 0x7f2c200a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:08.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.884+0000 7f2c36ffd700 1 --2- 172.21.15.52:0/1926332186 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f2c200a12c0 0x7f2c200a3770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c005950 tx=0x7f2c2c005cf0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:08.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.884+0000 7f2c49954b80 1 Processor -- start 2023-11-27T15:20:08.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.884+0000 7f2c49954b80 1 -- start start 2023-11-27T15:20:08.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.884+0000 7f2c49954b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967f647be0 0x55967f647fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:08.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.884+0000 7f2c49954b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55967f6484f0 0x55967f64c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:08.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.884+0000 7f2c49954b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55967f64c930 0x55967f64cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:08.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.884+0000 7f2c49954b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967f54d0e0 con 0x55967f647be0 2023-11-27T15:20:08.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.884+0000 7f2c49954b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967f59f920 con 0x55967f64c930 2023-11-27T15:20:08.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.885+0000 7f2c49954b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967f598db0 con 0x55967f6484f0 2023-11-27T15:20:08.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.885+0000 7f2c36ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967f647be0 0x55967f647fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:08.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.885+0000 7f2c36ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967f647be0 0x55967f647fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53326/0 (socket says 172.21.15.52:53326) 2023-11-27T15:20:08.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.885+0000 7f2c36ffd700 1 -- 172.21.15.52:0/1018817651 learned_addr learned my addr 172.21.15.52:0/1018817651 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:08.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.885+0000 7f2c377fe700 1 --2- 172.21.15.52:0/1018817651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55967f64c930 0x55967f64cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:08.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.885+0000 7f2c367fc700 1 --2- 172.21.15.52:0/1018817651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55967f6484f0 0x55967f64c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:08.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.885+0000 7f2c36ffd700 1 -- 172.21.15.52:0/1018817651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55967f6484f0 msgr2=0x55967f64c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:08.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.885+0000 7f2c36ffd700 1 --2- 172.21.15.52:0/1018817651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55967f6484f0 0x55967f64c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.885+0000 7f2c36ffd700 1 -- 172.21.15.52:0/1018817651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55967f64c930 msgr2=0x55967f64cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:08.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.885+0000 7f2c36ffd700 1 --2- 172.21.15.52:0/1018817651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55967f64c930 0x55967f64cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.885+0000 7f2c36ffd700 1 -- 172.21.15.52:0/1018817651 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55967f41a6b0 con 0x55967f647be0 2023-11-27T15:20:08.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.885+0000 7f2c367fc700 1 --2- 172.21.15.52:0/1018817651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55967f6484f0 0x55967f64c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:20:08.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.885+0000 7f2c36ffd700 1 --2- 172.21.15.52:0/1018817651 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967f647be0 0x55967f647fb0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c00dbe0 tx=0x7f2c2c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9c786eece0886 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:08.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.885+0000 7f2c1d7fa700 1 -- 172.21.15.52:0/1018817651 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c004a30 con 0x55967f647be0 2023-11-27T15:20:08.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.885+0000 7f2c1d7fa700 1 -- 172.21.15.52:0/1018817651 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c2c0054e0 con 0x55967f647be0 2023-11-27T15:20:08.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.885+0000 7f2c1d7fa700 1 -- 172.21.15.52:0/1018817651 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c00fb30 con 0x55967f647be0 2023-11-27T15:20:08.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.886+0000 7f2c49954b80 1 -- 172.21.15.52:0/1018817651 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967f647be0 msgr2=0x55967f647fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:08.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.886+0000 7f2c49954b80 1 --2- 172.21.15.52:0/1018817651 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967f647be0 0x55967f647fb0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c00dbe0 tx=0x7f2c2c010000 comp rx=0 tx=0).stop 2023-11-27T15:20:08.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.886+0000 7f2c49954b80 1 -- 172.21.15.52:0/1018817651 shutdown_connections 2023-11-27T15:20:08.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.886+0000 7f2c49954b80 1 --2- 172.21.15.52:0/1018817651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55967f64c930 0x55967f64cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.886+0000 7f2c49954b80 1 --2- 172.21.15.52:0/1018817651 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967f647be0 0x55967f647fb0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.886+0000 7f2c49954b80 1 --2- 172.21.15.52:0/1018817651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55967f6484f0 0x55967f64c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.886+0000 7f2c49954b80 1 -- 172.21.15.52:0/1018817651 >> 172.21.15.52:0/1018817651 conn(0x55967f63d3e0 msgr2=0x55967f642bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:08.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.886+0000 7f2c49954b80 1 -- 172.21.15.52:0/1018817651 shutdown_connections 2023-11-27T15:20:08.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.886+0000 7f2c49954b80 1 -- 172.21.15.52:0/1018817651 wait complete. 2023-11-27T15:20:08.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.886+0000 7f2c49954b80 1 -- start start 2023-11-27T15:20:08.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.887+0000 7f2c49954b80 1 Processor -- start 2023-11-27T15:20:08.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.887+0000 7f2c49954b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55967f647be0 0x55967f654070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:08.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.887+0000 7f2c49954b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55967f6484f0 0x55967f6512f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:08.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.887+0000 7f2c49954b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967f64c930 0x55967f651920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:08.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.887+0000 7f2c49954b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967f53a960 con 0x55967f64c930 2023-11-27T15:20:08.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.887+0000 7f2c49954b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967f53a600 con 0x55967f6484f0 2023-11-27T15:20:08.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.887+0000 7f2c49954b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967f41c9a0 con 0x55967f647be0 2023-11-27T15:20:08.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.887+0000 7f2c49954b80 10 client.0 _ll_register_callbacks cb 0x55967f558800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:20:08.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.887+0000 7f2c377fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967f64c930 0x55967f651920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:08.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.887+0000 7f2c377fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967f64c930 0x55967f651920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53336/0 (socket says 172.21.15.52:53336) 2023-11-27T15:20:08.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.887+0000 7f2c377fe700 1 -- 172.21.15.52:0/3621165526 learned_addr learned my addr 172.21.15.52:0/3621165526 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:08.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.887+0000 7f2c36ffd700 1 --2- 172.21.15.52:0/3621165526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55967f647be0 0x55967f654070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:08.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.887+0000 7f2c367fc700 1 --2- 172.21.15.52:0/3621165526 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55967f6484f0 0x55967f6512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:08.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.887+0000 7f2c377fe700 1 -- 172.21.15.52:0/3621165526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55967f647be0 msgr2=0x55967f654070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:08.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.887+0000 7f2c377fe700 1 --2- 172.21.15.52:0/3621165526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55967f647be0 0x55967f654070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.887+0000 7f2c377fe700 1 -- 172.21.15.52:0/3621165526 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55967f6484f0 msgr2=0x55967f6512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:08.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.887+0000 7f2c377fe700 1 --2- 172.21.15.52:0/3621165526 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55967f6484f0 0x55967f6512f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.887+0000 7f2c377fe700 1 -- 172.21.15.52:0/3621165526 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c2c00d890 con 0x55967f64c930 2023-11-27T15:20:08.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.888+0000 7f2c377fe700 1 --2- 172.21.15.52:0/3621165526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967f64c930 0x55967f651920 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2c3009b570 tx=0x7f2c3000b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f151357a4feb582 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:08.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.888+0000 7f2c1aff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:20:08.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.888+0000 7f2c1aff5700 1 -- 172.21.15.52:0/3621165526 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c30012e50 con 0x55967f64c930 2023-11-27T15:20:08.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.888+0000 7f2c1aff5700 1 -- 172.21.15.52:0/3621165526 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2c3009b5e0 con 0x55967f64c930 2023-11-27T15:20:08.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.888+0000 7f2c49954b80 1 -- 172.21.15.52:0/3621165526 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55967f63aba0 con 0x55967f64c930 2023-11-27T15:20:08.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.888+0000 7f2c1aff5700 1 -- 172.21.15.52:0/3621165526 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c300862c0 con 0x55967f64c930 2023-11-27T15:20:08.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.888+0000 7f2c49954b80 10 client.15960 fetch_fsmap learned FSMap version 23 2023-11-27T15:20:08.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.888+0000 7f2c49954b80 1 -- 172.21.15.52:0/3621165526 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55967f55f210 con 0x55967f64c930 2023-11-27T15:20:08.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.888+0000 7f2c1aff5700 1 -- 172.21.15.52:0/3621165526 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c300c9020 con 0x55967f64c930 2023-11-27T15:20:08.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.888+0000 7f2c1aff5700 1 -- 172.21.15.52:0/3621165526 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2c3009aad0 con 0x55967f64c930 2023-11-27T15:20:08.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.888+0000 7f2c49954b80 10 client.15960 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:20:08.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.888+0000 7f2c49954b80 10 client.15960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:20:08.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.888+0000 7f2c49954b80 20 client.15960 populate_metadata read hostname 'smithi052' 2023-11-27T15:20:08.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.888+0000 7f2c49954b80 1 --2- 172.21.15.52:0/3621165526 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55967f661780 0x55967f663b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:08.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.888+0000 7f2c49954b80 4 client.15960 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:20:08.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.888+0000 7f2c49954b80 1 -- 172.21.15.52:0/3621165526 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55967f55b120 con 0x55967f661780 2023-11-27T15:20:08.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.889+0000 7f2c36ffd700 1 --2- 172.21.15.52:0/3621165526 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55967f661780 0x55967f663b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:08.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.889+0000 7f2c1aff5700 10 client.15960 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:20:08.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.889+0000 7f2c36ffd700 1 --2- 172.21.15.52:0/3621165526 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55967f661780 0x55967f663b90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5432945a38b354c server_cookie=e838f257447c527c in_seq=0 out_seq=0 2023-11-27T15:20:08.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.890+0000 7f2c1aff5700 1 -- 172.21.15.52:0/3621165526 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55967f55b120 con 0x55967f661780 2023-11-27T15:20:08.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.890+0000 7f2c1aff5700 10 client.15960 handle_command_reply: tid=0 2023-11-27T15:20:08.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.890+0000 7f2c49954b80 1 client.15960 shutdown 2023-11-27T15:20:08.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.890+0000 7f2c49954b80 1 -- 172.21.15.52:0/3621165526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967f64c930 msgr2=0x55967f651920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:08.901 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.890+0000 7f2c49954b80 1 --2- 172.21.15.52:0/3621165526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967f64c930 0x55967f651920 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2c3009b570 tx=0x7f2c3000b3e0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.901 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.890+0000 7f2c49954b80 1 -- 172.21.15.52:0/3621165526 shutdown_connections 2023-11-27T15:20:08.901 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.890+0000 7f2c49954b80 1 --2- 172.21.15.52:0/3621165526 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55967f661780 0x55967f663b90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.901 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.890+0000 7f2c49954b80 1 --2- 172.21.15.52:0/3621165526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967f64c930 0x55967f651920 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.901 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.890+0000 7f2c1aff5700 0 client.15960 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:20:08.901 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.890+0000 7f2c49954b80 1 --2- 172.21.15.52:0/3621165526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55967f647be0 0x55967f654070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.901 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.890+0000 7f2c49954b80 1 --2- 172.21.15.52:0/3621165526 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55967f6484f0 0x55967f6512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.901 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.890+0000 7f2c49954b80 1 -- 172.21.15.52:0/3621165526 >> 172.21.15.52:0/3621165526 conn(0x55967f63d3e0 msgr2=0x55967f64dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:08.901 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.890+0000 7f2c49954b80 1 -- 172.21.15.52:0/3621165526 shutdown_connections 2023-11-27T15:20:08.901 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.890+0000 7f2c49954b80 1 -- 172.21.15.52:0/3621165526 wait complete. 2023-11-27T15:20:08.901 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.890+0000 7f2c49954b80 20 client.15960 trim_cache size 0 max 16384 2023-11-27T15:20:08.915 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.907+0000 7f2c49954b80 1 Processor -- start 2023-11-27T15:20:08.915 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.907+0000 7f2c49954b80 1 -- start start 2023-11-27T15:20:08.915 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.907+0000 7f2c49954b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967f647be0 0x55967f650ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:08.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.907+0000 7f2c49954b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55967f6484f0 0x55967f653c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:08.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.908+0000 7f2c49954b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55967f64c930 0x55967f642650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:08.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.908+0000 7f2c49954b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967f64f850 con 0x55967f647be0 2023-11-27T15:20:08.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.908+0000 7f2c49954b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967f64b4a0 con 0x55967f64c930 2023-11-27T15:20:08.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.908+0000 7f2c49954b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967f63a720 con 0x55967f6484f0 2023-11-27T15:20:08.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.908+0000 7f2c36ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967f647be0 0x55967f650ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:08.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.908+0000 7f2c36ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967f647be0 0x55967f650ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53340/0 (socket says 172.21.15.52:53340) 2023-11-27T15:20:08.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.908+0000 7f2c36ffd700 1 -- 172.21.15.52:0/1418836310 learned_addr learned my addr 172.21.15.52:0/1418836310 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:08.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.908+0000 7f2c377fe700 1 --2- 172.21.15.52:0/1418836310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55967f64c930 0x55967f642650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:08.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.908+0000 7f2c36ffd700 1 -- 172.21.15.52:0/1418836310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55967f6484f0 msgr2=0x55967f653c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:08.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.908+0000 7f2c367fc700 1 --2- 172.21.15.52:0/1418836310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55967f6484f0 0x55967f653c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:08.916 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.908+0000 7f2c36ffd700 1 --2- 172.21.15.52:0/1418836310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55967f6484f0 0x55967f653c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.917 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.908+0000 7f2c36ffd700 1 -- 172.21.15.52:0/1418836310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55967f64c930 msgr2=0x55967f642650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:08.917 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.908+0000 7f2c36ffd700 1 --2- 172.21.15.52:0/1418836310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55967f64c930 0x55967f642650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.917 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.908+0000 7f2c36ffd700 1 -- 172.21.15.52:0/1418836310 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55967f6418a0 con 0x55967f647be0 2023-11-27T15:20:08.917 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.908+0000 7f2c377fe700 1 --2- 172.21.15.52:0/1418836310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55967f64c930 0x55967f642650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:20:08.917 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.908+0000 7f2c367fc700 1 --2- 172.21.15.52:0/1418836310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55967f6484f0 0x55967f653c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:20:08.917 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.908+0000 7f2c36ffd700 1 --2- 172.21.15.52:0/1418836310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967f647be0 0x55967f650ab0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c00def0 tx=0x7f2c2c00b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2506e338ba44411c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:08.917 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.908+0000 7f2c1dffb700 1 -- 172.21.15.52:0/1418836310 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c033420 con 0x55967f647be0 2023-11-27T15:20:08.917 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.908+0000 7f2c1dffb700 1 -- 172.21.15.52:0/1418836310 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c2c033580 con 0x55967f647be0 2023-11-27T15:20:08.917 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.908+0000 7f2c1dffb700 1 -- 172.21.15.52:0/1418836310 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c00fe90 con 0x55967f647be0 2023-11-27T15:20:08.917 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.909+0000 7f2c49954b80 1 -- 172.21.15.52:0/1418836310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967f647be0 msgr2=0x55967f650ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:08.917 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.909+0000 7f2c49954b80 1 --2- 172.21.15.52:0/1418836310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967f647be0 0x55967f650ab0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c00def0 tx=0x7f2c2c00b7d0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.917 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.909+0000 7f2c49954b80 1 -- 172.21.15.52:0/1418836310 shutdown_connections 2023-11-27T15:20:08.917 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.909+0000 7f2c49954b80 1 --2- 172.21.15.52:0/1418836310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55967f64c930 0x55967f642650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.918 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.909+0000 7f2c49954b80 1 --2- 172.21.15.52:0/1418836310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967f647be0 0x55967f650ab0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.918 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.909+0000 7f2c49954b80 1 --2- 172.21.15.52:0/1418836310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55967f6484f0 0x55967f653c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.918 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.909+0000 7f2c49954b80 1 -- 172.21.15.52:0/1418836310 >> 172.21.15.52:0/1418836310 conn(0x55967f63d3e0 msgr2=0x55967f6512b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:08.918 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.909+0000 7f2c49954b80 1 -- 172.21.15.52:0/1418836310 shutdown_connections 2023-11-27T15:20:08.918 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.909+0000 7f2c49954b80 1 -- 172.21.15.52:0/1418836310 wait complete. 2023-11-27T15:20:08.918 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.909+0000 7f2c49954b80 1 -- start start 2023-11-27T15:20:08.918 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.909+0000 7f2c49954b80 1 Processor -- start 2023-11-27T15:20:08.918 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.910+0000 7f2c49954b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55967f647be0 0x55967f657970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:08.918 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.910+0000 7f2c49954b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967f6484f0 0x55967f657eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:08.918 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.910+0000 7f2c49954b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55967f64c930 0x55967f6b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:08.918 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.910+0000 7f2c49954b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967f6b3fe0 con 0x55967f6484f0 2023-11-27T15:20:08.918 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.910+0000 7f2c49954b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967f6b4150 con 0x55967f647be0 2023-11-27T15:20:08.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.910+0000 7f2c49954b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967f6b4310 con 0x55967f64c930 2023-11-27T15:20:08.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.910+0000 7f2c49954b80 10 client.0 _ll_register_callbacks cb 0x7f2c3009b3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:20:08.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.910+0000 7f2c367fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967f6484f0 0x55967f657eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:08.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.910+0000 7f2c367fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967f6484f0 0x55967f657eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53348/0 (socket says 172.21.15.52:53348) 2023-11-27T15:20:08.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.910+0000 7f2c367fc700 1 -- 172.21.15.52:0/1820858911 learned_addr learned my addr 172.21.15.52:0/1820858911 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:08.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.910+0000 7f2c36ffd700 1 --2- 172.21.15.52:0/1820858911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55967f647be0 0x55967f657970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:08.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.910+0000 7f2c377fe700 1 --2- 172.21.15.52:0/1820858911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55967f64c930 0x55967f6b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:08.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.910+0000 7f2c367fc700 1 -- 172.21.15.52:0/1820858911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55967f64c930 msgr2=0x55967f6b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:08.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.910+0000 7f2c367fc700 1 --2- 172.21.15.52:0/1820858911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55967f64c930 0x55967f6b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.910+0000 7f2c367fc700 1 -- 172.21.15.52:0/1820858911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55967f647be0 msgr2=0x55967f657970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:08.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.910+0000 7f2c367fc700 1 --2- 172.21.15.52:0/1820858911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55967f647be0 0x55967f657970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.910+0000 7f2c367fc700 1 -- 172.21.15.52:0/1820858911 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55967f41a6b0 con 0x55967f6484f0 2023-11-27T15:20:08.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.910+0000 7f2c367fc700 1 --2- 172.21.15.52:0/1820858911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967f6484f0 0x55967f657eb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2c3806a4a0 tx=0x7f2c2800dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b800c3cd9470e15 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:08.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.910+0000 7f2c1aff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:20:08.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.910+0000 7f2c1aff5700 1 -- 172.21.15.52:0/1820858911 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c28018070 con 0x55967f6484f0 2023-11-27T15:20:08.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.910+0000 7f2c49954b80 1 -- 172.21.15.52:0/1820858911 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c2c0054e0 con 0x55967f6484f0 2023-11-27T15:20:08.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.910+0000 7f2c1aff5700 1 -- 172.21.15.52:0/1820858911 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2c28003c50 con 0x55967f6484f0 2023-11-27T15:20:08.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.911+0000 7f2c1aff5700 1 -- 172.21.15.52:0/1820858911 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c2800de90 con 0x55967f6484f0 2023-11-27T15:20:08.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.911+0000 7f2c49954b80 10 client.15966 fetch_fsmap learned FSMap version 23 2023-11-27T15:20:08.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.911+0000 7f2c49954b80 1 -- 172.21.15.52:0/1820858911 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55967f3c6e60 con 0x55967f6484f0 2023-11-27T15:20:08.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.911+0000 7f2c1aff5700 1 -- 172.21.15.52:0/1820858911 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c2c0054e0 con 0x55967f6484f0 2023-11-27T15:20:08.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.911+0000 7f2c1aff5700 1 -- 172.21.15.52:0/1820858911 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2c2801d540 con 0x55967f6484f0 2023-11-27T15:20:08.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.911+0000 7f2c49954b80 10 client.15966 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:20:08.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.911+0000 7f2c49954b80 10 client.15966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:20:08.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.911+0000 7f2c49954b80 20 client.15966 populate_metadata read hostname 'smithi052' 2023-11-27T15:20:08.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.911+0000 7f2c49954b80 1 --2- 172.21.15.52:0/1820858911 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55967f6bbb50 0x55967f6bdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:08.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.911+0000 7f2c49954b80 4 client.15966 mds_command: new command op to 24536 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-27T15:20:08.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.911+0000 7f2c49954b80 1 -- 172.21.15.52:0/1820858911 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55967f5676e0 con 0x55967f6bbb50 2023-11-27T15:20:08.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.911+0000 7f2c36ffd700 1 --2- 172.21.15.52:0/1820858911 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55967f6bbb50 0x55967f6bdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:08.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.912+0000 7f2c1aff5700 10 client.15966 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:20:08.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.912+0000 7f2c36ffd700 1 --2- 172.21.15.52:0/1820858911 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55967f6bbb50 0x55967f6bdf60 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91a9081d9a05d1fb server_cookie=3ab3072e946d9e73 in_seq=0 out_seq=0 2023-11-27T15:20:08.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.913+0000 7f2c1aff5700 1 -- 172.21.15.52:0/1820858911 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55967f5676e0 con 0x55967f6bbb50 2023-11-27T15:20:08.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.913+0000 7f2c1aff5700 10 client.15966 handle_command_reply: tid=0 2023-11-27T15:20:08.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.913+0000 7f2c49954b80 1 client.15966 shutdown 2023-11-27T15:20:08.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.913+0000 7f2c49954b80 1 -- 172.21.15.52:0/1820858911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967f6484f0 msgr2=0x55967f657eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:08.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.913+0000 7f2c49954b80 1 --2- 172.21.15.52:0/1820858911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967f6484f0 0x55967f657eb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2c3806a4a0 tx=0x7f2c2800dd70 comp rx=0 tx=0).stop 2023-11-27T15:20:08.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.913+0000 7f2c49954b80 1 -- 172.21.15.52:0/1820858911 shutdown_connections 2023-11-27T15:20:08.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.913+0000 7f2c49954b80 1 --2- 172.21.15.52:0/1820858911 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55967f6bbb50 0x55967f6bdf60 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.913+0000 7f2c49954b80 1 --2- 172.21.15.52:0/1820858911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55967f647be0 0x55967f657970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.913+0000 7f2c49954b80 1 --2- 172.21.15.52:0/1820858911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55967f64c930 0x55967f6b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.913+0000 7f2c1aff5700 0 client.15966 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:20:08.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.913+0000 7f2c49954b80 1 --2- 172.21.15.52:0/1820858911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55967f6484f0 0x55967f657eb0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.913+0000 7f2c49954b80 1 -- 172.21.15.52:0/1820858911 >> 172.21.15.52:0/1820858911 conn(0x55967f63d3e0 msgr2=0x55967f64b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:08.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.913+0000 7f2c49954b80 1 -- 172.21.15.52:0/1820858911 shutdown_connections 2023-11-27T15:20:08.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.913+0000 7f2c49954b80 1 -- 172.21.15.52:0/1820858911 wait complete. 2023-11-27T15:20:08.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.913+0000 7f2c49954b80 20 client.15966 trim_cache size 0 max 16384 2023-11-27T15:20:08.923 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:20:08.924 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2023-11-27T15:20:08.924 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "27ec9b1e-7bb4-48fc-99ff-3f40b7797927", 2023-11-27T15:20:08.924 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2023-11-27T15:20:08.924 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:20:08.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.914+0000 7f2c3d7c8700 1 -- 172.21.15.52:0/1926332186 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f2c200a12c0 msgr2=0x7f2c200a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:08.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.914+0000 7f2c3d7c8700 1 --2- 172.21.15.52:0/1926332186 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f2c200a12c0 0x7f2c200a3770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c005950 tx=0x7f2c2c005cf0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.914+0000 7f2c3d7c8700 1 -- 172.21.15.52:0/1926332186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c38104490 msgr2=0x7f2c3819aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:08.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.914+0000 7f2c3d7c8700 1 --2- 172.21.15.52:0/1926332186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c38104490 0x7f2c3819aed0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2c30007870 tx=0x7f2c30007950 comp rx=0 tx=0).stop 2023-11-27T15:20:08.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.914+0000 7f2c3d7c8700 1 -- 172.21.15.52:0/1926332186 shutdown_connections 2023-11-27T15:20:08.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.914+0000 7f2c3d7c8700 1 --2- 172.21.15.52:0/1926332186 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f2c200a12c0 0x7f2c200a3770 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.914+0000 7f2c3d7c8700 1 --2- 172.21.15.52:0/1926332186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c38104490 0x7f2c3819aed0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.914+0000 7f2c3d7c8700 1 --2- 172.21.15.52:0/1926332186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c380ff450 0x7f2c38196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.914+0000 7f2c3d7c8700 1 --2- 172.21.15.52:0/1926332186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c380ffdf0 0x7f2c38196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.914+0000 7f2c3d7c8700 1 -- 172.21.15.52:0/1926332186 >> 172.21.15.52:0/1926332186 conn(0x7f2c380770b0 msgr2=0x7f2c38102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:08.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.914+0000 7f2c3d7c8700 1 -- 172.21.15.52:0/1926332186 shutdown_connections 2023-11-27T15:20:08.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:08.914+0000 7f2c3d7c8700 1 -- 172.21.15.52:0/1926332186 wait complete. 2023-11-27T15:20:08.932 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:20:09.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.064+0000 7f37c9556700 1 -- 172.21.15.52:0/766883584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f37c4104560 msgr2=0x7f37c410c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:09.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.064+0000 7f37c9556700 1 --2- 172.21.15.52:0/766883584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f37c4104560 0x7f37c410c130 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009f90 tx=0x7f37bc00b330 comp rx=0 tx=0).stop 2023-11-27T15:20:09.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.064+0000 7f37c9556700 1 -- 172.21.15.52:0/766883584 shutdown_connections 2023-11-27T15:20:09.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.064+0000 7f37c9556700 1 --2- 172.21.15.52:0/766883584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f37c4104560 0x7f37c410c130 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.064+0000 7f37c9556700 1 --2- 172.21.15.52:0/766883584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37c40ff520 0x7f37c40ff8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.064+0000 7f37c9556700 1 --2- 172.21.15.52:0/766883584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37c40ffec0 0x7f37c4103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.064+0000 7f37c9556700 1 -- 172.21.15.52:0/766883584 >> 172.21.15.52:0/766883584 conn(0x7f37c40760d0 msgr2=0x7f37c40764d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:09.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.064+0000 7f37c9556700 1 -- 172.21.15.52:0/766883584 shutdown_connections 2023-11-27T15:20:09.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.064+0000 7f37c9556700 1 -- 172.21.15.52:0/766883584 wait complete. 2023-11-27T15:20:09.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.065+0000 7f37c9556700 1 Processor -- start 2023-11-27T15:20:09.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.066+0000 7f37c9556700 1 -- start start 2023-11-27T15:20:09.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.066+0000 7f37c9556700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f37c40ff520 0x7f37c4195450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:09.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.066+0000 7f37c9556700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37c40ffec0 0x7f37c4195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:09.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.066+0000 7f37c9556700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37c4104560 0x7f37c4199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:09.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.066+0000 7f37c9556700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c419a430 con 0x7f37c4104560 2023-11-27T15:20:09.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.066+0000 7f37c9556700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c419a5a0 con 0x7f37c40ff520 2023-11-27T15:20:09.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.066+0000 7f37c9556700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c419a760 con 0x7f37c40ffec0 2023-11-27T15:20:09.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.066+0000 7f37c2ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f37c40ff520 0x7f37c4195450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:09.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.066+0000 7f37c2ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f37c40ff520 0x7f37c4195450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43018/0 (socket says 172.21.15.52:43018) 2023-11-27T15:20:09.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.066+0000 7f37c2ffd700 1 -- 172.21.15.52:0/2283185696 learned_addr learned my addr 172.21.15.52:0/2283185696 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:09.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.066+0000 7f37c37fe700 1 --2- 172.21.15.52:0/2283185696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37c4104560 0x7f37c4199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:09.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.066+0000 7f37c27fc700 1 --2- 172.21.15.52:0/2283185696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37c40ffec0 0x7f37c4195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:09.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.066+0000 7f37c2ffd700 1 -- 172.21.15.52:0/2283185696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37c40ffec0 msgr2=0x7f37c4195990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:09.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.066+0000 7f37c2ffd700 1 --2- 172.21.15.52:0/2283185696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37c40ffec0 0x7f37c4195990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.066+0000 7f37c2ffd700 1 -- 172.21.15.52:0/2283185696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37c4104560 msgr2=0x7f37c4199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:09.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.066+0000 7f37c2ffd700 1 --2- 172.21.15.52:0/2283185696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37c4104560 0x7f37c4199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.066+0000 7f37c2ffd700 1 -- 172.21.15.52:0/2283185696 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b40095f0 con 0x7f37c40ff520 2023-11-27T15:20:09.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.066+0000 7f37c37fe700 1 --2- 172.21.15.52:0/2283185696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37c4104560 0x7f37c4199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:09.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.067+0000 7f37c2ffd700 1 --2- 172.21.15.52:0/2283185696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f37c40ff520 0x7f37c4195450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f37b400ec10 tx=0x7f37b400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:09.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.067+0000 7f37abfff700 1 -- 172.21.15.52:0/2283185696 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b400cd40 con 0x7f37c40ff520 2023-11-27T15:20:09.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.067+0000 7f37abfff700 1 -- 172.21.15.52:0/2283185696 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37b40044e0 con 0x7f37c40ff520 2023-11-27T15:20:09.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.067+0000 7f37c9556700 1 -- 172.21.15.52:0/2283185696 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37bc009c40 con 0x7f37c40ff520 2023-11-27T15:20:09.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.067+0000 7f37abfff700 1 -- 172.21.15.52:0/2283185696 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b4010690 con 0x7f37c40ff520 2023-11-27T15:20:09.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.067+0000 7f37c9556700 1 -- 172.21.15.52:0/2283185696 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c410f390 con 0x7f37c40ff520 2023-11-27T15:20:09.074 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.069+0000 7f37abfff700 1 -- 172.21.15.52:0/2283185696 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f37b4003710 con 0x7f37c40ff520 2023-11-27T15:20:09.074 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.070+0000 7f37abfff700 1 --2- 172.21.15.52:0/2283185696 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f37ac0a12c0 0x7f37ac0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:09.074 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.070+0000 7f37abfff700 1 -- 172.21.15.52:0/2283185696 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f37b4014070 con 0x7f37c40ff520 2023-11-27T15:20:09.074 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.070+0000 7f37c27fc700 1 --2- 172.21.15.52:0/2283185696 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f37ac0a12c0 0x7f37ac0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:09.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.071+0000 7f37d56e2b80 1 Processor -- start 2023-11-27T15:20:09.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.071+0000 7f37c27fc700 1 --2- 172.21.15.52:0/2283185696 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f37ac0a12c0 0x7f37ac0a3770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f37b8005950 tx=0x7f37b80058e0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:09.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.071+0000 7f37d56e2b80 1 -- start start 2023-11-27T15:20:09.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.071+0000 7f37d56e2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563a296ed970 0x563a296edd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:09.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.071+0000 7f37d56e2b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a296ee280 0x563a296fbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:09.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.071+0000 7f37d56e2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a296fc530 0x563a296fe910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:09.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.071+0000 7f37d56e2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a295f3060 con 0x563a296fc530 2023-11-27T15:20:09.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.071+0000 7f37d56e2b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a296458a0 con 0x563a296ee280 2023-11-27T15:20:09.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.071+0000 7f37d56e2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a2963ed30 con 0x563a296ed970 2023-11-27T15:20:09.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.071+0000 7f37c27fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a296ee280 0x563a296fbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:09.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.071+0000 7f37c27fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a296ee280 0x563a296fbff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43028/0 (socket says 172.21.15.52:43028) 2023-11-27T15:20:09.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.071+0000 7f37c2ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563a296ed970 0x563a296edd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:09.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.071+0000 7f37c27fc700 1 -- 172.21.15.52:0/618899421 learned_addr learned my addr 172.21.15.52:0/618899421 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:09.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.072+0000 7f37c27fc700 1 -- 172.21.15.52:0/618899421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563a296ed970 msgr2=0x563a296edd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:09.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.072+0000 7f37c27fc700 1 --2- 172.21.15.52:0/618899421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563a296ed970 0x563a296edd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.072+0000 7f37c27fc700 1 -- 172.21.15.52:0/618899421 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a296fc530 msgr2=0x563a296fe910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:09.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.072+0000 7f37c27fc700 1 --2- 172.21.15.52:0/618899421 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a296fc530 0x563a296fe910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.072+0000 7f37c27fc700 1 -- 172.21.15.52:0/618899421 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a294c0630 con 0x563a296ee280 2023-11-27T15:20:09.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.072+0000 7f37c27fc700 1 --2- 172.21.15.52:0/618899421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a296ee280 0x563a296fbff0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f37b800fc10 tx=0x7f37b8012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=291ad3cd1c23cdb8 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:09.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.072+0000 7f37a97fa700 1 -- 172.21.15.52:0/618899421 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b8011ad0 con 0x563a296ee280 2023-11-27T15:20:09.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.072+0000 7f37a97fa700 1 -- 172.21.15.52:0/618899421 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37b8011c30 con 0x563a296ee280 2023-11-27T15:20:09.078 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.072+0000 7f37a97fa700 1 -- 172.21.15.52:0/618899421 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b801d610 con 0x563a296ee280 2023-11-27T15:20:09.081 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.073+0000 7f37d56e2b80 1 -- 172.21.15.52:0/618899421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a296ee280 msgr2=0x563a296fbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:09.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.073+0000 7f37d56e2b80 1 --2- 172.21.15.52:0/618899421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a296ee280 0x563a296fbff0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f37b800fc10 tx=0x7f37b8012000 comp rx=0 tx=0).stop 2023-11-27T15:20:09.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.073+0000 7f37d56e2b80 1 -- 172.21.15.52:0/618899421 shutdown_connections 2023-11-27T15:20:09.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.073+0000 7f37d56e2b80 1 --2- 172.21.15.52:0/618899421 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a296fc530 0x563a296fe910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.073+0000 7f37d56e2b80 1 --2- 172.21.15.52:0/618899421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563a296ed970 0x563a296edd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.073+0000 7f37d56e2b80 1 --2- 172.21.15.52:0/618899421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a296ee280 0x563a296fbff0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.073+0000 7f37d56e2b80 1 -- 172.21.15.52:0/618899421 >> 172.21.15.52:0/618899421 conn(0x563a296e3360 msgr2=0x563a296e8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:09.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.073+0000 7f37d56e2b80 1 -- 172.21.15.52:0/618899421 shutdown_connections 2023-11-27T15:20:09.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.073+0000 7f37d56e2b80 1 -- 172.21.15.52:0/618899421 wait complete. 2023-11-27T15:20:09.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.073+0000 7f37d56e2b80 1 -- start start 2023-11-27T15:20:09.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.073+0000 7f37d56e2b80 1 Processor -- start 2023-11-27T15:20:09.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.074+0000 7f37d56e2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563a296ed970 0x563a296fcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:09.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.074+0000 7f37d56e2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a296ee280 0x563a296fd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:09.083 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.074+0000 7f37d56e2b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a296fc530 0x563a29707980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:09.083 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.074+0000 7f37d56e2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a295e08e0 con 0x563a296ee280 2023-11-27T15:20:09.083 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.074+0000 7f37d56e2b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a295e0580 con 0x563a296fc530 2023-11-27T15:20:09.083 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.074+0000 7f37d56e2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a294c2920 con 0x563a296ed970 2023-11-27T15:20:09.083 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.074+0000 7f37d56e2b80 10 client.0 _ll_register_callbacks cb 0x563a295fe780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:20:09.083 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.074+0000 7f37c27fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a296ee280 0x563a296fd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:09.083 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.074+0000 7f37c27fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a296ee280 0x563a296fd030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53376/0 (socket says 172.21.15.52:53376) 2023-11-27T15:20:09.083 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.074+0000 7f37c27fc700 1 -- 172.21.15.52:0/4031669868 learned_addr learned my addr 172.21.15.52:0/4031669868 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:09.083 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.074+0000 7f37c37fe700 1 --2- 172.21.15.52:0/4031669868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a296fc530 0x563a29707980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:09.083 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.074+0000 7f37c2ffd700 1 --2- 172.21.15.52:0/4031669868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563a296ed970 0x563a296fcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:09.083 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.074+0000 7f37c27fc700 1 -- 172.21.15.52:0/4031669868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563a296ed970 msgr2=0x563a296fcaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:09.083 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.074+0000 7f37c27fc700 1 --2- 172.21.15.52:0/4031669868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563a296ed970 0x563a296fcaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.074+0000 7f37c27fc700 1 -- 172.21.15.52:0/4031669868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a296fc530 msgr2=0x563a29707980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:09.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.074+0000 7f37c27fc700 1 --2- 172.21.15.52:0/4031669868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a296fc530 0x563a29707980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.074+0000 7f37c27fc700 1 -- 172.21.15.52:0/4031669868 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b800f8f0 con 0x563a296ee280 2023-11-27T15:20:09.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.074+0000 7f37c27fc700 1 --2- 172.21.15.52:0/4031669868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a296ee280 0x563a296fd030 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f37b8011b00 tx=0x7f37b8011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ae55f818637f20 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:09.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.074+0000 7f37a6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:20:09.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.074+0000 7f37a6ff5700 1 -- 172.21.15.52:0/4031669868 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b802d070 con 0x563a296ee280 2023-11-27T15:20:09.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.074+0000 7f37a6ff5700 1 -- 172.21.15.52:0/4031669868 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f37b801e660 con 0x563a296ee280 2023-11-27T15:20:09.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.074+0000 7f37d56e2b80 1 -- 172.21.15.52:0/4031669868 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a296e0b20 con 0x563a296ee280 2023-11-27T15:20:09.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.075+0000 7f37a6ff5700 1 -- 172.21.15.52:0/4031669868 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37b801e820 con 0x563a296ee280 2023-11-27T15:20:09.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.075+0000 7f37d56e2b80 10 client.15975 fetch_fsmap learned FSMap version 23 2023-11-27T15:20:09.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.075+0000 7f37d56e2b80 1 -- 172.21.15.52:0/4031669868 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x563a29605190 con 0x563a296ee280 2023-11-27T15:20:09.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.075+0000 7f37a6ff5700 1 -- 172.21.15.52:0/4031669868 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37b803f020 con 0x563a296ee280 2023-11-27T15:20:09.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.075+0000 7f37a6ff5700 1 -- 172.21.15.52:0/4031669868 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f37b80103d0 con 0x563a296ee280 2023-11-27T15:20:09.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.075+0000 7f37d56e2b80 10 client.15975 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:20:09.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.075+0000 7f37d56e2b80 10 client.15975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:20:09.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.075+0000 7f37d56e2b80 20 client.15975 populate_metadata read hostname 'smithi052' 2023-11-27T15:20:09.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.075+0000 7f37d56e2b80 1 --2- 172.21.15.52:0/4031669868 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x563a2970bd30 0x563a2970e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:09.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.075+0000 7f37d56e2b80 4 client.15975 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:20:09.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.075+0000 7f37d56e2b80 1 -- 172.21.15.52:0/4031669868 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a296010a0 con 0x563a2970bd30 2023-11-27T15:20:09.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.075+0000 7f37c2ffd700 1 --2- 172.21.15.52:0/4031669868 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x563a2970bd30 0x563a2970e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:09.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.076+0000 7f37a6ff5700 10 client.15975 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:20:09.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.076+0000 7f37c2ffd700 1 --2- 172.21.15.52:0/4031669868 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x563a2970bd30 0x563a2970e140 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb71d28f0da96b82 server_cookie=69e401dcc8de1419 in_seq=0 out_seq=0 2023-11-27T15:20:09.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.077+0000 7f37a6ff5700 1 -- 172.21.15.52:0/4031669868 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563a296010a0 con 0x563a2970bd30 2023-11-27T15:20:09.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.077+0000 7f37a6ff5700 10 client.15975 handle_command_reply: tid=0 2023-11-27T15:20:09.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.077+0000 7f37d56e2b80 1 client.15975 shutdown 2023-11-27T15:20:09.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.077+0000 7f37d56e2b80 1 -- 172.21.15.52:0/4031669868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a296ee280 msgr2=0x563a296fd030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:09.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.077+0000 7f37d56e2b80 1 --2- 172.21.15.52:0/4031669868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a296ee280 0x563a296fd030 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f37b8011b00 tx=0x7f37b8011b30 comp rx=0 tx=0).stop 2023-11-27T15:20:09.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.077+0000 7f37d56e2b80 1 -- 172.21.15.52:0/4031669868 shutdown_connections 2023-11-27T15:20:09.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.077+0000 7f37d56e2b80 1 --2- 172.21.15.52:0/4031669868 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x563a2970bd30 0x563a2970e140 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.077+0000 7f37d56e2b80 1 --2- 172.21.15.52:0/4031669868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a296fc530 0x563a29707980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.077+0000 7f37d56e2b80 1 --2- 172.21.15.52:0/4031669868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563a296ed970 0x563a296fcaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.077+0000 7f37a6ff5700 0 client.15975 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:20:09.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.077+0000 7f37d56e2b80 1 --2- 172.21.15.52:0/4031669868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a296ee280 0x563a296fd030 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.077+0000 7f37d56e2b80 1 -- 172.21.15.52:0/4031669868 >> 172.21.15.52:0/4031669868 conn(0x563a296e3360 msgr2=0x563a296e7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:09.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.077+0000 7f37d56e2b80 1 -- 172.21.15.52:0/4031669868 shutdown_connections 2023-11-27T15:20:09.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.077+0000 7f37d56e2b80 1 -- 172.21.15.52:0/4031669868 wait complete. 2023-11-27T15:20:09.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.077+0000 7f37d56e2b80 20 client.15975 trim_cache size 0 max 16384 2023-11-27T15:20:09.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.096+0000 7f37d56e2b80 1 Processor -- start 2023-11-27T15:20:09.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.096+0000 7f37d56e2b80 1 -- start start 2023-11-27T15:20:09.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.097+0000 7f37d56e2b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a296ed970 0x563a297016e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:09.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.097+0000 7f37d56e2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563a296ee280 0x563a296fc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:09.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.097+0000 7f37d56e2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a296fc530 0x563a296fce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:09.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.097+0000 7f37d56e2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a296ef3e0 con 0x563a296fc530 2023-11-27T15:20:09.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.097+0000 7f37d56e2b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a296e0660 con 0x563a296ed970 2023-11-27T15:20:09.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.097+0000 7f37d56e2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a296e9950 con 0x563a296ee280 2023-11-27T15:20:09.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.097+0000 7f37c37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a296fc530 0x563a296fce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:09.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.097+0000 7f37c2ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a296ed970 0x563a297016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:09.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.097+0000 7f37c2ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a296ed970 0x563a297016e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43040/0 (socket says 172.21.15.52:43040) 2023-11-27T15:20:09.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.097+0000 7f37c2ffd700 1 -- 172.21.15.52:0/702938226 learned_addr learned my addr 172.21.15.52:0/702938226 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:09.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.097+0000 7f37c27fc700 1 --2- 172.21.15.52:0/702938226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563a296ee280 0x563a296fc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:09.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.097+0000 7f37c37fe700 1 -- 172.21.15.52:0/702938226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563a296ee280 msgr2=0x563a296fc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:09.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.097+0000 7f37c37fe700 1 --2- 172.21.15.52:0/702938226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563a296ee280 0x563a296fc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.097+0000 7f37c37fe700 1 -- 172.21.15.52:0/702938226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a296ed970 msgr2=0x563a297016e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:09.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.097+0000 7f37c37fe700 1 --2- 172.21.15.52:0/702938226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a296ed970 0x563a297016e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.097+0000 7f37c37fe700 1 -- 172.21.15.52:0/702938226 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a296fb310 con 0x563a296fc530 2023-11-27T15:20:09.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.097+0000 7f37c2ffd700 1 --2- 172.21.15.52:0/702938226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a296ed970 0x563a297016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:09.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.097+0000 7f37c37fe700 1 --2- 172.21.15.52:0/702938226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a296fc530 0x563a296fce40 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x563a296ea390 tx=0x7f37bc009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd21b6ab6c516d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:09.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.098+0000 7f37a9ffb700 1 -- 172.21.15.52:0/702938226 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc00e040 con 0x563a296fc530 2023-11-27T15:20:09.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.098+0000 7f37a9ffb700 1 -- 172.21.15.52:0/702938226 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37bc021070 con 0x563a296fc530 2023-11-27T15:20:09.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.098+0000 7f37a9ffb700 1 -- 172.21.15.52:0/702938226 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc0052a0 con 0x563a296fc530 2023-11-27T15:20:09.106 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.098+0000 7f37d56e2b80 1 -- 172.21.15.52:0/702938226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a296fc530 msgr2=0x563a296fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:09.106 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.098+0000 7f37d56e2b80 1 --2- 172.21.15.52:0/702938226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a296fc530 0x563a296fce40 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x563a296ea390 tx=0x7f37bc009510 comp rx=0 tx=0).stop 2023-11-27T15:20:09.106 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.099+0000 7f37d56e2b80 1 -- 172.21.15.52:0/702938226 shutdown_connections 2023-11-27T15:20:09.106 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.099+0000 7f37d56e2b80 1 --2- 172.21.15.52:0/702938226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a296ed970 0x563a297016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.106 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.099+0000 7f37d56e2b80 1 --2- 172.21.15.52:0/702938226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a296fc530 0x563a296fce40 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.106 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.099+0000 7f37d56e2b80 1 --2- 172.21.15.52:0/702938226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563a296ee280 0x563a296fc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.106 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.099+0000 7f37d56e2b80 1 -- 172.21.15.52:0/702938226 >> 172.21.15.52:0/702938226 conn(0x563a296e3360 msgr2=0x563a296f5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:09.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.099+0000 7f37d56e2b80 1 -- 172.21.15.52:0/702938226 shutdown_connections 2023-11-27T15:20:09.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.099+0000 7f37d56e2b80 1 -- 172.21.15.52:0/702938226 wait complete. 2023-11-27T15:20:09.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.099+0000 7f37d56e2b80 1 -- start start 2023-11-27T15:20:09.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.099+0000 7f37d56e2b80 1 Processor -- start 2023-11-27T15:20:09.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.099+0000 7f37d56e2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563a296ed970 0x563a297010d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:09.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.099+0000 7f37d56e2b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a296ee280 0x563a29701610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:09.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.099+0000 7f37d56e2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a296fc530 0x563a296ff600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:09.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.099+0000 7f37d56e2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a296ffbd0 con 0x563a296fc530 2023-11-27T15:20:09.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.099+0000 7f37d56e2b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a296ffd40 con 0x563a296ee280 2023-11-27T15:20:09.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.099+0000 7f37d56e2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a296fff00 con 0x563a296ed970 2023-11-27T15:20:09.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.099+0000 7f37d56e2b80 10 client.0 _ll_register_callbacks cb 0x7f37b4009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:20:09.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.099+0000 7f37c37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a296fc530 0x563a296ff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:09.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.100+0000 7f37c27fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a296ee280 0x563a29701610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:09.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.100+0000 7f37c37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a296fc530 0x563a296ff600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53392/0 (socket says 172.21.15.52:53392) 2023-11-27T15:20:09.108 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.100+0000 7f37c2ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563a296ed970 0x563a297010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:09.108 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.100+0000 7f37c37fe700 1 -- 172.21.15.52:0/394883295 learned_addr learned my addr 172.21.15.52:0/394883295 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:09.108 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.100+0000 7f37c27fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a296ee280 0x563a29701610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43048/0 (socket says 172.21.15.52:43048) 2023-11-27T15:20:09.108 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.100+0000 7f37c37fe700 1 -- 172.21.15.52:0/394883295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563a296ed970 msgr2=0x563a297010d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:09.108 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.100+0000 7f37c37fe700 1 --2- 172.21.15.52:0/394883295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563a296ed970 0x563a297010d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.108 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.100+0000 7f37c37fe700 1 -- 172.21.15.52:0/394883295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a296ee280 msgr2=0x563a29701610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:09.108 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.100+0000 7f37c37fe700 1 --2- 172.21.15.52:0/394883295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a296ee280 0x563a29701610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.108 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.100+0000 7f37c37fe700 1 -- 172.21.15.52:0/394883295 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c410f390 con 0x563a296fc530 2023-11-27T15:20:09.108 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.100+0000 7f37c27fc700 1 --2- 172.21.15.52:0/394883295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a296ee280 0x563a29701610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:09.108 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.100+0000 7f37c37fe700 1 --2- 172.21.15.52:0/394883295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a296fc530 0x563a296ff600 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00bd20 tx=0x7f37bc006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0956c74c1bddaa server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:09.108 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.100+0000 7f37a6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:20:09.108 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.101+0000 7f37a6ff5700 1 -- 172.21.15.52:0/394883295 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc00e040 con 0x563a296fc530 2023-11-27T15:20:09.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.101+0000 7f37a6ff5700 1 -- 172.21.15.52:0/394883295 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f37bc003e10 con 0x563a296fc530 2023-11-27T15:20:09.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.101+0000 7f37d56e2b80 1 -- 172.21.15.52:0/394883295 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f37b8011c30 con 0x563a296fc530 2023-11-27T15:20:09.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.101+0000 7f37a6ff5700 1 -- 172.21.15.52:0/394883295 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37bc005040 con 0x563a296fc530 2023-11-27T15:20:09.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.101+0000 7f37d56e2b80 10 client.15981 fetch_fsmap learned FSMap version 23 2023-11-27T15:20:09.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.101+0000 7f37d56e2b80 1 -- 172.21.15.52:0/394883295 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f37c410c970 con 0x563a296fc530 2023-11-27T15:20:09.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.101+0000 7f37a6ff5700 1 -- 172.21.15.52:0/394883295 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37b8011c30 con 0x563a296fc530 2023-11-27T15:20:09.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.102+0000 7f37a6ff5700 1 -- 172.21.15.52:0/394883295 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f37bc019070 con 0x563a296fc530 2023-11-27T15:20:09.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.102+0000 7f37d56e2b80 10 client.15981 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:20:09.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.102+0000 7f37d56e2b80 10 client.15981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:20:09.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.102+0000 7f37d56e2b80 20 client.15981 populate_metadata read hostname 'smithi052' 2023-11-27T15:20:09.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.102+0000 7f37d56e2b80 1 --2- 172.21.15.52:0/394883295 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x563a29760a50 0x563a29762e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:09.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.102+0000 7f37d56e2b80 4 client.15981 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:20:09.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.102+0000 7f37d56e2b80 1 -- 172.21.15.52:0/394883295 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563a2960d660 con 0x563a29760a50 2023-11-27T15:20:09.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.102+0000 7f37c2ffd700 1 --2- 172.21.15.52:0/394883295 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x563a29760a50 0x563a29762e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:09.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.102+0000 7f37c2ffd700 1 --2- 172.21.15.52:0/394883295 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x563a29760a50 0x563a29762e60 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db5a648f2999f179 server_cookie=f9f35e5fc9a83a46 in_seq=0 out_seq=0 2023-11-27T15:20:09.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.102+0000 7f37a6ff5700 10 client.15981 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:20:09.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.103+0000 7f37a6ff5700 1 -- 172.21.15.52:0/394883295 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x563a2960d660 con 0x563a29760a50 2023-11-27T15:20:09.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.103+0000 7f37a6ff5700 10 client.15981 handle_command_reply: tid=0 2023-11-27T15:20:09.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.103+0000 7f37d56e2b80 1 client.15981 shutdown 2023-11-27T15:20:09.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.103+0000 7f37d56e2b80 1 -- 172.21.15.52:0/394883295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a296fc530 msgr2=0x563a296ff600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:09.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.103+0000 7f37d56e2b80 1 --2- 172.21.15.52:0/394883295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a296fc530 0x563a296ff600 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00bd20 tx=0x7f37bc006000 comp rx=0 tx=0).stop 2023-11-27T15:20:09.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.104+0000 7f37d56e2b80 1 -- 172.21.15.52:0/394883295 shutdown_connections 2023-11-27T15:20:09.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.104+0000 7f37d56e2b80 1 --2- 172.21.15.52:0/394883295 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x563a29760a50 0x563a29762e60 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.104+0000 7f37d56e2b80 1 --2- 172.21.15.52:0/394883295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a296fc530 0x563a296ff600 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.104+0000 7f37d56e2b80 1 --2- 172.21.15.52:0/394883295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563a296ed970 0x563a297010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.104+0000 7f37a6ff5700 0 client.15981 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:20:09.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.104+0000 7f37d56e2b80 1 --2- 172.21.15.52:0/394883295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a296ee280 0x563a29701610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.104+0000 7f37d56e2b80 1 -- 172.21.15.52:0/394883295 >> 172.21.15.52:0/394883295 conn(0x563a296e3360 msgr2=0x563a296ec960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:09.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.104+0000 7f37d56e2b80 1 -- 172.21.15.52:0/394883295 shutdown_connections 2023-11-27T15:20:09.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.104+0000 7f37d56e2b80 1 -- 172.21.15.52:0/394883295 wait complete. 2023-11-27T15:20:09.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.104+0000 7f37d56e2b80 20 client.15981 trim_cache size 0 max 16384 2023-11-27T15:20:09.112 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:20:09.112 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "scrub active (0 inodes in the stack)", 2023-11-27T15:20:09.112 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2023-11-27T15:20:09.112 INFO:teuthology.orchestra.run.smithi052.stdout: "27ec9b1e-7bb4-48fc-99ff-3f40b7797927": { 2023-11-27T15:20:09.112 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2023-11-27T15:20:09.112 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "27ec9b1e-7bb4-48fc-99ff-3f40b7797927", 2023-11-27T15:20:09.113 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2023-11-27T15:20:09.113 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:20:09.113 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:20:09.113 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:20:09.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.104+0000 7f37c9556700 1 -- 172.21.15.52:0/2283185696 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f37ac0a12c0 msgr2=0x7f37ac0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:09.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.104+0000 7f37c9556700 1 --2- 172.21.15.52:0/2283185696 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f37ac0a12c0 0x7f37ac0a3770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f37b8005950 tx=0x7f37b80058e0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.104+0000 7f37c9556700 1 -- 172.21.15.52:0/2283185696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f37c40ff520 msgr2=0x7f37c4195450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:09.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.104+0000 7f37c9556700 1 --2- 172.21.15.52:0/2283185696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f37c40ff520 0x7f37c4195450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f37b400ec10 tx=0x7f37b400ef20 comp rx=0 tx=0).stop 2023-11-27T15:20:09.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.104+0000 7f37c9556700 1 -- 172.21.15.52:0/2283185696 shutdown_connections 2023-11-27T15:20:09.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.104+0000 7f37c9556700 1 --2- 172.21.15.52:0/2283185696 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f37ac0a12c0 0x7f37ac0a3770 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.104+0000 7f37c9556700 1 --2- 172.21.15.52:0/2283185696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f37c40ff520 0x7f37c4195450 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.104+0000 7f37c9556700 1 --2- 172.21.15.52:0/2283185696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37c4104560 0x7f37c4199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.104+0000 7f37c9556700 1 --2- 172.21.15.52:0/2283185696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37c40ffec0 0x7f37c4195990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:09.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.104+0000 7f37c9556700 1 -- 172.21.15.52:0/2283185696 >> 172.21.15.52:0/2283185696 conn(0x7f37c40760d0 msgr2=0x7f37c40fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:09.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.105+0000 7f37c9556700 1 -- 172.21.15.52:0/2283185696 shutdown_connections 2023-11-27T15:20:09.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:09.105+0000 7f37c9556700 1 -- 172.21.15.52:0/2283185696 wait complete. 2023-11-27T15:20:09.122 INFO:tasks.cephfs.filesystem:scrub status for tag:27ec9b1e-7bb4-48fc-99ff-3f40b7797927 - {'path': '/', 'tag': '27ec9b1e-7bb4-48fc-99ff-3f40b7797927', 'options': 'recursive,force'} 2023-11-27T15:20:10.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:10 smithi052 ceph-mon[99056]: scrub queued for path: / 2023-11-27T15:20:10.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:10 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:20:10.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:10 smithi052 ceph-mon[99056]: scrub summary: active paths [/] 2023-11-27T15:20:10.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:10 smithi052 ceph-mon[99056]: pgmap v70: 97 pgs: 97 active+clean; 2.9 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 42 KiB/s wr, 19 op/s 2023-11-27T15:20:10.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:10 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:20:10.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:10 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:20:10.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:10 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:20:10.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:10 smithi052 ceph-mon[99056]: scrub summary: idle 2023-11-27T15:20:10.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:10 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:10.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:10 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:20:10.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:10 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:20:10.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:10 smithi150 ceph-mon[100559]: scrub queued for path: / 2023-11-27T15:20:10.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:10 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:20:10.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:10 smithi150 ceph-mon[100559]: scrub summary: active paths [/] 2023-11-27T15:20:10.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:10 smithi150 ceph-mon[100559]: pgmap v70: 97 pgs: 97 active+clean; 2.9 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 42 KiB/s wr, 19 op/s 2023-11-27T15:20:10.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:10 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:20:10.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:10 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:20:10.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:10 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:20:10.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:10 smithi150 ceph-mon[100559]: scrub summary: idle 2023-11-27T15:20:10.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:10 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:10.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:10 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:20:10.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:10 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:20:10.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:10 smithi102 ceph-mon[100296]: scrub queued for path: / 2023-11-27T15:20:10.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:10 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:20:10.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:10 smithi102 ceph-mon[100296]: scrub summary: active paths [/] 2023-11-27T15:20:10.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:10 smithi102 ceph-mon[100296]: pgmap v70: 97 pgs: 97 active+clean; 2.9 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 42 KiB/s wr, 19 op/s 2023-11-27T15:20:10.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:10 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:20:10.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:10 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:20:10.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:10 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:20:10.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:10 smithi102 ceph-mon[100296]: scrub summary: idle 2023-11-27T15:20:10.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:10 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:10.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:10 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:20:10.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:10 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:20:11.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:11 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:20:11.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:11 smithi052 ceph-mon[99056]: Service mds.all has no related daemon already placed. 2023-11-27T15:20:11.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:11 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:20:11.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:11 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:20:11.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:11 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:20:11.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:11 smithi150 ceph-mon[100559]: Service mds.all has no related daemon already placed. 2023-11-27T15:20:11.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:11 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:20:11.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:11 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:20:11.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:11 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:20:11.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:11 smithi102 ceph-mon[100296]: Service mds.all has no related daemon already placed. 2023-11-27T15:20:11.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:11 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:20:11.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:11 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:20:12.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:12 smithi052 ceph-mon[99056]: pgmap v71: 97 pgs: 97 active+clean; 8.2 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 538 KiB/s wr, 34 op/s 2023-11-27T15:20:12.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:12 smithi150 ceph-mon[100559]: pgmap v71: 97 pgs: 97 active+clean; 8.2 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 538 KiB/s wr, 34 op/s 2023-11-27T15:20:12.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:12 smithi102 ceph-mon[100296]: pgmap v71: 97 pgs: 97 active+clean; 8.2 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 538 KiB/s wr, 34 op/s 2023-11-27T15:20:14.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:14 smithi052 ceph-mon[99056]: pgmap v72: 97 pgs: 97 active+clean; 8.2 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 503 KiB/s wr, 18 op/s 2023-11-27T15:20:14.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:14 smithi150 ceph-mon[100559]: pgmap v72: 97 pgs: 97 active+clean; 8.2 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 503 KiB/s wr, 18 op/s 2023-11-27T15:20:14.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:14 smithi102 ceph-mon[100296]: pgmap v72: 97 pgs: 97 active+clean; 8.2 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 503 KiB/s wr, 18 op/s 2023-11-27T15:20:16.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:16 smithi150 ceph-mon[100559]: pgmap v73: 97 pgs: 97 active+clean; 8.2 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 503 KiB/s wr, 18 op/s 2023-11-27T15:20:16.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:16 smithi150 ceph-mon[100559]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:20:16.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:16 smithi102 ceph-mon[100296]: pgmap v73: 97 pgs: 97 active+clean; 8.2 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 503 KiB/s wr, 18 op/s 2023-11-27T15:20:16.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:16 smithi102 ceph-mon[100296]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:20:16.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:16 smithi052 ceph-mon[99056]: pgmap v73: 97 pgs: 97 active+clean; 8.2 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 503 KiB/s wr, 18 op/s 2023-11-27T15:20:16.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:16 smithi052 ceph-mon[99056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:20:18.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:18 smithi150 ceph-mon[100559]: pgmap v74: 97 pgs: 97 active+clean; 20 MiB data, 112 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 79 op/s 2023-11-27T15:20:18.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:18 smithi102 ceph-mon[100296]: pgmap v74: 97 pgs: 97 active+clean; 20 MiB data, 112 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 79 op/s 2023-11-27T15:20:18.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:18 smithi052 ceph-mon[99056]: pgmap v74: 97 pgs: 97 active+clean; 20 MiB data, 112 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 79 op/s 2023-11-27T15:20:20.079 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:19 smithi102 ceph-mon[100296]: pgmap v75: 97 pgs: 97 active+clean; 20 MiB data, 112 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 76 op/s 2023-11-27T15:20:20.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:19 smithi052 ceph-mon[99056]: pgmap v75: 97 pgs: 97 active+clean; 20 MiB data, 112 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 76 op/s 2023-11-27T15:20:20.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:19 smithi150 ceph-mon[100559]: pgmap v75: 97 pgs: 97 active+clean; 20 MiB data, 112 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 76 op/s 2023-11-27T15:20:22.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:22 smithi150 ceph-mon[100559]: pgmap v76: 97 pgs: 97 active+clean; 35 MiB data, 131 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 106 op/s 2023-11-27T15:20:22.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:22 smithi102 ceph-mon[100296]: pgmap v76: 97 pgs: 97 active+clean; 35 MiB data, 131 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 106 op/s 2023-11-27T15:20:22.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:22 smithi052 ceph-mon[99056]: pgmap v76: 97 pgs: 97 active+clean; 35 MiB data, 131 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 106 op/s 2023-11-27T15:20:24.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:24 smithi150 ceph-mon[100559]: pgmap v77: 97 pgs: 97 active+clean; 35 MiB data, 131 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 90 op/s 2023-11-27T15:20:24.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:24 smithi102 ceph-mon[100296]: pgmap v77: 97 pgs: 97 active+clean; 35 MiB data, 131 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 90 op/s 2023-11-27T15:20:24.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:24 smithi052 ceph-mon[99056]: pgmap v77: 97 pgs: 97 active+clean; 35 MiB data, 131 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 90 op/s 2023-11-27T15:20:26.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:26 smithi150 ceph-mon[100559]: pgmap v78: 97 pgs: 97 active+clean; 35 MiB data, 131 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 90 op/s 2023-11-27T15:20:26.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:26 smithi102 ceph-mon[100296]: pgmap v78: 97 pgs: 97 active+clean; 35 MiB data, 131 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 90 op/s 2023-11-27T15:20:26.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:26 smithi052 ceph-mon[99056]: pgmap v78: 97 pgs: 97 active+clean; 35 MiB data, 131 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 90 op/s 2023-11-27T15:20:28.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:28 smithi150 ceph-mon[100559]: pgmap v79: 97 pgs: 97 active+clean; 55 MiB data, 150 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.4 MiB/s wr, 100 op/s 2023-11-27T15:20:28.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:28 smithi102 ceph-mon[100296]: pgmap v79: 97 pgs: 97 active+clean; 55 MiB data, 150 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.4 MiB/s wr, 100 op/s 2023-11-27T15:20:28.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:28 smithi052 ceph-mon[99056]: pgmap v79: 97 pgs: 97 active+clean; 55 MiB data, 150 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.4 MiB/s wr, 100 op/s 2023-11-27T15:20:30.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:29 smithi102 ceph-mon[100296]: pgmap v80: 97 pgs: 97 active+clean; 55 MiB data, 150 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 38 op/s 2023-11-27T15:20:30.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:29 smithi052 ceph-mon[99056]: pgmap v80: 97 pgs: 97 active+clean; 55 MiB data, 150 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 38 op/s 2023-11-27T15:20:30.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:29 smithi150 ceph-mon[100559]: pgmap v80: 97 pgs: 97 active+clean; 55 MiB data, 150 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 38 op/s 2023-11-27T15:20:32.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:32 smithi150 ceph-mon[100559]: pgmap v81: 97 pgs: 97 active+clean; 75 MiB data, 164 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.8 MiB/s wr, 49 op/s 2023-11-27T15:20:32.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:32 smithi102 ceph-mon[100296]: pgmap v81: 97 pgs: 97 active+clean; 75 MiB data, 164 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.8 MiB/s wr, 49 op/s 2023-11-27T15:20:32.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:32 smithi052 ceph-mon[99056]: pgmap v81: 97 pgs: 97 active+clean; 75 MiB data, 164 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.8 MiB/s wr, 49 op/s 2023-11-27T15:20:34.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:34 smithi150 ceph-mon[100559]: pgmap v82: 97 pgs: 97 active+clean; 75 MiB data, 164 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 20 op/s 2023-11-27T15:20:34.631 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:34 smithi052 ceph-mon[99056]: pgmap v82: 97 pgs: 97 active+clean; 75 MiB data, 164 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 20 op/s 2023-11-27T15:20:34.632 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:34 smithi102 ceph-mon[100296]: pgmap v82: 97 pgs: 97 active+clean; 75 MiB data, 164 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 20 op/s 2023-11-27T15:20:36.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:36 smithi150 ceph-mon[100559]: pgmap v83: 97 pgs: 97 active+clean; 75 MiB data, 164 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 20 op/s 2023-11-27T15:20:36.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:36 smithi102 ceph-mon[100296]: pgmap v83: 97 pgs: 97 active+clean; 75 MiB data, 164 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 20 op/s 2023-11-27T15:20:36.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:36 smithi052 ceph-mon[99056]: pgmap v83: 97 pgs: 97 active+clean; 75 MiB data, 164 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 20 op/s 2023-11-27T15:20:38.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:38 smithi150 ceph-mon[100559]: pgmap v84: 97 pgs: 97 active+clean; 90 MiB data, 264 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.0 MiB/s wr, 328 op/s 2023-11-27T15:20:38.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:38 smithi102 ceph-mon[100296]: pgmap v84: 97 pgs: 97 active+clean; 90 MiB data, 264 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.0 MiB/s wr, 328 op/s 2023-11-27T15:20:38.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:38 smithi052 ceph-mon[99056]: pgmap v84: 97 pgs: 97 active+clean; 90 MiB data, 264 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.0 MiB/s wr, 328 op/s 2023-11-27T15:20:39.123 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:20:39.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.265+0000 7fb3b1cb0700 1 -- 172.21.15.52:0/4129304388 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3ac0ff520 msgr2=0x7fb3ac0ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.265+0000 7fb3b1cb0700 1 --2- 172.21.15.52:0/4129304388 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3ac0ff520 0x7fb3ac0ff8f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009fc0 tx=0x7fb39c00b330 comp rx=0 tx=0).stop 2023-11-27T15:20:39.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.265+0000 7fb3b1cb0700 1 -- 172.21.15.52:0/4129304388 shutdown_connections 2023-11-27T15:20:39.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.265+0000 7fb3b1cb0700 1 --2- 172.21.15.52:0/4129304388 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3ac104560 0x7fb3ac10c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.265+0000 7fb3b1cb0700 1 --2- 172.21.15.52:0/4129304388 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3ac0ff520 0x7fb3ac0ff8f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.265+0000 7fb3b1cb0700 1 --2- 172.21.15.52:0/4129304388 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3ac0ffec0 0x7fb3ac103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.265+0000 7fb3b1cb0700 1 -- 172.21.15.52:0/4129304388 >> 172.21.15.52:0/4129304388 conn(0x7fb3ac0760d0 msgr2=0x7fb3ac0764d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:39.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.265+0000 7fb3b1cb0700 1 -- 172.21.15.52:0/4129304388 shutdown_connections 2023-11-27T15:20:39.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.265+0000 7fb3b1cb0700 1 -- 172.21.15.52:0/4129304388 wait complete. 2023-11-27T15:20:39.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.265+0000 7fb3b1cb0700 1 Processor -- start 2023-11-27T15:20:39.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.266+0000 7fb3b1cb0700 1 -- start start 2023-11-27T15:20:39.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.266+0000 7fb3b1cb0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3ac0ff520 0x7fb3ac195180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.266+0000 7fb3b1cb0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3ac0ffec0 0x7fb3ac1956c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.266+0000 7fb3b1cb0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3ac104560 0x7fb3ac195c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.266+0000 7fb3b1cb0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac1a49c0 con 0x7fb3ac0ff520 2023-11-27T15:20:39.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.266+0000 7fb3b1cb0700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac1a4b30 con 0x7fb3ac0ffec0 2023-11-27T15:20:39.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.266+0000 7fb3b1cb0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac1a4cf0 con 0x7fb3ac104560 2023-11-27T15:20:39.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.266+0000 7fb3ab7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3ac0ff520 0x7fb3ac195180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.266+0000 7fb3abfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3ac104560 0x7fb3ac195c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.266+0000 7fb3ab7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3ac0ff520 0x7fb3ac195180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53402/0 (socket says 172.21.15.52:53402) 2023-11-27T15:20:39.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.266+0000 7fb3ab7fe700 1 -- 172.21.15.52:0/2656865839 learned_addr learned my addr 172.21.15.52:0/2656865839 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:39.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.266+0000 7fb3aaffd700 1 --2- 172.21.15.52:0/2656865839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3ac0ffec0 0x7fb3ac1956c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.266+0000 7fb3abfff700 1 -- 172.21.15.52:0/2656865839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3ac0ffec0 msgr2=0x7fb3ac1956c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.266+0000 7fb3abfff700 1 --2- 172.21.15.52:0/2656865839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3ac0ffec0 0x7fb3ac1956c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.266+0000 7fb3abfff700 1 -- 172.21.15.52:0/2656865839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3ac0ff520 msgr2=0x7fb3ac195180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.266+0000 7fb3abfff700 1 --2- 172.21.15.52:0/2656865839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3ac0ff520 0x7fb3ac195180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.266+0000 7fb3abfff700 1 -- 172.21.15.52:0/2656865839 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a00095f0 con 0x7fb3ac104560 2023-11-27T15:20:39.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.267+0000 7fb3abfff700 1 --2- 172.21.15.52:0/2656865839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3ac104560 0x7fb3ac195c00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400e900 tx=0x7fb3a400ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:39.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.267+0000 7fb3a8ff9700 1 -- 172.21.15.52:0/2656865839 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a400cbd0 con 0x7fb3ac104560 2023-11-27T15:20:39.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.267+0000 7fb3a8ff9700 1 -- 172.21.15.52:0/2656865839 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3a400ee50 con 0x7fb3ac104560 2023-11-27T15:20:39.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.267+0000 7fb3b1cb0700 1 -- 172.21.15.52:0/2656865839 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3a4009520 con 0x7fb3ac104560 2023-11-27T15:20:39.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.267+0000 7fb3a8ff9700 1 -- 172.21.15.52:0/2656865839 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4003d30 con 0x7fb3ac104560 2023-11-27T15:20:39.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.267+0000 7fb3b1cb0700 1 -- 172.21.15.52:0/2656865839 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb39c009c40 con 0x7fb3ac104560 2023-11-27T15:20:39.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.270+0000 7fb3a8ff9700 1 -- 172.21.15.52:0/2656865839 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb3a400cd30 con 0x7fb3ac104560 2023-11-27T15:20:39.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.270+0000 7fb3a8ff9700 1 --2- 172.21.15.52:0/2656865839 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb3940a1340 0x7fb3940a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.270+0000 7fb3a8ff9700 1 -- 172.21.15.52:0/2656865839 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb3a4014070 con 0x7fb3ac104560 2023-11-27T15:20:39.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.270+0000 7fb3ab7fe700 1 --2- 172.21.15.52:0/2656865839 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb3940a1340 0x7fb3940a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.271+0000 7fb3ab7fe700 1 --2- 172.21.15.52:0/2656865839 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb3940a1340 0x7fb3940a37f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb39c006130 tx=0x7fb39c006080 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:39.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.271+0000 7fb3bde3cb80 1 Processor -- start 2023-11-27T15:20:39.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.271+0000 7fb3bde3cb80 1 -- start start 2023-11-27T15:20:39.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.271+0000 7fb3bde3cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c94436b60 0x562c94436f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.271+0000 7fb3bde3cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c94437470 0x562c9443b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.271+0000 7fb3bde3cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c9443b8b0 0x562c9443bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.271+0000 7fb3bde3cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c9433c060 con 0x562c94437470 2023-11-27T15:20:39.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.271+0000 7fb3bde3cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c9438e8a0 con 0x562c94436b60 2023-11-27T15:20:39.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.271+0000 7fb3bde3cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c94387d30 con 0x562c9443b8b0 2023-11-27T15:20:39.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.271+0000 7fb3ab7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c94436b60 0x562c94436f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.271+0000 7fb3ab7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c94436b60 0x562c94436f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43068/0 (socket says 172.21.15.52:43068) 2023-11-27T15:20:39.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.271+0000 7fb3ab7fe700 1 -- 172.21.15.52:0/2476434824 learned_addr learned my addr 172.21.15.52:0/2476434824 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:39.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.272+0000 7fb3abfff700 1 --2- 172.21.15.52:0/2476434824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c9443b8b0 0x562c9443bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.272+0000 7fb3aaffd700 1 --2- 172.21.15.52:0/2476434824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c94437470 0x562c9443b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.272+0000 7fb3ab7fe700 1 -- 172.21.15.52:0/2476434824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c9443b8b0 msgr2=0x562c9443bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:39.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.272+0000 7fb3ab7fe700 1 --2- 172.21.15.52:0/2476434824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c9443b8b0 0x562c9443bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.272+0000 7fb3ab7fe700 1 -- 172.21.15.52:0/2476434824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c94437470 msgr2=0x562c9443b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:39.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.272+0000 7fb3ab7fe700 1 --2- 172.21.15.52:0/2476434824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c94437470 0x562c9443b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.272+0000 7fb3ab7fe700 1 -- 172.21.15.52:0/2476434824 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c94209630 con 0x562c94436b60 2023-11-27T15:20:39.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.272+0000 7fb3aaffd700 1 --2- 172.21.15.52:0/2476434824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c94437470 0x562c9443b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:20:39.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.272+0000 7fb3ab7fe700 1 --2- 172.21.15.52:0/2476434824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c94436b60 0x562c94436f30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb39c007ed0 tx=0x7fb39c01b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26444a2d81b36434 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:39.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.272+0000 7fb391ffb700 1 -- 172.21.15.52:0/2476434824 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39c009ac0 con 0x562c94436b60 2023-11-27T15:20:39.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.272+0000 7fb391ffb700 1 -- 172.21.15.52:0/2476434824 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb39c0050b0 con 0x562c94436b60 2023-11-27T15:20:39.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.272+0000 7fb391ffb700 1 -- 172.21.15.52:0/2476434824 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39c028680 con 0x562c94436b60 2023-11-27T15:20:39.286 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.273+0000 7fb3bde3cb80 1 -- 172.21.15.52:0/2476434824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c94436b60 msgr2=0x562c94436f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.286 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.273+0000 7fb3bde3cb80 1 --2- 172.21.15.52:0/2476434824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c94436b60 0x562c94436f30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb39c007ed0 tx=0x7fb39c01b000 comp rx=0 tx=0).stop 2023-11-27T15:20:39.286 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.273+0000 7fb3bde3cb80 1 -- 172.21.15.52:0/2476434824 shutdown_connections 2023-11-27T15:20:39.286 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.273+0000 7fb3bde3cb80 1 --2- 172.21.15.52:0/2476434824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c94436b60 0x562c94436f30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.273+0000 7fb3bde3cb80 1 --2- 172.21.15.52:0/2476434824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c9443b8b0 0x562c9443bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.273+0000 7fb3bde3cb80 1 --2- 172.21.15.52:0/2476434824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c94437470 0x562c9443b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.273+0000 7fb3bde3cb80 1 -- 172.21.15.52:0/2476434824 >> 172.21.15.52:0/2476434824 conn(0x562c9442c360 msgr2=0x562c94431b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:39.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.273+0000 7fb3bde3cb80 1 -- 172.21.15.52:0/2476434824 shutdown_connections 2023-11-27T15:20:39.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.273+0000 7fb3bde3cb80 1 -- 172.21.15.52:0/2476434824 wait complete. 2023-11-27T15:20:39.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.273+0000 7fb3bde3cb80 1 -- start start 2023-11-27T15:20:39.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.274+0000 7fb3bde3cb80 1 Processor -- start 2023-11-27T15:20:39.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.274+0000 7fb3bde3cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c94436b60 0x562c944430d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.274+0000 7fb3bde3cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c94437470 0x562c94440350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.274+0000 7fb3bde3cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c9443b8b0 0x562c94440a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.274+0000 7fb3bde3cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c943298e0 con 0x562c94437470 2023-11-27T15:20:39.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.274+0000 7fb3bde3cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c94329580 con 0x562c9443b8b0 2023-11-27T15:20:39.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.274+0000 7fb3bde3cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c9420b920 con 0x562c94436b60 2023-11-27T15:20:39.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.274+0000 7fb3bde3cb80 10 client.0 _ll_register_callbacks cb 0x562c94347780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:20:39.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.274+0000 7fb3aaffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c94437470 0x562c94440350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.274+0000 7fb3ab7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c94436b60 0x562c944430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.274+0000 7fb3abfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c9443b8b0 0x562c94440a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.274+0000 7fb3aaffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c94437470 0x562c94440350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53418/0 (socket says 172.21.15.52:53418) 2023-11-27T15:20:39.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.274+0000 7fb3ab7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c94436b60 0x562c944430d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:40060/0 (socket says 172.21.15.52:40060) 2023-11-27T15:20:39.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.274+0000 7fb3aaffd700 1 -- 172.21.15.52:0/1132727150 learned_addr learned my addr 172.21.15.52:0/1132727150 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:39.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.275+0000 7fb3ab7fe700 1 -- 172.21.15.52:0/1132727150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c9443b8b0 msgr2=0x562c94440a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:39.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.275+0000 7fb3ab7fe700 1 --2- 172.21.15.52:0/1132727150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c9443b8b0 0x562c94440a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.275+0000 7fb3ab7fe700 1 -- 172.21.15.52:0/1132727150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c94437470 msgr2=0x562c94440350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:39.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.275+0000 7fb3ab7fe700 1 --2- 172.21.15.52:0/1132727150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c94437470 0x562c94440350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.275+0000 7fb3ab7fe700 1 -- 172.21.15.52:0/1132727150 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3ac10c970 con 0x562c94436b60 2023-11-27T15:20:39.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.275+0000 7fb3ab7fe700 1 --2- 172.21.15.52:0/1132727150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c94436b60 0x562c944430d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009550 tx=0x7fb39c007cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7fd8a24ccfd558e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:39.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.275+0000 7fb38affd700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2023-11-27T15:20:39.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.275+0000 7fb38affd700 1 -- 172.21.15.52:0/1132727150 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39c003c00 con 0x562c94436b60 2023-11-27T15:20:39.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.275+0000 7fb38affd700 1 -- 172.21.15.52:0/1132727150 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb39c003d60 con 0x562c94436b60 2023-11-27T15:20:39.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.275+0000 7fb3bde3cb80 1 -- 172.21.15.52:0/1132727150 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562c94429b20 con 0x562c94436b60 2023-11-27T15:20:39.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.275+0000 7fb38affd700 1 -- 172.21.15.52:0/1132727150 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb39c0077f0 con 0x562c94436b60 2023-11-27T15:20:39.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.275+0000 7fb3bde3cb80 10 client.25702 fetch_fsmap learned FSMap version 24 2023-11-27T15:20:39.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.275+0000 7fb3bde3cb80 1 -- 172.21.15.52:0/1132727150 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562c9434e190 con 0x562c94436b60 2023-11-27T15:20:39.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.276+0000 7fb38affd700 1 -- 172.21.15.52:0/1132727150 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562c94429b20 con 0x562c94436b60 2023-11-27T15:20:39.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.276+0000 7fb38affd700 1 -- 172.21.15.52:0/1132727150 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb39c03fc40 con 0x562c94436b60 2023-11-27T15:20:39.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.276+0000 7fb3bde3cb80 10 client.25702 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:20:39.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.276+0000 7fb3bde3cb80 10 client.25702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:20:39.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.276+0000 7fb3bde3cb80 20 client.25702 populate_metadata read hostname 'smithi052' 2023-11-27T15:20:39.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.276+0000 7fb3bde3cb80 1 --2- 172.21.15.52:0/1132727150 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x562c94450350 0x562c94452730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.276+0000 7fb3bde3cb80 4 client.25702 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:20:39.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.276+0000 7fb3bde3cb80 1 -- 172.21.15.52:0/1132727150 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c9434a0a0 con 0x562c94450350 2023-11-27T15:20:39.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.276+0000 7fb3aaffd700 1 --2- 172.21.15.52:0/1132727150 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x562c94450350 0x562c94452730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.276+0000 7fb38affd700 10 client.25702 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:20:39.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.276+0000 7fb3aaffd700 1 --2- 172.21.15.52:0/1132727150 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x562c94450350 0x562c94452730 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c82c50948a24726 server_cookie=6a22e84b8af77d70 in_seq=0 out_seq=0 2023-11-27T15:20:39.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.277+0000 7fb38affd700 1 -- 172.21.15.52:0/1132727150 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562c9434a0a0 con 0x562c94450350 2023-11-27T15:20:39.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.277+0000 7fb38affd700 10 client.25702 handle_command_reply: tid=0 2023-11-27T15:20:39.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.277+0000 7fb3bde3cb80 1 client.25702 shutdown 2023-11-27T15:20:39.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.277+0000 7fb3bde3cb80 1 -- 172.21.15.52:0/1132727150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c94436b60 msgr2=0x562c944430d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.277+0000 7fb3bde3cb80 1 --2- 172.21.15.52:0/1132727150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c94436b60 0x562c944430d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009550 tx=0x7fb39c007cb0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.277+0000 7fb3bde3cb80 1 -- 172.21.15.52:0/1132727150 shutdown_connections 2023-11-27T15:20:39.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.277+0000 7fb3bde3cb80 1 --2- 172.21.15.52:0/1132727150 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x562c94450350 0x562c94452730 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.277+0000 7fb3bde3cb80 1 --2- 172.21.15.52:0/1132727150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c9443b8b0 0x562c94440a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.277+0000 7fb38affd700 0 client.25702 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:20:39.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.277+0000 7fb3bde3cb80 1 --2- 172.21.15.52:0/1132727150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c94436b60 0x562c944430d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.277+0000 7fb3bde3cb80 1 --2- 172.21.15.52:0/1132727150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c94437470 0x562c94440350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.277+0000 7fb3bde3cb80 1 -- 172.21.15.52:0/1132727150 >> 172.21.15.52:0/1132727150 conn(0x562c9442c360 msgr2=0x562c94430de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:39.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.277+0000 7fb3bde3cb80 1 -- 172.21.15.52:0/1132727150 shutdown_connections 2023-11-27T15:20:39.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.277+0000 7fb3bde3cb80 1 -- 172.21.15.52:0/1132727150 wait complete. 2023-11-27T15:20:39.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.278+0000 7fb3bde3cb80 20 client.25702 trim_cache size 0 max 16384 2023-11-27T15:20:39.302 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.296+0000 7fb3bde3cb80 1 Processor -- start 2023-11-27T15:20:39.303 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.296+0000 7fb3bde3cb80 1 -- start start 2023-11-27T15:20:39.303 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.297+0000 7fb3bde3cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c94436b60 0x562c94447940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.303 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.297+0000 7fb3bde3cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c94437470 0x562c94439f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.303 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.297+0000 7fb3bde3cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c9443b8b0 0x562c94431560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.303 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.297+0000 7fb3bde3cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c9442bee0 con 0x562c9443b8b0 2023-11-27T15:20:39.303 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.297+0000 7fb3bde3cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c94429660 con 0x562c94436b60 2023-11-27T15:20:39.303 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.297+0000 7fb3bde3cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c94441900 con 0x562c94437470 2023-11-27T15:20:39.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.297+0000 7fb3ab7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c94436b60 0x562c94447940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.297+0000 7fb3ab7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c94436b60 0x562c94447940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43082/0 (socket says 172.21.15.52:43082) 2023-11-27T15:20:39.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.297+0000 7fb3ab7fe700 1 -- 172.21.15.52:0/3970607309 learned_addr learned my addr 172.21.15.52:0/3970607309 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:39.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.297+0000 7fb3abfff700 1 --2- 172.21.15.52:0/3970607309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c9443b8b0 0x562c94431560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.297+0000 7fb3ab7fe700 1 -- 172.21.15.52:0/3970607309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c94437470 msgr2=0x562c94439f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:39.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.297+0000 7fb3ab7fe700 1 --2- 172.21.15.52:0/3970607309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c94437470 0x562c94439f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.297+0000 7fb3ab7fe700 1 -- 172.21.15.52:0/3970607309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c9443b8b0 msgr2=0x562c94431560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:39.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.297+0000 7fb3ab7fe700 1 --2- 172.21.15.52:0/3970607309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c9443b8b0 0x562c94431560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.297+0000 7fb3ab7fe700 1 -- 172.21.15.52:0/3970607309 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c9442db00 con 0x562c94436b60 2023-11-27T15:20:39.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.297+0000 7fb3abfff700 1 --2- 172.21.15.52:0/3970607309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c9443b8b0 0x562c94431560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:39.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.298+0000 7fb3ab7fe700 1 --2- 172.21.15.52:0/3970607309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c94436b60 0x562c94447940 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb39c005680 tx=0x7fb39c028550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b575e1dc46a9e690 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:39.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.298+0000 7fb3927fc700 1 -- 172.21.15.52:0/3970607309 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39c044070 con 0x562c94436b60 2023-11-27T15:20:39.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.298+0000 7fb3927fc700 1 -- 172.21.15.52:0/3970607309 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb39c028650 con 0x562c94436b60 2023-11-27T15:20:39.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.298+0000 7fb3927fc700 1 -- 172.21.15.52:0/3970607309 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39c028b80 con 0x562c94436b60 2023-11-27T15:20:39.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.299+0000 7fb3bde3cb80 1 -- 172.21.15.52:0/3970607309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c94436b60 msgr2=0x562c94447940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.299+0000 7fb3bde3cb80 1 --2- 172.21.15.52:0/3970607309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c94436b60 0x562c94447940 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb39c005680 tx=0x7fb39c028550 comp rx=0 tx=0).stop 2023-11-27T15:20:39.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.299+0000 7fb3bde3cb80 1 -- 172.21.15.52:0/3970607309 shutdown_connections 2023-11-27T15:20:39.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.299+0000 7fb3bde3cb80 1 --2- 172.21.15.52:0/3970607309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c94436b60 0x562c94447940 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.299+0000 7fb3bde3cb80 1 --2- 172.21.15.52:0/3970607309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c9443b8b0 0x562c94431560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.299+0000 7fb3bde3cb80 1 --2- 172.21.15.52:0/3970607309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c94437470 0x562c94439f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.299+0000 7fb3bde3cb80 1 -- 172.21.15.52:0/3970607309 >> 172.21.15.52:0/3970607309 conn(0x562c9442c360 msgr2=0x562c9444cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:39.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.299+0000 7fb3bde3cb80 1 -- 172.21.15.52:0/3970607309 shutdown_connections 2023-11-27T15:20:39.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.299+0000 7fb3bde3cb80 1 -- 172.21.15.52:0/3970607309 wait complete. 2023-11-27T15:20:39.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.299+0000 7fb3bde3cb80 1 -- start start 2023-11-27T15:20:39.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.300+0000 7fb3bde3cb80 1 Processor -- start 2023-11-27T15:20:39.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.300+0000 7fb3bde3cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c94436b60 0x562c9443fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.300+0000 7fb3bde3cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c94437470 0x562c94440070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.300+0000 7fb3bde3cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c9443b8b0 0x562c9449f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.300+0000 7fb3bde3cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c94447dc0 con 0x562c94437470 2023-11-27T15:20:39.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.300+0000 7fb3bde3cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c94447f30 con 0x562c94436b60 2023-11-27T15:20:39.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.300+0000 7fb3bde3cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c9449f680 con 0x562c9443b8b0 2023-11-27T15:20:39.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.300+0000 7fb3bde3cb80 10 client.0 _ll_register_callbacks cb 0x7fb39c02b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:20:39.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.300+0000 7fb3ab7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c94436b60 0x562c9443fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.300+0000 7fb3ab7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c94436b60 0x562c9443fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43088/0 (socket says 172.21.15.52:43088) 2023-11-27T15:20:39.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.300+0000 7fb3ab7fe700 1 -- 172.21.15.52:0/2012651101 learned_addr learned my addr 172.21.15.52:0/2012651101 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:39.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.300+0000 7fb3aaffd700 1 --2- 172.21.15.52:0/2012651101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c94437470 0x562c94440070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.300+0000 7fb3abfff700 1 --2- 172.21.15.52:0/2012651101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c9443b8b0 0x562c9449f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.301+0000 7fb3ab7fe700 1 -- 172.21.15.52:0/2012651101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c9443b8b0 msgr2=0x562c9449f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:39.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.301+0000 7fb3ab7fe700 1 --2- 172.21.15.52:0/2012651101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c9443b8b0 0x562c9449f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.301+0000 7fb3ab7fe700 1 -- 172.21.15.52:0/2012651101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c94437470 msgr2=0x562c94440070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:39.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.301+0000 7fb3ab7fe700 1 --2- 172.21.15.52:0/2012651101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c94437470 0x562c94440070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.301+0000 7fb3ab7fe700 1 -- 172.21.15.52:0/2012651101 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb39c009c40 con 0x562c94436b60 2023-11-27T15:20:39.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.301+0000 7fb3ab7fe700 1 --2- 172.21.15.52:0/2012651101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c94436b60 0x562c9443fb30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb39c028520 tx=0x7fb39c005550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4ea8fa1b350e621 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:39.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.301+0000 7fb38affd700 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2023-11-27T15:20:39.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.301+0000 7fb38affd700 1 -- 172.21.15.52:0/2012651101 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39c044030 con 0x562c94436b60 2023-11-27T15:20:39.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.301+0000 7fb38affd700 1 -- 172.21.15.52:0/2012651101 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fb39c045070 con 0x562c94436b60 2023-11-27T15:20:39.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.301+0000 7fb3bde3cb80 1 -- 172.21.15.52:0/2012651101 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb39c0050b0 con 0x562c94436b60 2023-11-27T15:20:39.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.302+0000 7fb38affd700 1 -- 172.21.15.52:0/2012651101 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb39c007c30 con 0x562c94436b60 2023-11-27T15:20:39.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.302+0000 7fb3bde3cb80 10 client.25811 fetch_fsmap learned FSMap version 24 2023-11-27T15:20:39.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.302+0000 7fb3bde3cb80 1 -- 172.21.15.52:0/2012651101 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562c9434e190 con 0x562c94436b60 2023-11-27T15:20:39.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.302+0000 7fb38affd700 1 -- 172.21.15.52:0/2012651101 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb39c0050b0 con 0x562c94436b60 2023-11-27T15:20:39.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.302+0000 7fb38affd700 1 -- 172.21.15.52:0/2012651101 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb39c03fc90 con 0x562c94436b60 2023-11-27T15:20:39.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.302+0000 7fb3bde3cb80 10 client.25811 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:20:39.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.302+0000 7fb3bde3cb80 10 client.25811 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:20:39.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.302+0000 7fb3bde3cb80 20 client.25811 populate_metadata read hostname 'smithi052' 2023-11-27T15:20:39.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.302+0000 7fb3bde3cb80 1 --2- 172.21.15.52:0/2012651101 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x562c9443f110 0x562c94446950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.302+0000 7fb3bde3cb80 4 client.25811 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:20:39.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.302+0000 7fb3bde3cb80 1 -- 172.21.15.52:0/2012651101 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562c94356660 con 0x562c9443f110 2023-11-27T15:20:39.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.302+0000 7fb3aaffd700 1 --2- 172.21.15.52:0/2012651101 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x562c9443f110 0x562c94446950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.303+0000 7fb38affd700 10 client.25811 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:20:39.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.303+0000 7fb3aaffd700 1 --2- 172.21.15.52:0/2012651101 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x562c9443f110 0x562c94446950 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59fd26ad8df9598e server_cookie=cb9fcd8da26434c2 in_seq=0 out_seq=0 2023-11-27T15:20:39.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.308+0000 7fb38affd700 1 -- 172.21.15.52:0/2012651101 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562c94356660 con 0x562c9443f110 2023-11-27T15:20:39.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.308+0000 7fb38affd700 10 client.25811 handle_command_reply: tid=0 2023-11-27T15:20:39.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.308+0000 7fb3bde3cb80 1 client.25811 shutdown 2023-11-27T15:20:39.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.309+0000 7fb3bde3cb80 1 -- 172.21.15.52:0/2012651101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c94436b60 msgr2=0x562c9443fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.309+0000 7fb3bde3cb80 1 --2- 172.21.15.52:0/2012651101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c94436b60 0x562c9443fb30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb39c028520 tx=0x7fb39c005550 comp rx=0 tx=0).stop 2023-11-27T15:20:39.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.309+0000 7fb3bde3cb80 1 -- 172.21.15.52:0/2012651101 shutdown_connections 2023-11-27T15:20:39.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.309+0000 7fb3bde3cb80 1 --2- 172.21.15.52:0/2012651101 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x562c9443f110 0x562c94446950 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.309+0000 7fb3bde3cb80 1 --2- 172.21.15.52:0/2012651101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c94436b60 0x562c9443fb30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.309+0000 7fb3bde3cb80 1 --2- 172.21.15.52:0/2012651101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562c9443b8b0 0x562c9449f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.309+0000 7fb38affd700 0 client.25811 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:20:39.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.309+0000 7fb3bde3cb80 1 --2- 172.21.15.52:0/2012651101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c94437470 0x562c94440070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.309+0000 7fb3bde3cb80 1 -- 172.21.15.52:0/2012651101 >> 172.21.15.52:0/2012651101 conn(0x562c9442c360 msgr2=0x562c94439f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:39.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.309+0000 7fb3bde3cb80 1 -- 172.21.15.52:0/2012651101 shutdown_connections 2023-11-27T15:20:39.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.309+0000 7fb3bde3cb80 1 -- 172.21.15.52:0/2012651101 wait complete. 2023-11-27T15:20:39.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.309+0000 7fb3bde3cb80 20 client.25811 trim_cache size 0 max 16384 2023-11-27T15:20:39.316 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:20:39.316 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2023-11-27T15:20:39.316 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2023-11-27T15:20:39.317 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:20:39.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.309+0000 7fb3b1cb0700 1 -- 172.21.15.52:0/2656865839 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb3940a1340 msgr2=0x7fb3940a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.309+0000 7fb3b1cb0700 1 --2- 172.21.15.52:0/2656865839 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb3940a1340 0x7fb3940a37f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb39c006130 tx=0x7fb39c006080 comp rx=0 tx=0).stop 2023-11-27T15:20:39.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.309+0000 7fb3b1cb0700 1 -- 172.21.15.52:0/2656865839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3ac104560 msgr2=0x7fb3ac195c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.309+0000 7fb3b1cb0700 1 --2- 172.21.15.52:0/2656865839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3ac104560 0x7fb3ac195c00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400e900 tx=0x7fb3a400ec10 comp rx=0 tx=0).stop 2023-11-27T15:20:39.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.309+0000 7fb3b1cb0700 1 -- 172.21.15.52:0/2656865839 shutdown_connections 2023-11-27T15:20:39.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.309+0000 7fb3b1cb0700 1 --2- 172.21.15.52:0/2656865839 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb3940a1340 0x7fb3940a37f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.309+0000 7fb3b1cb0700 1 --2- 172.21.15.52:0/2656865839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3ac104560 0x7fb3ac195c00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.309+0000 7fb3b1cb0700 1 --2- 172.21.15.52:0/2656865839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb3ac0ff520 0x7fb3ac195180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.309+0000 7fb3b1cb0700 1 --2- 172.21.15.52:0/2656865839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3ac0ffec0 0x7fb3ac1956c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.309+0000 7fb3b1cb0700 1 -- 172.21.15.52:0/2656865839 >> 172.21.15.52:0/2656865839 conn(0x7fb3ac0760d0 msgr2=0x7fb3ac102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:39.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.310+0000 7fb3b1cb0700 1 -- 172.21.15.52:0/2656865839 shutdown_connections 2023-11-27T15:20:39.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.310+0000 7fb3b1cb0700 1 -- 172.21.15.52:0/2656865839 wait complete. 2023-11-27T15:20:39.329 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-27T15:20:39.329 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:20:39.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.465+0000 7f0142a7d700 1 -- 172.21.15.52:0/2987339579 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f013c105550 msgr2=0x7f013c10d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.465+0000 7f0142a7d700 1 --2- 172.21.15.52:0/2987339579 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f013c105550 0x7f013c10d120 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f013400b0d0 tx=0x7f013400b3e0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.465+0000 7f0142a7d700 1 -- 172.21.15.52:0/2987339579 shutdown_connections 2023-11-27T15:20:39.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.465+0000 7f0142a7d700 1 --2- 172.21.15.52:0/2987339579 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f013c100510 0x7f013c1008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.465+0000 7f0142a7d700 1 --2- 172.21.15.52:0/2987339579 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f013c105550 0x7f013c10d120 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.465+0000 7f0142a7d700 1 --2- 172.21.15.52:0/2987339579 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f013c100eb0 0x7f013c104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.465+0000 7f0142a7d700 1 -- 172.21.15.52:0/2987339579 >> 172.21.15.52:0/2987339579 conn(0x7f013c077050 msgr2=0x7f013c077450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:39.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.465+0000 7f0142a7d700 1 -- 172.21.15.52:0/2987339579 shutdown_connections 2023-11-27T15:20:39.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.465+0000 7f0142a7d700 1 -- 172.21.15.52:0/2987339579 wait complete. 2023-11-27T15:20:39.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.465+0000 7f0142a7d700 1 Processor -- start 2023-11-27T15:20:39.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.465+0000 7f0142a7d700 1 -- start start 2023-11-27T15:20:39.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.466+0000 7f0142a7d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f013c100510 0x7f013c1142c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.466+0000 7f0142a7d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f013c100eb0 0x7f013c114820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.466+0000 7f0142a7d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f013c105550 0x7f013c11a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.466+0000 7f0142a7d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013c10f420 con 0x7f013c100eb0 2023-11-27T15:20:39.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.466+0000 7f0142a7d700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013c10f590 con 0x7f013c100510 2023-11-27T15:20:39.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.466+0000 7f0142a7d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013c10f750 con 0x7f013c105550 2023-11-27T15:20:39.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.466+0000 7f013bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f013c100eb0 0x7f013c114820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.466+0000 7f0140819700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f013c100510 0x7f013c1142c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.466+0000 7f013bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f013c100eb0 0x7f013c114820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53446/0 (socket says 172.21.15.52:53446) 2023-11-27T15:20:39.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.466+0000 7f013bfff700 1 -- 172.21.15.52:0/3784274395 learned_addr learned my addr 172.21.15.52:0/3784274395 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:39.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.466+0000 7f014101a700 1 --2- 172.21.15.52:0/3784274395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f013c105550 0x7f013c11a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.466+0000 7f014101a700 1 -- 172.21.15.52:0/3784274395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f013c100510 msgr2=0x7f013c1142c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.466+0000 7f014101a700 1 --2- 172.21.15.52:0/3784274395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f013c100510 0x7f013c1142c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.466+0000 7f014101a700 1 -- 172.21.15.52:0/3784274395 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f013c100eb0 msgr2=0x7f013c114820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.466+0000 7f014101a700 1 --2- 172.21.15.52:0/3784274395 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f013c100eb0 0x7f013c114820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.466+0000 7f014101a700 1 -- 172.21.15.52:0/3784274395 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0134009d00 con 0x7f013c105550 2023-11-27T15:20:39.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.466+0000 7f0140819700 1 --2- 172.21.15.52:0/3784274395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f013c100510 0x7f013c1142c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:39.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.466+0000 7f013bfff700 1 --2- 172.21.15.52:0/3784274395 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f013c100eb0 0x7f013c114820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:39.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.466+0000 7f014101a700 1 --2- 172.21.15.52:0/3784274395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f013c105550 0x7f013c11a060 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0134009fd0 tx=0x7f01340077f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:39.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.467+0000 7f0139ffb700 1 -- 172.21.15.52:0/3784274395 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013400e040 con 0x7f013c105550 2023-11-27T15:20:39.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.467+0000 7f0142a7d700 1 -- 172.21.15.52:0/3784274395 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f013c10e190 con 0x7f013c105550 2023-11-27T15:20:39.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.467+0000 7f0139ffb700 1 -- 172.21.15.52:0/3784274395 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f013400bcb0 con 0x7f013c105550 2023-11-27T15:20:39.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.467+0000 7f0139ffb700 1 -- 172.21.15.52:0/3784274395 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134005840 con 0x7f013c105550 2023-11-27T15:20:39.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.467+0000 7f0142a7d700 1 -- 172.21.15.52:0/3784274395 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f013c10e000 con 0x7f013c105550 2023-11-27T15:20:39.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.468+0000 7f0142a7d700 1 -- 172.21.15.52:0/3784274395 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f013c10e320 con 0x7f013c105550 2023-11-27T15:20:39.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.469+0000 7f0139ffb700 1 -- 172.21.15.52:0/3784274395 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0134004a50 con 0x7f013c105550 2023-11-27T15:20:39.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.470+0000 7f0139ffb700 1 --2- 172.21.15.52:0/3784274395 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f012c0a12c0 0x7f012c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.470+0000 7f0139ffb700 1 -- 172.21.15.52:0/3784274395 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f01340cdad0 con 0x7f013c105550 2023-11-27T15:20:39.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.470+0000 7f0140819700 1 --2- 172.21.15.52:0/3784274395 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f012c0a12c0 0x7f012c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.470+0000 7f0140819700 1 --2- 172.21.15.52:0/3784274395 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f012c0a12c0 0x7f012c0a3770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0130011490 tx=0x7f0130009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:39.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.473+0000 7f0139ffb700 1 -- 172.21.15.52:0/3784274395 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0134096f10 con 0x7f013c105550 2023-11-27T15:20:39.762 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.757+0000 7f0142a7d700 1 -- 172.21.15.52:0/3784274395 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f013c110930 con 0x7f013c105550 2023-11-27T15:20:39.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.758+0000 7f0139ffb700 1 -- 172.21.15.52:0/3784274395 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10807 (secure 0 0 0) 0x7f01340ba030 con 0x7f013c105550 2023-11-27T15:20:39.763 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:20:39.764 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2328881376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2328881376},{"type":"v1","addr":"172.21.15.52:6839","nonce":2328881376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2799551570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2799551570},{"type":"v1","addr":"172.21.15.52:6841","nonce":2799551570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:16:09.319630+0000","modified":"2023-11-27T15:20:15.109211+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24494,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/215725838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":215725838},{"type":"v1","addr":"172.21.15.150:6839","nonce":215725838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3571830559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3571830559},{"type":"v1","addr":"172.21.15.102:6839","nonce":3571830559}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:20:39.766 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.761+0000 7f0142a7d700 1 -- 172.21.15.52:0/3784274395 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f012c0a12c0 msgr2=0x7f012c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.766 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.761+0000 7f0142a7d700 1 --2- 172.21.15.52:0/3784274395 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f012c0a12c0 0x7f012c0a3770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0130011490 tx=0x7f0130009260 comp rx=0 tx=0).stop 2023-11-27T15:20:39.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.761+0000 7f0142a7d700 1 -- 172.21.15.52:0/3784274395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f013c105550 msgr2=0x7f013c11a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.761+0000 7f0142a7d700 1 --2- 172.21.15.52:0/3784274395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f013c105550 0x7f013c11a060 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0134009fd0 tx=0x7f01340077f0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.761+0000 7f0142a7d700 1 -- 172.21.15.52:0/3784274395 shutdown_connections 2023-11-27T15:20:39.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.761+0000 7f0142a7d700 1 --2- 172.21.15.52:0/3784274395 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f012c0a12c0 0x7f012c0a3770 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.761+0000 7f0142a7d700 1 --2- 172.21.15.52:0/3784274395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f013c100510 0x7f013c1142c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.761+0000 7f0142a7d700 1 --2- 172.21.15.52:0/3784274395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f013c105550 0x7f013c11a060 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.761+0000 7f0142a7d700 1 --2- 172.21.15.52:0/3784274395 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f013c100eb0 0x7f013c114820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.761+0000 7f0142a7d700 1 -- 172.21.15.52:0/3784274395 >> 172.21.15.52:0/3784274395 conn(0x7f013c077050 msgr2=0x7f013c0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:39.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.761+0000 7f0142a7d700 1 -- 172.21.15.52:0/3784274395 shutdown_connections 2023-11-27T15:20:39.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.761+0000 7f0142a7d700 1 -- 172.21.15.52:0/3784274395 wait complete. 2023-11-27T15:20:39.768 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 24 2023-11-27T15:20:39.782 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-27T15:20:39.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.932+0000 7f2d08000700 1 -- 172.21.15.52:0/1867503512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d00104560 msgr2=0x7f2d0010c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.932+0000 7f2d08000700 1 --2- 172.21.15.52:0/1867503512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d00104560 0x7f2d0010c130 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8009f90 tx=0x7f2cf800b330 comp rx=0 tx=0).stop 2023-11-27T15:20:39.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.932+0000 7f2d08000700 1 -- 172.21.15.52:0/1867503512 shutdown_connections 2023-11-27T15:20:39.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.932+0000 7f2d08000700 1 --2- 172.21.15.52:0/1867503512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d00104560 0x7f2d0010c130 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.932+0000 7f2d08000700 1 --2- 172.21.15.52:0/1867503512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d000ff520 0x7f2d000ff8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.932+0000 7f2d08000700 1 --2- 172.21.15.52:0/1867503512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d000ffec0 0x7f2d00103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.932+0000 7f2d08000700 1 -- 172.21.15.52:0/1867503512 >> 172.21.15.52:0/1867503512 conn(0x7f2d000760d0 msgr2=0x7f2d000764d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:39.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.932+0000 7f2d08000700 1 -- 172.21.15.52:0/1867503512 shutdown_connections 2023-11-27T15:20:39.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.932+0000 7f2d08000700 1 -- 172.21.15.52:0/1867503512 wait complete. 2023-11-27T15:20:39.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.933+0000 7f2d08000700 1 Processor -- start 2023-11-27T15:20:39.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.933+0000 7f2d08000700 1 -- start start 2023-11-27T15:20:39.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.933+0000 7f2d08000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d000ff520 0x7f2d001951a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.933+0000 7f2d08000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d000ffec0 0x7f2d001956e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.933+0000 7f2d08000700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d00104560 0x7f2d00195c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.933+0000 7f2d08000700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d001a49e0 con 0x7f2d00104560 2023-11-27T15:20:39.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.933+0000 7f2d08000700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d001a4b50 con 0x7f2d000ff520 2023-11-27T15:20:39.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.933+0000 7f2d08000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d001a4d10 con 0x7f2d000ffec0 2023-11-27T15:20:39.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.933+0000 7f2d05d9c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d000ff520 0x7f2d001951a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.933+0000 7f2d0659d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d00104560 0x7f2d00195c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.933+0000 7f2d05d9c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d000ff520 0x7f2d001951a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43116/0 (socket says 172.21.15.52:43116) 2023-11-27T15:20:39.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.934+0000 7f2d05d9c700 1 -- 172.21.15.52:0/3378974379 learned_addr learned my addr 172.21.15.52:0/3378974379 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:39.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.934+0000 7f2d0559b700 1 --2- 172.21.15.52:0/3378974379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d000ffec0 0x7f2d001956e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.934+0000 7f2d05d9c700 1 -- 172.21.15.52:0/3378974379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d000ffec0 msgr2=0x7f2d001956e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.934+0000 7f2d05d9c700 1 --2- 172.21.15.52:0/3378974379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d000ffec0 0x7f2d001956e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.934+0000 7f2d05d9c700 1 -- 172.21.15.52:0/3378974379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d00104560 msgr2=0x7f2d00195c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.942 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.934+0000 7f2d05d9c700 1 --2- 172.21.15.52:0/3378974379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d00104560 0x7f2d00195c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.942 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.934+0000 7f2d05d9c700 1 -- 172.21.15.52:0/3378974379 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cfc0095f0 con 0x7f2d000ff520 2023-11-27T15:20:39.942 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.934+0000 7f2d05d9c700 1 --2- 172.21.15.52:0/3378974379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d000ff520 0x7f2d001951a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00ec10 tx=0x7f2cfc00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:39.942 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.934+0000 7f2cf6ffd700 1 -- 172.21.15.52:0/3378974379 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc00cc90 con 0x7f2d000ff520 2023-11-27T15:20:39.942 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.934+0000 7f2cf6ffd700 1 -- 172.21.15.52:0/3378974379 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cfc0044e0 con 0x7f2d000ff520 2023-11-27T15:20:39.942 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.934+0000 7f2d08000700 1 -- 172.21.15.52:0/3378974379 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf8009c40 con 0x7f2d000ff520 2023-11-27T15:20:39.942 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.934+0000 7f2cf6ffd700 1 -- 172.21.15.52:0/3378974379 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc010430 con 0x7f2d000ff520 2023-11-27T15:20:39.942 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.934+0000 7f2d08000700 1 -- 172.21.15.52:0/3378974379 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0010f390 con 0x7f2d000ff520 2023-11-27T15:20:39.943 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.937+0000 7f2cf6ffd700 1 -- 172.21.15.52:0/3378974379 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f2cfc003fb0 con 0x7f2d000ff520 2023-11-27T15:20:39.943 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.938+0000 7f2cf6ffd700 1 --2- 172.21.15.52:0/3378974379 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f2cec0a1340 0x7f2cec0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.945 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.938+0000 7f2d1418cb80 1 Processor -- start 2023-11-27T15:20:39.945 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.938+0000 7f2d0559b700 1 --2- 172.21.15.52:0/3378974379 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f2cec0a1340 0x7f2cec0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.945 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.938+0000 7f2cf6ffd700 1 -- 172.21.15.52:0/3378974379 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2cfc014070 con 0x7f2d000ff520 2023-11-27T15:20:39.945 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.939+0000 7f2d1418cb80 1 -- start start 2023-11-27T15:20:39.945 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.939+0000 7f2d1418cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5981db370 0x55a5981db740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.945 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.939+0000 7f2d1418cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5981dbc80 0x55a5981dfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.946 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.939+0000 7f2d1418cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5981e00c0 0x55a5981e0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.946 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.939+0000 7f2d1418cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5980d8060 con 0x55a5981db370 2023-11-27T15:20:39.946 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.939+0000 7f2d1418cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a59812a8a0 con 0x55a5981e00c0 2023-11-27T15:20:39.946 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.939+0000 7f2d1418cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a598123d30 con 0x55a5981dbc80 2023-11-27T15:20:39.946 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.939+0000 7f2d05d9c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5981db370 0x55a5981db740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.946 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.939+0000 7f2d05d9c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5981db370 0x55a5981db740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53466/0 (socket says 172.21.15.52:53466) 2023-11-27T15:20:39.946 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.939+0000 7f2d05d9c700 1 -- 172.21.15.52:0/1031721215 learned_addr learned my addr 172.21.15.52:0/1031721215 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:39.946 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.939+0000 7f2d0659d700 1 --2- 172.21.15.52:0/1031721215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5981e00c0 0x55a5981e0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.946 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.939+0000 7f2d0559b700 1 --2- 172.21.15.52:0/3378974379 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f2cec0a1340 0x7f2cec0a37f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2d00069770 tx=0x7f2cf0009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:39.946 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.939+0000 7f2d0559b700 1 --2- 172.21.15.52:0/1031721215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5981dbc80 0x55a5981dfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.939+0000 7f2d05d9c700 1 -- 172.21.15.52:0/1031721215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5981dbc80 msgr2=0x55a5981dfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:39.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.939+0000 7f2d05d9c700 1 --2- 172.21.15.52:0/1031721215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5981dbc80 0x55a5981dfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.939+0000 7f2d05d9c700 1 -- 172.21.15.52:0/1031721215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5981e00c0 msgr2=0x55a5981e0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:39.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.939+0000 7f2d05d9c700 1 --2- 172.21.15.52:0/1031721215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5981e00c0 0x55a5981e0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.939+0000 7f2d05d9c700 1 -- 172.21.15.52:0/1031721215 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a597fa5630 con 0x55a5981db370 2023-11-27T15:20:39.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.939+0000 7f2d0559b700 1 --2- 172.21.15.52:0/1031721215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5981dbc80 0x55a5981dfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:20:39.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.940+0000 7f2d05d9c700 1 --2- 172.21.15.52:0/1031721215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5981db370 0x55a5981db740 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc096cf0 tx=0x7f2cfc099dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5247330937b3f83e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:39.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.940+0000 7f2cebfff700 1 -- 172.21.15.52:0/1031721215 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc09e2b0 con 0x55a5981db370 2023-11-27T15:20:39.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.940+0000 7f2cebfff700 1 -- 172.21.15.52:0/1031721215 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cfc0a26d0 con 0x55a5981db370 2023-11-27T15:20:39.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.940+0000 7f2cebfff700 1 -- 172.21.15.52:0/1031721215 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc09a4e0 con 0x55a5981db370 2023-11-27T15:20:39.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.941+0000 7f2d1418cb80 1 -- 172.21.15.52:0/1031721215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5981db370 msgr2=0x55a5981db740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.941+0000 7f2d1418cb80 1 --2- 172.21.15.52:0/1031721215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5981db370 0x55a5981db740 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc096cf0 tx=0x7f2cfc099dc0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.941+0000 7f2d1418cb80 1 -- 172.21.15.52:0/1031721215 shutdown_connections 2023-11-27T15:20:39.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.941+0000 7f2d1418cb80 1 --2- 172.21.15.52:0/1031721215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5981e00c0 0x55a5981e0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.941+0000 7f2d1418cb80 1 --2- 172.21.15.52:0/1031721215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5981db370 0x55a5981db740 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.941+0000 7f2d1418cb80 1 --2- 172.21.15.52:0/1031721215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5981dbc80 0x55a5981dfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.941+0000 7f2d1418cb80 1 -- 172.21.15.52:0/1031721215 >> 172.21.15.52:0/1031721215 conn(0x55a5981c8360 msgr2=0x55a5981cdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:39.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.941+0000 7f2d1418cb80 1 -- 172.21.15.52:0/1031721215 shutdown_connections 2023-11-27T15:20:39.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.941+0000 7f2d1418cb80 1 -- 172.21.15.52:0/1031721215 wait complete. 2023-11-27T15:20:39.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.941+0000 7f2d1418cb80 1 -- start start 2023-11-27T15:20:39.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.941+0000 7f2d1418cb80 1 Processor -- start 2023-11-27T15:20:39.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.942+0000 7f2d1418cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5981db370 0x55a5981e7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.942+0000 7f2d1418cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5981dbc80 0x55a5981e4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.942+0000 7f2d1418cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5981e00c0 0x55a5981e5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.942+0000 7f2d1418cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5980c58e0 con 0x55a5981e00c0 2023-11-27T15:20:39.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.942+0000 7f2d1418cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5980c5580 con 0x55a5981db370 2023-11-27T15:20:39.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.942+0000 7f2d1418cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a597fa7920 con 0x55a5981dbc80 2023-11-27T15:20:39.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.942+0000 7f2d1418cb80 10 client.0 _ll_register_callbacks cb 0x55a5980e3780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:20:39.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.942+0000 7f2d0659d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5981e00c0 0x55a5981e5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.942+0000 7f2d05d9c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5981db370 0x55a5981e7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.942+0000 7f2d0659d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5981e00c0 0x55a5981e5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53476/0 (socket says 172.21.15.52:53476) 2023-11-27T15:20:39.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.942+0000 7f2d0659d700 1 -- 172.21.15.52:0/2225513250 learned_addr learned my addr 172.21.15.52:0/2225513250 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:39.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.942+0000 7f2d0559b700 1 --2- 172.21.15.52:0/2225513250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5981dbc80 0x55a5981e4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.942+0000 7f2d05d9c700 1 -- 172.21.15.52:0/2225513250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5981dbc80 msgr2=0x55a5981e4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:39.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.942+0000 7f2d05d9c700 1 --2- 172.21.15.52:0/2225513250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5981dbc80 0x55a5981e4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.942+0000 7f2d05d9c700 1 -- 172.21.15.52:0/2225513250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5981e00c0 msgr2=0x55a5981e5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:39.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.942+0000 7f2d05d9c700 1 --2- 172.21.15.52:0/2225513250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5981e00c0 0x55a5981e5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.942+0000 7f2d05d9c700 1 -- 172.21.15.52:0/2225513250 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0010f390 con 0x55a5981db370 2023-11-27T15:20:39.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.942+0000 7f2d05d9c700 1 --2- 172.21.15.52:0/2225513250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5981db370 0x55a5981e7880 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00cf30 tx=0x7f2cfc003910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef60a717a7751af8 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:39.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.942+0000 7f2ce97fa700 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2023-11-27T15:20:39.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.943+0000 7f2ce97fa700 1 -- 172.21.15.52:0/2225513250 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc009a10 con 0x55a5981db370 2023-11-27T15:20:39.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.943+0000 7f2ce97fa700 1 -- 172.21.15.52:0/2225513250 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2cfc09e2b0 con 0x55a5981db370 2023-11-27T15:20:39.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.943+0000 7f2d1418cb80 1 -- 172.21.15.52:0/2225513250 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a5981c5b20 con 0x55a5981db370 2023-11-27T15:20:39.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.943+0000 7f2ce97fa700 1 -- 172.21.15.52:0/2225513250 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cfc099970 con 0x55a5981db370 2023-11-27T15:20:39.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.943+0000 7f2d1418cb80 10 client.25829 fetch_fsmap learned FSMap version 24 2023-11-27T15:20:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.943+0000 7f2d1418cb80 1 -- 172.21.15.52:0/2225513250 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a5980ea190 con 0x55a5981db370 2023-11-27T15:20:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.943+0000 7f2ce97fa700 1 -- 172.21.15.52:0/2225513250 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a5981c5b20 con 0x55a5981db370 2023-11-27T15:20:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.943+0000 7f2ce97fa700 1 -- 172.21.15.52:0/2225513250 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2cfc0c4ab0 con 0x55a5981db370 2023-11-27T15:20:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.943+0000 7f2d1418cb80 10 client.25829 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:20:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.943+0000 7f2d1418cb80 10 client.25829 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:20:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.943+0000 7f2d1418cb80 20 client.25829 populate_metadata read hostname 'smithi052' 2023-11-27T15:20:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.943+0000 7f2d1418cb80 1 --2- 172.21.15.52:0/2225513250 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55a5981fd640 0x55a5981ffa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.943+0000 7f2d1418cb80 4 client.25829 mds_command: new command op to 24472 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:20:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.943+0000 7f2d1418cb80 1 -- 172.21.15.52:0/2225513250 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a5980e60a0 con 0x55a5981fd640 2023-11-27T15:20:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.943+0000 7f2d0559b700 1 --2- 172.21.15.52:0/2225513250 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55a5981fd640 0x55a5981ffa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.944+0000 7f2ce97fa700 10 client.25829 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:20:39.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.944+0000 7f2d0559b700 1 --2- 172.21.15.52:0/2225513250 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55a5981fd640 0x55a5981ffa20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=813275c2aadae1ca server_cookie=5731c3f4abaa63dc in_seq=0 out_seq=0 2023-11-27T15:20:39.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.944+0000 7f2ce97fa700 1 -- 172.21.15.52:0/2225513250 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a5980e60a0 con 0x55a5981fd640 2023-11-27T15:20:39.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.944+0000 7f2ce97fa700 10 client.25829 handle_command_reply: tid=0 2023-11-27T15:20:39.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.944+0000 7f2d1418cb80 1 client.25829 shutdown 2023-11-27T15:20:39.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.945+0000 7f2d1418cb80 1 -- 172.21.15.52:0/2225513250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5981db370 msgr2=0x55a5981e7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.945+0000 7f2d1418cb80 1 --2- 172.21.15.52:0/2225513250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5981db370 0x55a5981e7880 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00cf30 tx=0x7f2cfc003910 comp rx=0 tx=0).stop 2023-11-27T15:20:39.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.945+0000 7f2d1418cb80 1 -- 172.21.15.52:0/2225513250 shutdown_connections 2023-11-27T15:20:39.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.945+0000 7f2d1418cb80 1 --2- 172.21.15.52:0/2225513250 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55a5981fd640 0x55a5981ffa20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.945+0000 7f2d1418cb80 1 --2- 172.21.15.52:0/2225513250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5981db370 0x55a5981e7880 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.945+0000 7f2d1418cb80 1 --2- 172.21.15.52:0/2225513250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5981e00c0 0x55a5981e5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.945+0000 7f2d1418cb80 1 --2- 172.21.15.52:0/2225513250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5981dbc80 0x55a5981e4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.945+0000 7f2d1418cb80 1 -- 172.21.15.52:0/2225513250 >> 172.21.15.52:0/2225513250 conn(0x55a5981c8360 msgr2=0x55a5981e1570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:39.957 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.945+0000 7f2ce97fa700 0 client.25829 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:20:39.957 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.945+0000 7f2d1418cb80 1 -- 172.21.15.52:0/2225513250 shutdown_connections 2023-11-27T15:20:39.957 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.945+0000 7f2d1418cb80 1 -- 172.21.15.52:0/2225513250 wait complete. 2023-11-27T15:20:39.957 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.945+0000 7f2d1418cb80 20 client.25829 trim_cache size 0 max 16384 2023-11-27T15:20:39.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.961+0000 7f2d1418cb80 1 Processor -- start 2023-11-27T15:20:39.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.961+0000 7f2d1418cb80 1 -- start start 2023-11-27T15:20:39.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.961+0000 7f2d1418cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5981db370 0x55a5981d6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.961+0000 7f2d1418cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5981dbc80 0x55a5981d1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.961+0000 7f2d1418cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5981e00c0 0x55a5981e7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.961+0000 7f2d1418cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5980e60a0 con 0x55a5981e00c0 2023-11-27T15:20:39.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.961+0000 7f2d1418cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5981c81e0 con 0x55a5981dbc80 2023-11-27T15:20:39.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.961+0000 7f2d1418cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5981cf430 con 0x55a5981db370 2023-11-27T15:20:39.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.961+0000 7f2d0559b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5981dbc80 0x55a5981d1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.961+0000 7f2d0559b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5981dbc80 0x55a5981d1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43140/0 (socket says 172.21.15.52:43140) 2023-11-27T15:20:39.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.961+0000 7f2d0559b700 1 -- 172.21.15.52:0/8390200 learned_addr learned my addr 172.21.15.52:0/8390200 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:39.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.961+0000 7f2d05d9c700 1 --2- 172.21.15.52:0/8390200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5981db370 0x55a5981d6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.961+0000 7f2d0659d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5981e00c0 0x55a5981e7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.962+0000 7f2d0559b700 1 -- 172.21.15.52:0/8390200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5981db370 msgr2=0x55a5981d6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:39.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.962+0000 7f2d0559b700 1 --2- 172.21.15.52:0/8390200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5981db370 0x55a5981d6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.962+0000 7f2d0559b700 1 -- 172.21.15.52:0/8390200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5981e00c0 msgr2=0x55a5981e7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:39.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.962+0000 7f2d0559b700 1 --2- 172.21.15.52:0/8390200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5981e00c0 0x55a5981e7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.962+0000 7f2d0559b700 1 -- 172.21.15.52:0/8390200 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5980e9ee0 con 0x55a5981dbc80 2023-11-27T15:20:39.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.962+0000 7f2d05d9c700 1 --2- 172.21.15.52:0/8390200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5981db370 0x55a5981d6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:39.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.962+0000 7f2d0559b700 1 --2- 172.21.15.52:0/8390200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5981dbc80 0x55a5981d1960 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2d00069800 tx=0x7f2cf0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=945eed3b76d8696f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:39.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.962+0000 7f2cf4ff9700 1 -- 172.21.15.52:0/8390200 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0024070 con 0x55a5981dbc80 2023-11-27T15:20:39.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.962+0000 7f2cf4ff9700 1 -- 172.21.15.52:0/8390200 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cf0025040 con 0x55a5981dbc80 2023-11-27T15:20:39.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.962+0000 7f2cf4ff9700 1 -- 172.21.15.52:0/8390200 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0028780 con 0x55a5981dbc80 2023-11-27T15:20:39.975 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.963+0000 7f2d1418cb80 1 -- 172.21.15.52:0/8390200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5981dbc80 msgr2=0x55a5981d1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.975 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.963+0000 7f2d1418cb80 1 --2- 172.21.15.52:0/8390200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5981dbc80 0x55a5981d1960 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2d00069800 tx=0x7f2cf0014000 comp rx=0 tx=0).stop 2023-11-27T15:20:39.975 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.963+0000 7f2d1418cb80 1 -- 172.21.15.52:0/8390200 shutdown_connections 2023-11-27T15:20:39.975 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.963+0000 7f2d1418cb80 1 --2- 172.21.15.52:0/8390200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5981e00c0 0x55a5981e7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.975 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.963+0000 7f2d1418cb80 1 --2- 172.21.15.52:0/8390200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5981db370 0x55a5981d6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.975 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.963+0000 7f2d1418cb80 1 --2- 172.21.15.52:0/8390200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5981dbc80 0x55a5981d1960 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.975 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.963+0000 7f2d1418cb80 1 -- 172.21.15.52:0/8390200 >> 172.21.15.52:0/8390200 conn(0x55a5981c8360 msgr2=0x55a5981e0490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:39.975 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.963+0000 7f2d1418cb80 1 -- 172.21.15.52:0/8390200 shutdown_connections 2023-11-27T15:20:39.975 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.963+0000 7f2d1418cb80 1 -- 172.21.15.52:0/8390200 wait complete. 2023-11-27T15:20:39.976 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.963+0000 7f2d1418cb80 1 -- start start 2023-11-27T15:20:39.976 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.964+0000 7f2d1418cb80 1 Processor -- start 2023-11-27T15:20:39.976 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.964+0000 7f2d1418cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5981db370 0x55a5981d6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.976 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.964+0000 7f2d1418cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5981dbc80 0x55a5981e6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.976 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.964+0000 7f2d1418cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5981e00c0 0x55a5981e74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.976 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.964+0000 7f2d1418cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5981e7b70 con 0x55a5981db370 2023-11-27T15:20:39.976 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.964+0000 7f2d1418cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5981e7ce0 con 0x55a5981e00c0 2023-11-27T15:20:39.976 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.964+0000 7f2d1418cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5981cc550 con 0x55a5981dbc80 2023-11-27T15:20:39.976 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.964+0000 7f2d1418cb80 10 client.0 _ll_register_callbacks cb 0x7f2cfc0091e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:20:39.976 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.964+0000 7f2d05d9c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5981db370 0x55a5981d6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.976 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.964+0000 7f2d0659d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5981e00c0 0x55a5981e74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.977 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.964+0000 7f2d0659d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5981e00c0 0x55a5981e74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43148/0 (socket says 172.21.15.52:43148) 2023-11-27T15:20:39.977 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.964+0000 7f2d0659d700 1 -- 172.21.15.52:0/2091290161 learned_addr learned my addr 172.21.15.52:0/2091290161 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:39.977 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.964+0000 7f2d05d9c700 1 --2- 172.21.15.52:0/2091290161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5981db370 0x55a5981d6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53486/0 (socket says 172.21.15.52:53486) 2023-11-27T15:20:39.977 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.964+0000 7f2d0559b700 1 --2- 172.21.15.52:0/2091290161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5981dbc80 0x55a5981e6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.977 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.964+0000 7f2d05d9c700 1 -- 172.21.15.52:0/2091290161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5981dbc80 msgr2=0x55a5981e6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:39.977 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.964+0000 7f2d05d9c700 1 --2- 172.21.15.52:0/2091290161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5981dbc80 0x55a5981e6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.977 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.964+0000 7f2d05d9c700 1 -- 172.21.15.52:0/2091290161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5981e00c0 msgr2=0x55a5981e74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:39.977 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.964+0000 7f2d05d9c700 1 --2- 172.21.15.52:0/2091290161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5981e00c0 0x55a5981e74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.977 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.964+0000 7f2d05d9c700 1 -- 172.21.15.52:0/2091290161 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5980ea190 con 0x55a5981db370 2023-11-27T15:20:39.977 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.965+0000 7f2d0559b700 1 --2- 172.21.15.52:0/2091290161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5981dbc80 0x55a5981e6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:20:39.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.965+0000 7f2d0659d700 1 --2- 172.21.15.52:0/2091290161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5981e00c0 0x55a5981e74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:39.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.965+0000 7f2d05d9c700 1 --2- 172.21.15.52:0/2091290161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5981db370 0x55a5981d6d00 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc099f80 tx=0x7f2cfc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261cc9bebf8f76f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:39.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.965+0000 7f2ce97fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:20:39.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.965+0000 7f2ce97fa700 1 -- 172.21.15.52:0/2091290161 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc009a10 con 0x55a5981db370 2023-11-27T15:20:39.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.965+0000 7f2ce97fa700 1 -- 172.21.15.52:0/2091290161 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2cfc09e2b0 con 0x55a5981db370 2023-11-27T15:20:39.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.965+0000 7f2d1418cb80 1 -- 172.21.15.52:0/2091290161 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2cfc0a26d0 con 0x55a5981db370 2023-11-27T15:20:39.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.965+0000 7f2ce97fa700 1 -- 172.21.15.52:0/2091290161 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cfc099dc0 con 0x55a5981db370 2023-11-27T15:20:39.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.965+0000 7f2d1418cb80 10 client.16020 fetch_fsmap learned FSMap version 24 2023-11-27T15:20:39.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.965+0000 7f2d1418cb80 1 -- 172.21.15.52:0/2091290161 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f2cf0017070 con 0x55a5981db370 2023-11-27T15:20:39.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.965+0000 7f2ce97fa700 1 -- 172.21.15.52:0/2091290161 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cfc082080 con 0x55a5981db370 2023-11-27T15:20:39.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.966+0000 7f2ce97fa700 1 -- 172.21.15.52:0/2091290161 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2cfc083a80 con 0x55a5981db370 2023-11-27T15:20:39.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.966+0000 7f2d1418cb80 10 client.16020 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:20:39.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.966+0000 7f2d1418cb80 10 client.16020 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:20:39.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.966+0000 7f2d1418cb80 20 client.16020 populate_metadata read hostname 'smithi052' 2023-11-27T15:20:39.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.966+0000 7f2d1418cb80 1 --2- 172.21.15.52:0/2091290161 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55a598251730 0x55a598253b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.966+0000 7f2d1418cb80 4 client.16020 mds_command: new command op to 24472 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:20:39.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.966+0000 7f2d1418cb80 1 -- 172.21.15.52:0/2091290161 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a5980f2660 con 0x55a598251730 2023-11-27T15:20:39.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.966+0000 7f2d0559b700 1 --2- 172.21.15.52:0/2091290161 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55a598251730 0x55a598253b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.966+0000 7f2d0559b700 1 --2- 172.21.15.52:0/2091290161 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55a598251730 0x55a598253b10 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=34028867d4895eb6 server_cookie=d04c8236dcc32f0d in_seq=0 out_seq=0 2023-11-27T15:20:39.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.966+0000 7f2ce97fa700 10 client.16020 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:20:39.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.966+0000 7f2ce97fa700 1 -- 172.21.15.52:0/2091290161 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a5980f2660 con 0x55a598251730 2023-11-27T15:20:39.980 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.966+0000 7f2ce97fa700 10 client.16020 handle_command_reply: tid=0 2023-11-27T15:20:39.980 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.967+0000 7f2d1418cb80 1 client.16020 shutdown 2023-11-27T15:20:39.980 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.967+0000 7f2d1418cb80 1 -- 172.21.15.52:0/2091290161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5981db370 msgr2=0x55a5981d6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.980 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.967+0000 7f2d1418cb80 1 --2- 172.21.15.52:0/2091290161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5981db370 0x55a5981d6d00 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc099f80 tx=0x7f2cfc000bc0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.980 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.967+0000 7f2d1418cb80 1 -- 172.21.15.52:0/2091290161 shutdown_connections 2023-11-27T15:20:39.980 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.967+0000 7f2d1418cb80 1 --2- 172.21.15.52:0/2091290161 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55a598251730 0x55a598253b10 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.980 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.967+0000 7f2d1418cb80 1 --2- 172.21.15.52:0/2091290161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5981e00c0 0x55a5981e74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.980 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.967+0000 7f2d1418cb80 1 --2- 172.21.15.52:0/2091290161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5981db370 0x55a5981d6d00 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.980 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.967+0000 7f2d1418cb80 1 --2- 172.21.15.52:0/2091290161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a5981dbc80 0x55a5981e6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.980 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.967+0000 7f2d1418cb80 1 -- 172.21.15.52:0/2091290161 >> 172.21.15.52:0/2091290161 conn(0x55a5981c8360 msgr2=0x55a5981d9490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:39.980 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.967+0000 7f2ce97fa700 0 client.16020 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:20:39.981 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.967+0000 7f2d1418cb80 1 -- 172.21.15.52:0/2091290161 shutdown_connections 2023-11-27T15:20:39.981 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.967+0000 7f2d1418cb80 1 -- 172.21.15.52:0/2091290161 wait complete. 2023-11-27T15:20:39.981 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.967+0000 7f2d1418cb80 20 client.16020 trim_cache size 0 max 16384 2023-11-27T15:20:39.981 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:20:39.982 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.968+0000 7f2d08000700 1 -- 172.21.15.52:0/3378974379 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f2cec0a1340 msgr2=0x7f2cec0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.982 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.968+0000 7f2d08000700 1 --2- 172.21.15.52:0/3378974379 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f2cec0a1340 0x7f2cec0a37f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2d00069770 tx=0x7f2cf0009260 comp rx=0 tx=0).stop 2023-11-27T15:20:39.982 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.968+0000 7f2d08000700 1 -- 172.21.15.52:0/3378974379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d000ff520 msgr2=0x7f2d001951a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.982 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.968+0000 7f2d08000700 1 --2- 172.21.15.52:0/3378974379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d000ff520 0x7f2d001951a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00ec10 tx=0x7f2cfc00ef20 comp rx=0 tx=0).stop 2023-11-27T15:20:39.982 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.968+0000 7f2d08000700 1 -- 172.21.15.52:0/3378974379 shutdown_connections 2023-11-27T15:20:39.982 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.968+0000 7f2d08000700 1 --2- 172.21.15.52:0/3378974379 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f2cec0a1340 0x7f2cec0a37f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.982 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.968+0000 7f2d08000700 1 --2- 172.21.15.52:0/3378974379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d000ff520 0x7f2d001951a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.982 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.968+0000 7f2d08000700 1 --2- 172.21.15.52:0/3378974379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d00104560 0x7f2d00195c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.982 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.968+0000 7f2d08000700 1 --2- 172.21.15.52:0/3378974379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d000ffec0 0x7f2d001956e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.983 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.968+0000 7f2d08000700 1 -- 172.21.15.52:0/3378974379 >> 172.21.15.52:0/3378974379 conn(0x7f2d000760d0 msgr2=0x7f2d000fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:39.983 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.968+0000 7f2d08000700 1 -- 172.21.15.52:0/3378974379 shutdown_connections 2023-11-27T15:20:39.983 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:39.968+0000 7f2d08000700 1 -- 172.21.15.52:0/3378974379 wait complete. 2023-11-27T15:20:39.987 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-27T15:20:40.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:39 smithi052 ceph-mon[99056]: pgmap v85: 97 pgs: 97 active+clean; 90 MiB data, 264 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.2 MiB/s wr, 318 op/s 2023-11-27T15:20:40.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:39 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3784274395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:40.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.126+0000 7f4a40cff700 1 -- 172.21.15.52:0/4006422822 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a3c101400 msgr2=0x7f4a3c1017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.126+0000 7f4a40cff700 1 --2- 172.21.15.52:0/4006422822 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a3c101400 0x7f4a3c1017d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c009990 tx=0x7f4a2c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.126+0000 7f4a40cff700 1 -- 172.21.15.52:0/4006422822 shutdown_connections 2023-11-27T15:20:40.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.126+0000 7f4a40cff700 1 --2- 172.21.15.52:0/4006422822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a3c10a920 0x7f4a3c10cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.126+0000 7f4a40cff700 1 --2- 172.21.15.52:0/4006422822 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a3c101400 0x7f4a3c1017d0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.126+0000 7f4a40cff700 1 --2- 172.21.15.52:0/4006422822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a3c101d10 0x7f4a3c10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.126+0000 7f4a40cff700 1 -- 172.21.15.52:0/4006422822 >> 172.21.15.52:0/4006422822 conn(0x7f4a3c076200 msgr2=0x7f4a3c076600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:40.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.126+0000 7f4a40cff700 1 -- 172.21.15.52:0/4006422822 shutdown_connections 2023-11-27T15:20:40.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.126+0000 7f4a40cff700 1 -- 172.21.15.52:0/4006422822 wait complete. 2023-11-27T15:20:40.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.127+0000 7f4a40cff700 1 Processor -- start 2023-11-27T15:20:40.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.127+0000 7f4a40cff700 1 -- start start 2023-11-27T15:20:40.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.127+0000 7f4a40cff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a3c101400 0x7f4a3c196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.127+0000 7f4a40cff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a3c101d10 0x7f4a3c196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.127+0000 7f4a40cff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a3c10a920 0x7f4a3c19ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.127+0000 7f4a40cff700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c19b420 con 0x7f4a3c101400 2023-11-27T15:20:40.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.127+0000 7f4a40cff700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c19b590 con 0x7f4a3c101d10 2023-11-27T15:20:40.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.127+0000 7f4a40cff700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c19b750 con 0x7f4a3c10a920 2023-11-27T15:20:40.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.127+0000 7f4a3a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a3c101400 0x7f4a3c196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.127+0000 7f4a39d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a3c101d10 0x7f4a3c196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.127+0000 7f4a3a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a3c101400 0x7f4a3c196440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53500/0 (socket says 172.21.15.52:53500) 2023-11-27T15:20:40.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.127+0000 7f4a39d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a3c101d10 0x7f4a3c196980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43160/0 (socket says 172.21.15.52:43160) 2023-11-27T15:20:40.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.127+0000 7f4a3a59c700 1 -- 172.21.15.52:0/2147544406 learned_addr learned my addr 172.21.15.52:0/2147544406 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:40.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.127+0000 7f4a3ad9d700 1 --2- 172.21.15.52:0/2147544406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a3c10a920 0x7f4a3c19ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.128+0000 7f4a3a59c700 1 -- 172.21.15.52:0/2147544406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a3c10a920 msgr2=0x7f4a3c19ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.128+0000 7f4a3a59c700 1 --2- 172.21.15.52:0/2147544406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a3c10a920 0x7f4a3c19ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.128+0000 7f4a3a59c700 1 -- 172.21.15.52:0/2147544406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a3c101d10 msgr2=0x7f4a3c196980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.128+0000 7f4a3a59c700 1 --2- 172.21.15.52:0/2147544406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a3c101d10 0x7f4a3c196980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.128+0000 7f4a3a59c700 1 -- 172.21.15.52:0/2147544406 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a2c0095f0 con 0x7f4a3c101400 2023-11-27T15:20:40.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.128+0000 7f4a3ad9d700 1 --2- 172.21.15.52:0/2147544406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a3c10a920 0x7f4a3c19ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:40.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.128+0000 7f4a39d9b700 1 --2- 172.21.15.52:0/2147544406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a3c101d10 0x7f4a3c196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:40.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.128+0000 7f4a3a59c700 1 --2- 172.21.15.52:0/2147544406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a3c101400 0x7f4a3c196440 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c000c00 tx=0x7f4a2c004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:40.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.128+0000 7f4a2b7fe700 1 -- 172.21.15.52:0/2147544406 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c01d070 con 0x7f4a3c101400 2023-11-27T15:20:40.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.128+0000 7f4a40cff700 1 -- 172.21.15.52:0/2147544406 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a3c1103a0 con 0x7f4a3c101400 2023-11-27T15:20:40.137 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.128+0000 7f4a2b7fe700 1 -- 172.21.15.52:0/2147544406 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a2c021410 con 0x7f4a3c101400 2023-11-27T15:20:40.137 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.128+0000 7f4a2b7fe700 1 -- 172.21.15.52:0/2147544406 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c017650 con 0x7f4a3c101400 2023-11-27T15:20:40.137 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.128+0000 7f4a40cff700 1 -- 172.21.15.52:0/2147544406 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a3c110210 con 0x7f4a3c101400 2023-11-27T15:20:40.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.130+0000 7f4a2b7fe700 1 -- 172.21.15.52:0/2147544406 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f4a2c021580 con 0x7f4a3c101400 2023-11-27T15:20:40.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.131+0000 7f4a2b7fe700 1 --2- 172.21.15.52:0/2147544406 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f4a240a12c0 0x7f4a240a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.131+0000 7f4a2b7fe700 1 -- 172.21.15.52:0/2147544406 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4a2c0cdfa0 con 0x7f4a3c101400 2023-11-27T15:20:40.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.131+0000 7f4a39d9b700 1 --2- 172.21.15.52:0/2147544406 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f4a240a12c0 0x7f4a240a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.132+0000 7f4a39d9b700 1 --2- 172.21.15.52:0/2147544406 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f4a240a12c0 0x7f4a240a3770 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c197a50 tx=0x7f4a30012040 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:40.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.132+0000 7f4a4ce8bb80 1 Processor -- start 2023-11-27T15:20:40.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.132+0000 7f4a4ce8bb80 1 -- start start 2023-11-27T15:20:40.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.132+0000 7f4a4ce8bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3fd0d7b60 0x55e3fd0d7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.133+0000 7f4a4ce8bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3fd0d8470 0x55e3fd0dc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.133+0000 7f4a4ce8bb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3fd0dc8b0 0x55e3fd0dcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.133+0000 7f4a4ce8bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3fcfdd060 con 0x55e3fd0d8470 2023-11-27T15:20:40.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.133+0000 7f4a4ce8bb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3fd02f8a0 con 0x55e3fd0dc8b0 2023-11-27T15:20:40.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.133+0000 7f4a4ce8bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3fd028d30 con 0x55e3fd0d7b60 2023-11-27T15:20:40.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.133+0000 7f4a39d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3fd0d8470 0x55e3fd0dc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.133+0000 7f4a3a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3fd0d7b60 0x55e3fd0d7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.133+0000 7f4a3ad9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3fd0dc8b0 0x55e3fd0dcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.133+0000 7f4a3ad9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3fd0dc8b0 0x55e3fd0dcd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43168/0 (socket says 172.21.15.52:43168) 2023-11-27T15:20:40.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.133+0000 7f4a3a59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3fd0d7b60 0x55e3fd0d7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:40152/0 (socket says 172.21.15.52:40152) 2023-11-27T15:20:40.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.133+0000 7f4a39d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3fd0d8470 0x55e3fd0dc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53512/0 (socket says 172.21.15.52:53512) 2023-11-27T15:20:40.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.133+0000 7f4a3ad9d700 1 -- 172.21.15.52:0/2200043582 learned_addr learned my addr 172.21.15.52:0/2200043582 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:40.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.133+0000 7f4a3a59c700 1 -- 172.21.15.52:0/2200043582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3fd0dc8b0 msgr2=0x55e3fd0dcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:40.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.133+0000 7f4a3a59c700 1 --2- 172.21.15.52:0/2200043582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3fd0dc8b0 0x55e3fd0dcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.133+0000 7f4a3a59c700 1 -- 172.21.15.52:0/2200043582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3fd0d8470 msgr2=0x55e3fd0dc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:40.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.133+0000 7f4a3a59c700 1 --2- 172.21.15.52:0/2200043582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3fd0d8470 0x55e3fd0dc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.133+0000 7f4a3a59c700 1 -- 172.21.15.52:0/2200043582 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3fceaa630 con 0x55e3fd0d7b60 2023-11-27T15:20:40.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.133+0000 7f4a3ad9d700 1 --2- 172.21.15.52:0/2200043582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3fd0dc8b0 0x55e3fd0dcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:40.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.133+0000 7f4a3a59c700 1 --2- 172.21.15.52:0/2200043582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3fd0d7b60 0x55e3fd0d7f30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c0cb6f0 tx=0x7f4a2c09afa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4bfea1ad447f157 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:40.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.133+0000 7f4a28ff9700 1 -- 172.21.15.52:0/2200043582 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c021860 con 0x55e3fd0d7b60 2023-11-27T15:20:40.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.133+0000 7f4a28ff9700 1 -- 172.21.15.52:0/2200043582 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a2c09b100 con 0x55e3fd0d7b60 2023-11-27T15:20:40.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.134+0000 7f4a28ff9700 1 -- 172.21.15.52:0/2200043582 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c09a850 con 0x55e3fd0d7b60 2023-11-27T15:20:40.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.134+0000 7f4a4ce8bb80 1 -- 172.21.15.52:0/2200043582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3fd0d7b60 msgr2=0x55e3fd0d7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.146 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.134+0000 7f4a4ce8bb80 1 --2- 172.21.15.52:0/2200043582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3fd0d7b60 0x55e3fd0d7f30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c0cb6f0 tx=0x7f4a2c09afa0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.146 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.134+0000 7f4a4ce8bb80 1 -- 172.21.15.52:0/2200043582 shutdown_connections 2023-11-27T15:20:40.146 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.134+0000 7f4a4ce8bb80 1 --2- 172.21.15.52:0/2200043582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3fd0dc8b0 0x55e3fd0dcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.146 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.134+0000 7f4a4ce8bb80 1 --2- 172.21.15.52:0/2200043582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3fd0d7b60 0x55e3fd0d7f30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.146 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.134+0000 7f4a4ce8bb80 1 --2- 172.21.15.52:0/2200043582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3fd0d8470 0x55e3fd0dc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.146 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.134+0000 7f4a4ce8bb80 1 -- 172.21.15.52:0/2200043582 >> 172.21.15.52:0/2200043582 conn(0x55e3fd0cd360 msgr2=0x55e3fd0d2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:40.146 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.134+0000 7f4a4ce8bb80 1 -- 172.21.15.52:0/2200043582 shutdown_connections 2023-11-27T15:20:40.146 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.134+0000 7f4a4ce8bb80 1 -- 172.21.15.52:0/2200043582 wait complete. 2023-11-27T15:20:40.146 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.135+0000 7f4a4ce8bb80 1 -- start start 2023-11-27T15:20:40.147 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.135+0000 7f4a4ce8bb80 1 Processor -- start 2023-11-27T15:20:40.147 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.135+0000 7f4a4ce8bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3fd0d7b60 0x55e3fd0e3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.147 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.135+0000 7f4a4ce8bb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3fd0d8470 0x55e3fd0e11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.147 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.135+0000 7f4a4ce8bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3fd0dc8b0 0x55e3fd0e1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.147 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.135+0000 7f4a4ce8bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3fcfca8e0 con 0x55e3fd0dc8b0 2023-11-27T15:20:40.147 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.135+0000 7f4a4ce8bb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3fcfca580 con 0x55e3fd0d8470 2023-11-27T15:20:40.147 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.135+0000 7f4a4ce8bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3fceac920 con 0x55e3fd0d7b60 2023-11-27T15:20:40.147 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.135+0000 7f4a4ce8bb80 10 client.0 _ll_register_callbacks cb 0x55e3fcfe8780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:20:40.147 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.135+0000 7f4a39d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3fd0d8470 0x55e3fd0e11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.147 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.135+0000 7f4a39d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3fd0d8470 0x55e3fd0e11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43174/0 (socket says 172.21.15.52:43174) 2023-11-27T15:20:40.147 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.135+0000 7f4a39d9b700 1 -- 172.21.15.52:0/234106556 learned_addr learned my addr 172.21.15.52:0/234106556 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:40.147 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.135+0000 7f4a3ad9d700 1 --2- 172.21.15.52:0/234106556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3fd0dc8b0 0x55e3fd0e1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.148 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.135+0000 7f4a3a59c700 1 --2- 172.21.15.52:0/234106556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3fd0d7b60 0x55e3fd0e3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.148 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.135+0000 7f4a39d9b700 1 -- 172.21.15.52:0/234106556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3fd0d7b60 msgr2=0x55e3fd0e3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:40.148 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.136+0000 7f4a39d9b700 1 --2- 172.21.15.52:0/234106556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3fd0d7b60 0x55e3fd0e3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.148 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.136+0000 7f4a39d9b700 1 -- 172.21.15.52:0/234106556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3fd0dc8b0 msgr2=0x55e3fd0e1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:40.148 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.136+0000 7f4a39d9b700 1 --2- 172.21.15.52:0/234106556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3fd0dc8b0 0x55e3fd0e1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.148 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.136+0000 7f4a39d9b700 1 -- 172.21.15.52:0/234106556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a3c110210 con 0x55e3fd0d8470 2023-11-27T15:20:40.148 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.136+0000 7f4a3ad9d700 1 --2- 172.21.15.52:0/234106556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3fd0dc8b0 0x55e3fd0e1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:40.148 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.136+0000 7f4a39d9b700 1 --2- 172.21.15.52:0/234106556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3fd0d8470 0x55e3fd0e11a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4a30003ab0 tx=0x7f4a30004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7354f8dfd1ec4f4c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:40.148 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.136+0000 7f4a21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2023-11-27T15:20:40.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.136+0000 7f4a21ffb700 1 -- 172.21.15.52:0/234106556 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3001a810 con 0x55e3fd0d8470 2023-11-27T15:20:40.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.136+0000 7f4a21ffb700 1 -- 172.21.15.52:0/234106556 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4a3001a970 con 0x55e3fd0d8470 2023-11-27T15:20:40.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.136+0000 7f4a4ce8bb80 1 -- 172.21.15.52:0/234106556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e3fd0cab20 con 0x55e3fd0d8470 2023-11-27T15:20:40.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.136+0000 7f4a21ffb700 1 -- 172.21.15.52:0/234106556 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a3001ab30 con 0x55e3fd0d8470 2023-11-27T15:20:40.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.136+0000 7f4a4ce8bb80 10 client.25847 fetch_fsmap learned FSMap version 24 2023-11-27T15:20:40.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.136+0000 7f4a4ce8bb80 1 -- 172.21.15.52:0/234106556 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e3fcfef190 con 0x55e3fd0d8470 2023-11-27T15:20:40.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.136+0000 7f4a21ffb700 1 -- 172.21.15.52:0/234106556 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e3fd0cab20 con 0x55e3fd0d8470 2023-11-27T15:20:40.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.137+0000 7f4a21ffb700 1 -- 172.21.15.52:0/234106556 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4a3001c3f0 con 0x55e3fd0d8470 2023-11-27T15:20:40.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.137+0000 7f4a4ce8bb80 10 client.25847 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:20:40.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.137+0000 7f4a4ce8bb80 10 client.25847 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:20:40.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.137+0000 7f4a4ce8bb80 20 client.25847 populate_metadata read hostname 'smithi052' 2023-11-27T15:20:40.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.137+0000 7f4a4ce8bb80 1 --2- 172.21.15.52:0/234106556 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55e3fd0f1660 0x55e3fd0f3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.137+0000 7f4a4ce8bb80 4 client.25847 mds_command: new command op to 24494 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:20:40.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.137+0000 7f4a4ce8bb80 1 -- 172.21.15.52:0/234106556 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3fcfeb0a0 con 0x55e3fd0f1660 2023-11-27T15:20:40.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.137+0000 7f4a3a59c700 1 --2- 172.21.15.52:0/234106556 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55e3fd0f1660 0x55e3fd0f3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.137+0000 7f4a21ffb700 10 client.25847 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:20:40.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.137+0000 7f4a3a59c700 1 --2- 172.21.15.52:0/234106556 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55e3fd0f1660 0x55e3fd0f3a70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9a16ae9d25eb7ba1 server_cookie=4201cce0805d64b3 in_seq=0 out_seq=0 2023-11-27T15:20:40.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.138+0000 7f4a21ffb700 1 -- 172.21.15.52:0/234106556 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e3fcfeb0a0 con 0x55e3fd0f1660 2023-11-27T15:20:40.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.138+0000 7f4a21ffb700 10 client.25847 handle_command_reply: tid=0 2023-11-27T15:20:40.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.138+0000 7f4a4ce8bb80 1 client.25847 shutdown 2023-11-27T15:20:40.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.138+0000 7f4a4ce8bb80 1 -- 172.21.15.52:0/234106556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3fd0d8470 msgr2=0x55e3fd0e11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.138+0000 7f4a4ce8bb80 1 --2- 172.21.15.52:0/234106556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3fd0d8470 0x55e3fd0e11a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4a30003ab0 tx=0x7f4a30004750 comp rx=0 tx=0).stop 2023-11-27T15:20:40.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.139+0000 7f4a4ce8bb80 1 -- 172.21.15.52:0/234106556 shutdown_connections 2023-11-27T15:20:40.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.139+0000 7f4a4ce8bb80 1 --2- 172.21.15.52:0/234106556 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55e3fd0f1660 0x55e3fd0f3a70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.139+0000 7f4a4ce8bb80 1 --2- 172.21.15.52:0/234106556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3fd0dc8b0 0x55e3fd0e1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.139+0000 7f4a4ce8bb80 1 --2- 172.21.15.52:0/234106556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3fd0d7b60 0x55e3fd0e3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.139+0000 7f4a4ce8bb80 1 --2- 172.21.15.52:0/234106556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3fd0d8470 0x55e3fd0e11a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.139+0000 7f4a21ffb700 0 client.25847 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:20:40.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.139+0000 7f4a4ce8bb80 1 -- 172.21.15.52:0/234106556 >> 172.21.15.52:0/234106556 conn(0x55e3fd0cd360 msgr2=0x55e3fd0d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:40.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.139+0000 7f4a4ce8bb80 1 -- 172.21.15.52:0/234106556 shutdown_connections 2023-11-27T15:20:40.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.139+0000 7f4a4ce8bb80 1 -- 172.21.15.52:0/234106556 wait complete. 2023-11-27T15:20:40.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.139+0000 7f4a4ce8bb80 20 client.25847 trim_cache size 0 max 16384 2023-11-27T15:20:40.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.153+0000 7f4a4ce8bb80 1 Processor -- start 2023-11-27T15:20:40.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.153+0000 7f4a4ce8bb80 1 -- start start 2023-11-27T15:20:40.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.153+0000 7f4a4ce8bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3fd0d7b60 0x55e3fd0d2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.153+0000 7f4a4ce8bb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3fd0d8470 0x55e3fd0d2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.153+0000 7f4a4ce8bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3fd0dc8b0 0x55e3fd0e77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.153+0000 7f4a4ce8bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3fd0eec90 con 0x55e3fd0d7b60 2023-11-27T15:20:40.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.153+0000 7f4a4ce8bb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3fd0d8bc0 con 0x55e3fd0d8470 2023-11-27T15:20:40.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.153+0000 7f4a4ce8bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3fd0edf70 con 0x55e3fd0dc8b0 2023-11-27T15:20:40.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.153+0000 7f4a3a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3fd0d7b60 0x55e3fd0d2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.153+0000 7f4a39d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3fd0d8470 0x55e3fd0d2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.153+0000 7f4a3a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3fd0d7b60 0x55e3fd0d2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53522/0 (socket says 172.21.15.52:53522) 2023-11-27T15:20:40.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.153+0000 7f4a39d9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3fd0d8470 0x55e3fd0d2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43182/0 (socket says 172.21.15.52:43182) 2023-11-27T15:20:40.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.153+0000 7f4a3a59c700 1 -- 172.21.15.52:0/350338218 learned_addr learned my addr 172.21.15.52:0/350338218 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:40.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.153+0000 7f4a3ad9d700 1 --2- 172.21.15.52:0/350338218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3fd0dc8b0 0x55e3fd0e77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.153+0000 7f4a3a59c700 1 -- 172.21.15.52:0/350338218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3fd0dc8b0 msgr2=0x55e3fd0e77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:40.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.153+0000 7f4a3a59c700 1 --2- 172.21.15.52:0/350338218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3fd0dc8b0 0x55e3fd0e77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.153+0000 7f4a3a59c700 1 -- 172.21.15.52:0/350338218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3fd0d8470 msgr2=0x55e3fd0d2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:40.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.153+0000 7f4a3a59c700 1 --2- 172.21.15.52:0/350338218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3fd0d8470 0x55e3fd0d2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.153+0000 7f4a3a59c700 1 -- 172.21.15.52:0/350338218 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3fd0dcf50 con 0x55e3fd0d7b60 2023-11-27T15:20:40.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.154+0000 7f4a3a59c700 1 --2- 172.21.15.52:0/350338218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3fd0d7b60 0x55e3fd0d2560 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c005cd0 tx=0x7f4a2c021c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6344e2abf1e8f624 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:40.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.154+0000 7f4a297fa700 1 -- 172.21.15.52:0/350338218 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c0a3540 con 0x55e3fd0d7b60 2023-11-27T15:20:40.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.154+0000 7f4a297fa700 1 -- 172.21.15.52:0/350338218 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a2c09f130 con 0x55e3fd0d7b60 2023-11-27T15:20:40.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.154+0000 7f4a297fa700 1 -- 172.21.15.52:0/350338218 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c09b360 con 0x55e3fd0d7b60 2023-11-27T15:20:40.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.154+0000 7f4a4ce8bb80 1 -- 172.21.15.52:0/350338218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3fd0d7b60 msgr2=0x55e3fd0d2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.154+0000 7f4a4ce8bb80 1 --2- 172.21.15.52:0/350338218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3fd0d7b60 0x55e3fd0d2560 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c005cd0 tx=0x7f4a2c021c00 comp rx=0 tx=0).stop 2023-11-27T15:20:40.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.154+0000 7f4a4ce8bb80 1 -- 172.21.15.52:0/350338218 shutdown_connections 2023-11-27T15:20:40.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.154+0000 7f4a4ce8bb80 1 --2- 172.21.15.52:0/350338218 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3fd0dc8b0 0x55e3fd0e77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.154+0000 7f4a4ce8bb80 1 --2- 172.21.15.52:0/350338218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3fd0d7b60 0x55e3fd0d2560 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.154+0000 7f4a4ce8bb80 1 --2- 172.21.15.52:0/350338218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3fd0d8470 0x55e3fd0d2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.154+0000 7f4a4ce8bb80 1 -- 172.21.15.52:0/350338218 >> 172.21.15.52:0/350338218 conn(0x55e3fd0cd360 msgr2=0x55e3fd0d71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:40.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.155+0000 7f4a4ce8bb80 1 -- 172.21.15.52:0/350338218 shutdown_connections 2023-11-27T15:20:40.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.155+0000 7f4a4ce8bb80 1 -- 172.21.15.52:0/350338218 wait complete. 2023-11-27T15:20:40.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.155+0000 7f4a4ce8bb80 1 -- start start 2023-11-27T15:20:40.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.155+0000 7f4a4ce8bb80 1 Processor -- start 2023-11-27T15:20:40.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.155+0000 7f4a4ce8bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3fd0d7b60 0x55e3fd13dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.155+0000 7f4a4ce8bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3fd0d8470 0x55e3fd13e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.155+0000 7f4a4ce8bb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3fd0dc8b0 0x55e3fd0e7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.155+0000 7f4a4ce8bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3fd0e7f50 con 0x55e3fd0d7b60 2023-11-27T15:20:40.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.155+0000 7f4a4ce8bb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3fd0e80c0 con 0x55e3fd0dc8b0 2023-11-27T15:20:40.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.155+0000 7f4a4ce8bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3fd0e8280 con 0x55e3fd0d8470 2023-11-27T15:20:40.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.155+0000 7f4a4ce8bb80 10 client.0 _ll_register_callbacks cb 0x7f4a30003680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:20:40.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.155+0000 7f4a3a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3fd0d7b60 0x55e3fd13dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.155+0000 7f4a3a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3fd0d7b60 0x55e3fd13dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53528/0 (socket says 172.21.15.52:53528) 2023-11-27T15:20:40.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.155+0000 7f4a3a59c700 1 -- 172.21.15.52:0/1241059037 learned_addr learned my addr 172.21.15.52:0/1241059037 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:40.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.155+0000 7f4a3ad9d700 1 --2- 172.21.15.52:0/1241059037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3fd0dc8b0 0x55e3fd0e7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.155+0000 7f4a39d9b700 1 --2- 172.21.15.52:0/1241059037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3fd0d8470 0x55e3fd13e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.156+0000 7f4a3a59c700 1 -- 172.21.15.52:0/1241059037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3fd0d8470 msgr2=0x55e3fd13e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:40.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.156+0000 7f4a3a59c700 1 --2- 172.21.15.52:0/1241059037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3fd0d8470 0x55e3fd13e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.156+0000 7f4a3a59c700 1 -- 172.21.15.52:0/1241059037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3fd0dc8b0 msgr2=0x55e3fd0e7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:40.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.156+0000 7f4a3a59c700 1 --2- 172.21.15.52:0/1241059037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3fd0dc8b0 0x55e3fd0e7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.156+0000 7f4a3a59c700 1 -- 172.21.15.52:0/1241059037 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3fceaa630 con 0x55e3fd0d7b60 2023-11-27T15:20:40.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.156+0000 7f4a39d9b700 1 --2- 172.21.15.52:0/1241059037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3fd0d8470 0x55e3fd13e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:40.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.156+0000 7f4a3ad9d700 1 --2- 172.21.15.52:0/1241059037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3fd0dc8b0 0x55e3fd0e7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:40.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.156+0000 7f4a3a59c700 1 --2- 172.21.15.52:0/1241059037 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3fd0d7b60 0x55e3fd13dbf0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c0059d0 tx=0x7f4a2c09aef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258424ab76f705db server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:40.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.156+0000 7f4a21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:20:40.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.156+0000 7f4a21ffb700 1 -- 172.21.15.52:0/1241059037 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c0a3540 con 0x55e3fd0d7b60 2023-11-27T15:20:40.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.156+0000 7f4a21ffb700 1 -- 172.21.15.52:0/1241059037 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4a2c0cbe40 con 0x55e3fd0d7b60 2023-11-27T15:20:40.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.156+0000 7f4a4ce8bb80 1 -- 172.21.15.52:0/1241059037 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a2c09b100 con 0x55e3fd0d7b60 2023-11-27T15:20:40.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.156+0000 7f4a21ffb700 1 -- 172.21.15.52:0/1241059037 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a2c09b360 con 0x55e3fd0d7b60 2023-11-27T15:20:40.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.156+0000 7f4a4ce8bb80 10 client.16038 fetch_fsmap learned FSMap version 24 2023-11-27T15:20:40.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.156+0000 7f4a4ce8bb80 1 -- 172.21.15.52:0/1241059037 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e3fce56de0 con 0x55e3fd0d7b60 2023-11-27T15:20:40.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.156+0000 7f4a21ffb700 1 -- 172.21.15.52:0/1241059037 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a2c09b100 con 0x55e3fd0d7b60 2023-11-27T15:20:40.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.157+0000 7f4a21ffb700 1 -- 172.21.15.52:0/1241059037 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4a2c0a3030 con 0x55e3fd0d7b60 2023-11-27T15:20:40.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.157+0000 7f4a4ce8bb80 10 client.16038 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:20:40.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.157+0000 7f4a4ce8bb80 10 client.16038 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:20:40.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.157+0000 7f4a4ce8bb80 20 client.16038 populate_metadata read hostname 'smithi052' 2023-11-27T15:20:40.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.157+0000 7f4a4ce8bb80 1 --2- 172.21.15.52:0/1241059037 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55e3fd145530 0x55e3fd147910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.157+0000 7f4a4ce8bb80 4 client.16038 mds_command: new command op to 24494 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:20:40.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.157+0000 7f4a4ce8bb80 1 -- 172.21.15.52:0/1241059037 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e3fcff7660 con 0x55e3fd145530 2023-11-27T15:20:40.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.157+0000 7f4a39d9b700 1 --2- 172.21.15.52:0/1241059037 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55e3fd145530 0x55e3fd147910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.157+0000 7f4a21ffb700 10 client.16038 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:20:40.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.157+0000 7f4a39d9b700 1 --2- 172.21.15.52:0/1241059037 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55e3fd145530 0x55e3fd147910 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ed466ddf2098a4fb server_cookie=f86b5de6c0242c in_seq=0 out_seq=0 2023-11-27T15:20:40.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.159+0000 7f4a21ffb700 1 -- 172.21.15.52:0/1241059037 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e3fcff7660 con 0x55e3fd145530 2023-11-27T15:20:40.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.159+0000 7f4a21ffb700 10 client.16038 handle_command_reply: tid=0 2023-11-27T15:20:40.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.159+0000 7f4a4ce8bb80 1 client.16038 shutdown 2023-11-27T15:20:40.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.159+0000 7f4a4ce8bb80 1 -- 172.21.15.52:0/1241059037 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3fd0d7b60 msgr2=0x55e3fd13dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.159+0000 7f4a4ce8bb80 1 --2- 172.21.15.52:0/1241059037 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3fd0d7b60 0x55e3fd13dbf0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c0059d0 tx=0x7f4a2c09aef0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.159+0000 7f4a4ce8bb80 1 -- 172.21.15.52:0/1241059037 shutdown_connections 2023-11-27T15:20:40.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.159+0000 7f4a4ce8bb80 1 --2- 172.21.15.52:0/1241059037 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55e3fd145530 0x55e3fd147910 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.159+0000 7f4a4ce8bb80 1 --2- 172.21.15.52:0/1241059037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e3fd0dc8b0 0x55e3fd0e7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.159+0000 7f4a4ce8bb80 1 --2- 172.21.15.52:0/1241059037 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3fd0d7b60 0x55e3fd13dbf0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.159+0000 7f4a21ffb700 0 client.16038 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:20:40.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.159+0000 7f4a4ce8bb80 1 --2- 172.21.15.52:0/1241059037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e3fd0d8470 0x55e3fd13e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.159+0000 7f4a4ce8bb80 1 -- 172.21.15.52:0/1241059037 >> 172.21.15.52:0/1241059037 conn(0x55e3fd0cd360 msgr2=0x55e3fd0ecf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:40.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.159+0000 7f4a4ce8bb80 1 -- 172.21.15.52:0/1241059037 shutdown_connections 2023-11-27T15:20:40.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.159+0000 7f4a4ce8bb80 1 -- 172.21.15.52:0/1241059037 wait complete. 2023-11-27T15:20:40.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.159+0000 7f4a4ce8bb80 20 client.16038 trim_cache size 0 max 16384 2023-11-27T15:20:40.172 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:20:40.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.160+0000 7f4a40cff700 1 -- 172.21.15.52:0/2147544406 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f4a240a12c0 msgr2=0x7f4a240a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.160+0000 7f4a40cff700 1 --2- 172.21.15.52:0/2147544406 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f4a240a12c0 0x7f4a240a3770 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c197a50 tx=0x7f4a30012040 comp rx=0 tx=0).stop 2023-11-27T15:20:40.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.160+0000 7f4a40cff700 1 -- 172.21.15.52:0/2147544406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a3c101400 msgr2=0x7f4a3c196440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.160+0000 7f4a40cff700 1 --2- 172.21.15.52:0/2147544406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a3c101400 0x7f4a3c196440 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c000c00 tx=0x7f4a2c004790 comp rx=0 tx=0).stop 2023-11-27T15:20:40.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.160+0000 7f4a40cff700 1 -- 172.21.15.52:0/2147544406 shutdown_connections 2023-11-27T15:20:40.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.160+0000 7f4a40cff700 1 --2- 172.21.15.52:0/2147544406 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f4a240a12c0 0x7f4a240a3770 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.160+0000 7f4a40cff700 1 --2- 172.21.15.52:0/2147544406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a3c10a920 0x7f4a3c19ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.160+0000 7f4a40cff700 1 --2- 172.21.15.52:0/2147544406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a3c101400 0x7f4a3c196440 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.160+0000 7f4a40cff700 1 --2- 172.21.15.52:0/2147544406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a3c101d10 0x7f4a3c196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.160+0000 7f4a40cff700 1 -- 172.21.15.52:0/2147544406 >> 172.21.15.52:0/2147544406 conn(0x7f4a3c076200 msgr2=0x7f4a3c0fbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:40.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.160+0000 7f4a40cff700 1 -- 172.21.15.52:0/2147544406 shutdown_connections 2023-11-27T15:20:40.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.160+0000 7f4a40cff700 1 -- 172.21.15.52:0/2147544406 wait complete. 2023-11-27T15:20:40.179 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-27T15:20:40.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:39 smithi150 ceph-mon[100559]: pgmap v85: 97 pgs: 97 active+clean; 90 MiB data, 264 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.2 MiB/s wr, 318 op/s 2023-11-27T15:20:40.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:39 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3784274395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:40.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.311+0000 7f5d33db5700 1 -- 172.21.15.52:0/1138896965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d2c100f50 msgr2=0x7f5d2c104f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.311+0000 7f5d33db5700 1 --2- 172.21.15.52:0/1138896965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d2c100f50 0x7f5d2c104f10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5d280099c0 tx=0x7f5d28009cd0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.311+0000 7f5d33db5700 1 -- 172.21.15.52:0/1138896965 shutdown_connections 2023-11-27T15:20:40.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.311+0000 7f5d33db5700 1 --2- 172.21.15.52:0/1138896965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d2c1005b0 0x7f5d2c100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.311+0000 7f5d33db5700 1 --2- 172.21.15.52:0/1138896965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d2c1055f0 0x7f5d2c10d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.311+0000 7f5d33db5700 1 --2- 172.21.15.52:0/1138896965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d2c100f50 0x7f5d2c104f10 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.311+0000 7f5d33db5700 1 -- 172.21.15.52:0/1138896965 >> 172.21.15.52:0/1138896965 conn(0x7f5d2c077170 msgr2=0x7f5d2c077570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:40.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.311+0000 7f5d33db5700 1 -- 172.21.15.52:0/1138896965 shutdown_connections 2023-11-27T15:20:40.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.311+0000 7f5d33db5700 1 -- 172.21.15.52:0/1138896965 wait complete. 2023-11-27T15:20:40.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.312+0000 7f5d33db5700 1 Processor -- start 2023-11-27T15:20:40.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.312+0000 7f5d33db5700 1 -- start start 2023-11-27T15:20:40.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.312+0000 7f5d33db5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d2c1005b0 0x7f5d2c1964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.312+0000 7f5d33db5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d2c100f50 0x7f5d2c1969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.312+0000 7f5d33db5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d2c1055f0 0x7f5d2c19ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.312+0000 7f5d33db5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c19b490 con 0x7f5d2c1055f0 2023-11-27T15:20:40.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.312+0000 7f5d33db5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c19b600 con 0x7f5d2c100f50 2023-11-27T15:20:40.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.312+0000 7f5d33db5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c19b7c0 con 0x7f5d2c1005b0 2023-11-27T15:20:40.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.312+0000 7f5d32352700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d2c1055f0 0x7f5d2c19ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.312+0000 7f5d31b51700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d2c1005b0 0x7f5d2c1964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.312+0000 7f5d31350700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d2c100f50 0x7f5d2c1969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.312+0000 7f5d32352700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d2c1055f0 0x7f5d2c19ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53546/0 (socket says 172.21.15.52:53546) 2023-11-27T15:20:40.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.313+0000 7f5d31350700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d2c100f50 0x7f5d2c1969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43202/0 (socket says 172.21.15.52:43202) 2023-11-27T15:20:40.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.313+0000 7f5d32352700 1 -- 172.21.15.52:0/1448734356 learned_addr learned my addr 172.21.15.52:0/1448734356 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:40.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.313+0000 7f5d31350700 1 -- 172.21.15.52:0/1448734356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d2c1005b0 msgr2=0x7f5d2c1964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.313+0000 7f5d31350700 1 --2- 172.21.15.52:0/1448734356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d2c1005b0 0x7f5d2c1964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.313+0000 7f5d31350700 1 -- 172.21.15.52:0/1448734356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d2c1055f0 msgr2=0x7f5d2c19ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.313+0000 7f5d31350700 1 --2- 172.21.15.52:0/1448734356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d2c1055f0 0x7f5d2c19ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.313+0000 7f5d31350700 1 -- 172.21.15.52:0/1448734356 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d280095f0 con 0x7f5d2c100f50 2023-11-27T15:20:40.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.313+0000 7f5d32352700 1 --2- 172.21.15.52:0/1448734356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d2c1055f0 0x7f5d2c19ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:40.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.313+0000 7f5d31350700 1 --2- 172.21.15.52:0/1448734356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d2c100f50 0x7f5d2c1969f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5d28009fd0 tx=0x7f5d2800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:40.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.313+0000 7f5d22ffd700 1 -- 172.21.15.52:0/1448734356 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2801d070 con 0x7f5d2c100f50 2023-11-27T15:20:40.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.313+0000 7f5d33db5700 1 -- 172.21.15.52:0/1448734356 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d2c1103c0 con 0x7f5d2c100f50 2023-11-27T15:20:40.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.313+0000 7f5d22ffd700 1 -- 172.21.15.52:0/1448734356 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d28022470 con 0x7f5d2c100f50 2023-11-27T15:20:40.322 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.313+0000 7f5d22ffd700 1 -- 172.21.15.52:0/1448734356 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d28017650 con 0x7f5d2c100f50 2023-11-27T15:20:40.322 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.313+0000 7f5d33db5700 1 -- 172.21.15.52:0/1448734356 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2c110230 con 0x7f5d2c100f50 2023-11-27T15:20:40.324 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.316+0000 7f5d22ffd700 1 -- 172.21.15.52:0/1448734356 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f5d280225e0 con 0x7f5d2c100f50 2023-11-27T15:20:40.324 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.317+0000 7f5d22ffd700 1 --2- 172.21.15.52:0/1448734356 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f5d180a12c0 0x7f5d180a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.324 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.317+0000 7f5d31b51700 1 --2- 172.21.15.52:0/1448734356 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f5d180a12c0 0x7f5d180a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.324 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.317+0000 7f5d22ffd700 1 -- 172.21.15.52:0/1448734356 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5d280cd880 con 0x7f5d2c100f50 2023-11-27T15:20:40.324 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.317+0000 7f5d3ff41b80 1 Processor -- start 2023-11-27T15:20:40.325 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.317+0000 7f5d31b51700 1 --2- 172.21.15.52:0/1448734356 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f5d180a12c0 0x7f5d180a3770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5d2400f540 tx=0x7f5d2400b040 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:40.325 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.318+0000 7f5d3ff41b80 1 -- start start 2023-11-27T15:20:40.325 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.318+0000 7f5d3ff41b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dfdaff70 0x55d2dfdb0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.325 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.318+0000 7f5d3ff41b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2dfdb0880 0x55d2dfdb4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.325 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.318+0000 7f5d3ff41b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2dfdb4cc0 0x55d2dfdb5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.325 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.318+0000 7f5d3ff41b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2dfcb1060 con 0x55d2dfdb4cc0 2023-11-27T15:20:40.325 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.318+0000 7f5d3ff41b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2dfd038a0 con 0x55d2dfdb0880 2023-11-27T15:20:40.325 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.318+0000 7f5d3ff41b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2dfcfcd30 con 0x55d2dfdaff70 2023-11-27T15:20:40.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.318+0000 7f5d31b51700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dfdaff70 0x55d2dfdb0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.318+0000 7f5d32352700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2dfdb4cc0 0x55d2dfdb5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.318+0000 7f5d31350700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2dfdb0880 0x55d2dfdb4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.318+0000 7f5d31b51700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dfdaff70 0x55d2dfdb0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:40194/0 (socket says 172.21.15.52:40194) 2023-11-27T15:20:40.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.318+0000 7f5d31b51700 1 -- 172.21.15.52:0/2129690491 learned_addr learned my addr 172.21.15.52:0/2129690491 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:40.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.318+0000 7f5d31350700 1 -- 172.21.15.52:0/2129690491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dfdaff70 msgr2=0x55d2dfdb0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:40.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.318+0000 7f5d31350700 1 --2- 172.21.15.52:0/2129690491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dfdaff70 0x55d2dfdb0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.318+0000 7f5d31350700 1 -- 172.21.15.52:0/2129690491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2dfdb4cc0 msgr2=0x55d2dfdb5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:40.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.318+0000 7f5d31350700 1 --2- 172.21.15.52:0/2129690491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2dfdb4cc0 0x55d2dfdb5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.327 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.318+0000 7f5d31350700 1 -- 172.21.15.52:0/2129690491 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2dfb7e630 con 0x55d2dfdb0880 2023-11-27T15:20:40.327 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.318+0000 7f5d32352700 1 --2- 172.21.15.52:0/2129690491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2dfdb4cc0 0x55d2dfdb5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:40.327 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.319+0000 7f5d31350700 1 --2- 172.21.15.52:0/2129690491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2dfdb0880 0x55d2dfdb4780 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5d280caf70 tx=0x7f5d2809a2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcf87c24c75da0b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:40.327 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.319+0000 7f5d17fff700 1 -- 172.21.15.52:0/2129690491 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d280a2d40 con 0x55d2dfdb0880 2023-11-27T15:20:40.327 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.319+0000 7f5d17fff700 1 -- 172.21.15.52:0/2129690491 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d28022900 con 0x55d2dfdb0880 2023-11-27T15:20:40.327 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.319+0000 7f5d17fff700 1 -- 172.21.15.52:0/2129690491 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2809a0d0 con 0x55d2dfdb0880 2023-11-27T15:20:40.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:39 smithi102 ceph-mon[100296]: pgmap v85: 97 pgs: 97 active+clean; 90 MiB data, 264 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.2 MiB/s wr, 318 op/s 2023-11-27T15:20:40.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:39 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3784274395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:40.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.320+0000 7f5d3ff41b80 1 -- 172.21.15.52:0/2129690491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2dfdb0880 msgr2=0x55d2dfdb4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.320+0000 7f5d3ff41b80 1 --2- 172.21.15.52:0/2129690491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2dfdb0880 0x55d2dfdb4780 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5d280caf70 tx=0x7f5d2809a2e0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.320+0000 7f5d3ff41b80 1 -- 172.21.15.52:0/2129690491 shutdown_connections 2023-11-27T15:20:40.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.320+0000 7f5d3ff41b80 1 --2- 172.21.15.52:0/2129690491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2dfdb4cc0 0x55d2dfdb5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.320+0000 7f5d3ff41b80 1 --2- 172.21.15.52:0/2129690491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dfdaff70 0x55d2dfdb0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.320+0000 7f5d3ff41b80 1 --2- 172.21.15.52:0/2129690491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2dfdb0880 0x55d2dfdb4780 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.320+0000 7f5d3ff41b80 1 -- 172.21.15.52:0/2129690491 >> 172.21.15.52:0/2129690491 conn(0x55d2dfda1360 msgr2=0x55d2dfda6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:40.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.320+0000 7f5d3ff41b80 1 -- 172.21.15.52:0/2129690491 shutdown_connections 2023-11-27T15:20:40.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.320+0000 7f5d3ff41b80 1 -- 172.21.15.52:0/2129690491 wait complete. 2023-11-27T15:20:40.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.320+0000 7f5d3ff41b80 1 -- start start 2023-11-27T15:20:40.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.320+0000 7f5d3ff41b80 1 Processor -- start 2023-11-27T15:20:40.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.321+0000 7f5d3ff41b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2dfdaff70 0x55d2dfdbc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.321+0000 7f5d3ff41b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2dfdb0880 0x55d2dfdb95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.321+0000 7f5d3ff41b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dfdb4cc0 0x55d2dfdb9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.321+0000 7f5d3ff41b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2dfc9e8e0 con 0x55d2dfdaff70 2023-11-27T15:20:40.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.321+0000 7f5d3ff41b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2dfc9e580 con 0x55d2dfdb0880 2023-11-27T15:20:40.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.321+0000 7f5d3ff41b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2dfb80920 con 0x55d2dfdb4cc0 2023-11-27T15:20:40.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.321+0000 7f5d3ff41b80 10 client.0 _ll_register_callbacks cb 0x55d2dfcbc780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:20:40.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.321+0000 7f5d31b51700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2dfdaff70 0x55d2dfdbc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.321+0000 7f5d31350700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2dfdb0880 0x55d2dfdb95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.321+0000 7f5d31b51700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2dfdaff70 0x55d2dfdbc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53556/0 (socket says 172.21.15.52:53556) 2023-11-27T15:20:40.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.321+0000 7f5d31b51700 1 -- 172.21.15.52:0/2826292427 learned_addr learned my addr 172.21.15.52:0/2826292427 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:40.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.321+0000 7f5d32352700 1 --2- 172.21.15.52:0/2826292427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dfdb4cc0 0x55d2dfdb9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.321+0000 7f5d31b51700 1 -- 172.21.15.52:0/2826292427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dfdb4cc0 msgr2=0x55d2dfdb9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:40.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.321+0000 7f5d31b51700 1 --2- 172.21.15.52:0/2826292427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dfdb4cc0 0x55d2dfdb9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.321+0000 7f5d31b51700 1 -- 172.21.15.52:0/2826292427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2dfdb0880 msgr2=0x55d2dfdb95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:40.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.321+0000 7f5d31b51700 1 --2- 172.21.15.52:0/2826292427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2dfdb0880 0x55d2dfdb95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.321+0000 7f5d31b51700 1 -- 172.21.15.52:0/2826292427 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2c110230 con 0x55d2dfdaff70 2023-11-27T15:20:40.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.321+0000 7f5d31350700 1 --2- 172.21.15.52:0/2826292427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2dfdb0880 0x55d2dfdb95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:40.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.322+0000 7f5d31b51700 1 --2- 172.21.15.52:0/2826292427 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2dfdaff70 0x55d2dfdbc340 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5d24004660 tx=0x7f5d24004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c93a574d06b46e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:40.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.322+0000 7f5d157fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:20:40.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.322+0000 7f5d157fa700 1 -- 172.21.15.52:0/2826292427 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2400e040 con 0x55d2dfdaff70 2023-11-27T15:20:40.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.322+0000 7f5d157fa700 1 -- 172.21.15.52:0/2826292427 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5d24015c70 con 0x55d2dfdaff70 2023-11-27T15:20:40.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.322+0000 7f5d3ff41b80 1 -- 172.21.15.52:0/2826292427 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d2dfd9eb20 con 0x55d2dfdaff70 2023-11-27T15:20:40.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.322+0000 7f5d157fa700 1 -- 172.21.15.52:0/2826292427 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d24015e30 con 0x55d2dfdaff70 2023-11-27T15:20:40.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.322+0000 7f5d3ff41b80 10 client.16050 fetch_fsmap learned FSMap version 24 2023-11-27T15:20:40.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.322+0000 7f5d3ff41b80 1 -- 172.21.15.52:0/2826292427 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d2dfcc3190 con 0x55d2dfdaff70 2023-11-27T15:20:40.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.322+0000 7f5d157fa700 1 -- 172.21.15.52:0/2826292427 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d2dfd9eb20 con 0x55d2dfdaff70 2023-11-27T15:20:40.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.323+0000 7f5d157fa700 1 -- 172.21.15.52:0/2826292427 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5d24023520 con 0x55d2dfdaff70 2023-11-27T15:20:40.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.323+0000 7f5d3ff41b80 10 client.16050 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:20:40.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.323+0000 7f5d3ff41b80 10 client.16050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:20:40.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.323+0000 7f5d3ff41b80 20 client.16050 populate_metadata read hostname 'smithi052' 2023-11-27T15:20:40.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.323+0000 7f5d3ff41b80 1 --2- 172.21.15.52:0/2826292427 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d2dfdc9950 0x55d2dfdcbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.323+0000 7f5d3ff41b80 4 client.16050 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:20:40.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.323+0000 7f5d3ff41b80 1 -- 172.21.15.52:0/2826292427 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d2dfcbf0a0 con 0x55d2dfdc9950 2023-11-27T15:20:40.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.323+0000 7f5d31350700 1 --2- 172.21.15.52:0/2826292427 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d2dfdc9950 0x55d2dfdcbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.323+0000 7f5d157fa700 10 client.16050 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:20:40.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.323+0000 7f5d31350700 1 --2- 172.21.15.52:0/2826292427 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d2dfdc9950 0x55d2dfdcbd60 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a3263d13ba726b6 server_cookie=36293acbc1021dd3 in_seq=0 out_seq=0 2023-11-27T15:20:40.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.327+0000 7f5d157fa700 1 -- 172.21.15.52:0/2826292427 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d2dfcbf0a0 con 0x55d2dfdc9950 2023-11-27T15:20:40.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.327+0000 7f5d157fa700 10 client.16050 handle_command_reply: tid=0 2023-11-27T15:20:40.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.327+0000 7f5d3ff41b80 1 client.16050 shutdown 2023-11-27T15:20:40.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.327+0000 7f5d3ff41b80 1 -- 172.21.15.52:0/2826292427 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2dfdaff70 msgr2=0x55d2dfdbc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.327+0000 7f5d3ff41b80 1 --2- 172.21.15.52:0/2826292427 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2dfdaff70 0x55d2dfdbc340 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5d24004660 tx=0x7f5d24004ad0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.328+0000 7f5d3ff41b80 1 -- 172.21.15.52:0/2826292427 shutdown_connections 2023-11-27T15:20:40.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.328+0000 7f5d3ff41b80 1 --2- 172.21.15.52:0/2826292427 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d2dfdc9950 0x55d2dfdcbd60 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.328+0000 7f5d3ff41b80 1 --2- 172.21.15.52:0/2826292427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dfdb4cc0 0x55d2dfdb9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.328+0000 7f5d3ff41b80 1 --2- 172.21.15.52:0/2826292427 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2dfdaff70 0x55d2dfdbc340 secure :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5d24004660 tx=0x7f5d24004ad0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.328+0000 7f5d3ff41b80 1 --2- 172.21.15.52:0/2826292427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2dfdb0880 0x55d2dfdb95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.328+0000 7f5d157fa700 0 client.16050 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:20:40.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.328+0000 7f5d3ff41b80 1 -- 172.21.15.52:0/2826292427 >> 172.21.15.52:0/2826292427 conn(0x55d2dfda1360 msgr2=0x55d2dfdb6170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:40.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.328+0000 7f5d3ff41b80 1 -- 172.21.15.52:0/2826292427 shutdown_connections 2023-11-27T15:20:40.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.328+0000 7f5d3ff41b80 1 -- 172.21.15.52:0/2826292427 wait complete. 2023-11-27T15:20:40.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.328+0000 7f5d3ff41b80 20 client.16050 trim_cache size 0 max 16384 2023-11-27T15:20:40.350 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.344+0000 7f5d3ff41b80 1 Processor -- start 2023-11-27T15:20:40.351 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.344+0000 7f5d3ff41b80 1 -- start start 2023-11-27T15:20:40.351 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.345+0000 7f5d3ff41b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2dfdb0880 0x55d2dfdc0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.351 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.345+0000 7f5d3ff41b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2dfdb4cc0 0x55d2dfdbbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.351 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.345+0000 7f5d3ff41b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dfdb25b0 0x55d2dfdb2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.351 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.345+0000 7f5d3ff41b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2dfdc6370 con 0x55d2dfdb4cc0 2023-11-27T15:20:40.351 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.345+0000 7f5d3ff41b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2dfda7710 con 0x55d2dfdb0880 2023-11-27T15:20:40.351 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.345+0000 7f5d3ff41b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2dfdbabc0 con 0x55d2dfdb25b0 2023-11-27T15:20:40.351 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.345+0000 7f5d31b51700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2dfdb0880 0x55d2dfdc0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.351 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.345+0000 7f5d31350700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2dfdb4cc0 0x55d2dfdbbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.351 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.345+0000 7f5d31b51700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2dfdb0880 0x55d2dfdc0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43222/0 (socket says 172.21.15.52:43222) 2023-11-27T15:20:40.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.345+0000 7f5d31350700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2dfdb4cc0 0x55d2dfdbbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53566/0 (socket says 172.21.15.52:53566) 2023-11-27T15:20:40.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.345+0000 7f5d31b51700 1 -- 172.21.15.52:0/1356093345 learned_addr learned my addr 172.21.15.52:0/1356093345 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:40.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.345+0000 7f5d32352700 1 --2- 172.21.15.52:0/1356093345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dfdb25b0 0x55d2dfdb2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.345+0000 7f5d31b51700 1 -- 172.21.15.52:0/1356093345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dfdb25b0 msgr2=0x55d2dfdb2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:40.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.345+0000 7f5d31b51700 1 --2- 172.21.15.52:0/1356093345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dfdb25b0 0x55d2dfdb2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.345+0000 7f5d31b51700 1 -- 172.21.15.52:0/1356093345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2dfdb4cc0 msgr2=0x55d2dfdbbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:40.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.345+0000 7f5d31b51700 1 --2- 172.21.15.52:0/1356093345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2dfdb4cc0 0x55d2dfdbbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.345+0000 7f5d31b51700 1 -- 172.21.15.52:0/1356093345 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2dfda5820 con 0x55d2dfdb0880 2023-11-27T15:20:40.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.345+0000 7f5d32352700 1 --2- 172.21.15.52:0/1356093345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dfdb25b0 0x55d2dfdb2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:40.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.345+0000 7f5d31350700 1 --2- 172.21.15.52:0/1356093345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2dfdb4cc0 0x55d2dfdbbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:40.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.345+0000 7f5d31b51700 1 --2- 172.21.15.52:0/1356093345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2dfdb0880 0x55d2dfdc0b60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5d24010010 tx=0x7f5d24000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b2357d70e2b18b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:40.353 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.346+0000 7f5d20ff9700 1 -- 172.21.15.52:0/1356093345 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2403b070 con 0x55d2dfdb0880 2023-11-27T15:20:40.353 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.346+0000 7f5d20ff9700 1 -- 172.21.15.52:0/1356093345 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d2401b070 con 0x55d2dfdb0880 2023-11-27T15:20:40.353 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.346+0000 7f5d20ff9700 1 -- 172.21.15.52:0/1356093345 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2400e040 con 0x55d2dfdb0880 2023-11-27T15:20:40.356 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.346+0000 7f5d3ff41b80 1 -- 172.21.15.52:0/1356093345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2dfdb0880 msgr2=0x55d2dfdc0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.356 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.346+0000 7f5d3ff41b80 1 --2- 172.21.15.52:0/1356093345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2dfdb0880 0x55d2dfdc0b60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5d24010010 tx=0x7f5d24000f40 comp rx=0 tx=0).stop 2023-11-27T15:20:40.356 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.346+0000 7f5d3ff41b80 1 -- 172.21.15.52:0/1356093345 shutdown_connections 2023-11-27T15:20:40.356 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.346+0000 7f5d3ff41b80 1 --2- 172.21.15.52:0/1356093345 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2dfdb0880 0x55d2dfdc0b60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.356 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.346+0000 7f5d3ff41b80 1 --2- 172.21.15.52:0/1356093345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dfdb25b0 0x55d2dfdb2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.357 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.346+0000 7f5d3ff41b80 1 --2- 172.21.15.52:0/1356093345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2dfdb4cc0 0x55d2dfdbbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.357 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.346+0000 7f5d3ff41b80 1 -- 172.21.15.52:0/1356093345 >> 172.21.15.52:0/1356093345 conn(0x55d2dfda1360 msgr2=0x55d2dfdc5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:40.357 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.346+0000 7f5d3ff41b80 1 -- 172.21.15.52:0/1356093345 shutdown_connections 2023-11-27T15:20:40.357 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.346+0000 7f5d3ff41b80 1 -- 172.21.15.52:0/1356093345 wait complete. 2023-11-27T15:20:40.357 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.347+0000 7f5d3ff41b80 1 -- start start 2023-11-27T15:20:40.357 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.347+0000 7f5d3ff41b80 1 Processor -- start 2023-11-27T15:20:40.357 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.347+0000 7f5d3ff41b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dfdb0880 0x55d2dfdc0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.357 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.347+0000 7f5d3ff41b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2dfdb25b0 0x55d2dfdb3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.357 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.347+0000 7f5d3ff41b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2dfdb4cc0 0x55d2dfdb4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.357 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.347+0000 7f5d3ff41b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2dfdb4720 con 0x55d2dfdb4cc0 2023-11-27T15:20:40.357 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.347+0000 7f5d3ff41b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2dfdb4890 con 0x55d2dfdb25b0 2023-11-27T15:20:40.358 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.347+0000 7f5d3ff41b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2dfe1d7d0 con 0x55d2dfdb0880 2023-11-27T15:20:40.358 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.347+0000 7f5d3ff41b80 10 client.0 _ll_register_callbacks cb 0x55d2dfcbc780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:20:40.358 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.348+0000 7f5d31b51700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dfdb0880 0x55d2dfdc0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.358 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.348+0000 7f5d31350700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2dfdb25b0 0x55d2dfdb3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.358 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.348+0000 7f5d31b51700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dfdb0880 0x55d2dfdc0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:40214/0 (socket says 172.21.15.52:40214) 2023-11-27T15:20:40.358 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.348+0000 7f5d31b51700 1 -- 172.21.15.52:0/3454118755 learned_addr learned my addr 172.21.15.52:0/3454118755 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:40.358 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.348+0000 7f5d31350700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2dfdb25b0 0x55d2dfdb3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43230/0 (socket says 172.21.15.52:43230) 2023-11-27T15:20:40.358 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.348+0000 7f5d32352700 1 --2- 172.21.15.52:0/3454118755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2dfdb4cc0 0x55d2dfdb4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.358 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.348+0000 7f5d31b51700 1 -- 172.21.15.52:0/3454118755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2dfdb25b0 msgr2=0x55d2dfdb3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:40.358 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.348+0000 7f5d31b51700 1 --2- 172.21.15.52:0/3454118755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2dfdb25b0 0x55d2dfdb3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.358 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.348+0000 7f5d31b51700 1 -- 172.21.15.52:0/3454118755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2dfdb4cc0 msgr2=0x55d2dfdb4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:40.359 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.348+0000 7f5d31b51700 1 --2- 172.21.15.52:0/3454118755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2dfdb4cc0 0x55d2dfdb4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.359 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.348+0000 7f5d31b51700 1 -- 172.21.15.52:0/3454118755 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d2dfb7e630 con 0x55d2dfdb0880 2023-11-27T15:20:40.359 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.348+0000 7f5d31350700 1 --2- 172.21.15.52:0/3454118755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2dfdb25b0 0x55d2dfdb3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:40.359 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.348+0000 7f5d31b51700 1 --2- 172.21.15.52:0/3454118755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dfdb0880 0x55d2dfdc0cd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5d240061e0 tx=0x7f5d240237b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4ec9b547fe47e69 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:40.359 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.348+0000 7f5d157fa700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2023-11-27T15:20:40.359 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.348+0000 7f5d157fa700 1 -- 172.21.15.52:0/3454118755 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2403b070 con 0x55d2dfdb0880 2023-11-27T15:20:40.359 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.348+0000 7f5d157fa700 1 -- 172.21.15.52:0/3454118755 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f5d24023ba0 con 0x55d2dfdb0880 2023-11-27T15:20:40.359 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.348+0000 7f5d3ff41b80 1 -- 172.21.15.52:0/3454118755 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d28022900 con 0x55d2dfdb0880 2023-11-27T15:20:40.359 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.348+0000 7f5d157fa700 1 -- 172.21.15.52:0/3454118755 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d2400e040 con 0x55d2dfdb0880 2023-11-27T15:20:40.359 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.348+0000 7f5d3ff41b80 10 client.25765 fetch_fsmap learned FSMap version 24 2023-11-27T15:20:40.359 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.349+0000 7f5d3ff41b80 1 -- 172.21.15.52:0/3454118755 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d2dfcc3190 con 0x55d2dfdb0880 2023-11-27T15:20:40.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.349+0000 7f5d157fa700 1 -- 172.21.15.52:0/3454118755 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d2404c020 con 0x55d2dfdb0880 2023-11-27T15:20:40.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.349+0000 7f5d157fa700 1 -- 172.21.15.52:0/3454118755 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5d24004ba0 con 0x55d2dfdb0880 2023-11-27T15:20:40.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.349+0000 7f5d3ff41b80 10 client.25765 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:20:40.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.349+0000 7f5d3ff41b80 10 client.25765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:20:40.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.349+0000 7f5d3ff41b80 20 client.25765 populate_metadata read hostname 'smithi052' 2023-11-27T15:20:40.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.349+0000 7f5d3ff41b80 1 --2- 172.21.15.52:0/3454118755 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d2dfe1e9b0 0x55d2dfe20dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.349+0000 7f5d3ff41b80 4 client.25765 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:20:40.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.349+0000 7f5d3ff41b80 1 -- 172.21.15.52:0/3454118755 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d2dfccb660 con 0x55d2dfe1e9b0 2023-11-27T15:20:40.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.349+0000 7f5d31350700 1 --2- 172.21.15.52:0/3454118755 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d2dfe1e9b0 0x55d2dfe20dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.349+0000 7f5d31350700 1 --2- 172.21.15.52:0/3454118755 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d2dfe1e9b0 0x55d2dfe20dc0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef1a0dd9f2a25669 server_cookie=6dec37f126ea37a9 in_seq=0 out_seq=0 2023-11-27T15:20:40.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.349+0000 7f5d157fa700 10 client.25765 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:20:40.362 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.356+0000 7f5d157fa700 1 -- 172.21.15.52:0/3454118755 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d2dfccb660 con 0x55d2dfe1e9b0 2023-11-27T15:20:40.362 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.356+0000 7f5d157fa700 10 client.25765 handle_command_reply: tid=0 2023-11-27T15:20:40.362 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.356+0000 7f5d3ff41b80 1 client.25765 shutdown 2023-11-27T15:20:40.362 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.356+0000 7f5d3ff41b80 1 -- 172.21.15.52:0/3454118755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dfdb0880 msgr2=0x55d2dfdc0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.362 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.356+0000 7f5d3ff41b80 1 --2- 172.21.15.52:0/3454118755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dfdb0880 0x55d2dfdc0cd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5d240061e0 tx=0x7f5d240237b0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.362 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.356+0000 7f5d3ff41b80 1 -- 172.21.15.52:0/3454118755 shutdown_connections 2023-11-27T15:20:40.362 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.356+0000 7f5d3ff41b80 1 --2- 172.21.15.52:0/3454118755 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55d2dfe1e9b0 0x55d2dfe20dc0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.362 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.356+0000 7f5d3ff41b80 1 --2- 172.21.15.52:0/3454118755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2dfdb4cc0 0x55d2dfdb4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.363 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.356+0000 7f5d157fa700 0 client.25765 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:20:40.363 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.356+0000 7f5d3ff41b80 1 --2- 172.21.15.52:0/3454118755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d2dfdb0880 0x55d2dfdc0cd0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.363 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.356+0000 7f5d3ff41b80 1 --2- 172.21.15.52:0/3454118755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2dfdb25b0 0x55d2dfdb3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.363 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.356+0000 7f5d3ff41b80 1 -- 172.21.15.52:0/3454118755 >> 172.21.15.52:0/3454118755 conn(0x55d2dfda1360 msgr2=0x55d2dfdb2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:40.363 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.356+0000 7f5d3ff41b80 1 -- 172.21.15.52:0/3454118755 shutdown_connections 2023-11-27T15:20:40.363 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.356+0000 7f5d3ff41b80 1 -- 172.21.15.52:0/3454118755 wait complete. 2023-11-27T15:20:40.363 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.356+0000 7f5d3ff41b80 20 client.25765 trim_cache size 0 max 16384 2023-11-27T15:20:40.363 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:20:40.364 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.357+0000 7f5d33db5700 1 -- 172.21.15.52:0/1448734356 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f5d180a12c0 msgr2=0x7f5d180a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.364 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.357+0000 7f5d33db5700 1 --2- 172.21.15.52:0/1448734356 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f5d180a12c0 0x7f5d180a3770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5d2400f540 tx=0x7f5d2400b040 comp rx=0 tx=0).stop 2023-11-27T15:20:40.364 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.357+0000 7f5d33db5700 1 -- 172.21.15.52:0/1448734356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d2c100f50 msgr2=0x7f5d2c1969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.365 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.357+0000 7f5d33db5700 1 --2- 172.21.15.52:0/1448734356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d2c100f50 0x7f5d2c1969f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5d28009fd0 tx=0x7f5d2800f690 comp rx=0 tx=0).stop 2023-11-27T15:20:40.365 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.357+0000 7f5d33db5700 1 -- 172.21.15.52:0/1448734356 shutdown_connections 2023-11-27T15:20:40.365 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.357+0000 7f5d33db5700 1 --2- 172.21.15.52:0/1448734356 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f5d180a12c0 0x7f5d180a3770 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.365 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.357+0000 7f5d33db5700 1 --2- 172.21.15.52:0/1448734356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d2c1055f0 0x7f5d2c19ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.365 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.357+0000 7f5d33db5700 1 --2- 172.21.15.52:0/1448734356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d2c1005b0 0x7f5d2c1964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.365 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.357+0000 7f5d33db5700 1 --2- 172.21.15.52:0/1448734356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5d2c100f50 0x7f5d2c1969f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.365 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.357+0000 7f5d33db5700 1 -- 172.21.15.52:0/1448734356 >> 172.21.15.52:0/1448734356 conn(0x7f5d2c077170 msgr2=0x7f5d2c0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:40.365 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.357+0000 7f5d33db5700 1 -- 172.21.15.52:0/1448734356 shutdown_connections 2023-11-27T15:20:40.365 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:40.357+0000 7f5d33db5700 1 -- 172.21.15.52:0/1448734356 wait complete. 2023-11-27T15:20:41.377 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-27T15:20:41.377 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-27T15:20:41.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.509+0000 7fc8f25da700 1 -- 172.21.15.52:0/2872030439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8ec1005b0 msgr2=0x7fc8ec100980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:41.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.509+0000 7fc8f25da700 1 --2- 172.21.15.52:0/2872030439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8ec1005b0 0x7fc8ec100980 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc009990 tx=0x7fc8dc009ca0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.509+0000 7fc8f25da700 1 -- 172.21.15.52:0/2872030439 shutdown_connections 2023-11-27T15:20:41.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.509+0000 7fc8f25da700 1 --2- 172.21.15.52:0/2872030439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8ec1005b0 0x7fc8ec100980 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.509+0000 7fc8f25da700 1 --2- 172.21.15.52:0/2872030439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8ec1055f0 0x7fc8ec10d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.509+0000 7fc8f25da700 1 --2- 172.21.15.52:0/2872030439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8ec100f50 0x7fc8ec104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.509+0000 7fc8f25da700 1 -- 172.21.15.52:0/2872030439 >> 172.21.15.52:0/2872030439 conn(0x7fc8ec077170 msgr2=0x7fc8ec077570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:41.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.509+0000 7fc8f25da700 1 -- 172.21.15.52:0/2872030439 shutdown_connections 2023-11-27T15:20:41.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.509+0000 7fc8f25da700 1 -- 172.21.15.52:0/2872030439 wait complete. 2023-11-27T15:20:41.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.510+0000 7fc8f25da700 1 Processor -- start 2023-11-27T15:20:41.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.510+0000 7fc8f25da700 1 -- start start 2023-11-27T15:20:41.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.510+0000 7fc8f25da700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8ec1005b0 0x7fc8ec1963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.510+0000 7fc8f25da700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8ec100f50 0x7fc8ec196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.510+0000 7fc8f25da700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8ec1055f0 0x7fc8ec19ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.510+0000 7fc8f25da700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec19b3c0 con 0x7fc8ec100f50 2023-11-27T15:20:41.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.510+0000 7fc8f25da700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec19b530 con 0x7fc8ec1005b0 2023-11-27T15:20:41.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.510+0000 7fc8f25da700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec19b6f0 con 0x7fc8ec1055f0 2023-11-27T15:20:41.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.510+0000 7fc8ebfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8ec1005b0 0x7fc8ec1963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.510+0000 7fc8eb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8ec100f50 0x7fc8ec196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.510+0000 7fc8ebfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8ec1005b0 0x7fc8ec1963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43242/0 (socket says 172.21.15.52:43242) 2023-11-27T15:20:41.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.510+0000 7fc8ebfff700 1 -- 172.21.15.52:0/4038252584 learned_addr learned my addr 172.21.15.52:0/4038252584 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:41.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.510+0000 7fc8f0b77700 1 --2- 172.21.15.52:0/4038252584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8ec1055f0 0x7fc8ec19ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.510+0000 7fc8ebfff700 1 -- 172.21.15.52:0/4038252584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8ec1055f0 msgr2=0x7fc8ec19ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:41.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.510+0000 7fc8ebfff700 1 --2- 172.21.15.52:0/4038252584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8ec1055f0 0x7fc8ec19ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.510+0000 7fc8ebfff700 1 -- 172.21.15.52:0/4038252584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8ec100f50 msgr2=0x7fc8ec196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:41.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.510+0000 7fc8ebfff700 1 --2- 172.21.15.52:0/4038252584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8ec100f50 0x7fc8ec196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.510+0000 7fc8ebfff700 1 -- 172.21.15.52:0/4038252584 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8dc0095f0 con 0x7fc8ec1005b0 2023-11-27T15:20:41.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.510+0000 7fc8eb7fe700 1 --2- 172.21.15.52:0/4038252584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8ec100f50 0x7fc8ec196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:41.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.511+0000 7fc8ebfff700 1 --2- 172.21.15.52:0/4038252584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8ec1005b0 0x7fc8ec1963e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc000c00 tx=0x7fc8dc005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:41.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.511+0000 7fc8e97fa700 1 -- 172.21.15.52:0/4038252584 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc01d070 con 0x7fc8ec1005b0 2023-11-27T15:20:41.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.511+0000 7fc8e97fa700 1 -- 172.21.15.52:0/4038252584 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8dc022470 con 0x7fc8ec1005b0 2023-11-27T15:20:41.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.511+0000 7fc8e97fa700 1 -- 172.21.15.52:0/4038252584 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc017650 con 0x7fc8ec1005b0 2023-11-27T15:20:41.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.511+0000 7fc8f25da700 1 -- 172.21.15.52:0/4038252584 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8ec1103b0 con 0x7fc8ec1005b0 2023-11-27T15:20:41.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.511+0000 7fc8f25da700 1 -- 172.21.15.52:0/4038252584 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8ec110220 con 0x7fc8ec1005b0 2023-11-27T15:20:41.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.513+0000 7fc8e97fa700 1 -- 172.21.15.52:0/4038252584 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fc8dc0225e0 con 0x7fc8ec1005b0 2023-11-27T15:20:41.533 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.514+0000 7fc8e97fa700 1 --2- 172.21.15.52:0/4038252584 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fc8d80a1340 0x7fc8d80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.533 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.514+0000 7fc8e97fa700 1 -- 172.21.15.52:0/4038252584 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fc8dc0cdff0 con 0x7fc8ec1005b0 2023-11-27T15:20:41.533 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.514+0000 7fc8eb7fe700 1 --2- 172.21.15.52:0/4038252584 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fc8d80a1340 0x7fc8d80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.533 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.514+0000 7fc8eb7fe700 1 --2- 172.21.15.52:0/4038252584 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fc8d80a1340 0x7fc8d80a37f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec1979f0 tx=0x7fc8d4009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:41.533 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.515+0000 7fc8fe766b80 1 Processor -- start 2023-11-27T15:20:41.533 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.515+0000 7fc8fe766b80 1 -- start start 2023-11-27T15:20:41.533 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.515+0000 7fc8fe766b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565336b1bbe0 0x565336b1bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.534 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.515+0000 7fc8fe766b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565336b1c4f0 0x565336b203f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.534 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.515+0000 7fc8fe766b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565336b20930 0x565336b20d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.534 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.515+0000 7fc8fe766b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x565336a210e0 con 0x565336b1bbe0 2023-11-27T15:20:41.534 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.515+0000 7fc8fe766b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x565336a73920 con 0x565336b20930 2023-11-27T15:20:41.534 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.515+0000 7fc8fe766b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x565336a6cdb0 con 0x565336b1c4f0 2023-11-27T15:20:41.534 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.515+0000 7fc8ebfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565336b1bbe0 0x565336b1bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.515+0000 7fc8ebfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565336b1bbe0 0x565336b1bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53592/0 (socket says 172.21.15.52:53592) 2023-11-27T15:20:41.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.515+0000 7fc8ebfff700 1 -- 172.21.15.52:0/3523228919 learned_addr learned my addr 172.21.15.52:0/3523228919 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:41.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.515+0000 7fc8f0b77700 1 --2- 172.21.15.52:0/3523228919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565336b20930 0x565336b20d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.515+0000 7fc8eb7fe700 1 --2- 172.21.15.52:0/3523228919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565336b1c4f0 0x565336b203f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.515+0000 7fc8ebfff700 1 -- 172.21.15.52:0/3523228919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565336b1c4f0 msgr2=0x565336b203f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:41.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.515+0000 7fc8ebfff700 1 --2- 172.21.15.52:0/3523228919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565336b1c4f0 0x565336b203f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.515+0000 7fc8ebfff700 1 -- 172.21.15.52:0/3523228919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565336b20930 msgr2=0x565336b20d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:41.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.515+0000 7fc8ebfff700 1 --2- 172.21.15.52:0/3523228919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565336b20930 0x565336b20d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.515+0000 7fc8ebfff700 1 -- 172.21.15.52:0/3523228919 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653368ee6b0 con 0x565336b1bbe0 2023-11-27T15:20:41.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.515+0000 7fc8f0b77700 1 --2- 172.21.15.52:0/3523228919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565336b20930 0x565336b20d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:41.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.516+0000 7fc8ebfff700 1 --2- 172.21.15.52:0/3523228919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565336b1bbe0 0x565336b1bfb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc097650 tx=0x7fc8dc09ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace822ca44ea2af0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:41.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.516+0000 7fc8e27fc700 1 -- 172.21.15.52:0/3523228919 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc09f180 con 0x565336b1bbe0 2023-11-27T15:20:41.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.516+0000 7fc8e27fc700 1 -- 172.21.15.52:0/3523228919 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8dc0cbe90 con 0x565336b1bbe0 2023-11-27T15:20:41.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.516+0000 7fc8e27fc700 1 -- 172.21.15.52:0/3523228919 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc09a640 con 0x565336b1bbe0 2023-11-27T15:20:41.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.516+0000 7fc8fe766b80 1 -- 172.21.15.52:0/3523228919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565336b1bbe0 msgr2=0x565336b1bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:41.537 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.516+0000 7fc8fe766b80 1 --2- 172.21.15.52:0/3523228919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565336b1bbe0 0x565336b1bfb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc097650 tx=0x7fc8dc09ad80 comp rx=0 tx=0).stop 2023-11-27T15:20:41.537 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.516+0000 7fc8fe766b80 1 -- 172.21.15.52:0/3523228919 shutdown_connections 2023-11-27T15:20:41.537 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.517+0000 7fc8fe766b80 1 --2- 172.21.15.52:0/3523228919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565336b20930 0x565336b20d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.537 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.517+0000 7fc8fe766b80 1 --2- 172.21.15.52:0/3523228919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565336b1bbe0 0x565336b1bfb0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.537 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.517+0000 7fc8fe766b80 1 --2- 172.21.15.52:0/3523228919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565336b1c4f0 0x565336b203f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.537 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.517+0000 7fc8fe766b80 1 -- 172.21.15.52:0/3523228919 >> 172.21.15.52:0/3523228919 conn(0x565336b113e0 msgr2=0x565336b16bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:41.537 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.517+0000 7fc8fe766b80 1 -- 172.21.15.52:0/3523228919 shutdown_connections 2023-11-27T15:20:41.537 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.517+0000 7fc8fe766b80 1 -- 172.21.15.52:0/3523228919 wait complete. 2023-11-27T15:20:41.537 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.517+0000 7fc8fe766b80 1 -- start start 2023-11-27T15:20:41.538 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.517+0000 7fc8fe766b80 1 Processor -- start 2023-11-27T15:20:41.538 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.517+0000 7fc8fe766b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565336b1bbe0 0x565336b28020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.538 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.517+0000 7fc8fe766b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565336b1c4f0 0x565336b25230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.538 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.517+0000 7fc8fe766b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565336b20930 0x565336b257b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.538 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.517+0000 7fc8fe766b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x565336a0e960 con 0x565336b20930 2023-11-27T15:20:41.538 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.517+0000 7fc8fe766b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x565336a0e600 con 0x565336b1bbe0 2023-11-27T15:20:41.538 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.517+0000 7fc8fe766b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653368f09a0 con 0x565336b1c4f0 2023-11-27T15:20:41.539 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.517+0000 7fc8fe766b80 10 client.0 _ll_register_callbacks cb 0x565336a2c800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:20:41.539 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.518+0000 7fc8f0b77700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565336b20930 0x565336b257b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.539 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.518+0000 7fc8ebfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565336b1bbe0 0x565336b28020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.539 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.518+0000 7fc8f0b77700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565336b20930 0x565336b257b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53602/0 (socket says 172.21.15.52:53602) 2023-11-27T15:20:41.539 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.518+0000 7fc8ebfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565336b1bbe0 0x565336b28020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43256/0 (socket says 172.21.15.52:43256) 2023-11-27T15:20:41.539 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.518+0000 7fc8f0b77700 1 -- 172.21.15.52:0/469475079 learned_addr learned my addr 172.21.15.52:0/469475079 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:41.539 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.518+0000 7fc8eb7fe700 1 --2- 172.21.15.52:0/469475079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565336b1c4f0 0x565336b25230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.539 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.518+0000 7fc8f0b77700 1 -- 172.21.15.52:0/469475079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565336b1c4f0 msgr2=0x565336b25230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:41.539 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.518+0000 7fc8f0b77700 1 --2- 172.21.15.52:0/469475079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565336b1c4f0 0x565336b25230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.540 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.518+0000 7fc8f0b77700 1 -- 172.21.15.52:0/469475079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565336b1bbe0 msgr2=0x565336b28020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:41.540 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.518+0000 7fc8f0b77700 1 --2- 172.21.15.52:0/469475079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565336b1bbe0 0x565336b28020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.540 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.518+0000 7fc8f0b77700 1 -- 172.21.15.52:0/469475079 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8ec110220 con 0x565336b20930 2023-11-27T15:20:41.540 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.518+0000 7fc8ebfff700 1 --2- 172.21.15.52:0/469475079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565336b1bbe0 0x565336b28020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:41.540 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.518+0000 7fc8f0b77700 1 --2- 172.21.15.52:0/469475079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565336b20930 0x565336b257b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4011f70 tx=0x7fc8e4014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2a063d19596200 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:41.540 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.518+0000 7fc8cb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:20:41.540 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.518+0000 7fc8cb7fe700 1 -- 172.21.15.52:0/469475079 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e400e040 con 0x565336b20930 2023-11-27T15:20:41.541 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.518+0000 7fc8fe766b80 1 -- 172.21.15.52:0/469475079 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565336b0eba0 con 0x565336b20930 2023-11-27T15:20:41.541 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.518+0000 7fc8cb7fe700 1 -- 172.21.15.52:0/469475079 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fc8e401a070 con 0x565336b20930 2023-11-27T15:20:41.541 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.518+0000 7fc8cb7fe700 1 -- 172.21.15.52:0/469475079 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8e4014ba0 con 0x565336b20930 2023-11-27T15:20:41.541 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.518+0000 7fc8fe766b80 10 client.16068 fetch_fsmap learned FSMap version 24 2023-11-27T15:20:41.541 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.518+0000 7fc8fe766b80 1 -- 172.21.15.52:0/469475079 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x565336a33210 con 0x565336b20930 2023-11-27T15:20:41.541 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.519+0000 7fc8cb7fe700 1 -- 172.21.15.52:0/469475079 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x565336b0eba0 con 0x565336b20930 2023-11-27T15:20:41.541 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.519+0000 7fc8cb7fe700 1 -- 172.21.15.52:0/469475079 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc8e40043d0 con 0x565336b20930 2023-11-27T15:20:41.541 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.519+0000 7fc8fe766b80 10 client.16068 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:20:41.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.519+0000 7fc8fe766b80 10 client.16068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:20:41.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.519+0000 7fc8fe766b80 20 client.16068 populate_metadata read hostname 'smithi052' 2023-11-27T15:20:41.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.519+0000 7fc8fe766b80 1 --2- 172.21.15.52:0/469475079 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x565336b35670 0x565336b37a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.519+0000 7fc8fe766b80 4 client.16068 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:20:41.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.519+0000 7fc8fe766b80 1 -- 172.21.15.52:0/469475079 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565336a2f120 con 0x565336b35670 2023-11-27T15:20:41.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.519+0000 7fc8ebfff700 1 --2- 172.21.15.52:0/469475079 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x565336b35670 0x565336b37a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.519+0000 7fc8cb7fe700 10 client.16068 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:20:41.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.519+0000 7fc8ebfff700 1 --2- 172.21.15.52:0/469475079 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x565336b35670 0x565336b37a80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c333228e977a4168 server_cookie=4f26d85fc1f8176 in_seq=0 out_seq=0 2023-11-27T15:20:41.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.521+0000 7fc8cb7fe700 1 -- 172.21.15.52:0/469475079 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x565336a2f120 con 0x565336b35670 2023-11-27T15:20:41.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.521+0000 7fc8cb7fe700 10 client.16068 handle_command_reply: tid=0 2023-11-27T15:20:41.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.521+0000 7fc8fe766b80 1 client.16068 shutdown 2023-11-27T15:20:41.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.522+0000 7fc8fe766b80 1 -- 172.21.15.52:0/469475079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565336b20930 msgr2=0x565336b257b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:41.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.522+0000 7fc8fe766b80 1 --2- 172.21.15.52:0/469475079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565336b20930 0x565336b257b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4011f70 tx=0x7fc8e4014600 comp rx=0 tx=0).stop 2023-11-27T15:20:41.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.522+0000 7fc8fe766b80 1 -- 172.21.15.52:0/469475079 shutdown_connections 2023-11-27T15:20:41.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.522+0000 7fc8fe766b80 1 --2- 172.21.15.52:0/469475079 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x565336b35670 0x565336b37a80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.522+0000 7fc8fe766b80 1 --2- 172.21.15.52:0/469475079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565336b1bbe0 0x565336b28020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.522+0000 7fc8fe766b80 1 --2- 172.21.15.52:0/469475079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565336b20930 0x565336b257b0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.522+0000 7fc8fe766b80 1 --2- 172.21.15.52:0/469475079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565336b1c4f0 0x565336b25230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.522+0000 7fc8cb7fe700 0 client.16068 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:20:41.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.522+0000 7fc8fe766b80 1 -- 172.21.15.52:0/469475079 >> 172.21.15.52:0/469475079 conn(0x565336b113e0 msgr2=0x565336b21de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:41.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.522+0000 7fc8fe766b80 1 -- 172.21.15.52:0/469475079 shutdown_connections 2023-11-27T15:20:41.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.522+0000 7fc8fe766b80 1 -- 172.21.15.52:0/469475079 wait complete. 2023-11-27T15:20:41.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.522+0000 7fc8fe766b80 20 client.16068 trim_cache size 0 max 16384 2023-11-27T15:20:41.552 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.540+0000 7fc8fe766b80 1 Processor -- start 2023-11-27T15:20:41.552 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.540+0000 7fc8fe766b80 1 -- start start 2023-11-27T15:20:41.552 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.541+0000 7fc8fe766b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565336b1bbe0 0x565336b27b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.552 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.541+0000 7fc8fe766b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565336b1c4f0 0x565336b166d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.552 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.541+0000 7fc8fe766b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565336b20930 0x565336b16c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.552 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.541+0000 7fc8fe766b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x565336b17150 con 0x565336b1bbe0 2023-11-27T15:20:41.552 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.541+0000 7fc8fe766b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x565336b32c70 con 0x565336b1c4f0 2023-11-27T15:20:41.552 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.541+0000 7fc8fe766b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x565336b31fb0 con 0x565336b20930 2023-11-27T15:20:41.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.541+0000 7fc8ebfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565336b1bbe0 0x565336b27b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.541+0000 7fc8ebfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565336b1bbe0 0x565336b27b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53606/0 (socket says 172.21.15.52:53606) 2023-11-27T15:20:41.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.541+0000 7fc8ebfff700 1 -- 172.21.15.52:0/3880836739 learned_addr learned my addr 172.21.15.52:0/3880836739 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:41.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.541+0000 7fc8eb7fe700 1 --2- 172.21.15.52:0/3880836739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565336b1c4f0 0x565336b166d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.541+0000 7fc8f0b77700 1 --2- 172.21.15.52:0/3880836739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565336b20930 0x565336b16c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.541+0000 7fc8ebfff700 1 -- 172.21.15.52:0/3880836739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565336b20930 msgr2=0x565336b16c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:41.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.541+0000 7fc8ebfff700 1 --2- 172.21.15.52:0/3880836739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565336b20930 0x565336b16c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.541+0000 7fc8ebfff700 1 -- 172.21.15.52:0/3880836739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565336b1c4f0 msgr2=0x565336b166d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:41.554 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.541+0000 7fc8ebfff700 1 --2- 172.21.15.52:0/3880836739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565336b1c4f0 0x565336b166d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.554 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.541+0000 7fc8ebfff700 1 -- 172.21.15.52:0/3880836739 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565336b12b80 con 0x565336b1bbe0 2023-11-27T15:20:41.554 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.541+0000 7fc8f0b77700 1 --2- 172.21.15.52:0/3880836739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565336b20930 0x565336b16c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:20:41.554 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.541+0000 7fc8eb7fe700 1 --2- 172.21.15.52:0/3880836739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565336b1c4f0 0x565336b166d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:41.554 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.541+0000 7fc8ebfff700 1 --2- 172.21.15.52:0/3880836739 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565336b1bbe0 0x565336b27b50 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0cb7f0 tx=0x7fc8dc022c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79741d15869b282 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:41.554 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.542+0000 7fc8e2ffd700 1 -- 172.21.15.52:0/3880836739 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc09f180 con 0x565336b1bbe0 2023-11-27T15:20:41.554 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.542+0000 7fc8e2ffd700 1 -- 172.21.15.52:0/3880836739 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8dc09a640 con 0x565336b1bbe0 2023-11-27T15:20:41.554 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.542+0000 7fc8e2ffd700 1 -- 172.21.15.52:0/3880836739 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc0a32e0 con 0x565336b1bbe0 2023-11-27T15:20:41.555 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.542+0000 7fc8fe766b80 1 -- 172.21.15.52:0/3880836739 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565336b1bbe0 msgr2=0x565336b27b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:41.555 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.542+0000 7fc8fe766b80 1 --2- 172.21.15.52:0/3880836739 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565336b1bbe0 0x565336b27b50 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0cb7f0 tx=0x7fc8dc022c80 comp rx=0 tx=0).stop 2023-11-27T15:20:41.555 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.542+0000 7fc8fe766b80 1 -- 172.21.15.52:0/3880836739 shutdown_connections 2023-11-27T15:20:41.555 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.542+0000 7fc8fe766b80 1 --2- 172.21.15.52:0/3880836739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565336b20930 0x565336b16c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.555 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.542+0000 7fc8fe766b80 1 --2- 172.21.15.52:0/3880836739 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565336b1bbe0 0x565336b27b50 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.555 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.542+0000 7fc8fe766b80 1 --2- 172.21.15.52:0/3880836739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565336b1c4f0 0x565336b166d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.555 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.542+0000 7fc8fe766b80 1 -- 172.21.15.52:0/3880836739 >> 172.21.15.52:0/3880836739 conn(0x565336b113e0 msgr2=0x565336b1e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:41.555 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.542+0000 7fc8fe766b80 1 -- 172.21.15.52:0/3880836739 shutdown_connections 2023-11-27T15:20:41.555 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.542+0000 7fc8fe766b80 1 -- 172.21.15.52:0/3880836739 wait complete. 2023-11-27T15:20:41.556 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.543+0000 7fc8fe766b80 1 -- start start 2023-11-27T15:20:41.556 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.543+0000 7fc8fe766b80 1 Processor -- start 2023-11-27T15:20:41.556 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.543+0000 7fc8fe766b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565336b1bbe0 0x565336b2c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.556 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.543+0000 7fc8fe766b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565336b1c4f0 0x565336b2a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.556 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.543+0000 7fc8fe766b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565336b20930 0x565336b2aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.556 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.543+0000 7fc8fe766b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x565336b2b400 con 0x565336b1c4f0 2023-11-27T15:20:41.556 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.543+0000 7fc8fe766b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x565336b2b570 con 0x565336b20930 2023-11-27T15:20:41.556 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.543+0000 7fc8fe766b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x565336b2b730 con 0x565336b1bbe0 2023-11-27T15:20:41.556 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.543+0000 7fc8fe766b80 10 client.0 _ll_register_callbacks cb 0x7fc8e40118a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:20:41.557 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.543+0000 7fc8eb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565336b1c4f0 0x565336b2a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.557 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.543+0000 7fc8eb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565336b1c4f0 0x565336b2a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53614/0 (socket says 172.21.15.52:53614) 2023-11-27T15:20:41.557 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.543+0000 7fc8eb7fe700 1 -- 172.21.15.52:0/481760552 learned_addr learned my addr 172.21.15.52:0/481760552 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:41.557 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.543+0000 7fc8f0b77700 1 --2- 172.21.15.52:0/481760552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565336b20930 0x565336b2aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.557 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.544+0000 7fc8ebfff700 1 --2- 172.21.15.52:0/481760552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565336b1bbe0 0x565336b2c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.557 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.544+0000 7fc8eb7fe700 1 -- 172.21.15.52:0/481760552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565336b1bbe0 msgr2=0x565336b2c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:41.557 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.544+0000 7fc8eb7fe700 1 --2- 172.21.15.52:0/481760552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565336b1bbe0 0x565336b2c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.557 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.544+0000 7fc8eb7fe700 1 -- 172.21.15.52:0/481760552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565336b20930 msgr2=0x565336b2aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:41.558 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.544+0000 7fc8eb7fe700 1 --2- 172.21.15.52:0/481760552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565336b20930 0x565336b2aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.558 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.544+0000 7fc8eb7fe700 1 -- 172.21.15.52:0/481760552 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d4017070 con 0x565336b1c4f0 2023-11-27T15:20:41.558 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.544+0000 7fc8ebfff700 1 --2- 172.21.15.52:0/481760552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565336b1bbe0 0x565336b2c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:20:41.558 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.544+0000 7fc8f0b77700 1 --2- 172.21.15.52:0/481760552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565336b20930 0x565336b2aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:41.558 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.544+0000 7fc8eb7fe700 1 --2- 172.21.15.52:0/481760552 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565336b1c4f0 0x565336b2a980 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec06a370 tx=0x7fc8d40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f5335066256e2f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:41.558 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.544+0000 7fc8cb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:20:41.558 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.544+0000 7fc8cb7fe700 1 -- 172.21.15.52:0/481760552 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d401e6b0 con 0x565336b1c4f0 2023-11-27T15:20:41.558 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.544+0000 7fc8cb7fe700 1 -- 172.21.15.52:0/481760552 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fc8d401e810 con 0x565336b1c4f0 2023-11-27T15:20:41.559 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.544+0000 7fc8fe766b80 1 -- 172.21.15.52:0/481760552 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc8dc0cbe90 con 0x565336b1c4f0 2023-11-27T15:20:41.559 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.544+0000 7fc8cb7fe700 1 -- 172.21.15.52:0/481760552 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8d401e9d0 con 0x565336b1c4f0 2023-11-27T15:20:41.559 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.544+0000 7fc8fe766b80 10 client.16074 fetch_fsmap learned FSMap version 24 2023-11-27T15:20:41.559 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.544+0000 7fc8fe766b80 1 -- 172.21.15.52:0/481760552 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5653368ee6b0 con 0x565336b1c4f0 2023-11-27T15:20:41.559 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.544+0000 7fc8cb7fe700 1 -- 172.21.15.52:0/481760552 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8d4036020 con 0x565336b1c4f0 2023-11-27T15:20:41.559 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.545+0000 7fc8cb7fe700 1 -- 172.21.15.52:0/481760552 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc8d4029ce0 con 0x565336b1c4f0 2023-11-27T15:20:41.559 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.545+0000 7fc8fe766b80 10 client.16074 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:20:41.559 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.545+0000 7fc8fe766b80 10 client.16074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:20:41.559 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.545+0000 7fc8fe766b80 20 client.16074 populate_metadata read hostname 'smithi052' 2023-11-27T15:20:41.560 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.545+0000 7fc8fe766b80 1 --2- 172.21.15.52:0/481760552 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x565336b8aa70 0x565336b8ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.560 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.545+0000 7fc8fe766b80 4 client.16074 mds_command: new command op to 24536 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-27T15:20:41.560 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.545+0000 7fc8fe766b80 1 -- 172.21.15.52:0/481760552 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x565336a3b6e0 con 0x565336b8aa70 2023-11-27T15:20:41.560 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.545+0000 7fc8ebfff700 1 --2- 172.21.15.52:0/481760552 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x565336b8aa70 0x565336b8ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.560 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.545+0000 7fc8ebfff700 1 --2- 172.21.15.52:0/481760552 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x565336b8aa70 0x565336b8ce80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1ed244968212f82 server_cookie=11f7a06d68acbc3c in_seq=0 out_seq=0 2023-11-27T15:20:41.560 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.545+0000 7fc8cb7fe700 10 client.16074 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:20:41.579 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.573+0000 7fc8cb7fe700 1 -- 172.21.15.52:0/481760552 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x565336a3b6e0 con 0x565336b8aa70 2023-11-27T15:20:41.579 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.573+0000 7fc8cb7fe700 10 client.16074 handle_command_reply: tid=0 2023-11-27T15:20:41.579 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.573+0000 7fc8fe766b80 1 client.16074 shutdown 2023-11-27T15:20:41.579 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.573+0000 7fc8fe766b80 1 -- 172.21.15.52:0/481760552 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565336b1c4f0 msgr2=0x565336b2a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:41.579 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.573+0000 7fc8fe766b80 1 --2- 172.21.15.52:0/481760552 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565336b1c4f0 0x565336b2a980 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec06a370 tx=0x7fc8d40047c0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.579 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.573+0000 7fc8fe766b80 1 -- 172.21.15.52:0/481760552 shutdown_connections 2023-11-27T15:20:41.579 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.573+0000 7fc8fe766b80 1 --2- 172.21.15.52:0/481760552 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x565336b8aa70 0x565336b8ce80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.580 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.573+0000 7fc8fe766b80 1 --2- 172.21.15.52:0/481760552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565336b20930 0x565336b2aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.580 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.573+0000 7fc8cb7fe700 0 client.16074 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:20:41.580 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.573+0000 7fc8fe766b80 1 --2- 172.21.15.52:0/481760552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x565336b1bbe0 0x565336b2c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.580 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.573+0000 7fc8fe766b80 1 --2- 172.21.15.52:0/481760552 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565336b1c4f0 0x565336b2a980 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.580 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.573+0000 7fc8fe766b80 1 -- 172.21.15.52:0/481760552 >> 172.21.15.52:0/481760552 conn(0x565336b113e0 msgr2=0x565336b1b270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:41.580 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.573+0000 7fc8fe766b80 1 -- 172.21.15.52:0/481760552 shutdown_connections 2023-11-27T15:20:41.580 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.573+0000 7fc8fe766b80 1 -- 172.21.15.52:0/481760552 wait complete. 2023-11-27T15:20:41.580 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.573+0000 7fc8fe766b80 20 client.16074 trim_cache size 0 max 16384 2023-11-27T15:20:41.581 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:20:41.581 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2023-11-27T15:20:41.581 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "1a14f367-9a76-4e86-b4c8-bd13c3d4d297", 2023-11-27T15:20:41.581 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2023-11-27T15:20:41.581 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:20:41.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.574+0000 7fc8f25da700 1 -- 172.21.15.52:0/4038252584 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fc8d80a1340 msgr2=0x7fc8d80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:41.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.574+0000 7fc8f25da700 1 --2- 172.21.15.52:0/4038252584 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fc8d80a1340 0x7fc8d80a37f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec1979f0 tx=0x7fc8d4009260 comp rx=0 tx=0).stop 2023-11-27T15:20:41.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.574+0000 7fc8f25da700 1 -- 172.21.15.52:0/4038252584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8ec1005b0 msgr2=0x7fc8ec1963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:41.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.574+0000 7fc8f25da700 1 --2- 172.21.15.52:0/4038252584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8ec1005b0 0x7fc8ec1963e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc000c00 tx=0x7fc8dc005740 comp rx=0 tx=0).stop 2023-11-27T15:20:41.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.574+0000 7fc8f25da700 1 -- 172.21.15.52:0/4038252584 shutdown_connections 2023-11-27T15:20:41.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.574+0000 7fc8f25da700 1 --2- 172.21.15.52:0/4038252584 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fc8d80a1340 0x7fc8d80a37f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.574+0000 7fc8f25da700 1 --2- 172.21.15.52:0/4038252584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8ec1005b0 0x7fc8ec1963e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.574+0000 7fc8f25da700 1 --2- 172.21.15.52:0/4038252584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8ec1055f0 0x7fc8ec19ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.574+0000 7fc8f25da700 1 --2- 172.21.15.52:0/4038252584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8ec100f50 0x7fc8ec196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.574+0000 7fc8f25da700 1 -- 172.21.15.52:0/4038252584 >> 172.21.15.52:0/4038252584 conn(0x7fc8ec077170 msgr2=0x7fc8ec103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:41.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.574+0000 7fc8f25da700 1 -- 172.21.15.52:0/4038252584 shutdown_connections 2023-11-27T15:20:41.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.574+0000 7fc8f25da700 1 -- 172.21.15.52:0/4038252584 wait complete. 2023-11-27T15:20:41.594 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:20:41.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.724+0000 7f167bf9b700 1 -- 172.21.15.52:0/3554576557 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16740ff450 msgr2=0x7f16740ff820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:41.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.724+0000 7f167bf9b700 1 --2- 172.21.15.52:0/3554576557 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16740ff450 0x7f16740ff820 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f16640099c0 tx=0x7f1664009cd0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.724+0000 7f167bf9b700 1 -- 172.21.15.52:0/3554576557 shutdown_connections 2023-11-27T15:20:41.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.724+0000 7f167bf9b700 1 --2- 172.21.15.52:0/3554576557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1674104490 0x7f167410c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.724+0000 7f167bf9b700 1 --2- 172.21.15.52:0/3554576557 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16740ff450 0x7f16740ff820 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.724+0000 7f167bf9b700 1 --2- 172.21.15.52:0/3554576557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16740ffdf0 0x7f1674103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.724+0000 7f167bf9b700 1 -- 172.21.15.52:0/3554576557 >> 172.21.15.52:0/3554576557 conn(0x7f16740770b0 msgr2=0x7f16740774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:41.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.724+0000 7f167bf9b700 1 -- 172.21.15.52:0/3554576557 shutdown_connections 2023-11-27T15:20:41.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.724+0000 7f167bf9b700 1 -- 172.21.15.52:0/3554576557 wait complete. 2023-11-27T15:20:41.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.725+0000 7f167bf9b700 1 Processor -- start 2023-11-27T15:20:41.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.725+0000 7f167bf9b700 1 -- start start 2023-11-27T15:20:41.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.725+0000 7f167bf9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16740ff450 0x7f16741952d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.725+0000 7f167bf9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16740ffdf0 0x7f1674195810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.725+0000 7f167bf9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1674104490 0x7f1674199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.725+0000 7f167bf9b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167419a2b0 con 0x7f1674104490 2023-11-27T15:20:41.735 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.725+0000 7f167bf9b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167419a420 con 0x7f16740ffdf0 2023-11-27T15:20:41.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.725+0000 7f167bf9b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167419a5e0 con 0x7f16740ff450 2023-11-27T15:20:41.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.725+0000 7f1679d37700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16740ff450 0x7f16741952d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.725+0000 7f167a538700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1674104490 0x7f1674199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.725+0000 7f1679d37700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16740ff450 0x7f16741952d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:40270/0 (socket says 172.21.15.52:40270) 2023-11-27T15:20:41.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.725+0000 7f1679d37700 1 -- 172.21.15.52:0/183105109 learned_addr learned my addr 172.21.15.52:0/183105109 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:41.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.725+0000 7f1679536700 1 --2- 172.21.15.52:0/183105109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16740ffdf0 0x7f1674195810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.726+0000 7f167a538700 1 -- 172.21.15.52:0/183105109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16740ff450 msgr2=0x7f16741952d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:41.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.726+0000 7f167a538700 1 --2- 172.21.15.52:0/183105109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16740ff450 0x7f16741952d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.726+0000 7f167a538700 1 -- 172.21.15.52:0/183105109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16740ffdf0 msgr2=0x7f1674195810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:41.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.726+0000 7f167a538700 1 --2- 172.21.15.52:0/183105109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16740ffdf0 0x7f1674195810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.726+0000 7f167a538700 1 -- 172.21.15.52:0/183105109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16640095f0 con 0x7f1674104490 2023-11-27T15:20:41.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.726+0000 7f1679d37700 1 --2- 172.21.15.52:0/183105109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16740ff450 0x7f16741952d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:41.738 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.726+0000 7f1679536700 1 --2- 172.21.15.52:0/183105109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16740ffdf0 0x7f1674195810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:41.738 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.726+0000 7f167a538700 1 --2- 172.21.15.52:0/183105109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1674104490 0x7f1674199c70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f166c007f10 tx=0x7f166c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:41.738 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.726+0000 7f166affd700 1 -- 172.21.15.52:0/183105109 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c004db0 con 0x7f1674104490 2023-11-27T15:20:41.738 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.726+0000 7f167bf9b700 1 -- 172.21.15.52:0/183105109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f167410f250 con 0x7f1674104490 2023-11-27T15:20:41.738 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.726+0000 7f166affd700 1 -- 172.21.15.52:0/183105109 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f166c00f040 con 0x7f1674104490 2023-11-27T15:20:41.738 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.726+0000 7f166affd700 1 -- 172.21.15.52:0/183105109 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c005990 con 0x7f1674104490 2023-11-27T15:20:41.738 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.726+0000 7f167bf9b700 1 -- 172.21.15.52:0/183105109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f167410f0c0 con 0x7f1674104490 2023-11-27T15:20:41.738 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.728+0000 7f166affd700 1 -- 172.21.15.52:0/183105109 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f166c00db20 con 0x7f1674104490 2023-11-27T15:20:41.746 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.729+0000 7f166affd700 1 --2- 172.21.15.52:0/183105109 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f16600a12c0 0x7f16600a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.746 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.729+0000 7f166affd700 1 -- 172.21.15.52:0/183105109 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f166c0cc540 con 0x7f1674104490 2023-11-27T15:20:41.746 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.729+0000 7f1679d37700 1 --2- 172.21.15.52:0/183105109 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f16600a12c0 0x7f16600a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.746 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.729+0000 7f1679d37700 1 --2- 172.21.15.52:0/183105109 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f16600a12c0 0x7f16600a3770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1664004800 tx=0x7f166401c040 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:41.746 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.730+0000 7f1688127b80 1 Processor -- start 2023-11-27T15:20:41.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.730+0000 7f1688127b80 1 -- start start 2023-11-27T15:20:41.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.730+0000 7f1688127b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e54ffe6b60 0x55e54ffe6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.730+0000 7f1688127b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e54ffe7470 0x55e54ffeb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.730+0000 7f1688127b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e54ffeb8b0 0x55e54ffebd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.730+0000 7f1688127b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54feec060 con 0x55e54ffe7470 2023-11-27T15:20:41.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.730+0000 7f1688127b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54ff3e8a0 con 0x55e54ffe6b60 2023-11-27T15:20:41.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.730+0000 7f1688127b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54ff37d30 con 0x55e54ffeb8b0 2023-11-27T15:20:41.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.730+0000 7f1679d37700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e54ffe6b60 0x55e54ffe6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.730+0000 7f1679536700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e54ffe7470 0x55e54ffeb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.730+0000 7f1679d37700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e54ffe6b60 0x55e54ffe6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43292/0 (socket says 172.21.15.52:43292) 2023-11-27T15:20:41.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.730+0000 7f1679536700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e54ffe7470 0x55e54ffeb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53636/0 (socket says 172.21.15.52:53636) 2023-11-27T15:20:41.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.730+0000 7f1679d37700 1 -- 172.21.15.52:0/4269880293 learned_addr learned my addr 172.21.15.52:0/4269880293 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:41.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.730+0000 7f167a538700 1 --2- 172.21.15.52:0/4269880293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e54ffeb8b0 0x55e54ffebd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.731+0000 7f1679536700 1 -- 172.21.15.52:0/4269880293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e54ffeb8b0 msgr2=0x55e54ffebd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:41.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.731+0000 7f1679536700 1 --2- 172.21.15.52:0/4269880293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e54ffeb8b0 0x55e54ffebd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.731+0000 7f1679536700 1 -- 172.21.15.52:0/4269880293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e54ffe6b60 msgr2=0x55e54ffe6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:41.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.731+0000 7f1679536700 1 --2- 172.21.15.52:0/4269880293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e54ffe6b60 0x55e54ffe6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.731+0000 7f1679536700 1 -- 172.21.15.52:0/4269880293 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e54fdb9630 con 0x55e54ffe7470 2023-11-27T15:20:41.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.731+0000 7f1679d37700 1 --2- 172.21.15.52:0/4269880293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e54ffe6b60 0x55e54ffe6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:41.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.731+0000 7f1679536700 1 --2- 172.21.15.52:0/4269880293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e54ffe7470 0x55e54ffeb370 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f167406a4a0 tx=0x7f16700117a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3426eff3000b6e09 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:41.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.731+0000 7f165ffff700 1 -- 172.21.15.52:0/4269880293 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1670004230 con 0x55e54ffe7470 2023-11-27T15:20:41.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.731+0000 7f165ffff700 1 -- 172.21.15.52:0/4269880293 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1670004cf0 con 0x55e54ffe7470 2023-11-27T15:20:41.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.731+0000 7f165ffff700 1 -- 172.21.15.52:0/4269880293 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16700056c0 con 0x55e54ffe7470 2023-11-27T15:20:41.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.732+0000 7f1688127b80 1 -- 172.21.15.52:0/4269880293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e54ffe7470 msgr2=0x55e54ffeb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:41.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.732+0000 7f1688127b80 1 --2- 172.21.15.52:0/4269880293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e54ffe7470 0x55e54ffeb370 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f167406a4a0 tx=0x7f16700117a0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.732+0000 7f1688127b80 1 -- 172.21.15.52:0/4269880293 shutdown_connections 2023-11-27T15:20:41.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.732+0000 7f1688127b80 1 --2- 172.21.15.52:0/4269880293 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e54ffe6b60 0x55e54ffe6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.732+0000 7f1688127b80 1 --2- 172.21.15.52:0/4269880293 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e54ffeb8b0 0x55e54ffebd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.732+0000 7f1688127b80 1 --2- 172.21.15.52:0/4269880293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e54ffe7470 0x55e54ffeb370 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.732+0000 7f1688127b80 1 -- 172.21.15.52:0/4269880293 >> 172.21.15.52:0/4269880293 conn(0x55e54ffdc360 msgr2=0x55e54ffe1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:41.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.732+0000 7f1688127b80 1 -- 172.21.15.52:0/4269880293 shutdown_connections 2023-11-27T15:20:41.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.732+0000 7f1688127b80 1 -- 172.21.15.52:0/4269880293 wait complete. 2023-11-27T15:20:41.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.732+0000 7f1688127b80 1 -- start start 2023-11-27T15:20:41.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.732+0000 7f1688127b80 1 Processor -- start 2023-11-27T15:20:41.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.733+0000 7f1688127b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e54ffe6b60 0x55e54fff30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.733+0000 7f1688127b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e54ffe7470 0x55e54fff03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.733+0000 7f1688127b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e54ffeb8b0 0x55e54fff0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.733+0000 7f1688127b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54fed98e0 con 0x55e54ffeb8b0 2023-11-27T15:20:41.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.733+0000 7f1688127b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54fed9580 con 0x55e54ffe6b60 2023-11-27T15:20:41.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.733+0000 7f1688127b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54fdbb920 con 0x55e54ffe7470 2023-11-27T15:20:41.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.733+0000 7f1688127b80 10 client.0 _ll_register_callbacks cb 0x55e54fef7780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:20:41.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.733+0000 7f1679d37700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e54ffe6b60 0x55e54fff30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.733+0000 7f167a538700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e54ffeb8b0 0x55e54fff0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.733+0000 7f167a538700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e54ffeb8b0 0x55e54fff0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53644/0 (socket says 172.21.15.52:53644) 2023-11-27T15:20:41.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.733+0000 7f167a538700 1 -- 172.21.15.52:0/461342490 learned_addr learned my addr 172.21.15.52:0/461342490 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:41.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.733+0000 7f1679d37700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e54ffe6b60 0x55e54fff30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43298/0 (socket says 172.21.15.52:43298) 2023-11-27T15:20:41.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.733+0000 7f1679536700 1 --2- 172.21.15.52:0/461342490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e54ffe7470 0x55e54fff03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.733+0000 7f167a538700 1 -- 172.21.15.52:0/461342490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e54ffe7470 msgr2=0x55e54fff03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:41.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.733+0000 7f167a538700 1 --2- 172.21.15.52:0/461342490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e54ffe7470 0x55e54fff03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.733+0000 7f167a538700 1 -- 172.21.15.52:0/461342490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e54ffe6b60 msgr2=0x55e54fff30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:41.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.733+0000 7f167a538700 1 --2- 172.21.15.52:0/461342490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e54ffe6b60 0x55e54fff30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.733+0000 7f167a538700 1 -- 172.21.15.52:0/461342490 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1670009c90 con 0x55e54ffeb8b0 2023-11-27T15:20:41.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.733+0000 7f1679d37700 1 --2- 172.21.15.52:0/461342490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e54ffe6b60 0x55e54fff30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:41.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.733+0000 7f167a538700 1 --2- 172.21.15.52:0/461342490 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e54ffeb8b0 0x55e54fff0aa0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f166c099310 tx=0x7f166c0994b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77fe98fb77e2ae87 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:41.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.733+0000 7f165d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:20:41.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.733+0000 7f165d7fa700 1 -- 172.21.15.52:0/461342490 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c099610 con 0x55e54ffeb8b0 2023-11-27T15:20:41.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.734+0000 7f1688127b80 1 -- 172.21.15.52:0/461342490 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e54ffd9b20 con 0x55e54ffeb8b0 2023-11-27T15:20:41.754 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.733+0000 7f165d7fa700 1 -- 172.21.15.52:0/461342490 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f166c09d640 con 0x55e54ffeb8b0 2023-11-27T15:20:41.754 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.734+0000 7f165d7fa700 1 -- 172.21.15.52:0/461342490 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f166c0a1a60 con 0x55e54ffeb8b0 2023-11-27T15:20:41.754 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.734+0000 7f1688127b80 10 client.16086 fetch_fsmap learned FSMap version 24 2023-11-27T15:20:41.754 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.734+0000 7f1688127b80 1 -- 172.21.15.52:0/461342490 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e54fefe190 con 0x55e54ffeb8b0 2023-11-27T15:20:41.754 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.734+0000 7f165d7fa700 1 -- 172.21.15.52:0/461342490 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f166c0c7080 con 0x55e54ffeb8b0 2023-11-27T15:20:41.754 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.734+0000 7f165d7fa700 1 -- 172.21.15.52:0/461342490 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f166c099990 con 0x55e54ffeb8b0 2023-11-27T15:20:41.754 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.734+0000 7f1688127b80 10 client.16086 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:20:41.755 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.734+0000 7f1688127b80 10 client.16086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:20:41.755 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.734+0000 7f1688127b80 20 client.16086 populate_metadata read hostname 'smithi052' 2023-11-27T15:20:41.755 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.734+0000 7f1688127b80 1 --2- 172.21.15.52:0/461342490 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55e550000990 0x55e550002da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.755 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.734+0000 7f1688127b80 4 client.16086 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:20:41.755 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.734+0000 7f1688127b80 1 -- 172.21.15.52:0/461342490 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e54fefa0a0 con 0x55e550000990 2023-11-27T15:20:41.755 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.734+0000 7f1679d37700 1 --2- 172.21.15.52:0/461342490 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55e550000990 0x55e550002da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.755 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.735+0000 7f165d7fa700 10 client.16086 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:20:41.755 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.735+0000 7f1679d37700 1 --2- 172.21.15.52:0/461342490 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55e550000990 0x55e550002da0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=247c6c31b7554388 server_cookie=16e665da84b899cd in_seq=0 out_seq=0 2023-11-27T15:20:41.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.771+0000 7f165d7fa700 1 -- 172.21.15.52:0/461342490 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e54fefa0a0 con 0x55e550000990 2023-11-27T15:20:41.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.771+0000 7f165d7fa700 10 client.16086 handle_command_reply: tid=0 2023-11-27T15:20:41.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.771+0000 7f1688127b80 1 client.16086 shutdown 2023-11-27T15:20:41.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.771+0000 7f1688127b80 1 -- 172.21.15.52:0/461342490 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e54ffeb8b0 msgr2=0x55e54fff0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:41.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.771+0000 7f1688127b80 1 --2- 172.21.15.52:0/461342490 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e54ffeb8b0 0x55e54fff0aa0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f166c099310 tx=0x7f166c0994b0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.771+0000 7f1688127b80 1 -- 172.21.15.52:0/461342490 shutdown_connections 2023-11-27T15:20:41.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.771+0000 7f1688127b80 1 --2- 172.21.15.52:0/461342490 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55e550000990 0x55e550002da0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.771+0000 7f1688127b80 1 --2- 172.21.15.52:0/461342490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e54ffe6b60 0x55e54fff30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.771+0000 7f165d7fa700 0 client.16086 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:20:41.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.771+0000 7f1688127b80 1 --2- 172.21.15.52:0/461342490 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e54ffeb8b0 0x55e54fff0aa0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.771+0000 7f1688127b80 1 --2- 172.21.15.52:0/461342490 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e54ffe7470 0x55e54fff03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.771+0000 7f1688127b80 1 -- 172.21.15.52:0/461342490 >> 172.21.15.52:0/461342490 conn(0x55e54ffdc360 msgr2=0x55e54ffe0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:41.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.771+0000 7f1688127b80 1 -- 172.21.15.52:0/461342490 shutdown_connections 2023-11-27T15:20:41.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.771+0000 7f1688127b80 1 -- 172.21.15.52:0/461342490 wait complete. 2023-11-27T15:20:41.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.772+0000 7f1688127b80 20 client.16086 trim_cache size 0 max 16384 2023-11-27T15:20:41.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.789+0000 7f1688127b80 1 Processor -- start 2023-11-27T15:20:41.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.789+0000 7f1688127b80 1 -- start start 2023-11-27T15:20:41.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.789+0000 7f1688127b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e54ffe6b60 0x55e54ffeb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.789+0000 7f1688127b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e54ffe7470 0x55e54fff79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.789+0000 7f1688127b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e54ffeb8b0 0x55e54ffede00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.789+0000 7f1688127b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54fff0e80 con 0x55e54ffe7470 2023-11-27T15:20:41.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.789+0000 7f1688127b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54ffe7bf0 con 0x55e54ffe6b60 2023-11-27T15:20:41.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.789+0000 7f1688127b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54ffee910 con 0x55e54ffeb8b0 2023-11-27T15:20:41.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.789+0000 7f1679536700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e54ffe7470 0x55e54fff79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.789+0000 7f1679536700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e54ffe7470 0x55e54fff79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53650/0 (socket says 172.21.15.52:53650) 2023-11-27T15:20:41.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.789+0000 7f1679536700 1 -- 172.21.15.52:0/569248453 learned_addr learned my addr 172.21.15.52:0/569248453 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:41.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.789+0000 7f1679d37700 1 --2- 172.21.15.52:0/569248453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e54ffe6b60 0x55e54ffeb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.789+0000 7f167a538700 1 --2- 172.21.15.52:0/569248453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e54ffeb8b0 0x55e54ffede00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.789+0000 7f1679536700 1 -- 172.21.15.52:0/569248453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e54ffeb8b0 msgr2=0x55e54ffede00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:41.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.790+0000 7f1679536700 1 --2- 172.21.15.52:0/569248453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e54ffeb8b0 0x55e54ffede00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.790+0000 7f1679536700 1 -- 172.21.15.52:0/569248453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e54ffe6b60 msgr2=0x55e54ffeb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:41.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.790+0000 7f1679536700 1 --2- 172.21.15.52:0/569248453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e54ffe6b60 0x55e54ffeb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.790+0000 7f1679536700 1 -- 172.21.15.52:0/569248453 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e54fefdee0 con 0x55e54ffe7470 2023-11-27T15:20:41.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.790+0000 7f167a538700 1 --2- 172.21.15.52:0/569248453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e54ffeb8b0 0x55e54ffede00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:41.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.790+0000 7f1679d37700 1 --2- 172.21.15.52:0/569248453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e54ffe6b60 0x55e54ffeb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:41.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.790+0000 7f1679536700 1 --2- 172.21.15.52:0/569248453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e54ffe7470 0x55e54fff79a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1670004690 tx=0x7f1670005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac47d73c0feaa940 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:41.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.790+0000 7f1668ff9700 1 -- 172.21.15.52:0/569248453 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1670004ee0 con 0x55e54ffe7470 2023-11-27T15:20:41.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.790+0000 7f1668ff9700 1 -- 172.21.15.52:0/569248453 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1670004b80 con 0x55e54ffe7470 2023-11-27T15:20:41.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.790+0000 7f1668ff9700 1 -- 172.21.15.52:0/569248453 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1670024740 con 0x55e54ffe7470 2023-11-27T15:20:41.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.791+0000 7f1688127b80 1 -- 172.21.15.52:0/569248453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e54ffe7470 msgr2=0x55e54fff79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:41.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.791+0000 7f1688127b80 1 --2- 172.21.15.52:0/569248453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e54ffe7470 0x55e54fff79a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1670004690 tx=0x7f1670005b40 comp rx=0 tx=0).stop 2023-11-27T15:20:41.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.791+0000 7f1688127b80 1 -- 172.21.15.52:0/569248453 shutdown_connections 2023-11-27T15:20:41.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.791+0000 7f1688127b80 1 --2- 172.21.15.52:0/569248453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e54ffe6b60 0x55e54ffeb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.791+0000 7f1688127b80 1 --2- 172.21.15.52:0/569248453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e54ffeb8b0 0x55e54ffede00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.791+0000 7f1688127b80 1 --2- 172.21.15.52:0/569248453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e54ffe7470 0x55e54fff79a0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.791+0000 7f1688127b80 1 -- 172.21.15.52:0/569248453 >> 172.21.15.52:0/569248453 conn(0x55e54ffdc360 msgr2=0x55e54ffe5960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:41.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.791+0000 7f1688127b80 1 -- 172.21.15.52:0/569248453 shutdown_connections 2023-11-27T15:20:41.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.791+0000 7f1688127b80 1 -- 172.21.15.52:0/569248453 wait complete. 2023-11-27T15:20:41.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.791+0000 7f1688127b80 1 -- start start 2023-11-27T15:20:41.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.791+0000 7f1688127b80 1 Processor -- start 2023-11-27T15:20:41.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.792+0000 7f1688127b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e54ffe6b60 0x55e54ffeff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.792+0000 7f1688127b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e54ffe7470 0x55e54ffeeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.792+0000 7f1688127b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e54ffeb8b0 0x55e54ffef0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.792+0000 7f1688127b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54fff0530 con 0x55e54ffeb8b0 2023-11-27T15:20:41.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.792+0000 7f1688127b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54fff06a0 con 0x55e54ffe7470 2023-11-27T15:20:41.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.792+0000 7f1688127b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54ffef5e0 con 0x55e54ffe6b60 2023-11-27T15:20:41.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.792+0000 7f1688127b80 10 client.0 _ll_register_callbacks cb 0x7f166c095a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:20:41.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.792+0000 7f167a538700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e54ffeb8b0 0x55e54ffef0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.792+0000 7f1679536700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e54ffe7470 0x55e54ffeeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.792+0000 7f1679d37700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e54ffe6b60 0x55e54ffeff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.792+0000 7f1679536700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e54ffe7470 0x55e54ffeeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43314/0 (socket says 172.21.15.52:43314) 2023-11-27T15:20:41.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.792+0000 7f1679d37700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e54ffe6b60 0x55e54ffeff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:40298/0 (socket says 172.21.15.52:40298) 2023-11-27T15:20:41.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.792+0000 7f167a538700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e54ffeb8b0 0x55e54ffef0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53658/0 (socket says 172.21.15.52:53658) 2023-11-27T15:20:41.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.792+0000 7f1679536700 1 -- 172.21.15.52:0/1965420081 learned_addr learned my addr 172.21.15.52:0/1965420081 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:20:41.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.792+0000 7f1679536700 1 -- 172.21.15.52:0/1965420081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e54ffe6b60 msgr2=0x55e54ffeff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:41.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.792+0000 7f1679536700 1 --2- 172.21.15.52:0/1965420081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e54ffe6b60 0x55e54ffeff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.792+0000 7f1679536700 1 -- 172.21.15.52:0/1965420081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e54ffeb8b0 msgr2=0x55e54ffef0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:41.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.792+0000 7f1679536700 1 --2- 172.21.15.52:0/1965420081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e54ffeb8b0 0x55e54ffef0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.792+0000 7f1679536700 1 -- 172.21.15.52:0/1965420081 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1664037b30 con 0x55e54ffe7470 2023-11-27T15:20:41.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.792+0000 7f1679536700 1 --2- 172.21.15.52:0/1965420081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e54ffe7470 0x55e54ffeeb60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1670004aa0 tx=0x7f1670004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f70d5c123ffde466 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:41.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.792+0000 7f165d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2023-11-27T15:20:41.807 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.793+0000 7f165d7fa700 1 -- 172.21.15.52:0/1965420081 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1670004f60 con 0x55e54ffe7470 2023-11-27T15:20:41.807 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.793+0000 7f165d7fa700 1 -- 172.21.15.52:0/1965420081 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f16700050c0 con 0x55e54ffe7470 2023-11-27T15:20:41.807 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.793+0000 7f1688127b80 1 -- 172.21.15.52:0/1965420081 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1670004cf0 con 0x55e54ffe7470 2023-11-27T15:20:41.807 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.793+0000 7f165d7fa700 1 -- 172.21.15.52:0/1965420081 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1670005280 con 0x55e54ffe7470 2023-11-27T15:20:41.807 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.793+0000 7f1688127b80 10 client.25907 fetch_fsmap learned FSMap version 24 2023-11-27T15:20:41.807 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.793+0000 7f1688127b80 1 -- 172.21.15.52:0/1965420081 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e54fefe190 con 0x55e54ffe7470 2023-11-27T15:20:41.808 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.793+0000 7f165d7fa700 1 -- 172.21.15.52:0/1965420081 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1670041020 con 0x55e54ffe7470 2023-11-27T15:20:41.808 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.793+0000 7f165d7fa700 1 -- 172.21.15.52:0/1965420081 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f167002a020 con 0x55e54ffe7470 2023-11-27T15:20:41.808 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.793+0000 7f1688127b80 10 client.25907 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:20:41.808 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.793+0000 7f1688127b80 10 client.25907 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:20:41.808 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.793+0000 7f1688127b80 20 client.25907 populate_metadata read hostname 'smithi052' 2023-11-27T15:20:41.808 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.793+0000 7f1688127b80 1 --2- 172.21.15.52:0/1965420081 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55e550055480 0x55e550057890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:41.808 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.793+0000 7f1688127b80 4 client.25907 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:20:41.808 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.793+0000 7f1688127b80 1 -- 172.21.15.52:0/1965420081 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e54ff06660 con 0x55e550055480 2023-11-27T15:20:41.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.793+0000 7f1679d37700 1 --2- 172.21.15.52:0/1965420081 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55e550055480 0x55e550057890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:41.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.794+0000 7f165d7fa700 10 client.25907 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:20:41.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.794+0000 7f1679d37700 1 --2- 172.21.15.52:0/1965420081 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55e550055480 0x55e550057890 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e82601108d8b8088 server_cookie=b69623541b7cfac2 in_seq=0 out_seq=0 2023-11-27T15:20:41.810 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.797+0000 7f165d7fa700 1 -- 172.21.15.52:0/1965420081 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x55e54ff06660 con 0x55e550055480 2023-11-27T15:20:41.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.797+0000 7f165d7fa700 10 client.25907 handle_command_reply: tid=0 2023-11-27T15:20:41.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.797+0000 7f1688127b80 1 client.25907 shutdown 2023-11-27T15:20:41.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.797+0000 7f1688127b80 1 -- 172.21.15.52:0/1965420081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e54ffe7470 msgr2=0x55e54ffeeb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:41.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.797+0000 7f1688127b80 1 --2- 172.21.15.52:0/1965420081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e54ffe7470 0x55e54ffeeb60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1670004aa0 tx=0x7f1670004ad0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.797+0000 7f1688127b80 1 -- 172.21.15.52:0/1965420081 shutdown_connections 2023-11-27T15:20:41.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.797+0000 7f1688127b80 1 --2- 172.21.15.52:0/1965420081 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55e550055480 0x55e550057890 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.797+0000 7f1688127b80 1 --2- 172.21.15.52:0/1965420081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e54ffeb8b0 0x55e54ffef0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.797+0000 7f1688127b80 1 --2- 172.21.15.52:0/1965420081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55e54ffe6b60 0x55e54ffeff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.797+0000 7f1688127b80 1 --2- 172.21.15.52:0/1965420081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e54ffe7470 0x55e54ffeeb60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.797+0000 7f1688127b80 1 -- 172.21.15.52:0/1965420081 >> 172.21.15.52:0/1965420081 conn(0x55e54ffdc360 msgr2=0x55e54ffe5960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:41.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.797+0000 7f165d7fa700 0 client.25907 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:20:41.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.797+0000 7f1688127b80 1 -- 172.21.15.52:0/1965420081 shutdown_connections 2023-11-27T15:20:41.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.797+0000 7f1688127b80 1 -- 172.21.15.52:0/1965420081 wait complete. 2023-11-27T15:20:41.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.798+0000 7f1688127b80 20 client.25907 trim_cache size 0 max 16384 2023-11-27T15:20:41.812 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:20:41.813 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "scrub active (8441 inodes in the stack)", 2023-11-27T15:20:41.813 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2023-11-27T15:20:41.813 INFO:teuthology.orchestra.run.smithi052.stdout: "1a14f367-9a76-4e86-b4c8-bd13c3d4d297": { 2023-11-27T15:20:41.813 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2023-11-27T15:20:41.813 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "1a14f367-9a76-4e86-b4c8-bd13c3d4d297", 2023-11-27T15:20:41.813 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2023-11-27T15:20:41.813 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:20:41.814 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:20:41.814 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:20:41.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.798+0000 7f167bf9b700 1 -- 172.21.15.52:0/183105109 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f16600a12c0 msgr2=0x7f16600a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:41.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.798+0000 7f167bf9b700 1 --2- 172.21.15.52:0/183105109 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f16600a12c0 0x7f16600a3770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1664004800 tx=0x7f166401c040 comp rx=0 tx=0).stop 2023-11-27T15:20:41.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.798+0000 7f167bf9b700 1 -- 172.21.15.52:0/183105109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1674104490 msgr2=0x7f1674199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:41.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.798+0000 7f167bf9b700 1 --2- 172.21.15.52:0/183105109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1674104490 0x7f1674199c70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f166c007f10 tx=0x7f166c00d3b0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.798+0000 7f167bf9b700 1 -- 172.21.15.52:0/183105109 shutdown_connections 2023-11-27T15:20:41.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.798+0000 7f167bf9b700 1 --2- 172.21.15.52:0/183105109 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f16600a12c0 0x7f16600a3770 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.798+0000 7f167bf9b700 1 --2- 172.21.15.52:0/183105109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1674104490 0x7f1674199c70 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.798+0000 7f167bf9b700 1 --2- 172.21.15.52:0/183105109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16740ff450 0x7f16741952d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.798+0000 7f167bf9b700 1 --2- 172.21.15.52:0/183105109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f16740ffdf0 0x7f1674195810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:41.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.798+0000 7f167bf9b700 1 -- 172.21.15.52:0/183105109 >> 172.21.15.52:0/183105109 conn(0x7f16740770b0 msgr2=0x7f16740fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:41.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.798+0000 7f167bf9b700 1 -- 172.21.15.52:0/183105109 shutdown_connections 2023-11-27T15:20:41.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:20:41.798+0000 7f167bf9b700 1 -- 172.21.15.52:0/183105109 wait complete. 2023-11-27T15:20:41.817 INFO:tasks.cephfs.filesystem:scrub status for tag:1a14f367-9a76-4e86-b4c8-bd13c3d4d297 - {'path': '/', 'tag': '1a14f367-9a76-4e86-b4c8-bd13c3d4d297', 'options': 'recursive,force'} 2023-11-27T15:20:42.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:42 smithi150 ceph-mon[100559]: pgmap v86: 97 pgs: 97 active+clean; 105 MiB data, 324 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.2 MiB/s wr, 499 op/s 2023-11-27T15:20:42.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:42 smithi150 ceph-mon[100559]: scrub queued for path: / 2023-11-27T15:20:42.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:42 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:20:42.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:42 smithi150 ceph-mon[100559]: scrub summary: active paths [/] 2023-11-27T15:20:42.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:42 smithi102 ceph-mon[100296]: pgmap v86: 97 pgs: 97 active+clean; 105 MiB data, 324 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.2 MiB/s wr, 499 op/s 2023-11-27T15:20:42.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:42 smithi102 ceph-mon[100296]: scrub queued for path: / 2023-11-27T15:20:42.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:42 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:20:42.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:42 smithi102 ceph-mon[100296]: scrub summary: active paths [/] 2023-11-27T15:20:42.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:42 smithi052 ceph-mon[99056]: pgmap v86: 97 pgs: 97 active+clean; 105 MiB data, 324 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.2 MiB/s wr, 499 op/s 2023-11-27T15:20:42.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:42 smithi052 ceph-mon[99056]: scrub queued for path: / 2023-11-27T15:20:42.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:42 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:20:42.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:42 smithi052 ceph-mon[99056]: scrub summary: active paths [/] 2023-11-27T15:20:44.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:44 smithi150 ceph-mon[100559]: pgmap v87: 97 pgs: 97 active+clean; 105 MiB data, 324 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 488 op/s 2023-11-27T15:20:44.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:44 smithi102 ceph-mon[100296]: pgmap v87: 97 pgs: 97 active+clean; 105 MiB data, 324 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 488 op/s 2023-11-27T15:20:44.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:44 smithi052 ceph-mon[99056]: pgmap v87: 97 pgs: 97 active+clean; 105 MiB data, 324 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 488 op/s 2023-11-27T15:20:46.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:46 smithi150 ceph-mon[100559]: pgmap v88: 97 pgs: 97 active+clean; 105 MiB data, 324 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 488 op/s 2023-11-27T15:20:46.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:46 smithi150 ceph-mon[100559]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:20:46.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:46 smithi102 ceph-mon[100296]: pgmap v88: 97 pgs: 97 active+clean; 105 MiB data, 324 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 488 op/s 2023-11-27T15:20:46.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:46 smithi102 ceph-mon[100296]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:20:46.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:46 smithi052 ceph-mon[99056]: pgmap v88: 97 pgs: 97 active+clean; 105 MiB data, 324 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 488 op/s 2023-11-27T15:20:46.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:46 smithi052 ceph-mon[99056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:20:48.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:48 smithi150 ceph-mon[100559]: pgmap v89: 97 pgs: 97 active+clean; 122 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.0 MiB/s wr, 521 op/s 2023-11-27T15:20:48.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:48 smithi102 ceph-mon[100296]: pgmap v89: 97 pgs: 97 active+clean; 122 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.0 MiB/s wr, 521 op/s 2023-11-27T15:20:48.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:48 smithi052 ceph-mon[99056]: pgmap v89: 97 pgs: 97 active+clean; 122 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.0 MiB/s wr, 521 op/s 2023-11-27T15:20:50.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:49 smithi052 ceph-mon[99056]: pgmap v90: 97 pgs: 97 active+clean; 122 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 213 op/s 2023-11-27T15:20:50.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:49 smithi150 ceph-mon[100559]: pgmap v90: 97 pgs: 97 active+clean; 122 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 213 op/s 2023-11-27T15:20:50.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:49 smithi102 ceph-mon[100296]: pgmap v90: 97 pgs: 97 active+clean; 122 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 213 op/s 2023-11-27T15:20:51.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:50 smithi052 ceph-mon[99056]: scrub summary: active paths [/] 2023-11-27T15:20:51.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:50 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:20:51.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:50 smithi150 ceph-mon[100559]: scrub summary: active paths [/] 2023-11-27T15:20:51.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:50 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:20:51.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:50 smithi102 ceph-mon[100296]: scrub summary: active paths [/] 2023-11-27T15:20:51.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:50 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:20:52.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:51 smithi052 ceph-mon[99056]: pgmap v91: 97 pgs: 97 active+clean; 138 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.0 MiB/s wr, 225 op/s 2023-11-27T15:20:52.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:51 smithi150 ceph-mon[100559]: pgmap v91: 97 pgs: 97 active+clean; 138 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.0 MiB/s wr, 225 op/s 2023-11-27T15:20:52.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:51 smithi102 ceph-mon[100296]: pgmap v91: 97 pgs: 97 active+clean; 138 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.0 MiB/s wr, 225 op/s 2023-11-27T15:20:53.841 INFO:tasks.workunit.client.0.smithi052.stdout:created hierarchy, now cleaning up 2023-11-27T15:20:54.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:54 smithi150 ceph-mon[100559]: pgmap v92: 97 pgs: 97 active+clean; 138 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 44 op/s 2023-11-27T15:20:54.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:54 smithi102 ceph-mon[100296]: pgmap v92: 97 pgs: 97 active+clean; 138 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 44 op/s 2023-11-27T15:20:54.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:54 smithi052 ceph-mon[99056]: pgmap v92: 97 pgs: 97 active+clean; 138 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 44 op/s 2023-11-27T15:20:56.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:56 smithi150 ceph-mon[100559]: pgmap v93: 97 pgs: 97 active+clean; 138 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 44 op/s 2023-11-27T15:20:56.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:56 smithi102 ceph-mon[100296]: pgmap v93: 97 pgs: 97 active+clean; 138 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 44 op/s 2023-11-27T15:20:56.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:56 smithi052 ceph-mon[99056]: pgmap v93: 97 pgs: 97 active+clean; 138 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 44 op/s 2023-11-27T15:20:58.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:58 smithi150 ceph-mon[100559]: pgmap v94: 97 pgs: 97 active+clean; 149 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.9 MiB/s wr, 50 op/s 2023-11-27T15:20:58.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:58 smithi102 ceph-mon[100296]: pgmap v94: 97 pgs: 97 active+clean; 149 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.9 MiB/s wr, 50 op/s 2023-11-27T15:20:58.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:58 smithi052 ceph-mon[99056]: pgmap v94: 97 pgs: 97 active+clean; 149 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.9 MiB/s wr, 50 op/s 2023-11-27T15:21:00.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:20:59 smithi052 ceph-mon[99056]: pgmap v95: 97 pgs: 97 active+clean; 149 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 17 op/s 2023-11-27T15:21:00.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:20:59 smithi150 ceph-mon[100559]: pgmap v95: 97 pgs: 97 active+clean; 149 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 17 op/s 2023-11-27T15:21:00.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:20:59 smithi102 ceph-mon[100296]: pgmap v95: 97 pgs: 97 active+clean; 149 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 17 op/s 2023-11-27T15:21:02.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:01 smithi052 ceph-mon[99056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:21:02.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:01 smithi052 ceph-mon[99056]: pgmap v96: 97 pgs: 97 active+clean; 170 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 34 op/s 2023-11-27T15:21:02.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:01 smithi150 ceph-mon[100559]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:21:02.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:01 smithi150 ceph-mon[100559]: pgmap v96: 97 pgs: 97 active+clean; 170 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 34 op/s 2023-11-27T15:21:02.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:01 smithi102 ceph-mon[100296]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:21:02.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:01 smithi102 ceph-mon[100296]: pgmap v96: 97 pgs: 97 active+clean; 170 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 34 op/s 2023-11-27T15:21:04.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:04 smithi150 ceph-mon[100559]: pgmap v97: 97 pgs: 97 active+clean; 170 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 22 op/s 2023-11-27T15:21:04.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:04 smithi102 ceph-mon[100296]: pgmap v97: 97 pgs: 97 active+clean; 170 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 22 op/s 2023-11-27T15:21:04.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:04 smithi052 ceph-mon[99056]: pgmap v97: 97 pgs: 97 active+clean; 170 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 22 op/s 2023-11-27T15:21:06.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:06 smithi150 ceph-mon[100559]: pgmap v98: 97 pgs: 97 active+clean; 170 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 22 op/s 2023-11-27T15:21:06.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:06 smithi102 ceph-mon[100296]: pgmap v98: 97 pgs: 97 active+clean; 170 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 22 op/s 2023-11-27T15:21:06.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:06 smithi052 ceph-mon[99056]: pgmap v98: 97 pgs: 97 active+clean; 170 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 22 op/s 2023-11-27T15:21:08.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:08 smithi150 ceph-mon[100559]: pgmap v99: 97 pgs: 97 active+clean; 210 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.0 MiB/s wr, 49 op/s 2023-11-27T15:21:08.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:08 smithi102 ceph-mon[100296]: pgmap v99: 97 pgs: 97 active+clean; 210 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.0 MiB/s wr, 49 op/s 2023-11-27T15:21:08.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:08 smithi052 ceph-mon[99056]: pgmap v99: 97 pgs: 97 active+clean; 210 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.0 MiB/s wr, 49 op/s 2023-11-27T15:21:10.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:09 smithi052 ceph-mon[99056]: pgmap v100: 97 pgs: 97 active+clean; 210 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.2 MiB/s wr, 42 op/s 2023-11-27T15:21:10.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:09 smithi150 ceph-mon[100559]: pgmap v100: 97 pgs: 97 active+clean; 210 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.2 MiB/s wr, 42 op/s 2023-11-27T15:21:10.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:09 smithi102 ceph-mon[100296]: pgmap v100: 97 pgs: 97 active+clean; 210 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.2 MiB/s wr, 42 op/s 2023-11-27T15:21:11.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:10 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:21:11.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:10 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:21:11.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:10 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:21:11.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:10 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:21:11.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:10 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:21:11.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:10 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:21:11.302 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:10 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:21:11.302 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:10 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:21:11.302 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:10 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:21:11.303 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:10 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:21:11.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:10 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:21:11.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:10 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:21:11.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:10 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:21:11.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:10 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:21:11.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:10 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:21:11.818 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:21:11.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.947+0000 7f6aaa677700 1 -- 172.21.15.52:0/1971808904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6aa4104490 msgr2=0x7f6aa410c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:11.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.947+0000 7f6aaa677700 1 --2- 172.21.15.52:0/1971808904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6aa4104490 0x7f6aa410c060 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00b0d0 tx=0x7f6a9c00b3e0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.947+0000 7f6aaa677700 1 -- 172.21.15.52:0/1971808904 shutdown_connections 2023-11-27T15:21:11.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.947+0000 7f6aaa677700 1 --2- 172.21.15.52:0/1971808904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6aa4104490 0x7f6aa410c060 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.947+0000 7f6aaa677700 1 --2- 172.21.15.52:0/1971808904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6aa40ff450 0x7f6aa40ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.947+0000 7f6aaa677700 1 --2- 172.21.15.52:0/1971808904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6aa40ffdf0 0x7f6aa4103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.947+0000 7f6aaa677700 1 -- 172.21.15.52:0/1971808904 >> 172.21.15.52:0/1971808904 conn(0x7f6aa40770b0 msgr2=0x7f6aa40774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:11.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.947+0000 7f6aaa677700 1 -- 172.21.15.52:0/1971808904 shutdown_connections 2023-11-27T15:21:11.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.947+0000 7f6aaa677700 1 -- 172.21.15.52:0/1971808904 wait complete. 2023-11-27T15:21:11.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.947+0000 7f6aaa677700 1 Processor -- start 2023-11-27T15:21:11.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.947+0000 7f6aaa677700 1 -- start start 2023-11-27T15:21:11.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.948+0000 7f6aaa677700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6aa40ff450 0x7f6aa4195400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:11.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.948+0000 7f6aaa677700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6aa40ffdf0 0x7f6aa4195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:11.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.948+0000 7f6aaa677700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6aa4104490 0x7f6aa41a2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:11.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.948+0000 7f6aaa677700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa41a2800 con 0x7f6aa40ff450 2023-11-27T15:21:11.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.948+0000 7f6aaa677700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa41a2940 con 0x7f6aa4104490 2023-11-27T15:21:11.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.948+0000 7f6aaa677700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa41a2ab0 con 0x7f6aa40ffdf0 2023-11-27T15:21:11.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.948+0000 7f6aa3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6aa40ff450 0x7f6aa4195400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:11.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.948+0000 7f6aa3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6aa40ff450 0x7f6aa4195400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53670/0 (socket says 172.21.15.52:53670) 2023-11-27T15:21:11.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.948+0000 7f6aa3fff700 1 -- 172.21.15.52:0/3560317699 learned_addr learned my addr 172.21.15.52:0/3560317699 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:21:11.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.948+0000 7f6aa37fe700 1 --2- 172.21.15.52:0/3560317699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6aa40ffdf0 0x7f6aa4195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:11.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.948+0000 7f6aa8c14700 1 --2- 172.21.15.52:0/3560317699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6aa4104490 0x7f6aa41a2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:11.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.948+0000 7f6aa3fff700 1 -- 172.21.15.52:0/3560317699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6aa40ffdf0 msgr2=0x7f6aa4195940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:11.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.948+0000 7f6aa3fff700 1 --2- 172.21.15.52:0/3560317699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6aa40ffdf0 0x7f6aa4195940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.948+0000 7f6aa3fff700 1 -- 172.21.15.52:0/3560317699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6aa4104490 msgr2=0x7f6aa41a2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:11.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.948+0000 7f6aa3fff700 1 --2- 172.21.15.52:0/3560317699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6aa4104490 0x7f6aa41a2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.948+0000 7f6aa3fff700 1 -- 172.21.15.52:0/3560317699 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9c009d00 con 0x7f6aa40ff450 2023-11-27T15:21:11.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.948+0000 7f6aa3fff700 1 --2- 172.21.15.52:0/3560317699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6aa40ff450 0x7f6aa4195400 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400d980 tx=0x7f6a9400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:11.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.949+0000 7f6aa17fa700 1 -- 172.21.15.52:0/3560317699 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a94004220 con 0x7f6aa40ff450 2023-11-27T15:21:11.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.949+0000 7f6aa17fa700 1 -- 172.21.15.52:0/3560317699 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a94007640 con 0x7f6aa40ff450 2023-11-27T15:21:11.957 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.949+0000 7f6aa17fa700 1 -- 172.21.15.52:0/3560317699 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a94005690 con 0x7f6aa40ff450 2023-11-27T15:21:11.957 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.949+0000 7f6aaa677700 1 -- 172.21.15.52:0/3560317699 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa410f260 con 0x7f6aa40ff450 2023-11-27T15:21:11.957 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.949+0000 7f6aaa677700 1 -- 172.21.15.52:0/3560317699 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa410f0d0 con 0x7f6aa40ff450 2023-11-27T15:21:11.962 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.951+0000 7f6aa17fa700 1 -- 172.21.15.52:0/3560317699 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f6a94004750 con 0x7f6aa40ff450 2023-11-27T15:21:11.963 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.952+0000 7f6aa17fa700 1 --2- 172.21.15.52:0/3560317699 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f6a8c0a1340 0x7f6a8c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:11.963 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.952+0000 7f6aa17fa700 1 -- 172.21.15.52:0/3560317699 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f6a940cca70 con 0x7f6aa40ff450 2023-11-27T15:21:11.963 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.952+0000 7f6aa37fe700 1 --2- 172.21.15.52:0/3560317699 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f6a8c0a1340 0x7f6a8c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:11.963 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.952+0000 7f6aa37fe700 1 --2- 172.21.15.52:0/3560317699 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f6a8c0a1340 0x7f6a8c0a37f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4068eb0 tx=0x7f6a98009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:11.963 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.952+0000 7f6ab6803b80 1 Processor -- start 2023-11-27T15:21:11.963 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.952+0000 7f6ab6803b80 1 -- start start 2023-11-27T15:21:11.963 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.953+0000 7f6ab6803b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608ab9f6b60 0x5608ab9f6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:11.963 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.953+0000 7f6ab6803b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608ab9f7470 0x5608ab9fb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:11.964 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.953+0000 7f6ab6803b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608ab9fb8b0 0x5608ab9fbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:11.964 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.953+0000 7f6ab6803b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ab8fc060 con 0x5608ab9f7470 2023-11-27T15:21:11.964 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.953+0000 7f6ab6803b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ab94e8a0 con 0x5608ab9fb8b0 2023-11-27T15:21:11.964 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.953+0000 7f6ab6803b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ab947d30 con 0x5608ab9f6b60 2023-11-27T15:21:11.964 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.953+0000 7f6aa37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608ab9f7470 0x5608ab9fb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:11.964 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.953+0000 7f6aa3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608ab9f6b60 0x5608ab9f6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:11.964 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.953+0000 7f6aa3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608ab9f6b60 0x5608ab9f6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:40322/0 (socket says 172.21.15.52:40322) 2023-11-27T15:21:11.964 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.953+0000 7f6aa37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608ab9f7470 0x5608ab9fb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53680/0 (socket says 172.21.15.52:53680) 2023-11-27T15:21:11.964 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.953+0000 7f6aa3fff700 1 -- 172.21.15.52:0/1636396467 learned_addr learned my addr 172.21.15.52:0/1636396467 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:21:11.964 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.953+0000 7f6aa8c14700 1 --2- 172.21.15.52:0/1636396467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608ab9fb8b0 0x5608ab9fbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:11.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.953+0000 7f6aa37fe700 1 -- 172.21.15.52:0/1636396467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608ab9f6b60 msgr2=0x5608ab9f6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:21:11.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.953+0000 7f6aa37fe700 1 --2- 172.21.15.52:0/1636396467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608ab9f6b60 0x5608ab9f6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.953+0000 7f6aa37fe700 1 -- 172.21.15.52:0/1636396467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608ab9fb8b0 msgr2=0x5608ab9fbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:21:11.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.953+0000 7f6aa37fe700 1 --2- 172.21.15.52:0/1636396467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608ab9fb8b0 0x5608ab9fbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.953+0000 7f6aa37fe700 1 -- 172.21.15.52:0/1636396467 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608ab7c9630 con 0x5608ab9f7470 2023-11-27T15:21:11.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.953+0000 7f6aa3fff700 1 --2- 172.21.15.52:0/1636396467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608ab9f6b60 0x5608ab9f6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:21:11.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.953+0000 7f6aa8c14700 1 --2- 172.21.15.52:0/1636396467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608ab9fb8b0 0x5608ab9fbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:21:11.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.953+0000 7f6aa37fe700 1 --2- 172.21.15.52:0/1636396467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608ab9f7470 0x5608ab9fb370 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009fd0 tx=0x7f6a98014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8f091bbd025611 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:11.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.954+0000 7f6a8a7fc700 1 -- 172.21.15.52:0/1636396467 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a980119b0 con 0x5608ab9f7470 2023-11-27T15:21:11.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.954+0000 7f6a8a7fc700 1 -- 172.21.15.52:0/1636396467 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a980044e0 con 0x5608ab9f7470 2023-11-27T15:21:11.966 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.954+0000 7f6a8a7fc700 1 -- 172.21.15.52:0/1636396467 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98003920 con 0x5608ab9f7470 2023-11-27T15:21:11.966 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.954+0000 7f6ab6803b80 1 -- 172.21.15.52:0/1636396467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608ab9f7470 msgr2=0x5608ab9fb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:11.966 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.954+0000 7f6ab6803b80 1 --2- 172.21.15.52:0/1636396467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608ab9f7470 0x5608ab9fb370 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009fd0 tx=0x7f6a98014000 comp rx=0 tx=0).stop 2023-11-27T15:21:11.966 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.954+0000 7f6ab6803b80 1 -- 172.21.15.52:0/1636396467 shutdown_connections 2023-11-27T15:21:11.966 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.954+0000 7f6ab6803b80 1 --2- 172.21.15.52:0/1636396467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608ab9fb8b0 0x5608ab9fbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.966 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.954+0000 7f6ab6803b80 1 --2- 172.21.15.52:0/1636396467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608ab9f6b60 0x5608ab9f6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.966 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.954+0000 7f6ab6803b80 1 --2- 172.21.15.52:0/1636396467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608ab9f7470 0x5608ab9fb370 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.966 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.954+0000 7f6ab6803b80 1 -- 172.21.15.52:0/1636396467 >> 172.21.15.52:0/1636396467 conn(0x5608ab9ec360 msgr2=0x5608ab9f1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:11.966 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.955+0000 7f6ab6803b80 1 -- 172.21.15.52:0/1636396467 shutdown_connections 2023-11-27T15:21:11.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.955+0000 7f6ab6803b80 1 -- 172.21.15.52:0/1636396467 wait complete. 2023-11-27T15:21:11.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.955+0000 7f6ab6803b80 1 -- start start 2023-11-27T15:21:11.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.955+0000 7f6ab6803b80 1 Processor -- start 2023-11-27T15:21:11.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.956+0000 7f6ab6803b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608ab9f6b60 0x5608aba02fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:11.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.956+0000 7f6ab6803b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608ab9f7470 0x5608aba002d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:11.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.956+0000 7f6ab6803b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608ab9fb8b0 0x5608aba008e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:11.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.956+0000 7f6ab6803b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ab8e98e0 con 0x5608ab9f7470 2023-11-27T15:21:11.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.956+0000 7f6ab6803b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ab8e9580 con 0x5608ab9fb8b0 2023-11-27T15:21:11.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.956+0000 7f6ab6803b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ab7cb920 con 0x5608ab9f6b60 2023-11-27T15:21:11.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.956+0000 7f6ab6803b80 10 client.0 _ll_register_callbacks cb 0x5608ab907780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:21:11.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.956+0000 7f6aa37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608ab9f7470 0x5608aba002d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:11.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.956+0000 7f6aa3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608ab9f6b60 0x5608aba02fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:11.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.956+0000 7f6aa37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608ab9f7470 0x5608aba002d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53686/0 (socket says 172.21.15.52:53686) 2023-11-27T15:21:11.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.956+0000 7f6aa8c14700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608ab9fb8b0 0x5608aba008e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:11.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.956+0000 7f6aa37fe700 1 -- 172.21.15.52:0/2427291827 learned_addr learned my addr 172.21.15.52:0/2427291827 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:21:11.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.956+0000 7f6aa37fe700 1 -- 172.21.15.52:0/2427291827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608ab9f6b60 msgr2=0x5608aba02fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:21:11.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.956+0000 7f6aa37fe700 1 --2- 172.21.15.52:0/2427291827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608ab9f6b60 0x5608aba02fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.956+0000 7f6aa37fe700 1 -- 172.21.15.52:0/2427291827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608ab9fb8b0 msgr2=0x5608aba008e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:21:11.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.956+0000 7f6aa37fe700 1 --2- 172.21.15.52:0/2427291827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608ab9fb8b0 0x5608aba008e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.956+0000 7f6aa37fe700 1 -- 172.21.15.52:0/2427291827 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a98017070 con 0x5608ab9f7470 2023-11-27T15:21:11.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.956+0000 7f6aa37fe700 1 --2- 172.21.15.52:0/2427291827 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608ab9f7470 0x5608aba002d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6a98005950 tx=0x7f6a9801e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a5427840caee30 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:11.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.956+0000 7f6a837fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:21:11.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.956+0000 7f6a837fe700 1 -- 172.21.15.52:0/2427291827 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9802b070 con 0x5608ab9f7470 2023-11-27T15:21:11.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.956+0000 7f6a837fe700 1 -- 172.21.15.52:0/2427291827 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f6a9801e930 con 0x5608ab9f7470 2023-11-27T15:21:11.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.956+0000 7f6ab6803b80 1 -- 172.21.15.52:0/2427291827 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608ab9e9b20 con 0x5608ab9f7470 2023-11-27T15:21:11.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.957+0000 7f6a837fe700 1 -- 172.21.15.52:0/2427291827 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a980119b0 con 0x5608ab9f7470 2023-11-27T15:21:11.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.957+0000 7f6ab6803b80 10 client.16104 fetch_fsmap learned FSMap version 26 2023-11-27T15:21:11.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.957+0000 7f6ab6803b80 1 -- 172.21.15.52:0/2427291827 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5608ab90e190 con 0x5608ab9f7470 2023-11-27T15:21:11.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.957+0000 7f6a837fe700 1 -- 172.21.15.52:0/2427291827 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5608ab9e9b20 con 0x5608ab9f7470 2023-11-27T15:21:11.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.957+0000 7f6a837fe700 1 -- 172.21.15.52:0/2427291827 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f6a98024030 con 0x5608ab9f7470 2023-11-27T15:21:11.970 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.957+0000 7f6ab6803b80 10 client.16104 fetch_fsmap finished waiting for FSMap version 26 2023-11-27T15:21:11.970 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.957+0000 7f6ab6803b80 10 client.16104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:21:11.970 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.957+0000 7f6ab6803b80 20 client.16104 populate_metadata read hostname 'smithi052' 2023-11-27T15:21:11.970 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.957+0000 7f6ab6803b80 1 --2- 172.21.15.52:0/2427291827 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5608aba107a0 0x5608aba12bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:11.970 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.957+0000 7f6ab6803b80 4 client.16104 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:21:11.970 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.957+0000 7f6ab6803b80 1 -- 172.21.15.52:0/2427291827 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5608ab90a0a0 con 0x5608aba107a0 2023-11-27T15:21:11.970 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.957+0000 7f6aa3fff700 1 --2- 172.21.15.52:0/2427291827 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5608aba107a0 0x5608aba12bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:11.970 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.958+0000 7f6a837fe700 10 client.16104 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:21:11.970 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.958+0000 7f6aa3fff700 1 --2- 172.21.15.52:0/2427291827 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5608aba107a0 0x5608aba12bb0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40bfec7aed7e53ec server_cookie=32e1b14fb2f9a6f8 in_seq=0 out_seq=0 2023-11-27T15:21:11.971 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.960+0000 7f6a837fe700 1 -- 172.21.15.52:0/2427291827 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5608ab90a0a0 con 0x5608aba107a0 2023-11-27T15:21:11.972 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.960+0000 7f6a837fe700 10 client.16104 handle_command_reply: tid=0 2023-11-27T15:21:11.972 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.960+0000 7f6ab6803b80 1 client.16104 shutdown 2023-11-27T15:21:11.972 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.961+0000 7f6ab6803b80 1 -- 172.21.15.52:0/2427291827 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608ab9f7470 msgr2=0x5608aba002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:11.972 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.961+0000 7f6ab6803b80 1 --2- 172.21.15.52:0/2427291827 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608ab9f7470 0x5608aba002d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6a98005950 tx=0x7f6a9801e650 comp rx=0 tx=0).stop 2023-11-27T15:21:11.972 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.961+0000 7f6ab6803b80 1 -- 172.21.15.52:0/2427291827 shutdown_connections 2023-11-27T15:21:11.972 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.961+0000 7f6ab6803b80 1 --2- 172.21.15.52:0/2427291827 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5608aba107a0 0x5608aba12bb0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.972 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.961+0000 7f6ab6803b80 1 --2- 172.21.15.52:0/2427291827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608ab9fb8b0 0x5608aba008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.972 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.961+0000 7f6a837fe700 0 client.16104 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:21:11.972 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.961+0000 7f6ab6803b80 1 --2- 172.21.15.52:0/2427291827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608ab9f6b60 0x5608aba02fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.972 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.961+0000 7f6ab6803b80 1 --2- 172.21.15.52:0/2427291827 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608ab9f7470 0x5608aba002d0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.973 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.961+0000 7f6ab6803b80 1 -- 172.21.15.52:0/2427291827 >> 172.21.15.52:0/2427291827 conn(0x5608ab9ec360 msgr2=0x5608ab9fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:11.973 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.961+0000 7f6ab6803b80 1 -- 172.21.15.52:0/2427291827 shutdown_connections 2023-11-27T15:21:11.973 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.961+0000 7f6ab6803b80 1 -- 172.21.15.52:0/2427291827 wait complete. 2023-11-27T15:21:11.973 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.961+0000 7f6ab6803b80 20 client.16104 trim_cache size 0 max 16384 2023-11-27T15:21:11.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.978+0000 7f6ab6803b80 1 Processor -- start 2023-11-27T15:21:11.985 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.979+0000 7f6ab6803b80 1 -- start start 2023-11-27T15:21:11.985 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.979+0000 7f6ab6803b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608ab9f6b60 0x5608aba079a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:11.985 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.979+0000 7f6ab6803b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608ab9f7470 0x5608aba02b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:11.985 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.979+0000 7f6ab6803b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608ab9fb8b0 0x5608ab9f1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:11.985 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.979+0000 7f6ab6803b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ab9f7bc0 con 0x5608ab9f7470 2023-11-27T15:21:11.985 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.979+0000 7f6ab6803b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ab9fa200 con 0x5608ab9f6b60 2023-11-27T15:21:11.985 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.979+0000 7f6ab6803b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608ab9f31c0 con 0x5608ab9fb8b0 2023-11-27T15:21:11.985 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.979+0000 7f6aa37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608ab9f7470 0x5608aba02b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:11.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.979+0000 7f6aa3fff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608ab9f6b60 0x5608aba079a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:11.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.979+0000 7f6aa37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608ab9f7470 0x5608aba02b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53694/0 (socket says 172.21.15.52:53694) 2023-11-27T15:21:11.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.979+0000 7f6aa37fe700 1 -- 172.21.15.52:0/3725647675 learned_addr learned my addr 172.21.15.52:0/3725647675 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:21:11.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.979+0000 7f6aa8c14700 1 --2- 172.21.15.52:0/3725647675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608ab9fb8b0 0x5608ab9f1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:11.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.979+0000 7f6aa37fe700 1 -- 172.21.15.52:0/3725647675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608ab9fb8b0 msgr2=0x5608ab9f1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:21:11.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.979+0000 7f6aa37fe700 1 --2- 172.21.15.52:0/3725647675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608ab9fb8b0 0x5608ab9f1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.979+0000 7f6aa37fe700 1 -- 172.21.15.52:0/3725647675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608ab9f6b60 msgr2=0x5608aba079a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:21:11.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.979+0000 7f6aa37fe700 1 --2- 172.21.15.52:0/3725647675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608ab9f6b60 0x5608aba079a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.979+0000 7f6aa37fe700 1 -- 172.21.15.52:0/3725647675 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608ab9fbf50 con 0x5608ab9f7470 2023-11-27T15:21:11.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.979+0000 7f6aa8c14700 1 --2- 172.21.15.52:0/3725647675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608ab9fb8b0 0x5608ab9f1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:21:11.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.979+0000 7f6aa37fe700 1 --2- 172.21.15.52:0/3725647675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608ab9f7470 0x5608aba02b80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6a98011c70 tx=0x7f6a980119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c925a7b86a460d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:11.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.980+0000 7f6a8affd700 1 -- 172.21.15.52:0/3725647675 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9802b070 con 0x5608ab9f7470 2023-11-27T15:21:11.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.980+0000 7f6a8affd700 1 -- 172.21.15.52:0/3725647675 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a9800edb0 con 0x5608ab9f7470 2023-11-27T15:21:11.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.980+0000 7f6a8affd700 1 -- 172.21.15.52:0/3725647675 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98004f90 con 0x5608ab9f7470 2023-11-27T15:21:11.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.980+0000 7f6ab6803b80 1 -- 172.21.15.52:0/3725647675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608ab9f7470 msgr2=0x5608aba02b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:11.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.980+0000 7f6ab6803b80 1 --2- 172.21.15.52:0/3725647675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608ab9f7470 0x5608aba02b80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6a98011c70 tx=0x7f6a980119b0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.980+0000 7f6ab6803b80 1 -- 172.21.15.52:0/3725647675 shutdown_connections 2023-11-27T15:21:11.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.980+0000 7f6ab6803b80 1 --2- 172.21.15.52:0/3725647675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608ab9f6b60 0x5608aba079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.980+0000 7f6ab6803b80 1 --2- 172.21.15.52:0/3725647675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608ab9fb8b0 0x5608ab9f1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.980+0000 7f6ab6803b80 1 --2- 172.21.15.52:0/3725647675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608ab9f7470 0x5608aba02b80 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.980+0000 7f6ab6803b80 1 -- 172.21.15.52:0/3725647675 >> 172.21.15.52:0/3725647675 conn(0x5608ab9ec360 msgr2=0x5608aba0cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:11.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.980+0000 7f6ab6803b80 1 -- 172.21.15.52:0/3725647675 shutdown_connections 2023-11-27T15:21:11.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.981+0000 7f6ab6803b80 1 -- 172.21.15.52:0/3725647675 wait complete. 2023-11-27T15:21:11.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.981+0000 7f6ab6803b80 1 -- start start 2023-11-27T15:21:11.990 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.981+0000 7f6ab6803b80 1 Processor -- start 2023-11-27T15:21:11.990 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.981+0000 7f6ab6803b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608ab9f6b60 0x5608aba05990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:11.991 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.982+0000 7f6ab6803b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608ab9f7470 0x5608aba05ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:11.991 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.982+0000 7f6ab6803b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608ab9fb8b0 0x5608aba06410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:11.991 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.982+0000 7f6ab6803b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608aba63650 con 0x5608ab9fb8b0 2023-11-27T15:21:11.991 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.982+0000 7f6ab6803b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608aba637c0 con 0x5608ab9f7470 2023-11-27T15:21:11.991 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.982+0000 7f6ab6803b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608aba63980 con 0x5608ab9f6b60 2023-11-27T15:21:11.991 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.982+0000 7f6ab6803b80 10 client.0 _ll_register_callbacks cb 0x5608ab907780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:21:11.991 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.982+0000 7f6aa8c14700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608ab9fb8b0 0x5608aba06410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:11.991 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.982+0000 7f6aa3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608ab9f6b60 0x5608aba05990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:11.991 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.982+0000 7f6aa37fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608ab9f7470 0x5608aba05ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:11.991 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.982+0000 7f6aa37fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608ab9f7470 0x5608aba05ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43358/0 (socket says 172.21.15.52:43358) 2023-11-27T15:21:11.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.982+0000 7f6aa3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608ab9f6b60 0x5608aba05990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:40342/0 (socket says 172.21.15.52:40342) 2023-11-27T15:21:11.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.982+0000 7f6aa37fe700 1 -- 172.21.15.52:0/1640390364 learned_addr learned my addr 172.21.15.52:0/1640390364 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:21:11.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.982+0000 7f6aa8c14700 1 -- 172.21.15.52:0/1640390364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608ab9f6b60 msgr2=0x5608aba05990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:21:11.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.982+0000 7f6aa8c14700 1 --2- 172.21.15.52:0/1640390364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608ab9f6b60 0x5608aba05990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.982+0000 7f6aa8c14700 1 -- 172.21.15.52:0/1640390364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608ab9f7470 msgr2=0x5608aba05ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:21:11.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.982+0000 7f6aa8c14700 1 --2- 172.21.15.52:0/1640390364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608ab9f7470 0x5608aba05ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.982+0000 7f6aa8c14700 1 -- 172.21.15.52:0/1640390364 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a9c0093c0 con 0x5608ab9fb8b0 2023-11-27T15:21:11.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.982+0000 7f6aa8c14700 1 --2- 172.21.15.52:0/1640390364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608ab9fb8b0 0x5608aba06410 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c009fd0 tx=0x7f6a9c00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb81f243328b5f8a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:11.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.982+0000 7f6a837fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:21:11.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.982+0000 7f6a837fe700 1 -- 172.21.15.52:0/1640390364 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c00e040 con 0x5608ab9fb8b0 2023-11-27T15:21:11.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.983+0000 7f6a837fe700 1 -- 172.21.15.52:0/1640390364 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f6a9c0078f0 con 0x5608ab9fb8b0 2023-11-27T15:21:11.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.983+0000 7f6ab6803b80 1 -- 172.21.15.52:0/1640390364 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a980044e0 con 0x5608ab9fb8b0 2023-11-27T15:21:11.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.983+0000 7f6a837fe700 1 -- 172.21.15.52:0/1640390364 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a9c003a00 con 0x5608ab9fb8b0 2023-11-27T15:21:11.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.983+0000 7f6ab6803b80 10 client.16110 fetch_fsmap learned FSMap version 26 2023-11-27T15:21:11.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.983+0000 7f6ab6803b80 1 -- 172.21.15.52:0/1640390364 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5608ab90e190 con 0x5608ab9fb8b0 2023-11-27T15:21:11.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.983+0000 7f6a837fe700 1 -- 172.21.15.52:0/1640390364 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a9c033020 con 0x5608ab9fb8b0 2023-11-27T15:21:11.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.983+0000 7f6a837fe700 1 -- 172.21.15.52:0/1640390364 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f6a9c00b100 con 0x5608ab9fb8b0 2023-11-27T15:21:11.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.983+0000 7f6ab6803b80 10 client.16110 fetch_fsmap finished waiting for FSMap version 26 2023-11-27T15:21:11.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.983+0000 7f6ab6803b80 10 client.16110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:21:11.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.983+0000 7f6ab6803b80 20 client.16110 populate_metadata read hostname 'smithi052' 2023-11-27T15:21:11.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.983+0000 7f6ab6803b80 1 --2- 172.21.15.52:0/1640390364 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5608aba69530 0x5608aba6b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:11.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.983+0000 7f6ab6803b80 4 client.16110 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:21:11.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.983+0000 7f6ab6803b80 1 -- 172.21.15.52:0/1640390364 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5608ab916660 con 0x5608aba69530 2023-11-27T15:21:11.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.983+0000 7f6aa3fff700 1 --2- 172.21.15.52:0/1640390364 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5608aba69530 0x5608aba6b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:11.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.984+0000 7f6a837fe700 10 client.16110 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:21:11.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.984+0000 7f6aa3fff700 1 --2- 172.21.15.52:0/1640390364 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5608aba69530 0x5608aba6b940 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27cf373b8b2a15d5 server_cookie=5b6f2b97517284ae in_seq=0 out_seq=0 2023-11-27T15:21:11.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.989+0000 7f6a837fe700 1 -- 172.21.15.52:0/1640390364 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5608ab916660 con 0x5608aba69530 2023-11-27T15:21:11.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.990+0000 7f6a837fe700 10 client.16110 handle_command_reply: tid=0 2023-11-27T15:21:11.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.990+0000 7f6ab6803b80 1 client.16110 shutdown 2023-11-27T15:21:11.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.990+0000 7f6ab6803b80 1 -- 172.21.15.52:0/1640390364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608ab9fb8b0 msgr2=0x5608aba06410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:11.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.990+0000 7f6ab6803b80 1 --2- 172.21.15.52:0/1640390364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608ab9fb8b0 0x5608aba06410 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c009fd0 tx=0x7f6a9c00be10 comp rx=0 tx=0).stop 2023-11-27T15:21:11.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.990+0000 7f6ab6803b80 1 -- 172.21.15.52:0/1640390364 shutdown_connections 2023-11-27T15:21:11.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.990+0000 7f6ab6803b80 1 --2- 172.21.15.52:0/1640390364 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5608aba69530 0x5608aba6b940 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.990+0000 7f6ab6803b80 1 --2- 172.21.15.52:0/1640390364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608ab9fb8b0 0x5608aba06410 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.990+0000 7f6ab6803b80 1 --2- 172.21.15.52:0/1640390364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5608ab9f6b60 0x5608aba05990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.990+0000 7f6a837fe700 0 client.16110 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:21:11.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.990+0000 7f6ab6803b80 1 --2- 172.21.15.52:0/1640390364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608ab9f7470 0x5608aba05ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.990+0000 7f6ab6803b80 1 -- 172.21.15.52:0/1640390364 >> 172.21.15.52:0/1640390364 conn(0x5608ab9ec360 msgr2=0x5608ab9f5960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:11.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.990+0000 7f6ab6803b80 1 -- 172.21.15.52:0/1640390364 shutdown_connections 2023-11-27T15:21:11.996 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.990+0000 7f6ab6803b80 1 -- 172.21.15.52:0/1640390364 wait complete. 2023-11-27T15:21:11.997 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.990+0000 7f6ab6803b80 20 client.16110 trim_cache size 0 max 16384 2023-11-27T15:21:11.997 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:21:11.997 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-27T15:21:11.997 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2023-11-27T15:21:11.997 INFO:teuthology.orchestra.run.smithi052.stdout: "1a14f367-9a76-4e86-b4c8-bd13c3d4d297": { 2023-11-27T15:21:11.997 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2023-11-27T15:21:11.997 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "1a14f367-9a76-4e86-b4c8-bd13c3d4d297", 2023-11-27T15:21:11.997 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2023-11-27T15:21:11.998 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:21:11.998 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:21:11.998 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:21:11.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.990+0000 7f6aaa677700 1 -- 172.21.15.52:0/3560317699 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f6a8c0a1340 msgr2=0x7f6a8c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:11.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.991+0000 7f6aaa677700 1 --2- 172.21.15.52:0/3560317699 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f6a8c0a1340 0x7f6a8c0a37f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4068eb0 tx=0x7f6a98009260 comp rx=0 tx=0).stop 2023-11-27T15:21:11.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.991+0000 7f6aaa677700 1 -- 172.21.15.52:0/3560317699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6aa40ff450 msgr2=0x7f6aa4195400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:11.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.991+0000 7f6aaa677700 1 --2- 172.21.15.52:0/3560317699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6aa40ff450 0x7f6aa4195400 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400d980 tx=0x7f6a9400dc90 comp rx=0 tx=0).stop 2023-11-27T15:21:11.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.991+0000 7f6aaa677700 1 -- 172.21.15.52:0/3560317699 shutdown_connections 2023-11-27T15:21:11.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.991+0000 7f6aaa677700 1 --2- 172.21.15.52:0/3560317699 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f6a8c0a1340 0x7f6a8c0a37f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.991+0000 7f6aaa677700 1 --2- 172.21.15.52:0/3560317699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6aa4104490 0x7f6aa41a2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.999 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.991+0000 7f6aaa677700 1 --2- 172.21.15.52:0/3560317699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6aa40ff450 0x7f6aa4195400 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:12.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.991+0000 7f6aaa677700 1 --2- 172.21.15.52:0/3560317699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6aa40ffdf0 0x7f6aa4195940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:12.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.991+0000 7f6aaa677700 1 -- 172.21.15.52:0/3560317699 >> 172.21.15.52:0/3560317699 conn(0x7f6aa40770b0 msgr2=0x7f6aa40fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:12.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.991+0000 7f6aaa677700 1 -- 172.21.15.52:0/3560317699 shutdown_connections 2023-11-27T15:21:12.000 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:11.991+0000 7f6aaa677700 1 -- 172.21.15.52:0/3560317699 wait complete. 2023-11-27T15:21:12.007 INFO:tasks.cephfs.filesystem:scrub status for tag:1a14f367-9a76-4e86-b4c8-bd13c3d4d297 - {'path': '/', 'tag': '1a14f367-9a76-4e86-b4c8-bd13c3d4d297', 'options': 'recursive,force'} 2023-11-27T15:21:12.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:11 smithi052 ceph-mon[99056]: pgmap v101: 97 pgs: 97 active+clean; 260 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.3 MiB/s wr, 56 op/s 2023-11-27T15:21:12.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:11 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:12.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:11 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:21:12.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:11 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:21:12.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:11 smithi150 ceph-mon[100559]: pgmap v101: 97 pgs: 97 active+clean; 260 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.3 MiB/s wr, 56 op/s 2023-11-27T15:21:12.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:11 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:12.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:11 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:21:12.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:11 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:21:12.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:11 smithi102 ceph-mon[100296]: pgmap v101: 97 pgs: 97 active+clean; 260 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.3 MiB/s wr, 56 op/s 2023-11-27T15:21:12.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:11 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:12.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:11 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:21:12.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:11 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:21:13.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:12 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:21:13.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:12 smithi052 ceph-mon[99056]: Service mds.all has no related daemon already placed. 2023-11-27T15:21:13.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:12 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:21:13.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:12 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:21:13.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:12 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:21:13.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:12 smithi150 ceph-mon[100559]: Service mds.all has no related daemon already placed. 2023-11-27T15:21:13.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:12 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:21:13.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:12 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:21:13.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:12 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:21:13.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:12 smithi102 ceph-mon[100296]: Service mds.all has no related daemon already placed. 2023-11-27T15:21:13.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:12 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:21:13.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:12 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:21:14.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:13 smithi052 ceph-mon[99056]: pgmap v102: 97 pgs: 97 active+clean; 260 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.5 MiB/s wr, 39 op/s 2023-11-27T15:21:14.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:13 smithi150 ceph-mon[100559]: pgmap v102: 97 pgs: 97 active+clean; 260 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.5 MiB/s wr, 39 op/s 2023-11-27T15:21:14.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:13 smithi102 ceph-mon[100296]: pgmap v102: 97 pgs: 97 active+clean; 260 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.5 MiB/s wr, 39 op/s 2023-11-27T15:21:15.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:14 smithi052 ceph-mon[99056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:21:15.277 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:14 smithi150 ceph-mon[100559]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:21:15.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:14 smithi102 ceph-mon[100296]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:21:16.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:15 smithi052 ceph-mon[99056]: pgmap v103: 97 pgs: 97 active+clean; 260 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.5 MiB/s wr, 39 op/s 2023-11-27T15:21:16.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:15 smithi150 ceph-mon[100559]: pgmap v103: 97 pgs: 97 active+clean; 260 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.5 MiB/s wr, 39 op/s 2023-11-27T15:21:16.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:15 smithi102 ceph-mon[100296]: pgmap v103: 97 pgs: 97 active+clean; 260 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.5 MiB/s wr, 39 op/s 2023-11-27T15:21:18.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:18 smithi150 ceph-mon[100559]: pgmap v104: 97 pgs: 97 active+clean; 296 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 55 op/s 2023-11-27T15:21:18.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:18 smithi102 ceph-mon[100296]: pgmap v104: 97 pgs: 97 active+clean; 296 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 55 op/s 2023-11-27T15:21:18.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:18 smithi052 ceph-mon[99056]: pgmap v104: 97 pgs: 97 active+clean; 296 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 55 op/s 2023-11-27T15:21:20.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:20 smithi150 ceph-mon[100559]: pgmap v105: 97 pgs: 97 active+clean; 296 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 7.1 MiB/s wr, 29 op/s 2023-11-27T15:21:20.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:20 smithi150 ceph-mon[100559]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:21:20.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:20 smithi102 ceph-mon[100296]: pgmap v105: 97 pgs: 97 active+clean; 296 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 7.1 MiB/s wr, 29 op/s 2023-11-27T15:21:20.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:20 smithi102 ceph-mon[100296]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:21:20.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:20 smithi052 ceph-mon[99056]: pgmap v105: 97 pgs: 97 active+clean; 296 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 7.1 MiB/s wr, 29 op/s 2023-11-27T15:21:20.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:20 smithi052 ceph-mon[99056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:21:22.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:22 smithi150 ceph-mon[100559]: pgmap v106: 97 pgs: 97 active+clean; 330 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 45 op/s 2023-11-27T15:21:22.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:22 smithi102 ceph-mon[100296]: pgmap v106: 97 pgs: 97 active+clean; 330 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 45 op/s 2023-11-27T15:21:22.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:22 smithi052 ceph-mon[99056]: pgmap v106: 97 pgs: 97 active+clean; 330 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 45 op/s 2023-11-27T15:21:24.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:24 smithi150 ceph-mon[100559]: pgmap v107: 97 pgs: 97 active+clean; 330 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.9 MiB/s wr, 31 op/s 2023-11-27T15:21:24.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:24 smithi102 ceph-mon[100296]: pgmap v107: 97 pgs: 97 active+clean; 330 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.9 MiB/s wr, 31 op/s 2023-11-27T15:21:24.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:24 smithi052 ceph-mon[99056]: pgmap v107: 97 pgs: 97 active+clean; 330 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.9 MiB/s wr, 31 op/s 2023-11-27T15:21:26.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:26 smithi150 ceph-mon[100559]: pgmap v108: 97 pgs: 97 active+clean; 330 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.9 MiB/s wr, 31 op/s 2023-11-27T15:21:26.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:26 smithi102 ceph-mon[100296]: pgmap v108: 97 pgs: 97 active+clean; 330 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.9 MiB/s wr, 31 op/s 2023-11-27T15:21:26.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:26 smithi052 ceph-mon[99056]: pgmap v108: 97 pgs: 97 active+clean; 330 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.9 MiB/s wr, 31 op/s 2023-11-27T15:21:28.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:28 smithi150 ceph-mon[100559]: pgmap v109: 97 pgs: 97 active+clean; 359 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.3 MiB/s wr, 83 op/s 2023-11-27T15:21:28.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:28 smithi102 ceph-mon[100296]: pgmap v109: 97 pgs: 97 active+clean; 359 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.3 MiB/s wr, 83 op/s 2023-11-27T15:21:28.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:28 smithi052 ceph-mon[99056]: pgmap v109: 97 pgs: 97 active+clean; 359 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.3 MiB/s wr, 83 op/s 2023-11-27T15:21:30.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:29 smithi052 ceph-mon[99056]: pgmap v110: 97 pgs: 97 active+clean; 359 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.2 MiB/s wr, 68 op/s 2023-11-27T15:21:30.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:29 smithi150 ceph-mon[100559]: pgmap v110: 97 pgs: 97 active+clean; 359 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.2 MiB/s wr, 68 op/s 2023-11-27T15:21:30.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:29 smithi102 ceph-mon[100296]: pgmap v110: 97 pgs: 97 active+clean; 359 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.2 MiB/s wr, 68 op/s 2023-11-27T15:21:32.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:32 smithi150 ceph-mon[100559]: pgmap v111: 97 pgs: 97 active+clean; 381 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.1 MiB/s wr, 86 op/s 2023-11-27T15:21:32.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:32 smithi102 ceph-mon[100296]: pgmap v111: 97 pgs: 97 active+clean; 381 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.1 MiB/s wr, 86 op/s 2023-11-27T15:21:32.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:32 smithi052 ceph-mon[99056]: pgmap v111: 97 pgs: 97 active+clean; 381 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.1 MiB/s wr, 86 op/s 2023-11-27T15:21:34.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:34 smithi150 ceph-mon[100559]: pgmap v112: 97 pgs: 97 active+clean; 381 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.3 MiB/s wr, 70 op/s 2023-11-27T15:21:34.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:34 smithi102 ceph-mon[100296]: pgmap v112: 97 pgs: 97 active+clean; 381 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.3 MiB/s wr, 70 op/s 2023-11-27T15:21:34.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:34 smithi052 ceph-mon[99056]: pgmap v112: 97 pgs: 97 active+clean; 381 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.3 MiB/s wr, 70 op/s 2023-11-27T15:21:36.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:36 smithi150 ceph-mon[100559]: pgmap v113: 97 pgs: 97 active+clean; 381 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.3 MiB/s wr, 70 op/s 2023-11-27T15:21:36.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:36 smithi102 ceph-mon[100296]: pgmap v113: 97 pgs: 97 active+clean; 381 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.3 MiB/s wr, 70 op/s 2023-11-27T15:21:36.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:36 smithi052 ceph-mon[99056]: pgmap v113: 97 pgs: 97 active+clean; 381 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.3 MiB/s wr, 70 op/s 2023-11-27T15:21:38.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:38 smithi150 ceph-mon[100559]: pgmap v114: 97 pgs: 97 active+clean; 403 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.1 MiB/s wr, 94 op/s 2023-11-27T15:21:38.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:38 smithi102 ceph-mon[100296]: pgmap v114: 97 pgs: 97 active+clean; 403 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.1 MiB/s wr, 94 op/s 2023-11-27T15:21:38.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:38 smithi052 ceph-mon[99056]: pgmap v114: 97 pgs: 97 active+clean; 403 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.1 MiB/s wr, 94 op/s 2023-11-27T15:21:40.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:39 smithi052 ceph-mon[99056]: pgmap v115: 97 pgs: 97 active+clean; 403 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 42 op/s 2023-11-27T15:21:40.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:39 smithi150 ceph-mon[100559]: pgmap v115: 97 pgs: 97 active+clean; 403 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 42 op/s 2023-11-27T15:21:40.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:39 smithi102 ceph-mon[100296]: pgmap v115: 97 pgs: 97 active+clean; 403 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 42 op/s 2023-11-27T15:21:42.009 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:21:42.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.145+0000 7fa4820af700 1 -- 172.21.15.52:0/1493044308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa47c1009e0 msgr2=0x7fa47c100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:42.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.145+0000 7fa4820af700 1 --2- 172.21.15.52:0/1493044308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa47c1009e0 0x7fa47c100e30 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa470009990 tx=0x7fa470009ca0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.145+0000 7fa4820af700 1 -- 172.21.15.52:0/1493044308 shutdown_connections 2023-11-27T15:21:42.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.145+0000 7fa4820af700 1 --2- 172.21.15.52:0/1493044308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa47c101370 0x7fa47c10d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.145+0000 7fa4820af700 1 --2- 172.21.15.52:0/1493044308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa47c106410 0x7fa47c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.145+0000 7fa4820af700 1 --2- 172.21.15.52:0/1493044308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa47c1009e0 0x7fa47c100e30 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.145+0000 7fa4820af700 1 -- 172.21.15.52:0/1493044308 >> 172.21.15.52:0/1493044308 conn(0x7fa47c0775c0 msgr2=0x7fa47c0779c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:42.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.145+0000 7fa4820af700 1 -- 172.21.15.52:0/1493044308 shutdown_connections 2023-11-27T15:21:42.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.145+0000 7fa4820af700 1 -- 172.21.15.52:0/1493044308 wait complete. 2023-11-27T15:21:42.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.146+0000 7fa4820af700 1 Processor -- start 2023-11-27T15:21:42.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.146+0000 7fa4820af700 1 -- start start 2023-11-27T15:21:42.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.146+0000 7fa4820af700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa47c1009e0 0x7fa47c06b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:42.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.146+0000 7fa4820af700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa47c101370 0x7fa47c06b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:42.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.146+0000 7fa4820af700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa47c106410 0x7fa47c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:42.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.146+0000 7fa4820af700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c06f050 con 0x7fa47c101370 2023-11-27T15:21:42.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.146+0000 7fa4820af700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c06f1c0 con 0x7fa47c106410 2023-11-27T15:21:42.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.146+0000 7fa4820af700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c06f380 con 0x7fa47c1009e0 2023-11-27T15:21:42.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.146+0000 7fa47b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa47c1009e0 0x7fa47c06b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:42.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.146+0000 7fa47bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa47c106410 0x7fa47c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:42.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.146+0000 7fa47b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa47c1009e0 0x7fa47c06b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:40356/0 (socket says 172.21.15.52:40356) 2023-11-27T15:21:42.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.146+0000 7fa47b7fe700 1 -- 172.21.15.52:0/281286993 learned_addr learned my addr 172.21.15.52:0/281286993 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:21:42.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.146+0000 7fa47bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa47c106410 0x7fa47c06e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43372/0 (socket says 172.21.15.52:43372) 2023-11-27T15:21:42.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.146+0000 7fa47affd700 1 --2- 172.21.15.52:0/281286993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa47c101370 0x7fa47c06b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:42.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.147+0000 7fa47b7fe700 1 -- 172.21.15.52:0/281286993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa47c106410 msgr2=0x7fa47c06e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:42.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.147+0000 7fa47b7fe700 1 --2- 172.21.15.52:0/281286993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa47c106410 0x7fa47c06e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.147+0000 7fa47b7fe700 1 -- 172.21.15.52:0/281286993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa47c101370 msgr2=0x7fa47c06b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:42.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.147+0000 7fa47b7fe700 1 --2- 172.21.15.52:0/281286993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa47c101370 0x7fa47c06b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.147+0000 7fa47b7fe700 1 -- 172.21.15.52:0/281286993 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4700095f0 con 0x7fa47c1009e0 2023-11-27T15:21:42.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.147+0000 7fa47bfff700 1 --2- 172.21.15.52:0/281286993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa47c106410 0x7fa47c06e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:21:42.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.147+0000 7fa47b7fe700 1 --2- 172.21.15.52:0/281286993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa47c1009e0 0x7fa47c06b060 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa46c009cb0 tx=0x7fa46c00b2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:42.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.147+0000 7fa478ff9700 1 -- 172.21.15.52:0/281286993 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c015070 con 0x7fa47c1009e0 2023-11-27T15:21:42.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.147+0000 7fa478ff9700 1 -- 172.21.15.52:0/281286993 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa46c00bb50 con 0x7fa47c1009e0 2023-11-27T15:21:42.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.147+0000 7fa4820af700 1 -- 172.21.15.52:0/281286993 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa47c110430 con 0x7fa47c1009e0 2023-11-27T15:21:42.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.147+0000 7fa478ff9700 1 -- 172.21.15.52:0/281286993 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c018610 con 0x7fa47c1009e0 2023-11-27T15:21:42.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.147+0000 7fa4820af700 1 -- 172.21.15.52:0/281286993 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa47c1102a0 con 0x7fa47c1009e0 2023-11-27T15:21:42.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.150+0000 7fa478ff9700 1 -- 172.21.15.52:0/281286993 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fa46c00bcc0 con 0x7fa47c1009e0 2023-11-27T15:21:42.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.151+0000 7fa478ff9700 1 --2- 172.21.15.52:0/281286993 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fa4640a12c0 0x7fa4640a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:42.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.151+0000 7fa478ff9700 1 -- 172.21.15.52:0/281286993 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fa46c0d0470 con 0x7fa47c1009e0 2023-11-27T15:21:42.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.151+0000 7fa47affd700 1 --2- 172.21.15.52:0/281286993 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fa4640a12c0 0x7fa4640a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:42.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.151+0000 7fa47affd700 1 --2- 172.21.15.52:0/281286993 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fa4640a12c0 0x7fa4640a3770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa47c06c670 tx=0x7fa470005fd0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:42.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.151+0000 7fa48e23bb80 1 Processor -- start 2023-11-27T15:21:42.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.151+0000 7fa48e23bb80 1 -- start start 2023-11-27T15:21:42.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.152+0000 7fa48e23bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c50d01eb60 0x55c50d01ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:42.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.152+0000 7fa48e23bb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c50d01f470 0x55c50d023370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:42.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.152+0000 7fa48e23bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c50d0238b0 0x55c50d023d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:42.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.152+0000 7fa48e23bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50cf24060 con 0x55c50d0238b0 2023-11-27T15:21:42.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.152+0000 7fa48e23bb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50cf768a0 con 0x55c50d01f470 2023-11-27T15:21:42.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.152+0000 7fa48e23bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50cf6fd30 con 0x55c50d01eb60 2023-11-27T15:21:42.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.152+0000 7fa47bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c50d0238b0 0x55c50d023d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:42.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.152+0000 7fa47affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c50d01f470 0x55c50d023370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:42.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.152+0000 7fa47b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c50d01eb60 0x55c50d01ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:42.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.152+0000 7fa47bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c50d0238b0 0x55c50d023d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53724/0 (socket says 172.21.15.52:53724) 2023-11-27T15:21:42.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.152+0000 7fa47affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c50d01f470 0x55c50d023370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43380/0 (socket says 172.21.15.52:43380) 2023-11-27T15:21:42.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.152+0000 7fa47b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c50d01eb60 0x55c50d01ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:40364/0 (socket says 172.21.15.52:40364) 2023-11-27T15:21:42.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.152+0000 7fa47bfff700 1 -- 172.21.15.52:0/1922729861 learned_addr learned my addr 172.21.15.52:0/1922729861 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:21:42.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.152+0000 7fa47bfff700 1 -- 172.21.15.52:0/1922729861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c50d01eb60 msgr2=0x55c50d01ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:21:42.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.152+0000 7fa47bfff700 1 --2- 172.21.15.52:0/1922729861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c50d01eb60 0x55c50d01ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.152+0000 7fa47bfff700 1 -- 172.21.15.52:0/1922729861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c50d01f470 msgr2=0x55c50d023370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:21:42.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.152+0000 7fa47bfff700 1 --2- 172.21.15.52:0/1922729861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c50d01f470 0x55c50d023370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.152+0000 7fa47bfff700 1 -- 172.21.15.52:0/1922729861 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c50cdf1630 con 0x55c50d0238b0 2023-11-27T15:21:42.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.152+0000 7fa47affd700 1 --2- 172.21.15.52:0/1922729861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c50d01f470 0x55c50d023370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:21:42.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.152+0000 7fa47bfff700 1 --2- 172.21.15.52:0/1922729861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c50d0238b0 0x55c50d023d00 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa474003eb0 tx=0x7fa474009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df944c4103e70543 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:42.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.152+0000 7fa461ffb700 1 -- 172.21.15.52:0/1922729861 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4740092d0 con 0x55c50d0238b0 2023-11-27T15:21:42.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.153+0000 7fa461ffb700 1 -- 172.21.15.52:0/1922729861 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa47400f040 con 0x55c50d0238b0 2023-11-27T15:21:42.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.153+0000 7fa461ffb700 1 -- 172.21.15.52:0/1922729861 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47400e7e0 con 0x55c50d0238b0 2023-11-27T15:21:42.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.153+0000 7fa48e23bb80 1 -- 172.21.15.52:0/1922729861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c50d0238b0 msgr2=0x55c50d023d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:42.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.153+0000 7fa48e23bb80 1 --2- 172.21.15.52:0/1922729861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c50d0238b0 0x55c50d023d00 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa474003eb0 tx=0x7fa474009920 comp rx=0 tx=0).stop 2023-11-27T15:21:42.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.153+0000 7fa48e23bb80 1 -- 172.21.15.52:0/1922729861 shutdown_connections 2023-11-27T15:21:42.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.153+0000 7fa48e23bb80 1 --2- 172.21.15.52:0/1922729861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c50d0238b0 0x55c50d023d00 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.153+0000 7fa48e23bb80 1 --2- 172.21.15.52:0/1922729861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c50d01eb60 0x55c50d01ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.153+0000 7fa48e23bb80 1 --2- 172.21.15.52:0/1922729861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c50d01f470 0x55c50d023370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.153+0000 7fa48e23bb80 1 -- 172.21.15.52:0/1922729861 >> 172.21.15.52:0/1922729861 conn(0x55c50d014360 msgr2=0x55c50d019b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:42.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.154+0000 7fa48e23bb80 1 -- 172.21.15.52:0/1922729861 shutdown_connections 2023-11-27T15:21:42.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.154+0000 7fa48e23bb80 1 -- 172.21.15.52:0/1922729861 wait complete. 2023-11-27T15:21:42.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.154+0000 7fa48e23bb80 1 -- start start 2023-11-27T15:21:42.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.154+0000 7fa48e23bb80 1 Processor -- start 2023-11-27T15:21:42.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.155+0000 7fa48e23bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c50d01eb60 0x55c50d02afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:42.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.155+0000 7fa48e23bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c50d01f470 0x55c50d028250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:42.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.155+0000 7fa48e23bb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c50d0238b0 0x55c50d028880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:42.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.155+0000 7fa48e23bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50cf118e0 con 0x55c50d01eb60 2023-11-27T15:21:42.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.155+0000 7fa48e23bb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50cf11580 con 0x55c50d0238b0 2023-11-27T15:21:42.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.155+0000 7fa48e23bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50cdf3920 con 0x55c50d01f470 2023-11-27T15:21:42.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.155+0000 7fa48e23bb80 10 client.0 _ll_register_callbacks cb 0x55c50cf2f780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:21:42.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.155+0000 7fa47b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c50d01eb60 0x55c50d02afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:42.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.155+0000 7fa47bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c50d0238b0 0x55c50d028880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:42.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.155+0000 7fa47affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c50d01f470 0x55c50d028250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:42.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.155+0000 7fa47b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c50d01eb60 0x55c50d02afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53726/0 (socket says 172.21.15.52:53726) 2023-11-27T15:21:42.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.155+0000 7fa47b7fe700 1 -- 172.21.15.52:0/3395782648 learned_addr learned my addr 172.21.15.52:0/3395782648 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:21:42.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.155+0000 7fa47bfff700 1 -- 172.21.15.52:0/3395782648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c50d01f470 msgr2=0x55c50d028250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:21:42.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.155+0000 7fa47bfff700 1 --2- 172.21.15.52:0/3395782648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c50d01f470 0x55c50d028250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.155+0000 7fa47bfff700 1 -- 172.21.15.52:0/3395782648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c50d01eb60 msgr2=0x55c50d02afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:21:42.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.155+0000 7fa47bfff700 1 --2- 172.21.15.52:0/3395782648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c50d01eb60 0x55c50d02afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.155+0000 7fa47bfff700 1 -- 172.21.15.52:0/3395782648 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa474003b90 con 0x55c50d0238b0 2023-11-27T15:21:42.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.155+0000 7fa47b7fe700 1 --2- 172.21.15.52:0/3395782648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c50d01eb60 0x55c50d02afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:21:42.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.155+0000 7fa47affd700 1 --2- 172.21.15.52:0/3395782648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c50d01f470 0x55c50d028250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:21:42.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.155+0000 7fa47bfff700 1 --2- 172.21.15.52:0/3395782648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c50d0238b0 0x55c50d028880 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa474007e70 tx=0x7fa474004d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=656b09da16fd174a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:42.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.155+0000 7fa45affd700 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2023-11-27T15:21:42.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.156+0000 7fa45affd700 1 -- 172.21.15.52:0/3395782648 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa474007910 con 0x55c50d0238b0 2023-11-27T15:21:42.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.156+0000 7fa45affd700 1 -- 172.21.15.52:0/3395782648 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fa474007a70 con 0x55c50d0238b0 2023-11-27T15:21:42.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.156+0000 7fa48e23bb80 1 -- 172.21.15.52:0/3395782648 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c50d011b20 con 0x55c50d0238b0 2023-11-27T15:21:42.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.156+0000 7fa45affd700 1 -- 172.21.15.52:0/3395782648 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa474007c90 con 0x55c50d0238b0 2023-11-27T15:21:42.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.156+0000 7fa48e23bb80 10 client.25937 fetch_fsmap learned FSMap version 28 2023-11-27T15:21:42.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.156+0000 7fa48e23bb80 1 -- 172.21.15.52:0/3395782648 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c50cf36190 con 0x55c50d0238b0 2023-11-27T15:21:42.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.156+0000 7fa45affd700 1 -- 172.21.15.52:0/3395782648 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c50d011b20 con 0x55c50d0238b0 2023-11-27T15:21:42.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.156+0000 7fa45affd700 1 -- 172.21.15.52:0/3395782648 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa4740316b0 con 0x55c50d0238b0 2023-11-27T15:21:42.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.156+0000 7fa48e23bb80 10 client.25937 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:21:42.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.156+0000 7fa48e23bb80 10 client.25937 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:21:42.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.156+0000 7fa48e23bb80 20 client.25937 populate_metadata read hostname 'smithi052' 2023-11-27T15:21:42.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.156+0000 7fa48e23bb80 1 --2- 172.21.15.52:0/3395782648 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c50d0386e0 0x55c50d03aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:42.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.156+0000 7fa48e23bb80 4 client.25937 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:21:42.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.156+0000 7fa48e23bb80 1 -- 172.21.15.52:0/3395782648 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c50cf320a0 con 0x55c50d0386e0 2023-11-27T15:21:42.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.156+0000 7fa47b7fe700 1 --2- 172.21.15.52:0/3395782648 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c50d0386e0 0x55c50d03aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:42.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.157+0000 7fa45affd700 10 client.25937 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:21:42.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.157+0000 7fa47b7fe700 1 --2- 172.21.15.52:0/3395782648 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c50d0386e0 0x55c50d03aaf0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f89c5718ddc6c960 server_cookie=b2582c34ac1e68cd in_seq=0 out_seq=0 2023-11-27T15:21:42.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.163+0000 7fa45affd700 1 -- 172.21.15.52:0/3395782648 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c50cf320a0 con 0x55c50d0386e0 2023-11-27T15:21:42.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.163+0000 7fa45affd700 10 client.25937 handle_command_reply: tid=0 2023-11-27T15:21:42.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.163+0000 7fa48e23bb80 1 client.25937 shutdown 2023-11-27T15:21:42.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.163+0000 7fa48e23bb80 1 -- 172.21.15.52:0/3395782648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c50d0238b0 msgr2=0x55c50d028880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:42.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.163+0000 7fa48e23bb80 1 --2- 172.21.15.52:0/3395782648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c50d0238b0 0x55c50d028880 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa474007e70 tx=0x7fa474004d80 comp rx=0 tx=0).stop 2023-11-27T15:21:42.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.163+0000 7fa48e23bb80 1 -- 172.21.15.52:0/3395782648 shutdown_connections 2023-11-27T15:21:42.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.163+0000 7fa48e23bb80 1 --2- 172.21.15.52:0/3395782648 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c50d0386e0 0x55c50d03aaf0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.163+0000 7fa48e23bb80 1 --2- 172.21.15.52:0/3395782648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c50d0238b0 0x55c50d028880 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.163+0000 7fa48e23bb80 1 --2- 172.21.15.52:0/3395782648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c50d01eb60 0x55c50d02afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.163+0000 7fa48e23bb80 1 --2- 172.21.15.52:0/3395782648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c50d01f470 0x55c50d028250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.163+0000 7fa45affd700 0 client.25937 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:21:42.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.163+0000 7fa48e23bb80 1 -- 172.21.15.52:0/3395782648 >> 172.21.15.52:0/3395782648 conn(0x55c50d014360 msgr2=0x55c50d024d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:42.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.163+0000 7fa48e23bb80 1 -- 172.21.15.52:0/3395782648 shutdown_connections 2023-11-27T15:21:42.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.163+0000 7fa48e23bb80 1 -- 172.21.15.52:0/3395782648 wait complete. 2023-11-27T15:21:42.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.164+0000 7fa48e23bb80 20 client.25937 trim_cache size 0 max 16384 2023-11-27T15:21:42.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.181+0000 7fa48e23bb80 1 Processor -- start 2023-11-27T15:21:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.181+0000 7fa48e23bb80 1 -- start start 2023-11-27T15:21:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.182+0000 7fa48e23bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c50d01eb60 0x55c50d019c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.182+0000 7fa48e23bb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c50d01f470 0x55c50d02f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.182+0000 7fa48e23bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c50d0238b0 0x55c50d02ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.182+0000 7fa48e23bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50d013ee0 con 0x55c50d0238b0 2023-11-27T15:21:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.182+0000 7fa48e23bb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50d01a970 con 0x55c50d01f470 2023-11-27T15:21:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.182+0000 7fa48e23bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50d022140 con 0x55c50d01eb60 2023-11-27T15:21:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.182+0000 7fa47bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c50d0238b0 0x55c50d02ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.182+0000 7fa47b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c50d01eb60 0x55c50d019c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:42.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.182+0000 7fa47affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c50d01f470 0x55c50d02f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:42.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.182+0000 7fa47b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c50d01eb60 0x55c50d019c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:40378/0 (socket says 172.21.15.52:40378) 2023-11-27T15:21:42.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.182+0000 7fa47b7fe700 1 -- 172.21.15.52:0/1244525059 learned_addr learned my addr 172.21.15.52:0/1244525059 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:21:42.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.182+0000 7fa47b7fe700 1 -- 172.21.15.52:0/1244525059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c50d01f470 msgr2=0x55c50d02f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:21:42.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.182+0000 7fa47b7fe700 1 --2- 172.21.15.52:0/1244525059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c50d01f470 0x55c50d02f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.182+0000 7fa47b7fe700 1 -- 172.21.15.52:0/1244525059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c50d0238b0 msgr2=0x55c50d02ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:21:42.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.182+0000 7fa47b7fe700 1 --2- 172.21.15.52:0/1244525059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c50d0238b0 0x55c50d02ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.182+0000 7fa47b7fe700 1 -- 172.21.15.52:0/1244525059 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c50d018820 con 0x55c50d01eb60 2023-11-27T15:21:42.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.182+0000 7fa47bfff700 1 --2- 172.21.15.52:0/1244525059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c50d0238b0 0x55c50d02ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:21:42.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.182+0000 7fa47b7fe700 1 --2- 172.21.15.52:0/1244525059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c50d01eb60 0x55c50d019c10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa46c099a40 tx=0x7fa46c0bd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=737a8c229723e82b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:42.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.183+0000 7fa4627fc700 1 -- 172.21.15.52:0/1244525059 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c0b9070 con 0x55c50d01eb60 2023-11-27T15:21:42.189 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.183+0000 7fa4627fc700 1 -- 172.21.15.52:0/1244525059 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa46c0a1570 con 0x55c50d01eb60 2023-11-27T15:21:42.189 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.183+0000 7fa4627fc700 1 -- 172.21.15.52:0/1244525059 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c0a1380 con 0x55c50d01eb60 2023-11-27T15:21:42.193 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.183+0000 7fa48e23bb80 1 -- 172.21.15.52:0/1244525059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c50d01eb60 msgr2=0x55c50d019c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:42.194 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.183+0000 7fa48e23bb80 1 --2- 172.21.15.52:0/1244525059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c50d01eb60 0x55c50d019c10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa46c099a40 tx=0x7fa46c0bd040 comp rx=0 tx=0).stop 2023-11-27T15:21:42.194 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.183+0000 7fa48e23bb80 1 -- 172.21.15.52:0/1244525059 shutdown_connections 2023-11-27T15:21:42.194 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.183+0000 7fa48e23bb80 1 --2- 172.21.15.52:0/1244525059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c50d0238b0 0x55c50d02ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.194 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.183+0000 7fa48e23bb80 1 --2- 172.21.15.52:0/1244525059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c50d01eb60 0x55c50d019c10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.194 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.183+0000 7fa48e23bb80 1 --2- 172.21.15.52:0/1244525059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c50d01f470 0x55c50d02f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.194 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.183+0000 7fa48e23bb80 1 -- 172.21.15.52:0/1244525059 >> 172.21.15.52:0/1244525059 conn(0x55c50d014360 msgr2=0x55c50d034c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:42.194 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.183+0000 7fa48e23bb80 1 -- 172.21.15.52:0/1244525059 shutdown_connections 2023-11-27T15:21:42.194 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.184+0000 7fa48e23bb80 1 -- 172.21.15.52:0/1244525059 wait complete. 2023-11-27T15:21:42.194 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.184+0000 7fa48e23bb80 1 -- start start 2023-11-27T15:21:42.194 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.184+0000 7fa48e23bb80 1 Processor -- start 2023-11-27T15:21:42.194 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.185+0000 7fa48e23bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c50d01eb60 0x55c50d02f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:42.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.185+0000 7fa48e23bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c50d01f470 0x55c50d02fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:42.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.185+0000 7fa48e23bb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c50d0238b0 0x55c50d02d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:42.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.185+0000 7fa48e23bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50d02df70 con 0x55c50d01f470 2023-11-27T15:21:42.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.185+0000 7fa48e23bb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50d02e0e0 con 0x55c50d0238b0 2023-11-27T15:21:42.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.185+0000 7fa48e23bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50d02e2a0 con 0x55c50d01eb60 2023-11-27T15:21:42.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.185+0000 7fa48e23bb80 10 client.0 _ll_register_callbacks cb 0x7fa470005e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:21:42.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.185+0000 7fa47affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c50d01f470 0x55c50d02fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:42.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.185+0000 7fa47affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c50d01f470 0x55c50d02fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53742/0 (socket says 172.21.15.52:53742) 2023-11-27T15:21:42.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.185+0000 7fa47affd700 1 -- 172.21.15.52:0/4072166910 learned_addr learned my addr 172.21.15.52:0/4072166910 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:21:42.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.185+0000 7fa47bfff700 1 --2- 172.21.15.52:0/4072166910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c50d0238b0 0x55c50d02d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:42.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.185+0000 7fa47b7fe700 1 --2- 172.21.15.52:0/4072166910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c50d01eb60 0x55c50d02f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:42.195 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.185+0000 7fa47affd700 1 -- 172.21.15.52:0/4072166910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c50d01eb60 msgr2=0x55c50d02f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:21:42.196 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.185+0000 7fa47affd700 1 --2- 172.21.15.52:0/4072166910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c50d01eb60 0x55c50d02f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.196 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.185+0000 7fa47affd700 1 -- 172.21.15.52:0/4072166910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c50d0238b0 msgr2=0x55c50d02d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:21:42.196 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.185+0000 7fa47affd700 1 --2- 172.21.15.52:0/4072166910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c50d0238b0 0x55c50d02d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.196 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.185+0000 7fa47affd700 1 -- 172.21.15.52:0/4072166910 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c50cf36190 con 0x55c50d01f470 2023-11-27T15:21:42.196 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.185+0000 7fa47bfff700 1 --2- 172.21.15.52:0/4072166910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c50d0238b0 0x55c50d02d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:21:42.196 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.185+0000 7fa47affd700 1 --2- 172.21.15.52:0/4072166910 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c50d01f470 0x55c50d02fa70 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa4700098e0 tx=0x7fa470009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff87a2a045a966d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:42.196 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.185+0000 7fa45affd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:21:42.196 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.185+0000 7fa45affd700 1 -- 172.21.15.52:0/4072166910 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47002b040 con 0x55c50d01f470 2023-11-27T15:21:42.196 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.185+0000 7fa45affd700 1 -- 172.21.15.52:0/4072166910 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7fa47000f660 con 0x55c50d01f470 2023-11-27T15:21:42.196 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.186+0000 7fa48e23bb80 1 -- 172.21.15.52:0/4072166910 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa47400f040 con 0x55c50d01f470 2023-11-27T15:21:42.196 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.186+0000 7fa45affd700 1 -- 172.21.15.52:0/4072166910 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa47000f820 con 0x55c50d01f470 2023-11-27T15:21:42.196 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.186+0000 7fa48e23bb80 10 client.16128 fetch_fsmap learned FSMap version 28 2023-11-27T15:21:42.196 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.186+0000 7fa48e23bb80 1 -- 172.21.15.52:0/4072166910 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fa47c1102a0 con 0x55c50d01f470 2023-11-27T15:21:42.197 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.186+0000 7fa45affd700 1 -- 172.21.15.52:0/4072166910 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa47400f040 con 0x55c50d01f470 2023-11-27T15:21:42.197 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.186+0000 7fa45affd700 1 -- 172.21.15.52:0/4072166910 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa4700339e0 con 0x55c50d01f470 2023-11-27T15:21:42.197 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.186+0000 7fa48e23bb80 10 client.16128 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:21:42.197 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.186+0000 7fa48e23bb80 10 client.16128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:21:42.197 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.186+0000 7fa48e23bb80 20 client.16128 populate_metadata read hostname 'smithi052' 2023-11-27T15:21:42.197 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.186+0000 7fa48e23bb80 1 --2- 172.21.15.52:0/4072166910 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c50d08d810 0x55c50d08fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:42.197 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.186+0000 7fa48e23bb80 4 client.16128 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:21:42.197 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.186+0000 7fa48e23bb80 1 -- 172.21.15.52:0/4072166910 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c50cf3e660 con 0x55c50d08d810 2023-11-27T15:21:42.197 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.186+0000 7fa47b7fe700 1 --2- 172.21.15.52:0/4072166910 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c50d08d810 0x55c50d08fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:42.197 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.187+0000 7fa47b7fe700 1 --2- 172.21.15.52:0/4072166910 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c50d08d810 0x55c50d08fc20 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=813bfcab8da51c2e server_cookie=3037751e50f72b13 in_seq=0 out_seq=0 2023-11-27T15:21:42.197 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.187+0000 7fa45affd700 10 client.16128 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:21:42.197 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.187+0000 7fa45affd700 1 -- 172.21.15.52:0/4072166910 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c50cf3e660 con 0x55c50d08d810 2023-11-27T15:21:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.187+0000 7fa45affd700 10 client.16128 handle_command_reply: tid=0 2023-11-27T15:21:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.187+0000 7fa48e23bb80 1 client.16128 shutdown 2023-11-27T15:21:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.187+0000 7fa48e23bb80 1 -- 172.21.15.52:0/4072166910 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c50d01f470 msgr2=0x55c50d02fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.187+0000 7fa48e23bb80 1 --2- 172.21.15.52:0/4072166910 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c50d01f470 0x55c50d02fa70 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa4700098e0 tx=0x7fa470009ca0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.187+0000 7fa48e23bb80 1 -- 172.21.15.52:0/4072166910 shutdown_connections 2023-11-27T15:21:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.187+0000 7fa48e23bb80 1 --2- 172.21.15.52:0/4072166910 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c50d08d810 0x55c50d08fc20 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.187+0000 7fa48e23bb80 1 --2- 172.21.15.52:0/4072166910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c50d0238b0 0x55c50d02d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.187+0000 7fa48e23bb80 1 --2- 172.21.15.52:0/4072166910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c50d01eb60 0x55c50d02f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.187+0000 7fa48e23bb80 1 --2- 172.21.15.52:0/4072166910 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c50d01f470 0x55c50d02fa70 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.188+0000 7fa48e23bb80 1 -- 172.21.15.52:0/4072166910 >> 172.21.15.52:0/4072166910 conn(0x55c50d014360 msgr2=0x55c50d01d960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:42.198 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.188+0000 7fa45affd700 0 client.16128 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:21:42.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.188+0000 7fa48e23bb80 1 -- 172.21.15.52:0/4072166910 shutdown_connections 2023-11-27T15:21:42.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.188+0000 7fa48e23bb80 1 -- 172.21.15.52:0/4072166910 wait complete. 2023-11-27T15:21:42.199 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.188+0000 7fa48e23bb80 20 client.16128 trim_cache size 0 max 16384 2023-11-27T15:21:42.199 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:21:42.199 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-27T15:21:42.199 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2023-11-27T15:21:42.199 INFO:teuthology.orchestra.run.smithi052.stdout: "1a14f367-9a76-4e86-b4c8-bd13c3d4d297": { 2023-11-27T15:21:42.199 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2023-11-27T15:21:42.199 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "1a14f367-9a76-4e86-b4c8-bd13c3d4d297", 2023-11-27T15:21:42.200 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2023-11-27T15:21:42.200 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:21:42.200 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:21:42.200 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:21:42.200 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.188+0000 7fa4820af700 1 -- 172.21.15.52:0/281286993 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fa4640a12c0 msgr2=0x7fa4640a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:42.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.188+0000 7fa4820af700 1 --2- 172.21.15.52:0/281286993 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fa4640a12c0 0x7fa4640a3770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa47c06c670 tx=0x7fa470005fd0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.188+0000 7fa4820af700 1 -- 172.21.15.52:0/281286993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa47c1009e0 msgr2=0x7fa47c06b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:42.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.188+0000 7fa4820af700 1 --2- 172.21.15.52:0/281286993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa47c1009e0 0x7fa47c06b060 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa46c009cb0 tx=0x7fa46c00b2c0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.188+0000 7fa4820af700 1 -- 172.21.15.52:0/281286993 shutdown_connections 2023-11-27T15:21:42.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.188+0000 7fa4820af700 1 --2- 172.21.15.52:0/281286993 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fa4640a12c0 0x7fa4640a3770 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.188+0000 7fa4820af700 1 --2- 172.21.15.52:0/281286993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa47c106410 0x7fa47c06e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.188+0000 7fa4820af700 1 --2- 172.21.15.52:0/281286993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa47c1009e0 0x7fa47c06b060 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.188+0000 7fa4820af700 1 --2- 172.21.15.52:0/281286993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa47c101370 0x7fa47c06b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.188+0000 7fa4820af700 1 -- 172.21.15.52:0/281286993 >> 172.21.15.52:0/281286993 conn(0x7fa47c0775c0 msgr2=0x7fa47c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:42.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.188+0000 7fa4820af700 1 -- 172.21.15.52:0/281286993 shutdown_connections 2023-11-27T15:21:42.201 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:21:42.188+0000 7fa4820af700 1 -- 172.21.15.52:0/281286993 wait complete. 2023-11-27T15:21:42.205 INFO:tasks.cephfs.filesystem:scrub status for tag:1a14f367-9a76-4e86-b4c8-bd13c3d4d297 - {'path': '/', 'tag': '1a14f367-9a76-4e86-b4c8-bd13c3d4d297', 'options': 'recursive,force'} 2023-11-27T15:21:42.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:42 smithi052 ceph-mon[99056]: pgmap v116: 97 pgs: 97 active+clean; 424 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.5 MiB/s wr, 61 op/s 2023-11-27T15:21:42.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:42 smithi150 ceph-mon[100559]: pgmap v116: 97 pgs: 97 active+clean; 424 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.5 MiB/s wr, 61 op/s 2023-11-27T15:21:42.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:42 smithi102 ceph-mon[100296]: pgmap v116: 97 pgs: 97 active+clean; 424 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.5 MiB/s wr, 61 op/s 2023-11-27T15:21:44.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:44 smithi150 ceph-mon[100559]: pgmap v117: 97 pgs: 97 active+clean; 424 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 43 op/s 2023-11-27T15:21:44.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:44 smithi150 ceph-mon[100559]: scrub summary: active paths [/] 2023-11-27T15:21:44.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:44 smithi150 ceph-mon[100559]: scrub summary: active paths [/] 2023-11-27T15:21:44.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:44 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:21:44.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:44 smithi102 ceph-mon[100296]: pgmap v117: 97 pgs: 97 active+clean; 424 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 43 op/s 2023-11-27T15:21:44.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:44 smithi102 ceph-mon[100296]: scrub summary: active paths [/] 2023-11-27T15:21:44.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:44 smithi102 ceph-mon[100296]: scrub summary: active paths [/] 2023-11-27T15:21:44.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:44 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:21:44.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:44 smithi052 ceph-mon[99056]: pgmap v117: 97 pgs: 97 active+clean; 424 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 43 op/s 2023-11-27T15:21:44.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:44 smithi052 ceph-mon[99056]: scrub summary: active paths [/] 2023-11-27T15:21:44.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:44 smithi052 ceph-mon[99056]: scrub summary: active paths [/] 2023-11-27T15:21:44.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:44 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:21:46.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:46 smithi150 ceph-mon[100559]: pgmap v118: 97 pgs: 97 active+clean; 424 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 43 op/s 2023-11-27T15:21:46.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:46 smithi102 ceph-mon[100296]: pgmap v118: 97 pgs: 97 active+clean; 424 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 43 op/s 2023-11-27T15:21:46.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:46 smithi052 ceph-mon[99056]: pgmap v118: 97 pgs: 97 active+clean; 424 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 43 op/s 2023-11-27T15:21:48.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:48 smithi150 ceph-mon[100559]: pgmap v119: 97 pgs: 97 active+clean; 449 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.7 MiB/s wr, 67 op/s 2023-11-27T15:21:48.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:48 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:21:48.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:48 smithi102 ceph-mon[100296]: pgmap v119: 97 pgs: 97 active+clean; 449 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.7 MiB/s wr, 67 op/s 2023-11-27T15:21:48.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:48 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:21:48.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:48 smithi052 ceph-mon[99056]: pgmap v119: 97 pgs: 97 active+clean; 449 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.7 MiB/s wr, 67 op/s 2023-11-27T15:21:48.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:48 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:21:50.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:49 smithi052 ceph-mon[99056]: pgmap v120: 97 pgs: 97 active+clean; 449 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 43 op/s 2023-11-27T15:21:50.277 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:49 smithi150 ceph-mon[100559]: pgmap v120: 97 pgs: 97 active+clean; 449 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 43 op/s 2023-11-27T15:21:50.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:49 smithi102 ceph-mon[100296]: pgmap v120: 97 pgs: 97 active+clean; 449 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 43 op/s 2023-11-27T15:21:52.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:52 smithi150 ceph-mon[100559]: pgmap v121: 97 pgs: 97 active+clean; 476 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.1 MiB/s wr, 77 op/s 2023-11-27T15:21:52.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:52 smithi102 ceph-mon[100296]: pgmap v121: 97 pgs: 97 active+clean; 476 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.1 MiB/s wr, 77 op/s 2023-11-27T15:21:52.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:52 smithi052 ceph-mon[99056]: pgmap v121: 97 pgs: 97 active+clean; 476 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.1 MiB/s wr, 77 op/s 2023-11-27T15:21:54.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:54 smithi150 ceph-mon[100559]: pgmap v122: 97 pgs: 97 active+clean; 476 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.3 MiB/s wr, 58 op/s 2023-11-27T15:21:54.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:54 smithi102 ceph-mon[100296]: pgmap v122: 97 pgs: 97 active+clean; 476 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.3 MiB/s wr, 58 op/s 2023-11-27T15:21:54.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:54 smithi052 ceph-mon[99056]: pgmap v122: 97 pgs: 97 active+clean; 476 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.3 MiB/s wr, 58 op/s 2023-11-27T15:21:56.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:56 smithi150 ceph-mon[100559]: pgmap v123: 97 pgs: 97 active+clean; 476 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.3 MiB/s wr, 58 op/s 2023-11-27T15:21:56.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:56 smithi102 ceph-mon[100296]: pgmap v123: 97 pgs: 97 active+clean; 476 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.3 MiB/s wr, 58 op/s 2023-11-27T15:21:56.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:56 smithi052 ceph-mon[99056]: pgmap v123: 97 pgs: 97 active+clean; 476 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.3 MiB/s wr, 58 op/s 2023-11-27T15:21:58.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:58 smithi150 ceph-mon[100559]: pgmap v124: 97 pgs: 97 active+clean; 497 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.1 MiB/s wr, 83 op/s 2023-11-27T15:21:58.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:58 smithi102 ceph-mon[100296]: pgmap v124: 97 pgs: 97 active+clean; 497 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.1 MiB/s wr, 83 op/s 2023-11-27T15:21:58.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:58 smithi052 ceph-mon[99056]: pgmap v124: 97 pgs: 97 active+clean; 497 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.1 MiB/s wr, 83 op/s 2023-11-27T15:22:00.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:21:59 smithi052 ceph-mon[99056]: pgmap v125: 97 pgs: 97 active+clean; 497 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 59 op/s 2023-11-27T15:22:00.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:21:59 smithi150 ceph-mon[100559]: pgmap v125: 97 pgs: 97 active+clean; 497 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 59 op/s 2023-11-27T15:22:00.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:21:59 smithi102 ceph-mon[100296]: pgmap v125: 97 pgs: 97 active+clean; 497 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 59 op/s 2023-11-27T15:22:02.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:02 smithi150 ceph-mon[100559]: pgmap v126: 97 pgs: 97 active+clean; 521 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.0 MiB/s wr, 78 op/s 2023-11-27T15:22:02.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:02 smithi102 ceph-mon[100296]: pgmap v126: 97 pgs: 97 active+clean; 521 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.0 MiB/s wr, 78 op/s 2023-11-27T15:22:02.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:02 smithi052 ceph-mon[99056]: pgmap v126: 97 pgs: 97 active+clean; 521 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.0 MiB/s wr, 78 op/s 2023-11-27T15:22:04.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:04 smithi150 ceph-mon[100559]: pgmap v127: 97 pgs: 97 active+clean; 521 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 44 op/s 2023-11-27T15:22:04.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:04 smithi102 ceph-mon[100296]: pgmap v127: 97 pgs: 97 active+clean; 521 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 44 op/s 2023-11-27T15:22:04.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:04 smithi052 ceph-mon[99056]: pgmap v127: 97 pgs: 97 active+clean; 521 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 44 op/s 2023-11-27T15:22:06.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:06 smithi150 ceph-mon[100559]: pgmap v128: 97 pgs: 97 active+clean; 521 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 44 op/s 2023-11-27T15:22:06.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:06 smithi102 ceph-mon[100296]: pgmap v128: 97 pgs: 97 active+clean; 521 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 44 op/s 2023-11-27T15:22:06.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:06 smithi052 ceph-mon[99056]: pgmap v128: 97 pgs: 97 active+clean; 521 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 44 op/s 2023-11-27T15:22:08.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:08 smithi150 ceph-mon[100559]: pgmap v129: 97 pgs: 97 active+clean; 543 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.6 MiB/s wr, 75 op/s 2023-11-27T15:22:08.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:08 smithi102 ceph-mon[100296]: pgmap v129: 97 pgs: 97 active+clean; 543 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.6 MiB/s wr, 75 op/s 2023-11-27T15:22:08.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:08 smithi052 ceph-mon[99056]: pgmap v129: 97 pgs: 97 active+clean; 543 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.6 MiB/s wr, 75 op/s 2023-11-27T15:22:10.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:09 smithi052 ceph-mon[99056]: pgmap v130: 97 pgs: 97 active+clean; 543 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 50 op/s 2023-11-27T15:22:10.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:09 smithi150 ceph-mon[100559]: pgmap v130: 97 pgs: 97 active+clean; 543 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 50 op/s 2023-11-27T15:22:10.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:09 smithi102 ceph-mon[100296]: pgmap v130: 97 pgs: 97 active+clean; 543 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 50 op/s 2023-11-27T15:22:12.206 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:22:12.359 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.354+0000 7f74e51fa700 1 -- 172.21.15.52:0/4101739748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74e010cd50 msgr2=0x7f74e010f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:12.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.354+0000 7f74e51fa700 1 --2- 172.21.15.52:0/4101739748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74e010cd50 0x7f74e010f130 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f74d800b0a0 tx=0x7f74d800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.354+0000 7f74e51fa700 1 -- 172.21.15.52:0/4101739748 shutdown_connections 2023-11-27T15:22:12.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.354+0000 7f74e51fa700 1 --2- 172.21.15.52:0/4101739748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74e010cd50 0x7f74e010f130 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.354+0000 7f74e51fa700 1 --2- 172.21.15.52:0/4101739748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74e00ff1a0 0x7f74e00ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.354+0000 7f74e51fa700 1 --2- 172.21.15.52:0/4101739748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74e00ffab0 0x7f74e010c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.354+0000 7f74e51fa700 1 -- 172.21.15.52:0/4101739748 >> 172.21.15.52:0/4101739748 conn(0x7f74e00760d0 msgr2=0x7f74e00764d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:12.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.354+0000 7f74e51fa700 1 -- 172.21.15.52:0/4101739748 shutdown_connections 2023-11-27T15:22:12.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.354+0000 7f74e51fa700 1 -- 172.21.15.52:0/4101739748 wait complete. 2023-11-27T15:22:12.362 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.355+0000 7f74e51fa700 1 Processor -- start 2023-11-27T15:22:12.362 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.356+0000 7f74e51fa700 1 -- start start 2023-11-27T15:22:12.362 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.356+0000 7f74e51fa700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74e00ff1a0 0x7f74e0198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:12.362 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.356+0000 7f74e51fa700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74e00ffab0 0x7f74e0198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:12.363 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.356+0000 7f74e51fa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74e010cd50 0x7f74e019c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:12.363 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.356+0000 7f74e51fa700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e019c600 con 0x7f74e010cd50 2023-11-27T15:22:12.363 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.356+0000 7f74e51fa700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e019c770 con 0x7f74e00ffab0 2023-11-27T15:22:12.363 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.356+0000 7f74e51fa700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e019c930 con 0x7f74e00ff1a0 2023-11-27T15:22:12.363 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.356+0000 7f74df59e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74e010cd50 0x7f74e019c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:12.363 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.356+0000 7f74de59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74e00ffab0 0x7f74e0198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:12.363 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.356+0000 7f74ded9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74e00ff1a0 0x7f74e0198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:12.363 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.356+0000 7f74de59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74e00ffab0 0x7f74e0198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43410/0 (socket says 172.21.15.52:43410) 2023-11-27T15:22:12.363 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.356+0000 7f74ded9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74e00ff1a0 0x7f74e0198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:40396/0 (socket says 172.21.15.52:40396) 2023-11-27T15:22:12.363 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.356+0000 7f74de59c700 1 -- 172.21.15.52:0/1247469883 learned_addr learned my addr 172.21.15.52:0/1247469883 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:22:12.363 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.356+0000 7f74df59e700 1 -- 172.21.15.52:0/1247469883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74e00ff1a0 msgr2=0x7f74e0198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:12.363 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.356+0000 7f74df59e700 1 --2- 172.21.15.52:0/1247469883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74e00ff1a0 0x7f74e0198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.364 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.356+0000 7f74df59e700 1 -- 172.21.15.52:0/1247469883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74e00ffab0 msgr2=0x7f74e0198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:12.364 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.356+0000 7f74df59e700 1 --2- 172.21.15.52:0/1247469883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74e00ffab0 0x7f74e0198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.364 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.356+0000 7f74df59e700 1 -- 172.21.15.52:0/1247469883 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d4009520 con 0x7f74e010cd50 2023-11-27T15:22:12.364 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.356+0000 7f74ded9d700 1 --2- 172.21.15.52:0/1247469883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74e00ff1a0 0x7f74e0198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:12.364 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.356+0000 7f74de59c700 1 --2- 172.21.15.52:0/1247469883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74e00ffab0 0x7f74e0198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:12.364 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.357+0000 7f74df59e700 1 --2- 172.21.15.52:0/1247469883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74e010cd50 0x7f74e019c000 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f74d800b070 tx=0x7f74d8007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:12.364 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.357+0000 7f74c7fff700 1 -- 172.21.15.52:0/1247469883 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d800e040 con 0x7f74e010cd50 2023-11-27T15:22:12.364 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.357+0000 7f74e51fa700 1 -- 172.21.15.52:0/1247469883 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74d8009d00 con 0x7f74e010cd50 2023-11-27T15:22:12.364 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.357+0000 7f74c7fff700 1 -- 172.21.15.52:0/1247469883 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74d801d470 con 0x7f74e010cd50 2023-11-27T15:22:12.364 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.357+0000 7f74c7fff700 1 -- 172.21.15.52:0/1247469883 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d80052a0 con 0x7f74e010cd50 2023-11-27T15:22:12.364 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.357+0000 7f74e51fa700 1 -- 172.21.15.52:0/1247469883 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e0112630 con 0x7f74e010cd50 2023-11-27T15:22:12.365 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.360+0000 7f74c7fff700 1 -- 172.21.15.52:0/1247469883 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f74d801d5e0 con 0x7f74e010cd50 2023-11-27T15:22:12.366 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.361+0000 7f74c7fff700 1 --2- 172.21.15.52:0/1247469883 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f74c80a12c0 0x7f74c80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:12.366 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.361+0000 7f74c7fff700 1 -- 172.21.15.52:0/1247469883 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f74d80ce040 con 0x7f74e010cd50 2023-11-27T15:22:12.366 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.361+0000 7f74ded9d700 1 --2- 172.21.15.52:0/1247469883 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f74c80a12c0 0x7f74c80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:12.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.362+0000 7f74f1386b80 1 Processor -- start 2023-11-27T15:22:12.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.363+0000 7f74f1386b80 1 -- start start 2023-11-27T15:22:12.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.363+0000 7f74f1386b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c34616d890 0x55c346165950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:12.369 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.363+0000 7f74f1386b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c34616dc60 0x55c346165e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:12.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.363+0000 7f74f1386b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3461663d0 0x55c3461861b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:12.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.363+0000 7f74f1386b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c34606e060 con 0x55c34616d890 2023-11-27T15:22:12.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.363+0000 7f74f1386b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3460c08a0 con 0x55c34616dc60 2023-11-27T15:22:12.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.363+0000 7f74f1386b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3460b9d30 con 0x55c3461663d0 2023-11-27T15:22:12.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.363+0000 7f74de59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c34616dc60 0x55c346165e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:12.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.363+0000 7f74de59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c34616dc60 0x55c346165e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43418/0 (socket says 172.21.15.52:43418) 2023-11-27T15:22:12.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.363+0000 7f74de59c700 1 -- 172.21.15.52:0/316831096 learned_addr learned my addr 172.21.15.52:0/316831096 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:22:12.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.364+0000 7f74df59e700 1 --2- 172.21.15.52:0/316831096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3461663d0 0x55c3461861b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:12.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.364+0000 7f74ded9d700 1 --2- 172.21.15.52:0/316831096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c34616d890 0x55c346165950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:12.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.364+0000 7f74de59c700 1 -- 172.21.15.52:0/316831096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3461663d0 msgr2=0x55c3461861b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:12.370 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.364+0000 7f74de59c700 1 --2- 172.21.15.52:0/316831096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3461663d0 0x55c3461861b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.371 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.364+0000 7f74de59c700 1 -- 172.21.15.52:0/316831096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c34616d890 msgr2=0x55c346165950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:12.371 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.364+0000 7f74de59c700 1 --2- 172.21.15.52:0/316831096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c34616d890 0x55c346165950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.371 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.364+0000 7f74ded9d700 1 --2- 172.21.15.52:0/1247469883 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f74c80a12c0 0x7f74c80a3770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f74d0011490 tx=0x7f74d0009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:12.371 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.364+0000 7f74de59c700 1 -- 172.21.15.52:0/316831096 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c345f3b630 con 0x55c34616dc60 2023-11-27T15:22:12.371 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.365+0000 7f74de59c700 1 --2- 172.21.15.52:0/316831096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c34616dc60 0x55c346165e90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f74d4033850 tx=0x7f74d4033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=777f9243d2bad67a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:12.371 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.366+0000 7f74c57fa700 1 -- 172.21.15.52:0/316831096 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d4004020 con 0x55c34616dc60 2023-11-27T15:22:12.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.366+0000 7f74c57fa700 1 -- 172.21.15.52:0/316831096 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74d4004830 con 0x55c34616dc60 2023-11-27T15:22:12.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.366+0000 7f74c57fa700 1 -- 172.21.15.52:0/316831096 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d4005480 con 0x55c34616dc60 2023-11-27T15:22:12.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.367+0000 7f74f1386b80 1 -- 172.21.15.52:0/316831096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c34616dc60 msgr2=0x55c346165e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:12.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.367+0000 7f74f1386b80 1 --2- 172.21.15.52:0/316831096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c34616dc60 0x55c346165e90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f74d4033850 tx=0x7f74d4033b60 comp rx=0 tx=0).stop 2023-11-27T15:22:12.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.368+0000 7f74f1386b80 1 -- 172.21.15.52:0/316831096 shutdown_connections 2023-11-27T15:22:12.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.368+0000 7f74f1386b80 1 --2- 172.21.15.52:0/316831096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3461663d0 0x55c3461861b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.372 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.368+0000 7f74f1386b80 1 --2- 172.21.15.52:0/316831096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c34616d890 0x55c346165950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.373 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.368+0000 7f74f1386b80 1 --2- 172.21.15.52:0/316831096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c34616dc60 0x55c346165e90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.373 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.368+0000 7f74f1386b80 1 -- 172.21.15.52:0/316831096 >> 172.21.15.52:0/316831096 conn(0x55c34615e360 msgr2=0x55c346163b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:12.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.370+0000 7f74f1386b80 1 -- 172.21.15.52:0/316831096 shutdown_connections 2023-11-27T15:22:12.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.370+0000 7f74f1386b80 1 -- 172.21.15.52:0/316831096 wait complete. 2023-11-27T15:22:12.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.370+0000 7f74f1386b80 1 -- start start 2023-11-27T15:22:12.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.370+0000 7f74f1386b80 1 Processor -- start 2023-11-27T15:22:12.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.371+0000 7f74f1386b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3461663d0 0x55c3461772a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:12.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.371+0000 7f74f1386b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c34616d890 0x55c3461777e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:12.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.371+0000 7f74f1386b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c34616dc60 0x55c346172570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:12.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.371+0000 7f74f1386b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c34605b8e0 con 0x55c34616dc60 2023-11-27T15:22:12.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.371+0000 7f74f1386b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c34605b580 con 0x55c34616d890 2023-11-27T15:22:12.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.371+0000 7f74f1386b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c345f3d920 con 0x55c3461663d0 2023-11-27T15:22:12.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.371+0000 7f74f1386b80 10 client.0 _ll_register_callbacks cb 0x55c346079780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:22:12.376 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.371+0000 7f74df59e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c34616dc60 0x55c346172570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:12.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.371+0000 7f74de59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c34616d890 0x55c3461777e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:12.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.371+0000 7f74ded9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3461663d0 0x55c3461772a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:12.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.371+0000 7f74de59c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c34616d890 0x55c3461777e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43424/0 (socket says 172.21.15.52:43424) 2023-11-27T15:22:12.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.371+0000 7f74ded9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3461663d0 0x55c3461772a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:40410/0 (socket says 172.21.15.52:40410) 2023-11-27T15:22:12.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.371+0000 7f74df59e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c34616dc60 0x55c346172570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53770/0 (socket says 172.21.15.52:53770) 2023-11-27T15:22:12.377 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.371+0000 7f74de59c700 1 -- 172.21.15.52:0/3900293505 learned_addr learned my addr 172.21.15.52:0/3900293505 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:22:12.378 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.372+0000 7f74df59e700 1 -- 172.21.15.52:0/3900293505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3461663d0 msgr2=0x55c3461772a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:12.378 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.372+0000 7f74df59e700 1 --2- 172.21.15.52:0/3900293505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3461663d0 0x55c3461772a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.378 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.372+0000 7f74df59e700 1 -- 172.21.15.52:0/3900293505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c34616d890 msgr2=0x55c3461777e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:12.378 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.372+0000 7f74df59e700 1 --2- 172.21.15.52:0/3900293505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c34616d890 0x55c3461777e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.378 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.372+0000 7f74df59e700 1 -- 172.21.15.52:0/3900293505 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74d4033450 con 0x55c34616dc60 2023-11-27T15:22:12.379 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.372+0000 7f74de59c700 1 --2- 172.21.15.52:0/3900293505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c34616d890 0x55c3461777e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:12.379 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.372+0000 7f74df59e700 1 --2- 172.21.15.52:0/3900293505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c34616dc60 0x55c346172570 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f74d80cb790 tx=0x7f74d809ab00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd8c4371914a0f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:12.379 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.372+0000 7f74c2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:22:12.379 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.372+0000 7f74ded9d700 1 --2- 172.21.15.52:0/3900293505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3461663d0 0x55c3461772a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:22:12.379 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.373+0000 7f74c2ff5700 1 -- 172.21.15.52:0/3900293505 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d809b600 con 0x55c34616dc60 2023-11-27T15:22:12.379 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.373+0000 7f74c2ff5700 1 -- 172.21.15.52:0/3900293505 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f74d809f1d0 con 0x55c34616dc60 2023-11-27T15:22:12.379 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.373+0000 7f74f1386b80 1 -- 172.21.15.52:0/3900293505 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c34615bb20 con 0x55c34616dc60 2023-11-27T15:22:12.379 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.373+0000 7f74c2ff5700 1 -- 172.21.15.52:0/3900293505 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74d801daf0 con 0x55c34616dc60 2023-11-27T15:22:12.379 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.373+0000 7f74f1386b80 10 client.16140 fetch_fsmap learned FSMap version 28 2023-11-27T15:22:12.379 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.373+0000 7f74f1386b80 1 -- 172.21.15.52:0/3900293505 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c346080190 con 0x55c34616dc60 2023-11-27T15:22:12.380 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.374+0000 7f74c2ff5700 1 -- 172.21.15.52:0/3900293505 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c34615bb20 con 0x55c34616dc60 2023-11-27T15:22:12.380 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.374+0000 7f74c2ff5700 1 -- 172.21.15.52:0/3900293505 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f74d8096f90 con 0x55c34616dc60 2023-11-27T15:22:12.380 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.374+0000 7f74f1386b80 10 client.16140 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:22:12.380 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.374+0000 7f74f1386b80 10 client.16140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:22:12.381 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.374+0000 7f74f1386b80 20 client.16140 populate_metadata read hostname 'smithi052' 2023-11-27T15:22:12.381 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.374+0000 7f74f1386b80 1 --2- 172.21.15.52:0/3900293505 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c346182430 0x55c346184840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:12.381 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.374+0000 7f74f1386b80 4 client.16140 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:22:12.381 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.374+0000 7f74f1386b80 1 -- 172.21.15.52:0/3900293505 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c34607c0a0 con 0x55c346182430 2023-11-27T15:22:12.381 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.374+0000 7f74ded9d700 1 --2- 172.21.15.52:0/3900293505 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c346182430 0x55c346184840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:12.381 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.375+0000 7f74ded9d700 1 --2- 172.21.15.52:0/3900293505 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c346182430 0x55c346184840 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81eaad31a478b59d server_cookie=8fa6b9ec575822c2 in_seq=0 out_seq=0 2023-11-27T15:22:12.382 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.375+0000 7f74c2ff5700 10 client.16140 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:22:12.382 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.376+0000 7f74c2ff5700 1 -- 172.21.15.52:0/3900293505 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c34607c0a0 con 0x55c346182430 2023-11-27T15:22:12.382 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.376+0000 7f74c2ff5700 10 client.16140 handle_command_reply: tid=0 2023-11-27T15:22:12.382 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.376+0000 7f74f1386b80 1 client.16140 shutdown 2023-11-27T15:22:12.382 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.376+0000 7f74f1386b80 1 -- 172.21.15.52:0/3900293505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c34616dc60 msgr2=0x55c346172570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:12.382 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.376+0000 7f74f1386b80 1 --2- 172.21.15.52:0/3900293505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c34616dc60 0x55c346172570 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f74d80cb790 tx=0x7f74d809ab00 comp rx=0 tx=0).stop 2023-11-27T15:22:12.382 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.377+0000 7f74f1386b80 1 -- 172.21.15.52:0/3900293505 shutdown_connections 2023-11-27T15:22:12.382 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.377+0000 7f74f1386b80 1 --2- 172.21.15.52:0/3900293505 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c346182430 0x55c346184840 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.383 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.377+0000 7f74f1386b80 1 --2- 172.21.15.52:0/3900293505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c34616dc60 0x55c346172570 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.383 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.377+0000 7f74c2ff5700 0 client.16140 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:22:12.383 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.377+0000 7f74f1386b80 1 --2- 172.21.15.52:0/3900293505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3461663d0 0x55c3461772a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.383 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.377+0000 7f74f1386b80 1 --2- 172.21.15.52:0/3900293505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c34616d890 0x55c3461777e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.383 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.377+0000 7f74f1386b80 1 -- 172.21.15.52:0/3900293505 >> 172.21.15.52:0/3900293505 conn(0x55c34615e360 msgr2=0x55c346162550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:12.383 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.377+0000 7f74f1386b80 1 -- 172.21.15.52:0/3900293505 shutdown_connections 2023-11-27T15:22:12.383 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.377+0000 7f74f1386b80 1 -- 172.21.15.52:0/3900293505 wait complete. 2023-11-27T15:22:12.383 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.377+0000 7f74f1386b80 20 client.16140 trim_cache size 0 max 16384 2023-11-27T15:22:12.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.408+0000 7f74f1386b80 1 Processor -- start 2023-11-27T15:22:12.413 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.409+0000 7f74f1386b80 1 -- start start 2023-11-27T15:22:12.414 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.409+0000 7f74f1386b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3461663d0 0x55c34617e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:12.414 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.409+0000 7f74f1386b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c34616d890 0x55c34617ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:12.414 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.409+0000 7f74f1386b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c34616dc60 0x55c346171e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:12.414 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.409+0000 7f74f1386b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c346165180 con 0x55c34616d890 2023-11-27T15:22:12.414 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.409+0000 7f74f1386b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c34615b660 con 0x55c34616dc60 2023-11-27T15:22:12.414 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.409+0000 7f74f1386b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3461661e0 con 0x55c3461663d0 2023-11-27T15:22:12.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.409+0000 7f74df59e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c34616dc60 0x55c346171e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:12.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.409+0000 7f74df59e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c34616dc60 0x55c346171e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43434/0 (socket says 172.21.15.52:43434) 2023-11-27T15:22:12.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.409+0000 7f74df59e700 1 -- 172.21.15.52:0/978990258 learned_addr learned my addr 172.21.15.52:0/978990258 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:22:12.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.410+0000 7f74ded9d700 1 --2- 172.21.15.52:0/978990258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3461663d0 0x55c34617e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:12.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.410+0000 7f74df59e700 1 -- 172.21.15.52:0/978990258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3461663d0 msgr2=0x55c34617e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:12.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.410+0000 7f74df59e700 1 --2- 172.21.15.52:0/978990258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3461663d0 0x55c34617e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.410+0000 7f74df59e700 1 -- 172.21.15.52:0/978990258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c34616d890 msgr2=0x55c34617ec50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-27T15:22:12.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.410+0000 7f74df59e700 1 --2- 172.21.15.52:0/978990258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c34616d890 0x55c34617ec50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.410+0000 7f74df59e700 1 -- 172.21.15.52:0/978990258 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c34607fee0 con 0x55c34616dc60 2023-11-27T15:22:12.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.410+0000 7f74df59e700 1 --2- 172.21.15.52:0/978990258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c34616dc60 0x55c346171e10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f74d8097480 tx=0x7f74d800b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4383e075fffc8ff6 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:12.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.411+0000 7f74c5ffb700 1 -- 172.21.15.52:0/978990258 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d801daf0 con 0x55c34616dc60 2023-11-27T15:22:12.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.411+0000 7f74c5ffb700 1 -- 172.21.15.52:0/978990258 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74d80a3370 con 0x55c34616dc60 2023-11-27T15:22:12.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.411+0000 7f74c5ffb700 1 -- 172.21.15.52:0/978990258 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d809a690 con 0x55c34616dc60 2023-11-27T15:22:12.417 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.412+0000 7f74f1386b80 1 -- 172.21.15.52:0/978990258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c34616dc60 msgr2=0x55c346171e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:12.417 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.412+0000 7f74f1386b80 1 --2- 172.21.15.52:0/978990258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c34616dc60 0x55c346171e10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f74d8097480 tx=0x7f74d800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.417 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.412+0000 7f74f1386b80 1 -- 172.21.15.52:0/978990258 shutdown_connections 2023-11-27T15:22:12.417 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.412+0000 7f74f1386b80 1 --2- 172.21.15.52:0/978990258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c34616dc60 0x55c346171e10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.412+0000 7f74f1386b80 1 --2- 172.21.15.52:0/978990258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c3461663d0 0x55c34617e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.412+0000 7f74f1386b80 1 --2- 172.21.15.52:0/978990258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c34616d890 0x55c34617ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.412+0000 7f74f1386b80 1 -- 172.21.15.52:0/978990258 >> 172.21.15.52:0/978990258 conn(0x55c34615e360 msgr2=0x55c3461678c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:12.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.413+0000 7f74f1386b80 1 -- 172.21.15.52:0/978990258 shutdown_connections 2023-11-27T15:22:12.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.413+0000 7f74f1386b80 1 -- 172.21.15.52:0/978990258 wait complete. 2023-11-27T15:22:12.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.413+0000 7f74f1386b80 1 -- start start 2023-11-27T15:22:12.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.414+0000 7f74f1386b80 1 Processor -- start 2023-11-27T15:22:12.419 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.414+0000 7f74f1386b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3461663d0 0x55c34617bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:12.419 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.414+0000 7f74f1386b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c34616d890 0x55c346177fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:12.419 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.414+0000 7f74f1386b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c34616dc60 0x55c346178520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:12.419 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.414+0000 7f74f1386b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c346178a60 con 0x55c34616dc60 2023-11-27T15:22:12.419 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.414+0000 7f74f1386b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c346178bd0 con 0x55c3461663d0 2023-11-27T15:22:12.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.414+0000 7f74f1386b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c346178d10 con 0x55c34616d890 2023-11-27T15:22:12.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.414+0000 7f74f1386b80 10 client.0 _ll_register_callbacks cb 0x7f74d809b020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:22:12.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.414+0000 7f74de59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c34616d890 0x55c346177fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:12.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.414+0000 7f74de59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c34616d890 0x55c346177fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:40424/0 (socket says 172.21.15.52:40424) 2023-11-27T15:22:12.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.414+0000 7f74de59c700 1 -- 172.21.15.52:0/3315418607 learned_addr learned my addr 172.21.15.52:0/3315418607 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:22:12.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.414+0000 7f74ded9d700 1 --2- 172.21.15.52:0/3315418607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3461663d0 0x55c34617bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:12.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.415+0000 7f74de59c700 1 -- 172.21.15.52:0/3315418607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3461663d0 msgr2=0x55c34617bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:12.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.415+0000 7f74de59c700 1 --2- 172.21.15.52:0/3315418607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3461663d0 0x55c34617bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.415+0000 7f74de59c700 1 -- 172.21.15.52:0/3315418607 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c34616dc60 msgr2=0x55c346178520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:12.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.415+0000 7f74de59c700 1 --2- 172.21.15.52:0/3315418607 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c34616dc60 0x55c346178520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.415+0000 7f74de59c700 1 -- 172.21.15.52:0/3315418607 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c346080190 con 0x55c34616d890 2023-11-27T15:22:12.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.415+0000 7f74de59c700 1 --2- 172.21.15.52:0/3315418607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c34616d890 0x55c346177fe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f74d4033740 tx=0x7f74d4004650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b56788606adc334 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:12.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.416+0000 7f74c2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2023-11-27T15:22:12.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.416+0000 7f74c2ff5700 1 -- 172.21.15.52:0/3315418607 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d4004cb0 con 0x55c34616d890 2023-11-27T15:22:12.422 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.416+0000 7f74c2ff5700 1 -- 172.21.15.52:0/3315418607 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f74d4005160 con 0x55c34616d890 2023-11-27T15:22:12.422 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.416+0000 7f74f1386b80 1 -- 172.21.15.52:0/3315418607 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74d4004830 con 0x55c34616d890 2023-11-27T15:22:12.422 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.416+0000 7f74c2ff5700 1 -- 172.21.15.52:0/3315418607 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74d4005320 con 0x55c34616d890 2023-11-27T15:22:12.422 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.416+0000 7f74f1386b80 10 client.25849 fetch_fsmap learned FSMap version 28 2023-11-27T15:22:12.422 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.416+0000 7f74f1386b80 1 -- 172.21.15.52:0/3315418607 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c345ed0c60 con 0x55c34616d890 2023-11-27T15:22:12.422 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.417+0000 7f74c2ff5700 1 -- 172.21.15.52:0/3315418607 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74d4004830 con 0x55c34616d890 2023-11-27T15:22:12.422 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.417+0000 7f74c2ff5700 1 -- 172.21.15.52:0/3315418607 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f74d400f5a0 con 0x55c34616d890 2023-11-27T15:22:12.423 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.418+0000 7f74f1386b80 10 client.25849 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:22:12.423 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.418+0000 7f74f1386b80 10 client.25849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:22:12.423 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.418+0000 7f74f1386b80 20 client.25849 populate_metadata read hostname 'smithi052' 2023-11-27T15:22:12.424 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.418+0000 7f74f1386b80 1 --2- 172.21.15.52:0/3315418607 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c3461e7d00 0x55c3461ea110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:12.424 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.419+0000 7f74f1386b80 4 client.25849 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:22:12.424 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.419+0000 7f74f1386b80 1 -- 172.21.15.52:0/3315418607 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c346088660 con 0x55c3461e7d00 2023-11-27T15:22:12.424 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.419+0000 7f74ded9d700 1 --2- 172.21.15.52:0/3315418607 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c3461e7d00 0x55c3461ea110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:12.424 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.420+0000 7f74c2ff5700 10 client.25849 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:22:12.424 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.420+0000 7f74ded9d700 1 --2- 172.21.15.52:0/3315418607 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c3461e7d00 0x55c3461ea110 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd89dbecd567afb6 server_cookie=205c0a3d73d8d11f in_seq=0 out_seq=0 2023-11-27T15:22:12.425 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.420+0000 7f74c2ff5700 1 -- 172.21.15.52:0/3315418607 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c346088660 con 0x55c3461e7d00 2023-11-27T15:22:12.425 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.420+0000 7f74c2ff5700 10 client.25849 handle_command_reply: tid=0 2023-11-27T15:22:12.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.420+0000 7f74f1386b80 1 client.25849 shutdown 2023-11-27T15:22:12.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.420+0000 7f74f1386b80 1 -- 172.21.15.52:0/3315418607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c34616d890 msgr2=0x55c346177fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:12.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.420+0000 7f74f1386b80 1 --2- 172.21.15.52:0/3315418607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c34616d890 0x55c346177fe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f74d4033740 tx=0x7f74d4004650 comp rx=0 tx=0).stop 2023-11-27T15:22:12.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.420+0000 7f74f1386b80 1 -- 172.21.15.52:0/3315418607 shutdown_connections 2023-11-27T15:22:12.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.420+0000 7f74f1386b80 1 --2- 172.21.15.52:0/3315418607 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c3461e7d00 0x55c3461ea110 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.420+0000 7f74f1386b80 1 --2- 172.21.15.52:0/3315418607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c3461663d0 0x55c34617bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.420+0000 7f74f1386b80 1 --2- 172.21.15.52:0/3315418607 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c34616dc60 0x55c346178520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.420+0000 7f74c2ff5700 0 client.25849 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:22:12.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.420+0000 7f74f1386b80 1 --2- 172.21.15.52:0/3315418607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c34616d890 0x55c346177fe0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.420+0000 7f74f1386b80 1 -- 172.21.15.52:0/3315418607 >> 172.21.15.52:0/3315418607 conn(0x55c34615e360 msgr2=0x55c346171d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:12.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.421+0000 7f74f1386b80 1 -- 172.21.15.52:0/3315418607 shutdown_connections 2023-11-27T15:22:12.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.421+0000 7f74f1386b80 1 -- 172.21.15.52:0/3315418607 wait complete. 2023-11-27T15:22:12.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.421+0000 7f74f1386b80 20 client.25849 trim_cache size 0 max 16384 2023-11-27T15:22:12.427 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:22:12.427 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-27T15:22:12.427 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2023-11-27T15:22:12.427 INFO:teuthology.orchestra.run.smithi052.stdout: "1a14f367-9a76-4e86-b4c8-bd13c3d4d297": { 2023-11-27T15:22:12.427 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2023-11-27T15:22:12.427 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "1a14f367-9a76-4e86-b4c8-bd13c3d4d297", 2023-11-27T15:22:12.428 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2023-11-27T15:22:12.428 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:22:12.428 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:22:12.428 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:22:12.428 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.421+0000 7f74e51fa700 1 -- 172.21.15.52:0/1247469883 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f74c80a12c0 msgr2=0x7f74c80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:12.428 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.421+0000 7f74e51fa700 1 --2- 172.21.15.52:0/1247469883 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f74c80a12c0 0x7f74c80a3770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f74d0011490 tx=0x7f74d0009260 comp rx=0 tx=0).stop 2023-11-27T15:22:12.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.421+0000 7f74e51fa700 1 -- 172.21.15.52:0/1247469883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74e010cd50 msgr2=0x7f74e019c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:12.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.421+0000 7f74e51fa700 1 --2- 172.21.15.52:0/1247469883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74e010cd50 0x7f74e019c000 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f74d800b070 tx=0x7f74d8007930 comp rx=0 tx=0).stop 2023-11-27T15:22:12.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.422+0000 7f74e51fa700 1 -- 172.21.15.52:0/1247469883 shutdown_connections 2023-11-27T15:22:12.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.422+0000 7f74e51fa700 1 --2- 172.21.15.52:0/1247469883 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f74c80a12c0 0x7f74c80a3770 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.422+0000 7f74e51fa700 1 --2- 172.21.15.52:0/1247469883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74e010cd50 0x7f74e019c000 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.422+0000 7f74e51fa700 1 --2- 172.21.15.52:0/1247469883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74e00ff1a0 0x7f74e0198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.422+0000 7f74e51fa700 1 --2- 172.21.15.52:0/1247469883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f74e00ffab0 0x7f74e0198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.422+0000 7f74e51fa700 1 -- 172.21.15.52:0/1247469883 >> 172.21.15.52:0/1247469883 conn(0x7f74e00760d0 msgr2=0x7f74e00fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:12.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.422+0000 7f74e51fa700 1 -- 172.21.15.52:0/1247469883 shutdown_connections 2023-11-27T15:22:12.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:12.422+0000 7f74e51fa700 1 -- 172.21.15.52:0/1247469883 wait complete. 2023-11-27T15:22:12.446 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:12 smithi052 ceph-mon[99056]: pgmap v131: 97 pgs: 97 active+clean; 570 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.2 MiB/s wr, 80 op/s 2023-11-27T15:22:12.447 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:12 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:22:12.447 INFO:tasks.cephfs.filesystem:scrub status for tag:1a14f367-9a76-4e86-b4c8-bd13c3d4d297 - {'path': '/', 'tag': '1a14f367-9a76-4e86-b4c8-bd13c3d4d297', 'options': 'recursive,force'} 2023-11-27T15:22:12.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:12 smithi150 ceph-mon[100559]: pgmap v131: 97 pgs: 97 active+clean; 570 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.2 MiB/s wr, 80 op/s 2023-11-27T15:22:12.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:12 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:22:12.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:12 smithi102 ceph-mon[100296]: pgmap v131: 97 pgs: 97 active+clean; 570 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.2 MiB/s wr, 80 op/s 2023-11-27T15:22:12.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:12 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:22:13.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:13 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:13.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:13 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:22:13.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:13 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:22:13.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:13 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:13.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:13 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:22:13.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:13 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:22:13.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:13 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:13.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:13 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:22:13.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:13 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:22:14.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:14 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:22:14.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:14 smithi150 ceph-mon[100559]: Service mds.all has no related daemon already placed. 2023-11-27T15:22:14.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:14 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:22:14.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:14 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:22:14.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:14 smithi150 ceph-mon[100559]: pgmap v132: 97 pgs: 97 active+clean; 570 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.1 MiB/s wr, 60 op/s 2023-11-27T15:22:14.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:14 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:22:14.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:14 smithi102 ceph-mon[100296]: Service mds.all has no related daemon already placed. 2023-11-27T15:22:14.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:14 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:22:14.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:14 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:22:14.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:14 smithi102 ceph-mon[100296]: pgmap v132: 97 pgs: 97 active+clean; 570 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.1 MiB/s wr, 60 op/s 2023-11-27T15:22:14.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:14 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:22:14.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:14 smithi052 ceph-mon[99056]: Service mds.all has no related daemon already placed. 2023-11-27T15:22:14.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:14 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:22:14.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:14 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:22:14.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:14 smithi052 ceph-mon[99056]: pgmap v132: 97 pgs: 97 active+clean; 570 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.1 MiB/s wr, 60 op/s 2023-11-27T15:22:16.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:16 smithi150 ceph-mon[100559]: pgmap v133: 97 pgs: 97 active+clean; 570 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.1 MiB/s wr, 60 op/s 2023-11-27T15:22:16.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:16 smithi102 ceph-mon[100296]: pgmap v133: 97 pgs: 97 active+clean; 570 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.1 MiB/s wr, 60 op/s 2023-11-27T15:22:16.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:16 smithi052 ceph-mon[99056]: pgmap v133: 97 pgs: 97 active+clean; 570 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.1 MiB/s wr, 60 op/s 2023-11-27T15:22:18.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:18 smithi150 ceph-mon[100559]: pgmap v134: 97 pgs: 97 active+clean; 596 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.3 MiB/s wr, 78 op/s 2023-11-27T15:22:18.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:18 smithi102 ceph-mon[100296]: pgmap v134: 97 pgs: 97 active+clean; 596 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.3 MiB/s wr, 78 op/s 2023-11-27T15:22:18.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:18 smithi052 ceph-mon[99056]: pgmap v134: 97 pgs: 97 active+clean; 596 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.3 MiB/s wr, 78 op/s 2023-11-27T15:22:20.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:19 smithi052 ceph-mon[99056]: pgmap v135: 97 pgs: 97 active+clean; 596 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.5 MiB/s wr, 47 op/s 2023-11-27T15:22:20.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:19 smithi150 ceph-mon[100559]: pgmap v135: 97 pgs: 97 active+clean; 596 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.5 MiB/s wr, 47 op/s 2023-11-27T15:22:20.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:19 smithi102 ceph-mon[100296]: pgmap v135: 97 pgs: 97 active+clean; 596 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.5 MiB/s wr, 47 op/s 2023-11-27T15:22:22.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:22 smithi150 ceph-mon[100559]: pgmap v136: 97 pgs: 97 active+clean; 619 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.4 MiB/s wr, 66 op/s 2023-11-27T15:22:22.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:22 smithi102 ceph-mon[100296]: pgmap v136: 97 pgs: 97 active+clean; 619 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.4 MiB/s wr, 66 op/s 2023-11-27T15:22:22.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:22 smithi052 ceph-mon[99056]: pgmap v136: 97 pgs: 97 active+clean; 619 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.4 MiB/s wr, 66 op/s 2023-11-27T15:22:24.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:24 smithi150 ceph-mon[100559]: pgmap v137: 97 pgs: 97 active+clean; 619 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 36 op/s 2023-11-27T15:22:24.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:24 smithi102 ceph-mon[100296]: pgmap v137: 97 pgs: 97 active+clean; 619 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 36 op/s 2023-11-27T15:22:24.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:24 smithi052 ceph-mon[99056]: pgmap v137: 97 pgs: 97 active+clean; 619 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 36 op/s 2023-11-27T15:22:26.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:26 smithi150 ceph-mon[100559]: pgmap v138: 97 pgs: 97 active+clean; 619 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 36 op/s 2023-11-27T15:22:26.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:26 smithi102 ceph-mon[100296]: pgmap v138: 97 pgs: 97 active+clean; 619 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 36 op/s 2023-11-27T15:22:26.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:26 smithi052 ceph-mon[99056]: pgmap v138: 97 pgs: 97 active+clean; 619 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 36 op/s 2023-11-27T15:22:28.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:28 smithi150 ceph-mon[100559]: pgmap v139: 97 pgs: 97 active+clean; 641 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.9 MiB/s wr, 64 op/s 2023-11-27T15:22:28.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:28 smithi102 ceph-mon[100296]: pgmap v139: 97 pgs: 97 active+clean; 641 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.9 MiB/s wr, 64 op/s 2023-11-27T15:22:28.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:28 smithi052 ceph-mon[99056]: pgmap v139: 97 pgs: 97 active+clean; 641 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.9 MiB/s wr, 64 op/s 2023-11-27T15:22:30.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:29 smithi052 ceph-mon[99056]: pgmap v140: 97 pgs: 97 active+clean; 641 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 46 op/s 2023-11-27T15:22:30.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:29 smithi150 ceph-mon[100559]: pgmap v140: 97 pgs: 97 active+clean; 641 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 46 op/s 2023-11-27T15:22:30.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:29 smithi102 ceph-mon[100296]: pgmap v140: 97 pgs: 97 active+clean; 641 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 46 op/s 2023-11-27T15:22:32.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:32 smithi150 ceph-mon[100559]: pgmap v141: 97 pgs: 97 active+clean; 661 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.4 MiB/s wr, 75 op/s 2023-11-27T15:22:32.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:32 smithi102 ceph-mon[100296]: pgmap v141: 97 pgs: 97 active+clean; 661 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.4 MiB/s wr, 75 op/s 2023-11-27T15:22:32.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:32 smithi052 ceph-mon[99056]: pgmap v141: 97 pgs: 97 active+clean; 661 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.4 MiB/s wr, 75 op/s 2023-11-27T15:22:34.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:34 smithi150 ceph-mon[100559]: pgmap v142: 97 pgs: 97 active+clean; 661 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 56 op/s 2023-11-27T15:22:34.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:34 smithi102 ceph-mon[100296]: pgmap v142: 97 pgs: 97 active+clean; 661 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 56 op/s 2023-11-27T15:22:34.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:34 smithi052 ceph-mon[99056]: pgmap v142: 97 pgs: 97 active+clean; 661 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 56 op/s 2023-11-27T15:22:36.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:36 smithi150 ceph-mon[100559]: pgmap v143: 97 pgs: 97 active+clean; 661 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 56 op/s 2023-11-27T15:22:36.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:36 smithi102 ceph-mon[100296]: pgmap v143: 97 pgs: 97 active+clean; 661 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 56 op/s 2023-11-27T15:22:36.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:36 smithi052 ceph-mon[99056]: pgmap v143: 97 pgs: 97 active+clean; 661 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 56 op/s 2023-11-27T15:22:38.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:38 smithi150 ceph-mon[100559]: pgmap v144: 97 pgs: 97 active+clean; 674 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.6 MiB/s wr, 74 op/s 2023-11-27T15:22:38.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:38 smithi102 ceph-mon[100296]: pgmap v144: 97 pgs: 97 active+clean; 674 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.6 MiB/s wr, 74 op/s 2023-11-27T15:22:38.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:38 smithi052 ceph-mon[99056]: pgmap v144: 97 pgs: 97 active+clean; 674 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.6 MiB/s wr, 74 op/s 2023-11-27T15:22:40.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:39 smithi052 ceph-mon[99056]: pgmap v145: 97 pgs: 97 active+clean; 674 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 47 op/s 2023-11-27T15:22:40.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:39 smithi150 ceph-mon[100559]: pgmap v145: 97 pgs: 97 active+clean; 674 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 47 op/s 2023-11-27T15:22:40.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:39 smithi102 ceph-mon[100296]: pgmap v145: 97 pgs: 97 active+clean; 674 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 47 op/s 2023-11-27T15:22:42.448 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:22:42.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:42 smithi052 ceph-mon[99056]: pgmap v146: 97 pgs: 97 active+clean; 690 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 67 op/s 2023-11-27T15:22:42.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:42 smithi150 ceph-mon[100559]: pgmap v146: 97 pgs: 97 active+clean; 690 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 67 op/s 2023-11-27T15:22:42.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:42 smithi102 ceph-mon[100296]: pgmap v146: 97 pgs: 97 active+clean; 690 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 67 op/s 2023-11-27T15:22:42.580 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.576+0000 7f697d013700 1 -- 172.21.15.52:0/1376565565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69780ffdf0 msgr2=0x7f6978103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:42.580 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.576+0000 7f697d013700 1 --2- 172.21.15.52:0/1376565565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69780ffdf0 0x7f6978103db0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f696c0099c0 tx=0x7f696c009cd0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.580 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.576+0000 7f697d013700 1 -- 172.21.15.52:0/1376565565 shutdown_connections 2023-11-27T15:22:42.580 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.576+0000 7f697d013700 1 --2- 172.21.15.52:0/1376565565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f69780ff450 0x7f69780ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.580 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.576+0000 7f697d013700 1 --2- 172.21.15.52:0/1376565565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6978104490 0x7f697810c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.576+0000 7f697d013700 1 --2- 172.21.15.52:0/1376565565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69780ffdf0 0x7f6978103db0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.576+0000 7f697d013700 1 -- 172.21.15.52:0/1376565565 >> 172.21.15.52:0/1376565565 conn(0x7f69780770b0 msgr2=0x7f69780774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:42.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.576+0000 7f697d013700 1 -- 172.21.15.52:0/1376565565 shutdown_connections 2023-11-27T15:22:42.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.576+0000 7f697d013700 1 -- 172.21.15.52:0/1376565565 wait complete. 2023-11-27T15:22:42.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.577+0000 7f697d013700 1 Processor -- start 2023-11-27T15:22:42.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.577+0000 7f697d013700 1 -- start start 2023-11-27T15:22:42.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.577+0000 7f697d013700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69780ff450 0x7f6978100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.577+0000 7f697d013700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69780ffdf0 0x7f6978101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.577+0000 7f697d013700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6978104490 0x7f69781a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.577+0000 7f697d013700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6978101bc0 con 0x7f69780ff450 2023-11-27T15:22:42.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.577+0000 7f697d013700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69781a4b10 con 0x7f6978104490 2023-11-27T15:22:42.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.577+0000 7f697d013700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69781a4c50 con 0x7f69780ffdf0 2023-11-27T15:22:42.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.577+0000 7f6976d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69780ff450 0x7f6978100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.578+0000 7f697659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69780ffdf0 0x7f6978101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.578+0000 7f697759e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6978104490 0x7f69781a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.578+0000 7f6976d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69780ff450 0x7f6978100e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53792/0 (socket says 172.21.15.52:53792) 2023-11-27T15:22:42.584 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.578+0000 7f6976d9d700 1 -- 172.21.15.52:0/1044506730 learned_addr learned my addr 172.21.15.52:0/1044506730 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:22:42.584 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.578+0000 7f697759e700 1 -- 172.21.15.52:0/1044506730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69780ffdf0 msgr2=0x7f6978101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:42.584 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.578+0000 7f697759e700 1 --2- 172.21.15.52:0/1044506730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69780ffdf0 0x7f6978101390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.584 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.578+0000 7f697759e700 1 -- 172.21.15.52:0/1044506730 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69780ff450 msgr2=0x7f6978100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:42.584 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.578+0000 7f697759e700 1 --2- 172.21.15.52:0/1044506730 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69780ff450 0x7f6978100e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.584 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.578+0000 7f697759e700 1 -- 172.21.15.52:0/1044506730 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6970009c40 con 0x7f6978104490 2023-11-27T15:22:42.584 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.578+0000 7f6976d9d700 1 --2- 172.21.15.52:0/1044506730 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69780ff450 0x7f6978100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:42.584 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.578+0000 7f697759e700 1 --2- 172.21.15.52:0/1044506730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6978104490 0x7f69781a45d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f697000efb0 tx=0x7f697000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:42.584 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.578+0000 7f695ffff700 1 -- 172.21.15.52:0/1044506730 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6970003a00 con 0x7f6978104490 2023-11-27T15:22:42.584 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.578+0000 7f697d013700 1 -- 172.21.15.52:0/1044506730 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f696c0095f0 con 0x7f6978104490 2023-11-27T15:22:42.584 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.578+0000 7f695ffff700 1 -- 172.21.15.52:0/1044506730 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f697000f040 con 0x7f6978104490 2023-11-27T15:22:42.584 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.578+0000 7f695ffff700 1 -- 172.21.15.52:0/1044506730 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6970004df0 con 0x7f6978104490 2023-11-27T15:22:42.585 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.578+0000 7f697d013700 1 -- 172.21.15.52:0/1044506730 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697810f210 con 0x7f6978104490 2023-11-27T15:22:42.585 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.582+0000 7f695ffff700 1 -- 172.21.15.52:0/1044506730 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f6970003bf0 con 0x7f6978104490 2023-11-27T15:22:42.588 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.582+0000 7f695ffff700 1 --2- 172.21.15.52:0/1044506730 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f69600a12c0 0x7f69600a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.588 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.582+0000 7f698919fb80 1 Processor -- start 2023-11-27T15:22:42.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.582+0000 7f698919fb80 1 -- start start 2023-11-27T15:22:42.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.582+0000 7f6976d9d700 1 --2- 172.21.15.52:0/1044506730 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f69600a12c0 0x7f69600a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.582+0000 7f695ffff700 1 -- 172.21.15.52:0/1044506730 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f69700cca50 con 0x7f6978104490 2023-11-27T15:22:42.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.583+0000 7f698919fb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fd6b7d7370 0x55fd6b7d7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.583+0000 7f698919fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd6b7d7c80 0x55fd6b7dbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.583+0000 7f698919fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd6b7dc0c0 0x55fd6b7dc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.583+0000 7f698919fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6b6d4060 con 0x55fd6b7dc0c0 2023-11-27T15:22:42.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.583+0000 7f698919fb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6b7268a0 con 0x55fd6b7d7370 2023-11-27T15:22:42.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.583+0000 7f698919fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6b71fd30 con 0x55fd6b7d7c80 2023-11-27T15:22:42.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.583+0000 7f697759e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd6b7dc0c0 0x55fd6b7dc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.583+0000 7f697659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd6b7d7c80 0x55fd6b7dbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.583+0000 7f6976d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fd6b7d7370 0x55fd6b7d7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.583+0000 7f697659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd6b7d7c80 0x55fd6b7dbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:40446/0 (socket says 172.21.15.52:40446) 2023-11-27T15:22:42.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.583+0000 7f697759e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd6b7dc0c0 0x55fd6b7dc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53804/0 (socket says 172.21.15.52:53804) 2023-11-27T15:22:42.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.583+0000 7f697659c700 1 -- 172.21.15.52:0/3342002829 learned_addr learned my addr 172.21.15.52:0/3342002829 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:22:42.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.583+0000 7f6976d9d700 1 --2- 172.21.15.52:0/1044506730 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f69600a12c0 0x7f69600a3770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f69680115c0 tx=0x7f6968011980 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:42.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.583+0000 7f697659c700 1 -- 172.21.15.52:0/3342002829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fd6b7d7370 msgr2=0x55fd6b7d7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:42.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.583+0000 7f697659c700 1 --2- 172.21.15.52:0/3342002829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fd6b7d7370 0x55fd6b7d7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.583+0000 7f697659c700 1 -- 172.21.15.52:0/3342002829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd6b7dc0c0 msgr2=0x55fd6b7dc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:42.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.583+0000 7f697659c700 1 --2- 172.21.15.52:0/3342002829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd6b7dc0c0 0x55fd6b7dc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.583+0000 7f697659c700 1 -- 172.21.15.52:0/3342002829 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd6b5a1630 con 0x55fd6b7d7c80 2023-11-27T15:22:42.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.583+0000 7f697759e700 1 --2- 172.21.15.52:0/3342002829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd6b7dc0c0 0x55fd6b7dc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:42.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.583+0000 7f697659c700 1 --2- 172.21.15.52:0/3342002829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd6b7d7c80 0x55fd6b7dbb80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f696c0036c0 tx=0x7f696c005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28c3aac368f21420 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:42.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.584+0000 7f695d7fa700 1 -- 172.21.15.52:0/3342002829 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c026070 con 0x55fd6b7d7c80 2023-11-27T15:22:42.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.584+0000 7f695d7fa700 1 -- 172.21.15.52:0/3342002829 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f696c01e040 con 0x55fd6b7d7c80 2023-11-27T15:22:42.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.584+0000 7f695d7fa700 1 -- 172.21.15.52:0/3342002829 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c00fa50 con 0x55fd6b7d7c80 2023-11-27T15:22:42.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.584+0000 7f698919fb80 1 -- 172.21.15.52:0/3342002829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd6b7d7c80 msgr2=0x55fd6b7dbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:42.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.584+0000 7f698919fb80 1 --2- 172.21.15.52:0/3342002829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd6b7d7c80 0x55fd6b7dbb80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f696c0036c0 tx=0x7f696c005c00 comp rx=0 tx=0).stop 2023-11-27T15:22:42.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.584+0000 7f698919fb80 1 -- 172.21.15.52:0/3342002829 shutdown_connections 2023-11-27T15:22:42.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.584+0000 7f698919fb80 1 --2- 172.21.15.52:0/3342002829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fd6b7d7370 0x55fd6b7d7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.584+0000 7f698919fb80 1 --2- 172.21.15.52:0/3342002829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd6b7dc0c0 0x55fd6b7dc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.584+0000 7f698919fb80 1 --2- 172.21.15.52:0/3342002829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd6b7d7c80 0x55fd6b7dbb80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.584+0000 7f698919fb80 1 -- 172.21.15.52:0/3342002829 >> 172.21.15.52:0/3342002829 conn(0x55fd6b7c4360 msgr2=0x55fd6b7c9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:42.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.584+0000 7f698919fb80 1 -- 172.21.15.52:0/3342002829 shutdown_connections 2023-11-27T15:22:42.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.584+0000 7f698919fb80 1 -- 172.21.15.52:0/3342002829 wait complete. 2023-11-27T15:22:42.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.585+0000 7f698919fb80 1 -- start start 2023-11-27T15:22:42.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.585+0000 7f698919fb80 1 Processor -- start 2023-11-27T15:22:42.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.585+0000 7f698919fb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fd6b7d7370 0x55fd6b7e38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.585+0000 7f698919fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd6b7d7c80 0x55fd6b7e0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.585+0000 7f698919fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd6b7dc0c0 0x55fd6b7e1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.585+0000 7f698919fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6b6c18e0 con 0x55fd6b7d7c80 2023-11-27T15:22:42.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.585+0000 7f698919fb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6b6c1580 con 0x55fd6b7d7370 2023-11-27T15:22:42.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.585+0000 7f698919fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6b5a3920 con 0x55fd6b7dc0c0 2023-11-27T15:22:42.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.585+0000 7f698919fb80 10 client.0 _ll_register_callbacks cb 0x55fd6b6df780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:22:42.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.585+0000 7f697659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd6b7d7c80 0x55fd6b7e0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.585+0000 7f6976d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fd6b7d7370 0x55fd6b7e38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.585+0000 7f697659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd6b7d7c80 0x55fd6b7e0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53808/0 (socket says 172.21.15.52:53808) 2023-11-27T15:22:42.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.585+0000 7f697659c700 1 -- 172.21.15.52:0/2397540014 learned_addr learned my addr 172.21.15.52:0/2397540014 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:22:42.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.585+0000 7f697759e700 1 --2- 172.21.15.52:0/2397540014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd6b7dc0c0 0x55fd6b7e1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.586+0000 7f6976d9d700 1 -- 172.21.15.52:0/2397540014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd6b7dc0c0 msgr2=0x55fd6b7e1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:42.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.586+0000 7f6976d9d700 1 --2- 172.21.15.52:0/2397540014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd6b7dc0c0 0x55fd6b7e1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.586+0000 7f6976d9d700 1 -- 172.21.15.52:0/2397540014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd6b7d7c80 msgr2=0x55fd6b7e0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:42.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.586+0000 7f6976d9d700 1 --2- 172.21.15.52:0/2397540014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd6b7d7c80 0x55fd6b7e0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.586+0000 7f6976d9d700 1 -- 172.21.15.52:0/2397540014 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697810c8a0 con 0x55fd6b7d7370 2023-11-27T15:22:42.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.586+0000 7f697759e700 1 --2- 172.21.15.52:0/2397540014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd6b7dc0c0 0x55fd6b7e1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:42.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.586+0000 7f6976d9d700 1 --2- 172.21.15.52:0/2397540014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fd6b7d7370 0x55fd6b7e38d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f696801eb20 tx=0x7f69680047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfba7491958fde2f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:42.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.586+0000 7f695aff5700 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2023-11-27T15:22:42.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.586+0000 7f695aff5700 1 -- 172.21.15.52:0/2397540014 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696800f880 con 0x55fd6b7d7370 2023-11-27T15:22:42.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.586+0000 7f695aff5700 1 -- 172.21.15.52:0/2397540014 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f696800f9e0 con 0x55fd6b7d7370 2023-11-27T15:22:42.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.586+0000 7f698919fb80 1 -- 172.21.15.52:0/2397540014 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd6b7c1b20 con 0x55fd6b7d7370 2023-11-27T15:22:42.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.586+0000 7f695aff5700 1 -- 172.21.15.52:0/2397540014 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f696800fba0 con 0x55fd6b7d7370 2023-11-27T15:22:42.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.586+0000 7f698919fb80 10 client.25973 fetch_fsmap learned FSMap version 28 2023-11-27T15:22:42.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.586+0000 7f698919fb80 1 -- 172.21.15.52:0/2397540014 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55fd6b6e6190 con 0x55fd6b7d7370 2023-11-27T15:22:42.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.586+0000 7f695aff5700 1 -- 172.21.15.52:0/2397540014 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6968032080 con 0x55fd6b7d7370 2023-11-27T15:22:42.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.587+0000 7f695aff5700 1 -- 172.21.15.52:0/2397540014 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f696802c030 con 0x55fd6b7d7370 2023-11-27T15:22:42.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.587+0000 7f698919fb80 10 client.25973 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:22:42.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.587+0000 7f698919fb80 10 client.25973 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:22:42.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.587+0000 7f698919fb80 20 client.25973 populate_metadata read hostname 'smithi052' 2023-11-27T15:22:42.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.587+0000 7f698919fb80 1 --2- 172.21.15.52:0/2397540014 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55fd6b7f1250 0x55fd6b7f3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.587+0000 7f698919fb80 4 client.25973 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:22:42.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.587+0000 7f698919fb80 1 -- 172.21.15.52:0/2397540014 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd6b6e20a0 con 0x55fd6b7f1250 2023-11-27T15:22:42.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.587+0000 7f697659c700 1 --2- 172.21.15.52:0/2397540014 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55fd6b7f1250 0x55fd6b7f3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.587+0000 7f697659c700 1 --2- 172.21.15.52:0/2397540014 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55fd6b7f1250 0x55fd6b7f3660 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fd0bab641e22103 server_cookie=2fabcd84b1d4e9ff in_seq=0 out_seq=0 2023-11-27T15:22:42.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.587+0000 7f695aff5700 10 client.25973 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:22:42.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.588+0000 7f695aff5700 1 -- 172.21.15.52:0/2397540014 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fd6b6e20a0 con 0x55fd6b7f1250 2023-11-27T15:22:42.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.588+0000 7f695aff5700 10 client.25973 handle_command_reply: tid=0 2023-11-27T15:22:42.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.588+0000 7f698919fb80 1 client.25973 shutdown 2023-11-27T15:22:42.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.588+0000 7f698919fb80 1 -- 172.21.15.52:0/2397540014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fd6b7d7370 msgr2=0x55fd6b7e38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:42.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.588+0000 7f698919fb80 1 --2- 172.21.15.52:0/2397540014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fd6b7d7370 0x55fd6b7e38d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f696801eb20 tx=0x7f69680047c0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.588+0000 7f698919fb80 1 -- 172.21.15.52:0/2397540014 shutdown_connections 2023-11-27T15:22:42.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.588+0000 7f698919fb80 1 --2- 172.21.15.52:0/2397540014 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55fd6b7f1250 0x55fd6b7f3660 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.588+0000 7f698919fb80 1 --2- 172.21.15.52:0/2397540014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fd6b7d7370 0x55fd6b7e38d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.588+0000 7f698919fb80 1 --2- 172.21.15.52:0/2397540014 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd6b7dc0c0 0x55fd6b7e1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.588+0000 7f698919fb80 1 --2- 172.21.15.52:0/2397540014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd6b7d7c80 0x55fd6b7e0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.588+0000 7f698919fb80 1 -- 172.21.15.52:0/2397540014 >> 172.21.15.52:0/2397540014 conn(0x55fd6b7c4360 msgr2=0x55fd6b7dd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:42.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.588+0000 7f695aff5700 0 client.25973 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:22:42.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.588+0000 7f698919fb80 1 -- 172.21.15.52:0/2397540014 shutdown_connections 2023-11-27T15:22:42.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.588+0000 7f698919fb80 1 -- 172.21.15.52:0/2397540014 wait complete. 2023-11-27T15:22:42.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.588+0000 7f698919fb80 20 client.25973 trim_cache size 0 max 16384 2023-11-27T15:22:42.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.605+0000 7f698919fb80 1 Processor -- start 2023-11-27T15:22:42.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.605+0000 7f698919fb80 1 -- start start 2023-11-27T15:22:42.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.605+0000 7f698919fb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fd6b7d7370 0x55fd6b7e8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.605+0000 7f698919fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd6b7d7c80 0x55fd6b7ec900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.605+0000 7f698919fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd6b7dc0c0 0x55fd6b7e3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.605+0000 7f698919fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6b7df150 con 0x55fd6b7d7c80 2023-11-27T15:22:42.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.605+0000 7f698919fb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6b7dac80 con 0x55fd6b7d7370 2023-11-27T15:22:42.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.605+0000 7f698919fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6b7f3970 con 0x55fd6b7dc0c0 2023-11-27T15:22:42.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.605+0000 7f697659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd6b7d7c80 0x55fd6b7ec900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.605+0000 7f697759e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd6b7dc0c0 0x55fd6b7e3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.605+0000 7f697659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd6b7d7c80 0x55fd6b7ec900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53816/0 (socket says 172.21.15.52:53816) 2023-11-27T15:22:42.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.605+0000 7f6976d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fd6b7d7370 0x55fd6b7e8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.605+0000 7f697659c700 1 -- 172.21.15.52:0/1046121993 learned_addr learned my addr 172.21.15.52:0/1046121993 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:22:42.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.606+0000 7f697659c700 1 -- 172.21.15.52:0/1046121993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd6b7dc0c0 msgr2=0x55fd6b7e3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:42.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.606+0000 7f697659c700 1 --2- 172.21.15.52:0/1046121993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd6b7dc0c0 0x55fd6b7e3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.606+0000 7f697659c700 1 -- 172.21.15.52:0/1046121993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fd6b7d7370 msgr2=0x55fd6b7e8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:42.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.606+0000 7f697659c700 1 --2- 172.21.15.52:0/1046121993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fd6b7d7370 0x55fd6b7e8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.606+0000 7f697659c700 1 -- 172.21.15.52:0/1046121993 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd6b7c5b00 con 0x55fd6b7d7c80 2023-11-27T15:22:42.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.606+0000 7f697759e700 1 --2- 172.21.15.52:0/1046121993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd6b7dc0c0 0x55fd6b7e3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:42.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.606+0000 7f6976d9d700 1 --2- 172.21.15.52:0/1046121993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fd6b7d7370 0x55fd6b7e8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:42.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.606+0000 7f697659c700 1 --2- 172.21.15.52:0/1046121993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd6b7d7c80 0x55fd6b7ec900 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f696c0036c0 tx=0x7f696c00f630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a66da59c3788f58 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:42.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.606+0000 7f695dffb700 1 -- 172.21.15.52:0/1046121993 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c026040 con 0x55fd6b7d7c80 2023-11-27T15:22:42.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.606+0000 7f695dffb700 1 -- 172.21.15.52:0/1046121993 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f696c00b8b0 con 0x55fd6b7d7c80 2023-11-27T15:22:42.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.606+0000 7f695dffb700 1 -- 172.21.15.52:0/1046121993 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c0055f0 con 0x55fd6b7d7c80 2023-11-27T15:22:42.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.607+0000 7f698919fb80 1 -- 172.21.15.52:0/1046121993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd6b7d7c80 msgr2=0x55fd6b7ec900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:42.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.607+0000 7f698919fb80 1 --2- 172.21.15.52:0/1046121993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd6b7d7c80 0x55fd6b7ec900 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f696c0036c0 tx=0x7f696c00f630 comp rx=0 tx=0).stop 2023-11-27T15:22:42.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.607+0000 7f698919fb80 1 -- 172.21.15.52:0/1046121993 shutdown_connections 2023-11-27T15:22:42.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.607+0000 7f698919fb80 1 --2- 172.21.15.52:0/1046121993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fd6b7d7370 0x55fd6b7e8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.607+0000 7f698919fb80 1 --2- 172.21.15.52:0/1046121993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd6b7dc0c0 0x55fd6b7e3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.607+0000 7f698919fb80 1 --2- 172.21.15.52:0/1046121993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd6b7d7c80 0x55fd6b7ec900 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.607+0000 7f698919fb80 1 -- 172.21.15.52:0/1046121993 >> 172.21.15.52:0/1046121993 conn(0x55fd6b7c4360 msgr2=0x55fd6b7f3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:42.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.607+0000 7f698919fb80 1 -- 172.21.15.52:0/1046121993 shutdown_connections 2023-11-27T15:22:42.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.607+0000 7f698919fb80 1 -- 172.21.15.52:0/1046121993 wait complete. 2023-11-27T15:22:42.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.607+0000 7f698919fb80 1 -- start start 2023-11-27T15:22:42.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.607+0000 7f698919fb80 1 Processor -- start 2023-11-27T15:22:42.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.608+0000 7f698919fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd6b7d7370 0x55fd6b7e8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.608+0000 7f698919fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd6b7d7c80 0x55fd6b7e6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.608+0000 7f698919fb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fd6b7dc0c0 0x55fd6b7e6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.608+0000 7f698919fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6b7e6d50 con 0x55fd6b7d7370 2023-11-27T15:22:42.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.608+0000 7f698919fb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6b7e6ec0 con 0x55fd6b7dc0c0 2023-11-27T15:22:42.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.608+0000 7f698919fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6b8440d0 con 0x55fd6b7d7c80 2023-11-27T15:22:42.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.608+0000 7f698919fb80 10 client.0 _ll_register_callbacks cb 0x7f696801e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:22:42.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.608+0000 7f6976d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd6b7d7370 0x55fd6b7e8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.608+0000 7f697659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd6b7d7c80 0x55fd6b7e6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.608+0000 7f6976d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd6b7d7370 0x55fd6b7e8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53820/0 (socket says 172.21.15.52:53820) 2023-11-27T15:22:42.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.608+0000 7f6976d9d700 1 -- 172.21.15.52:0/2142331384 learned_addr learned my addr 172.21.15.52:0/2142331384 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:22:42.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.608+0000 7f697759e700 1 --2- 172.21.15.52:0/2142331384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fd6b7dc0c0 0x55fd6b7e6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.608+0000 7f697659c700 1 -- 172.21.15.52:0/2142331384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fd6b7dc0c0 msgr2=0x55fd6b7e6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:42.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.608+0000 7f697659c700 1 --2- 172.21.15.52:0/2142331384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fd6b7dc0c0 0x55fd6b7e6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.608+0000 7f697659c700 1 -- 172.21.15.52:0/2142331384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd6b7d7370 msgr2=0x55fd6b7e8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:42.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.608+0000 7f697659c700 1 --2- 172.21.15.52:0/2142331384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd6b7d7370 0x55fd6b7e8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.608+0000 7f697659c700 1 -- 172.21.15.52:0/2142331384 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f696c0095f0 con 0x55fd6b7d7c80 2023-11-27T15:22:42.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.608+0000 7f6976d9d700 1 --2- 172.21.15.52:0/2142331384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd6b7d7370 0x55fd6b7e8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:42.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.608+0000 7f697659c700 1 --2- 172.21.15.52:0/2142331384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd6b7d7c80 0x55fd6b7e6140 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f696c0058e0 tx=0x7f696c005050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=964c10f45eac69f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:42.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.608+0000 7f695aff5700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2023-11-27T15:22:42.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.608+0000 7f695aff5700 1 -- 172.21.15.52:0/2142331384 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c026070 con 0x55fd6b7d7c80 2023-11-27T15:22:42.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.608+0000 7f698919fb80 1 -- 172.21.15.52:0/2142331384 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f696c01e040 con 0x55fd6b7d7c80 2023-11-27T15:22:42.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.608+0000 7f695aff5700 1 -- 172.21.15.52:0/2142331384 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f696c031ac0 con 0x55fd6b7d7c80 2023-11-27T15:22:42.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.609+0000 7f695aff5700 1 -- 172.21.15.52:0/2142331384 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f696c031ca0 con 0x55fd6b7d7c80 2023-11-27T15:22:42.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.609+0000 7f698919fb80 10 client.25867 fetch_fsmap learned FSMap version 28 2023-11-27T15:22:42.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.609+0000 7f698919fb80 1 -- 172.21.15.52:0/2142331384 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55fd6b5a1630 con 0x55fd6b7d7c80 2023-11-27T15:22:42.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.609+0000 7f695aff5700 1 -- 172.21.15.52:0/2142331384 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f696c01e040 con 0x55fd6b7d7c80 2023-11-27T15:22:42.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.609+0000 7f695aff5700 1 -- 172.21.15.52:0/2142331384 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f696c021070 con 0x55fd6b7d7c80 2023-11-27T15:22:42.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.609+0000 7f698919fb80 10 client.25867 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:22:42.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.609+0000 7f698919fb80 10 client.25867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:22:42.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.609+0000 7f698919fb80 20 client.25867 populate_metadata read hostname 'smithi052' 2023-11-27T15:22:42.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.609+0000 7f698919fb80 1 --2- 172.21.15.52:0/2142331384 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55fd6b8451c0 0x55fd6b8475d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.609+0000 7f698919fb80 4 client.25867 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:22:42.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.609+0000 7f698919fb80 1 -- 172.21.15.52:0/2142331384 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fd6b6ee660 con 0x55fd6b8451c0 2023-11-27T15:22:42.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.609+0000 7f6976d9d700 1 --2- 172.21.15.52:0/2142331384 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55fd6b8451c0 0x55fd6b8475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.610+0000 7f695aff5700 10 client.25867 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:22:42.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.610+0000 7f6976d9d700 1 --2- 172.21.15.52:0/2142331384 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55fd6b8451c0 0x55fd6b8475d0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8683916c3574a2bd server_cookie=2b1c1595552a6d3c in_seq=0 out_seq=0 2023-11-27T15:22:42.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.610+0000 7f695aff5700 1 -- 172.21.15.52:0/2142331384 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55fd6b6ee660 con 0x55fd6b8451c0 2023-11-27T15:22:42.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.610+0000 7f695aff5700 10 client.25867 handle_command_reply: tid=0 2023-11-27T15:22:42.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.610+0000 7f698919fb80 1 client.25867 shutdown 2023-11-27T15:22:42.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.610+0000 7f698919fb80 1 -- 172.21.15.52:0/2142331384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd6b7d7c80 msgr2=0x55fd6b7e6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:42.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.610+0000 7f698919fb80 1 --2- 172.21.15.52:0/2142331384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd6b7d7c80 0x55fd6b7e6140 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f696c0058e0 tx=0x7f696c005050 comp rx=0 tx=0).stop 2023-11-27T15:22:42.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.610+0000 7f698919fb80 1 -- 172.21.15.52:0/2142331384 shutdown_connections 2023-11-27T15:22:42.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.610+0000 7f698919fb80 1 --2- 172.21.15.52:0/2142331384 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55fd6b8451c0 0x55fd6b8475d0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.610+0000 7f698919fb80 1 --2- 172.21.15.52:0/2142331384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fd6b7dc0c0 0x55fd6b7e6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.610+0000 7f695aff5700 0 client.25867 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:22:42.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.610+0000 7f698919fb80 1 --2- 172.21.15.52:0/2142331384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fd6b7d7370 0x55fd6b7e8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.610+0000 7f698919fb80 1 --2- 172.21.15.52:0/2142331384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd6b7d7c80 0x55fd6b7e6140 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.610+0000 7f698919fb80 1 -- 172.21.15.52:0/2142331384 >> 172.21.15.52:0/2142331384 conn(0x55fd6b7c4360 msgr2=0x55fd6b7ec900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:42.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.610+0000 7f698919fb80 1 -- 172.21.15.52:0/2142331384 shutdown_connections 2023-11-27T15:22:42.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.610+0000 7f698919fb80 1 -- 172.21.15.52:0/2142331384 wait complete. 2023-11-27T15:22:42.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.610+0000 7f698919fb80 20 client.25867 trim_cache size 0 max 16384 2023-11-27T15:22:42.621 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:22:42.621 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-27T15:22:42.621 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2023-11-27T15:22:42.622 INFO:teuthology.orchestra.run.smithi052.stdout: "1a14f367-9a76-4e86-b4c8-bd13c3d4d297": { 2023-11-27T15:22:42.622 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2023-11-27T15:22:42.622 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "1a14f367-9a76-4e86-b4c8-bd13c3d4d297", 2023-11-27T15:22:42.622 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2023-11-27T15:22:42.622 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:22:42.622 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:22:42.622 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:22:42.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.611+0000 7f697d013700 1 -- 172.21.15.52:0/1044506730 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f69600a12c0 msgr2=0x7f69600a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:42.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.611+0000 7f697d013700 1 --2- 172.21.15.52:0/1044506730 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f69600a12c0 0x7f69600a3770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f69680115c0 tx=0x7f6968011980 comp rx=0 tx=0).stop 2023-11-27T15:22:42.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.611+0000 7f697d013700 1 -- 172.21.15.52:0/1044506730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6978104490 msgr2=0x7f69781a45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:42.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.611+0000 7f697d013700 1 --2- 172.21.15.52:0/1044506730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6978104490 0x7f69781a45d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f697000efb0 tx=0x7f697000c3a0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.611+0000 7f697d013700 1 -- 172.21.15.52:0/1044506730 shutdown_connections 2023-11-27T15:22:42.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.611+0000 7f697d013700 1 --2- 172.21.15.52:0/1044506730 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f69600a12c0 0x7f69600a3770 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.611+0000 7f697d013700 1 --2- 172.21.15.52:0/1044506730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6978104490 0x7f69781a45d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.611+0000 7f697d013700 1 --2- 172.21.15.52:0/1044506730 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f69780ff450 0x7f6978100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.611+0000 7f697d013700 1 --2- 172.21.15.52:0/1044506730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69780ffdf0 0x7f6978101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.611+0000 7f697d013700 1 -- 172.21.15.52:0/1044506730 >> 172.21.15.52:0/1044506730 conn(0x7f69780770b0 msgr2=0x7f6978102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:42.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.611+0000 7f697d013700 1 -- 172.21.15.52:0/1044506730 shutdown_connections 2023-11-27T15:22:42.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:22:42.611+0000 7f697d013700 1 -- 172.21.15.52:0/1044506730 wait complete. 2023-11-27T15:22:42.627 INFO:tasks.cephfs.filesystem:scrub status for tag:1a14f367-9a76-4e86-b4c8-bd13c3d4d297 - {'path': '/', 'tag': '1a14f367-9a76-4e86-b4c8-bd13c3d4d297', 'options': 'recursive,force'} 2023-11-27T15:22:44.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:44 smithi150 ceph-mon[100559]: pgmap v147: 97 pgs: 97 active+clean; 690 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 38 op/s 2023-11-27T15:22:44.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:44 smithi150 ceph-mon[100559]: scrub summary: active paths [/] 2023-11-27T15:22:44.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:44 smithi102 ceph-mon[100296]: pgmap v147: 97 pgs: 97 active+clean; 690 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 38 op/s 2023-11-27T15:22:44.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:44 smithi102 ceph-mon[100296]: scrub summary: active paths [/] 2023-11-27T15:22:44.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:44 smithi052 ceph-mon[99056]: pgmap v147: 97 pgs: 97 active+clean; 690 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 38 op/s 2023-11-27T15:22:44.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:44 smithi052 ceph-mon[99056]: scrub summary: active paths [/] 2023-11-27T15:22:45.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:45 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:22:45.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:45 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:22:45.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:45 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:22:45.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:45 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:22:45.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:45 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:22:45.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:45 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:22:46.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:46 smithi150 ceph-mon[100559]: pgmap v148: 97 pgs: 97 active+clean; 690 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 38 op/s 2023-11-27T15:22:46.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:46 smithi102 ceph-mon[100296]: pgmap v148: 97 pgs: 97 active+clean; 690 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 38 op/s 2023-11-27T15:22:46.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:46 smithi052 ceph-mon[99056]: pgmap v148: 97 pgs: 97 active+clean; 690 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 38 op/s 2023-11-27T15:22:46.964 INFO:tasks.workunit.client.0.smithi052.stdout:cleaned up hierarchy 2023-11-27T15:22:48.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:48 smithi150 ceph-mon[100559]: pgmap v149: 97 pgs: 97 active+clean; 708 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 90 op/s 2023-11-27T15:22:48.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:48 smithi102 ceph-mon[100296]: pgmap v149: 97 pgs: 97 active+clean; 708 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 90 op/s 2023-11-27T15:22:48.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:48 smithi052 ceph-mon[99056]: pgmap v149: 97 pgs: 97 active+clean; 708 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 90 op/s 2023-11-27T15:22:50.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:49 smithi052 ceph-mon[99056]: pgmap v150: 97 pgs: 97 active+clean; 708 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 71 op/s 2023-11-27T15:22:50.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:49 smithi150 ceph-mon[100559]: pgmap v150: 97 pgs: 97 active+clean; 708 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 71 op/s 2023-11-27T15:22:50.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:49 smithi102 ceph-mon[100296]: pgmap v150: 97 pgs: 97 active+clean; 708 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 71 op/s 2023-11-27T15:22:51.932 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-27T15:22:51.932 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-27T15:22:51.971 INFO:tasks.workunit:Running workunit fs/misc/filelock_deadlock.py... 2023-11-27T15:22:51.972 DEBUG:teuthology.orchestra.run.smithi052:workunit test fs/misc/filelock_deadlock.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/filelock_deadlock.py 2023-11-27T15:22:52.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:52 smithi052 ceph-mon[99056]: pgmap v151: 97 pgs: 97 active+clean; 711 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 80 op/s 2023-11-27T15:22:52.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:52 smithi150 ceph-mon[100559]: pgmap v151: 97 pgs: 97 active+clean; 711 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 80 op/s 2023-11-27T15:22:52.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:52 smithi102 ceph-mon[100296]: pgmap v151: 97 pgs: 97 active+clean; 711 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 80 op/s 2023-11-27T15:22:54.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:54 smithi150 ceph-mon[100559]: pgmap v152: 97 pgs: 97 active+clean; 711 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 60 op/s 2023-11-27T15:22:54.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:54 smithi102 ceph-mon[100296]: pgmap v152: 97 pgs: 97 active+clean; 711 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 60 op/s 2023-11-27T15:22:54.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:54 smithi052 ceph-mon[99056]: pgmap v152: 97 pgs: 97 active+clean; 711 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 60 op/s 2023-11-27T15:22:55.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:55 smithi150 ceph-mon[100559]: scrub summary: idle 2023-11-27T15:22:55.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:55 smithi150 ceph-mon[100559]: scrub summary: idle 2023-11-27T15:22:55.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:55 smithi150 ceph-mon[100559]: scrub summary: idle 2023-11-27T15:22:55.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:55 smithi102 ceph-mon[100296]: scrub summary: idle 2023-11-27T15:22:55.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:55 smithi102 ceph-mon[100296]: scrub summary: idle 2023-11-27T15:22:55.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:55 smithi102 ceph-mon[100296]: scrub summary: idle 2023-11-27T15:22:55.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:55 smithi052 ceph-mon[99056]: scrub summary: idle 2023-11-27T15:22:55.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:55 smithi052 ceph-mon[99056]: scrub summary: idle 2023-11-27T15:22:55.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:55 smithi052 ceph-mon[99056]: scrub summary: idle 2023-11-27T15:22:56.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:56 smithi150 ceph-mon[100559]: pgmap v153: 97 pgs: 97 active+clean; 711 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 60 op/s 2023-11-27T15:22:56.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:56 smithi102 ceph-mon[100296]: pgmap v153: 97 pgs: 97 active+clean; 711 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 60 op/s 2023-11-27T15:22:56.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:56 smithi052 ceph-mon[99056]: pgmap v153: 97 pgs: 97 active+clean; 711 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 60 op/s 2023-11-27T15:22:58.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:58 smithi150 ceph-mon[100559]: pgmap v154: 97 pgs: 97 active+clean; 712 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 69 op/s 2023-11-27T15:22:58.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:58 smithi102 ceph-mon[100296]: pgmap v154: 97 pgs: 97 active+clean; 712 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 69 op/s 2023-11-27T15:22:58.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:58 smithi052 ceph-mon[99056]: pgmap v154: 97 pgs: 97 active+clean; 712 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 69 op/s 2023-11-27T15:23:00.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:22:59 smithi052 ceph-mon[99056]: pgmap v155: 97 pgs: 97 active+clean; 712 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 286 KiB/s wr, 18 op/s 2023-11-27T15:23:00.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:22:59 smithi150 ceph-mon[100559]: pgmap v155: 97 pgs: 97 active+clean; 712 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 286 KiB/s wr, 18 op/s 2023-11-27T15:23:00.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:22:59 smithi102 ceph-mon[100296]: pgmap v155: 97 pgs: 97 active+clean; 712 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 286 KiB/s wr, 18 op/s 2023-11-27T15:23:02.095 INFO:tasks.workunit.client.0.smithi052.stdout:ok 2023-11-27T15:23:02.100 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-27T15:23:02.101 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-27T15:23:02.157 INFO:tasks.workunit:Running workunit fs/misc/filelock_interrupt.py... 2023-11-27T15:23:02.157 DEBUG:teuthology.orchestra.run.smithi052:workunit test fs/misc/filelock_interrupt.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/filelock_interrupt.py 2023-11-27T15:23:02.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:02 smithi150 ceph-mon[100559]: pgmap v156: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 340 KiB/s wr, 34 op/s 2023-11-27T15:23:02.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:02 smithi102 ceph-mon[100296]: pgmap v156: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 340 KiB/s wr, 34 op/s 2023-11-27T15:23:02.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:02 smithi052 ceph-mon[99056]: pgmap v156: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 340 KiB/s wr, 34 op/s 2023-11-27T15:23:04.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:04 smithi150 ceph-mon[100559]: pgmap v157: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 108 KiB/s wr, 25 op/s 2023-11-27T15:23:04.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:04 smithi102 ceph-mon[100296]: pgmap v157: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 108 KiB/s wr, 25 op/s 2023-11-27T15:23:04.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:04 smithi052 ceph-mon[99056]: pgmap v157: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 108 KiB/s wr, 25 op/s 2023-11-27T15:23:06.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:06 smithi150 ceph-mon[100559]: pgmap v158: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 108 KiB/s wr, 25 op/s 2023-11-27T15:23:06.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:06 smithi102 ceph-mon[100296]: pgmap v158: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 108 KiB/s wr, 25 op/s 2023-11-27T15:23:06.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:06 smithi052 ceph-mon[99056]: pgmap v158: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 108 KiB/s wr, 25 op/s 2023-11-27T15:23:08.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:08 smithi150 ceph-mon[100559]: pgmap v159: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 115 KiB/s wr, 28 op/s 2023-11-27T15:23:08.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:08 smithi102 ceph-mon[100296]: pgmap v159: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 115 KiB/s wr, 28 op/s 2023-11-27T15:23:08.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:08 smithi052 ceph-mon[99056]: pgmap v159: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 115 KiB/s wr, 28 op/s 2023-11-27T15:23:10.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:09 smithi052 ceph-mon[99056]: pgmap v160: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 61 KiB/s wr, 19 op/s 2023-11-27T15:23:10.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:09 smithi150 ceph-mon[100559]: pgmap v160: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 61 KiB/s wr, 19 op/s 2023-11-27T15:23:10.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:09 smithi102 ceph-mon[100296]: pgmap v160: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 61 KiB/s wr, 19 op/s 2023-11-27T15:23:12.215 INFO:tasks.workunit.client.0.smithi052.stdout:ok 2023-11-27T15:23:12.220 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-27T15:23:12.220 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-27T15:23:12.268 INFO:tasks.workunit:Running workunit fs/misc/general_vxattrs.sh... 2023-11-27T15:23:12.268 DEBUG:teuthology.orchestra.run.smithi052:workunit test fs/misc/general_vxattrs.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh 2023-11-27T15:23:12.312 INFO:tasks.workunit.client.0.smithi052.stderr:+ mkdir -p dir 2023-11-27T15:23:12.314 INFO:tasks.workunit.client.0.smithi052.stderr:+ getfattr -n ceph.dir.pin dir 2023-11-27T15:23:12.315 INFO:tasks.workunit.client.0.smithi052.stderr:+ grep 'ceph.dir.pin="-1"' 2023-11-27T15:23:12.316 INFO:tasks.workunit.client.0.smithi052.stdout:ceph.dir.pin="-1" 2023-11-27T15:23:12.316 INFO:tasks.workunit.client.0.smithi052.stderr:+ setfattr -n ceph.dir.pin dir 2023-11-27T15:23:12.316 INFO:tasks.workunit.client.0.smithi052.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-11-27T15:23:12.317 INFO:tasks.workunit.client.0.smithi052.stdout:setfattr: dir: Invalid argument 2023-11-27T15:23:12.317 INFO:tasks.workunit.client.0.smithi052.stderr:+ setfattr -n ceph.dir.pin -v 1 dir 2023-11-27T15:23:12.319 INFO:tasks.workunit.client.0.smithi052.stderr:+ getfattr -n ceph.dir.pin dir 2023-11-27T15:23:12.319 INFO:tasks.workunit.client.0.smithi052.stderr:+ grep 'ceph.dir.pin="1"' 2023-11-27T15:23:12.320 INFO:tasks.workunit.client.0.smithi052.stdout:ceph.dir.pin="1" 2023-11-27T15:23:12.321 INFO:tasks.workunit.client.0.smithi052.stderr:+ setfattr -x ceph.dir.pin dir 2023-11-27T15:23:12.322 INFO:tasks.workunit.client.0.smithi052.stderr:+ getfattr -n ceph.dir.pin dir 2023-11-27T15:23:12.323 INFO:tasks.workunit.client.0.smithi052.stderr:+ grep 'ceph.dir.pin="-1"' 2023-11-27T15:23:12.323 INFO:tasks.workunit.client.0.smithi052.stdout:ceph.dir.pin="-1" 2023-11-27T15:23:12.324 INFO:tasks.workunit.client.0.smithi052.stderr:+ setfattr -n ceph.dir.pin.distributed dir 2023-11-27T15:23:12.324 INFO:tasks.workunit.client.0.smithi052.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-11-27T15:23:12.325 INFO:tasks.workunit.client.0.smithi052.stdout:setfattr: dir: Invalid argument 2023-11-27T15:23:12.325 INFO:tasks.workunit.client.0.smithi052.stderr:+ setfattr -n ceph.dir.pin.distributed -v 1 dir 2023-11-27T15:23:12.326 INFO:tasks.workunit.client.0.smithi052.stderr:+ setfattr -x ceph.dir.pin.distributed dir 2023-11-27T15:23:12.328 INFO:tasks.workunit.client.0.smithi052.stderr:+ setfattr -n ceph.dir.pin.random dir 2023-11-27T15:23:12.328 INFO:tasks.workunit.client.0.smithi052.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-11-27T15:23:12.329 INFO:tasks.workunit.client.0.smithi052.stdout:setfattr: dir: Invalid argument 2023-11-27T15:23:12.329 INFO:tasks.workunit.client.0.smithi052.stderr:+ setfattr -n ceph.dir.pin.random -v 0.01 dir 2023-11-27T15:23:12.331 INFO:tasks.workunit.client.0.smithi052.stderr:+ setfattr -x ceph.dir.pin.random dir 2023-11-27T15:23:12.333 INFO:tasks.workunit.client.0.smithi052.stderr:+ setfattr -n ceph.dir.subvolume dir 2023-11-27T15:23:12.333 INFO:tasks.workunit.client.0.smithi052.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-11-27T15:23:12.334 INFO:tasks.workunit.client.0.smithi052.stdout:setfattr: dir: Invalid argument 2023-11-27T15:23:12.334 INFO:tasks.workunit.client.0.smithi052.stderr:+ setfattr -n ceph.dir.subvolume -v 1 dir 2023-11-27T15:23:12.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T15:23:12.343 INFO:tasks.workunit.client.0.smithi052.stderr:setfattr: dir: Invalid argument 2023-11-27T15:23:12.344 INFO:tasks.workunit:Stopping ['fs/misc'] on client.0... 2023-11-27T15:23:12.344 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-11-27T15:23:12.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:12 smithi150 ceph-mon[100559]: pgmap v161: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 64 KiB/s wr, 23 op/s 2023-11-27T15:23:12.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:12 smithi102 ceph-mon[100296]: pgmap v161: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 64 KiB/s wr, 23 op/s 2023-11-27T15:23:12.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:12 smithi052 ceph-mon[99056]: pgmap v161: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 64 KiB/s wr, 23 op/s 2023-11-27T15:23:12.628 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:23:12.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.758+0000 7ff04304d700 1 -- 172.21.15.52:0/1004019208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff03c101370 msgr2=0x7ff03c10d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:12.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.758+0000 7ff04304d700 1 --2- 172.21.15.52:0/1004019208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff03c101370 0x7ff03c10d1d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff03400b0a0 tx=0x7ff03400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.758+0000 7ff04304d700 1 -- 172.21.15.52:0/1004019208 shutdown_connections 2023-11-27T15:23:12.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.758+0000 7ff04304d700 1 --2- 172.21.15.52:0/1004019208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff03c101370 0x7ff03c10d1d0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.758+0000 7ff04304d700 1 --2- 172.21.15.52:0/1004019208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff03c106410 0x7ff03c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.758+0000 7ff04304d700 1 --2- 172.21.15.52:0/1004019208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff03c1009e0 0x7ff03c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.758+0000 7ff04304d700 1 -- 172.21.15.52:0/1004019208 >> 172.21.15.52:0/1004019208 conn(0x7ff03c0775c0 msgr2=0x7ff03c0779c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:12.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.758+0000 7ff04304d700 1 -- 172.21.15.52:0/1004019208 shutdown_connections 2023-11-27T15:23:12.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.758+0000 7ff04304d700 1 -- 172.21.15.52:0/1004019208 wait complete. 2023-11-27T15:23:12.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.758+0000 7ff04304d700 1 Processor -- start 2023-11-27T15:23:12.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.759+0000 7ff04304d700 1 -- start start 2023-11-27T15:23:12.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.759+0000 7ff04304d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff03c1009e0 0x7ff03c072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:12.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.759+0000 7ff04304d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff03c101370 0x7ff03c073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:12.766 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.759+0000 7ff04304d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff03c106410 0x7ff03c1a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:12.766 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.759+0000 7ff04304d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c073b60 con 0x7ff03c106410 2023-11-27T15:23:12.766 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.759+0000 7ff04304d700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c1a4ca0 con 0x7ff03c101370 2023-11-27T15:23:12.766 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.759+0000 7ff04304d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c1a4de0 con 0x7ff03c1009e0 2023-11-27T15:23:12.766 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.759+0000 7ff0415ea700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff03c106410 0x7ff03c1a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:12.766 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.759+0000 7ff03bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff03c101370 0x7ff03c073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:12.766 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.759+0000 7ff040de9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff03c1009e0 0x7ff03c072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:12.766 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.759+0000 7ff040de9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff03c1009e0 0x7ff03c072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:40478/0 (socket says 172.21.15.52:40478) 2023-11-27T15:23:12.766 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.759+0000 7ff03bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff03c101370 0x7ff03c073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43494/0 (socket says 172.21.15.52:43494) 2023-11-27T15:23:12.766 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.759+0000 7ff040de9700 1 -- 172.21.15.52:0/2996248140 learned_addr learned my addr 172.21.15.52:0/2996248140 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:12.766 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.759+0000 7ff0415ea700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff03c106410 0x7ff03c1a4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53838/0 (socket says 172.21.15.52:53838) 2023-11-27T15:23:12.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.759+0000 7ff03bfff700 1 -- 172.21.15.52:0/2996248140 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff03c1009e0 msgr2=0x7ff03c072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:12.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.759+0000 7ff03bfff700 1 --2- 172.21.15.52:0/2996248140 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff03c1009e0 0x7ff03c072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.759+0000 7ff03bfff700 1 -- 172.21.15.52:0/2996248140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff03c106410 msgr2=0x7ff03c1a4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:12.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.759+0000 7ff03bfff700 1 --2- 172.21.15.52:0/2996248140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff03c106410 0x7ff03c1a4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.759+0000 7ff03bfff700 1 -- 172.21.15.52:0/2996248140 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff034009d00 con 0x7ff03c101370 2023-11-27T15:23:12.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.760+0000 7ff03bfff700 1 --2- 172.21.15.52:0/2996248140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff03c101370 0x7ff03c073330 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff02800e9f0 tx=0x7ff02800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:12.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.760+0000 7ff039ffb700 1 -- 172.21.15.52:0/2996248140 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02800c9c0 con 0x7ff03c101370 2023-11-27T15:23:12.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.760+0000 7ff04304d700 1 -- 172.21.15.52:0/2996248140 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff03c110450 con 0x7ff03c101370 2023-11-27T15:23:12.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.760+0000 7ff039ffb700 1 -- 172.21.15.52:0/2996248140 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff028010430 con 0x7ff03c101370 2023-11-27T15:23:12.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.760+0000 7ff039ffb700 1 -- 172.21.15.52:0/2996248140 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff028018680 con 0x7ff03c101370 2023-11-27T15:23:12.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.760+0000 7ff04304d700 1 -- 172.21.15.52:0/2996248140 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff03c1102c0 con 0x7ff03c101370 2023-11-27T15:23:12.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.763+0000 7ff039ffb700 1 -- 172.21.15.52:0/2996248140 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7ff028010860 con 0x7ff03c101370 2023-11-27T15:23:12.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.763+0000 7ff039ffb700 1 --2- 172.21.15.52:0/2996248140 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ff02c0a12c0 0x7ff02c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:12.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.764+0000 7ff039ffb700 1 -- 172.21.15.52:0/2996248140 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff028014070 con 0x7ff03c101370 2023-11-27T15:23:12.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.764+0000 7ff040de9700 1 --2- 172.21.15.52:0/2996248140 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ff02c0a12c0 0x7ff02c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:12.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.764+0000 7ff04f1d9b80 1 Processor -- start 2023-11-27T15:23:12.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.764+0000 7ff04f1d9b80 1 -- start start 2023-11-27T15:23:12.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.764+0000 7ff040de9700 1 --2- 172.21.15.52:0/2996248140 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ff02c0a12c0 0x7ff02c0a3770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff030009ca0 tx=0x7ff030009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:12.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.764+0000 7ff04f1d9b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559be11c9890 0x559be11c1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:12.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.764+0000 7ff04f1d9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559be11c9c60 0x559be11c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:12.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.764+0000 7ff04f1d9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559be11c23d0 0x559be11e21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:12.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.764+0000 7ff04f1d9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x559be10ca060 con 0x559be11c23d0 2023-11-27T15:23:12.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.764+0000 7ff04f1d9b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559be111c8a0 con 0x559be11c9890 2023-11-27T15:23:12.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.764+0000 7ff04f1d9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559be1115d30 con 0x559be11c9c60 2023-11-27T15:23:12.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.764+0000 7ff0415ea700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559be11c23d0 0x559be11e21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:12.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.765+0000 7ff0415ea700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559be11c23d0 0x559be11e21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53846/0 (socket says 172.21.15.52:53846) 2023-11-27T15:23:12.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.765+0000 7ff0415ea700 1 -- 172.21.15.52:0/186165737 learned_addr learned my addr 172.21.15.52:0/186165737 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:12.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.765+0000 7ff040de9700 1 --2- 172.21.15.52:0/186165737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559be11c9890 0x559be11c1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:12.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.765+0000 7ff0415ea700 1 -- 172.21.15.52:0/186165737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559be11c9c60 msgr2=0x559be11c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:12.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.765+0000 7ff0415ea700 1 --2- 172.21.15.52:0/186165737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559be11c9c60 0x559be11c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.765+0000 7ff0415ea700 1 -- 172.21.15.52:0/186165737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559be11c9890 msgr2=0x559be11c1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:12.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.765+0000 7ff0415ea700 1 --2- 172.21.15.52:0/186165737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559be11c9890 0x559be11c1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.765+0000 7ff0415ea700 1 -- 172.21.15.52:0/186165737 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559be0f97630 con 0x559be11c23d0 2023-11-27T15:23:12.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.765+0000 7ff0415ea700 1 --2- 172.21.15.52:0/186165737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559be11c23d0 0x559be11e21b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff034009c70 tx=0x7ff0340092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20229118fa79def server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:12.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.765+0000 7ff026ffd700 1 -- 172.21.15.52:0/186165737 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03400e040 con 0x559be11c23d0 2023-11-27T15:23:12.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.765+0000 7ff026ffd700 1 -- 172.21.15.52:0/186165737 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff034003a00 con 0x559be11c23d0 2023-11-27T15:23:12.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.765+0000 7ff026ffd700 1 -- 172.21.15.52:0/186165737 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03403e450 con 0x559be11c23d0 2023-11-27T15:23:12.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.766+0000 7ff04f1d9b80 1 -- 172.21.15.52:0/186165737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559be11c23d0 msgr2=0x559be11e21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:12.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.766+0000 7ff04f1d9b80 1 --2- 172.21.15.52:0/186165737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559be11c23d0 0x559be11e21b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff034009c70 tx=0x7ff0340092e0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.766+0000 7ff04f1d9b80 1 -- 172.21.15.52:0/186165737 shutdown_connections 2023-11-27T15:23:12.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.766+0000 7ff04f1d9b80 1 --2- 172.21.15.52:0/186165737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559be11c9890 0x559be11c1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.766+0000 7ff04f1d9b80 1 --2- 172.21.15.52:0/186165737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559be11c23d0 0x559be11e21b0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.766+0000 7ff04f1d9b80 1 --2- 172.21.15.52:0/186165737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559be11c9c60 0x559be11c1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.766+0000 7ff04f1d9b80 1 -- 172.21.15.52:0/186165737 >> 172.21.15.52:0/186165737 conn(0x559be11ba360 msgr2=0x559be11bfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:12.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.766+0000 7ff04f1d9b80 1 -- 172.21.15.52:0/186165737 shutdown_connections 2023-11-27T15:23:12.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.766+0000 7ff04f1d9b80 1 -- 172.21.15.52:0/186165737 wait complete. 2023-11-27T15:23:12.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.766+0000 7ff04f1d9b80 1 -- start start 2023-11-27T15:23:12.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.767+0000 7ff04f1d9b80 1 Processor -- start 2023-11-27T15:23:12.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.767+0000 7ff04f1d9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559be11c23d0 0x559be11d3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:12.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.767+0000 7ff04f1d9b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559be11c9890 0x559be11d37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:12.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.767+0000 7ff04f1d9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559be11c9c60 0x559be11ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:12.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.767+0000 7ff04f1d9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x559be10b78e0 con 0x559be11c9c60 2023-11-27T15:23:12.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.767+0000 7ff04f1d9b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559be10b7580 con 0x559be11c9890 2023-11-27T15:23:12.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.767+0000 7ff04f1d9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559be0f99920 con 0x559be11c23d0 2023-11-27T15:23:12.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.767+0000 7ff04f1d9b80 10 client.0 _ll_register_callbacks cb 0x559be10d5780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:12.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.767+0000 7ff0415ea700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559be11c9c60 0x559be11ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:12.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.767+0000 7ff040de9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559be11c23d0 0x559be11d3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:12.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.767+0000 7ff0415ea700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559be11c9c60 0x559be11ce540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53852/0 (socket says 172.21.15.52:53852) 2023-11-27T15:23:12.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.767+0000 7ff040de9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559be11c23d0 0x559be11d3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:40492/0 (socket says 172.21.15.52:40492) 2023-11-27T15:23:12.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.767+0000 7ff0415ea700 1 -- 172.21.15.52:0/1905060488 learned_addr learned my addr 172.21.15.52:0/1905060488 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:12.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.767+0000 7ff03bfff700 1 --2- 172.21.15.52:0/1905060488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559be11c9890 0x559be11d37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:12.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.768+0000 7ff040de9700 1 -- 172.21.15.52:0/1905060488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559be11c9890 msgr2=0x559be11d37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:12.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.768+0000 7ff040de9700 1 --2- 172.21.15.52:0/1905060488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559be11c9890 0x559be11d37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.768+0000 7ff040de9700 1 -- 172.21.15.52:0/1905060488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559be11c9c60 msgr2=0x559be11ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:12.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.768+0000 7ff040de9700 1 --2- 172.21.15.52:0/1905060488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559be11c9c60 0x559be11ce540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.768+0000 7ff040de9700 1 -- 172.21.15.52:0/1905060488 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff03c10dac0 con 0x559be11c23d0 2023-11-27T15:23:12.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.768+0000 7ff0415ea700 1 --2- 172.21.15.52:0/1905060488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559be11c9c60 0x559be11ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:12.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.768+0000 7ff03bfff700 1 --2- 172.21.15.52:0/1905060488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559be11c9890 0x559be11d37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:12.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.768+0000 7ff040de9700 1 --2- 172.21.15.52:0/1905060488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559be11c23d0 0x559be11d3270 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff030009b60 tx=0x7ff0300047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af0a07683ab31ccc server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:12.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.768+0000 7ff01bfff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2023-11-27T15:23:12.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.768+0000 7ff01bfff700 1 -- 172.21.15.52:0/1905060488 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff030022070 con 0x559be11c23d0 2023-11-27T15:23:12.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.768+0000 7ff01bfff700 1 -- 172.21.15.52:0/1905060488 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff03000e9a0 con 0x559be11c23d0 2023-11-27T15:23:12.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.768+0000 7ff04f1d9b80 1 -- 172.21.15.52:0/1905060488 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559be11b7b20 con 0x559be11c23d0 2023-11-27T15:23:12.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.768+0000 7ff01bfff700 1 -- 172.21.15.52:0/1905060488 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff03000eb60 con 0x559be11c23d0 2023-11-27T15:23:12.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.768+0000 7ff04f1d9b80 10 client.25876 fetch_fsmap learned FSMap version 28 2023-11-27T15:23:12.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.768+0000 7ff04f1d9b80 1 -- 172.21.15.52:0/1905060488 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559be10dc190 con 0x559be11c23d0 2023-11-27T15:23:12.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.769+0000 7ff01bfff700 1 -- 172.21.15.52:0/1905060488 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff030033080 con 0x559be11c23d0 2023-11-27T15:23:12.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.769+0000 7ff01bfff700 1 -- 172.21.15.52:0/1905060488 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff03002d030 con 0x559be11c23d0 2023-11-27T15:23:12.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.769+0000 7ff04f1d9b80 10 client.25876 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:23:12.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.769+0000 7ff04f1d9b80 10 client.25876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:12.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.769+0000 7ff04f1d9b80 20 client.25876 populate_metadata read hostname 'smithi052' 2023-11-27T15:23:12.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.769+0000 7ff04f1d9b80 1 --2- 172.21.15.52:0/1905060488 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x559be11de7f0 0x559be11e0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:12.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.769+0000 7ff04f1d9b80 4 client.25876 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:23:12.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.769+0000 7ff04f1d9b80 1 -- 172.21.15.52:0/1905060488 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559be10d80a0 con 0x559be11de7f0 2023-11-27T15:23:12.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.769+0000 7ff03bfff700 1 --2- 172.21.15.52:0/1905060488 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x559be11de7f0 0x559be11e0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:12.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.769+0000 7ff01bfff700 10 client.25876 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:12.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.769+0000 7ff03bfff700 1 --2- 172.21.15.52:0/1905060488 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x559be11de7f0 0x559be11e0c00 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aa9acf5e8903ae3 server_cookie=e877d71a05ba3c1b in_seq=0 out_seq=0 2023-11-27T15:23:12.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.770+0000 7ff01bfff700 1 -- 172.21.15.52:0/1905060488 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559be10d80a0 con 0x559be11de7f0 2023-11-27T15:23:12.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.770+0000 7ff01bfff700 10 client.25876 handle_command_reply: tid=0 2023-11-27T15:23:12.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.770+0000 7ff04f1d9b80 1 client.25876 shutdown 2023-11-27T15:23:12.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.770+0000 7ff04f1d9b80 1 -- 172.21.15.52:0/1905060488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559be11c23d0 msgr2=0x559be11d3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:12.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.770+0000 7ff04f1d9b80 1 --2- 172.21.15.52:0/1905060488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559be11c23d0 0x559be11d3270 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff030009b60 tx=0x7ff0300047c0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.770+0000 7ff04f1d9b80 1 -- 172.21.15.52:0/1905060488 shutdown_connections 2023-11-27T15:23:12.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.770+0000 7ff04f1d9b80 1 --2- 172.21.15.52:0/1905060488 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x559be11de7f0 0x559be11e0c00 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.770+0000 7ff04f1d9b80 1 --2- 172.21.15.52:0/1905060488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559be11c9c60 0x559be11ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.770+0000 7ff04f1d9b80 1 --2- 172.21.15.52:0/1905060488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559be11c23d0 0x559be11d3270 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.770+0000 7ff01bfff700 0 client.25876 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:12.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.770+0000 7ff04f1d9b80 1 --2- 172.21.15.52:0/1905060488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559be11c9890 0x559be11d37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.770+0000 7ff04f1d9b80 1 -- 172.21.15.52:0/1905060488 >> 172.21.15.52:0/1905060488 conn(0x559be11ba360 msgr2=0x559be11be550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:12.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.771+0000 7ff04f1d9b80 1 -- 172.21.15.52:0/1905060488 shutdown_connections 2023-11-27T15:23:12.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.771+0000 7ff04f1d9b80 1 -- 172.21.15.52:0/1905060488 wait complete. 2023-11-27T15:23:12.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.771+0000 7ff04f1d9b80 20 client.25876 trim_cache size 0 max 16384 2023-11-27T15:23:12.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.788+0000 7ff04f1d9b80 1 Processor -- start 2023-11-27T15:23:12.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.788+0000 7ff04f1d9b80 1 -- start start 2023-11-27T15:23:12.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.788+0000 7ff04f1d9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559be11c23d0 0x559be11ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:12.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.788+0000 7ff04f1d9b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559be11c9890 0x559be11e67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:12.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.789+0000 7ff04f1d9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559be11c9c60 0x559be11bf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:12.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.789+0000 7ff04f1d9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x559be11c2bd0 con 0x559be11c23d0 2023-11-27T15:23:12.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.789+0000 7ff04f1d9b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559be11cdbf0 con 0x559be11c9890 2023-11-27T15:23:12.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.789+0000 7ff04f1d9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559be11c2200 con 0x559be11c9c60 2023-11-27T15:23:12.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.789+0000 7ff040de9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559be11c23d0 0x559be11ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:12.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.789+0000 7ff040de9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559be11c23d0 0x559be11ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53856/0 (socket says 172.21.15.52:53856) 2023-11-27T15:23:12.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.789+0000 7ff040de9700 1 -- 172.21.15.52:0/698053709 learned_addr learned my addr 172.21.15.52:0/698053709 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:12.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.789+0000 7ff03bfff700 1 --2- 172.21.15.52:0/698053709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559be11c9890 0x559be11e67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:12.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.789+0000 7ff0415ea700 1 --2- 172.21.15.52:0/698053709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559be11c9c60 0x559be11bf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:12.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.789+0000 7ff040de9700 1 -- 172.21.15.52:0/698053709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559be11c9c60 msgr2=0x559be11bf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:12.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.789+0000 7ff040de9700 1 --2- 172.21.15.52:0/698053709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559be11c9c60 0x559be11bf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.789+0000 7ff040de9700 1 -- 172.21.15.52:0/698053709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559be11c9890 msgr2=0x559be11e67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:12.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.789+0000 7ff040de9700 1 --2- 172.21.15.52:0/698053709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559be11c9890 0x559be11e67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.789+0000 7ff040de9700 1 -- 172.21.15.52:0/698053709 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559be11bbb00 con 0x559be11c23d0 2023-11-27T15:23:12.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.789+0000 7ff040de9700 1 --2- 172.21.15.52:0/698053709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559be11c23d0 0x559be11ce4f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff030004790 tx=0x7ff03002f620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0355a5b7cb23cf server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:12.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.789+0000 7ff0277fe700 1 -- 172.21.15.52:0/698053709 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff030022050 con 0x559be11c23d0 2023-11-27T15:23:12.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.790+0000 7ff0277fe700 1 -- 172.21.15.52:0/698053709 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559be11b7b20 con 0x559be11c23d0 2023-11-27T15:23:12.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.790+0000 7ff0277fe700 1 -- 172.21.15.52:0/698053709 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff030027680 con 0x559be11c23d0 2023-11-27T15:23:12.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.790+0000 7ff04f1d9b80 1 -- 172.21.15.52:0/698053709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559be11c23d0 msgr2=0x559be11ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:12.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.790+0000 7ff04f1d9b80 1 --2- 172.21.15.52:0/698053709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559be11c23d0 0x559be11ce4f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff030004790 tx=0x7ff03002f620 comp rx=0 tx=0).stop 2023-11-27T15:23:12.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.790+0000 7ff04f1d9b80 1 -- 172.21.15.52:0/698053709 shutdown_connections 2023-11-27T15:23:12.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.790+0000 7ff04f1d9b80 1 --2- 172.21.15.52:0/698053709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559be11c9c60 0x559be11bf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.790+0000 7ff04f1d9b80 1 --2- 172.21.15.52:0/698053709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559be11c23d0 0x559be11ce4f0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.790+0000 7ff04f1d9b80 1 --2- 172.21.15.52:0/698053709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559be11c9890 0x559be11e67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.790+0000 7ff04f1d9b80 1 -- 172.21.15.52:0/698053709 >> 172.21.15.52:0/698053709 conn(0x559be11ba360 msgr2=0x559be11cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:12.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.790+0000 7ff04f1d9b80 1 -- 172.21.15.52:0/698053709 shutdown_connections 2023-11-27T15:23:12.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.790+0000 7ff04f1d9b80 1 -- 172.21.15.52:0/698053709 wait complete. 2023-11-27T15:23:12.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.791+0000 7ff04f1d9b80 1 -- start start 2023-11-27T15:23:12.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.791+0000 7ff04f1d9b80 1 Processor -- start 2023-11-27T15:23:12.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.791+0000 7ff04f1d9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559be11c23d0 0x559be11e31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:12.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.791+0000 7ff04f1d9b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559be11c9890 0x559be11e36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:12.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.791+0000 7ff04f1d9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559be11c9c60 0x559be1231960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:12.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.791+0000 7ff04f1d9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x559be11e3eb0 con 0x559be11c9c60 2023-11-27T15:23:12.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.791+0000 7ff04f1d9b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559be11e4020 con 0x559be11c9890 2023-11-27T15:23:12.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.791+0000 7ff04f1d9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559be1231ed0 con 0x559be11c23d0 2023-11-27T15:23:12.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.792+0000 7ff04f1d9b80 10 client.0 _ll_register_callbacks cb 0x559be10d5780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:12.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.792+0000 7ff040de9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559be11c23d0 0x559be11e31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:12.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.792+0000 7ff040de9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559be11c23d0 0x559be11e31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:40504/0 (socket says 172.21.15.52:40504) 2023-11-27T15:23:12.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.792+0000 7ff040de9700 1 -- 172.21.15.52:0/4123933879 learned_addr learned my addr 172.21.15.52:0/4123933879 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:12.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.792+0000 7ff040de9700 1 -- 172.21.15.52:0/4123933879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559be11c9890 msgr2=0x559be11e36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:12.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.792+0000 7ff040de9700 1 --2- 172.21.15.52:0/4123933879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559be11c9890 0x559be11e36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.792+0000 7ff040de9700 1 -- 172.21.15.52:0/4123933879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559be11c9c60 msgr2=0x559be1231960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:12.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.792+0000 7ff040de9700 1 --2- 172.21.15.52:0/4123933879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559be11c9c60 0x559be1231960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.792+0000 7ff040de9700 1 -- 172.21.15.52:0/4123933879 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559be10dc190 con 0x559be11c23d0 2023-11-27T15:23:12.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.792+0000 7ff040de9700 1 --2- 172.21.15.52:0/4123933879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559be11c23d0 0x559be11e31b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff03000e760 tx=0x7ff03000ef40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2d11eb2b32c024d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:12.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.792+0000 7ff01bfff700 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2023-11-27T15:23:12.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.793+0000 7ff01bfff700 1 -- 172.21.15.52:0/4123933879 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff030022040 con 0x559be11c23d0 2023-11-27T15:23:12.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.793+0000 7ff04f1d9b80 1 -- 172.21.15.52:0/4123933879 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff034003a00 con 0x559be11c23d0 2023-11-27T15:23:12.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.793+0000 7ff01bfff700 1 -- 172.21.15.52:0/4123933879 <== mon.1 v2:172.21.15.150:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7ff03002fd80 con 0x559be11c23d0 2023-11-27T15:23:12.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.793+0000 7ff01bfff700 1 -- 172.21.15.52:0/4123933879 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff03002f6a0 con 0x559be11c23d0 2023-11-27T15:23:12.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.793+0000 7ff04f1d9b80 10 client.25882 fetch_fsmap learned FSMap version 28 2023-11-27T15:23:12.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.793+0000 7ff04f1d9b80 1 -- 172.21.15.52:0/4123933879 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559be10dbee0 con 0x559be11c23d0 2023-11-27T15:23:12.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.793+0000 7ff01bfff700 1 -- 172.21.15.52:0/4123933879 <== mon.1 v2:172.21.15.150:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff034003a00 con 0x559be11c23d0 2023-11-27T15:23:12.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.793+0000 7ff01bfff700 1 -- 172.21.15.52:0/4123933879 <== mon.1 v2:172.21.15.150:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff03003b6c0 con 0x559be11c23d0 2023-11-27T15:23:12.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.793+0000 7ff04f1d9b80 10 client.25882 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:23:12.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.793+0000 7ff04f1d9b80 10 client.25882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:12.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.793+0000 7ff04f1d9b80 20 client.25882 populate_metadata read hostname 'smithi052' 2023-11-27T15:23:12.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.793+0000 7ff04f1d9b80 1 --2- 172.21.15.52:0/4123933879 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x559be1233320 0x559be1235750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:12.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.793+0000 7ff04f1d9b80 4 client.25882 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:23:12.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.793+0000 7ff04f1d9b80 1 -- 172.21.15.52:0/4123933879 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559be10e4660 con 0x559be1233320 2023-11-27T15:23:12.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.794+0000 7ff03bfff700 1 --2- 172.21.15.52:0/4123933879 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x559be1233320 0x559be1235750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:12.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.794+0000 7ff03bfff700 1 --2- 172.21.15.52:0/4123933879 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x559be1233320 0x559be1235750 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5c003a24b5ffc18 server_cookie=d94a7a466c1cdb41 in_seq=0 out_seq=0 2023-11-27T15:23:12.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.794+0000 7ff01bfff700 10 client.25882 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:12.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.795+0000 7ff01bfff700 1 -- 172.21.15.52:0/4123933879 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559be10e4660 con 0x559be1233320 2023-11-27T15:23:12.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.795+0000 7ff01bfff700 10 client.25882 handle_command_reply: tid=0 2023-11-27T15:23:12.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.795+0000 7ff04f1d9b80 1 client.25882 shutdown 2023-11-27T15:23:12.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.795+0000 7ff04f1d9b80 1 -- 172.21.15.52:0/4123933879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559be11c23d0 msgr2=0x559be11e31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:12.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.795+0000 7ff04f1d9b80 1 --2- 172.21.15.52:0/4123933879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559be11c23d0 0x559be11e31b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff03000e760 tx=0x7ff03000ef40 comp rx=0 tx=0).stop 2023-11-27T15:23:12.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.795+0000 7ff04f1d9b80 1 -- 172.21.15.52:0/4123933879 shutdown_connections 2023-11-27T15:23:12.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.795+0000 7ff04f1d9b80 1 --2- 172.21.15.52:0/4123933879 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x559be1233320 0x559be1235750 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.795+0000 7ff04f1d9b80 1 --2- 172.21.15.52:0/4123933879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559be11c9c60 0x559be1231960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.795+0000 7ff01bfff700 0 client.25882 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:12.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.795+0000 7ff04f1d9b80 1 --2- 172.21.15.52:0/4123933879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559be11c23d0 0x559be11e31b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.795+0000 7ff04f1d9b80 1 --2- 172.21.15.52:0/4123933879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559be11c9890 0x559be11e36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.795+0000 7ff04f1d9b80 1 -- 172.21.15.52:0/4123933879 >> 172.21.15.52:0/4123933879 conn(0x559be11ba360 msgr2=0x559be11cc150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:12.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.795+0000 7ff04f1d9b80 1 -- 172.21.15.52:0/4123933879 shutdown_connections 2023-11-27T15:23:12.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.795+0000 7ff04f1d9b80 1 -- 172.21.15.52:0/4123933879 wait complete. 2023-11-27T15:23:12.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.795+0000 7ff04f1d9b80 20 client.25882 trim_cache size 0 max 16384 2023-11-27T15:23:12.805 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:23:12.805 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2023-11-27T15:23:12.805 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2023-11-27T15:23:12.805 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:23:12.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.796+0000 7ff04304d700 1 -- 172.21.15.52:0/2996248140 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ff02c0a12c0 msgr2=0x7ff02c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:12.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.796+0000 7ff04304d700 1 --2- 172.21.15.52:0/2996248140 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ff02c0a12c0 0x7ff02c0a3770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff030009ca0 tx=0x7ff030009260 comp rx=0 tx=0).stop 2023-11-27T15:23:12.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.796+0000 7ff04304d700 1 -- 172.21.15.52:0/2996248140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff03c101370 msgr2=0x7ff03c073330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:12.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.796+0000 7ff04304d700 1 --2- 172.21.15.52:0/2996248140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff03c101370 0x7ff03c073330 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff02800e9f0 tx=0x7ff02800ed00 comp rx=0 tx=0).stop 2023-11-27T15:23:12.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.796+0000 7ff04304d700 1 -- 172.21.15.52:0/2996248140 shutdown_connections 2023-11-27T15:23:12.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.796+0000 7ff04304d700 1 --2- 172.21.15.52:0/2996248140 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ff02c0a12c0 0x7ff02c0a3770 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.796+0000 7ff04304d700 1 --2- 172.21.15.52:0/2996248140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff03c106410 0x7ff03c1a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.796+0000 7ff04304d700 1 --2- 172.21.15.52:0/2996248140 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff03c1009e0 0x7ff03c072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.807 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.796+0000 7ff04304d700 1 --2- 172.21.15.52:0/2996248140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff03c101370 0x7ff03c073330 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.807 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.796+0000 7ff04304d700 1 -- 172.21.15.52:0/2996248140 >> 172.21.15.52:0/2996248140 conn(0x7ff03c0775c0 msgr2=0x7ff03c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:12.807 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.796+0000 7ff04304d700 1 -- 172.21.15.52:0/2996248140 shutdown_connections 2023-11-27T15:23:12.807 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.796+0000 7ff04304d700 1 -- 172.21.15.52:0/2996248140 wait complete. 2023-11-27T15:23:12.814 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-27T15:23:12.814 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:23:12.946 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.940+0000 7f6e36999700 1 -- 172.21.15.52:0/1054363198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e300ffca0 msgr2=0x7f6e30103c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:12.946 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.940+0000 7f6e36999700 1 --2- 172.21.15.52:0/1054363198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e300ffca0 0x7f6e30103c60 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6e240099c0 tx=0x7f6e24009cd0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.946 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.941+0000 7f6e36999700 1 -- 172.21.15.52:0/1054363198 shutdown_connections 2023-11-27T15:23:12.946 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.941+0000 7f6e36999700 1 --2- 172.21.15.52:0/1054363198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e30104340 0x7f6e3010bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.946 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.941+0000 7f6e36999700 1 --2- 172.21.15.52:0/1054363198 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e300ff300 0x7f6e300ff6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.946 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.941+0000 7f6e36999700 1 --2- 172.21.15.52:0/1054363198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e300ffca0 0x7f6e30103c60 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.946 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.941+0000 7f6e36999700 1 -- 172.21.15.52:0/1054363198 >> 172.21.15.52:0/1054363198 conn(0x7f6e30076170 msgr2=0x7f6e30076570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:12.946 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.941+0000 7f6e36999700 1 -- 172.21.15.52:0/1054363198 shutdown_connections 2023-11-27T15:23:12.946 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.941+0000 7f6e36999700 1 -- 172.21.15.52:0/1054363198 wait complete. 2023-11-27T15:23:12.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.941+0000 7f6e36999700 1 Processor -- start 2023-11-27T15:23:12.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.941+0000 7f6e36999700 1 -- start start 2023-11-27T15:23:12.948 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.942+0000 7f6e36999700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e300ff300 0x7f6e30100d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:12.948 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.942+0000 7f6e36999700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e300ffca0 0x7f6e30101240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.942+0000 7f6e36999700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e30104340 0x7f6e301a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.942+0000 7f6e36999700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e30101a70 con 0x7f6e300ffca0 2023-11-27T15:23:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.942+0000 7f6e36999700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e301a4b00 con 0x7f6e300ff300 2023-11-27T15:23:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.942+0000 7f6e36999700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e301a4c40 con 0x7f6e30104340 2023-11-27T15:23:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.942+0000 7f6e2f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e300ffca0 0x7f6e30101240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.942+0000 7f6e34f36700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e30104340 0x7f6e301a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.942+0000 7f6e2f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e300ffca0 0x7f6e30101240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53876/0 (socket says 172.21.15.52:53876) 2023-11-27T15:23:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.942+0000 7f6e2f7fe700 1 -- 172.21.15.52:0/3058370988 learned_addr learned my addr 172.21.15.52:0/3058370988 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.942+0000 7f6e2ffff700 1 --2- 172.21.15.52:0/3058370988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e300ff300 0x7f6e30100d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.942+0000 7f6e2f7fe700 1 -- 172.21.15.52:0/3058370988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e30104340 msgr2=0x7f6e301a45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.942+0000 7f6e2f7fe700 1 --2- 172.21.15.52:0/3058370988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e30104340 0x7f6e301a45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.942+0000 7f6e2f7fe700 1 -- 172.21.15.52:0/3058370988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e300ff300 msgr2=0x7f6e30100d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:12.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.942+0000 7f6e2f7fe700 1 --2- 172.21.15.52:0/3058370988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e300ff300 0x7f6e30100d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.942+0000 7f6e2f7fe700 1 -- 172.21.15.52:0/3058370988 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e240095f0 con 0x7f6e300ffca0 2023-11-27T15:23:12.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.942+0000 7f6e34f36700 1 --2- 172.21.15.52:0/3058370988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e30104340 0x7f6e301a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:12.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.942+0000 7f6e2f7fe700 1 --2- 172.21.15.52:0/3058370988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e300ffca0 0x7f6e30101240 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009990 tx=0x7f6e2400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:12.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.943+0000 7f6e2d7fa700 1 -- 172.21.15.52:0/3058370988 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2401d070 con 0x7f6e300ffca0 2023-11-27T15:23:12.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.943+0000 7f6e2d7fa700 1 -- 172.21.15.52:0/3058370988 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e24005020 con 0x7f6e300ffca0 2023-11-27T15:23:12.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.943+0000 7f6e36999700 1 -- 172.21.15.52:0/3058370988 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e3010f110 con 0x7f6e300ffca0 2023-11-27T15:23:12.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.943+0000 7f6e2d7fa700 1 -- 172.21.15.52:0/3058370988 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24004a70 con 0x7f6e300ffca0 2023-11-27T15:23:12.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.943+0000 7f6e36999700 1 -- 172.21.15.52:0/3058370988 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e3010ef80 con 0x7f6e300ffca0 2023-11-27T15:23:12.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.944+0000 7f6e36999700 1 -- 172.21.15.52:0/3058370988 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e3010f2a0 con 0x7f6e300ffca0 2023-11-27T15:23:12.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.945+0000 7f6e2d7fa700 1 -- 172.21.15.52:0/3058370988 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f6e24005400 con 0x7f6e300ffca0 2023-11-27T15:23:12.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.945+0000 7f6e2d7fa700 1 --2- 172.21.15.52:0/3058370988 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f6e180a12c0 0x7f6e180a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:12.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.946+0000 7f6e2d7fa700 1 -- 172.21.15.52:0/3058370988 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f6e240cd970 con 0x7f6e300ffca0 2023-11-27T15:23:12.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.946+0000 7f6e2ffff700 1 --2- 172.21.15.52:0/3058370988 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f6e180a12c0 0x7f6e180a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:12.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.946+0000 7f6e2ffff700 1 --2- 172.21.15.52:0/3058370988 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f6e180a12c0 0x7f6e180a3770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6e20009c60 tx=0x7f6e20009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:12.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:12.949+0000 7f6e2d7fa700 1 -- 172.21.15.52:0/3058370988 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6e24096e10 con 0x7f6e300ffca0 2023-11-27T15:23:13.208 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.204+0000 7f6e36999700 1 -- 172.21.15.52:0/3058370988 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e300679f0 con 0x7f6e300ffca0 2023-11-27T15:23:13.209 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.205+0000 7f6e2d7fa700 1 -- 172.21.15.52:0/3058370988 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10813 (secure 0 0 0) 0x7f6e24005190 con 0x7f6e300ffca0 2023-11-27T15:23:13.209 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:23:13.210 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2328881376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2328881376},{"type":"v1","addr":"172.21.15.52:6839","nonce":2328881376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2799551570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2799551570},{"type":"v1","addr":"172.21.15.52:6841","nonce":2799551570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:16:09.319630+0000","modified":"2023-11-27T15:21:19.128122+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24494,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/215725838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":215725838},{"type":"v1","addr":"172.21.15.150:6839","nonce":215725838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3571830559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3571830559},{"type":"v1","addr":"172.21.15.102:6839","nonce":3571830559}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:23:13.212 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.207+0000 7f6e36999700 1 -- 172.21.15.52:0/3058370988 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f6e180a12c0 msgr2=0x7f6e180a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:13.212 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.207+0000 7f6e36999700 1 --2- 172.21.15.52:0/3058370988 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f6e180a12c0 0x7f6e180a3770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6e20009c60 tx=0x7f6e20009260 comp rx=0 tx=0).stop 2023-11-27T15:23:13.212 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.207+0000 7f6e36999700 1 -- 172.21.15.52:0/3058370988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e300ffca0 msgr2=0x7f6e30101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:13.212 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.207+0000 7f6e36999700 1 --2- 172.21.15.52:0/3058370988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e300ffca0 0x7f6e30101240 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009990 tx=0x7f6e2400f690 comp rx=0 tx=0).stop 2023-11-27T15:23:13.213 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.207+0000 7f6e36999700 1 -- 172.21.15.52:0/3058370988 shutdown_connections 2023-11-27T15:23:13.213 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.207+0000 7f6e36999700 1 --2- 172.21.15.52:0/3058370988 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f6e180a12c0 0x7f6e180a3770 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.213 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.207+0000 7f6e36999700 1 --2- 172.21.15.52:0/3058370988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e300ff300 0x7f6e30100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.213 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.207+0000 7f6e36999700 1 --2- 172.21.15.52:0/3058370988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e30104340 0x7f6e301a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.213 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.207+0000 7f6e36999700 1 --2- 172.21.15.52:0/3058370988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e300ffca0 0x7f6e30101240 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.213 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.207+0000 7f6e36999700 1 -- 172.21.15.52:0/3058370988 >> 172.21.15.52:0/3058370988 conn(0x7f6e30076170 msgr2=0x7f6e30067ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:13.213 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.209+0000 7f6e36999700 1 -- 172.21.15.52:0/3058370988 shutdown_connections 2023-11-27T15:23:13.213 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.209+0000 7f6e36999700 1 -- 172.21.15.52:0/3058370988 wait complete. 2023-11-27T15:23:13.214 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 28 2023-11-27T15:23:13.226 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-27T15:23:13.310 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_chrisphoffman_ceph_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_chrisphoffman_ceph_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_chrisphoffman_ceph_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/misc/general_vxattrs.sh) on smithi052 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh' 2023-11-27T15:23:13.405 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=8b59c8712d754e1481fcddae110def33 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_chrisphoffman_ceph_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_chrisphoffman_ceph_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_chrisphoffman_ceph_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/misc/general_vxattrs.sh) on smithi052 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh' 2023-11-27T15:23:13.408 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2023-11-27T15:23:13.417 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2023-11-27T15:23:13.417 INFO:teuthology.task.exec:Executing custom commands... 2023-11-27T15:23:13.417 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi052.front.sepia.ceph.com 2023-11-27T15:23:13.418 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2023-11-27T15:23:13.440 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.366+0000 7f710d3ca700 1 -- 172.21.15.52:0/4094771319 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7108105600 msgr2=0x7f710810d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:13.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.366+0000 7f710d3ca700 1 --2- 172.21.15.52:0/4094771319 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7108105600 0x7f710810d1d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f710000b0d0 tx=0x7f710000b3e0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.366+0000 7f710d3ca700 1 -- 172.21.15.52:0/4094771319 shutdown_connections 2023-11-27T15:23:13.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.366+0000 7f710d3ca700 1 --2- 172.21.15.52:0/4094771319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71080690e0 0x7f71080694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.366+0000 7f710d3ca700 1 --2- 172.21.15.52:0/4094771319 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7108105600 0x7f710810d1d0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.366+0000 7f710d3ca700 1 --2- 172.21.15.52:0/4094771319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7108069a80 0x7f7108104f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.366+0000 7f710d3ca700 1 -- 172.21.15.52:0/4094771319 >> 172.21.15.52:0/4094771319 conn(0x7f7108075470 msgr2=0x7f7108075870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:13.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.366+0000 7f710d3ca700 1 -- 172.21.15.52:0/4094771319 shutdown_connections 2023-11-27T15:23:13.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.366+0000 7f710d3ca700 1 -- 172.21.15.52:0/4094771319 wait complete. 2023-11-27T15:23:13.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.367+0000 7f710d3ca700 1 Processor -- start 2023-11-27T15:23:13.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.367+0000 7f710d3ca700 1 -- start start 2023-11-27T15:23:13.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.367+0000 7f710d3ca700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71080690e0 0x7f71081964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.367+0000 7f710d3ca700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7108069a80 0x7f7108196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.367+0000 7f710d3ca700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7108105600 0x7f710819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.367+0000 7f710d3ca700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710819b4a0 con 0x7f71080690e0 2023-11-27T15:23:13.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.367+0000 7f710d3ca700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710819b610 con 0x7f7108069a80 2023-11-27T15:23:13.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.367+0000 7f710d3ca700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710819b7d0 con 0x7f7108105600 2023-11-27T15:23:13.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.367+0000 7f7106ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71080690e0 0x7f71081964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.367+0000 7f7106ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71080690e0 0x7f71081964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53888/0 (socket says 172.21.15.52:53888) 2023-11-27T15:23:13.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.367+0000 7f7106ffd700 1 -- 172.21.15.52:0/3786300593 learned_addr learned my addr 172.21.15.52:0/3786300593 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:13.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.367+0000 7f71067fc700 1 --2- 172.21.15.52:0/3786300593 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7108069a80 0x7f7108196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.367+0000 7f71077fe700 1 --2- 172.21.15.52:0/3786300593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7108105600 0x7f710819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.368+0000 7f7106ffd700 1 -- 172.21.15.52:0/3786300593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7108105600 msgr2=0x7f710819ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:13.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.368+0000 7f7106ffd700 1 --2- 172.21.15.52:0/3786300593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7108105600 0x7f710819ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.368+0000 7f7106ffd700 1 -- 172.21.15.52:0/3786300593 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7108069a80 msgr2=0x7f7108196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:13.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.368+0000 7f7106ffd700 1 --2- 172.21.15.52:0/3786300593 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7108069a80 0x7f7108196a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.368+0000 7f7106ffd700 1 -- 172.21.15.52:0/3786300593 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7100009d00 con 0x7f71080690e0 2023-11-27T15:23:13.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.368+0000 7f71067fc700 1 --2- 172.21.15.52:0/3786300593 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7108069a80 0x7f7108196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:13.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.368+0000 7f7106ffd700 1 --2- 172.21.15.52:0/3786300593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71080690e0 0x7f71081964c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f70f800e9a0 tx=0x7f70f800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:13.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.368+0000 7f70effff700 1 -- 172.21.15.52:0/3786300593 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f8009790 con 0x7f71080690e0 2023-11-27T15:23:13.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.368+0000 7f710d3ca700 1 -- 172.21.15.52:0/3786300593 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71081103e0 con 0x7f71080690e0 2023-11-27T15:23:13.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.368+0000 7f70effff700 1 -- 172.21.15.52:0/3786300593 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70f8010430 con 0x7f71080690e0 2023-11-27T15:23:13.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.368+0000 7f70effff700 1 -- 172.21.15.52:0/3786300593 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f8018680 con 0x7f71080690e0 2023-11-27T15:23:13.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.368+0000 7f710d3ca700 1 -- 172.21.15.52:0/3786300593 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7108110250 con 0x7f71080690e0 2023-11-27T15:23:13.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.371+0000 7f70effff700 1 -- 172.21.15.52:0/3786300593 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f70f80105a0 con 0x7f71080690e0 2023-11-27T15:23:13.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.372+0000 7f70effff700 1 --2- 172.21.15.52:0/3786300593 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f70f00a12c0 0x7f70f00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.372+0000 7f71067fc700 1 --2- 172.21.15.52:0/3786300593 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f70f00a12c0 0x7f70f00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.372+0000 7f70effff700 1 -- 172.21.15.52:0/3786300593 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f70f8014070 con 0x7f71080690e0 2023-11-27T15:23:13.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.372+0000 7f7119556b80 1 Processor -- start 2023-11-27T15:23:13.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.372+0000 7f7119556b80 1 -- start start 2023-11-27T15:23:13.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.372+0000 7f7119556b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c9fcb60 0x55a42c9fcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.372+0000 7f71067fc700 1 --2- 172.21.15.52:0/3786300593 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f70f00a12c0 0x7f70f00a3770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7108197ad0 tx=0x7f70fc009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:13.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.372+0000 7f7119556b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a42c9fd470 0x55a42ca01370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.372+0000 7f7119556b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42ca018b0 0x55a42ca01d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.372+0000 7f7119556b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42c902060 con 0x55a42c9fd470 2023-11-27T15:23:13.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.372+0000 7f7119556b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42c9548a0 con 0x55a42ca018b0 2023-11-27T15:23:13.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.372+0000 7f7119556b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42c94dd30 con 0x55a42c9fcb60 2023-11-27T15:23:13.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.373+0000 7f71067fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a42c9fd470 0x55a42ca01370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.373+0000 7f71077fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42ca018b0 0x55a42ca01d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.373+0000 7f71067fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a42c9fd470 0x55a42ca01370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53896/0 (socket says 172.21.15.52:53896) 2023-11-27T15:23:13.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.373+0000 7f71067fc700 1 -- 172.21.15.52:0/2941968980 learned_addr learned my addr 172.21.15.52:0/2941968980 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:13.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.373+0000 7f7106ffd700 1 --2- 172.21.15.52:0/2941968980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c9fcb60 0x55a42c9fcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.373+0000 7f71067fc700 1 -- 172.21.15.52:0/2941968980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c9fcb60 msgr2=0x55a42c9fcf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:13.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.373+0000 7f71067fc700 1 --2- 172.21.15.52:0/2941968980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c9fcb60 0x55a42c9fcf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.373+0000 7f71067fc700 1 -- 172.21.15.52:0/2941968980 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42ca018b0 msgr2=0x55a42ca01d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:13.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.373+0000 7f71067fc700 1 --2- 172.21.15.52:0/2941968980 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42ca018b0 0x55a42ca01d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.373+0000 7f71067fc700 1 -- 172.21.15.52:0/2941968980 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a42c7cf630 con 0x55a42c9fd470 2023-11-27T15:23:13.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.373+0000 7f71077fe700 1 --2- 172.21.15.52:0/2941968980 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42ca018b0 0x55a42ca01d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:13.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.373+0000 7f71067fc700 1 --2- 172.21.15.52:0/2941968980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a42c9fd470 0x55a42ca01370 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00e6d0 tx=0x7f70fc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d701c1f40e8c46 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:13.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.373+0000 7f70ed7fa700 1 -- 172.21.15.52:0/2941968980 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc004220 con 0x55a42c9fd470 2023-11-27T15:23:13.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.373+0000 7f70ed7fa700 1 -- 172.21.15.52:0/2941968980 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70fc00ee70 con 0x55a42c9fd470 2023-11-27T15:23:13.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.373+0000 7f70ed7fa700 1 -- 172.21.15.52:0/2941968980 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc01e6b0 con 0x55a42c9fd470 2023-11-27T15:23:13.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.374+0000 7f7119556b80 1 -- 172.21.15.52:0/2941968980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a42c9fd470 msgr2=0x55a42ca01370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:13.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.374+0000 7f7119556b80 1 --2- 172.21.15.52:0/2941968980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a42c9fd470 0x55a42ca01370 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00e6d0 tx=0x7f70fc014000 comp rx=0 tx=0).stop 2023-11-27T15:23:13.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.374+0000 7f7119556b80 1 -- 172.21.15.52:0/2941968980 shutdown_connections 2023-11-27T15:23:13.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.374+0000 7f7119556b80 1 --2- 172.21.15.52:0/2941968980 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42ca018b0 0x55a42ca01d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.374+0000 7f7119556b80 1 --2- 172.21.15.52:0/2941968980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c9fcb60 0x55a42c9fcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.374+0000 7f7119556b80 1 --2- 172.21.15.52:0/2941968980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a42c9fd470 0x55a42ca01370 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.374+0000 7f7119556b80 1 -- 172.21.15.52:0/2941968980 >> 172.21.15.52:0/2941968980 conn(0x55a42c9f2360 msgr2=0x55a42c9f7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:13.450 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.374+0000 7f7119556b80 1 -- 172.21.15.52:0/2941968980 shutdown_connections 2023-11-27T15:23:13.450 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.374+0000 7f7119556b80 1 -- 172.21.15.52:0/2941968980 wait complete. 2023-11-27T15:23:13.450 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.374+0000 7f7119556b80 1 -- start start 2023-11-27T15:23:13.450 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.375+0000 7f7119556b80 1 Processor -- start 2023-11-27T15:23:13.450 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.375+0000 7f7119556b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42c9fcb60 0x55a42ca09060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.450 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.375+0000 7f7119556b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c9fd470 0x55a42ca062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.450 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.375+0000 7f7119556b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a42ca018b0 0x55a42ca06980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.375+0000 7f7119556b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42c8ef8e0 con 0x55a42ca018b0 2023-11-27T15:23:13.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.375+0000 7f7119556b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42c8ef580 con 0x55a42c9fcb60 2023-11-27T15:23:13.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.375+0000 7f7119556b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42c7d1920 con 0x55a42c9fd470 2023-11-27T15:23:13.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.375+0000 7f7119556b80 10 client.0 _ll_register_callbacks cb 0x55a42c90d780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:13.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.375+0000 7f71077fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a42ca018b0 0x55a42ca06980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.375+0000 7f71067fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c9fd470 0x55a42ca062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.375+0000 7f71077fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a42ca018b0 0x55a42ca06980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53906/0 (socket says 172.21.15.52:53906) 2023-11-27T15:23:13.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.375+0000 7f71067fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c9fd470 0x55a42ca062e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:40548/0 (socket says 172.21.15.52:40548) 2023-11-27T15:23:13.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.375+0000 7f71077fe700 1 -- 172.21.15.52:0/3718273476 learned_addr learned my addr 172.21.15.52:0/3718273476 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:13.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.375+0000 7f7106ffd700 1 --2- 172.21.15.52:0/3718273476 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42c9fcb60 0x55a42ca09060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.375+0000 7f71077fe700 1 -- 172.21.15.52:0/3718273476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c9fd470 msgr2=0x55a42ca062e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:13.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.375+0000 7f71077fe700 1 --2- 172.21.15.52:0/3718273476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c9fd470 0x55a42ca062e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.375+0000 7f71077fe700 1 -- 172.21.15.52:0/3718273476 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42c9fcb60 msgr2=0x55a42ca09060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:13.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.375+0000 7f71077fe700 1 --2- 172.21.15.52:0/3718273476 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42c9fcb60 0x55a42ca09060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.375+0000 7f71077fe700 1 -- 172.21.15.52:0/3718273476 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70fc017070 con 0x55a42ca018b0 2023-11-27T15:23:13.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.375+0000 7f7106ffd700 1 --2- 172.21.15.52:0/3718273476 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42c9fcb60 0x55a42ca09060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:13.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.375+0000 7f71067fc700 1 --2- 172.21.15.52:0/3718273476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c9fd470 0x55a42ca062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:13.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.376+0000 7f71077fe700 1 --2- 172.21.15.52:0/3718273476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a42ca018b0 0x55a42ca06980 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f7100008fb0 tx=0x7f71000060e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=105b4ac971a90d2d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:13.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.376+0000 7f70eaff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:23:13.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.376+0000 7f70eaff5700 1 -- 172.21.15.52:0/3718273476 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710000e040 con 0x55a42ca018b0 2023-11-27T15:23:13.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.376+0000 7f70eaff5700 1 -- 172.21.15.52:0/3718273476 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f71000076e0 con 0x55a42ca018b0 2023-11-27T15:23:13.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.376+0000 7f7119556b80 1 -- 172.21.15.52:0/3718273476 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a42c9efb20 con 0x55a42ca018b0 2023-11-27T15:23:13.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.376+0000 7f70eaff5700 1 -- 172.21.15.52:0/3718273476 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71000078c0 con 0x55a42ca018b0 2023-11-27T15:23:13.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.376+0000 7f7119556b80 10 client.16191 fetch_fsmap learned FSMap version 28 2023-11-27T15:23:13.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.376+0000 7f7119556b80 1 -- 172.21.15.52:0/3718273476 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a42c914190 con 0x55a42ca018b0 2023-11-27T15:23:13.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.376+0000 7f70eaff5700 1 -- 172.21.15.52:0/3718273476 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a42c9efb20 con 0x55a42ca018b0 2023-11-27T15:23:13.454 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.376+0000 7f70eaff5700 1 -- 172.21.15.52:0/3718273476 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f710000b100 con 0x55a42ca018b0 2023-11-27T15:23:13.454 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.376+0000 7f7119556b80 10 client.16191 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:23:13.454 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.376+0000 7f7119556b80 10 client.16191 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:23:13.454 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.376+0000 7f7119556b80 20 client.16191 populate_metadata read hostname 'smithi052' 2023-11-27T15:23:13.454 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.376+0000 7f7119556b80 1 --2- 172.21.15.52:0/3718273476 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55a42ca16960 0x55a42ca18d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.454 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.376+0000 7f7119556b80 4 client.16191 mds_command: new command op to 24472 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:23:13.454 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.376+0000 7f7119556b80 1 -- 172.21.15.52:0/3718273476 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a42c9100a0 con 0x55a42ca16960 2023-11-27T15:23:13.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.377+0000 7f7106ffd700 1 --2- 172.21.15.52:0/3718273476 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55a42ca16960 0x55a42ca18d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.377+0000 7f70eaff5700 10 client.16191 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:23:13.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.377+0000 7f7106ffd700 1 --2- 172.21.15.52:0/3718273476 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55a42ca16960 0x55a42ca18d70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a81ed9c6625e4085 server_cookie=9a495cd223b6f627 in_seq=0 out_seq=0 2023-11-27T15:23:13.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.378+0000 7f70eaff5700 1 -- 172.21.15.52:0/3718273476 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a42c9100a0 con 0x55a42ca16960 2023-11-27T15:23:13.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.378+0000 7f70eaff5700 10 client.16191 handle_command_reply: tid=0 2023-11-27T15:23:13.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.378+0000 7f7119556b80 1 client.16191 shutdown 2023-11-27T15:23:13.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.378+0000 7f7119556b80 1 -- 172.21.15.52:0/3718273476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a42ca018b0 msgr2=0x55a42ca06980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:13.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.378+0000 7f7119556b80 1 --2- 172.21.15.52:0/3718273476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a42ca018b0 0x55a42ca06980 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f7100008fb0 tx=0x7f71000060e0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.378+0000 7f7119556b80 1 -- 172.21.15.52:0/3718273476 shutdown_connections 2023-11-27T15:23:13.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.378+0000 7f7119556b80 1 --2- 172.21.15.52:0/3718273476 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55a42ca16960 0x55a42ca18d70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.378+0000 7f7119556b80 1 --2- 172.21.15.52:0/3718273476 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42c9fcb60 0x55a42ca09060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.378+0000 7f7119556b80 1 --2- 172.21.15.52:0/3718273476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a42ca018b0 0x55a42ca06980 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.378+0000 7f7119556b80 1 --2- 172.21.15.52:0/3718273476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42c9fd470 0x55a42ca062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.378+0000 7f70eaff5700 0 client.16191 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:23:13.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.378+0000 7f7119556b80 1 -- 172.21.15.52:0/3718273476 >> 172.21.15.52:0/3718273476 conn(0x55a42c9f2360 msgr2=0x55a42ca02d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:13.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.378+0000 7f7119556b80 1 -- 172.21.15.52:0/3718273476 shutdown_connections 2023-11-27T15:23:13.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.378+0000 7f7119556b80 1 -- 172.21.15.52:0/3718273476 wait complete. 2023-11-27T15:23:13.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.378+0000 7f7119556b80 20 client.16191 trim_cache size 0 max 16384 2023-11-27T15:23:13.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.391+0000 7f7119556b80 1 Processor -- start 2023-11-27T15:23:13.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.391+0000 7f7119556b80 1 -- start start 2023-11-27T15:23:13.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.391+0000 7f7119556b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a42c9fcb60 0x55a42ca12c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.391+0000 7f7119556b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42c9fd470 0x55a42c9f6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.391+0000 7f7119556b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42ca018b0 0x55a42ca08c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.391+0000 7f7119556b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42ca00230 con 0x55a42c9fcb60 2023-11-27T15:23:13.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.391+0000 7f7119556b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42c9fdbc0 con 0x55a42c9fd470 2023-11-27T15:23:13.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.391+0000 7f7119556b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42ca078c0 con 0x55a42ca018b0 2023-11-27T15:23:13.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.391+0000 7f7106ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a42c9fcb60 0x55a42ca12c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.391+0000 7f7106ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a42c9fcb60 0x55a42ca12c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53910/0 (socket says 172.21.15.52:53910) 2023-11-27T15:23:13.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.391+0000 7f7106ffd700 1 -- 172.21.15.52:0/1139385261 learned_addr learned my addr 172.21.15.52:0/1139385261 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:13.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.391+0000 7f71067fc700 1 --2- 172.21.15.52:0/1139385261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42c9fd470 0x55a42c9f6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.392+0000 7f71077fe700 1 --2- 172.21.15.52:0/1139385261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42ca018b0 0x55a42ca08c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.392+0000 7f7106ffd700 1 -- 172.21.15.52:0/1139385261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42ca018b0 msgr2=0x55a42ca08c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:13.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.392+0000 7f7106ffd700 1 --2- 172.21.15.52:0/1139385261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42ca018b0 0x55a42ca08c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.392+0000 7f7106ffd700 1 -- 172.21.15.52:0/1139385261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42c9fd470 msgr2=0x55a42c9f6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:13.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.392+0000 7f7106ffd700 1 --2- 172.21.15.52:0/1139385261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42c9fd470 0x55a42c9f6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.392+0000 7f7106ffd700 1 -- 172.21.15.52:0/1139385261 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a42ca01f50 con 0x55a42c9fcb60 2023-11-27T15:23:13.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.392+0000 7f71067fc700 1 --2- 172.21.15.52:0/1139385261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42c9fd470 0x55a42c9f6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:13.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.392+0000 7f7106ffd700 1 --2- 172.21.15.52:0/1139385261 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a42c9fcb60 0x55a42ca12c40 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f70f8096a20 tx=0x7f70f8010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f57a5f0e94b31 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:13.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.392+0000 7f70edffb700 1 -- 172.21.15.52:0/1139385261 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f80a2400 con 0x55a42c9fcb60 2023-11-27T15:23:13.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.392+0000 7f70edffb700 1 -- 172.21.15.52:0/1139385261 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70f8098350 con 0x55a42c9fcb60 2023-11-27T15:23:13.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.392+0000 7f70edffb700 1 -- 172.21.15.52:0/1139385261 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f8099700 con 0x55a42c9fcb60 2023-11-27T15:23:13.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.393+0000 7f7119556b80 1 -- 172.21.15.52:0/1139385261 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a42c9fcb60 msgr2=0x55a42ca12c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:13.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.393+0000 7f7119556b80 1 --2- 172.21.15.52:0/1139385261 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a42c9fcb60 0x55a42ca12c40 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f70f8096a20 tx=0x7f70f8010c80 comp rx=0 tx=0).stop 2023-11-27T15:23:13.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.393+0000 7f7119556b80 1 -- 172.21.15.52:0/1139385261 shutdown_connections 2023-11-27T15:23:13.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.393+0000 7f7119556b80 1 --2- 172.21.15.52:0/1139385261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42ca018b0 0x55a42ca08c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.393+0000 7f7119556b80 1 --2- 172.21.15.52:0/1139385261 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a42c9fcb60 0x55a42ca12c40 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.393+0000 7f7119556b80 1 --2- 172.21.15.52:0/1139385261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42c9fd470 0x55a42c9f6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.393+0000 7f7119556b80 1 -- 172.21.15.52:0/1139385261 >> 172.21.15.52:0/1139385261 conn(0x55a42c9f2360 msgr2=0x55a42c9f6550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:13.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.393+0000 7f7119556b80 1 -- 172.21.15.52:0/1139385261 shutdown_connections 2023-11-27T15:23:13.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.393+0000 7f7119556b80 1 -- 172.21.15.52:0/1139385261 wait complete. 2023-11-27T15:23:13.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.393+0000 7f7119556b80 1 -- start start 2023-11-27T15:23:13.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.393+0000 7f7119556b80 1 Processor -- start 2023-11-27T15:23:13.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.393+0000 7f7119556b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a42c9fcb60 0x55a42ca0b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.393+0000 7f7119556b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42c9fd470 0x55a42ca0bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.393+0000 7f7119556b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42ca018b0 0x55a42ca0c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.393+0000 7f7119556b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42ca69820 con 0x55a42c9fcb60 2023-11-27T15:23:13.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.393+0000 7f7119556b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42ca69990 con 0x55a42c9fd470 2023-11-27T15:23:13.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.393+0000 7f7119556b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42ca69b50 con 0x55a42ca018b0 2023-11-27T15:23:13.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.393+0000 7f7119556b80 10 client.0 _ll_register_callbacks cb 0x7f7100005a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:13.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.393+0000 7f71067fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42c9fd470 0x55a42ca0bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.393+0000 7f7106ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a42c9fcb60 0x55a42ca0b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.394+0000 7f71067fc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42c9fd470 0x55a42ca0bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43576/0 (socket says 172.21.15.52:43576) 2023-11-27T15:23:13.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.394+0000 7f71067fc700 1 -- 172.21.15.52:0/4081905924 learned_addr learned my addr 172.21.15.52:0/4081905924 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:13.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.394+0000 7f7106ffd700 1 --2- 172.21.15.52:0/4081905924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a42c9fcb60 0x55a42ca0b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53916/0 (socket says 172.21.15.52:53916) 2023-11-27T15:23:13.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.394+0000 7f71077fe700 1 --2- 172.21.15.52:0/4081905924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42ca018b0 0x55a42ca0c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.394+0000 7f71067fc700 1 -- 172.21.15.52:0/4081905924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42ca018b0 msgr2=0x55a42ca0c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:13.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.394+0000 7f71067fc700 1 --2- 172.21.15.52:0/4081905924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42ca018b0 0x55a42ca0c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.394+0000 7f71067fc700 1 -- 172.21.15.52:0/4081905924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a42c9fcb60 msgr2=0x55a42ca0b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:13.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.394+0000 7f71067fc700 1 --2- 172.21.15.52:0/4081905924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a42c9fcb60 0x55a42ca0b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.394+0000 7f71067fc700 1 -- 172.21.15.52:0/4081905924 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a42c7cf630 con 0x55a42c9fd470 2023-11-27T15:23:13.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.394+0000 7f7106ffd700 1 --2- 172.21.15.52:0/4081905924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a42c9fcb60 0x55a42ca0b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:13.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.394+0000 7f71067fc700 1 --2- 172.21.15.52:0/4081905924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42c9fd470 0x55a42ca0bed0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0097c0 tx=0x7f70fc027a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86bfca6298033dbc server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:13.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.394+0000 7f70eaff5700 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2023-11-27T15:23:13.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.394+0000 7f70eaff5700 1 -- 172.21.15.52:0/4081905924 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc02c040 con 0x55a42c9fd470 2023-11-27T15:23:13.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.394+0000 7f70eaff5700 1 -- 172.21.15.52:0/4081905924 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f70fc027e40 con 0x55a42c9fd470 2023-11-27T15:23:13.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.394+0000 7f7119556b80 1 -- 172.21.15.52:0/4081905924 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f70fc00ee70 con 0x55a42c9fd470 2023-11-27T15:23:13.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.394+0000 7f70eaff5700 1 -- 172.21.15.52:0/4081905924 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70fc003b90 con 0x55a42c9fd470 2023-11-27T15:23:13.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.394+0000 7f7119556b80 10 client.26018 fetch_fsmap learned FSMap version 28 2023-11-27T15:23:13.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.394+0000 7f7119556b80 1 -- 172.21.15.52:0/4081905924 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f7108110250 con 0x55a42c9fd470 2023-11-27T15:23:13.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.395+0000 7f70eaff5700 1 -- 172.21.15.52:0/4081905924 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70fc03f020 con 0x55a42c9fd470 2023-11-27T15:23:13.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.395+0000 7f70eaff5700 1 -- 172.21.15.52:0/4081905924 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f70fc024030 con 0x55a42c9fd470 2023-11-27T15:23:13.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.395+0000 7f7119556b80 10 client.26018 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:23:13.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.395+0000 7f7119556b80 10 client.26018 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:23:13.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.395+0000 7f7119556b80 20 client.26018 populate_metadata read hostname 'smithi052' 2023-11-27T15:23:13.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.395+0000 7f7119556b80 1 --2- 172.21.15.52:0/4081905924 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55a42ca6b200 0x55a42ca6d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.395+0000 7f7119556b80 4 client.26018 mds_command: new command op to 24472 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:23:13.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.395+0000 7f7119556b80 1 -- 172.21.15.52:0/4081905924 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a42c91c660 con 0x55a42ca6b200 2023-11-27T15:23:13.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.395+0000 7f7106ffd700 1 --2- 172.21.15.52:0/4081905924 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55a42ca6b200 0x55a42ca6d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.395+0000 7f70eaff5700 10 client.26018 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:23:13.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.395+0000 7f7106ffd700 1 --2- 172.21.15.52:0/4081905924 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55a42ca6b200 0x55a42ca6d610 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=97b2dba5e01bd722 server_cookie=e41de43b3c71e62e in_seq=0 out_seq=0 2023-11-27T15:23:13.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.396+0000 7f70eaff5700 1 -- 172.21.15.52:0/4081905924 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a42c91c660 con 0x55a42ca6b200 2023-11-27T15:23:13.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.396+0000 7f70eaff5700 10 client.26018 handle_command_reply: tid=0 2023-11-27T15:23:13.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.396+0000 7f7119556b80 1 client.26018 shutdown 2023-11-27T15:23:13.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.396+0000 7f7119556b80 1 -- 172.21.15.52:0/4081905924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42c9fd470 msgr2=0x55a42ca0bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:13.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.396+0000 7f7119556b80 1 --2- 172.21.15.52:0/4081905924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42c9fd470 0x55a42ca0bed0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0097c0 tx=0x7f70fc027a00 comp rx=0 tx=0).stop 2023-11-27T15:23:13.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.396+0000 7f7119556b80 1 -- 172.21.15.52:0/4081905924 shutdown_connections 2023-11-27T15:23:13.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.396+0000 7f7119556b80 1 --2- 172.21.15.52:0/4081905924 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55a42ca6b200 0x55a42ca6d610 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.396+0000 7f7119556b80 1 --2- 172.21.15.52:0/4081905924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a42ca018b0 0x55a42ca0c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.396+0000 7f70eaff5700 0 client.26018 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:23:13.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.396+0000 7f7119556b80 1 --2- 172.21.15.52:0/4081905924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a42c9fcb60 0x55a42ca0b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.396+0000 7f7119556b80 1 --2- 172.21.15.52:0/4081905924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42c9fd470 0x55a42ca0bed0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.396+0000 7f7119556b80 1 -- 172.21.15.52:0/4081905924 >> 172.21.15.52:0/4081905924 conn(0x55a42c9f2360 msgr2=0x55a42ca12c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:13.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.396+0000 7f7119556b80 1 -- 172.21.15.52:0/4081905924 shutdown_connections 2023-11-27T15:23:13.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.396+0000 7f7119556b80 1 -- 172.21.15.52:0/4081905924 wait complete. 2023-11-27T15:23:13.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.396+0000 7f7119556b80 20 client.26018 trim_cache size 0 max 16384 2023-11-27T15:23:13.468 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:23:13.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.396+0000 7f710d3ca700 1 -- 172.21.15.52:0/3786300593 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f70f00a12c0 msgr2=0x7f70f00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:13.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.396+0000 7f710d3ca700 1 --2- 172.21.15.52:0/3786300593 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f70f00a12c0 0x7f70f00a3770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7108197ad0 tx=0x7f70fc009260 comp rx=0 tx=0).stop 2023-11-27T15:23:13.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.396+0000 7f710d3ca700 1 -- 172.21.15.52:0/3786300593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71080690e0 msgr2=0x7f71081964c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:13.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.396+0000 7f710d3ca700 1 --2- 172.21.15.52:0/3786300593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71080690e0 0x7f71081964c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f70f800e9a0 tx=0x7f70f800ecb0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.397+0000 7f710d3ca700 1 -- 172.21.15.52:0/3786300593 shutdown_connections 2023-11-27T15:23:13.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.397+0000 7f710d3ca700 1 --2- 172.21.15.52:0/3786300593 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f70f00a12c0 0x7f70f00a3770 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.397+0000 7f710d3ca700 1 --2- 172.21.15.52:0/3786300593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7108105600 0x7f710819ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.397+0000 7f710d3ca700 1 --2- 172.21.15.52:0/3786300593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71080690e0 0x7f71081964c0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.397+0000 7f710d3ca700 1 --2- 172.21.15.52:0/3786300593 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7108069a80 0x7f7108196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.397+0000 7f710d3ca700 1 -- 172.21.15.52:0/3786300593 >> 172.21.15.52:0/3786300593 conn(0x7f7108075470 msgr2=0x7f71080fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:13.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.397+0000 7f710d3ca700 1 -- 172.21.15.52:0/3786300593 shutdown_connections 2023-11-27T15:23:13.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.397+0000 7f710d3ca700 1 -- 172.21.15.52:0/3786300593 wait complete. 2023-11-27T15:23:13.471 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-27T15:23:13.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:13 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:23:13.535 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:13 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:23:13.572 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2023-11-27T15:23:13.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:13 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:23:13.644 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.639+0000 7f8b0fd83700 1 -- 172.21.15.52:0/3132292980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b080fed00 msgr2=0x7f8b080ff0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:13.644 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.639+0000 7f8b0fd83700 1 --2- 172.21.15.52:0/3132292980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b080fed00 0x7f8b080ff0d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8af80099c0 tx=0x7f8af8009cd0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.644 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.639+0000 7f8b0fd83700 1 -- 172.21.15.52:0/3132292980 shutdown_connections 2023-11-27T15:23:13.644 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.639+0000 7f8b0fd83700 1 --2- 172.21.15.52:0/3132292980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b08105620 0x7f8b0810d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.644 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.639+0000 7f8b0fd83700 1 --2- 172.21.15.52:0/3132292980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b080fed00 0x7f8b080ff0d0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.644 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.639+0000 7f8b0fd83700 1 --2- 172.21.15.52:0/3132292980 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b080ff6a0 0x7f8b08104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.644 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.639+0000 7f8b0fd83700 1 -- 172.21.15.52:0/3132292980 >> 172.21.15.52:0/3132292980 conn(0x7f8b080748f0 msgr2=0x7f8b08074cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:13.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.639+0000 7f8b0fd83700 1 -- 172.21.15.52:0/3132292980 shutdown_connections 2023-11-27T15:23:13.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.639+0000 7f8b0fd83700 1 -- 172.21.15.52:0/3132292980 wait complete. 2023-11-27T15:23:13.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.639+0000 7f8b0fd83700 1 Processor -- start 2023-11-27T15:23:13.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.639+0000 7f8b0fd83700 1 -- start start 2023-11-27T15:23:13.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.640+0000 7f8b0fd83700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b080fed00 0x7f8b08196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.640+0000 7f8b0fd83700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b080ff6a0 0x7f8b081969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.640+0000 7f8b0fd83700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b08105620 0x7f8b0819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.640+0000 7f8b0fd83700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0819b460 con 0x7f8b080fed00 2023-11-27T15:23:13.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.640+0000 7f8b0fd83700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0819b5d0 con 0x7f8b08105620 2023-11-27T15:23:13.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.640+0000 7f8b0fd83700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0819b790 con 0x7f8b080ff6a0 2023-11-27T15:23:13.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.640+0000 7f8b0db1f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b080fed00 0x7f8b08196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.640+0000 7f8b0db1f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b080fed00 0x7f8b08196480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53930/0 (socket says 172.21.15.52:53930) 2023-11-27T15:23:13.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.640+0000 7f8b0db1f700 1 -- 172.21.15.52:0/2780726983 learned_addr learned my addr 172.21.15.52:0/2780726983 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:13.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.640+0000 7f8b0d31e700 1 --2- 172.21.15.52:0/2780726983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b080ff6a0 0x7f8b081969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.640+0000 7f8b0e320700 1 --2- 172.21.15.52:0/2780726983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b08105620 0x7f8b0819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.640+0000 7f8b0db1f700 1 -- 172.21.15.52:0/2780726983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b080ff6a0 msgr2=0x7f8b081969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:13.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.640+0000 7f8b0db1f700 1 --2- 172.21.15.52:0/2780726983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b080ff6a0 0x7f8b081969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.640+0000 7f8b0db1f700 1 -- 172.21.15.52:0/2780726983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b08105620 msgr2=0x7f8b0819ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:13.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.640+0000 7f8b0db1f700 1 --2- 172.21.15.52:0/2780726983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b08105620 0x7f8b0819ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.640+0000 7f8b0db1f700 1 -- 172.21.15.52:0/2780726983 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af80095f0 con 0x7f8b080fed00 2023-11-27T15:23:13.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.640+0000 7f8b0db1f700 1 --2- 172.21.15.52:0/2780726983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b080fed00 0x7f8b08196480 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8af800b580 tx=0x7f8af800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:13.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.640+0000 7f8afeffd700 1 -- 172.21.15.52:0/2780726983 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af801d070 con 0x7f8b080fed00 2023-11-27T15:23:13.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.640+0000 7f8afeffd700 1 -- 172.21.15.52:0/2780726983 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8af8022470 con 0x7f8b080fed00 2023-11-27T15:23:13.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.640+0000 7f8b0fd83700 1 -- 172.21.15.52:0/2780726983 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b08110390 con 0x7f8b080fed00 2023-11-27T15:23:13.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.640+0000 7f8afeffd700 1 -- 172.21.15.52:0/2780726983 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af8017650 con 0x7f8b080fed00 2023-11-27T15:23:13.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.640+0000 7f8b0fd83700 1 -- 172.21.15.52:0/2780726983 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b08110200 con 0x7f8b080fed00 2023-11-27T15:23:13.656 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.643+0000 7f8afeffd700 1 -- 172.21.15.52:0/2780726983 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f8af80225e0 con 0x7f8b080fed00 2023-11-27T15:23:13.657 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.643+0000 7f8afeffd700 1 --2- 172.21.15.52:0/2780726983 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f8af40a12c0 0x7f8af40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.657 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.644+0000 7f8afeffd700 1 -- 172.21.15.52:0/2780726983 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f8af80cdc50 con 0x7f8b080fed00 2023-11-27T15:23:13.657 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.644+0000 7f8b0d31e700 1 --2- 172.21.15.52:0/2780726983 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f8af40a12c0 0x7f8af40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.657 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.644+0000 7f8b0d31e700 1 --2- 172.21.15.52:0/2780726983 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f8af40a12c0 0x7f8af40a3770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8b08197a90 tx=0x7f8b04009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:13.657 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.644+0000 7f8b1bf0fb80 1 Processor -- start 2023-11-27T15:23:13.657 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.644+0000 7f8b1bf0fb80 1 -- start start 2023-11-27T15:23:13.658 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.645+0000 7f8b1bf0fb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fec620b60 0x562fec620f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.658 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.645+0000 7f8b1bf0fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562fec621470 0x562fec625370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.658 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.645+0000 7f8b1bf0fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562fec6258b0 0x562fec625d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.658 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.645+0000 7f8b1bf0fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fec526060 con 0x562fec6258b0 2023-11-27T15:23:13.658 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.645+0000 7f8b1bf0fb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fec5788a0 con 0x562fec620b60 2023-11-27T15:23:13.658 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.645+0000 7f8b1bf0fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fec571d30 con 0x562fec621470 2023-11-27T15:23:13.658 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.645+0000 7f8b0e320700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562fec6258b0 0x562fec625d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.659 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.645+0000 7f8b0e320700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562fec6258b0 0x562fec625d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53942/0 (socket says 172.21.15.52:53942) 2023-11-27T15:23:13.659 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.645+0000 7f8b0e320700 1 -- 172.21.15.52:0/3946151469 learned_addr learned my addr 172.21.15.52:0/3946151469 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:13.659 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.645+0000 7f8b0db1f700 1 --2- 172.21.15.52:0/3946151469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fec620b60 0x562fec620f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.659 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.645+0000 7f8b0d31e700 1 --2- 172.21.15.52:0/3946151469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562fec621470 0x562fec625370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.659 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.645+0000 7f8b0e320700 1 -- 172.21.15.52:0/3946151469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562fec621470 msgr2=0x562fec625370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:13.659 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.645+0000 7f8b0e320700 1 --2- 172.21.15.52:0/3946151469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562fec621470 0x562fec625370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.659 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.645+0000 7f8b0e320700 1 -- 172.21.15.52:0/3946151469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fec620b60 msgr2=0x562fec620f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:13.659 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.645+0000 7f8b0e320700 1 --2- 172.21.15.52:0/3946151469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fec620b60 0x562fec620f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.659 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.645+0000 7f8b0e320700 1 -- 172.21.15.52:0/3946151469 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562fec3f3630 con 0x562fec6258b0 2023-11-27T15:23:13.660 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.645+0000 7f8b0e320700 1 --2- 172.21.15.52:0/3946151469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562fec6258b0 0x562fec625d00 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8b00011680 tx=0x7f8b00009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0689d3466bd48f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:13.660 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.645+0000 7f8af3fff700 1 -- 172.21.15.52:0/3946151469 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0000e040 con 0x562fec6258b0 2023-11-27T15:23:13.660 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.646+0000 7f8af3fff700 1 -- 172.21.15.52:0/3946151469 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b0000f040 con 0x562fec6258b0 2023-11-27T15:23:13.660 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.646+0000 7f8af3fff700 1 -- 172.21.15.52:0/3946151469 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b00004850 con 0x562fec6258b0 2023-11-27T15:23:13.660 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.646+0000 7f8b1bf0fb80 1 -- 172.21.15.52:0/3946151469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562fec6258b0 msgr2=0x562fec625d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:13.660 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.646+0000 7f8b1bf0fb80 1 --2- 172.21.15.52:0/3946151469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562fec6258b0 0x562fec625d00 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8b00011680 tx=0x7f8b00009920 comp rx=0 tx=0).stop 2023-11-27T15:23:13.660 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.646+0000 7f8b1bf0fb80 1 -- 172.21.15.52:0/3946151469 shutdown_connections 2023-11-27T15:23:13.660 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.646+0000 7f8b1bf0fb80 1 --2- 172.21.15.52:0/3946151469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fec620b60 0x562fec620f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.646+0000 7f8b1bf0fb80 1 --2- 172.21.15.52:0/3946151469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562fec6258b0 0x562fec625d00 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.646+0000 7f8b1bf0fb80 1 --2- 172.21.15.52:0/3946151469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562fec621470 0x562fec625370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.646+0000 7f8b1bf0fb80 1 -- 172.21.15.52:0/3946151469 >> 172.21.15.52:0/3946151469 conn(0x562fec616360 msgr2=0x562fec61bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:13.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.646+0000 7f8b1bf0fb80 1 -- 172.21.15.52:0/3946151469 shutdown_connections 2023-11-27T15:23:13.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.646+0000 7f8b1bf0fb80 1 -- 172.21.15.52:0/3946151469 wait complete. 2023-11-27T15:23:13.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.647+0000 7f8b1bf0fb80 1 -- start start 2023-11-27T15:23:13.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.647+0000 7f8b1bf0fb80 1 Processor -- start 2023-11-27T15:23:13.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.647+0000 7f8b1bf0fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562fec620b60 0x562fec62d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.647+0000 7f8b1bf0fb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fec621470 0x562fec62a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.647+0000 7f8b1bf0fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562fec6258b0 0x562fec62a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.647+0000 7f8b1bf0fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fec5138e0 con 0x562fec6258b0 2023-11-27T15:23:13.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.647+0000 7f8b1bf0fb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fec513580 con 0x562fec621470 2023-11-27T15:23:13.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.647+0000 7f8b1bf0fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fec3f5920 con 0x562fec620b60 2023-11-27T15:23:13.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.647+0000 7f8b1bf0fb80 10 client.0 _ll_register_callbacks cb 0x562fec531780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:13.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.647+0000 7f8b0d31e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fec621470 0x562fec62a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.647+0000 7f8b0e320700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562fec6258b0 0x562fec62a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.647+0000 7f8b0db1f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562fec620b60 0x562fec62d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.647+0000 7f8b0d31e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fec621470 0x562fec62a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43604/0 (socket says 172.21.15.52:43604) 2023-11-27T15:23:13.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.647+0000 7f8b0e320700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562fec6258b0 0x562fec62a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53948/0 (socket says 172.21.15.52:53948) 2023-11-27T15:23:13.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.647+0000 7f8b0d31e700 1 -- 172.21.15.52:0/3344909298 learned_addr learned my addr 172.21.15.52:0/3344909298 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:13.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.648+0000 7f8b0d31e700 1 -- 172.21.15.52:0/3344909298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562fec620b60 msgr2=0x562fec62d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:13.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.648+0000 7f8b0d31e700 1 --2- 172.21.15.52:0/3344909298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562fec620b60 0x562fec62d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.648+0000 7f8b0d31e700 1 -- 172.21.15.52:0/3344909298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562fec6258b0 msgr2=0x562fec62a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:13.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.648+0000 7f8b0d31e700 1 --2- 172.21.15.52:0/3344909298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562fec6258b0 0x562fec62a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.648+0000 7f8b0d31e700 1 -- 172.21.15.52:0/3344909298 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b00011330 con 0x562fec621470 2023-11-27T15:23:13.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.648+0000 7f8b0e320700 1 --2- 172.21.15.52:0/3344909298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562fec6258b0 0x562fec62a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:13.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.648+0000 7f8b0d31e700 1 --2- 172.21.15.52:0/3344909298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fec621470 0x562fec62a320 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8b0401e7a0 tx=0x7f8b040047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3800d7831e87119c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:13.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.648+0000 7f8af17fa700 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2023-11-27T15:23:13.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.648+0000 7f8af17fa700 1 -- 172.21.15.52:0/3344909298 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0401c640 con 0x562fec621470 2023-11-27T15:23:13.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.648+0000 7f8b1bf0fb80 1 -- 172.21.15.52:0/3344909298 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562fec613b20 con 0x562fec621470 2023-11-27T15:23:13.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.648+0000 7f8af17fa700 1 -- 172.21.15.52:0/3344909298 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f8b0401c7a0 con 0x562fec621470 2023-11-27T15:23:13.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.648+0000 7f8af17fa700 1 -- 172.21.15.52:0/3344909298 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b0401c960 con 0x562fec621470 2023-11-27T15:23:13.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.648+0000 7f8b1bf0fb80 10 client.26030 fetch_fsmap learned FSMap version 28 2023-11-27T15:23:13.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.648+0000 7f8b1bf0fb80 1 -- 172.21.15.52:0/3344909298 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562fec538190 con 0x562fec621470 2023-11-27T15:23:13.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.648+0000 7f8af17fa700 1 -- 172.21.15.52:0/3344909298 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b04033080 con 0x562fec621470 2023-11-27T15:23:13.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.649+0000 7f8af17fa700 1 -- 172.21.15.52:0/3344909298 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8b0402d030 con 0x562fec621470 2023-11-27T15:23:13.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.649+0000 7f8b1bf0fb80 10 client.26030 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:23:13.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.649+0000 7f8b1bf0fb80 10 client.26030 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:23:13.666 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.649+0000 7f8b1bf0fb80 20 client.26030 populate_metadata read hostname 'smithi052' 2023-11-27T15:23:13.666 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.649+0000 7f8b1bf0fb80 1 --2- 172.21.15.52:0/3344909298 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x562fec63aa00 0x562fec63ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.666 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.649+0000 7f8b1bf0fb80 4 client.26030 mds_command: new command op to 24494 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:23:13.666 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.649+0000 7f8b1bf0fb80 1 -- 172.21.15.52:0/3344909298 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562fec5340a0 con 0x562fec63aa00 2023-11-27T15:23:13.666 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.649+0000 7f8b0db1f700 1 --2- 172.21.15.52:0/3344909298 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x562fec63aa00 0x562fec63ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.666 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.649+0000 7f8b0db1f700 1 --2- 172.21.15.52:0/3344909298 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x562fec63aa00 0x562fec63ce10 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2925409174cfcc10 server_cookie=dd2ef3e376635da1 in_seq=0 out_seq=0 2023-11-27T15:23:13.666 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.649+0000 7f8af17fa700 10 client.26030 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:23:13.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.650+0000 7f8af17fa700 1 -- 172.21.15.52:0/3344909298 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562fec5340a0 con 0x562fec63aa00 2023-11-27T15:23:13.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.650+0000 7f8af17fa700 10 client.26030 handle_command_reply: tid=0 2023-11-27T15:23:13.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.650+0000 7f8b1bf0fb80 1 client.26030 shutdown 2023-11-27T15:23:13.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.650+0000 7f8b1bf0fb80 1 -- 172.21.15.52:0/3344909298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fec621470 msgr2=0x562fec62a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:13.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.650+0000 7f8b1bf0fb80 1 --2- 172.21.15.52:0/3344909298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fec621470 0x562fec62a320 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8b0401e7a0 tx=0x7f8b040047c0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.650+0000 7f8b1bf0fb80 1 -- 172.21.15.52:0/3344909298 shutdown_connections 2023-11-27T15:23:13.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.650+0000 7f8b1bf0fb80 1 --2- 172.21.15.52:0/3344909298 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x562fec63aa00 0x562fec63ce10 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.650+0000 7f8b1bf0fb80 1 --2- 172.21.15.52:0/3344909298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562fec6258b0 0x562fec62a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.668 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.650+0000 7f8b1bf0fb80 1 --2- 172.21.15.52:0/3344909298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562fec620b60 0x562fec62d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.668 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.650+0000 7f8b1bf0fb80 1 --2- 172.21.15.52:0/3344909298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fec621470 0x562fec62a320 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.668 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.650+0000 7f8af17fa700 0 client.26030 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:23:13.668 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.650+0000 7f8b1bf0fb80 1 -- 172.21.15.52:0/3344909298 >> 172.21.15.52:0/3344909298 conn(0x562fec616360 msgr2=0x562fec626d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:13.668 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.650+0000 7f8b1bf0fb80 1 -- 172.21.15.52:0/3344909298 shutdown_connections 2023-11-27T15:23:13.668 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.650+0000 7f8b1bf0fb80 1 -- 172.21.15.52:0/3344909298 wait complete. 2023-11-27T15:23:13.668 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.650+0000 7f8b1bf0fb80 20 client.26030 trim_cache size 0 max 16384 2023-11-27T15:23:13.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.666+0000 7f8b1bf0fb80 1 Processor -- start 2023-11-27T15:23:13.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.666+0000 7f8b1bf0fb80 1 -- start start 2023-11-27T15:23:13.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.666+0000 7f8b1bf0fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562fec620b60 0x562fec6319a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.666+0000 7f8b1bf0fb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fec621470 0x562fec6238f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.666+0000 7f8b1bf0fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562fec6258b0 0x562fec623e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.666+0000 7f8b1bf0fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fec628880 con 0x562fec6258b0 2023-11-27T15:23:13.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.666+0000 7f8b1bf0fb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fec624450 con 0x562fec621470 2023-11-27T15:23:13.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.666+0000 7f8b1bf0fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fec621900 con 0x562fec620b60 2023-11-27T15:23:13.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.666+0000 7f8b0e320700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562fec6258b0 0x562fec623e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.666+0000 7f8b0e320700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562fec6258b0 0x562fec623e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53956/0 (socket says 172.21.15.52:53956) 2023-11-27T15:23:13.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.666+0000 7f8b0e320700 1 -- 172.21.15.52:0/3305439390 learned_addr learned my addr 172.21.15.52:0/3305439390 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:13.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.667+0000 7f8b0db1f700 1 --2- 172.21.15.52:0/3305439390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562fec620b60 0x562fec6319a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.667+0000 7f8b0d31e700 1 --2- 172.21.15.52:0/3305439390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fec621470 0x562fec6238f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.667+0000 7f8b0e320700 1 -- 172.21.15.52:0/3305439390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562fec620b60 msgr2=0x562fec6319a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:13.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.667+0000 7f8b0e320700 1 --2- 172.21.15.52:0/3305439390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562fec620b60 0x562fec6319a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.667+0000 7f8b0e320700 1 -- 172.21.15.52:0/3305439390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fec621470 msgr2=0x562fec6238f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:13.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.667+0000 7f8b0e320700 1 --2- 172.21.15.52:0/3305439390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fec621470 0x562fec6238f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.667+0000 7f8b0e320700 1 -- 172.21.15.52:0/3305439390 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562fec537ee0 con 0x562fec6258b0 2023-11-27T15:23:13.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.667+0000 7f8b0d31e700 1 --2- 172.21.15.52:0/3305439390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fec621470 0x562fec6238f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:13.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.667+0000 7f8b0e320700 1 --2- 172.21.15.52:0/3305439390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562fec6258b0 0x562fec623e30 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8b00011620 tx=0x7f8b00009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90dbd2045c0562ef server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:13.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.667+0000 7f8afcff9700 1 -- 172.21.15.52:0/3305439390 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0000e050 con 0x562fec6258b0 2023-11-27T15:23:13.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.667+0000 7f8afcff9700 1 -- 172.21.15.52:0/3305439390 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b000093e0 con 0x562fec6258b0 2023-11-27T15:23:13.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.667+0000 7f8afcff9700 1 -- 172.21.15.52:0/3305439390 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b00004e40 con 0x562fec6258b0 2023-11-27T15:23:13.676 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.668+0000 7f8b1bf0fb80 1 -- 172.21.15.52:0/3305439390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562fec6258b0 msgr2=0x562fec623e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:13.676 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.668+0000 7f8b1bf0fb80 1 --2- 172.21.15.52:0/3305439390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562fec6258b0 0x562fec623e30 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8b00011620 tx=0x7f8b00009dd0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.676 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.668+0000 7f8b1bf0fb80 1 -- 172.21.15.52:0/3305439390 shutdown_connections 2023-11-27T15:23:13.676 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.668+0000 7f8b1bf0fb80 1 --2- 172.21.15.52:0/3305439390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562fec6258b0 0x562fec623e30 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.676 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.668+0000 7f8b1bf0fb80 1 --2- 172.21.15.52:0/3305439390 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562fec620b60 0x562fec6319a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.676 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.668+0000 7f8b1bf0fb80 1 --2- 172.21.15.52:0/3305439390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fec621470 0x562fec6238f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.676 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.668+0000 7f8b1bf0fb80 1 -- 172.21.15.52:0/3305439390 >> 172.21.15.52:0/3305439390 conn(0x562fec616360 msgr2=0x562fec636150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:13.676 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.668+0000 7f8b1bf0fb80 1 -- 172.21.15.52:0/3305439390 shutdown_connections 2023-11-27T15:23:13.676 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.668+0000 7f8b1bf0fb80 1 -- 172.21.15.52:0/3305439390 wait complete. 2023-11-27T15:23:13.677 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.668+0000 7f8b1bf0fb80 1 -- start start 2023-11-27T15:23:13.677 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.668+0000 7f8b1bf0fb80 1 Processor -- start 2023-11-27T15:23:13.680 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.669+0000 7f8b1bf0fb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fec620b60 0x562fec62f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.680 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.669+0000 7f8b1bf0fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562fec621470 0x562fec62ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.680 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.669+0000 7f8b1bf0fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562fec6258b0 0x562fec68d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.680 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.669+0000 7f8b1bf0fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fec630870 con 0x562fec6258b0 2023-11-27T15:23:13.680 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.669+0000 7f8b1bf0fb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fec68de00 con 0x562fec620b60 2023-11-27T15:23:13.680 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.669+0000 7f8b1bf0fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fec68df90 con 0x562fec621470 2023-11-27T15:23:13.680 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.669+0000 7f8b1bf0fb80 10 client.0 _ll_register_callbacks cb 0x7f8b04011f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:13.680 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.669+0000 7f8b0e320700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562fec6258b0 0x562fec68d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.669+0000 7f8b0d31e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562fec621470 0x562fec62ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.669+0000 7f8b0e320700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562fec6258b0 0x562fec68d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53962/0 (socket says 172.21.15.52:53962) 2023-11-27T15:23:13.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.669+0000 7f8b0e320700 1 -- 172.21.15.52:0/3647779258 learned_addr learned my addr 172.21.15.52:0/3647779258 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:13.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.669+0000 7f8b0db1f700 1 --2- 172.21.15.52:0/3647779258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fec620b60 0x562fec62f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.669+0000 7f8b0e320700 1 -- 172.21.15.52:0/3647779258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562fec621470 msgr2=0x562fec62ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:13.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.669+0000 7f8b0e320700 1 --2- 172.21.15.52:0/3647779258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562fec621470 0x562fec62ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.669+0000 7f8b0e320700 1 -- 172.21.15.52:0/3647779258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fec620b60 msgr2=0x562fec62f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:13.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.669+0000 7f8b0e320700 1 --2- 172.21.15.52:0/3647779258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fec620b60 0x562fec62f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.682 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.669+0000 7f8b0e320700 1 -- 172.21.15.52:0/3647779258 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562fec3f3630 con 0x562fec6258b0 2023-11-27T15:23:13.682 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.669+0000 7f8b0e320700 1 --2- 172.21.15.52:0/3647779258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562fec6258b0 0x562fec68d8c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009550 tx=0x7f8b000209a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487ebe164fe98551 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:13.682 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.669+0000 7f8af17fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:23:13.682 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.669+0000 7f8af17fa700 1 -- 172.21.15.52:0/3647779258 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b00004820 con 0x562fec6258b0 2023-11-27T15:23:13.682 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.670+0000 7f8af17fa700 1 -- 172.21.15.52:0/3647779258 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f8b00020da0 con 0x562fec6258b0 2023-11-27T15:23:13.682 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.670+0000 7f8b1bf0fb80 1 -- 172.21.15.52:0/3647779258 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b0000f040 con 0x562fec6258b0 2023-11-27T15:23:13.682 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.670+0000 7f8af17fa700 1 -- 172.21.15.52:0/3647779258 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b0000e020 con 0x562fec6258b0 2023-11-27T15:23:13.682 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.670+0000 7f8b1bf0fb80 10 client.16215 fetch_fsmap learned FSMap version 28 2023-11-27T15:23:13.682 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.670+0000 7f8b1bf0fb80 1 -- 172.21.15.52:0/3647779258 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562fec388c60 con 0x562fec6258b0 2023-11-27T15:23:13.683 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.670+0000 7f8af17fa700 1 -- 172.21.15.52:0/3647779258 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b0003d020 con 0x562fec6258b0 2023-11-27T15:23:13.683 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.670+0000 7f8af17fa700 1 -- 172.21.15.52:0/3647779258 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8b00007830 con 0x562fec6258b0 2023-11-27T15:23:13.683 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.670+0000 7f8b1bf0fb80 10 client.16215 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:23:13.683 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.670+0000 7f8b1bf0fb80 10 client.16215 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:23:13.683 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.670+0000 7f8b1bf0fb80 20 client.16215 populate_metadata read hostname 'smithi052' 2023-11-27T15:23:13.683 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.670+0000 7f8b1bf0fb80 1 --2- 172.21.15.52:0/3647779258 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x562fec68f360 0x562fec691770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.683 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.670+0000 7f8b1bf0fb80 4 client.16215 mds_command: new command op to 24494 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:23:13.683 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.670+0000 7f8b1bf0fb80 1 -- 172.21.15.52:0/3647779258 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562fec540660 con 0x562fec68f360 2023-11-27T15:23:13.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.670+0000 7f8b0db1f700 1 --2- 172.21.15.52:0/3647779258 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x562fec68f360 0x562fec691770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.671+0000 7f8b0db1f700 1 --2- 172.21.15.52:0/3647779258 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x562fec68f360 0x562fec691770 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7353d924be5c723d server_cookie=bb0110cd19ead0ea in_seq=0 out_seq=0 2023-11-27T15:23:13.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.671+0000 7f8af17fa700 10 client.16215 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:23:13.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.671+0000 7f8af17fa700 1 -- 172.21.15.52:0/3647779258 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562fec540660 con 0x562fec68f360 2023-11-27T15:23:13.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.671+0000 7f8af17fa700 10 client.16215 handle_command_reply: tid=0 2023-11-27T15:23:13.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.671+0000 7f8b1bf0fb80 1 client.16215 shutdown 2023-11-27T15:23:13.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.671+0000 7f8b1bf0fb80 1 -- 172.21.15.52:0/3647779258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562fec6258b0 msgr2=0x562fec68d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:13.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.671+0000 7f8b1bf0fb80 1 --2- 172.21.15.52:0/3647779258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562fec6258b0 0x562fec68d8c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009550 tx=0x7f8b000209a0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.671+0000 7f8b1bf0fb80 1 -- 172.21.15.52:0/3647779258 shutdown_connections 2023-11-27T15:23:13.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.671+0000 7f8b1bf0fb80 1 --2- 172.21.15.52:0/3647779258 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x562fec68f360 0x562fec691770 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.671+0000 7f8b1bf0fb80 1 --2- 172.21.15.52:0/3647779258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fec620b60 0x562fec62f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.671+0000 7f8b1bf0fb80 1 --2- 172.21.15.52:0/3647779258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562fec6258b0 0x562fec68d8c0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.671+0000 7f8b1bf0fb80 1 --2- 172.21.15.52:0/3647779258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562fec621470 0x562fec62ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.671+0000 7f8af17fa700 0 client.16215 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:23:13.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.671+0000 7f8b1bf0fb80 1 -- 172.21.15.52:0/3647779258 >> 172.21.15.52:0/3647779258 conn(0x562fec616360 msgr2=0x562fec624860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:13.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.671+0000 7f8b1bf0fb80 1 -- 172.21.15.52:0/3647779258 shutdown_connections 2023-11-27T15:23:13.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.671+0000 7f8b1bf0fb80 1 -- 172.21.15.52:0/3647779258 wait complete. 2023-11-27T15:23:13.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.671+0000 7f8b1bf0fb80 20 client.16215 trim_cache size 0 max 16384 2023-11-27T15:23:13.686 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:23:13.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.672+0000 7f8b0fd83700 1 -- 172.21.15.52:0/2780726983 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f8af40a12c0 msgr2=0x7f8af40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:13.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.672+0000 7f8b0fd83700 1 --2- 172.21.15.52:0/2780726983 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f8af40a12c0 0x7f8af40a3770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8b08197a90 tx=0x7f8b04009260 comp rx=0 tx=0).stop 2023-11-27T15:23:13.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.672+0000 7f8b0fd83700 1 -- 172.21.15.52:0/2780726983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b080fed00 msgr2=0x7f8b08196480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:13.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.672+0000 7f8b0fd83700 1 --2- 172.21.15.52:0/2780726983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b080fed00 0x7f8b08196480 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8af800b580 tx=0x7f8af800bfd0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.672+0000 7f8b0fd83700 1 -- 172.21.15.52:0/2780726983 shutdown_connections 2023-11-27T15:23:13.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.672+0000 7f8b0fd83700 1 --2- 172.21.15.52:0/2780726983 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f8af40a12c0 0x7f8af40a3770 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.672+0000 7f8b0fd83700 1 --2- 172.21.15.52:0/2780726983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b08105620 0x7f8b0819ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.672+0000 7f8b0fd83700 1 --2- 172.21.15.52:0/2780726983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b080fed00 0x7f8b08196480 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.672+0000 7f8b0fd83700 1 --2- 172.21.15.52:0/2780726983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b080ff6a0 0x7f8b081969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.672+0000 7f8b0fd83700 1 -- 172.21.15.52:0/2780726983 >> 172.21.15.52:0/2780726983 conn(0x7f8b080748f0 msgr2=0x7f8b080fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:13.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.672+0000 7f8b0fd83700 1 -- 172.21.15.52:0/2780726983 shutdown_connections 2023-11-27T15:23:13.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.672+0000 7f8b0fd83700 1 -- 172.21.15.52:0/2780726983 wait complete. 2023-11-27T15:23:13.689 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-27T15:23:13.859 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.854+0000 7f5009222700 1 -- 172.21.15.52:0/337145247 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f500410a9f0 msgr2=0x7f500410ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:13.859 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.854+0000 7f5009222700 1 --2- 172.21.15.52:0/337145247 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f500410a9f0 0x7f500410ce30 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc00b0a0 tx=0x7f4ffc00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.859 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.854+0000 7f5009222700 1 -- 172.21.15.52:0/337145247 shutdown_connections 2023-11-27T15:23:13.859 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.854+0000 7f5009222700 1 --2- 172.21.15.52:0/337145247 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5004101620 0x7f50041019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.860 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.854+0000 7f5009222700 1 --2- 172.21.15.52:0/337145247 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f500410a9f0 0x7f500410ce30 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.860 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.854+0000 7f5009222700 1 --2- 172.21.15.52:0/337145247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5004101f30 0x7f500410a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.860 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.854+0000 7f5009222700 1 -- 172.21.15.52:0/337145247 >> 172.21.15.52:0/337145247 conn(0x7f5004076100 msgr2=0x7f5004076500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:13.860 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.854+0000 7f5009222700 1 -- 172.21.15.52:0/337145247 shutdown_connections 2023-11-27T15:23:13.860 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.854+0000 7f5009222700 1 -- 172.21.15.52:0/337145247 wait complete. 2023-11-27T15:23:13.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.855+0000 7f5009222700 1 Processor -- start 2023-11-27T15:23:13.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.855+0000 7f5009222700 1 -- start start 2023-11-27T15:23:13.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.855+0000 7f5009222700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5004101620 0x7f50040ff270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.855+0000 7f5009222700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5004101f30 0x7f50040ff7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.855+0000 7f5009222700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f500410a9f0 0x7f50041a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.855+0000 7f5009222700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50040fffe0 con 0x7f5004101620 2023-11-27T15:23:13.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.855+0000 7f5009222700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50041a4ce0 con 0x7f5004101f30 2023-11-27T15:23:13.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.855+0000 7f5009222700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50041a4e20 con 0x7f500410a9f0 2023-11-27T15:23:13.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.856+0000 7f5002d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5004101620 0x7f50040ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.856+0000 7f5002d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5004101620 0x7f50040ff270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53972/0 (socket says 172.21.15.52:53972) 2023-11-27T15:23:13.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.856+0000 7f5002d9d700 1 -- 172.21.15.52:0/971365113 learned_addr learned my addr 172.21.15.52:0/971365113 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:13.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.856+0000 7f500259c700 1 --2- 172.21.15.52:0/971365113 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5004101f30 0x7f50040ff7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.856+0000 7f500359e700 1 --2- 172.21.15.52:0/971365113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f500410a9f0 0x7f50041a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.856+0000 7f5002d9d700 1 -- 172.21.15.52:0/971365113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f500410a9f0 msgr2=0x7f50041a47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:13.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.856+0000 7f5002d9d700 1 --2- 172.21.15.52:0/971365113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f500410a9f0 0x7f50041a47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.864 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.856+0000 7f5002d9d700 1 -- 172.21.15.52:0/971365113 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5004101f30 msgr2=0x7f50040ff7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:13.864 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.856+0000 7f5002d9d700 1 --2- 172.21.15.52:0/971365113 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5004101f30 0x7f50040ff7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.864 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.856+0000 7f5002d9d700 1 -- 172.21.15.52:0/971365113 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ffc009d00 con 0x7f5004101620 2023-11-27T15:23:13.864 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.856+0000 7f5002d9d700 1 --2- 172.21.15.52:0/971365113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5004101620 0x7f50040ff270 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400e9a0 tx=0x7f4ff400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:13.864 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.856+0000 7f4febfff700 1 -- 172.21.15.52:0/971365113 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4009790 con 0x7f5004101620 2023-11-27T15:23:13.864 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.856+0000 7f5009222700 1 -- 172.21.15.52:0/971365113 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50041104e0 con 0x7f5004101620 2023-11-27T15:23:13.864 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.856+0000 7f4febfff700 1 -- 172.21.15.52:0/971365113 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ff4010430 con 0x7f5004101620 2023-11-27T15:23:13.864 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.856+0000 7f4febfff700 1 -- 172.21.15.52:0/971365113 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4018680 con 0x7f5004101620 2023-11-27T15:23:13.864 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.856+0000 7f5009222700 1 -- 172.21.15.52:0/971365113 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5004110350 con 0x7f5004101620 2023-11-27T15:23:13.867 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.859+0000 7f4febfff700 1 -- 172.21.15.52:0/971365113 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f4ff40105a0 con 0x7f5004101620 2023-11-27T15:23:13.867 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.860+0000 7f4febfff700 1 --2- 172.21.15.52:0/971365113 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f4fec0a12c0 0x7f4fec0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.867 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.860+0000 7f500259c700 1 --2- 172.21.15.52:0/971365113 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f4fec0a12c0 0x7f4fec0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.860+0000 7f4febfff700 1 -- 172.21.15.52:0/971365113 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4ff4014070 con 0x7f5004101620 2023-11-27T15:23:13.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.860+0000 7f50153aeb80 1 Processor -- start 2023-11-27T15:23:13.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.860+0000 7f50153aeb80 1 -- start start 2023-11-27T15:23:13.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.860+0000 7f50153aeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ca34f8b60 0x556ca34f8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.860+0000 7f50153aeb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ca34f9470 0x556ca34fd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.860+0000 7f50153aeb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ca34fd8b0 0x556ca34fdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.860+0000 7f500259c700 1 --2- 172.21.15.52:0/971365113 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f4fec0a12c0 0x7f4fec0a3770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5004100a60 tx=0x7f4ff8009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:13.868 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.860+0000 7f50153aeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca33fe060 con 0x556ca34f8b60 2023-11-27T15:23:13.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.860+0000 7f50153aeb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca34508a0 con 0x556ca34fd8b0 2023-11-27T15:23:13.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.860+0000 7f50153aeb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca3449d30 con 0x556ca34f9470 2023-11-27T15:23:13.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.860+0000 7f5002d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ca34f8b60 0x556ca34f8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.861+0000 7f5002d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ca34f8b60 0x556ca34f8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53980/0 (socket says 172.21.15.52:53980) 2023-11-27T15:23:13.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.861+0000 7f5002d9d700 1 -- 172.21.15.52:0/3085404745 learned_addr learned my addr 172.21.15.52:0/3085404745 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:13.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.861+0000 7f500359e700 1 --2- 172.21.15.52:0/3085404745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ca34fd8b0 0x556ca34fdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.861+0000 7f500259c700 1 --2- 172.21.15.52:0/3085404745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ca34f9470 0x556ca34fd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.861+0000 7f5002d9d700 1 -- 172.21.15.52:0/3085404745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ca34f9470 msgr2=0x556ca34fd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:13.869 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.861+0000 7f5002d9d700 1 --2- 172.21.15.52:0/3085404745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ca34f9470 0x556ca34fd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.861+0000 7f5002d9d700 1 -- 172.21.15.52:0/3085404745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ca34fd8b0 msgr2=0x556ca34fdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:13.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.861+0000 7f5002d9d700 1 --2- 172.21.15.52:0/3085404745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ca34fd8b0 0x556ca34fdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.861+0000 7f5002d9d700 1 -- 172.21.15.52:0/3085404745 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ca32cb630 con 0x556ca34f8b60 2023-11-27T15:23:13.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.861+0000 7f500359e700 1 --2- 172.21.15.52:0/3085404745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ca34fd8b0 0x556ca34fdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:13.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.861+0000 7f500259c700 1 --2- 172.21.15.52:0/3085404745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ca34f9470 0x556ca34fd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:23:13.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.861+0000 7f5002d9d700 1 --2- 172.21.15.52:0/3085404745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ca34f8b60 0x556ca34f8f30 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4096a20 tx=0x7f4ff4099c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=846fe17e08bcfbd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:13.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.861+0000 7f4fe97fa700 1 -- 172.21.15.52:0/3085404745 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff40a2400 con 0x556ca34f8b60 2023-11-27T15:23:13.870 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.861+0000 7f4fe97fa700 1 -- 172.21.15.52:0/3085404745 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ff4010880 con 0x556ca34f8b60 2023-11-27T15:23:13.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.861+0000 7f4fe97fa700 1 -- 172.21.15.52:0/3085404745 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff40a1f70 con 0x556ca34f8b60 2023-11-27T15:23:13.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.862+0000 7f50153aeb80 1 -- 172.21.15.52:0/3085404745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ca34f8b60 msgr2=0x556ca34f8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:13.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.862+0000 7f50153aeb80 1 --2- 172.21.15.52:0/3085404745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ca34f8b60 0x556ca34f8f30 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4096a20 tx=0x7f4ff4099c20 comp rx=0 tx=0).stop 2023-11-27T15:23:13.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.862+0000 7f50153aeb80 1 -- 172.21.15.52:0/3085404745 shutdown_connections 2023-11-27T15:23:13.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.862+0000 7f50153aeb80 1 --2- 172.21.15.52:0/3085404745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ca34fd8b0 0x556ca34fdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.862+0000 7f50153aeb80 1 --2- 172.21.15.52:0/3085404745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ca34f8b60 0x556ca34f8f30 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.862+0000 7f50153aeb80 1 --2- 172.21.15.52:0/3085404745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ca34f9470 0x556ca34fd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.871 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.862+0000 7f50153aeb80 1 -- 172.21.15.52:0/3085404745 >> 172.21.15.52:0/3085404745 conn(0x556ca34ee360 msgr2=0x556ca34f3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:13.872 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.862+0000 7f50153aeb80 1 -- 172.21.15.52:0/3085404745 shutdown_connections 2023-11-27T15:23:13.872 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.862+0000 7f50153aeb80 1 -- 172.21.15.52:0/3085404745 wait complete. 2023-11-27T15:23:13.872 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.863+0000 7f50153aeb80 1 -- start start 2023-11-27T15:23:13.872 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.863+0000 7f50153aeb80 1 Processor -- start 2023-11-27T15:23:13.875 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.863+0000 7f50153aeb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ca34f8b60 0x556ca35050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.875 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.863+0000 7f50153aeb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ca34f9470 0x556ca3502350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.875 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.863+0000 7f50153aeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ca34fd8b0 0x556ca3502a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.875 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.863+0000 7f50153aeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca33eb8e0 con 0x556ca34fd8b0 2023-11-27T15:23:13.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.863+0000 7f50153aeb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca33eb580 con 0x556ca34f9470 2023-11-27T15:23:13.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.863+0000 7f50153aeb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca32cd920 con 0x556ca34f8b60 2023-11-27T15:23:13.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.863+0000 7f50153aeb80 10 client.0 _ll_register_callbacks cb 0x556ca3409780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:13.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.863+0000 7f500359e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ca34fd8b0 0x556ca3502a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.863+0000 7f5002d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ca34f8b60 0x556ca35050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.863+0000 7f500359e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ca34fd8b0 0x556ca3502a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53990/0 (socket says 172.21.15.52:53990) 2023-11-27T15:23:13.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.863+0000 7f500359e700 1 -- 172.21.15.52:0/4062515332 learned_addr learned my addr 172.21.15.52:0/4062515332 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:13.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.863+0000 7f500259c700 1 --2- 172.21.15.52:0/4062515332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ca34f9470 0x556ca3502350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.864+0000 7f500359e700 1 -- 172.21.15.52:0/4062515332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ca34f8b60 msgr2=0x556ca35050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:13.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.864+0000 7f500359e700 1 --2- 172.21.15.52:0/4062515332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ca34f8b60 0x556ca35050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.864+0000 7f500359e700 1 -- 172.21.15.52:0/4062515332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ca34f9470 msgr2=0x556ca3502350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:13.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.864+0000 7f500359e700 1 --2- 172.21.15.52:0/4062515332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ca34f9470 0x556ca3502350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.864+0000 7f500359e700 1 -- 172.21.15.52:0/4062515332 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5004110350 con 0x556ca34fd8b0 2023-11-27T15:23:13.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.864+0000 7f500259c700 1 --2- 172.21.15.52:0/4062515332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ca34f9470 0x556ca3502350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:13.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.864+0000 7f500359e700 1 --2- 172.21.15.52:0/4062515332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ca34fd8b0 0x556ca3502a40 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc00b0a0 tx=0x7f4ffc009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab4dd24e5728acd server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:13.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.864+0000 7f4fe6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:23:13.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.864+0000 7f4fe6ff5700 1 -- 172.21.15.52:0/4062515332 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc00e040 con 0x556ca34fd8b0 2023-11-27T15:23:13.878 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.864+0000 7f4fe6ff5700 1 -- 172.21.15.52:0/4062515332 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4ffc003dc0 con 0x556ca34fd8b0 2023-11-27T15:23:13.878 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.864+0000 7f50153aeb80 1 -- 172.21.15.52:0/4062515332 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ca34ebb20 con 0x556ca34fd8b0 2023-11-27T15:23:13.878 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.864+0000 7f4fe6ff5700 1 -- 172.21.15.52:0/4062515332 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ffc0074e0 con 0x556ca34fd8b0 2023-11-27T15:23:13.878 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.864+0000 7f50153aeb80 10 client.16227 fetch_fsmap learned FSMap version 28 2023-11-27T15:23:13.878 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.864+0000 7f50153aeb80 1 -- 172.21.15.52:0/4062515332 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556ca3410190 con 0x556ca34fd8b0 2023-11-27T15:23:13.878 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.864+0000 7f4fe6ff5700 1 -- 172.21.15.52:0/4062515332 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556ca34ebb20 con 0x556ca34fd8b0 2023-11-27T15:23:13.878 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.864+0000 7f4fe6ff5700 1 -- 172.21.15.52:0/4062515332 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f4ffc019070 con 0x556ca34fd8b0 2023-11-27T15:23:13.878 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.865+0000 7f50153aeb80 10 client.16227 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:23:13.879 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.865+0000 7f50153aeb80 10 client.16227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:13.879 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.865+0000 7f50153aeb80 20 client.16227 populate_metadata read hostname 'smithi052' 2023-11-27T15:23:13.879 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.865+0000 7f50153aeb80 1 --2- 172.21.15.52:0/4062515332 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x556ca35128e0 0x556ca3514cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.879 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.865+0000 7f50153aeb80 4 client.16227 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:23:13.879 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.865+0000 7f50153aeb80 1 -- 172.21.15.52:0/4062515332 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ca340c0a0 con 0x556ca35128e0 2023-11-27T15:23:13.879 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.865+0000 7f5002d9d700 1 --2- 172.21.15.52:0/4062515332 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x556ca35128e0 0x556ca3514cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.879 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.865+0000 7f4fe6ff5700 10 client.16227 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:13.879 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.865+0000 7f5002d9d700 1 --2- 172.21.15.52:0/4062515332 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x556ca35128e0 0x556ca3514cf0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d790519d2991d95 server_cookie=bf7589d27d835fed in_seq=0 out_seq=0 2023-11-27T15:23:13.879 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.866+0000 7f4fe6ff5700 1 -- 172.21.15.52:0/4062515332 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556ca340c0a0 con 0x556ca35128e0 2023-11-27T15:23:13.880 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.866+0000 7f4fe6ff5700 10 client.16227 handle_command_reply: tid=0 2023-11-27T15:23:13.880 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.866+0000 7f50153aeb80 1 client.16227 shutdown 2023-11-27T15:23:13.880 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.866+0000 7f50153aeb80 1 -- 172.21.15.52:0/4062515332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ca34fd8b0 msgr2=0x556ca3502a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:13.880 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.866+0000 7f50153aeb80 1 --2- 172.21.15.52:0/4062515332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ca34fd8b0 0x556ca3502a40 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc00b0a0 tx=0x7f4ffc009590 comp rx=0 tx=0).stop 2023-11-27T15:23:13.880 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.866+0000 7f50153aeb80 1 -- 172.21.15.52:0/4062515332 shutdown_connections 2023-11-27T15:23:13.880 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.866+0000 7f50153aeb80 1 --2- 172.21.15.52:0/4062515332 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x556ca35128e0 0x556ca3514cf0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.880 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.866+0000 7f50153aeb80 1 --2- 172.21.15.52:0/4062515332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ca34fd8b0 0x556ca3502a40 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.880 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.866+0000 7f50153aeb80 1 --2- 172.21.15.52:0/4062515332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ca34f8b60 0x556ca35050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.881 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.866+0000 7f50153aeb80 1 --2- 172.21.15.52:0/4062515332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ca34f9470 0x556ca3502350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.881 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.866+0000 7f4fe6ff5700 0 client.16227 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:13.881 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.866+0000 7f50153aeb80 1 -- 172.21.15.52:0/4062515332 >> 172.21.15.52:0/4062515332 conn(0x556ca34ee360 msgr2=0x556ca34fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:13.881 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.866+0000 7f50153aeb80 1 -- 172.21.15.52:0/4062515332 shutdown_connections 2023-11-27T15:23:13.881 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.866+0000 7f50153aeb80 1 -- 172.21.15.52:0/4062515332 wait complete. 2023-11-27T15:23:13.881 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.866+0000 7f50153aeb80 20 client.16227 trim_cache size 0 max 16384 2023-11-27T15:23:13.886 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.879+0000 7f50153aeb80 1 Processor -- start 2023-11-27T15:23:13.886 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.879+0000 7f50153aeb80 1 -- start start 2023-11-27T15:23:13.886 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.880+0000 7f50153aeb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ca34f8b60 0x556ca3502950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.886 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.880+0000 7f50153aeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ca34f9470 0x556ca35018c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.887 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.880+0000 7f50153aeb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ca34fd8b0 0x556ca34fb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.887 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.880+0000 7f50153aeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca34f9bf0 con 0x556ca34f9470 2023-11-27T15:23:13.887 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.880+0000 7f50153aeb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca35037e0 con 0x556ca34fd8b0 2023-11-27T15:23:13.887 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.880+0000 7f50153aeb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca3500430 con 0x556ca34f8b60 2023-11-27T15:23:13.887 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.880+0000 7f500259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ca34f9470 0x556ca35018c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.887 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.880+0000 7f500259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ca34f9470 0x556ca35018c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53996/0 (socket says 172.21.15.52:53996) 2023-11-27T15:23:13.887 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.880+0000 7f500259c700 1 -- 172.21.15.52:0/576464551 learned_addr learned my addr 172.21.15.52:0/576464551 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:13.887 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.880+0000 7f5002d9d700 1 --2- 172.21.15.52:0/576464551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ca34f8b60 0x556ca3502950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.888 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.880+0000 7f500359e700 1 --2- 172.21.15.52:0/576464551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ca34fd8b0 0x556ca34fb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.888 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.880+0000 7f500259c700 1 -- 172.21.15.52:0/576464551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ca34f8b60 msgr2=0x556ca3502950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:13.888 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.880+0000 7f500259c700 1 --2- 172.21.15.52:0/576464551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ca34f8b60 0x556ca3502950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.888 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.880+0000 7f500259c700 1 -- 172.21.15.52:0/576464551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ca34fd8b0 msgr2=0x556ca34fb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:13.888 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.880+0000 7f500259c700 1 --2- 172.21.15.52:0/576464551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ca34fd8b0 0x556ca34fb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.888 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.880+0000 7f500259c700 1 -- 172.21.15.52:0/576464551 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ca34efb00 con 0x556ca34f9470 2023-11-27T15:23:13.888 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.880+0000 7f5002d9d700 1 --2- 172.21.15.52:0/576464551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ca34f8b60 0x556ca3502950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:13.888 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.880+0000 7f500259c700 1 --2- 172.21.15.52:0/576464551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ca34f9470 0x556ca35018c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5004067c40 tx=0x7f4ff8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba6af0049be42d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:13.888 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.880+0000 7f4fe9ffb700 1 -- 172.21.15.52:0/576464551 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8023070 con 0x556ca34f9470 2023-11-27T15:23:13.889 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.881+0000 7f4fe9ffb700 1 -- 172.21.15.52:0/576464551 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ff801b9b0 con 0x556ca34f9470 2023-11-27T15:23:13.889 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.881+0000 7f4fe9ffb700 1 -- 172.21.15.52:0/576464551 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff801d7d0 con 0x556ca34f9470 2023-11-27T15:23:13.889 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.881+0000 7f50153aeb80 1 -- 172.21.15.52:0/576464551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ca34f9470 msgr2=0x556ca35018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:13.889 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.881+0000 7f50153aeb80 1 --2- 172.21.15.52:0/576464551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ca34f9470 0x556ca35018c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5004067c40 tx=0x7f4ff8014000 comp rx=0 tx=0).stop 2023-11-27T15:23:13.889 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.881+0000 7f50153aeb80 1 -- 172.21.15.52:0/576464551 shutdown_connections 2023-11-27T15:23:13.889 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.881+0000 7f50153aeb80 1 --2- 172.21.15.52:0/576464551 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ca34fd8b0 0x556ca34fb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.889 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.881+0000 7f50153aeb80 1 --2- 172.21.15.52:0/576464551 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ca34f8b60 0x556ca3502950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.889 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.881+0000 7f50153aeb80 1 --2- 172.21.15.52:0/576464551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ca34f9470 0x556ca35018c0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.890 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.881+0000 7f50153aeb80 1 -- 172.21.15.52:0/576464551 >> 172.21.15.52:0/576464551 conn(0x556ca34ee360 msgr2=0x556ca3504d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:13.890 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.881+0000 7f50153aeb80 1 -- 172.21.15.52:0/576464551 shutdown_connections 2023-11-27T15:23:13.890 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.881+0000 7f50153aeb80 1 -- 172.21.15.52:0/576464551 wait complete. 2023-11-27T15:23:13.890 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.881+0000 7f50153aeb80 1 -- start start 2023-11-27T15:23:13.890 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.882+0000 7f50153aeb80 1 Processor -- start 2023-11-27T15:23:13.893 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.882+0000 7f50153aeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ca34f8b60 0x556ca3507990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.893 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.882+0000 7f50153aeb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ca34f9470 0x556ca3507ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.893 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.882+0000 7f50153aeb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ca34fd8b0 0x556ca3508410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.893 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.882+0000 7f50153aeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca3509fb0 con 0x556ca34f8b60 2023-11-27T15:23:13.893 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.882+0000 7f50153aeb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca35657a0 con 0x556ca34f9470 2023-11-27T15:23:13.894 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.882+0000 7f50153aeb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca3565960 con 0x556ca34fd8b0 2023-11-27T15:23:13.894 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.882+0000 7f5002d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ca34f8b60 0x556ca3507990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.894 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.882+0000 7f50153aeb80 10 client.0 _ll_register_callbacks cb 0x7f4ffc00bdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:13.894 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.882+0000 7f5002d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ca34f8b60 0x556ca3507990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54000/0 (socket says 172.21.15.52:54000) 2023-11-27T15:23:13.894 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.882+0000 7f5002d9d700 1 -- 172.21.15.52:0/78951909 learned_addr learned my addr 172.21.15.52:0/78951909 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:13.894 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.882+0000 7f500259c700 1 --2- 172.21.15.52:0/78951909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ca34f9470 0x556ca3507ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.894 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.882+0000 7f5002d9d700 1 -- 172.21.15.52:0/78951909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ca34fd8b0 msgr2=0x556ca3508410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:13.894 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.882+0000 7f5002d9d700 1 --2- 172.21.15.52:0/78951909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ca34fd8b0 0x556ca3508410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.882+0000 7f5002d9d700 1 -- 172.21.15.52:0/78951909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ca34f9470 msgr2=0x556ca3507ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:13.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.882+0000 7f5002d9d700 1 --2- 172.21.15.52:0/78951909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ca34f9470 0x556ca3507ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.882+0000 7f5002d9d700 1 -- 172.21.15.52:0/78951909 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff8017070 con 0x556ca34f8b60 2023-11-27T15:23:13.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.882+0000 7f500259c700 1 --2- 172.21.15.52:0/78951909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ca34f9470 0x556ca3507ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:13.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.882+0000 7f5002d9d700 1 --2- 172.21.15.52:0/78951909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ca34f8b60 0x556ca3507990 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4099b60 tx=0x7f4ff409a120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d484750eb633edd1 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:13.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.882+0000 7f4fe6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:23:13.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.882+0000 7f4fe6ff5700 1 -- 172.21.15.52:0/78951909 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff40b8370 con 0x556ca34f8b60 2023-11-27T15:23:13.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.883+0000 7f4fe6ff5700 1 -- 172.21.15.52:0/78951909 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f4ff40994a0 con 0x556ca34f8b60 2023-11-27T15:23:13.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.883+0000 7f50153aeb80 1 -- 172.21.15.52:0/78951909 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ff4010880 con 0x556ca34f8b60 2023-11-27T15:23:13.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.883+0000 7f4fe6ff5700 1 -- 172.21.15.52:0/78951909 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ff4099660 con 0x556ca34f8b60 2023-11-27T15:23:13.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.883+0000 7f50153aeb80 10 client.16233 fetch_fsmap learned FSMap version 28 2023-11-27T15:23:13.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.883+0000 7f50153aeb80 1 -- 172.21.15.52:0/78951909 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556ca3277de0 con 0x556ca34f8b60 2023-11-27T15:23:13.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.883+0000 7f4fe6ff5700 1 -- 172.21.15.52:0/78951909 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ff4063080 con 0x556ca34f8b60 2023-11-27T15:23:13.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.883+0000 7f4fe6ff5700 1 -- 172.21.15.52:0/78951909 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f4ff4094bc0 con 0x556ca34f8b60 2023-11-27T15:23:13.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.883+0000 7f50153aeb80 10 client.16233 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:23:13.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.883+0000 7f50153aeb80 10 client.16233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:13.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.883+0000 7f50153aeb80 20 client.16233 populate_metadata read hostname 'smithi052' 2023-11-27T15:23:13.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.883+0000 7f50153aeb80 1 --2- 172.21.15.52:0/78951909 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x556ca3566d50 0x556ca3569160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:13.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.883+0000 7f50153aeb80 4 client.16233 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:23:13.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.883+0000 7f50153aeb80 1 -- 172.21.15.52:0/78951909 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556ca3418660 con 0x556ca3566d50 2023-11-27T15:23:13.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.883+0000 7f500259c700 1 --2- 172.21.15.52:0/78951909 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x556ca3566d50 0x556ca3569160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:13.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.884+0000 7f4fe6ff5700 10 client.16233 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:13.897 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.884+0000 7f500259c700 1 --2- 172.21.15.52:0/78951909 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x556ca3566d50 0x556ca3569160 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff59681e10238c61 server_cookie=4c3e58c7717514f9 in_seq=0 out_seq=0 2023-11-27T15:23:13.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.884+0000 7f4fe6ff5700 1 -- 172.21.15.52:0/78951909 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556ca3418660 con 0x556ca3566d50 2023-11-27T15:23:13.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.884+0000 7f4fe6ff5700 10 client.16233 handle_command_reply: tid=0 2023-11-27T15:23:13.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.884+0000 7f50153aeb80 1 client.16233 shutdown 2023-11-27T15:23:13.898 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.884+0000 7f50153aeb80 1 -- 172.21.15.52:0/78951909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ca34f8b60 msgr2=0x556ca3507990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:13.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.884+0000 7f50153aeb80 1 --2- 172.21.15.52:0/78951909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ca34f8b60 0x556ca3507990 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4099b60 tx=0x7f4ff409a120 comp rx=0 tx=0).stop 2023-11-27T15:23:13.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.884+0000 7f50153aeb80 1 -- 172.21.15.52:0/78951909 shutdown_connections 2023-11-27T15:23:13.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.884+0000 7f50153aeb80 1 --2- 172.21.15.52:0/78951909 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x556ca3566d50 0x556ca3569160 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.899 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.884+0000 7f50153aeb80 1 --2- 172.21.15.52:0/78951909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556ca34fd8b0 0x556ca3508410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.884+0000 7f50153aeb80 1 --2- 172.21.15.52:0/78951909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ca34f8b60 0x556ca3507990 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.884+0000 7f50153aeb80 1 --2- 172.21.15.52:0/78951909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ca34f9470 0x556ca3507ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.884+0000 7f4fe6ff5700 0 client.16233 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:13.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.884+0000 7f50153aeb80 1 -- 172.21.15.52:0/78951909 >> 172.21.15.52:0/78951909 conn(0x556ca34ee360 msgr2=0x556ca34fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:13.900 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.884+0000 7f50153aeb80 1 -- 172.21.15.52:0/78951909 shutdown_connections 2023-11-27T15:23:13.901 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.884+0000 7f50153aeb80 1 -- 172.21.15.52:0/78951909 wait complete. 2023-11-27T15:23:13.901 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.884+0000 7f50153aeb80 20 client.16233 trim_cache size 0 max 16384 2023-11-27T15:23:13.901 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:23:13.902 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.885+0000 7f5009222700 1 -- 172.21.15.52:0/971365113 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f4fec0a12c0 msgr2=0x7f4fec0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:13.903 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.885+0000 7f5009222700 1 --2- 172.21.15.52:0/971365113 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f4fec0a12c0 0x7f4fec0a3770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5004100a60 tx=0x7f4ff8009260 comp rx=0 tx=0).stop 2023-11-27T15:23:13.903 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.885+0000 7f5009222700 1 -- 172.21.15.52:0/971365113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5004101620 msgr2=0x7f50040ff270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:13.903 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.885+0000 7f5009222700 1 --2- 172.21.15.52:0/971365113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5004101620 0x7f50040ff270 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400e9a0 tx=0x7f4ff400ecb0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.903 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.885+0000 7f5009222700 1 -- 172.21.15.52:0/971365113 shutdown_connections 2023-11-27T15:23:13.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.885+0000 7f5009222700 1 --2- 172.21.15.52:0/971365113 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f4fec0a12c0 0x7f4fec0a3770 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.885+0000 7f5009222700 1 --2- 172.21.15.52:0/971365113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f500410a9f0 0x7f50041a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.885+0000 7f5009222700 1 --2- 172.21.15.52:0/971365113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5004101620 0x7f50040ff270 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.885+0000 7f5009222700 1 --2- 172.21.15.52:0/971365113 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5004101f30 0x7f50040ff7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:13.905 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.885+0000 7f5009222700 1 -- 172.21.15.52:0/971365113 >> 172.21.15.52:0/971365113 conn(0x7f5004076100 msgr2=0x7f500410b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:13.905 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.885+0000 7f5009222700 1 -- 172.21.15.52:0/971365113 shutdown_connections 2023-11-27T15:23:13.905 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:13.885+0000 7f5009222700 1 -- 172.21.15.52:0/971365113 wait complete. 2023-11-27T15:23:14.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:14 smithi150 ceph-mon[100559]: pgmap v162: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 6 op/s 2023-11-27T15:23:14.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:14 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3058370988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:14.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:14 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:14.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:14 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:23:14.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:14 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:23:14.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:14 smithi102 ceph-mon[100296]: pgmap v162: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 6 op/s 2023-11-27T15:23:14.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:14 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3058370988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:14.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:14 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:14.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:14 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:23:14.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:14 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:23:14.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:14 smithi052 ceph-mon[99056]: pgmap v162: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 6 op/s 2023-11-27T15:23:14.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:14 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3058370988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:14.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:14 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:14.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:14 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:23:14.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:14 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:23:14.906 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-27T15:23:14.906 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-27T15:23:15.052 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.047+0000 7f2f3f863700 1 -- 172.21.15.52:0/4209515200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f380ff990 msgr2=0x7f2f380ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:15.052 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.047+0000 7f2f3f863700 1 --2- 172.21.15.52:0/4209515200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f380ff990 0x7f2f380ffe00 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2f280099c0 tx=0x7f2f28009cd0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.053 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.047+0000 7f2f3f863700 1 -- 172.21.15.52:0/4209515200 shutdown_connections 2023-11-27T15:23:15.053 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.047+0000 7f2f3f863700 1 --2- 172.21.15.52:0/4209515200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f38104180 0x7f2f38104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.053 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.047+0000 7f2f3f863700 1 --2- 172.21.15.52:0/4209515200 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f38100340 0x7f2f3810c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.053 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.047+0000 7f2f3f863700 1 --2- 172.21.15.52:0/4209515200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f380ff990 0x7f2f380ffe00 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.054 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.047+0000 7f2f3f863700 1 -- 172.21.15.52:0/4209515200 >> 172.21.15.52:0/4209515200 conn(0x7f2f38075920 msgr2=0x7f2f38075d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:15.054 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.047+0000 7f2f3f863700 1 -- 172.21.15.52:0/4209515200 shutdown_connections 2023-11-27T15:23:15.054 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.047+0000 7f2f3f863700 1 -- 172.21.15.52:0/4209515200 wait complete. 2023-11-27T15:23:15.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.048+0000 7f2f3f863700 1 Processor -- start 2023-11-27T15:23:15.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.048+0000 7f2f3f863700 1 -- start start 2023-11-27T15:23:15.057 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.048+0000 7f2f3f863700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f380ff990 0x7f2f38195390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.057 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.048+0000 7f2f3f863700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f38100340 0x7f2f381958d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.057 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.048+0000 7f2f3f863700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f38104180 0x7f2f38199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.057 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.048+0000 7f2f3f863700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3819a370 con 0x7f2f380ff990 2023-11-27T15:23:15.058 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.048+0000 7f2f3f863700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3819a4e0 con 0x7f2f38100340 2023-11-27T15:23:15.058 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.048+0000 7f2f3f863700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3819a6a0 con 0x7f2f38104180 2023-11-27T15:23:15.058 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.048+0000 7f2f3d5ff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f380ff990 0x7f2f38195390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.058 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.048+0000 7f2f3d5ff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f380ff990 0x7f2f38195390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54014/0 (socket says 172.21.15.52:54014) 2023-11-27T15:23:15.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.048+0000 7f2f3d5ff700 1 -- 172.21.15.52:0/4109480274 learned_addr learned my addr 172.21.15.52:0/4109480274 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:15.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.048+0000 7f2f3de00700 1 --2- 172.21.15.52:0/4109480274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f38104180 0x7f2f38199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.048+0000 7f2f3cdfe700 1 --2- 172.21.15.52:0/4109480274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f38100340 0x7f2f381958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.049+0000 7f2f3d5ff700 1 -- 172.21.15.52:0/4109480274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f38104180 msgr2=0x7f2f38199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:15.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.049+0000 7f2f3d5ff700 1 --2- 172.21.15.52:0/4109480274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f38104180 0x7f2f38199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.060 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.049+0000 7f2f3d5ff700 1 -- 172.21.15.52:0/4109480274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f38100340 msgr2=0x7f2f381958d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:15.060 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.049+0000 7f2f3d5ff700 1 --2- 172.21.15.52:0/4109480274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f38100340 0x7f2f381958d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.060 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.049+0000 7f2f3d5ff700 1 -- 172.21.15.52:0/4109480274 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f280095f0 con 0x7f2f380ff990 2023-11-27T15:23:15.060 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.049+0000 7f2f3cdfe700 1 --2- 172.21.15.52:0/4109480274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f38100340 0x7f2f381958d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:15.061 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.049+0000 7f2f3d5ff700 1 --2- 172.21.15.52:0/4109480274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f380ff990 0x7f2f38195390 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f2f3400d950 tx=0x7f2f3400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:15.061 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.049+0000 7f2f2e7fc700 1 -- 172.21.15.52:0/4109480274 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34009940 con 0x7f2f380ff990 2023-11-27T15:23:15.061 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.049+0000 7f2f2e7fc700 1 -- 172.21.15.52:0/4109480274 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f34010460 con 0x7f2f380ff990 2023-11-27T15:23:15.061 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.049+0000 7f2f3f863700 1 -- 172.21.15.52:0/4109480274 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f3810f210 con 0x7f2f380ff990 2023-11-27T15:23:15.061 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.049+0000 7f2f2e7fc700 1 -- 172.21.15.52:0/4109480274 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3400f620 con 0x7f2f380ff990 2023-11-27T15:23:15.062 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.049+0000 7f2f3f863700 1 -- 172.21.15.52:0/4109480274 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3810f080 con 0x7f2f380ff990 2023-11-27T15:23:15.062 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.051+0000 7f2f2e7fc700 1 -- 172.21.15.52:0/4109480274 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f2f340108d0 con 0x7f2f380ff990 2023-11-27T15:23:15.067 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.052+0000 7f2f2e7fc700 1 --2- 172.21.15.52:0/4109480274 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f2f240a12c0 0x7f2f240a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.067 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.052+0000 7f2f2e7fc700 1 -- 172.21.15.52:0/4109480274 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2f340ccb80 con 0x7f2f380ff990 2023-11-27T15:23:15.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.052+0000 7f2f3cdfe700 1 --2- 172.21.15.52:0/4109480274 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f2f240a12c0 0x7f2f240a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.053+0000 7f2f3cdfe700 1 --2- 172.21.15.52:0/4109480274 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f2f240a12c0 0x7f2f240a3770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2f381969a0 tx=0x7f2f28005fb0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:15.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.053+0000 7f2f4b9efb80 1 Processor -- start 2023-11-27T15:23:15.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.053+0000 7f2f4b9efb80 1 -- start start 2023-11-27T15:23:15.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.053+0000 7f2f4b9efb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ff3ae9be0 0x558ff3ae9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.053+0000 7f2f4b9efb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ff3aea4f0 0x558ff3aee3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.053+0000 7f2f4b9efb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff3aee930 0x558ff3aeed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.053+0000 7f2f4b9efb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ff39ef0e0 con 0x558ff3ae9be0 2023-11-27T15:23:15.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.053+0000 7f2f4b9efb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ff3a41920 con 0x558ff3aea4f0 2023-11-27T15:23:15.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.053+0000 7f2f4b9efb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ff3a3adb0 con 0x558ff3aee930 2023-11-27T15:23:15.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.053+0000 7f2f3d5ff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ff3ae9be0 0x558ff3ae9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.053+0000 7f2f3d5ff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ff3ae9be0 0x558ff3ae9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54022/0 (socket says 172.21.15.52:54022) 2023-11-27T15:23:15.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.053+0000 7f2f3d5ff700 1 -- 172.21.15.52:0/27348091 learned_addr learned my addr 172.21.15.52:0/27348091 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:15.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.053+0000 7f2f3de00700 1 --2- 172.21.15.52:0/27348091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff3aee930 0x558ff3aeed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.053+0000 7f2f3cdfe700 1 --2- 172.21.15.52:0/27348091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ff3aea4f0 0x558ff3aee3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.053+0000 7f2f3d5ff700 1 -- 172.21.15.52:0/27348091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff3aee930 msgr2=0x558ff3aeed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:15.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.053+0000 7f2f3d5ff700 1 --2- 172.21.15.52:0/27348091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff3aee930 0x558ff3aeed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.053+0000 7f2f3d5ff700 1 -- 172.21.15.52:0/27348091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ff3aea4f0 msgr2=0x558ff3aee3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:15.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.053+0000 7f2f3d5ff700 1 --2- 172.21.15.52:0/27348091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ff3aea4f0 0x558ff3aee3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.053+0000 7f2f3d5ff700 1 -- 172.21.15.52:0/27348091 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ff38bc6b0 con 0x558ff3ae9be0 2023-11-27T15:23:15.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.053+0000 7f2f3de00700 1 --2- 172.21.15.52:0/27348091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff3aee930 0x558ff3aeed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:15.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.054+0000 7f2f3d5ff700 1 --2- 172.21.15.52:0/27348091 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ff3ae9be0 0x558ff3ae9fb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2f34096660 tx=0x7f2f34099620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f25967c04df349df server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:15.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.054+0000 7f2f237fe700 1 -- 172.21.15.52:0/27348091 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34096300 con 0x558ff3ae9be0 2023-11-27T15:23:15.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.054+0000 7f2f237fe700 1 -- 172.21.15.52:0/27348091 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f34096460 con 0x558ff3ae9be0 2023-11-27T15:23:15.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.054+0000 7f2f237fe700 1 -- 172.21.15.52:0/27348091 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f340a1bb0 con 0x558ff3ae9be0 2023-11-27T15:23:15.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.054+0000 7f2f4b9efb80 1 -- 172.21.15.52:0/27348091 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ff3ae9be0 msgr2=0x558ff3ae9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:15.074 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.054+0000 7f2f4b9efb80 1 --2- 172.21.15.52:0/27348091 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ff3ae9be0 0x558ff3ae9fb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2f34096660 tx=0x7f2f34099620 comp rx=0 tx=0).stop 2023-11-27T15:23:15.074 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.055+0000 7f2f4b9efb80 1 -- 172.21.15.52:0/27348091 shutdown_connections 2023-11-27T15:23:15.074 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.055+0000 7f2f4b9efb80 1 --2- 172.21.15.52:0/27348091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff3aee930 0x558ff3aeed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.074 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.055+0000 7f2f4b9efb80 1 --2- 172.21.15.52:0/27348091 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ff3ae9be0 0x558ff3ae9fb0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.055+0000 7f2f4b9efb80 1 --2- 172.21.15.52:0/27348091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ff3aea4f0 0x558ff3aee3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.055+0000 7f2f4b9efb80 1 -- 172.21.15.52:0/27348091 >> 172.21.15.52:0/27348091 conn(0x558ff3adf3e0 msgr2=0x558ff3ae4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:15.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.055+0000 7f2f4b9efb80 1 -- 172.21.15.52:0/27348091 shutdown_connections 2023-11-27T15:23:15.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.055+0000 7f2f4b9efb80 1 -- 172.21.15.52:0/27348091 wait complete. 2023-11-27T15:23:15.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.055+0000 7f2f4b9efb80 1 -- start start 2023-11-27T15:23:15.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.055+0000 7f2f4b9efb80 1 Processor -- start 2023-11-27T15:23:15.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.055+0000 7f2f4b9efb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ff3ae9be0 0x558ff3af34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.055+0000 7f2f4b9efb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ff3aea4f0 0x558ff3af39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.055+0000 7f2f4b9efb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff3aee930 0x558ff3af3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.055+0000 7f2f4b9efb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ff39dc960 con 0x558ff3aea4f0 2023-11-27T15:23:15.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.055+0000 7f2f4b9efb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ff39dc600 con 0x558ff3ae9be0 2023-11-27T15:23:15.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.055+0000 7f2f4b9efb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ff38be9a0 con 0x558ff3aee930 2023-11-27T15:23:15.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.056+0000 7f2f4b9efb80 10 client.0 _ll_register_callbacks cb 0x558ff39fa800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:15.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.056+0000 7f2f3cdfe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ff3aea4f0 0x558ff3af39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.078 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.056+0000 7f2f3cdfe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ff3aea4f0 0x558ff3af39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54030/0 (socket says 172.21.15.52:54030) 2023-11-27T15:23:15.078 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.056+0000 7f2f3cdfe700 1 -- 172.21.15.52:0/2071446670 learned_addr learned my addr 172.21.15.52:0/2071446670 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:15.078 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.056+0000 7f2f3d5ff700 1 --2- 172.21.15.52:0/2071446670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ff3ae9be0 0x558ff3af34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.078 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.056+0000 7f2f3cdfe700 1 -- 172.21.15.52:0/2071446670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff3aee930 msgr2=0x558ff3af3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:15.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.056+0000 7f2f3cdfe700 1 --2- 172.21.15.52:0/2071446670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff3aee930 0x558ff3af3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.056+0000 7f2f3cdfe700 1 -- 172.21.15.52:0/2071446670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ff3ae9be0 msgr2=0x558ff3af34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:15.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.056+0000 7f2f3cdfe700 1 --2- 172.21.15.52:0/2071446670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ff3ae9be0 0x558ff3af34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.056+0000 7f2f3cdfe700 1 -- 172.21.15.52:0/2071446670 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3810f080 con 0x558ff3aea4f0 2023-11-27T15:23:15.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.056+0000 7f2f3cdfe700 1 --2- 172.21.15.52:0/2071446670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ff3aea4f0 0x558ff3af39e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2f280099c0 tx=0x7f2f28009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844fa78101bfc18d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:15.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.056+0000 7f2f20ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:23:15.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.056+0000 7f2f20ff9700 1 -- 172.21.15.52:0/2071446670 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f280040b0 con 0x558ff3aea4f0 2023-11-27T15:23:15.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.056+0000 7f2f20ff9700 1 -- 172.21.15.52:0/2071446670 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2f2800f460 con 0x558ff3aea4f0 2023-11-27T15:23:15.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.056+0000 7f2f4b9efb80 1 -- 172.21.15.52:0/2071446670 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ff3adcba0 con 0x558ff3aea4f0 2023-11-27T15:23:15.081 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.056+0000 7f2f20ff9700 1 -- 172.21.15.52:0/2071446670 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f28004210 con 0x558ff3aea4f0 2023-11-27T15:23:15.081 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.056+0000 7f2f4b9efb80 10 client.16245 fetch_fsmap learned FSMap version 28 2023-11-27T15:23:15.081 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.057+0000 7f2f4b9efb80 1 -- 172.21.15.52:0/2071446670 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558ff3a01210 con 0x558ff3aea4f0 2023-11-27T15:23:15.081 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.057+0000 7f2f20ff9700 1 -- 172.21.15.52:0/2071446670 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f2803d020 con 0x558ff3aea4f0 2023-11-27T15:23:15.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.057+0000 7f2f20ff9700 1 -- 172.21.15.52:0/2071446670 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2f28024610 con 0x558ff3aea4f0 2023-11-27T15:23:15.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.057+0000 7f2f4b9efb80 10 client.16245 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:23:15.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.057+0000 7f2f4b9efb80 10 client.16245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:15.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.057+0000 7f2f4b9efb80 20 client.16245 populate_metadata read hostname 'smithi052' 2023-11-27T15:23:15.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.057+0000 7f2f4b9efb80 1 --2- 172.21.15.52:0/2071446670 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x558ff3b03a80 0x558ff3b05e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.083 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.057+0000 7f2f4b9efb80 4 client.16245 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:23:15.083 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.057+0000 7f2f4b9efb80 1 -- 172.21.15.52:0/2071446670 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ff39fd120 con 0x558ff3b03a80 2023-11-27T15:23:15.083 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.057+0000 7f2f3d5ff700 1 --2- 172.21.15.52:0/2071446670 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x558ff3b03a80 0x558ff3b05e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.083 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.058+0000 7f2f20ff9700 10 client.16245 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:15.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.058+0000 7f2f3d5ff700 1 --2- 172.21.15.52:0/2071446670 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x558ff3b03a80 0x558ff3b05e90 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e52acfba26aff6f8 server_cookie=6274f7ce509f773 in_seq=0 out_seq=0 2023-11-27T15:23:15.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.058+0000 7f2f20ff9700 1 -- 172.21.15.52:0/2071446670 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558ff39fd120 con 0x558ff3b03a80 2023-11-27T15:23:15.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.058+0000 7f2f20ff9700 10 client.16245 handle_command_reply: tid=0 2023-11-27T15:23:15.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.059+0000 7f2f4b9efb80 1 client.16245 shutdown 2023-11-27T15:23:15.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.059+0000 7f2f4b9efb80 1 -- 172.21.15.52:0/2071446670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ff3aea4f0 msgr2=0x558ff3af39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:15.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.059+0000 7f2f4b9efb80 1 --2- 172.21.15.52:0/2071446670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ff3aea4f0 0x558ff3af39e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2f280099c0 tx=0x7f2f28009cd0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.059+0000 7f2f4b9efb80 1 -- 172.21.15.52:0/2071446670 shutdown_connections 2023-11-27T15:23:15.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.059+0000 7f2f4b9efb80 1 --2- 172.21.15.52:0/2071446670 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x558ff3b03a80 0x558ff3b05e90 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.059+0000 7f2f4b9efb80 1 --2- 172.21.15.52:0/2071446670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ff3ae9be0 0x558ff3af34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.059+0000 7f2f20ff9700 0 client.16245 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:15.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.059+0000 7f2f4b9efb80 1 --2- 172.21.15.52:0/2071446670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff3aee930 0x558ff3af3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.059+0000 7f2f4b9efb80 1 --2- 172.21.15.52:0/2071446670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ff3aea4f0 0x558ff3af39e0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.059+0000 7f2f4b9efb80 1 -- 172.21.15.52:0/2071446670 >> 172.21.15.52:0/2071446670 conn(0x558ff3adf3e0 msgr2=0x558ff3aefde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:15.087 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.059+0000 7f2f4b9efb80 1 -- 172.21.15.52:0/2071446670 shutdown_connections 2023-11-27T15:23:15.087 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.059+0000 7f2f4b9efb80 1 -- 172.21.15.52:0/2071446670 wait complete. 2023-11-27T15:23:15.087 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.059+0000 7f2f4b9efb80 20 client.16245 trim_cache size 0 max 16384 2023-11-27T15:23:15.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.076+0000 7f2f4b9efb80 1 Processor -- start 2023-11-27T15:23:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.076+0000 7f2f4b9efb80 1 -- start start 2023-11-27T15:23:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.076+0000 7f2f4b9efb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ff3ae9be0 0x558ff3aec220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.076+0000 7f2f4b9efb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ff3aea4f0 0x558ff3afa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.076+0000 7f2f4b9efb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff3aee930 0x558ff3aee140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.076+0000 7f2f4b9efb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ff3aef1a0 con 0x558ff3aea4f0 2023-11-27T15:23:15.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.076+0000 7f2f4b9efb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ff3aea980 con 0x558ff3ae9be0 2023-11-27T15:23:15.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.076+0000 7f2f4b9efb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ff3afaf10 con 0x558ff3aee930 2023-11-27T15:23:15.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.076+0000 7f2f3cdfe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ff3aea4f0 0x558ff3afa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.076+0000 7f2f3cdfe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ff3aea4f0 0x558ff3afa9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54038/0 (socket says 172.21.15.52:54038) 2023-11-27T15:23:15.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.076+0000 7f2f3cdfe700 1 -- 172.21.15.52:0/490405179 learned_addr learned my addr 172.21.15.52:0/490405179 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:15.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.077+0000 7f2f3d5ff700 1 --2- 172.21.15.52:0/490405179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ff3ae9be0 0x558ff3aec220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.077+0000 7f2f3de00700 1 --2- 172.21.15.52:0/490405179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff3aee930 0x558ff3aee140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.077+0000 7f2f3cdfe700 1 -- 172.21.15.52:0/490405179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff3aee930 msgr2=0x558ff3aee140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:15.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.077+0000 7f2f3cdfe700 1 --2- 172.21.15.52:0/490405179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff3aee930 0x558ff3aee140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.096 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.077+0000 7f2f3cdfe700 1 -- 172.21.15.52:0/490405179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ff3ae9be0 msgr2=0x558ff3aec220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:15.096 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.077+0000 7f2f3cdfe700 1 --2- 172.21.15.52:0/490405179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ff3ae9be0 0x558ff3aec220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.096 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.077+0000 7f2f3cdfe700 1 -- 172.21.15.52:0/490405179 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ff3ae62c0 con 0x558ff3aea4f0 2023-11-27T15:23:15.096 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.077+0000 7f2f3d5ff700 1 --2- 172.21.15.52:0/490405179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ff3ae9be0 0x558ff3aec220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:15.097 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.077+0000 7f2f3cdfe700 1 --2- 172.21.15.52:0/490405179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ff3aea4f0 0x558ff3afa9d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2f28024a40 tx=0x7f2f28009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea10d13e441c419 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:15.097 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.077+0000 7f2f23fff700 1 -- 172.21.15.52:0/490405179 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f28024410 con 0x558ff3aea4f0 2023-11-27T15:23:15.097 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.077+0000 7f2f23fff700 1 -- 172.21.15.52:0/490405179 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f2803d030 con 0x558ff3aea4f0 2023-11-27T15:23:15.097 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.077+0000 7f2f23fff700 1 -- 172.21.15.52:0/490405179 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f280429e0 con 0x558ff3aea4f0 2023-11-27T15:23:15.098 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.078+0000 7f2f4b9efb80 1 -- 172.21.15.52:0/490405179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ff3aea4f0 msgr2=0x558ff3afa9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:15.098 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.078+0000 7f2f4b9efb80 1 --2- 172.21.15.52:0/490405179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ff3aea4f0 0x558ff3afa9d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2f28024a40 tx=0x7f2f28009310 comp rx=0 tx=0).stop 2023-11-27T15:23:15.098 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.078+0000 7f2f4b9efb80 1 -- 172.21.15.52:0/490405179 shutdown_connections 2023-11-27T15:23:15.098 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.078+0000 7f2f4b9efb80 1 --2- 172.21.15.52:0/490405179 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ff3ae9be0 0x558ff3aec220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.098 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.078+0000 7f2f4b9efb80 1 --2- 172.21.15.52:0/490405179 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff3aee930 0x558ff3aee140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.099 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.078+0000 7f2f4b9efb80 1 --2- 172.21.15.52:0/490405179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ff3aea4f0 0x558ff3afa9d0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.099 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.078+0000 7f2f4b9efb80 1 -- 172.21.15.52:0/490405179 >> 172.21.15.52:0/490405179 conn(0x558ff3adf3e0 msgr2=0x558ff3aeda30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:15.099 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.078+0000 7f2f4b9efb80 1 -- 172.21.15.52:0/490405179 shutdown_connections 2023-11-27T15:23:15.099 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.078+0000 7f2f4b9efb80 1 -- 172.21.15.52:0/490405179 wait complete. 2023-11-27T15:23:15.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.078+0000 7f2f4b9efb80 1 -- start start 2023-11-27T15:23:15.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.078+0000 7f2f4b9efb80 1 Processor -- start 2023-11-27T15:23:15.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.079+0000 7f2f4b9efb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff3ae9be0 0x558ff3b55e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.079+0000 7f2f4b9efb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ff3aea4f0 0x558ff3b56360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.079+0000 7f2f4b9efb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ff3aee930 0x558ff3b501c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.079+0000 7f2f4b9efb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ff3b507a0 con 0x558ff3aea4f0 2023-11-27T15:23:15.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.079+0000 7f2f4b9efb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ff3b50910 con 0x558ff3aee930 2023-11-27T15:23:15.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.079+0000 7f2f4b9efb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ff3b50ad0 con 0x558ff3ae9be0 2023-11-27T15:23:15.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.079+0000 7f2f4b9efb80 10 client.0 _ll_register_callbacks cb 0x558ff39fa800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:15.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.079+0000 7f2f3cdfe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ff3aea4f0 0x558ff3b56360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.079+0000 7f2f3cdfe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ff3aea4f0 0x558ff3b56360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54044/0 (socket says 172.21.15.52:54044) 2023-11-27T15:23:15.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.079+0000 7f2f3cdfe700 1 -- 172.21.15.52:0/1564129854 learned_addr learned my addr 172.21.15.52:0/1564129854 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:15.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.079+0000 7f2f3d5ff700 1 --2- 172.21.15.52:0/1564129854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff3ae9be0 0x558ff3b55e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.079+0000 7f2f3de00700 1 --2- 172.21.15.52:0/1564129854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ff3aee930 0x558ff3b501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.079+0000 7f2f3cdfe700 1 -- 172.21.15.52:0/1564129854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff3ae9be0 msgr2=0x558ff3b55e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:15.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.079+0000 7f2f3cdfe700 1 --2- 172.21.15.52:0/1564129854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff3ae9be0 0x558ff3b55e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.079+0000 7f2f3cdfe700 1 -- 172.21.15.52:0/1564129854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ff3aee930 msgr2=0x558ff3b501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:15.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.079+0000 7f2f3cdfe700 1 --2- 172.21.15.52:0/1564129854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ff3aee930 0x558ff3b501c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.079+0000 7f2f3cdfe700 1 -- 172.21.15.52:0/1564129854 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ff3a01210 con 0x558ff3aea4f0 2023-11-27T15:23:15.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.079+0000 7f2f3de00700 1 --2- 172.21.15.52:0/1564129854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ff3aee930 0x558ff3b501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:15.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.079+0000 7f2f3cdfe700 1 --2- 172.21.15.52:0/1564129854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ff3aea4f0 0x558ff3b56360 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2f280099c0 tx=0x7f2f28024e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c53fa3ee95fd80 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:15.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.079+0000 7f2f20ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:23:15.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.079+0000 7f2f20ff9700 1 -- 172.21.15.52:0/1564129854 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f280040b0 con 0x558ff3aea4f0 2023-11-27T15:23:15.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.079+0000 7f2f20ff9700 1 -- 172.21.15.52:0/1564129854 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f2f2803e080 con 0x558ff3aea4f0 2023-11-27T15:23:15.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.079+0000 7f2f4b9efb80 1 -- 172.21.15.52:0/1564129854 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f34096460 con 0x558ff3aea4f0 2023-11-27T15:23:15.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.079+0000 7f2f20ff9700 1 -- 172.21.15.52:0/1564129854 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f28004210 con 0x558ff3aea4f0 2023-11-27T15:23:15.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.080+0000 7f2f4b9efb80 10 client.16251 fetch_fsmap learned FSMap version 28 2023-11-27T15:23:15.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.080+0000 7f2f4b9efb80 1 -- 172.21.15.52:0/1564129854 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558ff3868e60 con 0x558ff3aea4f0 2023-11-27T15:23:15.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.080+0000 7f2f20ff9700 1 -- 172.21.15.52:0/1564129854 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f34096460 con 0x558ff3aea4f0 2023-11-27T15:23:15.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.080+0000 7f2f20ff9700 1 -- 172.21.15.52:0/1564129854 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2f2800f690 con 0x558ff3aea4f0 2023-11-27T15:23:15.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.080+0000 7f2f4b9efb80 10 client.16251 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:23:15.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.080+0000 7f2f4b9efb80 10 client.16251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:15.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.080+0000 7f2f4b9efb80 20 client.16251 populate_metadata read hostname 'smithi052' 2023-11-27T15:23:15.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.080+0000 7f2f4b9efb80 1 --2- 172.21.15.52:0/1564129854 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x558ff3b5d950 0x558ff3b5fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.080+0000 7f2f4b9efb80 4 client.16251 mds_command: new command op to 24536 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-27T15:23:15.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.080+0000 7f2f4b9efb80 1 -- 172.21.15.52:0/1564129854 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x558ff3a096e0 con 0x558ff3b5d950 2023-11-27T15:23:15.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.080+0000 7f2f3d5ff700 1 --2- 172.21.15.52:0/1564129854 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x558ff3b5d950 0x558ff3b5fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.081+0000 7f2f20ff9700 10 client.16251 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:15.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.081+0000 7f2f3d5ff700 1 --2- 172.21.15.52:0/1564129854 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x558ff3b5d950 0x558ff3b5fd60 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=410603c0dfa9e070 server_cookie=a7fcfe8e0566f88d in_seq=0 out_seq=0 2023-11-27T15:23:15.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.082+0000 7f2f20ff9700 1 -- 172.21.15.52:0/1564129854 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558ff3a096e0 con 0x558ff3b5d950 2023-11-27T15:23:15.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.082+0000 7f2f20ff9700 10 client.16251 handle_command_reply: tid=0 2023-11-27T15:23:15.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.082+0000 7f2f4b9efb80 1 client.16251 shutdown 2023-11-27T15:23:15.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.082+0000 7f2f4b9efb80 1 -- 172.21.15.52:0/1564129854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ff3aea4f0 msgr2=0x558ff3b56360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:15.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.082+0000 7f2f4b9efb80 1 --2- 172.21.15.52:0/1564129854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ff3aea4f0 0x558ff3b56360 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2f280099c0 tx=0x7f2f28024e60 comp rx=0 tx=0).stop 2023-11-27T15:23:15.106 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.082+0000 7f2f4b9efb80 1 -- 172.21.15.52:0/1564129854 shutdown_connections 2023-11-27T15:23:15.106 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.082+0000 7f2f4b9efb80 1 --2- 172.21.15.52:0/1564129854 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x558ff3b5d950 0x558ff3b5fd60 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.106 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.082+0000 7f2f4b9efb80 1 --2- 172.21.15.52:0/1564129854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ff3aee930 0x558ff3b501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.106 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.082+0000 7f2f20ff9700 0 client.16251 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:15.106 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.082+0000 7f2f4b9efb80 1 --2- 172.21.15.52:0/1564129854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558ff3ae9be0 0x558ff3b55e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.106 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.082+0000 7f2f4b9efb80 1 --2- 172.21.15.52:0/1564129854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ff3aea4f0 0x558ff3b56360 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.082+0000 7f2f4b9efb80 1 -- 172.21.15.52:0/1564129854 >> 172.21.15.52:0/1564129854 conn(0x558ff3adf3e0 msgr2=0x558ff3ae89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:15.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.082+0000 7f2f4b9efb80 1 -- 172.21.15.52:0/1564129854 shutdown_connections 2023-11-27T15:23:15.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.082+0000 7f2f4b9efb80 1 -- 172.21.15.52:0/1564129854 wait complete. 2023-11-27T15:23:15.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.082+0000 7f2f4b9efb80 20 client.16251 trim_cache size 0 max 16384 2023-11-27T15:23:15.107 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:23:15.107 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2023-11-27T15:23:15.108 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "567b1959-c433-4da8-8bc2-2847ea622f57", 2023-11-27T15:23:15.108 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2023-11-27T15:23:15.108 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:23:15.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.083+0000 7f2f3f863700 1 -- 172.21.15.52:0/4109480274 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f2f240a12c0 msgr2=0x7f2f240a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:15.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.083+0000 7f2f3f863700 1 --2- 172.21.15.52:0/4109480274 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f2f240a12c0 0x7f2f240a3770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2f381969a0 tx=0x7f2f28005fb0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.083+0000 7f2f3f863700 1 -- 172.21.15.52:0/4109480274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f380ff990 msgr2=0x7f2f38195390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:15.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.083+0000 7f2f3f863700 1 --2- 172.21.15.52:0/4109480274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f380ff990 0x7f2f38195390 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f2f3400d950 tx=0x7f2f3400dc60 comp rx=0 tx=0).stop 2023-11-27T15:23:15.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.083+0000 7f2f3f863700 1 -- 172.21.15.52:0/4109480274 shutdown_connections 2023-11-27T15:23:15.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.083+0000 7f2f3f863700 1 --2- 172.21.15.52:0/4109480274 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f2f240a12c0 0x7f2f240a3770 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.083+0000 7f2f3f863700 1 --2- 172.21.15.52:0/4109480274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f38104180 0x7f2f38199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.083+0000 7f2f3f863700 1 --2- 172.21.15.52:0/4109480274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f380ff990 0x7f2f38195390 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.083+0000 7f2f3f863700 1 --2- 172.21.15.52:0/4109480274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f38100340 0x7f2f381958d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.083+0000 7f2f3f863700 1 -- 172.21.15.52:0/4109480274 >> 172.21.15.52:0/4109480274 conn(0x7f2f38075920 msgr2=0x7f2f380fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:15.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.083+0000 7f2f3f863700 1 -- 172.21.15.52:0/4109480274 shutdown_connections 2023-11-27T15:23:15.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.083+0000 7f2f3f863700 1 -- 172.21.15.52:0/4109480274 wait complete. 2023-11-27T15:23:15.111 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:23:15.240 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.235+0000 7f9b2fd87700 1 -- 172.21.15.52:0/723246046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b280ffdf0 msgr2=0x7f9b28103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:15.240 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.235+0000 7f9b2fd87700 1 --2- 172.21.15.52:0/723246046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b280ffdf0 0x7f9b28103db0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9b240099c0 tx=0x7f9b24009cd0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.240 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.235+0000 7f9b2fd87700 1 -- 172.21.15.52:0/723246046 shutdown_connections 2023-11-27T15:23:15.240 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.235+0000 7f9b2fd87700 1 --2- 172.21.15.52:0/723246046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b280ff450 0x7f9b280ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.240 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.235+0000 7f9b2fd87700 1 --2- 172.21.15.52:0/723246046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b28104490 0x7f9b2810c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.241 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.235+0000 7f9b2fd87700 1 --2- 172.21.15.52:0/723246046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b280ffdf0 0x7f9b28103db0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.241 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.235+0000 7f9b2fd87700 1 -- 172.21.15.52:0/723246046 >> 172.21.15.52:0/723246046 conn(0x7f9b280770b0 msgr2=0x7f9b280774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:15.241 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.235+0000 7f9b2fd87700 1 -- 172.21.15.52:0/723246046 shutdown_connections 2023-11-27T15:23:15.241 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.235+0000 7f9b2fd87700 1 -- 172.21.15.52:0/723246046 wait complete. 2023-11-27T15:23:15.243 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.236+0000 7f9b2fd87700 1 Processor -- start 2023-11-27T15:23:15.243 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.236+0000 7f9b2fd87700 1 -- start start 2023-11-27T15:23:15.243 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.236+0000 7f9b2fd87700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b280ff450 0x7f9b28198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.243 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.236+0000 7f9b2fd87700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b280ffdf0 0x7f9b28198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.243 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.236+0000 7f9b2fd87700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b28104490 0x7f9b2819d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.243 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.236+0000 7f9b2fd87700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2819d650 con 0x7f9b28104490 2023-11-27T15:23:15.243 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.236+0000 7f9b2fd87700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2819d7c0 con 0x7f9b280ffdf0 2023-11-27T15:23:15.243 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.236+0000 7f9b2fd87700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2819d980 con 0x7f9b280ff450 2023-11-27T15:23:15.244 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.236+0000 7f9b2e324700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b28104490 0x7f9b2819d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.244 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.236+0000 7f9b2db23700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b280ff450 0x7f9b28198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.244 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.236+0000 7f9b2e324700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b28104490 0x7f9b2819d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54060/0 (socket says 172.21.15.52:54060) 2023-11-27T15:23:15.244 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.236+0000 7f9b2db23700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b280ff450 0x7f9b28198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.52:40700/0 (socket says 172.21.15.52:40700) 2023-11-27T15:23:15.244 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.236+0000 7f9b2e324700 1 -- 172.21.15.52:0/1221966069 learned_addr learned my addr 172.21.15.52:0/1221966069 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:15.244 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.236+0000 7f9b2d322700 1 --2- 172.21.15.52:0/1221966069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b280ffdf0 0x7f9b28198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.244 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.236+0000 7f9b2e324700 1 -- 172.21.15.52:0/1221966069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b280ff450 msgr2=0x7f9b28198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:15.244 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.236+0000 7f9b2e324700 1 --2- 172.21.15.52:0/1221966069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b280ff450 0x7f9b28198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.244 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.236+0000 7f9b2e324700 1 -- 172.21.15.52:0/1221966069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b280ffdf0 msgr2=0x7f9b28198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:15.245 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.236+0000 7f9b2e324700 1 --2- 172.21.15.52:0/1221966069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b280ffdf0 0x7f9b28198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.245 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.236+0000 7f9b2e324700 1 -- 172.21.15.52:0/1221966069 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b240095f0 con 0x7f9b28104490 2023-11-27T15:23:15.245 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.237+0000 7f9b2e324700 1 --2- 172.21.15.52:0/1221966069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b28104490 0x7f9b2819d010 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000ef30 tx=0x7f9b2000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:15.245 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.237+0000 7f9b1effd700 1 -- 172.21.15.52:0/1221966069 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b20018070 con 0x7f9b28104490 2023-11-27T15:23:15.245 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.237+0000 7f9b1effd700 1 -- 172.21.15.52:0/1221966069 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b2000f040 con 0x7f9b28104490 2023-11-27T15:23:15.245 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.237+0000 7f9b2fd87700 1 -- 172.21.15.52:0/1221966069 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b2810f260 con 0x7f9b28104490 2023-11-27T15:23:15.245 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.237+0000 7f9b1effd700 1 -- 172.21.15.52:0/1221966069 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2000cd00 con 0x7f9b28104490 2023-11-27T15:23:15.245 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.237+0000 7f9b2fd87700 1 -- 172.21.15.52:0/1221966069 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b2810f0d0 con 0x7f9b28104490 2023-11-27T15:23:15.250 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.239+0000 7f9b1effd700 1 -- 172.21.15.52:0/1221966069 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9b20003a00 con 0x7f9b28104490 2023-11-27T15:23:15.251 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.240+0000 7f9b1effd700 1 --2- 172.21.15.52:0/1221966069 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f9b140a1340 0x7f9b140a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.251 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.240+0000 7f9b2db23700 1 --2- 172.21.15.52:0/1221966069 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f9b140a1340 0x7f9b140a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.251 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.240+0000 7f9b1effd700 1 -- 172.21.15.52:0/1221966069 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f9b200cc7d0 con 0x7f9b28104490 2023-11-27T15:23:15.251 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.240+0000 7f9b2db23700 1 --2- 172.21.15.52:0/1221966069 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f9b140a1340 0x7f9b140a37f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9b18007be0 tx=0x7f9b180073d0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:15.251 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.240+0000 7f9b3bf13b80 1 Processor -- start 2023-11-27T15:23:15.251 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.241+0000 7f9b3bf13b80 1 -- start start 2023-11-27T15:23:15.251 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.241+0000 7f9b3bf13b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a4fc180b60 0x55a4fc180f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.241+0000 7f9b3bf13b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a4fc181470 0x55a4fc185370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.241+0000 7f9b3bf13b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4fc1858b0 0x55a4fc185d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.241+0000 7f9b3bf13b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4fc086060 con 0x55a4fc181470 2023-11-27T15:23:15.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.241+0000 7f9b3bf13b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4fc0d88a0 con 0x55a4fc180b60 2023-11-27T15:23:15.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.241+0000 7f9b3bf13b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4fc0d1d30 con 0x55a4fc1858b0 2023-11-27T15:23:15.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.241+0000 7f9b2d322700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a4fc181470 0x55a4fc185370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.241+0000 7f9b2d322700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a4fc181470 0x55a4fc185370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54066/0 (socket says 172.21.15.52:54066) 2023-11-27T15:23:15.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.241+0000 7f9b2d322700 1 -- 172.21.15.52:0/1976078675 learned_addr learned my addr 172.21.15.52:0/1976078675 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:15.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.241+0000 7f9b2e324700 1 --2- 172.21.15.52:0/1976078675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4fc1858b0 0x55a4fc185d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.241+0000 7f9b2db23700 1 --2- 172.21.15.52:0/1976078675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a4fc180b60 0x55a4fc180f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.241+0000 7f9b2d322700 1 -- 172.21.15.52:0/1976078675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4fc1858b0 msgr2=0x55a4fc185d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:15.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.241+0000 7f9b2d322700 1 --2- 172.21.15.52:0/1976078675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4fc1858b0 0x55a4fc185d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.241+0000 7f9b2d322700 1 -- 172.21.15.52:0/1976078675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a4fc180b60 msgr2=0x55a4fc180f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:15.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.241+0000 7f9b2d322700 1 --2- 172.21.15.52:0/1976078675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a4fc180b60 0x55a4fc180f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.241+0000 7f9b2d322700 1 -- 172.21.15.52:0/1976078675 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4fbf53630 con 0x55a4fc181470 2023-11-27T15:23:15.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.241+0000 7f9b2e324700 1 --2- 172.21.15.52:0/1976078675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4fc1858b0 0x55a4fc185d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:15.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.241+0000 7f9b2d322700 1 --2- 172.21.15.52:0/1976078675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a4fc181470 0x55a4fc185370 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f9b240098e0 tx=0x7f9b24005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63dde3ef69fd70c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:15.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.242+0000 7f9b13fff700 1 -- 172.21.15.52:0/1976078675 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b24026070 con 0x55a4fc181470 2023-11-27T15:23:15.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.242+0000 7f9b13fff700 1 -- 172.21.15.52:0/1976078675 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b2401e040 con 0x55a4fc181470 2023-11-27T15:23:15.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.242+0000 7f9b13fff700 1 -- 172.21.15.52:0/1976078675 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2400fa50 con 0x55a4fc181470 2023-11-27T15:23:15.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.242+0000 7f9b3bf13b80 1 -- 172.21.15.52:0/1976078675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a4fc181470 msgr2=0x55a4fc185370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:15.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.242+0000 7f9b3bf13b80 1 --2- 172.21.15.52:0/1976078675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a4fc181470 0x55a4fc185370 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f9b240098e0 tx=0x7f9b24005c70 comp rx=0 tx=0).stop 2023-11-27T15:23:15.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.242+0000 7f9b3bf13b80 1 -- 172.21.15.52:0/1976078675 shutdown_connections 2023-11-27T15:23:15.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.242+0000 7f9b3bf13b80 1 --2- 172.21.15.52:0/1976078675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a4fc180b60 0x55a4fc180f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.255 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.242+0000 7f9b3bf13b80 1 --2- 172.21.15.52:0/1976078675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4fc1858b0 0x55a4fc185d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.255 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.242+0000 7f9b3bf13b80 1 --2- 172.21.15.52:0/1976078675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a4fc181470 0x55a4fc185370 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.255 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.242+0000 7f9b3bf13b80 1 -- 172.21.15.52:0/1976078675 >> 172.21.15.52:0/1976078675 conn(0x55a4fc176360 msgr2=0x55a4fc17bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:15.255 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.243+0000 7f9b3bf13b80 1 -- 172.21.15.52:0/1976078675 shutdown_connections 2023-11-27T15:23:15.255 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.243+0000 7f9b3bf13b80 1 -- 172.21.15.52:0/1976078675 wait complete. 2023-11-27T15:23:15.255 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.243+0000 7f9b3bf13b80 1 -- start start 2023-11-27T15:23:15.255 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.243+0000 7f9b3bf13b80 1 Processor -- start 2023-11-27T15:23:15.255 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.243+0000 7f9b3bf13b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a4fc180b60 0x55a4fc18d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.255 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.243+0000 7f9b3bf13b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a4fc181470 0x55a4fc18a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.256 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.243+0000 7f9b3bf13b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4fc1858b0 0x55a4fc18a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.256 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.243+0000 7f9b3bf13b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4fc0738e0 con 0x55a4fc180b60 2023-11-27T15:23:15.256 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.243+0000 7f9b3bf13b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4fc073580 con 0x55a4fc181470 2023-11-27T15:23:15.256 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.243+0000 7f9b3bf13b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4fbf55920 con 0x55a4fc1858b0 2023-11-27T15:23:15.256 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.243+0000 7f9b3bf13b80 10 client.0 _ll_register_callbacks cb 0x55a4fc091780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:15.256 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.243+0000 7f9b2db23700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a4fc180b60 0x55a4fc18d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.256 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.244+0000 7f9b2db23700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a4fc180b60 0x55a4fc18d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54070/0 (socket says 172.21.15.52:54070) 2023-11-27T15:23:15.256 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.244+0000 7f9b2db23700 1 -- 172.21.15.52:0/1324722752 learned_addr learned my addr 172.21.15.52:0/1324722752 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:15.257 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.244+0000 7f9b2d322700 1 --2- 172.21.15.52:0/1324722752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a4fc181470 0x55a4fc18a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.257 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.244+0000 7f9b2e324700 1 --2- 172.21.15.52:0/1324722752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4fc1858b0 0x55a4fc18a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.257 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.244+0000 7f9b2db23700 1 -- 172.21.15.52:0/1324722752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4fc1858b0 msgr2=0x55a4fc18a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:15.257 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.244+0000 7f9b2db23700 1 --2- 172.21.15.52:0/1324722752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4fc1858b0 0x55a4fc18a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.257 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.244+0000 7f9b2db23700 1 -- 172.21.15.52:0/1324722752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a4fc181470 msgr2=0x55a4fc18a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:15.257 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.244+0000 7f9b2db23700 1 --2- 172.21.15.52:0/1324722752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a4fc181470 0x55a4fc18a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.257 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.244+0000 7f9b2db23700 1 -- 172.21.15.52:0/1324722752 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b2810c8a0 con 0x55a4fc180b60 2023-11-27T15:23:15.257 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.244+0000 7f9b2d322700 1 --2- 172.21.15.52:0/1324722752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a4fc181470 0x55a4fc18a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:15.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.244+0000 7f9b2db23700 1 --2- 172.21.15.52:0/1324722752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a4fc180b60 0x55a4fc18d020 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9b180042e0 tx=0x7f9b180047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcdc013b2c3cde5 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:15.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.244+0000 7f9b117fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:23:15.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.244+0000 7f9b117fa700 1 -- 172.21.15.52:0/1324722752 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b18003b10 con 0x55a4fc180b60 2023-11-27T15:23:15.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.244+0000 7f9b117fa700 1 -- 172.21.15.52:0/1324722752 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f9b18017070 con 0x55a4fc180b60 2023-11-27T15:23:15.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.244+0000 7f9b3bf13b80 1 -- 172.21.15.52:0/1324722752 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a4fc173b20 con 0x55a4fc180b60 2023-11-27T15:23:15.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.244+0000 7f9b117fa700 1 -- 172.21.15.52:0/1324722752 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b18003c70 con 0x55a4fc180b60 2023-11-27T15:23:15.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.244+0000 7f9b3bf13b80 10 client.16263 fetch_fsmap learned FSMap version 28 2023-11-27T15:23:15.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.244+0000 7f9b3bf13b80 1 -- 172.21.15.52:0/1324722752 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a4fc098190 con 0x55a4fc180b60 2023-11-27T15:23:15.259 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.245+0000 7f9b117fa700 1 -- 172.21.15.52:0/1324722752 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a4fc173b20 con 0x55a4fc180b60 2023-11-27T15:23:15.259 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.245+0000 7f9b117fa700 1 -- 172.21.15.52:0/1324722752 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9b18020410 con 0x55a4fc180b60 2023-11-27T15:23:15.259 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.245+0000 7f9b3bf13b80 10 client.16263 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:23:15.259 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.245+0000 7f9b3bf13b80 10 client.16263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:15.259 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.245+0000 7f9b3bf13b80 20 client.16263 populate_metadata read hostname 'smithi052' 2023-11-27T15:23:15.259 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.245+0000 7f9b3bf13b80 1 --2- 172.21.15.52:0/1324722752 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55a4fc19a8c0 0x55a4fc19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.259 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.245+0000 7f9b3bf13b80 4 client.16263 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:23:15.259 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.245+0000 7f9b3bf13b80 1 -- 172.21.15.52:0/1324722752 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a4fc0940a0 con 0x55a4fc19a8c0 2023-11-27T15:23:15.259 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.245+0000 7f9b2d322700 1 --2- 172.21.15.52:0/1324722752 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55a4fc19a8c0 0x55a4fc19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.260 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.246+0000 7f9b117fa700 10 client.16263 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:15.260 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.246+0000 7f9b2d322700 1 --2- 172.21.15.52:0/1324722752 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55a4fc19a8c0 0x55a4fc19ccd0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e09bb37599dab8fe server_cookie=3e54b1175b1ed35 in_seq=0 out_seq=0 2023-11-27T15:23:15.261 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.246+0000 7f9b117fa700 1 -- 172.21.15.52:0/1324722752 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a4fc0940a0 con 0x55a4fc19a8c0 2023-11-27T15:23:15.261 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.246+0000 7f9b117fa700 10 client.16263 handle_command_reply: tid=0 2023-11-27T15:23:15.261 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.246+0000 7f9b3bf13b80 1 client.16263 shutdown 2023-11-27T15:23:15.261 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.247+0000 7f9b3bf13b80 1 -- 172.21.15.52:0/1324722752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a4fc180b60 msgr2=0x55a4fc18d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:15.261 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.247+0000 7f9b3bf13b80 1 --2- 172.21.15.52:0/1324722752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a4fc180b60 0x55a4fc18d020 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9b180042e0 tx=0x7f9b180047c0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.261 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.247+0000 7f9b3bf13b80 1 -- 172.21.15.52:0/1324722752 shutdown_connections 2023-11-27T15:23:15.261 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.247+0000 7f9b3bf13b80 1 --2- 172.21.15.52:0/1324722752 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55a4fc19a8c0 0x55a4fc19ccd0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.247+0000 7f9b3bf13b80 1 --2- 172.21.15.52:0/1324722752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4fc1858b0 0x55a4fc18a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.247+0000 7f9b117fa700 0 client.16263 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:15.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.247+0000 7f9b3bf13b80 1 --2- 172.21.15.52:0/1324722752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a4fc180b60 0x55a4fc18d020 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.247+0000 7f9b3bf13b80 1 --2- 172.21.15.52:0/1324722752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a4fc181470 0x55a4fc18a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.247+0000 7f9b3bf13b80 1 -- 172.21.15.52:0/1324722752 >> 172.21.15.52:0/1324722752 conn(0x55a4fc176360 msgr2=0x55a4fc186d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:15.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.247+0000 7f9b3bf13b80 1 -- 172.21.15.52:0/1324722752 shutdown_connections 2023-11-27T15:23:15.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.247+0000 7f9b3bf13b80 1 -- 172.21.15.52:0/1324722752 wait complete. 2023-11-27T15:23:15.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.247+0000 7f9b3bf13b80 20 client.16263 trim_cache size 0 max 16384 2023-11-27T15:23:15.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.264+0000 7f9b3bf13b80 1 Processor -- start 2023-11-27T15:23:15.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.264+0000 7f9b3bf13b80 1 -- start start 2023-11-27T15:23:15.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.264+0000 7f9b3bf13b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a4fc180b60 0x55a4fc18cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.264+0000 7f9b3bf13b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4fc181470 0x55a4fc17f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.264+0000 7f9b3bf13b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a4fc1858b0 0x55a4fc17fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.264+0000 7f9b3bf13b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4fc181bc0 con 0x55a4fc1858b0 2023-11-27T15:23:15.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.264+0000 7f9b3bf13b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4fc175ee0 con 0x55a4fc180b60 2023-11-27T15:23:15.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.264+0000 7f9b3bf13b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4fc18b200 con 0x55a4fc181470 2023-11-27T15:23:15.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.264+0000 7f9b2e324700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a4fc1858b0 0x55a4fc17fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.264+0000 7f9b2e324700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a4fc1858b0 0x55a4fc17fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54082/0 (socket says 172.21.15.52:54082) 2023-11-27T15:23:15.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.264+0000 7f9b2e324700 1 -- 172.21.15.52:0/2717805038 learned_addr learned my addr 172.21.15.52:0/2717805038 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:15.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.264+0000 7f9b2db23700 1 --2- 172.21.15.52:0/2717805038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a4fc180b60 0x55a4fc18cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.264+0000 7f9b2d322700 1 --2- 172.21.15.52:0/2717805038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4fc181470 0x55a4fc17f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.264+0000 7f9b2e324700 1 -- 172.21.15.52:0/2717805038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4fc181470 msgr2=0x55a4fc17f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:15.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.264+0000 7f9b2e324700 1 --2- 172.21.15.52:0/2717805038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4fc181470 0x55a4fc17f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.264+0000 7f9b2e324700 1 -- 172.21.15.52:0/2717805038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a4fc180b60 msgr2=0x55a4fc18cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:15.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.264+0000 7f9b2e324700 1 --2- 172.21.15.52:0/2717805038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a4fc180b60 0x55a4fc18cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.264+0000 7f9b2e324700 1 -- 172.21.15.52:0/2717805038 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4fc097ee0 con 0x55a4fc1858b0 2023-11-27T15:23:15.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.264+0000 7f9b2e324700 1 --2- 172.21.15.52:0/2717805038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a4fc1858b0 0x55a4fc17fea0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9b20096340 tx=0x7f9b200994e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7203f4887b484627 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:15.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.265+0000 7f9b1cff9700 1 -- 172.21.15.52:0/2717805038 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2000ce60 con 0x55a4fc1858b0 2023-11-27T15:23:15.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.265+0000 7f9b1cff9700 1 -- 172.21.15.52:0/2717805038 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b200998d0 con 0x55a4fc1858b0 2023-11-27T15:23:15.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.265+0000 7f9b1cff9700 1 -- 172.21.15.52:0/2717805038 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b20099d90 con 0x55a4fc1858b0 2023-11-27T15:23:15.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.265+0000 7f9b3bf13b80 1 -- 172.21.15.52:0/2717805038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a4fc1858b0 msgr2=0x55a4fc17fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:15.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.265+0000 7f9b3bf13b80 1 --2- 172.21.15.52:0/2717805038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a4fc1858b0 0x55a4fc17fea0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9b20096340 tx=0x7f9b200994e0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.265+0000 7f9b3bf13b80 1 -- 172.21.15.52:0/2717805038 shutdown_connections 2023-11-27T15:23:15.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.265+0000 7f9b3bf13b80 1 --2- 172.21.15.52:0/2717805038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a4fc180b60 0x55a4fc18cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.265+0000 7f9b3bf13b80 1 --2- 172.21.15.52:0/2717805038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a4fc1858b0 0x55a4fc17fea0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.265+0000 7f9b3bf13b80 1 --2- 172.21.15.52:0/2717805038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4fc181470 0x55a4fc17f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.265+0000 7f9b3bf13b80 1 -- 172.21.15.52:0/2717805038 >> 172.21.15.52:0/2717805038 conn(0x55a4fc176360 msgr2=0x55a4fc196be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:15.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.265+0000 7f9b3bf13b80 1 -- 172.21.15.52:0/2717805038 shutdown_connections 2023-11-27T15:23:15.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.265+0000 7f9b3bf13b80 1 -- 172.21.15.52:0/2717805038 wait complete. 2023-11-27T15:23:15.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.265+0000 7f9b3bf13b80 1 -- start start 2023-11-27T15:23:15.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.266+0000 7f9b3bf13b80 1 Processor -- start 2023-11-27T15:23:15.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.266+0000 7f9b3bf13b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a4fc180b60 0x55a4fc1e28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.266+0000 7f9b3bf13b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a4fc181470 0x55a4fc1e2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.266+0000 7f9b3bf13b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4fc1858b0 0x55a4fc1e7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.266+0000 7f9b3bf13b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4fc1e7c60 con 0x55a4fc180b60 2023-11-27T15:23:15.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.266+0000 7f9b3bf13b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4fc1e7dd0 con 0x55a4fc181470 2023-11-27T15:23:15.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.266+0000 7f9b3bf13b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4fc1e7f90 con 0x55a4fc1858b0 2023-11-27T15:23:15.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.266+0000 7f9b3bf13b80 10 client.0 _ll_register_callbacks cb 0x7f9b18010d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:15.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.266+0000 7f9b2db23700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a4fc180b60 0x55a4fc1e28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.266+0000 7f9b2db23700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a4fc180b60 0x55a4fc1e28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54084/0 (socket says 172.21.15.52:54084) 2023-11-27T15:23:15.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.266+0000 7f9b2db23700 1 -- 172.21.15.52:0/1766949575 learned_addr learned my addr 172.21.15.52:0/1766949575 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:15.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.266+0000 7f9b2d322700 1 --2- 172.21.15.52:0/1766949575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a4fc181470 0x55a4fc1e2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.266+0000 7f9b2e324700 1 --2- 172.21.15.52:0/1766949575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4fc1858b0 0x55a4fc1e7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.266+0000 7f9b2db23700 1 -- 172.21.15.52:0/1766949575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4fc1858b0 msgr2=0x55a4fc1e7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:15.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.266+0000 7f9b2db23700 1 --2- 172.21.15.52:0/1766949575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4fc1858b0 0x55a4fc1e7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.266+0000 7f9b2db23700 1 -- 172.21.15.52:0/1766949575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a4fc181470 msgr2=0x55a4fc1e2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:15.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.266+0000 7f9b2db23700 1 --2- 172.21.15.52:0/1766949575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a4fc181470 0x55a4fc1e2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.266+0000 7f9b2db23700 1 -- 172.21.15.52:0/1766949575 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b200b7e70 con 0x55a4fc180b60 2023-11-27T15:23:15.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.266+0000 7f9b2d322700 1 --2- 172.21.15.52:0/1766949575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a4fc181470 0x55a4fc1e2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:23:15.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.266+0000 7f9b2e324700 1 --2- 172.21.15.52:0/1766949575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4fc1858b0 0x55a4fc1e7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:23:15.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.267+0000 7f9b2db23700 1 --2- 172.21.15.52:0/1766949575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a4fc180b60 0x55a4fc1e28c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f9b18006010 tx=0x7f9b18004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e15cbb6f18bbb463 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:15.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.267+0000 7f9b117fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:23:15.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.267+0000 7f9b117fa700 1 -- 172.21.15.52:0/1766949575 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b18010970 con 0x55a4fc180b60 2023-11-27T15:23:15.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.267+0000 7f9b117fa700 1 -- 172.21.15.52:0/1766949575 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f9b18017040 con 0x55a4fc180b60 2023-11-27T15:23:15.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.267+0000 7f9b3bf13b80 1 -- 172.21.15.52:0/1766949575 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b2401e040 con 0x55a4fc180b60 2023-11-27T15:23:15.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.267+0000 7f9b117fa700 1 -- 172.21.15.52:0/1766949575 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b18020a10 con 0x55a4fc180b60 2023-11-27T15:23:15.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.267+0000 7f9b3bf13b80 10 client.16269 fetch_fsmap learned FSMap version 28 2023-11-27T15:23:15.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.267+0000 7f9b3bf13b80 1 -- 172.21.15.52:0/1766949575 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a4fbee8c60 con 0x55a4fc180b60 2023-11-27T15:23:15.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.267+0000 7f9b117fa700 1 -- 172.21.15.52:0/1766949575 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b2401e040 con 0x55a4fc180b60 2023-11-27T15:23:15.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.267+0000 7f9b117fa700 1 -- 172.21.15.52:0/1766949575 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9b18029cd0 con 0x55a4fc180b60 2023-11-27T15:23:15.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.267+0000 7f9b3bf13b80 10 client.16269 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:23:15.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.267+0000 7f9b3bf13b80 10 client.16269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:15.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.267+0000 7f9b3bf13b80 20 client.16269 populate_metadata read hostname 'smithi052' 2023-11-27T15:23:15.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.267+0000 7f9b3bf13b80 1 --2- 172.21.15.52:0/1766949575 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55a4fc1ef7f0 0x55a4fc1f1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:15.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.267+0000 7f9b3bf13b80 4 client.16269 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:23:15.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.267+0000 7f9b3bf13b80 1 -- 172.21.15.52:0/1766949575 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a4fc0a0660 con 0x55a4fc1ef7f0 2023-11-27T15:23:15.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.268+0000 7f9b2d322700 1 --2- 172.21.15.52:0/1766949575 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55a4fc1ef7f0 0x55a4fc1f1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:15.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.268+0000 7f9b117fa700 10 client.16269 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:15.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.268+0000 7f9b2d322700 1 --2- 172.21.15.52:0/1766949575 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55a4fc1ef7f0 0x55a4fc1f1c00 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5735dbc645097aef server_cookie=b9f5061622f6e42 in_seq=0 out_seq=0 2023-11-27T15:23:15.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.268+0000 7f9b117fa700 1 -- 172.21.15.52:0/1766949575 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a4fc0a0660 con 0x55a4fc1ef7f0 2023-11-27T15:23:15.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.268+0000 7f9b117fa700 10 client.16269 handle_command_reply: tid=0 2023-11-27T15:23:15.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.268+0000 7f9b3bf13b80 1 client.16269 shutdown 2023-11-27T15:23:15.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.268+0000 7f9b3bf13b80 1 -- 172.21.15.52:0/1766949575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a4fc180b60 msgr2=0x55a4fc1e28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:15.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.268+0000 7f9b3bf13b80 1 --2- 172.21.15.52:0/1766949575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a4fc180b60 0x55a4fc1e28c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f9b18006010 tx=0x7f9b18004750 comp rx=0 tx=0).stop 2023-11-27T15:23:15.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.268+0000 7f9b3bf13b80 1 -- 172.21.15.52:0/1766949575 shutdown_connections 2023-11-27T15:23:15.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.268+0000 7f9b3bf13b80 1 --2- 172.21.15.52:0/1766949575 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55a4fc1ef7f0 0x55a4fc1f1c00 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.268+0000 7f9b3bf13b80 1 --2- 172.21.15.52:0/1766949575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a4fc1858b0 0x55a4fc1e7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.268+0000 7f9b3bf13b80 1 --2- 172.21.15.52:0/1766949575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a4fc180b60 0x55a4fc1e28c0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.268+0000 7f9b3bf13b80 1 --2- 172.21.15.52:0/1766949575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a4fc181470 0x55a4fc1e2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.268+0000 7f9b117fa700 0 client.16269 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:15.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.268+0000 7f9b3bf13b80 1 -- 172.21.15.52:0/1766949575 >> 172.21.15.52:0/1766949575 conn(0x55a4fc176360 msgr2=0x55a4fc1847e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:15.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.268+0000 7f9b3bf13b80 1 -- 172.21.15.52:0/1766949575 shutdown_connections 2023-11-27T15:23:15.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.268+0000 7f9b3bf13b80 1 -- 172.21.15.52:0/1766949575 wait complete. 2023-11-27T15:23:15.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.269+0000 7f9b3bf13b80 20 client.16269 trim_cache size 0 max 16384 2023-11-27T15:23:15.283 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:23:15.284 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-27T15:23:15.284 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2023-11-27T15:23:15.284 INFO:teuthology.orchestra.run.smithi052.stdout: "567b1959-c433-4da8-8bc2-2847ea622f57": { 2023-11-27T15:23:15.284 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2023-11-27T15:23:15.284 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "567b1959-c433-4da8-8bc2-2847ea622f57", 2023-11-27T15:23:15.284 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2023-11-27T15:23:15.284 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:23:15.285 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:23:15.285 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:23:15.285 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.269+0000 7f9b2fd87700 1 -- 172.21.15.52:0/1221966069 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f9b140a1340 msgr2=0x7f9b140a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:15.286 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.269+0000 7f9b2fd87700 1 --2- 172.21.15.52:0/1221966069 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f9b140a1340 0x7f9b140a37f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9b18007be0 tx=0x7f9b180073d0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.286 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.269+0000 7f9b2fd87700 1 -- 172.21.15.52:0/1221966069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b28104490 msgr2=0x7f9b2819d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:15.286 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.269+0000 7f9b2fd87700 1 --2- 172.21.15.52:0/1221966069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b28104490 0x7f9b2819d010 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000ef30 tx=0x7f9b2000c3a0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.286 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.269+0000 7f9b2fd87700 1 -- 172.21.15.52:0/1221966069 shutdown_connections 2023-11-27T15:23:15.286 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.269+0000 7f9b2fd87700 1 --2- 172.21.15.52:0/1221966069 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f9b140a1340 0x7f9b140a37f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.286 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.269+0000 7f9b2fd87700 1 --2- 172.21.15.52:0/1221966069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9b28104490 0x7f9b2819d010 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.269+0000 7f9b2fd87700 1 --2- 172.21.15.52:0/1221966069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b280ff450 0x7f9b28198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.269+0000 7f9b2fd87700 1 --2- 172.21.15.52:0/1221966069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b280ffdf0 0x7f9b28198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.269+0000 7f9b2fd87700 1 -- 172.21.15.52:0/1221966069 >> 172.21.15.52:0/1221966069 conn(0x7f9b280770b0 msgr2=0x7f9b28067dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:15.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.269+0000 7f9b2fd87700 1 -- 172.21.15.52:0/1221966069 shutdown_connections 2023-11-27T15:23:15.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:15.269+0000 7f9b2fd87700 1 -- 172.21.15.52:0/1221966069 wait complete. 2023-11-27T15:23:15.287 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:15 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:23:15.288 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:15 smithi052 ceph-mon[99056]: Service mds.all has no related daemon already placed. 2023-11-27T15:23:15.288 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:15 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:23:15.288 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:15 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:23:15.288 INFO:tasks.cephfs.filesystem:scrub status for tag:567b1959-c433-4da8-8bc2-2847ea622f57 - {'path': '/', 'tag': '567b1959-c433-4da8-8bc2-2847ea622f57', 'options': 'recursive,force'} 2023-11-27T15:23:15.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:15 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:23:15.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:15 smithi150 ceph-mon[100559]: Service mds.all has no related daemon already placed. 2023-11-27T15:23:15.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:15 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:23:15.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:15 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:23:15.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:15 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:23:15.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:15 smithi102 ceph-mon[100296]: Service mds.all has no related daemon already placed. 2023-11-27T15:23:15.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:15 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:23:15.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:15 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:23:16.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:16 smithi150 ceph-mon[100559]: scrub queued for path: / 2023-11-27T15:23:16.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:16 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:23:16.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:16 smithi150 ceph-mon[100559]: scrub summary: active paths [/] 2023-11-27T15:23:16.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:16 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:23:16.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:16 smithi150 ceph-mon[100559]: pgmap v163: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 6 op/s 2023-11-27T15:23:16.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:16 smithi102 ceph-mon[100296]: scrub queued for path: / 2023-11-27T15:23:16.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:16 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:23:16.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:16 smithi102 ceph-mon[100296]: scrub summary: active paths [/] 2023-11-27T15:23:16.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:16 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:23:16.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:16 smithi102 ceph-mon[100296]: pgmap v163: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 6 op/s 2023-11-27T15:23:16.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:16 smithi052 ceph-mon[99056]: scrub queued for path: / 2023-11-27T15:23:16.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:16 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:23:16.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:16 smithi052 ceph-mon[99056]: scrub summary: active paths [/] 2023-11-27T15:23:16.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:16 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:23:16.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:16 smithi052 ceph-mon[99056]: pgmap v163: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 6 op/s 2023-11-27T15:23:18.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:18 smithi150 ceph-mon[100559]: pgmap v164: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 10 op/s 2023-11-27T15:23:18.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:18 smithi102 ceph-mon[100296]: pgmap v164: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 10 op/s 2023-11-27T15:23:18.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:18 smithi052 ceph-mon[99056]: pgmap v164: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 10 op/s 2023-11-27T15:23:20.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:19 smithi052 ceph-mon[99056]: pgmap v165: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 7 op/s 2023-11-27T15:23:20.117 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:19 smithi052 ceph-mon[99056]: scrub summary: idle 2023-11-27T15:23:20.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:19 smithi150 ceph-mon[100559]: pgmap v165: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 7 op/s 2023-11-27T15:23:20.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:19 smithi150 ceph-mon[100559]: scrub summary: idle 2023-11-27T15:23:20.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:19 smithi102 ceph-mon[100296]: pgmap v165: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 7 op/s 2023-11-27T15:23:20.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:19 smithi102 ceph-mon[100296]: scrub summary: idle 2023-11-27T15:23:22.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:22 smithi150 ceph-mon[100559]: pgmap v166: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 10 op/s 2023-11-27T15:23:22.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:22 smithi102 ceph-mon[100296]: pgmap v166: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 10 op/s 2023-11-27T15:23:22.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:22 smithi052 ceph-mon[99056]: pgmap v166: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 10 op/s 2023-11-27T15:23:24.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:24 smithi150 ceph-mon[100559]: pgmap v167: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 7 op/s 2023-11-27T15:23:24.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:24 smithi102 ceph-mon[100296]: pgmap v167: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 7 op/s 2023-11-27T15:23:24.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:24 smithi052 ceph-mon[99056]: pgmap v167: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 7 op/s 2023-11-27T15:23:26.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:26 smithi150 ceph-mon[100559]: pgmap v168: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 7 op/s 2023-11-27T15:23:26.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:26 smithi102 ceph-mon[100296]: pgmap v168: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 7 op/s 2023-11-27T15:23:26.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:26 smithi052 ceph-mon[99056]: pgmap v168: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 7 op/s 2023-11-27T15:23:28.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:28 smithi150 ceph-mon[100559]: pgmap v169: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.3 KiB/s wr, 9 op/s 2023-11-27T15:23:28.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:28 smithi102 ceph-mon[100296]: pgmap v169: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.3 KiB/s wr, 9 op/s 2023-11-27T15:23:28.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:28 smithi052 ceph-mon[99056]: pgmap v169: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.3 KiB/s wr, 9 op/s 2023-11-27T15:23:30.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:29 smithi052 ceph-mon[99056]: pgmap v170: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2023-11-27T15:23:30.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:29 smithi150 ceph-mon[100559]: pgmap v170: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2023-11-27T15:23:30.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:29 smithi102 ceph-mon[100296]: pgmap v170: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2023-11-27T15:23:32.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:32 smithi150 ceph-mon[100559]: pgmap v171: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2023-11-27T15:23:32.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:32 smithi102 ceph-mon[100296]: pgmap v171: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2023-11-27T15:23:32.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:32 smithi052 ceph-mon[99056]: pgmap v171: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2023-11-27T15:23:34.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:34 smithi150 ceph-mon[100559]: pgmap v172: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.4 KiB/s wr, 5 op/s 2023-11-27T15:23:34.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:34 smithi102 ceph-mon[100296]: pgmap v172: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.4 KiB/s wr, 5 op/s 2023-11-27T15:23:34.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:34 smithi052 ceph-mon[99056]: pgmap v172: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.4 KiB/s wr, 5 op/s 2023-11-27T15:23:36.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:36 smithi150 ceph-mon[100559]: pgmap v173: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.4 KiB/s wr, 5 op/s 2023-11-27T15:23:36.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:36 smithi102 ceph-mon[100296]: pgmap v173: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.4 KiB/s wr, 5 op/s 2023-11-27T15:23:36.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:36 smithi052 ceph-mon[99056]: pgmap v173: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.4 KiB/s wr, 5 op/s 2023-11-27T15:23:38.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:38 smithi150 ceph-mon[100559]: pgmap v174: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2023-11-27T15:23:38.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:38 smithi102 ceph-mon[100296]: pgmap v174: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2023-11-27T15:23:38.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:38 smithi052 ceph-mon[99056]: pgmap v174: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2023-11-27T15:23:40.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:39 smithi150 ceph-mon[100559]: pgmap v175: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2023-11-27T15:23:40.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:39 smithi102 ceph-mon[100296]: pgmap v175: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2023-11-27T15:23:40.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:39 smithi052 ceph-mon[99056]: pgmap v175: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2023-11-27T15:23:42.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:42 smithi150 ceph-mon[100559]: pgmap v176: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.9 KiB/s wr, 7 op/s 2023-11-27T15:23:42.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:42 smithi102 ceph-mon[100296]: pgmap v176: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.9 KiB/s wr, 7 op/s 2023-11-27T15:23:42.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:42 smithi052 ceph-mon[99056]: pgmap v176: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.9 KiB/s wr, 7 op/s 2023-11-27T15:23:44.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:44 smithi150 ceph-mon[100559]: pgmap v177: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2023-11-27T15:23:44.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:44 smithi102 ceph-mon[100296]: pgmap v177: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2023-11-27T15:23:44.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:44 smithi052 ceph-mon[99056]: pgmap v177: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2023-11-27T15:23:45.289 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:23:45.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.462+0000 7f98d35c7700 1 -- 172.21.15.52:0/2237593703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f98cc0ff450 msgr2=0x7f98cc0ff820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.462+0000 7f98d35c7700 1 --2- 172.21.15.52:0/2237593703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f98cc0ff450 0x7f98cc0ff820 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f98c80099c0 tx=0x7f98c8009cd0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.462+0000 7f98d35c7700 1 -- 172.21.15.52:0/2237593703 shutdown_connections 2023-11-27T15:23:45.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.462+0000 7f98d35c7700 1 --2- 172.21.15.52:0/2237593703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98cc104490 0x7f98cc10c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.462+0000 7f98d35c7700 1 --2- 172.21.15.52:0/2237593703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f98cc0ff450 0x7f98cc0ff820 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.462+0000 7f98d35c7700 1 --2- 172.21.15.52:0/2237593703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98cc0ffdf0 0x7f98cc103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.462+0000 7f98d35c7700 1 -- 172.21.15.52:0/2237593703 >> 172.21.15.52:0/2237593703 conn(0x7f98cc0770b0 msgr2=0x7f98cc0774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:45.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.462+0000 7f98d35c7700 1 -- 172.21.15.52:0/2237593703 shutdown_connections 2023-11-27T15:23:45.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.463+0000 7f98d35c7700 1 -- 172.21.15.52:0/2237593703 wait complete. 2023-11-27T15:23:45.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.463+0000 7f98d35c7700 1 Processor -- start 2023-11-27T15:23:45.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.463+0000 7f98d35c7700 1 -- start start 2023-11-27T15:23:45.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.463+0000 7f98d35c7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98cc0ff450 0x7f98cc1963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.463+0000 7f98d35c7700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98cc0ffdf0 0x7f98cc1968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.463+0000 7f98d35c7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f98cc104490 0x7f98cc19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.463+0000 7f98d35c7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc19b390 con 0x7f98cc104490 2023-11-27T15:23:45.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.463+0000 7f98d35c7700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc19b500 con 0x7f98cc0ffdf0 2023-11-27T15:23:45.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.463+0000 7f98d35c7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc19b6c0 con 0x7f98cc0ff450 2023-11-27T15:23:45.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.464+0000 7f98d1b64700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f98cc104490 0x7f98cc19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.464+0000 7f98d1b64700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f98cc104490 0x7f98cc19ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54102/0 (socket says 172.21.15.52:54102) 2023-11-27T15:23:45.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.464+0000 7f98d1b64700 1 -- 172.21.15.52:0/4294697052 learned_addr learned my addr 172.21.15.52:0/4294697052 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:45.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.464+0000 7f98d0b62700 1 --2- 172.21.15.52:0/4294697052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98cc0ffdf0 0x7f98cc1968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.464+0000 7f98d1363700 1 --2- 172.21.15.52:0/4294697052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98cc0ff450 0x7f98cc1963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.464+0000 7f98d1b64700 1 -- 172.21.15.52:0/4294697052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98cc0ff450 msgr2=0x7f98cc1963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.464+0000 7f98d1b64700 1 --2- 172.21.15.52:0/4294697052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98cc0ff450 0x7f98cc1963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.464+0000 7f98d1b64700 1 -- 172.21.15.52:0/4294697052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98cc0ffdf0 msgr2=0x7f98cc1968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.464+0000 7f98d1b64700 1 --2- 172.21.15.52:0/4294697052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98cc0ffdf0 0x7f98cc1968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.464+0000 7f98d1b64700 1 -- 172.21.15.52:0/4294697052 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c80095f0 con 0x7f98cc104490 2023-11-27T15:23:45.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.464+0000 7f98d1b64700 1 --2- 172.21.15.52:0/4294697052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f98cc104490 0x7f98cc19ad50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f98c400ada0 tx=0x7f98c400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:45.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.464+0000 7f98be7fc700 1 -- 172.21.15.52:0/4294697052 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c400f070 con 0x7f98cc104490 2023-11-27T15:23:45.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.464+0000 7f98be7fc700 1 -- 172.21.15.52:0/4294697052 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98c4007b30 con 0x7f98cc104490 2023-11-27T15:23:45.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.464+0000 7f98d35c7700 1 -- 172.21.15.52:0/4294697052 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98cc1103c0 con 0x7f98cc104490 2023-11-27T15:23:45.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.464+0000 7f98be7fc700 1 -- 172.21.15.52:0/4294697052 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c40053e0 con 0x7f98cc104490 2023-11-27T15:23:45.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.464+0000 7f98d35c7700 1 -- 172.21.15.52:0/4294697052 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98cc110230 con 0x7f98cc104490 2023-11-27T15:23:45.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.467+0000 7f98be7fc700 1 -- 172.21.15.52:0/4294697052 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f98c400c980 con 0x7f98cc104490 2023-11-27T15:23:45.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.467+0000 7f98be7fc700 1 --2- 172.21.15.52:0/4294697052 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f98b80a1340 0x7f98b80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.467+0000 7f98be7fc700 1 -- 172.21.15.52:0/4294697052 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f98c40cc240 con 0x7f98cc104490 2023-11-27T15:23:45.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.467+0000 7f98d1363700 1 --2- 172.21.15.52:0/4294697052 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f98b80a1340 0x7f98b80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.482 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.468+0000 7f98d1363700 1 --2- 172.21.15.52:0/4294697052 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f98b80a1340 0x7f98b80a37f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f98c8009fd0 tx=0x7f98c8005fb0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:45.482 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.468+0000 7f98df753b80 1 Processor -- start 2023-11-27T15:23:45.482 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.468+0000 7f98df753b80 1 -- start start 2023-11-27T15:23:45.482 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.469+0000 7f98df753b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557735782b60 0x557735782f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.483 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.469+0000 7f98df753b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557735783470 0x557735787370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.483 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.469+0000 7f98df753b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577357878b0 0x557735787d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.483 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.469+0000 7f98df753b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x557735688060 con 0x5577357878b0 2023-11-27T15:23:45.483 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.469+0000 7f98df753b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577356da8a0 con 0x557735783470 2023-11-27T15:23:45.483 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.469+0000 7f98df753b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577356d3d30 con 0x557735782b60 2023-11-27T15:23:45.484 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.469+0000 7f98d1b64700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577357878b0 0x557735787d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.484 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.469+0000 7f98d1b64700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577357878b0 0x557735787d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54110/0 (socket says 172.21.15.52:54110) 2023-11-27T15:23:45.484 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.469+0000 7f98d1b64700 1 -- 172.21.15.52:0/1373764454 learned_addr learned my addr 172.21.15.52:0/1373764454 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:45.484 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.469+0000 7f98d1363700 1 --2- 172.21.15.52:0/1373764454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557735782b60 0x557735782f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.485 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.469+0000 7f98d0b62700 1 --2- 172.21.15.52:0/1373764454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557735783470 0x557735787370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.485 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.469+0000 7f98d1b64700 1 -- 172.21.15.52:0/1373764454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557735782b60 msgr2=0x557735782f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:45.485 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.469+0000 7f98d1b64700 1 --2- 172.21.15.52:0/1373764454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557735782b60 0x557735782f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.485 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.469+0000 7f98d1b64700 1 -- 172.21.15.52:0/1373764454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557735783470 msgr2=0x557735787370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:45.485 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.469+0000 7f98d1b64700 1 --2- 172.21.15.52:0/1373764454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557735783470 0x557735787370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.486 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.469+0000 7f98d1b64700 1 -- 172.21.15.52:0/1373764454 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557735555630 con 0x5577357878b0 2023-11-27T15:23:45.486 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.469+0000 7f98d1363700 1 --2- 172.21.15.52:0/1373764454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557735782b60 0x557735782f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:45.486 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.469+0000 7f98d0b62700 1 --2- 172.21.15.52:0/1373764454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557735783470 0x557735787370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:23:45.486 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.469+0000 7f98d1b64700 1 --2- 172.21.15.52:0/1373764454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577357878b0 0x557735787d00 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f98c4099330 tx=0x7f98c4098d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bae04d60cc745e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:45.487 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.469+0000 7f98b77fe700 1 -- 172.21.15.52:0/1373764454 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c40993a0 con 0x5577357878b0 2023-11-27T15:23:45.487 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.470+0000 7f98b77fe700 1 -- 172.21.15.52:0/1373764454 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98c409d3d0 con 0x5577357878b0 2023-11-27T15:23:45.487 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.470+0000 7f98b77fe700 1 -- 172.21.15.52:0/1373764454 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c4095ab0 con 0x5577357878b0 2023-11-27T15:23:45.487 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.470+0000 7f98df753b80 1 -- 172.21.15.52:0/1373764454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577357878b0 msgr2=0x557735787d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.487 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.470+0000 7f98df753b80 1 --2- 172.21.15.52:0/1373764454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577357878b0 0x557735787d00 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f98c4099330 tx=0x7f98c4098d00 comp rx=0 tx=0).stop 2023-11-27T15:23:45.488 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.470+0000 7f98df753b80 1 -- 172.21.15.52:0/1373764454 shutdown_connections 2023-11-27T15:23:45.488 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.470+0000 7f98df753b80 1 --2- 172.21.15.52:0/1373764454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5577357878b0 0x557735787d00 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.488 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.470+0000 7f98df753b80 1 --2- 172.21.15.52:0/1373764454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557735782b60 0x557735782f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.488 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.470+0000 7f98df753b80 1 --2- 172.21.15.52:0/1373764454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557735783470 0x557735787370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.488 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.470+0000 7f98df753b80 1 -- 172.21.15.52:0/1373764454 >> 172.21.15.52:0/1373764454 conn(0x557735778360 msgr2=0x55773577db40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:45.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.470+0000 7f98df753b80 1 -- 172.21.15.52:0/1373764454 shutdown_connections 2023-11-27T15:23:45.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.470+0000 7f98df753b80 1 -- 172.21.15.52:0/1373764454 wait complete. 2023-11-27T15:23:45.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.471+0000 7f98df753b80 1 -- start start 2023-11-27T15:23:45.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.471+0000 7f98df753b80 1 Processor -- start 2023-11-27T15:23:45.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.471+0000 7f98df753b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557735782b60 0x55773578f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.490 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.471+0000 7f98df753b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557735783470 0x55773578c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.490 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.472+0000 7f98df753b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577357878b0 0x55773578caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.490 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.472+0000 7f98df753b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577356758e0 con 0x557735783470 2023-11-27T15:23:45.490 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.472+0000 7f98df753b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557735675580 con 0x557735782b60 2023-11-27T15:23:45.491 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.472+0000 7f98df753b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557735557920 con 0x5577357878b0 2023-11-27T15:23:45.491 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.472+0000 7f98df753b80 10 client.0 _ll_register_callbacks cb 0x557735693780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:45.491 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.472+0000 7f98d0b62700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557735783470 0x55773578c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.491 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.472+0000 7f98d0b62700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557735783470 0x55773578c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54114/0 (socket says 172.21.15.52:54114) 2023-11-27T15:23:45.491 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.472+0000 7f98d0b62700 1 -- 172.21.15.52:0/802178049 learned_addr learned my addr 172.21.15.52:0/802178049 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:45.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.472+0000 7f98d1363700 1 --2- 172.21.15.52:0/802178049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557735782b60 0x55773578f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.472+0000 7f98d1b64700 1 --2- 172.21.15.52:0/802178049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577357878b0 0x55773578caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.472+0000 7f98d0b62700 1 -- 172.21.15.52:0/802178049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577357878b0 msgr2=0x55773578caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:45.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.472+0000 7f98d0b62700 1 --2- 172.21.15.52:0/802178049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577357878b0 0x55773578caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.472+0000 7f98d0b62700 1 -- 172.21.15.52:0/802178049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557735782b60 msgr2=0x55773578f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:45.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.472+0000 7f98d0b62700 1 --2- 172.21.15.52:0/802178049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557735782b60 0x55773578f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.472+0000 7f98d0b62700 1 -- 172.21.15.52:0/802178049 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98cc110230 con 0x557735783470 2023-11-27T15:23:45.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.472+0000 7f98d0b62700 1 --2- 172.21.15.52:0/802178049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557735783470 0x55773578c3b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f98c00117a0 tx=0x7f98c0011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f90571b7c4da912 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:45.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.472+0000 7f98b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:23:45.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.472+0000 7f98b4ff9700 1 -- 172.21.15.52:0/802178049 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c0004220 con 0x557735783470 2023-11-27T15:23:45.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.472+0000 7f98b4ff9700 1 -- 172.21.15.52:0/802178049 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f98c0004380 con 0x557735783470 2023-11-27T15:23:45.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.472+0000 7f98df753b80 1 -- 172.21.15.52:0/802178049 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557735775b20 con 0x557735783470 2023-11-27T15:23:45.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.472+0000 7f98b4ff9700 1 -- 172.21.15.52:0/802178049 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98c0004540 con 0x557735783470 2023-11-27T15:23:45.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.472+0000 7f98df753b80 10 client.16281 fetch_fsmap learned FSMap version 28 2023-11-27T15:23:45.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.472+0000 7f98df753b80 1 -- 172.21.15.52:0/802178049 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55773569a190 con 0x557735783470 2023-11-27T15:23:45.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.473+0000 7f98b4ff9700 1 -- 172.21.15.52:0/802178049 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557735775b20 con 0x557735783470 2023-11-27T15:23:45.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.473+0000 7f98b4ff9700 1 -- 172.21.15.52:0/802178049 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f98c0003b40 con 0x557735783470 2023-11-27T15:23:45.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.473+0000 7f98df753b80 10 client.16281 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:23:45.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.473+0000 7f98df753b80 10 client.16281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:45.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.473+0000 7f98df753b80 20 client.16281 populate_metadata read hostname 'smithi052' 2023-11-27T15:23:45.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.473+0000 7f98df753b80 1 --2- 172.21.15.52:0/802178049 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55773579c970 0x55773579ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.473+0000 7f98df753b80 4 client.16281 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:23:45.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.473+0000 7f98df753b80 1 -- 172.21.15.52:0/802178049 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577356960a0 con 0x55773579c970 2023-11-27T15:23:45.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.473+0000 7f98d1363700 1 --2- 172.21.15.52:0/802178049 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55773579c970 0x55773579ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.474+0000 7f98b4ff9700 10 client.16281 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:45.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.474+0000 7f98d1363700 1 --2- 172.21.15.52:0/802178049 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55773579c970 0x55773579ed80 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab0cb70773587b50 server_cookie=a2e324cd51803e87 in_seq=0 out_seq=0 2023-11-27T15:23:45.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.474+0000 7f98b4ff9700 1 -- 172.21.15.52:0/802178049 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5577356960a0 con 0x55773579c970 2023-11-27T15:23:45.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.474+0000 7f98b4ff9700 10 client.16281 handle_command_reply: tid=0 2023-11-27T15:23:45.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.474+0000 7f98df753b80 1 client.16281 shutdown 2023-11-27T15:23:45.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.474+0000 7f98df753b80 1 -- 172.21.15.52:0/802178049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557735783470 msgr2=0x55773578c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.474+0000 7f98df753b80 1 --2- 172.21.15.52:0/802178049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557735783470 0x55773578c3b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f98c00117a0 tx=0x7f98c0011ab0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.474+0000 7f98df753b80 1 -- 172.21.15.52:0/802178049 shutdown_connections 2023-11-27T15:23:45.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.474+0000 7f98df753b80 1 --2- 172.21.15.52:0/802178049 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55773579c970 0x55773579ed80 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.474+0000 7f98df753b80 1 --2- 172.21.15.52:0/802178049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557735782b60 0x55773578f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.474+0000 7f98df753b80 1 --2- 172.21.15.52:0/802178049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577357878b0 0x55773578caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.474+0000 7f98df753b80 1 --2- 172.21.15.52:0/802178049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557735783470 0x55773578c3b0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.474+0000 7f98df753b80 1 -- 172.21.15.52:0/802178049 >> 172.21.15.52:0/802178049 conn(0x557735778360 msgr2=0x557735788d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:45.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.474+0000 7f98b4ff9700 0 client.16281 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:45.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.475+0000 7f98df753b80 1 -- 172.21.15.52:0/802178049 shutdown_connections 2023-11-27T15:23:45.501 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.475+0000 7f98df753b80 1 -- 172.21.15.52:0/802178049 wait complete. 2023-11-27T15:23:45.501 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.475+0000 7f98df753b80 20 client.16281 trim_cache size 0 max 16384 2023-11-27T15:23:45.506 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.492+0000 7f98df753b80 1 Processor -- start 2023-11-27T15:23:45.506 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.492+0000 7f98df753b80 1 -- start start 2023-11-27T15:23:45.507 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.492+0000 7f98df753b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557735782b60 0x55773578ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.507 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.492+0000 7f98df753b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557735783470 0x5577357859d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.507 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.492+0000 7f98df753b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577357878b0 0x557735785f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.507 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.492+0000 7f98df753b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577357881c0 con 0x557735783470 2023-11-27T15:23:45.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.492+0000 7f98df753b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557735777ee0 con 0x557735782b60 2023-11-27T15:23:45.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.492+0000 7f98df753b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557735786540 con 0x5577357878b0 2023-11-27T15:23:45.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.492+0000 7f98d0b62700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557735783470 0x5577357859d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.492+0000 7f98d0b62700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557735783470 0x5577357859d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54122/0 (socket says 172.21.15.52:54122) 2023-11-27T15:23:45.509 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.492+0000 7f98d0b62700 1 -- 172.21.15.52:0/3759688404 learned_addr learned my addr 172.21.15.52:0/3759688404 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:45.509 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.492+0000 7f98d1363700 1 --2- 172.21.15.52:0/3759688404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557735782b60 0x55773578ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.509 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.492+0000 7f98d1b64700 1 --2- 172.21.15.52:0/3759688404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577357878b0 0x557735785f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.509 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.492+0000 7f98d0b62700 1 -- 172.21.15.52:0/3759688404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577357878b0 msgr2=0x557735785f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:45.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.492+0000 7f98d0b62700 1 --2- 172.21.15.52:0/3759688404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577357878b0 0x557735785f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.492+0000 7f98d0b62700 1 -- 172.21.15.52:0/3759688404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557735782b60 msgr2=0x55773578ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:45.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.492+0000 7f98d0b62700 1 --2- 172.21.15.52:0/3759688404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557735782b60 0x55773578ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.492+0000 7f98d0b62700 1 -- 172.21.15.52:0/3759688404 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557735699ee0 con 0x557735783470 2023-11-27T15:23:45.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.492+0000 7f98d1363700 1 --2- 172.21.15.52:0/3759688404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557735782b60 0x55773578ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:45.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.492+0000 7f98d0b62700 1 --2- 172.21.15.52:0/3759688404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557735783470 0x5577357859d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f98c000f3b0 tx=0x7f98c0005eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec90df2e841913bc server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:45.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.492+0000 7f98b7fff700 1 -- 172.21.15.52:0/3759688404 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c001aa70 con 0x557735783470 2023-11-27T15:23:45.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.493+0000 7f98b7fff700 1 -- 172.21.15.52:0/3759688404 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98c001a660 con 0x557735783470 2023-11-27T15:23:45.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.493+0000 7f98b7fff700 1 -- 172.21.15.52:0/3759688404 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c0012be0 con 0x557735783470 2023-11-27T15:23:45.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.493+0000 7f98df753b80 1 -- 172.21.15.52:0/3759688404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557735783470 msgr2=0x5577357859d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.493+0000 7f98df753b80 1 --2- 172.21.15.52:0/3759688404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557735783470 0x5577357859d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f98c000f3b0 tx=0x7f98c0005eb0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.493+0000 7f98df753b80 1 -- 172.21.15.52:0/3759688404 shutdown_connections 2023-11-27T15:23:45.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.493+0000 7f98df753b80 1 --2- 172.21.15.52:0/3759688404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557735782b60 0x55773578ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.493+0000 7f98df753b80 1 --2- 172.21.15.52:0/3759688404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577357878b0 0x557735785f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.493+0000 7f98df753b80 1 --2- 172.21.15.52:0/3759688404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557735783470 0x5577357859d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.493+0000 7f98df753b80 1 -- 172.21.15.52:0/3759688404 >> 172.21.15.52:0/3759688404 conn(0x557735778360 msgr2=0x557735798ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:45.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.493+0000 7f98df753b80 1 -- 172.21.15.52:0/3759688404 shutdown_connections 2023-11-27T15:23:45.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.493+0000 7f98df753b80 1 -- 172.21.15.52:0/3759688404 wait complete. 2023-11-27T15:23:45.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.494+0000 7f98df753b80 1 -- start start 2023-11-27T15:23:45.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.494+0000 7f98df753b80 1 Processor -- start 2023-11-27T15:23:45.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.494+0000 7f98df753b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557735782b60 0x557735791990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.494+0000 7f98df753b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557735783470 0x557735791ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.494+0000 7f98df753b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577357878b0 0x557735792410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.494+0000 7f98df753b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577357f3cd0 con 0x557735783470 2023-11-27T15:23:45.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.494+0000 7f98df753b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577357f3e40 con 0x557735782b60 2023-11-27T15:23:45.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.494+0000 7f98df753b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577357f4000 con 0x5577357878b0 2023-11-27T15:23:45.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.494+0000 7f98df753b80 10 client.0 _ll_register_callbacks cb 0x7f98c000abf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:45.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.494+0000 7f98d0b62700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557735783470 0x557735791ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.494+0000 7f98d0b62700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557735783470 0x557735791ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54128/0 (socket says 172.21.15.52:54128) 2023-11-27T15:23:45.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.494+0000 7f98d0b62700 1 -- 172.21.15.52:0/695179799 learned_addr learned my addr 172.21.15.52:0/695179799 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:45.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.494+0000 7f98d1363700 1 --2- 172.21.15.52:0/695179799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557735782b60 0x557735791990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.494+0000 7f98d1b64700 1 --2- 172.21.15.52:0/695179799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577357878b0 0x557735792410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.495+0000 7f98d0b62700 1 -- 172.21.15.52:0/695179799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577357878b0 msgr2=0x557735792410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:45.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.495+0000 7f98d0b62700 1 --2- 172.21.15.52:0/695179799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577357878b0 0x557735792410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.495+0000 7f98d0b62700 1 -- 172.21.15.52:0/695179799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557735782b60 msgr2=0x557735791990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:45.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.495+0000 7f98d0b62700 1 --2- 172.21.15.52:0/695179799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557735782b60 0x557735791990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.495+0000 7f98d0b62700 1 -- 172.21.15.52:0/695179799 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55773569a190 con 0x557735783470 2023-11-27T15:23:45.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.495+0000 7f98d1363700 1 --2- 172.21.15.52:0/695179799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557735782b60 0x557735791990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:45.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.495+0000 7f98d0b62700 1 --2- 172.21.15.52:0/695179799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557735783470 0x557735791ed0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f98c0003f70 tx=0x7f98c001aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b2ed34ad1bc913 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:45.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.495+0000 7f98b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:23:45.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.495+0000 7f98b4ff9700 1 -- 172.21.15.52:0/695179799 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c0012780 con 0x557735783470 2023-11-27T15:23:45.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.495+0000 7f98b4ff9700 1 -- 172.21.15.52:0/695179799 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f98c0000e40 con 0x557735783470 2023-11-27T15:23:45.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.495+0000 7f98df753b80 1 -- 172.21.15.52:0/695179799 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f98c409d3d0 con 0x557735783470 2023-11-27T15:23:45.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.495+0000 7f98b4ff9700 1 -- 172.21.15.52:0/695179799 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98c00128e0 con 0x557735783470 2023-11-27T15:23:45.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.495+0000 7f98df753b80 10 client.16287 fetch_fsmap learned FSMap version 28 2023-11-27T15:23:45.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.495+0000 7f98df753b80 1 -- 172.21.15.52:0/695179799 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5577354eac60 con 0x557735783470 2023-11-27T15:23:45.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.495+0000 7f98b4ff9700 1 -- 172.21.15.52:0/695179799 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98c409d3d0 con 0x557735783470 2023-11-27T15:23:45.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.495+0000 7f98b4ff9700 1 -- 172.21.15.52:0/695179799 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f98c0036480 con 0x557735783470 2023-11-27T15:23:45.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.495+0000 7f98df753b80 10 client.16287 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:23:45.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.495+0000 7f98df753b80 10 client.16287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:45.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.495+0000 7f98df753b80 20 client.16287 populate_metadata read hostname 'smithi052' 2023-11-27T15:23:45.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.495+0000 7f98df753b80 1 --2- 172.21.15.52:0/695179799 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5577357f53e0 0x5577357f77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.495+0000 7f98df753b80 4 client.16287 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:23:45.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.496+0000 7f98df753b80 1 -- 172.21.15.52:0/695179799 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5577356a2660 con 0x5577357f53e0 2023-11-27T15:23:45.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.496+0000 7f98d1363700 1 --2- 172.21.15.52:0/695179799 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5577357f53e0 0x5577357f77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.496+0000 7f98d1363700 1 --2- 172.21.15.52:0/695179799 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5577357f53e0 0x5577357f77f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e16e3ca612ea335b server_cookie=3d73092a20f2281e in_seq=0 out_seq=0 2023-11-27T15:23:45.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.496+0000 7f98b4ff9700 10 client.16287 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:45.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.496+0000 7f98b4ff9700 1 -- 172.21.15.52:0/695179799 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5577356a2660 con 0x5577357f53e0 2023-11-27T15:23:45.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.496+0000 7f98b4ff9700 10 client.16287 handle_command_reply: tid=0 2023-11-27T15:23:45.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.496+0000 7f98df753b80 1 client.16287 shutdown 2023-11-27T15:23:45.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.496+0000 7f98df753b80 1 -- 172.21.15.52:0/695179799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557735783470 msgr2=0x557735791ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.496+0000 7f98df753b80 1 --2- 172.21.15.52:0/695179799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557735783470 0x557735791ed0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f98c0003f70 tx=0x7f98c001aed0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.496+0000 7f98df753b80 1 -- 172.21.15.52:0/695179799 shutdown_connections 2023-11-27T15:23:45.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.496+0000 7f98df753b80 1 --2- 172.21.15.52:0/695179799 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5577357f53e0 0x5577357f77f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.496+0000 7f98df753b80 1 --2- 172.21.15.52:0/695179799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557735782b60 0x557735791990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.496+0000 7f98df753b80 1 --2- 172.21.15.52:0/695179799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5577357878b0 0x557735792410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.496+0000 7f98df753b80 1 --2- 172.21.15.52:0/695179799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557735783470 0x557735791ed0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.496+0000 7f98b4ff9700 0 client.16287 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:45.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.496+0000 7f98df753b80 1 -- 172.21.15.52:0/695179799 >> 172.21.15.52:0/695179799 conn(0x557735778360 msgr2=0x5577357859d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:45.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.497+0000 7f98df753b80 1 -- 172.21.15.52:0/695179799 shutdown_connections 2023-11-27T15:23:45.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.497+0000 7f98df753b80 1 -- 172.21.15.52:0/695179799 wait complete. 2023-11-27T15:23:45.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.497+0000 7f98df753b80 20 client.16287 trim_cache size 0 max 16384 2023-11-27T15:23:45.526 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:23:45.526 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2023-11-27T15:23:45.526 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2023-11-27T15:23:45.526 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:23:45.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.497+0000 7f98d35c7700 1 -- 172.21.15.52:0/4294697052 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f98b80a1340 msgr2=0x7f98b80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.497+0000 7f98d35c7700 1 --2- 172.21.15.52:0/4294697052 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f98b80a1340 0x7f98b80a37f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f98c8009fd0 tx=0x7f98c8005fb0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.497+0000 7f98d35c7700 1 -- 172.21.15.52:0/4294697052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f98cc104490 msgr2=0x7f98cc19ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.497+0000 7f98d35c7700 1 --2- 172.21.15.52:0/4294697052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f98cc104490 0x7f98cc19ad50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f98c400ada0 tx=0x7f98c400c330 comp rx=0 tx=0).stop 2023-11-27T15:23:45.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.497+0000 7f98d35c7700 1 -- 172.21.15.52:0/4294697052 shutdown_connections 2023-11-27T15:23:45.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.497+0000 7f98d35c7700 1 --2- 172.21.15.52:0/4294697052 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f98b80a1340 0x7f98b80a37f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.497+0000 7f98d35c7700 1 --2- 172.21.15.52:0/4294697052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f98cc104490 0x7f98cc19ad50 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.497+0000 7f98d35c7700 1 --2- 172.21.15.52:0/4294697052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98cc0ff450 0x7f98cc1963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.497+0000 7f98d35c7700 1 --2- 172.21.15.52:0/4294697052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98cc0ffdf0 0x7f98cc1968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.497+0000 7f98d35c7700 1 -- 172.21.15.52:0/4294697052 >> 172.21.15.52:0/4294697052 conn(0x7f98cc0770b0 msgr2=0x7f98cc0fcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:45.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.497+0000 7f98d35c7700 1 -- 172.21.15.52:0/4294697052 shutdown_connections 2023-11-27T15:23:45.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.497+0000 7f98d35c7700 1 -- 172.21.15.52:0/4294697052 wait complete. 2023-11-27T15:23:45.530 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-27T15:23:45.530 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:23:45.656 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.652+0000 7facde0a4700 1 -- 172.21.15.52:0/3630911733 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facd8100500 msgr2=0x7facd8103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.657 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.652+0000 7facde0a4700 1 --2- 172.21.15.52:0/3630911733 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facd8100500 0x7facd8103fa0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7facc80099c0 tx=0x7facc8009cd0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.657 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.652+0000 7facde0a4700 1 -- 172.21.15.52:0/3630911733 shutdown_connections 2023-11-27T15:23:45.657 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.652+0000 7facde0a4700 1 --2- 172.21.15.52:0/3630911733 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7facd8104680 0x7facd810c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.657 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.652+0000 7facde0a4700 1 --2- 172.21.15.52:0/3630911733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facd80ffb60 0x7facd80fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.658 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.652+0000 7facde0a4700 1 --2- 172.21.15.52:0/3630911733 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facd8100500 0x7facd8103fa0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.658 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.652+0000 7facde0a4700 1 -- 172.21.15.52:0/3630911733 >> 172.21.15.52:0/3630911733 conn(0x7facd8075820 msgr2=0x7facd8075c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:45.658 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.652+0000 7facde0a4700 1 -- 172.21.15.52:0/3630911733 shutdown_connections 2023-11-27T15:23:45.658 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.652+0000 7facde0a4700 1 -- 172.21.15.52:0/3630911733 wait complete. 2023-11-27T15:23:45.659 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.653+0000 7facde0a4700 1 Processor -- start 2023-11-27T15:23:45.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.653+0000 7facde0a4700 1 -- start start 2023-11-27T15:23:45.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.653+0000 7facde0a4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facd80ffb60 0x7facd8101040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.653+0000 7facde0a4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7facd8100500 0x7facd8101580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.653+0000 7facde0a4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facd8104680 0x7facd81a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.653+0000 7facde0a4700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd8101db0 con 0x7facd8104680 2023-11-27T15:23:45.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.653+0000 7facde0a4700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd81a4cb0 con 0x7facd8100500 2023-11-27T15:23:45.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.653+0000 7facde0a4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd81a4df0 con 0x7facd80ffb60 2023-11-27T15:23:45.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.653+0000 7facd7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facd8104680 0x7facd81a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.653+0000 7facd7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facd8104680 0x7facd81a4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54144/0 (socket says 172.21.15.52:54144) 2023-11-27T15:23:45.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.653+0000 7facd7fff700 1 -- 172.21.15.52:0/1831310006 learned_addr learned my addr 172.21.15.52:0/1831310006 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:45.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.653+0000 7facd6ffd700 1 --2- 172.21.15.52:0/1831310006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7facd8100500 0x7facd8101580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.654+0000 7facd77fe700 1 --2- 172.21.15.52:0/1831310006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facd80ffb60 0x7facd8101040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.654+0000 7facd7fff700 1 -- 172.21.15.52:0/1831310006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facd80ffb60 msgr2=0x7facd8101040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.654+0000 7facd7fff700 1 --2- 172.21.15.52:0/1831310006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facd80ffb60 0x7facd8101040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.654+0000 7facd7fff700 1 -- 172.21.15.52:0/1831310006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7facd8100500 msgr2=0x7facd8101580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.654+0000 7facd7fff700 1 --2- 172.21.15.52:0/1831310006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7facd8100500 0x7facd8101580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.654+0000 7facd7fff700 1 -- 172.21.15.52:0/1831310006 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc80095f0 con 0x7facd8104680 2023-11-27T15:23:45.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.654+0000 7facd7fff700 1 --2- 172.21.15.52:0/1831310006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facd8104680 0x7facd81a4770 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7facd000ef30 tx=0x7facd000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:45.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.654+0000 7facd4ff9700 1 -- 172.21.15.52:0/1831310006 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd0018070 con 0x7facd8104680 2023-11-27T15:23:45.666 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.654+0000 7facd4ff9700 1 -- 172.21.15.52:0/1831310006 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facd000f040 con 0x7facd8104680 2023-11-27T15:23:45.666 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.654+0000 7facde0a4700 1 -- 172.21.15.52:0/1831310006 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd810f450 con 0x7facd8104680 2023-11-27T15:23:45.666 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.654+0000 7facd4ff9700 1 -- 172.21.15.52:0/1831310006 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd000cdb0 con 0x7facd8104680 2023-11-27T15:23:45.666 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.654+0000 7facde0a4700 1 -- 172.21.15.52:0/1831310006 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd810f2c0 con 0x7facd8104680 2023-11-27T15:23:45.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.656+0000 7facd4ff9700 1 -- 172.21.15.52:0/1831310006 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7facd0003a00 con 0x7facd8104680 2023-11-27T15:23:45.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.656+0000 7facde0a4700 1 -- 172.21.15.52:0/1831310006 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facd810f5e0 con 0x7facd8104680 2023-11-27T15:23:45.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.657+0000 7facd4ff9700 1 --2- 172.21.15.52:0/1831310006 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7facc00a12c0 0x7facc00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.657+0000 7facd4ff9700 1 -- 172.21.15.52:0/1831310006 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7facd00cc850 con 0x7facd8104680 2023-11-27T15:23:45.668 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.657+0000 7facd77fe700 1 --2- 172.21.15.52:0/1831310006 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7facc00a12c0 0x7facc00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.668 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.657+0000 7facd77fe700 1 --2- 172.21.15.52:0/1831310006 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7facc00a12c0 0x7facc00a3770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faccc005fd0 tx=0x7faccc005f60 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:45.668 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.661+0000 7facd4ff9700 1 -- 172.21.15.52:0/1831310006 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7facd0095cf0 con 0x7facd8104680 2023-11-27T15:23:45.914 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.910+0000 7facde0a4700 1 -- 172.21.15.52:0/1831310006 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7facd80671d0 con 0x7facd8104680 2023-11-27T15:23:45.914 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.911+0000 7facd4ff9700 1 -- 172.21.15.52:0/1831310006 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10813 (secure 0 0 0) 0x7facd0099310 con 0x7facd8104680 2023-11-27T15:23:45.915 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:23:45.915 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2328881376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2328881376},{"type":"v1","addr":"172.21.15.52:6839","nonce":2328881376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2799551570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2799551570},{"type":"v1","addr":"172.21.15.52:6841","nonce":2799551570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:16:09.319630+0000","modified":"2023-11-27T15:21:19.128122+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24494,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/215725838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":215725838},{"type":"v1","addr":"172.21.15.150:6839","nonce":215725838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3571830559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3571830559},{"type":"v1","addr":"172.21.15.102:6839","nonce":3571830559}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:23:45.917 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.913+0000 7facde0a4700 1 -- 172.21.15.52:0/1831310006 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7facc00a12c0 msgr2=0x7facc00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.917 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.913+0000 7facde0a4700 1 --2- 172.21.15.52:0/1831310006 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7facc00a12c0 0x7facc00a3770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faccc005fd0 tx=0x7faccc005f60 comp rx=0 tx=0).stop 2023-11-27T15:23:45.917 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.913+0000 7facde0a4700 1 -- 172.21.15.52:0/1831310006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facd8104680 msgr2=0x7facd81a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.918 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.913+0000 7facde0a4700 1 --2- 172.21.15.52:0/1831310006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facd8104680 0x7facd81a4770 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7facd000ef30 tx=0x7facd000c3a0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.918 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.913+0000 7facde0a4700 1 -- 172.21.15.52:0/1831310006 shutdown_connections 2023-11-27T15:23:45.918 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.913+0000 7facde0a4700 1 --2- 172.21.15.52:0/1831310006 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7facc00a12c0 0x7facc00a3770 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.918 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.913+0000 7facde0a4700 1 --2- 172.21.15.52:0/1831310006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7facd8104680 0x7facd81a4770 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.913+0000 7facde0a4700 1 --2- 172.21.15.52:0/1831310006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7facd80ffb60 0x7facd8101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.913+0000 7facde0a4700 1 --2- 172.21.15.52:0/1831310006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7facd8100500 0x7facd8101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.913+0000 7facde0a4700 1 -- 172.21.15.52:0/1831310006 >> 172.21.15.52:0/1831310006 conn(0x7facd8075820 msgr2=0x7facd81031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:45.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.913+0000 7facde0a4700 1 -- 172.21.15.52:0/1831310006 shutdown_connections 2023-11-27T15:23:45.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:45.913+0000 7facde0a4700 1 -- 172.21.15.52:0/1831310006 wait complete. 2023-11-27T15:23:45.920 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 28 2023-11-27T15:23:45.929 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-27T15:23:46.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.055+0000 7f1194622700 1 -- 172.21.15.52:0/1675832078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f118c1055f0 msgr2=0x7f118c10d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.055+0000 7f1194622700 1 --2- 172.21.15.52:0/1675832078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f118c1055f0 0x7f118c10d1c0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f118400b0d0 tx=0x7f118400b3e0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.055+0000 7f1194622700 1 -- 172.21.15.52:0/1675832078 shutdown_connections 2023-11-27T15:23:46.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.055+0000 7f1194622700 1 --2- 172.21.15.52:0/1675832078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f118c1005b0 0x7f118c100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.060 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.055+0000 7f1194622700 1 --2- 172.21.15.52:0/1675832078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f118c1055f0 0x7f118c10d1c0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.060 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.055+0000 7f1194622700 1 --2- 172.21.15.52:0/1675832078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f118c100f50 0x7f118c104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.060 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.055+0000 7f1194622700 1 -- 172.21.15.52:0/1675832078 >> 172.21.15.52:0/1675832078 conn(0x7f118c077170 msgr2=0x7f118c077570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:46.060 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.055+0000 7f1194622700 1 -- 172.21.15.52:0/1675832078 shutdown_connections 2023-11-27T15:23:46.061 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.055+0000 7f1194622700 1 -- 172.21.15.52:0/1675832078 wait complete. 2023-11-27T15:23:46.064 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.055+0000 7f1194622700 1 Processor -- start 2023-11-27T15:23:46.064 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.055+0000 7f1194622700 1 -- start start 2023-11-27T15:23:46.064 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.056+0000 7f1194622700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f118c1005b0 0x7f118c19e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.064 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.056+0000 7f1194622700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f118c100f50 0x7f118c19ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.065 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.056+0000 7f1194622700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f118c1055f0 0x7f118c198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.065 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.056+0000 7f1194622700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c198bf0 con 0x7f118c1005b0 2023-11-27T15:23:46.065 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.056+0000 7f1194622700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c198d60 con 0x7f118c100f50 2023-11-27T15:23:46.065 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.056+0000 7f1194622700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c198f20 con 0x7f118c1055f0 2023-11-27T15:23:46.065 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.056+0000 7f11923be700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f118c1005b0 0x7f118c19e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.066 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.056+0000 7f11923be700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f118c1005b0 0x7f118c19e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54152/0 (socket says 172.21.15.52:54152) 2023-11-27T15:23:46.066 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.056+0000 7f11923be700 1 -- 172.21.15.52:0/3143777432 learned_addr learned my addr 172.21.15.52:0/3143777432 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:46.066 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.056+0000 7f1191bbd700 1 --2- 172.21.15.52:0/3143777432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f118c100f50 0x7f118c19ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.066 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.056+0000 7f1192bbf700 1 --2- 172.21.15.52:0/3143777432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f118c1055f0 0x7f118c198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.067 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.056+0000 7f11923be700 1 -- 172.21.15.52:0/3143777432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f118c1055f0 msgr2=0x7f118c198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.067 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.056+0000 7f11923be700 1 --2- 172.21.15.52:0/3143777432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f118c1055f0 0x7f118c198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.067 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.056+0000 7f11923be700 1 -- 172.21.15.52:0/3143777432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f118c100f50 msgr2=0x7f118c19ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.067 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.056+0000 7f11923be700 1 --2- 172.21.15.52:0/3143777432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f118c100f50 0x7f118c19ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.056+0000 7f11923be700 1 -- 172.21.15.52:0/3143777432 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1184009d00 con 0x7f118c1005b0 2023-11-27T15:23:46.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.056+0000 7f1191bbd700 1 --2- 172.21.15.52:0/3143777432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f118c100f50 0x7f118c19ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:46.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.056+0000 7f11923be700 1 --2- 172.21.15.52:0/3143777432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f118c1005b0 0x7f118c19e510 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f118800e9a0 tx=0x7f118800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:46.068 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.056+0000 7f11837fe700 1 -- 172.21.15.52:0/3143777432 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1188009790 con 0x7f118c1005b0 2023-11-27T15:23:46.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.056+0000 7f11837fe700 1 -- 172.21.15.52:0/3143777432 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f118800cbd0 con 0x7f118c1005b0 2023-11-27T15:23:46.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.056+0000 7f1194622700 1 -- 172.21.15.52:0/3143777432 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f118c110420 con 0x7f118c1005b0 2023-11-27T15:23:46.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.056+0000 7f11837fe700 1 -- 172.21.15.52:0/3143777432 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1188018680 con 0x7f118c1005b0 2023-11-27T15:23:46.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.056+0000 7f1194622700 1 -- 172.21.15.52:0/3143777432 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f118c110290 con 0x7f118c1005b0 2023-11-27T15:23:46.069 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.059+0000 7f11837fe700 1 -- 172.21.15.52:0/3143777432 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f118800cd40 con 0x7f118c1005b0 2023-11-27T15:23:46.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.059+0000 7f11837fe700 1 --2- 172.21.15.52:0/3143777432 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f11780a1340 0x7f11780a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.059+0000 7f1191bbd700 1 --2- 172.21.15.52:0/3143777432 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f11780a1340 0x7f11780a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.059+0000 7f11837fe700 1 -- 172.21.15.52:0/3143777432 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f1188014070 con 0x7f118c1005b0 2023-11-27T15:23:46.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.060+0000 7f1191bbd700 1 --2- 172.21.15.52:0/3143777432 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f11780a1340 0x7f11780a37f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f117c005e10 tx=0x7f117c005d60 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:46.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.060+0000 7f11a07aeb80 1 Processor -- start 2023-11-27T15:23:46.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.060+0000 7f11a07aeb80 1 -- start start 2023-11-27T15:23:46.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.060+0000 7f11a07aeb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab77153b60 0x55ab77153f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.060+0000 7f11a07aeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab77154470 0x55ab77158370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.060+0000 7f11a07aeb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab771588b0 0x55ab77158d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.060+0000 7f11a07aeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab77059060 con 0x55ab77154470 2023-11-27T15:23:46.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.060+0000 7f11a07aeb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab770ab8a0 con 0x55ab771588b0 2023-11-27T15:23:46.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.060+0000 7f11a07aeb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab770a4d30 con 0x55ab77153b60 2023-11-27T15:23:46.078 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.061+0000 7f1191bbd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab77154470 0x55ab77158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.078 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.061+0000 7f1191bbd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab77154470 0x55ab77158370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54162/0 (socket says 172.21.15.52:54162) 2023-11-27T15:23:46.078 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.061+0000 7f1192bbf700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab771588b0 0x55ab77158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.078 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.061+0000 7f1191bbd700 1 -- 172.21.15.52:0/2080234853 learned_addr learned my addr 172.21.15.52:0/2080234853 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:46.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.061+0000 7f11923be700 1 --2- 172.21.15.52:0/2080234853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab77153b60 0x55ab77153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.061+0000 7f1191bbd700 1 -- 172.21.15.52:0/2080234853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab77153b60 msgr2=0x55ab77153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:46.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.061+0000 7f1191bbd700 1 --2- 172.21.15.52:0/2080234853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab77153b60 0x55ab77153f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.061+0000 7f1191bbd700 1 -- 172.21.15.52:0/2080234853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab771588b0 msgr2=0x55ab77158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:46.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.061+0000 7f1191bbd700 1 --2- 172.21.15.52:0/2080234853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab771588b0 0x55ab77158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.061+0000 7f1191bbd700 1 -- 172.21.15.52:0/2080234853 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab76f26630 con 0x55ab77154470 2023-11-27T15:23:46.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.061+0000 7f1192bbf700 1 --2- 172.21.15.52:0/2080234853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab771588b0 0x55ab77158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:46.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.061+0000 7f1191bbd700 1 --2- 172.21.15.52:0/2080234853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab77154470 0x55ab77158370 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f117c00b880 tx=0x7f117c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81da47df3be23433 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:46.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.061+0000 7f1180ff9700 1 -- 172.21.15.52:0/2080234853 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c017040 con 0x55ab77154470 2023-11-27T15:23:46.081 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.061+0000 7f1180ff9700 1 -- 172.21.15.52:0/2080234853 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f117c003b40 con 0x55ab77154470 2023-11-27T15:23:46.081 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.061+0000 7f1180ff9700 1 -- 172.21.15.52:0/2080234853 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c01fbb0 con 0x55ab77154470 2023-11-27T15:23:46.081 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.062+0000 7f11a07aeb80 1 -- 172.21.15.52:0/2080234853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab77154470 msgr2=0x55ab77158370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.081 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.062+0000 7f11a07aeb80 1 --2- 172.21.15.52:0/2080234853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab77154470 0x55ab77158370 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f117c00b880 tx=0x7f117c012000 comp rx=0 tx=0).stop 2023-11-27T15:23:46.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.062+0000 7f11a07aeb80 1 -- 172.21.15.52:0/2080234853 shutdown_connections 2023-11-27T15:23:46.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.062+0000 7f11a07aeb80 1 --2- 172.21.15.52:0/2080234853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab771588b0 0x55ab77158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.062+0000 7f11a07aeb80 1 --2- 172.21.15.52:0/2080234853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab77153b60 0x55ab77153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.062+0000 7f11a07aeb80 1 --2- 172.21.15.52:0/2080234853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab77154470 0x55ab77158370 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.062+0000 7f11a07aeb80 1 -- 172.21.15.52:0/2080234853 >> 172.21.15.52:0/2080234853 conn(0x55ab77149360 msgr2=0x55ab7714eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:46.083 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.062+0000 7f11a07aeb80 1 -- 172.21.15.52:0/2080234853 shutdown_connections 2023-11-27T15:23:46.083 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.062+0000 7f11a07aeb80 1 -- 172.21.15.52:0/2080234853 wait complete. 2023-11-27T15:23:46.083 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.062+0000 7f11a07aeb80 1 -- start start 2023-11-27T15:23:46.083 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.063+0000 7f11a07aeb80 1 Processor -- start 2023-11-27T15:23:46.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.063+0000 7f11a07aeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab77153b60 0x55ab77160060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.063+0000 7f11a07aeb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab77154470 0x55ab7715d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.063+0000 7f11a07aeb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab771588b0 0x55ab7715d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.063+0000 7f11a07aeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab770468e0 con 0x55ab77153b60 2023-11-27T15:23:46.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.063+0000 7f11a07aeb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab77046580 con 0x55ab771588b0 2023-11-27T15:23:46.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.063+0000 7f11a07aeb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab76f28920 con 0x55ab77154470 2023-11-27T15:23:46.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.063+0000 7f11a07aeb80 10 client.0 _ll_register_callbacks cb 0x55ab77064780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:46.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.063+0000 7f11923be700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab77153b60 0x55ab77160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.063+0000 7f11923be700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab77153b60 0x55ab77160060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54166/0 (socket says 172.21.15.52:54166) 2023-11-27T15:23:46.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.063+0000 7f11923be700 1 -- 172.21.15.52:0/114171676 learned_addr learned my addr 172.21.15.52:0/114171676 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:46.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.063+0000 7f1191bbd700 1 --2- 172.21.15.52:0/114171676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab77154470 0x55ab7715d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.087 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.063+0000 7f1192bbf700 1 --2- 172.21.15.52:0/114171676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab771588b0 0x55ab7715d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.087 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.063+0000 7f11923be700 1 -- 172.21.15.52:0/114171676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab77154470 msgr2=0x55ab7715d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:46.087 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.063+0000 7f11923be700 1 --2- 172.21.15.52:0/114171676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab77154470 0x55ab7715d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.087 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.063+0000 7f11923be700 1 -- 172.21.15.52:0/114171676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab771588b0 msgr2=0x55ab7715d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:46.088 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.063+0000 7f11923be700 1 --2- 172.21.15.52:0/114171676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab771588b0 0x55ab7715d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.088 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.063+0000 7f11923be700 1 -- 172.21.15.52:0/114171676 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f117c004170 con 0x55ab77153b60 2023-11-27T15:23:46.088 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.064+0000 7f11923be700 1 --2- 172.21.15.52:0/114171676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab77153b60 0x55ab77160060 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f118800c5b0 tx=0x7f118809e120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea8fad30532b629c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:46.088 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.064+0000 7f1175ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:23:46.089 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.064+0000 7f1175ffb700 1 -- 172.21.15.52:0/114171676 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11880a20b0 con 0x55ab77153b60 2023-11-27T15:23:46.089 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.064+0000 7f1175ffb700 1 -- 172.21.15.52:0/114171676 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f11880a2210 con 0x55ab77153b60 2023-11-27T15:23:46.089 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.064+0000 7f11a07aeb80 1 -- 172.21.15.52:0/114171676 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab77146b20 con 0x55ab77153b60 2023-11-27T15:23:46.089 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.064+0000 7f1175ffb700 1 -- 172.21.15.52:0/114171676 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1188095ee0 con 0x55ab77153b60 2023-11-27T15:23:46.089 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.064+0000 7f11a07aeb80 10 client.16305 fetch_fsmap learned FSMap version 28 2023-11-27T15:23:46.090 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.064+0000 7f11a07aeb80 1 -- 172.21.15.52:0/114171676 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ab7706b190 con 0x55ab77153b60 2023-11-27T15:23:46.090 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.064+0000 7f1175ffb700 1 -- 172.21.15.52:0/114171676 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ab77146b20 con 0x55ab77153b60 2023-11-27T15:23:46.090 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.064+0000 7f1175ffb700 1 -- 172.21.15.52:0/114171676 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f118809aed0 con 0x55ab77153b60 2023-11-27T15:23:46.090 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.064+0000 7f11a07aeb80 10 client.16305 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:23:46.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.064+0000 7f11a07aeb80 10 client.16305 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:23:46.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.064+0000 7f11a07aeb80 20 client.16305 populate_metadata read hostname 'smithi052' 2023-11-27T15:23:46.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.064+0000 7f11a07aeb80 1 --2- 172.21.15.52:0/114171676 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55ab7716d830 0x55ab7716fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.064+0000 7f11a07aeb80 4 client.16305 mds_command: new command op to 24472 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:23:46.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.064+0000 7f11a07aeb80 1 -- 172.21.15.52:0/114171676 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab770670a0 con 0x55ab7716d830 2023-11-27T15:23:46.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.065+0000 7f1191bbd700 1 --2- 172.21.15.52:0/114171676 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55ab7716d830 0x55ab7716fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.065+0000 7f1175ffb700 10 client.16305 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:23:46.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.065+0000 7f1191bbd700 1 --2- 172.21.15.52:0/114171676 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55ab7716d830 0x55ab7716fc40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e57cbdd4ab365a75 server_cookie=10d22bdb693252db in_seq=0 out_seq=0 2023-11-27T15:23:46.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.066+0000 7f1175ffb700 1 -- 172.21.15.52:0/114171676 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ab770670a0 con 0x55ab7716d830 2023-11-27T15:23:46.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.066+0000 7f1175ffb700 10 client.16305 handle_command_reply: tid=0 2023-11-27T15:23:46.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.066+0000 7f11a07aeb80 1 client.16305 shutdown 2023-11-27T15:23:46.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.066+0000 7f11a07aeb80 1 -- 172.21.15.52:0/114171676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab77153b60 msgr2=0x55ab77160060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.066+0000 7f11a07aeb80 1 --2- 172.21.15.52:0/114171676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab77153b60 0x55ab77160060 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f118800c5b0 tx=0x7f118809e120 comp rx=0 tx=0).stop 2023-11-27T15:23:46.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.066+0000 7f11a07aeb80 1 -- 172.21.15.52:0/114171676 shutdown_connections 2023-11-27T15:23:46.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.066+0000 7f11a07aeb80 1 --2- 172.21.15.52:0/114171676 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55ab7716d830 0x55ab7716fc40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.066+0000 7f11a07aeb80 1 --2- 172.21.15.52:0/114171676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab771588b0 0x55ab7715d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.066+0000 7f11a07aeb80 1 --2- 172.21.15.52:0/114171676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab77153b60 0x55ab77160060 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.066+0000 7f11a07aeb80 1 --2- 172.21.15.52:0/114171676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab77154470 0x55ab7715d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.066+0000 7f1175ffb700 0 client.16305 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:23:46.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.066+0000 7f11a07aeb80 1 -- 172.21.15.52:0/114171676 >> 172.21.15.52:0/114171676 conn(0x55ab77149360 msgr2=0x55ab77159d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:46.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.066+0000 7f11a07aeb80 1 -- 172.21.15.52:0/114171676 shutdown_connections 2023-11-27T15:23:46.096 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.066+0000 7f11a07aeb80 1 -- 172.21.15.52:0/114171676 wait complete. 2023-11-27T15:23:46.096 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.066+0000 7f11a07aeb80 20 client.16305 trim_cache size 0 max 16384 2023-11-27T15:23:46.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.079+0000 7f11a07aeb80 1 Processor -- start 2023-11-27T15:23:46.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.079+0000 7f11a07aeb80 1 -- start start 2023-11-27T15:23:46.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.079+0000 7f11a07aeb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab77153b60 0x55ab7715fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.079+0000 7f11a07aeb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab77154470 0x55ab7714e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.079+0000 7f11a07aeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab771588b0 0x55ab7714ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.079+0000 7f11a07aeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab7715e890 con 0x55ab771588b0 2023-11-27T15:23:46.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.079+0000 7f11a07aeb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab77154950 con 0x55ab77154470 2023-11-27T15:23:46.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.079+0000 7f11a07aeb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab771469a0 con 0x55ab77153b60 2023-11-27T15:23:46.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.080+0000 7f1192bbf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab771588b0 0x55ab7714ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.080+0000 7f1192bbf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab771588b0 0x55ab7714ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54178/0 (socket says 172.21.15.52:54178) 2023-11-27T15:23:46.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.080+0000 7f1192bbf700 1 -- 172.21.15.52:0/2687576498 learned_addr learned my addr 172.21.15.52:0/2687576498 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:46.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.080+0000 7f1191bbd700 1 --2- 172.21.15.52:0/2687576498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab77154470 0x55ab7714e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.080+0000 7f11923be700 1 --2- 172.21.15.52:0/2687576498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab77153b60 0x55ab7715fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.080+0000 7f1192bbf700 1 -- 172.21.15.52:0/2687576498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab77153b60 msgr2=0x55ab7715fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:46.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.080+0000 7f1192bbf700 1 --2- 172.21.15.52:0/2687576498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab77153b60 0x55ab7715fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.080+0000 7f1192bbf700 1 -- 172.21.15.52:0/2687576498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab77154470 msgr2=0x55ab7714e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:46.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.080+0000 7f1192bbf700 1 --2- 172.21.15.52:0/2687576498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab77154470 0x55ab7714e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.080+0000 7f1192bbf700 1 -- 172.21.15.52:0/2687576498 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab77158f50 con 0x55ab771588b0 2023-11-27T15:23:46.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.080+0000 7f11923be700 1 --2- 172.21.15.52:0/2687576498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab77153b60 0x55ab7715fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:23:46.106 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.080+0000 7f1192bbf700 1 --2- 172.21.15.52:0/2687576498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab771588b0 0x55ab7714ec30 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1184009520 tx=0x7f1184009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84934fe2b4785281 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:46.106 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.080+0000 7f11817fa700 1 -- 172.21.15.52:0/2687576498 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118400e040 con 0x55ab771588b0 2023-11-27T15:23:46.106 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.080+0000 7f11817fa700 1 -- 172.21.15.52:0/2687576498 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1184022070 con 0x55ab771588b0 2023-11-27T15:23:46.106 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.080+0000 7f11817fa700 1 -- 172.21.15.52:0/2687576498 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11840276d0 con 0x55ab771588b0 2023-11-27T15:23:46.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.081+0000 7f11a07aeb80 1 -- 172.21.15.52:0/2687576498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab771588b0 msgr2=0x55ab7714ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.081+0000 7f11a07aeb80 1 --2- 172.21.15.52:0/2687576498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab771588b0 0x55ab7714ec30 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1184009520 tx=0x7f1184009550 comp rx=0 tx=0).stop 2023-11-27T15:23:46.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.081+0000 7f11a07aeb80 1 -- 172.21.15.52:0/2687576498 shutdown_connections 2023-11-27T15:23:46.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.081+0000 7f11a07aeb80 1 --2- 172.21.15.52:0/2687576498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab771588b0 0x55ab7714ec30 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.108 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.081+0000 7f11a07aeb80 1 --2- 172.21.15.52:0/2687576498 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab77153b60 0x55ab7715fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.108 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.081+0000 7f11a07aeb80 1 --2- 172.21.15.52:0/2687576498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab77154470 0x55ab7714e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.108 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.081+0000 7f11a07aeb80 1 -- 172.21.15.52:0/2687576498 >> 172.21.15.52:0/2687576498 conn(0x55ab77149360 msgr2=0x55ab77169150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:46.108 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.081+0000 7f11a07aeb80 1 -- 172.21.15.52:0/2687576498 shutdown_connections 2023-11-27T15:23:46.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.081+0000 7f11a07aeb80 1 -- 172.21.15.52:0/2687576498 wait complete. 2023-11-27T15:23:46.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.081+0000 7f11a07aeb80 1 -- start start 2023-11-27T15:23:46.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.081+0000 7f11a07aeb80 1 Processor -- start 2023-11-27T15:23:46.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.082+0000 7f11a07aeb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab77153b60 0x55ab771b5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.082+0000 7f11a07aeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab77154470 0x55ab771b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.082+0000 7f11a07aeb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab771588b0 0x55ab771c0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.082+0000 7f11a07aeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab771b64d0 con 0x55ab77154470 2023-11-27T15:23:46.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.082+0000 7f11a07aeb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab771c0c40 con 0x55ab77153b60 2023-11-27T15:23:46.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.082+0000 7f11a07aeb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab771c0d80 con 0x55ab771588b0 2023-11-27T15:23:46.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.082+0000 7f11a07aeb80 10 client.0 _ll_register_callbacks cb 0x7f1188099a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:46.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.082+0000 7f1191bbd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab77154470 0x55ab771b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.082+0000 7f1191bbd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab77154470 0x55ab771b5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54182/0 (socket says 172.21.15.52:54182) 2023-11-27T15:23:46.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.082+0000 7f1191bbd700 1 -- 172.21.15.52:0/394875933 learned_addr learned my addr 172.21.15.52:0/394875933 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:46.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.082+0000 7f11923be700 1 --2- 172.21.15.52:0/394875933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab77153b60 0x55ab771b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.082+0000 7f1192bbf700 1 --2- 172.21.15.52:0/394875933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab771588b0 0x55ab771c0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.082+0000 7f1191bbd700 1 -- 172.21.15.52:0/394875933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab771588b0 msgr2=0x55ab771c0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:46.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.082+0000 7f1191bbd700 1 --2- 172.21.15.52:0/394875933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab771588b0 0x55ab771c0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.082+0000 7f1191bbd700 1 -- 172.21.15.52:0/394875933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab77153b60 msgr2=0x55ab771b5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:46.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.082+0000 7f1191bbd700 1 --2- 172.21.15.52:0/394875933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab77153b60 0x55ab771b5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.082+0000 7f1191bbd700 1 -- 172.21.15.52:0/394875933 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11840092e0 con 0x55ab77154470 2023-11-27T15:23:46.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.082+0000 7f1192bbf700 1 --2- 172.21.15.52:0/394875933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab771588b0 0x55ab771c0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:23:46.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.082+0000 7f11923be700 1 --2- 172.21.15.52:0/394875933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab77153b60 0x55ab771b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:23:46.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.082+0000 7f1191bbd700 1 --2- 172.21.15.52:0/394875933 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab77154470 0x55ab771b5bc0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f117c00b880 tx=0x7f117c01f4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35886d21c948ac0a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:46.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.082+0000 7f1175ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:23:46.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.082+0000 7f1175ffb700 1 -- 172.21.15.52:0/394875933 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c030070 con 0x55ab77154470 2023-11-27T15:23:46.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.082+0000 7f1175ffb700 1 -- 172.21.15.52:0/394875933 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f117c003880 con 0x55ab77154470 2023-11-27T15:23:46.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.082+0000 7f11a07aeb80 1 -- 172.21.15.52:0/394875933 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f117c003b40 con 0x55ab77154470 2023-11-27T15:23:46.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.083+0000 7f1175ffb700 1 -- 172.21.15.52:0/394875933 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f117c034780 con 0x55ab77154470 2023-11-27T15:23:46.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.083+0000 7f11a07aeb80 10 client.16311 fetch_fsmap learned FSMap version 28 2023-11-27T15:23:46.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.083+0000 7f11a07aeb80 1 -- 172.21.15.52:0/394875933 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ab76ed2de0 con 0x55ab77154470 2023-11-27T15:23:46.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.083+0000 7f1175ffb700 1 -- 172.21.15.52:0/394875933 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f117c003b40 con 0x55ab77154470 2023-11-27T15:23:46.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.083+0000 7f1175ffb700 1 -- 172.21.15.52:0/394875933 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f117c01d040 con 0x55ab77154470 2023-11-27T15:23:46.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.083+0000 7f11a07aeb80 10 client.16311 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:23:46.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.083+0000 7f11a07aeb80 10 client.16311 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:23:46.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.083+0000 7f11a07aeb80 20 client.16311 populate_metadata read hostname 'smithi052' 2023-11-27T15:23:46.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.083+0000 7f11a07aeb80 1 --2- 172.21.15.52:0/394875933 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55ab771c2430 0x55ab771c4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.083+0000 7f11a07aeb80 4 client.16311 mds_command: new command op to 24472 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:23:46.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.083+0000 7f11a07aeb80 1 -- 172.21.15.52:0/394875933 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ab77073660 con 0x55ab771c2430 2023-11-27T15:23:46.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.083+0000 7f11923be700 1 --2- 172.21.15.52:0/394875933 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55ab771c2430 0x55ab771c4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.084+0000 7f1175ffb700 10 client.16311 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:23:46.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.084+0000 7f11923be700 1 --2- 172.21.15.52:0/394875933 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55ab771c2430 0x55ab771c4840 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aea49c6637a027d0 server_cookie=9e98c18e03360b45 in_seq=0 out_seq=0 2023-11-27T15:23:46.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.084+0000 7f1175ffb700 1 -- 172.21.15.52:0/394875933 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ab77073660 con 0x55ab771c2430 2023-11-27T15:23:46.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.084+0000 7f1175ffb700 10 client.16311 handle_command_reply: tid=0 2023-11-27T15:23:46.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.084+0000 7f11a07aeb80 1 client.16311 shutdown 2023-11-27T15:23:46.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.084+0000 7f11a07aeb80 1 -- 172.21.15.52:0/394875933 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab77154470 msgr2=0x55ab771b5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.084+0000 7f11a07aeb80 1 --2- 172.21.15.52:0/394875933 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab77154470 0x55ab771b5bc0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f117c00b880 tx=0x7f117c01f4e0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.084+0000 7f11a07aeb80 1 -- 172.21.15.52:0/394875933 shutdown_connections 2023-11-27T15:23:46.120 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.084+0000 7f11a07aeb80 1 --2- 172.21.15.52:0/394875933 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55ab771c2430 0x55ab771c4840 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.120 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.084+0000 7f11a07aeb80 1 --2- 172.21.15.52:0/394875933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab77153b60 0x55ab771b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.120 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.084+0000 7f1175ffb700 0 client.16311 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:23:46.120 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.084+0000 7f11a07aeb80 1 --2- 172.21.15.52:0/394875933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ab771588b0 0x55ab771c0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.120 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.084+0000 7f11a07aeb80 1 --2- 172.21.15.52:0/394875933 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab77154470 0x55ab771b5bc0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.121 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.084+0000 7f11a07aeb80 1 -- 172.21.15.52:0/394875933 >> 172.21.15.52:0/394875933 conn(0x55ab77149360 msgr2=0x55ab7714e660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:46.121 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.084+0000 7f11a07aeb80 1 -- 172.21.15.52:0/394875933 shutdown_connections 2023-11-27T15:23:46.121 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.084+0000 7f11a07aeb80 1 -- 172.21.15.52:0/394875933 wait complete. 2023-11-27T15:23:46.121 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.084+0000 7f11a07aeb80 20 client.16311 trim_cache size 0 max 16384 2023-11-27T15:23:46.122 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:23:46.123 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.085+0000 7f1194622700 1 -- 172.21.15.52:0/3143777432 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f11780a1340 msgr2=0x7f11780a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.123 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.085+0000 7f1194622700 1 --2- 172.21.15.52:0/3143777432 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f11780a1340 0x7f11780a37f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f117c005e10 tx=0x7f117c005d60 comp rx=0 tx=0).stop 2023-11-27T15:23:46.124 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.085+0000 7f1194622700 1 -- 172.21.15.52:0/3143777432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f118c1005b0 msgr2=0x7f118c19e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.124 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.085+0000 7f1194622700 1 --2- 172.21.15.52:0/3143777432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f118c1005b0 0x7f118c19e510 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f118800e9a0 tx=0x7f118800ed60 comp rx=0 tx=0).stop 2023-11-27T15:23:46.124 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.085+0000 7f1194622700 1 -- 172.21.15.52:0/3143777432 shutdown_connections 2023-11-27T15:23:46.124 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.085+0000 7f1194622700 1 --2- 172.21.15.52:0/3143777432 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f11780a1340 0x7f11780a37f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.124 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.085+0000 7f1194622700 1 --2- 172.21.15.52:0/3143777432 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f118c1055f0 0x7f118c198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.124 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.085+0000 7f1194622700 1 --2- 172.21.15.52:0/3143777432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f118c1005b0 0x7f118c19e510 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.124 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.085+0000 7f1194622700 1 --2- 172.21.15.52:0/3143777432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f118c100f50 0x7f118c19ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.124 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.085+0000 7f1194622700 1 -- 172.21.15.52:0/3143777432 >> 172.21.15.52:0/3143777432 conn(0x7f118c077170 msgr2=0x7f118c0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:46.125 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.085+0000 7f1194622700 1 -- 172.21.15.52:0/3143777432 shutdown_connections 2023-11-27T15:23:46.125 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.085+0000 7f1194622700 1 -- 172.21.15.52:0/3143777432 wait complete. 2023-11-27T15:23:46.125 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-27T15:23:46.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.249+0000 7f8c534c9700 1 -- 172.21.15.52:0/55594904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c4c0ff990 msgr2=0x7f8c4c0ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.249+0000 7f8c534c9700 1 --2- 172.21.15.52:0/55594904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c4c0ff990 0x7f8c4c0ffe00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0099c0 tx=0x7f8c3c009cd0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.250+0000 7f8c534c9700 1 -- 172.21.15.52:0/55594904 shutdown_connections 2023-11-27T15:23:46.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.250+0000 7f8c534c9700 1 --2- 172.21.15.52:0/55594904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c4c100340 0x7f8c4c10c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.250+0000 7f8c534c9700 1 --2- 172.21.15.52:0/55594904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c4c104180 0x7f8c4c104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.250+0000 7f8c534c9700 1 --2- 172.21.15.52:0/55594904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c4c0ff990 0x7f8c4c0ffe00 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.250+0000 7f8c534c9700 1 -- 172.21.15.52:0/55594904 >> 172.21.15.52:0/55594904 conn(0x7f8c4c075920 msgr2=0x7f8c4c075d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:46.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.250+0000 7f8c534c9700 1 -- 172.21.15.52:0/55594904 shutdown_connections 2023-11-27T15:23:46.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.250+0000 7f8c534c9700 1 -- 172.21.15.52:0/55594904 wait complete. 2023-11-27T15:23:46.256 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.250+0000 7f8c534c9700 1 Processor -- start 2023-11-27T15:23:46.256 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.250+0000 7f8c534c9700 1 -- start start 2023-11-27T15:23:46.256 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.251+0000 7f8c534c9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c4c0ff990 0x7f8c4c195220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.257 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.251+0000 7f8c534c9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c4c100340 0x7f8c4c195760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.257 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.251+0000 7f8c534c9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c4c104180 0x7f8c4c199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.257 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.251+0000 7f8c534c9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c19a200 con 0x7f8c4c100340 2023-11-27T15:23:46.257 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.251+0000 7f8c534c9700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c19a370 con 0x7f8c4c0ff990 2023-11-27T15:23:46.257 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.251+0000 7f8c534c9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c19a530 con 0x7f8c4c104180 2023-11-27T15:23:46.257 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.251+0000 7f8c50a64700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c4c100340 0x7f8c4c195760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.257 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.251+0000 7f8c50a64700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c4c100340 0x7f8c4c195760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54196/0 (socket says 172.21.15.52:54196) 2023-11-27T15:23:46.257 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.251+0000 7f8c51265700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c4c0ff990 0x7f8c4c195220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.251+0000 7f8c50a64700 1 -- 172.21.15.52:0/1274996741 learned_addr learned my addr 172.21.15.52:0/1274996741 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:46.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.251+0000 7f8c51a66700 1 --2- 172.21.15.52:0/1274996741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c4c104180 0x7f8c4c199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.251+0000 7f8c50a64700 1 -- 172.21.15.52:0/1274996741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c4c104180 msgr2=0x7f8c4c199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.251+0000 7f8c50a64700 1 --2- 172.21.15.52:0/1274996741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c4c104180 0x7f8c4c199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.251+0000 7f8c50a64700 1 -- 172.21.15.52:0/1274996741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c4c0ff990 msgr2=0x7f8c4c195220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.251+0000 7f8c50a64700 1 --2- 172.21.15.52:0/1274996741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c4c0ff990 0x7f8c4c195220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.251+0000 7f8c50a64700 1 -- 172.21.15.52:0/1274996741 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3c0095f0 con 0x7f8c4c100340 2023-11-27T15:23:46.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.251+0000 7f8c50a64700 1 --2- 172.21.15.52:0/1274996741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c4c100340 0x7f8c4c195760 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c009990 tx=0x7f8c3c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:46.259 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.252+0000 7f8c427fc700 1 -- 172.21.15.52:0/1274996741 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c01d070 con 0x7f8c4c100340 2023-11-27T15:23:46.259 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.252+0000 7f8c427fc700 1 -- 172.21.15.52:0/1274996741 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c3c003f90 con 0x7f8c4c100340 2023-11-27T15:23:46.259 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.252+0000 7f8c534c9700 1 -- 172.21.15.52:0/1274996741 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c4c10f210 con 0x7f8c4c100340 2023-11-27T15:23:46.259 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.252+0000 7f8c427fc700 1 -- 172.21.15.52:0/1274996741 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c022ce0 con 0x7f8c4c100340 2023-11-27T15:23:46.259 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.252+0000 7f8c534c9700 1 -- 172.21.15.52:0/1274996741 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c4c10f080 con 0x7f8c4c100340 2023-11-27T15:23:46.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.255+0000 7f8c427fc700 1 -- 172.21.15.52:0/1274996741 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f8c3c022470 con 0x7f8c4c100340 2023-11-27T15:23:46.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.255+0000 7f8c427fc700 1 --2- 172.21.15.52:0/1274996741 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f8c380a1340 0x7f8c380a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.255+0000 7f8c427fc700 1 -- 172.21.15.52:0/1274996741 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f8c3c0cde80 con 0x7f8c4c100340 2023-11-27T15:23:46.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.255+0000 7f8c51265700 1 --2- 172.21.15.52:0/1274996741 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f8c380a1340 0x7f8c380a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.256+0000 7f8c5f655b80 1 Processor -- start 2023-11-27T15:23:46.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.256+0000 7f8c5f655b80 1 -- start start 2023-11-27T15:23:46.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.256+0000 7f8c51265700 1 --2- 172.21.15.52:0/1274996741 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f8c380a1340 0x7f8c380a37f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8c48005950 tx=0x7f8c4800a300 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:46.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.256+0000 7f8c5f655b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b47f60d70 0x564b47f61140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.256+0000 7f8c5f655b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b47f61680 0x564b47f65580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.256+0000 7f8c5f655b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b47f65ac0 0x564b47f65f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.256+0000 7f8c5f655b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b47e64060 con 0x564b47f65ac0 2023-11-27T15:23:46.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.256+0000 7f8c5f655b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b47eb68a0 con 0x564b47f60d70 2023-11-27T15:23:46.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.256+0000 7f8c5f655b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b47eafd30 con 0x564b47f61680 2023-11-27T15:23:46.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.256+0000 7f8c51a66700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b47f65ac0 0x564b47f65f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.256+0000 7f8c51a66700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b47f65ac0 0x564b47f65f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54206/0 (socket says 172.21.15.52:54206) 2023-11-27T15:23:46.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.256+0000 7f8c51a66700 1 -- 172.21.15.52:0/3794315745 learned_addr learned my addr 172.21.15.52:0/3794315745 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:46.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.256+0000 7f8c51265700 1 --2- 172.21.15.52:0/3794315745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b47f60d70 0x564b47f61140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.256+0000 7f8c50a64700 1 --2- 172.21.15.52:0/3794315745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b47f61680 0x564b47f65580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.257+0000 7f8c51a66700 1 -- 172.21.15.52:0/3794315745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b47f61680 msgr2=0x564b47f65580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:46.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.257+0000 7f8c51a66700 1 --2- 172.21.15.52:0/3794315745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b47f61680 0x564b47f65580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.257+0000 7f8c51a66700 1 -- 172.21.15.52:0/3794315745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b47f60d70 msgr2=0x564b47f61140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:46.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.257+0000 7f8c51a66700 1 --2- 172.21.15.52:0/3794315745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b47f60d70 0x564b47f61140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.257+0000 7f8c51a66700 1 -- 172.21.15.52:0/3794315745 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b47d31630 con 0x564b47f65ac0 2023-11-27T15:23:46.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.257+0000 7f8c51a66700 1 --2- 172.21.15.52:0/3794315745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b47f65ac0 0x564b47f65f10 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009d10 tx=0x7f8c440077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba830fa9fae951a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:46.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.257+0000 7f8c377fe700 1 -- 172.21.15.52:0/3794315745 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c44019070 con 0x564b47f65ac0 2023-11-27T15:23:46.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.257+0000 7f8c377fe700 1 -- 172.21.15.52:0/3794315745 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c4400f050 con 0x564b47f65ac0 2023-11-27T15:23:46.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.257+0000 7f8c377fe700 1 -- 172.21.15.52:0/3794315745 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c44015df0 con 0x564b47f65ac0 2023-11-27T15:23:46.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.258+0000 7f8c5f655b80 1 -- 172.21.15.52:0/3794315745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b47f65ac0 msgr2=0x564b47f65f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.258+0000 7f8c5f655b80 1 --2- 172.21.15.52:0/3794315745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b47f65ac0 0x564b47f65f10 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009d10 tx=0x7f8c440077e0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.258+0000 7f8c5f655b80 1 -- 172.21.15.52:0/3794315745 shutdown_connections 2023-11-27T15:23:46.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.258+0000 7f8c5f655b80 1 --2- 172.21.15.52:0/3794315745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b47f60d70 0x564b47f61140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.258+0000 7f8c5f655b80 1 --2- 172.21.15.52:0/3794315745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b47f65ac0 0x564b47f65f10 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.258+0000 7f8c5f655b80 1 --2- 172.21.15.52:0/3794315745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b47f61680 0x564b47f65580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.258+0000 7f8c5f655b80 1 -- 172.21.15.52:0/3794315745 >> 172.21.15.52:0/3794315745 conn(0x564b47f54360 msgr2=0x564b47f59b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:46.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.258+0000 7f8c5f655b80 1 -- 172.21.15.52:0/3794315745 shutdown_connections 2023-11-27T15:23:46.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.258+0000 7f8c5f655b80 1 -- 172.21.15.52:0/3794315745 wait complete. 2023-11-27T15:23:46.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.258+0000 7f8c5f655b80 1 -- start start 2023-11-27T15:23:46.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.258+0000 7f8c5f655b80 1 Processor -- start 2023-11-27T15:23:46.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.259+0000 7f8c5f655b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b47f60d70 0x564b47f6d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.259+0000 7f8c5f655b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b47f61680 0x564b47f6a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.259+0000 7f8c5f655b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b47f65ac0 0x564b47f6aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.259+0000 7f8c5f655b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b47e518e0 con 0x564b47f61680 2023-11-27T15:23:46.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.259+0000 7f8c5f655b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b47e51580 con 0x564b47f60d70 2023-11-27T15:23:46.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.259+0000 7f8c5f655b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b47d33920 con 0x564b47f65ac0 2023-11-27T15:23:46.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.259+0000 7f8c5f655b80 10 client.0 _ll_register_callbacks cb 0x564b47e6f780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:46.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.259+0000 7f8c50a64700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b47f61680 0x564b47f6a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.259+0000 7f8c51265700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b47f60d70 0x564b47f6d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.259+0000 7f8c50a64700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b47f61680 0x564b47f6a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54210/0 (socket says 172.21.15.52:54210) 2023-11-27T15:23:46.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.259+0000 7f8c51265700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b47f60d70 0x564b47f6d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:43866/0 (socket says 172.21.15.52:43866) 2023-11-27T15:23:46.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.259+0000 7f8c50a64700 1 -- 172.21.15.52:0/1596861485 learned_addr learned my addr 172.21.15.52:0/1596861485 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:46.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.259+0000 7f8c51a66700 1 --2- 172.21.15.52:0/1596861485 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b47f65ac0 0x564b47f6aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.259+0000 7f8c50a64700 1 -- 172.21.15.52:0/1596861485 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b47f65ac0 msgr2=0x564b47f6aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:46.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.259+0000 7f8c50a64700 1 --2- 172.21.15.52:0/1596861485 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b47f65ac0 0x564b47f6aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.259+0000 7f8c50a64700 1 -- 172.21.15.52:0/1596861485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b47f60d70 msgr2=0x564b47f6d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:46.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.259+0000 7f8c50a64700 1 --2- 172.21.15.52:0/1596861485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b47f60d70 0x564b47f6d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.259+0000 7f8c50a64700 1 -- 172.21.15.52:0/1596861485 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c44009940 con 0x564b47f61680 2023-11-27T15:23:46.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.259+0000 7f8c51265700 1 --2- 172.21.15.52:0/1596861485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b47f60d70 0x564b47f6d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:46.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.259+0000 7f8c50a64700 1 --2- 172.21.15.52:0/1596861485 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b47f61680 0x564b47f6a4a0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00c010 tx=0x7f8c3c09a940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8f9445d063cde1 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:46.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.259+0000 7f8c34ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:23:46.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.259+0000 7f8c34ff9700 1 -- 172.21.15.52:0/1596861485 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c09a6d0 con 0x564b47f61680 2023-11-27T15:23:46.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.259+0000 7f8c34ff9700 1 -- 172.21.15.52:0/1596861485 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f8c3c09f010 con 0x564b47f61680 2023-11-27T15:23:46.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.259+0000 7f8c5f655b80 1 -- 172.21.15.52:0/1596861485 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564b47f51b20 con 0x564b47f61680 2023-11-27T15:23:46.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.260+0000 7f8c34ff9700 1 -- 172.21.15.52:0/1596861485 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c3c0b9340 con 0x564b47f61680 2023-11-27T15:23:46.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.260+0000 7f8c5f655b80 10 client.16323 fetch_fsmap learned FSMap version 28 2023-11-27T15:23:46.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.260+0000 7f8c5f655b80 1 -- 172.21.15.52:0/1596861485 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564b47e76190 con 0x564b47f61680 2023-11-27T15:23:46.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.260+0000 7f8c34ff9700 1 -- 172.21.15.52:0/1596861485 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564b47f51b20 con 0x564b47f61680 2023-11-27T15:23:46.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.260+0000 7f8c34ff9700 1 -- 172.21.15.52:0/1596861485 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8c3c0cbd20 con 0x564b47f61680 2023-11-27T15:23:46.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.260+0000 7f8c5f655b80 10 client.16323 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:23:46.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.260+0000 7f8c5f655b80 10 client.16323 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:23:46.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.260+0000 7f8c5f655b80 20 client.16323 populate_metadata read hostname 'smithi052' 2023-11-27T15:23:46.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.260+0000 7f8c5f655b80 1 --2- 172.21.15.52:0/1596861485 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x564b47f5fc90 0x564b47f60090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.260+0000 7f8c5f655b80 4 client.16323 mds_command: new command op to 24494 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:23:46.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.260+0000 7f8c5f655b80 1 -- 172.21.15.52:0/1596861485 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b47e720a0 con 0x564b47f5fc90 2023-11-27T15:23:46.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.260+0000 7f8c51265700 1 --2- 172.21.15.52:0/1596861485 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x564b47f5fc90 0x564b47f60090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.261+0000 7f8c34ff9700 10 client.16323 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:23:46.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.261+0000 7f8c51265700 1 --2- 172.21.15.52:0/1596861485 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x564b47f5fc90 0x564b47f60090 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=98bf72781eed288f server_cookie=1e43a83f5ea7818d in_seq=0 out_seq=0 2023-11-27T15:23:46.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.261+0000 7f8c34ff9700 1 -- 172.21.15.52:0/1596861485 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564b47e720a0 con 0x564b47f5fc90 2023-11-27T15:23:46.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.261+0000 7f8c34ff9700 10 client.16323 handle_command_reply: tid=0 2023-11-27T15:23:46.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.261+0000 7f8c5f655b80 1 client.16323 shutdown 2023-11-27T15:23:46.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.262+0000 7f8c5f655b80 1 -- 172.21.15.52:0/1596861485 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b47f61680 msgr2=0x564b47f6a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.262+0000 7f8c5f655b80 1 --2- 172.21.15.52:0/1596861485 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b47f61680 0x564b47f6a4a0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00c010 tx=0x7f8c3c09a940 comp rx=0 tx=0).stop 2023-11-27T15:23:46.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.262+0000 7f8c5f655b80 1 -- 172.21.15.52:0/1596861485 shutdown_connections 2023-11-27T15:23:46.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.262+0000 7f8c5f655b80 1 --2- 172.21.15.52:0/1596861485 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x564b47f5fc90 0x564b47f60090 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.262+0000 7f8c5f655b80 1 --2- 172.21.15.52:0/1596861485 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b47f60d70 0x564b47f6d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.262+0000 7f8c5f655b80 1 --2- 172.21.15.52:0/1596861485 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b47f65ac0 0x564b47f6aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.262+0000 7f8c34ff9700 0 client.16323 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:23:46.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.262+0000 7f8c5f655b80 1 --2- 172.21.15.52:0/1596861485 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b47f61680 0x564b47f6a4a0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.262+0000 7f8c5f655b80 1 -- 172.21.15.52:0/1596861485 >> 172.21.15.52:0/1596861485 conn(0x564b47f54360 msgr2=0x564b47f66f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:46.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.262+0000 7f8c5f655b80 1 -- 172.21.15.52:0/1596861485 shutdown_connections 2023-11-27T15:23:46.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.262+0000 7f8c5f655b80 1 -- 172.21.15.52:0/1596861485 wait complete. 2023-11-27T15:23:46.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.262+0000 7f8c5f655b80 20 client.16323 trim_cache size 0 max 16384 2023-11-27T15:23:46.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.275+0000 7f8c5f655b80 1 Processor -- start 2023-11-27T15:23:46.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.275+0000 7f8c5f655b80 1 -- start start 2023-11-27T15:23:46.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.275+0000 7f8c5f655b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b47f60d70 0x564b47f59be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.275+0000 7f8c5f655b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b47f61680 0x564b47f7f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.275+0000 7f8c5f655b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b47f65ac0 0x564b47f69ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.275+0000 7f8c5f655b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b47f5b430 con 0x564b47f61680 2023-11-27T15:23:46.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.275+0000 7f8c5f655b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b47f5b1c0 con 0x564b47f65ac0 2023-11-27T15:23:46.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.275+0000 7f8c5f655b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b47f606f0 con 0x564b47f60d70 2023-11-27T15:23:46.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.275+0000 7f8c50a64700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b47f61680 0x564b47f7f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.275+0000 7f8c50a64700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b47f61680 0x564b47f7f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54218/0 (socket says 172.21.15.52:54218) 2023-11-27T15:23:46.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.275+0000 7f8c50a64700 1 -- 172.21.15.52:0/3307520784 learned_addr learned my addr 172.21.15.52:0/3307520784 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:46.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.275+0000 7f8c51a66700 1 --2- 172.21.15.52:0/3307520784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b47f65ac0 0x564b47f69ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.275+0000 7f8c51265700 1 --2- 172.21.15.52:0/3307520784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b47f60d70 0x564b47f59be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.276+0000 7f8c50a64700 1 -- 172.21.15.52:0/3307520784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b47f60d70 msgr2=0x564b47f59be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:46.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.276+0000 7f8c50a64700 1 --2- 172.21.15.52:0/3307520784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b47f60d70 0x564b47f59be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.276+0000 7f8c50a64700 1 -- 172.21.15.52:0/3307520784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b47f65ac0 msgr2=0x564b47f69ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:46.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.276+0000 7f8c50a64700 1 --2- 172.21.15.52:0/3307520784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b47f65ac0 0x564b47f69ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.276+0000 7f8c50a64700 1 -- 172.21.15.52:0/3307520784 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b47e75ee0 con 0x564b47f61680 2023-11-27T15:23:46.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.276+0000 7f8c51a66700 1 --2- 172.21.15.52:0/3307520784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b47f65ac0 0x564b47f69ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:46.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.276+0000 7f8c50a64700 1 --2- 172.21.15.52:0/3307520784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b47f61680 0x564b47f7f3e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0cbbd0 tx=0x7f8c3c099590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c49a4943ca9dfe server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:46.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.276+0000 7f8c37fff700 1 -- 172.21.15.52:0/3307520784 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c022ea0 con 0x564b47f61680 2023-11-27T15:23:46.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.276+0000 7f8c37fff700 1 -- 172.21.15.52:0/3307520784 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c3c097060 con 0x564b47f61680 2023-11-27T15:23:46.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.276+0000 7f8c37fff700 1 -- 172.21.15.52:0/3307520784 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c09a710 con 0x564b47f61680 2023-11-27T15:23:46.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.276+0000 7f8c5f655b80 1 -- 172.21.15.52:0/3307520784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b47f61680 msgr2=0x564b47f7f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.276+0000 7f8c5f655b80 1 --2- 172.21.15.52:0/3307520784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b47f61680 0x564b47f7f3e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0cbbd0 tx=0x7f8c3c099590 comp rx=0 tx=0).stop 2023-11-27T15:23:46.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.277+0000 7f8c5f655b80 1 -- 172.21.15.52:0/3307520784 shutdown_connections 2023-11-27T15:23:46.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.277+0000 7f8c5f655b80 1 --2- 172.21.15.52:0/3307520784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b47f65ac0 0x564b47f69ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.277+0000 7f8c5f655b80 1 --2- 172.21.15.52:0/3307520784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b47f60d70 0x564b47f59be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.277+0000 7f8c5f655b80 1 --2- 172.21.15.52:0/3307520784 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b47f61680 0x564b47f7f3e0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.277+0000 7f8c5f655b80 1 -- 172.21.15.52:0/3307520784 >> 172.21.15.52:0/3307520784 conn(0x564b47f54360 msgr2=0x564b47f5e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:46.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.277+0000 7f8c5f655b80 1 -- 172.21.15.52:0/3307520784 shutdown_connections 2023-11-27T15:23:46.285 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.277+0000 7f8c5f655b80 1 -- 172.21.15.52:0/3307520784 wait complete. 2023-11-27T15:23:46.285 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.277+0000 7f8c5f655b80 1 -- start start 2023-11-27T15:23:46.285 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.277+0000 7f8c5f655b80 1 Processor -- start 2023-11-27T15:23:46.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.277+0000 7f8c5f655b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b47f60d70 0x564b47f7b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.277+0000 7f8c5f655b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b47f61680 0x564b47f7b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.277+0000 7f8c5f655b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b47f65ac0 0x564b47f7be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.277+0000 7f8c5f655b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b47fcb520 con 0x564b47f60d70 2023-11-27T15:23:46.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.277+0000 7f8c5f655b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b47fcb660 con 0x564b47f61680 2023-11-27T15:23:46.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.277+0000 7f8c5f655b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b47fcb7a0 con 0x564b47f65ac0 2023-11-27T15:23:46.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.278+0000 7f8c5f655b80 10 client.0 _ll_register_callbacks cb 0x7f8c3c09ab20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:46.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.278+0000 7f8c51265700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b47f60d70 0x564b47f7b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.278+0000 7f8c51265700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b47f60d70 0x564b47f7b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54222/0 (socket says 172.21.15.52:54222) 2023-11-27T15:23:46.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.278+0000 7f8c51265700 1 -- 172.21.15.52:0/321861107 learned_addr learned my addr 172.21.15.52:0/321861107 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:46.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.278+0000 7f8c50a64700 1 --2- 172.21.15.52:0/321861107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b47f61680 0x564b47f7b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.278+0000 7f8c51a66700 1 --2- 172.21.15.52:0/321861107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b47f65ac0 0x564b47f7be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.278+0000 7f8c51265700 1 -- 172.21.15.52:0/321861107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b47f65ac0 msgr2=0x564b47f7be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:46.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.278+0000 7f8c51265700 1 --2- 172.21.15.52:0/321861107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b47f65ac0 0x564b47f7be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.278+0000 7f8c51265700 1 -- 172.21.15.52:0/321861107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b47f61680 msgr2=0x564b47f7b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:46.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.278+0000 7f8c51265700 1 --2- 172.21.15.52:0/321861107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b47f61680 0x564b47f7b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.278+0000 7f8c51265700 1 -- 172.21.15.52:0/321861107 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b47e76190 con 0x564b47f60d70 2023-11-27T15:23:46.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.278+0000 7f8c51265700 1 --2- 172.21.15.52:0/321861107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b47f60d70 0x564b47f7b3c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800a9d0 tx=0x7f8c480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99516236f6f5c50 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:46.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.278+0000 7f8c34ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:23:46.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.278+0000 7f8c34ff9700 1 -- 172.21.15.52:0/321861107 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c48026040 con 0x564b47f60d70 2023-11-27T15:23:46.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.278+0000 7f8c34ff9700 1 -- 172.21.15.52:0/321861107 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f8c48021420 con 0x564b47f60d70 2023-11-27T15:23:46.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.278+0000 7f8c5f655b80 1 -- 172.21.15.52:0/321861107 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c4400f050 con 0x564b47f60d70 2023-11-27T15:23:46.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.278+0000 7f8c34ff9700 1 -- 172.21.15.52:0/321861107 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c480215e0 con 0x564b47f60d70 2023-11-27T15:23:46.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.278+0000 7f8c5f655b80 10 client.16329 fetch_fsmap learned FSMap version 28 2023-11-27T15:23:46.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.278+0000 7f8c5f655b80 1 -- 172.21.15.52:0/321861107 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564b47cc6c60 con 0x564b47f60d70 2023-11-27T15:23:46.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.279+0000 7f8c34ff9700 1 -- 172.21.15.52:0/321861107 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c4400f050 con 0x564b47f60d70 2023-11-27T15:23:46.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.279+0000 7f8c34ff9700 1 -- 172.21.15.52:0/321861107 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8c48004bf0 con 0x564b47f60d70 2023-11-27T15:23:46.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.279+0000 7f8c5f655b80 10 client.16329 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:23:46.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.279+0000 7f8c5f655b80 10 client.16329 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:23:46.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.279+0000 7f8c5f655b80 20 client.16329 populate_metadata read hostname 'smithi052' 2023-11-27T15:23:46.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.279+0000 7f8c5f655b80 1 --2- 172.21.15.52:0/321861107 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x564b47fccf50 0x564b47fcf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.279+0000 7f8c5f655b80 4 client.16329 mds_command: new command op to 24494 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:23:46.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.279+0000 7f8c5f655b80 1 -- 172.21.15.52:0/321861107 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564b47e7e660 con 0x564b47fccf50 2023-11-27T15:23:46.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.279+0000 7f8c50a64700 1 --2- 172.21.15.52:0/321861107 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x564b47fccf50 0x564b47fcf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.279+0000 7f8c34ff9700 10 client.16329 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:23:46.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.279+0000 7f8c50a64700 1 --2- 172.21.15.52:0/321861107 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x564b47fccf50 0x564b47fcf360 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=91440bafcfab205 server_cookie=8000e327f4812ec2 in_seq=0 out_seq=0 2023-11-27T15:23:46.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.279+0000 7f8c34ff9700 1 -- 172.21.15.52:0/321861107 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564b47e7e660 con 0x564b47fccf50 2023-11-27T15:23:46.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.279+0000 7f8c34ff9700 10 client.16329 handle_command_reply: tid=0 2023-11-27T15:23:46.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.279+0000 7f8c5f655b80 1 client.16329 shutdown 2023-11-27T15:23:46.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.280+0000 7f8c5f655b80 1 -- 172.21.15.52:0/321861107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b47f60d70 msgr2=0x564b47f7b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.280+0000 7f8c5f655b80 1 --2- 172.21.15.52:0/321861107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b47f60d70 0x564b47f7b3c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800a9d0 tx=0x7f8c480047c0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.280+0000 7f8c5f655b80 1 -- 172.21.15.52:0/321861107 shutdown_connections 2023-11-27T15:23:46.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.280+0000 7f8c5f655b80 1 --2- 172.21.15.52:0/321861107 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x564b47fccf50 0x564b47fcf360 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.280+0000 7f8c5f655b80 1 --2- 172.21.15.52:0/321861107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564b47f65ac0 0x564b47f7be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.280+0000 7f8c5f655b80 1 --2- 172.21.15.52:0/321861107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b47f60d70 0x564b47f7b3c0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.280+0000 7f8c34ff9700 0 client.16329 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:23:46.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.280+0000 7f8c5f655b80 1 --2- 172.21.15.52:0/321861107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564b47f61680 0x564b47f7b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.280+0000 7f8c5f655b80 1 -- 172.21.15.52:0/321861107 >> 172.21.15.52:0/321861107 conn(0x564b47f54360 msgr2=0x564b47f5e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:46.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.280+0000 7f8c5f655b80 1 -- 172.21.15.52:0/321861107 shutdown_connections 2023-11-27T15:23:46.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.280+0000 7f8c5f655b80 1 -- 172.21.15.52:0/321861107 wait complete. 2023-11-27T15:23:46.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.280+0000 7f8c5f655b80 20 client.16329 trim_cache size 0 max 16384 2023-11-27T15:23:46.295 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:23:46.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.280+0000 7f8c534c9700 1 -- 172.21.15.52:0/1274996741 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f8c380a1340 msgr2=0x7f8c380a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.280+0000 7f8c534c9700 1 --2- 172.21.15.52:0/1274996741 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f8c380a1340 0x7f8c380a37f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8c48005950 tx=0x7f8c4800a300 comp rx=0 tx=0).stop 2023-11-27T15:23:46.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.280+0000 7f8c534c9700 1 -- 172.21.15.52:0/1274996741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c4c100340 msgr2=0x7f8c4c195760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.280+0000 7f8c534c9700 1 --2- 172.21.15.52:0/1274996741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c4c100340 0x7f8c4c195760 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c009990 tx=0x7f8c3c009580 comp rx=0 tx=0).stop 2023-11-27T15:23:46.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.280+0000 7f8c534c9700 1 -- 172.21.15.52:0/1274996741 shutdown_connections 2023-11-27T15:23:46.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.280+0000 7f8c534c9700 1 --2- 172.21.15.52:0/1274996741 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f8c380a1340 0x7f8c380a37f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.280+0000 7f8c534c9700 1 --2- 172.21.15.52:0/1274996741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c4c0ff990 0x7f8c4c195220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.280+0000 7f8c534c9700 1 --2- 172.21.15.52:0/1274996741 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c4c104180 0x7f8c4c199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.280+0000 7f8c534c9700 1 --2- 172.21.15.52:0/1274996741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c4c100340 0x7f8c4c195760 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.280+0000 7f8c534c9700 1 -- 172.21.15.52:0/1274996741 >> 172.21.15.52:0/1274996741 conn(0x7f8c4c075920 msgr2=0x7f8c4c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:46.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.280+0000 7f8c534c9700 1 -- 172.21.15.52:0/1274996741 shutdown_connections 2023-11-27T15:23:46.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.281+0000 7f8c534c9700 1 -- 172.21.15.52:0/1274996741 wait complete. 2023-11-27T15:23:46.297 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:46 smithi052 ceph-mon[99056]: pgmap v178: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2023-11-27T15:23:46.298 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:46 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1831310006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:46.298 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-27T15:23:46.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.422+0000 7fab0de9b700 1 -- 172.21.15.52:0/1179022064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab08100f50 msgr2=0x7fab08104f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.422+0000 7fab0de9b700 1 --2- 172.21.15.52:0/1179022064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab08100f50 0x7fab08104f10 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7faafc0099c0 tx=0x7faafc009cd0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.423+0000 7fab0de9b700 1 -- 172.21.15.52:0/1179022064 shutdown_connections 2023-11-27T15:23:46.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.423+0000 7fab0de9b700 1 --2- 172.21.15.52:0/1179022064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab081005b0 0x7fab08100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.423+0000 7fab0de9b700 1 --2- 172.21.15.52:0/1179022064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab081055f0 0x7fab0810d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.423+0000 7fab0de9b700 1 --2- 172.21.15.52:0/1179022064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab08100f50 0x7fab08104f10 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.423+0000 7fab0de9b700 1 -- 172.21.15.52:0/1179022064 >> 172.21.15.52:0/1179022064 conn(0x7fab08077170 msgr2=0x7fab08077570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:46.428 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.423+0000 7fab0de9b700 1 -- 172.21.15.52:0/1179022064 shutdown_connections 2023-11-27T15:23:46.428 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.423+0000 7fab0de9b700 1 -- 172.21.15.52:0/1179022064 wait complete. 2023-11-27T15:23:46.428 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.423+0000 7fab0de9b700 1 Processor -- start 2023-11-27T15:23:46.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.423+0000 7fab0de9b700 1 -- start start 2023-11-27T15:23:46.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.424+0000 7fab0de9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab081005b0 0x7fab081964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.424+0000 7fab0de9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab08100f50 0x7fab081969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.424+0000 7fab0de9b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab081055f0 0x7fab0819ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.424+0000 7fab0de9b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0819b490 con 0x7fab081005b0 2023-11-27T15:23:46.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.424+0000 7fab0de9b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0819b600 con 0x7fab081055f0 2023-11-27T15:23:46.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.424+0000 7fab0de9b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0819b7c0 con 0x7fab08100f50 2023-11-27T15:23:46.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.424+0000 7fab077fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab081005b0 0x7fab081964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.424+0000 7fab077fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab081005b0 0x7fab081964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54236/0 (socket says 172.21.15.52:54236) 2023-11-27T15:23:46.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.424+0000 7fab077fe700 1 -- 172.21.15.52:0/546611549 learned_addr learned my addr 172.21.15.52:0/546611549 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:46.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.424+0000 7fab06ffd700 1 --2- 172.21.15.52:0/546611549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab08100f50 0x7fab081969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.424+0000 7fab07fff700 1 --2- 172.21.15.52:0/546611549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab081055f0 0x7fab0819ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.424+0000 7fab077fe700 1 -- 172.21.15.52:0/546611549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab08100f50 msgr2=0x7fab081969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.424+0000 7fab077fe700 1 --2- 172.21.15.52:0/546611549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab08100f50 0x7fab081969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.424+0000 7fab077fe700 1 -- 172.21.15.52:0/546611549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab081055f0 msgr2=0x7fab0819ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.424+0000 7fab077fe700 1 --2- 172.21.15.52:0/546611549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab081055f0 0x7fab0819ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.424+0000 7fab077fe700 1 -- 172.21.15.52:0/546611549 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faafc0095f0 con 0x7fab081005b0 2023-11-27T15:23:46.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.424+0000 7fab06ffd700 1 --2- 172.21.15.52:0/546611549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab08100f50 0x7fab081969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:46.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.424+0000 7fab077fe700 1 --2- 172.21.15.52:0/546611549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab081005b0 0x7fab081964b0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7faaf800e9a0 tx=0x7faaf800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:46.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.424+0000 7fab04ff9700 1 -- 172.21.15.52:0/546611549 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf8009790 con 0x7fab081005b0 2023-11-27T15:23:46.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.424+0000 7fab04ff9700 1 -- 172.21.15.52:0/546611549 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faaf800cbd0 con 0x7fab081005b0 2023-11-27T15:23:46.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.424+0000 7fab0de9b700 1 -- 172.21.15.52:0/546611549 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab081103c0 con 0x7fab081005b0 2023-11-27T15:23:46.433 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.424+0000 7fab04ff9700 1 -- 172.21.15.52:0/546611549 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf8018680 con 0x7fab081005b0 2023-11-27T15:23:46.433 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.424+0000 7fab0de9b700 1 -- 172.21.15.52:0/546611549 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab08110230 con 0x7fab081005b0 2023-11-27T15:23:46.439 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.427+0000 7fab04ff9700 1 -- 172.21.15.52:0/546611549 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7faaf800cd40 con 0x7fab081005b0 2023-11-27T15:23:46.439 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.428+0000 7fab04ff9700 1 --2- 172.21.15.52:0/546611549 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7faaf00a12c0 0x7faaf00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.440 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.428+0000 7fab04ff9700 1 -- 172.21.15.52:0/546611549 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7faaf8014070 con 0x7fab081005b0 2023-11-27T15:23:46.440 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.428+0000 7fab06ffd700 1 --2- 172.21.15.52:0/546611549 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7faaf00a12c0 0x7faaf00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.440 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.428+0000 7fab06ffd700 1 --2- 172.21.15.52:0/546611549 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7faaf00a12c0 0x7faaf00a3770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faafc00b580 tx=0x7faafc01a040 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:46.440 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.429+0000 7fab1a027b80 1 Processor -- start 2023-11-27T15:23:46.440 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.429+0000 7fab1a027b80 1 -- start start 2023-11-27T15:23:46.440 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.429+0000 7fab1a027b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395b774b60 0x56395b774f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.440 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.429+0000 7fab1a027b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56395b775470 0x56395b779370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.429+0000 7fab1a027b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56395b7798b0 0x56395b779d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.429+0000 7fab1a027b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395b67a060 con 0x56395b7798b0 2023-11-27T15:23:46.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.429+0000 7fab1a027b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395b6cc8a0 con 0x56395b775470 2023-11-27T15:23:46.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.429+0000 7fab1a027b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395b6c5d30 con 0x56395b774b60 2023-11-27T15:23:46.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.429+0000 7fab07fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56395b7798b0 0x56395b779d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.429+0000 7fab07fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56395b7798b0 0x56395b779d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54248/0 (socket says 172.21.15.52:54248) 2023-11-27T15:23:46.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.429+0000 7fab07fff700 1 -- 172.21.15.52:0/4009875051 learned_addr learned my addr 172.21.15.52:0/4009875051 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:46.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.429+0000 7fab077fe700 1 --2- 172.21.15.52:0/4009875051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395b774b60 0x56395b774f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.429+0000 7fab06ffd700 1 --2- 172.21.15.52:0/4009875051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56395b775470 0x56395b779370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.429+0000 7fab07fff700 1 -- 172.21.15.52:0/4009875051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395b774b60 msgr2=0x56395b774f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:46.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.429+0000 7fab07fff700 1 --2- 172.21.15.52:0/4009875051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395b774b60 0x56395b774f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.429+0000 7fab07fff700 1 -- 172.21.15.52:0/4009875051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56395b775470 msgr2=0x56395b779370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:46.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.429+0000 7fab07fff700 1 --2- 172.21.15.52:0/4009875051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56395b775470 0x56395b779370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.429+0000 7fab07fff700 1 -- 172.21.15.52:0/4009875051 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56395b547630 con 0x56395b7798b0 2023-11-27T15:23:46.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.429+0000 7fab06ffd700 1 --2- 172.21.15.52:0/4009875051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56395b775470 0x56395b779370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:23:46.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.429+0000 7fab07fff700 1 --2- 172.21.15.52:0/4009875051 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56395b7798b0 0x56395b779d00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fab00009440 tx=0x7fab00009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e275edadd4a7e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:46.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.429+0000 7faaedffb700 1 -- 172.21.15.52:0/4009875051 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0000e070 con 0x56395b7798b0 2023-11-27T15:23:46.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.430+0000 7faaedffb700 1 -- 172.21.15.52:0/4009875051 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab0000f040 con 0x56395b7798b0 2023-11-27T15:23:46.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.430+0000 7faaedffb700 1 -- 172.21.15.52:0/4009875051 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab00015b00 con 0x56395b7798b0 2023-11-27T15:23:46.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.430+0000 7fab1a027b80 1 -- 172.21.15.52:0/4009875051 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56395b7798b0 msgr2=0x56395b779d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.430+0000 7fab1a027b80 1 --2- 172.21.15.52:0/4009875051 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56395b7798b0 0x56395b779d00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fab00009440 tx=0x7fab00009920 comp rx=0 tx=0).stop 2023-11-27T15:23:46.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.430+0000 7fab1a027b80 1 -- 172.21.15.52:0/4009875051 shutdown_connections 2023-11-27T15:23:46.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.430+0000 7fab1a027b80 1 --2- 172.21.15.52:0/4009875051 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56395b7798b0 0x56395b779d00 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.430+0000 7fab1a027b80 1 --2- 172.21.15.52:0/4009875051 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395b774b60 0x56395b774f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.430+0000 7fab1a027b80 1 --2- 172.21.15.52:0/4009875051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56395b775470 0x56395b779370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.430+0000 7fab1a027b80 1 -- 172.21.15.52:0/4009875051 >> 172.21.15.52:0/4009875051 conn(0x56395b76a360 msgr2=0x56395b76fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:46.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.430+0000 7fab1a027b80 1 -- 172.21.15.52:0/4009875051 shutdown_connections 2023-11-27T15:23:46.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.430+0000 7fab1a027b80 1 -- 172.21.15.52:0/4009875051 wait complete. 2023-11-27T15:23:46.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.431+0000 7fab1a027b80 1 -- start start 2023-11-27T15:23:46.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.431+0000 7fab1a027b80 1 Processor -- start 2023-11-27T15:23:46.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.431+0000 7fab1a027b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56395b774b60 0x56395b780fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.431+0000 7fab1a027b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395b775470 0x56395b77e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.431+0000 7fab1a027b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56395b7798b0 0x56395b77e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.431+0000 7fab1a027b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395b6678e0 con 0x56395b774b60 2023-11-27T15:23:46.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.431+0000 7fab1a027b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395b667580 con 0x56395b7798b0 2023-11-27T15:23:46.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.431+0000 7fab1a027b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395b549920 con 0x56395b775470 2023-11-27T15:23:46.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.431+0000 7fab1a027b80 10 client.0 _ll_register_callbacks cb 0x56395b685780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:46.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.431+0000 7fab077fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56395b774b60 0x56395b780fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.431+0000 7fab077fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56395b774b60 0x56395b780fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54250/0 (socket says 172.21.15.52:54250) 2023-11-27T15:23:46.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.431+0000 7fab077fe700 1 -- 172.21.15.52:0/4071724161 learned_addr learned my addr 172.21.15.52:0/4071724161 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:46.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.431+0000 7fab06ffd700 1 --2- 172.21.15.52:0/4071724161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395b775470 0x56395b77e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.431+0000 7fab077fe700 1 -- 172.21.15.52:0/4071724161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395b775470 msgr2=0x56395b77e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:46.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.431+0000 7fab077fe700 1 --2- 172.21.15.52:0/4071724161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395b775470 0x56395b77e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.431+0000 7fab07fff700 1 --2- 172.21.15.52:0/4071724161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56395b7798b0 0x56395b77e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.432+0000 7fab077fe700 1 -- 172.21.15.52:0/4071724161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56395b7798b0 msgr2=0x56395b77e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:46.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.432+0000 7fab077fe700 1 --2- 172.21.15.52:0/4071724161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56395b7798b0 0x56395b77e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.432+0000 7fab077fe700 1 -- 172.21.15.52:0/4071724161 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab000090f0 con 0x56395b774b60 2023-11-27T15:23:46.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.432+0000 7fab07fff700 1 --2- 172.21.15.52:0/4071724161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56395b7798b0 0x56395b77e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:23:46.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.432+0000 7fab077fe700 1 --2- 172.21.15.52:0/4071724161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56395b774b60 0x56395b780fd0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7faaf8096670 tx=0x7faaf809e250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da25be095f2b3b80 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:46.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.432+0000 7faae6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:23:46.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.432+0000 7faae6ffd700 1 -- 172.21.15.52:0/4071724161 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf80a2130 con 0x56395b774b60 2023-11-27T15:23:46.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.432+0000 7faae6ffd700 1 -- 172.21.15.52:0/4071724161 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7faaf80a2290 con 0x56395b774b60 2023-11-27T15:23:46.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.432+0000 7fab1a027b80 1 -- 172.21.15.52:0/4071724161 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56395b767b20 con 0x56395b774b60 2023-11-27T15:23:46.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.432+0000 7faae6ffd700 1 -- 172.21.15.52:0/4071724161 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7faaf8095f60 con 0x56395b774b60 2023-11-27T15:23:46.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.432+0000 7fab1a027b80 10 client.16341 fetch_fsmap learned FSMap version 28 2023-11-27T15:23:46.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.432+0000 7fab1a027b80 1 -- 172.21.15.52:0/4071724161 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56395b68c190 con 0x56395b774b60 2023-11-27T15:23:46.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.432+0000 7faae6ffd700 1 -- 172.21.15.52:0/4071724161 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faaf80b8020 con 0x56395b774b60 2023-11-27T15:23:46.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.432+0000 7faae6ffd700 1 -- 172.21.15.52:0/4071724161 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7faaf80b83e0 con 0x56395b774b60 2023-11-27T15:23:46.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.432+0000 7fab1a027b80 10 client.16341 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:23:46.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.432+0000 7fab1a027b80 10 client.16341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:46.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.432+0000 7fab1a027b80 20 client.16341 populate_metadata read hostname 'smithi052' 2023-11-27T15:23:46.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.432+0000 7fab1a027b80 1 --2- 172.21.15.52:0/4071724161 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x56395b78e820 0x56395b790c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.432+0000 7fab1a027b80 4 client.16341 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:23:46.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.432+0000 7fab1a027b80 1 -- 172.21.15.52:0/4071724161 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56395b6880a0 con 0x56395b78e820 2023-11-27T15:23:46.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.433+0000 7fab06ffd700 1 --2- 172.21.15.52:0/4071724161 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x56395b78e820 0x56395b790c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.433+0000 7faae6ffd700 10 client.16341 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:46.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.433+0000 7fab06ffd700 1 --2- 172.21.15.52:0/4071724161 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x56395b78e820 0x56395b790c30 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a27a20f3762d167 server_cookie=6d32f42f23b05ad7 in_seq=0 out_seq=0 2023-11-27T15:23:46.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.434+0000 7faae6ffd700 1 -- 172.21.15.52:0/4071724161 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56395b6880a0 con 0x56395b78e820 2023-11-27T15:23:46.450 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.434+0000 7faae6ffd700 10 client.16341 handle_command_reply: tid=0 2023-11-27T15:23:46.450 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.434+0000 7fab1a027b80 1 client.16341 shutdown 2023-11-27T15:23:46.450 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.434+0000 7fab1a027b80 1 -- 172.21.15.52:0/4071724161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56395b774b60 msgr2=0x56395b780fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.450 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.434+0000 7fab1a027b80 1 --2- 172.21.15.52:0/4071724161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56395b774b60 0x56395b780fd0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7faaf8096670 tx=0x7faaf809e250 comp rx=0 tx=0).stop 2023-11-27T15:23:46.450 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.434+0000 7fab1a027b80 1 -- 172.21.15.52:0/4071724161 shutdown_connections 2023-11-27T15:23:46.450 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.434+0000 7fab1a027b80 1 --2- 172.21.15.52:0/4071724161 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x56395b78e820 0x56395b790c30 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.450 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.434+0000 7fab1a027b80 1 --2- 172.21.15.52:0/4071724161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56395b7798b0 0x56395b77e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.434+0000 7faae6ffd700 0 client.16341 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:46.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.434+0000 7fab1a027b80 1 --2- 172.21.15.52:0/4071724161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56395b774b60 0x56395b780fd0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.434+0000 7fab1a027b80 1 --2- 172.21.15.52:0/4071724161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395b775470 0x56395b77e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.434+0000 7fab1a027b80 1 -- 172.21.15.52:0/4071724161 >> 172.21.15.52:0/4071724161 conn(0x56395b76a360 msgr2=0x56395b77ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:46.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.434+0000 7fab1a027b80 1 -- 172.21.15.52:0/4071724161 shutdown_connections 2023-11-27T15:23:46.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.434+0000 7fab1a027b80 1 -- 172.21.15.52:0/4071724161 wait complete. 2023-11-27T15:23:46.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.434+0000 7fab1a027b80 20 client.16341 trim_cache size 0 max 16384 2023-11-27T15:23:46.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.447+0000 7fab1a027b80 1 Processor -- start 2023-11-27T15:23:46.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.447+0000 7fab1a027b80 1 -- start start 2023-11-27T15:23:46.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.447+0000 7fab1a027b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395b774b60 0x56395b76fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.447+0000 7fab1a027b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56395b775470 0x56395b7859d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.447+0000 7fab1a027b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56395b7798b0 0x56395b780b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.447+0000 7fab1a027b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395b77f830 con 0x56395b775470 2023-11-27T15:23:46.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.447+0000 7fab1a027b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395b78e600 con 0x56395b7798b0 2023-11-27T15:23:46.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.447+0000 7fab1a027b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395b77c820 con 0x56395b774b60 2023-11-27T15:23:46.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.447+0000 7fab06ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56395b775470 0x56395b7859d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.447+0000 7fab06ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56395b775470 0x56395b7859d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54260/0 (socket says 172.21.15.52:54260) 2023-11-27T15:23:46.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.447+0000 7fab06ffd700 1 -- 172.21.15.52:0/2207773003 learned_addr learned my addr 172.21.15.52:0/2207773003 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:46.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.447+0000 7fab077fe700 1 --2- 172.21.15.52:0/2207773003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395b774b60 0x56395b76fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.447+0000 7fab07fff700 1 --2- 172.21.15.52:0/2207773003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56395b7798b0 0x56395b780b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.448+0000 7fab06ffd700 1 -- 172.21.15.52:0/2207773003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395b774b60 msgr2=0x56395b76fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:46.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.448+0000 7fab06ffd700 1 --2- 172.21.15.52:0/2207773003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395b774b60 0x56395b76fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.448+0000 7fab06ffd700 1 -- 172.21.15.52:0/2207773003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56395b7798b0 msgr2=0x56395b780b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:46.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.448+0000 7fab06ffd700 1 --2- 172.21.15.52:0/2207773003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56395b7798b0 0x56395b780b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.448+0000 7fab06ffd700 1 -- 172.21.15.52:0/2207773003 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56395b76e820 con 0x56395b775470 2023-11-27T15:23:46.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.448+0000 7fab06ffd700 1 --2- 172.21.15.52:0/2207773003 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56395b775470 0x56395b7859d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7faafc0099c0 tx=0x7faafc022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc7a373505acf0a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:46.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.448+0000 7faaee7fc700 1 -- 172.21.15.52:0/2207773003 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc01d040 con 0x56395b775470 2023-11-27T15:23:46.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.448+0000 7faaee7fc700 1 -- 172.21.15.52:0/2207773003 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faafc02b040 con 0x56395b775470 2023-11-27T15:23:46.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.448+0000 7faaee7fc700 1 -- 172.21.15.52:0/2207773003 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc005440 con 0x56395b775470 2023-11-27T15:23:46.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.448+0000 7fab1a027b80 1 -- 172.21.15.52:0/2207773003 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56395b775470 msgr2=0x56395b7859d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.448+0000 7fab1a027b80 1 --2- 172.21.15.52:0/2207773003 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56395b775470 0x56395b7859d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7faafc0099c0 tx=0x7faafc022000 comp rx=0 tx=0).stop 2023-11-27T15:23:46.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.448+0000 7fab1a027b80 1 -- 172.21.15.52:0/2207773003 shutdown_connections 2023-11-27T15:23:46.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.448+0000 7fab1a027b80 1 --2- 172.21.15.52:0/2207773003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56395b7798b0 0x56395b780b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.448+0000 7fab1a027b80 1 --2- 172.21.15.52:0/2207773003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395b774b60 0x56395b76fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.449+0000 7fab1a027b80 1 --2- 172.21.15.52:0/2207773003 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56395b775470 0x56395b7859d0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.449+0000 7fab1a027b80 1 -- 172.21.15.52:0/2207773003 >> 172.21.15.52:0/2207773003 conn(0x56395b76a360 msgr2=0x56395b77bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:46.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.449+0000 7fab1a027b80 1 -- 172.21.15.52:0/2207773003 shutdown_connections 2023-11-27T15:23:46.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.449+0000 7fab1a027b80 1 -- 172.21.15.52:0/2207773003 wait complete. 2023-11-27T15:23:46.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.449+0000 7fab1a027b80 1 -- start start 2023-11-27T15:23:46.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.449+0000 7fab1a027b80 1 Processor -- start 2023-11-27T15:23:46.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.450+0000 7fab1a027b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56395b774b60 0x56395b783bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.450+0000 7fab1a027b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56395b775470 0x56395b7840f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.450+0000 7fab1a027b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395b7798b0 0x56395b7e5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.450+0000 7fab1a027b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395b784890 con 0x56395b775470 2023-11-27T15:23:46.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.450+0000 7fab1a027b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395b7e6030 con 0x56395b774b60 2023-11-27T15:23:46.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.450+0000 7fab1a027b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56395b7e61c0 con 0x56395b7798b0 2023-11-27T15:23:46.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.450+0000 7fab1a027b80 10 client.0 _ll_register_callbacks cb 0x7faaf8099b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:46.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.450+0000 7fab06ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56395b775470 0x56395b7840f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.450+0000 7fab06ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56395b775470 0x56395b7840f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54266/0 (socket says 172.21.15.52:54266) 2023-11-27T15:23:46.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.450+0000 7fab06ffd700 1 -- 172.21.15.52:0/1999803834 learned_addr learned my addr 172.21.15.52:0/1999803834 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:46.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.450+0000 7fab07fff700 1 --2- 172.21.15.52:0/1999803834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395b7798b0 0x56395b7e5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.450+0000 7fab077fe700 1 --2- 172.21.15.52:0/1999803834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56395b774b60 0x56395b783bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.450+0000 7fab06ffd700 1 -- 172.21.15.52:0/1999803834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395b7798b0 msgr2=0x56395b7e5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:46.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.450+0000 7fab06ffd700 1 --2- 172.21.15.52:0/1999803834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395b7798b0 0x56395b7e5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.450+0000 7fab06ffd700 1 -- 172.21.15.52:0/1999803834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56395b774b60 msgr2=0x56395b783bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:46.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.450+0000 7fab06ffd700 1 --2- 172.21.15.52:0/1999803834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56395b774b60 0x56395b783bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.450+0000 7fab06ffd700 1 -- 172.21.15.52:0/1999803834 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab0810da00 con 0x56395b775470 2023-11-27T15:23:46.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.450+0000 7fab06ffd700 1 --2- 172.21.15.52:0/1999803834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56395b775470 0x56395b7840f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7faafc00c010 tx=0x7faafc0303b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af19cd77519c4bd8 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:46.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.450+0000 7faae6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:23:46.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.450+0000 7faae6ffd700 1 -- 172.21.15.52:0/1999803834 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc01d040 con 0x56395b775470 2023-11-27T15:23:46.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.450+0000 7faae6ffd700 1 -- 172.21.15.52:0/1999803834 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7faafc030d70 con 0x56395b775470 2023-11-27T15:23:46.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.450+0000 7fab1a027b80 1 -- 172.21.15.52:0/1999803834 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab0000f040 con 0x56395b775470 2023-11-27T15:23:46.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.451+0000 7faae6ffd700 1 -- 172.21.15.52:0/1999803834 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7faafc00bbe0 con 0x56395b775470 2023-11-27T15:23:46.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.451+0000 7fab1a027b80 10 client.16347 fetch_fsmap learned FSMap version 28 2023-11-27T15:23:46.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.451+0000 7fab1a027b80 1 -- 172.21.15.52:0/1999803834 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56395b4f3de0 con 0x56395b775470 2023-11-27T15:23:46.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.451+0000 7faae6ffd700 1 -- 172.21.15.52:0/1999803834 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faafc046020 con 0x56395b775470 2023-11-27T15:23:46.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.451+0000 7faae6ffd700 1 -- 172.21.15.52:0/1999803834 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7faafc024830 con 0x56395b775470 2023-11-27T15:23:46.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.451+0000 7fab1a027b80 10 client.16347 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:23:46.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.451+0000 7fab1a027b80 10 client.16347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:46.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.451+0000 7fab1a027b80 20 client.16347 populate_metadata read hostname 'smithi052' 2023-11-27T15:23:46.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.451+0000 7fab1a027b80 1 --2- 172.21.15.52:0/1999803834 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x56395b7e7520 0x56395b7e9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.451+0000 7fab1a027b80 4 client.16347 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:23:46.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.451+0000 7fab1a027b80 1 -- 172.21.15.52:0/1999803834 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56395b694660 con 0x56395b7e7520 2023-11-27T15:23:46.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.451+0000 7fab077fe700 1 --2- 172.21.15.52:0/1999803834 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x56395b7e7520 0x56395b7e9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.452+0000 7faae6ffd700 10 client.16347 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:46.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.452+0000 7fab077fe700 1 --2- 172.21.15.52:0/1999803834 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x56395b7e7520 0x56395b7e9930 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb6fc482ffe13808 server_cookie=be27a964432834cb in_seq=0 out_seq=0 2023-11-27T15:23:46.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.452+0000 7faae6ffd700 1 -- 172.21.15.52:0/1999803834 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56395b694660 con 0x56395b7e7520 2023-11-27T15:23:46.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.452+0000 7faae6ffd700 10 client.16347 handle_command_reply: tid=0 2023-11-27T15:23:46.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.452+0000 7fab1a027b80 1 client.16347 shutdown 2023-11-27T15:23:46.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.452+0000 7fab1a027b80 1 -- 172.21.15.52:0/1999803834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56395b775470 msgr2=0x56395b7840f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.452+0000 7fab1a027b80 1 --2- 172.21.15.52:0/1999803834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56395b775470 0x56395b7840f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7faafc00c010 tx=0x7faafc0303b0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.452+0000 7fab1a027b80 1 -- 172.21.15.52:0/1999803834 shutdown_connections 2023-11-27T15:23:46.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.452+0000 7fab1a027b80 1 --2- 172.21.15.52:0/1999803834 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x56395b7e7520 0x56395b7e9930 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.452+0000 7fab1a027b80 1 --2- 172.21.15.52:0/1999803834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56395b774b60 0x56395b783bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.452+0000 7faae6ffd700 0 client.16347 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:46.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.452+0000 7fab1a027b80 1 --2- 172.21.15.52:0/1999803834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56395b7798b0 0x56395b7e5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.452+0000 7fab1a027b80 1 --2- 172.21.15.52:0/1999803834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56395b775470 0x56395b7840f0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.452+0000 7fab1a027b80 1 -- 172.21.15.52:0/1999803834 >> 172.21.15.52:0/1999803834 conn(0x56395b76a360 msgr2=0x56395b7778b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:46.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.452+0000 7fab1a027b80 1 -- 172.21.15.52:0/1999803834 shutdown_connections 2023-11-27T15:23:46.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.452+0000 7fab1a027b80 1 -- 172.21.15.52:0/1999803834 wait complete. 2023-11-27T15:23:46.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.452+0000 7fab1a027b80 20 client.16347 trim_cache size 0 max 16384 2023-11-27T15:23:46.467 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:23:46.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.453+0000 7fab0de9b700 1 -- 172.21.15.52:0/546611549 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7faaf00a12c0 msgr2=0x7faaf00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.453+0000 7fab0de9b700 1 --2- 172.21.15.52:0/546611549 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7faaf00a12c0 0x7faaf00a3770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faafc00b580 tx=0x7faafc01a040 comp rx=0 tx=0).stop 2023-11-27T15:23:46.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.453+0000 7fab0de9b700 1 -- 172.21.15.52:0/546611549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab081005b0 msgr2=0x7fab081964b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.453+0000 7fab0de9b700 1 --2- 172.21.15.52:0/546611549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab081005b0 0x7fab081964b0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7faaf800e9a0 tx=0x7faaf800ed60 comp rx=0 tx=0).stop 2023-11-27T15:23:46.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.453+0000 7fab0de9b700 1 -- 172.21.15.52:0/546611549 shutdown_connections 2023-11-27T15:23:46.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.453+0000 7fab0de9b700 1 --2- 172.21.15.52:0/546611549 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7faaf00a12c0 0x7faaf00a3770 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.453+0000 7fab0de9b700 1 --2- 172.21.15.52:0/546611549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab081055f0 0x7fab0819ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.453+0000 7fab0de9b700 1 --2- 172.21.15.52:0/546611549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab081005b0 0x7fab081964b0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.453+0000 7fab0de9b700 1 --2- 172.21.15.52:0/546611549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab08100f50 0x7fab081969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.453+0000 7fab0de9b700 1 -- 172.21.15.52:0/546611549 >> 172.21.15.52:0/546611549 conn(0x7fab08077170 msgr2=0x7fab080fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:46.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.453+0000 7fab0de9b700 1 -- 172.21.15.52:0/546611549 shutdown_connections 2023-11-27T15:23:46.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:46.453+0000 7fab0de9b700 1 -- 172.21.15.52:0/546611549 wait complete. 2023-11-27T15:23:46.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:46 smithi150 ceph-mon[100559]: pgmap v178: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2023-11-27T15:23:46.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:46 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1831310006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:46.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:46 smithi102 ceph-mon[100296]: pgmap v178: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2023-11-27T15:23:46.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:46 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1831310006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:47.471 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-27T15:23:47.471 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-27T15:23:47.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.611+0000 7f926195d700 1 -- 172.21.15.52:0/1363306989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f925c104500 msgr2=0x7f925c10c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:47.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.611+0000 7f926195d700 1 --2- 172.21.15.52:0/1363306989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f925c104500 0x7f925c10c0d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f925400b0d0 tx=0x7f925400b3e0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.612+0000 7f926195d700 1 -- 172.21.15.52:0/1363306989 shutdown_connections 2023-11-27T15:23:47.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.612+0000 7f926195d700 1 --2- 172.21.15.52:0/1363306989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f925c0ff4c0 0x7f925c0ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.612+0000 7f926195d700 1 --2- 172.21.15.52:0/1363306989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f925c104500 0x7f925c10c0d0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.612+0000 7f926195d700 1 --2- 172.21.15.52:0/1363306989 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f925c0ffe60 0x7f925c103e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.612+0000 7f926195d700 1 -- 172.21.15.52:0/1363306989 >> 172.21.15.52:0/1363306989 conn(0x7f925c076200 msgr2=0x7f925c076600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:47.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.612+0000 7f926195d700 1 -- 172.21.15.52:0/1363306989 shutdown_connections 2023-11-27T15:23:47.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.612+0000 7f926195d700 1 -- 172.21.15.52:0/1363306989 wait complete. 2023-11-27T15:23:47.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.612+0000 7f926195d700 1 Processor -- start 2023-11-27T15:23:47.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.612+0000 7f926195d700 1 -- start start 2023-11-27T15:23:47.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.613+0000 7f926195d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f925c0ff4c0 0x7f925c196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.613+0000 7f926195d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f925c0ffe60 0x7f925c196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.613+0000 7f926195d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f925c104500 0x7f925c19aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.613+0000 7f926195d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925c19b4e0 con 0x7f925c0ffe60 2023-11-27T15:23:47.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.613+0000 7f926195d700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925c19b650 con 0x7f925c104500 2023-11-27T15:23:47.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.613+0000 7f926195d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925c19b810 con 0x7f925c0ff4c0 2023-11-27T15:23:47.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.613+0000 7f925a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f925c0ffe60 0x7f925c196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.613+0000 7f925a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f925c0ffe60 0x7f925c196ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54280/0 (socket says 172.21.15.52:54280) 2023-11-27T15:23:47.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.613+0000 7f925a7fc700 1 -- 172.21.15.52:0/3036097266 learned_addr learned my addr 172.21.15.52:0/3036097266 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:47.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.613+0000 7f925b7fe700 1 --2- 172.21.15.52:0/3036097266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f925c104500 0x7f925c19aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.613+0000 7f925affd700 1 --2- 172.21.15.52:0/3036097266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f925c0ff4c0 0x7f925c196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.613+0000 7f925a7fc700 1 -- 172.21.15.52:0/3036097266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f925c0ff4c0 msgr2=0x7f925c196570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:47.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.613+0000 7f925a7fc700 1 --2- 172.21.15.52:0/3036097266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f925c0ff4c0 0x7f925c196570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.613+0000 7f925a7fc700 1 -- 172.21.15.52:0/3036097266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f925c104500 msgr2=0x7f925c19aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:47.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.613+0000 7f925a7fc700 1 --2- 172.21.15.52:0/3036097266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f925c104500 0x7f925c19aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.613+0000 7f925a7fc700 1 -- 172.21.15.52:0/3036097266 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9254009d00 con 0x7f925c0ffe60 2023-11-27T15:23:47.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.613+0000 7f925a7fc700 1 --2- 172.21.15.52:0/3036097266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f925c0ffe60 0x7f925c196ab0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f925000e940 tx=0x7f925000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:47.622 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.614+0000 7f926095b700 1 -- 172.21.15.52:0/3036097266 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9250009730 con 0x7f925c0ffe60 2023-11-27T15:23:47.622 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.614+0000 7f926095b700 1 -- 172.21.15.52:0/3036097266 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9250010430 con 0x7f925c0ffe60 2023-11-27T15:23:47.622 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.614+0000 7f926195d700 1 -- 172.21.15.52:0/3036097266 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f925c10f330 con 0x7f925c0ffe60 2023-11-27T15:23:47.622 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.614+0000 7f926095b700 1 -- 172.21.15.52:0/3036097266 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925000cea0 con 0x7f925c0ffe60 2023-11-27T15:23:47.622 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.614+0000 7f926195d700 1 -- 172.21.15.52:0/3036097266 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f925c10f1a0 con 0x7f925c0ffe60 2023-11-27T15:23:47.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.616+0000 7f926095b700 1 -- 172.21.15.52:0/3036097266 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9250010810 con 0x7f925c0ffe60 2023-11-27T15:23:47.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.616+0000 7f926095b700 1 --2- 172.21.15.52:0/3036097266 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f92440a1340 0x7f92440a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.617+0000 7f926095b700 1 -- 172.21.15.52:0/3036097266 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f9250014070 con 0x7f925c0ffe60 2023-11-27T15:23:47.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.617+0000 7f925affd700 1 --2- 172.21.15.52:0/3036097266 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f92440a1340 0x7f92440a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.617+0000 7f925affd700 1 --2- 172.21.15.52:0/3036097266 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f92440a1340 0x7f92440a37f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f924c005950 tx=0x7f924c00b410 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:47.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.617+0000 7f926dae9b80 1 Processor -- start 2023-11-27T15:23:47.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.617+0000 7f926dae9b80 1 -- start start 2023-11-27T15:23:47.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.617+0000 7f926dae9b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c83d3bbe0 0x557c83d3bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.618+0000 7f926dae9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c83d3c4f0 0x557c83d403f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.618+0000 7f926dae9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c83d40930 0x557c83d40d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.618+0000 7f926dae9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c83c410e0 con 0x557c83d40930 2023-11-27T15:23:47.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.618+0000 7f926dae9b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c83c93920 con 0x557c83d3bbe0 2023-11-27T15:23:47.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.618+0000 7f926dae9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c83c8cdb0 con 0x557c83d3c4f0 2023-11-27T15:23:47.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.618+0000 7f925b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c83d40930 0x557c83d40d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.618+0000 7f925b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c83d40930 0x557c83d40d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54290/0 (socket says 172.21.15.52:54290) 2023-11-27T15:23:47.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.618+0000 7f925b7fe700 1 -- 172.21.15.52:0/475040371 learned_addr learned my addr 172.21.15.52:0/475040371 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:47.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.618+0000 7f925affd700 1 --2- 172.21.15.52:0/475040371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c83d3bbe0 0x557c83d3bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.618+0000 7f925a7fc700 1 --2- 172.21.15.52:0/475040371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c83d3c4f0 0x557c83d403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.618+0000 7f925b7fe700 1 -- 172.21.15.52:0/475040371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c83d3c4f0 msgr2=0x557c83d403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:47.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.618+0000 7f925b7fe700 1 --2- 172.21.15.52:0/475040371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c83d3c4f0 0x557c83d403f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.618+0000 7f925b7fe700 1 -- 172.21.15.52:0/475040371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c83d3bbe0 msgr2=0x557c83d3bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:47.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.618+0000 7f925b7fe700 1 --2- 172.21.15.52:0/475040371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c83d3bbe0 0x557c83d3bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.618+0000 7f925b7fe700 1 -- 172.21.15.52:0/475040371 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c83b0e6b0 con 0x557c83d40930 2023-11-27T15:23:47.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.618+0000 7f925affd700 1 --2- 172.21.15.52:0/475040371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c83d3bbe0 0x557c83d3bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:47.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.618+0000 7f925b7fe700 1 --2- 172.21.15.52:0/475040371 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c83d40930 0x557c83d40d80 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9254009b80 tx=0x7f9254009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a35945b9e85e12 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:47.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.618+0000 7f9241ffb700 1 -- 172.21.15.52:0/475040371 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925400e040 con 0x557c83d40930 2023-11-27T15:23:47.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.618+0000 7f9241ffb700 1 -- 172.21.15.52:0/475040371 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92540074e0 con 0x557c83d40930 2023-11-27T15:23:47.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.618+0000 7f9241ffb700 1 -- 172.21.15.52:0/475040371 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9254005070 con 0x557c83d40930 2023-11-27T15:23:47.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.619+0000 7f926dae9b80 1 -- 172.21.15.52:0/475040371 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c83d40930 msgr2=0x557c83d40d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:47.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.619+0000 7f926dae9b80 1 --2- 172.21.15.52:0/475040371 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c83d40930 0x557c83d40d80 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9254009b80 tx=0x7f9254009bb0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.619+0000 7f926dae9b80 1 -- 172.21.15.52:0/475040371 shutdown_connections 2023-11-27T15:23:47.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.619+0000 7f926dae9b80 1 --2- 172.21.15.52:0/475040371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c83d3bbe0 0x557c83d3bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.619+0000 7f926dae9b80 1 --2- 172.21.15.52:0/475040371 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c83d40930 0x557c83d40d80 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.619+0000 7f926dae9b80 1 --2- 172.21.15.52:0/475040371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c83d3c4f0 0x557c83d403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.619+0000 7f926dae9b80 1 -- 172.21.15.52:0/475040371 >> 172.21.15.52:0/475040371 conn(0x557c83d313e0 msgr2=0x557c83d36bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.619+0000 7f926dae9b80 1 -- 172.21.15.52:0/475040371 shutdown_connections 2023-11-27T15:23:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.619+0000 7f926dae9b80 1 -- 172.21.15.52:0/475040371 wait complete. 2023-11-27T15:23:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.620+0000 7f926dae9b80 1 -- start start 2023-11-27T15:23:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.620+0000 7f926dae9b80 1 Processor -- start 2023-11-27T15:23:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.620+0000 7f926dae9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c83d3bbe0 0x557c83d480b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.620+0000 7f926dae9b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c83d3c4f0 0x557c83d452c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.620+0000 7f926dae9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c83d40930 0x557c83d458b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.620+0000 7f926dae9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c83c2e960 con 0x557c83d40930 2023-11-27T15:23:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.620+0000 7f926dae9b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c83c2e600 con 0x557c83d3c4f0 2023-11-27T15:23:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.620+0000 7f926dae9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c83b109a0 con 0x557c83d3bbe0 2023-11-27T15:23:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.620+0000 7f926dae9b80 10 client.0 _ll_register_callbacks cb 0x557c83c4c800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.620+0000 7f925b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c83d40930 0x557c83d458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.620+0000 7f925b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c83d40930 0x557c83d458b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54296/0 (socket says 172.21.15.52:54296) 2023-11-27T15:23:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.620+0000 7f925b7fe700 1 -- 172.21.15.52:0/2209867832 learned_addr learned my addr 172.21.15.52:0/2209867832 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.620+0000 7f925affd700 1 --2- 172.21.15.52:0/2209867832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c83d3bbe0 0x557c83d480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.620+0000 7f925a7fc700 1 --2- 172.21.15.52:0/2209867832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c83d3c4f0 0x557c83d452c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.621+0000 7f925b7fe700 1 -- 172.21.15.52:0/2209867832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c83d3bbe0 msgr2=0x557c83d480b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.621+0000 7f925b7fe700 1 --2- 172.21.15.52:0/2209867832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c83d3bbe0 0x557c83d480b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.621+0000 7f925b7fe700 1 -- 172.21.15.52:0/2209867832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c83d3c4f0 msgr2=0x557c83d452c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.621+0000 7f925b7fe700 1 --2- 172.21.15.52:0/2209867832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c83d3c4f0 0x557c83d452c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.621+0000 7f925b7fe700 1 -- 172.21.15.52:0/2209867832 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f925c10c910 con 0x557c83d40930 2023-11-27T15:23:47.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.621+0000 7f925b7fe700 1 --2- 172.21.15.52:0/2209867832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c83d40930 0x557c83d458b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f925400b0d0 tx=0x7f9254007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48725cc2f687ea47 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:47.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.621+0000 7f923affd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:23:47.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.621+0000 7f923affd700 1 -- 172.21.15.52:0/2209867832 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925400e040 con 0x557c83d40930 2023-11-27T15:23:47.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.621+0000 7f926dae9b80 1 -- 172.21.15.52:0/2209867832 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c83d2eba0 con 0x557c83d40930 2023-11-27T15:23:47.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.621+0000 7f923affd700 1 -- 172.21.15.52:0/2209867832 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f925400bca0 con 0x557c83d40930 2023-11-27T15:23:47.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.621+0000 7f923affd700 1 -- 172.21.15.52:0/2209867832 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f925400be60 con 0x557c83d40930 2023-11-27T15:23:47.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.621+0000 7f926dae9b80 10 client.16359 fetch_fsmap learned FSMap version 28 2023-11-27T15:23:47.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.621+0000 7f926dae9b80 1 -- 172.21.15.52:0/2209867832 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557c83c53210 con 0x557c83d40930 2023-11-27T15:23:47.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.621+0000 7f923affd700 1 -- 172.21.15.52:0/2209867832 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557c83d2eba0 con 0x557c83d40930 2023-11-27T15:23:47.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.622+0000 7f923affd700 1 -- 172.21.15.52:0/2209867832 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f925400b100 con 0x557c83d40930 2023-11-27T15:23:47.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.622+0000 7f926dae9b80 10 client.16359 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:23:47.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.622+0000 7f926dae9b80 10 client.16359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:47.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.622+0000 7f926dae9b80 20 client.16359 populate_metadata read hostname 'smithi052' 2023-11-27T15:23:47.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.622+0000 7f926dae9b80 1 --2- 172.21.15.52:0/2209867832 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557c83d553e0 0x557c83d577f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.622+0000 7f926dae9b80 4 client.16359 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:23:47.638 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.622+0000 7f926dae9b80 1 -- 172.21.15.52:0/2209867832 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c83c4f120 con 0x557c83d553e0 2023-11-27T15:23:47.638 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.622+0000 7f925affd700 1 --2- 172.21.15.52:0/2209867832 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557c83d553e0 0x557c83d577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.638 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.622+0000 7f923affd700 10 client.16359 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:47.638 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.622+0000 7f925affd700 1 --2- 172.21.15.52:0/2209867832 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557c83d553e0 0x557c83d577f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d80588b7c6931b43 server_cookie=7e8d285a5075c3b7 in_seq=0 out_seq=0 2023-11-27T15:23:47.638 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.623+0000 7f923affd700 1 -- 172.21.15.52:0/2209867832 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557c83c4f120 con 0x557c83d553e0 2023-11-27T15:23:47.638 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.623+0000 7f923affd700 10 client.16359 handle_command_reply: tid=0 2023-11-27T15:23:47.638 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.623+0000 7f926dae9b80 1 client.16359 shutdown 2023-11-27T15:23:47.638 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.623+0000 7f926dae9b80 1 -- 172.21.15.52:0/2209867832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c83d40930 msgr2=0x557c83d458b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:47.639 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.623+0000 7f926dae9b80 1 --2- 172.21.15.52:0/2209867832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c83d40930 0x557c83d458b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f925400b0d0 tx=0x7f9254007650 comp rx=0 tx=0).stop 2023-11-27T15:23:47.639 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.623+0000 7f926dae9b80 1 -- 172.21.15.52:0/2209867832 shutdown_connections 2023-11-27T15:23:47.639 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.623+0000 7f926dae9b80 1 --2- 172.21.15.52:0/2209867832 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557c83d553e0 0x557c83d577f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.639 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.623+0000 7f926dae9b80 1 --2- 172.21.15.52:0/2209867832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c83d40930 0x557c83d458b0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.639 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.623+0000 7f923affd700 0 client.16359 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:47.639 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.623+0000 7f926dae9b80 1 --2- 172.21.15.52:0/2209867832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c83d3bbe0 0x557c83d480b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.639 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.623+0000 7f926dae9b80 1 --2- 172.21.15.52:0/2209867832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c83d3c4f0 0x557c83d452c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.640 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.623+0000 7f926dae9b80 1 -- 172.21.15.52:0/2209867832 >> 172.21.15.52:0/2209867832 conn(0x557c83d313e0 msgr2=0x557c83d41de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:47.640 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.623+0000 7f926dae9b80 1 -- 172.21.15.52:0/2209867832 shutdown_connections 2023-11-27T15:23:47.640 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.623+0000 7f926dae9b80 1 -- 172.21.15.52:0/2209867832 wait complete. 2023-11-27T15:23:47.640 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.623+0000 7f926dae9b80 20 client.16359 trim_cache size 0 max 16384 2023-11-27T15:23:47.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.640+0000 7f926dae9b80 1 Processor -- start 2023-11-27T15:23:47.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.640+0000 7f926dae9b80 1 -- start start 2023-11-27T15:23:47.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.641+0000 7f926dae9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c83d3bbe0 0x557c83d36d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.641+0000 7f926dae9b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c83d3c4f0 0x557c83d4c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.641+0000 7f926dae9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c83d40930 0x557c83d47be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.641+0000 7f926dae9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c83d38280 con 0x557c83d40930 2023-11-27T15:23:47.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.641+0000 7f926dae9b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c83d3cc70 con 0x557c83d3c4f0 2023-11-27T15:23:47.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.641+0000 7f926dae9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c83d45f10 con 0x557c83d3bbe0 2023-11-27T15:23:47.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.641+0000 7f925b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c83d40930 0x557c83d47be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.641+0000 7f925b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c83d40930 0x557c83d47be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54304/0 (socket says 172.21.15.52:54304) 2023-11-27T15:23:47.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.641+0000 7f925b7fe700 1 -- 172.21.15.52:0/1970450043 learned_addr learned my addr 172.21.15.52:0/1970450043 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:47.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.641+0000 7f925a7fc700 1 --2- 172.21.15.52:0/1970450043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c83d3c4f0 0x557c83d4c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.641+0000 7f925affd700 1 --2- 172.21.15.52:0/1970450043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c83d3bbe0 0x557c83d36d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.641+0000 7f925b7fe700 1 -- 172.21.15.52:0/1970450043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c83d3bbe0 msgr2=0x557c83d36d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:47.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.641+0000 7f925b7fe700 1 --2- 172.21.15.52:0/1970450043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c83d3bbe0 0x557c83d36d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.641+0000 7f925b7fe700 1 -- 172.21.15.52:0/1970450043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c83d3c4f0 msgr2=0x557c83d4c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:47.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.641+0000 7f925b7fe700 1 --2- 172.21.15.52:0/1970450043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c83d3c4f0 0x557c83d4c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.641+0000 7f925b7fe700 1 -- 172.21.15.52:0/1970450043 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c83d358a0 con 0x557c83d40930 2023-11-27T15:23:47.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.641+0000 7f925b7fe700 1 --2- 172.21.15.52:0/1970450043 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c83d40930 0x557c83d47be0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f925400b060 tx=0x7f9254000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d32162ef1d9638 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:47.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.641+0000 7f92427fc700 1 -- 172.21.15.52:0/1970450043 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925400e040 con 0x557c83d40930 2023-11-27T15:23:47.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.642+0000 7f92427fc700 1 -- 172.21.15.52:0/1970450043 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92540076c0 con 0x557c83d40930 2023-11-27T15:23:47.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.642+0000 7f92427fc700 1 -- 172.21.15.52:0/1970450043 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925402f3a0 con 0x557c83d40930 2023-11-27T15:23:47.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.642+0000 7f926dae9b80 1 -- 172.21.15.52:0/1970450043 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c83d40930 msgr2=0x557c83d47be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:47.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.642+0000 7f926dae9b80 1 --2- 172.21.15.52:0/1970450043 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c83d40930 0x557c83d47be0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f925400b060 tx=0x7f9254000f40 comp rx=0 tx=0).stop 2023-11-27T15:23:47.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.642+0000 7f926dae9b80 1 -- 172.21.15.52:0/1970450043 shutdown_connections 2023-11-27T15:23:47.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.642+0000 7f926dae9b80 1 --2- 172.21.15.52:0/1970450043 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c83d40930 0x557c83d47be0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.642+0000 7f926dae9b80 1 --2- 172.21.15.52:0/1970450043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c83d3bbe0 0x557c83d36d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.642+0000 7f926dae9b80 1 --2- 172.21.15.52:0/1970450043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c83d3c4f0 0x557c83d4c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.655 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.642+0000 7f926dae9b80 1 -- 172.21.15.52:0/1970450043 >> 172.21.15.52:0/1970450043 conn(0x557c83d313e0 msgr2=0x557c83d3b270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:47.655 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.642+0000 7f926dae9b80 1 -- 172.21.15.52:0/1970450043 shutdown_connections 2023-11-27T15:23:47.655 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.642+0000 7f926dae9b80 1 -- 172.21.15.52:0/1970450043 wait complete. 2023-11-27T15:23:47.655 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.642+0000 7f926dae9b80 1 -- start start 2023-11-27T15:23:47.655 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.643+0000 7f926dae9b80 1 Processor -- start 2023-11-27T15:23:47.656 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.643+0000 7f926dae9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c83d3bbe0 0x557c83d48fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.656 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.643+0000 7f926dae9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c83d3c4f0 0x557c83d49500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.656 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.643+0000 7f926dae9b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c83d40930 0x557c83d49a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.656 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.643+0000 7f926dae9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c83dad6b0 con 0x557c83d3c4f0 2023-11-27T15:23:47.656 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.643+0000 7f926dae9b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c83dad7f0 con 0x557c83d40930 2023-11-27T15:23:47.657 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.643+0000 7f926dae9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c83dad930 con 0x557c83d3bbe0 2023-11-27T15:23:47.657 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.643+0000 7f926dae9b80 10 client.0 _ll_register_callbacks cb 0x7f9250010f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:47.657 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.643+0000 7f925a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c83d3c4f0 0x557c83d49500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.657 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.643+0000 7f925a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c83d3c4f0 0x557c83d49500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54308/0 (socket says 172.21.15.52:54308) 2023-11-27T15:23:47.658 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.643+0000 7f925a7fc700 1 -- 172.21.15.52:0/3982780635 learned_addr learned my addr 172.21.15.52:0/3982780635 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:47.658 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.643+0000 7f925affd700 1 --2- 172.21.15.52:0/3982780635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c83d3bbe0 0x557c83d48fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.658 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.643+0000 7f925b7fe700 1 --2- 172.21.15.52:0/3982780635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c83d40930 0x557c83d49a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.658 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.643+0000 7f925a7fc700 1 -- 172.21.15.52:0/3982780635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c83d3bbe0 msgr2=0x557c83d48fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:47.659 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.643+0000 7f925a7fc700 1 --2- 172.21.15.52:0/3982780635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c83d3bbe0 0x557c83d48fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.659 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.643+0000 7f925a7fc700 1 -- 172.21.15.52:0/3982780635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c83d40930 msgr2=0x557c83d49a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:47.659 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.643+0000 7f925a7fc700 1 --2- 172.21.15.52:0/3982780635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c83d40930 0x557c83d49a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.659 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.643+0000 7f925a7fc700 1 -- 172.21.15.52:0/3982780635 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c83c53210 con 0x557c83d3c4f0 2023-11-27T15:23:47.659 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.644+0000 7f925a7fc700 1 --2- 172.21.15.52:0/3982780635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c83d3c4f0 0x557c83d49500 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f92500968b0 tx=0x7f9250099f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8238cb74b4ee45 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:47.660 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.644+0000 7f923affd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:23:47.660 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.644+0000 7f923affd700 1 -- 172.21.15.52:0/3982780635 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9250099530 con 0x557c83d3c4f0 2023-11-27T15:23:47.660 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.644+0000 7f923affd700 1 -- 172.21.15.52:0/3982780635 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f92500a2290 con 0x557c83d3c4f0 2023-11-27T15:23:47.660 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.644+0000 7f926dae9b80 1 -- 172.21.15.52:0/3982780635 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92540074e0 con 0x557c83d3c4f0 2023-11-27T15:23:47.660 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.644+0000 7f923affd700 1 -- 172.21.15.52:0/3982780635 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92500a1e00 con 0x557c83d3c4f0 2023-11-27T15:23:47.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.644+0000 7f926dae9b80 10 client.16365 fetch_fsmap learned FSMap version 28 2023-11-27T15:23:47.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.644+0000 7f926dae9b80 1 -- 172.21.15.52:0/3982780635 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557c83abae60 con 0x557c83d3c4f0 2023-11-27T15:23:47.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.644+0000 7f923affd700 1 -- 172.21.15.52:0/3982780635 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92540074e0 con 0x557c83d3c4f0 2023-11-27T15:23:47.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.644+0000 7f923affd700 1 -- 172.21.15.52:0/3982780635 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f92500b8020 con 0x557c83d3c4f0 2023-11-27T15:23:47.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.644+0000 7f926dae9b80 10 client.16365 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:23:47.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.644+0000 7f926dae9b80 10 client.16365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:47.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.644+0000 7f926dae9b80 20 client.16365 populate_metadata read hostname 'smithi052' 2023-11-27T15:23:47.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.644+0000 7f926dae9b80 1 --2- 172.21.15.52:0/3982780635 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557c83daf130 0x557c83db1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.644+0000 7f926dae9b80 4 client.16365 mds_command: new command op to 24536 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-27T15:23:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.644+0000 7f926dae9b80 1 -- 172.21.15.52:0/3982780635 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557c83c5b6e0 con 0x557c83daf130 2023-11-27T15:23:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.644+0000 7f925affd700 1 --2- 172.21.15.52:0/3982780635 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557c83daf130 0x557c83db1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.645+0000 7f925affd700 1 --2- 172.21.15.52:0/3982780635 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557c83daf130 0x557c83db1540 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdcb084ce1f330d7 server_cookie=1142c119a61c313a in_seq=0 out_seq=0 2023-11-27T15:23:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.645+0000 7f923affd700 10 client.16365 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:47.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.646+0000 7f923affd700 1 -- 172.21.15.52:0/3982780635 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557c83c5b6e0 con 0x557c83daf130 2023-11-27T15:23:47.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.646+0000 7f923affd700 10 client.16365 handle_command_reply: tid=0 2023-11-27T15:23:47.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.646+0000 7f926dae9b80 1 client.16365 shutdown 2023-11-27T15:23:47.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.646+0000 7f926dae9b80 1 -- 172.21.15.52:0/3982780635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c83d3c4f0 msgr2=0x557c83d49500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:47.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.646+0000 7f926dae9b80 1 --2- 172.21.15.52:0/3982780635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c83d3c4f0 0x557c83d49500 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f92500968b0 tx=0x7f9250099f00 comp rx=0 tx=0).stop 2023-11-27T15:23:47.665 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.646+0000 7f926dae9b80 1 -- 172.21.15.52:0/3982780635 shutdown_connections 2023-11-27T15:23:47.666 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.646+0000 7f926dae9b80 1 --2- 172.21.15.52:0/3982780635 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557c83daf130 0x557c83db1540 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.666 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.646+0000 7f926dae9b80 1 --2- 172.21.15.52:0/3982780635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c83d40930 0x557c83d49a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.666 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.646+0000 7f926dae9b80 1 --2- 172.21.15.52:0/3982780635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c83d3bbe0 0x557c83d48fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.666 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.646+0000 7f923affd700 0 client.16365 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:47.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.646+0000 7f926dae9b80 1 --2- 172.21.15.52:0/3982780635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c83d3c4f0 0x557c83d49500 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.646+0000 7f926dae9b80 1 -- 172.21.15.52:0/3982780635 >> 172.21.15.52:0/3982780635 conn(0x557c83d313e0 msgr2=0x557c83d3f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:47.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.646+0000 7f926dae9b80 1 -- 172.21.15.52:0/3982780635 shutdown_connections 2023-11-27T15:23:47.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.646+0000 7f926dae9b80 1 -- 172.21.15.52:0/3982780635 wait complete. 2023-11-27T15:23:47.668 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.646+0000 7f926dae9b80 20 client.16365 trim_cache size 0 max 16384 2023-11-27T15:23:47.668 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:23:47.668 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2023-11-27T15:23:47.668 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "155e3093-febc-4937-a68a-c6c9f7993adf", 2023-11-27T15:23:47.669 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2023-11-27T15:23:47.669 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:23:47.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.647+0000 7f926195d700 1 -- 172.21.15.52:0/3036097266 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f92440a1340 msgr2=0x7f92440a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:47.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.647+0000 7f926195d700 1 --2- 172.21.15.52:0/3036097266 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f92440a1340 0x7f92440a37f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f924c005950 tx=0x7f924c00b410 comp rx=0 tx=0).stop 2023-11-27T15:23:47.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.647+0000 7f926195d700 1 -- 172.21.15.52:0/3036097266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f925c0ffe60 msgr2=0x7f925c196ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:47.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.647+0000 7f926195d700 1 --2- 172.21.15.52:0/3036097266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f925c0ffe60 0x7f925c196ab0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f925000e940 tx=0x7f925000ec50 comp rx=0 tx=0).stop 2023-11-27T15:23:47.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.647+0000 7f926195d700 1 -- 172.21.15.52:0/3036097266 shutdown_connections 2023-11-27T15:23:47.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.647+0000 7f926195d700 1 --2- 172.21.15.52:0/3036097266 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f92440a1340 0x7f92440a37f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.647+0000 7f926195d700 1 --2- 172.21.15.52:0/3036097266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f925c104500 0x7f925c19aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.647+0000 7f926195d700 1 --2- 172.21.15.52:0/3036097266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f925c0ff4c0 0x7f925c196570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.647+0000 7f926195d700 1 --2- 172.21.15.52:0/3036097266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f925c0ffe60 0x7f925c196ab0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.647+0000 7f926195d700 1 -- 172.21.15.52:0/3036097266 >> 172.21.15.52:0/3036097266 conn(0x7f925c076200 msgr2=0x7f925c103000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:47.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.647+0000 7f926195d700 1 -- 172.21.15.52:0/3036097266 shutdown_connections 2023-11-27T15:23:47.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.647+0000 7f926195d700 1 -- 172.21.15.52:0/3036097266 wait complete. 2023-11-27T15:23:47.673 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:23:47.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.798+0000 7f6165825700 1 -- 172.21.15.52:0/1332584525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61601055f0 msgr2=0x7f616010d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:47.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.798+0000 7f6165825700 1 --2- 172.21.15.52:0/1332584525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61601055f0 0x7f616010d1c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f615800b0d0 tx=0x7f615800b3e0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.798+0000 7f6165825700 1 -- 172.21.15.52:0/1332584525 shutdown_connections 2023-11-27T15:23:47.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.798+0000 7f6165825700 1 --2- 172.21.15.52:0/1332584525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61601055f0 0x7f616010d1c0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.798+0000 7f6165825700 1 --2- 172.21.15.52:0/1332584525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61601005b0 0x7f6160100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.798+0000 7f6165825700 1 --2- 172.21.15.52:0/1332584525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6160100f50 0x7f6160104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.798+0000 7f6165825700 1 -- 172.21.15.52:0/1332584525 >> 172.21.15.52:0/1332584525 conn(0x7f6160077170 msgr2=0x7f6160077570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:47.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.798+0000 7f6165825700 1 -- 172.21.15.52:0/1332584525 shutdown_connections 2023-11-27T15:23:47.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.798+0000 7f6165825700 1 -- 172.21.15.52:0/1332584525 wait complete. 2023-11-27T15:23:47.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.798+0000 7f6165825700 1 Processor -- start 2023-11-27T15:23:47.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.799+0000 7f6165825700 1 -- start start 2023-11-27T15:23:47.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.799+0000 7f6165825700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61601005b0 0x7f6160072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.807 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.799+0000 7f6165825700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6160100f50 0x7f6160073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.807 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.799+0000 7f6165825700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61601055f0 0x7f61601a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.807 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.799+0000 7f6165825700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6160073b60 con 0x7f6160100f50 2023-11-27T15:23:47.807 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.799+0000 7f6165825700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61601a4c20 con 0x7f61601005b0 2023-11-27T15:23:47.807 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.799+0000 7f6165825700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61601a4d60 con 0x7f61601055f0 2023-11-27T15:23:47.808 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.799+0000 7f615e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6160100f50 0x7f6160073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.808 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.799+0000 7f615e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6160100f50 0x7f6160073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54322/0 (socket says 172.21.15.52:54322) 2023-11-27T15:23:47.808 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.799+0000 7f615e7fc700 1 -- 172.21.15.52:0/2874667931 learned_addr learned my addr 172.21.15.52:0/2874667931 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:47.808 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.799+0000 7f615effd700 1 --2- 172.21.15.52:0/2874667931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61601005b0 0x7f6160072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.799+0000 7f615f7fe700 1 --2- 172.21.15.52:0/2874667931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61601055f0 0x7f61601a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.799+0000 7f615e7fc700 1 -- 172.21.15.52:0/2874667931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61601055f0 msgr2=0x7f61601a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:47.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.799+0000 7f615e7fc700 1 --2- 172.21.15.52:0/2874667931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61601055f0 0x7f61601a46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.799+0000 7f615e7fc700 1 -- 172.21.15.52:0/2874667931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61601005b0 msgr2=0x7f6160072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:47.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.799+0000 7f615e7fc700 1 --2- 172.21.15.52:0/2874667931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61601005b0 0x7f6160072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.810 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.799+0000 7f615e7fc700 1 -- 172.21.15.52:0/2874667931 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6158009d00 con 0x7f6160100f50 2023-11-27T15:23:47.810 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.799+0000 7f615effd700 1 --2- 172.21.15.52:0/2874667931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61601005b0 0x7f6160072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:23:47.810 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.799+0000 7f615e7fc700 1 --2- 172.21.15.52:0/2874667931 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6160100f50 0x7f6160073330 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f615000d950 tx=0x7f615000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:47.810 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.800+0000 7f6164823700 1 -- 172.21.15.52:0/2874667931 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6150009b40 con 0x7f6160100f50 2023-11-27T15:23:47.810 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.800+0000 7f6165825700 1 -- 172.21.15.52:0/2874667931 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61601103d0 con 0x7f6160100f50 2023-11-27T15:23:47.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.800+0000 7f6164823700 1 -- 172.21.15.52:0/2874667931 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f615000de30 con 0x7f6160100f50 2023-11-27T15:23:47.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.800+0000 7f6164823700 1 -- 172.21.15.52:0/2874667931 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615000f620 con 0x7f6160100f50 2023-11-27T15:23:47.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.800+0000 7f6165825700 1 -- 172.21.15.52:0/2874667931 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6160110240 con 0x7f6160100f50 2023-11-27T15:23:47.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.802+0000 7f6164823700 1 -- 172.21.15.52:0/2874667931 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f6150010460 con 0x7f6160100f50 2023-11-27T15:23:47.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.803+0000 7f6164823700 1 --2- 172.21.15.52:0/2874667931 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f61480a12c0 0x7f61480a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.817 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.803+0000 7f615effd700 1 --2- 172.21.15.52:0/2874667931 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f61480a12c0 0x7f61480a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.818 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.803+0000 7f6164823700 1 -- 172.21.15.52:0/2874667931 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f61500cc940 con 0x7f6160100f50 2023-11-27T15:23:47.818 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.803+0000 7f61719b1b80 1 Processor -- start 2023-11-27T15:23:47.818 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.803+0000 7f61719b1b80 1 -- start start 2023-11-27T15:23:47.818 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.803+0000 7f615effd700 1 --2- 172.21.15.52:0/2874667931 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f61480a12c0 0x7f61480a3770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6154005fd0 tx=0x7f6154005ee0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:47.818 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.804+0000 7f61719b1b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d5f6e9b60 0x560d5f6e9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.804+0000 7f61719b1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d5f6ea470 0x560d5f6ee370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.804+0000 7f61719b1b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d5f6ee8b0 0x560d5f6eed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.804+0000 7f61719b1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5f5ef060 con 0x560d5f6ea470 2023-11-27T15:23:47.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.804+0000 7f61719b1b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5f6418a0 con 0x560d5f6e9b60 2023-11-27T15:23:47.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.804+0000 7f61719b1b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5f63ad30 con 0x560d5f6ee8b0 2023-11-27T15:23:47.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.804+0000 7f615e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d5f6ea470 0x560d5f6ee370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.804+0000 7f615e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d5f6ea470 0x560d5f6ee370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54332/0 (socket says 172.21.15.52:54332) 2023-11-27T15:23:47.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.804+0000 7f615e7fc700 1 -- 172.21.15.52:0/4029753150 learned_addr learned my addr 172.21.15.52:0/4029753150 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:47.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.804+0000 7f615effd700 1 --2- 172.21.15.52:0/4029753150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d5f6e9b60 0x560d5f6e9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.804+0000 7f615f7fe700 1 --2- 172.21.15.52:0/4029753150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d5f6ee8b0 0x560d5f6eed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.804+0000 7f615e7fc700 1 -- 172.21.15.52:0/4029753150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d5f6ee8b0 msgr2=0x560d5f6eed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:47.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.804+0000 7f615e7fc700 1 --2- 172.21.15.52:0/4029753150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d5f6ee8b0 0x560d5f6eed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.804+0000 7f615e7fc700 1 -- 172.21.15.52:0/4029753150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d5f6e9b60 msgr2=0x560d5f6e9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:47.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.804+0000 7f615e7fc700 1 --2- 172.21.15.52:0/4029753150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d5f6e9b60 0x560d5f6e9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.804+0000 7f615e7fc700 1 -- 172.21.15.52:0/4029753150 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d5f4bc630 con 0x560d5f6ea470 2023-11-27T15:23:47.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.804+0000 7f615e7fc700 1 --2- 172.21.15.52:0/4029753150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d5f6ea470 0x560d5f6ee370 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f6150095eb0 tx=0x7f6150099390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958b2f77738843fb server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:47.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.804+0000 7f6145ffb700 1 -- 172.21.15.52:0/4029753150 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61500a1e00 con 0x560d5f6ea470 2023-11-27T15:23:47.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.804+0000 7f6145ffb700 1 -- 172.21.15.52:0/4029753150 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6150097d50 con 0x560d5f6ea470 2023-11-27T15:23:47.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.804+0000 7f6145ffb700 1 -- 172.21.15.52:0/4029753150 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61500a1970 con 0x560d5f6ea470 2023-11-27T15:23:47.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.805+0000 7f61719b1b80 1 -- 172.21.15.52:0/4029753150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d5f6ea470 msgr2=0x560d5f6ee370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:47.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.805+0000 7f61719b1b80 1 --2- 172.21.15.52:0/4029753150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d5f6ea470 0x560d5f6ee370 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f6150095eb0 tx=0x7f6150099390 comp rx=0 tx=0).stop 2023-11-27T15:23:47.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.805+0000 7f61719b1b80 1 -- 172.21.15.52:0/4029753150 shutdown_connections 2023-11-27T15:23:47.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.805+0000 7f61719b1b80 1 --2- 172.21.15.52:0/4029753150 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d5f6e9b60 0x560d5f6e9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.805+0000 7f61719b1b80 1 --2- 172.21.15.52:0/4029753150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d5f6ee8b0 0x560d5f6eed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.805+0000 7f61719b1b80 1 --2- 172.21.15.52:0/4029753150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d5f6ea470 0x560d5f6ee370 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.805+0000 7f61719b1b80 1 -- 172.21.15.52:0/4029753150 >> 172.21.15.52:0/4029753150 conn(0x560d5f6df360 msgr2=0x560d5f6e4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:47.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.805+0000 7f61719b1b80 1 -- 172.21.15.52:0/4029753150 shutdown_connections 2023-11-27T15:23:47.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.805+0000 7f61719b1b80 1 -- 172.21.15.52:0/4029753150 wait complete. 2023-11-27T15:23:47.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.806+0000 7f61719b1b80 1 -- start start 2023-11-27T15:23:47.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.806+0000 7f61719b1b80 1 Processor -- start 2023-11-27T15:23:47.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.806+0000 7f61719b1b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d5f6e9b60 0x560d5f6f6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.806+0000 7f61719b1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d5f6ea470 0x560d5f6f3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.806+0000 7f61719b1b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d5f6ee8b0 0x560d5f6f3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.806+0000 7f61719b1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5f5dc8e0 con 0x560d5f6ea470 2023-11-27T15:23:47.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.806+0000 7f61719b1b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5f5dc580 con 0x560d5f6ee8b0 2023-11-27T15:23:47.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.806+0000 7f61719b1b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5f4be920 con 0x560d5f6e9b60 2023-11-27T15:23:47.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.806+0000 7f61719b1b80 10 client.0 _ll_register_callbacks cb 0x560d5f5fa780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:47.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.806+0000 7f615e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d5f6ea470 0x560d5f6f3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.806+0000 7f615e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d5f6ea470 0x560d5f6f3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54336/0 (socket says 172.21.15.52:54336) 2023-11-27T15:23:47.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.806+0000 7f615e7fc700 1 -- 172.21.15.52:0/2062747207 learned_addr learned my addr 172.21.15.52:0/2062747207 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:47.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.807+0000 7f615f7fe700 1 --2- 172.21.15.52:0/2062747207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d5f6ee8b0 0x560d5f6f3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.807+0000 7f615effd700 1 --2- 172.21.15.52:0/2062747207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d5f6e9b60 0x560d5f6f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.807+0000 7f615e7fc700 1 -- 172.21.15.52:0/2062747207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d5f6e9b60 msgr2=0x560d5f6f6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:47.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.807+0000 7f615e7fc700 1 --2- 172.21.15.52:0/2062747207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d5f6e9b60 0x560d5f6f6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.807+0000 7f615e7fc700 1 -- 172.21.15.52:0/2062747207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d5f6ee8b0 msgr2=0x560d5f6f3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:47.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.807+0000 7f615e7fc700 1 --2- 172.21.15.52:0/2062747207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d5f6ee8b0 0x560d5f6f3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.807+0000 7f615e7fc700 1 -- 172.21.15.52:0/2062747207 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6160110240 con 0x560d5f6ea470 2023-11-27T15:23:47.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.807+0000 7f615f7fe700 1 --2- 172.21.15.52:0/2062747207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d5f6ee8b0 0x560d5f6f3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:23:47.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.807+0000 7f615e7fc700 1 --2- 172.21.15.52:0/2062747207 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d5f6ea470 0x560d5f6f3250 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f6150009520 tx=0x7f61500a1e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc4a9bbd3b723a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:47.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.807+0000 7f613effd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:23:47.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.807+0000 7f613effd700 1 -- 172.21.15.52:0/2062747207 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6150098ea0 con 0x560d5f6ea470 2023-11-27T15:23:47.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.807+0000 7f613effd700 1 -- 172.21.15.52:0/2062747207 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f61500b1070 con 0x560d5f6ea470 2023-11-27T15:23:47.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.807+0000 7f61719b1b80 1 -- 172.21.15.52:0/2062747207 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560d5f6dcb20 con 0x560d5f6ea470 2023-11-27T15:23:47.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.807+0000 7f613effd700 1 -- 172.21.15.52:0/2062747207 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6150099000 con 0x560d5f6ea470 2023-11-27T15:23:47.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.807+0000 7f61719b1b80 10 client.16377 fetch_fsmap learned FSMap version 28 2023-11-27T15:23:47.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.807+0000 7f61719b1b80 1 -- 172.21.15.52:0/2062747207 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x560d5f601190 con 0x560d5f6ea470 2023-11-27T15:23:47.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.807+0000 7f613effd700 1 -- 172.21.15.52:0/2062747207 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6150063080 con 0x560d5f6ea470 2023-11-27T15:23:47.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.808+0000 7f613effd700 1 -- 172.21.15.52:0/2062747207 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6150098910 con 0x560d5f6ea470 2023-11-27T15:23:47.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.808+0000 7f61719b1b80 10 client.16377 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:23:47.832 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.808+0000 7f61719b1b80 10 client.16377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:47.832 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.808+0000 7f61719b1b80 20 client.16377 populate_metadata read hostname 'smithi052' 2023-11-27T15:23:47.832 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.808+0000 7f61719b1b80 1 --2- 172.21.15.52:0/2062747207 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560d5f7033e0 0x560d5f7057f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.832 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.808+0000 7f61719b1b80 4 client.16377 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:23:47.832 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.808+0000 7f61719b1b80 1 -- 172.21.15.52:0/2062747207 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560d5f5fd0a0 con 0x560d5f7033e0 2023-11-27T15:23:47.832 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.808+0000 7f615effd700 1 --2- 172.21.15.52:0/2062747207 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560d5f7033e0 0x560d5f7057f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.832 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.808+0000 7f613effd700 10 client.16377 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:47.832 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.808+0000 7f615effd700 1 --2- 172.21.15.52:0/2062747207 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560d5f7033e0 0x560d5f7057f0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd026453dc0bd1f1 server_cookie=d1f3fc960acb2d35 in_seq=0 out_seq=0 2023-11-27T15:23:47.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.809+0000 7f613effd700 1 -- 172.21.15.52:0/2062747207 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560d5f5fd0a0 con 0x560d5f7033e0 2023-11-27T15:23:47.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.809+0000 7f613effd700 10 client.16377 handle_command_reply: tid=0 2023-11-27T15:23:47.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.809+0000 7f61719b1b80 1 client.16377 shutdown 2023-11-27T15:23:47.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.809+0000 7f61719b1b80 1 -- 172.21.15.52:0/2062747207 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d5f6ea470 msgr2=0x560d5f6f3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:47.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.809+0000 7f61719b1b80 1 --2- 172.21.15.52:0/2062747207 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d5f6ea470 0x560d5f6f3250 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f6150009520 tx=0x7f61500a1e00 comp rx=0 tx=0).stop 2023-11-27T15:23:47.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.809+0000 7f61719b1b80 1 -- 172.21.15.52:0/2062747207 shutdown_connections 2023-11-27T15:23:47.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.809+0000 7f61719b1b80 1 --2- 172.21.15.52:0/2062747207 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560d5f7033e0 0x560d5f7057f0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.809+0000 7f61719b1b80 1 --2- 172.21.15.52:0/2062747207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d5f6ee8b0 0x560d5f6f3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.834 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.809+0000 7f61719b1b80 1 --2- 172.21.15.52:0/2062747207 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d5f6e9b60 0x560d5f6f6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.834 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.809+0000 7f613effd700 0 client.16377 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:47.834 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.809+0000 7f61719b1b80 1 --2- 172.21.15.52:0/2062747207 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d5f6ea470 0x560d5f6f3250 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.834 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.809+0000 7f61719b1b80 1 -- 172.21.15.52:0/2062747207 >> 172.21.15.52:0/2062747207 conn(0x560d5f6df360 msgr2=0x560d5f6e3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:47.834 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.809+0000 7f61719b1b80 1 -- 172.21.15.52:0/2062747207 shutdown_connections 2023-11-27T15:23:47.834 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.809+0000 7f61719b1b80 1 -- 172.21.15.52:0/2062747207 wait complete. 2023-11-27T15:23:47.834 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.809+0000 7f61719b1b80 20 client.16377 trim_cache size 0 max 16384 2023-11-27T15:23:47.839 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.826+0000 7f61719b1b80 1 Processor -- start 2023-11-27T15:23:47.839 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.826+0000 7f61719b1b80 1 -- start start 2023-11-27T15:23:47.839 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.826+0000 7f61719b1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d5f6e9b60 0x560d5f6f0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.826+0000 7f61719b1b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d5f6ea470 0x560d5f6fa9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.826+0000 7f61719b1b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d5f6ee8b0 0x560d5f6f5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.826+0000 7f61719b1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5f6f3e60 con 0x560d5f6e9b60 2023-11-27T15:23:47.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.826+0000 7f61719b1b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5f6ea970 con 0x560d5f6ee8b0 2023-11-27T15:23:47.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.826+0000 7f61719b1b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5f6f1240 con 0x560d5f6ea470 2023-11-27T15:23:47.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.826+0000 7f615effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d5f6e9b60 0x560d5f6f0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.826+0000 7f615effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d5f6e9b60 0x560d5f6f0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54342/0 (socket says 172.21.15.52:54342) 2023-11-27T15:23:47.840 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.826+0000 7f615effd700 1 -- 172.21.15.52:0/3046222665 learned_addr learned my addr 172.21.15.52:0/3046222665 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:47.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.826+0000 7f615f7fe700 1 --2- 172.21.15.52:0/3046222665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d5f6ee8b0 0x560d5f6f5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.827+0000 7f615e7fc700 1 --2- 172.21.15.52:0/3046222665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d5f6ea470 0x560d5f6fa9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.827+0000 7f615effd700 1 -- 172.21.15.52:0/3046222665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d5f6ea470 msgr2=0x560d5f6fa9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:47.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.827+0000 7f615effd700 1 --2- 172.21.15.52:0/3046222665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d5f6ea470 0x560d5f6fa9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.827+0000 7f615effd700 1 -- 172.21.15.52:0/3046222665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d5f6ee8b0 msgr2=0x560d5f6f5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:47.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.827+0000 7f615effd700 1 --2- 172.21.15.52:0/3046222665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d5f6ee8b0 0x560d5f6f5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.827+0000 7f615effd700 1 -- 172.21.15.52:0/3046222665 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d5f600ee0 con 0x560d5f6e9b60 2023-11-27T15:23:47.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.827+0000 7f615f7fe700 1 --2- 172.21.15.52:0/3046222665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d5f6ee8b0 0x560d5f6f5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:47.842 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.827+0000 7f615effd700 1 --2- 172.21.15.52:0/3046222665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d5f6e9b60 0x560d5f6f0d00 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f615400e680 tx=0x7f6154014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced1af69956315a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:47.842 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.827+0000 7f61467fc700 1 -- 172.21.15.52:0/3046222665 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6154025040 con 0x560d5f6e9b60 2023-11-27T15:23:47.842 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.827+0000 7f61467fc700 1 -- 172.21.15.52:0/3046222665 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6154011e10 con 0x560d5f6e9b60 2023-11-27T15:23:47.842 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.827+0000 7f61467fc700 1 -- 172.21.15.52:0/3046222665 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6154004bb0 con 0x560d5f6e9b60 2023-11-27T15:23:47.842 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.827+0000 7f61719b1b80 1 -- 172.21.15.52:0/3046222665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d5f6e9b60 msgr2=0x560d5f6f0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:47.842 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.827+0000 7f61719b1b80 1 --2- 172.21.15.52:0/3046222665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d5f6e9b60 0x560d5f6f0d00 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f615400e680 tx=0x7f6154014000 comp rx=0 tx=0).stop 2023-11-27T15:23:47.842 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.828+0000 7f61719b1b80 1 -- 172.21.15.52:0/3046222665 shutdown_connections 2023-11-27T15:23:47.842 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.828+0000 7f61719b1b80 1 --2- 172.21.15.52:0/3046222665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d5f6ee8b0 0x560d5f6f5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.828+0000 7f61719b1b80 1 --2- 172.21.15.52:0/3046222665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d5f6e9b60 0x560d5f6f0d00 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.828+0000 7f61719b1b80 1 --2- 172.21.15.52:0/3046222665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d5f6ea470 0x560d5f6fa9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.828+0000 7f61719b1b80 1 -- 172.21.15.52:0/3046222665 >> 172.21.15.52:0/3046222665 conn(0x560d5f6df360 msgr2=0x560d5f6ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:47.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.828+0000 7f61719b1b80 1 -- 172.21.15.52:0/3046222665 shutdown_connections 2023-11-27T15:23:47.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.828+0000 7f61719b1b80 1 -- 172.21.15.52:0/3046222665 wait complete. 2023-11-27T15:23:47.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.828+0000 7f61719b1b80 1 -- start start 2023-11-27T15:23:47.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.828+0000 7f61719b1b80 1 Processor -- start 2023-11-27T15:23:47.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.828+0000 7f61719b1b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d5f6e9b60 0x560d5f6fa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.828+0000 7f61719b1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d5f6ea470 0x560d5f6ef460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.828+0000 7f61719b1b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d5f6ee8b0 0x560d5f6ef9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.828+0000 7f61719b1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5f6eff20 con 0x560d5f6ea470 2023-11-27T15:23:47.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.828+0000 7f61719b1b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5f6f0060 con 0x560d5f6ee8b0 2023-11-27T15:23:47.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.828+0000 7f61719b1b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d5f6f0220 con 0x560d5f6e9b60 2023-11-27T15:23:47.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.828+0000 7f61719b1b80 10 client.0 _ll_register_callbacks cb 0x7f6150099580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:47.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.828+0000 7f615e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d5f6ea470 0x560d5f6ef460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.829+0000 7f615e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d5f6ea470 0x560d5f6ef460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54350/0 (socket says 172.21.15.52:54350) 2023-11-27T15:23:47.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.829+0000 7f615e7fc700 1 -- 172.21.15.52:0/4072759525 learned_addr learned my addr 172.21.15.52:0/4072759525 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:23:47.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.829+0000 7f615effd700 1 --2- 172.21.15.52:0/4072759525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d5f6e9b60 0x560d5f6fa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.829+0000 7f615e7fc700 1 -- 172.21.15.52:0/4072759525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d5f6e9b60 msgr2=0x560d5f6fa9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:47.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.829+0000 7f615e7fc700 1 --2- 172.21.15.52:0/4072759525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d5f6e9b60 0x560d5f6fa9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.829+0000 7f615e7fc700 1 -- 172.21.15.52:0/4072759525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d5f6ee8b0 msgr2=0x560d5f6ef9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-27T15:23:47.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.829+0000 7f615e7fc700 1 --2- 172.21.15.52:0/4072759525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d5f6ee8b0 0x560d5f6ef9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.829+0000 7f615e7fc700 1 -- 172.21.15.52:0/4072759525 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f615400ea60 con 0x560d5f6ea470 2023-11-27T15:23:47.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.829+0000 7f615e7fc700 1 --2- 172.21.15.52:0/4072759525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d5f6ea470 0x560d5f6ef460 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f61500cc170 tx=0x7f61500ca130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6833471c5c5f54c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:47.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.829+0000 7f613effd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:23:47.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.829+0000 7f613effd700 1 -- 172.21.15.52:0/4072759525 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61500a1970 con 0x560d5f6ea470 2023-11-27T15:23:47.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.829+0000 7f613effd700 1 -- 172.21.15.52:0/4072759525 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8527+0+0 (secure 0 0 0) 0x7f6150064060 con 0x560d5f6ea470 2023-11-27T15:23:47.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.829+0000 7f61719b1b80 1 -- 172.21.15.52:0/4072759525 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6150097d50 con 0x560d5f6ea470 2023-11-27T15:23:47.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.829+0000 7f613effd700 1 -- 172.21.15.52:0/4072759525 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61500a1ad0 con 0x560d5f6ea470 2023-11-27T15:23:47.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.829+0000 7f61719b1b80 10 client.16383 fetch_fsmap learned FSMap version 28 2023-11-27T15:23:47.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.829+0000 7f61719b1b80 1 -- 172.21.15.52:0/4072759525 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x560d5f451c60 con 0x560d5f6ea470 2023-11-27T15:23:47.848 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.829+0000 7f613effd700 1 -- 172.21.15.52:0/4072759525 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6150097d50 con 0x560d5f6ea470 2023-11-27T15:23:47.848 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.830+0000 7f613effd700 1 -- 172.21.15.52:0/4072759525 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f61500b90b0 con 0x560d5f6ea470 2023-11-27T15:23:47.848 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.830+0000 7f61719b1b80 10 client.16383 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:23:47.848 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.830+0000 7f61719b1b80 10 client.16383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:47.848 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.830+0000 7f61719b1b80 20 client.16383 populate_metadata read hostname 'smithi052' 2023-11-27T15:23:47.849 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.830+0000 7f61719b1b80 1 --2- 172.21.15.52:0/4072759525 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560d5f758340 0x560d5f75a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.849 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.830+0000 7f61719b1b80 4 client.16383 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:23:47.849 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.830+0000 7f61719b1b80 1 -- 172.21.15.52:0/4072759525 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560d5f609660 con 0x560d5f758340 2023-11-27T15:23:47.849 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.830+0000 7f615effd700 1 --2- 172.21.15.52:0/4072759525 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560d5f758340 0x560d5f75a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.849 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.830+0000 7f613effd700 10 client.16383 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:47.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.830+0000 7f615effd700 1 --2- 172.21.15.52:0/4072759525 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560d5f758340 0x560d5f75a750 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3151908afe8dac5 server_cookie=b6fa872a17985905 in_seq=0 out_seq=0 2023-11-27T15:23:47.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.830+0000 7f613effd700 1 -- 172.21.15.52:0/4072759525 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560d5f609660 con 0x560d5f758340 2023-11-27T15:23:47.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.830+0000 7f613effd700 10 client.16383 handle_command_reply: tid=0 2023-11-27T15:23:47.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.830+0000 7f61719b1b80 1 client.16383 shutdown 2023-11-27T15:23:47.851 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.831+0000 7f61719b1b80 1 -- 172.21.15.52:0/4072759525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d5f6ea470 msgr2=0x560d5f6ef460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:47.851 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.831+0000 7f61719b1b80 1 --2- 172.21.15.52:0/4072759525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d5f6ea470 0x560d5f6ef460 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f61500cc170 tx=0x7f61500ca130 comp rx=0 tx=0).stop 2023-11-27T15:23:47.851 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.831+0000 7f61719b1b80 1 -- 172.21.15.52:0/4072759525 shutdown_connections 2023-11-27T15:23:47.851 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.831+0000 7f61719b1b80 1 --2- 172.21.15.52:0/4072759525 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560d5f758340 0x560d5f75a750 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.851 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.831+0000 7f61719b1b80 1 --2- 172.21.15.52:0/4072759525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560d5f6ee8b0 0x560d5f6ef9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.852 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.831+0000 7f613effd700 0 client.16383 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:23:47.852 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.831+0000 7f61719b1b80 1 --2- 172.21.15.52:0/4072759525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560d5f6e9b60 0x560d5f6fa9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.852 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.831+0000 7f61719b1b80 1 --2- 172.21.15.52:0/4072759525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d5f6ea470 0x560d5f6ef460 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.852 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.831+0000 7f61719b1b80 1 -- 172.21.15.52:0/4072759525 >> 172.21.15.52:0/4072759525 conn(0x560d5f6df360 msgr2=0x560d5f6e8960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:47.853 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.831+0000 7f61719b1b80 1 -- 172.21.15.52:0/4072759525 shutdown_connections 2023-11-27T15:23:47.853 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.831+0000 7f61719b1b80 1 -- 172.21.15.52:0/4072759525 wait complete. 2023-11-27T15:23:47.853 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.831+0000 7f61719b1b80 20 client.16383 trim_cache size 0 max 16384 2023-11-27T15:23:47.853 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:23:47.854 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-27T15:23:47.854 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2023-11-27T15:23:47.854 INFO:teuthology.orchestra.run.smithi052.stdout: "155e3093-febc-4937-a68a-c6c9f7993adf": { 2023-11-27T15:23:47.854 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2023-11-27T15:23:47.855 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "155e3093-febc-4937-a68a-c6c9f7993adf", 2023-11-27T15:23:47.855 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2023-11-27T15:23:47.855 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:23:47.855 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:23:47.856 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:23:47.857 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.831+0000 7f6165825700 1 -- 172.21.15.52:0/2874667931 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f61480a12c0 msgr2=0x7f61480a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:47.857 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.831+0000 7f6165825700 1 --2- 172.21.15.52:0/2874667931 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f61480a12c0 0x7f61480a3770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6154005fd0 tx=0x7f6154005ee0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.857 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.831+0000 7f6165825700 1 -- 172.21.15.52:0/2874667931 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6160100f50 msgr2=0x7f6160073330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:47.857 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.831+0000 7f6165825700 1 --2- 172.21.15.52:0/2874667931 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6160100f50 0x7f6160073330 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f615000d950 tx=0x7f615000dd10 comp rx=0 tx=0).stop 2023-11-27T15:23:47.858 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.831+0000 7f6165825700 1 -- 172.21.15.52:0/2874667931 shutdown_connections 2023-11-27T15:23:47.858 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.831+0000 7f6165825700 1 --2- 172.21.15.52:0/2874667931 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f61480a12c0 0x7f61480a3770 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.858 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.831+0000 7f6165825700 1 --2- 172.21.15.52:0/2874667931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61601005b0 0x7f6160072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.858 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.831+0000 7f6165825700 1 --2- 172.21.15.52:0/2874667931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61601055f0 0x7f61601a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.859 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.831+0000 7f6165825700 1 --2- 172.21.15.52:0/2874667931 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6160100f50 0x7f6160073330 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.859 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.831+0000 7f6165825700 1 -- 172.21.15.52:0/2874667931 >> 172.21.15.52:0/2874667931 conn(0x7f6160077170 msgr2=0x7f61600fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:47.859 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.832+0000 7f6165825700 1 -- 172.21.15.52:0/2874667931 shutdown_connections 2023-11-27T15:23:47.859 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:23:47.832+0000 7f6165825700 1 -- 172.21.15.52:0/2874667931 wait complete. 2023-11-27T15:23:47.860 INFO:tasks.cephfs.filesystem:scrub status for tag:155e3093-febc-4937-a68a-c6c9f7993adf - {'path': '/', 'tag': '155e3093-febc-4937-a68a-c6c9f7993adf', 'options': 'recursive,force'} 2023-11-27T15:23:48.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:48 smithi150 ceph-mon[100559]: pgmap v179: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-11-27T15:23:48.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:48 smithi150 ceph-mon[100559]: scrub queued for path: / 2023-11-27T15:23:48.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:48 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:23:48.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:48 smithi150 ceph-mon[100559]: scrub summary: active paths [/] 2023-11-27T15:23:48.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:48 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:23:48.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:48 smithi102 ceph-mon[100296]: pgmap v179: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-11-27T15:23:48.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:48 smithi102 ceph-mon[100296]: scrub queued for path: / 2023-11-27T15:23:48.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:48 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:23:48.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:48 smithi102 ceph-mon[100296]: scrub summary: active paths [/] 2023-11-27T15:23:48.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:48 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:23:48.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:48 smithi052 ceph-mon[99056]: pgmap v179: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-11-27T15:23:48.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:48 smithi052 ceph-mon[99056]: scrub queued for path: / 2023-11-27T15:23:48.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:48 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:23:48.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:48 smithi052 ceph-mon[99056]: scrub summary: active paths [/] 2023-11-27T15:23:48.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:48 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:23:50.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:49 smithi150 ceph-mon[100559]: pgmap v180: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2023-11-27T15:23:50.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:49 smithi150 ceph-mon[100559]: scrub summary: idle 2023-11-27T15:23:50.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:49 smithi102 ceph-mon[100296]: pgmap v180: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2023-11-27T15:23:50.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:49 smithi102 ceph-mon[100296]: scrub summary: idle 2023-11-27T15:23:50.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:49 smithi052 ceph-mon[99056]: pgmap v180: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2023-11-27T15:23:50.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:49 smithi052 ceph-mon[99056]: scrub summary: idle 2023-11-27T15:23:52.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:52 smithi150 ceph-mon[100559]: pgmap v181: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-11-27T15:23:52.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:52 smithi102 ceph-mon[100296]: pgmap v181: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-11-27T15:23:52.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:52 smithi052 ceph-mon[99056]: pgmap v181: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-11-27T15:23:54.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:54 smithi150 ceph-mon[100559]: pgmap v182: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:23:54.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:54 smithi102 ceph-mon[100296]: pgmap v182: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:23:54.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:54 smithi052 ceph-mon[99056]: pgmap v182: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:23:56.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:56 smithi150 ceph-mon[100559]: pgmap v183: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:23:56.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:56 smithi102 ceph-mon[100296]: pgmap v183: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:23:56.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:56 smithi052 ceph-mon[99056]: pgmap v183: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:23:58.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:58 smithi150 ceph-mon[100559]: pgmap v184: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:23:58.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:58 smithi102 ceph-mon[100296]: pgmap v184: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:23:58.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:58 smithi052 ceph-mon[99056]: pgmap v184: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:24:00.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:23:59 smithi150 ceph-mon[100559]: pgmap v185: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:24:00.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:23:59 smithi102 ceph-mon[100296]: pgmap v185: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:24:00.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:23:59 smithi052 ceph-mon[99056]: pgmap v185: 97 pgs: 97 active+clean; 712 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:24:02.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:02 smithi150 ceph-mon[100559]: pgmap v186: 97 pgs: 97 active+clean; 712 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-27T15:24:02.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:02 smithi102 ceph-mon[100296]: pgmap v186: 97 pgs: 97 active+clean; 712 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-27T15:24:02.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:02 smithi052 ceph-mon[99056]: pgmap v186: 97 pgs: 97 active+clean; 712 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-27T15:24:04.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:04 smithi150 ceph-mon[100559]: pgmap v187: 97 pgs: 97 active+clean; 712 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-27T15:24:04.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:04 smithi102 ceph-mon[100296]: pgmap v187: 97 pgs: 97 active+clean; 712 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-27T15:24:04.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:04 smithi052 ceph-mon[99056]: pgmap v187: 97 pgs: 97 active+clean; 712 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-27T15:24:06.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:06 smithi150 ceph-mon[100559]: pgmap v188: 97 pgs: 97 active+clean; 712 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-27T15:24:06.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:06 smithi102 ceph-mon[100296]: pgmap v188: 97 pgs: 97 active+clean; 712 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-27T15:24:06.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:06 smithi052 ceph-mon[99056]: pgmap v188: 97 pgs: 97 active+clean; 712 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-27T15:24:08.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:08 smithi150 ceph-mon[100559]: pgmap v189: 97 pgs: 97 active+clean; 712 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-27T15:24:08.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:08 smithi102 ceph-mon[100296]: pgmap v189: 97 pgs: 97 active+clean; 712 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-27T15:24:08.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:08 smithi052 ceph-mon[99056]: pgmap v189: 97 pgs: 97 active+clean; 712 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-27T15:24:10.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:09 smithi150 ceph-mon[100559]: pgmap v190: 97 pgs: 97 active+clean; 712 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-27T15:24:10.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:09 smithi102 ceph-mon[100296]: pgmap v190: 97 pgs: 97 active+clean; 712 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-27T15:24:10.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:09 smithi052 ceph-mon[99056]: pgmap v190: 97 pgs: 97 active+clean; 712 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-27T15:24:11.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:10 smithi150 ceph-mon[100559]: osdmap e87: 12 total, 12 up, 12 in 2023-11-27T15:24:11.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:10 smithi102 ceph-mon[100296]: osdmap e87: 12 total, 12 up, 12 in 2023-11-27T15:24:11.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:10 smithi052 ceph-mon[99056]: osdmap e87: 12 total, 12 up, 12 in 2023-11-27T15:24:12.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:11 smithi150 ceph-mon[100559]: pgmap v192: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:24:12.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:11 smithi102 ceph-mon[100296]: pgmap v192: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:24:12.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:11 smithi052 ceph-mon[99056]: pgmap v192: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:24:14.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:14 smithi150 ceph-mon[100559]: pgmap v193: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:24:14.988 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:14 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:24:14.988 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:14 smithi150 ceph-mon[100559]: osdmap e88: 12 total, 12 up, 12 in 2023-11-27T15:24:14.990 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:14 smithi052 ceph-mon[99056]: pgmap v193: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:24:14.991 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:14 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:24:14.991 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:14 smithi052 ceph-mon[99056]: osdmap e88: 12 total, 12 up, 12 in 2023-11-27T15:24:14.992 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:14 smithi102 ceph-mon[100296]: pgmap v193: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:24:14.992 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:14 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:24:14.992 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:14 smithi102 ceph-mon[100296]: osdmap e88: 12 total, 12 up, 12 in 2023-11-27T15:24:15.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:15 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:24:15.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:15 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:24:15.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:15 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:24:15.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:15 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:24:15.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:15 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:24:15.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:15 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:24:15.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:15 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:24:15.624 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:15 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:24:15.625 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:15 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:24:16.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:16 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:24:16.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:16 smithi150 ceph-mon[100559]: Service mds.all has no related daemon already placed. 2023-11-27T15:24:16.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:16 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:24:16.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:16 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:24:16.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:16 smithi150 ceph-mon[100559]: pgmap v195: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2023-11-27T15:24:16.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:16 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:24:16.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:16 smithi102 ceph-mon[100296]: Service mds.all has no related daemon already placed. 2023-11-27T15:24:16.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:16 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:24:16.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:16 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:24:16.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:16 smithi102 ceph-mon[100296]: pgmap v195: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2023-11-27T15:24:16.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:16 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:24:16.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:16 smithi052 ceph-mon[99056]: Service mds.all has no related daemon already placed. 2023-11-27T15:24:16.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:16 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:24:16.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:16 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:24:16.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:16 smithi052 ceph-mon[99056]: pgmap v195: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2023-11-27T15:24:17.861 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:24:18.006 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.002+0000 7f1234c95700 1 -- 172.21.15.52:0/2525779013 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12300ffec0 msgr2=0x7f1230103e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.006 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.002+0000 7f1234c95700 1 --2- 172.21.15.52:0/2525779013 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12300ffec0 0x7f1230103e80 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f12200099c0 tx=0x7f1220009cd0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.007 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.002+0000 7f1234c95700 1 -- 172.21.15.52:0/2525779013 shutdown_connections 2023-11-27T15:24:18.007 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.002+0000 7f1234c95700 1 --2- 172.21.15.52:0/2525779013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1230104560 0x7f123010c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.007 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.002+0000 7f1234c95700 1 --2- 172.21.15.52:0/2525779013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12300ff520 0x7f12300ff8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.007 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.002+0000 7f1234c95700 1 --2- 172.21.15.52:0/2525779013 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12300ffec0 0x7f1230103e80 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.007 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.002+0000 7f1234c95700 1 -- 172.21.15.52:0/2525779013 >> 172.21.15.52:0/2525779013 conn(0x7f12300760d0 msgr2=0x7f12300764d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.007 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.002+0000 7f1234c95700 1 -- 172.21.15.52:0/2525779013 shutdown_connections 2023-11-27T15:24:18.007 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.002+0000 7f1234c95700 1 -- 172.21.15.52:0/2525779013 wait complete. 2023-11-27T15:24:18.009 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.003+0000 7f1234c95700 1 Processor -- start 2023-11-27T15:24:18.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.003+0000 7f1234c95700 1 -- start start 2023-11-27T15:24:18.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.003+0000 7f1234c95700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12300ff520 0x7f1230197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.003+0000 7f1234c95700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12300ffec0 0x7f1230197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.003+0000 7f1234c95700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1230104560 0x7f12301a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.003+0000 7f1234c95700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1230198090 con 0x7f12300ffec0 2023-11-27T15:24:18.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.003+0000 7f1234c95700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12301a2800 con 0x7f1230104560 2023-11-27T15:24:18.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.003+0000 7f1234c95700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12301a2970 con 0x7f12300ff520 2023-11-27T15:24:18.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.003+0000 7f122dd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12300ffec0 0x7f1230197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.003+0000 7f122dd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12300ffec0 0x7f1230197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54364/0 (socket says 172.21.15.52:54364) 2023-11-27T15:24:18.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.003+0000 7f122dd9b700 1 -- 172.21.15.52:0/4056374840 learned_addr learned my addr 172.21.15.52:0/4056374840 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:18.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.003+0000 7f122e59c700 1 --2- 172.21.15.52:0/4056374840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12300ff520 0x7f1230197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.003+0000 7f122ed9d700 1 --2- 172.21.15.52:0/4056374840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1230104560 0x7f12301a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.004+0000 7f122dd9b700 1 -- 172.21.15.52:0/4056374840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12300ff520 msgr2=0x7f1230197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.004+0000 7f122dd9b700 1 --2- 172.21.15.52:0/4056374840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12300ff520 0x7f1230197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.004+0000 7f122dd9b700 1 -- 172.21.15.52:0/4056374840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1230104560 msgr2=0x7f12301a22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.004+0000 7f122dd9b700 1 --2- 172.21.15.52:0/4056374840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1230104560 0x7f12301a22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.004+0000 7f122dd9b700 1 -- 172.21.15.52:0/4056374840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12200095f0 con 0x7f12300ffec0 2023-11-27T15:24:18.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.004+0000 7f122dd9b700 1 --2- 172.21.15.52:0/4056374840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12300ffec0 0x7f1230197780 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1220009fd0 tx=0x7f1220009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.004+0000 7f121f7fe700 1 -- 172.21.15.52:0/4056374840 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122001d070 con 0x7f12300ffec0 2023-11-27T15:24:18.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.004+0000 7f121f7fe700 1 -- 172.21.15.52:0/4056374840 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1220005020 con 0x7f12300ffec0 2023-11-27T15:24:18.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.004+0000 7f1234c95700 1 -- 172.21.15.52:0/4056374840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f123010f2e0 con 0x7f12300ffec0 2023-11-27T15:24:18.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.004+0000 7f121f7fe700 1 -- 172.21.15.52:0/4056374840 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1220004aa0 con 0x7f12300ffec0 2023-11-27T15:24:18.013 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.004+0000 7f1234c95700 1 -- 172.21.15.52:0/4056374840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123010f150 con 0x7f12300ffec0 2023-11-27T15:24:18.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.006+0000 7f121f7fe700 1 -- 172.21.15.52:0/4056374840 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f12200054b0 con 0x7f12300ffec0 2023-11-27T15:24:18.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.007+0000 7f121f7fe700 1 --2- 172.21.15.52:0/4056374840 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f12180a12c0 0x7f12180a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.007+0000 7f122e59c700 1 --2- 172.21.15.52:0/4056374840 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f12180a12c0 0x7f12180a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.007+0000 7f121f7fe700 1 -- 172.21.15.52:0/4056374840 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f12200cd890 con 0x7f12300ffec0 2023-11-27T15:24:18.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.008+0000 7f122e59c700 1 --2- 172.21.15.52:0/4056374840 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f12180a12c0 0x7f12180a3770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1224005950 tx=0x7f122400b410 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.008+0000 7f1240e21b80 1 Processor -- start 2023-11-27T15:24:18.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.008+0000 7f1240e21b80 1 -- start start 2023-11-27T15:24:18.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.009+0000 7f1240e21b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609e0d49b60 0x5609e0d49f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.009+0000 7f1240e21b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609e0d4a470 0x5609e0d4e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.009+0000 7f1240e21b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5609e0d4e8b0 0x5609e0d4ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.009+0000 7f1240e21b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e0c4f060 con 0x5609e0d4e8b0 2023-11-27T15:24:18.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.009+0000 7f1240e21b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e0ca18a0 con 0x5609e0d49b60 2023-11-27T15:24:18.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.009+0000 7f1240e21b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e0c9ad30 con 0x5609e0d4a470 2023-11-27T15:24:18.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.009+0000 7f122ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5609e0d4e8b0 0x5609e0d4ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.009+0000 7f122ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5609e0d4e8b0 0x5609e0d4ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54374/0 (socket says 172.21.15.52:54374) 2023-11-27T15:24:18.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.009+0000 7f122ed9d700 1 -- 172.21.15.52:0/484436764 learned_addr learned my addr 172.21.15.52:0/484436764 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:18.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.009+0000 7f122dd9b700 1 --2- 172.21.15.52:0/484436764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609e0d4a470 0x5609e0d4e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.009+0000 7f122e59c700 1 --2- 172.21.15.52:0/484436764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609e0d49b60 0x5609e0d49f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.009+0000 7f122ed9d700 1 -- 172.21.15.52:0/484436764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609e0d4a470 msgr2=0x5609e0d4e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.009+0000 7f122ed9d700 1 --2- 172.21.15.52:0/484436764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609e0d4a470 0x5609e0d4e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.009+0000 7f122ed9d700 1 -- 172.21.15.52:0/484436764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609e0d49b60 msgr2=0x5609e0d49f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.009+0000 7f122ed9d700 1 --2- 172.21.15.52:0/484436764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609e0d49b60 0x5609e0d49f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.009+0000 7f122ed9d700 1 -- 172.21.15.52:0/484436764 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609e0b1c630 con 0x5609e0d4e8b0 2023-11-27T15:24:18.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.009+0000 7f122ed9d700 1 --2- 172.21.15.52:0/484436764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5609e0d4e8b0 0x5609e0d4ed00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1228009440 tx=0x7f1228009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ff19687dbf8ee2 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.009+0000 7f121cff9700 1 -- 172.21.15.52:0/484436764 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122800e070 con 0x5609e0d4e8b0 2023-11-27T15:24:18.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.009+0000 7f121cff9700 1 -- 172.21.15.52:0/484436764 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f122800f040 con 0x5609e0d4e8b0 2023-11-27T15:24:18.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.009+0000 7f121cff9700 1 -- 172.21.15.52:0/484436764 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1228015b20 con 0x5609e0d4e8b0 2023-11-27T15:24:18.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.010+0000 7f1240e21b80 1 -- 172.21.15.52:0/484436764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5609e0d4e8b0 msgr2=0x5609e0d4ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.010+0000 7f1240e21b80 1 --2- 172.21.15.52:0/484436764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5609e0d4e8b0 0x5609e0d4ed00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1228009440 tx=0x7f1228009920 comp rx=0 tx=0).stop 2023-11-27T15:24:18.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.010+0000 7f1240e21b80 1 -- 172.21.15.52:0/484436764 shutdown_connections 2023-11-27T15:24:18.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.010+0000 7f1240e21b80 1 --2- 172.21.15.52:0/484436764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609e0d49b60 0x5609e0d49f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.010+0000 7f1240e21b80 1 --2- 172.21.15.52:0/484436764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5609e0d4e8b0 0x5609e0d4ed00 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.010+0000 7f1240e21b80 1 --2- 172.21.15.52:0/484436764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609e0d4a470 0x5609e0d4e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.010+0000 7f1240e21b80 1 -- 172.21.15.52:0/484436764 >> 172.21.15.52:0/484436764 conn(0x5609e0d3f360 msgr2=0x5609e0d44b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.010+0000 7f1240e21b80 1 -- 172.21.15.52:0/484436764 shutdown_connections 2023-11-27T15:24:18.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.010+0000 7f1240e21b80 1 -- 172.21.15.52:0/484436764 wait complete. 2023-11-27T15:24:18.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.011+0000 7f1240e21b80 1 -- start start 2023-11-27T15:24:18.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.011+0000 7f1240e21b80 1 Processor -- start 2023-11-27T15:24:18.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.011+0000 7f1240e21b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5609e0d49b60 0x5609e0d55fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.011+0000 7f1240e21b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609e0d4a470 0x5609e0d53290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.011+0000 7f1240e21b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609e0d4e8b0 0x5609e0d53870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.011+0000 7f1240e21b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e0c3c8e0 con 0x5609e0d49b60 2023-11-27T15:24:18.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.011+0000 7f1240e21b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e0c3c580 con 0x5609e0d4a470 2023-11-27T15:24:18.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.011+0000 7f1240e21b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e0b1e920 con 0x5609e0d4e8b0 2023-11-27T15:24:18.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.011+0000 7f1240e21b80 10 client.0 _ll_register_callbacks cb 0x5609e0c5a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:18.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.011+0000 7f122e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5609e0d49b60 0x5609e0d55fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.011+0000 7f122e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5609e0d49b60 0x5609e0d55fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54376/0 (socket says 172.21.15.52:54376) 2023-11-27T15:24:18.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.011+0000 7f122e59c700 1 -- 172.21.15.52:0/3569688746 learned_addr learned my addr 172.21.15.52:0/3569688746 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:18.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.011+0000 7f122ed9d700 1 --2- 172.21.15.52:0/3569688746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609e0d4e8b0 0x5609e0d53870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.011+0000 7f122dd9b700 1 --2- 172.21.15.52:0/3569688746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609e0d4a470 0x5609e0d53290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.011+0000 7f122e59c700 1 -- 172.21.15.52:0/3569688746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609e0d4e8b0 msgr2=0x5609e0d53870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.011+0000 7f122e59c700 1 --2- 172.21.15.52:0/3569688746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609e0d4e8b0 0x5609e0d53870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.011+0000 7f122e59c700 1 -- 172.21.15.52:0/3569688746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609e0d4a470 msgr2=0x5609e0d53290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.011+0000 7f122e59c700 1 --2- 172.21.15.52:0/3569688746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609e0d4a470 0x5609e0d53290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.011+0000 7f122e59c700 1 -- 172.21.15.52:0/3569688746 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12280090f0 con 0x5609e0d49b60 2023-11-27T15:24:18.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.011+0000 7f122ed9d700 1 --2- 172.21.15.52:0/3569688746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609e0d4e8b0 0x5609e0d53870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:24:18.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.011+0000 7f122dd9b700 1 --2- 172.21.15.52:0/3569688746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609e0d4a470 0x5609e0d53290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:24:18.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.012+0000 7f122e59c700 1 --2- 172.21.15.52:0/3569688746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5609e0d49b60 0x5609e0d55fa0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f12240116f0 tx=0x7f12240047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b0f73dabac4413 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.012+0000 7f1215ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:24:18.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.012+0000 7f1215ffb700 1 -- 172.21.15.52:0/3569688746 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1224028670 con 0x5609e0d49b60 2023-11-27T15:24:18.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.012+0000 7f1215ffb700 1 -- 172.21.15.52:0/3569688746 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f1224017070 con 0x5609e0d49b60 2023-11-27T15:24:18.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.012+0000 7f1240e21b80 1 -- 172.21.15.52:0/3569688746 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609e0d3cb20 con 0x5609e0d49b60 2023-11-27T15:24:18.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.012+0000 7f1215ffb700 1 -- 172.21.15.52:0/3569688746 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12240287d0 con 0x5609e0d49b60 2023-11-27T15:24:18.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.012+0000 7f1240e21b80 10 client.16395 fetch_fsmap learned FSMap version 28 2023-11-27T15:24:18.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.012+0000 7f1240e21b80 1 -- 172.21.15.52:0/3569688746 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5609e0c61190 con 0x5609e0d49b60 2023-11-27T15:24:18.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.012+0000 7f1215ffb700 1 -- 172.21.15.52:0/3569688746 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5609e0d3cb20 con 0x5609e0d49b60 2023-11-27T15:24:18.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.012+0000 7f1215ffb700 1 -- 172.21.15.52:0/3569688746 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f122401fa00 con 0x5609e0d49b60 2023-11-27T15:24:18.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.012+0000 7f1240e21b80 10 client.16395 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:24:18.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.012+0000 7f1240e21b80 10 client.16395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:18.028 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.012+0000 7f1240e21b80 20 client.16395 populate_metadata read hostname 'smithi052' 2023-11-27T15:24:18.028 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.012+0000 7f1240e21b80 1 --2- 172.21.15.52:0/3569688746 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5609e0d638e0 0x5609e0d65cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.028 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.012+0000 7f1240e21b80 4 client.16395 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:24:18.028 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.012+0000 7f1240e21b80 1 -- 172.21.15.52:0/3569688746 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609e0c5d0a0 con 0x5609e0d638e0 2023-11-27T15:24:18.028 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.013+0000 7f122dd9b700 1 --2- 172.21.15.52:0/3569688746 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5609e0d638e0 0x5609e0d65cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.028 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.013+0000 7f122dd9b700 1 --2- 172.21.15.52:0/3569688746 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5609e0d638e0 0x5609e0d65cf0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3da896281792e3c1 server_cookie=7b24b1655f63fe66 in_seq=0 out_seq=0 2023-11-27T15:24:18.028 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.013+0000 7f1215ffb700 10 client.16395 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:18.028 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.014+0000 7f1215ffb700 1 -- 172.21.15.52:0/3569688746 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5609e0c5d0a0 con 0x5609e0d638e0 2023-11-27T15:24:18.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.014+0000 7f1215ffb700 10 client.16395 handle_command_reply: tid=0 2023-11-27T15:24:18.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.014+0000 7f1240e21b80 1 client.16395 shutdown 2023-11-27T15:24:18.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.014+0000 7f1240e21b80 1 -- 172.21.15.52:0/3569688746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5609e0d49b60 msgr2=0x5609e0d55fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.014+0000 7f1240e21b80 1 --2- 172.21.15.52:0/3569688746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5609e0d49b60 0x5609e0d55fa0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f12240116f0 tx=0x7f12240047c0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.014+0000 7f1240e21b80 1 -- 172.21.15.52:0/3569688746 shutdown_connections 2023-11-27T15:24:18.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.014+0000 7f1240e21b80 1 --2- 172.21.15.52:0/3569688746 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5609e0d638e0 0x5609e0d65cf0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.014+0000 7f1240e21b80 1 --2- 172.21.15.52:0/3569688746 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609e0d4e8b0 0x5609e0d53870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.014+0000 7f1240e21b80 1 --2- 172.21.15.52:0/3569688746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5609e0d49b60 0x5609e0d55fa0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.030 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.014+0000 7f1240e21b80 1 --2- 172.21.15.52:0/3569688746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609e0d4a470 0x5609e0d53290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.030 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.014+0000 7f1215ffb700 0 client.16395 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:18.030 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.014+0000 7f1240e21b80 1 -- 172.21.15.52:0/3569688746 >> 172.21.15.52:0/3569688746 conn(0x5609e0d3f360 msgr2=0x5609e0d4fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.030 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.014+0000 7f1240e21b80 1 -- 172.21.15.52:0/3569688746 shutdown_connections 2023-11-27T15:24:18.030 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.014+0000 7f1240e21b80 1 -- 172.21.15.52:0/3569688746 wait complete. 2023-11-27T15:24:18.030 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.014+0000 7f1240e21b80 20 client.16395 trim_cache size 0 max 16384 2023-11-27T15:24:18.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.032+0000 7f1240e21b80 1 Processor -- start 2023-11-27T15:24:18.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.032+0000 7f1240e21b80 1 -- start start 2023-11-27T15:24:18.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.032+0000 7f1240e21b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5609e0d49b60 0x5609e0d5a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.032+0000 7f1240e21b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609e0d4a470 0x5609e0d55b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.032+0000 7f1240e21b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609e0d4e8b0 0x5609e0d44560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.032+0000 7f1240e21b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e0d4d400 con 0x5609e0d49b60 2023-11-27T15:24:18.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.032+0000 7f1240e21b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e0d4a900 con 0x5609e0d4e8b0 2023-11-27T15:24:18.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.032+0000 7f1240e21b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e0d3c960 con 0x5609e0d4a470 2023-11-27T15:24:18.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.032+0000 7f122e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5609e0d49b60 0x5609e0d5a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.032+0000 7f122e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5609e0d49b60 0x5609e0d5a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54384/0 (socket says 172.21.15.52:54384) 2023-11-27T15:24:18.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.032+0000 7f122e59c700 1 -- 172.21.15.52:0/3272082607 learned_addr learned my addr 172.21.15.52:0/3272082607 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:18.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.032+0000 7f122dd9b700 1 --2- 172.21.15.52:0/3272082607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609e0d4a470 0x5609e0d55b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.032+0000 7f122ed9d700 1 --2- 172.21.15.52:0/3272082607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609e0d4e8b0 0x5609e0d44560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.032+0000 7f122e59c700 1 -- 172.21.15.52:0/3272082607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609e0d4a470 msgr2=0x5609e0d55b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.032+0000 7f122e59c700 1 --2- 172.21.15.52:0/3272082607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609e0d4a470 0x5609e0d55b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.032+0000 7f122e59c700 1 -- 172.21.15.52:0/3272082607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609e0d4e8b0 msgr2=0x5609e0d44560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.032+0000 7f122e59c700 1 --2- 172.21.15.52:0/3272082607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609e0d4e8b0 0x5609e0d44560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.032+0000 7f122e59c700 1 -- 172.21.15.52:0/3272082607 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609e0d46490 con 0x5609e0d49b60 2023-11-27T15:24:18.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.032+0000 7f122ed9d700 1 --2- 172.21.15.52:0/3272082607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609e0d4e8b0 0x5609e0d44560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:24:18.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.033+0000 7f122e59c700 1 --2- 172.21.15.52:0/3272082607 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5609e0d49b60 0x5609e0d5a9a0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f122400ba80 tx=0x7f12240046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1f9d0b4e81aceb server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.033+0000 7f121d7fa700 1 -- 172.21.15.52:0/3272082607 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1224011b60 con 0x5609e0d49b60 2023-11-27T15:24:18.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.033+0000 7f121d7fa700 1 -- 172.21.15.52:0/3272082607 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1224015070 con 0x5609e0d49b60 2023-11-27T15:24:18.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.033+0000 7f121d7fa700 1 -- 172.21.15.52:0/3272082607 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122401ec90 con 0x5609e0d49b60 2023-11-27T15:24:18.044 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.033+0000 7f1240e21b80 1 -- 172.21.15.52:0/3272082607 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5609e0d49b60 msgr2=0x5609e0d5a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.044 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.033+0000 7f1240e21b80 1 --2- 172.21.15.52:0/3272082607 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5609e0d49b60 0x5609e0d5a9a0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f122400ba80 tx=0x7f12240046d0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.044 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.033+0000 7f1240e21b80 1 -- 172.21.15.52:0/3272082607 shutdown_connections 2023-11-27T15:24:18.044 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.033+0000 7f1240e21b80 1 --2- 172.21.15.52:0/3272082607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609e0d4e8b0 0x5609e0d44560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.044 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.033+0000 7f1240e21b80 1 --2- 172.21.15.52:0/3272082607 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5609e0d49b60 0x5609e0d5a9a0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.044 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.033+0000 7f1240e21b80 1 --2- 172.21.15.52:0/3272082607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609e0d4a470 0x5609e0d55b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.044 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.033+0000 7f1240e21b80 1 -- 172.21.15.52:0/3272082607 >> 172.21.15.52:0/3272082607 conn(0x5609e0d3f360 msgr2=0x5609e0d4c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.044 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.033+0000 7f1240e21b80 1 -- 172.21.15.52:0/3272082607 shutdown_connections 2023-11-27T15:24:18.045 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.034+0000 7f1240e21b80 1 -- 172.21.15.52:0/3272082607 wait complete. 2023-11-27T15:24:18.045 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.034+0000 7f1240e21b80 1 -- start start 2023-11-27T15:24:18.045 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.034+0000 7f1240e21b80 1 Processor -- start 2023-11-27T15:24:18.045 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.034+0000 7f1240e21b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609e0d49b60 0x5609e0d5a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.045 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.034+0000 7f1240e21b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5609e0d4a470 0x5609e0d5abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.045 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.034+0000 7f1240e21b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609e0d4e8b0 0x5609e0d58b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.045 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.034+0000 7f1240e21b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e0d59260 con 0x5609e0d4a470 2023-11-27T15:24:18.046 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.034+0000 7f1240e21b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e0d593d0 con 0x5609e0d4e8b0 2023-11-27T15:24:18.046 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.034+0000 7f1240e21b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e0d59590 con 0x5609e0d49b60 2023-11-27T15:24:18.046 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.034+0000 7f1240e21b80 10 client.0 _ll_register_callbacks cb 0x7f1224010dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:18.046 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.034+0000 7f122dd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5609e0d4a470 0x5609e0d5abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.046 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.034+0000 7f122dd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5609e0d4a470 0x5609e0d5abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54392/0 (socket says 172.21.15.52:54392) 2023-11-27T15:24:18.046 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.034+0000 7f122dd9b700 1 -- 172.21.15.52:0/2050731068 learned_addr learned my addr 172.21.15.52:0/2050731068 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:18.046 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.034+0000 7f122ed9d700 1 --2- 172.21.15.52:0/2050731068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609e0d4e8b0 0x5609e0d58b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.046 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.034+0000 7f122e59c700 1 --2- 172.21.15.52:0/2050731068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609e0d49b60 0x5609e0d5a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.047 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.034+0000 7f122dd9b700 1 -- 172.21.15.52:0/2050731068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609e0d49b60 msgr2=0x5609e0d5a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.047 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.034+0000 7f122dd9b700 1 --2- 172.21.15.52:0/2050731068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609e0d49b60 0x5609e0d5a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.047 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.034+0000 7f122dd9b700 1 -- 172.21.15.52:0/2050731068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609e0d4e8b0 msgr2=0x5609e0d58b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.047 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.035+0000 7f122dd9b700 1 --2- 172.21.15.52:0/2050731068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609e0d4e8b0 0x5609e0d58b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.047 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.035+0000 7f122dd9b700 1 -- 172.21.15.52:0/2050731068 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609e0c61190 con 0x5609e0d4a470 2023-11-27T15:24:18.047 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.035+0000 7f122dd9b700 1 --2- 172.21.15.52:0/2050731068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5609e0d4a470 0x5609e0d5abb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f1220097410 tx=0x7f1220009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4c5b91d6f088a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.047 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.035+0000 7f1215ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:24:18.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.035+0000 7f1215ffb700 1 -- 172.21.15.52:0/2050731068 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12200a2de0 con 0x5609e0d4a470 2023-11-27T15:24:18.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.035+0000 7f1215ffb700 1 -- 172.21.15.52:0/2050731068 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f12200b8aa0 con 0x5609e0d4a470 2023-11-27T15:24:18.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.035+0000 7f1240e21b80 1 -- 172.21.15.52:0/2050731068 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f122800f040 con 0x5609e0d4a470 2023-11-27T15:24:18.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.035+0000 7f1215ffb700 1 -- 172.21.15.52:0/2050731068 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12200b8ca0 con 0x5609e0d4a470 2023-11-27T15:24:18.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.035+0000 7f1240e21b80 10 client.16401 fetch_fsmap learned FSMap version 28 2023-11-27T15:24:18.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.035+0000 7f1240e21b80 1 -- 172.21.15.52:0/2050731068 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5609e0ac8de0 con 0x5609e0d4a470 2023-11-27T15:24:18.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.035+0000 7f1215ffb700 1 -- 172.21.15.52:0/2050731068 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1220064080 con 0x5609e0d4a470 2023-11-27T15:24:18.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.035+0000 7f1215ffb700 1 -- 172.21.15.52:0/2050731068 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1220099e90 con 0x5609e0d4a470 2023-11-27T15:24:18.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.035+0000 7f1240e21b80 10 client.16401 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:24:18.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.035+0000 7f1240e21b80 10 client.16401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:18.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.035+0000 7f1240e21b80 20 client.16401 populate_metadata read hostname 'smithi052' 2023-11-27T15:24:18.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.035+0000 7f1240e21b80 1 --2- 172.21.15.52:0/2050731068 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5609e0dbca10 0x5609e0dbee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.035+0000 7f1240e21b80 4 client.16401 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:24:18.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.035+0000 7f1240e21b80 1 -- 172.21.15.52:0/2050731068 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5609e0c69660 con 0x5609e0dbca10 2023-11-27T15:24:18.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.036+0000 7f122e59c700 1 --2- 172.21.15.52:0/2050731068 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5609e0dbca10 0x5609e0dbee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.036+0000 7f1215ffb700 10 client.16401 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:18.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.036+0000 7f122e59c700 1 --2- 172.21.15.52:0/2050731068 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5609e0dbca10 0x5609e0dbee20 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c880e223e8fd2890 server_cookie=b7b07af0e7efa646 in_seq=0 out_seq=0 2023-11-27T15:24:18.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.036+0000 7f1215ffb700 1 -- 172.21.15.52:0/2050731068 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5609e0c69660 con 0x5609e0dbca10 2023-11-27T15:24:18.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.036+0000 7f1215ffb700 10 client.16401 handle_command_reply: tid=0 2023-11-27T15:24:18.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.036+0000 7f1240e21b80 1 client.16401 shutdown 2023-11-27T15:24:18.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.036+0000 7f1240e21b80 1 -- 172.21.15.52:0/2050731068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5609e0d4a470 msgr2=0x5609e0d5abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.036+0000 7f1240e21b80 1 --2- 172.21.15.52:0/2050731068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5609e0d4a470 0x5609e0d5abb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f1220097410 tx=0x7f1220009310 comp rx=0 tx=0).stop 2023-11-27T15:24:18.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.036+0000 7f1240e21b80 1 -- 172.21.15.52:0/2050731068 shutdown_connections 2023-11-27T15:24:18.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.036+0000 7f1240e21b80 1 --2- 172.21.15.52:0/2050731068 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5609e0dbca10 0x5609e0dbee20 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.036+0000 7f1240e21b80 1 --2- 172.21.15.52:0/2050731068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609e0d4e8b0 0x5609e0d58b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.036+0000 7f1240e21b80 1 --2- 172.21.15.52:0/2050731068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5609e0d49b60 0x5609e0d5a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.036+0000 7f1240e21b80 1 --2- 172.21.15.52:0/2050731068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5609e0d4a470 0x5609e0d5abb0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.036+0000 7f1215ffb700 0 client.16401 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:18.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.036+0000 7f1240e21b80 1 -- 172.21.15.52:0/2050731068 >> 172.21.15.52:0/2050731068 conn(0x5609e0d3f360 msgr2=0x5609e0d4d810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.037+0000 7f1240e21b80 1 -- 172.21.15.52:0/2050731068 shutdown_connections 2023-11-27T15:24:18.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.037+0000 7f1240e21b80 1 -- 172.21.15.52:0/2050731068 wait complete. 2023-11-27T15:24:18.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.037+0000 7f1240e21b80 20 client.16401 trim_cache size 0 max 16384 2023-11-27T15:24:18.052 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:24:18.052 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2023-11-27T15:24:18.052 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2023-11-27T15:24:18.052 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:24:18.053 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.037+0000 7f1234c95700 1 -- 172.21.15.52:0/4056374840 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f12180a12c0 msgr2=0x7f12180a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.054 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.037+0000 7f1234c95700 1 --2- 172.21.15.52:0/4056374840 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f12180a12c0 0x7f12180a3770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1224005950 tx=0x7f122400b410 comp rx=0 tx=0).stop 2023-11-27T15:24:18.054 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.037+0000 7f1234c95700 1 -- 172.21.15.52:0/4056374840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12300ffec0 msgr2=0x7f1230197780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.054 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.037+0000 7f1234c95700 1 --2- 172.21.15.52:0/4056374840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12300ffec0 0x7f1230197780 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1220009fd0 tx=0x7f1220009580 comp rx=0 tx=0).stop 2023-11-27T15:24:18.054 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.037+0000 7f1234c95700 1 -- 172.21.15.52:0/4056374840 shutdown_connections 2023-11-27T15:24:18.054 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.037+0000 7f1234c95700 1 --2- 172.21.15.52:0/4056374840 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f12180a12c0 0x7f12180a3770 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.054 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.037+0000 7f1234c95700 1 --2- 172.21.15.52:0/4056374840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1230104560 0x7f12301a22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.037+0000 7f1234c95700 1 --2- 172.21.15.52:0/4056374840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12300ff520 0x7f1230197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.037+0000 7f1234c95700 1 --2- 172.21.15.52:0/4056374840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12300ffec0 0x7f1230197780 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.037+0000 7f1234c95700 1 -- 172.21.15.52:0/4056374840 >> 172.21.15.52:0/4056374840 conn(0x7f12300760d0 msgr2=0x7f12300fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.037+0000 7f1234c95700 1 -- 172.21.15.52:0/4056374840 shutdown_connections 2023-11-27T15:24:18.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.037+0000 7f1234c95700 1 -- 172.21.15.52:0/4056374840 wait complete. 2023-11-27T15:24:18.055 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-27T15:24:18.055 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:24:18.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.177+0000 7fe1c7361700 1 -- 172.21.15.52:0/2923443474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c00ffdd0 msgr2=0x7fe1c0103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.177+0000 7fe1c7361700 1 --2- 172.21.15.52:0/2923443474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c00ffdd0 0x7fe1c0103d90 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40099c0 tx=0x7fe1b4009cd0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.177+0000 7fe1c7361700 1 -- 172.21.15.52:0/2923443474 shutdown_connections 2023-11-27T15:24:18.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.177+0000 7fe1c7361700 1 --2- 172.21.15.52:0/2923443474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1c0104470 0x7fe1c010c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.177+0000 7fe1c7361700 1 --2- 172.21.15.52:0/2923443474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1c00ff430 0x7fe1c00ff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.177+0000 7fe1c7361700 1 --2- 172.21.15.52:0/2923443474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c00ffdd0 0x7fe1c0103d90 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.177+0000 7fe1c7361700 1 -- 172.21.15.52:0/2923443474 >> 172.21.15.52:0/2923443474 conn(0x7fe1c0076340 msgr2=0x7fe1c0076740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.177+0000 7fe1c7361700 1 -- 172.21.15.52:0/2923443474 shutdown_connections 2023-11-27T15:24:18.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.178+0000 7fe1c7361700 1 -- 172.21.15.52:0/2923443474 wait complete. 2023-11-27T15:24:18.184 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.178+0000 7fe1c7361700 1 Processor -- start 2023-11-27T15:24:18.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.178+0000 7fe1c7361700 1 -- start start 2023-11-27T15:24:18.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.178+0000 7fe1c7361700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1c00ff430 0x7fe1c0194fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.178+0000 7fe1c7361700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c00ffdd0 0x7fe1c0195500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.178+0000 7fe1c7361700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1c0104470 0x7fe1c0195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.178+0000 7fe1c7361700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c01a4800 con 0x7fe1c00ffdd0 2023-11-27T15:24:18.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.178+0000 7fe1c7361700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c01a4970 con 0x7fe1c0104470 2023-11-27T15:24:18.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.178+0000 7fe1c7361700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c01a4b30 con 0x7fe1c00ff430 2023-11-27T15:24:18.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.178+0000 7fe1c48fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c00ffdd0 0x7fe1c0195500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.179+0000 7fe1c48fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c00ffdd0 0x7fe1c0195500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54404/0 (socket says 172.21.15.52:54404) 2023-11-27T15:24:18.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.179+0000 7fe1c48fc700 1 -- 172.21.15.52:0/3343858193 learned_addr learned my addr 172.21.15.52:0/3343858193 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:18.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.179+0000 7fe1c58fe700 1 --2- 172.21.15.52:0/3343858193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1c0104470 0x7fe1c0195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.179+0000 7fe1c50fd700 1 --2- 172.21.15.52:0/3343858193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1c00ff430 0x7fe1c0194fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.179+0000 7fe1c48fc700 1 -- 172.21.15.52:0/3343858193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1c00ff430 msgr2=0x7fe1c0194fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.179+0000 7fe1c48fc700 1 --2- 172.21.15.52:0/3343858193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1c00ff430 0x7fe1c0194fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.179+0000 7fe1c48fc700 1 -- 172.21.15.52:0/3343858193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1c0104470 msgr2=0x7fe1c0195a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.179+0000 7fe1c48fc700 1 --2- 172.21.15.52:0/3343858193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1c0104470 0x7fe1c0195a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.179+0000 7fe1c48fc700 1 -- 172.21.15.52:0/3343858193 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b40095f0 con 0x7fe1c00ffdd0 2023-11-27T15:24:18.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.179+0000 7fe1c48fc700 1 --2- 172.21.15.52:0/3343858193 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c00ffdd0 0x7fe1c0195500 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40098e0 tx=0x7fe1b400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.179+0000 7fe1b27fc700 1 -- 172.21.15.52:0/3343858193 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b401d070 con 0x7fe1c00ffdd0 2023-11-27T15:24:18.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.179+0000 7fe1b27fc700 1 -- 172.21.15.52:0/3343858193 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1b400fb80 con 0x7fe1c00ffdd0 2023-11-27T15:24:18.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.179+0000 7fe1c7361700 1 -- 172.21.15.52:0/3343858193 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c010f210 con 0x7fe1c00ffdd0 2023-11-27T15:24:18.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.179+0000 7fe1b27fc700 1 -- 172.21.15.52:0/3343858193 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b40173f0 con 0x7fe1c00ffdd0 2023-11-27T15:24:18.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.179+0000 7fe1c7361700 1 -- 172.21.15.52:0/3343858193 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c010f080 con 0x7fe1c00ffdd0 2023-11-27T15:24:18.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.181+0000 7fe1c7361700 1 -- 172.21.15.52:0/3343858193 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c010f3a0 con 0x7fe1c00ffdd0 2023-11-27T15:24:18.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.181+0000 7fe1b27fc700 1 -- 172.21.15.52:0/3343858193 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe1b4022470 con 0x7fe1c00ffdd0 2023-11-27T15:24:18.189 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.182+0000 7fe1b27fc700 1 --2- 172.21.15.52:0/3343858193 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe1ac0a12c0 0x7fe1ac0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.189 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.182+0000 7fe1b27fc700 1 -- 172.21.15.52:0/3343858193 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fe1b40cda40 con 0x7fe1c00ffdd0 2023-11-27T15:24:18.189 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.182+0000 7fe1c50fd700 1 --2- 172.21.15.52:0/3343858193 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe1ac0a12c0 0x7fe1ac0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.189 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.182+0000 7fe1c50fd700 1 --2- 172.21.15.52:0/3343858193 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe1ac0a12c0 0x7fe1ac0a3770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc009bf0 tx=0x7fe1bc009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.190 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.186+0000 7fe1b27fc700 1 -- 172.21.15.52:0/3343858193 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe1b40cfe60 con 0x7fe1c00ffdd0 2023-11-27T15:24:18.433 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.429+0000 7fe1c7361700 1 -- 172.21.15.52:0/3343858193 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1c00671d0 con 0x7fe1c00ffdd0 2023-11-27T15:24:18.433 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.430+0000 7fe1b27fc700 1 -- 172.21.15.52:0/3343858193 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10813 (secure 0 0 0) 0x7fe1b4096eb0 con 0x7fe1c00ffdd0 2023-11-27T15:24:18.434 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:24:18.434 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2328881376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2328881376},{"type":"v1","addr":"172.21.15.52:6839","nonce":2328881376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2799551570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2799551570},{"type":"v1","addr":"172.21.15.52:6841","nonce":2799551570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:16:09.319630+0000","modified":"2023-11-27T15:21:19.128122+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24494,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/215725838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":215725838},{"type":"v1","addr":"172.21.15.150:6839","nonce":215725838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3571830559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3571830559},{"type":"v1","addr":"172.21.15.102:6839","nonce":3571830559}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:24:18.436 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.432+0000 7fe1c7361700 1 -- 172.21.15.52:0/3343858193 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe1ac0a12c0 msgr2=0x7fe1ac0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.436 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.432+0000 7fe1c7361700 1 --2- 172.21.15.52:0/3343858193 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe1ac0a12c0 0x7fe1ac0a3770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc009bf0 tx=0x7fe1bc009260 comp rx=0 tx=0).stop 2023-11-27T15:24:18.436 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.432+0000 7fe1c7361700 1 -- 172.21.15.52:0/3343858193 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c00ffdd0 msgr2=0x7fe1c0195500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.436 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.432+0000 7fe1c7361700 1 --2- 172.21.15.52:0/3343858193 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c00ffdd0 0x7fe1c0195500 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40098e0 tx=0x7fe1b400f690 comp rx=0 tx=0).stop 2023-11-27T15:24:18.437 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.432+0000 7fe1c7361700 1 -- 172.21.15.52:0/3343858193 shutdown_connections 2023-11-27T15:24:18.437 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.432+0000 7fe1c7361700 1 --2- 172.21.15.52:0/3343858193 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe1ac0a12c0 0x7fe1ac0a3770 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.437 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.432+0000 7fe1c7361700 1 --2- 172.21.15.52:0/3343858193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1c0104470 0x7fe1c0195a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.437 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.432+0000 7fe1c7361700 1 --2- 172.21.15.52:0/3343858193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1c00ff430 0x7fe1c0194fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.437 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.432+0000 7fe1c7361700 1 --2- 172.21.15.52:0/3343858193 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c00ffdd0 0x7fe1c0195500 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.437 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.432+0000 7fe1c7361700 1 -- 172.21.15.52:0/3343858193 >> 172.21.15.52:0/3343858193 conn(0x7fe1c0076340 msgr2=0x7fe1c00fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.437 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.432+0000 7fe1c7361700 1 -- 172.21.15.52:0/3343858193 shutdown_connections 2023-11-27T15:24:18.438 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.432+0000 7fe1c7361700 1 -- 172.21.15.52:0/3343858193 wait complete. 2023-11-27T15:24:18.438 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 28 2023-11-27T15:24:18.447 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:18 smithi052 ceph-mon[99056]: pgmap v196: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 383 B/s wr, 8 op/s 2023-11-27T15:24:18.448 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-27T15:24:18.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:18 smithi150 ceph-mon[100559]: pgmap v196: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 383 B/s wr, 8 op/s 2023-11-27T15:24:18.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.572+0000 7fd16bd0c700 1 -- 172.21.15.52:0/618686912 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd164100f50 msgr2=0x7fd164104f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.572+0000 7fd16bd0c700 1 --2- 172.21.15.52:0/618686912 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd164100f50 0x7fd164104f10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd1600099c0 tx=0x7fd160009cd0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.572+0000 7fd16bd0c700 1 -- 172.21.15.52:0/618686912 shutdown_connections 2023-11-27T15:24:18.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.572+0000 7fd16bd0c700 1 --2- 172.21.15.52:0/618686912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1641055f0 0x7fd16410d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.572+0000 7fd16bd0c700 1 --2- 172.21.15.52:0/618686912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1641005b0 0x7fd164100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.572+0000 7fd16bd0c700 1 --2- 172.21.15.52:0/618686912 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd164100f50 0x7fd164104f10 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.572+0000 7fd16bd0c700 1 -- 172.21.15.52:0/618686912 >> 172.21.15.52:0/618686912 conn(0x7fd164077170 msgr2=0x7fd164077570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.572+0000 7fd16bd0c700 1 -- 172.21.15.52:0/618686912 shutdown_connections 2023-11-27T15:24:18.578 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.573+0000 7fd16bd0c700 1 -- 172.21.15.52:0/618686912 wait complete. 2023-11-27T15:24:18.578 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.573+0000 7fd16bd0c700 1 Processor -- start 2023-11-27T15:24:18.578 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.573+0000 7fd16bd0c700 1 -- start start 2023-11-27T15:24:18.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:18 smithi102 ceph-mon[100296]: pgmap v196: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 383 B/s wr, 8 op/s 2023-11-27T15:24:18.580 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.573+0000 7fd16bd0c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1641005b0 0x7fd1641964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.580 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.573+0000 7fd16bd0c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd164100f50 0x7fd164196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.580 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.573+0000 7fd16bd0c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1641055f0 0x7fd16419ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.580 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.573+0000 7fd16bd0c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16419b4d0 con 0x7fd1641055f0 2023-11-27T15:24:18.580 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.573+0000 7fd16bd0c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16419b640 con 0x7fd164100f50 2023-11-27T15:24:18.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.573+0000 7fd16bd0c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16419b800 con 0x7fd1641005b0 2023-11-27T15:24:18.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.573+0000 7fd16a2a9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1641055f0 0x7fd16419ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.574+0000 7fd1692a7700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd164100f50 0x7fd164196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.574+0000 7fd16a2a9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1641055f0 0x7fd16419ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54420/0 (socket says 172.21.15.52:54420) 2023-11-27T15:24:18.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.574+0000 7fd16a2a9700 1 -- 172.21.15.52:0/1573466483 learned_addr learned my addr 172.21.15.52:0/1573466483 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:18.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.574+0000 7fd169aa8700 1 --2- 172.21.15.52:0/1573466483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1641005b0 0x7fd1641964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.574+0000 7fd16a2a9700 1 -- 172.21.15.52:0/1573466483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1641005b0 msgr2=0x7fd1641964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.574+0000 7fd16a2a9700 1 --2- 172.21.15.52:0/1573466483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1641005b0 0x7fd1641964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.574+0000 7fd16a2a9700 1 -- 172.21.15.52:0/1573466483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd164100f50 msgr2=0x7fd164196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.574+0000 7fd16a2a9700 1 --2- 172.21.15.52:0/1573466483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd164100f50 0x7fd164196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.574+0000 7fd16a2a9700 1 -- 172.21.15.52:0/1573466483 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1600095f0 con 0x7fd1641055f0 2023-11-27T15:24:18.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.574+0000 7fd16a2a9700 1 --2- 172.21.15.52:0/1573466483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1641055f0 0x7fd16419ae90 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00ef30 tx=0x7fd15c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.574+0000 7fd15affd700 1 -- 172.21.15.52:0/1573466483 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c018070 con 0x7fd1641055f0 2023-11-27T15:24:18.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.574+0000 7fd15affd700 1 -- 172.21.15.52:0/1573466483 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd15c00f040 con 0x7fd1641055f0 2023-11-27T15:24:18.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.574+0000 7fd15affd700 1 -- 172.21.15.52:0/1573466483 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c00cdb0 con 0x7fd1641055f0 2023-11-27T15:24:18.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.574+0000 7fd16bd0c700 1 -- 172.21.15.52:0/1573466483 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd164110370 con 0x7fd1641055f0 2023-11-27T15:24:18.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.574+0000 7fd16bd0c700 1 -- 172.21.15.52:0/1573466483 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1641101e0 con 0x7fd1641055f0 2023-11-27T15:24:18.588 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.577+0000 7fd15affd700 1 -- 172.21.15.52:0/1573466483 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fd15c003a00 con 0x7fd1641055f0 2023-11-27T15:24:18.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.578+0000 7fd15affd700 1 --2- 172.21.15.52:0/1573466483 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd1500a1340 0x7fd1500a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.578+0000 7fd15affd700 1 -- 172.21.15.52:0/1573466483 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fd15c0cc8e0 con 0x7fd1641055f0 2023-11-27T15:24:18.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.578+0000 7fd169aa8700 1 --2- 172.21.15.52:0/1573466483 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd1500a1340 0x7fd1500a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.578+0000 7fd169aa8700 1 --2- 172.21.15.52:0/1573466483 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd1500a1340 0x7fd1500a37f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd154005fd0 tx=0x7fd15400a560 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.578+0000 7fd177e98b80 1 Processor -- start 2023-11-27T15:24:18.589 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.579+0000 7fd177e98b80 1 -- start start 2023-11-27T15:24:18.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.579+0000 7fd177e98b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1bf194b60 0x55b1bf194f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.579+0000 7fd177e98b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1bf195470 0x55b1bf199370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.579+0000 7fd177e98b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1bf1998b0 0x55b1bf199d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.579+0000 7fd177e98b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1bf09a060 con 0x55b1bf194b60 2023-11-27T15:24:18.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.579+0000 7fd177e98b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1bf0ec8a0 con 0x55b1bf195470 2023-11-27T15:24:18.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.579+0000 7fd177e98b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1bf0e5d30 con 0x55b1bf1998b0 2023-11-27T15:24:18.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.579+0000 7fd169aa8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1bf194b60 0x55b1bf194f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.590 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.579+0000 7fd169aa8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1bf194b60 0x55b1bf194f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54424/0 (socket says 172.21.15.52:54424) 2023-11-27T15:24:18.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.579+0000 7fd169aa8700 1 -- 172.21.15.52:0/2188355335 learned_addr learned my addr 172.21.15.52:0/2188355335 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:18.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.579+0000 7fd16a2a9700 1 --2- 172.21.15.52:0/2188355335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1bf1998b0 0x55b1bf199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.579+0000 7fd1692a7700 1 --2- 172.21.15.52:0/2188355335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1bf195470 0x55b1bf199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.579+0000 7fd169aa8700 1 -- 172.21.15.52:0/2188355335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1bf1998b0 msgr2=0x55b1bf199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.579+0000 7fd169aa8700 1 --2- 172.21.15.52:0/2188355335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1bf1998b0 0x55b1bf199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.579+0000 7fd169aa8700 1 -- 172.21.15.52:0/2188355335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1bf195470 msgr2=0x55b1bf199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.579+0000 7fd169aa8700 1 --2- 172.21.15.52:0/2188355335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1bf195470 0x55b1bf199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.591 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.579+0000 7fd169aa8700 1 -- 172.21.15.52:0/2188355335 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1bef67630 con 0x55b1bf194b60 2023-11-27T15:24:18.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.579+0000 7fd169aa8700 1 --2- 172.21.15.52:0/2188355335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1bf194b60 0x55b1bf194f30 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd15400ec80 tx=0x7fd154011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239a7fdd5a17b4b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.580+0000 7fd14ffff700 1 -- 172.21.15.52:0/2188355335 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd154010cd0 con 0x55b1bf194b60 2023-11-27T15:24:18.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.580+0000 7fd14ffff700 1 -- 172.21.15.52:0/2188355335 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd154010e30 con 0x55b1bf194b60 2023-11-27T15:24:18.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.580+0000 7fd14ffff700 1 -- 172.21.15.52:0/2188355335 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15400f780 con 0x55b1bf194b60 2023-11-27T15:24:18.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.580+0000 7fd177e98b80 1 -- 172.21.15.52:0/2188355335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1bf194b60 msgr2=0x55b1bf194f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.580+0000 7fd177e98b80 1 --2- 172.21.15.52:0/2188355335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1bf194b60 0x55b1bf194f30 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd15400ec80 tx=0x7fd154011000 comp rx=0 tx=0).stop 2023-11-27T15:24:18.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.580+0000 7fd177e98b80 1 -- 172.21.15.52:0/2188355335 shutdown_connections 2023-11-27T15:24:18.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.580+0000 7fd177e98b80 1 --2- 172.21.15.52:0/2188355335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1bf1998b0 0x55b1bf199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.580+0000 7fd177e98b80 1 --2- 172.21.15.52:0/2188355335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1bf194b60 0x55b1bf194f30 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.580+0000 7fd177e98b80 1 --2- 172.21.15.52:0/2188355335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1bf195470 0x55b1bf199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.580+0000 7fd177e98b80 1 -- 172.21.15.52:0/2188355335 >> 172.21.15.52:0/2188355335 conn(0x55b1bf18a360 msgr2=0x55b1bf18fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.580+0000 7fd177e98b80 1 -- 172.21.15.52:0/2188355335 shutdown_connections 2023-11-27T15:24:18.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.580+0000 7fd177e98b80 1 -- 172.21.15.52:0/2188355335 wait complete. 2023-11-27T15:24:18.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.581+0000 7fd177e98b80 1 -- start start 2023-11-27T15:24:18.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.581+0000 7fd177e98b80 1 Processor -- start 2023-11-27T15:24:18.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.581+0000 7fd177e98b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1bf194b60 0x55b1bf1a1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.581+0000 7fd177e98b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1bf195470 0x55b1bf19e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.581+0000 7fd177e98b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1bf1998b0 0x55b1bf19e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.581+0000 7fd177e98b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1bf0878e0 con 0x55b1bf195470 2023-11-27T15:24:18.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.581+0000 7fd177e98b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1bf087580 con 0x55b1bf1998b0 2023-11-27T15:24:18.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.581+0000 7fd177e98b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1bef69920 con 0x55b1bf194b60 2023-11-27T15:24:18.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.581+0000 7fd177e98b80 10 client.0 _ll_register_callbacks cb 0x55b1bf0a5780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:18.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.581+0000 7fd1692a7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1bf195470 0x55b1bf19e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.581+0000 7fd1692a7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1bf195470 0x55b1bf19e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54432/0 (socket says 172.21.15.52:54432) 2023-11-27T15:24:18.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.581+0000 7fd1692a7700 1 -- 172.21.15.52:0/3678808946 learned_addr learned my addr 172.21.15.52:0/3678808946 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:18.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.582+0000 7fd16a2a9700 1 --2- 172.21.15.52:0/3678808946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1bf1998b0 0x55b1bf19e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.582+0000 7fd169aa8700 1 --2- 172.21.15.52:0/3678808946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1bf194b60 0x55b1bf1a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.582+0000 7fd1692a7700 1 -- 172.21.15.52:0/3678808946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1bf194b60 msgr2=0x55b1bf1a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.582+0000 7fd1692a7700 1 --2- 172.21.15.52:0/3678808946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1bf194b60 0x55b1bf1a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.582+0000 7fd1692a7700 1 -- 172.21.15.52:0/3678808946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1bf1998b0 msgr2=0x55b1bf19e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.582+0000 7fd1692a7700 1 --2- 172.21.15.52:0/3678808946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1bf1998b0 0x55b1bf19e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.582+0000 7fd1692a7700 1 -- 172.21.15.52:0/3678808946 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd15400e930 con 0x55b1bf195470 2023-11-27T15:24:18.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.582+0000 7fd1692a7700 1 --2- 172.21.15.52:0/3678808946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1bf195470 0x55b1bf19e330 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd1600036c0 tx=0x7fd160003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a9722e2068a1841 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.582+0000 7fd14d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:24:18.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.582+0000 7fd14d7fa700 1 -- 172.21.15.52:0/3678808946 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd160005580 con 0x55b1bf195470 2023-11-27T15:24:18.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.582+0000 7fd14d7fa700 1 -- 172.21.15.52:0/3678808946 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fd16000b790 con 0x55b1bf195470 2023-11-27T15:24:18.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.582+0000 7fd177e98b80 1 -- 172.21.15.52:0/3678808946 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b1bf187b20 con 0x55b1bf195470 2023-11-27T15:24:18.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.582+0000 7fd14d7fa700 1 -- 172.21.15.52:0/3678808946 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1600056e0 con 0x55b1bf195470 2023-11-27T15:24:18.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.582+0000 7fd177e98b80 10 client.16419 fetch_fsmap learned FSMap version 28 2023-11-27T15:24:18.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.582+0000 7fd177e98b80 1 -- 172.21.15.52:0/3678808946 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b1bf0ac190 con 0x55b1bf195470 2023-11-27T15:24:18.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.582+0000 7fd14d7fa700 1 -- 172.21.15.52:0/3678808946 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd160030020 con 0x55b1bf195470 2023-11-27T15:24:18.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.583+0000 7fd14d7fa700 1 -- 172.21.15.52:0/3678808946 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd16000faa0 con 0x55b1bf195470 2023-11-27T15:24:18.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.583+0000 7fd177e98b80 10 client.16419 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:24:18.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.583+0000 7fd177e98b80 10 client.16419 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:24:18.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.583+0000 7fd177e98b80 20 client.16419 populate_metadata read hostname 'smithi052' 2023-11-27T15:24:18.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.583+0000 7fd177e98b80 1 --2- 172.21.15.52:0/3678808946 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55b1bf1ae950 0x55b1bf1b0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.583+0000 7fd177e98b80 4 client.16419 mds_command: new command op to 24472 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:24:18.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.583+0000 7fd177e98b80 1 -- 172.21.15.52:0/3678808946 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b1bf0a80a0 con 0x55b1bf1ae950 2023-11-27T15:24:18.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.583+0000 7fd169aa8700 1 --2- 172.21.15.52:0/3678808946 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55b1bf1ae950 0x55b1bf1b0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.584+0000 7fd14d7fa700 10 client.16419 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:24:18.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.584+0000 7fd169aa8700 1 --2- 172.21.15.52:0/3678808946 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55b1bf1ae950 0x55b1bf1b0d60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7e788207b9617849 server_cookie=b8627c0cb1f54d15 in_seq=0 out_seq=0 2023-11-27T15:24:18.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.584+0000 7fd14d7fa700 1 -- 172.21.15.52:0/3678808946 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b1bf0a80a0 con 0x55b1bf1ae950 2023-11-27T15:24:18.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.584+0000 7fd14d7fa700 10 client.16419 handle_command_reply: tid=0 2023-11-27T15:24:18.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.584+0000 7fd177e98b80 1 client.16419 shutdown 2023-11-27T15:24:18.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.584+0000 7fd177e98b80 1 -- 172.21.15.52:0/3678808946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1bf195470 msgr2=0x55b1bf19e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.584+0000 7fd177e98b80 1 --2- 172.21.15.52:0/3678808946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1bf195470 0x55b1bf19e330 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd1600036c0 tx=0x7fd160003f90 comp rx=0 tx=0).stop 2023-11-27T15:24:18.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.585+0000 7fd177e98b80 1 -- 172.21.15.52:0/3678808946 shutdown_connections 2023-11-27T15:24:18.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.585+0000 7fd177e98b80 1 --2- 172.21.15.52:0/3678808946 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55b1bf1ae950 0x55b1bf1b0d60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.585+0000 7fd177e98b80 1 --2- 172.21.15.52:0/3678808946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1bf1998b0 0x55b1bf19e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.585+0000 7fd177e98b80 1 --2- 172.21.15.52:0/3678808946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1bf194b60 0x55b1bf1a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.585+0000 7fd177e98b80 1 --2- 172.21.15.52:0/3678808946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1bf195470 0x55b1bf19e330 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.585+0000 7fd14d7fa700 0 client.16419 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:24:18.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.585+0000 7fd177e98b80 1 -- 172.21.15.52:0/3678808946 >> 172.21.15.52:0/3678808946 conn(0x55b1bf18a360 msgr2=0x55b1bf18ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.585+0000 7fd177e98b80 1 -- 172.21.15.52:0/3678808946 shutdown_connections 2023-11-27T15:24:18.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.585+0000 7fd177e98b80 1 -- 172.21.15.52:0/3678808946 wait complete. 2023-11-27T15:24:18.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.585+0000 7fd177e98b80 20 client.16419 trim_cache size 0 max 16384 2023-11-27T15:24:18.601 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.598+0000 7fd177e98b80 1 Processor -- start 2023-11-27T15:24:18.602 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.598+0000 7fd177e98b80 1 -- start start 2023-11-27T15:24:18.607 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.598+0000 7fd177e98b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1bf194b60 0x55b1bf1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.607 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.598+0000 7fd177e98b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1bf195470 0x55b1bf1aabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.607 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.598+0000 7fd177e98b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1bf1998b0 0x55b1bf198a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.607 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.598+0000 7fd177e98b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1bf0a80a0 con 0x55b1bf194b60 2023-11-27T15:24:18.607 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.598+0000 7fd177e98b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1bf1abf80 con 0x55b1bf1998b0 2023-11-27T15:24:18.608 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.598+0000 7fd177e98b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1bf190950 con 0x55b1bf195470 2023-11-27T15:24:18.608 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.598+0000 7fd169aa8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1bf194b60 0x55b1bf1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.608 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.598+0000 7fd169aa8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1bf194b60 0x55b1bf1981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54438/0 (socket says 172.21.15.52:54438) 2023-11-27T15:24:18.608 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.598+0000 7fd169aa8700 1 -- 172.21.15.52:0/4061875161 learned_addr learned my addr 172.21.15.52:0/4061875161 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:18.608 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.599+0000 7fd16a2a9700 1 --2- 172.21.15.52:0/4061875161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1bf1998b0 0x55b1bf198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.608 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.599+0000 7fd1692a7700 1 --2- 172.21.15.52:0/4061875161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1bf195470 0x55b1bf1aabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.608 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.599+0000 7fd169aa8700 1 -- 172.21.15.52:0/4061875161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1bf195470 msgr2=0x55b1bf1aabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.608 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.599+0000 7fd169aa8700 1 --2- 172.21.15.52:0/4061875161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1bf195470 0x55b1bf1aabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.599+0000 7fd169aa8700 1 -- 172.21.15.52:0/4061875161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1bf1998b0 msgr2=0x55b1bf198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.599+0000 7fd169aa8700 1 --2- 172.21.15.52:0/4061875161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1bf1998b0 0x55b1bf198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.599+0000 7fd169aa8700 1 -- 172.21.15.52:0/4061875161 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1bf0abee0 con 0x55b1bf194b60 2023-11-27T15:24:18.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.599+0000 7fd1692a7700 1 --2- 172.21.15.52:0/4061875161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1bf195470 0x55b1bf1aabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:24:18.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.599+0000 7fd169aa8700 1 --2- 172.21.15.52:0/4061875161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1bf194b60 0x55b1bf1981a0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd15400f010 tx=0x7fd15400a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad2ab73cea3b1c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.599+0000 7fd158ff9700 1 -- 172.21.15.52:0/4061875161 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd154033960 con 0x55b1bf194b60 2023-11-27T15:24:18.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.599+0000 7fd158ff9700 1 -- 172.21.15.52:0/4061875161 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd15402e040 con 0x55b1bf194b60 2023-11-27T15:24:18.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.599+0000 7fd158ff9700 1 -- 172.21.15.52:0/4061875161 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1540046f0 con 0x55b1bf194b60 2023-11-27T15:24:18.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.600+0000 7fd177e98b80 1 -- 172.21.15.52:0/4061875161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1bf194b60 msgr2=0x55b1bf1981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.600+0000 7fd177e98b80 1 --2- 172.21.15.52:0/4061875161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1bf194b60 0x55b1bf1981a0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd15400f010 tx=0x7fd15400a2d0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.600+0000 7fd177e98b80 1 -- 172.21.15.52:0/4061875161 shutdown_connections 2023-11-27T15:24:18.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.600+0000 7fd177e98b80 1 --2- 172.21.15.52:0/4061875161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1bf1998b0 0x55b1bf198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.600+0000 7fd177e98b80 1 --2- 172.21.15.52:0/4061875161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1bf194b60 0x55b1bf1981a0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.600+0000 7fd177e98b80 1 --2- 172.21.15.52:0/4061875161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1bf195470 0x55b1bf1aabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.600+0000 7fd177e98b80 1 -- 172.21.15.52:0/4061875161 >> 172.21.15.52:0/4061875161 conn(0x55b1bf18a360 msgr2=0x55b1bf199c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.600+0000 7fd177e98b80 1 -- 172.21.15.52:0/4061875161 shutdown_connections 2023-11-27T15:24:18.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.600+0000 7fd177e98b80 1 -- 172.21.15.52:0/4061875161 wait complete. 2023-11-27T15:24:18.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.600+0000 7fd177e98b80 1 -- start start 2023-11-27T15:24:18.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.600+0000 7fd177e98b80 1 Processor -- start 2023-11-27T15:24:18.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.600+0000 7fd177e98b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1bf194b60 0x55b1bf1a1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.600+0000 7fd177e98b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1bf195470 0x55b1bf1a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.600+0000 7fd177e98b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1bf1998b0 0x55b1bf2017c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.600+0000 7fd177e98b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1bf1a2630 con 0x55b1bf195470 2023-11-27T15:24:18.612 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.600+0000 7fd177e98b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1bf1a27a0 con 0x55b1bf194b60 2023-11-27T15:24:18.612 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.600+0000 7fd177e98b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1bf201d00 con 0x55b1bf1998b0 2023-11-27T15:24:18.612 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.601+0000 7fd177e98b80 10 client.0 _ll_register_callbacks cb 0x7fd160003ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:18.612 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.601+0000 7fd169aa8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1bf194b60 0x55b1bf1a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.612 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.601+0000 7fd1692a7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1bf195470 0x55b1bf1a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.612 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.601+0000 7fd169aa8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1bf194b60 0x55b1bf1a1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:44102/0 (socket says 172.21.15.52:44102) 2023-11-27T15:24:18.612 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.601+0000 7fd169aa8700 1 -- 172.21.15.52:0/1548306030 learned_addr learned my addr 172.21.15.52:0/1548306030 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:18.612 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.601+0000 7fd16a2a9700 1 --2- 172.21.15.52:0/1548306030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1bf1998b0 0x55b1bf2017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.601+0000 7fd1692a7700 1 -- 172.21.15.52:0/1548306030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1bf1998b0 msgr2=0x55b1bf2017c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.601+0000 7fd1692a7700 1 --2- 172.21.15.52:0/1548306030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1bf1998b0 0x55b1bf2017c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.601+0000 7fd1692a7700 1 -- 172.21.15.52:0/1548306030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1bf194b60 msgr2=0x55b1bf1a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.601+0000 7fd1692a7700 1 --2- 172.21.15.52:0/1548306030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1bf194b60 0x55b1bf1a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.601+0000 7fd1692a7700 1 -- 172.21.15.52:0/1548306030 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b1bef67630 con 0x55b1bf195470 2023-11-27T15:24:18.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.601+0000 7fd169aa8700 1 --2- 172.21.15.52:0/1548306030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1bf194b60 0x55b1bf1a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:18.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.601+0000 7fd16a2a9700 1 --2- 172.21.15.52:0/1548306030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1bf1998b0 0x55b1bf2017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:24:18.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.601+0000 7fd1692a7700 1 --2- 172.21.15.52:0/1548306030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1bf195470 0x55b1bf1a1ec0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x55b1bf19ebc0 tx=0x7fd160005630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e99e172b71263b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.601+0000 7fd14d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:24:18.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.601+0000 7fd14d7fa700 1 -- 172.21.15.52:0/1548306030 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16000f660 con 0x55b1bf195470 2023-11-27T15:24:18.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.601+0000 7fd14d7fa700 1 -- 172.21.15.52:0/1548306030 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fd160031080 con 0x55b1bf195470 2023-11-27T15:24:18.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.601+0000 7fd177e98b80 1 -- 172.21.15.52:0/1548306030 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd154010e30 con 0x55b1bf195470 2023-11-27T15:24:18.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.601+0000 7fd14d7fa700 1 -- 172.21.15.52:0/1548306030 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd160023500 con 0x55b1bf195470 2023-11-27T15:24:18.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.601+0000 7fd177e98b80 10 client.16425 fetch_fsmap learned FSMap version 28 2023-11-27T15:24:18.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.602+0000 7fd177e98b80 1 -- 172.21.15.52:0/1548306030 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b1beefcc60 con 0x55b1bf195470 2023-11-27T15:24:18.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.602+0000 7fd14d7fa700 1 -- 172.21.15.52:0/1548306030 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd160041020 con 0x55b1bf195470 2023-11-27T15:24:18.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.602+0000 7fd14d7fa700 1 -- 172.21.15.52:0/1548306030 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd160033070 con 0x55b1bf195470 2023-11-27T15:24:18.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.602+0000 7fd177e98b80 10 client.16425 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:24:18.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.602+0000 7fd177e98b80 10 client.16425 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:24:18.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.602+0000 7fd177e98b80 20 client.16425 populate_metadata read hostname 'smithi052' 2023-11-27T15:24:18.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.602+0000 7fd177e98b80 1 --2- 172.21.15.52:0/1548306030 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55b1bf203030 0x55b1bf205440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.602+0000 7fd177e98b80 4 client.16425 mds_command: new command op to 24472 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:24:18.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.602+0000 7fd177e98b80 1 -- 172.21.15.52:0/1548306030 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b1bf0b4660 con 0x55b1bf203030 2023-11-27T15:24:18.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.602+0000 7fd169aa8700 1 --2- 172.21.15.52:0/1548306030 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55b1bf203030 0x55b1bf205440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.603+0000 7fd14d7fa700 10 client.16425 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:24:18.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.603+0000 7fd169aa8700 1 --2- 172.21.15.52:0/1548306030 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55b1bf203030 0x55b1bf205440 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=304486e384d61d12 server_cookie=125009403fdf0295 in_seq=0 out_seq=0 2023-11-27T15:24:18.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.603+0000 7fd14d7fa700 1 -- 172.21.15.52:0/1548306030 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b1bf0b4660 con 0x55b1bf203030 2023-11-27T15:24:18.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.603+0000 7fd14d7fa700 10 client.16425 handle_command_reply: tid=0 2023-11-27T15:24:18.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.603+0000 7fd177e98b80 1 client.16425 shutdown 2023-11-27T15:24:18.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.603+0000 7fd177e98b80 1 -- 172.21.15.52:0/1548306030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1bf195470 msgr2=0x55b1bf1a1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.603+0000 7fd177e98b80 1 --2- 172.21.15.52:0/1548306030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1bf195470 0x55b1bf1a1ec0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x55b1bf19ebc0 tx=0x7fd160005630 comp rx=0 tx=0).stop 2023-11-27T15:24:18.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.603+0000 7fd177e98b80 1 -- 172.21.15.52:0/1548306030 shutdown_connections 2023-11-27T15:24:18.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.603+0000 7fd177e98b80 1 --2- 172.21.15.52:0/1548306030 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55b1bf203030 0x55b1bf205440 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.603+0000 7fd177e98b80 1 --2- 172.21.15.52:0/1548306030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1bf194b60 0x55b1bf1a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.603+0000 7fd177e98b80 1 --2- 172.21.15.52:0/1548306030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1bf1998b0 0x55b1bf2017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.603+0000 7fd177e98b80 1 --2- 172.21.15.52:0/1548306030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1bf195470 0x55b1bf1a1ec0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.603+0000 7fd177e98b80 1 -- 172.21.15.52:0/1548306030 >> 172.21.15.52:0/1548306030 conn(0x55b1bf18a360 msgr2=0x55b1bf1aabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.603+0000 7fd14d7fa700 0 client.16425 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:24:18.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.603+0000 7fd177e98b80 1 -- 172.21.15.52:0/1548306030 shutdown_connections 2023-11-27T15:24:18.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.603+0000 7fd177e98b80 1 -- 172.21.15.52:0/1548306030 wait complete. 2023-11-27T15:24:18.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.603+0000 7fd177e98b80 20 client.16425 trim_cache size 0 max 16384 2023-11-27T15:24:18.618 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:24:18.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.604+0000 7fd16bd0c700 1 -- 172.21.15.52:0/1573466483 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd1500a1340 msgr2=0x7fd1500a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.604+0000 7fd16bd0c700 1 --2- 172.21.15.52:0/1573466483 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd1500a1340 0x7fd1500a37f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd154005fd0 tx=0x7fd15400a560 comp rx=0 tx=0).stop 2023-11-27T15:24:18.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.604+0000 7fd16bd0c700 1 -- 172.21.15.52:0/1573466483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1641055f0 msgr2=0x7fd16419ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.604+0000 7fd16bd0c700 1 --2- 172.21.15.52:0/1573466483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1641055f0 0x7fd16419ae90 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00ef30 tx=0x7fd15c00c3a0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.604+0000 7fd16bd0c700 1 -- 172.21.15.52:0/1573466483 shutdown_connections 2023-11-27T15:24:18.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.604+0000 7fd16bd0c700 1 --2- 172.21.15.52:0/1573466483 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd1500a1340 0x7fd1500a37f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.604+0000 7fd16bd0c700 1 --2- 172.21.15.52:0/1573466483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1641055f0 0x7fd16419ae90 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.604+0000 7fd16bd0c700 1 --2- 172.21.15.52:0/1573466483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1641005b0 0x7fd1641964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.604+0000 7fd16bd0c700 1 --2- 172.21.15.52:0/1573466483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd164100f50 0x7fd164196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.604+0000 7fd16bd0c700 1 -- 172.21.15.52:0/1573466483 >> 172.21.15.52:0/1573466483 conn(0x7fd164077170 msgr2=0x7fd1640fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.604+0000 7fd16bd0c700 1 -- 172.21.15.52:0/1573466483 shutdown_connections 2023-11-27T15:24:18.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.604+0000 7fd16bd0c700 1 -- 172.21.15.52:0/1573466483 wait complete. 2023-11-27T15:24:18.621 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-27T15:24:18.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.745+0000 7f0622c13700 1 -- 172.21.15.52:0/429601987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f061c100340 msgr2=0x7f061c10c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.745+0000 7f0622c13700 1 --2- 172.21.15.52:0/429601987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f061c100340 0x7f061c10c010 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f061400b0d0 tx=0x7f061400b3e0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.745+0000 7f0622c13700 1 -- 172.21.15.52:0/429601987 shutdown_connections 2023-11-27T15:24:18.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.745+0000 7f0622c13700 1 --2- 172.21.15.52:0/429601987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f061c100340 0x7f061c10c010 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.745+0000 7f0622c13700 1 --2- 172.21.15.52:0/429601987 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f061c104180 0x7f061c104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.745+0000 7f0622c13700 1 --2- 172.21.15.52:0/429601987 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f061c0ff990 0x7f061c0ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.745+0000 7f0622c13700 1 -- 172.21.15.52:0/429601987 >> 172.21.15.52:0/429601987 conn(0x7f061c075920 msgr2=0x7f061c075d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.745+0000 7f0622c13700 1 -- 172.21.15.52:0/429601987 shutdown_connections 2023-11-27T15:24:18.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.745+0000 7f0622c13700 1 -- 172.21.15.52:0/429601987 wait complete. 2023-11-27T15:24:18.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.746+0000 7f0622c13700 1 Processor -- start 2023-11-27T15:24:18.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.746+0000 7f0622c13700 1 -- start start 2023-11-27T15:24:18.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.746+0000 7f0622c13700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f061c0ff990 0x7f061c195440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.746+0000 7f0622c13700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f061c100340 0x7f061c195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.746+0000 7f0622c13700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f061c104180 0x7f061c199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.746+0000 7f0622c13700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c19a420 con 0x7f061c0ff990 2023-11-27T15:24:18.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.746+0000 7f0622c13700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c19a590 con 0x7f061c104180 2023-11-27T15:24:18.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.746+0000 7f0622c13700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c19a750 con 0x7f061c100340 2023-11-27T15:24:18.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.746+0000 7f06209af700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f061c0ff990 0x7f061c195440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.746+0000 7f06209af700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f061c0ff990 0x7f061c195440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54458/0 (socket says 172.21.15.52:54458) 2023-11-27T15:24:18.754 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.746+0000 7f06209af700 1 -- 172.21.15.52:0/4251786322 learned_addr learned my addr 172.21.15.52:0/4251786322 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:18.754 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.746+0000 7f06211b0700 1 --2- 172.21.15.52:0/4251786322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f061c104180 0x7f061c199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.754 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.746+0000 7f06209af700 1 -- 172.21.15.52:0/4251786322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f061c100340 msgr2=0x7f061c195980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.754 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.746+0000 7f061bfff700 1 --2- 172.21.15.52:0/4251786322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f061c100340 0x7f061c195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.754 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.746+0000 7f06209af700 1 --2- 172.21.15.52:0/4251786322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f061c100340 0x7f061c195980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.754 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.746+0000 7f06209af700 1 -- 172.21.15.52:0/4251786322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f061c104180 msgr2=0x7f061c199de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.754 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.746+0000 7f06209af700 1 --2- 172.21.15.52:0/4251786322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f061c104180 0x7f061c199de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.755 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.746+0000 7f06209af700 1 -- 172.21.15.52:0/4251786322 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0614009d00 con 0x7f061c0ff990 2023-11-27T15:24:18.755 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.747+0000 7f06209af700 1 --2- 172.21.15.52:0/4251786322 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f061c0ff990 0x7f061c195440 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f061000d950 tx=0x7f061000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.755 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.747+0000 7f0619ffb700 1 -- 172.21.15.52:0/4251786322 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0610009940 con 0x7f061c0ff990 2023-11-27T15:24:18.755 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.747+0000 7f0619ffb700 1 -- 172.21.15.52:0/4251786322 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0610010460 con 0x7f061c0ff990 2023-11-27T15:24:18.755 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.747+0000 7f0622c13700 1 -- 172.21.15.52:0/4251786322 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f061c10f260 con 0x7f061c0ff990 2023-11-27T15:24:18.755 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.747+0000 7f0619ffb700 1 -- 172.21.15.52:0/4251786322 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061000f620 con 0x7f061c0ff990 2023-11-27T15:24:18.755 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.747+0000 7f0622c13700 1 -- 172.21.15.52:0/4251786322 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f061c10f0d0 con 0x7f061c0ff990 2023-11-27T15:24:18.761 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.750+0000 7f0619ffb700 1 -- 172.21.15.52:0/4251786322 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f06100108d0 con 0x7f061c0ff990 2023-11-27T15:24:18.762 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.750+0000 7f0619ffb700 1 --2- 172.21.15.52:0/4251786322 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f060c0a12c0 0x7f060c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.762 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.750+0000 7f0619ffb700 1 -- 172.21.15.52:0/4251786322 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f06100ccb50 con 0x7f061c0ff990 2023-11-27T15:24:18.762 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.750+0000 7f061bfff700 1 --2- 172.21.15.52:0/4251786322 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f060c0a12c0 0x7f060c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.762 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.751+0000 7f061bfff700 1 --2- 172.21.15.52:0/4251786322 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f060c0a12c0 0x7f060c0a3770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0608005950 tx=0x7f06080058e0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.762 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.751+0000 7f062ed9fb80 1 Processor -- start 2023-11-27T15:24:18.762 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.751+0000 7f062ed9fb80 1 -- start start 2023-11-27T15:24:18.762 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.751+0000 7f062ed9fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3449c7940 0x55d3449c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.751+0000 7f062ed9fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d3449c8250 0x55d3449c1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.751+0000 7f062ed9fb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3449c20c0 0x55d3449c2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.751+0000 7f062ed9fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3448c5060 con 0x55d3449c8250 2023-11-27T15:24:18.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.751+0000 7f062ed9fb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3449178a0 con 0x55d3449c20c0 2023-11-27T15:24:18.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.751+0000 7f062ed9fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d344910d30 con 0x55d3449c7940 2023-11-27T15:24:18.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.751+0000 7f061bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d3449c8250 0x55d3449c1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.752+0000 7f061bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d3449c8250 0x55d3449c1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54468/0 (socket says 172.21.15.52:54468) 2023-11-27T15:24:18.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.752+0000 7f061bfff700 1 -- 172.21.15.52:0/3416838410 learned_addr learned my addr 172.21.15.52:0/3416838410 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:18.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.752+0000 7f06211b0700 1 --2- 172.21.15.52:0/3416838410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3449c20c0 0x55d3449c2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.752+0000 7f06209af700 1 --2- 172.21.15.52:0/3416838410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3449c7940 0x55d3449c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.752+0000 7f061bfff700 1 -- 172.21.15.52:0/3416838410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3449c7940 msgr2=0x55d3449c7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.752+0000 7f061bfff700 1 --2- 172.21.15.52:0/3416838410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3449c7940 0x55d3449c7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.752+0000 7f061bfff700 1 -- 172.21.15.52:0/3416838410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3449c20c0 msgr2=0x55d3449c2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.752+0000 7f061bfff700 1 --2- 172.21.15.52:0/3416838410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3449c20c0 0x55d3449c2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.752+0000 7f061bfff700 1 -- 172.21.15.52:0/3416838410 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d344792630 con 0x55d3449c8250 2023-11-27T15:24:18.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.752+0000 7f06211b0700 1 --2- 172.21.15.52:0/3416838410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3449c20c0 0x55d3449c2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:24:18.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.752+0000 7f061bfff700 1 --2- 172.21.15.52:0/3416838410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d3449c8250 0x55d3449c1b80 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0608009450 tx=0x7f0608011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47240b470231e718 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.752+0000 7f0606ffd700 1 -- 172.21.15.52:0/3416838410 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0608011b00 con 0x55d3449c8250 2023-11-27T15:24:18.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.752+0000 7f0606ffd700 1 -- 172.21.15.52:0/3416838410 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0608011c60 con 0x55d3449c8250 2023-11-27T15:24:18.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.752+0000 7f0606ffd700 1 -- 172.21.15.52:0/3416838410 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06080105f0 con 0x55d3449c8250 2023-11-27T15:24:18.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.753+0000 7f062ed9fb80 1 -- 172.21.15.52:0/3416838410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d3449c8250 msgr2=0x55d3449c1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.753+0000 7f062ed9fb80 1 --2- 172.21.15.52:0/3416838410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d3449c8250 0x55d3449c1b80 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0608009450 tx=0x7f0608011000 comp rx=0 tx=0).stop 2023-11-27T15:24:18.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.753+0000 7f062ed9fb80 1 -- 172.21.15.52:0/3416838410 shutdown_connections 2023-11-27T15:24:18.766 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.753+0000 7f062ed9fb80 1 --2- 172.21.15.52:0/3416838410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3449c20c0 0x55d3449c2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.766 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.753+0000 7f062ed9fb80 1 --2- 172.21.15.52:0/3416838410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3449c7940 0x55d3449c7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.766 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.753+0000 7f062ed9fb80 1 --2- 172.21.15.52:0/3416838410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d3449c8250 0x55d3449c1b80 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.766 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.753+0000 7f062ed9fb80 1 -- 172.21.15.52:0/3416838410 >> 172.21.15.52:0/3416838410 conn(0x55d3449b5360 msgr2=0x55d3449bab40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.766 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.753+0000 7f062ed9fb80 1 -- 172.21.15.52:0/3416838410 shutdown_connections 2023-11-27T15:24:18.766 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.753+0000 7f062ed9fb80 1 -- 172.21.15.52:0/3416838410 wait complete. 2023-11-27T15:24:18.766 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.753+0000 7f062ed9fb80 1 -- start start 2023-11-27T15:24:18.766 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.753+0000 7f062ed9fb80 1 Processor -- start 2023-11-27T15:24:18.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.754+0000 7f062ed9fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d3449c20c0 0x55d3449ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.754+0000 7f062ed9fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3449c7940 0x55d3449cb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.754+0000 7f062ed9fb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3449c8250 0x55d3449cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.754+0000 7f062ed9fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3448b28e0 con 0x55d3449c20c0 2023-11-27T15:24:18.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.754+0000 7f062ed9fb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3448b2580 con 0x55d3449c8250 2023-11-27T15:24:18.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.754+0000 7f062ed9fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d344794920 con 0x55d3449c7940 2023-11-27T15:24:18.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.754+0000 7f062ed9fb80 10 client.0 _ll_register_callbacks cb 0x55d3448d0780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:18.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.754+0000 7f06209af700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d3449c20c0 0x55d3449ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.754+0000 7f06209af700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d3449c20c0 0x55d3449ce1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54472/0 (socket says 172.21.15.52:54472) 2023-11-27T15:24:18.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.754+0000 7f06209af700 1 -- 172.21.15.52:0/2405958693 learned_addr learned my addr 172.21.15.52:0/2405958693 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:18.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.754+0000 7f061bfff700 1 --2- 172.21.15.52:0/2405958693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3449c7940 0x55d3449cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.754+0000 7f06211b0700 1 --2- 172.21.15.52:0/2405958693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3449c8250 0x55d3449cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.754+0000 7f06209af700 1 -- 172.21.15.52:0/2405958693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3449c7940 msgr2=0x55d3449cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.754+0000 7f06209af700 1 --2- 172.21.15.52:0/2405958693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3449c7940 0x55d3449cb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.754+0000 7f06209af700 1 -- 172.21.15.52:0/2405958693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3449c8250 msgr2=0x55d3449cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.754+0000 7f06209af700 1 --2- 172.21.15.52:0/2405958693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3449c8250 0x55d3449cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.754+0000 7f06209af700 1 -- 172.21.15.52:0/2405958693 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060800e940 con 0x55d3449c20c0 2023-11-27T15:24:18.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.754+0000 7f06211b0700 1 --2- 172.21.15.52:0/2405958693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3449c8250 0x55d3449cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:24:18.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.754+0000 7f06209af700 1 --2- 172.21.15.52:0/2405958693 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d3449c20c0 0x55d3449ce1f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f06100960a0 tx=0x7f061009dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f09e44868af74b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.754+0000 7f05fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:24:18.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.755+0000 7f05fbfff700 1 -- 172.21.15.52:0/2405958693 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0610096480 con 0x55d3449c20c0 2023-11-27T15:24:18.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.755+0000 7f05fbfff700 1 -- 172.21.15.52:0/2405958693 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f06100b2200 con 0x55d3449c20c0 2023-11-27T15:24:18.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.755+0000 7f062ed9fb80 1 -- 172.21.15.52:0/2405958693 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3449b2b20 con 0x55d3449c20c0 2023-11-27T15:24:18.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.755+0000 7f05fbfff700 1 -- 172.21.15.52:0/2405958693 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06100b23c0 con 0x55d3449c20c0 2023-11-27T15:24:18.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.755+0000 7f062ed9fb80 10 client.16437 fetch_fsmap learned FSMap version 28 2023-11-27T15:24:18.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.755+0000 7f062ed9fb80 1 -- 172.21.15.52:0/2405958693 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d3448d7190 con 0x55d3449c20c0 2023-11-27T15:24:18.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.755+0000 7f05fbfff700 1 -- 172.21.15.52:0/2405958693 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06100c8080 con 0x55d3449c20c0 2023-11-27T15:24:18.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.755+0000 7f05fbfff700 1 -- 172.21.15.52:0/2405958693 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0610098b50 con 0x55d3449c20c0 2023-11-27T15:24:18.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.755+0000 7f062ed9fb80 10 client.16437 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:24:18.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.755+0000 7f062ed9fb80 10 client.16437 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:24:18.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.755+0000 7f062ed9fb80 20 client.16437 populate_metadata read hostname 'smithi052' 2023-11-27T15:24:18.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.755+0000 7f062ed9fb80 1 --2- 172.21.15.52:0/2405958693 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55d3449db980 0x55d3449ddd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.755+0000 7f062ed9fb80 4 client.16437 mds_command: new command op to 24494 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:24:18.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.755+0000 7f062ed9fb80 1 -- 172.21.15.52:0/2405958693 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3448d30a0 con 0x55d3449db980 2023-11-27T15:24:18.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.755+0000 7f061bfff700 1 --2- 172.21.15.52:0/2405958693 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55d3449db980 0x55d3449ddd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.756+0000 7f05fbfff700 10 client.16437 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:24:18.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.756+0000 7f061bfff700 1 --2- 172.21.15.52:0/2405958693 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55d3449db980 0x55d3449ddd90 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=550eacc698a8e2b9 server_cookie=bdd6cbed4a744b12 in_seq=0 out_seq=0 2023-11-27T15:24:18.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.756+0000 7f05fbfff700 1 -- 172.21.15.52:0/2405958693 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d3448d30a0 con 0x55d3449db980 2023-11-27T15:24:18.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.756+0000 7f05fbfff700 10 client.16437 handle_command_reply: tid=0 2023-11-27T15:24:18.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.756+0000 7f062ed9fb80 1 client.16437 shutdown 2023-11-27T15:24:18.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.757+0000 7f062ed9fb80 1 -- 172.21.15.52:0/2405958693 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d3449c20c0 msgr2=0x55d3449ce1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.757+0000 7f062ed9fb80 1 --2- 172.21.15.52:0/2405958693 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d3449c20c0 0x55d3449ce1f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f06100960a0 tx=0x7f061009dca0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.757+0000 7f062ed9fb80 1 -- 172.21.15.52:0/2405958693 shutdown_connections 2023-11-27T15:24:18.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.757+0000 7f062ed9fb80 1 --2- 172.21.15.52:0/2405958693 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55d3449db980 0x55d3449ddd90 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.757+0000 7f062ed9fb80 1 --2- 172.21.15.52:0/2405958693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3449c8250 0x55d3449cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.757+0000 7f062ed9fb80 1 --2- 172.21.15.52:0/2405958693 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d3449c20c0 0x55d3449ce1f0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.757+0000 7f062ed9fb80 1 --2- 172.21.15.52:0/2405958693 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3449c7940 0x55d3449cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.757+0000 7f05fbfff700 0 client.16437 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:24:18.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.757+0000 7f062ed9fb80 1 -- 172.21.15.52:0/2405958693 >> 172.21.15.52:0/2405958693 conn(0x55d3449b5360 msgr2=0x55d3449b9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.757+0000 7f062ed9fb80 1 -- 172.21.15.52:0/2405958693 shutdown_connections 2023-11-27T15:24:18.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.757+0000 7f062ed9fb80 1 -- 172.21.15.52:0/2405958693 wait complete. 2023-11-27T15:24:18.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.757+0000 7f062ed9fb80 20 client.16437 trim_cache size 0 max 16384 2023-11-27T15:24:18.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.770+0000 7f062ed9fb80 1 Processor -- start 2023-11-27T15:24:18.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.770+0000 7f062ed9fb80 1 -- start start 2023-11-27T15:24:18.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.770+0000 7f062ed9fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d3449c20c0 0x55d3449d2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.770+0000 7f062ed9fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3449c7940 0x55d3449cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.770+0000 7f062ed9fb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3449c8250 0x55d3449c5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.770+0000 7f062ed9fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3449d90c0 con 0x55d3449c20c0 2023-11-27T15:24:18.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.770+0000 7f062ed9fb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3449d7a20 con 0x55d3449c8250 2023-11-27T15:24:18.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.770+0000 7f062ed9fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3449c9960 con 0x55d3449c7940 2023-11-27T15:24:18.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.770+0000 7f06209af700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d3449c20c0 0x55d3449d2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.770+0000 7f06209af700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d3449c20c0 0x55d3449d2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54480/0 (socket says 172.21.15.52:54480) 2023-11-27T15:24:18.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.770+0000 7f06209af700 1 -- 172.21.15.52:0/1886723522 learned_addr learned my addr 172.21.15.52:0/1886723522 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:18.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.770+0000 7f06211b0700 1 --2- 172.21.15.52:0/1886723522 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3449c8250 0x55d3449c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.771+0000 7f061bfff700 1 --2- 172.21.15.52:0/1886723522 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3449c7940 0x55d3449cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.771+0000 7f06209af700 1 -- 172.21.15.52:0/1886723522 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3449c7940 msgr2=0x55d3449cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.771+0000 7f06209af700 1 --2- 172.21.15.52:0/1886723522 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3449c7940 0x55d3449cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.771+0000 7f06209af700 1 -- 172.21.15.52:0/1886723522 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3449c8250 msgr2=0x55d3449c5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.771+0000 7f06209af700 1 --2- 172.21.15.52:0/1886723522 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3449c8250 0x55d3449c5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.771+0000 7f06209af700 1 -- 172.21.15.52:0/1886723522 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3448d6ee0 con 0x55d3449c20c0 2023-11-27T15:24:18.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.771+0000 7f06209af700 1 --2- 172.21.15.52:0/1886723522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d3449c20c0 0x55d3449d2b60 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f06100ca870 tx=0x7f0610005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b53ce63e8d6f50a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.771+0000 7f06077fe700 1 -- 172.21.15.52:0/1886723522 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0610014070 con 0x55d3449c20c0 2023-11-27T15:24:18.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.771+0000 7f06077fe700 1 -- 172.21.15.52:0/1886723522 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d3449b2b20 con 0x55d3449c20c0 2023-11-27T15:24:18.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.771+0000 7f06077fe700 1 -- 172.21.15.52:0/1886723522 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0610098cd0 con 0x55d3449c20c0 2023-11-27T15:24:18.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.772+0000 7f062ed9fb80 1 -- 172.21.15.52:0/1886723522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d3449c20c0 msgr2=0x55d3449d2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.772+0000 7f062ed9fb80 1 --2- 172.21.15.52:0/1886723522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d3449c20c0 0x55d3449d2b60 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f06100ca870 tx=0x7f0610005910 comp rx=0 tx=0).stop 2023-11-27T15:24:18.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.772+0000 7f062ed9fb80 1 -- 172.21.15.52:0/1886723522 shutdown_connections 2023-11-27T15:24:18.783 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.772+0000 7f062ed9fb80 1 --2- 172.21.15.52:0/1886723522 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3449c8250 0x55d3449c5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.783 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.772+0000 7f062ed9fb80 1 --2- 172.21.15.52:0/1886723522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d3449c20c0 0x55d3449d2b60 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.783 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.772+0000 7f062ed9fb80 1 --2- 172.21.15.52:0/1886723522 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3449c7940 0x55d3449cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.783 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.772+0000 7f062ed9fb80 1 -- 172.21.15.52:0/1886723522 >> 172.21.15.52:0/1886723522 conn(0x55d3449b5360 msgr2=0x55d3449d7310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.783 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.772+0000 7f062ed9fb80 1 -- 172.21.15.52:0/1886723522 shutdown_connections 2023-11-27T15:24:18.783 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.772+0000 7f062ed9fb80 1 -- 172.21.15.52:0/1886723522 wait complete. 2023-11-27T15:24:18.783 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.772+0000 7f062ed9fb80 1 -- start start 2023-11-27T15:24:18.783 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.772+0000 7f062ed9fb80 1 Processor -- start 2023-11-27T15:24:18.784 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.772+0000 7f062ed9fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3449c20c0 0x55d3449c33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.784 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.772+0000 7f062ed9fb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3449c7940 0x55d3449c38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.784 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.772+0000 7f062ed9fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d3449c8250 0x55d3449c3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.784 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.772+0000 7f062ed9fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3449c43a0 con 0x55d3449c8250 2023-11-27T15:24:18.784 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.773+0000 7f062ed9fb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3449c4510 con 0x55d3449c7940 2023-11-27T15:24:18.784 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.773+0000 7f062ed9fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3449c46d0 con 0x55d3449c20c0 2023-11-27T15:24:18.784 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.773+0000 7f062ed9fb80 10 client.0 _ll_register_callbacks cb 0x55d3448d0780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:18.784 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.773+0000 7f06211b0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d3449c8250 0x55d3449c3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.785 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.773+0000 7f06211b0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d3449c8250 0x55d3449c3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54490/0 (socket says 172.21.15.52:54490) 2023-11-27T15:24:18.785 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.773+0000 7f061bfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3449c7940 0x55d3449c38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.785 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.773+0000 7f06211b0700 1 -- 172.21.15.52:0/954600743 learned_addr learned my addr 172.21.15.52:0/954600743 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:18.785 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.773+0000 7f06209af700 1 --2- 172.21.15.52:0/954600743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3449c20c0 0x55d3449c33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.785 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.773+0000 7f06211b0700 1 -- 172.21.15.52:0/954600743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3449c20c0 msgr2=0x55d3449c33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.785 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.773+0000 7f06211b0700 1 --2- 172.21.15.52:0/954600743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3449c20c0 0x55d3449c33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.785 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.773+0000 7f06211b0700 1 -- 172.21.15.52:0/954600743 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3449c7940 msgr2=0x55d3449c38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.786 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.773+0000 7f06211b0700 1 --2- 172.21.15.52:0/954600743 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3449c7940 0x55d3449c38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.786 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.773+0000 7f06211b0700 1 -- 172.21.15.52:0/954600743 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3448d7190 con 0x55d3449c8250 2023-11-27T15:24:18.786 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.773+0000 7f06209af700 1 --2- 172.21.15.52:0/954600743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3449c20c0 0x55d3449c33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:24:18.786 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.773+0000 7f06211b0700 1 --2- 172.21.15.52:0/954600743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d3449c8250 0x55d3449c3e20 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f061c104db0 tx=0x7f0614003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1574bd78f0e899a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.786 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.773+0000 7f05fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:24:18.786 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.773+0000 7f05fbfff700 1 -- 172.21.15.52:0/954600743 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0614021070 con 0x55d3449c8250 2023-11-27T15:24:18.786 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.773+0000 7f062ed9fb80 1 -- 172.21.15.52:0/954600743 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0608011c60 con 0x55d3449c8250 2023-11-27T15:24:18.787 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.773+0000 7f05fbfff700 1 -- 172.21.15.52:0/954600743 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f06140074e0 con 0x55d3449c8250 2023-11-27T15:24:18.787 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.773+0000 7f05fbfff700 1 -- 172.21.15.52:0/954600743 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f061400e040 con 0x55d3449c8250 2023-11-27T15:24:18.787 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.774+0000 7f062ed9fb80 10 client.16443 fetch_fsmap learned FSMap version 28 2023-11-27T15:24:18.787 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.774+0000 7f062ed9fb80 1 -- 172.21.15.52:0/954600743 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d344727c60 con 0x55d3449c8250 2023-11-27T15:24:18.787 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.774+0000 7f05fbfff700 1 -- 172.21.15.52:0/954600743 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0614032020 con 0x55d3449c8250 2023-11-27T15:24:18.787 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.774+0000 7f05fbfff700 1 -- 172.21.15.52:0/954600743 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f061400b100 con 0x55d3449c8250 2023-11-27T15:24:18.787 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.774+0000 7f062ed9fb80 10 client.16443 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:24:18.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.774+0000 7f062ed9fb80 10 client.16443 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:24:18.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.774+0000 7f062ed9fb80 20 client.16443 populate_metadata read hostname 'smithi052' 2023-11-27T15:24:18.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.774+0000 7f062ed9fb80 1 --2- 172.21.15.52:0/954600743 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55d344a304c0 0x55d344a328d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.774+0000 7f062ed9fb80 4 client.16443 mds_command: new command op to 24494 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:24:18.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.774+0000 7f062ed9fb80 1 -- 172.21.15.52:0/954600743 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d3448df660 con 0x55d344a304c0 2023-11-27T15:24:18.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.774+0000 7f06209af700 1 --2- 172.21.15.52:0/954600743 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55d344a304c0 0x55d344a328d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.774+0000 7f05fbfff700 10 client.16443 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:24:18.788 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.774+0000 7f06209af700 1 --2- 172.21.15.52:0/954600743 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55d344a304c0 0x55d344a328d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9dcf1ada989e41b9 server_cookie=652d241ad887af2f in_seq=0 out_seq=0 2023-11-27T15:24:18.789 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.775+0000 7f05fbfff700 1 -- 172.21.15.52:0/954600743 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d3448df660 con 0x55d344a304c0 2023-11-27T15:24:18.789 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.775+0000 7f05fbfff700 10 client.16443 handle_command_reply: tid=0 2023-11-27T15:24:18.789 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.775+0000 7f062ed9fb80 1 client.16443 shutdown 2023-11-27T15:24:18.789 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.775+0000 7f062ed9fb80 1 -- 172.21.15.52:0/954600743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d3449c8250 msgr2=0x55d3449c3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.789 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.775+0000 7f062ed9fb80 1 --2- 172.21.15.52:0/954600743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d3449c8250 0x55d3449c3e20 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f061c104db0 tx=0x7f0614003c30 comp rx=0 tx=0).stop 2023-11-27T15:24:18.789 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.775+0000 7f062ed9fb80 1 -- 172.21.15.52:0/954600743 shutdown_connections 2023-11-27T15:24:18.789 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.775+0000 7f062ed9fb80 1 --2- 172.21.15.52:0/954600743 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55d344a304c0 0x55d344a328d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.789 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.775+0000 7f062ed9fb80 1 --2- 172.21.15.52:0/954600743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d3449c8250 0x55d3449c3e20 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.775+0000 7f062ed9fb80 1 --2- 172.21.15.52:0/954600743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55d3449c20c0 0x55d3449c33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.775+0000 7f062ed9fb80 1 --2- 172.21.15.52:0/954600743 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3449c7940 0x55d3449c38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.775+0000 7f05fbfff700 0 client.16443 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:24:18.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.775+0000 7f062ed9fb80 1 -- 172.21.15.52:0/954600743 >> 172.21.15.52:0/954600743 conn(0x55d3449b5360 msgr2=0x55d3449c5560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.775+0000 7f062ed9fb80 1 -- 172.21.15.52:0/954600743 shutdown_connections 2023-11-27T15:24:18.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.775+0000 7f062ed9fb80 1 -- 172.21.15.52:0/954600743 wait complete. 2023-11-27T15:24:18.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.775+0000 7f062ed9fb80 20 client.16443 trim_cache size 0 max 16384 2023-11-27T15:24:18.791 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:24:18.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.776+0000 7f0622c13700 1 -- 172.21.15.52:0/4251786322 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f060c0a12c0 msgr2=0x7f060c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.776+0000 7f0622c13700 1 --2- 172.21.15.52:0/4251786322 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f060c0a12c0 0x7f060c0a3770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0608005950 tx=0x7f06080058e0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.776+0000 7f0622c13700 1 -- 172.21.15.52:0/4251786322 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f061c0ff990 msgr2=0x7f061c195440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.776+0000 7f0622c13700 1 --2- 172.21.15.52:0/4251786322 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f061c0ff990 0x7f061c195440 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f061000d950 tx=0x7f061000dc60 comp rx=0 tx=0).stop 2023-11-27T15:24:18.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.776+0000 7f0622c13700 1 -- 172.21.15.52:0/4251786322 shutdown_connections 2023-11-27T15:24:18.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.776+0000 7f0622c13700 1 --2- 172.21.15.52:0/4251786322 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f060c0a12c0 0x7f060c0a3770 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.776+0000 7f0622c13700 1 --2- 172.21.15.52:0/4251786322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f061c104180 0x7f061c199de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.776+0000 7f0622c13700 1 --2- 172.21.15.52:0/4251786322 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f061c0ff990 0x7f061c195440 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.776+0000 7f0622c13700 1 --2- 172.21.15.52:0/4251786322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f061c100340 0x7f061c195980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.776+0000 7f0622c13700 1 -- 172.21.15.52:0/4251786322 >> 172.21.15.52:0/4251786322 conn(0x7f061c075920 msgr2=0x7f061c0fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.776+0000 7f0622c13700 1 -- 172.21.15.52:0/4251786322 shutdown_connections 2023-11-27T15:24:18.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.776+0000 7f0622c13700 1 -- 172.21.15.52:0/4251786322 wait complete. 2023-11-27T15:24:18.794 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-27T15:24:18.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.918+0000 7f87b5fd0700 1 -- 172.21.15.52:0/2407583268 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87b0101390 msgr2=0x7f87b0101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.918+0000 7f87b5fd0700 1 --2- 172.21.15.52:0/2407583268 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87b0101390 0x7f87b0101760 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f87a0009990 tx=0x7f87a0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.918+0000 7f87b5fd0700 1 -- 172.21.15.52:0/2407583268 shutdown_connections 2023-11-27T15:24:18.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.918+0000 7f87b5fd0700 1 --2- 172.21.15.52:0/2407583268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87b010a8b0 0x7f87b010ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.918+0000 7f87b5fd0700 1 --2- 172.21.15.52:0/2407583268 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87b0101390 0x7f87b0101760 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.918+0000 7f87b5fd0700 1 --2- 172.21.15.52:0/2407583268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87b0101ca0 0x7f87b010a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.918+0000 7f87b5fd0700 1 -- 172.21.15.52:0/2407583268 >> 172.21.15.52:0/2407583268 conn(0x7f87b00770b0 msgr2=0x7f87b00774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.918+0000 7f87b5fd0700 1 -- 172.21.15.52:0/2407583268 shutdown_connections 2023-11-27T15:24:18.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.918+0000 7f87b5fd0700 1 -- 172.21.15.52:0/2407583268 wait complete. 2023-11-27T15:24:18.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.919+0000 7f87b5fd0700 1 Processor -- start 2023-11-27T15:24:18.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.919+0000 7f87b5fd0700 1 -- start start 2023-11-27T15:24:18.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.919+0000 7f87b5fd0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87b0101390 0x7f87b01963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.919+0000 7f87b5fd0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87b0101ca0 0x7f87b01968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.919+0000 7f87b5fd0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87b010a8b0 0x7f87b019ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.926 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.919+0000 7f87b5fd0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b019b390 con 0x7f87b0101ca0 2023-11-27T15:24:18.926 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.919+0000 7f87b5fd0700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b019b500 con 0x7f87b0101390 2023-11-27T15:24:18.926 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.919+0000 7f87b5fd0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b019b6c0 con 0x7f87b010a8b0 2023-11-27T15:24:18.926 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.919+0000 7f87aeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87b0101ca0 0x7f87b01968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.926 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.919+0000 7f87aeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87b0101ca0 0x7f87b01968f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54502/0 (socket says 172.21.15.52:54502) 2023-11-27T15:24:18.926 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.919+0000 7f87aeffd700 1 -- 172.21.15.52:0/3159209093 learned_addr learned my addr 172.21.15.52:0/3159209093 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:18.926 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.919+0000 7f87af7fe700 1 --2- 172.21.15.52:0/3159209093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87b0101390 0x7f87b01963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.926 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.920+0000 7f87affff700 1 --2- 172.21.15.52:0/3159209093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87b010a8b0 0x7f87b019ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.920+0000 7f87aeffd700 1 -- 172.21.15.52:0/3159209093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87b010a8b0 msgr2=0x7f87b019ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.920+0000 7f87aeffd700 1 --2- 172.21.15.52:0/3159209093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87b010a8b0 0x7f87b019ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.920+0000 7f87aeffd700 1 -- 172.21.15.52:0/3159209093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87b0101390 msgr2=0x7f87b01963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.920+0000 7f87aeffd700 1 --2- 172.21.15.52:0/3159209093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87b0101390 0x7f87b01963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.920+0000 7f87aeffd700 1 -- 172.21.15.52:0/3159209093 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a00095f0 con 0x7f87b0101ca0 2023-11-27T15:24:18.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.920+0000 7f87affff700 1 --2- 172.21.15.52:0/3159209093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87b010a8b0 0x7f87b019ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:24:18.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.920+0000 7f87aeffd700 1 --2- 172.21.15.52:0/3159209093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87b0101ca0 0x7f87b01968f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f879800c9b0 tx=0x7f879800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.920+0000 7f87acff9700 1 -- 172.21.15.52:0/3159209093 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8798007b00 con 0x7f87b0101ca0 2023-11-27T15:24:18.928 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.920+0000 7f87acff9700 1 -- 172.21.15.52:0/3159209093 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8798007c60 con 0x7f87b0101ca0 2023-11-27T15:24:18.928 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.920+0000 7f87b5fd0700 1 -- 172.21.15.52:0/3159209093 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b0110330 con 0x7f87b0101ca0 2023-11-27T15:24:18.928 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.920+0000 7f87acff9700 1 -- 172.21.15.52:0/3159209093 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8798018660 con 0x7f87b0101ca0 2023-11-27T15:24:18.928 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.920+0000 7f87b5fd0700 1 -- 172.21.15.52:0/3159209093 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b01101a0 con 0x7f87b0101ca0 2023-11-27T15:24:18.934 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.922+0000 7f87acff9700 1 -- 172.21.15.52:0/3159209093 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f879800f450 con 0x7f87b0101ca0 2023-11-27T15:24:18.934 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.923+0000 7f87acff9700 1 --2- 172.21.15.52:0/3159209093 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f879c0a1340 0x7f879c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.934 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.923+0000 7f87af7fe700 1 --2- 172.21.15.52:0/3159209093 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f879c0a1340 0x7f879c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.934 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.923+0000 7f87acff9700 1 -- 172.21.15.52:0/3159209093 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f87980cccd0 con 0x7f87b0101ca0 2023-11-27T15:24:18.934 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.924+0000 7f87af7fe700 1 --2- 172.21.15.52:0/3159209093 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f879c0a1340 0x7f879c0a37f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f87a000b2e0 tx=0x7f87a0005960 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.924+0000 7f87c215cb80 1 Processor -- start 2023-11-27T15:24:18.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.924+0000 7f87c215cb80 1 -- start start 2023-11-27T15:24:18.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.924+0000 7f87c215cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f8b965b60 0x557f8b965f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.924+0000 7f87c215cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8b966470 0x557f8b96a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.924+0000 7f87c215cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f8b96a8b0 0x557f8b96ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.924+0000 7f87c215cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8b86b060 con 0x557f8b965b60 2023-11-27T15:24:18.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.924+0000 7f87c215cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8b8bd8a0 con 0x557f8b96a8b0 2023-11-27T15:24:18.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.924+0000 7f87c215cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8b8b6d30 con 0x557f8b966470 2023-11-27T15:24:18.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.924+0000 7f87af7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f8b965b60 0x557f8b965f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.924+0000 7f87af7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f8b965b60 0x557f8b965f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54508/0 (socket says 172.21.15.52:54508) 2023-11-27T15:24:18.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.924+0000 7f87af7fe700 1 -- 172.21.15.52:0/3996250940 learned_addr learned my addr 172.21.15.52:0/3996250940 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:18.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.924+0000 7f87aeffd700 1 --2- 172.21.15.52:0/3996250940 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8b966470 0x557f8b96a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.924+0000 7f87affff700 1 --2- 172.21.15.52:0/3996250940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f8b96a8b0 0x557f8b96ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.924+0000 7f87af7fe700 1 -- 172.21.15.52:0/3996250940 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8b966470 msgr2=0x557f8b96a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.924+0000 7f87af7fe700 1 --2- 172.21.15.52:0/3996250940 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8b966470 0x557f8b96a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.924+0000 7f87af7fe700 1 -- 172.21.15.52:0/3996250940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f8b96a8b0 msgr2=0x557f8b96ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.924+0000 7f87af7fe700 1 --2- 172.21.15.52:0/3996250940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f8b96a8b0 0x557f8b96ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.924+0000 7f87af7fe700 1 -- 172.21.15.52:0/3996250940 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f8b738630 con 0x557f8b965b60 2023-11-27T15:24:18.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.924+0000 7f87aeffd700 1 --2- 172.21.15.52:0/3996250940 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8b966470 0x557f8b96a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:24:18.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.925+0000 7f87af7fe700 1 --2- 172.21.15.52:0/3996250940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f8b965b60 0x557f8b965f30 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f87a000b8c0 tx=0x7f87a0020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221fdd382b6dd7a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.925+0000 7f87a5ffb700 1 -- 172.21.15.52:0/3996250940 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a002b070 con 0x557f8b965b60 2023-11-27T15:24:18.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.925+0000 7f87a5ffb700 1 -- 172.21.15.52:0/3996250940 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87a001e040 con 0x557f8b965b60 2023-11-27T15:24:18.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.925+0000 7f87a5ffb700 1 -- 172.21.15.52:0/3996250940 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a000f860 con 0x557f8b965b60 2023-11-27T15:24:18.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.925+0000 7f87c215cb80 1 -- 172.21.15.52:0/3996250940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f8b965b60 msgr2=0x557f8b965f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.925+0000 7f87c215cb80 1 --2- 172.21.15.52:0/3996250940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f8b965b60 0x557f8b965f30 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f87a000b8c0 tx=0x7f87a0020000 comp rx=0 tx=0).stop 2023-11-27T15:24:18.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.926+0000 7f87c215cb80 1 -- 172.21.15.52:0/3996250940 shutdown_connections 2023-11-27T15:24:18.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.926+0000 7f87c215cb80 1 --2- 172.21.15.52:0/3996250940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f8b96a8b0 0x557f8b96ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.926+0000 7f87c215cb80 1 --2- 172.21.15.52:0/3996250940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f8b965b60 0x557f8b965f30 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.926+0000 7f87c215cb80 1 --2- 172.21.15.52:0/3996250940 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8b966470 0x557f8b96a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.926+0000 7f87c215cb80 1 -- 172.21.15.52:0/3996250940 >> 172.21.15.52:0/3996250940 conn(0x557f8b95b360 msgr2=0x557f8b960b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.926+0000 7f87c215cb80 1 -- 172.21.15.52:0/3996250940 shutdown_connections 2023-11-27T15:24:18.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.926+0000 7f87c215cb80 1 -- 172.21.15.52:0/3996250940 wait complete. 2023-11-27T15:24:18.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.926+0000 7f87c215cb80 1 -- start start 2023-11-27T15:24:18.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.926+0000 7f87c215cb80 1 Processor -- start 2023-11-27T15:24:18.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.926+0000 7f87c215cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8b965b60 0x557f8b971ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.926+0000 7f87c215cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f8b966470 0x557f8b96f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.926+0000 7f87c215cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f8b96a8b0 0x557f8b96f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.926+0000 7f87c215cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8b8588e0 con 0x557f8b96a8b0 2023-11-27T15:24:18.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.926+0000 7f87c215cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8b858580 con 0x557f8b966470 2023-11-27T15:24:18.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.926+0000 7f87c215cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8b73a920 con 0x557f8b965b60 2023-11-27T15:24:18.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.927+0000 7f87c215cb80 10 client.0 _ll_register_callbacks cb 0x557f8b876780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:18.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.927+0000 7f87affff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f8b96a8b0 0x557f8b96f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.927+0000 7f87affff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f8b96a8b0 0x557f8b96f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54518/0 (socket says 172.21.15.52:54518) 2023-11-27T15:24:18.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.927+0000 7f87affff700 1 -- 172.21.15.52:0/2150101953 learned_addr learned my addr 172.21.15.52:0/2150101953 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:18.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.927+0000 7f87af7fe700 1 --2- 172.21.15.52:0/2150101953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8b965b60 0x557f8b971ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.927+0000 7f87aeffd700 1 --2- 172.21.15.52:0/2150101953 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f8b966470 0x557f8b96f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.927+0000 7f87affff700 1 -- 172.21.15.52:0/2150101953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8b965b60 msgr2=0x557f8b971ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.927+0000 7f87affff700 1 --2- 172.21.15.52:0/2150101953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8b965b60 0x557f8b971ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.927+0000 7f87affff700 1 -- 172.21.15.52:0/2150101953 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f8b966470 msgr2=0x557f8b96f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.927+0000 7f87affff700 1 --2- 172.21.15.52:0/2150101953 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f8b966470 0x557f8b96f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.927+0000 7f87affff700 1 -- 172.21.15.52:0/2150101953 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b010d990 con 0x557f8b96a8b0 2023-11-27T15:24:18.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.927+0000 7f87affff700 1 --2- 172.21.15.52:0/2150101953 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f8b96a8b0 0x557f8b96f8a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f87a8007ab0 tx=0x7f87a8007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602626335094ce9c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.927+0000 7f878effd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:24:18.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.927+0000 7f878effd700 1 -- 172.21.15.52:0/2150101953 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a801a070 con 0x557f8b96a8b0 2023-11-27T15:24:18.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.927+0000 7f878effd700 1 -- 172.21.15.52:0/2150101953 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f87a80106e0 con 0x557f8b96a8b0 2023-11-27T15:24:18.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.927+0000 7f87c215cb80 1 -- 172.21.15.52:0/2150101953 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f8b958b20 con 0x557f8b96a8b0 2023-11-27T15:24:18.942 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.927+0000 7f878effd700 1 -- 172.21.15.52:0/2150101953 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87a80108a0 con 0x557f8b96a8b0 2023-11-27T15:24:18.942 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.927+0000 7f87c215cb80 10 client.16455 fetch_fsmap learned FSMap version 28 2023-11-27T15:24:18.942 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.927+0000 7f87c215cb80 1 -- 172.21.15.52:0/2150101953 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557f8b87d190 con 0x557f8b96a8b0 2023-11-27T15:24:18.942 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.928+0000 7f878effd700 1 -- 172.21.15.52:0/2150101953 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557f8b958b20 con 0x557f8b96a8b0 2023-11-27T15:24:18.942 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.928+0000 7f878effd700 1 -- 172.21.15.52:0/2150101953 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f87a801d020 con 0x557f8b96a8b0 2023-11-27T15:24:18.942 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.928+0000 7f87c215cb80 10 client.16455 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:24:18.942 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.928+0000 7f87c215cb80 10 client.16455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:18.943 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.928+0000 7f87c215cb80 20 client.16455 populate_metadata read hostname 'smithi052' 2023-11-27T15:24:18.943 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.928+0000 7f87c215cb80 1 --2- 172.21.15.52:0/2150101953 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557f8b97f7f0 0x557f8b981c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.943 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.928+0000 7f87c215cb80 4 client.16455 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:24:18.943 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.928+0000 7f87c215cb80 1 -- 172.21.15.52:0/2150101953 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f8b8790a0 con 0x557f8b97f7f0 2023-11-27T15:24:18.943 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.928+0000 7f87af7fe700 1 --2- 172.21.15.52:0/2150101953 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557f8b97f7f0 0x557f8b981c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.943 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.928+0000 7f87af7fe700 1 --2- 172.21.15.52:0/2150101953 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557f8b97f7f0 0x557f8b981c00 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24626328dd5b6a server_cookie=9723c83d7fd1c0da in_seq=0 out_seq=0 2023-11-27T15:24:18.943 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.928+0000 7f878effd700 10 client.16455 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:18.943 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.929+0000 7f878effd700 1 -- 172.21.15.52:0/2150101953 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557f8b8790a0 con 0x557f8b97f7f0 2023-11-27T15:24:18.943 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.929+0000 7f878effd700 10 client.16455 handle_command_reply: tid=0 2023-11-27T15:24:18.944 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.929+0000 7f87c215cb80 1 client.16455 shutdown 2023-11-27T15:24:18.944 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.929+0000 7f87c215cb80 1 -- 172.21.15.52:0/2150101953 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f8b96a8b0 msgr2=0x557f8b96f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.944 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.929+0000 7f87c215cb80 1 --2- 172.21.15.52:0/2150101953 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f8b96a8b0 0x557f8b96f8a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f87a8007ab0 tx=0x7f87a8007dc0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.944 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.929+0000 7f87c215cb80 1 -- 172.21.15.52:0/2150101953 shutdown_connections 2023-11-27T15:24:18.944 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.929+0000 7f87c215cb80 1 --2- 172.21.15.52:0/2150101953 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557f8b97f7f0 0x557f8b981c00 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.944 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.929+0000 7f87c215cb80 1 --2- 172.21.15.52:0/2150101953 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f8b96a8b0 0x557f8b96f8a0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.944 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.929+0000 7f87c215cb80 1 --2- 172.21.15.52:0/2150101953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8b965b60 0x557f8b971ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.944 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.929+0000 7f87c215cb80 1 --2- 172.21.15.52:0/2150101953 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f8b966470 0x557f8b96f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.945 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.929+0000 7f878effd700 0 client.16455 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:18.945 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.929+0000 7f87c215cb80 1 -- 172.21.15.52:0/2150101953 >> 172.21.15.52:0/2150101953 conn(0x557f8b95b360 msgr2=0x557f8b96bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.945 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.929+0000 7f87c215cb80 1 -- 172.21.15.52:0/2150101953 shutdown_connections 2023-11-27T15:24:18.945 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.929+0000 7f87c215cb80 1 -- 172.21.15.52:0/2150101953 wait complete. 2023-11-27T15:24:18.945 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.929+0000 7f87c215cb80 20 client.16455 trim_cache size 0 max 16384 2023-11-27T15:24:18.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.942+0000 7f87c215cb80 1 Processor -- start 2023-11-27T15:24:18.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.942+0000 7f87c215cb80 1 -- start start 2023-11-27T15:24:18.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.943+0000 7f87c215cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f8b965b60 0x557f8b9768f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.943+0000 7f87c215cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8b966470 0x557f8b97bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.943+0000 7f87c215cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f8b96a8b0 0x557f8b97b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.943+0000 7f87c215cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8b96fe80 con 0x557f8b965b60 2023-11-27T15:24:18.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.943+0000 7f87c215cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8b96d7d0 con 0x557f8b96a8b0 2023-11-27T15:24:18.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.943+0000 7f87c215cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8b969420 con 0x557f8b966470 2023-11-27T15:24:18.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.943+0000 7f87af7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f8b965b60 0x557f8b9768f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.943+0000 7f87af7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f8b965b60 0x557f8b9768f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54522/0 (socket says 172.21.15.52:54522) 2023-11-27T15:24:18.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.943+0000 7f87af7fe700 1 -- 172.21.15.52:0/3611894032 learned_addr learned my addr 172.21.15.52:0/3611894032 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:18.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.943+0000 7f87affff700 1 --2- 172.21.15.52:0/3611894032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f8b96a8b0 0x557f8b97b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.943+0000 7f87aeffd700 1 --2- 172.21.15.52:0/3611894032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8b966470 0x557f8b97bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.943+0000 7f87af7fe700 1 -- 172.21.15.52:0/3611894032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8b966470 msgr2=0x557f8b97bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.943+0000 7f87af7fe700 1 --2- 172.21.15.52:0/3611894032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8b966470 0x557f8b97bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.943+0000 7f87af7fe700 1 -- 172.21.15.52:0/3611894032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f8b96a8b0 msgr2=0x557f8b97b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.943+0000 7f87af7fe700 1 --2- 172.21.15.52:0/3611894032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f8b96a8b0 0x557f8b97b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.943+0000 7f87af7fe700 1 -- 172.21.15.52:0/3611894032 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f8b87cee0 con 0x557f8b965b60 2023-11-27T15:24:18.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.943+0000 7f87af7fe700 1 --2- 172.21.15.52:0/3611894032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f8b965b60 0x557f8b9768f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f87a0000c00 tx=0x7f87a0004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48929031fa28957e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.943+0000 7f87a67fc700 1 -- 172.21.15.52:0/3611894032 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a0029050 con 0x557f8b965b60 2023-11-27T15:24:18.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.943+0000 7f87a67fc700 1 -- 172.21.15.52:0/3611894032 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87a003a590 con 0x557f8b965b60 2023-11-27T15:24:18.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.943+0000 7f87a67fc700 1 -- 172.21.15.52:0/3611894032 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a002b040 con 0x557f8b965b60 2023-11-27T15:24:18.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.944+0000 7f87c215cb80 1 -- 172.21.15.52:0/3611894032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f8b965b60 msgr2=0x557f8b9768f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.944+0000 7f87c215cb80 1 --2- 172.21.15.52:0/3611894032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f8b965b60 0x557f8b9768f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f87a0000c00 tx=0x7f87a0004fa0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.944+0000 7f87c215cb80 1 -- 172.21.15.52:0/3611894032 shutdown_connections 2023-11-27T15:24:18.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.944+0000 7f87c215cb80 1 --2- 172.21.15.52:0/3611894032 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f8b96a8b0 0x557f8b97b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.944+0000 7f87c215cb80 1 --2- 172.21.15.52:0/3611894032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f8b965b60 0x557f8b9768f0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.944+0000 7f87c215cb80 1 --2- 172.21.15.52:0/3611894032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8b966470 0x557f8b97bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.944+0000 7f87c215cb80 1 -- 172.21.15.52:0/3611894032 >> 172.21.15.52:0/3611894032 conn(0x557f8b95b360 msgr2=0x557f8b96cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.944+0000 7f87c215cb80 1 -- 172.21.15.52:0/3611894032 shutdown_connections 2023-11-27T15:24:18.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.944+0000 7f87c215cb80 1 -- 172.21.15.52:0/3611894032 wait complete. 2023-11-27T15:24:18.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.944+0000 7f87c215cb80 1 -- start start 2023-11-27T15:24:18.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.945+0000 7f87c215cb80 1 Processor -- start 2023-11-27T15:24:18.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.945+0000 7f87c215cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8b965b60 0x557f8b9c7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.945+0000 7f87c215cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f8b966470 0x557f8b9c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.945+0000 7f87c215cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f8b96a8b0 0x557f8b9c8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.945+0000 7f87c215cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8b9d2720 con 0x557f8b966470 2023-11-27T15:24:18.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.945+0000 7f87c215cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8b9d2860 con 0x557f8b96a8b0 2023-11-27T15:24:18.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.945+0000 7f87c215cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8b9d2a20 con 0x557f8b965b60 2023-11-27T15:24:18.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.945+0000 7f87c215cb80 10 client.0 _ll_register_callbacks cb 0x7f87a8009450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:18.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.945+0000 7f87aeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f8b966470 0x557f8b9c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.945+0000 7f87aeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f8b966470 0x557f8b9c7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54530/0 (socket says 172.21.15.52:54530) 2023-11-27T15:24:18.957 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.945+0000 7f87af7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8b965b60 0x557f8b9c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.957 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.945+0000 7f87aeffd700 1 -- 172.21.15.52:0/1061802678 learned_addr learned my addr 172.21.15.52:0/1061802678 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:18.957 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.945+0000 7f87affff700 1 --2- 172.21.15.52:0/1061802678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f8b96a8b0 0x557f8b9c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.957 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.945+0000 7f87aeffd700 1 -- 172.21.15.52:0/1061802678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8b965b60 msgr2=0x557f8b9c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.957 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.945+0000 7f87aeffd700 1 --2- 172.21.15.52:0/1061802678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8b965b60 0x557f8b9c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.957 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.945+0000 7f87aeffd700 1 -- 172.21.15.52:0/1061802678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f8b96a8b0 msgr2=0x557f8b9c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.957 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.945+0000 7f87aeffd700 1 --2- 172.21.15.52:0/1061802678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f8b96a8b0 0x557f8b9c8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.957 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.945+0000 7f87aeffd700 1 -- 172.21.15.52:0/1061802678 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f8b738630 con 0x557f8b966470 2023-11-27T15:24:18.958 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.945+0000 7f87affff700 1 --2- 172.21.15.52:0/1061802678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f8b96a8b0 0x557f8b9c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:24:18.958 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.945+0000 7f87aeffd700 1 --2- 172.21.15.52:0/1061802678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f8b966470 0x557f8b9c7b40 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f8798099d00 tx=0x7f879809ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8462601a9bdd362 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.958 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.945+0000 7f878effd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:24:18.958 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.946+0000 7f878effd700 1 -- 172.21.15.52:0/1061802678 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879800de00 con 0x557f8b966470 2023-11-27T15:24:18.958 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.946+0000 7f878effd700 1 -- 172.21.15.52:0/1061802678 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f879800f700 con 0x557f8b966470 2023-11-27T15:24:18.958 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.946+0000 7f87c215cb80 1 -- 172.21.15.52:0/1061802678 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87a001e040 con 0x557f8b966470 2023-11-27T15:24:18.958 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.946+0000 7f878effd700 1 -- 172.21.15.52:0/1061802678 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f879800f8c0 con 0x557f8b966470 2023-11-27T15:24:18.959 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.946+0000 7f87c215cb80 10 client.16461 fetch_fsmap learned FSMap version 28 2023-11-27T15:24:18.959 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.946+0000 7f87c215cb80 1 -- 172.21.15.52:0/1061802678 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557f8b6cdc60 con 0x557f8b966470 2023-11-27T15:24:18.959 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.946+0000 7f878effd700 1 -- 172.21.15.52:0/1061802678 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87a001e040 con 0x557f8b966470 2023-11-27T15:24:18.959 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.946+0000 7f878effd700 1 -- 172.21.15.52:0/1061802678 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f87980a1d30 con 0x557f8b966470 2023-11-27T15:24:18.959 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.946+0000 7f87c215cb80 10 client.16461 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:24:18.959 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.946+0000 7f87c215cb80 10 client.16461 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:18.959 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.946+0000 7f87c215cb80 20 client.16461 populate_metadata read hostname 'smithi052' 2023-11-27T15:24:18.959 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.946+0000 7f87c215cb80 1 --2- 172.21.15.52:0/1061802678 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557f8b9d4230 0x557f8b9d6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.960 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.946+0000 7f87c215cb80 4 client.16461 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:24:18.960 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.946+0000 7f87c215cb80 1 -- 172.21.15.52:0/1061802678 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557f8b885660 con 0x557f8b9d4230 2023-11-27T15:24:18.960 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.946+0000 7f87af7fe700 1 --2- 172.21.15.52:0/1061802678 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557f8b9d4230 0x557f8b9d6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.960 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.947+0000 7f878effd700 10 client.16461 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:18.960 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.947+0000 7f87af7fe700 1 --2- 172.21.15.52:0/1061802678 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557f8b9d4230 0x557f8b9d6640 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e268e30e1e85866c server_cookie=e08b22841ae53e9e in_seq=0 out_seq=0 2023-11-27T15:24:18.960 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.947+0000 7f878effd700 1 -- 172.21.15.52:0/1061802678 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557f8b885660 con 0x557f8b9d4230 2023-11-27T15:24:18.960 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.947+0000 7f878effd700 10 client.16461 handle_command_reply: tid=0 2023-11-27T15:24:18.960 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.947+0000 7f87c215cb80 1 client.16461 shutdown 2023-11-27T15:24:18.961 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.947+0000 7f87c215cb80 1 -- 172.21.15.52:0/1061802678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f8b966470 msgr2=0x557f8b9c7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.961 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.947+0000 7f87c215cb80 1 --2- 172.21.15.52:0/1061802678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f8b966470 0x557f8b9c7b40 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f8798099d00 tx=0x7f879809ddf0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.961 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.947+0000 7f87c215cb80 1 -- 172.21.15.52:0/1061802678 shutdown_connections 2023-11-27T15:24:18.961 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.947+0000 7f87c215cb80 1 --2- 172.21.15.52:0/1061802678 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557f8b9d4230 0x557f8b9d6640 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.961 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.947+0000 7f87c215cb80 1 --2- 172.21.15.52:0/1061802678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f8b96a8b0 0x557f8b9c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.961 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.947+0000 7f87c215cb80 1 --2- 172.21.15.52:0/1061802678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8b965b60 0x557f8b9c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.961 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.947+0000 7f87c215cb80 1 --2- 172.21.15.52:0/1061802678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f8b966470 0x557f8b9c7b40 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.961 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.947+0000 7f878effd700 0 client.16461 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:18.962 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.947+0000 7f87c215cb80 1 -- 172.21.15.52:0/1061802678 >> 172.21.15.52:0/1061802678 conn(0x557f8b95b360 msgr2=0x557f8b97bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.962 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.947+0000 7f87c215cb80 1 -- 172.21.15.52:0/1061802678 shutdown_connections 2023-11-27T15:24:18.962 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.947+0000 7f87c215cb80 1 -- 172.21.15.52:0/1061802678 wait complete. 2023-11-27T15:24:18.962 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.947+0000 7f87c215cb80 20 client.16461 trim_cache size 0 max 16384 2023-11-27T15:24:18.962 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:24:18.963 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.948+0000 7f87b5fd0700 1 -- 172.21.15.52:0/3159209093 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f879c0a1340 msgr2=0x7f879c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.964 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.948+0000 7f87b5fd0700 1 --2- 172.21.15.52:0/3159209093 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f879c0a1340 0x7f879c0a37f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f87a000b2e0 tx=0x7f87a0005960 comp rx=0 tx=0).stop 2023-11-27T15:24:18.964 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.948+0000 7f87b5fd0700 1 -- 172.21.15.52:0/3159209093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87b0101ca0 msgr2=0x7f87b01968f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.964 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.948+0000 7f87b5fd0700 1 --2- 172.21.15.52:0/3159209093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87b0101ca0 0x7f87b01968f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f879800c9b0 tx=0x7f879800ccc0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.964 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.948+0000 7f87b5fd0700 1 -- 172.21.15.52:0/3159209093 shutdown_connections 2023-11-27T15:24:18.964 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.948+0000 7f87b5fd0700 1 --2- 172.21.15.52:0/3159209093 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f879c0a1340 0x7f879c0a37f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.964 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.948+0000 7f87b5fd0700 1 --2- 172.21.15.52:0/3159209093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87b0101390 0x7f87b01963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.964 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.948+0000 7f87b5fd0700 1 --2- 172.21.15.52:0/3159209093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f87b010a8b0 0x7f87b019ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.948+0000 7f87b5fd0700 1 --2- 172.21.15.52:0/3159209093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87b0101ca0 0x7f87b01968f0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.948+0000 7f87b5fd0700 1 -- 172.21.15.52:0/3159209093 >> 172.21.15.52:0/3159209093 conn(0x7f87b00770b0 msgr2=0x7f87b00fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.948+0000 7f87b5fd0700 1 -- 172.21.15.52:0/3159209093 shutdown_connections 2023-11-27T15:24:18.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:18.948+0000 7f87b5fd0700 1 -- 172.21.15.52:0/3159209093 wait complete. 2023-11-27T15:24:19.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:19 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3343858193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:24:19.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:19 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3343858193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:24:19.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:19 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3343858193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:24:19.966 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-27T15:24:19.966 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-27T15:24:20.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.107+0000 7f92f194e700 1 -- 172.21.15.52:0/3409827080 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92ec0ffc30 msgr2=0x7f92ec100000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:20.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.107+0000 7f92f194e700 1 --2- 172.21.15.52:0/3409827080 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92ec0ffc30 0x7f92ec100000 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0099c0 tx=0x7f92dc009cd0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.107+0000 7f92f194e700 1 -- 172.21.15.52:0/3409827080 shutdown_connections 2023-11-27T15:24:20.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.107+0000 7f92f194e700 1 --2- 172.21.15.52:0/3409827080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92ec104590 0x7f92ec10c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.107+0000 7f92f194e700 1 --2- 172.21.15.52:0/3409827080 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92ec0ffc30 0x7f92ec100000 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.107+0000 7f92f194e700 1 --2- 172.21.15.52:0/3409827080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92ec1005d0 0x7f92ec103e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.107+0000 7f92f194e700 1 -- 172.21.15.52:0/3409827080 >> 172.21.15.52:0/3409827080 conn(0x7f92ec075470 msgr2=0x7f92ec075870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:20.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.107+0000 7f92f194e700 1 -- 172.21.15.52:0/3409827080 shutdown_connections 2023-11-27T15:24:20.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.107+0000 7f92f194e700 1 -- 172.21.15.52:0/3409827080 wait complete. 2023-11-27T15:24:20.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.108+0000 7f92f194e700 1 Processor -- start 2023-11-27T15:24:20.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.108+0000 7f92f194e700 1 -- start start 2023-11-27T15:24:20.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.108+0000 7f92f194e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92ec0ffc30 0x7f92ec197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.108+0000 7f92f194e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92ec1005d0 0x7f92ec197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.108+0000 7f92f194e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92ec104590 0x7f92ec1a22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.108+0000 7f92f194e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec1980d0 con 0x7f92ec104590 2023-11-27T15:24:20.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.108+0000 7f92f194e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec1a2810 con 0x7f92ec1005d0 2023-11-27T15:24:20.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.108+0000 7f92f194e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec1a29d0 con 0x7f92ec0ffc30 2023-11-27T15:24:20.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.108+0000 7f92eb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92ec104590 0x7f92ec1a22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.108+0000 7f92eb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92ec104590 0x7f92ec1a22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54544/0 (socket says 172.21.15.52:54544) 2023-11-27T15:24:20.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.108+0000 7f92eb7fe700 1 -- 172.21.15.52:0/2316425133 learned_addr learned my addr 172.21.15.52:0/2316425133 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:20.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.108+0000 7f92ea7fc700 1 --2- 172.21.15.52:0/2316425133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92ec1005d0 0x7f92ec197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.109+0000 7f92eaffd700 1 --2- 172.21.15.52:0/2316425133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92ec0ffc30 0x7f92ec197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.109+0000 7f92eb7fe700 1 -- 172.21.15.52:0/2316425133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92ec0ffc30 msgr2=0x7f92ec197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:20.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.109+0000 7f92eb7fe700 1 --2- 172.21.15.52:0/2316425133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92ec0ffc30 0x7f92ec197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.109+0000 7f92eb7fe700 1 -- 172.21.15.52:0/2316425133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92ec1005d0 msgr2=0x7f92ec197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:20.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.109+0000 7f92eb7fe700 1 --2- 172.21.15.52:0/2316425133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92ec1005d0 0x7f92ec197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.109+0000 7f92eb7fe700 1 -- 172.21.15.52:0/2316425133 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92dc0095f0 con 0x7f92ec104590 2023-11-27T15:24:20.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.109+0000 7f92eaffd700 1 --2- 172.21.15.52:0/2316425133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92ec0ffc30 0x7f92ec197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:24:20.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.109+0000 7f92eb7fe700 1 --2- 172.21.15.52:0/2316425133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92ec104590 0x7f92ec1a22d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f92e4007ee0 tx=0x7f92e400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:20.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.109+0000 7f92f094c700 1 -- 172.21.15.52:0/2316425133 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e4017070 con 0x7f92ec104590 2023-11-27T15:24:20.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.109+0000 7f92f094c700 1 -- 172.21.15.52:0/2316425133 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92e400f040 con 0x7f92ec104590 2023-11-27T15:24:20.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.109+0000 7f92f094c700 1 -- 172.21.15.52:0/2316425133 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e4005740 con 0x7f92ec104590 2023-11-27T15:24:20.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.109+0000 7f92f194e700 1 -- 172.21.15.52:0/2316425133 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92ec10f300 con 0x7f92ec104590 2023-11-27T15:24:20.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.109+0000 7f92f194e700 1 -- 172.21.15.52:0/2316425133 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92ec10f170 con 0x7f92ec104590 2023-11-27T15:24:20.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.111+0000 7f92f094c700 1 -- 172.21.15.52:0/2316425133 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f92e400dc60 con 0x7f92ec104590 2023-11-27T15:24:20.125 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.112+0000 7f92f094c700 1 --2- 172.21.15.52:0/2316425133 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f92d40a1340 0x7f92d40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.125 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.112+0000 7f92f094c700 1 -- 172.21.15.52:0/2316425133 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f92e40cc420 con 0x7f92ec104590 2023-11-27T15:24:20.125 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.112+0000 7f92eaffd700 1 --2- 172.21.15.52:0/2316425133 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f92d40a1340 0x7f92d40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.126 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.113+0000 7f92eaffd700 1 --2- 172.21.15.52:0/2316425133 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f92d40a1340 0x7f92d40a37f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009fd0 tx=0x7f92dc005fb0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:20.126 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.113+0000 7f92fdadab80 1 Processor -- start 2023-11-27T15:24:20.126 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.113+0000 7f92fdadab80 1 -- start start 2023-11-27T15:24:20.126 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.113+0000 7f92fdadab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555849ce0ff0 0x555849ce13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.126 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.113+0000 7f92fdadab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555849ce1900 0x555849ce5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.126 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.113+0000 7f92fdadab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555849ce5d40 0x555849ce6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.126 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.113+0000 7f92fdadab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x555849be20e0 con 0x555849ce0ff0 2023-11-27T15:24:20.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.113+0000 7f92fdadab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555849c34920 con 0x555849ce5d40 2023-11-27T15:24:20.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.113+0000 7f92fdadab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555849c2ddb0 con 0x555849ce1900 2023-11-27T15:24:20.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.113+0000 7f92eaffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555849ce0ff0 0x555849ce13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.113+0000 7f92eaffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555849ce0ff0 0x555849ce13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54550/0 (socket says 172.21.15.52:54550) 2023-11-27T15:24:20.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.113+0000 7f92eaffd700 1 -- 172.21.15.52:0/1592742386 learned_addr learned my addr 172.21.15.52:0/1592742386 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:20.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.113+0000 7f92eb7fe700 1 --2- 172.21.15.52:0/1592742386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555849ce5d40 0x555849ce6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.114+0000 7f92ea7fc700 1 --2- 172.21.15.52:0/1592742386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555849ce1900 0x555849ce5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.114+0000 7f92eaffd700 1 -- 172.21.15.52:0/1592742386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555849ce1900 msgr2=0x555849ce5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:20.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.114+0000 7f92eaffd700 1 --2- 172.21.15.52:0/1592742386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555849ce1900 0x555849ce5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.114+0000 7f92eaffd700 1 -- 172.21.15.52:0/1592742386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555849ce5d40 msgr2=0x555849ce6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:20.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.114+0000 7f92eaffd700 1 --2- 172.21.15.52:0/1592742386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555849ce5d40 0x555849ce6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.114+0000 7f92eaffd700 1 -- 172.21.15.52:0/1592742386 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555849aaf6b0 con 0x555849ce0ff0 2023-11-27T15:24:20.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.114+0000 7f92eaffd700 1 --2- 172.21.15.52:0/1592742386 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555849ce0ff0 0x555849ce13c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0099c0 tx=0x7f92dc01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5479ace4aedd674 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:20.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.114+0000 7f92d1ffb700 1 -- 172.21.15.52:0/1592742386 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc02b070 con 0x555849ce0ff0 2023-11-27T15:24:20.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.114+0000 7f92d1ffb700 1 -- 172.21.15.52:0/1592742386 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92dc01e040 con 0x555849ce0ff0 2023-11-27T15:24:20.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.114+0000 7f92d1ffb700 1 -- 172.21.15.52:0/1592742386 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc004be0 con 0x555849ce0ff0 2023-11-27T15:24:20.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.115+0000 7f92fdadab80 1 -- 172.21.15.52:0/1592742386 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555849ce0ff0 msgr2=0x555849ce13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:20.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.115+0000 7f92fdadab80 1 --2- 172.21.15.52:0/1592742386 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555849ce0ff0 0x555849ce13c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0099c0 tx=0x7f92dc01f000 comp rx=0 tx=0).stop 2023-11-27T15:24:20.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.115+0000 7f92fdadab80 1 -- 172.21.15.52:0/1592742386 shutdown_connections 2023-11-27T15:24:20.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.115+0000 7f92fdadab80 1 --2- 172.21.15.52:0/1592742386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555849ce5d40 0x555849ce6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.115+0000 7f92fdadab80 1 --2- 172.21.15.52:0/1592742386 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555849ce0ff0 0x555849ce13c0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.115+0000 7f92fdadab80 1 --2- 172.21.15.52:0/1592742386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555849ce1900 0x555849ce5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.115+0000 7f92fdadab80 1 -- 172.21.15.52:0/1592742386 >> 172.21.15.52:0/1592742386 conn(0x555849cd23e0 msgr2=0x555849cd7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:20.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.115+0000 7f92fdadab80 1 -- 172.21.15.52:0/1592742386 shutdown_connections 2023-11-27T15:24:20.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.115+0000 7f92fdadab80 1 -- 172.21.15.52:0/1592742386 wait complete. 2023-11-27T15:24:20.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.115+0000 7f92fdadab80 1 -- start start 2023-11-27T15:24:20.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.115+0000 7f92fdadab80 1 Processor -- start 2023-11-27T15:24:20.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.116+0000 7f92fdadab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555849ce0ff0 0x555849ced400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.116+0000 7f92fdadab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555849ce1900 0x555849cea6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.116+0000 7f92fdadab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555849ce5d40 0x555849ceac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.116+0000 7f92fdadab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x555849bcf960 con 0x555849ce0ff0 2023-11-27T15:24:20.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.116+0000 7f92fdadab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555849bcf600 con 0x555849ce5d40 2023-11-27T15:24:20.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.116+0000 7f92fdadab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555849ab19a0 con 0x555849ce1900 2023-11-27T15:24:20.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.116+0000 7f92fdadab80 10 client.0 _ll_register_callbacks cb 0x555849bed800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:20.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.116+0000 7f92eaffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555849ce0ff0 0x555849ced400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.116+0000 7f92eaffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555849ce0ff0 0x555849ced400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54556/0 (socket says 172.21.15.52:54556) 2023-11-27T15:24:20.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.116+0000 7f92eaffd700 1 -- 172.21.15.52:0/2558002546 learned_addr learned my addr 172.21.15.52:0/2558002546 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:20.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.116+0000 7f92ea7fc700 1 --2- 172.21.15.52:0/2558002546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555849ce1900 0x555849cea6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.116+0000 7f92eb7fe700 1 --2- 172.21.15.52:0/2558002546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555849ce5d40 0x555849ceac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.116+0000 7f92eaffd700 1 -- 172.21.15.52:0/2558002546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555849ce1900 msgr2=0x555849cea6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:20.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.116+0000 7f92eaffd700 1 --2- 172.21.15.52:0/2558002546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555849ce1900 0x555849cea6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.116+0000 7f92eaffd700 1 -- 172.21.15.52:0/2558002546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555849ce5d40 msgr2=0x555849ceac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:20.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.116+0000 7f92eaffd700 1 --2- 172.21.15.52:0/2558002546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555849ce5d40 0x555849ceac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.116+0000 7f92eaffd700 1 -- 172.21.15.52:0/2558002546 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92ec10c950 con 0x555849ce0ff0 2023-11-27T15:24:20.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.116+0000 7f92eb7fe700 1 --2- 172.21.15.52:0/2558002546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555849ce5d40 0x555849ceac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:24:20.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.116+0000 7f92ea7fc700 1 --2- 172.21.15.52:0/2558002546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555849ce1900 0x555849cea6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:24:20.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.116+0000 7f92eaffd700 1 --2- 172.21.15.52:0/2558002546 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555849ce0ff0 0x555849ced400 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009990 tx=0x7f92dc0053b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b712600b5278b08 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:20.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.116+0000 7f92caffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:24:20.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.117+0000 7f92caffd700 1 -- 172.21.15.52:0/2558002546 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc02b030 con 0x555849ce0ff0 2023-11-27T15:24:20.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.117+0000 7f92caffd700 1 -- 172.21.15.52:0/2558002546 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f92dc02c070 con 0x555849ce0ff0 2023-11-27T15:24:20.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.117+0000 7f92fdadab80 1 -- 172.21.15.52:0/2558002546 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555849ccfba0 con 0x555849ce0ff0 2023-11-27T15:24:20.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.117+0000 7f92caffd700 1 -- 172.21.15.52:0/2558002546 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92dc004ea0 con 0x555849ce0ff0 2023-11-27T15:24:20.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.117+0000 7f92fdadab80 10 client.16473 fetch_fsmap learned FSMap version 28 2023-11-27T15:24:20.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.117+0000 7f92fdadab80 1 -- 172.21.15.52:0/2558002546 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555849bf4210 con 0x555849ce0ff0 2023-11-27T15:24:20.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.117+0000 7f92caffd700 1 -- 172.21.15.52:0/2558002546 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555849ccfba0 con 0x555849ce0ff0 2023-11-27T15:24:20.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.117+0000 7f92caffd700 1 -- 172.21.15.52:0/2558002546 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f92dc030580 con 0x555849ce0ff0 2023-11-27T15:24:20.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.117+0000 7f92fdadab80 10 client.16473 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:24:20.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.117+0000 7f92fdadab80 10 client.16473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:20.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.117+0000 7f92fdadab80 20 client.16473 populate_metadata read hostname 'smithi052' 2023-11-27T15:24:20.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.117+0000 7f92fdadab80 1 --2- 172.21.15.52:0/2558002546 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x555849cfa890 0x555849cfcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.117+0000 7f92fdadab80 4 client.16473 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:24:20.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.117+0000 7f92fdadab80 1 -- 172.21.15.52:0/2558002546 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555849bf0120 con 0x555849cfa890 2023-11-27T15:24:20.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.117+0000 7f92ea7fc700 1 --2- 172.21.15.52:0/2558002546 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x555849cfa890 0x555849cfcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.118+0000 7f92ea7fc700 1 --2- 172.21.15.52:0/2558002546 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x555849cfa890 0x555849cfcca0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a4eaa54dbe81ecd server_cookie=564a75d88bd53931 in_seq=0 out_seq=0 2023-11-27T15:24:20.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.118+0000 7f92caffd700 10 client.16473 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:20.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.119+0000 7f92caffd700 1 -- 172.21.15.52:0/2558002546 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555849bf0120 con 0x555849cfa890 2023-11-27T15:24:20.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.119+0000 7f92caffd700 10 client.16473 handle_command_reply: tid=0 2023-11-27T15:24:20.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.119+0000 7f92fdadab80 1 client.16473 shutdown 2023-11-27T15:24:20.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.119+0000 7f92fdadab80 1 -- 172.21.15.52:0/2558002546 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555849ce0ff0 msgr2=0x555849ced400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:20.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.119+0000 7f92fdadab80 1 --2- 172.21.15.52:0/2558002546 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555849ce0ff0 0x555849ced400 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009990 tx=0x7f92dc0053b0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.119+0000 7f92fdadab80 1 -- 172.21.15.52:0/2558002546 shutdown_connections 2023-11-27T15:24:20.137 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.119+0000 7f92fdadab80 1 --2- 172.21.15.52:0/2558002546 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x555849cfa890 0x555849cfcca0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.137 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.119+0000 7f92fdadab80 1 --2- 172.21.15.52:0/2558002546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555849ce5d40 0x555849ceac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.137 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.119+0000 7f92caffd700 0 client.16473 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:20.137 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.119+0000 7f92fdadab80 1 --2- 172.21.15.52:0/2558002546 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555849ce0ff0 0x555849ced400 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.137 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.119+0000 7f92fdadab80 1 --2- 172.21.15.52:0/2558002546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555849ce1900 0x555849cea6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.137 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.119+0000 7f92fdadab80 1 -- 172.21.15.52:0/2558002546 >> 172.21.15.52:0/2558002546 conn(0x555849cd23e0 msgr2=0x555849ce71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:20.137 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.119+0000 7f92fdadab80 1 -- 172.21.15.52:0/2558002546 shutdown_connections 2023-11-27T15:24:20.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.119+0000 7f92fdadab80 1 -- 172.21.15.52:0/2558002546 wait complete. 2023-11-27T15:24:20.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.119+0000 7f92fdadab80 20 client.16473 trim_cache size 0 max 16384 2023-11-27T15:24:20.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.136+0000 7f92fdadab80 1 Processor -- start 2023-11-27T15:24:20.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.136+0000 7f92fdadab80 1 -- start start 2023-11-27T15:24:20.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.136+0000 7f92fdadab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555849ce0ff0 0x555849ce3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.136+0000 7f92fdadab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555849ce1900 0x555849cf1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.136+0000 7f92fdadab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555849ce5d40 0x555849cecfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.136+0000 7f92fdadab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x555849ccfa70 con 0x555849ce5d40 2023-11-27T15:24:20.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.136+0000 7f92fdadab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555849cf7630 con 0x555849ce1900 2023-11-27T15:24:20.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.136+0000 7f92fdadab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555849cf2370 con 0x555849ce0ff0 2023-11-27T15:24:20.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.136+0000 7f92eb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555849ce5d40 0x555849cecfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.136+0000 7f92eb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555849ce5d40 0x555849cecfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54568/0 (socket says 172.21.15.52:54568) 2023-11-27T15:24:20.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.136+0000 7f92eb7fe700 1 -- 172.21.15.52:0/1458633901 learned_addr learned my addr 172.21.15.52:0/1458633901 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:20.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.137+0000 7f92ea7fc700 1 --2- 172.21.15.52:0/1458633901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555849ce1900 0x555849cf1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.137+0000 7f92eaffd700 1 --2- 172.21.15.52:0/1458633901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555849ce0ff0 0x555849ce3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.137+0000 7f92eb7fe700 1 -- 172.21.15.52:0/1458633901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555849ce0ff0 msgr2=0x555849ce3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:20.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.137+0000 7f92eb7fe700 1 --2- 172.21.15.52:0/1458633901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555849ce0ff0 0x555849ce3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.137+0000 7f92eb7fe700 1 -- 172.21.15.52:0/1458633901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555849ce1900 msgr2=0x555849cf1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:20.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.137+0000 7f92eb7fe700 1 --2- 172.21.15.52:0/1458633901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555849ce1900 0x555849cf1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.137+0000 7f92eb7fe700 1 -- 172.21.15.52:0/1458633901 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555849bf3f60 con 0x555849ce5d40 2023-11-27T15:24:20.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.137+0000 7f92eb7fe700 1 --2- 172.21.15.52:0/1458633901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555849ce5d40 0x555849cecfa0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f92e40c9bf0 tx=0x7f92e400dbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ec900bb019d8b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:20.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.137+0000 7f92d27fc700 1 -- 172.21.15.52:0/1458633901 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e40994c0 con 0x555849ce5d40 2023-11-27T15:24:20.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.137+0000 7f92d27fc700 1 -- 172.21.15.52:0/1458633901 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92e409d4f0 con 0x555849ce5d40 2023-11-27T15:24:20.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.137+0000 7f92d27fc700 1 -- 172.21.15.52:0/1458633901 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e4005b00 con 0x555849ce5d40 2023-11-27T15:24:20.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.138+0000 7f92fdadab80 1 -- 172.21.15.52:0/1458633901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555849ce5d40 msgr2=0x555849cecfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:20.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.138+0000 7f92fdadab80 1 --2- 172.21.15.52:0/1458633901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555849ce5d40 0x555849cecfa0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f92e40c9bf0 tx=0x7f92e400dbd0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.138+0000 7f92fdadab80 1 -- 172.21.15.52:0/1458633901 shutdown_connections 2023-11-27T15:24:20.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.138+0000 7f92fdadab80 1 --2- 172.21.15.52:0/1458633901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555849ce5d40 0x555849cecfa0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.138+0000 7f92fdadab80 1 --2- 172.21.15.52:0/1458633901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555849ce0ff0 0x555849ce3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.138+0000 7f92fdadab80 1 --2- 172.21.15.52:0/1458633901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555849ce1900 0x555849cf1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.146 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.138+0000 7f92fdadab80 1 -- 172.21.15.52:0/1458633901 >> 172.21.15.52:0/1458633901 conn(0x555849cd23e0 msgr2=0x555849ce3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:20.146 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.138+0000 7f92fdadab80 1 -- 172.21.15.52:0/1458633901 shutdown_connections 2023-11-27T15:24:20.146 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.138+0000 7f92fdadab80 1 -- 172.21.15.52:0/1458633901 wait complete. 2023-11-27T15:24:20.146 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.138+0000 7f92fdadab80 1 -- start start 2023-11-27T15:24:20.146 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.138+0000 7f92fdadab80 1 Processor -- start 2023-11-27T15:24:20.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.139+0000 7f92fdadab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555849ce0ff0 0x555849cf1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.139+0000 7f92fdadab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555849ce1900 0x555849cf0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.139+0000 7f92fdadab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555849ce5d40 0x555849cf0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.139+0000 7f92fdadab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x555849cf0df0 con 0x555849ce0ff0 2023-11-27T15:24:20.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.139+0000 7f92fdadab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555849d4e720 con 0x555849ce5d40 2023-11-27T15:24:20.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.139+0000 7f92fdadab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555849d4e860 con 0x555849ce1900 2023-11-27T15:24:20.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.139+0000 7f92fdadab80 10 client.0 _ll_register_callbacks cb 0x7f92e4095140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:20.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.139+0000 7f92eaffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555849ce0ff0 0x555849cf1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.139+0000 7f92eaffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555849ce0ff0 0x555849cf1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54570/0 (socket says 172.21.15.52:54570) 2023-11-27T15:24:20.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.139+0000 7f92eaffd700 1 -- 172.21.15.52:0/945893413 learned_addr learned my addr 172.21.15.52:0/945893413 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:20.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.139+0000 7f92eb7fe700 1 --2- 172.21.15.52:0/945893413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555849ce5d40 0x555849cf0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.139+0000 7f92ea7fc700 1 --2- 172.21.15.52:0/945893413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555849ce1900 0x555849cf0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.139+0000 7f92eaffd700 1 -- 172.21.15.52:0/945893413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555849ce1900 msgr2=0x555849cf0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:20.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.139+0000 7f92eaffd700 1 --2- 172.21.15.52:0/945893413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555849ce1900 0x555849cf0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.139+0000 7f92eaffd700 1 -- 172.21.15.52:0/945893413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555849ce5d40 msgr2=0x555849cf0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:20.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.139+0000 7f92eaffd700 1 --2- 172.21.15.52:0/945893413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555849ce5d40 0x555849cf0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.139+0000 7f92eaffd700 1 -- 172.21.15.52:0/945893413 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92ec10f170 con 0x555849ce0ff0 2023-11-27T15:24:20.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.139+0000 7f92ea7fc700 1 --2- 172.21.15.52:0/945893413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555849ce1900 0x555849cf0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:24:20.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.139+0000 7f92eaffd700 1 --2- 172.21.15.52:0/945893413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555849ce0ff0 0x555849cf1d90 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f92dc005f50 tx=0x7f92dc0054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1433cd22592d450 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:20.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.139+0000 7f92caffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:24:20.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.139+0000 7f92caffd700 1 -- 172.21.15.52:0/945893413 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc02b040 con 0x555849ce0ff0 2023-11-27T15:24:20.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.139+0000 7f92caffd700 1 -- 172.21.15.52:0/945893413 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f92dc004e90 con 0x555849ce0ff0 2023-11-27T15:24:20.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.139+0000 7f92fdadab80 1 -- 172.21.15.52:0/945893413 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92dc01e040 con 0x555849ce0ff0 2023-11-27T15:24:20.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.140+0000 7f92caffd700 1 -- 172.21.15.52:0/945893413 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92dc030660 con 0x555849ce0ff0 2023-11-27T15:24:20.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.140+0000 7f92fdadab80 10 client.16479 fetch_fsmap learned FSMap version 28 2023-11-27T15:24:20.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.140+0000 7f92fdadab80 1 -- 172.21.15.52:0/945893413 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555849a44ce0 con 0x555849ce0ff0 2023-11-27T15:24:20.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.140+0000 7f92caffd700 1 -- 172.21.15.52:0/945893413 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92dc01e040 con 0x555849ce0ff0 2023-11-27T15:24:20.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.140+0000 7f92caffd700 1 -- 172.21.15.52:0/945893413 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f92dc044430 con 0x555849ce0ff0 2023-11-27T15:24:20.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.140+0000 7f92fdadab80 10 client.16479 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:24:20.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.140+0000 7f92fdadab80 10 client.16479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:20.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.140+0000 7f92fdadab80 20 client.16479 populate_metadata read hostname 'smithi052' 2023-11-27T15:24:20.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.140+0000 7f92fdadab80 1 --2- 172.21.15.52:0/945893413 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x555849cfa890 0x555849d51b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.140+0000 7f92fdadab80 4 client.16479 mds_command: new command op to 24536 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-27T15:24:20.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.140+0000 7f92fdadab80 1 -- 172.21.15.52:0/945893413 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555849bfc6e0 con 0x555849cfa890 2023-11-27T15:24:20.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.140+0000 7f92ea7fc700 1 --2- 172.21.15.52:0/945893413 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x555849cfa890 0x555849d51b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.141+0000 7f92caffd700 10 client.16479 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:20.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.141+0000 7f92ea7fc700 1 --2- 172.21.15.52:0/945893413 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x555849cfa890 0x555849d51b50 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b17297d8eed83ce server_cookie=5cb1df92d0d43f2c in_seq=0 out_seq=0 2023-11-27T15:24:20.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.142+0000 7f92caffd700 1 -- 172.21.15.52:0/945893413 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555849bfc6e0 con 0x555849cfa890 2023-11-27T15:24:20.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.142+0000 7f92caffd700 10 client.16479 handle_command_reply: tid=0 2023-11-27T15:24:20.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.142+0000 7f92fdadab80 1 client.16479 shutdown 2023-11-27T15:24:20.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.142+0000 7f92fdadab80 1 -- 172.21.15.52:0/945893413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555849ce0ff0 msgr2=0x555849cf1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:20.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.142+0000 7f92fdadab80 1 --2- 172.21.15.52:0/945893413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555849ce0ff0 0x555849cf1d90 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f92dc005f50 tx=0x7f92dc0054d0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.142+0000 7f92fdadab80 1 -- 172.21.15.52:0/945893413 shutdown_connections 2023-11-27T15:24:20.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.142+0000 7f92fdadab80 1 --2- 172.21.15.52:0/945893413 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x555849cfa890 0x555849d51b50 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.142+0000 7f92fdadab80 1 --2- 172.21.15.52:0/945893413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555849ce5d40 0x555849cf0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.142+0000 7f92caffd700 0 client.16479 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:20.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.142+0000 7f92fdadab80 1 --2- 172.21.15.52:0/945893413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555849ce0ff0 0x555849cf1d90 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.142+0000 7f92fdadab80 1 --2- 172.21.15.52:0/945893413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555849ce1900 0x555849cf0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.142+0000 7f92fdadab80 1 -- 172.21.15.52:0/945893413 >> 172.21.15.52:0/945893413 conn(0x555849cd23e0 msgr2=0x555849cdb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:20.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.142+0000 7f92fdadab80 1 -- 172.21.15.52:0/945893413 shutdown_connections 2023-11-27T15:24:20.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.142+0000 7f92fdadab80 1 -- 172.21.15.52:0/945893413 wait complete. 2023-11-27T15:24:20.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.142+0000 7f92fdadab80 20 client.16479 trim_cache size 0 max 16384 2023-11-27T15:24:20.157 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:24:20.158 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2023-11-27T15:24:20.158 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "cbc22cb2-af74-4ea2-822e-d40d2a21058b", 2023-11-27T15:24:20.158 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2023-11-27T15:24:20.158 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:24:20.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.143+0000 7f92f194e700 1 -- 172.21.15.52:0/2316425133 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f92d40a1340 msgr2=0x7f92d40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:20.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.143+0000 7f92f194e700 1 --2- 172.21.15.52:0/2316425133 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f92d40a1340 0x7f92d40a37f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009fd0 tx=0x7f92dc005fb0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.143+0000 7f92f194e700 1 -- 172.21.15.52:0/2316425133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92ec104590 msgr2=0x7f92ec1a22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:20.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.143+0000 7f92f194e700 1 --2- 172.21.15.52:0/2316425133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92ec104590 0x7f92ec1a22d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f92e4007ee0 tx=0x7f92e400d3b0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.143+0000 7f92f194e700 1 -- 172.21.15.52:0/2316425133 shutdown_connections 2023-11-27T15:24:20.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.143+0000 7f92f194e700 1 --2- 172.21.15.52:0/2316425133 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f92d40a1340 0x7f92d40a37f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.143+0000 7f92f194e700 1 --2- 172.21.15.52:0/2316425133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92ec104590 0x7f92ec1a22d0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.143+0000 7f92f194e700 1 --2- 172.21.15.52:0/2316425133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92ec0ffc30 0x7f92ec197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.143+0000 7f92f194e700 1 --2- 172.21.15.52:0/2316425133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92ec1005d0 0x7f92ec197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.143+0000 7f92f194e700 1 -- 172.21.15.52:0/2316425133 >> 172.21.15.52:0/2316425133 conn(0x7f92ec075470 msgr2=0x7f92ec0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:20.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.143+0000 7f92f194e700 1 -- 172.21.15.52:0/2316425133 shutdown_connections 2023-11-27T15:24:20.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.143+0000 7f92f194e700 1 -- 172.21.15.52:0/2316425133 wait complete. 2023-11-27T15:24:20.162 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:24:20.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.287+0000 7f170ae7e700 1 -- 172.21.15.52:0/2771780062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f170410a8d0 msgr2=0x7f170410cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:20.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.287+0000 7f170ae7e700 1 --2- 172.21.15.52:0/2771780062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f170410a8d0 0x7f170410cd40 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00b0a0 tx=0x7f16fc00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.287+0000 7f170ae7e700 1 -- 172.21.15.52:0/2771780062 shutdown_connections 2023-11-27T15:24:20.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.287+0000 7f170ae7e700 1 --2- 172.21.15.52:0/2771780062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f170410a8d0 0x7f170410cd40 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.287+0000 7f170ae7e700 1 --2- 172.21.15.52:0/2771780062 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1704101590 0x7f1704101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.287+0000 7f170ae7e700 1 --2- 172.21.15.52:0/2771780062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1704101ea0 0x7f170410a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.287+0000 7f170ae7e700 1 -- 172.21.15.52:0/2771780062 >> 172.21.15.52:0/2771780062 conn(0x7f1704076070 msgr2=0x7f1704076470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:20.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.287+0000 7f170ae7e700 1 -- 172.21.15.52:0/2771780062 shutdown_connections 2023-11-27T15:24:20.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.287+0000 7f170ae7e700 1 -- 172.21.15.52:0/2771780062 wait complete. 2023-11-27T15:24:20.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.287+0000 7f170ae7e700 1 Processor -- start 2023-11-27T15:24:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.287+0000 7f170ae7e700 1 -- start start 2023-11-27T15:24:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.287+0000 7f170ae7e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1704101590 0x7f1704196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.288+0000 7f170ae7e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1704101ea0 0x7f17041969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.288+0000 7f170ae7e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f170410a8d0 0x7f170419ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.288+0000 7f170ae7e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170419b460 con 0x7f170410a8d0 2023-11-27T15:24:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.288+0000 7f170ae7e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170419b5d0 con 0x7f1704101ea0 2023-11-27T15:24:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.288+0000 7f170ae7e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170419b790 con 0x7f1704101590 2023-11-27T15:24:20.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.288+0000 7f170941b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f170410a8d0 0x7f170419ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.288+0000 7f170941b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f170410a8d0 0x7f170419ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54588/0 (socket says 172.21.15.52:54588) 2023-11-27T15:24:20.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.288+0000 7f170941b700 1 -- 172.21.15.52:0/2770401087 learned_addr learned my addr 172.21.15.52:0/2770401087 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:20.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.288+0000 7f1708c1a700 1 --2- 172.21.15.52:0/2770401087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1704101590 0x7f1704196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.288+0000 7f1703fff700 1 --2- 172.21.15.52:0/2770401087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1704101ea0 0x7f17041969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.288+0000 7f170941b700 1 -- 172.21.15.52:0/2770401087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1704101590 msgr2=0x7f1704196480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:20.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.288+0000 7f170941b700 1 --2- 172.21.15.52:0/2770401087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1704101590 0x7f1704196480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.288+0000 7f170941b700 1 -- 172.21.15.52:0/2770401087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1704101ea0 msgr2=0x7f17041969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:20.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.288+0000 7f170941b700 1 --2- 172.21.15.52:0/2770401087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1704101ea0 0x7f17041969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.288+0000 7f170941b700 1 -- 172.21.15.52:0/2770401087 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16fc009d00 con 0x7f170410a8d0 2023-11-27T15:24:20.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.288+0000 7f170941b700 1 --2- 172.21.15.52:0/2770401087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f170410a8d0 0x7f170419ae20 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00b780 tx=0x7f16fc00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:20.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.288+0000 7f1701ffb700 1 -- 172.21.15.52:0/2770401087 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc00e040 con 0x7f170410a8d0 2023-11-27T15:24:20.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.288+0000 7f1701ffb700 1 -- 172.21.15.52:0/2770401087 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16fc0094a0 con 0x7f170410a8d0 2023-11-27T15:24:20.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.288+0000 7f170ae7e700 1 -- 172.21.15.52:0/2770401087 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1704110370 con 0x7f170410a8d0 2023-11-27T15:24:20.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.288+0000 7f1701ffb700 1 -- 172.21.15.52:0/2770401087 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0052a0 con 0x7f170410a8d0 2023-11-27T15:24:20.303 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.289+0000 7f170ae7e700 1 -- 172.21.15.52:0/2770401087 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17041101e0 con 0x7f170410a8d0 2023-11-27T15:24:20.303 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.291+0000 7f1701ffb700 1 -- 172.21.15.52:0/2770401087 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f16fc01d470 con 0x7f170410a8d0 2023-11-27T15:24:20.303 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.291+0000 7f1701ffb700 1 --2- 172.21.15.52:0/2770401087 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f16f40a12c0 0x7f16f40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.303 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.292+0000 7f1701ffb700 1 -- 172.21.15.52:0/2770401087 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f16fc0ce290 con 0x7f170410a8d0 2023-11-27T15:24:20.303 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.292+0000 7f1708c1a700 1 --2- 172.21.15.52:0/2770401087 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f16f40a12c0 0x7f16f40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.303 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.292+0000 7f1708c1a700 1 --2- 172.21.15.52:0/2770401087 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f16f40a12c0 0x7f16f40a3770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f16f8005950 tx=0x7f16f800b410 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:20.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.292+0000 7f171700ab80 1 Processor -- start 2023-11-27T15:24:20.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.292+0000 7f171700ab80 1 -- start start 2023-11-27T15:24:20.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.292+0000 7f171700ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55600effbb60 0x55600effbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.292+0000 7f171700ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55600effc470 0x55600f000370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.292+0000 7f171700ab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55600f0008b0 0x55600f000d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.292+0000 7f171700ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600ef01060 con 0x55600effbb60 2023-11-27T15:24:20.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.292+0000 7f171700ab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600ef538a0 con 0x55600f0008b0 2023-11-27T15:24:20.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.293+0000 7f171700ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600ef4cd30 con 0x55600effc470 2023-11-27T15:24:20.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.293+0000 7f1708c1a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55600effbb60 0x55600effbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.293+0000 7f1708c1a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55600effbb60 0x55600effbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54592/0 (socket says 172.21.15.52:54592) 2023-11-27T15:24:20.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.293+0000 7f1708c1a700 1 -- 172.21.15.52:0/671095010 learned_addr learned my addr 172.21.15.52:0/671095010 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:20.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.293+0000 7f170941b700 1 --2- 172.21.15.52:0/671095010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55600f0008b0 0x55600f000d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.293+0000 7f1708c1a700 1 -- 172.21.15.52:0/671095010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55600effc470 msgr2=0x55600f000370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:20.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.293+0000 7f1703fff700 1 --2- 172.21.15.52:0/671095010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55600effc470 0x55600f000370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.293+0000 7f1708c1a700 1 --2- 172.21.15.52:0/671095010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55600effc470 0x55600f000370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.293+0000 7f1708c1a700 1 -- 172.21.15.52:0/671095010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55600f0008b0 msgr2=0x55600f000d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:20.306 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.293+0000 7f1708c1a700 1 --2- 172.21.15.52:0/671095010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55600f0008b0 0x55600f000d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.306 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.293+0000 7f1708c1a700 1 -- 172.21.15.52:0/671095010 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55600edce630 con 0x55600effbb60 2023-11-27T15:24:20.306 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.293+0000 7f170941b700 1 --2- 172.21.15.52:0/671095010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55600f0008b0 0x55600f000d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:24:20.306 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.293+0000 7f1708c1a700 1 --2- 172.21.15.52:0/671095010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55600effbb60 0x55600effbf30 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f16f8010fa0 tx=0x7f16f8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d0ad62ef0f2f53 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:20.306 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.293+0000 7f16eeffd700 1 -- 172.21.15.52:0/671095010 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f801e620 con 0x55600effbb60 2023-11-27T15:24:20.306 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.293+0000 7f16eeffd700 1 -- 172.21.15.52:0/671095010 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16f8015070 con 0x55600effbb60 2023-11-27T15:24:20.306 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.293+0000 7f16eeffd700 1 -- 172.21.15.52:0/671095010 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f801ca20 con 0x55600effbb60 2023-11-27T15:24:20.306 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.294+0000 7f171700ab80 1 -- 172.21.15.52:0/671095010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55600effbb60 msgr2=0x55600effbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:20.307 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.294+0000 7f171700ab80 1 --2- 172.21.15.52:0/671095010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55600effbb60 0x55600effbf30 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f16f8010fa0 tx=0x7f16f8013000 comp rx=0 tx=0).stop 2023-11-27T15:24:20.307 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.294+0000 7f171700ab80 1 -- 172.21.15.52:0/671095010 shutdown_connections 2023-11-27T15:24:20.307 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.294+0000 7f171700ab80 1 --2- 172.21.15.52:0/671095010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55600f0008b0 0x55600f000d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.307 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.294+0000 7f171700ab80 1 --2- 172.21.15.52:0/671095010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55600effbb60 0x55600effbf30 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.307 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.294+0000 7f171700ab80 1 --2- 172.21.15.52:0/671095010 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55600effc470 0x55600f000370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.307 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.294+0000 7f171700ab80 1 -- 172.21.15.52:0/671095010 >> 172.21.15.52:0/671095010 conn(0x55600eff1360 msgr2=0x55600eff6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:20.307 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.294+0000 7f171700ab80 1 -- 172.21.15.52:0/671095010 shutdown_connections 2023-11-27T15:24:20.308 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.294+0000 7f171700ab80 1 -- 172.21.15.52:0/671095010 wait complete. 2023-11-27T15:24:20.308 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.294+0000 7f171700ab80 1 -- start start 2023-11-27T15:24:20.308 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.295+0000 7f171700ab80 1 Processor -- start 2023-11-27T15:24:20.308 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.295+0000 7f171700ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55600effbb60 0x55600f007ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.308 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.295+0000 7f171700ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55600effc470 0x55600f005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.308 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.295+0000 7f171700ab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55600f0008b0 0x55600f0058a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.308 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.295+0000 7f171700ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600eeee8e0 con 0x55600effc470 2023-11-27T15:24:20.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.295+0000 7f171700ab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600eeee580 con 0x55600f0008b0 2023-11-27T15:24:20.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.295+0000 7f171700ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600edd0920 con 0x55600effbb60 2023-11-27T15:24:20.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.295+0000 7f171700ab80 10 client.0 _ll_register_callbacks cb 0x55600ef0c780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:20.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.295+0000 7f1703fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55600effc470 0x55600f005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.295+0000 7f1703fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55600effc470 0x55600f005270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54600/0 (socket says 172.21.15.52:54600) 2023-11-27T15:24:20.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.295+0000 7f1703fff700 1 -- 172.21.15.52:0/926982489 learned_addr learned my addr 172.21.15.52:0/926982489 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:20.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.295+0000 7f1708c1a700 1 --2- 172.21.15.52:0/926982489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55600effbb60 0x55600f007ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.295+0000 7f170941b700 1 --2- 172.21.15.52:0/926982489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55600f0008b0 0x55600f0058a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.295+0000 7f1703fff700 1 -- 172.21.15.52:0/926982489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55600effbb60 msgr2=0x55600f007ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:20.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.295+0000 7f1703fff700 1 --2- 172.21.15.52:0/926982489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55600effbb60 0x55600f007ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.295+0000 7f1703fff700 1 -- 172.21.15.52:0/926982489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55600f0008b0 msgr2=0x55600f0058a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:20.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.295+0000 7f1703fff700 1 --2- 172.21.15.52:0/926982489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55600f0008b0 0x55600f0058a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.295+0000 7f1703fff700 1 -- 172.21.15.52:0/926982489 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f8010c50 con 0x55600effc470 2023-11-27T15:24:20.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.295+0000 7f170941b700 1 --2- 172.21.15.52:0/926982489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55600f0008b0 0x55600f0058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:24:20.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.296+0000 7f1703fff700 1 --2- 172.21.15.52:0/926982489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55600effc470 0x55600f005270 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f16f0012700 tx=0x7f16f0012a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7e28b27f4bce16 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:20.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.296+0000 7f16e3fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:24:20.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.296+0000 7f16e3fff700 1 -- 172.21.15.52:0/926982489 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f0004520 con 0x55600effc470 2023-11-27T15:24:20.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.296+0000 7f16e3fff700 1 -- 172.21.15.52:0/926982489 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f16f0003680 con 0x55600effc470 2023-11-27T15:24:20.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.296+0000 7f171700ab80 1 -- 172.21.15.52:0/926982489 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55600efeeb20 con 0x55600effc470 2023-11-27T15:24:20.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.296+0000 7f16e3fff700 1 -- 172.21.15.52:0/926982489 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16f0003880 con 0x55600effc470 2023-11-27T15:24:20.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.296+0000 7f171700ab80 10 client.16491 fetch_fsmap learned FSMap version 28 2023-11-27T15:24:20.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.296+0000 7f171700ab80 1 -- 172.21.15.52:0/926982489 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55600ef13190 con 0x55600effc470 2023-11-27T15:24:20.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.296+0000 7f16e3fff700 1 -- 172.21.15.52:0/926982489 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16f0028020 con 0x55600effc470 2023-11-27T15:24:20.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.296+0000 7f16e3fff700 1 -- 172.21.15.52:0/926982489 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f16f0010b80 con 0x55600effc470 2023-11-27T15:24:20.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.296+0000 7f171700ab80 10 client.16491 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:24:20.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.296+0000 7f171700ab80 10 client.16491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:20.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.296+0000 7f171700ab80 20 client.16491 populate_metadata read hostname 'smithi052' 2023-11-27T15:24:20.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.296+0000 7f171700ab80 1 --2- 172.21.15.52:0/926982489 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55600f015a40 0x55600f017e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.296+0000 7f171700ab80 4 client.16491 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:24:20.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.296+0000 7f171700ab80 1 -- 172.21.15.52:0/926982489 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55600ef0f0a0 con 0x55600f015a40 2023-11-27T15:24:20.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.297+0000 7f1708c1a700 1 --2- 172.21.15.52:0/926982489 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55600f015a40 0x55600f017e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.297+0000 7f16e3fff700 10 client.16491 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:20.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.297+0000 7f1708c1a700 1 --2- 172.21.15.52:0/926982489 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55600f015a40 0x55600f017e50 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f9c596524f3371b server_cookie=4e8c0c745edc8e58 in_seq=0 out_seq=0 2023-11-27T15:24:20.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.297+0000 7f16e3fff700 1 -- 172.21.15.52:0/926982489 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55600ef0f0a0 con 0x55600f015a40 2023-11-27T15:24:20.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.298+0000 7f16e3fff700 10 client.16491 handle_command_reply: tid=0 2023-11-27T15:24:20.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.298+0000 7f171700ab80 1 client.16491 shutdown 2023-11-27T15:24:20.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.298+0000 7f171700ab80 1 -- 172.21.15.52:0/926982489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55600effc470 msgr2=0x55600f005270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:20.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.298+0000 7f171700ab80 1 --2- 172.21.15.52:0/926982489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55600effc470 0x55600f005270 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f16f0012700 tx=0x7f16f0012a10 comp rx=0 tx=0).stop 2023-11-27T15:24:20.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.298+0000 7f171700ab80 1 -- 172.21.15.52:0/926982489 shutdown_connections 2023-11-27T15:24:20.314 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.298+0000 7f171700ab80 1 --2- 172.21.15.52:0/926982489 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55600f015a40 0x55600f017e50 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.314 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.298+0000 7f171700ab80 1 --2- 172.21.15.52:0/926982489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55600f0008b0 0x55600f0058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.314 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.298+0000 7f16e3fff700 0 client.16491 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:20.314 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.298+0000 7f171700ab80 1 --2- 172.21.15.52:0/926982489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55600effbb60 0x55600f007ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.314 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.298+0000 7f171700ab80 1 --2- 172.21.15.52:0/926982489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55600effc470 0x55600f005270 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.314 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.298+0000 7f171700ab80 1 -- 172.21.15.52:0/926982489 >> 172.21.15.52:0/926982489 conn(0x55600eff1360 msgr2=0x55600f001d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:20.314 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.298+0000 7f171700ab80 1 -- 172.21.15.52:0/926982489 shutdown_connections 2023-11-27T15:24:20.314 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.298+0000 7f171700ab80 1 -- 172.21.15.52:0/926982489 wait complete. 2023-11-27T15:24:20.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.298+0000 7f171700ab80 20 client.16491 trim_cache size 0 max 16384 2023-11-27T15:24:20.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.315+0000 7f171700ab80 1 Processor -- start 2023-11-27T15:24:20.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.315+0000 7f171700ab80 1 -- start start 2023-11-27T15:24:20.323 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.315+0000 7f171700ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55600effbb60 0x55600f011be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.323 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.315+0000 7f171700ab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55600effc470 0x55600f011150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.323 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.315+0000 7f171700ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55600f0008b0 0x55600effe8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.323 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.316+0000 7f171700ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600f012f40 con 0x55600f0008b0 2023-11-27T15:24:20.324 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.316+0000 7f171700ab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600f006380 con 0x55600effc470 2023-11-27T15:24:20.324 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.316+0000 7f171700ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600efff420 con 0x55600effbb60 2023-11-27T15:24:20.324 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.316+0000 7f170941b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55600f0008b0 0x55600effe8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.324 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.316+0000 7f170941b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55600f0008b0 0x55600effe8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54610/0 (socket says 172.21.15.52:54610) 2023-11-27T15:24:20.324 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.316+0000 7f170941b700 1 -- 172.21.15.52:0/2426979419 learned_addr learned my addr 172.21.15.52:0/2426979419 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:20.324 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.316+0000 7f1703fff700 1 --2- 172.21.15.52:0/2426979419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55600effc470 0x55600f011150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.324 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.316+0000 7f170941b700 1 -- 172.21.15.52:0/2426979419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55600effbb60 msgr2=0x55600f011be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:20.325 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.316+0000 7f170941b700 1 --2- 172.21.15.52:0/2426979419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55600effbb60 0x55600f011be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.325 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.316+0000 7f170941b700 1 -- 172.21.15.52:0/2426979419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55600effc470 msgr2=0x55600f011150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:20.325 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.316+0000 7f170941b700 1 --2- 172.21.15.52:0/2426979419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55600effc470 0x55600f011150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.325 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.316+0000 7f170941b700 1 -- 172.21.15.52:0/2426979419 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55600ef12ee0 con 0x55600f0008b0 2023-11-27T15:24:20.325 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.316+0000 7f170941b700 1 --2- 172.21.15.52:0/2426979419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55600f0008b0 0x55600effe8b0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0cb9b0 tx=0x7f16fc007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2724cb21289915 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:20.325 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.316+0000 7f16ef7fe700 1 -- 172.21.15.52:0/2426979419 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc09aea0 con 0x55600f0008b0 2023-11-27T15:24:20.325 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.316+0000 7f16ef7fe700 1 -- 172.21.15.52:0/2426979419 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16fc09acc0 con 0x55600f0008b0 2023-11-27T15:24:20.325 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.316+0000 7f16ef7fe700 1 -- 172.21.15.52:0/2426979419 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc097a70 con 0x55600f0008b0 2023-11-27T15:24:20.325 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.317+0000 7f171700ab80 1 -- 172.21.15.52:0/2426979419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55600f0008b0 msgr2=0x55600effe8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:20.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.317+0000 7f171700ab80 1 --2- 172.21.15.52:0/2426979419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55600f0008b0 0x55600effe8b0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0cb9b0 tx=0x7f16fc007930 comp rx=0 tx=0).stop 2023-11-27T15:24:20.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.317+0000 7f171700ab80 1 -- 172.21.15.52:0/2426979419 shutdown_connections 2023-11-27T15:24:20.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.317+0000 7f171700ab80 1 --2- 172.21.15.52:0/2426979419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55600f0008b0 0x55600effe8b0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.317+0000 7f171700ab80 1 --2- 172.21.15.52:0/2426979419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55600effbb60 0x55600f011be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.317+0000 7f171700ab80 1 --2- 172.21.15.52:0/2426979419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55600effc470 0x55600f011150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.317+0000 7f171700ab80 1 -- 172.21.15.52:0/2426979419 >> 172.21.15.52:0/2426979419 conn(0x55600eff1360 msgr2=0x55600f002c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:20.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.317+0000 7f171700ab80 1 -- 172.21.15.52:0/2426979419 shutdown_connections 2023-11-27T15:24:20.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.317+0000 7f171700ab80 1 -- 172.21.15.52:0/2426979419 wait complete. 2023-11-27T15:24:20.327 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.317+0000 7f171700ab80 1 -- start start 2023-11-27T15:24:20.327 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.317+0000 7f171700ab80 1 Processor -- start 2023-11-27T15:24:20.327 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.318+0000 7f171700ab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55600effbb60 0x55600f00ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.327 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.318+0000 7f171700ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55600effc470 0x55600f00b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.327 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.318+0000 7f171700ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55600f0008b0 0x55600f062890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.327 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.318+0000 7f171700ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600f062dd0 con 0x55600f0008b0 2023-11-27T15:24:20.327 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.318+0000 7f171700ab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600f062f40 con 0x55600effbb60 2023-11-27T15:24:20.328 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.318+0000 7f171700ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600f063100 con 0x55600effc470 2023-11-27T15:24:20.328 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.318+0000 7f171700ab80 10 client.0 _ll_register_callbacks cb 0x7f16f0009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:20.328 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.318+0000 7f170941b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55600f0008b0 0x55600f062890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.328 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.318+0000 7f170941b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55600f0008b0 0x55600f062890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54616/0 (socket says 172.21.15.52:54616) 2023-11-27T15:24:20.328 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.318+0000 7f170941b700 1 -- 172.21.15.52:0/1165119419 learned_addr learned my addr 172.21.15.52:0/1165119419 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:20.328 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.318+0000 7f1708c1a700 1 --2- 172.21.15.52:0/1165119419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55600effbb60 0x55600f00ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.328 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.318+0000 7f1703fff700 1 --2- 172.21.15.52:0/1165119419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55600effc470 0x55600f00b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.328 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.318+0000 7f170941b700 1 -- 172.21.15.52:0/1165119419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55600effc470 msgr2=0x55600f00b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:20.329 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.318+0000 7f170941b700 1 --2- 172.21.15.52:0/1165119419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55600effc470 0x55600f00b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.329 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.318+0000 7f170941b700 1 -- 172.21.15.52:0/1165119419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55600effbb60 msgr2=0x55600f00ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:20.329 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.318+0000 7f170941b700 1 --2- 172.21.15.52:0/1165119419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55600effbb60 0x55600f00ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.329 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.318+0000 7f170941b700 1 -- 172.21.15.52:0/1165119419 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17041101e0 con 0x55600f0008b0 2023-11-27T15:24:20.329 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.318+0000 7f170941b700 1 --2- 172.21.15.52:0/1165119419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55600f0008b0 0x55600f062890 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f16fc097860 tx=0x7f16fc009a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd2c06d23667601 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:20.329 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.318+0000 7f16e3fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:24:20.329 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.318+0000 7f16e3fff700 1 -- 172.21.15.52:0/1165119419 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc09f390 con 0x55600f0008b0 2023-11-27T15:24:20.329 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.319+0000 7f16e3fff700 1 -- 172.21.15.52:0/1165119419 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f16fc09b360 con 0x55600f0008b0 2023-11-27T15:24:20.329 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.319+0000 7f171700ab80 1 -- 172.21.15.52:0/1165119419 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16f8015070 con 0x55600f0008b0 2023-11-27T15:24:20.330 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.319+0000 7f16e3fff700 1 -- 172.21.15.52:0/1165119419 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16fc09b5c0 con 0x55600f0008b0 2023-11-27T15:24:20.330 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.319+0000 7f171700ab80 10 client.16497 fetch_fsmap learned FSMap version 28 2023-11-27T15:24:20.330 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.319+0000 7f171700ab80 1 -- 172.21.15.52:0/1165119419 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55600ed63c60 con 0x55600f0008b0 2023-11-27T15:24:20.330 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.319+0000 7f16e3fff700 1 -- 172.21.15.52:0/1165119419 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16f8015070 con 0x55600f0008b0 2023-11-27T15:24:20.330 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.319+0000 7f16e3fff700 1 -- 172.21.15.52:0/1165119419 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f16fc0a33b0 con 0x55600f0008b0 2023-11-27T15:24:20.330 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.319+0000 7f171700ab80 10 client.16497 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:24:20.330 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.319+0000 7f171700ab80 10 client.16497 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:20.330 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.319+0000 7f171700ab80 20 client.16497 populate_metadata read hostname 'smithi052' 2023-11-27T15:24:20.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.319+0000 7f171700ab80 1 --2- 172.21.15.52:0/1165119419 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55600f06a330 0x55600f06c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:20.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.319+0000 7f171700ab80 4 client.16497 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:24:20.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.319+0000 7f171700ab80 1 -- 172.21.15.52:0/1165119419 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55600ef1b660 con 0x55600f06a330 2023-11-27T15:24:20.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.319+0000 7f1708c1a700 1 --2- 172.21.15.52:0/1165119419 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55600f06a330 0x55600f06c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:20.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.320+0000 7f16e3fff700 10 client.16497 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:20.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.320+0000 7f1708c1a700 1 --2- 172.21.15.52:0/1165119419 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55600f06a330 0x55600f06c710 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdb203b48d5dd328 server_cookie=1256e167c2798d5 in_seq=0 out_seq=0 2023-11-27T15:24:20.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.320+0000 7f16e3fff700 1 -- 172.21.15.52:0/1165119419 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55600ef1b660 con 0x55600f06a330 2023-11-27T15:24:20.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.320+0000 7f16e3fff700 10 client.16497 handle_command_reply: tid=0 2023-11-27T15:24:20.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.320+0000 7f171700ab80 1 client.16497 shutdown 2023-11-27T15:24:20.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.320+0000 7f171700ab80 1 -- 172.21.15.52:0/1165119419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55600f0008b0 msgr2=0x55600f062890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:20.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.320+0000 7f171700ab80 1 --2- 172.21.15.52:0/1165119419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55600f0008b0 0x55600f062890 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f16fc097860 tx=0x7f16fc009a60 comp rx=0 tx=0).stop 2023-11-27T15:24:20.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.320+0000 7f171700ab80 1 -- 172.21.15.52:0/1165119419 shutdown_connections 2023-11-27T15:24:20.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.320+0000 7f171700ab80 1 --2- 172.21.15.52:0/1165119419 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55600f06a330 0x55600f06c710 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.320+0000 7f171700ab80 1 --2- 172.21.15.52:0/1165119419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55600effbb60 0x55600f00ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.320+0000 7f171700ab80 1 --2- 172.21.15.52:0/1165119419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55600f0008b0 0x55600f062890 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.320+0000 7f16e3fff700 0 client.16497 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:20.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.320+0000 7f171700ab80 1 --2- 172.21.15.52:0/1165119419 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55600effc470 0x55600f00b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.320+0000 7f171700ab80 1 -- 172.21.15.52:0/1165119419 >> 172.21.15.52:0/1165119419 conn(0x55600eff1360 msgr2=0x55600f011150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:20.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.320+0000 7f171700ab80 1 -- 172.21.15.52:0/1165119419 shutdown_connections 2023-11-27T15:24:20.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.320+0000 7f171700ab80 1 -- 172.21.15.52:0/1165119419 wait complete. 2023-11-27T15:24:20.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.320+0000 7f171700ab80 20 client.16497 trim_cache size 0 max 16384 2023-11-27T15:24:20.334 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:24:20.335 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-27T15:24:20.335 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2023-11-27T15:24:20.335 INFO:teuthology.orchestra.run.smithi052.stdout: "cbc22cb2-af74-4ea2-822e-d40d2a21058b": { 2023-11-27T15:24:20.335 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2023-11-27T15:24:20.335 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "cbc22cb2-af74-4ea2-822e-d40d2a21058b", 2023-11-27T15:24:20.335 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2023-11-27T15:24:20.335 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:24:20.335 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:24:20.335 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:24:20.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.321+0000 7f170ae7e700 1 -- 172.21.15.52:0/2770401087 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f16f40a12c0 msgr2=0x7f16f40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:20.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.321+0000 7f170ae7e700 1 --2- 172.21.15.52:0/2770401087 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f16f40a12c0 0x7f16f40a3770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f16f8005950 tx=0x7f16f800b410 comp rx=0 tx=0).stop 2023-11-27T15:24:20.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.321+0000 7f170ae7e700 1 -- 172.21.15.52:0/2770401087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f170410a8d0 msgr2=0x7f170419ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:20.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.321+0000 7f170ae7e700 1 --2- 172.21.15.52:0/2770401087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f170410a8d0 0x7f170419ae20 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00b780 tx=0x7f16fc00bd60 comp rx=0 tx=0).stop 2023-11-27T15:24:20.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.321+0000 7f170ae7e700 1 -- 172.21.15.52:0/2770401087 shutdown_connections 2023-11-27T15:24:20.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.321+0000 7f170ae7e700 1 --2- 172.21.15.52:0/2770401087 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f16f40a12c0 0x7f16f40a3770 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.321+0000 7f170ae7e700 1 --2- 172.21.15.52:0/2770401087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f170410a8d0 0x7f170419ae20 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.321+0000 7f170ae7e700 1 --2- 172.21.15.52:0/2770401087 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1704101590 0x7f1704196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.321+0000 7f170ae7e700 1 --2- 172.21.15.52:0/2770401087 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1704101ea0 0x7f17041969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:20.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.321+0000 7f170ae7e700 1 -- 172.21.15.52:0/2770401087 >> 172.21.15.52:0/2770401087 conn(0x7f1704076070 msgr2=0x7f1704102330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:20.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.321+0000 7f170ae7e700 1 -- 172.21.15.52:0/2770401087 shutdown_connections 2023-11-27T15:24:20.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:20.321+0000 7f170ae7e700 1 -- 172.21.15.52:0/2770401087 wait complete. 2023-11-27T15:24:20.338 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:20 smithi052 ceph-mon[99056]: pgmap v197: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 332 B/s wr, 7 op/s 2023-11-27T15:24:20.339 INFO:tasks.cephfs.filesystem:scrub status for tag:cbc22cb2-af74-4ea2-822e-d40d2a21058b - {'path': '/', 'tag': 'cbc22cb2-af74-4ea2-822e-d40d2a21058b', 'options': 'recursive,force'} 2023-11-27T15:24:20.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:20 smithi150 ceph-mon[100559]: pgmap v197: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 332 B/s wr, 7 op/s 2023-11-27T15:24:20.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:20 smithi102 ceph-mon[100296]: pgmap v197: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 332 B/s wr, 7 op/s 2023-11-27T15:24:21.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:21 smithi150 ceph-mon[100559]: scrub queued for path: / 2023-11-27T15:24:21.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:21 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:24:21.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:21 smithi150 ceph-mon[100559]: scrub summary: active paths [/] 2023-11-27T15:24:21.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:21 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:24:21.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:21 smithi102 ceph-mon[100296]: scrub queued for path: / 2023-11-27T15:24:21.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:21 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:24:21.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:21 smithi102 ceph-mon[100296]: scrub summary: active paths [/] 2023-11-27T15:24:21.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:21 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:24:21.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:21 smithi052 ceph-mon[99056]: scrub queued for path: / 2023-11-27T15:24:21.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:21 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:24:21.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:21 smithi052 ceph-mon[99056]: scrub summary: active paths [/] 2023-11-27T15:24:21.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:21 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:24:22.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:22 smithi150 ceph-mon[100559]: pgmap v198: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:24:22.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:22 smithi102 ceph-mon[100296]: pgmap v198: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:24:22.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:22 smithi052 ceph-mon[99056]: pgmap v198: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:24:24.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:24 smithi150 ceph-mon[100559]: pgmap v199: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:24:24.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:24 smithi102 ceph-mon[100296]: pgmap v199: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:24:24.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:24 smithi052 ceph-mon[99056]: pgmap v199: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:24:25.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:25 smithi150 ceph-mon[100559]: scrub summary: idle 2023-11-27T15:24:25.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:25 smithi102 ceph-mon[100296]: scrub summary: idle 2023-11-27T15:24:25.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:25 smithi052 ceph-mon[99056]: scrub summary: idle 2023-11-27T15:24:26.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:26 smithi150 ceph-mon[100559]: pgmap v200: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 181 B/s wr, 5 op/s 2023-11-27T15:24:26.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:26 smithi102 ceph-mon[100296]: pgmap v200: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 181 B/s wr, 5 op/s 2023-11-27T15:24:26.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:26 smithi052 ceph-mon[99056]: pgmap v200: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 181 B/s wr, 5 op/s 2023-11-27T15:24:28.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:28 smithi150 ceph-mon[100559]: pgmap v201: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 170 B/s wr, 7 op/s 2023-11-27T15:24:28.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:28 smithi102 ceph-mon[100296]: pgmap v201: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 170 B/s wr, 7 op/s 2023-11-27T15:24:28.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:28 smithi052 ceph-mon[99056]: pgmap v201: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 170 B/s wr, 7 op/s 2023-11-27T15:24:30.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:29 smithi150 ceph-mon[100559]: pgmap v202: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:24:30.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:29 smithi102 ceph-mon[100296]: pgmap v202: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:24:30.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:29 smithi052 ceph-mon[99056]: pgmap v202: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:24:32.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:32 smithi150 ceph-mon[100559]: pgmap v203: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:24:32.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:32 smithi102 ceph-mon[100296]: pgmap v203: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:24:32.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:32 smithi052 ceph-mon[99056]: pgmap v203: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:24:34.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:34 smithi150 ceph-mon[100559]: pgmap v204: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:24:34.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:34 smithi102 ceph-mon[100296]: pgmap v204: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:24:34.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:34 smithi052 ceph-mon[99056]: pgmap v204: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:24:36.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:36 smithi150 ceph-mon[100559]: pgmap v205: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:24:36.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:36 smithi102 ceph-mon[100296]: pgmap v205: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:24:36.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:36 smithi052 ceph-mon[99056]: pgmap v205: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:24:38.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:38 smithi150 ceph-mon[100559]: pgmap v206: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:24:38.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:38 smithi102 ceph-mon[100296]: pgmap v206: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:24:38.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:38 smithi052 ceph-mon[99056]: pgmap v206: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:24:40.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:39 smithi150 ceph-mon[100559]: pgmap v207: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:24:40.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:39 smithi102 ceph-mon[100296]: pgmap v207: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:24:40.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:39 smithi052 ceph-mon[99056]: pgmap v207: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:24:42.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:42 smithi150 ceph-mon[100559]: pgmap v208: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:24:42.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:42 smithi102 ceph-mon[100296]: pgmap v208: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:24:42.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:42 smithi052 ceph-mon[99056]: pgmap v208: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:24:44.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:44 smithi150 ceph-mon[100559]: pgmap v209: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:24:44.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:44 smithi102 ceph-mon[100296]: pgmap v209: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:24:44.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:44 smithi052 ceph-mon[99056]: pgmap v209: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:24:46.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:46 smithi150 ceph-mon[100559]: pgmap v210: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:24:46.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:46 smithi102 ceph-mon[100296]: pgmap v210: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:24:46.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:46 smithi052 ceph-mon[99056]: pgmap v210: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:24:48.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:48 smithi150 ceph-mon[100559]: pgmap v211: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:24:48.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:48 smithi102 ceph-mon[100296]: pgmap v211: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:24:48.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:48 smithi052 ceph-mon[99056]: pgmap v211: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:24:50.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:49 smithi150 ceph-mon[100559]: pgmap v212: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:24:50.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:49 smithi102 ceph-mon[100296]: pgmap v212: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:24:50.340 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:24:50.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:49 smithi052 ceph-mon[99056]: pgmap v212: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:24:50.485 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.478+0000 7fd2d5fc2700 1 -- 172.21.15.52:0/4130743745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd2d01045d0 msgr2=0x7fd2d010c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:50.486 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.478+0000 7fd2d5fc2700 1 --2- 172.21.15.52:0/4130743745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd2d01045d0 0x7fd2d010c1a0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800b0d0 tx=0x7fd2c800b3e0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.486 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.478+0000 7fd2d5fc2700 1 -- 172.21.15.52:0/4130743745 shutdown_connections 2023-11-27T15:24:50.486 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.478+0000 7fd2d5fc2700 1 --2- 172.21.15.52:0/4130743745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd2d01045d0 0x7fd2d010c1a0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.486 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.478+0000 7fd2d5fc2700 1 --2- 172.21.15.52:0/4130743745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2d00ff590 0x7fd2d00ff960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.486 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.478+0000 7fd2d5fc2700 1 --2- 172.21.15.52:0/4130743745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2d00fff30 0x7fd2d0103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.487 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.478+0000 7fd2d5fc2700 1 -- 172.21.15.52:0/4130743745 >> 172.21.15.52:0/4130743745 conn(0x7fd2d00760d0 msgr2=0x7fd2d00764d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:50.487 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.478+0000 7fd2d5fc2700 1 -- 172.21.15.52:0/4130743745 shutdown_connections 2023-11-27T15:24:50.487 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.479+0000 7fd2d5fc2700 1 -- 172.21.15.52:0/4130743745 wait complete. 2023-11-27T15:24:50.487 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.479+0000 7fd2d5fc2700 1 Processor -- start 2023-11-27T15:24:50.487 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.479+0000 7fd2d5fc2700 1 -- start start 2023-11-27T15:24:50.487 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.479+0000 7fd2d5fc2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2d00ff590 0x7fd2d0196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.487 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.480+0000 7fd2d5fc2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2d00fff30 0x7fd2d0196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.488 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.480+0000 7fd2d5fc2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd2d01045d0 0x7fd2d019aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.488 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.480+0000 7fd2d5fc2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d019b530 con 0x7fd2d01045d0 2023-11-27T15:24:50.488 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.480+0000 7fd2d5fc2700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d019b6a0 con 0x7fd2d00ff590 2023-11-27T15:24:50.488 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.480+0000 7fd2d5fc2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d019b860 con 0x7fd2d00fff30 2023-11-27T15:24:50.488 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.480+0000 7fd2cffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd2d01045d0 0x7fd2d019aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.488 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.480+0000 7fd2cffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd2d01045d0 0x7fd2d019aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54630/0 (socket says 172.21.15.52:54630) 2023-11-27T15:24:50.488 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.480+0000 7fd2cffff700 1 -- 172.21.15.52:0/3535667266 learned_addr learned my addr 172.21.15.52:0/3535667266 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:50.488 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.480+0000 7fd2cf7fe700 1 --2- 172.21.15.52:0/3535667266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2d00ff590 0x7fd2d0196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.480+0000 7fd2ceffd700 1 --2- 172.21.15.52:0/3535667266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2d00fff30 0x7fd2d0196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.480+0000 7fd2cffff700 1 -- 172.21.15.52:0/3535667266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2d00fff30 msgr2=0x7fd2d0196a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:50.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.480+0000 7fd2cffff700 1 --2- 172.21.15.52:0/3535667266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2d00fff30 0x7fd2d0196a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.480+0000 7fd2cffff700 1 -- 172.21.15.52:0/3535667266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2d00ff590 msgr2=0x7fd2d0196550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:50.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.480+0000 7fd2cffff700 1 --2- 172.21.15.52:0/3535667266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2d00ff590 0x7fd2d0196550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.480+0000 7fd2cffff700 1 -- 172.21.15.52:0/3535667266 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c8009d00 con 0x7fd2d01045d0 2023-11-27T15:24:50.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.480+0000 7fd2cffff700 1 --2- 172.21.15.52:0/3535667266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd2d01045d0 0x7fd2d019aef0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8007870 tx=0x7fd2c80078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:50.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.480+0000 7fd2ccff9700 1 -- 172.21.15.52:0/3535667266 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c800e040 con 0x7fd2d01045d0 2023-11-27T15:24:50.490 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.480+0000 7fd2ccff9700 1 -- 172.21.15.52:0/3535667266 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2c8005070 con 0x7fd2d01045d0 2023-11-27T15:24:50.490 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.480+0000 7fd2d5fc2700 1 -- 172.21.15.52:0/3535667266 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d010f3a0 con 0x7fd2d01045d0 2023-11-27T15:24:50.490 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.480+0000 7fd2ccff9700 1 -- 172.21.15.52:0/3535667266 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c801b6c0 con 0x7fd2d01045d0 2023-11-27T15:24:50.490 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.480+0000 7fd2d5fc2700 1 -- 172.21.15.52:0/3535667266 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d010f210 con 0x7fd2d01045d0 2023-11-27T15:24:50.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.483+0000 7fd2ccff9700 1 -- 172.21.15.52:0/3535667266 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fd2c80051e0 con 0x7fd2d01045d0 2023-11-27T15:24:50.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.483+0000 7fd2ccff9700 1 --2- 172.21.15.52:0/3535667266 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd2bc0a1340 0x7fd2bc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.483+0000 7fd2cf7fe700 1 --2- 172.21.15.52:0/3535667266 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd2bc0a1340 0x7fd2bc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.483+0000 7fd2ccff9700 1 -- 172.21.15.52:0/3535667266 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fd2c80cde80 con 0x7fd2d01045d0 2023-11-27T15:24:50.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.484+0000 7fd2e214eb80 1 Processor -- start 2023-11-27T15:24:50.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.484+0000 7fd2cf7fe700 1 --2- 172.21.15.52:0/3535667266 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd2bc0a1340 0x7fd2bc0a37f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0005950 tx=0x7fd2c000b410 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:50.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.484+0000 7fd2e214eb80 1 -- start start 2023-11-27T15:24:50.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.484+0000 7fd2e214eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f7ac1cb60 0x557f7ac1cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.484+0000 7fd2e214eb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f7ac1d470 0x557f7ac21370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.484+0000 7fd2e214eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f7ac218b0 0x557f7ac21d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.484+0000 7fd2e214eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f7ab22060 con 0x557f7ac1cb60 2023-11-27T15:24:50.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.484+0000 7fd2e214eb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f7ab748a0 con 0x557f7ac1d470 2023-11-27T15:24:50.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.484+0000 7fd2e214eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f7ab6dd30 con 0x557f7ac218b0 2023-11-27T15:24:50.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.484+0000 7fd2cf7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f7ac1cb60 0x557f7ac1cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.484+0000 7fd2cf7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f7ac1cb60 0x557f7ac1cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54634/0 (socket says 172.21.15.52:54634) 2023-11-27T15:24:50.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.484+0000 7fd2cf7fe700 1 -- 172.21.15.52:0/3295497917 learned_addr learned my addr 172.21.15.52:0/3295497917 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:50.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.484+0000 7fd2ceffd700 1 --2- 172.21.15.52:0/3295497917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f7ac1d470 0x557f7ac21370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.485+0000 7fd2cf7fe700 1 -- 172.21.15.52:0/3295497917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f7ac218b0 msgr2=0x557f7ac21d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:50.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.485+0000 7fd2cffff700 1 --2- 172.21.15.52:0/3295497917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f7ac218b0 0x557f7ac21d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.485+0000 7fd2cf7fe700 1 --2- 172.21.15.52:0/3295497917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f7ac218b0 0x557f7ac21d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.485+0000 7fd2cf7fe700 1 -- 172.21.15.52:0/3295497917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f7ac1d470 msgr2=0x557f7ac21370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:50.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.485+0000 7fd2cf7fe700 1 --2- 172.21.15.52:0/3295497917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f7ac1d470 0x557f7ac21370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.485+0000 7fd2cf7fe700 1 -- 172.21.15.52:0/3295497917 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f7a9ef630 con 0x557f7ac1cb60 2023-11-27T15:24:50.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.485+0000 7fd2cffff700 1 --2- 172.21.15.52:0/3295497917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f7ac218b0 0x557f7ac21d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:24:50.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.485+0000 7fd2cf7fe700 1 --2- 172.21.15.52:0/3295497917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f7ac1cb60 0x557f7ac1cf30 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0010fa0 tx=0x7fd2c0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e03a2a7894b6a31 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:50.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.485+0000 7fd2c5ffb700 1 -- 172.21.15.52:0/3295497917 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0011db0 con 0x557f7ac1cb60 2023-11-27T15:24:50.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.485+0000 7fd2c5ffb700 1 -- 172.21.15.52:0/3295497917 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2c0015070 con 0x557f7ac1cb60 2023-11-27T15:24:50.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.485+0000 7fd2c5ffb700 1 -- 172.21.15.52:0/3295497917 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c001c610 con 0x557f7ac1cb60 2023-11-27T15:24:50.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.486+0000 7fd2e214eb80 1 -- 172.21.15.52:0/3295497917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f7ac1cb60 msgr2=0x557f7ac1cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:50.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.486+0000 7fd2e214eb80 1 --2- 172.21.15.52:0/3295497917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f7ac1cb60 0x557f7ac1cf30 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0010fa0 tx=0x7fd2c0013000 comp rx=0 tx=0).stop 2023-11-27T15:24:50.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.486+0000 7fd2e214eb80 1 -- 172.21.15.52:0/3295497917 shutdown_connections 2023-11-27T15:24:50.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.486+0000 7fd2e214eb80 1 --2- 172.21.15.52:0/3295497917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f7ac218b0 0x557f7ac21d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.486+0000 7fd2e214eb80 1 --2- 172.21.15.52:0/3295497917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f7ac1cb60 0x557f7ac1cf30 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.486+0000 7fd2e214eb80 1 --2- 172.21.15.52:0/3295497917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f7ac1d470 0x557f7ac21370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.486+0000 7fd2e214eb80 1 -- 172.21.15.52:0/3295497917 >> 172.21.15.52:0/3295497917 conn(0x557f7ac12360 msgr2=0x557f7ac17b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:50.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.486+0000 7fd2e214eb80 1 -- 172.21.15.52:0/3295497917 shutdown_connections 2023-11-27T15:24:50.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.486+0000 7fd2e214eb80 1 -- 172.21.15.52:0/3295497917 wait complete. 2023-11-27T15:24:50.501 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.486+0000 7fd2e214eb80 1 -- start start 2023-11-27T15:24:50.501 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.486+0000 7fd2e214eb80 1 Processor -- start 2023-11-27T15:24:50.501 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.487+0000 7fd2e214eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f7ac1cb60 0x557f7ac29010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.501 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.487+0000 7fd2e214eb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f7ac1d470 0x557f7ac26290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.501 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.487+0000 7fd2e214eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f7ac218b0 0x557f7ac26910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.501 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.487+0000 7fd2e214eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f7ab0f8e0 con 0x557f7ac1cb60 2023-11-27T15:24:50.502 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.487+0000 7fd2e214eb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f7ab0f580 con 0x557f7ac1d470 2023-11-27T15:24:50.502 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.487+0000 7fd2e214eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f7a9f1920 con 0x557f7ac218b0 2023-11-27T15:24:50.502 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.487+0000 7fd2e214eb80 10 client.0 _ll_register_callbacks cb 0x557f7ab2d780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:50.502 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.487+0000 7fd2cf7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f7ac1cb60 0x557f7ac29010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.502 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.487+0000 7fd2cf7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f7ac1cb60 0x557f7ac29010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54640/0 (socket says 172.21.15.52:54640) 2023-11-27T15:24:50.503 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.487+0000 7fd2cf7fe700 1 -- 172.21.15.52:0/168810720 learned_addr learned my addr 172.21.15.52:0/168810720 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:50.503 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.487+0000 7fd2ceffd700 1 --2- 172.21.15.52:0/168810720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f7ac1d470 0x557f7ac26290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.503 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.487+0000 7fd2cf7fe700 1 -- 172.21.15.52:0/168810720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f7ac218b0 msgr2=0x557f7ac26910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:50.503 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.487+0000 7fd2cf7fe700 1 --2- 172.21.15.52:0/168810720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f7ac218b0 0x557f7ac26910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.503 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.487+0000 7fd2cf7fe700 1 -- 172.21.15.52:0/168810720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f7ac1d470 msgr2=0x557f7ac26290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:50.503 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.487+0000 7fd2cf7fe700 1 --2- 172.21.15.52:0/168810720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f7ac1d470 0x557f7ac26290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.503 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.487+0000 7fd2cf7fe700 1 -- 172.21.15.52:0/168810720 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c0010c50 con 0x557f7ac1cb60 2023-11-27T15:24:50.503 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.487+0000 7fd2cf7fe700 1 --2- 172.21.15.52:0/168810720 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f7ac1cb60 0x557f7ac29010 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd2c001ecf0 tx=0x7fd2c0011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bffb62cca23c4860 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:50.504 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.487+0000 7fd2aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:24:50.504 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.487+0000 7fd2aeffd700 1 -- 172.21.15.52:0/168810720 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c002c070 con 0x557f7ac1cb60 2023-11-27T15:24:50.504 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.487+0000 7fd2aeffd700 1 -- 172.21.15.52:0/168810720 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fd2c0017070 con 0x557f7ac1cb60 2023-11-27T15:24:50.504 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.487+0000 7fd2e214eb80 1 -- 172.21.15.52:0/168810720 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f7ac0fb20 con 0x557f7ac1cb60 2023-11-27T15:24:50.504 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.488+0000 7fd2aeffd700 1 -- 172.21.15.52:0/168810720 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2c001e770 con 0x557f7ac1cb60 2023-11-27T15:24:50.504 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.488+0000 7fd2e214eb80 10 client.16509 fetch_fsmap learned FSMap version 28 2023-11-27T15:24:50.504 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.488+0000 7fd2e214eb80 1 -- 172.21.15.52:0/168810720 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557f7ab34190 con 0x557f7ac1cb60 2023-11-27T15:24:50.504 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.488+0000 7fd2aeffd700 1 -- 172.21.15.52:0/168810720 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2c003d080 con 0x557f7ac1cb60 2023-11-27T15:24:50.505 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.488+0000 7fd2aeffd700 1 -- 172.21.15.52:0/168810720 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd2c0004c60 con 0x557f7ac1cb60 2023-11-27T15:24:50.505 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.488+0000 7fd2e214eb80 10 client.16509 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:24:50.505 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.488+0000 7fd2e214eb80 10 client.16509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:50.505 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.488+0000 7fd2e214eb80 20 client.16509 populate_metadata read hostname 'smithi052' 2023-11-27T15:24:50.505 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.488+0000 7fd2e214eb80 1 --2- 172.21.15.52:0/168810720 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557f7ac36850 0x557f7ac38c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.505 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.488+0000 7fd2e214eb80 4 client.16509 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:24:50.505 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.488+0000 7fd2e214eb80 1 -- 172.21.15.52:0/168810720 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f7ab300a0 con 0x557f7ac36850 2023-11-27T15:24:50.505 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.488+0000 7fd2ceffd700 1 --2- 172.21.15.52:0/168810720 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557f7ac36850 0x557f7ac38c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.506 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.489+0000 7fd2aeffd700 10 client.16509 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:50.506 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.489+0000 7fd2ceffd700 1 --2- 172.21.15.52:0/168810720 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557f7ac36850 0x557f7ac38c60 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ea1120546c96dd server_cookie=32007d1eb42b7ce7 in_seq=0 out_seq=0 2023-11-27T15:24:50.506 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.489+0000 7fd2aeffd700 1 -- 172.21.15.52:0/168810720 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557f7ab300a0 con 0x557f7ac36850 2023-11-27T15:24:50.506 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.489+0000 7fd2aeffd700 10 client.16509 handle_command_reply: tid=0 2023-11-27T15:24:50.506 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.491+0000 7fd2e214eb80 1 client.16509 shutdown 2023-11-27T15:24:50.506 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.491+0000 7fd2e214eb80 1 -- 172.21.15.52:0/168810720 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f7ac1cb60 msgr2=0x557f7ac29010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:50.506 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.491+0000 7fd2e214eb80 1 --2- 172.21.15.52:0/168810720 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f7ac1cb60 0x557f7ac29010 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd2c001ecf0 tx=0x7fd2c0011e90 comp rx=0 tx=0).stop 2023-11-27T15:24:50.506 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.491+0000 7fd2e214eb80 1 -- 172.21.15.52:0/168810720 shutdown_connections 2023-11-27T15:24:50.507 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.491+0000 7fd2e214eb80 1 --2- 172.21.15.52:0/168810720 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557f7ac36850 0x557f7ac38c60 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.507 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.491+0000 7fd2e214eb80 1 --2- 172.21.15.52:0/168810720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f7ac218b0 0x557f7ac26910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.507 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.491+0000 7fd2e214eb80 1 --2- 172.21.15.52:0/168810720 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f7ac1cb60 0x557f7ac29010 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.507 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.491+0000 7fd2e214eb80 1 --2- 172.21.15.52:0/168810720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f7ac1d470 0x557f7ac26290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.507 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.491+0000 7fd2e214eb80 1 -- 172.21.15.52:0/168810720 >> 172.21.15.52:0/168810720 conn(0x557f7ac12360 msgr2=0x557f7ac22d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:50.507 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.491+0000 7fd2aeffd700 0 client.16509 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:50.507 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.491+0000 7fd2e214eb80 1 -- 172.21.15.52:0/168810720 shutdown_connections 2023-11-27T15:24:50.507 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.491+0000 7fd2e214eb80 1 -- 172.21.15.52:0/168810720 wait complete. 2023-11-27T15:24:50.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.491+0000 7fd2e214eb80 20 client.16509 trim_cache size 0 max 16384 2023-11-27T15:24:50.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.508+0000 7fd2e214eb80 1 Processor -- start 2023-11-27T15:24:50.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.508+0000 7fd2e214eb80 1 -- start start 2023-11-27T15:24:50.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.508+0000 7fd2e214eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f7ac1cb60 0x557f7ac17be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.508+0000 7fd2e214eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f7ac1d470 0x557f7ac2d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.508+0000 7fd2e214eb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f7ac218b0 0x557f7ac28bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.508+0000 7fd2e214eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f7ac11ee0 con 0x557f7ac1d470 2023-11-27T15:24:50.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.508+0000 7fd2e214eb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f7ac191c0 con 0x557f7ac218b0 2023-11-27T15:24:50.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.508+0000 7fd2e214eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f7ac20470 con 0x557f7ac1cb60 2023-11-27T15:24:50.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.508+0000 7fd2ceffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f7ac1d470 0x557f7ac2d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.508+0000 7fd2ceffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f7ac1d470 0x557f7ac2d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54650/0 (socket says 172.21.15.52:54650) 2023-11-27T15:24:50.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.508+0000 7fd2ceffd700 1 -- 172.21.15.52:0/4122142790 learned_addr learned my addr 172.21.15.52:0/4122142790 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:50.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.508+0000 7fd2ceffd700 1 -- 172.21.15.52:0/4122142790 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f7ac1cb60 msgr2=0x557f7ac17be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:50.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.508+0000 7fd2ceffd700 1 --2- 172.21.15.52:0/4122142790 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f7ac1cb60 0x557f7ac17be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.508+0000 7fd2cffff700 1 --2- 172.21.15.52:0/4122142790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f7ac218b0 0x557f7ac28bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.508+0000 7fd2ceffd700 1 -- 172.21.15.52:0/4122142790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f7ac218b0 msgr2=0x557f7ac28bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:50.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.508+0000 7fd2ceffd700 1 --2- 172.21.15.52:0/4122142790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f7ac218b0 0x557f7ac28bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.508+0000 7fd2ceffd700 1 -- 172.21.15.52:0/4122142790 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f7ab33ee0 con 0x557f7ac1d470 2023-11-27T15:24:50.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.508+0000 7fd2cffff700 1 --2- 172.21.15.52:0/4122142790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f7ac218b0 0x557f7ac28bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:24:50.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.508+0000 7fd2ceffd700 1 --2- 172.21.15.52:0/4122142790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f7ac1d470 0x557f7ac2d9e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd2d006a460 tx=0x7fd2b8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f365328d2426685 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:50.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.508+0000 7fd2c67fc700 1 -- 172.21.15.52:0/4122142790 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b801fd00 con 0x557f7ac1d470 2023-11-27T15:24:50.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.508+0000 7fd2c67fc700 1 -- 172.21.15.52:0/4122142790 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2b8018040 con 0x557f7ac1d470 2023-11-27T15:24:50.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.508+0000 7fd2c67fc700 1 -- 172.21.15.52:0/4122142790 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8005020 con 0x557f7ac1d470 2023-11-27T15:24:50.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.508+0000 7fd2e214eb80 1 -- 172.21.15.52:0/4122142790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f7ac1d470 msgr2=0x557f7ac2d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:50.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.508+0000 7fd2e214eb80 1 --2- 172.21.15.52:0/4122142790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f7ac1d470 0x557f7ac2d9e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd2d006a460 tx=0x7fd2b8009260 comp rx=0 tx=0).stop 2023-11-27T15:24:50.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.509+0000 7fd2e214eb80 1 -- 172.21.15.52:0/4122142790 shutdown_connections 2023-11-27T15:24:50.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.509+0000 7fd2e214eb80 1 --2- 172.21.15.52:0/4122142790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f7ac218b0 0x557f7ac28bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.509+0000 7fd2e214eb80 1 --2- 172.21.15.52:0/4122142790 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f7ac1cb60 0x557f7ac17be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.509+0000 7fd2e214eb80 1 --2- 172.21.15.52:0/4122142790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f7ac1d470 0x557f7ac2d9e0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.509+0000 7fd2e214eb80 1 -- 172.21.15.52:0/4122142790 >> 172.21.15.52:0/4122142790 conn(0x557f7ac12360 msgr2=0x557f7ac38860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:50.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.509+0000 7fd2e214eb80 1 -- 172.21.15.52:0/4122142790 shutdown_connections 2023-11-27T15:24:50.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.509+0000 7fd2e214eb80 1 -- 172.21.15.52:0/4122142790 wait complete. 2023-11-27T15:24:50.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.509+0000 7fd2e214eb80 1 -- start start 2023-11-27T15:24:50.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.509+0000 7fd2e214eb80 1 Processor -- start 2023-11-27T15:24:50.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.509+0000 7fd2e214eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f7ac1cb60 0x557f7ac7e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.509+0000 7fd2e214eb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f7ac1d470 0x557f7ac7eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.509+0000 7fd2e214eb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f7ac218b0 0x557f7ac7f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.509+0000 7fd2e214eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f7ac897c0 con 0x557f7ac1cb60 2023-11-27T15:24:50.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.509+0000 7fd2e214eb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f7ac89900 con 0x557f7ac1d470 2023-11-27T15:24:50.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.509+0000 7fd2e214eb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f7ac89ac0 con 0x557f7ac218b0 2023-11-27T15:24:50.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.509+0000 7fd2e214eb80 10 client.0 _ll_register_callbacks cb 0x7fd2b8009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:50.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.509+0000 7fd2cf7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f7ac1cb60 0x557f7ac7e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.510+0000 7fd2cf7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f7ac1cb60 0x557f7ac7e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54654/0 (socket says 172.21.15.52:54654) 2023-11-27T15:24:50.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.510+0000 7fd2cf7fe700 1 -- 172.21.15.52:0/1514120338 learned_addr learned my addr 172.21.15.52:0/1514120338 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:50.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.510+0000 7fd2ceffd700 1 --2- 172.21.15.52:0/1514120338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f7ac1d470 0x557f7ac7eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.510+0000 7fd2cffff700 1 --2- 172.21.15.52:0/1514120338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f7ac218b0 0x557f7ac7f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.510+0000 7fd2cf7fe700 1 -- 172.21.15.52:0/1514120338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f7ac218b0 msgr2=0x557f7ac7f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:50.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.510+0000 7fd2cf7fe700 1 --2- 172.21.15.52:0/1514120338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f7ac218b0 0x557f7ac7f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.510+0000 7fd2cf7fe700 1 -- 172.21.15.52:0/1514120338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f7ac1d470 msgr2=0x557f7ac7eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:50.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.510+0000 7fd2cf7fe700 1 --2- 172.21.15.52:0/1514120338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f7ac1d470 0x557f7ac7eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.510+0000 7fd2cf7fe700 1 -- 172.21.15.52:0/1514120338 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b8009e60 con 0x557f7ac1cb60 2023-11-27T15:24:50.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.510+0000 7fd2cf7fe700 1 --2- 172.21.15.52:0/1514120338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f7ac1cb60 0x557f7ac7e660 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0011010 tx=0x7fd2c00398d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2ed17d98bc7a79 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:50.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.510+0000 7fd2aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:24:50.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.510+0000 7fd2aeffd700 1 -- 172.21.15.52:0/1514120338 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c002c030 con 0x557f7ac1cb60 2023-11-27T15:24:50.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.510+0000 7fd2aeffd700 1 -- 172.21.15.52:0/1514120338 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fd2c002d070 con 0x557f7ac1cb60 2023-11-27T15:24:50.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.510+0000 7fd2e214eb80 1 -- 172.21.15.52:0/1514120338 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2c0015070 con 0x557f7ac1cb60 2023-11-27T15:24:50.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.510+0000 7fd2aeffd700 1 -- 172.21.15.52:0/1514120338 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2c0039bc0 con 0x557f7ac1cb60 2023-11-27T15:24:50.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.510+0000 7fd2e214eb80 10 client.16515 fetch_fsmap learned FSMap version 28 2023-11-27T15:24:50.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.510+0000 7fd2e214eb80 1 -- 172.21.15.52:0/1514120338 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557f7a984c60 con 0x557f7ac1cb60 2023-11-27T15:24:50.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.510+0000 7fd2aeffd700 1 -- 172.21.15.52:0/1514120338 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2c0047080 con 0x557f7ac1cb60 2023-11-27T15:24:50.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.511+0000 7fd2aeffd700 1 -- 172.21.15.52:0/1514120338 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd2c0039d20 con 0x557f7ac1cb60 2023-11-27T15:24:50.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.511+0000 7fd2e214eb80 10 client.16515 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:24:50.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.511+0000 7fd2e214eb80 10 client.16515 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:50.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.511+0000 7fd2e214eb80 20 client.16515 populate_metadata read hostname 'smithi052' 2023-11-27T15:24:50.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.511+0000 7fd2e214eb80 1 --2- 172.21.15.52:0/1514120338 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557f7ac8aec0 0x557f7ac8d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.511+0000 7fd2e214eb80 4 client.16515 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:24:50.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.511+0000 7fd2e214eb80 1 -- 172.21.15.52:0/1514120338 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557f7ab3c660 con 0x557f7ac8aec0 2023-11-27T15:24:50.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.511+0000 7fd2ceffd700 1 --2- 172.21.15.52:0/1514120338 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557f7ac8aec0 0x557f7ac8d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.511+0000 7fd2ceffd700 1 --2- 172.21.15.52:0/1514120338 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557f7ac8aec0 0x557f7ac8d2d0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e910dd7b0039b7c2 server_cookie=1dc7d7c2df2d8791 in_seq=0 out_seq=0 2023-11-27T15:24:50.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.511+0000 7fd2aeffd700 10 client.16515 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:50.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.513+0000 7fd2aeffd700 1 -- 172.21.15.52:0/1514120338 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557f7ab3c660 con 0x557f7ac8aec0 2023-11-27T15:24:50.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.513+0000 7fd2aeffd700 10 client.16515 handle_command_reply: tid=0 2023-11-27T15:24:50.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.513+0000 7fd2e214eb80 1 client.16515 shutdown 2023-11-27T15:24:50.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.513+0000 7fd2e214eb80 1 -- 172.21.15.52:0/1514120338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f7ac1cb60 msgr2=0x557f7ac7e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:50.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.513+0000 7fd2e214eb80 1 --2- 172.21.15.52:0/1514120338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f7ac1cb60 0x557f7ac7e660 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0011010 tx=0x7fd2c00398d0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.513+0000 7fd2e214eb80 1 -- 172.21.15.52:0/1514120338 shutdown_connections 2023-11-27T15:24:50.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.513+0000 7fd2e214eb80 1 --2- 172.21.15.52:0/1514120338 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557f7ac8aec0 0x557f7ac8d2d0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.513+0000 7fd2e214eb80 1 --2- 172.21.15.52:0/1514120338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f7ac218b0 0x557f7ac7f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.513+0000 7fd2e214eb80 1 --2- 172.21.15.52:0/1514120338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f7ac1cb60 0x557f7ac7e660 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.513+0000 7fd2aeffd700 0 client.16515 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:50.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.513+0000 7fd2e214eb80 1 --2- 172.21.15.52:0/1514120338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557f7ac1d470 0x557f7ac7eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.513+0000 7fd2e214eb80 1 -- 172.21.15.52:0/1514120338 >> 172.21.15.52:0/1514120338 conn(0x557f7ac12360 msgr2=0x557f7ac1f970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:50.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.513+0000 7fd2e214eb80 1 -- 172.21.15.52:0/1514120338 shutdown_connections 2023-11-27T15:24:50.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.513+0000 7fd2e214eb80 1 -- 172.21.15.52:0/1514120338 wait complete. 2023-11-27T15:24:50.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.513+0000 7fd2e214eb80 20 client.16515 trim_cache size 0 max 16384 2023-11-27T15:24:50.528 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:24:50.528 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2023-11-27T15:24:50.528 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2023-11-27T15:24:50.528 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:24:50.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.513+0000 7fd2d5fc2700 1 -- 172.21.15.52:0/3535667266 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd2bc0a1340 msgr2=0x7fd2bc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:50.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.513+0000 7fd2d5fc2700 1 --2- 172.21.15.52:0/3535667266 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd2bc0a1340 0x7fd2bc0a37f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0005950 tx=0x7fd2c000b410 comp rx=0 tx=0).stop 2023-11-27T15:24:50.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.513+0000 7fd2d5fc2700 1 -- 172.21.15.52:0/3535667266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd2d01045d0 msgr2=0x7fd2d019aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:50.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.513+0000 7fd2d5fc2700 1 --2- 172.21.15.52:0/3535667266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd2d01045d0 0x7fd2d019aef0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8007870 tx=0x7fd2c80078a0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.513+0000 7fd2d5fc2700 1 -- 172.21.15.52:0/3535667266 shutdown_connections 2023-11-27T15:24:50.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.513+0000 7fd2d5fc2700 1 --2- 172.21.15.52:0/3535667266 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd2bc0a1340 0x7fd2bc0a37f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.513+0000 7fd2d5fc2700 1 --2- 172.21.15.52:0/3535667266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2d00ff590 0x7fd2d0196550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.513+0000 7fd2d5fc2700 1 --2- 172.21.15.52:0/3535667266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd2d01045d0 0x7fd2d019aef0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.513+0000 7fd2d5fc2700 1 --2- 172.21.15.52:0/3535667266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2d00fff30 0x7fd2d0196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.513+0000 7fd2d5fc2700 1 -- 172.21.15.52:0/3535667266 >> 172.21.15.52:0/3535667266 conn(0x7fd2d00760d0 msgr2=0x7fd2d0068de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:50.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.513+0000 7fd2d5fc2700 1 -- 172.21.15.52:0/3535667266 shutdown_connections 2023-11-27T15:24:50.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.513+0000 7fd2d5fc2700 1 -- 172.21.15.52:0/3535667266 wait complete. 2023-11-27T15:24:50.531 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-27T15:24:50.531 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:24:50.695 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.691+0000 7f41aff5b700 1 -- 172.21.15.52:0/1225488341 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41a80ff470 msgr2=0x7f41a80ff840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:50.696 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.691+0000 7f41aff5b700 1 --2- 172.21.15.52:0/1225488341 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41a80ff470 0x7f41a80ff840 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f41a40099c0 tx=0x7f41a4009cd0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.696 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.691+0000 7f41aff5b700 1 -- 172.21.15.52:0/1225488341 shutdown_connections 2023-11-27T15:24:50.696 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.691+0000 7f41aff5b700 1 --2- 172.21.15.52:0/1225488341 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41a8104460 0x7f41a810c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.696 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.691+0000 7f41aff5b700 1 --2- 172.21.15.52:0/1225488341 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41a80ff470 0x7f41a80ff840 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.696 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.691+0000 7f41aff5b700 1 --2- 172.21.15.52:0/1225488341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41a80ffe10 0x7f41a8103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.696 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.691+0000 7f41aff5b700 1 -- 172.21.15.52:0/1225488341 >> 172.21.15.52:0/1225488341 conn(0x7f41a8076100 msgr2=0x7f41a8076500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:50.696 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.691+0000 7f41aff5b700 1 -- 172.21.15.52:0/1225488341 shutdown_connections 2023-11-27T15:24:50.697 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.691+0000 7f41aff5b700 1 -- 172.21.15.52:0/1225488341 wait complete. 2023-11-27T15:24:50.699 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.692+0000 7f41aff5b700 1 Processor -- start 2023-11-27T15:24:50.699 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.692+0000 7f41aff5b700 1 -- start start 2023-11-27T15:24:50.699 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.692+0000 7f41aff5b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41a80ff470 0x7f41a8195360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.699 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.692+0000 7f41aff5b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41a80ffe10 0x7f41a81958a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.692+0000 7f41aff5b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41a8104460 0x7f41a8199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.693+0000 7f41aff5b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a819a2a0 con 0x7f41a8104460 2023-11-27T15:24:50.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.693+0000 7f41aff5b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a819a410 con 0x7f41a80ffe10 2023-11-27T15:24:50.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.693+0000 7f41aff5b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a819a5d0 con 0x7f41a80ff470 2023-11-27T15:24:50.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.693+0000 7f41ae4f8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41a8104460 0x7f41a8199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.693+0000 7f41ae4f8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41a8104460 0x7f41a8199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54672/0 (socket says 172.21.15.52:54672) 2023-11-27T15:24:50.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.693+0000 7f41ae4f8700 1 -- 172.21.15.52:0/1262119141 learned_addr learned my addr 172.21.15.52:0/1262119141 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:50.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.693+0000 7f41adcf7700 1 --2- 172.21.15.52:0/1262119141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41a80ff470 0x7f41a8195360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.693+0000 7f41ad4f6700 1 --2- 172.21.15.52:0/1262119141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41a80ffe10 0x7f41a81958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.693+0000 7f41ae4f8700 1 -- 172.21.15.52:0/1262119141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41a80ff470 msgr2=0x7f41a8195360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:50.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.693+0000 7f41ae4f8700 1 --2- 172.21.15.52:0/1262119141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41a80ff470 0x7f41a8195360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.693+0000 7f41ae4f8700 1 -- 172.21.15.52:0/1262119141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41a80ffe10 msgr2=0x7f41a81958a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:50.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.693+0000 7f41ae4f8700 1 --2- 172.21.15.52:0/1262119141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41a80ffe10 0x7f41a81958a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.693+0000 7f41ae4f8700 1 -- 172.21.15.52:0/1262119141 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a40095f0 con 0x7f41a8104460 2023-11-27T15:24:50.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.693+0000 7f41ae4f8700 1 --2- 172.21.15.52:0/1262119141 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41a8104460 0x7f41a8199c60 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f41a0007ee0 tx=0x7f41a000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:50.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.693+0000 7f419affd700 1 -- 172.21.15.52:0/1262119141 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a0017070 con 0x7f41a8104460 2023-11-27T15:24:50.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.693+0000 7f419affd700 1 -- 172.21.15.52:0/1262119141 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41a000f040 con 0x7f41a8104460 2023-11-27T15:24:50.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.693+0000 7f41aff5b700 1 -- 172.21.15.52:0/1262119141 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41a810f270 con 0x7f41a8104460 2023-11-27T15:24:50.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.693+0000 7f419affd700 1 -- 172.21.15.52:0/1262119141 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a0005580 con 0x7f41a8104460 2023-11-27T15:24:50.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.693+0000 7f41aff5b700 1 -- 172.21.15.52:0/1262119141 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a810f0e0 con 0x7f41a8104460 2023-11-27T15:24:50.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.694+0000 7f41aff5b700 1 -- 172.21.15.52:0/1262119141 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41a810f400 con 0x7f41a8104460 2023-11-27T15:24:50.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.696+0000 7f419affd700 1 -- 172.21.15.52:0/1262119141 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f41a000db20 con 0x7f41a8104460 2023-11-27T15:24:50.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.696+0000 7f419affd700 1 --2- 172.21.15.52:0/1262119141 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f41940a1340 0x7f41940a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.696+0000 7f419affd700 1 -- 172.21.15.52:0/1262119141 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f41a00cc390 con 0x7f41a8104460 2023-11-27T15:24:50.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.696+0000 7f41adcf7700 1 --2- 172.21.15.52:0/1262119141 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f41940a1340 0x7f41940a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.697+0000 7f41adcf7700 1 --2- 172.21.15.52:0/1262119141 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f41940a1340 0x7f41940a37f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f41a4009990 tx=0x7f41a4005fb0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:50.704 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.701+0000 7f419affd700 1 -- 172.21.15.52:0/1262119141 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f41a0095770 con 0x7f41a8104460 2023-11-27T15:24:50.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.947+0000 7f41aff5b700 1 -- 172.21.15.52:0/1262119141 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41a8067eb0 con 0x7f41a8104460 2023-11-27T15:24:50.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.948+0000 7f419affd700 1 -- 172.21.15.52:0/1262119141 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10813 (secure 0 0 0) 0x7f41a0098d90 con 0x7f41a8104460 2023-11-27T15:24:50.951 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:24:50.952 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2328881376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2328881376},{"type":"v1","addr":"172.21.15.52:6839","nonce":2328881376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2799551570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2799551570},{"type":"v1","addr":"172.21.15.52:6841","nonce":2799551570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:16:09.319630+0000","modified":"2023-11-27T15:21:19.128122+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24494,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/215725838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":215725838},{"type":"v1","addr":"172.21.15.150:6839","nonce":215725838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3571830559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3571830559},{"type":"v1","addr":"172.21.15.102:6839","nonce":3571830559}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:24:50.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.950+0000 7f41aff5b700 1 -- 172.21.15.52:0/1262119141 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f41940a1340 msgr2=0x7f41940a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:50.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.950+0000 7f41aff5b700 1 --2- 172.21.15.52:0/1262119141 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f41940a1340 0x7f41940a37f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f41a4009990 tx=0x7f41a4005fb0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.950+0000 7f41aff5b700 1 -- 172.21.15.52:0/1262119141 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41a8104460 msgr2=0x7f41a8199c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:50.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.950+0000 7f41aff5b700 1 --2- 172.21.15.52:0/1262119141 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41a8104460 0x7f41a8199c60 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f41a0007ee0 tx=0x7f41a000d3b0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.950+0000 7f41aff5b700 1 -- 172.21.15.52:0/1262119141 shutdown_connections 2023-11-27T15:24:50.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.950+0000 7f41aff5b700 1 --2- 172.21.15.52:0/1262119141 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f41940a1340 0x7f41940a37f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.950+0000 7f41aff5b700 1 --2- 172.21.15.52:0/1262119141 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41a8104460 0x7f41a8199c60 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.950+0000 7f41aff5b700 1 --2- 172.21.15.52:0/1262119141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41a80ff470 0x7f41a8195360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.950+0000 7f41aff5b700 1 --2- 172.21.15.52:0/1262119141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41a80ffe10 0x7f41a81958a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.950+0000 7f41aff5b700 1 -- 172.21.15.52:0/1262119141 >> 172.21.15.52:0/1262119141 conn(0x7f41a8076100 msgr2=0x7f41a80fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:50.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.950+0000 7f41aff5b700 1 -- 172.21.15.52:0/1262119141 shutdown_connections 2023-11-27T15:24:50.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:50.950+0000 7f41aff5b700 1 -- 172.21.15.52:0/1262119141 wait complete. 2023-11-27T15:24:50.956 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 28 2023-11-27T15:24:50.966 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-27T15:24:51.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.091+0000 7fa0c0965700 1 -- 172.21.15.52:0/446363016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0bc100f50 msgr2=0x7fa0bc104f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.096 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.091+0000 7fa0c0965700 1 --2- 172.21.15.52:0/446363016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0bc100f50 0x7fa0bc104f10 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fa0b00099c0 tx=0x7fa0b0009cd0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.096 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.091+0000 7fa0c0965700 1 -- 172.21.15.52:0/446363016 shutdown_connections 2023-11-27T15:24:51.096 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.091+0000 7fa0c0965700 1 --2- 172.21.15.52:0/446363016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0bc1005b0 0x7fa0bc100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.096 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.091+0000 7fa0c0965700 1 --2- 172.21.15.52:0/446363016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0bc1055f0 0x7fa0bc10d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.096 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.091+0000 7fa0c0965700 1 --2- 172.21.15.52:0/446363016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0bc100f50 0x7fa0bc104f10 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.096 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.091+0000 7fa0c0965700 1 -- 172.21.15.52:0/446363016 >> 172.21.15.52:0/446363016 conn(0x7fa0bc077170 msgr2=0x7fa0bc077570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:51.096 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.092+0000 7fa0c0965700 1 -- 172.21.15.52:0/446363016 shutdown_connections 2023-11-27T15:24:51.097 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.092+0000 7fa0c0965700 1 -- 172.21.15.52:0/446363016 wait complete. 2023-11-27T15:24:51.099 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.092+0000 7fa0c0965700 1 Processor -- start 2023-11-27T15:24:51.099 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.093+0000 7fa0c0965700 1 -- start start 2023-11-27T15:24:51.099 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.093+0000 7fa0c0965700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0bc1005b0 0x7fa0bc1963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.099 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.093+0000 7fa0c0965700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0bc100f50 0x7fa0bc196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.099 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.093+0000 7fa0c0965700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0bc1055f0 0x7fa0bc19ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.099 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.093+0000 7fa0c0965700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc19b3c0 con 0x7fa0bc1055f0 2023-11-27T15:24:51.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.093+0000 7fa0c0965700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc19b530 con 0x7fa0bc1005b0 2023-11-27T15:24:51.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.093+0000 7fa0c0965700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc19b6f0 con 0x7fa0bc100f50 2023-11-27T15:24:51.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.093+0000 7fa0bad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0bc1055f0 0x7fa0bc19ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.093+0000 7fa0bad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0bc1055f0 0x7fa0bc19ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54686/0 (socket says 172.21.15.52:54686) 2023-11-27T15:24:51.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.093+0000 7fa0bad9d700 1 -- 172.21.15.52:0/3132196318 learned_addr learned my addr 172.21.15.52:0/3132196318 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:51.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.093+0000 7fa0b9d9b700 1 --2- 172.21.15.52:0/3132196318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0bc100f50 0x7fa0bc196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.093+0000 7fa0ba59c700 1 --2- 172.21.15.52:0/3132196318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0bc1005b0 0x7fa0bc1963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.093+0000 7fa0bad9d700 1 -- 172.21.15.52:0/3132196318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0bc100f50 msgr2=0x7fa0bc196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.093+0000 7fa0bad9d700 1 --2- 172.21.15.52:0/3132196318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0bc100f50 0x7fa0bc196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.093+0000 7fa0bad9d700 1 -- 172.21.15.52:0/3132196318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0bc1005b0 msgr2=0x7fa0bc1963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.093+0000 7fa0bad9d700 1 --2- 172.21.15.52:0/3132196318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0bc1005b0 0x7fa0bc1963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.093+0000 7fa0bad9d700 1 -- 172.21.15.52:0/3132196318 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b00095f0 con 0x7fa0bc1055f0 2023-11-27T15:24:51.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.094+0000 7fa0bad9d700 1 --2- 172.21.15.52:0/3132196318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0bc1055f0 0x7fa0bc19ad80 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4007ee0 tx=0x7fa0b400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:51.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.094+0000 7fa0ab7fe700 1 -- 172.21.15.52:0/3132196318 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b400ddd0 con 0x7fa0bc1055f0 2023-11-27T15:24:51.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.094+0000 7fa0ab7fe700 1 -- 172.21.15.52:0/3132196318 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0b400f040 con 0x7fa0bc1055f0 2023-11-27T15:24:51.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.094+0000 7fa0c0965700 1 -- 172.21.15.52:0/3132196318 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0bc110370 con 0x7fa0bc1055f0 2023-11-27T15:24:51.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.094+0000 7fa0ab7fe700 1 -- 172.21.15.52:0/3132196318 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b40048e0 con 0x7fa0bc1055f0 2023-11-27T15:24:51.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.094+0000 7fa0c0965700 1 -- 172.21.15.52:0/3132196318 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0bc1101e0 con 0x7fa0bc1055f0 2023-11-27T15:24:51.108 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.096+0000 7fa0ab7fe700 1 -- 172.21.15.52:0/3132196318 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fa0b4003a00 con 0x7fa0bc1055f0 2023-11-27T15:24:51.108 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.096+0000 7fa0ab7fe700 1 --2- 172.21.15.52:0/3132196318 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fa0a40a1340 0x7fa0a40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.108 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.097+0000 7fa0ab7fe700 1 -- 172.21.15.52:0/3132196318 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fa0b40cc590 con 0x7fa0bc1055f0 2023-11-27T15:24:51.108 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.097+0000 7fa0ba59c700 1 --2- 172.21.15.52:0/3132196318 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fa0a40a1340 0x7fa0a40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.108 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.097+0000 7fa0ba59c700 1 --2- 172.21.15.52:0/3132196318 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fa0a40a1340 0x7fa0a40a37f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac005950 tx=0x7fa0ac0058e0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:51.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.097+0000 7fa0ccaf1b80 1 Processor -- start 2023-11-27T15:24:51.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.098+0000 7fa0ccaf1b80 1 -- start start 2023-11-27T15:24:51.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.098+0000 7fa0ccaf1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624786f0b60 0x5624786f0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.098+0000 7fa0ccaf1b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624786f1470 0x5624786f5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.098+0000 7fa0ccaf1b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5624786f58b0 0x5624786f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.098+0000 7fa0ccaf1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624785f6060 con 0x5624786f0b60 2023-11-27T15:24:51.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.098+0000 7fa0ccaf1b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624786488a0 con 0x5624786f58b0 2023-11-27T15:24:51.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.098+0000 7fa0ccaf1b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562478641d30 con 0x5624786f1470 2023-11-27T15:24:51.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.098+0000 7fa0ba59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624786f0b60 0x5624786f0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.098+0000 7fa0ba59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624786f0b60 0x5624786f0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54690/0 (socket says 172.21.15.52:54690) 2023-11-27T15:24:51.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.098+0000 7fa0ba59c700 1 -- 172.21.15.52:0/2850232277 learned_addr learned my addr 172.21.15.52:0/2850232277 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:51.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.098+0000 7fa0b9d9b700 1 --2- 172.21.15.52:0/2850232277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624786f1470 0x5624786f5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.098+0000 7fa0bad9d700 1 --2- 172.21.15.52:0/2850232277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5624786f58b0 0x5624786f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.098+0000 7fa0ba59c700 1 -- 172.21.15.52:0/2850232277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624786f1470 msgr2=0x5624786f5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:51.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.098+0000 7fa0ba59c700 1 --2- 172.21.15.52:0/2850232277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624786f1470 0x5624786f5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.098+0000 7fa0ba59c700 1 -- 172.21.15.52:0/2850232277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5624786f58b0 msgr2=0x5624786f5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:51.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.098+0000 7fa0ba59c700 1 --2- 172.21.15.52:0/2850232277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5624786f58b0 0x5624786f5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.098+0000 7fa0ba59c700 1 -- 172.21.15.52:0/2850232277 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624784c3630 con 0x5624786f0b60 2023-11-27T15:24:51.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.098+0000 7fa0b9d9b700 1 --2- 172.21.15.52:0/2850232277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624786f1470 0x5624786f5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:24:51.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.098+0000 7fa0bad9d700 1 --2- 172.21.15.52:0/2850232277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5624786f58b0 0x5624786f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:24:51.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.098+0000 7fa0ba59c700 1 --2- 172.21.15.52:0/2850232277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624786f0b60 0x5624786f0f30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00fd70 tx=0x7fa0ac012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d35c8955747df5 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:51.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.099+0000 7fa0a8ff9700 1 -- 172.21.15.52:0/2850232277 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac011c60 con 0x5624786f0b60 2023-11-27T15:24:51.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.099+0000 7fa0a8ff9700 1 -- 172.21.15.52:0/2850232277 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0ac011dc0 con 0x5624786f0b60 2023-11-27T15:24:51.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.099+0000 7fa0a8ff9700 1 -- 172.21.15.52:0/2850232277 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac01d750 con 0x5624786f0b60 2023-11-27T15:24:51.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.099+0000 7fa0ccaf1b80 1 -- 172.21.15.52:0/2850232277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624786f0b60 msgr2=0x5624786f0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.099+0000 7fa0ccaf1b80 1 --2- 172.21.15.52:0/2850232277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624786f0b60 0x5624786f0f30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00fd70 tx=0x7fa0ac012000 comp rx=0 tx=0).stop 2023-11-27T15:24:51.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.099+0000 7fa0ccaf1b80 1 -- 172.21.15.52:0/2850232277 shutdown_connections 2023-11-27T15:24:51.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.099+0000 7fa0ccaf1b80 1 --2- 172.21.15.52:0/2850232277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5624786f58b0 0x5624786f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.099+0000 7fa0ccaf1b80 1 --2- 172.21.15.52:0/2850232277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624786f0b60 0x5624786f0f30 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.099+0000 7fa0ccaf1b80 1 --2- 172.21.15.52:0/2850232277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624786f1470 0x5624786f5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.099+0000 7fa0ccaf1b80 1 -- 172.21.15.52:0/2850232277 >> 172.21.15.52:0/2850232277 conn(0x5624786e6360 msgr2=0x5624786ebb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:51.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.100+0000 7fa0ccaf1b80 1 -- 172.21.15.52:0/2850232277 shutdown_connections 2023-11-27T15:24:51.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.100+0000 7fa0ccaf1b80 1 -- 172.21.15.52:0/2850232277 wait complete. 2023-11-27T15:24:51.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.100+0000 7fa0ccaf1b80 1 -- start start 2023-11-27T15:24:51.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.100+0000 7fa0ccaf1b80 1 Processor -- start 2023-11-27T15:24:51.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.100+0000 7fa0ccaf1b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5624786f0b60 0x5624786fd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.100+0000 7fa0ccaf1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624786f1470 0x5624786fa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.100+0000 7fa0ccaf1b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624786f58b0 0x5624786fa870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.100+0000 7fa0ccaf1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624785e38e0 con 0x5624786f1470 2023-11-27T15:24:51.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.100+0000 7fa0ccaf1b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624785e3580 con 0x5624786f0b60 2023-11-27T15:24:51.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.100+0000 7fa0ccaf1b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624784c5920 con 0x5624786f58b0 2023-11-27T15:24:51.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.101+0000 7fa0ccaf1b80 10 client.0 _ll_register_callbacks cb 0x562478601780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:51.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.101+0000 7fa0b9d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624786f1470 0x5624786fa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.101+0000 7fa0b9d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624786f1470 0x5624786fa260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54698/0 (socket says 172.21.15.52:54698) 2023-11-27T15:24:51.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.101+0000 7fa0b9d9b700 1 -- 172.21.15.52:0/3659378584 learned_addr learned my addr 172.21.15.52:0/3659378584 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:51.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.101+0000 7fa0ba59c700 1 --2- 172.21.15.52:0/3659378584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5624786f0b60 0x5624786fd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.101+0000 7fa0bad9d700 1 --2- 172.21.15.52:0/3659378584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624786f58b0 0x5624786fa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.101+0000 7fa0b9d9b700 1 -- 172.21.15.52:0/3659378584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624786f58b0 msgr2=0x5624786fa870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:51.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.101+0000 7fa0b9d9b700 1 --2- 172.21.15.52:0/3659378584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624786f58b0 0x5624786fa870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.101+0000 7fa0b9d9b700 1 -- 172.21.15.52:0/3659378584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5624786f0b60 msgr2=0x5624786fd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:51.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.101+0000 7fa0b9d9b700 1 --2- 172.21.15.52:0/3659378584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5624786f0b60 0x5624786fd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.101+0000 7fa0b9d9b700 1 -- 172.21.15.52:0/3659378584 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0ac00fa20 con 0x5624786f1470 2023-11-27T15:24:51.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.101+0000 7fa0b9d9b700 1 --2- 172.21.15.52:0/3659378584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624786f1470 0x5624786fa260 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0000c00 tx=0x7fa0b0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf4ee2ff4b1324b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:51.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.101+0000 7fa0a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:24:51.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.101+0000 7fa0a1ffb700 1 -- 172.21.15.52:0/3659378584 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b000bba0 con 0x5624786f1470 2023-11-27T15:24:51.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.101+0000 7fa0a1ffb700 1 -- 172.21.15.52:0/3659378584 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fa0b000bd00 con 0x5624786f1470 2023-11-27T15:24:51.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.101+0000 7fa0ccaf1b80 1 -- 172.21.15.52:0/3659378584 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5624786e3b20 con 0x5624786f1470 2023-11-27T15:24:51.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.101+0000 7fa0a1ffb700 1 -- 172.21.15.52:0/3659378584 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0b000bec0 con 0x5624786f1470 2023-11-27T15:24:51.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.101+0000 7fa0ccaf1b80 10 client.16533 fetch_fsmap learned FSMap version 28 2023-11-27T15:24:51.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.101+0000 7fa0ccaf1b80 1 -- 172.21.15.52:0/3659378584 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562478608190 con 0x5624786f1470 2023-11-27T15:24:51.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.102+0000 7fa0a1ffb700 1 -- 172.21.15.52:0/3659378584 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5624786e3b20 con 0x5624786f1470 2023-11-27T15:24:51.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.102+0000 7fa0a1ffb700 1 -- 172.21.15.52:0/3659378584 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa0b0004620 con 0x5624786f1470 2023-11-27T15:24:51.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.102+0000 7fa0ccaf1b80 10 client.16533 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:24:51.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.102+0000 7fa0ccaf1b80 10 client.16533 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:24:51.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.102+0000 7fa0ccaf1b80 20 client.16533 populate_metadata read hostname 'smithi052' 2023-11-27T15:24:51.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.102+0000 7fa0ccaf1b80 1 --2- 172.21.15.52:0/3659378584 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x56247870a860 0x56247870cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.102+0000 7fa0ccaf1b80 4 client.16533 mds_command: new command op to 24472 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:24:51.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.102+0000 7fa0ccaf1b80 1 -- 172.21.15.52:0/3659378584 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5624786040a0 con 0x56247870a860 2023-11-27T15:24:51.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.102+0000 7fa0ba59c700 1 --2- 172.21.15.52:0/3659378584 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x56247870a860 0x56247870cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.103+0000 7fa0a1ffb700 10 client.16533 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:24:51.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.103+0000 7fa0ba59c700 1 --2- 172.21.15.52:0/3659378584 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x56247870a860 0x56247870cc70 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e2a715ee84891d11 server_cookie=d75cd22ac0642ed4 in_seq=0 out_seq=0 2023-11-27T15:24:51.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.104+0000 7fa0a1ffb700 1 -- 172.21.15.52:0/3659378584 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5624786040a0 con 0x56247870a860 2023-11-27T15:24:51.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.104+0000 7fa0a1ffb700 10 client.16533 handle_command_reply: tid=0 2023-11-27T15:24:51.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.104+0000 7fa0ccaf1b80 1 client.16533 shutdown 2023-11-27T15:24:51.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.104+0000 7fa0ccaf1b80 1 -- 172.21.15.52:0/3659378584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624786f1470 msgr2=0x5624786fa260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.104+0000 7fa0ccaf1b80 1 --2- 172.21.15.52:0/3659378584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624786f1470 0x5624786fa260 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0000c00 tx=0x7fa0b0005e80 comp rx=0 tx=0).stop 2023-11-27T15:24:51.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.104+0000 7fa0ccaf1b80 1 -- 172.21.15.52:0/3659378584 shutdown_connections 2023-11-27T15:24:51.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.104+0000 7fa0ccaf1b80 1 --2- 172.21.15.52:0/3659378584 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x56247870a860 0x56247870cc70 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.104+0000 7fa0ccaf1b80 1 --2- 172.21.15.52:0/3659378584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5624786f0b60 0x5624786fd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.104+0000 7fa0a1ffb700 0 client.16533 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:24:51.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.104+0000 7fa0ccaf1b80 1 --2- 172.21.15.52:0/3659378584 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624786f58b0 0x5624786fa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.104+0000 7fa0ccaf1b80 1 --2- 172.21.15.52:0/3659378584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624786f1470 0x5624786fa260 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.104+0000 7fa0ccaf1b80 1 -- 172.21.15.52:0/3659378584 >> 172.21.15.52:0/3659378584 conn(0x5624786e6360 msgr2=0x5624786f6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:51.120 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.104+0000 7fa0ccaf1b80 1 -- 172.21.15.52:0/3659378584 shutdown_connections 2023-11-27T15:24:51.120 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.104+0000 7fa0ccaf1b80 1 -- 172.21.15.52:0/3659378584 wait complete. 2023-11-27T15:24:51.120 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.104+0000 7fa0ccaf1b80 20 client.16533 trim_cache size 0 max 16384 2023-11-27T15:24:51.126 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.117+0000 7fa0ccaf1b80 1 Processor -- start 2023-11-27T15:24:51.126 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.117+0000 7fa0ccaf1b80 1 -- start start 2023-11-27T15:24:51.126 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.117+0000 7fa0ccaf1b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624786f0b60 0x5624787019a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.126 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.117+0000 7fa0ccaf1b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5624786f1470 0x5624786fcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.126 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.117+0000 7fa0ccaf1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624786f58b0 0x5624786f38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.126 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.117+0000 7fa0ccaf1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624786f87c0 con 0x5624786f58b0 2023-11-27T15:24:51.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.117+0000 7fa0ccaf1b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624786f43e0 con 0x5624786f1470 2023-11-27T15:24:51.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.117+0000 7fa0ccaf1b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624786fae70 con 0x5624786f0b60 2023-11-27T15:24:51.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.117+0000 7fa0bad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624786f58b0 0x5624786f38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.117+0000 7fa0bad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624786f58b0 0x5624786f38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54708/0 (socket says 172.21.15.52:54708) 2023-11-27T15:24:51.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.117+0000 7fa0bad9d700 1 -- 172.21.15.52:0/2109894133 learned_addr learned my addr 172.21.15.52:0/2109894133 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:51.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.117+0000 7fa0ba59c700 1 --2- 172.21.15.52:0/2109894133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624786f0b60 0x5624787019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.117+0000 7fa0b9d9b700 1 --2- 172.21.15.52:0/2109894133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5624786f1470 0x5624786fcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.117+0000 7fa0bad9d700 1 -- 172.21.15.52:0/2109894133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624786f0b60 msgr2=0x5624787019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:51.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.117+0000 7fa0bad9d700 1 --2- 172.21.15.52:0/2109894133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624786f0b60 0x5624787019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.117+0000 7fa0bad9d700 1 -- 172.21.15.52:0/2109894133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5624786f1470 msgr2=0x5624786fcb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:51.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.117+0000 7fa0bad9d700 1 --2- 172.21.15.52:0/2109894133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5624786f1470 0x5624786fcb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.117+0000 7fa0bad9d700 1 -- 172.21.15.52:0/2109894133 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562478607ee0 con 0x5624786f58b0 2023-11-27T15:24:51.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.118+0000 7fa0bad9d700 1 --2- 172.21.15.52:0/2109894133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624786f58b0 0x5624786f38b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40c9d60 tx=0x7fa0b4095330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a05e3aae1346b1fc server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:51.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.118+0000 7fa0a97fa700 1 -- 172.21.15.52:0/2109894133 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4098f90 con 0x5624786f58b0 2023-11-27T15:24:51.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.118+0000 7fa0a97fa700 1 -- 172.21.15.52:0/2109894133 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0b4099630 con 0x5624786f58b0 2023-11-27T15:24:51.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.118+0000 7fa0a97fa700 1 -- 172.21.15.52:0/2109894133 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4095d40 con 0x5624786f58b0 2023-11-27T15:24:51.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.118+0000 7fa0ccaf1b80 1 -- 172.21.15.52:0/2109894133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624786f58b0 msgr2=0x5624786f38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.118+0000 7fa0ccaf1b80 1 --2- 172.21.15.52:0/2109894133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624786f58b0 0x5624786f38b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40c9d60 tx=0x7fa0b4095330 comp rx=0 tx=0).stop 2023-11-27T15:24:51.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.118+0000 7fa0ccaf1b80 1 -- 172.21.15.52:0/2109894133 shutdown_connections 2023-11-27T15:24:51.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.118+0000 7fa0ccaf1b80 1 --2- 172.21.15.52:0/2109894133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624786f58b0 0x5624786f38b0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.118+0000 7fa0ccaf1b80 1 --2- 172.21.15.52:0/2109894133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624786f0b60 0x5624787019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.118+0000 7fa0ccaf1b80 1 --2- 172.21.15.52:0/2109894133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5624786f1470 0x5624786fcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.118+0000 7fa0ccaf1b80 1 -- 172.21.15.52:0/2109894133 >> 172.21.15.52:0/2109894133 conn(0x5624786e6360 msgr2=0x5624787115d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:51.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.118+0000 7fa0ccaf1b80 1 -- 172.21.15.52:0/2109894133 shutdown_connections 2023-11-27T15:24:51.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.118+0000 7fa0ccaf1b80 1 -- 172.21.15.52:0/2109894133 wait complete. 2023-11-27T15:24:51.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.119+0000 7fa0ccaf1b80 1 -- start start 2023-11-27T15:24:51.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.119+0000 7fa0ccaf1b80 1 Processor -- start 2023-11-27T15:24:51.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.119+0000 7fa0ccaf1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624786f0b60 0x56247875adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.119+0000 7fa0ccaf1b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5624786f1470 0x56247875b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.119+0000 7fa0ccaf1b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624786f58b0 0x5624786fd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.119+0000 7fa0ccaf1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624786fdfa0 con 0x5624786f0b60 2023-11-27T15:24:51.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.119+0000 7fa0ccaf1b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624786fe110 con 0x5624786f1470 2023-11-27T15:24:51.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.119+0000 7fa0ccaf1b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624786fe2d0 con 0x5624786f58b0 2023-11-27T15:24:51.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.119+0000 7fa0ccaf1b80 10 client.0 _ll_register_callbacks cb 0x7fa0b00053c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:51.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.119+0000 7fa0ba59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624786f0b60 0x56247875adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.119+0000 7fa0ba59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624786f0b60 0x56247875adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54710/0 (socket says 172.21.15.52:54710) 2023-11-27T15:24:51.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.119+0000 7fa0ba59c700 1 -- 172.21.15.52:0/1876123403 learned_addr learned my addr 172.21.15.52:0/1876123403 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:51.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.119+0000 7fa0bad9d700 1 --2- 172.21.15.52:0/1876123403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624786f58b0 0x5624786fd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.119+0000 7fa0b9d9b700 1 --2- 172.21.15.52:0/1876123403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5624786f1470 0x56247875b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.119+0000 7fa0ba59c700 1 -- 172.21.15.52:0/1876123403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624786f58b0 msgr2=0x5624786fd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:51.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.119+0000 7fa0ba59c700 1 --2- 172.21.15.52:0/1876123403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624786f58b0 0x5624786fd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.119+0000 7fa0ba59c700 1 -- 172.21.15.52:0/1876123403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5624786f1470 msgr2=0x56247875b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:51.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.119+0000 7fa0ba59c700 1 --2- 172.21.15.52:0/1876123403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5624786f1470 0x56247875b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.119+0000 7fa0ba59c700 1 -- 172.21.15.52:0/1876123403 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0bc1101e0 con 0x5624786f0b60 2023-11-27T15:24:51.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.120+0000 7fa0ba59c700 1 --2- 172.21.15.52:0/1876123403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624786f0b60 0x56247875adb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00fd10 tx=0x7fa0ac005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939ea934dc9a18 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:51.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.120+0000 7fa0a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:24:51.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.120+0000 7fa0a1ffb700 1 -- 172.21.15.52:0/1876123403 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac02e040 con 0x5624786f0b60 2023-11-27T15:24:51.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.120+0000 7fa0a1ffb700 1 -- 172.21.15.52:0/1876123403 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fa0ac036df0 con 0x5624786f0b60 2023-11-27T15:24:51.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.120+0000 7fa0ccaf1b80 1 -- 172.21.15.52:0/1876123403 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa0ac011dc0 con 0x5624786f0b60 2023-11-27T15:24:51.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.120+0000 7fa0a1ffb700 1 -- 172.21.15.52:0/1876123403 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0ac036610 con 0x5624786f0b60 2023-11-27T15:24:51.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.120+0000 7fa0ccaf1b80 10 client.16539 fetch_fsmap learned FSMap version 28 2023-11-27T15:24:51.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.120+0000 7fa0ccaf1b80 1 -- 172.21.15.52:0/1876123403 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562478458c60 con 0x5624786f0b60 2023-11-27T15:24:51.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.120+0000 7fa0a1ffb700 1 -- 172.21.15.52:0/1876123403 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0ac011dc0 con 0x5624786f0b60 2023-11-27T15:24:51.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.120+0000 7fa0a1ffb700 1 -- 172.21.15.52:0/1876123403 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa0ac004cb0 con 0x5624786f0b60 2023-11-27T15:24:51.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.120+0000 7fa0ccaf1b80 10 client.16539 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:24:51.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.120+0000 7fa0ccaf1b80 10 client.16539 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:24:51.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.120+0000 7fa0ccaf1b80 20 client.16539 populate_metadata read hostname 'smithi052' 2023-11-27T15:24:51.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.120+0000 7fa0ccaf1b80 1 --2- 172.21.15.52:0/1876123403 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x562478763d40 0x562478766150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.120+0000 7fa0ccaf1b80 4 client.16539 mds_command: new command op to 24472 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:24:51.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.120+0000 7fa0ccaf1b80 1 -- 172.21.15.52:0/1876123403 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562478610660 con 0x562478763d40 2023-11-27T15:24:51.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.121+0000 7fa0b9d9b700 1 --2- 172.21.15.52:0/1876123403 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x562478763d40 0x562478766150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.121+0000 7fa0b9d9b700 1 --2- 172.21.15.52:0/1876123403 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x562478763d40 0x562478766150 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4ee4369615e7e0e1 server_cookie=4df111d8e3d28a46 in_seq=0 out_seq=0 2023-11-27T15:24:51.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.121+0000 7fa0a1ffb700 10 client.16539 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:24:51.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.121+0000 7fa0a1ffb700 1 -- 172.21.15.52:0/1876123403 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562478610660 con 0x562478763d40 2023-11-27T15:24:51.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.121+0000 7fa0a1ffb700 10 client.16539 handle_command_reply: tid=0 2023-11-27T15:24:51.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.121+0000 7fa0ccaf1b80 1 client.16539 shutdown 2023-11-27T15:24:51.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.121+0000 7fa0ccaf1b80 1 -- 172.21.15.52:0/1876123403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624786f0b60 msgr2=0x56247875adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.121+0000 7fa0ccaf1b80 1 --2- 172.21.15.52:0/1876123403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624786f0b60 0x56247875adb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00fd10 tx=0x7fa0ac005b90 comp rx=0 tx=0).stop 2023-11-27T15:24:51.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.121+0000 7fa0ccaf1b80 1 -- 172.21.15.52:0/1876123403 shutdown_connections 2023-11-27T15:24:51.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.121+0000 7fa0ccaf1b80 1 --2- 172.21.15.52:0/1876123403 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x562478763d40 0x562478766150 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.121+0000 7fa0ccaf1b80 1 --2- 172.21.15.52:0/1876123403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5624786f58b0 0x5624786fd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.121+0000 7fa0ccaf1b80 1 --2- 172.21.15.52:0/1876123403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5624786f0b60 0x56247875adb0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.121+0000 7fa0a1ffb700 0 client.16539 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:24:51.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.121+0000 7fa0ccaf1b80 1 --2- 172.21.15.52:0/1876123403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5624786f1470 0x56247875b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.121+0000 7fa0ccaf1b80 1 -- 172.21.15.52:0/1876123403 >> 172.21.15.52:0/1876123403 conn(0x5624786e6360 msgr2=0x5624786f38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:51.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.121+0000 7fa0ccaf1b80 1 -- 172.21.15.52:0/1876123403 shutdown_connections 2023-11-27T15:24:51.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.121+0000 7fa0ccaf1b80 1 -- 172.21.15.52:0/1876123403 wait complete. 2023-11-27T15:24:51.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.122+0000 7fa0ccaf1b80 20 client.16539 trim_cache size 0 max 16384 2023-11-27T15:24:51.137 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:24:51.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.122+0000 7fa0c0965700 1 -- 172.21.15.52:0/3132196318 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fa0a40a1340 msgr2=0x7fa0a40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.122+0000 7fa0c0965700 1 --2- 172.21.15.52:0/3132196318 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fa0a40a1340 0x7fa0a40a37f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac005950 tx=0x7fa0ac0058e0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.122+0000 7fa0c0965700 1 -- 172.21.15.52:0/3132196318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0bc1055f0 msgr2=0x7fa0bc19ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.122+0000 7fa0c0965700 1 --2- 172.21.15.52:0/3132196318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0bc1055f0 0x7fa0bc19ad80 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4007ee0 tx=0x7fa0b400d3b0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.122+0000 7fa0c0965700 1 -- 172.21.15.52:0/3132196318 shutdown_connections 2023-11-27T15:24:51.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.122+0000 7fa0c0965700 1 --2- 172.21.15.52:0/3132196318 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fa0a40a1340 0x7fa0a40a37f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.122+0000 7fa0c0965700 1 --2- 172.21.15.52:0/3132196318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa0bc1005b0 0x7fa0bc1963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.122+0000 7fa0c0965700 1 --2- 172.21.15.52:0/3132196318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0bc1055f0 0x7fa0bc19ad80 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.122+0000 7fa0c0965700 1 --2- 172.21.15.52:0/3132196318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0bc100f50 0x7fa0bc196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.122+0000 7fa0c0965700 1 -- 172.21.15.52:0/3132196318 >> 172.21.15.52:0/3132196318 conn(0x7fa0bc077170 msgr2=0x7fa0bc0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:51.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.122+0000 7fa0c0965700 1 -- 172.21.15.52:0/3132196318 shutdown_connections 2023-11-27T15:24:51.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.122+0000 7fa0c0965700 1 -- 172.21.15.52:0/3132196318 wait complete. 2023-11-27T15:24:51.140 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-27T15:24:51.268 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.264+0000 7f019b2a1700 1 -- 172.21.15.52:0/1809480830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0194100f50 msgr2=0x7f0194104f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.268 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.264+0000 7f019b2a1700 1 --2- 172.21.15.52:0/1809480830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0194100f50 0x7f0194104f10 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f01900099c0 tx=0x7f0190009cd0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.268 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.264+0000 7f019b2a1700 1 -- 172.21.15.52:0/1809480830 shutdown_connections 2023-11-27T15:24:51.268 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.264+0000 7f019b2a1700 1 --2- 172.21.15.52:0/1809480830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f01941055f0 0x7f019410d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.268 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.264+0000 7f019b2a1700 1 --2- 172.21.15.52:0/1809480830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01941005b0 0x7f0194100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.264+0000 7f019b2a1700 1 --2- 172.21.15.52:0/1809480830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0194100f50 0x7f0194104f10 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.264+0000 7f019b2a1700 1 -- 172.21.15.52:0/1809480830 >> 172.21.15.52:0/1809480830 conn(0x7f0194077170 msgr2=0x7f0194077570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:51.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.264+0000 7f019b2a1700 1 -- 172.21.15.52:0/1809480830 shutdown_connections 2023-11-27T15:24:51.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.264+0000 7f019b2a1700 1 -- 172.21.15.52:0/1809480830 wait complete. 2023-11-27T15:24:51.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.265+0000 7f019b2a1700 1 Processor -- start 2023-11-27T15:24:51.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.265+0000 7f019b2a1700 1 -- start start 2023-11-27T15:24:51.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.265+0000 7f019b2a1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01941005b0 0x7f0194196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.265+0000 7f019b2a1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0194100f50 0x7f0194196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.265+0000 7f019b2a1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01941055f0 0x7f019419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.265+0000 7f019b2a1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019419b400 con 0x7f01941055f0 2023-11-27T15:24:51.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.265+0000 7f019b2a1700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019419b570 con 0x7f0194100f50 2023-11-27T15:24:51.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.265+0000 7f019b2a1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019419b730 con 0x7f01941005b0 2023-11-27T15:24:51.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.265+0000 7f019983e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01941055f0 0x7f019419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.265+0000 7f019983e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01941055f0 0x7f019419adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54728/0 (socket says 172.21.15.52:54728) 2023-11-27T15:24:51.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.265+0000 7f019983e700 1 -- 172.21.15.52:0/4250754132 learned_addr learned my addr 172.21.15.52:0/4250754132 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:51.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.265+0000 7f019903d700 1 --2- 172.21.15.52:0/4250754132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01941005b0 0x7f0194196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.265+0000 7f019883c700 1 --2- 172.21.15.52:0/4250754132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0194100f50 0x7f0194196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.265+0000 7f019983e700 1 -- 172.21.15.52:0/4250754132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01941005b0 msgr2=0x7f0194196420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.265+0000 7f019983e700 1 --2- 172.21.15.52:0/4250754132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01941005b0 0x7f0194196420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.265+0000 7f019983e700 1 -- 172.21.15.52:0/4250754132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0194100f50 msgr2=0x7f0194196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.265+0000 7f019983e700 1 --2- 172.21.15.52:0/4250754132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0194100f50 0x7f0194196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.265+0000 7f019983e700 1 -- 172.21.15.52:0/4250754132 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01900095f0 con 0x7f01941055f0 2023-11-27T15:24:51.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.266+0000 7f019983e700 1 --2- 172.21.15.52:0/4250754132 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01941055f0 0x7f019419adc0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f018c007ee0 tx=0x7f018c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:51.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.266+0000 7f018a7fc700 1 -- 172.21.15.52:0/4250754132 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018c017070 con 0x7f01941055f0 2023-11-27T15:24:51.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.266+0000 7f019b2a1700 1 -- 172.21.15.52:0/4250754132 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0194110390 con 0x7f01941055f0 2023-11-27T15:24:51.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.266+0000 7f018a7fc700 1 -- 172.21.15.52:0/4250754132 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f018c00f040 con 0x7f01941055f0 2023-11-27T15:24:51.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.266+0000 7f018a7fc700 1 -- 172.21.15.52:0/4250754132 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018c005740 con 0x7f01941055f0 2023-11-27T15:24:51.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.266+0000 7f019b2a1700 1 -- 172.21.15.52:0/4250754132 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0194110200 con 0x7f01941055f0 2023-11-27T15:24:51.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.268+0000 7f018a7fc700 1 -- 172.21.15.52:0/4250754132 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f018c0058a0 con 0x7f01941055f0 2023-11-27T15:24:51.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.269+0000 7f018a7fc700 1 --2- 172.21.15.52:0/4250754132 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f01800a12c0 0x7f01800a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.269+0000 7f019903d700 1 --2- 172.21.15.52:0/4250754132 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f01800a12c0 0x7f01800a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.269+0000 7f018a7fc700 1 -- 172.21.15.52:0/4250754132 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f018c0cc540 con 0x7f01941055f0 2023-11-27T15:24:51.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.269+0000 7f019903d700 1 --2- 172.21.15.52:0/4250754132 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f01800a12c0 0x7f01800a3770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0184009bf0 tx=0x7f0184009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:51.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.270+0000 7f01a742db80 1 Processor -- start 2023-11-27T15:24:51.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.270+0000 7f01a742db80 1 -- start start 2023-11-27T15:24:51.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.270+0000 7f01a742db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b13d42eb60 0x55b13d42ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.270+0000 7f01a742db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b13d42f470 0x55b13d433370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.270+0000 7f01a742db80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b13d4338b0 0x55b13d433d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.270+0000 7f01a742db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13d334060 con 0x55b13d42eb60 2023-11-27T15:24:51.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.270+0000 7f01a742db80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13d3868a0 con 0x55b13d4338b0 2023-11-27T15:24:51.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.270+0000 7f01a742db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13d37fd30 con 0x55b13d42f470 2023-11-27T15:24:51.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.270+0000 7f019903d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b13d42eb60 0x55b13d42ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.270+0000 7f019903d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b13d42eb60 0x55b13d42ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54732/0 (socket says 172.21.15.52:54732) 2023-11-27T15:24:51.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.270+0000 7f019903d700 1 -- 172.21.15.52:0/1941581399 learned_addr learned my addr 172.21.15.52:0/1941581399 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:51.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.271+0000 7f019983e700 1 --2- 172.21.15.52:0/1941581399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b13d4338b0 0x55b13d433d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.271+0000 7f019883c700 1 --2- 172.21.15.52:0/1941581399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b13d42f470 0x55b13d433370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.271+0000 7f019903d700 1 -- 172.21.15.52:0/1941581399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b13d42f470 msgr2=0x55b13d433370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:51.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.271+0000 7f019903d700 1 --2- 172.21.15.52:0/1941581399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b13d42f470 0x55b13d433370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.271+0000 7f019903d700 1 -- 172.21.15.52:0/1941581399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b13d4338b0 msgr2=0x55b13d433d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:51.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.271+0000 7f019903d700 1 --2- 172.21.15.52:0/1941581399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b13d4338b0 0x55b13d433d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.271+0000 7f019903d700 1 -- 172.21.15.52:0/1941581399 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b13d201630 con 0x55b13d42eb60 2023-11-27T15:24:51.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.271+0000 7f019903d700 1 --2- 172.21.15.52:0/1941581399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b13d42eb60 0x55b13d42ef30 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0184011e00 tx=0x7f0184014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb9b8032ad9bf27 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:51.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.271+0000 7f017f7fe700 1 -- 172.21.15.52:0/1941581399 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0184012bc0 con 0x55b13d42eb60 2023-11-27T15:24:51.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.271+0000 7f017f7fe700 1 -- 172.21.15.52:0/1941581399 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0184016070 con 0x55b13d42eb60 2023-11-27T15:24:51.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.271+0000 7f017f7fe700 1 -- 172.21.15.52:0/1941581399 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0184003cd0 con 0x55b13d42eb60 2023-11-27T15:24:51.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.272+0000 7f01a742db80 1 -- 172.21.15.52:0/1941581399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b13d42eb60 msgr2=0x55b13d42ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.272+0000 7f01a742db80 1 --2- 172.21.15.52:0/1941581399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b13d42eb60 0x55b13d42ef30 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0184011e00 tx=0x7f0184014000 comp rx=0 tx=0).stop 2023-11-27T15:24:51.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.272+0000 7f01a742db80 1 -- 172.21.15.52:0/1941581399 shutdown_connections 2023-11-27T15:24:51.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.272+0000 7f01a742db80 1 --2- 172.21.15.52:0/1941581399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b13d4338b0 0x55b13d433d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.272+0000 7f01a742db80 1 --2- 172.21.15.52:0/1941581399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b13d42eb60 0x55b13d42ef30 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.272+0000 7f01a742db80 1 --2- 172.21.15.52:0/1941581399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b13d42f470 0x55b13d433370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.272+0000 7f01a742db80 1 -- 172.21.15.52:0/1941581399 >> 172.21.15.52:0/1941581399 conn(0x55b13d424360 msgr2=0x55b13d429b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:51.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.272+0000 7f01a742db80 1 -- 172.21.15.52:0/1941581399 shutdown_connections 2023-11-27T15:24:51.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.272+0000 7f01a742db80 1 -- 172.21.15.52:0/1941581399 wait complete. 2023-11-27T15:24:51.285 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.272+0000 7f01a742db80 1 -- start start 2023-11-27T15:24:51.285 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.272+0000 7f01a742db80 1 Processor -- start 2023-11-27T15:24:51.285 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.273+0000 7f01a742db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b13d42eb60 0x55b13d43afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.285 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.273+0000 7f01a742db80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b13d42f470 0x55b13d438240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.285 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.273+0000 7f01a742db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b13d4338b0 0x55b13d438870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.285 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.273+0000 7f01a742db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13d3218e0 con 0x55b13d42eb60 2023-11-27T15:24:51.285 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.273+0000 7f01a742db80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13d321580 con 0x55b13d42f470 2023-11-27T15:24:51.285 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.273+0000 7f01a742db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13d203920 con 0x55b13d4338b0 2023-11-27T15:24:51.286 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.273+0000 7f01a742db80 10 client.0 _ll_register_callbacks cb 0x55b13d33f780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:51.286 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.273+0000 7f019903d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b13d42eb60 0x55b13d43afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.286 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.273+0000 7f019903d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b13d42eb60 0x55b13d43afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54738/0 (socket says 172.21.15.52:54738) 2023-11-27T15:24:51.286 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.273+0000 7f019903d700 1 -- 172.21.15.52:0/55075296 learned_addr learned my addr 172.21.15.52:0/55075296 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:51.286 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.273+0000 7f019983e700 1 --2- 172.21.15.52:0/55075296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b13d4338b0 0x55b13d438870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.286 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.273+0000 7f019883c700 1 --2- 172.21.15.52:0/55075296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b13d42f470 0x55b13d438240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.286 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.273+0000 7f019903d700 1 -- 172.21.15.52:0/55075296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b13d4338b0 msgr2=0x55b13d438870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:51.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.273+0000 7f019903d700 1 --2- 172.21.15.52:0/55075296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b13d4338b0 0x55b13d438870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.273+0000 7f019903d700 1 -- 172.21.15.52:0/55075296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b13d42f470 msgr2=0x55b13d438240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:51.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.273+0000 7f019903d700 1 --2- 172.21.15.52:0/55075296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b13d42f470 0x55b13d438240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.273+0000 7f019903d700 1 -- 172.21.15.52:0/55075296 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0184011ae0 con 0x55b13d42eb60 2023-11-27T15:24:51.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.273+0000 7f019983e700 1 --2- 172.21.15.52:0/55075296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b13d4338b0 0x55b13d438870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:24:51.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.273+0000 7f019903d700 1 --2- 172.21.15.52:0/55075296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b13d42eb60 0x55b13d43afc0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0184012f50 tx=0x7f01840274a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a294a9bf150ea8 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:51.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.273+0000 7f017cff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:24:51.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.273+0000 7f017cff9700 1 -- 172.21.15.52:0/55075296 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0184027b80 con 0x55b13d42eb60 2023-11-27T15:24:51.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.274+0000 7f017cff9700 1 -- 172.21.15.52:0/55075296 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f0184018070 con 0x55b13d42eb60 2023-11-27T15:24:51.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.274+0000 7f01a742db80 1 -- 172.21.15.52:0/55075296 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b13d421b20 con 0x55b13d42eb60 2023-11-27T15:24:51.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.274+0000 7f017cff9700 1 -- 172.21.15.52:0/55075296 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0184027ce0 con 0x55b13d42eb60 2023-11-27T15:24:51.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.274+0000 7f01a742db80 10 client.16551 fetch_fsmap learned FSMap version 28 2023-11-27T15:24:51.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.274+0000 7f01a742db80 1 -- 172.21.15.52:0/55075296 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b13d346190 con 0x55b13d42eb60 2023-11-27T15:24:51.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.274+0000 7f017cff9700 1 -- 172.21.15.52:0/55075296 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f018403e020 con 0x55b13d42eb60 2023-11-27T15:24:51.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.274+0000 7f017cff9700 1 -- 172.21.15.52:0/55075296 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0184032400 con 0x55b13d42eb60 2023-11-27T15:24:51.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.274+0000 7f01a742db80 10 client.16551 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:24:51.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.274+0000 7f01a742db80 10 client.16551 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:24:51.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.274+0000 7f01a742db80 20 client.16551 populate_metadata read hostname 'smithi052' 2023-11-27T15:24:51.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.274+0000 7f01a742db80 1 --2- 172.21.15.52:0/55075296 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55b13d448880 0x55b13d44ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.274+0000 7f01a742db80 4 client.16551 mds_command: new command op to 24494 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:24:51.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.274+0000 7f01a742db80 1 -- 172.21.15.52:0/55075296 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b13d3420a0 con 0x55b13d448880 2023-11-27T15:24:51.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.274+0000 7f019883c700 1 --2- 172.21.15.52:0/55075296 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55b13d448880 0x55b13d44ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.275+0000 7f017cff9700 10 client.16551 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:24:51.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.275+0000 7f019883c700 1 --2- 172.21.15.52:0/55075296 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55b13d448880 0x55b13d44ac90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fdd5f5075c3f812d server_cookie=14b522d42d2f25d2 in_seq=0 out_seq=0 2023-11-27T15:24:51.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.275+0000 7f017cff9700 1 -- 172.21.15.52:0/55075296 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b13d3420a0 con 0x55b13d448880 2023-11-27T15:24:51.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.275+0000 7f017cff9700 10 client.16551 handle_command_reply: tid=0 2023-11-27T15:24:51.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.276+0000 7f01a742db80 1 client.16551 shutdown 2023-11-27T15:24:51.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.276+0000 7f01a742db80 1 -- 172.21.15.52:0/55075296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b13d42eb60 msgr2=0x55b13d43afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.276+0000 7f01a742db80 1 --2- 172.21.15.52:0/55075296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b13d42eb60 0x55b13d43afc0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0184012f50 tx=0x7f01840274a0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.276+0000 7f01a742db80 1 -- 172.21.15.52:0/55075296 shutdown_connections 2023-11-27T15:24:51.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.276+0000 7f01a742db80 1 --2- 172.21.15.52:0/55075296 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55b13d448880 0x55b13d44ac90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.276+0000 7f01a742db80 1 --2- 172.21.15.52:0/55075296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b13d4338b0 0x55b13d438870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.276+0000 7f01a742db80 1 --2- 172.21.15.52:0/55075296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b13d42eb60 0x55b13d43afc0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.276+0000 7f017cff9700 0 client.16551 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:24:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.276+0000 7f01a742db80 1 --2- 172.21.15.52:0/55075296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b13d42f470 0x55b13d438240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.276+0000 7f01a742db80 1 -- 172.21.15.52:0/55075296 >> 172.21.15.52:0/55075296 conn(0x55b13d424360 msgr2=0x55b13d434d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.276+0000 7f01a742db80 1 -- 172.21.15.52:0/55075296 shutdown_connections 2023-11-27T15:24:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.276+0000 7f01a742db80 1 -- 172.21.15.52:0/55075296 wait complete. 2023-11-27T15:24:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.276+0000 7f01a742db80 20 client.16551 trim_cache size 0 max 16384 2023-11-27T15:24:51.292 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:51 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1262119141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:24:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.289+0000 7f01a742db80 1 Processor -- start 2023-11-27T15:24:51.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.289+0000 7f01a742db80 1 -- start start 2023-11-27T15:24:51.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.289+0000 7f01a742db80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b13d42eb60 0x55b13d43f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.289+0000 7f01a742db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b13d42f470 0x55b13d428550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.289+0000 7f01a742db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b13d4338b0 0x55b13d428d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.289+0000 7f01a742db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13d439870 con 0x55b13d42f470 2023-11-27T15:24:51.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.289+0000 7f01a742db80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13d423f10 con 0x55b13d42eb60 2023-11-27T15:24:51.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.289+0000 7f01a742db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13d4367d0 con 0x55b13d4338b0 2023-11-27T15:24:51.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.289+0000 7f019883c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b13d42f470 0x55b13d428550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.289+0000 7f019883c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b13d42f470 0x55b13d428550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54748/0 (socket says 172.21.15.52:54748) 2023-11-27T15:24:51.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.289+0000 7f019883c700 1 -- 172.21.15.52:0/4140340189 learned_addr learned my addr 172.21.15.52:0/4140340189 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:51.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.289+0000 7f019903d700 1 --2- 172.21.15.52:0/4140340189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b13d42eb60 0x55b13d43f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.289+0000 7f019883c700 1 -- 172.21.15.52:0/4140340189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b13d4338b0 msgr2=0x55b13d428d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:51.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.289+0000 7f019883c700 1 --2- 172.21.15.52:0/4140340189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b13d4338b0 0x55b13d428d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.289+0000 7f019883c700 1 -- 172.21.15.52:0/4140340189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b13d42eb60 msgr2=0x55b13d43f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:51.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.289+0000 7f019883c700 1 --2- 172.21.15.52:0/4140340189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b13d42eb60 0x55b13d43f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.289+0000 7f019883c700 1 -- 172.21.15.52:0/4140340189 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b13d425b00 con 0x55b13d42f470 2023-11-27T15:24:51.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.289+0000 7f019903d700 1 --2- 172.21.15.52:0/4140340189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b13d42eb60 0x55b13d43f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:24:51.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.290+0000 7f019883c700 1 --2- 172.21.15.52:0/4140340189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b13d42f470 0x55b13d428550 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f01900098e0 tx=0x7f019001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c5156144eb8844 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:51.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.290+0000 7f017ffff700 1 -- 172.21.15.52:0/4140340189 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019001d040 con 0x55b13d42f470 2023-11-27T15:24:51.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.290+0000 7f017ffff700 1 -- 172.21.15.52:0/4140340189 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f019002b070 con 0x55b13d42f470 2023-11-27T15:24:51.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.290+0000 7f017ffff700 1 -- 172.21.15.52:0/4140340189 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0190003a30 con 0x55b13d42f470 2023-11-27T15:24:51.300 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.290+0000 7f01a742db80 1 -- 172.21.15.52:0/4140340189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b13d42f470 msgr2=0x55b13d428550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.300 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.290+0000 7f01a742db80 1 --2- 172.21.15.52:0/4140340189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b13d42f470 0x55b13d428550 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f01900098e0 tx=0x7f019001a040 comp rx=0 tx=0).stop 2023-11-27T15:24:51.300 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.290+0000 7f01a742db80 1 -- 172.21.15.52:0/4140340189 shutdown_connections 2023-11-27T15:24:51.300 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.290+0000 7f01a742db80 1 --2- 172.21.15.52:0/4140340189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b13d42eb60 0x55b13d43f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.301 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.290+0000 7f01a742db80 1 --2- 172.21.15.52:0/4140340189 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b13d4338b0 0x55b13d428d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.301 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.290+0000 7f01a742db80 1 --2- 172.21.15.52:0/4140340189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b13d42f470 0x55b13d428550 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.301 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.290+0000 7f01a742db80 1 -- 172.21.15.52:0/4140340189 >> 172.21.15.52:0/4140340189 conn(0x55b13d424360 msgr2=0x55b13d433c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:51.301 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.290+0000 7f01a742db80 1 -- 172.21.15.52:0/4140340189 shutdown_connections 2023-11-27T15:24:51.301 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.290+0000 7f01a742db80 1 -- 172.21.15.52:0/4140340189 wait complete. 2023-11-27T15:24:51.301 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.291+0000 7f01a742db80 1 -- start start 2023-11-27T15:24:51.301 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.291+0000 7f01a742db80 1 Processor -- start 2023-11-27T15:24:51.302 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.291+0000 7f01a742db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b13d42eb60 0x55b13d43b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.302 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.291+0000 7f01a742db80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b13d42f470 0x55b13d43bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.302 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.291+0000 7f01a742db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b13d4338b0 0x55b13d43c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.302 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.291+0000 7f01a742db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13d43ffe0 con 0x55b13d4338b0 2023-11-27T15:24:51.302 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.291+0000 7f01a742db80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13d49fb30 con 0x55b13d42f470 2023-11-27T15:24:51.302 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.291+0000 7f01a742db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13d49fcf0 con 0x55b13d42eb60 2023-11-27T15:24:51.302 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.291+0000 7f01a742db80 10 client.0 _ll_register_callbacks cb 0x7f018c0952e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:51.303 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.291+0000 7f019983e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b13d4338b0 0x55b13d43c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.303 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.291+0000 7f019983e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b13d4338b0 0x55b13d43c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54756/0 (socket says 172.21.15.52:54756) 2023-11-27T15:24:51.303 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.291+0000 7f019983e700 1 -- 172.21.15.52:0/2661964473 learned_addr learned my addr 172.21.15.52:0/2661964473 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:51.303 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.292+0000 7f019883c700 1 --2- 172.21.15.52:0/2661964473 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b13d42f470 0x55b13d43bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.303 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.292+0000 7f019903d700 1 --2- 172.21.15.52:0/2661964473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b13d42eb60 0x55b13d43b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.303 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.292+0000 7f019983e700 1 -- 172.21.15.52:0/2661964473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b13d42eb60 msgr2=0x55b13d43b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:51.303 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.292+0000 7f019983e700 1 --2- 172.21.15.52:0/2661964473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b13d42eb60 0x55b13d43b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.303 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.292+0000 7f019983e700 1 -- 172.21.15.52:0/2661964473 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b13d42f470 msgr2=0x55b13d43bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:51.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.292+0000 7f019983e700 1 --2- 172.21.15.52:0/2661964473 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b13d42f470 0x55b13d43bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.292+0000 7f019983e700 1 -- 172.21.15.52:0/2661964473 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f019410da00 con 0x55b13d4338b0 2023-11-27T15:24:51.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.292+0000 7f019983e700 1 --2- 172.21.15.52:0/2661964473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b13d4338b0 0x55b13d43c400 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f018c00d7b0 tx=0x7f018c095920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3e8ef6233f3d6b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:51.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.292+0000 7f017cff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:24:51.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.292+0000 7f017cff9700 1 -- 172.21.15.52:0/2661964473 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018c00ddd0 con 0x55b13d4338b0 2023-11-27T15:24:51.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.292+0000 7f017cff9700 1 -- 172.21.15.52:0/2661964473 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f018c09d610 con 0x55b13d4338b0 2023-11-27T15:24:51.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.292+0000 7f01a742db80 1 -- 172.21.15.52:0/2661964473 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0184016070 con 0x55b13d4338b0 2023-11-27T15:24:51.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.292+0000 7f017cff9700 1 -- 172.21.15.52:0/2661964473 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f018c0842c0 con 0x55b13d4338b0 2023-11-27T15:24:51.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.292+0000 7f01a742db80 10 client.16557 fetch_fsmap learned FSMap version 28 2023-11-27T15:24:51.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.292+0000 7f01a742db80 1 -- 172.21.15.52:0/2661964473 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b13d1adde0 con 0x55b13d4338b0 2023-11-27T15:24:51.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.292+0000 7f017cff9700 1 -- 172.21.15.52:0/2661964473 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0184016070 con 0x55b13d4338b0 2023-11-27T15:24:51.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.292+0000 7f017cff9700 1 -- 172.21.15.52:0/2661964473 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f018c098b60 con 0x55b13d4338b0 2023-11-27T15:24:51.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.293+0000 7f01a742db80 10 client.16557 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:24:51.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.293+0000 7f01a742db80 10 client.16557 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:24:51.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.293+0000 7f01a742db80 20 client.16557 populate_metadata read hostname 'smithi052' 2023-11-27T15:24:51.306 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.293+0000 7f01a742db80 1 --2- 172.21.15.52:0/2661964473 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55b13d4a14d0 0x55b13d4a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.306 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.293+0000 7f01a742db80 4 client.16557 mds_command: new command op to 24494 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:24:51.306 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.293+0000 7f01a742db80 1 -- 172.21.15.52:0/2661964473 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b13d34e660 con 0x55b13d4a14d0 2023-11-27T15:24:51.306 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.293+0000 7f019903d700 1 --2- 172.21.15.52:0/2661964473 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55b13d4a14d0 0x55b13d4a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.306 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.293+0000 7f017cff9700 10 client.16557 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:24:51.306 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.293+0000 7f019903d700 1 --2- 172.21.15.52:0/2661964473 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55b13d4a14d0 0x55b13d4a38e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5d45700a7e6928ce server_cookie=49af38aff6a9c50e in_seq=0 out_seq=0 2023-11-27T15:24:51.306 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.293+0000 7f017cff9700 1 -- 172.21.15.52:0/2661964473 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b13d34e660 con 0x55b13d4a14d0 2023-11-27T15:24:51.306 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.293+0000 7f017cff9700 10 client.16557 handle_command_reply: tid=0 2023-11-27T15:24:51.307 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.293+0000 7f01a742db80 1 client.16557 shutdown 2023-11-27T15:24:51.307 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.293+0000 7f01a742db80 1 -- 172.21.15.52:0/2661964473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b13d4338b0 msgr2=0x55b13d43c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.307 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.293+0000 7f01a742db80 1 --2- 172.21.15.52:0/2661964473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b13d4338b0 0x55b13d43c400 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f018c00d7b0 tx=0x7f018c095920 comp rx=0 tx=0).stop 2023-11-27T15:24:51.307 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.294+0000 7f01a742db80 1 -- 172.21.15.52:0/2661964473 shutdown_connections 2023-11-27T15:24:51.307 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.294+0000 7f01a742db80 1 --2- 172.21.15.52:0/2661964473 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55b13d4a14d0 0x55b13d4a38e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.307 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.294+0000 7f01a742db80 1 --2- 172.21.15.52:0/2661964473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b13d4338b0 0x55b13d43c400 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.307 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.294+0000 7f017cff9700 0 client.16557 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:24:51.307 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.294+0000 7f01a742db80 1 --2- 172.21.15.52:0/2661964473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b13d42eb60 0x55b13d43b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.308 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.294+0000 7f01a742db80 1 --2- 172.21.15.52:0/2661964473 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b13d42f470 0x55b13d43bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.308 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.294+0000 7f01a742db80 1 -- 172.21.15.52:0/2661964473 >> 172.21.15.52:0/2661964473 conn(0x55b13d424360 msgr2=0x55b13d428550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:51.308 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.294+0000 7f01a742db80 1 -- 172.21.15.52:0/2661964473 shutdown_connections 2023-11-27T15:24:51.308 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.294+0000 7f01a742db80 1 -- 172.21.15.52:0/2661964473 wait complete. 2023-11-27T15:24:51.308 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.294+0000 7f01a742db80 20 client.16557 trim_cache size 0 max 16384 2023-11-27T15:24:51.308 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:24:51.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.294+0000 7f019b2a1700 1 -- 172.21.15.52:0/4250754132 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f01800a12c0 msgr2=0x7f01800a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.294+0000 7f019b2a1700 1 --2- 172.21.15.52:0/4250754132 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f01800a12c0 0x7f01800a3770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0184009bf0 tx=0x7f0184009260 comp rx=0 tx=0).stop 2023-11-27T15:24:51.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.294+0000 7f019b2a1700 1 -- 172.21.15.52:0/4250754132 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01941055f0 msgr2=0x7f019419adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.294+0000 7f019b2a1700 1 --2- 172.21.15.52:0/4250754132 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01941055f0 0x7f019419adc0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f018c007ee0 tx=0x7f018c00d3b0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.294+0000 7f019b2a1700 1 -- 172.21.15.52:0/4250754132 shutdown_connections 2023-11-27T15:24:51.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.294+0000 7f019b2a1700 1 --2- 172.21.15.52:0/4250754132 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f01800a12c0 0x7f01800a3770 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.294+0000 7f019b2a1700 1 --2- 172.21.15.52:0/4250754132 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01941055f0 0x7f019419adc0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.294+0000 7f019b2a1700 1 --2- 172.21.15.52:0/4250754132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01941005b0 0x7f0194196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.294+0000 7f019b2a1700 1 --2- 172.21.15.52:0/4250754132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0194100f50 0x7f0194196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.294+0000 7f019b2a1700 1 -- 172.21.15.52:0/4250754132 >> 172.21.15.52:0/4250754132 conn(0x7f0194077170 msgr2=0x7f01940fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:51.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.294+0000 7f019b2a1700 1 -- 172.21.15.52:0/4250754132 shutdown_connections 2023-11-27T15:24:51.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.294+0000 7f019b2a1700 1 -- 172.21.15.52:0/4250754132 wait complete. 2023-11-27T15:24:51.311 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:50 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1262119141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:24:51.312 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-27T15:24:51.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:51 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1262119141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:24:51.440 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.436+0000 7f3371dca700 1 -- 172.21.15.52:0/12406144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f336c0ffe70 msgr2=0x7f336c103e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.440 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.436+0000 7f3371dca700 1 --2- 172.21.15.52:0/12406144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f336c0ffe70 0x7f336c103e30 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f335c0099c0 tx=0x7f335c009cd0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.440 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.436+0000 7f3371dca700 1 -- 172.21.15.52:0/12406144 shutdown_connections 2023-11-27T15:24:51.440 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.436+0000 7f3371dca700 1 --2- 172.21.15.52:0/12406144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f336c104510 0x7f336c10c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.440 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.436+0000 7f3371dca700 1 --2- 172.21.15.52:0/12406144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f336c0ff4d0 0x7f336c0ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.436+0000 7f3371dca700 1 --2- 172.21.15.52:0/12406144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f336c0ffe70 0x7f336c103e30 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.436+0000 7f3371dca700 1 -- 172.21.15.52:0/12406144 >> 172.21.15.52:0/12406144 conn(0x7f336c076070 msgr2=0x7f336c076470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:51.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.436+0000 7f3371dca700 1 -- 172.21.15.52:0/12406144 shutdown_connections 2023-11-27T15:24:51.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.436+0000 7f3371dca700 1 -- 172.21.15.52:0/12406144 wait complete. 2023-11-27T15:24:51.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.436+0000 7f3371dca700 1 Processor -- start 2023-11-27T15:24:51.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.437+0000 7f3371dca700 1 -- start start 2023-11-27T15:24:51.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.437+0000 7f3371dca700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f336c0ff4d0 0x7f336c1952f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.437+0000 7f3371dca700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f336c0ffe70 0x7f336c195830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.437+0000 7f3371dca700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f336c104510 0x7f336c199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.437+0000 7f3371dca700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336c19a2d0 con 0x7f336c104510 2023-11-27T15:24:51.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.437+0000 7f3371dca700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336c19a440 con 0x7f336c0ffe70 2023-11-27T15:24:51.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.437+0000 7f3371dca700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336c19a600 con 0x7f336c0ff4d0 2023-11-27T15:24:51.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.437+0000 7f336bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f336c104510 0x7f336c199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.437+0000 7f336bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f336c104510 0x7f336c199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54770/0 (socket says 172.21.15.52:54770) 2023-11-27T15:24:51.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.437+0000 7f336bfff700 1 -- 172.21.15.52:0/3556858636 learned_addr learned my addr 172.21.15.52:0/3556858636 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:51.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.437+0000 7f336b7fe700 1 --2- 172.21.15.52:0/3556858636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f336c0ff4d0 0x7f336c1952f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.437+0000 7f336affd700 1 --2- 172.21.15.52:0/3556858636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f336c0ffe70 0x7f336c195830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.437+0000 7f336bfff700 1 -- 172.21.15.52:0/3556858636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f336c0ff4d0 msgr2=0x7f336c1952f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.437+0000 7f336bfff700 1 --2- 172.21.15.52:0/3556858636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f336c0ff4d0 0x7f336c1952f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.437+0000 7f336bfff700 1 -- 172.21.15.52:0/3556858636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f336c0ffe70 msgr2=0x7f336c195830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.437+0000 7f336bfff700 1 --2- 172.21.15.52:0/3556858636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f336c0ffe70 0x7f336c195830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.437+0000 7f336bfff700 1 -- 172.21.15.52:0/3556858636 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f335c0095f0 con 0x7f336c104510 2023-11-27T15:24:51.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.437+0000 7f336affd700 1 --2- 172.21.15.52:0/3556858636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f336c0ffe70 0x7f336c195830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:24:51.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.438+0000 7f336bfff700 1 --2- 172.21.15.52:0/3556858636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f336c104510 0x7f336c199c90 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f336400ef30 tx=0x7f336400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:51.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.438+0000 7f3368ff9700 1 -- 172.21.15.52:0/3556858636 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3364018070 con 0x7f336c104510 2023-11-27T15:24:51.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.438+0000 7f3368ff9700 1 -- 172.21.15.52:0/3556858636 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f336400f040 con 0x7f336c104510 2023-11-27T15:24:51.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.438+0000 7f3371dca700 1 -- 172.21.15.52:0/3556858636 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f336c10f290 con 0x7f336c104510 2023-11-27T15:24:51.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.438+0000 7f3368ff9700 1 -- 172.21.15.52:0/3556858636 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336400cdb0 con 0x7f336c104510 2023-11-27T15:24:51.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.438+0000 7f3371dca700 1 -- 172.21.15.52:0/3556858636 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f336c10f100 con 0x7f336c104510 2023-11-27T15:24:51.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.440+0000 7f3368ff9700 1 -- 172.21.15.52:0/3556858636 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f3364003a00 con 0x7f336c104510 2023-11-27T15:24:51.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.441+0000 7f3368ff9700 1 --2- 172.21.15.52:0/3556858636 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f33540a12c0 0x7f33540a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.441+0000 7f336b7fe700 1 --2- 172.21.15.52:0/3556858636 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f33540a12c0 0x7f33540a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.441+0000 7f3368ff9700 1 -- 172.21.15.52:0/3556858636 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f33640cc940 con 0x7f336c104510 2023-11-27T15:24:51.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.441+0000 7f336b7fe700 1 --2- 172.21.15.52:0/3556858636 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f33540a12c0 0x7f33540a3770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3360005950 tx=0x7f336000b410 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:51.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.441+0000 7f337df56b80 1 Processor -- start 2023-11-27T15:24:51.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.441+0000 7f337df56b80 1 -- start start 2023-11-27T15:24:51.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.442+0000 7f337df56b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556bdef40b60 0x556bdef40f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.442+0000 7f337df56b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556bdef41470 0x556bdef45370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.442+0000 7f337df56b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556bdef458b0 0x556bdef45d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.442+0000 7f337df56b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdee46060 con 0x556bdef40b60 2023-11-27T15:24:51.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.442+0000 7f337df56b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdee988a0 con 0x556bdef41470 2023-11-27T15:24:51.454 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.442+0000 7f337df56b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdee91d30 con 0x556bdef458b0 2023-11-27T15:24:51.454 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.442+0000 7f336b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556bdef40b60 0x556bdef40f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.454 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.442+0000 7f336b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556bdef40b60 0x556bdef40f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54774/0 (socket says 172.21.15.52:54774) 2023-11-27T15:24:51.454 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.442+0000 7f336b7fe700 1 -- 172.21.15.52:0/3318870814 learned_addr learned my addr 172.21.15.52:0/3318870814 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:51.454 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.442+0000 7f336affd700 1 --2- 172.21.15.52:0/3318870814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556bdef41470 0x556bdef45370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.454 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.442+0000 7f336b7fe700 1 -- 172.21.15.52:0/3318870814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556bdef458b0 msgr2=0x556bdef45d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:51.454 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.442+0000 7f336bfff700 1 --2- 172.21.15.52:0/3318870814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556bdef458b0 0x556bdef45d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.442+0000 7f336b7fe700 1 --2- 172.21.15.52:0/3318870814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556bdef458b0 0x556bdef45d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.442+0000 7f336b7fe700 1 -- 172.21.15.52:0/3318870814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556bdef41470 msgr2=0x556bdef45370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:51.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.442+0000 7f336b7fe700 1 --2- 172.21.15.52:0/3318870814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556bdef41470 0x556bdef45370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.442+0000 7f336b7fe700 1 -- 172.21.15.52:0/3318870814 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556bded13630 con 0x556bdef40b60 2023-11-27T15:24:51.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.442+0000 7f336bfff700 1 --2- 172.21.15.52:0/3318870814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556bdef458b0 0x556bdef45d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:24:51.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.442+0000 7f336affd700 1 --2- 172.21.15.52:0/3318870814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556bdef41470 0x556bdef45370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:24:51.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.442+0000 7f336b7fe700 1 --2- 172.21.15.52:0/3318870814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556bdef40b60 0x556bdef40f30 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f3360010fa0 tx=0x7f3360013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874448278aab6836 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:51.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.443+0000 7f3351ffb700 1 -- 172.21.15.52:0/3318870814 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3360021070 con 0x556bdef40b60 2023-11-27T15:24:51.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.443+0000 7f3351ffb700 1 -- 172.21.15.52:0/3318870814 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3360015070 con 0x556bdef40b60 2023-11-27T15:24:51.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.443+0000 7f3351ffb700 1 -- 172.21.15.52:0/3318870814 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336001abd0 con 0x556bdef40b60 2023-11-27T15:24:51.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.443+0000 7f337df56b80 1 -- 172.21.15.52:0/3318870814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556bdef40b60 msgr2=0x556bdef40f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.443+0000 7f337df56b80 1 --2- 172.21.15.52:0/3318870814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556bdef40b60 0x556bdef40f30 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f3360010fa0 tx=0x7f3360013000 comp rx=0 tx=0).stop 2023-11-27T15:24:51.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.443+0000 7f337df56b80 1 -- 172.21.15.52:0/3318870814 shutdown_connections 2023-11-27T15:24:51.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.443+0000 7f337df56b80 1 --2- 172.21.15.52:0/3318870814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556bdef458b0 0x556bdef45d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.443+0000 7f337df56b80 1 --2- 172.21.15.52:0/3318870814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556bdef40b60 0x556bdef40f30 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.443+0000 7f337df56b80 1 --2- 172.21.15.52:0/3318870814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556bdef41470 0x556bdef45370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.443+0000 7f337df56b80 1 -- 172.21.15.52:0/3318870814 >> 172.21.15.52:0/3318870814 conn(0x556bdef36360 msgr2=0x556bdef3bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:51.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.443+0000 7f337df56b80 1 -- 172.21.15.52:0/3318870814 shutdown_connections 2023-11-27T15:24:51.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.443+0000 7f337df56b80 1 -- 172.21.15.52:0/3318870814 wait complete. 2023-11-27T15:24:51.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.444+0000 7f337df56b80 1 -- start start 2023-11-27T15:24:51.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.444+0000 7f337df56b80 1 Processor -- start 2023-11-27T15:24:51.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.444+0000 7f337df56b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556bdef40b60 0x556bdef4cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.444+0000 7f337df56b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556bdef41470 0x556bdef4a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.444+0000 7f337df56b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556bdef458b0 0x556bdef4a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.444+0000 7f337df56b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdee338e0 con 0x556bdef41470 2023-11-27T15:24:51.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.445+0000 7f337df56b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdee33580 con 0x556bdef40b60 2023-11-27T15:24:51.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.445+0000 7f337df56b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bded15920 con 0x556bdef458b0 2023-11-27T15:24:51.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.445+0000 7f337df56b80 10 client.0 _ll_register_callbacks cb 0x556bdee51780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:51.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.445+0000 7f336affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556bdef41470 0x556bdef4a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.445+0000 7f336affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556bdef41470 0x556bdef4a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54782/0 (socket says 172.21.15.52:54782) 2023-11-27T15:24:51.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.445+0000 7f336affd700 1 -- 172.21.15.52:0/3155397005 learned_addr learned my addr 172.21.15.52:0/3155397005 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:51.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.445+0000 7f336b7fe700 1 --2- 172.21.15.52:0/3155397005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556bdef40b60 0x556bdef4cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.445+0000 7f336bfff700 1 --2- 172.21.15.52:0/3155397005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556bdef458b0 0x556bdef4a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.445+0000 7f336affd700 1 -- 172.21.15.52:0/3155397005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556bdef458b0 msgr2=0x556bdef4a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:51.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.445+0000 7f336affd700 1 --2- 172.21.15.52:0/3155397005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556bdef458b0 0x556bdef4a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.445+0000 7f336affd700 1 -- 172.21.15.52:0/3155397005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556bdef40b60 msgr2=0x556bdef4cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:51.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.445+0000 7f336affd700 1 --2- 172.21.15.52:0/3155397005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556bdef40b60 0x556bdef4cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.445+0000 7f336affd700 1 -- 172.21.15.52:0/3155397005 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3360010c50 con 0x556bdef41470 2023-11-27T15:24:51.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.445+0000 7f336affd700 1 --2- 172.21.15.52:0/3155397005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556bdef41470 0x556bdef4a250 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f335c0036c0 tx=0x7f335c0057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311db02063b9961 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:51.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.445+0000 7f334affd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:24:51.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.445+0000 7f334affd700 1 -- 172.21.15.52:0/3155397005 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c01c050 con 0x556bdef41470 2023-11-27T15:24:51.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.445+0000 7f334affd700 1 -- 172.21.15.52:0/3155397005 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f335c003be0 con 0x556bdef41470 2023-11-27T15:24:51.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.445+0000 7f337df56b80 1 -- 172.21.15.52:0/3155397005 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556bdef33b20 con 0x556bdef41470 2023-11-27T15:24:51.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.446+0000 7f334affd700 1 -- 172.21.15.52:0/3155397005 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f335c003da0 con 0x556bdef41470 2023-11-27T15:24:51.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.446+0000 7f337df56b80 10 client.16569 fetch_fsmap learned FSMap version 28 2023-11-27T15:24:51.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.446+0000 7f337df56b80 1 -- 172.21.15.52:0/3155397005 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556bdee58190 con 0x556bdef41470 2023-11-27T15:24:51.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.446+0000 7f334affd700 1 -- 172.21.15.52:0/3155397005 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556bdef33b20 con 0x556bdef41470 2023-11-27T15:24:51.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.446+0000 7f334affd700 1 -- 172.21.15.52:0/3155397005 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f335c022700 con 0x556bdef41470 2023-11-27T15:24:51.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.446+0000 7f337df56b80 10 client.16569 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:24:51.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.446+0000 7f337df56b80 10 client.16569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:51.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.446+0000 7f337df56b80 20 client.16569 populate_metadata read hostname 'smithi052' 2023-11-27T15:24:51.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.446+0000 7f337df56b80 1 --2- 172.21.15.52:0/3155397005 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x556bdef5a800 0x556bdef5cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.446+0000 7f337df56b80 4 client.16569 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:24:51.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.446+0000 7f337df56b80 1 -- 172.21.15.52:0/3155397005 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556bdee540a0 con 0x556bdef5a800 2023-11-27T15:24:51.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.446+0000 7f336b7fe700 1 --2- 172.21.15.52:0/3155397005 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x556bdef5a800 0x556bdef5cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.447+0000 7f334affd700 10 client.16569 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:51.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.447+0000 7f336b7fe700 1 --2- 172.21.15.52:0/3155397005 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x556bdef5a800 0x556bdef5cc10 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=156d2dc80ffceb71 server_cookie=e69e5c70b31808b6 in_seq=0 out_seq=0 2023-11-27T15:24:51.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.447+0000 7f334affd700 1 -- 172.21.15.52:0/3155397005 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556bdee540a0 con 0x556bdef5a800 2023-11-27T15:24:51.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.447+0000 7f334affd700 10 client.16569 handle_command_reply: tid=0 2023-11-27T15:24:51.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.447+0000 7f337df56b80 1 client.16569 shutdown 2023-11-27T15:24:51.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.448+0000 7f337df56b80 1 -- 172.21.15.52:0/3155397005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556bdef41470 msgr2=0x556bdef4a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.448+0000 7f337df56b80 1 --2- 172.21.15.52:0/3155397005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556bdef41470 0x556bdef4a250 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f335c0036c0 tx=0x7f335c0057e0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.448+0000 7f337df56b80 1 -- 172.21.15.52:0/3155397005 shutdown_connections 2023-11-27T15:24:51.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.448+0000 7f337df56b80 1 --2- 172.21.15.52:0/3155397005 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x556bdef5a800 0x556bdef5cc10 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.448+0000 7f337df56b80 1 --2- 172.21.15.52:0/3155397005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556bdef40b60 0x556bdef4cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.448+0000 7f337df56b80 1 --2- 172.21.15.52:0/3155397005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556bdef458b0 0x556bdef4a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.448+0000 7f334affd700 0 client.16569 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:51.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.448+0000 7f337df56b80 1 --2- 172.21.15.52:0/3155397005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556bdef41470 0x556bdef4a250 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.448+0000 7f337df56b80 1 -- 172.21.15.52:0/3155397005 >> 172.21.15.52:0/3155397005 conn(0x556bdef36360 msgr2=0x556bdef3ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:51.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.448+0000 7f337df56b80 1 -- 172.21.15.52:0/3155397005 shutdown_connections 2023-11-27T15:24:51.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.448+0000 7f337df56b80 1 -- 172.21.15.52:0/3155397005 wait complete. 2023-11-27T15:24:51.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.448+0000 7f337df56b80 20 client.16569 trim_cache size 0 max 16384 2023-11-27T15:24:51.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.461+0000 7f337df56b80 1 Processor -- start 2023-11-27T15:24:51.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.461+0000 7f337df56b80 1 -- start start 2023-11-27T15:24:51.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.461+0000 7f337df56b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556bdef40b60 0x556bdef441a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.461+0000 7f337df56b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556bdef41470 0x556bdef51950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.461+0000 7f337df56b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556bdef458b0 0x556bdef44a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.461+0000 7f337df56b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdef4d730 con 0x556bdef40b60 2023-11-27T15:24:51.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.461+0000 7f337df56b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdee540a0 con 0x556bdef458b0 2023-11-27T15:24:51.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.461+0000 7f337df56b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdef336b0 con 0x556bdef41470 2023-11-27T15:24:51.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.461+0000 7f336b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556bdef40b60 0x556bdef441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.461+0000 7f336b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556bdef40b60 0x556bdef441a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54788/0 (socket says 172.21.15.52:54788) 2023-11-27T15:24:51.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.461+0000 7f336b7fe700 1 -- 172.21.15.52:0/654143558 learned_addr learned my addr 172.21.15.52:0/654143558 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:51.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.461+0000 7f336b7fe700 1 -- 172.21.15.52:0/654143558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556bdef41470 msgr2=0x556bdef51950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:51.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.461+0000 7f336affd700 1 --2- 172.21.15.52:0/654143558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556bdef41470 0x556bdef51950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.461+0000 7f336bfff700 1 --2- 172.21.15.52:0/654143558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556bdef458b0 0x556bdef44a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.461+0000 7f336b7fe700 1 --2- 172.21.15.52:0/654143558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556bdef41470 0x556bdef51950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.461+0000 7f336b7fe700 1 -- 172.21.15.52:0/654143558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556bdef458b0 msgr2=0x556bdef44a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:51.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.461+0000 7f336b7fe700 1 --2- 172.21.15.52:0/654143558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556bdef458b0 0x556bdef44a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.461+0000 7f336b7fe700 1 -- 172.21.15.52:0/654143558 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556bdef37b00 con 0x556bdef40b60 2023-11-27T15:24:51.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.461+0000 7f336bfff700 1 --2- 172.21.15.52:0/654143558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556bdef458b0 0x556bdef44a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:24:51.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.461+0000 7f336b7fe700 1 --2- 172.21.15.52:0/654143558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556bdef40b60 0x556bdef441a0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f3360010a40 tx=0x7f3360010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e092d80a114d41 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:51.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.462+0000 7f33527fc700 1 -- 172.21.15.52:0/654143558 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3360021040 con 0x556bdef40b60 2023-11-27T15:24:51.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.462+0000 7f33527fc700 1 -- 172.21.15.52:0/654143558 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f336001ae90 con 0x556bdef40b60 2023-11-27T15:24:51.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.462+0000 7f33527fc700 1 -- 172.21.15.52:0/654143558 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3360005040 con 0x556bdef40b60 2023-11-27T15:24:51.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.462+0000 7f337df56b80 1 -- 172.21.15.52:0/654143558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556bdef40b60 msgr2=0x556bdef441a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.462+0000 7f337df56b80 1 --2- 172.21.15.52:0/654143558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556bdef40b60 0x556bdef441a0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f3360010a40 tx=0x7f3360010fd0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.462+0000 7f337df56b80 1 -- 172.21.15.52:0/654143558 shutdown_connections 2023-11-27T15:24:51.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.462+0000 7f337df56b80 1 --2- 172.21.15.52:0/654143558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556bdef458b0 0x556bdef44a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.462+0000 7f337df56b80 1 --2- 172.21.15.52:0/654143558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556bdef40b60 0x556bdef441a0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.462+0000 7f337df56b80 1 --2- 172.21.15.52:0/654143558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556bdef41470 0x556bdef51950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.462+0000 7f337df56b80 1 -- 172.21.15.52:0/654143558 >> 172.21.15.52:0/654143558 conn(0x556bdef36360 msgr2=0x556bdef560f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:51.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.462+0000 7f337df56b80 1 -- 172.21.15.52:0/654143558 shutdown_connections 2023-11-27T15:24:51.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.462+0000 7f337df56b80 1 -- 172.21.15.52:0/654143558 wait complete. 2023-11-27T15:24:51.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.463+0000 7f337df56b80 1 -- start start 2023-11-27T15:24:51.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.463+0000 7f337df56b80 1 Processor -- start 2023-11-27T15:24:51.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.463+0000 7f337df56b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556bdef40b60 0x556bdef4f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.463+0000 7f337df56b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556bdef41470 0x556bdef4f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.463+0000 7f337df56b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556bdef458b0 0x556bdefad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.463+0000 7f337df56b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdef501b0 con 0x556bdef41470 2023-11-27T15:24:51.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.463+0000 7f337df56b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdefadc00 con 0x556bdef40b60 2023-11-27T15:24:51.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.463+0000 7f337df56b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdefadd40 con 0x556bdef458b0 2023-11-27T15:24:51.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.463+0000 7f337df56b80 10 client.0 _ll_register_callbacks cb 0x7f33640097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:51.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.463+0000 7f336affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556bdef41470 0x556bdef4f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.463+0000 7f336affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556bdef41470 0x556bdef4f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54796/0 (socket says 172.21.15.52:54796) 2023-11-27T15:24:51.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.463+0000 7f336affd700 1 -- 172.21.15.52:0/2895542792 learned_addr learned my addr 172.21.15.52:0/2895542792 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:51.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.463+0000 7f336bfff700 1 --2- 172.21.15.52:0/2895542792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556bdef458b0 0x556bdefad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.463+0000 7f336b7fe700 1 --2- 172.21.15.52:0/2895542792 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556bdef40b60 0x556bdef4f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.463+0000 7f336affd700 1 -- 172.21.15.52:0/2895542792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556bdef458b0 msgr2=0x556bdefad6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:51.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.463+0000 7f336affd700 1 --2- 172.21.15.52:0/2895542792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556bdef458b0 0x556bdefad6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.463+0000 7f336affd700 1 -- 172.21.15.52:0/2895542792 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556bdef40b60 msgr2=0x556bdef4f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:51.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.463+0000 7f336affd700 1 --2- 172.21.15.52:0/2895542792 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556bdef40b60 0x556bdef4f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.463+0000 7f336affd700 1 -- 172.21.15.52:0/2895542792 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556bded13630 con 0x556bdef41470 2023-11-27T15:24:51.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.463+0000 7f336b7fe700 1 --2- 172.21.15.52:0/2895542792 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556bdef40b60 0x556bdef4f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:24:51.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.464+0000 7f336affd700 1 --2- 172.21.15.52:0/2895542792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556bdef41470 0x556bdef4f8f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f335c0098e0 tx=0x7f335c0227c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c32e4ceb9cc21745 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:51.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.464+0000 7f334affd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:24:51.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.464+0000 7f334affd700 1 -- 172.21.15.52:0/2895542792 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c01c030 con 0x556bdef41470 2023-11-27T15:24:51.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.464+0000 7f334affd700 1 -- 172.21.15.52:0/2895542792 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f335c000e40 con 0x556bdef41470 2023-11-27T15:24:51.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.464+0000 7f337df56b80 1 -- 172.21.15.52:0/2895542792 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3360015070 con 0x556bdef41470 2023-11-27T15:24:51.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.464+0000 7f334affd700 1 -- 172.21.15.52:0/2895542792 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f335c005c00 con 0x556bdef41470 2023-11-27T15:24:51.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.464+0000 7f337df56b80 10 client.16575 fetch_fsmap learned FSMap version 28 2023-11-27T15:24:51.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.464+0000 7f337df56b80 1 -- 172.21.15.52:0/2895542792 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556bdecbfde0 con 0x556bdef41470 2023-11-27T15:24:51.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.464+0000 7f334affd700 1 -- 172.21.15.52:0/2895542792 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3360015070 con 0x556bdef41470 2023-11-27T15:24:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.464+0000 7f334affd700 1 -- 172.21.15.52:0/2895542792 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f335c03d070 con 0x556bdef41470 2023-11-27T15:24:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.464+0000 7f337df56b80 10 client.16575 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:24:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.464+0000 7f337df56b80 10 client.16575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.464+0000 7f337df56b80 20 client.16575 populate_metadata read hostname 'smithi052' 2023-11-27T15:24:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.464+0000 7f337df56b80 1 --2- 172.21.15.52:0/2895542792 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x556bdefaf030 0x556bdefb1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.464+0000 7f337df56b80 4 client.16575 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:24:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.464+0000 7f337df56b80 1 -- 172.21.15.52:0/2895542792 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556bdee60660 con 0x556bdefaf030 2023-11-27T15:24:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.465+0000 7f336b7fe700 1 --2- 172.21.15.52:0/2895542792 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x556bdefaf030 0x556bdefb1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.465+0000 7f334affd700 10 client.16575 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:51.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.465+0000 7f336b7fe700 1 --2- 172.21.15.52:0/2895542792 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x556bdefaf030 0x556bdefb1440 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a9621598277d3e3 server_cookie=28f75b326b2be16b in_seq=0 out_seq=0 2023-11-27T15:24:51.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.465+0000 7f334affd700 1 -- 172.21.15.52:0/2895542792 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556bdee60660 con 0x556bdefaf030 2023-11-27T15:24:51.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.465+0000 7f334affd700 10 client.16575 handle_command_reply: tid=0 2023-11-27T15:24:51.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.465+0000 7f337df56b80 1 client.16575 shutdown 2023-11-27T15:24:51.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.465+0000 7f337df56b80 1 -- 172.21.15.52:0/2895542792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556bdef41470 msgr2=0x556bdef4f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.465+0000 7f337df56b80 1 --2- 172.21.15.52:0/2895542792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556bdef41470 0x556bdef4f8f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f335c0098e0 tx=0x7f335c0227c0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.465+0000 7f337df56b80 1 -- 172.21.15.52:0/2895542792 shutdown_connections 2023-11-27T15:24:51.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.465+0000 7f337df56b80 1 --2- 172.21.15.52:0/2895542792 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x556bdefaf030 0x556bdefb1440 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.465+0000 7f337df56b80 1 --2- 172.21.15.52:0/2895542792 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556bdef40b60 0x556bdef4f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.465+0000 7f337df56b80 1 --2- 172.21.15.52:0/2895542792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556bdef458b0 0x556bdefad6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.465+0000 7f334affd700 0 client.16575 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:51.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.465+0000 7f337df56b80 1 --2- 172.21.15.52:0/2895542792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556bdef41470 0x556bdef4f8f0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.465+0000 7f337df56b80 1 -- 172.21.15.52:0/2895542792 >> 172.21.15.52:0/2895542792 conn(0x556bdef36360 msgr2=0x556bdef44a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:51.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.465+0000 7f337df56b80 1 -- 172.21.15.52:0/2895542792 shutdown_connections 2023-11-27T15:24:51.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.465+0000 7f337df56b80 1 -- 172.21.15.52:0/2895542792 wait complete. 2023-11-27T15:24:51.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.466+0000 7f337df56b80 20 client.16575 trim_cache size 0 max 16384 2023-11-27T15:24:51.481 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:24:51.482 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.466+0000 7f3371dca700 1 -- 172.21.15.52:0/3556858636 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f33540a12c0 msgr2=0x7f33540a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.482 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.466+0000 7f3371dca700 1 --2- 172.21.15.52:0/3556858636 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f33540a12c0 0x7f33540a3770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3360005950 tx=0x7f336000b410 comp rx=0 tx=0).stop 2023-11-27T15:24:51.483 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.466+0000 7f3371dca700 1 -- 172.21.15.52:0/3556858636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f336c104510 msgr2=0x7f336c199c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.483 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.466+0000 7f3371dca700 1 --2- 172.21.15.52:0/3556858636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f336c104510 0x7f336c199c90 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f336400ef30 tx=0x7f336400c3a0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.483 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.466+0000 7f3371dca700 1 -- 172.21.15.52:0/3556858636 shutdown_connections 2023-11-27T15:24:51.483 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.466+0000 7f3371dca700 1 --2- 172.21.15.52:0/3556858636 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f33540a12c0 0x7f33540a3770 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.483 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.466+0000 7f3371dca700 1 --2- 172.21.15.52:0/3556858636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f336c104510 0x7f336c199c90 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.483 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.466+0000 7f3371dca700 1 --2- 172.21.15.52:0/3556858636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f336c0ff4d0 0x7f336c1952f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.483 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.466+0000 7f3371dca700 1 --2- 172.21.15.52:0/3556858636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f336c0ffe70 0x7f336c195830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.484 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.466+0000 7f3371dca700 1 -- 172.21.15.52:0/3556858636 >> 172.21.15.52:0/3556858636 conn(0x7f336c076070 msgr2=0x7f336c0fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:51.484 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.466+0000 7f3371dca700 1 -- 172.21.15.52:0/3556858636 shutdown_connections 2023-11-27T15:24:51.484 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:51.466+0000 7f3371dca700 1 -- 172.21.15.52:0/3556858636 wait complete. 2023-11-27T15:24:52.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:52 smithi150 ceph-mon[100559]: pgmap v213: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:24:52.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:52 smithi102 ceph-mon[100296]: pgmap v213: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:24:52.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:52 smithi052 ceph-mon[99056]: pgmap v213: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:24:52.485 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-27T15:24:52.485 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-27T15:24:52.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.627+0000 7fb1cdf37700 1 -- 172.21.15.52:0/1445926832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1c80ffdf0 msgr2=0x7fb1c8103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:52.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.627+0000 7fb1cdf37700 1 --2- 172.21.15.52:0/1445926832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1c80ffdf0 0x7fb1c8103db0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb1b00099c0 tx=0x7fb1b0009cd0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.627+0000 7fb1cdf37700 1 -- 172.21.15.52:0/1445926832 shutdown_connections 2023-11-27T15:24:52.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.627+0000 7fb1cdf37700 1 --2- 172.21.15.52:0/1445926832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1c8104490 0x7fb1c810c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.627+0000 7fb1cdf37700 1 --2- 172.21.15.52:0/1445926832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1c80ff450 0x7fb1c80ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.627+0000 7fb1cdf37700 1 --2- 172.21.15.52:0/1445926832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1c80ffdf0 0x7fb1c8103db0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.627+0000 7fb1cdf37700 1 -- 172.21.15.52:0/1445926832 >> 172.21.15.52:0/1445926832 conn(0x7fb1c80770b0 msgr2=0x7fb1c80774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:52.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.627+0000 7fb1cdf37700 1 -- 172.21.15.52:0/1445926832 shutdown_connections 2023-11-27T15:24:52.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.627+0000 7fb1cdf37700 1 -- 172.21.15.52:0/1445926832 wait complete. 2023-11-27T15:24:52.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.628+0000 7fb1cdf37700 1 Processor -- start 2023-11-27T15:24:52.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.628+0000 7fb1cdf37700 1 -- start start 2023-11-27T15:24:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.629+0000 7fb1cdf37700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1c80ff450 0x7fb1c8195310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.629+0000 7fb1cdf37700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1c80ffdf0 0x7fb1c8195850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.629+0000 7fb1cdf37700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1c8104490 0x7fb1c8199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.629+0000 7fb1cdf37700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c819a2f0 con 0x7fb1c80ff450 2023-11-27T15:24:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.629+0000 7fb1cdf37700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c819a460 con 0x7fb1c8104490 2023-11-27T15:24:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.629+0000 7fb1cdf37700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c819a620 con 0x7fb1c80ffdf0 2023-11-27T15:24:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.629+0000 7fb1c77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1c80ff450 0x7fb1c8195310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.629+0000 7fb1c77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1c80ff450 0x7fb1c8195310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54808/0 (socket says 172.21.15.52:54808) 2023-11-27T15:24:52.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.629+0000 7fb1c77fe700 1 -- 172.21.15.52:0/3443275410 learned_addr learned my addr 172.21.15.52:0/3443275410 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:52.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.629+0000 7fb1c6ffd700 1 --2- 172.21.15.52:0/3443275410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1c80ffdf0 0x7fb1c8195850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.629+0000 7fb1c7fff700 1 --2- 172.21.15.52:0/3443275410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1c8104490 0x7fb1c8199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.629+0000 7fb1c77fe700 1 -- 172.21.15.52:0/3443275410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1c80ffdf0 msgr2=0x7fb1c8195850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:52.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.629+0000 7fb1c77fe700 1 --2- 172.21.15.52:0/3443275410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1c80ffdf0 0x7fb1c8195850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.629+0000 7fb1c77fe700 1 -- 172.21.15.52:0/3443275410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1c8104490 msgr2=0x7fb1c8199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:52.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.629+0000 7fb1c77fe700 1 --2- 172.21.15.52:0/3443275410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1c8104490 0x7fb1c8199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.629+0000 7fb1c77fe700 1 -- 172.21.15.52:0/3443275410 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b00095f0 con 0x7fb1c80ff450 2023-11-27T15:24:52.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.629+0000 7fb1c77fe700 1 --2- 172.21.15.52:0/3443275410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1c80ff450 0x7fb1c8195310 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800d950 tx=0x7fb1b800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:52.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.629+0000 7fb1c4ff9700 1 -- 172.21.15.52:0/3443275410 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8009940 con 0x7fb1c80ff450 2023-11-27T15:24:52.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.629+0000 7fb1c4ff9700 1 -- 172.21.15.52:0/3443275410 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1b8010460 con 0x7fb1c80ff450 2023-11-27T15:24:52.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.629+0000 7fb1cdf37700 1 -- 172.21.15.52:0/3443275410 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c810f210 con 0x7fb1c80ff450 2023-11-27T15:24:52.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.630+0000 7fb1c4ff9700 1 -- 172.21.15.52:0/3443275410 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b800f620 con 0x7fb1c80ff450 2023-11-27T15:24:52.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.630+0000 7fb1cdf37700 1 -- 172.21.15.52:0/3443275410 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c810f080 con 0x7fb1c80ff450 2023-11-27T15:24:52.644 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.632+0000 7fb1c4ff9700 1 -- 172.21.15.52:0/3443275410 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb1b80108d0 con 0x7fb1c80ff450 2023-11-27T15:24:52.644 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.633+0000 7fb1c4ff9700 1 --2- 172.21.15.52:0/3443275410 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb1b40a1340 0x7fb1b40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.644 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.633+0000 7fb1c4ff9700 1 -- 172.21.15.52:0/3443275410 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fb1b80ccbb0 con 0x7fb1c80ff450 2023-11-27T15:24:52.644 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.633+0000 7fb1c6ffd700 1 --2- 172.21.15.52:0/3443275410 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb1b40a1340 0x7fb1b40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.644 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.634+0000 7fb1da0c3b80 1 Processor -- start 2023-11-27T15:24:52.644 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.634+0000 7fb1c6ffd700 1 --2- 172.21.15.52:0/3443275410 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb1b40a1340 0x7fb1b40a37f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb1b00098e0 tx=0x7fb1b0000bc0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:52.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.634+0000 7fb1da0c3b80 1 -- start start 2023-11-27T15:24:52.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.634+0000 7fb1da0c3b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626a648ebe0 0x5626a648efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.634+0000 7fb1da0c3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626a648f4f0 0x5626a64933f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.634+0000 7fb1da0c3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626a6493930 0x5626a6493d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.634+0000 7fb1da0c3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a63940e0 con 0x5626a648f4f0 2023-11-27T15:24:52.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.634+0000 7fb1da0c3b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a63e6920 con 0x5626a648ebe0 2023-11-27T15:24:52.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.634+0000 7fb1da0c3b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a63dfdb0 con 0x5626a6493930 2023-11-27T15:24:52.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.634+0000 7fb1c6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626a648f4f0 0x5626a64933f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.634+0000 7fb1c6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626a648f4f0 0x5626a64933f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54818/0 (socket says 172.21.15.52:54818) 2023-11-27T15:24:52.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.634+0000 7fb1c6ffd700 1 -- 172.21.15.52:0/1014507771 learned_addr learned my addr 172.21.15.52:0/1014507771 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:52.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.634+0000 7fb1c7fff700 1 --2- 172.21.15.52:0/1014507771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626a6493930 0x5626a6493d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.634+0000 7fb1c77fe700 1 --2- 172.21.15.52:0/1014507771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626a648ebe0 0x5626a648efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.634+0000 7fb1c6ffd700 1 -- 172.21.15.52:0/1014507771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626a6493930 msgr2=0x5626a6493d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:52.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.634+0000 7fb1c6ffd700 1 --2- 172.21.15.52:0/1014507771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626a6493930 0x5626a6493d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.634+0000 7fb1c6ffd700 1 -- 172.21.15.52:0/1014507771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626a648ebe0 msgr2=0x5626a648efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:52.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.634+0000 7fb1c6ffd700 1 --2- 172.21.15.52:0/1014507771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626a648ebe0 0x5626a648efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.634+0000 7fb1c6ffd700 1 -- 172.21.15.52:0/1014507771 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626a62616b0 con 0x5626a648f4f0 2023-11-27T15:24:52.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.635+0000 7fb1c6ffd700 1 --2- 172.21.15.52:0/1014507771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626a648f4f0 0x5626a64933f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb1b00099c0 tx=0x7fb1b001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d4e41dca460637 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:52.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.635+0000 7fb1bdffb700 1 -- 172.21.15.52:0/1014507771 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b000b990 con 0x5626a648f4f0 2023-11-27T15:24:52.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.635+0000 7fb1bdffb700 1 -- 172.21.15.52:0/1014507771 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1b001e040 con 0x5626a648f4f0 2023-11-27T15:24:52.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.635+0000 7fb1bdffb700 1 -- 172.21.15.52:0/1014507771 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b00306e0 con 0x5626a648f4f0 2023-11-27T15:24:52.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.635+0000 7fb1da0c3b80 1 -- 172.21.15.52:0/1014507771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626a648f4f0 msgr2=0x5626a64933f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:52.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.635+0000 7fb1da0c3b80 1 --2- 172.21.15.52:0/1014507771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626a648f4f0 0x5626a64933f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb1b00099c0 tx=0x7fb1b001f000 comp rx=0 tx=0).stop 2023-11-27T15:24:52.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.635+0000 7fb1da0c3b80 1 -- 172.21.15.52:0/1014507771 shutdown_connections 2023-11-27T15:24:52.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.635+0000 7fb1da0c3b80 1 --2- 172.21.15.52:0/1014507771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626a648ebe0 0x5626a648efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.635+0000 7fb1da0c3b80 1 --2- 172.21.15.52:0/1014507771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626a6493930 0x5626a6493d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.635+0000 7fb1da0c3b80 1 --2- 172.21.15.52:0/1014507771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626a648f4f0 0x5626a64933f0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.636+0000 7fb1da0c3b80 1 -- 172.21.15.52:0/1014507771 >> 172.21.15.52:0/1014507771 conn(0x5626a64843e0 msgr2=0x5626a6489bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:52.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.636+0000 7fb1da0c3b80 1 -- 172.21.15.52:0/1014507771 shutdown_connections 2023-11-27T15:24:52.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.636+0000 7fb1da0c3b80 1 -- 172.21.15.52:0/1014507771 wait complete. 2023-11-27T15:24:52.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.636+0000 7fb1da0c3b80 1 -- start start 2023-11-27T15:24:52.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.636+0000 7fb1da0c3b80 1 Processor -- start 2023-11-27T15:24:52.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.636+0000 7fb1da0c3b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626a648ebe0 0x5626a649b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.636+0000 7fb1da0c3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626a648f4f0 0x5626a6498330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.636+0000 7fb1da0c3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626a6493930 0x5626a6498a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.637+0000 7fb1da0c3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a6381960 con 0x5626a6493930 2023-11-27T15:24:52.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.637+0000 7fb1da0c3b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a6381600 con 0x5626a648ebe0 2023-11-27T15:24:52.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.637+0000 7fb1da0c3b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a62639a0 con 0x5626a648f4f0 2023-11-27T15:24:52.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.637+0000 7fb1da0c3b80 10 client.0 _ll_register_callbacks cb 0x5626a639f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:52.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.637+0000 7fb1c7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626a6493930 0x5626a6498a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.637+0000 7fb1c77fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626a648ebe0 0x5626a649b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.637+0000 7fb1c7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626a6493930 0x5626a6498a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54826/0 (socket says 172.21.15.52:54826) 2023-11-27T15:24:52.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.637+0000 7fb1c77fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626a648ebe0 0x5626a649b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:44480/0 (socket says 172.21.15.52:44480) 2023-11-27T15:24:52.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.637+0000 7fb1c7fff700 1 -- 172.21.15.52:0/3217114549 learned_addr learned my addr 172.21.15.52:0/3217114549 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:52.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.637+0000 7fb1c6ffd700 1 --2- 172.21.15.52:0/3217114549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626a648f4f0 0x5626a6498330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.637+0000 7fb1c7fff700 1 -- 172.21.15.52:0/3217114549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626a648f4f0 msgr2=0x5626a6498330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:52.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.637+0000 7fb1c7fff700 1 --2- 172.21.15.52:0/3217114549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626a648f4f0 0x5626a6498330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.637+0000 7fb1c7fff700 1 -- 172.21.15.52:0/3217114549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626a648ebe0 msgr2=0x5626a649b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:52.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.637+0000 7fb1c7fff700 1 --2- 172.21.15.52:0/3217114549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626a648ebe0 0x5626a649b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.637+0000 7fb1c7fff700 1 -- 172.21.15.52:0/3217114549 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c810c8a0 con 0x5626a6493930 2023-11-27T15:24:52.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.637+0000 7fb1c77fe700 1 --2- 172.21.15.52:0/3217114549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626a648ebe0 0x5626a649b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:52.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.637+0000 7fb1c7fff700 1 --2- 172.21.15.52:0/3217114549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626a6493930 0x5626a6498a70 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009fd0 tx=0x7fb1c0007ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde974887c20eca9 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:52.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.637+0000 7fb1a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:24:52.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.637+0000 7fb1a6ffd700 1 -- 172.21.15.52:0/3217114549 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c000e070 con 0x5626a6493930 2023-11-27T15:24:52.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.637+0000 7fb1a6ffd700 1 -- 172.21.15.52:0/3217114549 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fb1c00093c0 con 0x5626a6493930 2023-11-27T15:24:52.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.637+0000 7fb1da0c3b80 1 -- 172.21.15.52:0/3217114549 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626a6481ba0 con 0x5626a6493930 2023-11-27T15:24:52.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.638+0000 7fb1a6ffd700 1 -- 172.21.15.52:0/3217114549 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1c0013b30 con 0x5626a6493930 2023-11-27T15:24:52.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.638+0000 7fb1da0c3b80 10 client.16587 fetch_fsmap learned FSMap version 28 2023-11-27T15:24:52.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.638+0000 7fb1da0c3b80 1 -- 172.21.15.52:0/3217114549 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5626a63a6210 con 0x5626a6493930 2023-11-27T15:24:52.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.638+0000 7fb1a6ffd700 1 -- 172.21.15.52:0/3217114549 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5626a6481ba0 con 0x5626a6493930 2023-11-27T15:24:52.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.638+0000 7fb1a6ffd700 1 -- 172.21.15.52:0/3217114549 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb1c0015390 con 0x5626a6493930 2023-11-27T15:24:52.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.638+0000 7fb1da0c3b80 10 client.16587 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:24:52.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.638+0000 7fb1da0c3b80 10 client.16587 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:52.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.638+0000 7fb1da0c3b80 20 client.16587 populate_metadata read hostname 'smithi052' 2023-11-27T15:24:52.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.638+0000 7fb1da0c3b80 1 --2- 172.21.15.52:0/3217114549 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5626a64a8900 0x5626a64aad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.638+0000 7fb1da0c3b80 4 client.16587 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:24:52.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.638+0000 7fb1da0c3b80 1 -- 172.21.15.52:0/3217114549 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626a63a2120 con 0x5626a64a8900 2023-11-27T15:24:52.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.638+0000 7fb1c77fe700 1 --2- 172.21.15.52:0/3217114549 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5626a64a8900 0x5626a64aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.639+0000 7fb1a6ffd700 10 client.16587 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:52.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.639+0000 7fb1c77fe700 1 --2- 172.21.15.52:0/3217114549 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5626a64a8900 0x5626a64aad10 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ca1e0525c95d0d7 server_cookie=4bb7b601e53ef6d8 in_seq=0 out_seq=0 2023-11-27T15:24:52.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.639+0000 7fb1a6ffd700 1 -- 172.21.15.52:0/3217114549 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5626a63a2120 con 0x5626a64a8900 2023-11-27T15:24:52.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.639+0000 7fb1a6ffd700 10 client.16587 handle_command_reply: tid=0 2023-11-27T15:24:52.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.639+0000 7fb1da0c3b80 1 client.16587 shutdown 2023-11-27T15:24:52.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.640+0000 7fb1da0c3b80 1 -- 172.21.15.52:0/3217114549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626a6493930 msgr2=0x5626a6498a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:52.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.640+0000 7fb1da0c3b80 1 --2- 172.21.15.52:0/3217114549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626a6493930 0x5626a6498a70 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009fd0 tx=0x7fb1c0007ed0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.640+0000 7fb1da0c3b80 1 -- 172.21.15.52:0/3217114549 shutdown_connections 2023-11-27T15:24:52.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.640+0000 7fb1da0c3b80 1 --2- 172.21.15.52:0/3217114549 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5626a64a8900 0x5626a64aad10 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.640+0000 7fb1da0c3b80 1 --2- 172.21.15.52:0/3217114549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626a648ebe0 0x5626a649b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.640+0000 7fb1a6ffd700 0 client.16587 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:52.655 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.640+0000 7fb1da0c3b80 1 --2- 172.21.15.52:0/3217114549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626a6493930 0x5626a6498a70 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.655 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.640+0000 7fb1da0c3b80 1 --2- 172.21.15.52:0/3217114549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626a648f4f0 0x5626a6498330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.655 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.640+0000 7fb1da0c3b80 1 -- 172.21.15.52:0/3217114549 >> 172.21.15.52:0/3217114549 conn(0x5626a64843e0 msgr2=0x5626a6488e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:52.655 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.640+0000 7fb1da0c3b80 1 -- 172.21.15.52:0/3217114549 shutdown_connections 2023-11-27T15:24:52.655 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.640+0000 7fb1da0c3b80 1 -- 172.21.15.52:0/3217114549 wait complete. 2023-11-27T15:24:52.655 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.640+0000 7fb1da0c3b80 20 client.16587 trim_cache size 0 max 16384 2023-11-27T15:24:52.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.657+0000 7fb1da0c3b80 1 Processor -- start 2023-11-27T15:24:52.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.657+0000 7fb1da0c3b80 1 -- start start 2023-11-27T15:24:52.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.657+0000 7fb1da0c3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626a648ebe0 0x5626a6495de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.657+0000 7fb1da0c3b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626a648f4f0 0x5626a649f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.657+0000 7fb1da0c3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626a6493930 0x5626a649ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.657+0000 7fb1da0c3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a6496320 con 0x5626a6493930 2023-11-27T15:24:52.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.657+0000 7fb1da0c3b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a6498dd0 con 0x5626a648f4f0 2023-11-27T15:24:52.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.657+0000 7fb1da0c3b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a6481a20 con 0x5626a648ebe0 2023-11-27T15:24:52.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.657+0000 7fb1c7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626a6493930 0x5626a649ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.657+0000 7fb1c7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626a6493930 0x5626a649ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54834/0 (socket says 172.21.15.52:54834) 2023-11-27T15:24:52.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.657+0000 7fb1c7fff700 1 -- 172.21.15.52:0/3773127697 learned_addr learned my addr 172.21.15.52:0/3773127697 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:52.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.657+0000 7fb1c77fe700 1 --2- 172.21.15.52:0/3773127697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626a648ebe0 0x5626a6495de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.657+0000 7fb1c6ffd700 1 --2- 172.21.15.52:0/3773127697 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626a648f4f0 0x5626a649f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.657+0000 7fb1c7fff700 1 -- 172.21.15.52:0/3773127697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626a648ebe0 msgr2=0x5626a6495de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:52.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.657+0000 7fb1c7fff700 1 --2- 172.21.15.52:0/3773127697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626a648ebe0 0x5626a6495de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.663 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.657+0000 7fb1c7fff700 1 -- 172.21.15.52:0/3773127697 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626a648f4f0 msgr2=0x5626a649f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:52.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.657+0000 7fb1c7fff700 1 --2- 172.21.15.52:0/3773127697 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626a648f4f0 0x5626a649f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.657+0000 7fb1c7fff700 1 -- 172.21.15.52:0/3773127697 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626a6493fd0 con 0x5626a6493930 2023-11-27T15:24:52.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.657+0000 7fb1c6ffd700 1 --2- 172.21.15.52:0/3773127697 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626a648f4f0 0x5626a649f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:24:52.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.658+0000 7fb1c7fff700 1 --2- 172.21.15.52:0/3773127697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626a6493930 0x5626a649ac50 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb1c00157a0 tx=0x7fb1c0013650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0aa5ef81fdd5c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:52.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.658+0000 7fb1be7fc700 1 -- 172.21.15.52:0/3773127697 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c000e040 con 0x5626a6493930 2023-11-27T15:24:52.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.658+0000 7fb1be7fc700 1 -- 172.21.15.52:0/3773127697 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1c000f040 con 0x5626a6493930 2023-11-27T15:24:52.664 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.658+0000 7fb1be7fc700 1 -- 172.21.15.52:0/3773127697 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0013b90 con 0x5626a6493930 2023-11-27T15:24:52.668 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.658+0000 7fb1da0c3b80 1 -- 172.21.15.52:0/3773127697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626a6493930 msgr2=0x5626a649ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:52.668 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.658+0000 7fb1da0c3b80 1 --2- 172.21.15.52:0/3773127697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626a6493930 0x5626a649ac50 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb1c00157a0 tx=0x7fb1c0013650 comp rx=0 tx=0).stop 2023-11-27T15:24:52.669 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.658+0000 7fb1da0c3b80 1 -- 172.21.15.52:0/3773127697 shutdown_connections 2023-11-27T15:24:52.669 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.658+0000 7fb1da0c3b80 1 --2- 172.21.15.52:0/3773127697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626a6493930 0x5626a649ac50 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.669 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.658+0000 7fb1da0c3b80 1 --2- 172.21.15.52:0/3773127697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626a648ebe0 0x5626a6495de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.669 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.658+0000 7fb1da0c3b80 1 --2- 172.21.15.52:0/3773127697 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626a648f4f0 0x5626a649f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.669 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.658+0000 7fb1da0c3b80 1 -- 172.21.15.52:0/3773127697 >> 172.21.15.52:0/3773127697 conn(0x5626a64843e0 msgr2=0x5626a648e270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:52.669 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.658+0000 7fb1da0c3b80 1 -- 172.21.15.52:0/3773127697 shutdown_connections 2023-11-27T15:24:52.669 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.658+0000 7fb1da0c3b80 1 -- 172.21.15.52:0/3773127697 wait complete. 2023-11-27T15:24:52.669 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.659+0000 7fb1da0c3b80 1 -- start start 2023-11-27T15:24:52.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.659+0000 7fb1da0c3b80 1 Processor -- start 2023-11-27T15:24:52.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.659+0000 7fb1da0c3b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626a648ebe0 0x5626a64977b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.659+0000 7fb1da0c3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626a648f4f0 0x5626a6497cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.659+0000 7fb1da0c3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626a6493930 0x5626a64fab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.659+0000 7fb1da0c3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a64fb0d0 con 0x5626a648f4f0 2023-11-27T15:24:52.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.659+0000 7fb1da0c3b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a64fb240 con 0x5626a648ebe0 2023-11-27T15:24:52.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.659+0000 7fb1da0c3b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626a64fb400 con 0x5626a6493930 2023-11-27T15:24:52.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.659+0000 7fb1da0c3b80 10 client.0 _ll_register_callbacks cb 0x7fb1c00078a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:52.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.659+0000 7fb1c6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626a648f4f0 0x5626a6497cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.659+0000 7fb1c6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626a648f4f0 0x5626a6497cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54838/0 (socket says 172.21.15.52:54838) 2023-11-27T15:24:52.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.659+0000 7fb1c6ffd700 1 -- 172.21.15.52:0/4097125513 learned_addr learned my addr 172.21.15.52:0/4097125513 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:52.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.659+0000 7fb1c7fff700 1 --2- 172.21.15.52:0/4097125513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626a6493930 0x5626a64fab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.659+0000 7fb1c77fe700 1 --2- 172.21.15.52:0/4097125513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626a648ebe0 0x5626a64977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.660+0000 7fb1c6ffd700 1 -- 172.21.15.52:0/4097125513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626a6493930 msgr2=0x5626a64fab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:52.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.660+0000 7fb1c6ffd700 1 --2- 172.21.15.52:0/4097125513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626a6493930 0x5626a64fab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.660+0000 7fb1c6ffd700 1 -- 172.21.15.52:0/4097125513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626a648ebe0 msgr2=0x5626a64977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:52.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.660+0000 7fb1c6ffd700 1 --2- 172.21.15.52:0/4097125513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626a648ebe0 0x5626a64977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.660+0000 7fb1c6ffd700 1 -- 172.21.15.52:0/4097125513 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626a63a6210 con 0x5626a648f4f0 2023-11-27T15:24:52.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.660+0000 7fb1c6ffd700 1 --2- 172.21.15.52:0/4097125513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626a648f4f0 0x5626a6497cf0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0003a30 tx=0x7fb1b000bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8459ac9825424e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:52.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.660+0000 7fb1a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:24:52.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.660+0000 7fb1a6ffd700 1 -- 172.21.15.52:0/4097125513 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b00046c0 con 0x5626a648f4f0 2023-11-27T15:24:52.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.660+0000 7fb1a6ffd700 1 -- 172.21.15.52:0/4097125513 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fb1b0030480 con 0x5626a648f4f0 2023-11-27T15:24:52.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.660+0000 7fb1da0c3b80 1 -- 172.21.15.52:0/4097125513 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb1b001e040 con 0x5626a648f4f0 2023-11-27T15:24:52.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.660+0000 7fb1a6ffd700 1 -- 172.21.15.52:0/4097125513 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1b0004820 con 0x5626a648f4f0 2023-11-27T15:24:52.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.660+0000 7fb1da0c3b80 10 client.16593 fetch_fsmap learned FSMap version 28 2023-11-27T15:24:52.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.660+0000 7fb1da0c3b80 1 -- 172.21.15.52:0/4097125513 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5626a620de60 con 0x5626a648f4f0 2023-11-27T15:24:52.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.660+0000 7fb1a6ffd700 1 -- 172.21.15.52:0/4097125513 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1b0047020 con 0x5626a648f4f0 2023-11-27T15:24:52.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.660+0000 7fb1a6ffd700 1 -- 172.21.15.52:0/4097125513 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb1b000f730 con 0x5626a648f4f0 2023-11-27T15:24:52.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.660+0000 7fb1da0c3b80 10 client.16593 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:24:52.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.660+0000 7fb1da0c3b80 10 client.16593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:52.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.660+0000 7fb1da0c3b80 20 client.16593 populate_metadata read hostname 'smithi052' 2023-11-27T15:24:52.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.660+0000 7fb1da0c3b80 1 --2- 172.21.15.52:0/4097125513 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5626a6502750 0x5626a6504b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.660+0000 7fb1da0c3b80 4 client.16593 mds_command: new command op to 24536 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-27T15:24:52.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.660+0000 7fb1da0c3b80 1 -- 172.21.15.52:0/4097125513 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5626a63ae6e0 con 0x5626a6502750 2023-11-27T15:24:52.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.661+0000 7fb1c77fe700 1 --2- 172.21.15.52:0/4097125513 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5626a6502750 0x5626a6504b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.661+0000 7fb1a6ffd700 10 client.16593 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:52.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.661+0000 7fb1c77fe700 1 --2- 172.21.15.52:0/4097125513 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5626a6502750 0x5626a6504b30 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1702a555015feb2 server_cookie=731668c93c1b250b in_seq=0 out_seq=0 2023-11-27T15:24:52.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.662+0000 7fb1a6ffd700 1 -- 172.21.15.52:0/4097125513 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5626a63ae6e0 con 0x5626a6502750 2023-11-27T15:24:52.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.662+0000 7fb1a6ffd700 10 client.16593 handle_command_reply: tid=0 2023-11-27T15:24:52.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.662+0000 7fb1da0c3b80 1 client.16593 shutdown 2023-11-27T15:24:52.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.662+0000 7fb1da0c3b80 1 -- 172.21.15.52:0/4097125513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626a648f4f0 msgr2=0x5626a6497cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:52.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.662+0000 7fb1da0c3b80 1 --2- 172.21.15.52:0/4097125513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626a648f4f0 0x5626a6497cf0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0003a30 tx=0x7fb1b000bf70 comp rx=0 tx=0).stop 2023-11-27T15:24:52.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.663+0000 7fb1da0c3b80 1 -- 172.21.15.52:0/4097125513 shutdown_connections 2023-11-27T15:24:52.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.663+0000 7fb1da0c3b80 1 --2- 172.21.15.52:0/4097125513 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5626a6502750 0x5626a6504b30 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.663+0000 7fb1da0c3b80 1 --2- 172.21.15.52:0/4097125513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626a648ebe0 0x5626a64977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.663+0000 7fb1a6ffd700 0 client.16593 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:52.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.663+0000 7fb1da0c3b80 1 --2- 172.21.15.52:0/4097125513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5626a6493930 0x5626a64fab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.676 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.663+0000 7fb1da0c3b80 1 --2- 172.21.15.52:0/4097125513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5626a648f4f0 0x5626a6497cf0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.676 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.663+0000 7fb1da0c3b80 1 -- 172.21.15.52:0/4097125513 >> 172.21.15.52:0/4097125513 conn(0x5626a64843e0 msgr2=0x5626a64928b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:52.676 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.663+0000 7fb1da0c3b80 1 -- 172.21.15.52:0/4097125513 shutdown_connections 2023-11-27T15:24:52.676 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.663+0000 7fb1da0c3b80 1 -- 172.21.15.52:0/4097125513 wait complete. 2023-11-27T15:24:52.676 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.663+0000 7fb1da0c3b80 20 client.16593 trim_cache size 0 max 16384 2023-11-27T15:24:52.676 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:24:52.677 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2023-11-27T15:24:52.677 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "2155fb61-dca4-4c9a-9b19-32b5f67c7159", 2023-11-27T15:24:52.677 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2023-11-27T15:24:52.677 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:24:52.678 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.663+0000 7fb1cdf37700 1 -- 172.21.15.52:0/3443275410 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb1b40a1340 msgr2=0x7fb1b40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:52.678 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.663+0000 7fb1cdf37700 1 --2- 172.21.15.52:0/3443275410 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb1b40a1340 0x7fb1b40a37f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb1b00098e0 tx=0x7fb1b0000bc0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.678 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.663+0000 7fb1cdf37700 1 -- 172.21.15.52:0/3443275410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1c80ff450 msgr2=0x7fb1c8195310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:52.678 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.663+0000 7fb1cdf37700 1 --2- 172.21.15.52:0/3443275410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1c80ff450 0x7fb1c8195310 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800d950 tx=0x7fb1b800dc60 comp rx=0 tx=0).stop 2023-11-27T15:24:52.678 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.663+0000 7fb1cdf37700 1 -- 172.21.15.52:0/3443275410 shutdown_connections 2023-11-27T15:24:52.678 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.663+0000 7fb1cdf37700 1 --2- 172.21.15.52:0/3443275410 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb1b40a1340 0x7fb1b40a37f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.679 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.663+0000 7fb1cdf37700 1 --2- 172.21.15.52:0/3443275410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1c8104490 0x7fb1c8199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.679 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.663+0000 7fb1cdf37700 1 --2- 172.21.15.52:0/3443275410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1c80ff450 0x7fb1c8195310 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.679 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.663+0000 7fb1cdf37700 1 --2- 172.21.15.52:0/3443275410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1c80ffdf0 0x7fb1c8195850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.679 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.663+0000 7fb1cdf37700 1 -- 172.21.15.52:0/3443275410 >> 172.21.15.52:0/3443275410 conn(0x7fb1c80770b0 msgr2=0x7fb1c80fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:52.679 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.663+0000 7fb1cdf37700 1 -- 172.21.15.52:0/3443275410 shutdown_connections 2023-11-27T15:24:52.679 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.663+0000 7fb1cdf37700 1 -- 172.21.15.52:0/3443275410 wait complete. 2023-11-27T15:24:52.680 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:24:52.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.805+0000 7fd98531e700 1 -- 172.21.15.52:0/3155727711 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd980100570 msgr2=0x7fd980100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:52.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.805+0000 7fd98531e700 1 --2- 172.21.15.52:0/3155727711 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd980100570 0x7fd980100940 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fd9740099c0 tx=0x7fd974009cd0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.805+0000 7fd98531e700 1 -- 172.21.15.52:0/3155727711 shutdown_connections 2023-11-27T15:24:52.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.805+0000 7fd98531e700 1 --2- 172.21.15.52:0/3155727711 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9801055b0 0x7fd98010d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.805+0000 7fd98531e700 1 --2- 172.21.15.52:0/3155727711 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd980100570 0x7fd980100940 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.805+0000 7fd98531e700 1 --2- 172.21.15.52:0/3155727711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd980100f10 0x7fd980104ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.810 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.805+0000 7fd98531e700 1 -- 172.21.15.52:0/3155727711 >> 172.21.15.52:0/3155727711 conn(0x7fd9800770e0 msgr2=0x7fd9800774e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:52.810 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.805+0000 7fd98531e700 1 -- 172.21.15.52:0/3155727711 shutdown_connections 2023-11-27T15:24:52.810 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.805+0000 7fd98531e700 1 -- 172.21.15.52:0/3155727711 wait complete. 2023-11-27T15:24:52.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.805+0000 7fd98531e700 1 Processor -- start 2023-11-27T15:24:52.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.806+0000 7fd98531e700 1 -- start start 2023-11-27T15:24:52.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.806+0000 7fd98531e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd980100570 0x7fd980196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.806+0000 7fd98531e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd980100f10 0x7fd9801969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.806+0000 7fd98531e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9801055b0 0x7fd98019ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.806+0000 7fd98531e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98019b440 con 0x7fd980100f10 2023-11-27T15:24:52.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.806+0000 7fd98531e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98019b5b0 con 0x7fd980100570 2023-11-27T15:24:52.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.806+0000 7fd98531e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98019b770 con 0x7fd9801055b0 2023-11-27T15:24:52.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.806+0000 7fd97e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd980100f10 0x7fd9801969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.806+0000 7fd97e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd980100f10 0x7fd9801969a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54850/0 (socket says 172.21.15.52:54850) 2023-11-27T15:24:52.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.806+0000 7fd97e7fc700 1 -- 172.21.15.52:0/1337860360 learned_addr learned my addr 172.21.15.52:0/1337860360 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:52.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.806+0000 7fd97f7fe700 1 --2- 172.21.15.52:0/1337860360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9801055b0 0x7fd98019ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.806+0000 7fd97effd700 1 --2- 172.21.15.52:0/1337860360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd980100570 0x7fd980196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.806+0000 7fd97e7fc700 1 -- 172.21.15.52:0/1337860360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9801055b0 msgr2=0x7fd98019ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:52.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.806+0000 7fd97e7fc700 1 --2- 172.21.15.52:0/1337860360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9801055b0 0x7fd98019ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.806+0000 7fd97e7fc700 1 -- 172.21.15.52:0/1337860360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd980100570 msgr2=0x7fd980196460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:52.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.806+0000 7fd97e7fc700 1 --2- 172.21.15.52:0/1337860360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd980100570 0x7fd980196460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.806+0000 7fd97e7fc700 1 -- 172.21.15.52:0/1337860360 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9740095f0 con 0x7fd980100f10 2023-11-27T15:24:52.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.806+0000 7fd97e7fc700 1 --2- 172.21.15.52:0/1337860360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd980100f10 0x7fd9801969a0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd97000d950 tx=0x7fd97000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:52.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.807+0000 7fd967fff700 1 -- 172.21.15.52:0/1337860360 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd970009b40 con 0x7fd980100f10 2023-11-27T15:24:52.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.807+0000 7fd98531e700 1 -- 172.21.15.52:0/1337860360 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd980110370 con 0x7fd980100f10 2023-11-27T15:24:52.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.807+0000 7fd967fff700 1 -- 172.21.15.52:0/1337860360 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd97000de30 con 0x7fd980100f10 2023-11-27T15:24:52.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.807+0000 7fd967fff700 1 -- 172.21.15.52:0/1337860360 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97000f620 con 0x7fd980100f10 2023-11-27T15:24:52.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.807+0000 7fd98531e700 1 -- 172.21.15.52:0/1337860360 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9801101e0 con 0x7fd980100f10 2023-11-27T15:24:52.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.809+0000 7fd967fff700 1 -- 172.21.15.52:0/1337860360 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fd970010460 con 0x7fd980100f10 2023-11-27T15:24:52.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.809+0000 7fd967fff700 1 --2- 172.21.15.52:0/1337860360 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd9680a12c0 0x7fd9680a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.810+0000 7fd967fff700 1 -- 172.21.15.52:0/1337860360 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fd9700cc980 con 0x7fd980100f10 2023-11-27T15:24:52.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.810+0000 7fd97effd700 1 --2- 172.21.15.52:0/1337860360 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd9680a12c0 0x7fd9680a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.810+0000 7fd97effd700 1 --2- 172.21.15.52:0/1337860360 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd9680a12c0 0x7fd9680a3770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd974009fd0 tx=0x7fd97401a040 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:52.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.810+0000 7fd9914aab80 1 Processor -- start 2023-11-27T15:24:52.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.810+0000 7fd9914aab80 1 -- start start 2023-11-27T15:24:52.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.811+0000 7fd9914aab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bf2c63b60 0x564bf2c63f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.811+0000 7fd9914aab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bf2c64470 0x564bf2c68370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.811+0000 7fd9914aab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564bf2c688b0 0x564bf2c68d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.811+0000 7fd9914aab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bf2b69060 con 0x564bf2c64470 2023-11-27T15:24:52.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.811+0000 7fd9914aab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bf2bbb8a0 con 0x564bf2c688b0 2023-11-27T15:24:52.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.811+0000 7fd9914aab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bf2bb4d30 con 0x564bf2c63b60 2023-11-27T15:24:52.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.811+0000 7fd97e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bf2c64470 0x564bf2c68370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.811+0000 7fd97e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bf2c64470 0x564bf2c68370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54858/0 (socket says 172.21.15.52:54858) 2023-11-27T15:24:52.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.811+0000 7fd97e7fc700 1 -- 172.21.15.52:0/2053762359 learned_addr learned my addr 172.21.15.52:0/2053762359 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:52.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.811+0000 7fd97effd700 1 --2- 172.21.15.52:0/2053762359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bf2c63b60 0x564bf2c63f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.811+0000 7fd97f7fe700 1 --2- 172.21.15.52:0/2053762359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564bf2c688b0 0x564bf2c68d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.811+0000 7fd97e7fc700 1 -- 172.21.15.52:0/2053762359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bf2c63b60 msgr2=0x564bf2c63f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:52.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.811+0000 7fd97e7fc700 1 --2- 172.21.15.52:0/2053762359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bf2c63b60 0x564bf2c63f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.811+0000 7fd97e7fc700 1 -- 172.21.15.52:0/2053762359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564bf2c688b0 msgr2=0x564bf2c68d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:52.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.811+0000 7fd97e7fc700 1 --2- 172.21.15.52:0/2053762359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564bf2c688b0 0x564bf2c68d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.811+0000 7fd97e7fc700 1 -- 172.21.15.52:0/2053762359 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564bf2a36630 con 0x564bf2c64470 2023-11-27T15:24:52.823 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.811+0000 7fd97f7fe700 1 --2- 172.21.15.52:0/2053762359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564bf2c688b0 0x564bf2c68d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:24:52.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.811+0000 7fd97e7fc700 1 --2- 172.21.15.52:0/2053762359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bf2c64470 0x564bf2c68370 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fd970095f20 tx=0x7fd970099380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb85d660b1676f2d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:52.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.811+0000 7fd9657fa700 1 -- 172.21.15.52:0/2053762359 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9700a1e70 con 0x564bf2c64470 2023-11-27T15:24:52.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.811+0000 7fd9657fa700 1 -- 172.21.15.52:0/2053762359 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd970097dc0 con 0x564bf2c64470 2023-11-27T15:24:52.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.811+0000 7fd9657fa700 1 -- 172.21.15.52:0/2053762359 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9700a19e0 con 0x564bf2c64470 2023-11-27T15:24:52.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.812+0000 7fd9914aab80 1 -- 172.21.15.52:0/2053762359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bf2c64470 msgr2=0x564bf2c68370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:52.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.812+0000 7fd9914aab80 1 --2- 172.21.15.52:0/2053762359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bf2c64470 0x564bf2c68370 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fd970095f20 tx=0x7fd970099380 comp rx=0 tx=0).stop 2023-11-27T15:24:52.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.812+0000 7fd9914aab80 1 -- 172.21.15.52:0/2053762359 shutdown_connections 2023-11-27T15:24:52.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.812+0000 7fd9914aab80 1 --2- 172.21.15.52:0/2053762359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564bf2c688b0 0x564bf2c68d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.812+0000 7fd9914aab80 1 --2- 172.21.15.52:0/2053762359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bf2c63b60 0x564bf2c63f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.812+0000 7fd9914aab80 1 --2- 172.21.15.52:0/2053762359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bf2c64470 0x564bf2c68370 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.812+0000 7fd9914aab80 1 -- 172.21.15.52:0/2053762359 >> 172.21.15.52:0/2053762359 conn(0x564bf2c59360 msgr2=0x564bf2c5eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:52.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.812+0000 7fd9914aab80 1 -- 172.21.15.52:0/2053762359 shutdown_connections 2023-11-27T15:24:52.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.812+0000 7fd9914aab80 1 -- 172.21.15.52:0/2053762359 wait complete. 2023-11-27T15:24:52.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.812+0000 7fd9914aab80 1 -- start start 2023-11-27T15:24:52.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.813+0000 7fd9914aab80 1 Processor -- start 2023-11-27T15:24:52.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.813+0000 7fd9914aab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bf2c63b60 0x564bf2c70110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.813+0000 7fd9914aab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bf2c64470 0x564bf2c6d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.813+0000 7fd9914aab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564bf2c688b0 0x564bf2c6da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.813+0000 7fd9914aab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bf2b568e0 con 0x564bf2c63b60 2023-11-27T15:24:52.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.813+0000 7fd9914aab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bf2b56580 con 0x564bf2c688b0 2023-11-27T15:24:52.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.813+0000 7fd9914aab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bf2a38920 con 0x564bf2c64470 2023-11-27T15:24:52.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.813+0000 7fd97effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bf2c63b60 0x564bf2c70110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.813+0000 7fd9914aab80 10 client.0 _ll_register_callbacks cb 0x564bf2b74780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:52.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.813+0000 7fd97effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bf2c63b60 0x564bf2c70110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54862/0 (socket says 172.21.15.52:54862) 2023-11-27T15:24:52.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.813+0000 7fd97effd700 1 -- 172.21.15.52:0/1492478049 learned_addr learned my addr 172.21.15.52:0/1492478049 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:52.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.813+0000 7fd97e7fc700 1 --2- 172.21.15.52:0/1492478049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bf2c64470 0x564bf2c6d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.813+0000 7fd97effd700 1 -- 172.21.15.52:0/1492478049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bf2c64470 msgr2=0x564bf2c6d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:52.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.813+0000 7fd97effd700 1 --2- 172.21.15.52:0/1492478049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bf2c64470 0x564bf2c6d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.813+0000 7fd97f7fe700 1 --2- 172.21.15.52:0/1492478049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564bf2c688b0 0x564bf2c6da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.813+0000 7fd97effd700 1 -- 172.21.15.52:0/1492478049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564bf2c688b0 msgr2=0x564bf2c6da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:52.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.813+0000 7fd97effd700 1 --2- 172.21.15.52:0/1492478049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564bf2c688b0 0x564bf2c6da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.813+0000 7fd97effd700 1 -- 172.21.15.52:0/1492478049 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9801101e0 con 0x564bf2c63b60 2023-11-27T15:24:52.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.813+0000 7fd97e7fc700 1 --2- 172.21.15.52:0/1492478049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bf2c64470 0x564bf2c6d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:24:52.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.814+0000 7fd97effd700 1 --2- 172.21.15.52:0/1492478049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bf2c63b60 0x564bf2c70110 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fd97400b580 tx=0x7fd974009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a04f21bcd530ee8 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:52.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.814+0000 7fd962ff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:24:52.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.814+0000 7fd962ff5700 1 -- 172.21.15.52:0/1492478049 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97401d040 con 0x564bf2c63b60 2023-11-27T15:24:52.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.814+0000 7fd962ff5700 1 -- 172.21.15.52:0/1492478049 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fd974023820 con 0x564bf2c63b60 2023-11-27T15:24:52.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.814+0000 7fd9914aab80 1 -- 172.21.15.52:0/1492478049 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564bf2c56b20 con 0x564bf2c63b60 2023-11-27T15:24:52.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.814+0000 7fd962ff5700 1 -- 172.21.15.52:0/1492478049 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9740239e0 con 0x564bf2c63b60 2023-11-27T15:24:52.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.814+0000 7fd9914aab80 10 client.16605 fetch_fsmap learned FSMap version 28 2023-11-27T15:24:52.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.814+0000 7fd9914aab80 1 -- 172.21.15.52:0/1492478049 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564bf2b7b190 con 0x564bf2c63b60 2023-11-27T15:24:52.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.814+0000 7fd962ff5700 1 -- 172.21.15.52:0/1492478049 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd97403c020 con 0x564bf2c63b60 2023-11-27T15:24:52.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.814+0000 7fd962ff5700 1 -- 172.21.15.52:0/1492478049 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd97402e070 con 0x564bf2c63b60 2023-11-27T15:24:52.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.814+0000 7fd9914aab80 10 client.16605 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:24:52.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.814+0000 7fd9914aab80 10 client.16605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:52.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.814+0000 7fd9914aab80 20 client.16605 populate_metadata read hostname 'smithi052' 2023-11-27T15:24:52.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.814+0000 7fd9914aab80 1 --2- 172.21.15.52:0/1492478049 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x564bf2c7d900 0x564bf2c7fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.814+0000 7fd9914aab80 4 client.16605 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:24:52.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.814+0000 7fd9914aab80 1 -- 172.21.15.52:0/1492478049 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564bf2b770a0 con 0x564bf2c7d900 2023-11-27T15:24:52.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.815+0000 7fd97e7fc700 1 --2- 172.21.15.52:0/1492478049 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x564bf2c7d900 0x564bf2c7fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.815+0000 7fd962ff5700 10 client.16605 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:52.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.815+0000 7fd97e7fc700 1 --2- 172.21.15.52:0/1492478049 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x564bf2c7d900 0x564bf2c7fd10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91d268aad264238d server_cookie=5d31d8f5f26fff4d in_seq=0 out_seq=0 2023-11-27T15:24:52.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.815+0000 7fd962ff5700 1 -- 172.21.15.52:0/1492478049 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564bf2b770a0 con 0x564bf2c7d900 2023-11-27T15:24:52.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.816+0000 7fd962ff5700 10 client.16605 handle_command_reply: tid=0 2023-11-27T15:24:52.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.816+0000 7fd9914aab80 1 client.16605 shutdown 2023-11-27T15:24:52.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.816+0000 7fd9914aab80 1 -- 172.21.15.52:0/1492478049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bf2c63b60 msgr2=0x564bf2c70110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:52.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.816+0000 7fd9914aab80 1 --2- 172.21.15.52:0/1492478049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bf2c63b60 0x564bf2c70110 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fd97400b580 tx=0x7fd974009cd0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.816+0000 7fd9914aab80 1 -- 172.21.15.52:0/1492478049 shutdown_connections 2023-11-27T15:24:52.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.816+0000 7fd9914aab80 1 --2- 172.21.15.52:0/1492478049 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x564bf2c7d900 0x564bf2c7fd10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.816+0000 7fd9914aab80 1 --2- 172.21.15.52:0/1492478049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564bf2c688b0 0x564bf2c6da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.816+0000 7fd9914aab80 1 --2- 172.21.15.52:0/1492478049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bf2c63b60 0x564bf2c70110 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.816+0000 7fd9914aab80 1 --2- 172.21.15.52:0/1492478049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bf2c64470 0x564bf2c6d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.816+0000 7fd962ff5700 0 client.16605 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:52.832 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.816+0000 7fd9914aab80 1 -- 172.21.15.52:0/1492478049 >> 172.21.15.52:0/1492478049 conn(0x564bf2c59360 msgr2=0x564bf2c69d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:52.832 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.816+0000 7fd9914aab80 1 -- 172.21.15.52:0/1492478049 shutdown_connections 2023-11-27T15:24:52.832 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.816+0000 7fd9914aab80 1 -- 172.21.15.52:0/1492478049 wait complete. 2023-11-27T15:24:52.832 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.816+0000 7fd9914aab80 20 client.16605 trim_cache size 0 max 16384 2023-11-27T15:24:52.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.833+0000 7fd9914aab80 1 Processor -- start 2023-11-27T15:24:52.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.833+0000 7fd9914aab80 1 -- start start 2023-11-27T15:24:52.842 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.833+0000 7fd9914aab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bf2c63b60 0x564bf2c6fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.842 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.833+0000 7fd9914aab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bf2c64470 0x564bf2c5e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.842 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.833+0000 7fd9914aab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564bf2c688b0 0x564bf2c5ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.842 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.833+0000 7fd9914aab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bf2c58f30 con 0x564bf2c64470 2023-11-27T15:24:52.842 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.833+0000 7fd9914aab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bf2c5f8b0 con 0x564bf2c688b0 2023-11-27T15:24:52.842 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.833+0000 7fd9914aab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bf2c67440 con 0x564bf2c63b60 2023-11-27T15:24:52.842 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.833+0000 7fd97e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bf2c64470 0x564bf2c5e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.833+0000 7fd97e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bf2c64470 0x564bf2c5e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54872/0 (socket says 172.21.15.52:54872) 2023-11-27T15:24:52.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.833+0000 7fd97e7fc700 1 -- 172.21.15.52:0/1084876708 learned_addr learned my addr 172.21.15.52:0/1084876708 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:52.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.833+0000 7fd97effd700 1 --2- 172.21.15.52:0/1084876708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bf2c63b60 0x564bf2c6fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.833+0000 7fd97f7fe700 1 --2- 172.21.15.52:0/1084876708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564bf2c688b0 0x564bf2c5ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.834+0000 7fd97e7fc700 1 -- 172.21.15.52:0/1084876708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bf2c63b60 msgr2=0x564bf2c6fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:52.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.834+0000 7fd97e7fc700 1 --2- 172.21.15.52:0/1084876708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bf2c63b60 0x564bf2c6fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.834+0000 7fd97e7fc700 1 -- 172.21.15.52:0/1084876708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564bf2c688b0 msgr2=0x564bf2c5ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:52.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.834+0000 7fd97e7fc700 1 --2- 172.21.15.52:0/1084876708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564bf2c688b0 0x564bf2c5ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.834+0000 7fd97e7fc700 1 -- 172.21.15.52:0/1084876708 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564bf2b7aee0 con 0x564bf2c64470 2023-11-27T15:24:52.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.834+0000 7fd97f7fe700 1 --2- 172.21.15.52:0/1084876708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564bf2c688b0 0x564bf2c5ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:24:52.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.834+0000 7fd97e7fc700 1 --2- 172.21.15.52:0/1084876708 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bf2c64470 0x564bf2c5e730 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fd9700ca620 tx=0x7fd970010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38b374f438ec2e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:52.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.834+0000 7fd965ffb700 1 -- 172.21.15.52:0/1084876708 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9700a1e70 con 0x564bf2c64470 2023-11-27T15:24:52.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.834+0000 7fd965ffb700 1 -- 172.21.15.52:0/1084876708 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9700b1070 con 0x564bf2c64470 2023-11-27T15:24:52.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.834+0000 7fd965ffb700 1 -- 172.21.15.52:0/1084876708 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd970099170 con 0x564bf2c64470 2023-11-27T15:24:52.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.834+0000 7fd9914aab80 1 -- 172.21.15.52:0/1084876708 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bf2c64470 msgr2=0x564bf2c5e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:52.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.834+0000 7fd9914aab80 1 --2- 172.21.15.52:0/1084876708 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bf2c64470 0x564bf2c5e730 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fd9700ca620 tx=0x7fd970010c80 comp rx=0 tx=0).stop 2023-11-27T15:24:52.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.834+0000 7fd9914aab80 1 -- 172.21.15.52:0/1084876708 shutdown_connections 2023-11-27T15:24:52.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.834+0000 7fd9914aab80 1 --2- 172.21.15.52:0/1084876708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564bf2c688b0 0x564bf2c5ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.834+0000 7fd9914aab80 1 --2- 172.21.15.52:0/1084876708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bf2c63b60 0x564bf2c6fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.834+0000 7fd9914aab80 1 --2- 172.21.15.52:0/1084876708 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bf2c64470 0x564bf2c5e730 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.834+0000 7fd9914aab80 1 -- 172.21.15.52:0/1084876708 >> 172.21.15.52:0/1084876708 conn(0x564bf2c59360 msgr2=0x564bf2c66930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:52.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.835+0000 7fd9914aab80 1 -- 172.21.15.52:0/1084876708 shutdown_connections 2023-11-27T15:24:52.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.835+0000 7fd9914aab80 1 -- 172.21.15.52:0/1084876708 wait complete. 2023-11-27T15:24:52.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.835+0000 7fd9914aab80 1 -- start start 2023-11-27T15:24:52.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.835+0000 7fd9914aab80 1 Processor -- start 2023-11-27T15:24:52.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.835+0000 7fd9914aab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564bf2c63b60 0x564bf2c72990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.835+0000 7fd9914aab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bf2c64470 0x564bf2c72ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.835+0000 7fd9914aab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bf2c688b0 0x564bf2cd07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.835+0000 7fd9914aab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bf2c73810 con 0x564bf2c688b0 2023-11-27T15:24:52.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.835+0000 7fd9914aab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bf2cd0d20 con 0x564bf2c63b60 2023-11-27T15:24:52.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.835+0000 7fd9914aab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bf2cd0e60 con 0x564bf2c64470 2023-11-27T15:24:52.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.835+0000 7fd9914aab80 10 client.0 _ll_register_callbacks cb 0x7fd97400bf90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:52.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.835+0000 7fd97f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bf2c688b0 0x564bf2cd07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.835+0000 7fd97f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bf2c688b0 0x564bf2cd07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54880/0 (socket says 172.21.15.52:54880) 2023-11-27T15:24:52.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.835+0000 7fd97f7fe700 1 -- 172.21.15.52:0/2482807101 learned_addr learned my addr 172.21.15.52:0/2482807101 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:24:52.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.835+0000 7fd97effd700 1 --2- 172.21.15.52:0/2482807101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564bf2c63b60 0x564bf2c72990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.835+0000 7fd97e7fc700 1 --2- 172.21.15.52:0/2482807101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bf2c64470 0x564bf2c72ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.836+0000 7fd97f7fe700 1 -- 172.21.15.52:0/2482807101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bf2c64470 msgr2=0x564bf2c72ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:52.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.836+0000 7fd97f7fe700 1 --2- 172.21.15.52:0/2482807101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bf2c64470 0x564bf2c72ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.848 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.836+0000 7fd97f7fe700 1 -- 172.21.15.52:0/2482807101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564bf2c63b60 msgr2=0x564bf2c72990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:52.848 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.836+0000 7fd97f7fe700 1 --2- 172.21.15.52:0/2482807101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564bf2c63b60 0x564bf2c72990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.848 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.836+0000 7fd97f7fe700 1 -- 172.21.15.52:0/2482807101 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564bf2a36630 con 0x564bf2c688b0 2023-11-27T15:24:52.848 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.836+0000 7fd97e7fc700 1 --2- 172.21.15.52:0/2482807101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bf2c64470 0x564bf2c72ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:24:52.848 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.836+0000 7fd97f7fe700 1 --2- 172.21.15.52:0/2482807101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bf2c688b0 0x564bf2cd07e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd978007ab0 tx=0x7fd978007e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103a0f61e6a7eaa0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:52.848 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.836+0000 7fd962ff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:24:52.848 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.836+0000 7fd962ff5700 1 -- 172.21.15.52:0/2482807101 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97800e040 con 0x564bf2c688b0 2023-11-27T15:24:52.848 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.836+0000 7fd962ff5700 1 -- 172.21.15.52:0/2482807101 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fd978004db0 con 0x564bf2c688b0 2023-11-27T15:24:52.849 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.836+0000 7fd9914aab80 1 -- 172.21.15.52:0/2482807101 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd970097dc0 con 0x564bf2c688b0 2023-11-27T15:24:52.849 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.836+0000 7fd962ff5700 1 -- 172.21.15.52:0/2482807101 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd978012980 con 0x564bf2c688b0 2023-11-27T15:24:52.849 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.836+0000 7fd9914aab80 10 client.16611 fetch_fsmap learned FSMap version 28 2023-11-27T15:24:52.849 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.836+0000 7fd9914aab80 1 -- 172.21.15.52:0/2482807101 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564bf29cbc60 con 0x564bf2c688b0 2023-11-27T15:24:52.849 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.836+0000 7fd962ff5700 1 -- 172.21.15.52:0/2482807101 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd970097dc0 con 0x564bf2c688b0 2023-11-27T15:24:52.849 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.836+0000 7fd962ff5700 1 -- 172.21.15.52:0/2482807101 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd978014040 con 0x564bf2c688b0 2023-11-27T15:24:52.849 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.836+0000 7fd9914aab80 10 client.16611 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:24:52.849 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.836+0000 7fd9914aab80 10 client.16611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:52.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.836+0000 7fd9914aab80 20 client.16611 populate_metadata read hostname 'smithi052' 2023-11-27T15:24:52.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.836+0000 7fd9914aab80 1 --2- 172.21.15.52:0/2482807101 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x564bf2cd2550 0x564bf2cd4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:52.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.836+0000 7fd9914aab80 4 client.16611 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:24:52.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.836+0000 7fd9914aab80 1 -- 172.21.15.52:0/2482807101 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564bf2b83660 con 0x564bf2cd2550 2023-11-27T15:24:52.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.837+0000 7fd97effd700 1 --2- 172.21.15.52:0/2482807101 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x564bf2cd2550 0x564bf2cd4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:52.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.837+0000 7fd962ff5700 10 client.16611 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:52.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.837+0000 7fd97effd700 1 --2- 172.21.15.52:0/2482807101 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x564bf2cd2550 0x564bf2cd4960 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa0bd76736a8b463 server_cookie=dcbaa808d1f4a081 in_seq=0 out_seq=0 2023-11-27T15:24:52.850 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.837+0000 7fd962ff5700 1 -- 172.21.15.52:0/2482807101 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564bf2b83660 con 0x564bf2cd2550 2023-11-27T15:24:52.851 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.837+0000 7fd962ff5700 10 client.16611 handle_command_reply: tid=0 2023-11-27T15:24:52.851 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.837+0000 7fd9914aab80 1 client.16611 shutdown 2023-11-27T15:24:52.851 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.837+0000 7fd9914aab80 1 -- 172.21.15.52:0/2482807101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bf2c688b0 msgr2=0x564bf2cd07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:52.851 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.837+0000 7fd9914aab80 1 --2- 172.21.15.52:0/2482807101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bf2c688b0 0x564bf2cd07e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd978007ab0 tx=0x7fd978007e70 comp rx=0 tx=0).stop 2023-11-27T15:24:52.851 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.837+0000 7fd9914aab80 1 -- 172.21.15.52:0/2482807101 shutdown_connections 2023-11-27T15:24:52.851 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.837+0000 7fd9914aab80 1 --2- 172.21.15.52:0/2482807101 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x564bf2cd2550 0x564bf2cd4960 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.851 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.837+0000 7fd9914aab80 1 --2- 172.21.15.52:0/2482807101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564bf2c63b60 0x564bf2c72990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.851 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.837+0000 7fd962ff5700 0 client.16611 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:24:52.852 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.837+0000 7fd9914aab80 1 --2- 172.21.15.52:0/2482807101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564bf2c688b0 0x564bf2cd07e0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.852 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.838+0000 7fd9914aab80 1 --2- 172.21.15.52:0/2482807101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564bf2c64470 0x564bf2c72ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.852 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.838+0000 7fd9914aab80 1 -- 172.21.15.52:0/2482807101 >> 172.21.15.52:0/2482807101 conn(0x564bf2c59360 msgr2=0x564bf2c6cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:52.852 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.838+0000 7fd9914aab80 1 -- 172.21.15.52:0/2482807101 shutdown_connections 2023-11-27T15:24:52.852 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.838+0000 7fd9914aab80 1 -- 172.21.15.52:0/2482807101 wait complete. 2023-11-27T15:24:52.852 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.838+0000 7fd9914aab80 20 client.16611 trim_cache size 0 max 16384 2023-11-27T15:24:52.853 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:24:52.853 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-27T15:24:52.853 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2023-11-27T15:24:52.853 INFO:teuthology.orchestra.run.smithi052.stdout: "2155fb61-dca4-4c9a-9b19-32b5f67c7159": { 2023-11-27T15:24:52.853 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2023-11-27T15:24:52.853 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "2155fb61-dca4-4c9a-9b19-32b5f67c7159", 2023-11-27T15:24:52.853 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2023-11-27T15:24:52.854 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:24:52.854 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:24:52.854 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:24:52.855 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.838+0000 7fd98531e700 1 -- 172.21.15.52:0/1337860360 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd9680a12c0 msgr2=0x7fd9680a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:52.855 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.838+0000 7fd98531e700 1 --2- 172.21.15.52:0/1337860360 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd9680a12c0 0x7fd9680a3770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd974009fd0 tx=0x7fd97401a040 comp rx=0 tx=0).stop 2023-11-27T15:24:52.855 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.838+0000 7fd98531e700 1 -- 172.21.15.52:0/1337860360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd980100f10 msgr2=0x7fd9801969a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:52.855 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.838+0000 7fd98531e700 1 --2- 172.21.15.52:0/1337860360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd980100f10 0x7fd9801969a0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd97000d950 tx=0x7fd97000dd10 comp rx=0 tx=0).stop 2023-11-27T15:24:52.856 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.838+0000 7fd98531e700 1 -- 172.21.15.52:0/1337860360 shutdown_connections 2023-11-27T15:24:52.856 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.838+0000 7fd98531e700 1 --2- 172.21.15.52:0/1337860360 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd9680a12c0 0x7fd9680a3770 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.856 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.838+0000 7fd98531e700 1 --2- 172.21.15.52:0/1337860360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd980100570 0x7fd980196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.856 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.838+0000 7fd98531e700 1 --2- 172.21.15.52:0/1337860360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9801055b0 0x7fd98019ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.856 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.838+0000 7fd98531e700 1 --2- 172.21.15.52:0/1337860360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd980100f10 0x7fd9801969a0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:52.856 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.838+0000 7fd98531e700 1 -- 172.21.15.52:0/1337860360 >> 172.21.15.52:0/1337860360 conn(0x7fd9800770e0 msgr2=0x7fd9800fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:52.856 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.838+0000 7fd98531e700 1 -- 172.21.15.52:0/1337860360 shutdown_connections 2023-11-27T15:24:52.856 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:24:52.838+0000 7fd98531e700 1 -- 172.21.15.52:0/1337860360 wait complete. 2023-11-27T15:24:52.857 INFO:tasks.cephfs.filesystem:scrub status for tag:2155fb61-dca4-4c9a-9b19-32b5f67c7159 - {'path': '/', 'tag': '2155fb61-dca4-4c9a-9b19-32b5f67c7159', 'options': 'recursive,force'} 2023-11-27T15:24:53.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:53 smithi052 ceph-mon[99056]: scrub queued for path: / 2023-11-27T15:24:53.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:53 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:24:53.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:53 smithi052 ceph-mon[99056]: scrub summary: active paths [/] 2023-11-27T15:24:53.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:53 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:24:53.277 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:53 smithi150 ceph-mon[100559]: scrub queued for path: / 2023-11-27T15:24:53.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:53 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:24:53.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:53 smithi150 ceph-mon[100559]: scrub summary: active paths [/] 2023-11-27T15:24:53.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:53 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:24:53.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:53 smithi102 ceph-mon[100296]: scrub queued for path: / 2023-11-27T15:24:53.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:53 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:24:53.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:53 smithi102 ceph-mon[100296]: scrub summary: active paths [/] 2023-11-27T15:24:53.328 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:53 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:24:54.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:54 smithi150 ceph-mon[100559]: pgmap v214: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:24:54.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:54 smithi102 ceph-mon[100296]: pgmap v214: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:24:54.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:54 smithi052 ceph-mon[99056]: pgmap v214: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:24:55.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:55 smithi150 ceph-mon[100559]: scrub summary: idle 2023-11-27T15:24:55.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:55 smithi102 ceph-mon[100296]: scrub summary: idle 2023-11-27T15:24:55.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:55 smithi052 ceph-mon[99056]: scrub summary: idle 2023-11-27T15:24:56.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:56 smithi150 ceph-mon[100559]: pgmap v215: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:24:56.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:56 smithi102 ceph-mon[100296]: pgmap v215: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:24:56.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:56 smithi052 ceph-mon[99056]: pgmap v215: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:24:58.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:58 smithi150 ceph-mon[100559]: pgmap v216: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:24:58.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:58 smithi102 ceph-mon[100296]: pgmap v216: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:24:58.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:58 smithi052 ceph-mon[99056]: pgmap v216: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:25:00.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:24:59 smithi150 ceph-mon[100559]: pgmap v217: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:25:00.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:24:59 smithi102 ceph-mon[100296]: pgmap v217: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:25:00.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:24:59 smithi052 ceph-mon[99056]: pgmap v217: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:25:02.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:02 smithi150 ceph-mon[100559]: pgmap v218: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-27T15:25:02.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:02 smithi102 ceph-mon[100296]: pgmap v218: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-27T15:25:02.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:02 smithi052 ceph-mon[99056]: pgmap v218: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-27T15:25:04.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:04 smithi150 ceph-mon[100559]: pgmap v219: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-27T15:25:04.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:04 smithi102 ceph-mon[100296]: pgmap v219: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-27T15:25:04.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:04 smithi052 ceph-mon[99056]: pgmap v219: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-27T15:25:06.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:06 smithi150 ceph-mon[100559]: pgmap v220: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-27T15:25:06.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:06 smithi150 ceph-mon[100559]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:25:06.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:06 smithi102 ceph-mon[100296]: pgmap v220: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-27T15:25:06.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:06 smithi102 ceph-mon[100296]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:25:06.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:06 smithi052 ceph-mon[99056]: pgmap v220: 97 pgs: 97 active+clean; 712 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-27T15:25:06.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:06 smithi052 ceph-mon[99056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:25:08.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:08 smithi150 ceph-mon[100559]: pgmap v221: 97 pgs: 97 active+clean; 712 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-27T15:25:08.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:08 smithi102 ceph-mon[100296]: pgmap v221: 97 pgs: 97 active+clean; 712 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-27T15:25:08.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:08 smithi052 ceph-mon[99056]: pgmap v221: 97 pgs: 97 active+clean; 712 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-27T15:25:10.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:09 smithi150 ceph-mon[100559]: pgmap v222: 97 pgs: 97 active+clean; 712 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-27T15:25:10.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:09 smithi102 ceph-mon[100296]: pgmap v222: 97 pgs: 97 active+clean; 712 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-27T15:25:10.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:09 smithi052 ceph-mon[99056]: pgmap v222: 97 pgs: 97 active+clean; 712 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-27T15:25:11.277 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:10 smithi150 ceph-mon[100559]: osdmap e89: 12 total, 12 up, 12 in 2023-11-27T15:25:11.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:10 smithi102 ceph-mon[100296]: osdmap e89: 12 total, 12 up, 12 in 2023-11-27T15:25:11.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:10 smithi052 ceph-mon[99056]: osdmap e89: 12 total, 12 up, 12 in 2023-11-27T15:25:12.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:11 smithi150 ceph-mon[100559]: pgmap v224: 97 pgs: 97 active+clean; 712 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:25:12.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:11 smithi102 ceph-mon[100296]: pgmap v224: 97 pgs: 97 active+clean; 712 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:25:12.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:11 smithi052 ceph-mon[99056]: pgmap v224: 97 pgs: 97 active+clean; 712 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:25:14.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:14 smithi150 ceph-mon[100559]: pgmap v225: 97 pgs: 97 active+clean; 712 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:25:14.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:14 smithi150 ceph-mon[100559]: osdmap e90: 12 total, 12 up, 12 in 2023-11-27T15:25:14.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:14 smithi102 ceph-mon[100296]: pgmap v225: 97 pgs: 97 active+clean; 712 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:25:14.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:14 smithi102 ceph-mon[100296]: osdmap e90: 12 total, 12 up, 12 in 2023-11-27T15:25:14.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:14 smithi052 ceph-mon[99056]: pgmap v225: 97 pgs: 97 active+clean; 712 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:25:14.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:14 smithi052 ceph-mon[99056]: osdmap e90: 12 total, 12 up, 12 in 2023-11-27T15:25:15.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:15 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:25:15.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:15 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:25:15.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:15 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:25:16.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:16 smithi150 ceph-mon[100559]: pgmap v227: 97 pgs: 97 active+clean; 712 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2023-11-27T15:25:16.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:16 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:25:16.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:16 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:25:16.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:16 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:25:16.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:16 smithi102 ceph-mon[100296]: pgmap v227: 97 pgs: 97 active+clean; 712 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2023-11-27T15:25:16.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:16 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:25:16.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:16 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:25:16.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:16 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:25:16.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:16 smithi052 ceph-mon[99056]: pgmap v227: 97 pgs: 97 active+clean; 712 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2023-11-27T15:25:16.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:16 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:25:16.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:16 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:25:16.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:16 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:25:17.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:17 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:25:17.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:17 smithi150 ceph-mon[100559]: Service mds.all has no related daemon already placed. 2023-11-27T15:25:17.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:17 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:25:17.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:17 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:25:17.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:17 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:25:17.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:17 smithi102 ceph-mon[100296]: Service mds.all has no related daemon already placed. 2023-11-27T15:25:17.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:17 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:25:17.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:17 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:25:17.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:17 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:25:17.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:17 smithi052 ceph-mon[99056]: Service mds.all has no related daemon already placed. 2023-11-27T15:25:17.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:17 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:25:17.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:17 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:25:18.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:18 smithi150 ceph-mon[100559]: pgmap v228: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-27T15:25:18.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:18 smithi102 ceph-mon[100296]: pgmap v228: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-27T15:25:18.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:18 smithi052 ceph-mon[99056]: pgmap v228: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-27T15:25:20.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:19 smithi150 ceph-mon[100559]: pgmap v229: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 221 B/s wr, 6 op/s 2023-11-27T15:25:20.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:19 smithi102 ceph-mon[100296]: pgmap v229: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 221 B/s wr, 6 op/s 2023-11-27T15:25:20.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:19 smithi052 ceph-mon[99056]: pgmap v229: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 221 B/s wr, 6 op/s 2023-11-27T15:25:22.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:21 smithi150 ceph-mon[100559]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:25:22.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:21 smithi150 ceph-mon[100559]: pgmap v230: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-11-27T15:25:22.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:21 smithi102 ceph-mon[100296]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:25:22.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:21 smithi102 ceph-mon[100296]: pgmap v230: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-11-27T15:25:22.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:21 smithi052 ceph-mon[99056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:25:22.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:21 smithi052 ceph-mon[99056]: pgmap v230: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-11-27T15:25:22.858 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:25:23.003 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:22.999+0000 7fe7bc810700 1 -- 172.21.15.52:0/4046094683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7b8101600 msgr2=0x7fe7b81019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.003 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:22.999+0000 7fe7bc810700 1 --2- 172.21.15.52:0/4046094683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7b8101600 0x7fe7b81019d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009990 tx=0x7fe7a8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.003 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:22.999+0000 7fe7bc810700 1 -- 172.21.15.52:0/4046094683 shutdown_connections 2023-11-27T15:25:23.004 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:22.999+0000 7fe7bc810700 1 --2- 172.21.15.52:0/4046094683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7b810a940 0x7fe7b810cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.004 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:22.999+0000 7fe7bc810700 1 --2- 172.21.15.52:0/4046094683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7b8101600 0x7fe7b81019d0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.004 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:22.999+0000 7fe7bc810700 1 --2- 172.21.15.52:0/4046094683 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7b8101f10 0x7fe7b810a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.004 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:22.999+0000 7fe7bc810700 1 -- 172.21.15.52:0/4046094683 >> 172.21.15.52:0/4046094683 conn(0x7fe7b80760d0 msgr2=0x7fe7b80764d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:23.004 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:22.999+0000 7fe7bc810700 1 -- 172.21.15.52:0/4046094683 shutdown_connections 2023-11-27T15:25:23.004 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:22.999+0000 7fe7bc810700 1 -- 172.21.15.52:0/4046094683 wait complete. 2023-11-27T15:25:23.008 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.000+0000 7fe7bc810700 1 Processor -- start 2023-11-27T15:25:23.008 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.000+0000 7fe7bc810700 1 -- start start 2023-11-27T15:25:23.008 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.000+0000 7fe7bc810700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7b8101600 0x7fe7b81963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.009 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.000+0000 7fe7bc810700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7b8101f10 0x7fe7b81968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.009 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.000+0000 7fe7bc810700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7b810a940 0x7fe7b819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.009 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.000+0000 7fe7bc810700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b819b390 con 0x7fe7b8101f10 2023-11-27T15:25:23.009 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.000+0000 7fe7bc810700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b819b500 con 0x7fe7b810a940 2023-11-27T15:25:23.009 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.000+0000 7fe7bc810700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b819b6c0 con 0x7fe7b8101600 2023-11-27T15:25:23.009 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.000+0000 7fe7b5d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7b8101f10 0x7fe7b81968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.000+0000 7fe7b5d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7b8101f10 0x7fe7b81968f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54892/0 (socket says 172.21.15.52:54892) 2023-11-27T15:25:23.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.000+0000 7fe7b5d9b700 1 -- 172.21.15.52:0/3344500798 learned_addr learned my addr 172.21.15.52:0/3344500798 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:23.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.000+0000 7fe7b659c700 1 --2- 172.21.15.52:0/3344500798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7b8101600 0x7fe7b81963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.000+0000 7fe7b6d9d700 1 --2- 172.21.15.52:0/3344500798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7b810a940 0x7fe7b819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.000+0000 7fe7b5d9b700 1 -- 172.21.15.52:0/3344500798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7b8101600 msgr2=0x7fe7b81963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.001+0000 7fe7b5d9b700 1 --2- 172.21.15.52:0/3344500798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7b8101600 0x7fe7b81963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.001+0000 7fe7b5d9b700 1 -- 172.21.15.52:0/3344500798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7b810a940 msgr2=0x7fe7b819ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.001+0000 7fe7b5d9b700 1 --2- 172.21.15.52:0/3344500798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7b810a940 0x7fe7b819ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.001+0000 7fe7b5d9b700 1 -- 172.21.15.52:0/3344500798 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a80095f0 con 0x7fe7b8101f10 2023-11-27T15:25:23.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.001+0000 7fe7b659c700 1 --2- 172.21.15.52:0/3344500798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7b8101600 0x7fe7b81963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:23.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.001+0000 7fe7b5d9b700 1 --2- 172.21.15.52:0/3344500798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7b8101f10 0x7fe7b81968f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00e9a0 tx=0x7fe7ac00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:23.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.001+0000 7fe7a77fe700 1 -- 172.21.15.52:0/3344500798 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac009790 con 0x7fe7b8101f10 2023-11-27T15:25:23.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.001+0000 7fe7a77fe700 1 -- 172.21.15.52:0/3344500798 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7ac00cbd0 con 0x7fe7b8101f10 2023-11-27T15:25:23.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.001+0000 7fe7bc810700 1 -- 172.21.15.52:0/3344500798 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7b81103d0 con 0x7fe7b8101f10 2023-11-27T15:25:23.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.001+0000 7fe7a77fe700 1 -- 172.21.15.52:0/3344500798 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac018680 con 0x7fe7b8101f10 2023-11-27T15:25:23.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.001+0000 7fe7bc810700 1 -- 172.21.15.52:0/3344500798 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b8110240 con 0x7fe7b8101f10 2023-11-27T15:25:23.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.003+0000 7fe7a77fe700 1 -- 172.21.15.52:0/3344500798 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe7ac00cd40 con 0x7fe7b8101f10 2023-11-27T15:25:23.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.004+0000 7fe7a77fe700 1 --2- 172.21.15.52:0/3344500798 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe7a00a12c0 0x7fe7a00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.004+0000 7fe7a77fe700 1 -- 172.21.15.52:0/3344500798 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fe7ac014070 con 0x7fe7b8101f10 2023-11-27T15:25:23.013 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.004+0000 7fe7b659c700 1 --2- 172.21.15.52:0/3344500798 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe7a00a12c0 0x7fe7a00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.013 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.004+0000 7fe7b659c700 1 --2- 172.21.15.52:0/3344500798 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe7a00a12c0 0x7fe7a00a3770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8006010 tx=0x7fe7a800b260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:23.013 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.005+0000 7fe7c899cb80 1 Processor -- start 2023-11-27T15:25:23.019 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.005+0000 7fe7c899cb80 1 -- start start 2023-11-27T15:25:23.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.005+0000 7fe7c899cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d0a9b60 0x55988d0a9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.005+0000 7fe7c899cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988d0aa470 0x55988d0ae370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.005+0000 7fe7c899cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55988d0ae8b0 0x55988d0aed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.005+0000 7fe7c899cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988cfaf060 con 0x55988d0aa470 2023-11-27T15:25:23.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.005+0000 7fe7c899cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988d0018a0 con 0x55988d0ae8b0 2023-11-27T15:25:23.020 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.005+0000 7fe7c899cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988cffad30 con 0x55988d0a9b60 2023-11-27T15:25:23.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.005+0000 7fe7b5d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988d0aa470 0x55988d0ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.005+0000 7fe7b5d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988d0aa470 0x55988d0ae370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54900/0 (socket says 172.21.15.52:54900) 2023-11-27T15:25:23.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.005+0000 7fe7b5d9b700 1 -- 172.21.15.52:0/899383827 learned_addr learned my addr 172.21.15.52:0/899383827 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:23.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.006+0000 7fe7b6d9d700 1 --2- 172.21.15.52:0/899383827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55988d0ae8b0 0x55988d0aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.006+0000 7fe7b659c700 1 --2- 172.21.15.52:0/899383827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d0a9b60 0x55988d0a9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.006+0000 7fe7b5d9b700 1 -- 172.21.15.52:0/899383827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d0a9b60 msgr2=0x55988d0a9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.021 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.006+0000 7fe7b5d9b700 1 --2- 172.21.15.52:0/899383827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d0a9b60 0x55988d0a9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.006+0000 7fe7b5d9b700 1 -- 172.21.15.52:0/899383827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55988d0ae8b0 msgr2=0x55988d0aed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.006+0000 7fe7b5d9b700 1 --2- 172.21.15.52:0/899383827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55988d0ae8b0 0x55988d0aed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.006+0000 7fe7b5d9b700 1 -- 172.21.15.52:0/899383827 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55988ce7c630 con 0x55988d0aa470 2023-11-27T15:25:23.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.006+0000 7fe7b6d9d700 1 --2- 172.21.15.52:0/899383827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55988d0ae8b0 0x55988d0aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:23.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.006+0000 7fe7b5d9b700 1 --2- 172.21.15.52:0/899383827 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988d0aa470 0x55988d0ae370 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00c5b0 tx=0x7fe7ac099ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769c1fc1ad6335fd server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:23.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.006+0000 7fe7a4ff9700 1 -- 172.21.15.52:0/899383827 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac0a2530 con 0x55988d0aa470 2023-11-27T15:25:23.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.006+0000 7fe7a4ff9700 1 -- 172.21.15.52:0/899383827 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7ac09a340 con 0x55988d0aa470 2023-11-27T15:25:23.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.006+0000 7fe7a4ff9700 1 -- 172.21.15.52:0/899383827 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac0a22c0 con 0x55988d0aa470 2023-11-27T15:25:23.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.007+0000 7fe7c899cb80 1 -- 172.21.15.52:0/899383827 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988d0aa470 msgr2=0x55988d0ae370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.007+0000 7fe7c899cb80 1 --2- 172.21.15.52:0/899383827 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988d0aa470 0x55988d0ae370 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00c5b0 tx=0x7fe7ac099ac0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.007+0000 7fe7c899cb80 1 -- 172.21.15.52:0/899383827 shutdown_connections 2023-11-27T15:25:23.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.007+0000 7fe7c899cb80 1 --2- 172.21.15.52:0/899383827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55988d0ae8b0 0x55988d0aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.007+0000 7fe7c899cb80 1 --2- 172.21.15.52:0/899383827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d0a9b60 0x55988d0a9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.007+0000 7fe7c899cb80 1 --2- 172.21.15.52:0/899383827 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988d0aa470 0x55988d0ae370 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.007+0000 7fe7c899cb80 1 -- 172.21.15.52:0/899383827 >> 172.21.15.52:0/899383827 conn(0x55988d09f360 msgr2=0x55988d0a4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:23.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.007+0000 7fe7c899cb80 1 -- 172.21.15.52:0/899383827 shutdown_connections 2023-11-27T15:25:23.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.007+0000 7fe7c899cb80 1 -- 172.21.15.52:0/899383827 wait complete. 2023-11-27T15:25:23.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.007+0000 7fe7c899cb80 1 -- start start 2023-11-27T15:25:23.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.007+0000 7fe7c899cb80 1 Processor -- start 2023-11-27T15:25:23.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.008+0000 7fe7c899cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55988d0a9b60 0x55988d0b6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.008+0000 7fe7c899cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988d0aa470 0x55988d0b3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.008+0000 7fe7c899cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d0ae8b0 0x55988d0b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.008+0000 7fe7c899cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988cf9c8e0 con 0x55988d0aa470 2023-11-27T15:25:23.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.008+0000 7fe7c899cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988cf9c580 con 0x55988d0a9b60 2023-11-27T15:25:23.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.008+0000 7fe7c899cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988ce7e920 con 0x55988d0ae8b0 2023-11-27T15:25:23.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.008+0000 7fe7c899cb80 10 client.0 _ll_register_callbacks cb 0x55988cfba780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:23.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.008+0000 7fe7b5d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988d0aa470 0x55988d0b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.008+0000 7fe7b5d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988d0aa470 0x55988d0b3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54906/0 (socket says 172.21.15.52:54906) 2023-11-27T15:25:23.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.008+0000 7fe7b5d9b700 1 -- 172.21.15.52:0/3434492973 learned_addr learned my addr 172.21.15.52:0/3434492973 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:23.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.008+0000 7fe7b659c700 1 --2- 172.21.15.52:0/3434492973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55988d0a9b60 0x55988d0b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.008+0000 7fe7b6d9d700 1 --2- 172.21.15.52:0/3434492973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d0ae8b0 0x55988d0b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.008+0000 7fe7b5d9b700 1 -- 172.21.15.52:0/3434492973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d0ae8b0 msgr2=0x55988d0b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.008+0000 7fe7b5d9b700 1 --2- 172.21.15.52:0/3434492973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d0ae8b0 0x55988d0b3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.008+0000 7fe7b5d9b700 1 -- 172.21.15.52:0/3434492973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55988d0a9b60 msgr2=0x55988d0b6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.008+0000 7fe7b5d9b700 1 --2- 172.21.15.52:0/3434492973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55988d0a9b60 0x55988d0b6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.008+0000 7fe7b5d9b700 1 -- 172.21.15.52:0/3434492973 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b8110240 con 0x55988d0aa470 2023-11-27T15:25:23.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.008+0000 7fe7b5d9b700 1 --2- 172.21.15.52:0/3434492973 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988d0aa470 0x55988d0b3320 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac010fd0 tx=0x7fe7ac0a25e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b399a77e8b22eb7b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:23.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.008+0000 7fe79dffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:25:23.028 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.008+0000 7fe79dffb700 1 -- 172.21.15.52:0/3434492973 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac098500 con 0x55988d0aa470 2023-11-27T15:25:23.028 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.008+0000 7fe79dffb700 1 -- 172.21.15.52:0/3434492973 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fe7ac0a22b0 con 0x55988d0aa470 2023-11-27T15:25:23.028 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.008+0000 7fe7c899cb80 1 -- 172.21.15.52:0/3434492973 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55988d09cb20 con 0x55988d0aa470 2023-11-27T15:25:23.028 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.009+0000 7fe79dffb700 1 -- 172.21.15.52:0/3434492973 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7ac0ca360 con 0x55988d0aa470 2023-11-27T15:25:23.028 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.009+0000 7fe7c899cb80 10 client.16623 fetch_fsmap learned FSMap version 30 2023-11-27T15:25:23.028 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.009+0000 7fe7c899cb80 1 -- 172.21.15.52:0/3434492973 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55988cfc1190 con 0x55988d0aa470 2023-11-27T15:25:23.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.009+0000 7fe79dffb700 1 -- 172.21.15.52:0/3434492973 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55988d09cb20 con 0x55988d0aa470 2023-11-27T15:25:23.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.009+0000 7fe79dffb700 1 -- 172.21.15.52:0/3434492973 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe7ac099040 con 0x55988d0aa470 2023-11-27T15:25:23.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.009+0000 7fe7c899cb80 10 client.16623 fetch_fsmap finished waiting for FSMap version 30 2023-11-27T15:25:23.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.009+0000 7fe7c899cb80 10 client.16623 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:23.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.009+0000 7fe7c899cb80 20 client.16623 populate_metadata read hostname 'smithi052' 2023-11-27T15:25:23.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.009+0000 7fe7c899cb80 1 --2- 172.21.15.52:0/3434492973 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55988d0c3580 0x55988d0c5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.009+0000 7fe7c899cb80 4 client.16623 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:23.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.009+0000 7fe7c899cb80 1 -- 172.21.15.52:0/3434492973 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55988cfbd0a0 con 0x55988d0c3580 2023-11-27T15:25:23.030 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.009+0000 7fe7b659c700 1 --2- 172.21.15.52:0/3434492973 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55988d0c3580 0x55988d0c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.030 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.010+0000 7fe79dffb700 10 client.16623 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:23.030 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.010+0000 7fe7b659c700 1 --2- 172.21.15.52:0/3434492973 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55988d0c3580 0x55988d0c5990 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=559c557dbef18ed6 server_cookie=4bec96d48393ab67 in_seq=0 out_seq=0 2023-11-27T15:25:23.030 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.010+0000 7fe79dffb700 1 -- 172.21.15.52:0/3434492973 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55988cfbd0a0 con 0x55988d0c3580 2023-11-27T15:25:23.030 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.010+0000 7fe79dffb700 10 client.16623 handle_command_reply: tid=0 2023-11-27T15:25:23.030 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.010+0000 7fe7c899cb80 1 client.16623 shutdown 2023-11-27T15:25:23.031 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.010+0000 7fe7c899cb80 1 -- 172.21.15.52:0/3434492973 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988d0aa470 msgr2=0x55988d0b3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.031 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.010+0000 7fe7c899cb80 1 --2- 172.21.15.52:0/3434492973 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988d0aa470 0x55988d0b3320 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac010fd0 tx=0x7fe7ac0a25e0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.031 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.011+0000 7fe7c899cb80 1 -- 172.21.15.52:0/3434492973 shutdown_connections 2023-11-27T15:25:23.031 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.011+0000 7fe7c899cb80 1 --2- 172.21.15.52:0/3434492973 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55988d0c3580 0x55988d0c5990 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.031 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.011+0000 7fe7c899cb80 1 --2- 172.21.15.52:0/3434492973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55988d0a9b60 0x55988d0b6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.032 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.011+0000 7fe7c899cb80 1 --2- 172.21.15.52:0/3434492973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d0ae8b0 0x55988d0b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.032 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.011+0000 7fe79dffb700 0 client.16623 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:23.032 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.011+0000 7fe7c899cb80 1 --2- 172.21.15.52:0/3434492973 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988d0aa470 0x55988d0b3320 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.032 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.011+0000 7fe7c899cb80 1 -- 172.21.15.52:0/3434492973 >> 172.21.15.52:0/3434492973 conn(0x55988d09f360 msgr2=0x55988d0afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:23.032 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.011+0000 7fe7c899cb80 1 -- 172.21.15.52:0/3434492973 shutdown_connections 2023-11-27T15:25:23.032 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.011+0000 7fe7c899cb80 1 -- 172.21.15.52:0/3434492973 wait complete. 2023-11-27T15:25:23.033 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.011+0000 7fe7c899cb80 20 client.16623 trim_cache size 0 max 16384 2023-11-27T15:25:23.039 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.027+0000 7fe7c899cb80 1 Processor -- start 2023-11-27T15:25:23.039 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.028+0000 7fe7c899cb80 1 -- start start 2023-11-27T15:25:23.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.028+0000 7fe7c899cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988d0a9b60 0x55988d0b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.028+0000 7fe7c899cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55988d0aa470 0x55988d0b6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.028+0000 7fe7c899cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d0ae8b0 0x55988d0ac930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.028+0000 7fe7c899cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988d09ef50 con 0x55988d0a9b60 2023-11-27T15:25:23.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.028+0000 7fe7c899cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988cfbd0d0 con 0x55988d0aa470 2023-11-27T15:25:23.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.028+0000 7fe7c899cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988d0aabc0 con 0x55988d0ae8b0 2023-11-27T15:25:23.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.028+0000 7fe7b659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988d0a9b60 0x55988d0b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.028+0000 7fe7b659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988d0a9b60 0x55988d0b5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54912/0 (socket says 172.21.15.52:54912) 2023-11-27T15:25:23.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.028+0000 7fe7b659c700 1 -- 172.21.15.52:0/3998116439 learned_addr learned my addr 172.21.15.52:0/3998116439 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:23.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.028+0000 7fe7b6d9d700 1 --2- 172.21.15.52:0/3998116439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d0ae8b0 0x55988d0ac930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.028+0000 7fe7b5d9b700 1 --2- 172.21.15.52:0/3998116439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55988d0aa470 0x55988d0b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.028+0000 7fe7b659c700 1 -- 172.21.15.52:0/3998116439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d0ae8b0 msgr2=0x55988d0ac930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.028+0000 7fe7b659c700 1 --2- 172.21.15.52:0/3998116439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d0ae8b0 0x55988d0ac930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.028+0000 7fe7b659c700 1 -- 172.21.15.52:0/3998116439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55988d0aa470 msgr2=0x55988d0b6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.028+0000 7fe7b659c700 1 --2- 172.21.15.52:0/3998116439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55988d0aa470 0x55988d0b6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.028+0000 7fe7b659c700 1 -- 172.21.15.52:0/3998116439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55988d0a0b00 con 0x55988d0a9b60 2023-11-27T15:25:23.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.028+0000 7fe7b659c700 1 --2- 172.21.15.52:0/3998116439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988d0a9b60 0x55988d0b5c40 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009990 tx=0x7fe7a8022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b0945b6af60de0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:23.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.029+0000 7fe7a57fa700 1 -- 172.21.15.52:0/3998116439 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8005160 con 0x55988d0a9b60 2023-11-27T15:25:23.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.029+0000 7fe7a57fa700 1 -- 172.21.15.52:0/3998116439 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7a802c050 con 0x55988d0a9b60 2023-11-27T15:25:23.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.029+0000 7fe7a57fa700 1 -- 172.21.15.52:0/3998116439 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8004860 con 0x55988d0a9b60 2023-11-27T15:25:23.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.029+0000 7fe7c899cb80 1 -- 172.21.15.52:0/3998116439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988d0a9b60 msgr2=0x55988d0b5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.029+0000 7fe7c899cb80 1 --2- 172.21.15.52:0/3998116439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988d0a9b60 0x55988d0b5c40 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009990 tx=0x7fe7a8022000 comp rx=0 tx=0).stop 2023-11-27T15:25:23.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.029+0000 7fe7c899cb80 1 -- 172.21.15.52:0/3998116439 shutdown_connections 2023-11-27T15:25:23.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.029+0000 7fe7c899cb80 1 --2- 172.21.15.52:0/3998116439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d0ae8b0 0x55988d0ac930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.029+0000 7fe7c899cb80 1 --2- 172.21.15.52:0/3998116439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988d0a9b60 0x55988d0b5c40 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.044 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.029+0000 7fe7c899cb80 1 --2- 172.21.15.52:0/3998116439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55988d0aa470 0x55988d0b6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.044 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.029+0000 7fe7c899cb80 1 -- 172.21.15.52:0/3998116439 >> 172.21.15.52:0/3998116439 conn(0x55988d09f360 msgr2=0x55988d0bfc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:23.044 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.029+0000 7fe7c899cb80 1 -- 172.21.15.52:0/3998116439 shutdown_connections 2023-11-27T15:25:23.044 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.029+0000 7fe7c899cb80 1 -- 172.21.15.52:0/3998116439 wait complete. 2023-11-27T15:25:23.044 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.030+0000 7fe7c899cb80 1 -- start start 2023-11-27T15:25:23.044 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.030+0000 7fe7c899cb80 1 Processor -- start 2023-11-27T15:25:23.044 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.030+0000 7fe7c899cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d0a9b60 0x55988d0ba650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.045 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.030+0000 7fe7c899cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988d0aa470 0x55988d0bab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.045 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.030+0000 7fe7c899cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55988d0ae8b0 0x55988d0b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.045 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.030+0000 7fe7c899cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988d0b9240 con 0x55988d0aa470 2023-11-27T15:25:23.045 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.030+0000 7fe7c899cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988d0b93b0 con 0x55988d0ae8b0 2023-11-27T15:25:23.045 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.030+0000 7fe7c899cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988d0b9570 con 0x55988d0a9b60 2023-11-27T15:25:23.045 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.030+0000 7fe7c899cb80 10 client.0 _ll_register_callbacks cb 0x7fe7ac099c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:23.045 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.030+0000 7fe7b5d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988d0aa470 0x55988d0bab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.046 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.030+0000 7fe7b5d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988d0aa470 0x55988d0bab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54920/0 (socket says 172.21.15.52:54920) 2023-11-27T15:25:23.046 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.030+0000 7fe7b5d9b700 1 -- 172.21.15.52:0/1615327250 learned_addr learned my addr 172.21.15.52:0/1615327250 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:23.046 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.030+0000 7fe7b6d9d700 1 --2- 172.21.15.52:0/1615327250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55988d0ae8b0 0x55988d0b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.046 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.030+0000 7fe7b659c700 1 --2- 172.21.15.52:0/1615327250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d0a9b60 0x55988d0ba650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.046 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.030+0000 7fe7b5d9b700 1 -- 172.21.15.52:0/1615327250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d0a9b60 msgr2=0x55988d0ba650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.046 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.030+0000 7fe7b5d9b700 1 --2- 172.21.15.52:0/1615327250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d0a9b60 0x55988d0ba650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.046 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.030+0000 7fe7b5d9b700 1 -- 172.21.15.52:0/1615327250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55988d0ae8b0 msgr2=0x55988d0b8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.047 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.030+0000 7fe7b5d9b700 1 --2- 172.21.15.52:0/1615327250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55988d0ae8b0 0x55988d0b8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.047 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.031+0000 7fe7b5d9b700 1 -- 172.21.15.52:0/1615327250 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b810da60 con 0x55988d0aa470 2023-11-27T15:25:23.047 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.031+0000 7fe7b5d9b700 1 --2- 172.21.15.52:0/1615327250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988d0aa470 0x55988d0bab90 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac099420 tx=0x7fe7ac099d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7c137f63dcce9 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:23.047 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.031+0000 7fe79dffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:25:23.047 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.031+0000 7fe79dffb700 1 -- 172.21.15.52:0/1615327250 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac099040 con 0x55988d0aa470 2023-11-27T15:25:23.047 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.031+0000 7fe79dffb700 1 -- 172.21.15.52:0/1615327250 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fe7ac0ca360 con 0x55988d0aa470 2023-11-27T15:25:23.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.031+0000 7fe7c899cb80 1 -- 172.21.15.52:0/1615327250 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe7ac09a340 con 0x55988d0aa470 2023-11-27T15:25:23.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.031+0000 7fe79dffb700 1 -- 172.21.15.52:0/1615327250 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7ac0991a0 con 0x55988d0aa470 2023-11-27T15:25:23.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.031+0000 7fe7c899cb80 10 client.16629 fetch_fsmap learned FSMap version 30 2023-11-27T15:25:23.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.031+0000 7fe7c899cb80 1 -- 172.21.15.52:0/1615327250 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55988ce28de0 con 0x55988d0aa470 2023-11-27T15:25:23.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.031+0000 7fe79dffb700 1 -- 172.21.15.52:0/1615327250 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7ac09a340 con 0x55988d0aa470 2023-11-27T15:25:23.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.031+0000 7fe79dffb700 1 -- 172.21.15.52:0/1615327250 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe7ac0b2b50 con 0x55988d0aa470 2023-11-27T15:25:23.048 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.031+0000 7fe7c899cb80 10 client.16629 fetch_fsmap finished waiting for FSMap version 30 2023-11-27T15:25:23.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.031+0000 7fe7c899cb80 10 client.16629 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:23.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.031+0000 7fe7c899cb80 20 client.16629 populate_metadata read hostname 'smithi052' 2023-11-27T15:25:23.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.031+0000 7fe7c899cb80 1 --2- 172.21.15.52:0/1615327250 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55988d1182b0 0x55988d11a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.031+0000 7fe7c899cb80 4 client.16629 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:25:23.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.031+0000 7fe7c899cb80 1 -- 172.21.15.52:0/1615327250 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55988cfc9660 con 0x55988d1182b0 2023-11-27T15:25:23.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.032+0000 7fe7b659c700 1 --2- 172.21.15.52:0/1615327250 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55988d1182b0 0x55988d11a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.032+0000 7fe79dffb700 10 client.16629 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:23.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.032+0000 7fe7b659c700 1 --2- 172.21.15.52:0/1615327250 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55988d1182b0 0x55988d11a6c0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be44905315af42ff server_cookie=8d97ddb5a45c53bf in_seq=0 out_seq=0 2023-11-27T15:25:23.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.032+0000 7fe79dffb700 1 -- 172.21.15.52:0/1615327250 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55988cfc9660 con 0x55988d1182b0 2023-11-27T15:25:23.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.032+0000 7fe79dffb700 10 client.16629 handle_command_reply: tid=0 2023-11-27T15:25:23.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.032+0000 7fe7c899cb80 1 client.16629 shutdown 2023-11-27T15:25:23.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.032+0000 7fe7c899cb80 1 -- 172.21.15.52:0/1615327250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988d0aa470 msgr2=0x55988d0bab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.032+0000 7fe7c899cb80 1 --2- 172.21.15.52:0/1615327250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988d0aa470 0x55988d0bab90 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac099420 tx=0x7fe7ac099d40 comp rx=0 tx=0).stop 2023-11-27T15:25:23.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.033+0000 7fe7c899cb80 1 -- 172.21.15.52:0/1615327250 shutdown_connections 2023-11-27T15:25:23.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.033+0000 7fe7c899cb80 1 --2- 172.21.15.52:0/1615327250 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55988d1182b0 0x55988d11a6c0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.033+0000 7fe7c899cb80 1 --2- 172.21.15.52:0/1615327250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55988d0ae8b0 0x55988d0b8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.033+0000 7fe7c899cb80 1 --2- 172.21.15.52:0/1615327250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55988d0a9b60 0x55988d0ba650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.033+0000 7fe7c899cb80 1 --2- 172.21.15.52:0/1615327250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988d0aa470 0x55988d0bab90 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.033+0000 7fe79dffb700 0 client.16629 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:23.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.033+0000 7fe7c899cb80 1 -- 172.21.15.52:0/1615327250 >> 172.21.15.52:0/1615327250 conn(0x55988d09f360 msgr2=0x55988d0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:23.052 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.033+0000 7fe7c899cb80 1 -- 172.21.15.52:0/1615327250 shutdown_connections 2023-11-27T15:25:23.052 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.033+0000 7fe7c899cb80 1 -- 172.21.15.52:0/1615327250 wait complete. 2023-11-27T15:25:23.052 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.033+0000 7fe7c899cb80 20 client.16629 trim_cache size 0 max 16384 2023-11-27T15:25:23.052 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:25:23.052 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2023-11-27T15:25:23.053 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2023-11-27T15:25:23.053 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:25:23.054 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.033+0000 7fe7bc810700 1 -- 172.21.15.52:0/3344500798 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe7a00a12c0 msgr2=0x7fe7a00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.054 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.033+0000 7fe7bc810700 1 --2- 172.21.15.52:0/3344500798 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe7a00a12c0 0x7fe7a00a3770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8006010 tx=0x7fe7a800b260 comp rx=0 tx=0).stop 2023-11-27T15:25:23.054 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.033+0000 7fe7bc810700 1 -- 172.21.15.52:0/3344500798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7b8101f10 msgr2=0x7fe7b81968f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.033+0000 7fe7bc810700 1 --2- 172.21.15.52:0/3344500798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7b8101f10 0x7fe7b81968f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00e9a0 tx=0x7fe7ac00ed60 comp rx=0 tx=0).stop 2023-11-27T15:25:23.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.033+0000 7fe7bc810700 1 -- 172.21.15.52:0/3344500798 shutdown_connections 2023-11-27T15:25:23.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.033+0000 7fe7bc810700 1 --2- 172.21.15.52:0/3344500798 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe7a00a12c0 0x7fe7a00a3770 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.033+0000 7fe7bc810700 1 --2- 172.21.15.52:0/3344500798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7b810a940 0x7fe7b819ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.033+0000 7fe7bc810700 1 --2- 172.21.15.52:0/3344500798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7b8101600 0x7fe7b81963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.033+0000 7fe7bc810700 1 --2- 172.21.15.52:0/3344500798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7b8101f10 0x7fe7b81968f0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.033+0000 7fe7bc810700 1 -- 172.21.15.52:0/3344500798 >> 172.21.15.52:0/3344500798 conn(0x7fe7b80760d0 msgr2=0x7fe7b81023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:23.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.033+0000 7fe7bc810700 1 -- 172.21.15.52:0/3344500798 shutdown_connections 2023-11-27T15:25:23.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.033+0000 7fe7bc810700 1 -- 172.21.15.52:0/3344500798 wait complete. 2023-11-27T15:25:23.056 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-27T15:25:23.056 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:25:23.184 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.177+0000 7f2c4d9c5700 1 -- 172.21.15.52:0/1603424947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c480ffe00 msgr2=0x7f2c48103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.184 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.177+0000 7f2c4d9c5700 1 --2- 172.21.15.52:0/1603424947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c480ffe00 0x7f2c48103dc0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c0099c0 tx=0x7f2c3c009cd0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.184 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.178+0000 7f2c4d9c5700 1 -- 172.21.15.52:0/1603424947 shutdown_connections 2023-11-27T15:25:23.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.178+0000 7f2c4d9c5700 1 --2- 172.21.15.52:0/1603424947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c481044a0 0x7f2c4810c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.178+0000 7f2c4d9c5700 1 --2- 172.21.15.52:0/1603424947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c480ff460 0x7f2c480ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.178+0000 7f2c4d9c5700 1 --2- 172.21.15.52:0/1603424947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c480ffe00 0x7f2c48103dc0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.178+0000 7f2c4d9c5700 1 -- 172.21.15.52:0/1603424947 >> 172.21.15.52:0/1603424947 conn(0x7f2c480763a0 msgr2=0x7f2c480767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:23.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.178+0000 7f2c4d9c5700 1 -- 172.21.15.52:0/1603424947 shutdown_connections 2023-11-27T15:25:23.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.178+0000 7f2c4d9c5700 1 -- 172.21.15.52:0/1603424947 wait complete. 2023-11-27T15:25:23.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.178+0000 7f2c4d9c5700 1 Processor -- start 2023-11-27T15:25:23.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.178+0000 7f2c4d9c5700 1 -- start start 2023-11-27T15:25:23.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.179+0000 7f2c4d9c5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c480ff460 0x7f2c481952d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.179+0000 7f2c4d9c5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c480ffe00 0x7f2c48195810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.179+0000 7f2c4d9c5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c481044a0 0x7f2c48199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.179+0000 7f2c4d9c5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4819a2b0 con 0x7f2c480ffe00 2023-11-27T15:25:23.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.179+0000 7f2c4d9c5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4819a420 con 0x7f2c480ff460 2023-11-27T15:25:23.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.179+0000 7f2c4d9c5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4819a5e0 con 0x7f2c481044a0 2023-11-27T15:25:23.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.179+0000 7f2c467fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c480ffe00 0x7f2c48195810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.179+0000 7f2c467fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c480ffe00 0x7f2c48195810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54934/0 (socket says 172.21.15.52:54934) 2023-11-27T15:25:23.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.179+0000 7f2c467fc700 1 -- 172.21.15.52:0/4065274981 learned_addr learned my addr 172.21.15.52:0/4065274981 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:23.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.179+0000 7f2c46ffd700 1 --2- 172.21.15.52:0/4065274981 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c480ff460 0x7f2c481952d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.179+0000 7f2c477fe700 1 --2- 172.21.15.52:0/4065274981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c481044a0 0x7f2c48199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.179+0000 7f2c467fc700 1 -- 172.21.15.52:0/4065274981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c481044a0 msgr2=0x7f2c48199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.179+0000 7f2c467fc700 1 --2- 172.21.15.52:0/4065274981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c481044a0 0x7f2c48199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.179+0000 7f2c467fc700 1 -- 172.21.15.52:0/4065274981 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c480ff460 msgr2=0x7f2c481952d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.179+0000 7f2c467fc700 1 --2- 172.21.15.52:0/4065274981 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c480ff460 0x7f2c481952d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.189 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.179+0000 7f2c467fc700 1 -- 172.21.15.52:0/4065274981 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3c0095f0 con 0x7f2c480ffe00 2023-11-27T15:25:23.189 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.179+0000 7f2c477fe700 1 --2- 172.21.15.52:0/4065274981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c481044a0 0x7f2c48199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:23.189 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.179+0000 7f2c467fc700 1 --2- 172.21.15.52:0/4065274981 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c480ffe00 0x7f2c48195810 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009990 tx=0x7f2c3c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:23.189 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.179+0000 7f2c4c9c3700 1 -- 172.21.15.52:0/4065274981 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c01f070 con 0x7f2c480ffe00 2023-11-27T15:25:23.189 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.179+0000 7f2c4c9c3700 1 -- 172.21.15.52:0/4065274981 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c3c01ab50 con 0x7f2c480ffe00 2023-11-27T15:25:23.189 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.179+0000 7f2c4d9c5700 1 -- 172.21.15.52:0/4065274981 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c4810f270 con 0x7f2c480ffe00 2023-11-27T15:25:23.189 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.179+0000 7f2c4c9c3700 1 -- 172.21.15.52:0/4065274981 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c004150 con 0x7f2c480ffe00 2023-11-27T15:25:23.190 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.180+0000 7f2c4d9c5700 1 -- 172.21.15.52:0/4065274981 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c4810f0e0 con 0x7f2c480ffe00 2023-11-27T15:25:23.190 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.181+0000 7f2c4d9c5700 1 -- 172.21.15.52:0/4065274981 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c4810f400 con 0x7f2c480ffe00 2023-11-27T15:25:23.191 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.181+0000 7f2c4c9c3700 1 -- 172.21.15.52:0/4065274981 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f2c3c023070 con 0x7f2c480ffe00 2023-11-27T15:25:23.191 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.182+0000 7f2c4c9c3700 1 --2- 172.21.15.52:0/4065274981 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f2c300a12c0 0x7f2c300a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.191 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.182+0000 7f2c4c9c3700 1 -- 172.21.15.52:0/4065274981 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f2c3c0cfe40 con 0x7f2c480ffe00 2023-11-27T15:25:23.191 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.182+0000 7f2c46ffd700 1 --2- 172.21.15.52:0/4065274981 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f2c300a12c0 0x7f2c300a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.192 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.182+0000 7f2c46ffd700 1 --2- 172.21.15.52:0/4065274981 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f2c300a12c0 0x7f2c300a3770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2c38005950 tx=0x7f2c380058e0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:23.192 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.186+0000 7f2c4c9c3700 1 -- 172.21.15.52:0/4065274981 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2c3c0992b0 con 0x7f2c480ffe00 2023-11-27T15:25:23.433 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.429+0000 7f2c4d9c5700 1 -- 172.21.15.52:0/4065274981 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c48069470 con 0x7f2c480ffe00 2023-11-27T15:25:23.433 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.430+0000 7f2c4c9c3700 1 -- 172.21.15.52:0/4065274981 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10810 (secure 0 0 0) 0x7f2c3c01acc0 con 0x7f2c480ffe00 2023-11-27T15:25:23.434 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:25:23.434 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2328881376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2328881376},{"type":"v1","addr":"172.21.15.52:6839","nonce":2328881376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2799551570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2799551570},{"type":"v1","addr":"172.21.15.52:6841","nonce":2799551570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:16:09.319630+0000","modified":"2023-11-27T15:25:20.893592+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24494,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/215725838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":215725838},{"type":"v1","addr":"172.21.15.150:6839","nonce":215725838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3571830559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3571830559},{"type":"v1","addr":"172.21.15.102:6839","nonce":3571830559}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:25:23.436 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.431+0000 7f2c4d9c5700 1 -- 172.21.15.52:0/4065274981 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f2c300a12c0 msgr2=0x7f2c300a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.436 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.431+0000 7f2c4d9c5700 1 --2- 172.21.15.52:0/4065274981 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f2c300a12c0 0x7f2c300a3770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2c38005950 tx=0x7f2c380058e0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.436 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.432+0000 7f2c4d9c5700 1 -- 172.21.15.52:0/4065274981 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c480ffe00 msgr2=0x7f2c48195810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.436 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.432+0000 7f2c4d9c5700 1 --2- 172.21.15.52:0/4065274981 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c480ffe00 0x7f2c48195810 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009990 tx=0x7f2c3c009580 comp rx=0 tx=0).stop 2023-11-27T15:25:23.436 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.432+0000 7f2c4d9c5700 1 -- 172.21.15.52:0/4065274981 shutdown_connections 2023-11-27T15:25:23.437 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.432+0000 7f2c4d9c5700 1 --2- 172.21.15.52:0/4065274981 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f2c300a12c0 0x7f2c300a3770 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.437 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.432+0000 7f2c4d9c5700 1 --2- 172.21.15.52:0/4065274981 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c480ff460 0x7f2c481952d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.437 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.432+0000 7f2c4d9c5700 1 --2- 172.21.15.52:0/4065274981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c481044a0 0x7f2c48199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.437 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.432+0000 7f2c4d9c5700 1 --2- 172.21.15.52:0/4065274981 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c480ffe00 0x7f2c48195810 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.437 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.432+0000 7f2c4d9c5700 1 -- 172.21.15.52:0/4065274981 >> 172.21.15.52:0/4065274981 conn(0x7f2c480763a0 msgr2=0x7f2c480fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:23.437 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.432+0000 7f2c4d9c5700 1 -- 172.21.15.52:0/4065274981 shutdown_connections 2023-11-27T15:25:23.438 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.432+0000 7f2c4d9c5700 1 -- 172.21.15.52:0/4065274981 wait complete. 2023-11-27T15:25:23.438 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2023-11-27T15:25:23.448 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-27T15:25:23.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.572+0000 7f1bf470a700 1 -- 172.21.15.52:0/955200953 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bec0ff520 msgr2=0x7f1bec0ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.572+0000 7f1bf470a700 1 --2- 172.21.15.52:0/955200953 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bec0ff520 0x7f1bec0ff8f0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f1be80099c0 tx=0x7f1be8009cd0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.572+0000 7f1bf470a700 1 -- 172.21.15.52:0/955200953 shutdown_connections 2023-11-27T15:25:23.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.572+0000 7f1bf470a700 1 --2- 172.21.15.52:0/955200953 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bec104560 0x7f1bec10c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.572+0000 7f1bf470a700 1 --2- 172.21.15.52:0/955200953 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bec0ff520 0x7f1bec0ff8f0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.572+0000 7f1bf470a700 1 --2- 172.21.15.52:0/955200953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bec0ffec0 0x7f1bec103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.572+0000 7f1bf470a700 1 -- 172.21.15.52:0/955200953 >> 172.21.15.52:0/955200953 conn(0x7f1bec0760d0 msgr2=0x7f1bec0764d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:23.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.572+0000 7f1bf470a700 1 -- 172.21.15.52:0/955200953 shutdown_connections 2023-11-27T15:25:23.578 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.572+0000 7f1bf470a700 1 -- 172.21.15.52:0/955200953 wait complete. 2023-11-27T15:25:23.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.573+0000 7f1bf470a700 1 Processor -- start 2023-11-27T15:25:23.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.573+0000 7f1bf470a700 1 -- start start 2023-11-27T15:25:23.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.573+0000 7f1bf470a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bec0ff520 0x7f1bec1953b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.573+0000 7f1bf470a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bec0ffec0 0x7f1bec1958f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.573+0000 7f1bf470a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bec104560 0x7f1bec199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.573+0000 7f1bf470a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec19a390 con 0x7f1bec0ffec0 2023-11-27T15:25:23.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.573+0000 7f1bf470a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec19a500 con 0x7f1bec0ff520 2023-11-27T15:25:23.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.573+0000 7f1bf470a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec19a6c0 con 0x7f1bec104560 2023-11-27T15:25:23.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.574+0000 7f1bf1ca5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bec0ffec0 0x7f1bec1958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.574+0000 7f1bf1ca5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bec0ffec0 0x7f1bec1958f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54948/0 (socket says 172.21.15.52:54948) 2023-11-27T15:25:23.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.574+0000 7f1bf1ca5700 1 -- 172.21.15.52:0/2259410270 learned_addr learned my addr 172.21.15.52:0/2259410270 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:23.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.574+0000 7f1bf2ca7700 1 --2- 172.21.15.52:0/2259410270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bec104560 0x7f1bec199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.574+0000 7f1bf24a6700 1 --2- 172.21.15.52:0/2259410270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bec0ff520 0x7f1bec1953b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.574+0000 7f1bf1ca5700 1 -- 172.21.15.52:0/2259410270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bec104560 msgr2=0x7f1bec199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.574+0000 7f1bf1ca5700 1 --2- 172.21.15.52:0/2259410270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bec104560 0x7f1bec199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.574+0000 7f1bf1ca5700 1 -- 172.21.15.52:0/2259410270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bec0ff520 msgr2=0x7f1bec1953b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.574+0000 7f1bf1ca5700 1 --2- 172.21.15.52:0/2259410270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bec0ff520 0x7f1bec1953b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.583 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.574+0000 7f1bf1ca5700 1 -- 172.21.15.52:0/2259410270 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be80095f0 con 0x7f1bec0ffec0 2023-11-27T15:25:23.584 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.574+0000 7f1bf2ca7700 1 --2- 172.21.15.52:0/2259410270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bec104560 0x7f1bec199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:23.584 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.574+0000 7f1bf24a6700 1 --2- 172.21.15.52:0/2259410270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bec0ff520 0x7f1bec1953b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:23.584 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.574+0000 7f1bf1ca5700 1 --2- 172.21.15.52:0/2259410270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bec0ffec0 0x7f1bec1958f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc00e940 tx=0x7f1bdc00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:23.584 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.574+0000 7f1be37fe700 1 -- 172.21.15.52:0/2259410270 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc009730 con 0x7f1bec0ffec0 2023-11-27T15:25:23.584 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.574+0000 7f1bf470a700 1 -- 172.21.15.52:0/2259410270 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bec10f370 con 0x7f1bec0ffec0 2023-11-27T15:25:23.584 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.574+0000 7f1be37fe700 1 -- 172.21.15.52:0/2259410270 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bdc00cbd0 con 0x7f1bec0ffec0 2023-11-27T15:25:23.585 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.574+0000 7f1be37fe700 1 -- 172.21.15.52:0/2259410270 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc018680 con 0x7f1bec0ffec0 2023-11-27T15:25:23.585 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.574+0000 7f1bf470a700 1 -- 172.21.15.52:0/2259410270 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bec10f1e0 con 0x7f1bec0ffec0 2023-11-27T15:25:23.585 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.576+0000 7f1be37fe700 1 -- 172.21.15.52:0/2259410270 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f1bdc00cd40 con 0x7f1bec0ffec0 2023-11-27T15:25:23.585 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.577+0000 7f1be37fe700 1 --2- 172.21.15.52:0/2259410270 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f1bd80a12c0 0x7f1bd80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.585 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.577+0000 7f1be37fe700 1 -- 172.21.15.52:0/2259410270 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f1bdc014070 con 0x7f1bec0ffec0 2023-11-27T15:25:23.585 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.577+0000 7f1bf24a6700 1 --2- 172.21.15.52:0/2259410270 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f1bd80a12c0 0x7f1bd80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.578+0000 7f1bf24a6700 1 --2- 172.21.15.52:0/2259410270 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f1bd80a12c0 0x7f1bd80a3770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1be8009990 tx=0x7f1be801a040 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:23.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.578+0000 7f1c00896b80 1 Processor -- start 2023-11-27T15:25:23.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.578+0000 7f1c00896b80 1 -- start start 2023-11-27T15:25:23.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.578+0000 7f1c00896b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f580948b60 0x55f580948f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.578+0000 7f1c00896b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f580949470 0x55f58094d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.578+0000 7f1c00896b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f58094d8b0 0x55f58094dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.578+0000 7f1c00896b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58084e060 con 0x55f580948b60 2023-11-27T15:25:23.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.578+0000 7f1c00896b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5808a08a0 con 0x55f58094d8b0 2023-11-27T15:25:23.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.578+0000 7f1c00896b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f580899d30 con 0x55f580949470 2023-11-27T15:25:23.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.578+0000 7f1bf24a6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f580948b60 0x55f580948f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.578+0000 7f1bf24a6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f580948b60 0x55f580948f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54954/0 (socket says 172.21.15.52:54954) 2023-11-27T15:25:23.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.578+0000 7f1bf24a6700 1 -- 172.21.15.52:0/2449899581 learned_addr learned my addr 172.21.15.52:0/2449899581 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:23.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.578+0000 7f1bf2ca7700 1 --2- 172.21.15.52:0/2449899581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f58094d8b0 0x55f58094dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.579+0000 7f1bf1ca5700 1 --2- 172.21.15.52:0/2449899581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f580949470 0x55f58094d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.579+0000 7f1bf24a6700 1 -- 172.21.15.52:0/2449899581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f580949470 msgr2=0x55f58094d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.579+0000 7f1bf24a6700 1 --2- 172.21.15.52:0/2449899581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f580949470 0x55f58094d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.579+0000 7f1bf24a6700 1 -- 172.21.15.52:0/2449899581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f58094d8b0 msgr2=0x55f58094dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.579+0000 7f1bf24a6700 1 --2- 172.21.15.52:0/2449899581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f58094d8b0 0x55f58094dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.579+0000 7f1bf24a6700 1 -- 172.21.15.52:0/2449899581 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f58071b630 con 0x55f580948b60 2023-11-27T15:25:23.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.579+0000 7f1bf2ca7700 1 --2- 172.21.15.52:0/2449899581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f58094d8b0 0x55f58094dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:23.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.579+0000 7f1bf1ca5700 1 --2- 172.21.15.52:0/2449899581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f580949470 0x55f58094d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:23.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.579+0000 7f1bf24a6700 1 --2- 172.21.15.52:0/2449899581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f580948b60 0x55f580948f30 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f1be8015000 tx=0x7f1be801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ae035d8e092fdd server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:23.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.579+0000 7f1be0ff9700 1 -- 172.21.15.52:0/2449899581 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be802a070 con 0x55f580948b60 2023-11-27T15:25:23.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.579+0000 7f1be0ff9700 1 -- 172.21.15.52:0/2449899581 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1be8005500 con 0x55f580948b60 2023-11-27T15:25:23.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.579+0000 7f1be0ff9700 1 -- 172.21.15.52:0/2449899581 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be801d040 con 0x55f580948b60 2023-11-27T15:25:23.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.580+0000 7f1c00896b80 1 -- 172.21.15.52:0/2449899581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f580948b60 msgr2=0x55f580948f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.580+0000 7f1c00896b80 1 --2- 172.21.15.52:0/2449899581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f580948b60 0x55f580948f30 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f1be8015000 tx=0x7f1be801e000 comp rx=0 tx=0).stop 2023-11-27T15:25:23.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.580+0000 7f1c00896b80 1 -- 172.21.15.52:0/2449899581 shutdown_connections 2023-11-27T15:25:23.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.580+0000 7f1c00896b80 1 --2- 172.21.15.52:0/2449899581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f58094d8b0 0x55f58094dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.580+0000 7f1c00896b80 1 --2- 172.21.15.52:0/2449899581 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f580948b60 0x55f580948f30 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.580+0000 7f1c00896b80 1 --2- 172.21.15.52:0/2449899581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f580949470 0x55f58094d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.580+0000 7f1c00896b80 1 -- 172.21.15.52:0/2449899581 >> 172.21.15.52:0/2449899581 conn(0x55f58093e360 msgr2=0x55f580943b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:23.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.580+0000 7f1c00896b80 1 -- 172.21.15.52:0/2449899581 shutdown_connections 2023-11-27T15:25:23.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.580+0000 7f1c00896b80 1 -- 172.21.15.52:0/2449899581 wait complete. 2023-11-27T15:25:23.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.580+0000 7f1c00896b80 1 -- start start 2023-11-27T15:25:23.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.580+0000 7f1c00896b80 1 Processor -- start 2023-11-27T15:25:23.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.580+0000 7f1c00896b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f580948b60 0x55f580955050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.580+0000 7f1c00896b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f580949470 0x55f580952260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.580+0000 7f1c00896b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f58094d8b0 0x55f580952870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.580+0000 7f1c00896b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58083b8e0 con 0x55f580949470 2023-11-27T15:25:23.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.580+0000 7f1c00896b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58083b580 con 0x55f58094d8b0 2023-11-27T15:25:23.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.580+0000 7f1c00896b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58071d920 con 0x55f580948b60 2023-11-27T15:25:23.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.581+0000 7f1c00896b80 10 client.0 _ll_register_callbacks cb 0x55f580859780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:23.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.581+0000 7f1bf1ca5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f580949470 0x55f580952260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.581+0000 7f1bf1ca5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f580949470 0x55f580952260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54964/0 (socket says 172.21.15.52:54964) 2023-11-27T15:25:23.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.581+0000 7f1bf1ca5700 1 -- 172.21.15.52:0/3964601175 learned_addr learned my addr 172.21.15.52:0/3964601175 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:23.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.581+0000 7f1bf2ca7700 1 --2- 172.21.15.52:0/3964601175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f58094d8b0 0x55f580952870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.581+0000 7f1bf24a6700 1 --2- 172.21.15.52:0/3964601175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f580948b60 0x55f580955050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.581+0000 7f1bf1ca5700 1 -- 172.21.15.52:0/3964601175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f580948b60 msgr2=0x55f580955050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.581+0000 7f1bf1ca5700 1 --2- 172.21.15.52:0/3964601175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f580948b60 0x55f580955050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.581+0000 7f1bf1ca5700 1 -- 172.21.15.52:0/3964601175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f58094d8b0 msgr2=0x55f580952870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.581+0000 7f1bf1ca5700 1 --2- 172.21.15.52:0/3964601175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f58094d8b0 0x55f580952870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.581+0000 7f1bf1ca5700 1 -- 172.21.15.52:0/3964601175 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bec10c970 con 0x55f580949470 2023-11-27T15:25:23.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.581+0000 7f1bf1ca5700 1 --2- 172.21.15.52:0/3964601175 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f580949470 0x55f580952260 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc00e910 tx=0x7f1bdc09a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2aa54314e546f44 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:23.601 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.581+0000 7f1bd5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:25:23.601 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.581+0000 7f1bd5ffb700 1 -- 172.21.15.52:0/3964601175 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc0b1070 con 0x55f580949470 2023-11-27T15:25:23.601 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.581+0000 7f1c00896b80 1 -- 172.21.15.52:0/3964601175 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f58093bb20 con 0x55f580949470 2023-11-27T15:25:23.601 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.581+0000 7f1bd5ffb700 1 -- 172.21.15.52:0/3964601175 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f1bdc0a2540 con 0x55f580949470 2023-11-27T15:25:23.601 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.582+0000 7f1bd5ffb700 1 -- 172.21.15.52:0/3964601175 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bdc0997f0 con 0x55f580949470 2023-11-27T15:25:23.602 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.582+0000 7f1c00896b80 10 client.16647 fetch_fsmap learned FSMap version 30 2023-11-27T15:25:23.602 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.582+0000 7f1c00896b80 1 -- 172.21.15.52:0/3964601175 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f580860190 con 0x55f580949470 2023-11-27T15:25:23.602 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.582+0000 7f1bd5ffb700 1 -- 172.21.15.52:0/3964601175 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bdc0c8080 con 0x55f580949470 2023-11-27T15:25:23.602 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.582+0000 7f1bd5ffb700 1 -- 172.21.15.52:0/3964601175 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f1bdc099050 con 0x55f580949470 2023-11-27T15:25:23.602 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.582+0000 7f1c00896b80 10 client.16647 fetch_fsmap finished waiting for FSMap version 30 2023-11-27T15:25:23.603 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.582+0000 7f1c00896b80 10 client.16647 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:25:23.603 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.582+0000 7f1c00896b80 20 client.16647 populate_metadata read hostname 'smithi052' 2023-11-27T15:25:23.603 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.582+0000 7f1c00896b80 1 --2- 172.21.15.52:0/3964601175 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55f5809626f0 0x55f580964b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.603 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.582+0000 7f1c00896b80 4 client.16647 mds_command: new command op to 24472 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:23.603 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.582+0000 7f1c00896b80 1 -- 172.21.15.52:0/3964601175 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f58085c0a0 con 0x55f5809626f0 2023-11-27T15:25:23.603 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.582+0000 7f1bf24a6700 1 --2- 172.21.15.52:0/3964601175 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55f5809626f0 0x55f580964b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.604 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.583+0000 7f1bd5ffb700 10 client.16647 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:25:23.604 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.583+0000 7f1bf24a6700 1 --2- 172.21.15.52:0/3964601175 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55f5809626f0 0x55f580964b00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f8d306c8e97ca278 server_cookie=a7c8ebea50cfa8ef in_seq=0 out_seq=0 2023-11-27T15:25:23.604 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.583+0000 7f1bd5ffb700 1 -- 172.21.15.52:0/3964601175 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f58085c0a0 con 0x55f5809626f0 2023-11-27T15:25:23.604 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.583+0000 7f1bd5ffb700 10 client.16647 handle_command_reply: tid=0 2023-11-27T15:25:23.604 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.583+0000 7f1c00896b80 1 client.16647 shutdown 2023-11-27T15:25:23.605 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.583+0000 7f1c00896b80 1 -- 172.21.15.52:0/3964601175 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f580949470 msgr2=0x55f580952260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.605 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.583+0000 7f1c00896b80 1 --2- 172.21.15.52:0/3964601175 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f580949470 0x55f580952260 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc00e910 tx=0x7f1bdc09a0f0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.605 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.583+0000 7f1c00896b80 1 -- 172.21.15.52:0/3964601175 shutdown_connections 2023-11-27T15:25:23.605 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.583+0000 7f1c00896b80 1 --2- 172.21.15.52:0/3964601175 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55f5809626f0 0x55f580964b00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.605 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.583+0000 7f1c00896b80 1 --2- 172.21.15.52:0/3964601175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f58094d8b0 0x55f580952870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.605 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.583+0000 7f1bd5ffb700 0 client.16647 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:25:23.605 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.583+0000 7f1c00896b80 1 --2- 172.21.15.52:0/3964601175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f580948b60 0x55f580955050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.583+0000 7f1c00896b80 1 --2- 172.21.15.52:0/3964601175 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f580949470 0x55f580952260 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.583+0000 7f1c00896b80 1 -- 172.21.15.52:0/3964601175 >> 172.21.15.52:0/3964601175 conn(0x55f58093e360 msgr2=0x55f58094ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:23.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.583+0000 7f1c00896b80 1 -- 172.21.15.52:0/3964601175 shutdown_connections 2023-11-27T15:25:23.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.583+0000 7f1c00896b80 1 -- 172.21.15.52:0/3964601175 wait complete. 2023-11-27T15:25:23.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.584+0000 7f1c00896b80 20 client.16647 trim_cache size 0 max 16384 2023-11-27T15:25:23.612 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.596+0000 7f1c00896b80 1 Processor -- start 2023-11-27T15:25:23.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.597+0000 7f1c00896b80 1 -- start start 2023-11-27T15:25:23.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.597+0000 7f1c00896b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f580948b60 0x55f5809598e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.597+0000 7f1c00896b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f580949470 0x55f580954b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.597+0000 7f1c00896b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f58094d8b0 0x55f580943560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.597+0000 7f1c00896b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58095fdb0 con 0x55f580948b60 2023-11-27T15:25:23.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.597+0000 7f1c00896b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58093b710 con 0x55f58094d8b0 2023-11-27T15:25:23.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.597+0000 7f1c00896b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5809507c0 con 0x55f580949470 2023-11-27T15:25:23.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.597+0000 7f1bf24a6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f580948b60 0x55f5809598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.597+0000 7f1bf24a6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f580948b60 0x55f5809598e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54968/0 (socket says 172.21.15.52:54968) 2023-11-27T15:25:23.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.597+0000 7f1bf24a6700 1 -- 172.21.15.52:0/2026937555 learned_addr learned my addr 172.21.15.52:0/2026937555 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:23.614 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.597+0000 7f1bf1ca5700 1 --2- 172.21.15.52:0/2026937555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f580949470 0x55f580954b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.597+0000 7f1bf2ca7700 1 --2- 172.21.15.52:0/2026937555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f58094d8b0 0x55f580943560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.597+0000 7f1bf24a6700 1 -- 172.21.15.52:0/2026937555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f580949470 msgr2=0x55f580954b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.597+0000 7f1bf24a6700 1 --2- 172.21.15.52:0/2026937555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f580949470 0x55f580954b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.597+0000 7f1bf24a6700 1 -- 172.21.15.52:0/2026937555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f58094d8b0 msgr2=0x55f580943560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.597+0000 7f1bf24a6700 1 --2- 172.21.15.52:0/2026937555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f58094d8b0 0x55f580943560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.597+0000 7f1bf24a6700 1 -- 172.21.15.52:0/2026937555 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f58085fee0 con 0x55f580948b60 2023-11-27T15:25:23.615 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.597+0000 7f1bf1ca5700 1 --2- 172.21.15.52:0/2026937555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f580949470 0x55f580954b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:23.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.597+0000 7f1bf2ca7700 1 --2- 172.21.15.52:0/2026937555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f58094d8b0 0x55f580943560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:23.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.597+0000 7f1bf24a6700 1 --2- 172.21.15.52:0/2026937555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f580948b60 0x55f5809598e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f1be80058e0 tx=0x7f1be8025040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dfba6fd51345af6 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:23.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.598+0000 7f1be17fa700 1 -- 172.21.15.52:0/2026937555 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be801d050 con 0x55f580948b60 2023-11-27T15:25:23.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.598+0000 7f1be17fa700 1 -- 172.21.15.52:0/2026937555 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1be803acc0 con 0x55f580948b60 2023-11-27T15:25:23.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.598+0000 7f1be17fa700 1 -- 172.21.15.52:0/2026937555 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be802a040 con 0x55f580948b60 2023-11-27T15:25:23.616 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.598+0000 7f1c00896b80 1 -- 172.21.15.52:0/2026937555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f580948b60 msgr2=0x55f5809598e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.598+0000 7f1c00896b80 1 --2- 172.21.15.52:0/2026937555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f580948b60 0x55f5809598e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f1be80058e0 tx=0x7f1be8025040 comp rx=0 tx=0).stop 2023-11-27T15:25:23.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.598+0000 7f1c00896b80 1 -- 172.21.15.52:0/2026937555 shutdown_connections 2023-11-27T15:25:23.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.598+0000 7f1c00896b80 1 --2- 172.21.15.52:0/2026937555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f58094d8b0 0x55f580943560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.598+0000 7f1c00896b80 1 --2- 172.21.15.52:0/2026937555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f580948b60 0x55f5809598e0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.598+0000 7f1c00896b80 1 --2- 172.21.15.52:0/2026937555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f580949470 0x55f580954b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.598+0000 7f1c00896b80 1 -- 172.21.15.52:0/2026937555 >> 172.21.15.52:0/2026937555 conn(0x55f58093e360 msgr2=0x55f58095eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:23.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.598+0000 7f1c00896b80 1 -- 172.21.15.52:0/2026937555 shutdown_connections 2023-11-27T15:25:23.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.598+0000 7f1c00896b80 1 -- 172.21.15.52:0/2026937555 wait complete. 2023-11-27T15:25:23.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.598+0000 7f1c00896b80 1 -- start start 2023-11-27T15:25:23.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.599+0000 7f1c00896b80 1 Processor -- start 2023-11-27T15:25:23.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.599+0000 7f1c00896b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f580948b60 0x55f5809558c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.599+0000 7f1c00896b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f580949470 0x55f580955e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.599+0000 7f1c00896b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f58094d8b0 0x55f580956340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.599+0000 7f1c00896b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5809b5670 con 0x55f580948b60 2023-11-27T15:25:23.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.599+0000 7f1c00896b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5809b57e0 con 0x55f580949470 2023-11-27T15:25:23.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.599+0000 7f1c00896b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5809b59a0 con 0x55f58094d8b0 2023-11-27T15:25:23.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.599+0000 7f1c00896b80 10 client.0 _ll_register_callbacks cb 0x7f1bdc099c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:23.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.599+0000 7f1bf24a6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f580948b60 0x55f5809558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.599+0000 7f1bf24a6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f580948b60 0x55f5809558c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54974/0 (socket says 172.21.15.52:54974) 2023-11-27T15:25:23.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.599+0000 7f1bf24a6700 1 -- 172.21.15.52:0/3729134084 learned_addr learned my addr 172.21.15.52:0/3729134084 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:23.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.599+0000 7f1bf1ca5700 1 --2- 172.21.15.52:0/3729134084 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f580949470 0x55f580955e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.599+0000 7f1bf2ca7700 1 --2- 172.21.15.52:0/3729134084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f58094d8b0 0x55f580956340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.599+0000 7f1bf24a6700 1 -- 172.21.15.52:0/3729134084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f58094d8b0 msgr2=0x55f580956340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.599+0000 7f1bf24a6700 1 --2- 172.21.15.52:0/3729134084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f58094d8b0 0x55f580956340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.599+0000 7f1bf24a6700 1 -- 172.21.15.52:0/3729134084 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f580949470 msgr2=0x55f580955e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.599+0000 7f1bf24a6700 1 --2- 172.21.15.52:0/3729134084 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f580949470 0x55f580955e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.599+0000 7f1bf24a6700 1 -- 172.21.15.52:0/3729134084 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f58071b630 con 0x55f580948b60 2023-11-27T15:25:23.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.599+0000 7f1bf1ca5700 1 --2- 172.21.15.52:0/3729134084 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f580949470 0x55f580955e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:23.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.599+0000 7f1bf24a6700 1 --2- 172.21.15.52:0/3729134084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f580948b60 0x55f5809558c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f1be8009350 tx=0x7f1be803ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dedf9c9ed9cf2be server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:23.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.599+0000 7f1bd5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:25:23.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.600+0000 7f1bd5ffb700 1 -- 172.21.15.52:0/3729134084 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be802a030 con 0x55f580948b60 2023-11-27T15:25:23.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.600+0000 7f1bd5ffb700 1 -- 172.21.15.52:0/3729134084 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f1be8028040 con 0x55f580948b60 2023-11-27T15:25:23.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.600+0000 7f1c00896b80 1 -- 172.21.15.52:0/3729134084 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1be8005500 con 0x55f580948b60 2023-11-27T15:25:23.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.600+0000 7f1bd5ffb700 1 -- 172.21.15.52:0/3729134084 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1be801d020 con 0x55f580948b60 2023-11-27T15:25:23.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.600+0000 7f1c00896b80 10 client.16653 fetch_fsmap learned FSMap version 30 2023-11-27T15:25:23.622 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.600+0000 7f1c00896b80 1 -- 172.21.15.52:0/3729134084 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f5806b0c60 con 0x55f580948b60 2023-11-27T15:25:23.622 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.600+0000 7f1bd5ffb700 1 -- 172.21.15.52:0/3729134084 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1be8005500 con 0x55f580948b60 2023-11-27T15:25:23.622 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.600+0000 7f1bd5ffb700 1 -- 172.21.15.52:0/3729134084 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f1be804b030 con 0x55f580948b60 2023-11-27T15:25:23.622 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.600+0000 7f1c00896b80 10 client.16653 fetch_fsmap finished waiting for FSMap version 30 2023-11-27T15:25:23.622 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.600+0000 7f1c00896b80 10 client.16653 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:25:23.622 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.600+0000 7f1c00896b80 20 client.16653 populate_metadata read hostname 'smithi052' 2023-11-27T15:25:23.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.600+0000 7f1c00896b80 1 --2- 172.21.15.52:0/3729134084 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55f5809626f0 0x55f5809b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.600+0000 7f1c00896b80 4 client.16653 mds_command: new command op to 24472 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:25:23.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.600+0000 7f1c00896b80 1 -- 172.21.15.52:0/3729134084 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f580868660 con 0x55f5809626f0 2023-11-27T15:25:23.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.600+0000 7f1bf1ca5700 1 --2- 172.21.15.52:0/3729134084 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55f5809626f0 0x55f5809b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.601+0000 7f1bd5ffb700 10 client.16653 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:25:23.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.601+0000 7f1bf1ca5700 1 --2- 172.21.15.52:0/3729134084 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55f5809626f0 0x55f5809b8c90 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ac33f3455c83a06b server_cookie=ddbd597e7d8b0e01 in_seq=0 out_seq=0 2023-11-27T15:25:23.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.601+0000 7f1bd5ffb700 1 -- 172.21.15.52:0/3729134084 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f580868660 con 0x55f5809626f0 2023-11-27T15:25:23.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.601+0000 7f1bd5ffb700 10 client.16653 handle_command_reply: tid=0 2023-11-27T15:25:23.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.601+0000 7f1c00896b80 1 client.16653 shutdown 2023-11-27T15:25:23.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.601+0000 7f1c00896b80 1 -- 172.21.15.52:0/3729134084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f580948b60 msgr2=0x55f5809558c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.601+0000 7f1c00896b80 1 --2- 172.21.15.52:0/3729134084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f580948b60 0x55f5809558c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f1be8009350 tx=0x7f1be803ab70 comp rx=0 tx=0).stop 2023-11-27T15:25:23.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.601+0000 7f1c00896b80 1 -- 172.21.15.52:0/3729134084 shutdown_connections 2023-11-27T15:25:23.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.601+0000 7f1c00896b80 1 --2- 172.21.15.52:0/3729134084 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55f5809626f0 0x55f5809b8c90 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.601+0000 7f1c00896b80 1 --2- 172.21.15.52:0/3729134084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f58094d8b0 0x55f580956340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.601+0000 7f1c00896b80 1 --2- 172.21.15.52:0/3729134084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f580948b60 0x55f5809558c0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.601+0000 7f1c00896b80 1 --2- 172.21.15.52:0/3729134084 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f580949470 0x55f580955e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.601+0000 7f1c00896b80 1 -- 172.21.15.52:0/3729134084 >> 172.21.15.52:0/3729134084 conn(0x55f58093e360 msgr2=0x55f580947960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:23.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.601+0000 7f1bd5ffb700 0 client.16653 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:25:23.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.601+0000 7f1c00896b80 1 -- 172.21.15.52:0/3729134084 shutdown_connections 2023-11-27T15:25:23.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.601+0000 7f1c00896b80 1 -- 172.21.15.52:0/3729134084 wait complete. 2023-11-27T15:25:23.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.601+0000 7f1c00896b80 20 client.16653 trim_cache size 0 max 16384 2023-11-27T15:25:23.626 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:25:23.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.602+0000 7f1bf470a700 1 -- 172.21.15.52:0/2259410270 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f1bd80a12c0 msgr2=0x7f1bd80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.602+0000 7f1bf470a700 1 --2- 172.21.15.52:0/2259410270 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f1bd80a12c0 0x7f1bd80a3770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1be8009990 tx=0x7f1be801a040 comp rx=0 tx=0).stop 2023-11-27T15:25:23.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.602+0000 7f1bf470a700 1 -- 172.21.15.52:0/2259410270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bec0ffec0 msgr2=0x7f1bec1958f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.602+0000 7f1bf470a700 1 --2- 172.21.15.52:0/2259410270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bec0ffec0 0x7f1bec1958f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc00e940 tx=0x7f1bdc00ed00 comp rx=0 tx=0).stop 2023-11-27T15:25:23.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.602+0000 7f1bf470a700 1 -- 172.21.15.52:0/2259410270 shutdown_connections 2023-11-27T15:25:23.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.602+0000 7f1bf470a700 1 --2- 172.21.15.52:0/2259410270 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f1bd80a12c0 0x7f1bd80a3770 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.602+0000 7f1bf470a700 1 --2- 172.21.15.52:0/2259410270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bec0ff520 0x7f1bec1953b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.602+0000 7f1bf470a700 1 --2- 172.21.15.52:0/2259410270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bec104560 0x7f1bec199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.602+0000 7f1bf470a700 1 --2- 172.21.15.52:0/2259410270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bec0ffec0 0x7f1bec1958f0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.602+0000 7f1bf470a700 1 -- 172.21.15.52:0/2259410270 >> 172.21.15.52:0/2259410270 conn(0x7f1bec0760d0 msgr2=0x7f1bec0fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:23.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.602+0000 7f1bf470a700 1 -- 172.21.15.52:0/2259410270 shutdown_connections 2023-11-27T15:25:23.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.602+0000 7f1bf470a700 1 -- 172.21.15.52:0/2259410270 wait complete. 2023-11-27T15:25:23.629 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-27T15:25:23.759 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.753+0000 7f422d5b8700 1 -- 172.21.15.52:0/587464284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42281041f0 msgr2=0x7f42281045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.760 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.753+0000 7f422d5b8700 1 --2- 172.21.15.52:0/587464284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42281041f0 0x7f42281045c0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f4218009990 tx=0x7f4218009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.760 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.753+0000 7f422d5b8700 1 -- 172.21.15.52:0/587464284 shutdown_connections 2023-11-27T15:25:23.760 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.753+0000 7f422d5b8700 1 --2- 172.21.15.52:0/587464284 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4228100470 0x7f422810c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.760 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.753+0000 7f422d5b8700 1 --2- 172.21.15.52:0/587464284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42281041f0 0x7f42281045c0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.760 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.753+0000 7f422d5b8700 1 --2- 172.21.15.52:0/587464284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42280ffac0 0x7f42280fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.760 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.753+0000 7f422d5b8700 1 -- 172.21.15.52:0/587464284 >> 172.21.15.52:0/587464284 conn(0x7f42280763a0 msgr2=0x7f42280767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:23.761 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.753+0000 7f422d5b8700 1 -- 172.21.15.52:0/587464284 shutdown_connections 2023-11-27T15:25:23.761 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.753+0000 7f422d5b8700 1 -- 172.21.15.52:0/587464284 wait complete. 2023-11-27T15:25:23.761 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.754+0000 7f422d5b8700 1 Processor -- start 2023-11-27T15:25:23.761 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.754+0000 7f422d5b8700 1 -- start start 2023-11-27T15:25:23.761 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.754+0000 7f422d5b8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42280ffac0 0x7f42281962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.761 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.754+0000 7f422d5b8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4228100470 0x7f4228196830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.762 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.754+0000 7f422d5b8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42281041f0 0x7f422819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.762 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.754+0000 7f422d5b8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422819b2d0 con 0x7f42280ffac0 2023-11-27T15:25:23.762 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.754+0000 7f422d5b8700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422819b440 con 0x7f42281041f0 2023-11-27T15:25:23.762 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.754+0000 7f422d5b8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422819b600 con 0x7f4228100470 2023-11-27T15:25:23.762 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.754+0000 7f4226ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42280ffac0 0x7f42281962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.754+0000 7f4226ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42280ffac0 0x7f42281962f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54988/0 (socket says 172.21.15.52:54988) 2023-11-27T15:25:23.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.754+0000 7f4226ffd700 1 -- 172.21.15.52:0/2793140174 learned_addr learned my addr 172.21.15.52:0/2793140174 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:23.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.755+0000 7f42277fe700 1 --2- 172.21.15.52:0/2793140174 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42281041f0 0x7f422819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.755+0000 7f42267fc700 1 --2- 172.21.15.52:0/2793140174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4228100470 0x7f4228196830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.755+0000 7f4226ffd700 1 -- 172.21.15.52:0/2793140174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4228100470 msgr2=0x7f4228196830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.755+0000 7f4226ffd700 1 --2- 172.21.15.52:0/2793140174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4228100470 0x7f4228196830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.755+0000 7f4226ffd700 1 -- 172.21.15.52:0/2793140174 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42281041f0 msgr2=0x7f422819ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.755+0000 7f4226ffd700 1 --2- 172.21.15.52:0/2793140174 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42281041f0 0x7f422819ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.755+0000 7f4226ffd700 1 -- 172.21.15.52:0/2793140174 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42180095f0 con 0x7f42280ffac0 2023-11-27T15:25:23.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.755+0000 7f4226ffd700 1 --2- 172.21.15.52:0/2793140174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42280ffac0 0x7f42281962f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f42180095c0 tx=0x7f421800f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:23.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.755+0000 7f420ffff700 1 -- 172.21.15.52:0/2793140174 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421801d070 con 0x7f42280ffac0 2023-11-27T15:25:23.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.755+0000 7f420ffff700 1 -- 172.21.15.52:0/2793140174 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f421800fc00 con 0x7f42280ffac0 2023-11-27T15:25:23.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.755+0000 7f422d5b8700 1 -- 172.21.15.52:0/2793140174 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f422810f260 con 0x7f42280ffac0 2023-11-27T15:25:23.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.755+0000 7f420ffff700 1 -- 172.21.15.52:0/2793140174 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42180173f0 con 0x7f42280ffac0 2023-11-27T15:25:23.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.755+0000 7f422d5b8700 1 -- 172.21.15.52:0/2793140174 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f422810f0d0 con 0x7f42280ffac0 2023-11-27T15:25:23.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.757+0000 7f420ffff700 1 -- 172.21.15.52:0/2793140174 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f4218022470 con 0x7f42280ffac0 2023-11-27T15:25:23.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.758+0000 7f420ffff700 1 --2- 172.21.15.52:0/2793140174 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f42100a1340 0x7f42100a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.758+0000 7f420ffff700 1 -- 172.21.15.52:0/2793140174 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f42180cdf00 con 0x7f42280ffac0 2023-11-27T15:25:23.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.758+0000 7f42267fc700 1 --2- 172.21.15.52:0/2793140174 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f42100a1340 0x7f42100a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.758+0000 7f42267fc700 1 --2- 172.21.15.52:0/2793140174 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f42100a1340 0x7f42100a37f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f421c005950 tx=0x7f421c0058e0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:23.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.758+0000 7f4239744b80 1 Processor -- start 2023-11-27T15:25:23.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.758+0000 7f4239744b80 1 -- start start 2023-11-27T15:25:23.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.759+0000 7f4239744b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df469657c0 0x55df46961970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.759+0000 7f4239744b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df46961eb0 0x55df46962320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.759+0000 7f4239744b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df469745d0 0x55df469769b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.759+0000 7f4239744b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df46867060 con 0x55df469745d0 2023-11-27T15:25:23.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.759+0000 7f4239744b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df468b98a0 con 0x55df469657c0 2023-11-27T15:25:23.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.759+0000 7f4239744b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df468b2d30 con 0x55df46961eb0 2023-11-27T15:25:23.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.759+0000 7f42277fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df469745d0 0x55df469769b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.759+0000 7f42277fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df469745d0 0x55df469769b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55000/0 (socket says 172.21.15.52:55000) 2023-11-27T15:25:23.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.759+0000 7f42277fe700 1 -- 172.21.15.52:0/2444620804 learned_addr learned my addr 172.21.15.52:0/2444620804 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:23.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.759+0000 7f4226ffd700 1 --2- 172.21.15.52:0/2444620804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df469657c0 0x55df46961970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.759+0000 7f42267fc700 1 --2- 172.21.15.52:0/2444620804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df46961eb0 0x55df46962320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.759+0000 7f42277fe700 1 -- 172.21.15.52:0/2444620804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df46961eb0 msgr2=0x55df46962320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.759+0000 7f42277fe700 1 --2- 172.21.15.52:0/2444620804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df46961eb0 0x55df46962320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.759+0000 7f42277fe700 1 -- 172.21.15.52:0/2444620804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df469657c0 msgr2=0x55df46961970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.759+0000 7f42277fe700 1 --2- 172.21.15.52:0/2444620804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df469657c0 0x55df46961970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.775 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.759+0000 7f42277fe700 1 -- 172.21.15.52:0/2444620804 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df46734630 con 0x55df469745d0 2023-11-27T15:25:23.775 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.759+0000 7f4226ffd700 1 --2- 172.21.15.52:0/2444620804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df469657c0 0x55df46961970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:23.775 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.759+0000 7f42277fe700 1 --2- 172.21.15.52:0/2444620804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df469745d0 0x55df469769b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f4220009dc0 tx=0x7f42200077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b348f08226d411 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:23.775 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.760+0000 7f420d7fa700 1 -- 172.21.15.52:0/2444620804 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4220014610 con 0x55df469745d0 2023-11-27T15:25:23.775 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.760+0000 7f420d7fa700 1 -- 172.21.15.52:0/2444620804 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f422000f050 con 0x55df469745d0 2023-11-27T15:25:23.775 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.760+0000 7f420d7fa700 1 -- 172.21.15.52:0/2444620804 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4220013960 con 0x55df469745d0 2023-11-27T15:25:23.775 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.760+0000 7f4239744b80 1 -- 172.21.15.52:0/2444620804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df469745d0 msgr2=0x55df469769b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.760+0000 7f4239744b80 1 --2- 172.21.15.52:0/2444620804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df469745d0 0x55df469769b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f4220009dc0 tx=0x7f42200077e0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.760+0000 7f4239744b80 1 -- 172.21.15.52:0/2444620804 shutdown_connections 2023-11-27T15:25:23.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.760+0000 7f4239744b80 1 --2- 172.21.15.52:0/2444620804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df469657c0 0x55df46961970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.760+0000 7f4239744b80 1 --2- 172.21.15.52:0/2444620804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df469745d0 0x55df469769b0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.760+0000 7f4239744b80 1 --2- 172.21.15.52:0/2444620804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df46961eb0 0x55df46962320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.760+0000 7f4239744b80 1 -- 172.21.15.52:0/2444620804 >> 172.21.15.52:0/2444620804 conn(0x55df46957360 msgr2=0x55df4695cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:23.776 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.760+0000 7f4239744b80 1 -- 172.21.15.52:0/2444620804 shutdown_connections 2023-11-27T15:25:23.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.760+0000 7f4239744b80 1 -- 172.21.15.52:0/2444620804 wait complete. 2023-11-27T15:25:23.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.761+0000 7f4239744b80 1 -- start start 2023-11-27T15:25:23.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.761+0000 7f4239744b80 1 Processor -- start 2023-11-27T15:25:23.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.761+0000 7f4239744b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df46961eb0 0x55df46974030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.761+0000 7f4239744b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df469657c0 0x55df46971240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.761+0000 7f4239744b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df469745d0 0x55df46971980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.777 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.761+0000 7f4239744b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df468548e0 con 0x55df46961eb0 2023-11-27T15:25:23.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.761+0000 7f4239744b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df46854580 con 0x55df469657c0 2023-11-27T15:25:23.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.761+0000 7f4239744b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df46736920 con 0x55df469745d0 2023-11-27T15:25:23.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.761+0000 7f4239744b80 10 client.0 _ll_register_callbacks cb 0x55df46872780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:23.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.761+0000 7f4226ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df46961eb0 0x55df46974030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.761+0000 7f4226ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df46961eb0 0x55df46974030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55002/0 (socket says 172.21.15.52:55002) 2023-11-27T15:25:23.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.761+0000 7f4226ffd700 1 -- 172.21.15.52:0/3688237117 learned_addr learned my addr 172.21.15.52:0/3688237117 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:23.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.761+0000 7f42277fe700 1 --2- 172.21.15.52:0/3688237117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df469745d0 0x55df46971980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.761+0000 7f42267fc700 1 --2- 172.21.15.52:0/3688237117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df469657c0 0x55df46971240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.761+0000 7f4226ffd700 1 -- 172.21.15.52:0/3688237117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df469745d0 msgr2=0x55df46971980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.761+0000 7f4226ffd700 1 --2- 172.21.15.52:0/3688237117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df469745d0 0x55df46971980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.761+0000 7f4226ffd700 1 -- 172.21.15.52:0/3688237117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df469657c0 msgr2=0x55df46971240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.761+0000 7f4226ffd700 1 --2- 172.21.15.52:0/3688237117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df469657c0 0x55df46971240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.761+0000 7f4226ffd700 1 -- 172.21.15.52:0/3688237117 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4220009a20 con 0x55df46961eb0 2023-11-27T15:25:23.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.761+0000 7f42267fc700 1 --2- 172.21.15.52:0/3688237117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df469657c0 0x55df46971240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:23.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.761+0000 7f42277fe700 1 --2- 172.21.15.52:0/3688237117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df469745d0 0x55df46971980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:23.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.762+0000 7f4226ffd700 1 --2- 172.21.15.52:0/3688237117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df46961eb0 0x55df46974030 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f4218000c00 tx=0x7f421809abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b64a428eb19d332 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:23.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.762+0000 7f420aff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:25:23.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.762+0000 7f420aff5700 1 -- 172.21.15.52:0/3688237117 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421809b5b0 con 0x55df46961eb0 2023-11-27T15:25:23.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.762+0000 7f420aff5700 1 -- 172.21.15.52:0/3688237117 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f42180a3510 con 0x55df46961eb0 2023-11-27T15:25:23.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.762+0000 7f4239744b80 1 -- 172.21.15.52:0/3688237117 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df46954b20 con 0x55df46961eb0 2023-11-27T15:25:23.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.762+0000 7f420aff5700 1 -- 172.21.15.52:0/3688237117 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42180b9330 con 0x55df46961eb0 2023-11-27T15:25:23.780 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.762+0000 7f4239744b80 10 client.16665 fetch_fsmap learned FSMap version 30 2023-11-27T15:25:23.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.762+0000 7f4239744b80 1 -- 172.21.15.52:0/3688237117 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55df46879190 con 0x55df46961eb0 2023-11-27T15:25:23.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.762+0000 7f420aff5700 1 -- 172.21.15.52:0/3688237117 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55df46954b20 con 0x55df46961eb0 2023-11-27T15:25:23.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.762+0000 7f420aff5700 1 -- 172.21.15.52:0/3688237117 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f42180a3080 con 0x55df46961eb0 2023-11-27T15:25:23.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.762+0000 7f4239744b80 10 client.16665 fetch_fsmap finished waiting for FSMap version 30 2023-11-27T15:25:23.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.762+0000 7f4239744b80 10 client.16665 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:25:23.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.762+0000 7f4239744b80 20 client.16665 populate_metadata read hostname 'smithi052' 2023-11-27T15:25:23.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.762+0000 7f4239744b80 1 --2- 172.21.15.52:0/3688237117 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55df46983cc0 0x55df469860d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.762+0000 7f4239744b80 4 client.16665 mds_command: new command op to 24494 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:23.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.762+0000 7f4239744b80 1 -- 172.21.15.52:0/3688237117 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df468750a0 con 0x55df46983cc0 2023-11-27T15:25:23.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.763+0000 7f42267fc700 1 --2- 172.21.15.52:0/3688237117 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55df46983cc0 0x55df469860d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.763+0000 7f42267fc700 1 --2- 172.21.15.52:0/3688237117 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55df46983cc0 0x55df469860d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2a941ed39a97d29c server_cookie=ae89cd65f7f351b5 in_seq=0 out_seq=0 2023-11-27T15:25:23.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.763+0000 7f420aff5700 10 client.16665 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:25:23.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.764+0000 7f420aff5700 1 -- 172.21.15.52:0/3688237117 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55df468750a0 con 0x55df46983cc0 2023-11-27T15:25:23.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.764+0000 7f420aff5700 10 client.16665 handle_command_reply: tid=0 2023-11-27T15:25:23.783 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.764+0000 7f4239744b80 1 client.16665 shutdown 2023-11-27T15:25:23.783 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.764+0000 7f4239744b80 1 -- 172.21.15.52:0/3688237117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df46961eb0 msgr2=0x55df46974030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.783 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.764+0000 7f4239744b80 1 --2- 172.21.15.52:0/3688237117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df46961eb0 0x55df46974030 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f4218000c00 tx=0x7f421809abe0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.783 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.764+0000 7f4239744b80 1 -- 172.21.15.52:0/3688237117 shutdown_connections 2023-11-27T15:25:23.783 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.764+0000 7f4239744b80 1 --2- 172.21.15.52:0/3688237117 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55df46983cc0 0x55df469860d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.783 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.764+0000 7f4239744b80 1 --2- 172.21.15.52:0/3688237117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df469745d0 0x55df46971980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.783 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.764+0000 7f4239744b80 1 --2- 172.21.15.52:0/3688237117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df46961eb0 0x55df46974030 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.783 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.764+0000 7f420aff5700 0 client.16665 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:25:23.784 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.764+0000 7f4239744b80 1 --2- 172.21.15.52:0/3688237117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df469657c0 0x55df46971240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.784 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.764+0000 7f4239744b80 1 -- 172.21.15.52:0/3688237117 >> 172.21.15.52:0/3688237117 conn(0x55df46957360 msgr2=0x55df4695bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:23.784 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.764+0000 7f4239744b80 1 -- 172.21.15.52:0/3688237117 shutdown_connections 2023-11-27T15:25:23.784 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.764+0000 7f4239744b80 1 -- 172.21.15.52:0/3688237117 wait complete. 2023-11-27T15:25:23.784 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.764+0000 7f4239744b80 20 client.16665 trim_cache size 0 max 16384 2023-11-27T15:25:23.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.777+0000 7f4239744b80 1 Processor -- start 2023-11-27T15:25:23.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.777+0000 7f4239744b80 1 -- start start 2023-11-27T15:25:23.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.777+0000 7f4239744b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df46961eb0 0x55df469711f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.777+0000 7f4239744b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df469657c0 0x55df46973b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.777+0000 7f4239744b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df469745d0 0x55df4695c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.777+0000 7f4239744b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df46962bc0 con 0x55df469745d0 2023-11-27T15:25:23.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.777+0000 7f4239744b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df46972820 con 0x55df46961eb0 2023-11-27T15:25:23.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.777+0000 7f4239744b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df469653f0 con 0x55df469657c0 2023-11-27T15:25:23.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.777+0000 7f42277fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df469745d0 0x55df4695c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.777+0000 7f42277fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df469745d0 0x55df4695c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55014/0 (socket says 172.21.15.52:55014) 2023-11-27T15:25:23.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.777+0000 7f42277fe700 1 -- 172.21.15.52:0/3888678246 learned_addr learned my addr 172.21.15.52:0/3888678246 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:23.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.778+0000 7f4226ffd700 1 --2- 172.21.15.52:0/3888678246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df46961eb0 0x55df469711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.778+0000 7f42267fc700 1 --2- 172.21.15.52:0/3888678246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df469657c0 0x55df46973b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.778+0000 7f42277fe700 1 -- 172.21.15.52:0/3888678246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df469657c0 msgr2=0x55df46973b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.778+0000 7f42277fe700 1 --2- 172.21.15.52:0/3888678246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df469657c0 0x55df46973b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.778+0000 7f42277fe700 1 -- 172.21.15.52:0/3888678246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df46961eb0 msgr2=0x55df469711f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.778+0000 7f42277fe700 1 --2- 172.21.15.52:0/3888678246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df46961eb0 0x55df469711f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.778+0000 7f42277fe700 1 -- 172.21.15.52:0/3888678246 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df46980770 con 0x55df469745d0 2023-11-27T15:25:23.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.778+0000 7f42267fc700 1 --2- 172.21.15.52:0/3888678246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df469657c0 0x55df46973b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:23.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.778+0000 7f42277fe700 1 --2- 172.21.15.52:0/3888678246 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df469745d0 0x55df4695c560 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f4220018000 tx=0x7f42200098c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c109983fb632d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:23.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.778+0000 7f420dffb700 1 -- 172.21.15.52:0/3888678246 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4220004c50 con 0x55df469745d0 2023-11-27T15:25:23.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.778+0000 7f420dffb700 1 -- 172.21.15.52:0/3888678246 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4220023070 con 0x55df469745d0 2023-11-27T15:25:23.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.778+0000 7f420dffb700 1 -- 172.21.15.52:0/3888678246 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4220013770 con 0x55df469745d0 2023-11-27T15:25:23.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.778+0000 7f4239744b80 1 -- 172.21.15.52:0/3888678246 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df469745d0 msgr2=0x55df4695c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.778+0000 7f4239744b80 1 --2- 172.21.15.52:0/3888678246 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df469745d0 0x55df4695c560 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f4220018000 tx=0x7f42200098c0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.779+0000 7f4239744b80 1 -- 172.21.15.52:0/3888678246 shutdown_connections 2023-11-27T15:25:23.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.779+0000 7f4239744b80 1 --2- 172.21.15.52:0/3888678246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df46961eb0 0x55df469711f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.779+0000 7f4239744b80 1 --2- 172.21.15.52:0/3888678246 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df469745d0 0x55df4695c560 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.779+0000 7f4239744b80 1 --2- 172.21.15.52:0/3888678246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df469657c0 0x55df46973b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.779+0000 7f4239744b80 1 -- 172.21.15.52:0/3888678246 >> 172.21.15.52:0/3888678246 conn(0x55df46957360 msgr2=0x55df46970ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:23.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.779+0000 7f4239744b80 1 -- 172.21.15.52:0/3888678246 shutdown_connections 2023-11-27T15:25:23.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.779+0000 7f4239744b80 1 -- 172.21.15.52:0/3888678246 wait complete. 2023-11-27T15:25:23.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.779+0000 7f4239744b80 1 -- start start 2023-11-27T15:25:23.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.779+0000 7f4239744b80 1 Processor -- start 2023-11-27T15:25:23.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.779+0000 7f4239744b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df46961eb0 0x55df4697aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.779+0000 7f4239744b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df469657c0 0x55df4697b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.779+0000 7f4239744b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df469745d0 0x55df46979410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.779+0000 7f4239744b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df46979b20 con 0x55df46961eb0 2023-11-27T15:25:23.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.779+0000 7f4239744b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df46979c90 con 0x55df469657c0 2023-11-27T15:25:23.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.779+0000 7f4239744b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df46979e50 con 0x55df469745d0 2023-11-27T15:25:23.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.780+0000 7f4239744b80 10 client.0 _ll_register_callbacks cb 0x7f421809ac90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:23.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.780+0000 7f4226ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df46961eb0 0x55df4697aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.780+0000 7f4226ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df46961eb0 0x55df4697aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55016/0 (socket says 172.21.15.52:55016) 2023-11-27T15:25:23.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.780+0000 7f4226ffd700 1 -- 172.21.15.52:0/1520242489 learned_addr learned my addr 172.21.15.52:0/1520242489 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:23.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.780+0000 7f42267fc700 1 --2- 172.21.15.52:0/1520242489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df469657c0 0x55df4697b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.780+0000 7f4226ffd700 1 -- 172.21.15.52:0/1520242489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df469745d0 msgr2=0x55df46979410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.780+0000 7f4226ffd700 1 --2- 172.21.15.52:0/1520242489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df469745d0 0x55df46979410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.780+0000 7f4226ffd700 1 -- 172.21.15.52:0/1520242489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df469657c0 msgr2=0x55df4697b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.780+0000 7f4226ffd700 1 --2- 172.21.15.52:0/1520242489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df469657c0 0x55df4697b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.780+0000 7f4226ffd700 1 -- 172.21.15.52:0/1520242489 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df46734630 con 0x55df46961eb0 2023-11-27T15:25:23.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.780+0000 7f4226ffd700 1 --2- 172.21.15.52:0/1520242489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df46961eb0 0x55df4697aed0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f42180a3480 tx=0x7f4218022a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb71c869e874fa1a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:23.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.780+0000 7f420aff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:25:23.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.780+0000 7f420aff5700 1 -- 172.21.15.52:0/1520242489 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42180b1f00 con 0x55df46961eb0 2023-11-27T15:25:23.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.780+0000 7f420aff5700 1 -- 172.21.15.52:0/1520242489 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f42180a3510 con 0x55df46961eb0 2023-11-27T15:25:23.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.780+0000 7f4239744b80 1 -- 172.21.15.52:0/1520242489 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f422000f050 con 0x55df46961eb0 2023-11-27T15:25:23.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.780+0000 7f420aff5700 1 -- 172.21.15.52:0/1520242489 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42180cb330 con 0x55df46961eb0 2023-11-27T15:25:23.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.780+0000 7f4239744b80 10 client.16671 fetch_fsmap learned FSMap version 30 2023-11-27T15:25:23.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.780+0000 7f4239744b80 1 -- 172.21.15.52:0/1520242489 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55df466e0de0 con 0x55df46961eb0 2023-11-27T15:25:23.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.780+0000 7f420aff5700 1 -- 172.21.15.52:0/1520242489 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f422000f050 con 0x55df46961eb0 2023-11-27T15:25:23.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.781+0000 7f420aff5700 1 -- 172.21.15.52:0/1520242489 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f421807e020 con 0x55df46961eb0 2023-11-27T15:25:23.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.781+0000 7f4239744b80 10 client.16671 fetch_fsmap finished waiting for FSMap version 30 2023-11-27T15:25:23.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.781+0000 7f4239744b80 10 client.16671 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:25:23.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.781+0000 7f4239744b80 20 client.16671 populate_metadata read hostname 'smithi052' 2023-11-27T15:25:23.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.781+0000 7f4239744b80 1 --2- 172.21.15.52:0/1520242489 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55df469d8a30 0x55df469dae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.781+0000 7f4239744b80 4 client.16671 mds_command: new command op to 24494 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:25:23.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.781+0000 7f4239744b80 1 -- 172.21.15.52:0/1520242489 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55df46881660 con 0x55df469d8a30 2023-11-27T15:25:23.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.781+0000 7f42267fc700 1 --2- 172.21.15.52:0/1520242489 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55df469d8a30 0x55df469dae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.781+0000 7f420aff5700 10 client.16671 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:25:23.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.781+0000 7f42267fc700 1 --2- 172.21.15.52:0/1520242489 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55df469d8a30 0x55df469dae40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f09d49818a184ce1 server_cookie=fa1815e81e302161 in_seq=0 out_seq=0 2023-11-27T15:25:23.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.781+0000 7f420aff5700 1 -- 172.21.15.52:0/1520242489 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55df46881660 con 0x55df469d8a30 2023-11-27T15:25:23.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.781+0000 7f420aff5700 10 client.16671 handle_command_reply: tid=0 2023-11-27T15:25:23.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.782+0000 7f4239744b80 1 client.16671 shutdown 2023-11-27T15:25:23.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.782+0000 7f4239744b80 1 -- 172.21.15.52:0/1520242489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df46961eb0 msgr2=0x55df4697aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.782+0000 7f4239744b80 1 --2- 172.21.15.52:0/1520242489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df46961eb0 0x55df4697aed0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f42180a3480 tx=0x7f4218022a80 comp rx=0 tx=0).stop 2023-11-27T15:25:23.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.782+0000 7f4239744b80 1 -- 172.21.15.52:0/1520242489 shutdown_connections 2023-11-27T15:25:23.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.782+0000 7f4239744b80 1 --2- 172.21.15.52:0/1520242489 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55df469d8a30 0x55df469dae40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.782+0000 7f4239744b80 1 --2- 172.21.15.52:0/1520242489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55df469745d0 0x55df46979410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.782+0000 7f4239744b80 1 --2- 172.21.15.52:0/1520242489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df46961eb0 0x55df4697aed0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.782+0000 7f420aff5700 0 client.16671 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:25:23.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.782+0000 7f4239744b80 1 --2- 172.21.15.52:0/1520242489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55df469657c0 0x55df4697b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.782+0000 7f4239744b80 1 -- 172.21.15.52:0/1520242489 >> 172.21.15.52:0/1520242489 conn(0x55df46957360 msgr2=0x55df4696e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:23.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.782+0000 7f4239744b80 1 -- 172.21.15.52:0/1520242489 shutdown_connections 2023-11-27T15:25:23.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.782+0000 7f4239744b80 1 -- 172.21.15.52:0/1520242489 wait complete. 2023-11-27T15:25:23.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.782+0000 7f4239744b80 20 client.16671 trim_cache size 0 max 16384 2023-11-27T15:25:23.802 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:25:23.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.782+0000 7f422d5b8700 1 -- 172.21.15.52:0/2793140174 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f42100a1340 msgr2=0x7f42100a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.782+0000 7f422d5b8700 1 --2- 172.21.15.52:0/2793140174 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f42100a1340 0x7f42100a37f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f421c005950 tx=0x7f421c0058e0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.782+0000 7f422d5b8700 1 -- 172.21.15.52:0/2793140174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42280ffac0 msgr2=0x7f42281962f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.782+0000 7f422d5b8700 1 --2- 172.21.15.52:0/2793140174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42280ffac0 0x7f42281962f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f42180095c0 tx=0x7f421800f710 comp rx=0 tx=0).stop 2023-11-27T15:25:23.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.782+0000 7f422d5b8700 1 -- 172.21.15.52:0/2793140174 shutdown_connections 2023-11-27T15:25:23.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.782+0000 7f422d5b8700 1 --2- 172.21.15.52:0/2793140174 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f42100a1340 0x7f42100a37f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.782+0000 7f422d5b8700 1 --2- 172.21.15.52:0/2793140174 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f42281041f0 0x7f422819ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.782+0000 7f422d5b8700 1 --2- 172.21.15.52:0/2793140174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42280ffac0 0x7f42281962f0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.782+0000 7f422d5b8700 1 --2- 172.21.15.52:0/2793140174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4228100470 0x7f4228196830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.782+0000 7f422d5b8700 1 -- 172.21.15.52:0/2793140174 >> 172.21.15.52:0/2793140174 conn(0x7f42280763a0 msgr2=0x7f4228069a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:23.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.782+0000 7f422d5b8700 1 -- 172.21.15.52:0/2793140174 shutdown_connections 2023-11-27T15:25:23.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.783+0000 7f422d5b8700 1 -- 172.21.15.52:0/2793140174 wait complete. 2023-11-27T15:25:23.805 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-27T15:25:23.934 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.930+0000 7f8041b8e700 1 -- 172.21.15.52:0/4094601248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f803c104500 msgr2=0x7f803c10c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.930+0000 7f8041b8e700 1 --2- 172.21.15.52:0/4094601248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f803c104500 0x7f803c10c0d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f803400b0d0 tx=0x7f803400b3e0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.930+0000 7f8041b8e700 1 -- 172.21.15.52:0/4094601248 shutdown_connections 2023-11-27T15:25:23.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.930+0000 7f8041b8e700 1 --2- 172.21.15.52:0/4094601248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f803c0ff4c0 0x7f803c0ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.930+0000 7f8041b8e700 1 --2- 172.21.15.52:0/4094601248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f803c104500 0x7f803c10c0d0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.930+0000 7f8041b8e700 1 --2- 172.21.15.52:0/4094601248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f803c0ffe60 0x7f803c103e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.930+0000 7f8041b8e700 1 -- 172.21.15.52:0/4094601248 >> 172.21.15.52:0/4094601248 conn(0x7f803c076200 msgr2=0x7f803c076600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:23.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.930+0000 7f8041b8e700 1 -- 172.21.15.52:0/4094601248 shutdown_connections 2023-11-27T15:25:23.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.930+0000 7f8041b8e700 1 -- 172.21.15.52:0/4094601248 wait complete. 2023-11-27T15:25:23.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.931+0000 7f8041b8e700 1 Processor -- start 2023-11-27T15:25:23.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.931+0000 7f8041b8e700 1 -- start start 2023-11-27T15:25:23.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.931+0000 7f8041b8e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f803c0ff4c0 0x7f803c19e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.931+0000 7f8041b8e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f803c0ffe60 0x7f803c19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.931+0000 7f8041b8e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f803c104500 0x7f803c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.931+0000 7f8041b8e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803c198c00 con 0x7f803c104500 2023-11-27T15:25:23.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.931+0000 7f8041b8e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803c198d70 con 0x7f803c0ff4c0 2023-11-27T15:25:23.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.931+0000 7f8041b8e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803c198f30 con 0x7f803c0ffe60 2023-11-27T15:25:23.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.932+0000 7f803bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f803c104500 0x7f803c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.932+0000 7f803b7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f803c0ff4c0 0x7f803c19e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.932+0000 7f803b7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f803c0ff4c0 0x7f803c19e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:44688/0 (socket says 172.21.15.52:44688) 2023-11-27T15:25:23.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.932+0000 7f803b7fe700 1 -- 172.21.15.52:0/2026293870 learned_addr learned my addr 172.21.15.52:0/2026293870 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:23.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.932+0000 7f803affd700 1 --2- 172.21.15.52:0/2026293870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f803c0ffe60 0x7f803c19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.932+0000 7f803bfff700 1 -- 172.21.15.52:0/2026293870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f803c0ffe60 msgr2=0x7f803c19ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.932+0000 7f803bfff700 1 --2- 172.21.15.52:0/2026293870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f803c0ffe60 0x7f803c19ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.932+0000 7f803bfff700 1 -- 172.21.15.52:0/2026293870 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f803c0ff4c0 msgr2=0x7f803c19e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.932+0000 7f803bfff700 1 --2- 172.21.15.52:0/2026293870 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f803c0ff4c0 0x7f803c19e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.932+0000 7f803bfff700 1 -- 172.21.15.52:0/2026293870 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c009520 con 0x7f803c104500 2023-11-27T15:25:23.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.932+0000 7f803b7fe700 1 --2- 172.21.15.52:0/2026293870 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f803c0ff4c0 0x7f803c19e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:23.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.932+0000 7f803affd700 1 --2- 172.21.15.52:0/2026293870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f803c0ffe60 0x7f803c19ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:23.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.932+0000 7f803bfff700 1 --2- 172.21.15.52:0/2026293870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f803c104500 0x7f803c198590 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f803400b770 tx=0x7f8034007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:23.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.932+0000 7f8038ff9700 1 -- 172.21.15.52:0/2026293870 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803400e040 con 0x7f803c104500 2023-11-27T15:25:23.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.932+0000 7f8038ff9700 1 -- 172.21.15.52:0/2026293870 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f803401d930 con 0x7f803c104500 2023-11-27T15:25:23.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.932+0000 7f8041b8e700 1 -- 172.21.15.52:0/2026293870 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8034009d00 con 0x7f803c104500 2023-11-27T15:25:23.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.932+0000 7f8038ff9700 1 -- 172.21.15.52:0/2026293870 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803401b920 con 0x7f803c104500 2023-11-27T15:25:23.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.932+0000 7f8041b8e700 1 -- 172.21.15.52:0/2026293870 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f803c10f2e0 con 0x7f803c104500 2023-11-27T15:25:23.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.935+0000 7f8038ff9700 1 -- 172.21.15.52:0/2026293870 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f803401daa0 con 0x7f803c104500 2023-11-27T15:25:23.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.935+0000 7f8038ff9700 1 --2- 172.21.15.52:0/2026293870 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f80240a12c0 0x7f80240a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.948 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.935+0000 7f8038ff9700 1 -- 172.21.15.52:0/2026293870 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f80340ce2a0 con 0x7f803c104500 2023-11-27T15:25:23.948 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.935+0000 7f803b7fe700 1 --2- 172.21.15.52:0/2026293870 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f80240a12c0 0x7f80240a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.948 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.936+0000 7f803b7fe700 1 --2- 172.21.15.52:0/2026293870 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f80240a12c0 0x7f80240a3770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f802c009fd0 tx=0x7f802c009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:23.948 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.936+0000 7f804dd1ab80 1 Processor -- start 2023-11-27T15:25:23.948 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.936+0000 7f804dd1ab80 1 -- start start 2023-11-27T15:25:23.948 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.936+0000 7f804dd1ab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559dcfd84b60 0x559dcfd84f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.949 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.936+0000 7f804dd1ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559dcfd85470 0x559dcfd89370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.949 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.936+0000 7f804dd1ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559dcfd898b0 0x559dcfd89d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.949 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.936+0000 7f804dd1ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dcfc8a060 con 0x559dcfd898b0 2023-11-27T15:25:23.949 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.936+0000 7f804dd1ab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dcfcdc8a0 con 0x559dcfd84b60 2023-11-27T15:25:23.949 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.936+0000 7f804dd1ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dcfcd5d30 con 0x559dcfd85470 2023-11-27T15:25:23.949 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.936+0000 7f803bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559dcfd898b0 0x559dcfd89d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.949 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.936+0000 7f803bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559dcfd898b0 0x559dcfd89d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55042/0 (socket says 172.21.15.52:55042) 2023-11-27T15:25:23.949 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.936+0000 7f803bfff700 1 -- 172.21.15.52:0/3519827171 learned_addr learned my addr 172.21.15.52:0/3519827171 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:23.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.936+0000 7f803b7fe700 1 --2- 172.21.15.52:0/3519827171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559dcfd84b60 0x559dcfd84f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.937+0000 7f803affd700 1 --2- 172.21.15.52:0/3519827171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559dcfd85470 0x559dcfd89370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.937+0000 7f803bfff700 1 -- 172.21.15.52:0/3519827171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559dcfd85470 msgr2=0x559dcfd89370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.937+0000 7f803bfff700 1 --2- 172.21.15.52:0/3519827171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559dcfd85470 0x559dcfd89370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.937+0000 7f803bfff700 1 -- 172.21.15.52:0/3519827171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559dcfd84b60 msgr2=0x559dcfd84f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.937+0000 7f803bfff700 1 --2- 172.21.15.52:0/3519827171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559dcfd84b60 0x559dcfd84f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.937+0000 7f803bfff700 1 -- 172.21.15.52:0/3519827171 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559dcfb57630 con 0x559dcfd898b0 2023-11-27T15:25:23.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.937+0000 7f803affd700 1 --2- 172.21.15.52:0/3519827171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559dcfd85470 0x559dcfd89370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:23.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.937+0000 7f803bfff700 1 --2- 172.21.15.52:0/3519827171 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559dcfd898b0 0x559dcfd89d00 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f803409b360 tx=0x7f80340099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d17151bb8b85fd server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:23.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.937+0000 7f8021ffb700 1 -- 172.21.15.52:0/3519827171 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80340ce0d0 con 0x559dcfd898b0 2023-11-27T15:25:23.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.937+0000 7f8021ffb700 1 -- 172.21.15.52:0/3519827171 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f803409f400 con 0x559dcfd898b0 2023-11-27T15:25:23.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.937+0000 7f8021ffb700 1 -- 172.21.15.52:0/3519827171 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8034097cc0 con 0x559dcfd898b0 2023-11-27T15:25:23.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.938+0000 7f804dd1ab80 1 -- 172.21.15.52:0/3519827171 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559dcfd898b0 msgr2=0x559dcfd89d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.938+0000 7f804dd1ab80 1 --2- 172.21.15.52:0/3519827171 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559dcfd898b0 0x559dcfd89d00 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f803409b360 tx=0x7f80340099e0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.938+0000 7f804dd1ab80 1 -- 172.21.15.52:0/3519827171 shutdown_connections 2023-11-27T15:25:23.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.938+0000 7f804dd1ab80 1 --2- 172.21.15.52:0/3519827171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559dcfd84b60 0x559dcfd84f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.938+0000 7f804dd1ab80 1 --2- 172.21.15.52:0/3519827171 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559dcfd898b0 0x559dcfd89d00 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.938+0000 7f804dd1ab80 1 --2- 172.21.15.52:0/3519827171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559dcfd85470 0x559dcfd89370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.938+0000 7f804dd1ab80 1 -- 172.21.15.52:0/3519827171 >> 172.21.15.52:0/3519827171 conn(0x559dcfd7a360 msgr2=0x559dcfd7fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:23.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.938+0000 7f804dd1ab80 1 -- 172.21.15.52:0/3519827171 shutdown_connections 2023-11-27T15:25:23.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.938+0000 7f804dd1ab80 1 -- 172.21.15.52:0/3519827171 wait complete. 2023-11-27T15:25:23.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.938+0000 7f804dd1ab80 1 -- start start 2023-11-27T15:25:23.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.938+0000 7f804dd1ab80 1 Processor -- start 2023-11-27T15:25:23.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.938+0000 7f804dd1ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559dcfd84b60 0x559dcfd91050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.938+0000 7f804dd1ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559dcfd85470 0x559dcfd8e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.938+0000 7f804dd1ab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559dcfd898b0 0x559dcfd8e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.939+0000 7f804dd1ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dcfc778e0 con 0x559dcfd84b60 2023-11-27T15:25:23.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.939+0000 7f804dd1ab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dcfc77580 con 0x559dcfd898b0 2023-11-27T15:25:23.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.939+0000 7f804dd1ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dcfb59920 con 0x559dcfd85470 2023-11-27T15:25:23.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.939+0000 7f804dd1ab80 10 client.0 _ll_register_callbacks cb 0x559dcfc95780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:23.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.939+0000 7f803b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559dcfd84b60 0x559dcfd91050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.939+0000 7f803b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559dcfd84b60 0x559dcfd91050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55044/0 (socket says 172.21.15.52:55044) 2023-11-27T15:25:23.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.939+0000 7f803b7fe700 1 -- 172.21.15.52:0/1271022625 learned_addr learned my addr 172.21.15.52:0/1271022625 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:23.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.939+0000 7f803bfff700 1 --2- 172.21.15.52:0/1271022625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559dcfd898b0 0x559dcfd8e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.939+0000 7f803affd700 1 --2- 172.21.15.52:0/1271022625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559dcfd85470 0x559dcfd8e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.939+0000 7f803b7fe700 1 -- 172.21.15.52:0/1271022625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559dcfd85470 msgr2=0x559dcfd8e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.939+0000 7f803b7fe700 1 --2- 172.21.15.52:0/1271022625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559dcfd85470 0x559dcfd8e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.939+0000 7f803b7fe700 1 -- 172.21.15.52:0/1271022625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559dcfd898b0 msgr2=0x559dcfd8e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.939+0000 7f803b7fe700 1 --2- 172.21.15.52:0/1271022625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559dcfd898b0 0x559dcfd8e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.939+0000 7f803b7fe700 1 -- 172.21.15.52:0/1271022625 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f803c10f2e0 con 0x559dcfd84b60 2023-11-27T15:25:23.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.939+0000 7f803bfff700 1 --2- 172.21.15.52:0/1271022625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559dcfd898b0 0x559dcfd8e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:23.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.939+0000 7f803b7fe700 1 --2- 172.21.15.52:0/1271022625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559dcfd84b60 0x559dcfd91050 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f802c00fc90 tx=0x7f802c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd650c0a94623db5 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:23.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.939+0000 7f801affd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:25:23.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.939+0000 7f801affd700 1 -- 172.21.15.52:0/1271022625 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c021070 con 0x559dcfd84b60 2023-11-27T15:25:23.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.939+0000 7f801affd700 1 -- 172.21.15.52:0/1271022625 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f802c01cb30 con 0x559dcfd84b60 2023-11-27T15:25:23.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.939+0000 7f804dd1ab80 1 -- 172.21.15.52:0/1271022625 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559dcfd77b20 con 0x559dcfd84b60 2023-11-27T15:25:23.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.939+0000 7f801affd700 1 -- 172.21.15.52:0/1271022625 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f802c01ccf0 con 0x559dcfd84b60 2023-11-27T15:25:23.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.939+0000 7f804dd1ab80 10 client.16683 fetch_fsmap learned FSMap version 30 2023-11-27T15:25:23.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.939+0000 7f804dd1ab80 1 -- 172.21.15.52:0/1271022625 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559dcfc9c190 con 0x559dcfd84b60 2023-11-27T15:25:23.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.940+0000 7f801affd700 1 -- 172.21.15.52:0/1271022625 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f802c032020 con 0x559dcfd84b60 2023-11-27T15:25:23.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.940+0000 7f801affd700 1 -- 172.21.15.52:0/1271022625 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f802c01b410 con 0x559dcfd84b60 2023-11-27T15:25:23.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.940+0000 7f804dd1ab80 10 client.16683 fetch_fsmap finished waiting for FSMap version 30 2023-11-27T15:25:23.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.940+0000 7f804dd1ab80 10 client.16683 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:23.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.940+0000 7f804dd1ab80 20 client.16683 populate_metadata read hostname 'smithi052' 2023-11-27T15:25:23.956 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.940+0000 7f804dd1ab80 1 --2- 172.21.15.52:0/1271022625 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x559dcfd9e6d0 0x559dcfda0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.957 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.940+0000 7f804dd1ab80 4 client.16683 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:23.957 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.940+0000 7f804dd1ab80 1 -- 172.21.15.52:0/1271022625 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559dcfc980a0 con 0x559dcfd9e6d0 2023-11-27T15:25:23.957 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.940+0000 7f803affd700 1 --2- 172.21.15.52:0/1271022625 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x559dcfd9e6d0 0x559dcfda0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.957 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.941+0000 7f803affd700 1 --2- 172.21.15.52:0/1271022625 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x559dcfd9e6d0 0x559dcfda0ae0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=237fdc5630b65a92 server_cookie=d8ef5268ac5cfc5d in_seq=0 out_seq=0 2023-11-27T15:25:23.957 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.941+0000 7f801affd700 10 client.16683 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:23.957 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.941+0000 7f801affd700 1 -- 172.21.15.52:0/1271022625 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559dcfc980a0 con 0x559dcfd9e6d0 2023-11-27T15:25:23.957 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.941+0000 7f801affd700 10 client.16683 handle_command_reply: tid=0 2023-11-27T15:25:23.958 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.941+0000 7f804dd1ab80 1 client.16683 shutdown 2023-11-27T15:25:23.958 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.941+0000 7f804dd1ab80 1 -- 172.21.15.52:0/1271022625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559dcfd84b60 msgr2=0x559dcfd91050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.958 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.942+0000 7f804dd1ab80 1 --2- 172.21.15.52:0/1271022625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559dcfd84b60 0x559dcfd91050 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f802c00fc90 tx=0x7f802c0047c0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.958 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.942+0000 7f804dd1ab80 1 -- 172.21.15.52:0/1271022625 shutdown_connections 2023-11-27T15:25:23.958 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.942+0000 7f804dd1ab80 1 --2- 172.21.15.52:0/1271022625 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x559dcfd9e6d0 0x559dcfda0ae0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.958 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.942+0000 7f804dd1ab80 1 --2- 172.21.15.52:0/1271022625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559dcfd898b0 0x559dcfd8e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.958 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.942+0000 7f801affd700 0 client.16683 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:23.958 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.942+0000 7f804dd1ab80 1 --2- 172.21.15.52:0/1271022625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559dcfd84b60 0x559dcfd91050 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.959 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.942+0000 7f804dd1ab80 1 --2- 172.21.15.52:0/1271022625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559dcfd85470 0x559dcfd8e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.959 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.942+0000 7f804dd1ab80 1 -- 172.21.15.52:0/1271022625 >> 172.21.15.52:0/1271022625 conn(0x559dcfd7a360 msgr2=0x559dcfd8ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:23.959 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.942+0000 7f804dd1ab80 1 -- 172.21.15.52:0/1271022625 shutdown_connections 2023-11-27T15:25:23.959 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.942+0000 7f804dd1ab80 1 -- 172.21.15.52:0/1271022625 wait complete. 2023-11-27T15:25:23.959 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.942+0000 7f804dd1ab80 20 client.16683 trim_cache size 0 max 16384 2023-11-27T15:25:23.964 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.955+0000 7f804dd1ab80 1 Processor -- start 2023-11-27T15:25:23.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.955+0000 7f804dd1ab80 1 -- start start 2023-11-27T15:25:23.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.955+0000 7f804dd1ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559dcfd84b60 0x559dcfd95750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.955+0000 7f804dd1ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559dcfd85470 0x559dcfd90bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.955+0000 7f804dd1ab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559dcfd898b0 0x559dcfd7f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.955+0000 7f804dd1ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dcfd85bc0 con 0x559dcfd84b60 2023-11-27T15:25:23.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.955+0000 7f804dd1ab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dcfd8f260 con 0x559dcfd898b0 2023-11-27T15:25:23.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.955+0000 7f804dd1ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dcfd8f8b0 con 0x559dcfd85470 2023-11-27T15:25:23.966 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.955+0000 7f803b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559dcfd84b60 0x559dcfd95750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.966 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.955+0000 7f803b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559dcfd84b60 0x559dcfd95750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55052/0 (socket says 172.21.15.52:55052) 2023-11-27T15:25:23.966 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.955+0000 7f803b7fe700 1 -- 172.21.15.52:0/3225149759 learned_addr learned my addr 172.21.15.52:0/3225149759 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:23.966 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.955+0000 7f803affd700 1 --2- 172.21.15.52:0/3225149759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559dcfd85470 0x559dcfd90bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.966 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.955+0000 7f803bfff700 1 --2- 172.21.15.52:0/3225149759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559dcfd898b0 0x559dcfd7f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.966 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.955+0000 7f803b7fe700 1 -- 172.21.15.52:0/3225149759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559dcfd85470 msgr2=0x559dcfd90bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.966 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.955+0000 7f803b7fe700 1 --2- 172.21.15.52:0/3225149759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559dcfd85470 0x559dcfd90bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.955+0000 7f803b7fe700 1 -- 172.21.15.52:0/3225149759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559dcfd898b0 msgr2=0x559dcfd7f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.955+0000 7f803b7fe700 1 --2- 172.21.15.52:0/3225149759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559dcfd898b0 0x559dcfd7f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.955+0000 7f803b7fe700 1 -- 172.21.15.52:0/3225149759 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559dcfd7bb00 con 0x559dcfd84b60 2023-11-27T15:25:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.955+0000 7f803affd700 1 --2- 172.21.15.52:0/3225149759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559dcfd85470 0x559dcfd90bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.955+0000 7f803bfff700 1 --2- 172.21.15.52:0/3225149759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559dcfd898b0 0x559dcfd7f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.956+0000 7f803b7fe700 1 --2- 172.21.15.52:0/3225149759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559dcfd84b60 0x559dcfd95750 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f802c005d60 tx=0x7f802c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8cb9884fa0f571 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.956+0000 7f80227fc700 1 -- 172.21.15.52:0/3225149759 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c021070 con 0x559dcfd84b60 2023-11-27T15:25:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.956+0000 7f80227fc700 1 -- 172.21.15.52:0/3225149759 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f802c032070 con 0x559dcfd84b60 2023-11-27T15:25:23.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.956+0000 7f80227fc700 1 -- 172.21.15.52:0/3225149759 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c02e6a0 con 0x559dcfd84b60 2023-11-27T15:25:23.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.956+0000 7f804dd1ab80 1 -- 172.21.15.52:0/3225149759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559dcfd84b60 msgr2=0x559dcfd95750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.956+0000 7f804dd1ab80 1 --2- 172.21.15.52:0/3225149759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559dcfd84b60 0x559dcfd95750 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f802c005d60 tx=0x7f802c0046d0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.956+0000 7f804dd1ab80 1 -- 172.21.15.52:0/3225149759 shutdown_connections 2023-11-27T15:25:23.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.956+0000 7f804dd1ab80 1 --2- 172.21.15.52:0/3225149759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559dcfd898b0 0x559dcfd7f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.956+0000 7f804dd1ab80 1 --2- 172.21.15.52:0/3225149759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559dcfd84b60 0x559dcfd95750 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.956+0000 7f804dd1ab80 1 --2- 172.21.15.52:0/3225149759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559dcfd85470 0x559dcfd90bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.956+0000 7f804dd1ab80 1 -- 172.21.15.52:0/3225149759 >> 172.21.15.52:0/3225149759 conn(0x559dcfd7a360 msgr2=0x559dcfd9a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:23.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.956+0000 7f804dd1ab80 1 -- 172.21.15.52:0/3225149759 shutdown_connections 2023-11-27T15:25:23.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.956+0000 7f804dd1ab80 1 -- 172.21.15.52:0/3225149759 wait complete. 2023-11-27T15:25:23.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.957+0000 7f804dd1ab80 1 -- start start 2023-11-27T15:25:23.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.957+0000 7f804dd1ab80 1 Processor -- start 2023-11-27T15:25:23.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.957+0000 7f804dd1ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559dcfd84b60 0x559dcfd8dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.957+0000 7f804dd1ab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559dcfd85470 0x559dcfd8e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.957+0000 7f804dd1ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559dcfd898b0 0x559dcfd8ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.957+0000 7f804dd1ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dcfd95d30 con 0x559dcfd84b60 2023-11-27T15:25:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.957+0000 7f804dd1ab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dcfd8cfc0 con 0x559dcfd85470 2023-11-27T15:25:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.957+0000 7f804dd1ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dcfd8d180 con 0x559dcfd898b0 2023-11-27T15:25:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.957+0000 7f804dd1ab80 10 client.0 _ll_register_callbacks cb 0x7f802c00fa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.957+0000 7f803b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559dcfd84b60 0x559dcfd8dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.957+0000 7f803b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559dcfd84b60 0x559dcfd8dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55058/0 (socket says 172.21.15.52:55058) 2023-11-27T15:25:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.957+0000 7f803b7fe700 1 -- 172.21.15.52:0/1522920030 learned_addr learned my addr 172.21.15.52:0/1522920030 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.957+0000 7f803affd700 1 --2- 172.21.15.52:0/1522920030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559dcfd85470 0x559dcfd8e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.971 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.957+0000 7f803bfff700 1 --2- 172.21.15.52:0/1522920030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559dcfd898b0 0x559dcfd8ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.971 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.957+0000 7f803b7fe700 1 -- 172.21.15.52:0/1522920030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559dcfd898b0 msgr2=0x559dcfd8ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.971 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.958+0000 7f803b7fe700 1 --2- 172.21.15.52:0/1522920030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559dcfd898b0 0x559dcfd8ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.971 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.958+0000 7f803b7fe700 1 -- 172.21.15.52:0/1522920030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559dcfd85470 msgr2=0x559dcfd8e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:23.971 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.958+0000 7f803b7fe700 1 --2- 172.21.15.52:0/1522920030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559dcfd85470 0x559dcfd8e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.971 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.958+0000 7f803b7fe700 1 -- 172.21.15.52:0/1522920030 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559dcfc9c190 con 0x559dcfd84b60 2023-11-27T15:25:23.971 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.958+0000 7f803b7fe700 1 --2- 172.21.15.52:0/1522920030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559dcfd84b60 0x559dcfd8dbe0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f802c016040 tx=0x7f802c01bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22921810c08c0d07 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:23.971 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.958+0000 7f801affd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:25:23.972 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.958+0000 7f801affd700 1 -- 172.21.15.52:0/1522920030 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c021070 con 0x559dcfd84b60 2023-11-27T15:25:23.972 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.958+0000 7f801affd700 1 -- 172.21.15.52:0/1522920030 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f802c033080 con 0x559dcfd84b60 2023-11-27T15:25:23.972 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.958+0000 7f804dd1ab80 1 -- 172.21.15.52:0/1522920030 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f803409f400 con 0x559dcfd84b60 2023-11-27T15:25:23.972 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.958+0000 7f801affd700 1 -- 172.21.15.52:0/1522920030 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f802c01cc90 con 0x559dcfd84b60 2023-11-27T15:25:23.972 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.958+0000 7f804dd1ab80 10 client.16689 fetch_fsmap learned FSMap version 30 2023-11-27T15:25:23.972 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.958+0000 7f804dd1ab80 1 -- 172.21.15.52:0/1522920030 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559dcfb03de0 con 0x559dcfd84b60 2023-11-27T15:25:23.972 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.958+0000 7f801affd700 1 -- 172.21.15.52:0/1522920030 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f802c045020 con 0x559dcfd84b60 2023-11-27T15:25:23.972 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.958+0000 7f801affd700 1 -- 172.21.15.52:0/1522920030 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f802c004d40 con 0x559dcfd84b60 2023-11-27T15:25:23.973 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.958+0000 7f804dd1ab80 10 client.16689 fetch_fsmap finished waiting for FSMap version 30 2023-11-27T15:25:23.973 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.958+0000 7f804dd1ab80 10 client.16689 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:23.973 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.958+0000 7f804dd1ab80 20 client.16689 populate_metadata read hostname 'smithi052' 2023-11-27T15:25:23.973 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.958+0000 7f804dd1ab80 1 --2- 172.21.15.52:0/1522920030 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x559dcfdf2f50 0x559dcfdf5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:23.973 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.958+0000 7f804dd1ab80 4 client.16689 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:25:23.973 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.958+0000 7f804dd1ab80 1 -- 172.21.15.52:0/1522920030 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559dcfca4660 con 0x559dcfdf2f50 2023-11-27T15:25:23.973 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.959+0000 7f803affd700 1 --2- 172.21.15.52:0/1522920030 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x559dcfdf2f50 0x559dcfdf5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:23.973 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.959+0000 7f801affd700 10 client.16689 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:23.974 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.959+0000 7f803affd700 1 --2- 172.21.15.52:0/1522920030 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x559dcfdf2f50 0x559dcfdf5360 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=135618d625990558 server_cookie=7ecb9027519dac39 in_seq=0 out_seq=0 2023-11-27T15:25:23.974 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.959+0000 7f801affd700 1 -- 172.21.15.52:0/1522920030 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559dcfca4660 con 0x559dcfdf2f50 2023-11-27T15:25:23.974 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.959+0000 7f801affd700 10 client.16689 handle_command_reply: tid=0 2023-11-27T15:25:23.974 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.959+0000 7f804dd1ab80 1 client.16689 shutdown 2023-11-27T15:25:23.974 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.959+0000 7f804dd1ab80 1 -- 172.21.15.52:0/1522920030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559dcfd84b60 msgr2=0x559dcfd8dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.974 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.959+0000 7f804dd1ab80 1 --2- 172.21.15.52:0/1522920030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559dcfd84b60 0x559dcfd8dbe0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f802c016040 tx=0x7f802c01bf30 comp rx=0 tx=0).stop 2023-11-27T15:25:23.974 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.959+0000 7f804dd1ab80 1 -- 172.21.15.52:0/1522920030 shutdown_connections 2023-11-27T15:25:23.974 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.959+0000 7f804dd1ab80 1 --2- 172.21.15.52:0/1522920030 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x559dcfdf2f50 0x559dcfdf5360 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.975 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.959+0000 7f804dd1ab80 1 --2- 172.21.15.52:0/1522920030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559dcfd898b0 0x559dcfd8ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.975 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.959+0000 7f804dd1ab80 1 --2- 172.21.15.52:0/1522920030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559dcfd84b60 0x559dcfd8dbe0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.975 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.959+0000 7f804dd1ab80 1 --2- 172.21.15.52:0/1522920030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559dcfd85470 0x559dcfd8e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.975 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.959+0000 7f801affd700 0 client.16689 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:23.975 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.959+0000 7f804dd1ab80 1 -- 172.21.15.52:0/1522920030 >> 172.21.15.52:0/1522920030 conn(0x559dcfd7a360 msgr2=0x559dcfd8b570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:23.975 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.960+0000 7f804dd1ab80 1 -- 172.21.15.52:0/1522920030 shutdown_connections 2023-11-27T15:25:23.975 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.960+0000 7f804dd1ab80 1 -- 172.21.15.52:0/1522920030 wait complete. 2023-11-27T15:25:23.976 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.960+0000 7f804dd1ab80 20 client.16689 trim_cache size 0 max 16384 2023-11-27T15:25:23.976 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:25:23.977 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.960+0000 7f8041b8e700 1 -- 172.21.15.52:0/2026293870 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f80240a12c0 msgr2=0x7f80240a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.977 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.960+0000 7f8041b8e700 1 --2- 172.21.15.52:0/2026293870 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f80240a12c0 0x7f80240a3770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f802c009fd0 tx=0x7f802c009260 comp rx=0 tx=0).stop 2023-11-27T15:25:23.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.960+0000 7f8041b8e700 1 -- 172.21.15.52:0/2026293870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f803c104500 msgr2=0x7f803c198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:23.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.960+0000 7f8041b8e700 1 --2- 172.21.15.52:0/2026293870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f803c104500 0x7f803c198590 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f803400b770 tx=0x7f8034007870 comp rx=0 tx=0).stop 2023-11-27T15:25:23.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.960+0000 7f8041b8e700 1 -- 172.21.15.52:0/2026293870 shutdown_connections 2023-11-27T15:25:23.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.960+0000 7f8041b8e700 1 --2- 172.21.15.52:0/2026293870 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f80240a12c0 0x7f80240a3770 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.960+0000 7f8041b8e700 1 --2- 172.21.15.52:0/2026293870 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f803c0ff4c0 0x7f803c19e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.960+0000 7f8041b8e700 1 --2- 172.21.15.52:0/2026293870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f803c104500 0x7f803c198590 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.960+0000 7f8041b8e700 1 --2- 172.21.15.52:0/2026293870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f803c0ffe60 0x7f803c19ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:23.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.960+0000 7f8041b8e700 1 -- 172.21.15.52:0/2026293870 >> 172.21.15.52:0/2026293870 conn(0x7f803c076200 msgr2=0x7f803c069ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:23.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.960+0000 7f8041b8e700 1 -- 172.21.15.52:0/2026293870 shutdown_connections 2023-11-27T15:25:23.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:23.960+0000 7f8041b8e700 1 -- 172.21.15.52:0/2026293870 wait complete. 2023-11-27T15:25:24.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:24 smithi150 ceph-mon[100559]: pgmap v231: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-11-27T15:25:24.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:24 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/4065274981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:25:24.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:24 smithi102 ceph-mon[100296]: pgmap v231: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-11-27T15:25:24.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:24 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/4065274981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:25:24.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:24 smithi052 ceph-mon[99056]: pgmap v231: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-11-27T15:25:24.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:24 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/4065274981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:25:24.980 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-27T15:25:24.980 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-27T15:25:25.124 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.120+0000 7f3f94a24700 1 -- 172.21.15.52:0/1248947257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f900ff450 msgr2=0x7f3f900ff820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:25.124 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.120+0000 7f3f94a24700 1 --2- 172.21.15.52:0/1248947257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f900ff450 0x7f3f900ff820 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f3f800099c0 tx=0x7f3f80009cd0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.124 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.120+0000 7f3f94a24700 1 -- 172.21.15.52:0/1248947257 shutdown_connections 2023-11-27T15:25:25.125 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.120+0000 7f3f94a24700 1 --2- 172.21.15.52:0/1248947257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f90104490 0x7f3f9010c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.125 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.120+0000 7f3f94a24700 1 --2- 172.21.15.52:0/1248947257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f900ff450 0x7f3f900ff820 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.125 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.120+0000 7f3f94a24700 1 --2- 172.21.15.52:0/1248947257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f900ffdf0 0x7f3f90103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.125 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.120+0000 7f3f94a24700 1 -- 172.21.15.52:0/1248947257 >> 172.21.15.52:0/1248947257 conn(0x7f3f900770b0 msgr2=0x7f3f900774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:25.125 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.120+0000 7f3f94a24700 1 -- 172.21.15.52:0/1248947257 shutdown_connections 2023-11-27T15:25:25.125 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.120+0000 7f3f94a24700 1 -- 172.21.15.52:0/1248947257 wait complete. 2023-11-27T15:25:25.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.121+0000 7f3f94a24700 1 Processor -- start 2023-11-27T15:25:25.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.121+0000 7f3f94a24700 1 -- start start 2023-11-27T15:25:25.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.122+0000 7f3f94a24700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f900ff450 0x7f3f901953b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.122+0000 7f3f94a24700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f900ffdf0 0x7f3f901958f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.122+0000 7f3f94a24700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f90104490 0x7f3f90199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.122+0000 7f3f94a24700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9019a390 con 0x7f3f900ff450 2023-11-27T15:25:25.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.122+0000 7f3f94a24700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9019a500 con 0x7f3f90104490 2023-11-27T15:25:25.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.122+0000 7f3f94a24700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9019a6c0 con 0x7f3f900ffdf0 2023-11-27T15:25:25.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.122+0000 7f3f8e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f900ff450 0x7f3f901953b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.122+0000 7f3f8e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f900ff450 0x7f3f901953b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55072/0 (socket says 172.21.15.52:55072) 2023-11-27T15:25:25.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.122+0000 7f3f8e59c700 1 -- 172.21.15.52:0/2857561050 learned_addr learned my addr 172.21.15.52:0/2857561050 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:25.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.122+0000 7f3f8dd9b700 1 --2- 172.21.15.52:0/2857561050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f900ffdf0 0x7f3f901958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.122+0000 7f3f8ed9d700 1 --2- 172.21.15.52:0/2857561050 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f90104490 0x7f3f90199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.122+0000 7f3f8e59c700 1 -- 172.21.15.52:0/2857561050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f900ffdf0 msgr2=0x7f3f901958f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:25.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.122+0000 7f3f8e59c700 1 --2- 172.21.15.52:0/2857561050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f900ffdf0 0x7f3f901958f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.122+0000 7f3f8e59c700 1 -- 172.21.15.52:0/2857561050 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f90104490 msgr2=0x7f3f90199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:25.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.122+0000 7f3f8e59c700 1 --2- 172.21.15.52:0/2857561050 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f90104490 0x7f3f90199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.122+0000 7f3f8e59c700 1 -- 172.21.15.52:0/2857561050 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f800095f0 con 0x7f3f900ff450 2023-11-27T15:25:25.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.122+0000 7f3f8dd9b700 1 --2- 172.21.15.52:0/2857561050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f900ffdf0 0x7f3f901958f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:25.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.122+0000 7f3f8ed9d700 1 --2- 172.21.15.52:0/2857561050 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f90104490 0x7f3f90199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:25.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.122+0000 7f3f8e59c700 1 --2- 172.21.15.52:0/2857561050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f900ff450 0x7f3f901953b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f3f80004730 tx=0x7f3f80005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:25.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.122+0000 7f3f7f7fe700 1 -- 172.21.15.52:0/2857561050 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8001d070 con 0x7f3f900ff450 2023-11-27T15:25:25.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.123+0000 7f3f94a24700 1 -- 172.21.15.52:0/2857561050 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f9010f2a0 con 0x7f3f900ff450 2023-11-27T15:25:25.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.123+0000 7f3f7f7fe700 1 -- 172.21.15.52:0/2857561050 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f80004cd0 con 0x7f3f900ff450 2023-11-27T15:25:25.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.123+0000 7f3f7f7fe700 1 -- 172.21.15.52:0/2857561050 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f80005290 con 0x7f3f900ff450 2023-11-27T15:25:25.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.123+0000 7f3f94a24700 1 -- 172.21.15.52:0/2857561050 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f9010f110 con 0x7f3f900ff450 2023-11-27T15:25:25.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.125+0000 7f3f7f7fe700 1 -- 172.21.15.52:0/2857561050 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f3f80003c50 con 0x7f3f900ff450 2023-11-27T15:25:25.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.125+0000 7f3f7f7fe700 1 --2- 172.21.15.52:0/2857561050 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f3f780a12c0 0x7f3f780a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.126+0000 7f3f7f7fe700 1 -- 172.21.15.52:0/2857561050 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f3f800cdaa0 con 0x7f3f900ff450 2023-11-27T15:25:25.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.126+0000 7f3f8dd9b700 1 --2- 172.21.15.52:0/2857561050 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f3f780a12c0 0x7f3f780a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.126+0000 7f3f8dd9b700 1 --2- 172.21.15.52:0/2857561050 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f3f780a12c0 0x7f3f780a3770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3f901969c0 tx=0x7f3f84005f50 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:25.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.126+0000 7f3fa0bb0b80 1 Processor -- start 2023-11-27T15:25:25.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.127+0000 7f3fa0bb0b80 1 -- start start 2023-11-27T15:25:25.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.127+0000 7f3fa0bb0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568932099f0 0x556893209dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.127+0000 7f3fa0bb0b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55689320a300 0x556893218070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.127+0000 7f3fa0bb0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568932185b0 0x55689321a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.127+0000 7f3fa0bb0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55689310f0e0 con 0x5568932099f0 2023-11-27T15:25:25.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.127+0000 7f3fa0bb0b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x556893161920 con 0x55689320a300 2023-11-27T15:25:25.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.127+0000 7f3fa0bb0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55689315adb0 con 0x5568932185b0 2023-11-27T15:25:25.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.127+0000 7f3f8e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568932099f0 0x556893209dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.127+0000 7f3f8e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568932099f0 0x556893209dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55080/0 (socket says 172.21.15.52:55080) 2023-11-27T15:25:25.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.127+0000 7f3f8e59c700 1 -- 172.21.15.52:0/1998247806 learned_addr learned my addr 172.21.15.52:0/1998247806 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:25.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.127+0000 7f3f8ed9d700 1 --2- 172.21.15.52:0/1998247806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568932185b0 0x55689321a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.127+0000 7f3f8dd9b700 1 --2- 172.21.15.52:0/1998247806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55689320a300 0x556893218070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.127+0000 7f3f8e59c700 1 -- 172.21.15.52:0/1998247806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568932185b0 msgr2=0x55689321a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:25.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.127+0000 7f3f8e59c700 1 --2- 172.21.15.52:0/1998247806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568932185b0 0x55689321a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.127+0000 7f3f8e59c700 1 -- 172.21.15.52:0/1998247806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55689320a300 msgr2=0x556893218070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:25.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.127+0000 7f3f8e59c700 1 --2- 172.21.15.52:0/1998247806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55689320a300 0x556893218070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.127+0000 7f3f8e59c700 1 -- 172.21.15.52:0/1998247806 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556892fdc6b0 con 0x5568932099f0 2023-11-27T15:25:25.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.127+0000 7f3f8ed9d700 1 --2- 172.21.15.52:0/1998247806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568932185b0 0x55689321a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:25.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.128+0000 7f3f8e59c700 1 --2- 172.21.15.52:0/1998247806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568932099f0 0x556893209dc0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f3f80005b20 tx=0x7f3f8009aa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60cdad86bb41a7eb server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:25.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.128+0000 7f3f7cff9700 1 -- 172.21.15.52:0/1998247806 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8009ab70 con 0x5568932099f0 2023-11-27T15:25:25.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.128+0000 7f3f7cff9700 1 -- 172.21.15.52:0/1998247806 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f8009eba0 con 0x5568932099f0 2023-11-27T15:25:25.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.128+0000 7f3f7cff9700 1 -- 172.21.15.52:0/1998247806 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8009add0 con 0x5568932099f0 2023-11-27T15:25:25.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.128+0000 7f3fa0bb0b80 1 -- 172.21.15.52:0/1998247806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568932099f0 msgr2=0x556893209dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:25.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.128+0000 7f3fa0bb0b80 1 --2- 172.21.15.52:0/1998247806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568932099f0 0x556893209dc0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f3f80005b20 tx=0x7f3f8009aa70 comp rx=0 tx=0).stop 2023-11-27T15:25:25.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.128+0000 7f3fa0bb0b80 1 -- 172.21.15.52:0/1998247806 shutdown_connections 2023-11-27T15:25:25.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.129+0000 7f3fa0bb0b80 1 --2- 172.21.15.52:0/1998247806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5568932185b0 0x55689321a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.129+0000 7f3fa0bb0b80 1 --2- 172.21.15.52:0/1998247806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568932099f0 0x556893209dc0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.129+0000 7f3fa0bb0b80 1 --2- 172.21.15.52:0/1998247806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55689320a300 0x556893218070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.129+0000 7f3fa0bb0b80 1 -- 172.21.15.52:0/1998247806 >> 172.21.15.52:0/1998247806 conn(0x5568931ff3e0 msgr2=0x556893204bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:25.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.129+0000 7f3fa0bb0b80 1 -- 172.21.15.52:0/1998247806 shutdown_connections 2023-11-27T15:25:25.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.129+0000 7f3fa0bb0b80 1 -- 172.21.15.52:0/1998247806 wait complete. 2023-11-27T15:25:25.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.129+0000 7f3fa0bb0b80 1 -- start start 2023-11-27T15:25:25.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.129+0000 7f3fa0bb0b80 1 Processor -- start 2023-11-27T15:25:25.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.129+0000 7f3fa0bb0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568932099f0 0x55689321d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.129+0000 7f3fa0bb0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55689320a300 0x55689321da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.129+0000 7f3fa0bb0b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5568932185b0 0x556893217440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.129+0000 7f3fa0bb0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568930fc960 con 0x5568932099f0 2023-11-27T15:25:25.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.129+0000 7f3fa0bb0b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568930fc600 con 0x5568932185b0 2023-11-27T15:25:25.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.129+0000 7f3fa0bb0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556892fde9a0 con 0x55689320a300 2023-11-27T15:25:25.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.129+0000 7f3fa0bb0b80 10 client.0 _ll_register_callbacks cb 0x55689311a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:25.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.129+0000 7f3f8e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568932099f0 0x55689321d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.129+0000 7f3f8e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568932099f0 0x55689321d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55086/0 (socket says 172.21.15.52:55086) 2023-11-27T15:25:25.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.129+0000 7f3f8e59c700 1 -- 172.21.15.52:0/2123670205 learned_addr learned my addr 172.21.15.52:0/2123670205 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:25.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.130+0000 7f3f8ed9d700 1 --2- 172.21.15.52:0/2123670205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5568932185b0 0x556893217440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.130+0000 7f3f8dd9b700 1 --2- 172.21.15.52:0/2123670205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55689320a300 0x55689321da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.130+0000 7f3f8e59c700 1 -- 172.21.15.52:0/2123670205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55689320a300 msgr2=0x55689321da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:25.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.130+0000 7f3f8e59c700 1 --2- 172.21.15.52:0/2123670205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55689320a300 0x55689321da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.130+0000 7f3f8e59c700 1 -- 172.21.15.52:0/2123670205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5568932185b0 msgr2=0x556893217440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:25.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.130+0000 7f3f8e59c700 1 --2- 172.21.15.52:0/2123670205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5568932185b0 0x556893217440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.130+0000 7f3f8e59c700 1 -- 172.21.15.52:0/2123670205 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f9010f110 con 0x5568932099f0 2023-11-27T15:25:25.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.130+0000 7f3f8ed9d700 1 --2- 172.21.15.52:0/2123670205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5568932185b0 0x556893217440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:25.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.130+0000 7f3f8e59c700 1 --2- 172.21.15.52:0/2123670205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568932099f0 0x55689321d4e0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f3f80006010 tx=0x7f3f80003ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7067090939bd26 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:25.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.130+0000 7f3f75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:25:25.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.130+0000 7f3f75ffb700 1 -- 172.21.15.52:0/2123670205 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8009a060 con 0x5568932099f0 2023-11-27T15:25:25.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.130+0000 7f3f75ffb700 1 -- 172.21.15.52:0/2123670205 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f3f8009a1c0 con 0x5568932099f0 2023-11-27T15:25:25.146 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.130+0000 7f3fa0bb0b80 1 -- 172.21.15.52:0/2123670205 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5568931fcba0 con 0x5568932099f0 2023-11-27T15:25:25.146 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.130+0000 7f3f75ffb700 1 -- 172.21.15.52:0/2123670205 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f8009add0 con 0x5568932099f0 2023-11-27T15:25:25.146 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.130+0000 7f3fa0bb0b80 10 client.16701 fetch_fsmap learned FSMap version 30 2023-11-27T15:25:25.146 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.130+0000 7f3fa0bb0b80 1 -- 172.21.15.52:0/2123670205 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556893121210 con 0x5568932099f0 2023-11-27T15:25:25.146 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.130+0000 7f3f75ffb700 1 -- 172.21.15.52:0/2123670205 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f80064080 con 0x5568932099f0 2023-11-27T15:25:25.146 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.131+0000 7f3f75ffb700 1 -- 172.21.15.52:0/2123670205 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3f80096960 con 0x5568932099f0 2023-11-27T15:25:25.146 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.131+0000 7f3fa0bb0b80 10 client.16701 fetch_fsmap finished waiting for FSMap version 30 2023-11-27T15:25:25.146 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.131+0000 7f3fa0bb0b80 10 client.16701 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:25.147 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.131+0000 7f3fa0bb0b80 20 client.16701 populate_metadata read hostname 'smithi052' 2023-11-27T15:25:25.147 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.131+0000 7f3fa0bb0b80 1 --2- 172.21.15.52:0/2123670205 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5568932274a0 0x5568932298b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.147 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.131+0000 7f3fa0bb0b80 4 client.16701 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:25.147 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.131+0000 7f3fa0bb0b80 1 -- 172.21.15.52:0/2123670205 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55689311d120 con 0x5568932274a0 2023-11-27T15:25:25.147 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.131+0000 7f3f8dd9b700 1 --2- 172.21.15.52:0/2123670205 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5568932274a0 0x5568932298b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.147 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.132+0000 7f3f8dd9b700 1 --2- 172.21.15.52:0/2123670205 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5568932274a0 0x5568932298b0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ddeeea436919688 server_cookie=ed3e09f3530a4060 in_seq=0 out_seq=0 2023-11-27T15:25:25.147 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.132+0000 7f3f75ffb700 10 client.16701 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:25.147 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.132+0000 7f3f75ffb700 1 -- 172.21.15.52:0/2123670205 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55689311d120 con 0x5568932274a0 2023-11-27T15:25:25.148 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.132+0000 7f3f75ffb700 10 client.16701 handle_command_reply: tid=0 2023-11-27T15:25:25.148 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.132+0000 7f3fa0bb0b80 1 client.16701 shutdown 2023-11-27T15:25:25.148 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.132+0000 7f3fa0bb0b80 1 -- 172.21.15.52:0/2123670205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568932099f0 msgr2=0x55689321d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:25.148 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.132+0000 7f3fa0bb0b80 1 --2- 172.21.15.52:0/2123670205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568932099f0 0x55689321d4e0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f3f80006010 tx=0x7f3f80003ff0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.148 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.132+0000 7f3fa0bb0b80 1 -- 172.21.15.52:0/2123670205 shutdown_connections 2023-11-27T15:25:25.148 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.132+0000 7f3fa0bb0b80 1 --2- 172.21.15.52:0/2123670205 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5568932274a0 0x5568932298b0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.148 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.132+0000 7f3fa0bb0b80 1 --2- 172.21.15.52:0/2123670205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5568932185b0 0x556893217440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.132+0000 7f3fa0bb0b80 1 --2- 172.21.15.52:0/2123670205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568932099f0 0x55689321d4e0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.132+0000 7f3f75ffb700 0 client.16701 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:25.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.132+0000 7f3fa0bb0b80 1 --2- 172.21.15.52:0/2123670205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55689320a300 0x55689321da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.132+0000 7f3fa0bb0b80 1 -- 172.21.15.52:0/2123670205 >> 172.21.15.52:0/2123670205 conn(0x5568931ff3e0 msgr2=0x556893203e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:25.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.133+0000 7f3fa0bb0b80 1 -- 172.21.15.52:0/2123670205 shutdown_connections 2023-11-27T15:25:25.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.133+0000 7f3fa0bb0b80 1 -- 172.21.15.52:0/2123670205 wait complete. 2023-11-27T15:25:25.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.133+0000 7f3fa0bb0b80 20 client.16701 trim_cache size 0 max 16384 2023-11-27T15:25:25.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.150+0000 7f3fa0bb0b80 1 Processor -- start 2023-11-27T15:25:25.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.150+0000 7f3fa0bb0b80 1 -- start start 2023-11-27T15:25:25.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.150+0000 7f3fa0bb0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568932099f0 0x556893209270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.150+0000 7f3fa0bb0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55689320a300 0x556893214710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.150+0000 7f3fa0bb0b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5568932185b0 0x556893214c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.150+0000 7f3fa0bb0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568931fef60 con 0x5568932099f0 2023-11-27T15:25:25.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.150+0000 7f3fa0bb0b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55689320b460 con 0x5568932185b0 2023-11-27T15:25:25.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.150+0000 7f3fa0bb0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556893212270 con 0x55689320a300 2023-11-27T15:25:25.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.150+0000 7f3f8e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568932099f0 0x556893209270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.150+0000 7f3f8e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568932099f0 0x556893209270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55094/0 (socket says 172.21.15.52:55094) 2023-11-27T15:25:25.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.150+0000 7f3f8e59c700 1 -- 172.21.15.52:0/2471619317 learned_addr learned my addr 172.21.15.52:0/2471619317 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:25.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.150+0000 7f3f8dd9b700 1 --2- 172.21.15.52:0/2471619317 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55689320a300 0x556893214710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.150+0000 7f3f8e59c700 1 -- 172.21.15.52:0/2471619317 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55689320a300 msgr2=0x556893214710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:25.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.150+0000 7f3f8e59c700 1 --2- 172.21.15.52:0/2471619317 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55689320a300 0x556893214710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.150+0000 7f3f8ed9d700 1 --2- 172.21.15.52:0/2471619317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5568932185b0 0x556893214c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.150+0000 7f3f8e59c700 1 -- 172.21.15.52:0/2471619317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5568932185b0 msgr2=0x556893214c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:25.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.151+0000 7f3f8e59c700 1 --2- 172.21.15.52:0/2471619317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5568932185b0 0x556893214c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.151+0000 7f3f8e59c700 1 -- 172.21.15.52:0/2471619317 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556893120f60 con 0x5568932099f0 2023-11-27T15:25:25.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.151+0000 7f3f8ed9d700 1 --2- 172.21.15.52:0/2471619317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5568932185b0 0x556893214c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:25.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.151+0000 7f3f8e59c700 1 --2- 172.21.15.52:0/2471619317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568932099f0 0x556893209270 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f3f80098ec0 tx=0x7f3f80009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a53153a5fe1d189 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:25.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.151+0000 7f3f7d7fa700 1 -- 172.21.15.52:0/2471619317 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f800cb8b0 con 0x5568932099f0 2023-11-27T15:25:25.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.151+0000 7f3f7d7fa700 1 -- 172.21.15.52:0/2471619317 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5568931fcba0 con 0x5568932099f0 2023-11-27T15:25:25.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.151+0000 7f3f7d7fa700 1 -- 172.21.15.52:0/2471619317 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f800a2aa0 con 0x5568932099f0 2023-11-27T15:25:25.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.151+0000 7f3fa0bb0b80 1 -- 172.21.15.52:0/2471619317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568932099f0 msgr2=0x556893209270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:25.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.151+0000 7f3fa0bb0b80 1 --2- 172.21.15.52:0/2471619317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568932099f0 0x556893209270 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f3f80098ec0 tx=0x7f3f80009cd0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.151+0000 7f3fa0bb0b80 1 -- 172.21.15.52:0/2471619317 shutdown_connections 2023-11-27T15:25:25.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.151+0000 7f3fa0bb0b80 1 --2- 172.21.15.52:0/2471619317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5568932185b0 0x556893214c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.151+0000 7f3fa0bb0b80 1 --2- 172.21.15.52:0/2471619317 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568932099f0 0x556893209270 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.151+0000 7f3fa0bb0b80 1 --2- 172.21.15.52:0/2471619317 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55689320a300 0x556893214710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.151+0000 7f3fa0bb0b80 1 -- 172.21.15.52:0/2471619317 >> 172.21.15.52:0/2471619317 conn(0x5568931ff3e0 msgr2=0x556893217360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:25.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.152+0000 7f3fa0bb0b80 1 -- 172.21.15.52:0/2471619317 shutdown_connections 2023-11-27T15:25:25.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.152+0000 7f3fa0bb0b80 1 -- 172.21.15.52:0/2471619317 wait complete. 2023-11-27T15:25:25.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.152+0000 7f3fa0bb0b80 1 -- start start 2023-11-27T15:25:25.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.152+0000 7f3fa0bb0b80 1 Processor -- start 2023-11-27T15:25:25.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.152+0000 7f3fa0bb0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568932099f0 0x556893274ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.152+0000 7f3fa0bb0b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55689320a300 0x556893275430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.152+0000 7f3fa0bb0b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5568932185b0 0x55689326f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.152+0000 7f3fa0bb0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55689326f900 con 0x5568932099f0 2023-11-27T15:25:25.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.152+0000 7f3fa0bb0b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55689326fa70 con 0x5568932185b0 2023-11-27T15:25:25.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.152+0000 7f3fa0bb0b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55689326fc30 con 0x55689320a300 2023-11-27T15:25:25.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.152+0000 7f3fa0bb0b80 10 client.0 _ll_register_callbacks cb 0x7f3f8009a940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:25.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.152+0000 7f3f8e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568932099f0 0x556893274ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.152+0000 7f3f8e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568932099f0 0x556893274ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55100/0 (socket says 172.21.15.52:55100) 2023-11-27T15:25:25.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.152+0000 7f3f8e59c700 1 -- 172.21.15.52:0/1785892570 learned_addr learned my addr 172.21.15.52:0/1785892570 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:25.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.152+0000 7f3f8ed9d700 1 --2- 172.21.15.52:0/1785892570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5568932185b0 0x55689326f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.152+0000 7f3f8dd9b700 1 --2- 172.21.15.52:0/1785892570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55689320a300 0x556893275430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.153+0000 7f3f8e59c700 1 -- 172.21.15.52:0/1785892570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55689320a300 msgr2=0x556893275430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:25.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.153+0000 7f3f8e59c700 1 --2- 172.21.15.52:0/1785892570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55689320a300 0x556893275430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.153+0000 7f3f8e59c700 1 -- 172.21.15.52:0/1785892570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5568932185b0 msgr2=0x55689326f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:25.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.153+0000 7f3f8e59c700 1 --2- 172.21.15.52:0/1785892570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5568932185b0 0x55689326f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.153+0000 7f3f8e59c700 1 -- 172.21.15.52:0/1785892570 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556893121210 con 0x5568932099f0 2023-11-27T15:25:25.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.153+0000 7f3f8ed9d700 1 --2- 172.21.15.52:0/1785892570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5568932185b0 0x55689326f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:25.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.153+0000 7f3f8e59c700 1 --2- 172.21.15.52:0/1785892570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568932099f0 0x556893274ef0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f3f800a2a00 tx=0x7f3f8009a9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acffd7f96d7bd944 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:25.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.153+0000 7f3f75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:25:25.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.153+0000 7f3f75ffb700 1 -- 172.21.15.52:0/1785892570 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f80096b70 con 0x5568932099f0 2023-11-27T15:25:25.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.153+0000 7f3f75ffb700 1 -- 172.21.15.52:0/1785892570 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f3f8009add0 con 0x5568932099f0 2023-11-27T15:25:25.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.153+0000 7f3fa0bb0b80 1 -- 172.21.15.52:0/1785892570 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f8009eba0 con 0x5568932099f0 2023-11-27T15:25:25.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.153+0000 7f3f75ffb700 1 -- 172.21.15.52:0/1785892570 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f8009afd0 con 0x5568932099f0 2023-11-27T15:25:25.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.153+0000 7f3fa0bb0b80 10 client.16707 fetch_fsmap learned FSMap version 30 2023-11-27T15:25:25.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.153+0000 7f3fa0bb0b80 1 -- 172.21.15.52:0/1785892570 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x556892f71ce0 con 0x5568932099f0 2023-11-27T15:25:25.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.153+0000 7f3f75ffb700 1 -- 172.21.15.52:0/1785892570 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f8009eba0 con 0x5568932099f0 2023-11-27T15:25:25.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.153+0000 7f3f75ffb700 1 -- 172.21.15.52:0/1785892570 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3f80005510 con 0x5568932099f0 2023-11-27T15:25:25.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.153+0000 7f3fa0bb0b80 10 client.16707 fetch_fsmap finished waiting for FSMap version 30 2023-11-27T15:25:25.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.153+0000 7f3fa0bb0b80 10 client.16707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:25.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.153+0000 7f3fa0bb0b80 20 client.16707 populate_metadata read hostname 'smithi052' 2023-11-27T15:25:25.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.153+0000 7f3fa0bb0b80 1 --2- 172.21.15.52:0/1785892570 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55689327c7a0 0x55689327eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.153+0000 7f3fa0bb0b80 4 client.16707 mds_command: new command op to 24536 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-27T15:25:25.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.153+0000 7f3fa0bb0b80 1 -- 172.21.15.52:0/1785892570 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5568931296e0 con 0x55689327c7a0 2023-11-27T15:25:25.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.154+0000 7f3f8dd9b700 1 --2- 172.21.15.52:0/1785892570 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55689327c7a0 0x55689327eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.154+0000 7f3f75ffb700 10 client.16707 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:25.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.154+0000 7f3f8dd9b700 1 --2- 172.21.15.52:0/1785892570 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55689327c7a0 0x55689327eb80 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea1309bee7815073 server_cookie=fc8bb6957ed51ce6 in_seq=0 out_seq=0 2023-11-27T15:25:25.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.155+0000 7f3f75ffb700 1 -- 172.21.15.52:0/1785892570 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5568931296e0 con 0x55689327c7a0 2023-11-27T15:25:25.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.155+0000 7f3f75ffb700 10 client.16707 handle_command_reply: tid=0 2023-11-27T15:25:25.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.155+0000 7f3fa0bb0b80 1 client.16707 shutdown 2023-11-27T15:25:25.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.155+0000 7f3fa0bb0b80 1 -- 172.21.15.52:0/1785892570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568932099f0 msgr2=0x556893274ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:25.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.155+0000 7f3fa0bb0b80 1 --2- 172.21.15.52:0/1785892570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568932099f0 0x556893274ef0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f3f800a2a00 tx=0x7f3f8009a9f0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.155+0000 7f3fa0bb0b80 1 -- 172.21.15.52:0/1785892570 shutdown_connections 2023-11-27T15:25:25.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.155+0000 7f3fa0bb0b80 1 --2- 172.21.15.52:0/1785892570 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55689327c7a0 0x55689327eb80 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.155+0000 7f3fa0bb0b80 1 --2- 172.21.15.52:0/1785892570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5568932185b0 0x55689326f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.155+0000 7f3fa0bb0b80 1 --2- 172.21.15.52:0/1785892570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568932099f0 0x556893274ef0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.155+0000 7f3f75ffb700 0 client.16707 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.155+0000 7f3fa0bb0b80 1 --2- 172.21.15.52:0/1785892570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55689320a300 0x556893275430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.155+0000 7f3fa0bb0b80 1 -- 172.21.15.52:0/1785892570 >> 172.21.15.52:0/1785892570 conn(0x5568931ff3e0 msgr2=0x556893209270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.156+0000 7f3fa0bb0b80 1 -- 172.21.15.52:0/1785892570 shutdown_connections 2023-11-27T15:25:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.156+0000 7f3fa0bb0b80 1 -- 172.21.15.52:0/1785892570 wait complete. 2023-11-27T15:25:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.156+0000 7f3fa0bb0b80 20 client.16707 trim_cache size 0 max 16384 2023-11-27T15:25:25.171 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:25:25.171 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2023-11-27T15:25:25.171 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "da59e58e-0e8b-42de-9bd9-b2899ee7567d", 2023-11-27T15:25:25.171 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2023-11-27T15:25:25.171 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:25:25.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.156+0000 7f3f94a24700 1 -- 172.21.15.52:0/2857561050 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f3f780a12c0 msgr2=0x7f3f780a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:25.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.156+0000 7f3f94a24700 1 --2- 172.21.15.52:0/2857561050 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f3f780a12c0 0x7f3f780a3770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3f901969c0 tx=0x7f3f84005f50 comp rx=0 tx=0).stop 2023-11-27T15:25:25.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.156+0000 7f3f94a24700 1 -- 172.21.15.52:0/2857561050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f900ff450 msgr2=0x7f3f901953b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:25.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.156+0000 7f3f94a24700 1 --2- 172.21.15.52:0/2857561050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f900ff450 0x7f3f901953b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f3f80004730 tx=0x7f3f80005e80 comp rx=0 tx=0).stop 2023-11-27T15:25:25.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.156+0000 7f3f94a24700 1 -- 172.21.15.52:0/2857561050 shutdown_connections 2023-11-27T15:25:25.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.156+0000 7f3f94a24700 1 --2- 172.21.15.52:0/2857561050 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f3f780a12c0 0x7f3f780a3770 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.156+0000 7f3f94a24700 1 --2- 172.21.15.52:0/2857561050 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f90104490 0x7f3f90199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.156+0000 7f3f94a24700 1 --2- 172.21.15.52:0/2857561050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f900ff450 0x7f3f901953b0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.156+0000 7f3f94a24700 1 --2- 172.21.15.52:0/2857561050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f900ffdf0 0x7f3f901958f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.156+0000 7f3f94a24700 1 -- 172.21.15.52:0/2857561050 >> 172.21.15.52:0/2857561050 conn(0x7f3f900770b0 msgr2=0x7f3f900fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:25.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.156+0000 7f3f94a24700 1 -- 172.21.15.52:0/2857561050 shutdown_connections 2023-11-27T15:25:25.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.156+0000 7f3f94a24700 1 -- 172.21.15.52:0/2857561050 wait complete. 2023-11-27T15:25:25.174 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:25:25.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.298+0000 7fefc2e7a700 1 -- 172.21.15.52:0/2294265248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefbc104590 msgr2=0x7fefbc10c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:25.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.298+0000 7fefc2e7a700 1 --2- 172.21.15.52:0/2294265248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefbc104590 0x7fefbc10c110 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fefb400b0d0 tx=0x7fefb400b3e0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.298+0000 7fefc2e7a700 1 -- 172.21.15.52:0/2294265248 shutdown_connections 2023-11-27T15:25:25.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.298+0000 7fefc2e7a700 1 --2- 172.21.15.52:0/2294265248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefbc104590 0x7fefbc10c110 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.298+0000 7fefc2e7a700 1 --2- 172.21.15.52:0/2294265248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefbc0ffc30 0x7fefbc100000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.298+0000 7fefc2e7a700 1 --2- 172.21.15.52:0/2294265248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fefbc1005d0 0x7fefbc103e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.298+0000 7fefc2e7a700 1 -- 172.21.15.52:0/2294265248 >> 172.21.15.52:0/2294265248 conn(0x7fefbc075470 msgr2=0x7fefbc075870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:25.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.298+0000 7fefc2e7a700 1 -- 172.21.15.52:0/2294265248 shutdown_connections 2023-11-27T15:25:25.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.298+0000 7fefc2e7a700 1 -- 172.21.15.52:0/2294265248 wait complete. 2023-11-27T15:25:25.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.299+0000 7fefc2e7a700 1 Processor -- start 2023-11-27T15:25:25.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.299+0000 7fefc2e7a700 1 -- start start 2023-11-27T15:25:25.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.299+0000 7fefc2e7a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefbc0ffc30 0x7fefbc19e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.306 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.299+0000 7fefc2e7a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefbc1005d0 0x7fefbc19e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.306 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.299+0000 7fefc2e7a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fefbc104590 0x7fefbc198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.306 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.299+0000 7fefc2e7a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc199260 con 0x7fefbc0ffc30 2023-11-27T15:25:25.306 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.299+0000 7fefc2e7a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc1993d0 con 0x7fefbc104590 2023-11-27T15:25:25.306 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.299+0000 7fefc2e7a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc199590 con 0x7fefbc1005d0 2023-11-27T15:25:25.306 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.299+0000 7fefc0c16700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefbc0ffc30 0x7fefbc19e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.306 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.299+0000 7fefc0c16700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefbc0ffc30 0x7fefbc19e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55114/0 (socket says 172.21.15.52:55114) 2023-11-27T15:25:25.307 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.299+0000 7fefc0c16700 1 -- 172.21.15.52:0/3736692255 learned_addr learned my addr 172.21.15.52:0/3736692255 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:25.307 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.299+0000 7fefbbfff700 1 --2- 172.21.15.52:0/3736692255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefbc1005d0 0x7fefbc19e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.307 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.299+0000 7fefc1417700 1 --2- 172.21.15.52:0/3736692255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fefbc104590 0x7fefbc198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.307 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.299+0000 7fefc0c16700 1 -- 172.21.15.52:0/3736692255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefbc1005d0 msgr2=0x7fefbc19e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:25.307 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.299+0000 7fefc0c16700 1 --2- 172.21.15.52:0/3736692255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefbc1005d0 0x7fefbc19e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.307 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.299+0000 7fefc0c16700 1 -- 172.21.15.52:0/3736692255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fefbc104590 msgr2=0x7fefbc198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:25.307 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.299+0000 7fefc0c16700 1 --2- 172.21.15.52:0/3736692255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fefbc104590 0x7fefbc198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.307 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.299+0000 7fefc0c16700 1 -- 172.21.15.52:0/3736692255 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb4009d00 con 0x7fefbc0ffc30 2023-11-27T15:25:25.308 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.299+0000 7fefc1417700 1 --2- 172.21.15.52:0/3736692255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fefbc104590 0x7fefbc198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:25.308 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.300+0000 7fefc0c16700 1 --2- 172.21.15.52:0/3736692255 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefbc0ffc30 0x7fefbc19e480 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fefb000d950 tx=0x7fefb000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:25.308 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.300+0000 7fefb9ffb700 1 -- 172.21.15.52:0/3736692255 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb0009940 con 0x7fefbc0ffc30 2023-11-27T15:25:25.308 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.300+0000 7fefb9ffb700 1 -- 172.21.15.52:0/3736692255 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefb0010460 con 0x7fefbc0ffc30 2023-11-27T15:25:25.308 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.300+0000 7fefc2e7a700 1 -- 172.21.15.52:0/3736692255 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefbc10f320 con 0x7fefbc0ffc30 2023-11-27T15:25:25.308 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.300+0000 7fefb9ffb700 1 -- 172.21.15.52:0/3736692255 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb000f620 con 0x7fefbc0ffc30 2023-11-27T15:25:25.308 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.300+0000 7fefc2e7a700 1 -- 172.21.15.52:0/3736692255 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefbc10f190 con 0x7fefbc0ffc30 2023-11-27T15:25:25.314 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.302+0000 7fefb9ffb700 1 -- 172.21.15.52:0/3736692255 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fefb00108d0 con 0x7fefbc0ffc30 2023-11-27T15:25:25.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.303+0000 7fefb9ffb700 1 --2- 172.21.15.52:0/3736692255 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fefac0a12c0 0x7fefac0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.303+0000 7fefb9ffb700 1 -- 172.21.15.52:0/3736692255 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fefb00ccc20 con 0x7fefbc0ffc30 2023-11-27T15:25:25.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.303+0000 7fefbbfff700 1 --2- 172.21.15.52:0/3736692255 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fefac0a12c0 0x7fefac0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.303+0000 7fefbbfff700 1 --2- 172.21.15.52:0/3736692255 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fefac0a12c0 0x7fefac0a3770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fefbc10c570 tx=0x7fefa8009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:25.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.304+0000 7fefcf006b80 1 Processor -- start 2023-11-27T15:25:25.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.304+0000 7fefcf006b80 1 -- start start 2023-11-27T15:25:25.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.304+0000 7fefcf006b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578e10beb60 0x5578e10bef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.304+0000 7fefcf006b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578e10bf470 0x5578e10c3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.304+0000 7fefcf006b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578e10c38b0 0x5578e10c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.304+0000 7fefcf006b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578e0fc4060 con 0x5578e10c38b0 2023-11-27T15:25:25.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.304+0000 7fefcf006b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578e10168a0 con 0x5578e10beb60 2023-11-27T15:25:25.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.304+0000 7fefcf006b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578e100fd30 con 0x5578e10bf470 2023-11-27T15:25:25.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.304+0000 7fefc1417700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578e10c38b0 0x5578e10c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.304+0000 7fefc1417700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578e10c38b0 0x5578e10c3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55126/0 (socket says 172.21.15.52:55126) 2023-11-27T15:25:25.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.304+0000 7fefc1417700 1 -- 172.21.15.52:0/3768865350 learned_addr learned my addr 172.21.15.52:0/3768865350 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:25.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.304+0000 7fefbbfff700 1 --2- 172.21.15.52:0/3768865350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578e10bf470 0x5578e10c3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.304+0000 7fefc0c16700 1 --2- 172.21.15.52:0/3768865350 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578e10beb60 0x5578e10bef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.304+0000 7fefc1417700 1 -- 172.21.15.52:0/3768865350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578e10bf470 msgr2=0x5578e10c3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:25.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.304+0000 7fefc1417700 1 --2- 172.21.15.52:0/3768865350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578e10bf470 0x5578e10c3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.304+0000 7fefc1417700 1 -- 172.21.15.52:0/3768865350 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578e10beb60 msgr2=0x5578e10bef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:25.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.304+0000 7fefc1417700 1 --2- 172.21.15.52:0/3768865350 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578e10beb60 0x5578e10bef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.304+0000 7fefc1417700 1 -- 172.21.15.52:0/3768865350 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578e0e91630 con 0x5578e10c38b0 2023-11-27T15:25:25.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.304+0000 7fefc0c16700 1 --2- 172.21.15.52:0/3768865350 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578e10beb60 0x5578e10bef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:25.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.304+0000 7fefc1417700 1 --2- 172.21.15.52:0/3768865350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578e10c38b0 0x5578e10c3d00 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fefb4009310 tx=0x7fefb4009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c498fc019d30f715 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:25.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.305+0000 7fefa6ffd700 1 -- 172.21.15.52:0/3768865350 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb400e040 con 0x5578e10c38b0 2023-11-27T15:25:25.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.305+0000 7fefa6ffd700 1 -- 172.21.15.52:0/3768865350 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefb400bca0 con 0x5578e10c38b0 2023-11-27T15:25:25.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.305+0000 7fefa6ffd700 1 -- 172.21.15.52:0/3768865350 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb401c6a0 con 0x5578e10c38b0 2023-11-27T15:25:25.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.305+0000 7fefcf006b80 1 -- 172.21.15.52:0/3768865350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578e10c38b0 msgr2=0x5578e10c3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:25.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.305+0000 7fefcf006b80 1 --2- 172.21.15.52:0/3768865350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578e10c38b0 0x5578e10c3d00 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fefb4009310 tx=0x7fefb4009340 comp rx=0 tx=0).stop 2023-11-27T15:25:25.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.305+0000 7fefcf006b80 1 -- 172.21.15.52:0/3768865350 shutdown_connections 2023-11-27T15:25:25.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.305+0000 7fefcf006b80 1 --2- 172.21.15.52:0/3768865350 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578e10beb60 0x5578e10bef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.305+0000 7fefcf006b80 1 --2- 172.21.15.52:0/3768865350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578e10c38b0 0x5578e10c3d00 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.305+0000 7fefcf006b80 1 --2- 172.21.15.52:0/3768865350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578e10bf470 0x5578e10c3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.305+0000 7fefcf006b80 1 -- 172.21.15.52:0/3768865350 >> 172.21.15.52:0/3768865350 conn(0x5578e10b4360 msgr2=0x5578e10b9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:25.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.305+0000 7fefcf006b80 1 -- 172.21.15.52:0/3768865350 shutdown_connections 2023-11-27T15:25:25.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.305+0000 7fefcf006b80 1 -- 172.21.15.52:0/3768865350 wait complete. 2023-11-27T15:25:25.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.306+0000 7fefcf006b80 1 -- start start 2023-11-27T15:25:25.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.306+0000 7fefcf006b80 1 Processor -- start 2023-11-27T15:25:25.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.306+0000 7fefcf006b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578e10beb60 0x5578e10cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.306+0000 7fefcf006b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578e10bf470 0x5578e10c8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.306+0000 7fefcf006b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578e10c38b0 0x5578e10c8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.306+0000 7fefcf006b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578e0fb18e0 con 0x5578e10bf470 2023-11-27T15:25:25.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.306+0000 7fefcf006b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578e0fb1580 con 0x5578e10c38b0 2023-11-27T15:25:25.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.306+0000 7fefcf006b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578e0e93920 con 0x5578e10beb60 2023-11-27T15:25:25.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.306+0000 7fefcf006b80 10 client.0 _ll_register_callbacks cb 0x5578e0fcf780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:25.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.306+0000 7fefbbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578e10bf470 0x5578e10c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.307+0000 7fefbbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578e10bf470 0x5578e10c8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55128/0 (socket says 172.21.15.52:55128) 2023-11-27T15:25:25.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.307+0000 7fefbbfff700 1 -- 172.21.15.52:0/1685319678 learned_addr learned my addr 172.21.15.52:0/1685319678 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:25.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.307+0000 7fefc1417700 1 --2- 172.21.15.52:0/1685319678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578e10c38b0 0x5578e10c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.307+0000 7fefc0c16700 1 --2- 172.21.15.52:0/1685319678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578e10beb60 0x5578e10cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.307+0000 7fefbbfff700 1 -- 172.21.15.52:0/1685319678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578e10beb60 msgr2=0x5578e10cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:25.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.307+0000 7fefbbfff700 1 --2- 172.21.15.52:0/1685319678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578e10beb60 0x5578e10cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.307+0000 7fefbbfff700 1 -- 172.21.15.52:0/1685319678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578e10c38b0 msgr2=0x5578e10c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:25.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.307+0000 7fefbbfff700 1 --2- 172.21.15.52:0/1685319678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578e10c38b0 0x5578e10c8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.322 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.307+0000 7fefbbfff700 1 -- 172.21.15.52:0/1685319678 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefbc10c950 con 0x5578e10bf470 2023-11-27T15:25:25.322 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.307+0000 7fefc0c16700 1 --2- 172.21.15.52:0/1685319678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578e10beb60 0x5578e10cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:25.322 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.307+0000 7fefbbfff700 1 --2- 172.21.15.52:0/1685319678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578e10bf470 0x5578e10c8290 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fefbc0768b0 tx=0x7fefa80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50272d7137bfe02 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:25.322 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.307+0000 7fef9bfff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:25:25.322 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.307+0000 7fef9bfff700 1 -- 172.21.15.52:0/1685319678 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa8004260 con 0x5578e10bf470 2023-11-27T15:25:25.322 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.307+0000 7fef9bfff700 1 -- 172.21.15.52:0/1685319678 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fefa80043c0 con 0x5578e10bf470 2023-11-27T15:25:25.322 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.307+0000 7fefcf006b80 1 -- 172.21.15.52:0/1685319678 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5578e10b1b20 con 0x5578e10bf470 2023-11-27T15:25:25.322 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.307+0000 7fef9bfff700 1 -- 172.21.15.52:0/1685319678 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefa8004580 con 0x5578e10bf470 2023-11-27T15:25:25.323 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.307+0000 7fefcf006b80 10 client.16719 fetch_fsmap learned FSMap version 30 2023-11-27T15:25:25.323 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.307+0000 7fefcf006b80 1 -- 172.21.15.52:0/1685319678 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5578e0fd6190 con 0x5578e10bf470 2023-11-27T15:25:25.323 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.308+0000 7fef9bfff700 1 -- 172.21.15.52:0/1685319678 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefa8034080 con 0x5578e10bf470 2023-11-27T15:25:25.323 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.308+0000 7fef9bfff700 1 -- 172.21.15.52:0/1685319678 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fefa801d8f0 con 0x5578e10bf470 2023-11-27T15:25:25.323 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.308+0000 7fefcf006b80 10 client.16719 fetch_fsmap finished waiting for FSMap version 30 2023-11-27T15:25:25.323 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.308+0000 7fefcf006b80 10 client.16719 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:25.323 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.308+0000 7fefcf006b80 20 client.16719 populate_metadata read hostname 'smithi052' 2023-11-27T15:25:25.323 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.308+0000 7fefcf006b80 1 --2- 172.21.15.52:0/1685319678 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5578e10d88a0 0x5578e10dacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.324 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.308+0000 7fefcf006b80 4 client.16719 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:25.324 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.308+0000 7fefcf006b80 1 -- 172.21.15.52:0/1685319678 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5578e0fd20a0 con 0x5578e10d88a0 2023-11-27T15:25:25.324 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.308+0000 7fefc0c16700 1 --2- 172.21.15.52:0/1685319678 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5578e10d88a0 0x5578e10dacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.324 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.308+0000 7fef9bfff700 10 client.16719 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:25.324 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.308+0000 7fefc0c16700 1 --2- 172.21.15.52:0/1685319678 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5578e10d88a0 0x5578e10dacb0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b703b4b94c8ff30a server_cookie=98be1b1aeed50f06 in_seq=0 out_seq=0 2023-11-27T15:25:25.324 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.309+0000 7fef9bfff700 1 -- 172.21.15.52:0/1685319678 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5578e0fd20a0 con 0x5578e10d88a0 2023-11-27T15:25:25.324 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.309+0000 7fef9bfff700 10 client.16719 handle_command_reply: tid=0 2023-11-27T15:25:25.324 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.309+0000 7fefcf006b80 1 client.16719 shutdown 2023-11-27T15:25:25.325 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.309+0000 7fefcf006b80 1 -- 172.21.15.52:0/1685319678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578e10bf470 msgr2=0x5578e10c8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:25.325 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.309+0000 7fefcf006b80 1 --2- 172.21.15.52:0/1685319678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578e10bf470 0x5578e10c8290 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fefbc0768b0 tx=0x7fefa80047c0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.325 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.309+0000 7fefcf006b80 1 -- 172.21.15.52:0/1685319678 shutdown_connections 2023-11-27T15:25:25.325 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.309+0000 7fefcf006b80 1 --2- 172.21.15.52:0/1685319678 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5578e10d88a0 0x5578e10dacb0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.325 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.309+0000 7fefcf006b80 1 --2- 172.21.15.52:0/1685319678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578e10c38b0 0x5578e10c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.325 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.309+0000 7fefcf006b80 1 --2- 172.21.15.52:0/1685319678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578e10beb60 0x5578e10cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.325 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.309+0000 7fef9bfff700 0 client.16719 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:25.325 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.309+0000 7fefcf006b80 1 --2- 172.21.15.52:0/1685319678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578e10bf470 0x5578e10c8290 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.309+0000 7fefcf006b80 1 -- 172.21.15.52:0/1685319678 >> 172.21.15.52:0/1685319678 conn(0x5578e10b4360 msgr2=0x5578e10c4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:25.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.309+0000 7fefcf006b80 1 -- 172.21.15.52:0/1685319678 shutdown_connections 2023-11-27T15:25:25.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.309+0000 7fefcf006b80 1 -- 172.21.15.52:0/1685319678 wait complete. 2023-11-27T15:25:25.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.309+0000 7fefcf006b80 20 client.16719 trim_cache size 0 max 16384 2023-11-27T15:25:25.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.326+0000 7fefcf006b80 1 Processor -- start 2023-11-27T15:25:25.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.326+0000 7fefcf006b80 1 -- start start 2023-11-27T15:25:25.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.326+0000 7fefcf006b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578e10beb60 0x5578e10cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.326+0000 7fefcf006b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578e10bf470 0x5578e10cab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.326+0000 7fefcf006b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578e10c38b0 0x5578e10b9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.326+0000 7fefcf006b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578e10b1710 con 0x5578e10beb60 2023-11-27T15:25:25.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.326+0000 7fefcf006b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578e10b4170 con 0x5578e10c38b0 2023-11-27T15:25:25.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.326+0000 7fefcf006b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578e10c6830 con 0x5578e10bf470 2023-11-27T15:25:25.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.326+0000 7fefc0c16700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578e10beb60 0x5578e10cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.326+0000 7fefc0c16700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578e10beb60 0x5578e10cf940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55136/0 (socket says 172.21.15.52:55136) 2023-11-27T15:25:25.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.326+0000 7fefc0c16700 1 -- 172.21.15.52:0/16283073 learned_addr learned my addr 172.21.15.52:0/16283073 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:25.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.326+0000 7fefc1417700 1 --2- 172.21.15.52:0/16283073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578e10c38b0 0x5578e10b9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.327+0000 7fefbbfff700 1 --2- 172.21.15.52:0/16283073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578e10bf470 0x5578e10cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.327+0000 7fefc0c16700 1 -- 172.21.15.52:0/16283073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578e10bf470 msgr2=0x5578e10cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:25.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.327+0000 7fefc0c16700 1 --2- 172.21.15.52:0/16283073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578e10bf470 0x5578e10cab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.327+0000 7fefc0c16700 1 -- 172.21.15.52:0/16283073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578e10c38b0 msgr2=0x5578e10b9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:25.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.327+0000 7fefc0c16700 1 --2- 172.21.15.52:0/16283073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578e10c38b0 0x5578e10b9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.327+0000 7fefc0c16700 1 -- 172.21.15.52:0/16283073 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578e0fd5ee0 con 0x5578e10beb60 2023-11-27T15:25:25.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.327+0000 7fefc0c16700 1 --2- 172.21.15.52:0/16283073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578e10beb60 0x5578e10cf940 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fefb0009520 tx=0x7fefb0099b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81522f8e7019b93d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:25.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.327+0000 7fefa77fe700 1 -- 172.21.15.52:0/16283073 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb009d040 con 0x5578e10beb60 2023-11-27T15:25:25.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.327+0000 7fefa77fe700 1 -- 172.21.15.52:0/16283073 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefb0014050 con 0x5578e10beb60 2023-11-27T15:25:25.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.327+0000 7fefa77fe700 1 -- 172.21.15.52:0/16283073 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb009a120 con 0x5578e10beb60 2023-11-27T15:25:25.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.327+0000 7fefcf006b80 1 -- 172.21.15.52:0/16283073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578e10beb60 msgr2=0x5578e10cf940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:25.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.327+0000 7fefcf006b80 1 --2- 172.21.15.52:0/16283073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578e10beb60 0x5578e10cf940 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fefb0009520 tx=0x7fefb0099b00 comp rx=0 tx=0).stop 2023-11-27T15:25:25.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.328+0000 7fefcf006b80 1 -- 172.21.15.52:0/16283073 shutdown_connections 2023-11-27T15:25:25.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.328+0000 7fefcf006b80 1 --2- 172.21.15.52:0/16283073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578e10c38b0 0x5578e10b9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.328+0000 7fefcf006b80 1 --2- 172.21.15.52:0/16283073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578e10beb60 0x5578e10cf940 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.328+0000 7fefcf006b80 1 --2- 172.21.15.52:0/16283073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578e10bf470 0x5578e10cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.328+0000 7fefcf006b80 1 -- 172.21.15.52:0/16283073 >> 172.21.15.52:0/16283073 conn(0x5578e10b4360 msgr2=0x5578e10d4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:25.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.328+0000 7fefcf006b80 1 -- 172.21.15.52:0/16283073 shutdown_connections 2023-11-27T15:25:25.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.328+0000 7fefcf006b80 1 -- 172.21.15.52:0/16283073 wait complete. 2023-11-27T15:25:25.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.328+0000 7fefcf006b80 1 -- start start 2023-11-27T15:25:25.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.328+0000 7fefcf006b80 1 Processor -- start 2023-11-27T15:25:25.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.328+0000 7fefcf006b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578e10beb60 0x5578e10cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.328+0000 7fefcf006b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578e10bf470 0x5578e10cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.328+0000 7fefcf006b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578e10c38b0 0x5578e10cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.328+0000 7fefcf006b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578e10ce4b0 con 0x5578e10beb60 2023-11-27T15:25:25.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.328+0000 7fefcf006b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578e10ce620 con 0x5578e10bf470 2023-11-27T15:25:25.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.328+0000 7fefcf006b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578e10ce790 con 0x5578e10c38b0 2023-11-27T15:25:25.340 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.328+0000 7fefcf006b80 10 client.0 _ll_register_callbacks cb 0x7fefa801f4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:25.340 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.329+0000 7fefc0c16700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578e10beb60 0x5578e10cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.340 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.329+0000 7fefc0c16700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578e10beb60 0x5578e10cf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55142/0 (socket says 172.21.15.52:55142) 2023-11-27T15:25:25.340 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.329+0000 7fefc0c16700 1 -- 172.21.15.52:0/1857326234 learned_addr learned my addr 172.21.15.52:0/1857326234 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:25.340 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.329+0000 7fefc1417700 1 --2- 172.21.15.52:0/1857326234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578e10c38b0 0x5578e10cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.340 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.329+0000 7fefbbfff700 1 --2- 172.21.15.52:0/1857326234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578e10bf470 0x5578e10cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.340 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.329+0000 7fefc0c16700 1 -- 172.21.15.52:0/1857326234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578e10c38b0 msgr2=0x5578e10cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:25.341 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.329+0000 7fefc0c16700 1 --2- 172.21.15.52:0/1857326234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578e10c38b0 0x5578e10cdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.341 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.329+0000 7fefc0c16700 1 -- 172.21.15.52:0/1857326234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578e10bf470 msgr2=0x5578e10cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:25.341 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.329+0000 7fefc0c16700 1 --2- 172.21.15.52:0/1857326234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578e10bf470 0x5578e10cda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.341 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.329+0000 7fefc0c16700 1 -- 172.21.15.52:0/1857326234 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefbc10f190 con 0x5578e10beb60 2023-11-27T15:25:25.341 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.329+0000 7fefbbfff700 1 --2- 172.21.15.52:0/1857326234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578e10bf470 0x5578e10cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:25.341 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.329+0000 7fefc0c16700 1 --2- 172.21.15.52:0/1857326234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578e10beb60 0x5578e10cf770 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fefb0099bf0 tx=0x7fefb00a2170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e870f7f0ff8bbf3a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:25.341 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.329+0000 7fef9bfff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:25:25.341 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.329+0000 7fef9bfff700 1 -- 172.21.15.52:0/1857326234 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb00b8070 con 0x5578e10beb60 2023-11-27T15:25:25.342 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.329+0000 7fef9bfff700 1 -- 172.21.15.52:0/1857326234 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fefb0099150 con 0x5578e10beb60 2023-11-27T15:25:25.342 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.329+0000 7fefcf006b80 1 -- 172.21.15.52:0/1857326234 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fefb400bca0 con 0x5578e10beb60 2023-11-27T15:25:25.342 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.329+0000 7fef9bfff700 1 -- 172.21.15.52:0/1857326234 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefb009d070 con 0x5578e10beb60 2023-11-27T15:25:25.342 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.329+0000 7fefcf006b80 10 client.16725 fetch_fsmap learned FSMap version 30 2023-11-27T15:25:25.342 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.329+0000 7fefcf006b80 1 -- 172.21.15.52:0/1857326234 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5578e0e26c60 con 0x5578e10beb60 2023-11-27T15:25:25.342 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.329+0000 7fef9bfff700 1 -- 172.21.15.52:0/1857326234 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefb400bca0 con 0x5578e10beb60 2023-11-27T15:25:25.342 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.330+0000 7fef9bfff700 1 -- 172.21.15.52:0/1857326234 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fefb00a1c20 con 0x5578e10beb60 2023-11-27T15:25:25.343 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.330+0000 7fefcf006b80 10 client.16725 fetch_fsmap finished waiting for FSMap version 30 2023-11-27T15:25:25.343 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.330+0000 7fefcf006b80 10 client.16725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:25.343 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.330+0000 7fefcf006b80 20 client.16725 populate_metadata read hostname 'smithi052' 2023-11-27T15:25:25.343 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.330+0000 7fefcf006b80 1 --2- 172.21.15.52:0/1857326234 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5578e112d890 0x5578e112fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:25.343 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.330+0000 7fefcf006b80 4 client.16725 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:25:25.343 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.330+0000 7fefcf006b80 1 -- 172.21.15.52:0/1857326234 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5578e0fde660 con 0x5578e112d890 2023-11-27T15:25:25.343 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.330+0000 7fefbbfff700 1 --2- 172.21.15.52:0/1857326234 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5578e112d890 0x5578e112fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:25.343 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.330+0000 7fef9bfff700 10 client.16725 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:25.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.330+0000 7fefbbfff700 1 --2- 172.21.15.52:0/1857326234 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5578e112d890 0x5578e112fca0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=179113b60d062e6f server_cookie=3994bd3dc1d62c49 in_seq=0 out_seq=0 2023-11-27T15:25:25.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.331+0000 7fef9bfff700 1 -- 172.21.15.52:0/1857326234 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5578e0fde660 con 0x5578e112d890 2023-11-27T15:25:25.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.331+0000 7fef9bfff700 10 client.16725 handle_command_reply: tid=0 2023-11-27T15:25:25.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.331+0000 7fefcf006b80 1 client.16725 shutdown 2023-11-27T15:25:25.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.331+0000 7fefcf006b80 1 -- 172.21.15.52:0/1857326234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578e10beb60 msgr2=0x5578e10cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:25.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.331+0000 7fefcf006b80 1 --2- 172.21.15.52:0/1857326234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578e10beb60 0x5578e10cf770 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fefb0099bf0 tx=0x7fefb00a2170 comp rx=0 tx=0).stop 2023-11-27T15:25:25.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.331+0000 7fefcf006b80 1 -- 172.21.15.52:0/1857326234 shutdown_connections 2023-11-27T15:25:25.345 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.331+0000 7fefcf006b80 1 --2- 172.21.15.52:0/1857326234 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5578e112d890 0x5578e112fca0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.345 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.331+0000 7fefcf006b80 1 --2- 172.21.15.52:0/1857326234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5578e10c38b0 0x5578e10cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.345 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.331+0000 7fefcf006b80 1 --2- 172.21.15.52:0/1857326234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578e10beb60 0x5578e10cf770 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.345 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.331+0000 7fef9bfff700 0 client.16725 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:25.345 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.331+0000 7fefcf006b80 1 --2- 172.21.15.52:0/1857326234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5578e10bf470 0x5578e10cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.345 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.331+0000 7fefcf006b80 1 -- 172.21.15.52:0/1857326234 >> 172.21.15.52:0/1857326234 conn(0x5578e10b4360 msgr2=0x5578e10bd960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:25.345 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.331+0000 7fefcf006b80 1 -- 172.21.15.52:0/1857326234 shutdown_connections 2023-11-27T15:25:25.345 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.331+0000 7fefcf006b80 1 -- 172.21.15.52:0/1857326234 wait complete. 2023-11-27T15:25:25.346 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.331+0000 7fefcf006b80 20 client.16725 trim_cache size 0 max 16384 2023-11-27T15:25:25.346 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:25:25.346 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-27T15:25:25.346 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2023-11-27T15:25:25.346 INFO:teuthology.orchestra.run.smithi052.stdout: "da59e58e-0e8b-42de-9bd9-b2899ee7567d": { 2023-11-27T15:25:25.347 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2023-11-27T15:25:25.347 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "da59e58e-0e8b-42de-9bd9-b2899ee7567d", 2023-11-27T15:25:25.347 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2023-11-27T15:25:25.347 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:25:25.347 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:25:25.347 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:25:25.348 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.331+0000 7fefc2e7a700 1 -- 172.21.15.52:0/3736692255 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fefac0a12c0 msgr2=0x7fefac0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:25.349 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.331+0000 7fefc2e7a700 1 --2- 172.21.15.52:0/3736692255 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fefac0a12c0 0x7fefac0a3770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fefbc10c570 tx=0x7fefa8009260 comp rx=0 tx=0).stop 2023-11-27T15:25:25.349 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.331+0000 7fefc2e7a700 1 -- 172.21.15.52:0/3736692255 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefbc0ffc30 msgr2=0x7fefbc19e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:25.349 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.331+0000 7fefc2e7a700 1 --2- 172.21.15.52:0/3736692255 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefbc0ffc30 0x7fefbc19e480 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fefb000d950 tx=0x7fefb000dc60 comp rx=0 tx=0).stop 2023-11-27T15:25:25.349 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.331+0000 7fefc2e7a700 1 -- 172.21.15.52:0/3736692255 shutdown_connections 2023-11-27T15:25:25.349 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.331+0000 7fefc2e7a700 1 --2- 172.21.15.52:0/3736692255 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fefac0a12c0 0x7fefac0a3770 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.349 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.331+0000 7fefc2e7a700 1 --2- 172.21.15.52:0/3736692255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fefbc104590 0x7fefbc198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.349 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.331+0000 7fefc2e7a700 1 --2- 172.21.15.52:0/3736692255 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefbc0ffc30 0x7fefbc19e480 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.350 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.331+0000 7fefc2e7a700 1 --2- 172.21.15.52:0/3736692255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fefbc1005d0 0x7fefbc19e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:25.350 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.331+0000 7fefc2e7a700 1 -- 172.21.15.52:0/3736692255 >> 172.21.15.52:0/3736692255 conn(0x7fefbc075470 msgr2=0x7fefbc102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:25.350 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.332+0000 7fefc2e7a700 1 -- 172.21.15.52:0/3736692255 shutdown_connections 2023-11-27T15:25:25.350 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:25.332+0000 7fefc2e7a700 1 -- 172.21.15.52:0/3736692255 wait complete. 2023-11-27T15:25:25.350 INFO:tasks.cephfs.filesystem:scrub status for tag:da59e58e-0e8b-42de-9bd9-b2899ee7567d - {'path': '/', 'tag': 'da59e58e-0e8b-42de-9bd9-b2899ee7567d', 'options': 'recursive,force'} 2023-11-27T15:25:26.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:26 smithi150 ceph-mon[100559]: pgmap v232: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 90 B/s wr, 5 op/s 2023-11-27T15:25:26.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:26 smithi150 ceph-mon[100559]: scrub queued for path: / 2023-11-27T15:25:26.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:26 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:25:26.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:26 smithi150 ceph-mon[100559]: scrub summary: active paths [/] 2023-11-27T15:25:26.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:26 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:25:26.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:26 smithi102 ceph-mon[100296]: pgmap v232: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 90 B/s wr, 5 op/s 2023-11-27T15:25:26.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:26 smithi102 ceph-mon[100296]: scrub queued for path: / 2023-11-27T15:25:26.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:26 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:25:26.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:26 smithi102 ceph-mon[100296]: scrub summary: active paths [/] 2023-11-27T15:25:26.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:26 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:25:26.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:26 smithi052 ceph-mon[99056]: pgmap v232: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 90 B/s wr, 5 op/s 2023-11-27T15:25:26.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:26 smithi052 ceph-mon[99056]: scrub queued for path: / 2023-11-27T15:25:26.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:26 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:25:26.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:26 smithi052 ceph-mon[99056]: scrub summary: active paths [/] 2023-11-27T15:25:26.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:26 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:25:28.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:28 smithi150 ceph-mon[100559]: pgmap v233: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 7 op/s 2023-11-27T15:25:28.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:28 smithi102 ceph-mon[100296]: pgmap v233: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 7 op/s 2023-11-27T15:25:28.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:28 smithi052 ceph-mon[99056]: pgmap v233: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 7 op/s 2023-11-27T15:25:30.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:29 smithi150 ceph-mon[100559]: pgmap v234: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:25:30.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:29 smithi150 ceph-mon[100559]: scrub summary: idle 2023-11-27T15:25:30.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:29 smithi102 ceph-mon[100296]: pgmap v234: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:25:30.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:29 smithi102 ceph-mon[100296]: scrub summary: idle 2023-11-27T15:25:30.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:29 smithi052 ceph-mon[99056]: pgmap v234: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:25:30.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:29 smithi052 ceph-mon[99056]: scrub summary: idle 2023-11-27T15:25:32.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:32 smithi150 ceph-mon[100559]: pgmap v235: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:25:32.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:32 smithi102 ceph-mon[100296]: pgmap v235: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:25:32.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:32 smithi052 ceph-mon[99056]: pgmap v235: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:25:34.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:34 smithi150 ceph-mon[100559]: pgmap v236: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:25:34.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:34 smithi102 ceph-mon[100296]: pgmap v236: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:25:34.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:34 smithi052 ceph-mon[99056]: pgmap v236: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:25:36.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:36 smithi150 ceph-mon[100559]: pgmap v237: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:25:36.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:36 smithi102 ceph-mon[100296]: pgmap v237: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:25:36.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:36 smithi052 ceph-mon[99056]: pgmap v237: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:25:38.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:38 smithi150 ceph-mon[100559]: pgmap v238: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:25:38.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:38 smithi102 ceph-mon[100296]: pgmap v238: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:25:38.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:38 smithi052 ceph-mon[99056]: pgmap v238: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:25:40.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:39 smithi150 ceph-mon[100559]: pgmap v239: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:25:40.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:39 smithi102 ceph-mon[100296]: pgmap v239: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:25:40.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:39 smithi052 ceph-mon[99056]: pgmap v239: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:25:42.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:42 smithi150 ceph-mon[100559]: pgmap v240: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:25:42.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:42 smithi102 ceph-mon[100296]: pgmap v240: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:25:42.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:42 smithi052 ceph-mon[99056]: pgmap v240: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:25:44.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:44 smithi150 ceph-mon[100559]: pgmap v241: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:25:44.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:44 smithi102 ceph-mon[100296]: pgmap v241: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:25:44.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:44 smithi052 ceph-mon[99056]: pgmap v241: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:25:46.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:46 smithi150 ceph-mon[100559]: pgmap v242: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:25:46.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:46 smithi102 ceph-mon[100296]: pgmap v242: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:25:46.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:46 smithi052 ceph-mon[99056]: pgmap v242: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:25:48.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:48 smithi150 ceph-mon[100559]: pgmap v243: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:25:48.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:48 smithi102 ceph-mon[100296]: pgmap v243: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:25:48.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:48 smithi052 ceph-mon[99056]: pgmap v243: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:25:49.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:49 smithi150 ceph-mon[100559]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:25:49.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:49 smithi102 ceph-mon[100296]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:25:49.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:49 smithi052 ceph-mon[99056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:25:50.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:50 smithi150 ceph-mon[100559]: pgmap v244: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:25:50.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:50 smithi102 ceph-mon[100296]: pgmap v244: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:25:50.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:50 smithi052 ceph-mon[99056]: pgmap v244: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:25:51.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:51 smithi150 ceph-mon[100559]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:25:51.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:51 smithi102 ceph-mon[100296]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:25:51.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:51 smithi052 ceph-mon[99056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:25:52.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:52 smithi150 ceph-mon[100559]: pgmap v245: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:25:52.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:52 smithi102 ceph-mon[100296]: pgmap v245: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:25:52.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:52 smithi052 ceph-mon[99056]: pgmap v245: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:25:54.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:54 smithi150 ceph-mon[100559]: pgmap v246: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:25:54.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:54 smithi102 ceph-mon[100296]: pgmap v246: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:25:54.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:54 smithi052 ceph-mon[99056]: pgmap v246: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:25:55.351 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:25:55.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.489+0000 7f8c6bab0700 1 -- 172.21.15.52:0/241133183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c640fff30 msgr2=0x7f8c64103ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:55.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.489+0000 7f8c6bab0700 1 --2- 172.21.15.52:0/241133183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c640fff30 0x7f8c64103ef0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f8c540099c0 tx=0x7f8c54009cd0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.489+0000 7f8c6bab0700 1 -- 172.21.15.52:0/241133183 shutdown_connections 2023-11-27T15:25:55.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.489+0000 7f8c6bab0700 1 --2- 172.21.15.52:0/241133183 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c640ff590 0x7f8c640ff960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.489+0000 7f8c6bab0700 1 --2- 172.21.15.52:0/241133183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c641045d0 0x7f8c6410c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.489+0000 7f8c6bab0700 1 --2- 172.21.15.52:0/241133183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c640fff30 0x7f8c64103ef0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.489+0000 7f8c6bab0700 1 -- 172.21.15.52:0/241133183 >> 172.21.15.52:0/241133183 conn(0x7f8c640760d0 msgr2=0x7f8c640764d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:55.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.489+0000 7f8c6bab0700 1 -- 172.21.15.52:0/241133183 shutdown_connections 2023-11-27T15:25:55.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.490+0000 7f8c6bab0700 1 -- 172.21.15.52:0/241133183 wait complete. 2023-11-27T15:25:55.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.490+0000 7f8c6bab0700 1 Processor -- start 2023-11-27T15:25:55.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.490+0000 7f8c6bab0700 1 -- start start 2023-11-27T15:25:55.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.490+0000 7f8c6bab0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c640ff590 0x7f8c64196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:55.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.490+0000 7f8c6bab0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c640fff30 0x7f8c64196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:55.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.490+0000 7f8c6bab0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c641045d0 0x7f8c6419aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:55.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.490+0000 7f8c6bab0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6419b4f0 con 0x7f8c640fff30 2023-11-27T15:25:55.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.490+0000 7f8c6bab0700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6419b660 con 0x7f8c641045d0 2023-11-27T15:25:55.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.490+0000 7f8c6bab0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6419b820 con 0x7f8c640ff590 2023-11-27T15:25:55.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.490+0000 7f8c6904b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c640fff30 0x7f8c64196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:55.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.490+0000 7f8c6904b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c640fff30 0x7f8c64196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55158/0 (socket says 172.21.15.52:55158) 2023-11-27T15:25:55.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.490+0000 7f8c6904b700 1 -- 172.21.15.52:0/3981766256 learned_addr learned my addr 172.21.15.52:0/3981766256 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:55.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.491+0000 7f8c6984c700 1 --2- 172.21.15.52:0/3981766256 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c640ff590 0x7f8c64196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:55.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.491+0000 7f8c6a04d700 1 --2- 172.21.15.52:0/3981766256 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c641045d0 0x7f8c6419aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:55.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.491+0000 7f8c6904b700 1 -- 172.21.15.52:0/3981766256 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c640ff590 msgr2=0x7f8c64196510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:55.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.491+0000 7f8c6904b700 1 --2- 172.21.15.52:0/3981766256 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c640ff590 0x7f8c64196510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.491+0000 7f8c6904b700 1 -- 172.21.15.52:0/3981766256 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c641045d0 msgr2=0x7f8c6419aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:55.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.491+0000 7f8c6904b700 1 --2- 172.21.15.52:0/3981766256 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c641045d0 0x7f8c6419aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.491+0000 7f8c6904b700 1 -- 172.21.15.52:0/3981766256 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c540095f0 con 0x7f8c640fff30 2023-11-27T15:25:55.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.491+0000 7f8c6904b700 1 --2- 172.21.15.52:0/3981766256 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c640fff30 0x7f8c64196a50 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8c54009990 tx=0x7f8c5400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:55.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.491+0000 7f8c5affd700 1 -- 172.21.15.52:0/3981766256 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5401d070 con 0x7f8c640fff30 2023-11-27T15:25:55.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.491+0000 7f8c6bab0700 1 -- 172.21.15.52:0/3981766256 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c6410f3a0 con 0x7f8c640fff30 2023-11-27T15:25:55.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.491+0000 7f8c5affd700 1 -- 172.21.15.52:0/3981766256 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c54005020 con 0x7f8c640fff30 2023-11-27T15:25:55.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.491+0000 7f8c5affd700 1 -- 172.21.15.52:0/3981766256 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c54004a70 con 0x7f8c640fff30 2023-11-27T15:25:55.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.491+0000 7f8c6bab0700 1 -- 172.21.15.52:0/3981766256 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c6410f210 con 0x7f8c640fff30 2023-11-27T15:25:55.507 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.493+0000 7f8c5affd700 1 -- 172.21.15.52:0/3981766256 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f8c54005400 con 0x7f8c640fff30 2023-11-27T15:25:55.507 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.494+0000 7f8c5affd700 1 --2- 172.21.15.52:0/3981766256 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f8c500a12c0 0x7f8c500a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:55.507 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.494+0000 7f8c5affd700 1 -- 172.21.15.52:0/3981766256 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f8c540cd780 con 0x7f8c640fff30 2023-11-27T15:25:55.507 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.494+0000 7f8c6984c700 1 --2- 172.21.15.52:0/3981766256 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f8c500a12c0 0x7f8c500a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:55.507 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.494+0000 7f8c6984c700 1 --2- 172.21.15.52:0/3981766256 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f8c500a12c0 0x7f8c500a3770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8c60009c60 tx=0x7f8c60009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:55.507 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.495+0000 7f8c77c3cb80 1 Processor -- start 2023-11-27T15:25:55.507 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.495+0000 7f8c77c3cb80 1 -- start start 2023-11-27T15:25:55.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.495+0000 7f8c77c3cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560564dbdb60 0x560564dbdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:55.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.495+0000 7f8c77c3cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560564dbe470 0x560564dc2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:55.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.495+0000 7f8c77c3cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560564dc28b0 0x560564dc2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:55.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.495+0000 7f8c77c3cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x560564cc3060 con 0x560564dc28b0 2023-11-27T15:25:55.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.495+0000 7f8c77c3cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560564d158a0 con 0x560564dbdb60 2023-11-27T15:25:55.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.495+0000 7f8c77c3cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560564d0ed30 con 0x560564dbe470 2023-11-27T15:25:55.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.496+0000 7f8c6a04d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560564dc28b0 0x560564dc2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:55.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.496+0000 7f8c6a04d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560564dc28b0 0x560564dc2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55168/0 (socket says 172.21.15.52:55168) 2023-11-27T15:25:55.509 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.496+0000 7f8c6984c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560564dbdb60 0x560564dbdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:55.509 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.496+0000 7f8c6a04d700 1 -- 172.21.15.52:0/95323291 learned_addr learned my addr 172.21.15.52:0/95323291 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:55.509 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.496+0000 7f8c6904b700 1 --2- 172.21.15.52:0/95323291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560564dbe470 0x560564dc2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:55.509 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.496+0000 7f8c6a04d700 1 -- 172.21.15.52:0/95323291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560564dbe470 msgr2=0x560564dc2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:55.509 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.496+0000 7f8c6a04d700 1 --2- 172.21.15.52:0/95323291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560564dbe470 0x560564dc2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.509 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.496+0000 7f8c6a04d700 1 -- 172.21.15.52:0/95323291 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560564dbdb60 msgr2=0x560564dbdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:55.509 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.496+0000 7f8c6a04d700 1 --2- 172.21.15.52:0/95323291 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560564dbdb60 0x560564dbdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.509 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.496+0000 7f8c6a04d700 1 -- 172.21.15.52:0/95323291 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560564b90630 con 0x560564dc28b0 2023-11-27T15:25:55.509 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.496+0000 7f8c6a04d700 1 --2- 172.21.15.52:0/95323291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560564dc28b0 0x560564dc2d00 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c003fa0 tx=0x7f8c5c009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e58b7e5ebe756b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:55.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.496+0000 7f8c4ffff700 1 -- 172.21.15.52:0/95323291 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c019070 con 0x560564dc28b0 2023-11-27T15:25:55.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.496+0000 7f8c4ffff700 1 -- 172.21.15.52:0/95323291 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c5c00f040 con 0x560564dc28b0 2023-11-27T15:25:55.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.496+0000 7f8c4ffff700 1 -- 172.21.15.52:0/95323291 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c007a00 con 0x560564dc28b0 2023-11-27T15:25:55.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.497+0000 7f8c77c3cb80 1 -- 172.21.15.52:0/95323291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560564dc28b0 msgr2=0x560564dc2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:55.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.497+0000 7f8c77c3cb80 1 --2- 172.21.15.52:0/95323291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560564dc28b0 0x560564dc2d00 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c003fa0 tx=0x7f8c5c009ba0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.497+0000 7f8c77c3cb80 1 -- 172.21.15.52:0/95323291 shutdown_connections 2023-11-27T15:25:55.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.497+0000 7f8c77c3cb80 1 --2- 172.21.15.52:0/95323291 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560564dbdb60 0x560564dbdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.497+0000 7f8c77c3cb80 1 --2- 172.21.15.52:0/95323291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560564dc28b0 0x560564dc2d00 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.497+0000 7f8c77c3cb80 1 --2- 172.21.15.52:0/95323291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560564dbe470 0x560564dc2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.497+0000 7f8c77c3cb80 1 -- 172.21.15.52:0/95323291 >> 172.21.15.52:0/95323291 conn(0x560564db3360 msgr2=0x560564db8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:55.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.497+0000 7f8c77c3cb80 1 -- 172.21.15.52:0/95323291 shutdown_connections 2023-11-27T15:25:55.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.497+0000 7f8c77c3cb80 1 -- 172.21.15.52:0/95323291 wait complete. 2023-11-27T15:25:55.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.497+0000 7f8c77c3cb80 1 -- start start 2023-11-27T15:25:55.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.497+0000 7f8c77c3cb80 1 Processor -- start 2023-11-27T15:25:55.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.498+0000 7f8c77c3cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560564dbdb60 0x560564dc9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:55.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.498+0000 7f8c77c3cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560564dbe470 0x560564dc7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:55.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.498+0000 7f8c77c3cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560564dc28b0 0x560564dc7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:55.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.498+0000 7f8c77c3cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x560564cb08e0 con 0x560564dbe470 2023-11-27T15:25:55.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.498+0000 7f8c77c3cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560564cb0580 con 0x560564dc28b0 2023-11-27T15:25:55.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.498+0000 7f8c77c3cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560564b92920 con 0x560564dbdb60 2023-11-27T15:25:55.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.498+0000 7f8c77c3cb80 10 client.0 _ll_register_callbacks cb 0x560564cce780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:55.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.498+0000 7f8c6904b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560564dbe470 0x560564dc7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:55.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.498+0000 7f8c6904b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560564dbe470 0x560564dc7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55172/0 (socket says 172.21.15.52:55172) 2023-11-27T15:25:55.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.498+0000 7f8c6904b700 1 -- 172.21.15.52:0/1314691979 learned_addr learned my addr 172.21.15.52:0/1314691979 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:55.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.498+0000 7f8c6984c700 1 --2- 172.21.15.52:0/1314691979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560564dbdb60 0x560564dc9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:55.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.498+0000 7f8c6a04d700 1 --2- 172.21.15.52:0/1314691979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560564dc28b0 0x560564dc7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:55.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.498+0000 7f8c6904b700 1 -- 172.21.15.52:0/1314691979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560564dbdb60 msgr2=0x560564dc9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:55.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.498+0000 7f8c6904b700 1 --2- 172.21.15.52:0/1314691979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560564dbdb60 0x560564dc9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.498+0000 7f8c6904b700 1 -- 172.21.15.52:0/1314691979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560564dc28b0 msgr2=0x560564dc7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:55.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.498+0000 7f8c6904b700 1 --2- 172.21.15.52:0/1314691979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560564dc28b0 0x560564dc7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.498+0000 7f8c6904b700 1 -- 172.21.15.52:0/1314691979 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c5c003bd0 con 0x560564dbe470 2023-11-27T15:25:55.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.499+0000 7f8c6904b700 1 --2- 172.21.15.52:0/1314691979 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560564dbe470 0x560564dc7290 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8c5409a5b0 tx=0x7f8c54005190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0deb438d69caa37 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:55.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.499+0000 7f8c4d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:25:55.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.499+0000 7f8c4d7fa700 1 -- 172.21.15.52:0/1314691979 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c540b2070 con 0x560564dbe470 2023-11-27T15:25:55.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.499+0000 7f8c4d7fa700 1 -- 172.21.15.52:0/1314691979 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f8c540ba2d0 con 0x560564dbe470 2023-11-27T15:25:55.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.499+0000 7f8c77c3cb80 1 -- 172.21.15.52:0/1314691979 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560564db0b20 con 0x560564dbe470 2023-11-27T15:25:55.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.499+0000 7f8c4d7fa700 1 -- 172.21.15.52:0/1314691979 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c5409e8e0 con 0x560564dbe470 2023-11-27T15:25:55.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.499+0000 7f8c77c3cb80 10 client.16737 fetch_fsmap learned FSMap version 32 2023-11-27T15:25:55.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.499+0000 7f8c77c3cb80 1 -- 172.21.15.52:0/1314691979 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x560564cd5190 con 0x560564dbe470 2023-11-27T15:25:55.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.499+0000 7f8c4d7fa700 1 -- 172.21.15.52:0/1314691979 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560564db0b20 con 0x560564dbe470 2023-11-27T15:25:55.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.499+0000 7f8c4d7fa700 1 -- 172.21.15.52:0/1314691979 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8c54004bd0 con 0x560564dbe470 2023-11-27T15:25:55.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.499+0000 7f8c77c3cb80 10 client.16737 fetch_fsmap finished waiting for FSMap version 32 2023-11-27T15:25:55.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.499+0000 7f8c77c3cb80 10 client.16737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:55.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.499+0000 7f8c77c3cb80 20 client.16737 populate_metadata read hostname 'smithi052' 2023-11-27T15:25:55.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.499+0000 7f8c77c3cb80 1 --2- 172.21.15.52:0/1314691979 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560564dd7410 0x560564dd9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:55.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.499+0000 7f8c77c3cb80 4 client.16737 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:55.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.499+0000 7f8c77c3cb80 1 -- 172.21.15.52:0/1314691979 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560564cd10a0 con 0x560564dd7410 2023-11-27T15:25:55.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.500+0000 7f8c6984c700 1 --2- 172.21.15.52:0/1314691979 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560564dd7410 0x560564dd9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:55.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.500+0000 7f8c4d7fa700 10 client.16737 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:55.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.500+0000 7f8c6984c700 1 --2- 172.21.15.52:0/1314691979 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560564dd7410 0x560564dd9820 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=524f01d6b8c8e82d server_cookie=51c5ecbbeb5bfcd2 in_seq=0 out_seq=0 2023-11-27T15:25:55.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.501+0000 7f8c4d7fa700 1 -- 172.21.15.52:0/1314691979 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560564cd10a0 con 0x560564dd7410 2023-11-27T15:25:55.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.501+0000 7f8c4d7fa700 10 client.16737 handle_command_reply: tid=0 2023-11-27T15:25:55.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.501+0000 7f8c77c3cb80 1 client.16737 shutdown 2023-11-27T15:25:55.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.501+0000 7f8c77c3cb80 1 -- 172.21.15.52:0/1314691979 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560564dbe470 msgr2=0x560564dc7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:55.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.501+0000 7f8c77c3cb80 1 --2- 172.21.15.52:0/1314691979 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560564dbe470 0x560564dc7290 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8c5409a5b0 tx=0x7f8c54005190 comp rx=0 tx=0).stop 2023-11-27T15:25:55.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.501+0000 7f8c77c3cb80 1 -- 172.21.15.52:0/1314691979 shutdown_connections 2023-11-27T15:25:55.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.501+0000 7f8c77c3cb80 1 --2- 172.21.15.52:0/1314691979 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560564dd7410 0x560564dd9820 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.501+0000 7f8c77c3cb80 1 --2- 172.21.15.52:0/1314691979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560564dc28b0 0x560564dc7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.501+0000 7f8c77c3cb80 1 --2- 172.21.15.52:0/1314691979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560564dbdb60 0x560564dc9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.501+0000 7f8c4d7fa700 0 client.16737 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:55.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.501+0000 7f8c77c3cb80 1 --2- 172.21.15.52:0/1314691979 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560564dbe470 0x560564dc7290 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.501+0000 7f8c77c3cb80 1 -- 172.21.15.52:0/1314691979 >> 172.21.15.52:0/1314691979 conn(0x560564db3360 msgr2=0x560564dc3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:55.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.501+0000 7f8c77c3cb80 1 -- 172.21.15.52:0/1314691979 shutdown_connections 2023-11-27T15:25:55.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.501+0000 7f8c77c3cb80 1 -- 172.21.15.52:0/1314691979 wait complete. 2023-11-27T15:25:55.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.501+0000 7f8c77c3cb80 20 client.16737 trim_cache size 0 max 16384 2023-11-27T15:25:55.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.518+0000 7f8c77c3cb80 1 Processor -- start 2023-11-27T15:25:55.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.518+0000 7f8c77c3cb80 1 -- start start 2023-11-27T15:25:55.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.518+0000 7f8c77c3cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560564dbdb60 0x560564dce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:55.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.518+0000 7f8c77c3cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560564dbe470 0x560564dc9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:55.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.518+0000 7f8c77c3cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560564dc28b0 0x560564db8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:55.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.518+0000 7f8c77c3cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x560564dc1470 con 0x560564dbdb60 2023-11-27T15:25:55.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.518+0000 7f8c77c3cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560564db0960 con 0x560564dc28b0 2023-11-27T15:25:55.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.518+0000 7f8c77c3cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560564dc1180 con 0x560564dbe470 2023-11-27T15:25:55.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.519+0000 7f8c6984c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560564dbdb60 0x560564dce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:55.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.519+0000 7f8c6984c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560564dbdb60 0x560564dce9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55178/0 (socket says 172.21.15.52:55178) 2023-11-27T15:25:55.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.519+0000 7f8c6984c700 1 -- 172.21.15.52:0/3228020474 learned_addr learned my addr 172.21.15.52:0/3228020474 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:55.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.519+0000 7f8c6a04d700 1 --2- 172.21.15.52:0/3228020474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560564dc28b0 0x560564db8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:55.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.519+0000 7f8c6904b700 1 --2- 172.21.15.52:0/3228020474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560564dbe470 0x560564dc9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:55.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.519+0000 7f8c6984c700 1 -- 172.21.15.52:0/3228020474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560564dbe470 msgr2=0x560564dc9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:55.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.519+0000 7f8c6984c700 1 --2- 172.21.15.52:0/3228020474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560564dbe470 0x560564dc9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.519+0000 7f8c6984c700 1 -- 172.21.15.52:0/3228020474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560564dc28b0 msgr2=0x560564db8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:55.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.519+0000 7f8c6984c700 1 --2- 172.21.15.52:0/3228020474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560564dc28b0 0x560564db8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.519+0000 7f8c6984c700 1 -- 172.21.15.52:0/3228020474 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560564dba240 con 0x560564dbdb60 2023-11-27T15:25:55.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.519+0000 7f8c6a04d700 1 --2- 172.21.15.52:0/3228020474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560564dc28b0 0x560564db8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:55.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.519+0000 7f8c6904b700 1 --2- 172.21.15.52:0/3228020474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560564dbe470 0x560564dc9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:55.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.519+0000 7f8c6984c700 1 --2- 172.21.15.52:0/3228020474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560564dbdb60 0x560564dce9a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8c60013010 tx=0x7f8c60014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee67ec4b97f8b701 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:55.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.519+0000 7f8c58dac700 1 -- 172.21.15.52:0/3228020474 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c60004570 con 0x560564dbdb60 2023-11-27T15:25:55.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.519+0000 7f8c58dac700 1 -- 172.21.15.52:0/3228020474 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c60023040 con 0x560564dbdb60 2023-11-27T15:25:55.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.519+0000 7f8c58dac700 1 -- 172.21.15.52:0/3228020474 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6001d790 con 0x560564dbdb60 2023-11-27T15:25:55.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.520+0000 7f8c77c3cb80 1 -- 172.21.15.52:0/3228020474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560564dbdb60 msgr2=0x560564dce9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:55.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.520+0000 7f8c77c3cb80 1 --2- 172.21.15.52:0/3228020474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560564dbdb60 0x560564dce9a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8c60013010 tx=0x7f8c60014000 comp rx=0 tx=0).stop 2023-11-27T15:25:55.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.520+0000 7f8c77c3cb80 1 -- 172.21.15.52:0/3228020474 shutdown_connections 2023-11-27T15:25:55.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.520+0000 7f8c77c3cb80 1 --2- 172.21.15.52:0/3228020474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560564dc28b0 0x560564db8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.520+0000 7f8c77c3cb80 1 --2- 172.21.15.52:0/3228020474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560564dbdb60 0x560564dce9a0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.520+0000 7f8c77c3cb80 1 --2- 172.21.15.52:0/3228020474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560564dbe470 0x560564dc9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.520+0000 7f8c77c3cb80 1 -- 172.21.15.52:0/3228020474 >> 172.21.15.52:0/3228020474 conn(0x560564db3360 msgr2=0x560564dd3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:55.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.520+0000 7f8c77c3cb80 1 -- 172.21.15.52:0/3228020474 shutdown_connections 2023-11-27T15:25:55.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.520+0000 7f8c77c3cb80 1 -- 172.21.15.52:0/3228020474 wait complete. 2023-11-27T15:25:55.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.520+0000 7f8c77c3cb80 1 -- start start 2023-11-27T15:25:55.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.520+0000 7f8c77c3cb80 1 Processor -- start 2023-11-27T15:25:55.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.521+0000 7f8c77c3cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560564dbdb60 0x560564dce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:55.533 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.521+0000 7f8c77c3cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560564dbe470 0x560564dcc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:55.533 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.521+0000 7f8c77c3cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560564dc28b0 0x560564dccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:55.533 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.521+0000 7f8c77c3cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x560564dcd540 con 0x560564dbe470 2023-11-27T15:25:55.533 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.521+0000 7f8c77c3cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x560564dcd6b0 con 0x560564dbdb60 2023-11-27T15:25:55.533 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.521+0000 7f8c77c3cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560564dcd870 con 0x560564dc28b0 2023-11-27T15:25:55.533 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.521+0000 7f8c77c3cb80 10 client.0 _ll_register_callbacks cb 0x7f8c5409a4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:55.533 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.521+0000 7f8c6904b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560564dbe470 0x560564dcc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:55.533 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.521+0000 7f8c6904b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560564dbe470 0x560564dcc9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55186/0 (socket says 172.21.15.52:55186) 2023-11-27T15:25:55.534 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.521+0000 7f8c6904b700 1 -- 172.21.15.52:0/2189781830 learned_addr learned my addr 172.21.15.52:0/2189781830 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:55.534 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.521+0000 7f8c6984c700 1 --2- 172.21.15.52:0/2189781830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560564dbdb60 0x560564dce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:55.534 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.521+0000 7f8c6a04d700 1 --2- 172.21.15.52:0/2189781830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560564dc28b0 0x560564dccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:55.534 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.521+0000 7f8c6904b700 1 -- 172.21.15.52:0/2189781830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560564dc28b0 msgr2=0x560564dccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:55.534 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.521+0000 7f8c6904b700 1 --2- 172.21.15.52:0/2189781830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560564dc28b0 0x560564dccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.534 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.521+0000 7f8c6904b700 1 -- 172.21.15.52:0/2189781830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560564dbdb60 msgr2=0x560564dce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:55.534 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.521+0000 7f8c6904b700 1 --2- 172.21.15.52:0/2189781830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560564dbdb60 0x560564dce6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.521+0000 7f8c6904b700 1 -- 172.21.15.52:0/2189781830 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c60017070 con 0x560564dbe470 2023-11-27T15:25:55.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.521+0000 7f8c6a04d700 1 --2- 172.21.15.52:0/2189781830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560564dc28b0 0x560564dccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:55.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.521+0000 7f8c6904b700 1 --2- 172.21.15.52:0/2189781830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560564dbe470 0x560564dcc9f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8c540036c0 tx=0x7f8c54009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8e6359d2007912 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:55.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.521+0000 7f8c4d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:25:55.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.521+0000 7f8c4d7fa700 1 -- 172.21.15.52:0/2189781830 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c540b2070 con 0x560564dbe470 2023-11-27T15:25:55.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.521+0000 7f8c4d7fa700 1 -- 172.21.15.52:0/2189781830 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f8c54096740 con 0x560564dbe470 2023-11-27T15:25:55.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.521+0000 7f8c77c3cb80 1 -- 172.21.15.52:0/2189781830 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c5c00f040 con 0x560564dbe470 2023-11-27T15:25:55.535 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.522+0000 7f8c4d7fa700 1 -- 172.21.15.52:0/2189781830 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c540ba2d0 con 0x560564dbe470 2023-11-27T15:25:55.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.522+0000 7f8c77c3cb80 10 client.16743 fetch_fsmap learned FSMap version 32 2023-11-27T15:25:55.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.522+0000 7f8c77c3cb80 1 -- 172.21.15.52:0/2189781830 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x560564b3cde0 con 0x560564dbe470 2023-11-27T15:25:55.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.522+0000 7f8c4d7fa700 1 -- 172.21.15.52:0/2189781830 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c5407d020 con 0x560564dbe470 2023-11-27T15:25:55.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.522+0000 7f8c4d7fa700 1 -- 172.21.15.52:0/2189781830 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8c54066560 con 0x560564dbe470 2023-11-27T15:25:55.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.522+0000 7f8c77c3cb80 10 client.16743 fetch_fsmap finished waiting for FSMap version 32 2023-11-27T15:25:55.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.522+0000 7f8c77c3cb80 10 client.16743 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:55.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.522+0000 7f8c77c3cb80 20 client.16743 populate_metadata read hostname 'smithi052' 2023-11-27T15:25:55.536 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.522+0000 7f8c77c3cb80 1 --2- 172.21.15.52:0/2189781830 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560564e2c2c0 0x560564e2e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.522+0000 7f8c77c3cb80 4 client.16743 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:25:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.522+0000 7f8c77c3cb80 1 -- 172.21.15.52:0/2189781830 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560564cdd660 con 0x560564e2c2c0 2023-11-27T15:25:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.522+0000 7f8c6984c700 1 --2- 172.21.15.52:0/2189781830 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560564e2c2c0 0x560564e2e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.523+0000 7f8c4d7fa700 10 client.16743 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.523+0000 7f8c6984c700 1 --2- 172.21.15.52:0/2189781830 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560564e2c2c0 0x560564e2e6d0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5938b73e502b55af server_cookie=85711dfd750db833 in_seq=0 out_seq=0 2023-11-27T15:25:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.523+0000 7f8c4d7fa700 1 -- 172.21.15.52:0/2189781830 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560564cdd660 con 0x560564e2c2c0 2023-11-27T15:25:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.523+0000 7f8c4d7fa700 10 client.16743 handle_command_reply: tid=0 2023-11-27T15:25:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.523+0000 7f8c77c3cb80 1 client.16743 shutdown 2023-11-27T15:25:55.537 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.523+0000 7f8c77c3cb80 1 -- 172.21.15.52:0/2189781830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560564dbe470 msgr2=0x560564dcc9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.523+0000 7f8c77c3cb80 1 --2- 172.21.15.52:0/2189781830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560564dbe470 0x560564dcc9f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8c540036c0 tx=0x7f8c54009380 comp rx=0 tx=0).stop 2023-11-27T15:25:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.523+0000 7f8c77c3cb80 1 -- 172.21.15.52:0/2189781830 shutdown_connections 2023-11-27T15:25:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.523+0000 7f8c77c3cb80 1 --2- 172.21.15.52:0/2189781830 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x560564e2c2c0 0x560564e2e6d0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.523+0000 7f8c77c3cb80 1 --2- 172.21.15.52:0/2189781830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560564dbdb60 0x560564dce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.523+0000 7f8c77c3cb80 1 --2- 172.21.15.52:0/2189781830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560564dc28b0 0x560564dccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.523+0000 7f8c4d7fa700 0 client.16743 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:55.538 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.523+0000 7f8c77c3cb80 1 --2- 172.21.15.52:0/2189781830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560564dbe470 0x560564dcc9f0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.539 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.523+0000 7f8c77c3cb80 1 -- 172.21.15.52:0/2189781830 >> 172.21.15.52:0/2189781830 conn(0x560564db3360 msgr2=0x560564dc1810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:55.539 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.523+0000 7f8c77c3cb80 1 -- 172.21.15.52:0/2189781830 shutdown_connections 2023-11-27T15:25:55.539 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.523+0000 7f8c77c3cb80 1 -- 172.21.15.52:0/2189781830 wait complete. 2023-11-27T15:25:55.539 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.523+0000 7f8c77c3cb80 20 client.16743 trim_cache size 0 max 16384 2023-11-27T15:25:55.539 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:25:55.539 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2023-11-27T15:25:55.539 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2023-11-27T15:25:55.540 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:25:55.541 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.524+0000 7f8c6bab0700 1 -- 172.21.15.52:0/3981766256 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f8c500a12c0 msgr2=0x7f8c500a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:55.541 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.524+0000 7f8c6bab0700 1 --2- 172.21.15.52:0/3981766256 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f8c500a12c0 0x7f8c500a3770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8c60009c60 tx=0x7f8c60009260 comp rx=0 tx=0).stop 2023-11-27T15:25:55.541 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.524+0000 7f8c6bab0700 1 -- 172.21.15.52:0/3981766256 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c640fff30 msgr2=0x7f8c64196a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:55.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.524+0000 7f8c6bab0700 1 --2- 172.21.15.52:0/3981766256 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c640fff30 0x7f8c64196a50 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8c54009990 tx=0x7f8c5400f690 comp rx=0 tx=0).stop 2023-11-27T15:25:55.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.524+0000 7f8c6bab0700 1 -- 172.21.15.52:0/3981766256 shutdown_connections 2023-11-27T15:25:55.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.524+0000 7f8c6bab0700 1 --2- 172.21.15.52:0/3981766256 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f8c500a12c0 0x7f8c500a3770 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.524+0000 7f8c6bab0700 1 --2- 172.21.15.52:0/3981766256 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c641045d0 0x7f8c6419aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.524+0000 7f8c6bab0700 1 --2- 172.21.15.52:0/3981766256 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c640ff590 0x7f8c64196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.524+0000 7f8c6bab0700 1 --2- 172.21.15.52:0/3981766256 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c640fff30 0x7f8c64196a50 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.524+0000 7f8c6bab0700 1 -- 172.21.15.52:0/3981766256 >> 172.21.15.52:0/3981766256 conn(0x7f8c640760d0 msgr2=0x7f8c64068e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:55.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.524+0000 7f8c6bab0700 1 -- 172.21.15.52:0/3981766256 shutdown_connections 2023-11-27T15:25:55.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.524+0000 7f8c6bab0700 1 -- 172.21.15.52:0/3981766256 wait complete. 2023-11-27T15:25:55.543 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-27T15:25:55.543 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:25:55.669 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.664+0000 7f0f819a8700 1 -- 172.21.15.52:0/2579563300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f7c104460 msgr2=0x7f0f7c10c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:55.669 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.664+0000 7f0f819a8700 1 --2- 172.21.15.52:0/2579563300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f7c104460 0x7f0f7c10c030 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f0f7400b0d0 tx=0x7f0f7400b3e0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.664+0000 7f0f819a8700 1 -- 172.21.15.52:0/2579563300 shutdown_connections 2023-11-27T15:25:55.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.664+0000 7f0f819a8700 1 --2- 172.21.15.52:0/2579563300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f7c0ff420 0x7f0f7c0ff7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.664+0000 7f0f819a8700 1 --2- 172.21.15.52:0/2579563300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f7c104460 0x7f0f7c10c030 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.664+0000 7f0f819a8700 1 --2- 172.21.15.52:0/2579563300 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f7c0ffdc0 0x7f0f7c103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.664+0000 7f0f819a8700 1 -- 172.21.15.52:0/2579563300 >> 172.21.15.52:0/2579563300 conn(0x7f0f7c076310 msgr2=0x7f0f7c076710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:55.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.664+0000 7f0f819a8700 1 -- 172.21.15.52:0/2579563300 shutdown_connections 2023-11-27T15:25:55.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.664+0000 7f0f819a8700 1 -- 172.21.15.52:0/2579563300 wait complete. 2023-11-27T15:25:55.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.665+0000 7f0f819a8700 1 Processor -- start 2023-11-27T15:25:55.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.665+0000 7f0f819a8700 1 -- start start 2023-11-27T15:25:55.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.665+0000 7f0f819a8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f7c0ff420 0x7f0f7c100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:55.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.665+0000 7f0f819a8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f7c0ffdc0 0x7f0f7c101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:55.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.665+0000 7f0f819a8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f7c104460 0x7f0f7c1a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:55.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.665+0000 7f0f819a8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c101b90 con 0x7f0f7c104460 2023-11-27T15:25:55.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.665+0000 7f0f819a8700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c1a4bb0 con 0x7f0f7c0ff420 2023-11-27T15:25:55.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.665+0000 7f0f819a8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c1a4cf0 con 0x7f0f7c0ffdc0 2023-11-27T15:25:55.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.665+0000 7f0f7b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f7c104460 0x7f0f7c1a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:55.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.665+0000 7f0f7affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f7c0ff420 0x7f0f7c100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:55.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.665+0000 7f0f7b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f7c104460 0x7f0f7c1a4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55202/0 (socket says 172.21.15.52:55202) 2023-11-27T15:25:55.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.665+0000 7f0f7b7fe700 1 -- 172.21.15.52:0/3321175590 learned_addr learned my addr 172.21.15.52:0/3321175590 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:55.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.666+0000 7f0f7a7fc700 1 --2- 172.21.15.52:0/3321175590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f7c0ffdc0 0x7f0f7c101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:55.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.666+0000 7f0f7b7fe700 1 -- 172.21.15.52:0/3321175590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f7c0ffdc0 msgr2=0x7f0f7c101360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:55.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.666+0000 7f0f7b7fe700 1 --2- 172.21.15.52:0/3321175590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f7c0ffdc0 0x7f0f7c101360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.666+0000 7f0f7b7fe700 1 -- 172.21.15.52:0/3321175590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f7c0ff420 msgr2=0x7f0f7c100e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:55.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.666+0000 7f0f7b7fe700 1 --2- 172.21.15.52:0/3321175590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f7c0ff420 0x7f0f7c100e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.666+0000 7f0f7b7fe700 1 -- 172.21.15.52:0/3321175590 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f74009d00 con 0x7f0f7c104460 2023-11-27T15:25:55.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.666+0000 7f0f7affd700 1 --2- 172.21.15.52:0/3321175590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f7c0ff420 0x7f0f7c100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:55.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.666+0000 7f0f7b7fe700 1 --2- 172.21.15.52:0/3321175590 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f7c104460 0x7f0f7c1a4670 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0f74007870 tx=0x7f0f740078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:55.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.666+0000 7f0f809a6700 1 -- 172.21.15.52:0/3321175590 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7400e040 con 0x7f0f7c104460 2023-11-27T15:25:55.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.666+0000 7f0f809a6700 1 -- 172.21.15.52:0/3321175590 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f74005070 con 0x7f0f7c104460 2023-11-27T15:25:55.676 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.666+0000 7f0f819a8700 1 -- 172.21.15.52:0/3321175590 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f7c10f230 con 0x7f0f7c104460 2023-11-27T15:25:55.676 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.666+0000 7f0f809a6700 1 -- 172.21.15.52:0/3321175590 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7401b6c0 con 0x7f0f7c104460 2023-11-27T15:25:55.676 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.666+0000 7f0f819a8700 1 -- 172.21.15.52:0/3321175590 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f7c10f0a0 con 0x7f0f7c104460 2023-11-27T15:25:55.676 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.667+0000 7f0f819a8700 1 -- 172.21.15.52:0/3321175590 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f7c10f3c0 con 0x7f0f7c104460 2023-11-27T15:25:55.677 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.669+0000 7f0f809a6700 1 -- 172.21.15.52:0/3321175590 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0f740051e0 con 0x7f0f7c104460 2023-11-27T15:25:55.677 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.669+0000 7f0f809a6700 1 --2- 172.21.15.52:0/3321175590 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f0f640a12c0 0x7f0f640a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:55.677 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.669+0000 7f0f809a6700 1 -- 172.21.15.52:0/3321175590 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f0f740ce160 con 0x7f0f7c104460 2023-11-27T15:25:55.677 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.670+0000 7f0f7affd700 1 --2- 172.21.15.52:0/3321175590 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f0f640a12c0 0x7f0f640a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:55.677 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.670+0000 7f0f7affd700 1 --2- 172.21.15.52:0/3321175590 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f0f640a12c0 0x7f0f640a3770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c007fd0 tx=0x7f0f6c0073d0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:55.678 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.673+0000 7f0f809a6700 1 -- 172.21.15.52:0/3321175590 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0f740975d0 con 0x7f0f7c104460 2023-11-27T15:25:55.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.923+0000 7f0f819a8700 1 -- 172.21.15.52:0/3321175590 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f7c067a60 con 0x7f0f7c104460 2023-11-27T15:25:55.928 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.923+0000 7f0f809a6700 1 -- 172.21.15.52:0/3321175590 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+10806 (secure 0 0 0) 0x7f0f7409abf0 con 0x7f0f7c104460 2023-11-27T15:25:55.928 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:25:55.928 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2328881376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2328881376},{"type":"v1","addr":"172.21.15.52:6839","nonce":2328881376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2799551570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2799551570},{"type":"v1","addr":"172.21.15.52:6841","nonce":2799551570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:16:09.319630+0000","modified":"2023-11-27T15:25:50.220260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24494,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/215725838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":215725838},{"type":"v1","addr":"172.21.15.150:6839","nonce":215725838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3571830559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3571830559},{"type":"v1","addr":"172.21.15.102:6839","nonce":3571830559}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:25:55.931 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.926+0000 7f0f819a8700 1 -- 172.21.15.52:0/3321175590 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f0f640a12c0 msgr2=0x7f0f640a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:55.931 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.926+0000 7f0f819a8700 1 --2- 172.21.15.52:0/3321175590 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f0f640a12c0 0x7f0f640a3770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c007fd0 tx=0x7f0f6c0073d0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.931 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.926+0000 7f0f819a8700 1 -- 172.21.15.52:0/3321175590 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f7c104460 msgr2=0x7f0f7c1a4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:55.931 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.926+0000 7f0f819a8700 1 --2- 172.21.15.52:0/3321175590 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f7c104460 0x7f0f7c1a4670 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0f74007870 tx=0x7f0f740078a0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.931 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.926+0000 7f0f819a8700 1 -- 172.21.15.52:0/3321175590 shutdown_connections 2023-11-27T15:25:55.931 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.926+0000 7f0f819a8700 1 --2- 172.21.15.52:0/3321175590 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f0f640a12c0 0x7f0f640a3770 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.931 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.926+0000 7f0f819a8700 1 --2- 172.21.15.52:0/3321175590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f7c0ff420 0x7f0f7c100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.932 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.926+0000 7f0f819a8700 1 --2- 172.21.15.52:0/3321175590 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f7c104460 0x7f0f7c1a4670 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.932 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.926+0000 7f0f819a8700 1 --2- 172.21.15.52:0/3321175590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0f7c0ffdc0 0x7f0f7c101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:55.932 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.926+0000 7f0f819a8700 1 -- 172.21.15.52:0/3321175590 >> 172.21.15.52:0/3321175590 conn(0x7f0f7c076310 msgr2=0x7f0f7c067dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:55.932 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.926+0000 7f0f819a8700 1 -- 172.21.15.52:0/3321175590 shutdown_connections 2023-11-27T15:25:55.932 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:55.926+0000 7f0f819a8700 1 -- 172.21.15.52:0/3321175590 wait complete. 2023-11-27T15:25:55.932 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 32 2023-11-27T15:25:55.943 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-27T15:25:56.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.067+0000 7f8fa5b2a700 1 -- 172.21.15.52:0/4096264812 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fa00ff990 msgr2=0x7f8fa00ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:56.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.067+0000 7f8fa5b2a700 1 --2- 172.21.15.52:0/4096264812 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fa00ff990 0x7f8fa00ffe00 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8f940099c0 tx=0x7f8f94009cd0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.067+0000 7f8fa5b2a700 1 -- 172.21.15.52:0/4096264812 shutdown_connections 2023-11-27T15:25:56.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.067+0000 7f8fa5b2a700 1 --2- 172.21.15.52:0/4096264812 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fa0100340 0x7f8fa010c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.067+0000 7f8fa5b2a700 1 --2- 172.21.15.52:0/4096264812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fa0104180 0x7f8fa0104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.067+0000 7f8fa5b2a700 1 --2- 172.21.15.52:0/4096264812 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fa00ff990 0x7f8fa00ffe00 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.067+0000 7f8fa5b2a700 1 -- 172.21.15.52:0/4096264812 >> 172.21.15.52:0/4096264812 conn(0x7f8fa0075920 msgr2=0x7f8fa0075d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:56.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.067+0000 7f8fa5b2a700 1 -- 172.21.15.52:0/4096264812 shutdown_connections 2023-11-27T15:25:56.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.067+0000 7f8fa5b2a700 1 -- 172.21.15.52:0/4096264812 wait complete. 2023-11-27T15:25:56.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.068+0000 7f8fa5b2a700 1 Processor -- start 2023-11-27T15:25:56.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.068+0000 7f8fa5b2a700 1 -- start start 2023-11-27T15:25:56.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.068+0000 7f8fa5b2a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fa00ff990 0x7f8fa01963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.068+0000 7f8fa5b2a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fa0100340 0x7f8fa01968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.068+0000 7f8fa5b2a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fa0104180 0x7f8fa019ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.068+0000 7f8fa5b2a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa019b390 con 0x7f8fa0104180 2023-11-27T15:25:56.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.068+0000 7f8fa5b2a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa019b500 con 0x7f8fa00ff990 2023-11-27T15:25:56.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.068+0000 7f8fa5b2a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa019b6c0 con 0x7f8fa0100340 2023-11-27T15:25:56.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.068+0000 7f8f9ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fa0104180 0x7f8fa019ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.068+0000 7f8f9ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fa0104180 0x7f8fa019ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55216/0 (socket says 172.21.15.52:55216) 2023-11-27T15:25:56.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.068+0000 7f8f9ffff700 1 -- 172.21.15.52:0/3818655333 learned_addr learned my addr 172.21.15.52:0/3818655333 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:56.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.068+0000 7f8f9effd700 1 --2- 172.21.15.52:0/3818655333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fa0100340 0x7f8fa01968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.068+0000 7f8f9f7fe700 1 --2- 172.21.15.52:0/3818655333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fa00ff990 0x7f8fa01963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.068+0000 7f8f9ffff700 1 -- 172.21.15.52:0/3818655333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fa0100340 msgr2=0x7f8fa01968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:56.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.068+0000 7f8f9ffff700 1 --2- 172.21.15.52:0/3818655333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fa0100340 0x7f8fa01968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.068+0000 7f8f9ffff700 1 -- 172.21.15.52:0/3818655333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fa00ff990 msgr2=0x7f8fa01963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:56.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.068+0000 7f8f9ffff700 1 --2- 172.21.15.52:0/3818655333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fa00ff990 0x7f8fa01963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.068+0000 7f8f9ffff700 1 -- 172.21.15.52:0/3818655333 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f940095f0 con 0x7f8fa0104180 2023-11-27T15:25:56.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.068+0000 7f8f9f7fe700 1 --2- 172.21.15.52:0/3818655333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fa00ff990 0x7f8fa01963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:56.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.069+0000 7f8f9ffff700 1 --2- 172.21.15.52:0/3818655333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fa0104180 0x7f8fa019ad50 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f8f98007ee0 tx=0x7f8f9800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:56.078 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.069+0000 7f8f9cff9700 1 -- 172.21.15.52:0/3818655333 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9800ddd0 con 0x7f8fa0104180 2023-11-27T15:25:56.078 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.069+0000 7f8fa5b2a700 1 -- 172.21.15.52:0/3818655333 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa010f260 con 0x7f8fa0104180 2023-11-27T15:25:56.078 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.069+0000 7f8f9cff9700 1 -- 172.21.15.52:0/3818655333 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f9800f040 con 0x7f8fa0104180 2023-11-27T15:25:56.078 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.069+0000 7f8f9cff9700 1 -- 172.21.15.52:0/3818655333 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f98005a40 con 0x7f8fa0104180 2023-11-27T15:25:56.078 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.069+0000 7f8fa5b2a700 1 -- 172.21.15.52:0/3818655333 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa010f0d0 con 0x7f8fa0104180 2023-11-27T15:25:56.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.071+0000 7f8f9cff9700 1 -- 172.21.15.52:0/3818655333 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f8f98004ad0 con 0x7f8fa0104180 2023-11-27T15:25:56.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.072+0000 7f8f9cff9700 1 --2- 172.21.15.52:0/3818655333 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f8f880a12c0 0x7f8f880a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.072+0000 7f8f9cff9700 1 -- 172.21.15.52:0/3818655333 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f8f980cc760 con 0x7f8fa0104180 2023-11-27T15:25:56.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.072+0000 7f8f9f7fe700 1 --2- 172.21.15.52:0/3818655333 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f8f880a12c0 0x7f8f880a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.073+0000 7f8f9f7fe700 1 --2- 172.21.15.52:0/3818655333 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f8f880a12c0 0x7f8f880a3770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8f90005fd0 tx=0x7f8f90005dc0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:56.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.073+0000 7f8fb1cb6b80 1 Processor -- start 2023-11-27T15:25:56.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.073+0000 7f8fb1cb6b80 1 -- start start 2023-11-27T15:25:56.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.073+0000 7f8fb1cb6b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d3cb31b60 0x564d3cb31f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.073+0000 7f8fb1cb6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d3cb32470 0x564d3cb36370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.073+0000 7f8fb1cb6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d3cb368b0 0x564d3cb36d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.073+0000 7f8fb1cb6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3ca37060 con 0x564d3cb368b0 2023-11-27T15:25:56.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.073+0000 7f8fb1cb6b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3ca898a0 con 0x564d3cb31b60 2023-11-27T15:25:56.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.073+0000 7f8fb1cb6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3ca82d30 con 0x564d3cb32470 2023-11-27T15:25:56.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.073+0000 7f8f9ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d3cb368b0 0x564d3cb36d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.073+0000 7f8f9ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d3cb368b0 0x564d3cb36d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55224/0 (socket says 172.21.15.52:55224) 2023-11-27T15:25:56.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.073+0000 7f8f9ffff700 1 -- 172.21.15.52:0/2503170329 learned_addr learned my addr 172.21.15.52:0/2503170329 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:56.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.073+0000 7f8f9effd700 1 --2- 172.21.15.52:0/2503170329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d3cb32470 0x564d3cb36370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.087 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.073+0000 7f8f9f7fe700 1 --2- 172.21.15.52:0/2503170329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d3cb31b60 0x564d3cb31f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.087 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.073+0000 7f8f9ffff700 1 -- 172.21.15.52:0/2503170329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d3cb32470 msgr2=0x564d3cb36370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:56.087 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.073+0000 7f8f9ffff700 1 --2- 172.21.15.52:0/2503170329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d3cb32470 0x564d3cb36370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.087 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.073+0000 7f8f9ffff700 1 -- 172.21.15.52:0/2503170329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d3cb31b60 msgr2=0x564d3cb31f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:56.087 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.073+0000 7f8f9ffff700 1 --2- 172.21.15.52:0/2503170329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d3cb31b60 0x564d3cb31f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.087 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.073+0000 7f8f9ffff700 1 -- 172.21.15.52:0/2503170329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d3c904630 con 0x564d3cb368b0 2023-11-27T15:25:56.087 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.073+0000 7f8f9f7fe700 1 --2- 172.21.15.52:0/2503170329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d3cb31b60 0x564d3cb31f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:56.087 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.074+0000 7f8f9ffff700 1 --2- 172.21.15.52:0/2503170329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d3cb368b0 0x564d3cb36d00 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8f98095c00 tx=0x7f8f98099600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4b64aaae520ad6 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:56.088 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.074+0000 7f8f85ffb700 1 -- 172.21.15.52:0/2503170329 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f980a1b50 con 0x564d3cb368b0 2023-11-27T15:25:56.088 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.074+0000 7f8f85ffb700 1 -- 172.21.15.52:0/2503170329 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f98097aa0 con 0x564d3cb368b0 2023-11-27T15:25:56.088 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.074+0000 7f8f85ffb700 1 -- 172.21.15.52:0/2503170329 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f98098e10 con 0x564d3cb368b0 2023-11-27T15:25:56.088 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.074+0000 7f8fb1cb6b80 1 -- 172.21.15.52:0/2503170329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d3cb368b0 msgr2=0x564d3cb36d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:56.088 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.074+0000 7f8fb1cb6b80 1 --2- 172.21.15.52:0/2503170329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d3cb368b0 0x564d3cb36d00 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8f98095c00 tx=0x7f8f98099600 comp rx=0 tx=0).stop 2023-11-27T15:25:56.088 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.074+0000 7f8fb1cb6b80 1 -- 172.21.15.52:0/2503170329 shutdown_connections 2023-11-27T15:25:56.088 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.074+0000 7f8fb1cb6b80 1 --2- 172.21.15.52:0/2503170329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d3cb31b60 0x564d3cb31f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.089 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.074+0000 7f8fb1cb6b80 1 --2- 172.21.15.52:0/2503170329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d3cb368b0 0x564d3cb36d00 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.089 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.074+0000 7f8fb1cb6b80 1 --2- 172.21.15.52:0/2503170329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d3cb32470 0x564d3cb36370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.089 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.074+0000 7f8fb1cb6b80 1 -- 172.21.15.52:0/2503170329 >> 172.21.15.52:0/2503170329 conn(0x564d3cb27360 msgr2=0x564d3cb2cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:56.089 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.074+0000 7f8fb1cb6b80 1 -- 172.21.15.52:0/2503170329 shutdown_connections 2023-11-27T15:25:56.089 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.074+0000 7f8fb1cb6b80 1 -- 172.21.15.52:0/2503170329 wait complete. 2023-11-27T15:25:56.089 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.075+0000 7f8fb1cb6b80 1 -- start start 2023-11-27T15:25:56.089 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.075+0000 7f8fb1cb6b80 1 Processor -- start 2023-11-27T15:25:56.089 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.075+0000 7f8fb1cb6b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d3cb31b60 0x564d3cb3b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.089 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.075+0000 7f8fb1cb6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d3cb32470 0x564d3cb3b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.090 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.075+0000 7f8fb1cb6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d3cb368b0 0x564d3cb3bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.090 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.075+0000 7f8fb1cb6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3ca248e0 con 0x564d3cb368b0 2023-11-27T15:25:56.090 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.075+0000 7f8fb1cb6b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3ca24580 con 0x564d3cb31b60 2023-11-27T15:25:56.090 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.075+0000 7f8fb1cb6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3c906920 con 0x564d3cb32470 2023-11-27T15:25:56.090 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.075+0000 7f8fb1cb6b80 10 client.0 _ll_register_callbacks cb 0x564d3ca42780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:56.090 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.075+0000 7f8f9ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d3cb368b0 0x564d3cb3bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.090 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.075+0000 7f8f9f7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d3cb31b60 0x564d3cb3b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.090 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.075+0000 7f8f9f7fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d3cb31b60 0x564d3cb3b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:44884/0 (socket says 172.21.15.52:44884) 2023-11-27T15:25:56.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.075+0000 7f8f9f7fe700 1 -- 172.21.15.52:0/948933695 learned_addr learned my addr 172.21.15.52:0/948933695 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:56.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.075+0000 7f8f9ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d3cb368b0 0x564d3cb3bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55230/0 (socket says 172.21.15.52:55230) 2023-11-27T15:25:56.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.076+0000 7f8f9effd700 1 --2- 172.21.15.52:0/948933695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d3cb32470 0x564d3cb3b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.076+0000 7f8f9ffff700 1 -- 172.21.15.52:0/948933695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d3cb32470 msgr2=0x564d3cb3b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:56.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.076+0000 7f8f9ffff700 1 --2- 172.21.15.52:0/948933695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d3cb32470 0x564d3cb3b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.076+0000 7f8f9ffff700 1 -- 172.21.15.52:0/948933695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d3cb31b60 msgr2=0x564d3cb3b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:56.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.076+0000 7f8f9ffff700 1 --2- 172.21.15.52:0/948933695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d3cb31b60 0x564d3cb3b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.091 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.076+0000 7f8f9ffff700 1 -- 172.21.15.52:0/948933695 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa010f0d0 con 0x564d3cb368b0 2023-11-27T15:25:56.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.076+0000 7f8f9ffff700 1 --2- 172.21.15.52:0/948933695 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d3cb368b0 0x564d3cb3bdd0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f8f98095c00 tx=0x7f8f98009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f95a2f753cb21f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:56.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.076+0000 7f8f7effd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:25:56.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.076+0000 7f8f7effd700 1 -- 172.21.15.52:0/948933695 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f98099b60 con 0x564d3cb368b0 2023-11-27T15:25:56.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.076+0000 7f8f7effd700 1 -- 172.21.15.52:0/948933695 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f8f980a1b50 con 0x564d3cb368b0 2023-11-27T15:25:56.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.076+0000 7f8fb1cb6b80 1 -- 172.21.15.52:0/948933695 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564d3cb24b20 con 0x564d3cb368b0 2023-11-27T15:25:56.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.076+0000 7f8f7effd700 1 -- 172.21.15.52:0/948933695 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f980a16c0 con 0x564d3cb368b0 2023-11-27T15:25:56.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.076+0000 7f8fb1cb6b80 10 client.16761 fetch_fsmap learned FSMap version 32 2023-11-27T15:25:56.092 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.076+0000 7f8fb1cb6b80 1 -- 172.21.15.52:0/948933695 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564d3ca49190 con 0x564d3cb368b0 2023-11-27T15:25:56.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.076+0000 7f8f7effd700 1 -- 172.21.15.52:0/948933695 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564d3cb24b20 con 0x564d3cb368b0 2023-11-27T15:25:56.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.077+0000 7f8f7effd700 1 -- 172.21.15.52:0/948933695 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8f9809ab60 con 0x564d3cb368b0 2023-11-27T15:25:56.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.077+0000 7f8fb1cb6b80 10 client.16761 fetch_fsmap finished waiting for FSMap version 32 2023-11-27T15:25:56.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.077+0000 7f8fb1cb6b80 10 client.16761 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:25:56.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.077+0000 7f8fb1cb6b80 20 client.16761 populate_metadata read hostname 'smithi052' 2023-11-27T15:25:56.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.077+0000 7f8fb1cb6b80 1 --2- 172.21.15.52:0/948933695 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x564d3cb4b430 0x564d3cb4d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.077+0000 7f8fb1cb6b80 4 client.16761 mds_command: new command op to 24472 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:56.093 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.077+0000 7f8fb1cb6b80 1 -- 172.21.15.52:0/948933695 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d3ca450a0 con 0x564d3cb4b430 2023-11-27T15:25:56.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.077+0000 7f8f9f7fe700 1 --2- 172.21.15.52:0/948933695 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x564d3cb4b430 0x564d3cb4d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.077+0000 7f8f7effd700 10 client.16761 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:25:56.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.077+0000 7f8f9f7fe700 1 --2- 172.21.15.52:0/948933695 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x564d3cb4b430 0x564d3cb4d810 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d59b976d5d7a39c0 server_cookie=6231edbcef17daf4 in_seq=0 out_seq=0 2023-11-27T15:25:56.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.078+0000 7f8f7effd700 1 -- 172.21.15.52:0/948933695 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564d3ca450a0 con 0x564d3cb4b430 2023-11-27T15:25:56.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.078+0000 7f8f7effd700 10 client.16761 handle_command_reply: tid=0 2023-11-27T15:25:56.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.078+0000 7f8fb1cb6b80 1 client.16761 shutdown 2023-11-27T15:25:56.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.078+0000 7f8fb1cb6b80 1 -- 172.21.15.52:0/948933695 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d3cb368b0 msgr2=0x564d3cb3bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:56.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.078+0000 7f8fb1cb6b80 1 --2- 172.21.15.52:0/948933695 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d3cb368b0 0x564d3cb3bdd0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f8f98095c00 tx=0x7f8f98009ee0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.078+0000 7f8fb1cb6b80 1 -- 172.21.15.52:0/948933695 shutdown_connections 2023-11-27T15:25:56.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.079+0000 7f8fb1cb6b80 1 --2- 172.21.15.52:0/948933695 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x564d3cb4b430 0x564d3cb4d810 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.079+0000 7f8fb1cb6b80 1 --2- 172.21.15.52:0/948933695 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d3cb31b60 0x564d3cb3b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.079+0000 7f8fb1cb6b80 1 --2- 172.21.15.52:0/948933695 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d3cb368b0 0x564d3cb3bdd0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.079+0000 7f8f7effd700 0 client.16761 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:25:56.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.079+0000 7f8fb1cb6b80 1 --2- 172.21.15.52:0/948933695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d3cb32470 0x564d3cb3b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.079+0000 7f8fb1cb6b80 1 -- 172.21.15.52:0/948933695 >> 172.21.15.52:0/948933695 conn(0x564d3cb27360 msgr2=0x564d3cb37d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:56.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.079+0000 7f8fb1cb6b80 1 -- 172.21.15.52:0/948933695 shutdown_connections 2023-11-27T15:25:56.096 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.079+0000 7f8fb1cb6b80 1 -- 172.21.15.52:0/948933695 wait complete. 2023-11-27T15:25:56.096 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.079+0000 7f8fb1cb6b80 20 client.16761 trim_cache size 0 max 16384 2023-11-27T15:25:56.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.092+0000 7f8fb1cb6b80 1 Processor -- start 2023-11-27T15:25:56.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.092+0000 7f8fb1cb6b80 1 -- start start 2023-11-27T15:25:56.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.092+0000 7f8fb1cb6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d3cb31b60 0x564d3cb42940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.092+0000 7f8fb1cb6b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d3cb32470 0x564d3cb3bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.092+0000 7f8fb1cb6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d3cb368b0 0x564d3cb35890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.092+0000 7f8fb1cb6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3cb482a0 con 0x564d3cb368b0 2023-11-27T15:25:56.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.092+0000 7f8fb1cb6b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3cb4de70 con 0x564d3cb32470 2023-11-27T15:25:56.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.092+0000 7f8fb1cb6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3cb39900 con 0x564d3cb31b60 2023-11-27T15:25:56.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.092+0000 7f8f9ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d3cb368b0 0x564d3cb35890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.092+0000 7f8f9ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d3cb368b0 0x564d3cb35890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55238/0 (socket says 172.21.15.52:55238) 2023-11-27T15:25:56.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.092+0000 7f8f9ffff700 1 -- 172.21.15.52:0/87058668 learned_addr learned my addr 172.21.15.52:0/87058668 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:56.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.092+0000 7f8f9f7fe700 1 --2- 172.21.15.52:0/87058668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d3cb31b60 0x564d3cb42940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.092+0000 7f8f9effd700 1 --2- 172.21.15.52:0/87058668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d3cb32470 0x564d3cb3bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.092+0000 7f8f9ffff700 1 -- 172.21.15.52:0/87058668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d3cb31b60 msgr2=0x564d3cb42940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:56.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.092+0000 7f8f9ffff700 1 --2- 172.21.15.52:0/87058668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d3cb31b60 0x564d3cb42940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.092+0000 7f8f9ffff700 1 -- 172.21.15.52:0/87058668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d3cb32470 msgr2=0x564d3cb3bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:56.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.092+0000 7f8f9ffff700 1 --2- 172.21.15.52:0/87058668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d3cb32470 0x564d3cb3bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.092+0000 7f8f9ffff700 1 -- 172.21.15.52:0/87058668 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d3ca48ee0 con 0x564d3cb368b0 2023-11-27T15:25:56.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.093+0000 7f8f9ffff700 1 --2- 172.21.15.52:0/87058668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d3cb368b0 0x564d3cb35890 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f8f98096170 tx=0x7f8f98084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=777d751d3d33b5d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:56.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.093+0000 7f8f867fc700 1 -- 172.21.15.52:0/87058668 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9809d200 con 0x564d3cb368b0 2023-11-27T15:25:56.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.093+0000 7f8f867fc700 1 -- 172.21.15.52:0/87058668 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f980a1b50 con 0x564d3cb368b0 2023-11-27T15:25:56.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.093+0000 7f8f867fc700 1 -- 172.21.15.52:0/87058668 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f980652a0 con 0x564d3cb368b0 2023-11-27T15:25:56.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.093+0000 7f8fb1cb6b80 1 -- 172.21.15.52:0/87058668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d3cb368b0 msgr2=0x564d3cb35890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:56.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.093+0000 7f8fb1cb6b80 1 --2- 172.21.15.52:0/87058668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d3cb368b0 0x564d3cb35890 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f8f98096170 tx=0x7f8f98084040 comp rx=0 tx=0).stop 2023-11-27T15:25:56.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.093+0000 7f8fb1cb6b80 1 -- 172.21.15.52:0/87058668 shutdown_connections 2023-11-27T15:25:56.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.093+0000 7f8fb1cb6b80 1 --2- 172.21.15.52:0/87058668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d3cb368b0 0x564d3cb35890 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.093+0000 7f8fb1cb6b80 1 --2- 172.21.15.52:0/87058668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d3cb31b60 0x564d3cb42940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.093+0000 7f8fb1cb6b80 1 --2- 172.21.15.52:0/87058668 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d3cb32470 0x564d3cb3bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.093+0000 7f8fb1cb6b80 1 -- 172.21.15.52:0/87058668 >> 172.21.15.52:0/87058668 conn(0x564d3cb27360 msgr2=0x564d3cb38cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:56.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.093+0000 7f8fb1cb6b80 1 -- 172.21.15.52:0/87058668 shutdown_connections 2023-11-27T15:25:56.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.093+0000 7f8fb1cb6b80 1 -- 172.21.15.52:0/87058668 wait complete. 2023-11-27T15:25:56.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.094+0000 7f8fb1cb6b80 1 -- start start 2023-11-27T15:25:56.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.094+0000 7f8fb1cb6b80 1 Processor -- start 2023-11-27T15:25:56.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.094+0000 7f8fb1cb6b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d3cb31b60 0x564d3cb40930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.094+0000 7f8fb1cb6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d3cb32470 0x564d3cb40e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.094+0000 7f8fb1cb6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d3cb368b0 0x564d3cb413b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.094+0000 7f8fb1cb6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3cba2730 con 0x564d3cb32470 2023-11-27T15:25:56.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.094+0000 7f8fb1cb6b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3cba2870 con 0x564d3cb31b60 2023-11-27T15:25:56.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.094+0000 7f8fb1cb6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3cba2a30 con 0x564d3cb368b0 2023-11-27T15:25:56.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.094+0000 7f8fb1cb6b80 10 client.0 _ll_register_callbacks cb 0x7f8f94009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:56.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.094+0000 7f8f9effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d3cb32470 0x564d3cb40e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.094+0000 7f8f9effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d3cb32470 0x564d3cb40e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55242/0 (socket says 172.21.15.52:55242) 2023-11-27T15:25:56.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.094+0000 7f8f9effd700 1 -- 172.21.15.52:0/2851766389 learned_addr learned my addr 172.21.15.52:0/2851766389 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:56.106 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.094+0000 7f8f9ffff700 1 --2- 172.21.15.52:0/2851766389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d3cb368b0 0x564d3cb413b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.106 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.095+0000 7f8f9f7fe700 1 --2- 172.21.15.52:0/2851766389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d3cb31b60 0x564d3cb40930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.106 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.095+0000 7f8f9effd700 1 -- 172.21.15.52:0/2851766389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d3cb368b0 msgr2=0x564d3cb413b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:56.106 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.095+0000 7f8f9effd700 1 --2- 172.21.15.52:0/2851766389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d3cb368b0 0x564d3cb413b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.106 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.095+0000 7f8f9effd700 1 -- 172.21.15.52:0/2851766389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d3cb31b60 msgr2=0x564d3cb40930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:56.106 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.095+0000 7f8f9effd700 1 --2- 172.21.15.52:0/2851766389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d3cb31b60 0x564d3cb40930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.106 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.095+0000 7f8f9effd700 1 -- 172.21.15.52:0/2851766389 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d3ca49190 con 0x564d3cb32470 2023-11-27T15:25:56.106 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.095+0000 7f8f9effd700 1 --2- 172.21.15.52:0/2851766389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d3cb32470 0x564d3cb40e70 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f8f940099c0 tx=0x7f8f940057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26ab7becee2f07a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:56.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.095+0000 7f8f7effd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:25:56.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.095+0000 7f8f7effd700 1 -- 172.21.15.52:0/2851766389 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9400be10 con 0x564d3cb32470 2023-11-27T15:25:56.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.095+0000 7f8f7effd700 1 -- 172.21.15.52:0/2851766389 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f8f940038c0 con 0x564d3cb32470 2023-11-27T15:25:56.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.095+0000 7f8fb1cb6b80 1 -- 172.21.15.52:0/2851766389 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f98097aa0 con 0x564d3cb32470 2023-11-27T15:25:56.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.095+0000 7f8f7effd700 1 -- 172.21.15.52:0/2851766389 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f94003a80 con 0x564d3cb32470 2023-11-27T15:25:56.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.095+0000 7f8fb1cb6b80 10 client.16767 fetch_fsmap learned FSMap version 32 2023-11-27T15:25:56.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.095+0000 7f8fb1cb6b80 1 -- 172.21.15.52:0/2851766389 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564d3c899c60 con 0x564d3cb32470 2023-11-27T15:25:56.107 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.095+0000 7f8f7effd700 1 -- 172.21.15.52:0/2851766389 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f98097aa0 con 0x564d3cb32470 2023-11-27T15:25:56.108 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.095+0000 7f8f7effd700 1 -- 172.21.15.52:0/2851766389 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8f9402b020 con 0x564d3cb32470 2023-11-27T15:25:56.108 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.095+0000 7f8fb1cb6b80 10 client.16767 fetch_fsmap finished waiting for FSMap version 32 2023-11-27T15:25:56.108 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.095+0000 7f8fb1cb6b80 10 client.16767 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:25:56.108 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.095+0000 7f8fb1cb6b80 20 client.16767 populate_metadata read hostname 'smithi052' 2023-11-27T15:25:56.108 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.095+0000 7f8fb1cb6b80 1 --2- 172.21.15.52:0/2851766389 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x564d3cba40b0 0x564d3cba64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.108 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.096+0000 7f8fb1cb6b80 4 client.16767 mds_command: new command op to 24472 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:25:56.108 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.096+0000 7f8fb1cb6b80 1 -- 172.21.15.52:0/2851766389 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564d3ca51660 con 0x564d3cba40b0 2023-11-27T15:25:56.108 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.096+0000 7f8f9f7fe700 1 --2- 172.21.15.52:0/2851766389 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x564d3cba40b0 0x564d3cba64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.096+0000 7f8f7effd700 10 client.16767 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:25:56.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.096+0000 7f8f9f7fe700 1 --2- 172.21.15.52:0/2851766389 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x564d3cba40b0 0x564d3cba64c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=34acb4c49a6d87a1 server_cookie=2f5a25d3d16b1f27 in_seq=0 out_seq=0 2023-11-27T15:25:56.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.096+0000 7f8f7effd700 1 -- 172.21.15.52:0/2851766389 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564d3ca51660 con 0x564d3cba40b0 2023-11-27T15:25:56.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.096+0000 7f8f7effd700 10 client.16767 handle_command_reply: tid=0 2023-11-27T15:25:56.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.096+0000 7f8fb1cb6b80 1 client.16767 shutdown 2023-11-27T15:25:56.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.097+0000 7f8fb1cb6b80 1 -- 172.21.15.52:0/2851766389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d3cb32470 msgr2=0x564d3cb40e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:56.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.097+0000 7f8fb1cb6b80 1 --2- 172.21.15.52:0/2851766389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d3cb32470 0x564d3cb40e70 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f8f940099c0 tx=0x7f8f940057e0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.097+0000 7f8fb1cb6b80 1 -- 172.21.15.52:0/2851766389 shutdown_connections 2023-11-27T15:25:56.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.097+0000 7f8fb1cb6b80 1 --2- 172.21.15.52:0/2851766389 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x564d3cba40b0 0x564d3cba64c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.097+0000 7f8fb1cb6b80 1 --2- 172.21.15.52:0/2851766389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d3cb31b60 0x564d3cb40930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.097+0000 7f8fb1cb6b80 1 --2- 172.21.15.52:0/2851766389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564d3cb368b0 0x564d3cb413b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.097+0000 7f8fb1cb6b80 1 --2- 172.21.15.52:0/2851766389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564d3cb32470 0x564d3cb40e70 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.097+0000 7f8f7effd700 0 client.16767 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:25:56.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.097+0000 7f8fb1cb6b80 1 -- 172.21.15.52:0/2851766389 >> 172.21.15.52:0/2851766389 conn(0x564d3cb27360 msgr2=0x564d3cb39b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:56.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.097+0000 7f8fb1cb6b80 1 -- 172.21.15.52:0/2851766389 shutdown_connections 2023-11-27T15:25:56.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.097+0000 7f8fb1cb6b80 1 -- 172.21.15.52:0/2851766389 wait complete. 2023-11-27T15:25:56.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.097+0000 7f8fb1cb6b80 20 client.16767 trim_cache size 0 max 16384 2023-11-27T15:25:56.112 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:25:56.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.097+0000 7f8fa5b2a700 1 -- 172.21.15.52:0/3818655333 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f8f880a12c0 msgr2=0x7f8f880a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:56.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.097+0000 7f8fa5b2a700 1 --2- 172.21.15.52:0/3818655333 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f8f880a12c0 0x7f8f880a3770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8f90005fd0 tx=0x7f8f90005dc0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.097+0000 7f8fa5b2a700 1 -- 172.21.15.52:0/3818655333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fa0104180 msgr2=0x7f8fa019ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:56.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.097+0000 7f8fa5b2a700 1 --2- 172.21.15.52:0/3818655333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fa0104180 0x7f8fa019ad50 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f8f98007ee0 tx=0x7f8f9800d3b0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.097+0000 7f8fa5b2a700 1 -- 172.21.15.52:0/3818655333 shutdown_connections 2023-11-27T15:25:56.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.097+0000 7f8fa5b2a700 1 --2- 172.21.15.52:0/3818655333 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f8f880a12c0 0x7f8f880a3770 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.097+0000 7f8fa5b2a700 1 --2- 172.21.15.52:0/3818655333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fa00ff990 0x7f8fa01963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.097+0000 7f8fa5b2a700 1 --2- 172.21.15.52:0/3818655333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fa0104180 0x7f8fa019ad50 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.097+0000 7f8fa5b2a700 1 --2- 172.21.15.52:0/3818655333 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fa0100340 0x7f8fa01968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.097+0000 7f8fa5b2a700 1 -- 172.21.15.52:0/3818655333 >> 172.21.15.52:0/3818655333 conn(0x7f8fa0075920 msgr2=0x7f8fa00fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:56.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.098+0000 7f8fa5b2a700 1 -- 172.21.15.52:0/3818655333 shutdown_connections 2023-11-27T15:25:56.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.098+0000 7f8fa5b2a700 1 -- 172.21.15.52:0/3818655333 wait complete. 2023-11-27T15:25:56.115 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-27T15:25:56.243 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.239+0000 7f9735671700 1 -- 172.21.15.52:0/521155611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97301041f0 msgr2=0x7f97301045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:56.244 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.239+0000 7f9735671700 1 --2- 172.21.15.52:0/521155611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97301041f0 0x7f97301045c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f972000b0a0 tx=0x7f972000b3b0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.244 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.239+0000 7f9735671700 1 -- 172.21.15.52:0/521155611 shutdown_connections 2023-11-27T15:25:56.244 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.239+0000 7f9735671700 1 --2- 172.21.15.52:0/521155611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9730100470 0x7f973010c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.244 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.239+0000 7f9735671700 1 --2- 172.21.15.52:0/521155611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97301041f0 0x7f97301045c0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.244 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.239+0000 7f9735671700 1 --2- 172.21.15.52:0/521155611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97300ffac0 0x7f97300fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.244 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.239+0000 7f9735671700 1 -- 172.21.15.52:0/521155611 >> 172.21.15.52:0/521155611 conn(0x7f97300763a0 msgr2=0x7f97300767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:56.245 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.239+0000 7f9735671700 1 -- 172.21.15.52:0/521155611 shutdown_connections 2023-11-27T15:25:56.245 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.239+0000 7f9735671700 1 -- 172.21.15.52:0/521155611 wait complete. 2023-11-27T15:25:56.246 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.239+0000 7f9735671700 1 Processor -- start 2023-11-27T15:25:56.246 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.240+0000 7f9735671700 1 -- start start 2023-11-27T15:25:56.246 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.240+0000 7f9735671700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97300ffac0 0x7f9730197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.247 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.240+0000 7f9735671700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9730100470 0x7f97301977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.247 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.240+0000 7f9735671700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97301041f0 0x7f97301a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.247 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.240+0000 7f9735671700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97301980e0 con 0x7f97301041f0 2023-11-27T15:25:56.247 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.240+0000 7f9735671700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97301a2850 con 0x7f9730100470 2023-11-27T15:25:56.247 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.240+0000 7f9735671700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97301a29c0 con 0x7f97300ffac0 2023-11-27T15:25:56.247 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.240+0000 7f972f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97301041f0 0x7f97301a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.247 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.240+0000 7f972f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97301041f0 0x7f97301a2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55258/0 (socket says 172.21.15.52:55258) 2023-11-27T15:25:56.247 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.240+0000 7f972f7fe700 1 -- 172.21.15.52:0/161685717 learned_addr learned my addr 172.21.15.52:0/161685717 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:56.248 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.240+0000 7f972e7fc700 1 --2- 172.21.15.52:0/161685717 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9730100470 0x7f97301977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.248 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.240+0000 7f972effd700 1 --2- 172.21.15.52:0/161685717 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97300ffac0 0x7f9730197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.248 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.240+0000 7f972f7fe700 1 -- 172.21.15.52:0/161685717 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97300ffac0 msgr2=0x7f9730197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:56.248 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.240+0000 7f972f7fe700 1 --2- 172.21.15.52:0/161685717 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97300ffac0 0x7f9730197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.248 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.240+0000 7f972f7fe700 1 -- 172.21.15.52:0/161685717 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9730100470 msgr2=0x7f97301977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:56.248 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.240+0000 7f972f7fe700 1 --2- 172.21.15.52:0/161685717 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9730100470 0x7f97301977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.248 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.240+0000 7f972f7fe700 1 -- 172.21.15.52:0/161685717 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9720009d00 con 0x7f97301041f0 2023-11-27T15:25:56.248 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.241+0000 7f972f7fe700 1 --2- 172.21.15.52:0/161685717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97301041f0 0x7f97301a2310 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f972800e940 tx=0x7f972800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:56.251 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.242+0000 7f9717fff700 1 -- 172.21.15.52:0/161685717 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97280097b0 con 0x7f97301041f0 2023-11-27T15:25:56.251 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.242+0000 7f9717fff700 1 -- 172.21.15.52:0/161685717 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9728005020 con 0x7f97301041f0 2023-11-27T15:25:56.251 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.242+0000 7f9735671700 1 -- 172.21.15.52:0/161685717 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f973010f260 con 0x7f97301041f0 2023-11-27T15:25:56.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.242+0000 7f9717fff700 1 -- 172.21.15.52:0/161685717 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972800cc40 con 0x7f97301041f0 2023-11-27T15:25:56.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.242+0000 7f9735671700 1 -- 172.21.15.52:0/161685717 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973010f0d0 con 0x7f97301041f0 2023-11-27T15:25:56.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.244+0000 7f9717fff700 1 -- 172.21.15.52:0/161685717 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9728005400 con 0x7f97301041f0 2023-11-27T15:25:56.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.245+0000 7f9717fff700 1 --2- 172.21.15.52:0/161685717 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f97180a1340 0x7f97180a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.245+0000 7f972effd700 1 --2- 172.21.15.52:0/161685717 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f97180a1340 0x7f97180a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.245+0000 7f9717fff700 1 -- 172.21.15.52:0/161685717 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f9728014070 con 0x7f97301041f0 2023-11-27T15:25:56.252 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.245+0000 7f972effd700 1 --2- 172.21.15.52:0/161685717 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f97180a1340 0x7f97180a37f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9720009fd0 tx=0x7f972000b700 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:56.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.246+0000 7f97417fdb80 1 Processor -- start 2023-11-27T15:25:56.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.246+0000 7f97417fdb80 1 -- start start 2023-11-27T15:25:56.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.246+0000 7f97417fdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f269ab5f70 0x55f269ab6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.246+0000 7f97417fdb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f269ab6880 0x55f269aba780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.246+0000 7f97417fdb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f269abacc0 0x55f269abb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.246+0000 7f97417fdb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2699b7060 con 0x55f269abacc0 2023-11-27T15:25:56.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.246+0000 7f97417fdb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f269a098a0 con 0x55f269ab6880 2023-11-27T15:25:56.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.246+0000 7f97417fdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f269a02d30 con 0x55f269ab5f70 2023-11-27T15:25:56.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.246+0000 7f972f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f269abacc0 0x55f269abb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.246+0000 7f972f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f269abacc0 0x55f269abb110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55266/0 (socket says 172.21.15.52:55266) 2023-11-27T15:25:56.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.246+0000 7f972f7fe700 1 -- 172.21.15.52:0/1497824483 learned_addr learned my addr 172.21.15.52:0/1497824483 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:56.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.246+0000 7f972effd700 1 --2- 172.21.15.52:0/1497824483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f269ab5f70 0x55f269ab6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.246+0000 7f972e7fc700 1 --2- 172.21.15.52:0/1497824483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f269ab6880 0x55f269aba780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.246+0000 7f972f7fe700 1 -- 172.21.15.52:0/1497824483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f269ab5f70 msgr2=0x55f269ab6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:56.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.246+0000 7f972f7fe700 1 --2- 172.21.15.52:0/1497824483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f269ab5f70 0x55f269ab6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.246+0000 7f972f7fe700 1 -- 172.21.15.52:0/1497824483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f269ab6880 msgr2=0x55f269aba780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:56.255 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.246+0000 7f972f7fe700 1 --2- 172.21.15.52:0/1497824483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f269ab6880 0x55f269aba780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.255 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.246+0000 7f972f7fe700 1 -- 172.21.15.52:0/1497824483 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f269884630 con 0x55f269abacc0 2023-11-27T15:25:56.255 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.246+0000 7f972f7fe700 1 --2- 172.21.15.52:0/1497824483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f269abacc0 0x55f269abb110 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f972800f010 tx=0x7f9728099b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ea21757ccd207e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:56.255 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.247+0000 7f97157fa700 1 -- 172.21.15.52:0/1497824483 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97280a2350 con 0x55f269abacc0 2023-11-27T15:25:56.255 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.247+0000 7f97157fa700 1 -- 172.21.15.52:0/1497824483 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f972800cdc0 con 0x55f269abacc0 2023-11-27T15:25:56.255 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.247+0000 7f97157fa700 1 -- 172.21.15.52:0/1497824483 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9728099650 con 0x55f269abacc0 2023-11-27T15:25:56.259 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.247+0000 7f97417fdb80 1 -- 172.21.15.52:0/1497824483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f269abacc0 msgr2=0x55f269abb110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:56.259 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.247+0000 7f97417fdb80 1 --2- 172.21.15.52:0/1497824483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f269abacc0 0x55f269abb110 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f972800f010 tx=0x7f9728099b70 comp rx=0 tx=0).stop 2023-11-27T15:25:56.260 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.247+0000 7f97417fdb80 1 -- 172.21.15.52:0/1497824483 shutdown_connections 2023-11-27T15:25:56.260 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.247+0000 7f97417fdb80 1 --2- 172.21.15.52:0/1497824483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f269abacc0 0x55f269abb110 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.260 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.247+0000 7f97417fdb80 1 --2- 172.21.15.52:0/1497824483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f269ab5f70 0x55f269ab6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.260 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.247+0000 7f97417fdb80 1 --2- 172.21.15.52:0/1497824483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f269ab6880 0x55f269aba780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.260 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.247+0000 7f97417fdb80 1 -- 172.21.15.52:0/1497824483 >> 172.21.15.52:0/1497824483 conn(0x55f269aa7360 msgr2=0x55f269aacb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:56.260 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.247+0000 7f97417fdb80 1 -- 172.21.15.52:0/1497824483 shutdown_connections 2023-11-27T15:25:56.260 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.247+0000 7f97417fdb80 1 -- 172.21.15.52:0/1497824483 wait complete. 2023-11-27T15:25:56.261 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.248+0000 7f97417fdb80 1 -- start start 2023-11-27T15:25:56.261 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.248+0000 7f97417fdb80 1 Processor -- start 2023-11-27T15:25:56.261 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.248+0000 7f97417fdb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f269ab5f70 0x55f269ac24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.261 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.248+0000 7f97417fdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f269ab6880 0x55f269abf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.261 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.248+0000 7f97417fdb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f269abacc0 0x55f269abfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.261 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.248+0000 7f97417fdb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2699a48e0 con 0x55f269abacc0 2023-11-27T15:25:56.261 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.248+0000 7f97417fdb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2699a4580 con 0x55f269ab5f70 2023-11-27T15:25:56.261 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.248+0000 7f97417fdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f269886920 con 0x55f269ab6880 2023-11-27T15:25:56.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.248+0000 7f97417fdb80 10 client.0 _ll_register_callbacks cb 0x55f2699c2780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:56.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.248+0000 7f972f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f269abacc0 0x55f269abfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.248+0000 7f972f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f269abacc0 0x55f269abfe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55272/0 (socket says 172.21.15.52:55272) 2023-11-27T15:25:56.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.248+0000 7f972f7fe700 1 -- 172.21.15.52:0/1197465719 learned_addr learned my addr 172.21.15.52:0/1197465719 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:56.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.248+0000 7f972effd700 1 --2- 172.21.15.52:0/1197465719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f269ab5f70 0x55f269ac24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.248+0000 7f972e7fc700 1 --2- 172.21.15.52:0/1197465719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f269ab6880 0x55f269abf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.249+0000 7f972f7fe700 1 -- 172.21.15.52:0/1197465719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f269ab6880 msgr2=0x55f269abf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:56.262 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.249+0000 7f972f7fe700 1 --2- 172.21.15.52:0/1197465719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f269ab6880 0x55f269abf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.249+0000 7f972f7fe700 1 -- 172.21.15.52:0/1197465719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f269ab5f70 msgr2=0x55f269ac24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:56.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.249+0000 7f972f7fe700 1 --2- 172.21.15.52:0/1197465719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f269ab5f70 0x55f269ac24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.249+0000 7f972f7fe700 1 -- 172.21.15.52:0/1197465719 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973010f0d0 con 0x55f269abacc0 2023-11-27T15:25:56.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.249+0000 7f972e7fc700 1 --2- 172.21.15.52:0/1197465719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f269ab6880 0x55f269abf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:56.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.249+0000 7f972f7fe700 1 --2- 172.21.15.52:0/1197465719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f269abacc0 0x55f269abfe40 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f97280963b0 tx=0x7f97280047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e031b5ce3e6aafa6 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:56.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.249+0000 7f9712ff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:25:56.263 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.249+0000 7f9712ff5700 1 -- 172.21.15.52:0/1197465719 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97280a2350 con 0x55f269abacc0 2023-11-27T15:25:56.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.249+0000 7f9712ff5700 1 -- 172.21.15.52:0/1197465719 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f972809a160 con 0x55f269abacc0 2023-11-27T15:25:56.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.249+0000 7f97417fdb80 1 -- 172.21.15.52:0/1197465719 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f269aa4b20 con 0x55f269abacc0 2023-11-27T15:25:56.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.249+0000 7f9712ff5700 1 -- 172.21.15.52:0/1197465719 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f972809a320 con 0x55f269abacc0 2023-11-27T15:25:56.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.249+0000 7f97417fdb80 10 client.16779 fetch_fsmap learned FSMap version 32 2023-11-27T15:25:56.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.249+0000 7f97417fdb80 1 -- 172.21.15.52:0/1197465719 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f2699c9190 con 0x55f269abacc0 2023-11-27T15:25:56.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.249+0000 7f9712ff5700 1 -- 172.21.15.52:0/1197465719 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f269aa4b20 con 0x55f269abacc0 2023-11-27T15:25:56.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.249+0000 7f9712ff5700 1 -- 172.21.15.52:0/1197465719 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9728094c10 con 0x55f269abacc0 2023-11-27T15:25:56.264 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.249+0000 7f97417fdb80 10 client.16779 fetch_fsmap finished waiting for FSMap version 32 2023-11-27T15:25:56.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.249+0000 7f97417fdb80 10 client.16779 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:25:56.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.249+0000 7f97417fdb80 20 client.16779 populate_metadata read hostname 'smithi052' 2023-11-27T15:25:56.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.249+0000 7f97417fdb80 1 --2- 172.21.15.52:0/1197465719 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55f269acf9f0 0x55f269ad1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.249+0000 7f97417fdb80 4 client.16779 mds_command: new command op to 24494 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:56.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.249+0000 7f97417fdb80 1 -- 172.21.15.52:0/1197465719 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2699c50a0 con 0x55f269acf9f0 2023-11-27T15:25:56.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.250+0000 7f972effd700 1 --2- 172.21.15.52:0/1197465719 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55f269acf9f0 0x55f269ad1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.250+0000 7f9712ff5700 10 client.16779 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:25:56.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.250+0000 7f972effd700 1 --2- 172.21.15.52:0/1197465719 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55f269acf9f0 0x55f269ad1e00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b4dcd67ac999b744 server_cookie=27b4344730d81d8c in_seq=0 out_seq=0 2023-11-27T15:25:56.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.251+0000 7f9712ff5700 1 -- 172.21.15.52:0/1197465719 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f2699c50a0 con 0x55f269acf9f0 2023-11-27T15:25:56.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.251+0000 7f9712ff5700 10 client.16779 handle_command_reply: tid=0 2023-11-27T15:25:56.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.251+0000 7f97417fdb80 1 client.16779 shutdown 2023-11-27T15:25:56.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.251+0000 7f97417fdb80 1 -- 172.21.15.52:0/1197465719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f269abacc0 msgr2=0x55f269abfe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:56.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.251+0000 7f97417fdb80 1 --2- 172.21.15.52:0/1197465719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f269abacc0 0x55f269abfe40 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f97280963b0 tx=0x7f97280047c0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.251+0000 7f97417fdb80 1 -- 172.21.15.52:0/1197465719 shutdown_connections 2023-11-27T15:25:56.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.251+0000 7f97417fdb80 1 --2- 172.21.15.52:0/1197465719 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55f269acf9f0 0x55f269ad1e00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.267 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.251+0000 7f97417fdb80 1 --2- 172.21.15.52:0/1197465719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f269ab5f70 0x55f269ac24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.267 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.251+0000 7f9712ff5700 0 client.16779 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:25:56.267 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.251+0000 7f97417fdb80 1 --2- 172.21.15.52:0/1197465719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f269abacc0 0x55f269abfe40 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.267 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.251+0000 7f97417fdb80 1 --2- 172.21.15.52:0/1197465719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f269ab6880 0x55f269abf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.267 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.251+0000 7f97417fdb80 1 -- 172.21.15.52:0/1197465719 >> 172.21.15.52:0/1197465719 conn(0x55f269aa7360 msgr2=0x55f269abc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:56.267 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.251+0000 7f97417fdb80 1 -- 172.21.15.52:0/1197465719 shutdown_connections 2023-11-27T15:25:56.267 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.251+0000 7f97417fdb80 1 -- 172.21.15.52:0/1197465719 wait complete. 2023-11-27T15:25:56.267 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.251+0000 7f97417fdb80 20 client.16779 trim_cache size 0 max 16384 2023-11-27T15:25:56.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.264+0000 7f97417fdb80 1 Processor -- start 2023-11-27T15:25:56.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.264+0000 7f97417fdb80 1 -- start start 2023-11-27T15:25:56.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.264+0000 7f97417fdb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f269ab5f70 0x55f269abef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.264+0000 7f97417fdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f269ab6880 0x55f269ab8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.264+0000 7f97417fdb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f269abacc0 0x55f269ab9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.264+0000 7f97417fdb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f269abdcb0 con 0x55f269abacc0 2023-11-27T15:25:56.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.264+0000 7f97417fdb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f269ab9870 con 0x55f269ab5f70 2023-11-27T15:25:56.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.264+0000 7f97417fdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f269acbc70 con 0x55f269ab6880 2023-11-27T15:25:56.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.264+0000 7f972f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f269abacc0 0x55f269ab9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.264+0000 7f972f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f269abacc0 0x55f269ab9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55280/0 (socket says 172.21.15.52:55280) 2023-11-27T15:25:56.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.264+0000 7f972f7fe700 1 -- 172.21.15.52:0/3483334927 learned_addr learned my addr 172.21.15.52:0/3483334927 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:56.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.265+0000 7f972effd700 1 --2- 172.21.15.52:0/3483334927 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f269ab5f70 0x55f269abef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.265+0000 7f972e7fc700 1 --2- 172.21.15.52:0/3483334927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f269ab6880 0x55f269ab8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.265+0000 7f972f7fe700 1 -- 172.21.15.52:0/3483334927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f269ab6880 msgr2=0x55f269ab8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:56.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.265+0000 7f972f7fe700 1 --2- 172.21.15.52:0/3483334927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f269ab6880 0x55f269ab8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.265+0000 7f972f7fe700 1 -- 172.21.15.52:0/3483334927 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f269ab5f70 msgr2=0x55f269abef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:56.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.265+0000 7f972f7fe700 1 --2- 172.21.15.52:0/3483334927 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f269ab5f70 0x55f269abef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.265+0000 7f972f7fe700 1 -- 172.21.15.52:0/3483334927 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2699c8ee0 con 0x55f269abacc0 2023-11-27T15:25:56.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.265+0000 7f972e7fc700 1 --2- 172.21.15.52:0/3483334927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f269ab6880 0x55f269ab8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:56.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.265+0000 7f972f7fe700 1 --2- 172.21.15.52:0/3483334927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f269abacc0 0x55f269ab9280 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f9728096970 tx=0x7f972809e0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b809874c147217 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:56.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.265+0000 7f9715ffb700 1 -- 172.21.15.52:0/3483334927 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9728099cd0 con 0x55f269abacc0 2023-11-27T15:25:56.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.265+0000 7f9715ffb700 1 -- 172.21.15.52:0/3483334927 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9728083400 con 0x55f269abacc0 2023-11-27T15:25:56.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.265+0000 7f9715ffb700 1 -- 172.21.15.52:0/3483334927 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972809a160 con 0x55f269abacc0 2023-11-27T15:25:56.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.265+0000 7f97417fdb80 1 -- 172.21.15.52:0/3483334927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f269abacc0 msgr2=0x55f269ab9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:56.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.265+0000 7f97417fdb80 1 --2- 172.21.15.52:0/3483334927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f269abacc0 0x55f269ab9280 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f9728096970 tx=0x7f972809e0b0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.266+0000 7f97417fdb80 1 -- 172.21.15.52:0/3483334927 shutdown_connections 2023-11-27T15:25:56.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.266+0000 7f97417fdb80 1 --2- 172.21.15.52:0/3483334927 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f269ab5f70 0x55f269abef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.266+0000 7f97417fdb80 1 --2- 172.21.15.52:0/3483334927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f269abacc0 0x55f269ab9280 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.266+0000 7f97417fdb80 1 --2- 172.21.15.52:0/3483334927 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f269ab6880 0x55f269ab8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.277 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.266+0000 7f97417fdb80 1 -- 172.21.15.52:0/3483334927 >> 172.21.15.52:0/3483334927 conn(0x55f269aa7360 msgr2=0x55f269acb560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:56.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.266+0000 7f97417fdb80 1 -- 172.21.15.52:0/3483334927 shutdown_connections 2023-11-27T15:25:56.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.266+0000 7f97417fdb80 1 -- 172.21.15.52:0/3483334927 wait complete. 2023-11-27T15:25:56.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.266+0000 7f97417fdb80 1 -- start start 2023-11-27T15:25:56.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.266+0000 7f97417fdb80 1 Processor -- start 2023-11-27T15:25:56.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.266+0000 7f97417fdb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f269ab5f70 0x55f269ac6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.266+0000 7f97417fdb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f269ab6880 0x55f269ac4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.266+0000 7f97417fdb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f269abacc0 0x55f269ac5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.266+0000 7f97417fdb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f269ac5b70 con 0x55f269ab6880 2023-11-27T15:25:56.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.266+0000 7f97417fdb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f269b228b0 con 0x55f269abacc0 2023-11-27T15:25:56.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.266+0000 7f97417fdb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f269b229f0 con 0x55f269ab5f70 2023-11-27T15:25:56.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.266+0000 7f97417fdb80 10 client.0 _ll_register_callbacks cb 0x7f9728099910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:56.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.266+0000 7f972e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f269ab6880 0x55f269ac4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.266+0000 7f972e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f269ab6880 0x55f269ac4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55284/0 (socket says 172.21.15.52:55284) 2023-11-27T15:25:56.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.266+0000 7f972e7fc700 1 -- 172.21.15.52:0/383692041 learned_addr learned my addr 172.21.15.52:0/383692041 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:56.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.266+0000 7f972effd700 1 --2- 172.21.15.52:0/383692041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f269ab5f70 0x55f269ac6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.267+0000 7f972f7fe700 1 --2- 172.21.15.52:0/383692041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f269abacc0 0x55f269ac5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.267+0000 7f972e7fc700 1 -- 172.21.15.52:0/383692041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f269ab5f70 msgr2=0x55f269ac6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:56.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.267+0000 7f972e7fc700 1 --2- 172.21.15.52:0/383692041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f269ab5f70 0x55f269ac6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.267+0000 7f972e7fc700 1 -- 172.21.15.52:0/383692041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f269abacc0 msgr2=0x55f269ac5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:56.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.267+0000 7f972e7fc700 1 --2- 172.21.15.52:0/383692041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f269abacc0 0x55f269ac5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.267+0000 7f972e7fc700 1 -- 172.21.15.52:0/383692041 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2699c9190 con 0x55f269ab6880 2023-11-27T15:25:56.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.267+0000 7f972e7fc700 1 --2- 172.21.15.52:0/383692041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f269ab6880 0x55f269ac4d70 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f9730069a50 tx=0x7f9724010a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d47c601714cf95 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:56.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.267+0000 7f9712ff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:25:56.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.267+0000 7f9712ff5700 1 -- 172.21.15.52:0/383692041 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9724004a30 con 0x55f269ab6880 2023-11-27T15:25:56.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.267+0000 7f9712ff5700 1 -- 172.21.15.52:0/383692041 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f9724004b90 con 0x55f269ab6880 2023-11-27T15:25:56.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.267+0000 7f97417fdb80 1 -- 172.21.15.52:0/383692041 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f972800cdc0 con 0x55f269ab6880 2023-11-27T15:25:56.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.267+0000 7f9712ff5700 1 -- 172.21.15.52:0/383692041 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9724004d50 con 0x55f269ab6880 2023-11-27T15:25:56.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.267+0000 7f97417fdb80 10 client.16785 fetch_fsmap learned FSMap version 32 2023-11-27T15:25:56.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.267+0000 7f97417fdb80 1 -- 172.21.15.52:0/383692041 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f269819c60 con 0x55f269ab6880 2023-11-27T15:25:56.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.267+0000 7f9712ff5700 1 -- 172.21.15.52:0/383692041 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9724027020 con 0x55f269ab6880 2023-11-27T15:25:56.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.268+0000 7f9712ff5700 1 -- 172.21.15.52:0/383692041 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f972400fbe0 con 0x55f269ab6880 2023-11-27T15:25:56.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.268+0000 7f97417fdb80 10 client.16785 fetch_fsmap finished waiting for FSMap version 32 2023-11-27T15:25:56.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.268+0000 7f97417fdb80 10 client.16785 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:25:56.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.268+0000 7f97417fdb80 20 client.16785 populate_metadata read hostname 'smithi052' 2023-11-27T15:25:56.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.268+0000 7f97417fdb80 1 --2- 172.21.15.52:0/383692041 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55f269acf9f0 0x55f269b26110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.268+0000 7f97417fdb80 4 client.16785 mds_command: new command op to 24494 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:25:56.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.268+0000 7f97417fdb80 1 -- 172.21.15.52:0/383692041 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f2699d1660 con 0x55f269acf9f0 2023-11-27T15:25:56.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.268+0000 7f972effd700 1 --2- 172.21.15.52:0/383692041 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55f269acf9f0 0x55f269b26110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.268+0000 7f9712ff5700 10 client.16785 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:25:56.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.268+0000 7f972effd700 1 --2- 172.21.15.52:0/383692041 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55f269acf9f0 0x55f269b26110 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=adc3b79b618a64e1 server_cookie=f858d176bce04247 in_seq=0 out_seq=0 2023-11-27T15:25:56.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.269+0000 7f9712ff5700 1 -- 172.21.15.52:0/383692041 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f2699d1660 con 0x55f269acf9f0 2023-11-27T15:25:56.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.269+0000 7f9712ff5700 10 client.16785 handle_command_reply: tid=0 2023-11-27T15:25:56.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.269+0000 7f97417fdb80 1 client.16785 shutdown 2023-11-27T15:25:56.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.269+0000 7f97417fdb80 1 -- 172.21.15.52:0/383692041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f269ab6880 msgr2=0x55f269ac4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:56.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.269+0000 7f97417fdb80 1 --2- 172.21.15.52:0/383692041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f269ab6880 0x55f269ac4d70 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f9730069a50 tx=0x7f9724010a00 comp rx=0 tx=0).stop 2023-11-27T15:25:56.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.269+0000 7f97417fdb80 1 -- 172.21.15.52:0/383692041 shutdown_connections 2023-11-27T15:25:56.283 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.269+0000 7f97417fdb80 1 --2- 172.21.15.52:0/383692041 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55f269acf9f0 0x55f269b26110 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.269+0000 7f97417fdb80 1 --2- 172.21.15.52:0/383692041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f269abacc0 0x55f269ac5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.269+0000 7f9712ff5700 0 client.16785 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:25:56.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.269+0000 7f97417fdb80 1 --2- 172.21.15.52:0/383692041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f269ab5f70 0x55f269ac6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.269+0000 7f97417fdb80 1 --2- 172.21.15.52:0/383692041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f269ab6880 0x55f269ac4d70 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.269+0000 7f97417fdb80 1 -- 172.21.15.52:0/383692041 >> 172.21.15.52:0/383692041 conn(0x55f269aa7360 msgr2=0x55f269aac580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:56.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.269+0000 7f97417fdb80 1 -- 172.21.15.52:0/383692041 shutdown_connections 2023-11-27T15:25:56.284 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.269+0000 7f97417fdb80 1 -- 172.21.15.52:0/383692041 wait complete. 2023-11-27T15:25:56.285 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.269+0000 7f97417fdb80 20 client.16785 trim_cache size 0 max 16384 2023-11-27T15:25:56.285 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:25:56.286 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.269+0000 7f9735671700 1 -- 172.21.15.52:0/161685717 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f97180a1340 msgr2=0x7f97180a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:56.286 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.269+0000 7f9735671700 1 --2- 172.21.15.52:0/161685717 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f97180a1340 0x7f97180a37f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9720009fd0 tx=0x7f972000b700 comp rx=0 tx=0).stop 2023-11-27T15:25:56.286 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.269+0000 7f9735671700 1 -- 172.21.15.52:0/161685717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97301041f0 msgr2=0x7f97301a2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:56.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.269+0000 7f9735671700 1 --2- 172.21.15.52:0/161685717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97301041f0 0x7f97301a2310 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f972800e940 tx=0x7f972800ec50 comp rx=0 tx=0).stop 2023-11-27T15:25:56.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.269+0000 7f9735671700 1 -- 172.21.15.52:0/161685717 shutdown_connections 2023-11-27T15:25:56.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.269+0000 7f9735671700 1 --2- 172.21.15.52:0/161685717 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f97180a1340 0x7f97180a37f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.269+0000 7f9735671700 1 --2- 172.21.15.52:0/161685717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97301041f0 0x7f97301a2310 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.269+0000 7f9735671700 1 --2- 172.21.15.52:0/161685717 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97300ffac0 0x7f9730197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.269+0000 7f9735671700 1 --2- 172.21.15.52:0/161685717 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9730100470 0x7f97301977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.269+0000 7f9735671700 1 -- 172.21.15.52:0/161685717 >> 172.21.15.52:0/161685717 conn(0x7f97300763a0 msgr2=0x7f97300fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:56.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.270+0000 7f9735671700 1 -- 172.21.15.52:0/161685717 shutdown_connections 2023-11-27T15:25:56.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.270+0000 7f9735671700 1 -- 172.21.15.52:0/161685717 wait complete. 2023-11-27T15:25:56.288 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-27T15:25:56.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.410+0000 7fb53130d700 1 -- 172.21.15.52:0/1514825522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb52c10a920 msgr2=0x7fb52c10cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:56.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.410+0000 7fb53130d700 1 --2- 172.21.15.52:0/1514825522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb52c10a920 0x7fb52c10cd40 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb52400b0a0 tx=0x7fb52400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.410+0000 7fb53130d700 1 -- 172.21.15.52:0/1514825522 shutdown_connections 2023-11-27T15:25:56.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.410+0000 7fb53130d700 1 --2- 172.21.15.52:0/1514825522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb52c10a920 0x7fb52c10cd40 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.410+0000 7fb53130d700 1 --2- 172.21.15.52:0/1514825522 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb52c101400 0x7fb52c1017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.410+0000 7fb53130d700 1 --2- 172.21.15.52:0/1514825522 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb52c101d10 0x7fb52c10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.410+0000 7fb53130d700 1 -- 172.21.15.52:0/1514825522 >> 172.21.15.52:0/1514825522 conn(0x7fb52c076200 msgr2=0x7fb52c076600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:56.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.410+0000 7fb53130d700 1 -- 172.21.15.52:0/1514825522 shutdown_connections 2023-11-27T15:25:56.416 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.410+0000 7fb53130d700 1 -- 172.21.15.52:0/1514825522 wait complete. 2023-11-27T15:25:56.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.411+0000 7fb53130d700 1 Processor -- start 2023-11-27T15:25:56.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.411+0000 7fb53130d700 1 -- start start 2023-11-27T15:25:56.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.411+0000 7fb53130d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb52c101400 0x7fb52c0ff230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.411+0000 7fb53130d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb52c101d10 0x7fb52c0ff770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.411+0000 7fb53130d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb52c10a920 0x7fb52c1a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.418 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.411+0000 7fb53130d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c0fffa0 con 0x7fb52c10a920 2023-11-27T15:25:56.419 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.411+0000 7fb53130d700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c1a4c50 con 0x7fb52c101400 2023-11-27T15:25:56.419 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.411+0000 7fb53130d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c1a4d90 con 0x7fb52c101d10 2023-11-27T15:25:56.419 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.411+0000 7fb52b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb52c10a920 0x7fb52c1a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.419 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.411+0000 7fb52b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb52c10a920 0x7fb52c1a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55300/0 (socket says 172.21.15.52:55300) 2023-11-27T15:25:56.419 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.411+0000 7fb52b7fe700 1 -- 172.21.15.52:0/3555747555 learned_addr learned my addr 172.21.15.52:0/3555747555 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:56.419 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.411+0000 7fb52a7fc700 1 --2- 172.21.15.52:0/3555747555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb52c101d10 0x7fb52c0ff770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.419 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.411+0000 7fb52affd700 1 --2- 172.21.15.52:0/3555747555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb52c101400 0x7fb52c0ff230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.419 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.411+0000 7fb52b7fe700 1 -- 172.21.15.52:0/3555747555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb52c101d10 msgr2=0x7fb52c0ff770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:56.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.411+0000 7fb52b7fe700 1 --2- 172.21.15.52:0/3555747555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb52c101d10 0x7fb52c0ff770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.411+0000 7fb52b7fe700 1 -- 172.21.15.52:0/3555747555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb52c101400 msgr2=0x7fb52c0ff230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:56.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.411+0000 7fb52b7fe700 1 --2- 172.21.15.52:0/3555747555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb52c101400 0x7fb52c0ff230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.412+0000 7fb52b7fe700 1 -- 172.21.15.52:0/3555747555 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb524009d00 con 0x7fb52c10a920 2023-11-27T15:25:56.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.412+0000 7fb52affd700 1 --2- 172.21.15.52:0/3555747555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb52c101400 0x7fb52c0ff230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:56.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.412+0000 7fb52a7fc700 1 --2- 172.21.15.52:0/3555747555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb52c101d10 0x7fb52c0ff770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:56.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.412+0000 7fb52b7fe700 1 --2- 172.21.15.52:0/3555747555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb52c10a920 0x7fb52c1a4710 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fb5240062a0 tx=0x7fb524007a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:56.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.412+0000 7fb513fff700 1 -- 172.21.15.52:0/3555747555 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52400e040 con 0x7fb52c10a920 2023-11-27T15:25:56.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.412+0000 7fb53130d700 1 -- 172.21.15.52:0/3555747555 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52c110370 con 0x7fb52c10a920 2023-11-27T15:25:56.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.412+0000 7fb513fff700 1 -- 172.21.15.52:0/3555747555 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb524004e10 con 0x7fb52c10a920 2023-11-27T15:25:56.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.412+0000 7fb513fff700 1 -- 172.21.15.52:0/3555747555 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52401d480 con 0x7fb52c10a920 2023-11-27T15:25:56.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.412+0000 7fb53130d700 1 -- 172.21.15.52:0/3555747555 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52c1101e0 con 0x7fb52c10a920 2023-11-27T15:25:56.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.414+0000 7fb513fff700 1 -- 172.21.15.52:0/3555747555 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb524004020 con 0x7fb52c10a920 2023-11-27T15:25:56.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.415+0000 7fb513fff700 1 --2- 172.21.15.52:0/3555747555 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb5140a12c0 0x7fb5140a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.415+0000 7fb513fff700 1 -- 172.21.15.52:0/3555747555 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fb5240ce560 con 0x7fb52c10a920 2023-11-27T15:25:56.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.415+0000 7fb52affd700 1 --2- 172.21.15.52:0/3555747555 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb5140a12c0 0x7fb5140a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.415+0000 7fb52affd700 1 --2- 172.21.15.52:0/3555747555 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb5140a12c0 0x7fb5140a3770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb520005fd0 tx=0x7fb520005dc0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:56.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.415+0000 7fb53d499b80 1 Processor -- start 2023-11-27T15:25:56.428 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.416+0000 7fb53d499b80 1 -- start start 2023-11-27T15:25:56.428 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.416+0000 7fb53d499b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558eeef7fb60 0x558eeef7ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.428 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.416+0000 7fb53d499b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558eeef80470 0x558eeef84370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.428 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.416+0000 7fb53d499b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558eeef848b0 0x558eeef84d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.428 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.416+0000 7fb53d499b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eeee85060 con 0x558eeef7fb60 2023-11-27T15:25:56.428 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.416+0000 7fb53d499b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eeeed78a0 con 0x558eeef80470 2023-11-27T15:25:56.428 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.416+0000 7fb53d499b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eeeed0d30 con 0x558eeef848b0 2023-11-27T15:25:56.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.416+0000 7fb52affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558eeef7fb60 0x558eeef7ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.416+0000 7fb52affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558eeef7fb60 0x558eeef7ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55304/0 (socket says 172.21.15.52:55304) 2023-11-27T15:25:56.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.416+0000 7fb52affd700 1 -- 172.21.15.52:0/3722161642 learned_addr learned my addr 172.21.15.52:0/3722161642 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:56.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.416+0000 7fb52a7fc700 1 --2- 172.21.15.52:0/3722161642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558eeef80470 0x558eeef84370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.416+0000 7fb52b7fe700 1 --2- 172.21.15.52:0/3722161642 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558eeef848b0 0x558eeef84d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.416+0000 7fb52affd700 1 -- 172.21.15.52:0/3722161642 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558eeef848b0 msgr2=0x558eeef84d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:56.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.416+0000 7fb52affd700 1 --2- 172.21.15.52:0/3722161642 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558eeef848b0 0x558eeef84d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.416+0000 7fb52affd700 1 -- 172.21.15.52:0/3722161642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558eeef80470 msgr2=0x558eeef84370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:56.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.416+0000 7fb52affd700 1 --2- 172.21.15.52:0/3722161642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558eeef80470 0x558eeef84370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.416+0000 7fb52affd700 1 -- 172.21.15.52:0/3722161642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558eeed52630 con 0x558eeef7fb60 2023-11-27T15:25:56.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.416+0000 7fb52b7fe700 1 --2- 172.21.15.52:0/3722161642 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558eeef848b0 0x558eeef84d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:56.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.416+0000 7fb52affd700 1 --2- 172.21.15.52:0/3722161642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558eeef7fb60 0x558eeef7ff30 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb52000db90 tx=0x7fb520010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff0b29946fa8c10 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:56.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.417+0000 7fb5117fa700 1 -- 172.21.15.52:0/3722161642 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52000f930 con 0x558eeef7fb60 2023-11-27T15:25:56.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.417+0000 7fb5117fa700 1 -- 172.21.15.52:0/3722161642 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb52001c450 con 0x558eeef7fb60 2023-11-27T15:25:56.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.417+0000 7fb5117fa700 1 -- 172.21.15.52:0/3722161642 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52001b610 con 0x558eeef7fb60 2023-11-27T15:25:56.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.417+0000 7fb53d499b80 1 -- 172.21.15.52:0/3722161642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558eeef7fb60 msgr2=0x558eeef7ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:56.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.417+0000 7fb53d499b80 1 --2- 172.21.15.52:0/3722161642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558eeef7fb60 0x558eeef7ff30 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb52000db90 tx=0x7fb520010000 comp rx=0 tx=0).stop 2023-11-27T15:25:56.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.417+0000 7fb53d499b80 1 -- 172.21.15.52:0/3722161642 shutdown_connections 2023-11-27T15:25:56.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.417+0000 7fb53d499b80 1 --2- 172.21.15.52:0/3722161642 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558eeef848b0 0x558eeef84d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.417+0000 7fb53d499b80 1 --2- 172.21.15.52:0/3722161642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558eeef7fb60 0x558eeef7ff30 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.417+0000 7fb53d499b80 1 --2- 172.21.15.52:0/3722161642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558eeef80470 0x558eeef84370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.417+0000 7fb53d499b80 1 -- 172.21.15.52:0/3722161642 >> 172.21.15.52:0/3722161642 conn(0x558eeef75360 msgr2=0x558eeef7ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:56.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.417+0000 7fb53d499b80 1 -- 172.21.15.52:0/3722161642 shutdown_connections 2023-11-27T15:25:56.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.417+0000 7fb53d499b80 1 -- 172.21.15.52:0/3722161642 wait complete. 2023-11-27T15:25:56.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.418+0000 7fb53d499b80 1 -- start start 2023-11-27T15:25:56.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.418+0000 7fb53d499b80 1 Processor -- start 2023-11-27T15:25:56.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.418+0000 7fb53d499b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558eeef7fb60 0x558eeef8c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.418+0000 7fb53d499b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558eeef80470 0x558eeef892d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.418+0000 7fb53d499b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558eeef848b0 0x558eeef89950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.418+0000 7fb53d499b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eeee728e0 con 0x558eeef80470 2023-11-27T15:25:56.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.418+0000 7fb53d499b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eeee72580 con 0x558eeef7fb60 2023-11-27T15:25:56.433 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.418+0000 7fb53d499b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eeed54920 con 0x558eeef848b0 2023-11-27T15:25:56.433 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.418+0000 7fb53d499b80 10 client.0 _ll_register_callbacks cb 0x558eeee90780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:56.433 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.418+0000 7fb52a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558eeef80470 0x558eeef892d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.433 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.418+0000 7fb52affd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558eeef7fb60 0x558eeef8c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.433 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.418+0000 7fb52a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558eeef80470 0x558eeef892d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55312/0 (socket says 172.21.15.52:55312) 2023-11-27T15:25:56.433 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.418+0000 7fb52a7fc700 1 -- 172.21.15.52:0/3743026703 learned_addr learned my addr 172.21.15.52:0/3743026703 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:56.433 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.419+0000 7fb52b7fe700 1 --2- 172.21.15.52:0/3743026703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558eeef848b0 0x558eeef89950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.433 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.419+0000 7fb52a7fc700 1 -- 172.21.15.52:0/3743026703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558eeef848b0 msgr2=0x558eeef89950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:56.434 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.419+0000 7fb52a7fc700 1 --2- 172.21.15.52:0/3743026703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558eeef848b0 0x558eeef89950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.434 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.419+0000 7fb52a7fc700 1 -- 172.21.15.52:0/3743026703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558eeef7fb60 msgr2=0x558eeef8c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:56.434 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.419+0000 7fb52a7fc700 1 --2- 172.21.15.52:0/3743026703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558eeef7fb60 0x558eeef8c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.434 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.419+0000 7fb52a7fc700 1 -- 172.21.15.52:0/3743026703 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52000d840 con 0x558eeef80470 2023-11-27T15:25:56.434 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.419+0000 7fb52a7fc700 1 --2- 172.21.15.52:0/3743026703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558eeef80470 0x558eeef892d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00de20 tx=0x7fb51c00f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0ef15fabd6559f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:56.434 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.419+0000 7fb50eff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:25:56.435 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.419+0000 7fb50eff5700 1 -- 172.21.15.52:0/3743026703 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c004220 con 0x558eeef80470 2023-11-27T15:25:56.435 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.419+0000 7fb50eff5700 1 -- 172.21.15.52:0/3743026703 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fb51c015070 con 0x558eeef80470 2023-11-27T15:25:56.435 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.419+0000 7fb53d499b80 1 -- 172.21.15.52:0/3743026703 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558eeef72b20 con 0x558eeef80470 2023-11-27T15:25:56.435 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.419+0000 7fb50eff5700 1 -- 172.21.15.52:0/3743026703 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb51c00fe60 con 0x558eeef80470 2023-11-27T15:25:56.435 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.419+0000 7fb53d499b80 10 client.16797 fetch_fsmap learned FSMap version 32 2023-11-27T15:25:56.435 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.419+0000 7fb53d499b80 1 -- 172.21.15.52:0/3743026703 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558eeee97190 con 0x558eeef80470 2023-11-27T15:25:56.435 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.419+0000 7fb50eff5700 1 -- 172.21.15.52:0/3743026703 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb51c025080 con 0x558eeef80470 2023-11-27T15:25:56.435 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.420+0000 7fb50eff5700 1 -- 172.21.15.52:0/3743026703 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb51c011480 con 0x558eeef80470 2023-11-27T15:25:56.436 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.420+0000 7fb53d499b80 10 client.16797 fetch_fsmap finished waiting for FSMap version 32 2023-11-27T15:25:56.436 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.420+0000 7fb53d499b80 10 client.16797 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:56.436 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.420+0000 7fb53d499b80 20 client.16797 populate_metadata read hostname 'smithi052' 2023-11-27T15:25:56.436 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.420+0000 7fb53d499b80 1 --2- 172.21.15.52:0/3743026703 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x558eeef99870 0x558eeef9bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.436 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.420+0000 7fb53d499b80 4 client.16797 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:56.436 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.420+0000 7fb53d499b80 1 -- 172.21.15.52:0/3743026703 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558eeee930a0 con 0x558eeef99870 2023-11-27T15:25:56.436 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.420+0000 7fb52affd700 1 --2- 172.21.15.52:0/3743026703 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x558eeef99870 0x558eeef9bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.436 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.420+0000 7fb52affd700 1 --2- 172.21.15.52:0/3743026703 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x558eeef99870 0x558eeef9bc80 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed41049cda2b71e2 server_cookie=a92ce00a5da0dd42 in_seq=0 out_seq=0 2023-11-27T15:25:56.437 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.420+0000 7fb50eff5700 10 client.16797 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:56.437 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.421+0000 7fb50eff5700 1 -- 172.21.15.52:0/3743026703 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558eeee930a0 con 0x558eeef99870 2023-11-27T15:25:56.437 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.421+0000 7fb50eff5700 10 client.16797 handle_command_reply: tid=0 2023-11-27T15:25:56.437 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.421+0000 7fb53d499b80 1 client.16797 shutdown 2023-11-27T15:25:56.437 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.421+0000 7fb53d499b80 1 -- 172.21.15.52:0/3743026703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558eeef80470 msgr2=0x558eeef892d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:56.437 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.421+0000 7fb53d499b80 1 --2- 172.21.15.52:0/3743026703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558eeef80470 0x558eeef892d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00de20 tx=0x7fb51c00f640 comp rx=0 tx=0).stop 2023-11-27T15:25:56.437 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.421+0000 7fb53d499b80 1 -- 172.21.15.52:0/3743026703 shutdown_connections 2023-11-27T15:25:56.438 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.421+0000 7fb53d499b80 1 --2- 172.21.15.52:0/3743026703 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x558eeef99870 0x558eeef9bc80 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.438 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.421+0000 7fb53d499b80 1 --2- 172.21.15.52:0/3743026703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558eeef7fb60 0x558eeef8c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.438 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.421+0000 7fb50eff5700 0 client.16797 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:56.438 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.421+0000 7fb53d499b80 1 --2- 172.21.15.52:0/3743026703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558eeef848b0 0x558eeef89950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.438 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.421+0000 7fb53d499b80 1 --2- 172.21.15.52:0/3743026703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558eeef80470 0x558eeef892d0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.438 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.421+0000 7fb53d499b80 1 -- 172.21.15.52:0/3743026703 >> 172.21.15.52:0/3743026703 conn(0x558eeef75360 msgr2=0x558eeef79de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:56.438 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.421+0000 7fb53d499b80 1 -- 172.21.15.52:0/3743026703 shutdown_connections 2023-11-27T15:25:56.439 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.421+0000 7fb53d499b80 1 -- 172.21.15.52:0/3743026703 wait complete. 2023-11-27T15:25:56.439 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.421+0000 7fb53d499b80 20 client.16797 trim_cache size 0 max 16384 2023-11-27T15:25:56.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.434+0000 7fb53d499b80 1 Processor -- start 2023-11-27T15:25:56.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.434+0000 7fb53d499b80 1 -- start start 2023-11-27T15:25:56.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.435+0000 7fb53d499b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558eeef7fb60 0x558eeef831a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.435+0000 7fb53d499b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558eeef80470 0x558eeef95be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.435+0000 7fb53d499b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558eeef848b0 0x558eeef83a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.435+0000 7fb53d499b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eeef96f40 con 0x558eeef848b0 2023-11-27T15:25:56.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.435+0000 7fb53d499b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eeef75170 con 0x558eeef80470 2023-11-27T15:25:56.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.435+0000 7fb53d499b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eeef7b950 con 0x558eeef7fb60 2023-11-27T15:25:56.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.435+0000 7fb52b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558eeef848b0 0x558eeef83a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.435+0000 7fb52b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558eeef848b0 0x558eeef83a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55322/0 (socket says 172.21.15.52:55322) 2023-11-27T15:25:56.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.435+0000 7fb52b7fe700 1 -- 172.21.15.52:0/3262002541 learned_addr learned my addr 172.21.15.52:0/3262002541 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:56.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.435+0000 7fb52a7fc700 1 --2- 172.21.15.52:0/3262002541 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558eeef80470 0x558eeef95be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.435+0000 7fb52affd700 1 --2- 172.21.15.52:0/3262002541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558eeef7fb60 0x558eeef831a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.435+0000 7fb52b7fe700 1 -- 172.21.15.52:0/3262002541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558eeef7fb60 msgr2=0x558eeef831a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:56.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.435+0000 7fb52b7fe700 1 --2- 172.21.15.52:0/3262002541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558eeef7fb60 0x558eeef831a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.435+0000 7fb52b7fe700 1 -- 172.21.15.52:0/3262002541 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558eeef80470 msgr2=0x558eeef95be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:56.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.435+0000 7fb52b7fe700 1 --2- 172.21.15.52:0/3262002541 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558eeef80470 0x558eeef95be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.435+0000 7fb52b7fe700 1 -- 172.21.15.52:0/3262002541 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558eeef76b00 con 0x558eeef848b0 2023-11-27T15:25:56.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.435+0000 7fb52a7fc700 1 --2- 172.21.15.52:0/3262002541 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558eeef80470 0x558eeef95be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:56.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.435+0000 7fb52b7fe700 1 --2- 172.21.15.52:0/3262002541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558eeef848b0 0x558eeef83a30 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fb524097b90 tx=0x7fb524009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7eb92b69649312 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:56.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.435+0000 7fb511ffb700 1 -- 172.21.15.52:0/3262002541 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52409f6c0 con 0x558eeef848b0 2023-11-27T15:25:56.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.435+0000 7fb511ffb700 1 -- 172.21.15.52:0/3262002541 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb524084e90 con 0x558eeef848b0 2023-11-27T15:25:56.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.435+0000 7fb511ffb700 1 -- 172.21.15.52:0/3262002541 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52409bb10 con 0x558eeef848b0 2023-11-27T15:25:56.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.436+0000 7fb53d499b80 1 -- 172.21.15.52:0/3262002541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558eeef848b0 msgr2=0x558eeef83a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:56.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.436+0000 7fb53d499b80 1 --2- 172.21.15.52:0/3262002541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558eeef848b0 0x558eeef83a30 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fb524097b90 tx=0x7fb524009040 comp rx=0 tx=0).stop 2023-11-27T15:25:56.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.436+0000 7fb53d499b80 1 -- 172.21.15.52:0/3262002541 shutdown_connections 2023-11-27T15:25:56.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.436+0000 7fb53d499b80 1 --2- 172.21.15.52:0/3262002541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558eeef848b0 0x558eeef83a30 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.436+0000 7fb53d499b80 1 --2- 172.21.15.52:0/3262002541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558eeef7fb60 0x558eeef831a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.436+0000 7fb53d499b80 1 --2- 172.21.15.52:0/3262002541 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558eeef80470 0x558eeef95be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.436+0000 7fb53d499b80 1 -- 172.21.15.52:0/3262002541 >> 172.21.15.52:0/3262002541 conn(0x558eeef75360 msgr2=0x558eeef84c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:56.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.436+0000 7fb53d499b80 1 -- 172.21.15.52:0/3262002541 shutdown_connections 2023-11-27T15:25:56.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.436+0000 7fb53d499b80 1 -- 172.21.15.52:0/3262002541 wait complete. 2023-11-27T15:25:56.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.436+0000 7fb53d499b80 1 -- start start 2023-11-27T15:25:56.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.436+0000 7fb53d499b80 1 Processor -- start 2023-11-27T15:25:56.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.437+0000 7fb53d499b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558eeef7fb60 0x558eeef8c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.437+0000 7fb53d499b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558eeef80470 0x558eeef8cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.437+0000 7fb53d499b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558eeef848b0 0x558eeef8d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.450 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.437+0000 7fb53d499b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eeefec720 con 0x558eeef7fb60 2023-11-27T15:25:56.450 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.437+0000 7fb53d499b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eeefec860 con 0x558eeef848b0 2023-11-27T15:25:56.450 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.437+0000 7fb53d499b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eeefec9a0 con 0x558eeef80470 2023-11-27T15:25:56.450 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.437+0000 7fb53d499b80 10 client.0 _ll_register_callbacks cb 0x7fb51c00d870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:56.450 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.437+0000 7fb52affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558eeef7fb60 0x558eeef8c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.450 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.437+0000 7fb52affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558eeef7fb60 0x558eeef8c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55324/0 (socket says 172.21.15.52:55324) 2023-11-27T15:25:56.450 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.437+0000 7fb52affd700 1 -- 172.21.15.52:0/3880158103 learned_addr learned my addr 172.21.15.52:0/3880158103 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:56.450 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.437+0000 7fb52a7fc700 1 --2- 172.21.15.52:0/3880158103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558eeef80470 0x558eeef8cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.437+0000 7fb52b7fe700 1 --2- 172.21.15.52:0/3880158103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558eeef848b0 0x558eeef8d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.437+0000 7fb52affd700 1 -- 172.21.15.52:0/3880158103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558eeef80470 msgr2=0x558eeef8cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:56.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.437+0000 7fb52affd700 1 --2- 172.21.15.52:0/3880158103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558eeef80470 0x558eeef8cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.437+0000 7fb52affd700 1 -- 172.21.15.52:0/3880158103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558eeef848b0 msgr2=0x558eeef8d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:56.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.437+0000 7fb52affd700 1 --2- 172.21.15.52:0/3880158103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558eeef848b0 0x558eeef8d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.437+0000 7fb52affd700 1 -- 172.21.15.52:0/3880158103 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52c1101e0 con 0x558eeef7fb60 2023-11-27T15:25:56.451 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.437+0000 7fb52b7fe700 1 --2- 172.21.15.52:0/3880158103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558eeef848b0 0x558eeef8d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:56.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.437+0000 7fb52affd700 1 --2- 172.21.15.52:0/3880158103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558eeef7fb60 0x558eeef8c980 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fb52001cdb0 tx=0x7fb52001cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bffd647229a872b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:56.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.437+0000 7fb50eff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:25:56.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.438+0000 7fb50eff5700 1 -- 172.21.15.52:0/3880158103 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52002b040 con 0x558eeef7fb60 2023-11-27T15:25:56.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.438+0000 7fb50eff5700 1 -- 172.21.15.52:0/3880158103 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fb52001c8f0 con 0x558eeef7fb60 2023-11-27T15:25:56.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.438+0000 7fb53d499b80 1 -- 172.21.15.52:0/3880158103 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb52001c450 con 0x558eeef7fb60 2023-11-27T15:25:56.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.438+0000 7fb50eff5700 1 -- 172.21.15.52:0/3880158103 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb520034cb0 con 0x558eeef7fb60 2023-11-27T15:25:56.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.438+0000 7fb53d499b80 10 client.16803 fetch_fsmap learned FSMap version 32 2023-11-27T15:25:56.452 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.438+0000 7fb53d499b80 1 -- 172.21.15.52:0/3880158103 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558eeecfede0 con 0x558eeef7fb60 2023-11-27T15:25:56.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.438+0000 7fb50eff5700 1 -- 172.21.15.52:0/3880158103 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb52001c450 con 0x558eeef7fb60 2023-11-27T15:25:56.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.438+0000 7fb50eff5700 1 -- 172.21.15.52:0/3880158103 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb52000fbb0 con 0x558eeef7fb60 2023-11-27T15:25:56.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.438+0000 7fb53d499b80 10 client.16803 fetch_fsmap finished waiting for FSMap version 32 2023-11-27T15:25:56.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.438+0000 7fb53d499b80 10 client.16803 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:56.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.438+0000 7fb53d499b80 20 client.16803 populate_metadata read hostname 'smithi052' 2023-11-27T15:25:56.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.438+0000 7fb53d499b80 1 --2- 172.21.15.52:0/3880158103 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x558eeefedee0 0x558eeeff02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:56.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.438+0000 7fb53d499b80 4 client.16803 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:25:56.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.438+0000 7fb53d499b80 1 -- 172.21.15.52:0/3880158103 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558eeee9f660 con 0x558eeefedee0 2023-11-27T15:25:56.453 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.438+0000 7fb52a7fc700 1 --2- 172.21.15.52:0/3880158103 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x558eeefedee0 0x558eeeff02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:56.454 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.439+0000 7fb50eff5700 10 client.16803 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:56.454 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.439+0000 7fb52a7fc700 1 --2- 172.21.15.52:0/3880158103 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x558eeefedee0 0x558eeeff02f0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30390cab21f9df31 server_cookie=93dc31986b09c2cb in_seq=0 out_seq=0 2023-11-27T15:25:56.454 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.439+0000 7fb50eff5700 1 -- 172.21.15.52:0/3880158103 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558eeee9f660 con 0x558eeefedee0 2023-11-27T15:25:56.454 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.439+0000 7fb50eff5700 10 client.16803 handle_command_reply: tid=0 2023-11-27T15:25:56.454 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.439+0000 7fb53d499b80 1 client.16803 shutdown 2023-11-27T15:25:56.454 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.439+0000 7fb53d499b80 1 -- 172.21.15.52:0/3880158103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558eeef7fb60 msgr2=0x558eeef8c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:56.454 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.439+0000 7fb53d499b80 1 --2- 172.21.15.52:0/3880158103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558eeef7fb60 0x558eeef8c980 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fb52001cdb0 tx=0x7fb52001cde0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.439+0000 7fb53d499b80 1 -- 172.21.15.52:0/3880158103 shutdown_connections 2023-11-27T15:25:56.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.439+0000 7fb53d499b80 1 --2- 172.21.15.52:0/3880158103 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x558eeefedee0 0x558eeeff02f0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.439+0000 7fb53d499b80 1 --2- 172.21.15.52:0/3880158103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558eeef848b0 0x558eeef8d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.439+0000 7fb53d499b80 1 --2- 172.21.15.52:0/3880158103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558eeef7fb60 0x558eeef8c980 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.439+0000 7fb53d499b80 1 --2- 172.21.15.52:0/3880158103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558eeef80470 0x558eeef8cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.439+0000 7fb53d499b80 1 -- 172.21.15.52:0/3880158103 >> 172.21.15.52:0/3880158103 conn(0x558eeef75360 msgr2=0x558eeef95be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:56.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.439+0000 7fb50eff5700 0 client.16803 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:56.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.439+0000 7fb53d499b80 1 -- 172.21.15.52:0/3880158103 shutdown_connections 2023-11-27T15:25:56.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.439+0000 7fb53d499b80 1 -- 172.21.15.52:0/3880158103 wait complete. 2023-11-27T15:25:56.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.439+0000 7fb53d499b80 20 client.16803 trim_cache size 0 max 16384 2023-11-27T15:25:56.456 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:25:56.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.440+0000 7fb53130d700 1 -- 172.21.15.52:0/3555747555 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb5140a12c0 msgr2=0x7fb5140a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:56.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.440+0000 7fb53130d700 1 --2- 172.21.15.52:0/3555747555 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb5140a12c0 0x7fb5140a3770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb520005fd0 tx=0x7fb520005dc0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.440+0000 7fb53130d700 1 -- 172.21.15.52:0/3555747555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb52c10a920 msgr2=0x7fb52c1a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:56.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.440+0000 7fb53130d700 1 --2- 172.21.15.52:0/3555747555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb52c10a920 0x7fb52c1a4710 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fb5240062a0 tx=0x7fb524007a60 comp rx=0 tx=0).stop 2023-11-27T15:25:56.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.440+0000 7fb53130d700 1 -- 172.21.15.52:0/3555747555 shutdown_connections 2023-11-27T15:25:56.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.440+0000 7fb53130d700 1 --2- 172.21.15.52:0/3555747555 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb5140a12c0 0x7fb5140a3770 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.440+0000 7fb53130d700 1 --2- 172.21.15.52:0/3555747555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb52c101400 0x7fb52c0ff230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.440+0000 7fb53130d700 1 --2- 172.21.15.52:0/3555747555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb52c10a920 0x7fb52c1a4710 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.440+0000 7fb53130d700 1 --2- 172.21.15.52:0/3555747555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb52c101d10 0x7fb52c0ff770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.440+0000 7fb53130d700 1 -- 172.21.15.52:0/3555747555 >> 172.21.15.52:0/3555747555 conn(0x7fb52c076200 msgr2=0x7fb52c0fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:56.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.440+0000 7fb53130d700 1 -- 172.21.15.52:0/3555747555 shutdown_connections 2023-11-27T15:25:56.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:56.440+0000 7fb53130d700 1 -- 172.21.15.52:0/3555747555 wait complete. 2023-11-27T15:25:56.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:56 smithi150 ceph-mon[100559]: pgmap v247: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:25:56.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:56 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3321175590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:25:56.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:56 smithi102 ceph-mon[100296]: pgmap v247: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:25:56.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:56 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3321175590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:25:56.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:56 smithi052 ceph-mon[99056]: pgmap v247: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:25:56.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:56 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3321175590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:25:57.459 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-27T15:25:57.460 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-27T15:25:57.607 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.599+0000 7fbb38bc9700 1 -- 172.21.15.52:0/4149376004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb341055f0 msgr2=0x7fbb3410d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:57.607 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.599+0000 7fbb38bc9700 1 --2- 172.21.15.52:0/4149376004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb341055f0 0x7fbb3410d1c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00b0d0 tx=0x7fbb2c00b3e0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.608 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.600+0000 7fbb38bc9700 1 -- 172.21.15.52:0/4149376004 shutdown_connections 2023-11-27T15:25:57.608 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.600+0000 7fbb38bc9700 1 --2- 172.21.15.52:0/4149376004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb341055f0 0x7fbb3410d1c0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.608 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.600+0000 7fbb38bc9700 1 --2- 172.21.15.52:0/4149376004 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb341005b0 0x7fbb34100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.608 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.600+0000 7fbb38bc9700 1 --2- 172.21.15.52:0/4149376004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb34100f50 0x7fbb34104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.608 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.600+0000 7fbb38bc9700 1 -- 172.21.15.52:0/4149376004 >> 172.21.15.52:0/4149376004 conn(0x7fbb34077140 msgr2=0x7fbb34077540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:57.608 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.600+0000 7fbb38bc9700 1 -- 172.21.15.52:0/4149376004 shutdown_connections 2023-11-27T15:25:57.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.600+0000 7fbb38bc9700 1 -- 172.21.15.52:0/4149376004 wait complete. 2023-11-27T15:25:57.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.600+0000 7fbb38bc9700 1 Processor -- start 2023-11-27T15:25:57.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.600+0000 7fbb38bc9700 1 -- start start 2023-11-27T15:25:57.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.601+0000 7fbb38bc9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb341005b0 0x7fbb34198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.601+0000 7fbb38bc9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb34100f50 0x7fbb34198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.601+0000 7fbb38bc9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb341055f0 0x7fbb3419bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.601+0000 7fbb38bc9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3419c580 con 0x7fbb341055f0 2023-11-27T15:25:57.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.601+0000 7fbb38bc9700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3419c6f0 con 0x7fbb34100f50 2023-11-27T15:25:57.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.601+0000 7fbb38bc9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3419c8b0 con 0x7fbb341005b0 2023-11-27T15:25:57.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.601+0000 7fbb32d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb341055f0 0x7fbb3419bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.601+0000 7fbb32d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb341055f0 0x7fbb3419bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55342/0 (socket says 172.21.15.52:55342) 2023-11-27T15:25:57.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.601+0000 7fbb32d9d700 1 -- 172.21.15.52:0/689501874 learned_addr learned my addr 172.21.15.52:0/689501874 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:57.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.601+0000 7fbb31d9b700 1 --2- 172.21.15.52:0/689501874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb34100f50 0x7fbb34198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.601+0000 7fbb3259c700 1 --2- 172.21.15.52:0/689501874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb341005b0 0x7fbb34198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.601+0000 7fbb32d9d700 1 -- 172.21.15.52:0/689501874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb341005b0 msgr2=0x7fbb34198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:57.612 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.601+0000 7fbb32d9d700 1 --2- 172.21.15.52:0/689501874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb341005b0 0x7fbb34198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.612 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.601+0000 7fbb32d9d700 1 -- 172.21.15.52:0/689501874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb34100f50 msgr2=0x7fbb34198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:57.612 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.601+0000 7fbb32d9d700 1 --2- 172.21.15.52:0/689501874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb34100f50 0x7fbb34198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.612 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.601+0000 7fbb32d9d700 1 -- 172.21.15.52:0/689501874 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2c009d00 con 0x7fbb341055f0 2023-11-27T15:25:57.612 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.601+0000 7fbb32d9d700 1 --2- 172.21.15.52:0/689501874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb341055f0 0x7fbb3419bf80 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c007870 tx=0x7fbb2c0078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:57.612 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.601+0000 7fbb2b7fe700 1 -- 172.21.15.52:0/689501874 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c00e040 con 0x7fbb341055f0 2023-11-27T15:25:57.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.602+0000 7fbb38bc9700 1 -- 172.21.15.52:0/689501874 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb341103d0 con 0x7fbb341055f0 2023-11-27T15:25:57.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.602+0000 7fbb2b7fe700 1 -- 172.21.15.52:0/689501874 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb2c003e10 con 0x7fbb341055f0 2023-11-27T15:25:57.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.602+0000 7fbb2b7fe700 1 -- 172.21.15.52:0/689501874 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c01b8a0 con 0x7fbb341055f0 2023-11-27T15:25:57.613 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.602+0000 7fbb38bc9700 1 -- 172.21.15.52:0/689501874 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb34110240 con 0x7fbb341055f0 2023-11-27T15:25:57.622 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.604+0000 7fbb2b7fe700 1 -- 172.21.15.52:0/689501874 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fbb2c005090 con 0x7fbb341055f0 2023-11-27T15:25:57.622 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.604+0000 7fbb2b7fe700 1 --2- 172.21.15.52:0/689501874 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fbb200a12c0 0x7fbb200a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.622 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.604+0000 7fbb2b7fe700 1 -- 172.21.15.52:0/689501874 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fbb2c0cde80 con 0x7fbb341055f0 2023-11-27T15:25:57.622 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.604+0000 7fbb3259c700 1 --2- 172.21.15.52:0/689501874 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fbb200a12c0 0x7fbb200a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.605+0000 7fbb3259c700 1 --2- 172.21.15.52:0/689501874 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fbb200a12c0 0x7fbb200a3770 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbb24005ea0 tx=0x7fbb24005e30 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:57.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.605+0000 7fbb44d55b80 1 Processor -- start 2023-11-27T15:25:57.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.606+0000 7fbb44d55b80 1 -- start start 2023-11-27T15:25:57.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.606+0000 7fbb44d55b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575a5669be0 0x5575a5669fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.606+0000 7fbb44d55b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575a566a4f0 0x5575a566e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.606+0000 7fbb44d55b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575a566e930 0x5575a566ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.606+0000 7fbb44d55b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575a556f0e0 con 0x5575a566e930 2023-11-27T15:25:57.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.606+0000 7fbb44d55b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575a55c1920 con 0x5575a566a4f0 2023-11-27T15:25:57.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.606+0000 7fbb44d55b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575a55badb0 con 0x5575a5669be0 2023-11-27T15:25:57.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.606+0000 7fbb32d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575a566e930 0x5575a566ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.606+0000 7fbb32d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575a566e930 0x5575a566ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55350/0 (socket says 172.21.15.52:55350) 2023-11-27T15:25:57.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.606+0000 7fbb32d9d700 1 -- 172.21.15.52:0/2405161160 learned_addr learned my addr 172.21.15.52:0/2405161160 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:57.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.606+0000 7fbb3259c700 1 --2- 172.21.15.52:0/2405161160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575a5669be0 0x5575a5669fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.606+0000 7fbb32d9d700 1 -- 172.21.15.52:0/2405161160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575a5669be0 msgr2=0x5575a5669fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:57.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.606+0000 7fbb32d9d700 1 --2- 172.21.15.52:0/2405161160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575a5669be0 0x5575a5669fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.606+0000 7fbb31d9b700 1 --2- 172.21.15.52:0/2405161160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575a566a4f0 0x5575a566e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.606+0000 7fbb32d9d700 1 -- 172.21.15.52:0/2405161160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575a566a4f0 msgr2=0x5575a566e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:57.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.606+0000 7fbb32d9d700 1 --2- 172.21.15.52:0/2405161160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575a566a4f0 0x5575a566e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.606+0000 7fbb32d9d700 1 -- 172.21.15.52:0/2405161160 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575a543c6b0 con 0x5575a566e930 2023-11-27T15:25:57.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.607+0000 7fbb31d9b700 1 --2- 172.21.15.52:0/2405161160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575a566a4f0 0x5575a566e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:57.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.607+0000 7fbb32d9d700 1 --2- 172.21.15.52:0/2405161160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575a566e930 0x5575a566ed80 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c005060 tx=0x7fbb2c00b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ae1cea4921b7b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:57.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.607+0000 7fbb28ff9700 1 -- 172.21.15.52:0/2405161160 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c019e60 con 0x5575a566e930 2023-11-27T15:25:57.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.607+0000 7fbb28ff9700 1 -- 172.21.15.52:0/2405161160 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb2c09afb0 con 0x5575a566e930 2023-11-27T15:25:57.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.607+0000 7fbb28ff9700 1 -- 172.21.15.52:0/2405161160 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c09b4a0 con 0x5575a566e930 2023-11-27T15:25:57.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.607+0000 7fbb44d55b80 1 -- 172.21.15.52:0/2405161160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575a566e930 msgr2=0x5575a566ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:57.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.608+0000 7fbb44d55b80 1 --2- 172.21.15.52:0/2405161160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575a566e930 0x5575a566ed80 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c005060 tx=0x7fbb2c00b3e0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.608+0000 7fbb44d55b80 1 -- 172.21.15.52:0/2405161160 shutdown_connections 2023-11-27T15:25:57.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.608+0000 7fbb44d55b80 1 --2- 172.21.15.52:0/2405161160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575a566e930 0x5575a566ed80 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.608+0000 7fbb44d55b80 1 --2- 172.21.15.52:0/2405161160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575a5669be0 0x5575a5669fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.608+0000 7fbb44d55b80 1 --2- 172.21.15.52:0/2405161160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575a566a4f0 0x5575a566e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.608+0000 7fbb44d55b80 1 -- 172.21.15.52:0/2405161160 >> 172.21.15.52:0/2405161160 conn(0x5575a565f3e0 msgr2=0x5575a5664bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:57.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.608+0000 7fbb44d55b80 1 -- 172.21.15.52:0/2405161160 shutdown_connections 2023-11-27T15:25:57.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.608+0000 7fbb44d55b80 1 -- 172.21.15.52:0/2405161160 wait complete. 2023-11-27T15:25:57.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.608+0000 7fbb44d55b80 1 -- start start 2023-11-27T15:25:57.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.608+0000 7fbb44d55b80 1 Processor -- start 2023-11-27T15:25:57.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.609+0000 7fbb44d55b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575a5669be0 0x5575a5676120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.609+0000 7fbb44d55b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575a566a4f0 0x5575a56733a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.609+0000 7fbb44d55b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575a566e930 0x5575a5673ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.609+0000 7fbb44d55b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575a555c960 con 0x5575a566a4f0 2023-11-27T15:25:57.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.609+0000 7fbb44d55b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575a555c600 con 0x5575a566e930 2023-11-27T15:25:57.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.609+0000 7fbb44d55b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575a543e9a0 con 0x5575a5669be0 2023-11-27T15:25:57.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.609+0000 7fbb44d55b80 10 client.0 _ll_register_callbacks cb 0x5575a557a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:57.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.609+0000 7fbb31d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575a566a4f0 0x5575a56733a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.609+0000 7fbb31d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575a566a4f0 0x5575a56733a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55354/0 (socket says 172.21.15.52:55354) 2023-11-27T15:25:57.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.609+0000 7fbb31d9b700 1 -- 172.21.15.52:0/1978092984 learned_addr learned my addr 172.21.15.52:0/1978092984 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:57.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.609+0000 7fbb3259c700 1 --2- 172.21.15.52:0/1978092984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575a5669be0 0x5575a5676120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.609+0000 7fbb32d9d700 1 --2- 172.21.15.52:0/1978092984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575a566e930 0x5575a5673ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.609+0000 7fbb31d9b700 1 -- 172.21.15.52:0/1978092984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575a5669be0 msgr2=0x5575a5676120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:57.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.609+0000 7fbb31d9b700 1 --2- 172.21.15.52:0/1978092984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575a5669be0 0x5575a5676120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.609+0000 7fbb31d9b700 1 -- 172.21.15.52:0/1978092984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575a566e930 msgr2=0x5575a5673ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:57.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.609+0000 7fbb31d9b700 1 --2- 172.21.15.52:0/1978092984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575a566e930 0x5575a5673ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.609+0000 7fbb31d9b700 1 -- 172.21.15.52:0/1978092984 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb34110240 con 0x5575a566a4f0 2023-11-27T15:25:57.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.609+0000 7fbb32d9d700 1 --2- 172.21.15.52:0/1978092984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575a566e930 0x5575a5673ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:57.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.609+0000 7fbb31d9b700 1 --2- 172.21.15.52:0/1978092984 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575a566a4f0 0x5575a56733a0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c00b990 tx=0x7fbb1c00bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1cceeaf4386b2b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:57.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.609+0000 7fbb19ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:25:57.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.609+0000 7fbb19ffb700 1 -- 172.21.15.52:0/1978092984 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c003d70 con 0x5575a566a4f0 2023-11-27T15:25:57.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.609+0000 7fbb19ffb700 1 -- 172.21.15.52:0/1978092984 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fbb1c004830 con 0x5575a566a4f0 2023-11-27T15:25:57.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.609+0000 7fbb44d55b80 1 -- 172.21.15.52:0/1978092984 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5575a565cba0 con 0x5575a566a4f0 2023-11-27T15:25:57.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.610+0000 7fbb19ffb700 1 -- 172.21.15.52:0/1978092984 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb1c0049f0 con 0x5575a566a4f0 2023-11-27T15:25:57.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.610+0000 7fbb44d55b80 10 client.16815 fetch_fsmap learned FSMap version 32 2023-11-27T15:25:57.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.610+0000 7fbb44d55b80 1 -- 172.21.15.52:0/1978092984 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5575a5581210 con 0x5575a566a4f0 2023-11-27T15:25:57.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.610+0000 7fbb19ffb700 1 -- 172.21.15.52:0/1978092984 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5575a565cba0 con 0x5575a566a4f0 2023-11-27T15:25:57.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.610+0000 7fbb19ffb700 1 -- 172.21.15.52:0/1978092984 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbb1c01a540 con 0x5575a566a4f0 2023-11-27T15:25:57.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.610+0000 7fbb44d55b80 10 client.16815 fetch_fsmap finished waiting for FSMap version 32 2023-11-27T15:25:57.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.610+0000 7fbb44d55b80 10 client.16815 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:57.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.610+0000 7fbb44d55b80 20 client.16815 populate_metadata read hostname 'smithi052' 2023-11-27T15:25:57.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.610+0000 7fbb44d55b80 1 --2- 172.21.15.52:0/1978092984 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5575a56837d0 0x5575a5685be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.610+0000 7fbb44d55b80 4 client.16815 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:57.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.610+0000 7fbb44d55b80 1 -- 172.21.15.52:0/1978092984 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5575a557d120 con 0x5575a56837d0 2023-11-27T15:25:57.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.610+0000 7fbb3259c700 1 --2- 172.21.15.52:0/1978092984 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5575a56837d0 0x5575a5685be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.611+0000 7fbb19ffb700 10 client.16815 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:57.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.611+0000 7fbb3259c700 1 --2- 172.21.15.52:0/1978092984 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5575a56837d0 0x5575a5685be0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30db88737649414d server_cookie=ad382ee5d23fff5b in_seq=0 out_seq=0 2023-11-27T15:25:57.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.611+0000 7fbb19ffb700 1 -- 172.21.15.52:0/1978092984 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5575a557d120 con 0x5575a56837d0 2023-11-27T15:25:57.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.611+0000 7fbb19ffb700 10 client.16815 handle_command_reply: tid=0 2023-11-27T15:25:57.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.611+0000 7fbb44d55b80 1 client.16815 shutdown 2023-11-27T15:25:57.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.611+0000 7fbb44d55b80 1 -- 172.21.15.52:0/1978092984 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575a566a4f0 msgr2=0x5575a56733a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:57.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.612+0000 7fbb44d55b80 1 --2- 172.21.15.52:0/1978092984 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575a566a4f0 0x5575a56733a0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c00b990 tx=0x7fbb1c00bca0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.612+0000 7fbb44d55b80 1 -- 172.21.15.52:0/1978092984 shutdown_connections 2023-11-27T15:25:57.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.612+0000 7fbb44d55b80 1 --2- 172.21.15.52:0/1978092984 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5575a56837d0 0x5575a5685be0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.612+0000 7fbb44d55b80 1 --2- 172.21.15.52:0/1978092984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575a566e930 0x5575a5673ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.612+0000 7fbb19ffb700 0 client.16815 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:57.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.612+0000 7fbb44d55b80 1 --2- 172.21.15.52:0/1978092984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575a5669be0 0x5575a5676120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.612+0000 7fbb44d55b80 1 --2- 172.21.15.52:0/1978092984 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575a566a4f0 0x5575a56733a0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.612+0000 7fbb44d55b80 1 -- 172.21.15.52:0/1978092984 >> 172.21.15.52:0/1978092984 conn(0x5575a565f3e0 msgr2=0x5575a566fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:57.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.612+0000 7fbb44d55b80 1 -- 172.21.15.52:0/1978092984 shutdown_connections 2023-11-27T15:25:57.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.612+0000 7fbb44d55b80 1 -- 172.21.15.52:0/1978092984 wait complete. 2023-11-27T15:25:57.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.612+0000 7fbb44d55b80 20 client.16815 trim_cache size 0 max 16384 2023-11-27T15:25:57.644 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.629+0000 7fbb44d55b80 1 Processor -- start 2023-11-27T15:25:57.644 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.629+0000 7fbb44d55b80 1 -- start start 2023-11-27T15:25:57.644 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.629+0000 7fbb44d55b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575a5669be0 0x5575a567a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.644 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.629+0000 7fbb44d55b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575a566a4f0 0x5575a5672b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.629+0000 7fbb44d55b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575a566e930 0x5575a56730a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.629+0000 7fbb44d55b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575a5674320 con 0x5575a566e930 2023-11-27T15:25:57.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.629+0000 7fbb44d55b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575a5671900 con 0x5575a5669be0 2023-11-27T15:25:57.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.629+0000 7fbb44d55b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575a566d4c0 con 0x5575a566a4f0 2023-11-27T15:25:57.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.629+0000 7fbb32d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575a566e930 0x5575a56730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.629+0000 7fbb32d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575a566e930 0x5575a56730a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55364/0 (socket says 172.21.15.52:55364) 2023-11-27T15:25:57.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.629+0000 7fbb32d9d700 1 -- 172.21.15.52:0/2518944373 learned_addr learned my addr 172.21.15.52:0/2518944373 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:57.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.629+0000 7fbb3259c700 1 --2- 172.21.15.52:0/2518944373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575a5669be0 0x5575a567a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.629+0000 7fbb31d9b700 1 --2- 172.21.15.52:0/2518944373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575a566a4f0 0x5575a5672b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.629+0000 7fbb32d9d700 1 -- 172.21.15.52:0/2518944373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575a566a4f0 msgr2=0x5575a5672b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:57.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.629+0000 7fbb32d9d700 1 --2- 172.21.15.52:0/2518944373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575a566a4f0 0x5575a5672b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.629+0000 7fbb32d9d700 1 -- 172.21.15.52:0/2518944373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575a5669be0 msgr2=0x5575a567a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:57.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.629+0000 7fbb32d9d700 1 --2- 172.21.15.52:0/2518944373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575a5669be0 0x5575a567a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.629+0000 7fbb32d9d700 1 -- 172.21.15.52:0/2518944373 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575a566f220 con 0x5575a566e930 2023-11-27T15:25:57.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.629+0000 7fbb3259c700 1 --2- 172.21.15.52:0/2518944373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575a5669be0 0x5575a567a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:57.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.630+0000 7fbb32d9d700 1 --2- 172.21.15.52:0/2518944373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575a566e930 0x5575a56730a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c099300 tx=0x7fbb2c00b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71223a9b37e3fe18 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:57.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.630+0000 7fbb297fa700 1 -- 172.21.15.52:0/2518944373 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c0b7070 con 0x5575a566e930 2023-11-27T15:25:57.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.630+0000 7fbb297fa700 1 -- 172.21.15.52:0/2518944373 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb2c09efe0 con 0x5575a566e930 2023-11-27T15:25:57.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.630+0000 7fbb297fa700 1 -- 172.21.15.52:0/2518944373 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c09a4a0 con 0x5575a566e930 2023-11-27T15:25:57.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.630+0000 7fbb44d55b80 1 -- 172.21.15.52:0/2518944373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575a566e930 msgr2=0x5575a56730a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:57.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.630+0000 7fbb44d55b80 1 --2- 172.21.15.52:0/2518944373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575a566e930 0x5575a56730a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c099300 tx=0x7fbb2c00b9d0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.630+0000 7fbb44d55b80 1 -- 172.21.15.52:0/2518944373 shutdown_connections 2023-11-27T15:25:57.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.630+0000 7fbb44d55b80 1 --2- 172.21.15.52:0/2518944373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575a5669be0 0x5575a567a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.630+0000 7fbb44d55b80 1 --2- 172.21.15.52:0/2518944373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575a566e930 0x5575a56730a0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.630+0000 7fbb44d55b80 1 --2- 172.21.15.52:0/2518944373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575a566a4f0 0x5575a5672b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.630+0000 7fbb44d55b80 1 -- 172.21.15.52:0/2518944373 >> 172.21.15.52:0/2518944373 conn(0x5575a565f3e0 msgr2=0x5575a566c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:57.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.630+0000 7fbb44d55b80 1 -- 172.21.15.52:0/2518944373 shutdown_connections 2023-11-27T15:25:57.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.630+0000 7fbb44d55b80 1 -- 172.21.15.52:0/2518944373 wait complete. 2023-11-27T15:25:57.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.631+0000 7fbb44d55b80 1 -- start start 2023-11-27T15:25:57.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.631+0000 7fbb44d55b80 1 Processor -- start 2023-11-27T15:25:57.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.631+0000 7fbb44d55b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575a5669be0 0x5575a5678e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.631+0000 7fbb44d55b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575a566a4f0 0x5575a5679380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.631+0000 7fbb44d55b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575a566e930 0x5575a56798c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.631+0000 7fbb44d55b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575a56d7660 con 0x5575a566a4f0 2023-11-27T15:25:57.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.631+0000 7fbb44d55b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575a56d77a0 con 0x5575a5669be0 2023-11-27T15:25:57.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.631+0000 7fbb44d55b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575a56d78e0 con 0x5575a566e930 2023-11-27T15:25:57.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.631+0000 7fbb44d55b80 10 client.0 _ll_register_callbacks cb 0x5575a557a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:57.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.631+0000 7fbb31d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575a566a4f0 0x5575a5679380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.631+0000 7fbb31d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575a566a4f0 0x5575a5679380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55368/0 (socket says 172.21.15.52:55368) 2023-11-27T15:25:57.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.631+0000 7fbb31d9b700 1 -- 172.21.15.52:0/4238180197 learned_addr learned my addr 172.21.15.52:0/4238180197 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:57.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.631+0000 7fbb32d9d700 1 --2- 172.21.15.52:0/4238180197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575a566e930 0x5575a56798c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.631+0000 7fbb3259c700 1 --2- 172.21.15.52:0/4238180197 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575a5669be0 0x5575a5678e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.632+0000 7fbb31d9b700 1 -- 172.21.15.52:0/4238180197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575a566e930 msgr2=0x5575a56798c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:57.651 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.632+0000 7fbb31d9b700 1 --2- 172.21.15.52:0/4238180197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575a566e930 0x5575a56798c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.632+0000 7fbb31d9b700 1 -- 172.21.15.52:0/4238180197 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575a5669be0 msgr2=0x5575a5678e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:57.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.632+0000 7fbb31d9b700 1 --2- 172.21.15.52:0/4238180197 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575a5669be0 0x5575a5678e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.632+0000 7fbb31d9b700 1 -- 172.21.15.52:0/4238180197 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5575a543c6b0 con 0x5575a566a4f0 2023-11-27T15:25:57.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.632+0000 7fbb31d9b700 1 --2- 172.21.15.52:0/4238180197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575a566a4f0 0x5575a5679380 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c001eb0 tx=0x7fbb1c01bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a47a912e4447bf server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:57.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.632+0000 7fbb19ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:25:57.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.632+0000 7fbb19ffb700 1 -- 172.21.15.52:0/4238180197 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c02f070 con 0x5575a566a4f0 2023-11-27T15:25:57.652 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.632+0000 7fbb19ffb700 1 -- 172.21.15.52:0/4238180197 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fbb1c003b70 con 0x5575a566a4f0 2023-11-27T15:25:57.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.632+0000 7fbb44d55b80 1 -- 172.21.15.52:0/4238180197 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb2c09afb0 con 0x5575a566a4f0 2023-11-27T15:25:57.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.632+0000 7fbb19ffb700 1 -- 172.21.15.52:0/4238180197 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb1c003d50 con 0x5575a566a4f0 2023-11-27T15:25:57.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.632+0000 7fbb44d55b80 10 client.16821 fetch_fsmap learned FSMap version 32 2023-11-27T15:25:57.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.632+0000 7fbb44d55b80 1 -- 172.21.15.52:0/4238180197 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5575a53e8e60 con 0x5575a566a4f0 2023-11-27T15:25:57.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.632+0000 7fbb19ffb700 1 -- 172.21.15.52:0/4238180197 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb2c09afb0 con 0x5575a566a4f0 2023-11-27T15:25:57.653 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.632+0000 7fbb19ffb700 1 -- 172.21.15.52:0/4238180197 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbb1c032020 con 0x5575a566a4f0 2023-11-27T15:25:57.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.632+0000 7fbb44d55b80 10 client.16821 fetch_fsmap finished waiting for FSMap version 32 2023-11-27T15:25:57.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.632+0000 7fbb44d55b80 10 client.16821 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:57.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.632+0000 7fbb44d55b80 20 client.16821 populate_metadata read hostname 'smithi052' 2023-11-27T15:25:57.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.633+0000 7fbb44d55b80 1 --2- 172.21.15.52:0/4238180197 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5575a56837d0 0x5575a56db000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.633+0000 7fbb44d55b80 4 client.16821 mds_command: new command op to 24536 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-27T15:25:57.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.633+0000 7fbb44d55b80 1 -- 172.21.15.52:0/4238180197 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5575a55896e0 con 0x5575a56837d0 2023-11-27T15:25:57.654 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.633+0000 7fbb3259c700 1 --2- 172.21.15.52:0/4238180197 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5575a56837d0 0x5575a56db000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.655 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.633+0000 7fbb19ffb700 10 client.16821 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:57.655 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.633+0000 7fbb3259c700 1 --2- 172.21.15.52:0/4238180197 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5575a56837d0 0x5575a56db000 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d92373a3686f86fb server_cookie=1f24399809f2112e in_seq=0 out_seq=0 2023-11-27T15:25:57.655 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.634+0000 7fbb19ffb700 1 -- 172.21.15.52:0/4238180197 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5575a55896e0 con 0x5575a56837d0 2023-11-27T15:25:57.655 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.634+0000 7fbb19ffb700 10 client.16821 handle_command_reply: tid=0 2023-11-27T15:25:57.655 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.634+0000 7fbb44d55b80 1 client.16821 shutdown 2023-11-27T15:25:57.655 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.635+0000 7fbb44d55b80 1 -- 172.21.15.52:0/4238180197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575a566a4f0 msgr2=0x5575a5679380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:57.656 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.635+0000 7fbb44d55b80 1 --2- 172.21.15.52:0/4238180197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575a566a4f0 0x5575a5679380 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c001eb0 tx=0x7fbb1c01bf90 comp rx=0 tx=0).stop 2023-11-27T15:25:57.656 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.635+0000 7fbb44d55b80 1 -- 172.21.15.52:0/4238180197 shutdown_connections 2023-11-27T15:25:57.656 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.635+0000 7fbb44d55b80 1 --2- 172.21.15.52:0/4238180197 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5575a56837d0 0x5575a56db000 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.656 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.635+0000 7fbb44d55b80 1 --2- 172.21.15.52:0/4238180197 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575a5669be0 0x5575a5678e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.656 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.635+0000 7fbb44d55b80 1 --2- 172.21.15.52:0/4238180197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5575a566e930 0x5575a56798c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.656 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.635+0000 7fbb44d55b80 1 --2- 172.21.15.52:0/4238180197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575a566a4f0 0x5575a5679380 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.657 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.635+0000 7fbb44d55b80 1 -- 172.21.15.52:0/4238180197 >> 172.21.15.52:0/4238180197 conn(0x5575a565f3e0 msgr2=0x5575a56689e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:57.657 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.635+0000 7fbb19ffb700 0 client.16821 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:57.657 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.635+0000 7fbb44d55b80 1 -- 172.21.15.52:0/4238180197 shutdown_connections 2023-11-27T15:25:57.657 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.635+0000 7fbb44d55b80 1 -- 172.21.15.52:0/4238180197 wait complete. 2023-11-27T15:25:57.657 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.635+0000 7fbb44d55b80 20 client.16821 trim_cache size 0 max 16384 2023-11-27T15:25:57.658 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:25:57.658 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2023-11-27T15:25:57.658 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "754cc4b7-561a-4e09-8c3c-e73dcacaddbb", 2023-11-27T15:25:57.658 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2023-11-27T15:25:57.658 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:25:57.660 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.635+0000 7fbb38bc9700 1 -- 172.21.15.52:0/689501874 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fbb200a12c0 msgr2=0x7fbb200a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:57.660 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.635+0000 7fbb38bc9700 1 --2- 172.21.15.52:0/689501874 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fbb200a12c0 0x7fbb200a3770 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbb24005ea0 tx=0x7fbb24005e30 comp rx=0 tx=0).stop 2023-11-27T15:25:57.660 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.635+0000 7fbb38bc9700 1 -- 172.21.15.52:0/689501874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb341055f0 msgr2=0x7fbb3419bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:57.660 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.635+0000 7fbb38bc9700 1 --2- 172.21.15.52:0/689501874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb341055f0 0x7fbb3419bf80 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c007870 tx=0x7fbb2c0078a0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.635+0000 7fbb38bc9700 1 -- 172.21.15.52:0/689501874 shutdown_connections 2023-11-27T15:25:57.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.635+0000 7fbb38bc9700 1 --2- 172.21.15.52:0/689501874 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fbb200a12c0 0x7fbb200a3770 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.635+0000 7fbb38bc9700 1 --2- 172.21.15.52:0/689501874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb341055f0 0x7fbb3419bf80 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.635+0000 7fbb38bc9700 1 --2- 172.21.15.52:0/689501874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb341005b0 0x7fbb34198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.635+0000 7fbb38bc9700 1 --2- 172.21.15.52:0/689501874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb34100f50 0x7fbb34198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.635+0000 7fbb38bc9700 1 -- 172.21.15.52:0/689501874 >> 172.21.15.52:0/689501874 conn(0x7fbb34077140 msgr2=0x7fbb3410b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:57.661 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.636+0000 7fbb38bc9700 1 -- 172.21.15.52:0/689501874 shutdown_connections 2023-11-27T15:25:57.662 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.636+0000 7fbb38bc9700 1 -- 172.21.15.52:0/689501874 wait complete. 2023-11-27T15:25:57.662 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:25:57.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.785+0000 7f7d35ebc700 1 -- 172.21.15.52:0/944746364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d301041f0 msgr2=0x7f7d301045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:57.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.785+0000 7f7d35ebc700 1 --2- 172.21.15.52:0/944746364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d301041f0 0x7f7d301045c0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009990 tx=0x7f7d20009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.786+0000 7f7d35ebc700 1 -- 172.21.15.52:0/944746364 shutdown_connections 2023-11-27T15:25:57.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.786+0000 7f7d35ebc700 1 --2- 172.21.15.52:0/944746364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d30100470 0x7f7d3010c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.786+0000 7f7d35ebc700 1 --2- 172.21.15.52:0/944746364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d301041f0 0x7f7d301045c0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.786+0000 7f7d35ebc700 1 --2- 172.21.15.52:0/944746364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d300ffac0 0x7f7d300fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.786+0000 7f7d35ebc700 1 -- 172.21.15.52:0/944746364 >> 172.21.15.52:0/944746364 conn(0x7f7d300763a0 msgr2=0x7f7d300767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:57.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.786+0000 7f7d35ebc700 1 -- 172.21.15.52:0/944746364 shutdown_connections 2023-11-27T15:25:57.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.786+0000 7f7d35ebc700 1 -- 172.21.15.52:0/944746364 wait complete. 2023-11-27T15:25:57.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.786+0000 7f7d35ebc700 1 Processor -- start 2023-11-27T15:25:57.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.786+0000 7f7d35ebc700 1 -- start start 2023-11-27T15:25:57.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.787+0000 7f7d35ebc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d300ffac0 0x7f7d30100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.787+0000 7f7d35ebc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d30100470 0x7f7d30101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.787+0000 7f7d35ebc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d301041f0 0x7f7d301a44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.787+0000 7f7d35ebc700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d30101bc0 con 0x7f7d300ffac0 2023-11-27T15:25:57.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.787+0000 7f7d35ebc700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d301a4a00 con 0x7f7d30100470 2023-11-27T15:25:57.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.787+0000 7f7d35ebc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d301a4b40 con 0x7f7d301041f0 2023-11-27T15:25:57.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.787+0000 7f7d2f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d300ffac0 0x7f7d30100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.787+0000 7f7d2f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d300ffac0 0x7f7d30100e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55380/0 (socket says 172.21.15.52:55380) 2023-11-27T15:25:57.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.787+0000 7f7d2f7fe700 1 -- 172.21.15.52:0/2543332814 learned_addr learned my addr 172.21.15.52:0/2543332814 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:57.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.787+0000 7f7d2ffff700 1 --2- 172.21.15.52:0/2543332814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d301041f0 0x7f7d301a44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.787+0000 7f7d2effd700 1 --2- 172.21.15.52:0/2543332814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d30100470 0x7f7d30101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.787+0000 7f7d2f7fe700 1 -- 172.21.15.52:0/2543332814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d301041f0 msgr2=0x7f7d301a44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:57.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.787+0000 7f7d2f7fe700 1 --2- 172.21.15.52:0/2543332814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d301041f0 0x7f7d301a44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.787+0000 7f7d2f7fe700 1 -- 172.21.15.52:0/2543332814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d30100470 msgr2=0x7f7d30101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:57.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.787+0000 7f7d2f7fe700 1 --2- 172.21.15.52:0/2543332814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d30100470 0x7f7d30101390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.787+0000 7f7d2f7fe700 1 -- 172.21.15.52:0/2543332814 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d200095f0 con 0x7f7d300ffac0 2023-11-27T15:25:57.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.787+0000 7f7d2effd700 1 --2- 172.21.15.52:0/2543332814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d30100470 0x7f7d30101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:57.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.787+0000 7f7d2ffff700 1 --2- 172.21.15.52:0/2543332814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d301041f0 0x7f7d301a44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:57.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.787+0000 7f7d2f7fe700 1 --2- 172.21.15.52:0/2543332814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d300ffac0 0x7f7d30100e50 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f7d20000c00 tx=0x7f7d20005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:57.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.787+0000 7f7d2cff9700 1 -- 172.21.15.52:0/2543332814 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2001d070 con 0x7f7d300ffac0 2023-11-27T15:25:57.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.787+0000 7f7d2cff9700 1 -- 172.21.15.52:0/2543332814 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d2000fbc0 con 0x7f7d300ffac0 2023-11-27T15:25:57.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.787+0000 7f7d35ebc700 1 -- 172.21.15.52:0/2543332814 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d3010f260 con 0x7f7d300ffac0 2023-11-27T15:25:57.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.787+0000 7f7d2cff9700 1 -- 172.21.15.52:0/2543332814 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d200173f0 con 0x7f7d300ffac0 2023-11-27T15:25:57.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.787+0000 7f7d35ebc700 1 -- 172.21.15.52:0/2543332814 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d3010f0d0 con 0x7f7d300ffac0 2023-11-27T15:25:57.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.789+0000 7f7d2cff9700 1 -- 172.21.15.52:0/2543332814 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7d20021410 con 0x7f7d300ffac0 2023-11-27T15:25:57.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.790+0000 7f7d2cff9700 1 --2- 172.21.15.52:0/2543332814 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f7d180a1340 0x7f7d180a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.790+0000 7f7d2effd700 1 --2- 172.21.15.52:0/2543332814 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f7d180a1340 0x7f7d180a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.790+0000 7f7d2cff9700 1 -- 172.21.15.52:0/2543332814 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f7d200ce020 con 0x7f7d300ffac0 2023-11-27T15:25:57.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.791+0000 7f7d2effd700 1 --2- 172.21.15.52:0/2543332814 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f7d180a1340 0x7f7d180a37f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7d30102640 tx=0x7f7d24009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:57.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.791+0000 7f7d42048b80 1 Processor -- start 2023-11-27T15:25:57.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.791+0000 7f7d42048b80 1 -- start start 2023-11-27T15:25:57.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.791+0000 7f7d42048b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564117437b60 0x564117437f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.791+0000 7f7d42048b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564117438470 0x56411743c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.791+0000 7f7d42048b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56411743c8b0 0x56411743cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.791+0000 7f7d42048b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x56411733d060 con 0x56411743c8b0 2023-11-27T15:25:57.807 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.791+0000 7f7d42048b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56411738f8a0 con 0x564117438470 2023-11-27T15:25:57.807 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.791+0000 7f7d42048b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564117388d30 con 0x564117437b60 2023-11-27T15:25:57.807 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.791+0000 7f7d2ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56411743c8b0 0x56411743cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.807 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.791+0000 7f7d2ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56411743c8b0 0x56411743cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55392/0 (socket says 172.21.15.52:55392) 2023-11-27T15:25:57.807 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.791+0000 7f7d2ffff700 1 -- 172.21.15.52:0/3255881318 learned_addr learned my addr 172.21.15.52:0/3255881318 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:57.807 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.791+0000 7f7d2f7fe700 1 --2- 172.21.15.52:0/3255881318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564117437b60 0x564117437f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.807 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.791+0000 7f7d2effd700 1 --2- 172.21.15.52:0/3255881318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564117438470 0x56411743c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.808 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.791+0000 7f7d2ffff700 1 -- 172.21.15.52:0/3255881318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564117437b60 msgr2=0x564117437f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:57.808 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.791+0000 7f7d2ffff700 1 --2- 172.21.15.52:0/3255881318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564117437b60 0x564117437f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.808 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.791+0000 7f7d2ffff700 1 -- 172.21.15.52:0/3255881318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564117438470 msgr2=0x56411743c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:57.808 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.792+0000 7f7d2ffff700 1 --2- 172.21.15.52:0/3255881318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564117438470 0x56411743c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.808 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.792+0000 7f7d2ffff700 1 -- 172.21.15.52:0/3255881318 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56411720a630 con 0x56411743c8b0 2023-11-27T15:25:57.808 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.792+0000 7f7d2effd700 1 --2- 172.21.15.52:0/3255881318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564117438470 0x56411743c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:57.808 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.792+0000 7f7d2ffff700 1 --2- 172.21.15.52:0/3255881318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56411743c8b0 0x56411743cd00 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f7d280093f0 tx=0x7f7d28009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d82e21bc18b7915 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:57.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.792+0000 7f7d15ffb700 1 -- 172.21.15.52:0/3255881318 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2800e030 con 0x56411743c8b0 2023-11-27T15:25:57.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.792+0000 7f7d15ffb700 1 -- 172.21.15.52:0/3255881318 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d2800f040 con 0x56411743c8b0 2023-11-27T15:25:57.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.792+0000 7f7d15ffb700 1 -- 172.21.15.52:0/3255881318 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d28015a30 con 0x56411743c8b0 2023-11-27T15:25:57.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.792+0000 7f7d42048b80 1 -- 172.21.15.52:0/3255881318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56411743c8b0 msgr2=0x56411743cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:57.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.792+0000 7f7d42048b80 1 --2- 172.21.15.52:0/3255881318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56411743c8b0 0x56411743cd00 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f7d280093f0 tx=0x7f7d28009920 comp rx=0 tx=0).stop 2023-11-27T15:25:57.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.793+0000 7f7d42048b80 1 -- 172.21.15.52:0/3255881318 shutdown_connections 2023-11-27T15:25:57.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.793+0000 7f7d42048b80 1 --2- 172.21.15.52:0/3255881318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56411743c8b0 0x56411743cd00 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.809 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.793+0000 7f7d42048b80 1 --2- 172.21.15.52:0/3255881318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564117437b60 0x564117437f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.810 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.793+0000 7f7d42048b80 1 --2- 172.21.15.52:0/3255881318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564117438470 0x56411743c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.810 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.793+0000 7f7d42048b80 1 -- 172.21.15.52:0/3255881318 >> 172.21.15.52:0/3255881318 conn(0x56411742d360 msgr2=0x564117432b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:57.810 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.793+0000 7f7d42048b80 1 -- 172.21.15.52:0/3255881318 shutdown_connections 2023-11-27T15:25:57.810 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.793+0000 7f7d42048b80 1 -- 172.21.15.52:0/3255881318 wait complete. 2023-11-27T15:25:57.810 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.793+0000 7f7d42048b80 1 -- start start 2023-11-27T15:25:57.810 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.793+0000 7f7d42048b80 1 Processor -- start 2023-11-27T15:25:57.810 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.793+0000 7f7d42048b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564117437b60 0x564117444040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.793+0000 7f7d42048b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564117438470 0x564117441250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.793+0000 7f7d42048b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56411743c8b0 0x564117441860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.793+0000 7f7d42048b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x56411732a8e0 con 0x564117438470 2023-11-27T15:25:57.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.794+0000 7f7d42048b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56411732a580 con 0x564117437b60 2023-11-27T15:25:57.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.794+0000 7f7d42048b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56411720c920 con 0x56411743c8b0 2023-11-27T15:25:57.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.794+0000 7f7d42048b80 10 client.0 _ll_register_callbacks cb 0x564117348780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:57.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.794+0000 7f7d2effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564117438470 0x564117441250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.794+0000 7f7d2effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564117438470 0x564117441250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55396/0 (socket says 172.21.15.52:55396) 2023-11-27T15:25:57.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.794+0000 7f7d2effd700 1 -- 172.21.15.52:0/486392767 learned_addr learned my addr 172.21.15.52:0/486392767 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:57.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.794+0000 7f7d2f7fe700 1 --2- 172.21.15.52:0/486392767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564117437b60 0x564117444040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.794+0000 7f7d2ffff700 1 --2- 172.21.15.52:0/486392767 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56411743c8b0 0x564117441860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.794+0000 7f7d2effd700 1 -- 172.21.15.52:0/486392767 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56411743c8b0 msgr2=0x564117441860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:57.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.794+0000 7f7d2effd700 1 --2- 172.21.15.52:0/486392767 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56411743c8b0 0x564117441860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.794+0000 7f7d2effd700 1 -- 172.21.15.52:0/486392767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564117437b60 msgr2=0x564117444040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:57.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.794+0000 7f7d2effd700 1 --2- 172.21.15.52:0/486392767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564117437b60 0x564117444040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.794+0000 7f7d2effd700 1 -- 172.21.15.52:0/486392767 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d280090d0 con 0x564117438470 2023-11-27T15:25:57.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.794+0000 7f7d2effd700 1 --2- 172.21.15.52:0/486392767 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564117438470 0x564117441250 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f7d24010f10 tx=0x7f7d24004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f90d0b03acf8c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:57.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.794+0000 7f7d0effd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:25:57.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.794+0000 7f7d0effd700 1 -- 172.21.15.52:0/486392767 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d24009890 con 0x564117438470 2023-11-27T15:25:57.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.794+0000 7f7d0effd700 1 -- 172.21.15.52:0/486392767 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f7d24012cf0 con 0x564117438470 2023-11-27T15:25:57.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.794+0000 7f7d42048b80 1 -- 172.21.15.52:0/486392767 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56411742ab20 con 0x564117438470 2023-11-27T15:25:57.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.795+0000 7f7d0effd700 1 -- 172.21.15.52:0/486392767 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d24004030 con 0x564117438470 2023-11-27T15:25:57.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.795+0000 7f7d42048b80 10 client.16833 fetch_fsmap learned FSMap version 32 2023-11-27T15:25:57.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.795+0000 7f7d42048b80 1 -- 172.21.15.52:0/486392767 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56411734f190 con 0x564117438470 2023-11-27T15:25:57.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.795+0000 7f7d0effd700 1 -- 172.21.15.52:0/486392767 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56411742ab20 con 0x564117438470 2023-11-27T15:25:57.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.795+0000 7f7d0effd700 1 -- 172.21.15.52:0/486392767 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7d24024020 con 0x564117438470 2023-11-27T15:25:57.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.795+0000 7f7d42048b80 10 client.16833 fetch_fsmap finished waiting for FSMap version 32 2023-11-27T15:25:57.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.795+0000 7f7d42048b80 10 client.16833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:57.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.795+0000 7f7d42048b80 20 client.16833 populate_metadata read hostname 'smithi052' 2023-11-27T15:25:57.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.795+0000 7f7d42048b80 1 --2- 172.21.15.52:0/486392767 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x564117451820 0x564117453c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.795+0000 7f7d42048b80 4 client.16833 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:57.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.795+0000 7f7d42048b80 1 -- 172.21.15.52:0/486392767 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56411734b0a0 con 0x564117451820 2023-11-27T15:25:57.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.795+0000 7f7d2f7fe700 1 --2- 172.21.15.52:0/486392767 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x564117451820 0x564117453c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.796+0000 7f7d0effd700 10 client.16833 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:57.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.796+0000 7f7d2f7fe700 1 --2- 172.21.15.52:0/486392767 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x564117451820 0x564117453c30 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c7d21224422a789 server_cookie=204e79ba7efa43d9 in_seq=0 out_seq=0 2023-11-27T15:25:57.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.796+0000 7f7d0effd700 1 -- 172.21.15.52:0/486392767 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56411734b0a0 con 0x564117451820 2023-11-27T15:25:57.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.796+0000 7f7d0effd700 10 client.16833 handle_command_reply: tid=0 2023-11-27T15:25:57.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.796+0000 7f7d42048b80 1 client.16833 shutdown 2023-11-27T15:25:57.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.796+0000 7f7d42048b80 1 -- 172.21.15.52:0/486392767 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564117438470 msgr2=0x564117441250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:57.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.797+0000 7f7d42048b80 1 --2- 172.21.15.52:0/486392767 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564117438470 0x564117441250 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f7d24010f10 tx=0x7f7d24004750 comp rx=0 tx=0).stop 2023-11-27T15:25:57.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.797+0000 7f7d42048b80 1 -- 172.21.15.52:0/486392767 shutdown_connections 2023-11-27T15:25:57.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.797+0000 7f7d42048b80 1 --2- 172.21.15.52:0/486392767 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x564117451820 0x564117453c30 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.797+0000 7f7d42048b80 1 --2- 172.21.15.52:0/486392767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564117437b60 0x564117444040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.797+0000 7f7d0effd700 0 client.16833 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:57.817 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.797+0000 7f7d42048b80 1 --2- 172.21.15.52:0/486392767 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56411743c8b0 0x564117441860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.817 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.797+0000 7f7d42048b80 1 --2- 172.21.15.52:0/486392767 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564117438470 0x564117441250 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.817 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.797+0000 7f7d42048b80 1 -- 172.21.15.52:0/486392767 >> 172.21.15.52:0/486392767 conn(0x56411742d360 msgr2=0x56411743dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:57.817 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.797+0000 7f7d42048b80 1 -- 172.21.15.52:0/486392767 shutdown_connections 2023-11-27T15:25:57.817 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.797+0000 7f7d42048b80 1 -- 172.21.15.52:0/486392767 wait complete. 2023-11-27T15:25:57.817 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.797+0000 7f7d42048b80 20 client.16833 trim_cache size 0 max 16384 2023-11-27T15:25:57.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.814+0000 7f7d42048b80 1 Processor -- start 2023-11-27T15:25:57.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.814+0000 7f7d42048b80 1 -- start start 2023-11-27T15:25:57.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.814+0000 7f7d42048b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564117437b60 0x5641174489a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.814+0000 7f7d42048b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564117438470 0x564117443b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.814+0000 7f7d42048b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56411743c8b0 0x564117440af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.814+0000 7f7d42048b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x564117438840 con 0x564117438470 2023-11-27T15:25:57.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.814+0000 7f7d42048b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56411743f7d0 con 0x564117437b60 2023-11-27T15:25:57.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.814+0000 7f7d42048b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56411743b3c0 con 0x56411743c8b0 2023-11-27T15:25:57.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.814+0000 7f7d2effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564117438470 0x564117443b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.814+0000 7f7d2effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564117438470 0x564117443b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55404/0 (socket says 172.21.15.52:55404) 2023-11-27T15:25:57.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.814+0000 7f7d2effd700 1 -- 172.21.15.52:0/3579347155 learned_addr learned my addr 172.21.15.52:0/3579347155 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:57.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.814+0000 7f7d2ffff700 1 --2- 172.21.15.52:0/3579347155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56411743c8b0 0x564117440af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.814+0000 7f7d2f7fe700 1 --2- 172.21.15.52:0/3579347155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564117437b60 0x5641174489a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.814+0000 7f7d2effd700 1 -- 172.21.15.52:0/3579347155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56411743c8b0 msgr2=0x564117440af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:57.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.814+0000 7f7d2effd700 1 --2- 172.21.15.52:0/3579347155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56411743c8b0 0x564117440af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.814+0000 7f7d2effd700 1 -- 172.21.15.52:0/3579347155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564117437b60 msgr2=0x5641174489a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:57.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.814+0000 7f7d2effd700 1 --2- 172.21.15.52:0/3579347155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564117437b60 0x5641174489a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.814+0000 7f7d2effd700 1 -- 172.21.15.52:0/3579347155 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56411742eb00 con 0x564117438470 2023-11-27T15:25:57.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.814+0000 7f7d2effd700 1 --2- 172.21.15.52:0/3579347155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564117438470 0x564117443b70 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f7d24010780 tx=0x7f7d24012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae806a0c2791a73 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:57.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.814+0000 7f7d167fc700 1 -- 172.21.15.52:0/3579347155 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d24012af0 con 0x564117438470 2023-11-27T15:25:57.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.815+0000 7f7d167fc700 1 -- 172.21.15.52:0/3579347155 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d24004460 con 0x564117438470 2023-11-27T15:25:57.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.815+0000 7f7d167fc700 1 -- 172.21.15.52:0/3579347155 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2401edd0 con 0x564117438470 2023-11-27T15:25:57.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.815+0000 7f7d42048b80 1 -- 172.21.15.52:0/3579347155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564117438470 msgr2=0x564117443b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:57.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.815+0000 7f7d42048b80 1 --2- 172.21.15.52:0/3579347155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564117438470 0x564117443b70 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f7d24010780 tx=0x7f7d24012730 comp rx=0 tx=0).stop 2023-11-27T15:25:57.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.815+0000 7f7d42048b80 1 -- 172.21.15.52:0/3579347155 shutdown_connections 2023-11-27T15:25:57.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.815+0000 7f7d42048b80 1 --2- 172.21.15.52:0/3579347155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564117437b60 0x5641174489a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.815+0000 7f7d42048b80 1 --2- 172.21.15.52:0/3579347155 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56411743c8b0 0x564117440af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.815+0000 7f7d42048b80 1 --2- 172.21.15.52:0/3579347155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564117438470 0x564117443b70 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.815+0000 7f7d42048b80 1 -- 172.21.15.52:0/3579347155 >> 172.21.15.52:0/3579347155 conn(0x56411742d360 msgr2=0x56411743a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:57.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.815+0000 7f7d42048b80 1 -- 172.21.15.52:0/3579347155 shutdown_connections 2023-11-27T15:25:57.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.815+0000 7f7d42048b80 1 -- 172.21.15.52:0/3579347155 wait complete. 2023-11-27T15:25:57.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.815+0000 7f7d42048b80 1 -- start start 2023-11-27T15:25:57.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.816+0000 7f7d42048b80 1 Processor -- start 2023-11-27T15:25:57.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.816+0000 7f7d42048b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564117437b60 0x564117446b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.816+0000 7f7d42048b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564117438470 0x5641174470c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.816+0000 7f7d42048b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56411743c8b0 0x56411749e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.816+0000 7f7d42048b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x56411749ebf0 con 0x56411743c8b0 2023-11-27T15:25:57.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.816+0000 7f7d42048b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56411749ed60 con 0x564117437b60 2023-11-27T15:25:57.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.816+0000 7f7d42048b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56411749ef20 con 0x564117438470 2023-11-27T15:25:57.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.816+0000 7f7d42048b80 10 client.0 _ll_register_callbacks cb 0x7f7d24010af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:57.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.816+0000 7f7d2ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56411743c8b0 0x56411749e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.832 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.816+0000 7f7d2ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56411743c8b0 0x56411749e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55412/0 (socket says 172.21.15.52:55412) 2023-11-27T15:25:57.832 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.816+0000 7f7d2ffff700 1 -- 172.21.15.52:0/599640253 learned_addr learned my addr 172.21.15.52:0/599640253 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:25:57.832 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.816+0000 7f7d2effd700 1 --2- 172.21.15.52:0/599640253 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564117438470 0x5641174470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.832 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.816+0000 7f7d2f7fe700 1 --2- 172.21.15.52:0/599640253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564117437b60 0x564117446b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.832 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.816+0000 7f7d2ffff700 1 -- 172.21.15.52:0/599640253 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564117438470 msgr2=0x5641174470c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:57.832 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.816+0000 7f7d2ffff700 1 --2- 172.21.15.52:0/599640253 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564117438470 0x5641174470c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.816+0000 7f7d2ffff700 1 -- 172.21.15.52:0/599640253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564117437b60 msgr2=0x564117446b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:57.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.816+0000 7f7d2ffff700 1 --2- 172.21.15.52:0/599640253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564117437b60 0x564117446b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.816+0000 7f7d2ffff700 1 -- 172.21.15.52:0/599640253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56411734f190 con 0x56411743c8b0 2023-11-27T15:25:57.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.816+0000 7f7d2effd700 1 --2- 172.21.15.52:0/599640253 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564117438470 0x5641174470c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:57.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.817+0000 7f7d2ffff700 1 --2- 172.21.15.52:0/599640253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56411743c8b0 0x56411749e6b0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f7d28000f80 tx=0x7f7d28014960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c100faee4ab3a942 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:57.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.817+0000 7f7d0effd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:25:57.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.817+0000 7f7d0effd700 1 -- 172.21.15.52:0/599640253 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2800e040 con 0x56411743c8b0 2023-11-27T15:25:57.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.817+0000 7f7d0effd700 1 -- 172.21.15.52:0/599640253 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f7d28007e40 con 0x56411743c8b0 2023-11-27T15:25:57.834 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.817+0000 7f7d42048b80 1 -- 172.21.15.52:0/599640253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d2800f040 con 0x56411743c8b0 2023-11-27T15:25:57.834 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.817+0000 7f7d0effd700 1 -- 172.21.15.52:0/599640253 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d28014de0 con 0x56411743c8b0 2023-11-27T15:25:57.834 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.817+0000 7f7d42048b80 10 client.16839 fetch_fsmap learned FSMap version 32 2023-11-27T15:25:57.834 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.817+0000 7f7d42048b80 1 -- 172.21.15.52:0/599640253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5641171b6de0 con 0x56411743c8b0 2023-11-27T15:25:57.834 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.817+0000 7f7d0effd700 1 -- 172.21.15.52:0/599640253 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d2800f040 con 0x56411743c8b0 2023-11-27T15:25:57.834 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.817+0000 7f7d0effd700 1 -- 172.21.15.52:0/599640253 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7d28004660 con 0x56411743c8b0 2023-11-27T15:25:57.834 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.817+0000 7f7d42048b80 10 client.16839 fetch_fsmap finished waiting for FSMap version 32 2023-11-27T15:25:57.835 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.817+0000 7f7d42048b80 10 client.16839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:57.835 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.817+0000 7f7d42048b80 20 client.16839 populate_metadata read hostname 'smithi052' 2023-11-27T15:25:57.835 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.817+0000 7f7d42048b80 1 --2- 172.21.15.52:0/599640253 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5641174a66a0 0x5641174a8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:57.835 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.817+0000 7f7d42048b80 4 client.16839 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:25:57.835 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.817+0000 7f7d42048b80 1 -- 172.21.15.52:0/599640253 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564117357660 con 0x5641174a66a0 2023-11-27T15:25:57.835 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.817+0000 7f7d2f7fe700 1 --2- 172.21.15.52:0/599640253 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5641174a66a0 0x5641174a8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:57.835 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.818+0000 7f7d0effd700 10 client.16839 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:57.836 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.818+0000 7f7d2f7fe700 1 --2- 172.21.15.52:0/599640253 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5641174a66a0 0x5641174a8ab0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e124024081936138 server_cookie=c9c0e10804eb83c0 in_seq=0 out_seq=0 2023-11-27T15:25:57.836 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.818+0000 7f7d0effd700 1 -- 172.21.15.52:0/599640253 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564117357660 con 0x5641174a66a0 2023-11-27T15:25:57.836 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.818+0000 7f7d0effd700 10 client.16839 handle_command_reply: tid=0 2023-11-27T15:25:57.836 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.818+0000 7f7d42048b80 1 client.16839 shutdown 2023-11-27T15:25:57.836 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.818+0000 7f7d42048b80 1 -- 172.21.15.52:0/599640253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56411743c8b0 msgr2=0x56411749e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:57.836 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.818+0000 7f7d42048b80 1 --2- 172.21.15.52:0/599640253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56411743c8b0 0x56411749e6b0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f7d28000f80 tx=0x7f7d28014960 comp rx=0 tx=0).stop 2023-11-27T15:25:57.836 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.818+0000 7f7d42048b80 1 -- 172.21.15.52:0/599640253 shutdown_connections 2023-11-27T15:25:57.837 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.818+0000 7f7d42048b80 1 --2- 172.21.15.52:0/599640253 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5641174a66a0 0x5641174a8ab0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.837 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.818+0000 7f7d42048b80 1 --2- 172.21.15.52:0/599640253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564117437b60 0x564117446b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.837 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.818+0000 7f7d42048b80 1 --2- 172.21.15.52:0/599640253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56411743c8b0 0x56411749e6b0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.837 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.818+0000 7f7d42048b80 1 --2- 172.21.15.52:0/599640253 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564117438470 0x5641174470c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.837 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.818+0000 7f7d42048b80 1 -- 172.21.15.52:0/599640253 >> 172.21.15.52:0/599640253 conn(0x56411742d360 msgr2=0x564117436960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:57.837 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.818+0000 7f7d0effd700 0 client.16839 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:25:57.837 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.818+0000 7f7d42048b80 1 -- 172.21.15.52:0/599640253 shutdown_connections 2023-11-27T15:25:57.838 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.818+0000 7f7d42048b80 1 -- 172.21.15.52:0/599640253 wait complete. 2023-11-27T15:25:57.838 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.819+0000 7f7d42048b80 20 client.16839 trim_cache size 0 max 16384 2023-11-27T15:25:57.838 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:25:57.838 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-27T15:25:57.839 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2023-11-27T15:25:57.839 INFO:teuthology.orchestra.run.smithi052.stdout: "754cc4b7-561a-4e09-8c3c-e73dcacaddbb": { 2023-11-27T15:25:57.839 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2023-11-27T15:25:57.839 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "754cc4b7-561a-4e09-8c3c-e73dcacaddbb", 2023-11-27T15:25:57.839 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2023-11-27T15:25:57.839 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:25:57.839 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:25:57.840 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:25:57.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.819+0000 7f7d35ebc700 1 -- 172.21.15.52:0/2543332814 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f7d180a1340 msgr2=0x7f7d180a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:57.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.819+0000 7f7d35ebc700 1 --2- 172.21.15.52:0/2543332814 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f7d180a1340 0x7f7d180a37f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7d30102640 tx=0x7f7d24009260 comp rx=0 tx=0).stop 2023-11-27T15:25:57.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.819+0000 7f7d35ebc700 1 -- 172.21.15.52:0/2543332814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d300ffac0 msgr2=0x7f7d30100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:57.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.819+0000 7f7d35ebc700 1 --2- 172.21.15.52:0/2543332814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d300ffac0 0x7f7d30100e50 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f7d20000c00 tx=0x7f7d20005ef0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.841 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.819+0000 7f7d35ebc700 1 -- 172.21.15.52:0/2543332814 shutdown_connections 2023-11-27T15:25:57.842 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.819+0000 7f7d35ebc700 1 --2- 172.21.15.52:0/2543332814 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f7d180a1340 0x7f7d180a37f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.842 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.819+0000 7f7d35ebc700 1 --2- 172.21.15.52:0/2543332814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d301041f0 0x7f7d301a44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.842 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.819+0000 7f7d35ebc700 1 --2- 172.21.15.52:0/2543332814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d300ffac0 0x7f7d30100e50 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.842 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.819+0000 7f7d35ebc700 1 --2- 172.21.15.52:0/2543332814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d30100470 0x7f7d30101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:57.842 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.819+0000 7f7d35ebc700 1 -- 172.21.15.52:0/2543332814 >> 172.21.15.52:0/2543332814 conn(0x7f7d300763a0 msgr2=0x7f7d300691a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:57.842 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.819+0000 7f7d35ebc700 1 -- 172.21.15.52:0/2543332814 shutdown_connections 2023-11-27T15:25:57.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:25:57.819+0000 7f7d35ebc700 1 -- 172.21.15.52:0/2543332814 wait complete. 2023-11-27T15:25:57.843 INFO:tasks.cephfs.filesystem:scrub status for tag:754cc4b7-561a-4e09-8c3c-e73dcacaddbb - {'path': '/', 'tag': '754cc4b7-561a-4e09-8c3c-e73dcacaddbb', 'options': 'recursive,force'} 2023-11-27T15:25:58.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:58 smithi150 ceph-mon[100559]: pgmap v248: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:25:58.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:58 smithi150 ceph-mon[100559]: scrub queued for path: / 2023-11-27T15:25:58.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:58 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:25:58.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:58 smithi150 ceph-mon[100559]: scrub summary: active paths [/] 2023-11-27T15:25:58.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:58 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:25:58.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:58 smithi102 ceph-mon[100296]: pgmap v248: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:25:58.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:58 smithi102 ceph-mon[100296]: scrub queued for path: / 2023-11-27T15:25:58.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:58 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:25:58.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:58 smithi102 ceph-mon[100296]: scrub summary: active paths [/] 2023-11-27T15:25:58.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:58 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:25:58.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:58 smithi052 ceph-mon[99056]: pgmap v248: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:25:58.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:58 smithi052 ceph-mon[99056]: scrub queued for path: / 2023-11-27T15:25:58.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:58 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:25:58.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:58 smithi052 ceph-mon[99056]: scrub summary: active paths [/] 2023-11-27T15:25:58.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:58 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:26:00.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:59 smithi150 ceph-mon[100559]: pgmap v249: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:26:00.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:25:59 smithi150 ceph-mon[100559]: scrub summary: idle 2023-11-27T15:26:00.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:59 smithi102 ceph-mon[100296]: pgmap v249: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:26:00.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:25:59 smithi102 ceph-mon[100296]: scrub summary: idle 2023-11-27T15:26:00.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:59 smithi052 ceph-mon[99056]: pgmap v249: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:26:00.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:25:59 smithi052 ceph-mon[99056]: scrub summary: idle 2023-11-27T15:26:02.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:02 smithi150 ceph-mon[100559]: pgmap v250: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-27T15:26:02.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:02 smithi102 ceph-mon[100296]: pgmap v250: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-27T15:26:02.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:02 smithi052 ceph-mon[99056]: pgmap v250: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-27T15:26:04.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:04 smithi150 ceph-mon[100559]: pgmap v251: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-27T15:26:04.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:04 smithi102 ceph-mon[100296]: pgmap v251: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-27T15:26:04.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:04 smithi052 ceph-mon[99056]: pgmap v251: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-27T15:26:06.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:06 smithi150 ceph-mon[100559]: pgmap v252: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-27T15:26:06.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:06 smithi102 ceph-mon[100296]: pgmap v252: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-27T15:26:06.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:06 smithi052 ceph-mon[99056]: pgmap v252: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-27T15:26:08.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:08 smithi150 ceph-mon[100559]: pgmap v253: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-27T15:26:08.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:08 smithi102 ceph-mon[100296]: pgmap v253: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-27T15:26:08.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:08 smithi052 ceph-mon[99056]: pgmap v253: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-27T15:26:09.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:09 smithi150 ceph-mon[100559]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:26:09.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:09 smithi102 ceph-mon[100296]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:26:09.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:09 smithi052 ceph-mon[99056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:26:10.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:10 smithi150 ceph-mon[100559]: pgmap v254: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-27T15:26:10.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:10 smithi102 ceph-mon[100296]: pgmap v254: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-27T15:26:10.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:10 smithi052 ceph-mon[99056]: pgmap v254: 97 pgs: 97 active+clean; 712 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-27T15:26:11.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:11 smithi150 ceph-mon[100559]: osdmap e91: 12 total, 12 up, 12 in 2023-11-27T15:26:11.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:11 smithi102 ceph-mon[100296]: osdmap e91: 12 total, 12 up, 12 in 2023-11-27T15:26:11.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:11 smithi052 ceph-mon[99056]: osdmap e91: 12 total, 12 up, 12 in 2023-11-27T15:26:12.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:12 smithi150 ceph-mon[100559]: pgmap v256: 97 pgs: 97 active+clean; 712 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:26:12.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:12 smithi102 ceph-mon[100296]: pgmap v256: 97 pgs: 97 active+clean; 712 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:26:12.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:12 smithi052 ceph-mon[99056]: pgmap v256: 97 pgs: 97 active+clean; 712 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:26:14.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:14 smithi150 ceph-mon[100559]: pgmap v257: 97 pgs: 97 active+clean; 712 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:26:14.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:14 smithi150 ceph-mon[100559]: osdmap e92: 12 total, 12 up, 12 in 2023-11-27T15:26:14.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:14 smithi102 ceph-mon[100296]: pgmap v257: 97 pgs: 97 active+clean; 712 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:26:14.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:14 smithi102 ceph-mon[100296]: osdmap e92: 12 total, 12 up, 12 in 2023-11-27T15:26:14.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:14 smithi052 ceph-mon[99056]: pgmap v257: 97 pgs: 97 active+clean; 712 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:26:14.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:14 smithi052 ceph-mon[99056]: osdmap e92: 12 total, 12 up, 12 in 2023-11-27T15:26:16.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:16 smithi150 ceph-mon[100559]: pgmap v259: 97 pgs: 97 active+clean; 712 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2023-11-27T15:26:16.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:16 smithi150 ceph-mon[100559]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:26:16.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:16 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:26:16.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:16 smithi102 ceph-mon[100296]: pgmap v259: 97 pgs: 97 active+clean; 712 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2023-11-27T15:26:16.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:16 smithi102 ceph-mon[100296]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:26:16.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:16 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:26:16.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:16 smithi052 ceph-mon[99056]: pgmap v259: 97 pgs: 97 active+clean; 712 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2023-11-27T15:26:16.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:16 smithi052 ceph-mon[99056]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:26:16.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:16 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:26:17.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:17 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:26:17.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:17 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:26:17.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:17 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:26:17.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:17 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:26:17.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:17 smithi150 ceph-mon[100559]: Service mds.all has no related daemon already placed. 2023-11-27T15:26:17.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:17 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:26:17.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:17 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:26:17.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:17 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:26:17.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:17 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:26:17.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:17 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:26:17.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:17 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:26:17.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:17 smithi102 ceph-mon[100296]: Service mds.all has no related daemon already placed. 2023-11-27T15:26:17.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:17 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:26:17.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:17 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:26:17.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:17 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:26:17.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:17 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:26:17.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:17 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:26:17.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:17 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:26:17.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:17 smithi052 ceph-mon[99056]: Service mds.all has no related daemon already placed. 2023-11-27T15:26:17.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:17 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:26:17.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:17 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:26:18.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:18 smithi052 ceph-mon[99056]: pgmap v260: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-27T15:26:18.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:18 smithi150 ceph-mon[100559]: pgmap v260: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-27T15:26:18.827 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:18 smithi102 ceph-mon[100296]: pgmap v260: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-27T15:26:20.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:19 smithi150 ceph-mon[100559]: pgmap v261: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 229 B/s wr, 7 op/s 2023-11-27T15:26:20.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:19 smithi102 ceph-mon[100296]: pgmap v261: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 229 B/s wr, 7 op/s 2023-11-27T15:26:20.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:19 smithi052 ceph-mon[99056]: pgmap v261: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 229 B/s wr, 7 op/s 2023-11-27T15:26:22.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:22 smithi150 ceph-mon[100559]: pgmap v262: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-11-27T15:26:22.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:22 smithi102 ceph-mon[100296]: pgmap v262: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-11-27T15:26:22.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:22 smithi052 ceph-mon[99056]: pgmap v262: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-11-27T15:26:24.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:24 smithi150 ceph-mon[100559]: pgmap v263: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-11-27T15:26:24.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:24 smithi102 ceph-mon[100296]: pgmap v263: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-11-27T15:26:24.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:24 smithi052 ceph-mon[99056]: pgmap v263: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-11-27T15:26:26.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:26 smithi150 ceph-mon[100559]: pgmap v264: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 90 B/s wr, 5 op/s 2023-11-27T15:26:26.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:26 smithi102 ceph-mon[100296]: pgmap v264: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 90 B/s wr, 5 op/s 2023-11-27T15:26:26.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:26 smithi052 ceph-mon[99056]: pgmap v264: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 90 B/s wr, 5 op/s 2023-11-27T15:26:27.844 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:26:27.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.984+0000 7fe88156e700 1 -- 172.21.15.52:0/2126768706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe87c104510 msgr2=0x7fe87c10c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:27.989 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.984+0000 7fe88156e700 1 --2- 172.21.15.52:0/2126768706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe87c104510 0x7fe87c10c0e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fe87400b0d0 tx=0x7fe87400b3e0 comp rx=0 tx=0).stop 2023-11-27T15:26:27.989 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.985+0000 7fe88156e700 1 -- 172.21.15.52:0/2126768706 shutdown_connections 2023-11-27T15:26:27.989 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.985+0000 7fe88156e700 1 --2- 172.21.15.52:0/2126768706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe87c0ff4d0 0x7fe87c0ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:27.989 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.985+0000 7fe88156e700 1 --2- 172.21.15.52:0/2126768706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe87c104510 0x7fe87c10c0e0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:27.989 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.985+0000 7fe88156e700 1 --2- 172.21.15.52:0/2126768706 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe87c0ffe70 0x7fe87c103e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:27.990 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.985+0000 7fe88156e700 1 -- 172.21.15.52:0/2126768706 >> 172.21.15.52:0/2126768706 conn(0x7fe87c076070 msgr2=0x7fe87c076470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:27.990 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.985+0000 7fe88156e700 1 -- 172.21.15.52:0/2126768706 shutdown_connections 2023-11-27T15:26:27.990 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.985+0000 7fe88156e700 1 -- 172.21.15.52:0/2126768706 wait complete. 2023-11-27T15:26:27.990 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.985+0000 7fe88156e700 1 Processor -- start 2023-11-27T15:26:27.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.985+0000 7fe88156e700 1 -- start start 2023-11-27T15:26:27.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.986+0000 7fe88156e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe87c0ff4d0 0x7fe87c10c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:27.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.986+0000 7fe88156e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe87c0ffe70 0x7fe87c10cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:27.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.986+0000 7fe88156e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe87c104510 0x7fe87c1a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:27.992 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.986+0000 7fe88156e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c10d760 con 0x7fe87c0ffe70 2023-11-27T15:26:27.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.986+0000 7fe88156e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c1a4c10 con 0x7fe87c0ff4d0 2023-11-27T15:26:27.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.986+0000 7fe88156e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c1a4d50 con 0x7fe87c104510 2023-11-27T15:26:27.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.986+0000 7fe87a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe87c0ffe70 0x7fe87c10cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:27.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.986+0000 7fe87a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe87c0ffe70 0x7fe87c10cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55422/0 (socket says 172.21.15.52:55422) 2023-11-27T15:26:27.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.986+0000 7fe87a7fc700 1 -- 172.21.15.52:0/90638081 learned_addr learned my addr 172.21.15.52:0/90638081 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:27.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.986+0000 7fe87affd700 1 --2- 172.21.15.52:0/90638081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe87c0ff4d0 0x7fe87c10c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:27.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.986+0000 7fe87b7fe700 1 --2- 172.21.15.52:0/90638081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe87c104510 0x7fe87c1a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:27.993 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.986+0000 7fe87a7fc700 1 -- 172.21.15.52:0/90638081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe87c104510 msgr2=0x7fe87c1a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:27.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.986+0000 7fe87a7fc700 1 --2- 172.21.15.52:0/90638081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe87c104510 0x7fe87c1a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:27.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.986+0000 7fe87a7fc700 1 -- 172.21.15.52:0/90638081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe87c0ff4d0 msgr2=0x7fe87c10c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:27.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.986+0000 7fe87a7fc700 1 --2- 172.21.15.52:0/90638081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe87c0ff4d0 0x7fe87c10c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:27.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.986+0000 7fe87a7fc700 1 -- 172.21.15.52:0/90638081 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe874009d00 con 0x7fe87c0ffe70 2023-11-27T15:26:27.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.986+0000 7fe87a7fc700 1 --2- 172.21.15.52:0/90638081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe87c0ffe70 0x7fe87c10cf30 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fe87000c9b0 tx=0x7fe87000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:27.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.986+0000 7fe863fff700 1 -- 172.21.15.52:0/90638081 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe870007aa0 con 0x7fe87c0ffe70 2023-11-27T15:26:27.994 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.987+0000 7fe863fff700 1 -- 172.21.15.52:0/90638081 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe870007c00 con 0x7fe87c0ffe70 2023-11-27T15:26:27.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.987+0000 7fe88156e700 1 -- 172.21.15.52:0/90638081 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe87c1104a0 con 0x7fe87c0ffe70 2023-11-27T15:26:27.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.987+0000 7fe863fff700 1 -- 172.21.15.52:0/90638081 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe870018660 con 0x7fe87c0ffe70 2023-11-27T15:26:27.995 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.987+0000 7fe88156e700 1 -- 172.21.15.52:0/90638081 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe87c110310 con 0x7fe87c0ffe70 2023-11-27T15:26:28.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.989+0000 7fe863fff700 1 -- 172.21.15.52:0/90638081 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe87000f450 con 0x7fe87c0ffe70 2023-11-27T15:26:28.001 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.990+0000 7fe863fff700 1 --2- 172.21.15.52:0/90638081 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe8640a1340 0x7fe8640a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.002 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.990+0000 7fe863fff700 1 -- 172.21.15.52:0/90638081 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fe8700cccd0 con 0x7fe87c0ffe70 2023-11-27T15:26:28.002 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.990+0000 7fe87affd700 1 --2- 172.21.15.52:0/90638081 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe8640a1340 0x7fe8640a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.002 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.991+0000 7fe87affd700 1 --2- 172.21.15.52:0/90638081 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe8640a1340 0x7fe8640a37f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe86c005950 tx=0x7fe86c0058e0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:28.002 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.991+0000 7fe88d6fab80 1 Processor -- start 2023-11-27T15:26:28.002 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.991+0000 7fe88d6fab80 1 -- start start 2023-11-27T15:26:28.002 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.991+0000 7fe88d6fab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557554793b60 0x557554793f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.002 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.991+0000 7fe88d6fab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557554794470 0x557554798370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.003 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.991+0000 7fe88d6fab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575547988b0 0x557554798d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.003 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.991+0000 7fe88d6fab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x557554699060 con 0x557554794470 2023-11-27T15:26:28.003 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.991+0000 7fe88d6fab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575546eb8a0 con 0x5575547988b0 2023-11-27T15:26:28.003 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.991+0000 7fe88d6fab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575546e4d30 con 0x557554793b60 2023-11-27T15:26:28.003 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.991+0000 7fe87a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557554794470 0x557554798370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.003 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.991+0000 7fe87a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557554794470 0x557554798370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55430/0 (socket says 172.21.15.52:55430) 2023-11-27T15:26:28.003 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.991+0000 7fe87a7fc700 1 -- 172.21.15.52:0/1264365121 learned_addr learned my addr 172.21.15.52:0/1264365121 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:28.003 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.992+0000 7fe87b7fe700 1 --2- 172.21.15.52:0/1264365121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575547988b0 0x557554798d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.004 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.992+0000 7fe87a7fc700 1 -- 172.21.15.52:0/1264365121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557554793b60 msgr2=0x557554793f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.004 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.992+0000 7fe87affd700 1 --2- 172.21.15.52:0/1264365121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557554793b60 0x557554793f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.004 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.992+0000 7fe87a7fc700 1 --2- 172.21.15.52:0/1264365121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557554793b60 0x557554793f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.004 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.992+0000 7fe87a7fc700 1 -- 172.21.15.52:0/1264365121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575547988b0 msgr2=0x557554798d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.004 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.992+0000 7fe87a7fc700 1 --2- 172.21.15.52:0/1264365121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575547988b0 0x557554798d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.004 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.992+0000 7fe87a7fc700 1 -- 172.21.15.52:0/1264365121 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557554566630 con 0x557554794470 2023-11-27T15:26:28.004 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.992+0000 7fe87b7fe700 1 --2- 172.21.15.52:0/1264365121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575547988b0 0x557554798d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:26:28.004 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.992+0000 7fe87a7fc700 1 --2- 172.21.15.52:0/1264365121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557554794470 0x557554798370 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fe8700a1c30 tx=0x7fe870099760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecff42e9210a5796 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:28.005 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.992+0000 7fe8617fa700 1 -- 172.21.15.52:0/1264365121 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8700980b0 con 0x557554794470 2023-11-27T15:26:28.005 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.992+0000 7fe8617fa700 1 -- 172.21.15.52:0/1264365121 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe870007d70 con 0x557554794470 2023-11-27T15:26:28.005 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.992+0000 7fe8617fa700 1 -- 172.21.15.52:0/1264365121 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8700b2400 con 0x557554794470 2023-11-27T15:26:28.005 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.993+0000 7fe88d6fab80 1 -- 172.21.15.52:0/1264365121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557554794470 msgr2=0x557554798370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.005 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.993+0000 7fe88d6fab80 1 --2- 172.21.15.52:0/1264365121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557554794470 0x557554798370 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fe8700a1c30 tx=0x7fe870099760 comp rx=0 tx=0).stop 2023-11-27T15:26:28.005 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.993+0000 7fe88d6fab80 1 -- 172.21.15.52:0/1264365121 shutdown_connections 2023-11-27T15:26:28.005 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.993+0000 7fe88d6fab80 1 --2- 172.21.15.52:0/1264365121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575547988b0 0x557554798d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.005 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.993+0000 7fe88d6fab80 1 --2- 172.21.15.52:0/1264365121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557554793b60 0x557554793f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.005 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.993+0000 7fe88d6fab80 1 --2- 172.21.15.52:0/1264365121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557554794470 0x557554798370 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.006 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.993+0000 7fe88d6fab80 1 -- 172.21.15.52:0/1264365121 >> 172.21.15.52:0/1264365121 conn(0x557554789360 msgr2=0x55755478eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:28.006 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.993+0000 7fe88d6fab80 1 -- 172.21.15.52:0/1264365121 shutdown_connections 2023-11-27T15:26:28.006 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.993+0000 7fe88d6fab80 1 -- 172.21.15.52:0/1264365121 wait complete. 2023-11-27T15:26:28.006 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.993+0000 7fe88d6fab80 1 -- start start 2023-11-27T15:26:28.006 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.993+0000 7fe88d6fab80 1 Processor -- start 2023-11-27T15:26:28.006 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.994+0000 7fe88d6fab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557554793b60 0x5575547a00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.006 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.994+0000 7fe88d6fab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557554794470 0x55755479d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.007 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.994+0000 7fe88d6fab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575547988b0 0x55755479da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.007 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.994+0000 7fe88d6fab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575546868e0 con 0x557554793b60 2023-11-27T15:26:28.007 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.994+0000 7fe88d6fab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557554686580 con 0x5575547988b0 2023-11-27T15:26:28.007 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.994+0000 7fe88d6fab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557554568920 con 0x557554794470 2023-11-27T15:26:28.007 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.994+0000 7fe88d6fab80 10 client.0 _ll_register_callbacks cb 0x5575546a4780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:28.007 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.994+0000 7fe87affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557554793b60 0x5575547a00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.007 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.994+0000 7fe87affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557554793b60 0x5575547a00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55434/0 (socket says 172.21.15.52:55434) 2023-11-27T15:26:28.007 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.994+0000 7fe87affd700 1 -- 172.21.15.52:0/3918984302 learned_addr learned my addr 172.21.15.52:0/3918984302 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:28.008 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.994+0000 7fe87a7fc700 1 --2- 172.21.15.52:0/3918984302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557554794470 0x55755479d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.008 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.994+0000 7fe87b7fe700 1 --2- 172.21.15.52:0/3918984302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575547988b0 0x55755479da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.008 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.994+0000 7fe87affd700 1 -- 172.21.15.52:0/3918984302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557554794470 msgr2=0x55755479d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.008 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.994+0000 7fe87affd700 1 --2- 172.21.15.52:0/3918984302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557554794470 0x55755479d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.008 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.994+0000 7fe87affd700 1 -- 172.21.15.52:0/3918984302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575547988b0 msgr2=0x55755479da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.008 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.994+0000 7fe87affd700 1 --2- 172.21.15.52:0/3918984302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575547988b0 0x55755479da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.008 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.994+0000 7fe87affd700 1 -- 172.21.15.52:0/3918984302 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe87c110310 con 0x557554793b60 2023-11-27T15:26:28.008 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.994+0000 7fe87affd700 1 --2- 172.21.15.52:0/3918984302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557554793b60 0x5575547a00d0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fe86c00fe50 tx=0x7fe86c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=643b83ac6d96c2c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:28.009 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.994+0000 7fe85eff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:26:28.009 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.994+0000 7fe85eff5700 1 -- 172.21.15.52:0/3918984302 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c01e620 con 0x557554793b60 2023-11-27T15:26:28.009 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.994+0000 7fe85eff5700 1 -- 172.21.15.52:0/3918984302 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fe86c00fe80 con 0x557554793b60 2023-11-27T15:26:28.009 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.994+0000 7fe88d6fab80 1 -- 172.21.15.52:0/3918984302 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557554786b20 con 0x557554793b60 2023-11-27T15:26:28.009 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.995+0000 7fe85eff5700 1 -- 172.21.15.52:0/3918984302 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe86c011e70 con 0x557554793b60 2023-11-27T15:26:28.009 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.995+0000 7fe88d6fab80 10 client.16851 fetch_fsmap learned FSMap version 34 2023-11-27T15:26:28.009 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.995+0000 7fe88d6fab80 1 -- 172.21.15.52:0/3918984302 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5575546ab190 con 0x557554793b60 2023-11-27T15:26:28.009 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.995+0000 7fe85eff5700 1 -- 172.21.15.52:0/3918984302 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557554786b20 con 0x557554793b60 2023-11-27T15:26:28.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.995+0000 7fe85eff5700 1 -- 172.21.15.52:0/3918984302 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe86c024020 con 0x557554793b60 2023-11-27T15:26:28.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.995+0000 7fe88d6fab80 10 client.16851 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:26:28.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.995+0000 7fe88d6fab80 10 client.16851 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:26:28.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.995+0000 7fe88d6fab80 20 client.16851 populate_metadata read hostname 'smithi052' 2023-11-27T15:26:28.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.995+0000 7fe88d6fab80 1 --2- 172.21.15.52:0/3918984302 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5575547ad8e0 0x5575547afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.995+0000 7fe88d6fab80 4 client.16851 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:26:28.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.995+0000 7fe88d6fab80 1 -- 172.21.15.52:0/3918984302 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5575546a70a0 con 0x5575547ad8e0 2023-11-27T15:26:28.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.995+0000 7fe87a7fc700 1 --2- 172.21.15.52:0/3918984302 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5575547ad8e0 0x5575547afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.010 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.996+0000 7fe85eff5700 10 client.16851 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:26:28.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.996+0000 7fe87a7fc700 1 --2- 172.21.15.52:0/3918984302 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5575547ad8e0 0x5575547afcf0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1a1daf50bd86366 server_cookie=f470aafb5013ec12 in_seq=0 out_seq=0 2023-11-27T15:26:28.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.996+0000 7fe85eff5700 1 -- 172.21.15.52:0/3918984302 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5575546a70a0 con 0x5575547ad8e0 2023-11-27T15:26:28.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.996+0000 7fe85eff5700 10 client.16851 handle_command_reply: tid=0 2023-11-27T15:26:28.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.996+0000 7fe88d6fab80 1 client.16851 shutdown 2023-11-27T15:26:28.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.997+0000 7fe88d6fab80 1 -- 172.21.15.52:0/3918984302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557554793b60 msgr2=0x5575547a00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.997+0000 7fe88d6fab80 1 --2- 172.21.15.52:0/3918984302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557554793b60 0x5575547a00d0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fe86c00fe50 tx=0x7fe86c004750 comp rx=0 tx=0).stop 2023-11-27T15:26:28.011 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.997+0000 7fe88d6fab80 1 -- 172.21.15.52:0/3918984302 shutdown_connections 2023-11-27T15:26:28.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.997+0000 7fe88d6fab80 1 --2- 172.21.15.52:0/3918984302 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5575547ad8e0 0x5575547afcf0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.997+0000 7fe88d6fab80 1 --2- 172.21.15.52:0/3918984302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575547988b0 0x55755479da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.997+0000 7fe88d6fab80 1 --2- 172.21.15.52:0/3918984302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557554793b60 0x5575547a00d0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.997+0000 7fe85eff5700 0 client.16851 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:26:28.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.997+0000 7fe88d6fab80 1 --2- 172.21.15.52:0/3918984302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557554794470 0x55755479d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.997+0000 7fe88d6fab80 1 -- 172.21.15.52:0/3918984302 >> 172.21.15.52:0/3918984302 conn(0x557554789360 msgr2=0x557554799d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:28.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.997+0000 7fe88d6fab80 1 -- 172.21.15.52:0/3918984302 shutdown_connections 2023-11-27T15:26:28.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.997+0000 7fe88d6fab80 1 -- 172.21.15.52:0/3918984302 wait complete. 2023-11-27T15:26:28.012 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:27.997+0000 7fe88d6fab80 20 client.16851 trim_cache size 0 max 16384 2023-11-27T15:26:28.022 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.014+0000 7fe88d6fab80 1 Processor -- start 2023-11-27T15:26:28.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.014+0000 7fe88d6fab80 1 -- start start 2023-11-27T15:26:28.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.014+0000 7fe88d6fab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557554793b60 0x55755478ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.014+0000 7fe88d6fab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557554794470 0x55755479fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.014+0000 7fe88d6fab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575547988b0 0x55755479cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.014+0000 7fe88d6fab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x557554794840 con 0x557554793b60 2023-11-27T15:26:28.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.014+0000 7fe88d6fab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575547ab080 con 0x5575547988b0 2023-11-27T15:26:28.023 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.014+0000 7fe88d6fab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55755479e950 con 0x557554794470 2023-11-27T15:26:28.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.014+0000 7fe87affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557554793b60 0x55755478ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.014+0000 7fe87affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557554793b60 0x55755478ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55442/0 (socket says 172.21.15.52:55442) 2023-11-27T15:26:28.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.014+0000 7fe87affd700 1 -- 172.21.15.52:0/311696142 learned_addr learned my addr 172.21.15.52:0/311696142 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:28.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.014+0000 7fe87b7fe700 1 --2- 172.21.15.52:0/311696142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575547988b0 0x55755479cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.014+0000 7fe87a7fc700 1 --2- 172.21.15.52:0/311696142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557554794470 0x55755479fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.014+0000 7fe87affd700 1 -- 172.21.15.52:0/311696142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557554794470 msgr2=0x55755479fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.024 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.014+0000 7fe87affd700 1 --2- 172.21.15.52:0/311696142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557554794470 0x55755479fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.014+0000 7fe87affd700 1 -- 172.21.15.52:0/311696142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575547988b0 msgr2=0x55755479cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.014+0000 7fe87affd700 1 --2- 172.21.15.52:0/311696142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575547988b0 0x55755479cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.014+0000 7fe87affd700 1 -- 172.21.15.52:0/311696142 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575546aaee0 con 0x557554793b60 2023-11-27T15:26:28.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.014+0000 7fe87b7fe700 1 --2- 172.21.15.52:0/311696142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575547988b0 0x55755479cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:26:28.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.014+0000 7fe87a7fc700 1 --2- 172.21.15.52:0/311696142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557554794470 0x55755479fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:26:28.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.015+0000 7fe87affd700 1 --2- 172.21.15.52:0/311696142 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557554793b60 0x55755478ebe0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe86c0269c0 tx=0x7fe86c00b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d1a067b1ae53a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:28.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.015+0000 7fe861ffb700 1 -- 172.21.15.52:0/311696142 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c011ab0 con 0x557554793b60 2023-11-27T15:26:28.025 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.015+0000 7fe861ffb700 1 -- 172.21.15.52:0/311696142 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe86c00fe80 con 0x557554793b60 2023-11-27T15:26:28.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.015+0000 7fe861ffb700 1 -- 172.21.15.52:0/311696142 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c036b90 con 0x557554793b60 2023-11-27T15:26:28.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.015+0000 7fe88d6fab80 1 -- 172.21.15.52:0/311696142 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557554793b60 msgr2=0x55755478ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.015+0000 7fe88d6fab80 1 --2- 172.21.15.52:0/311696142 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557554793b60 0x55755478ebe0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe86c0269c0 tx=0x7fe86c00b4e0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.015+0000 7fe88d6fab80 1 -- 172.21.15.52:0/311696142 shutdown_connections 2023-11-27T15:26:28.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.015+0000 7fe88d6fab80 1 --2- 172.21.15.52:0/311696142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575547988b0 0x55755479cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.015+0000 7fe88d6fab80 1 --2- 172.21.15.52:0/311696142 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557554793b60 0x55755478ebe0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.026 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.015+0000 7fe88d6fab80 1 --2- 172.21.15.52:0/311696142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557554794470 0x55755479fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.015+0000 7fe88d6fab80 1 -- 172.21.15.52:0/311696142 >> 172.21.15.52:0/311696142 conn(0x557554789360 msgr2=0x557554796970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:28.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.015+0000 7fe88d6fab80 1 -- 172.21.15.52:0/311696142 shutdown_connections 2023-11-27T15:26:28.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.015+0000 7fe88d6fab80 1 -- 172.21.15.52:0/311696142 wait complete. 2023-11-27T15:26:28.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.016+0000 7fe88d6fab80 1 -- start start 2023-11-27T15:26:28.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.016+0000 7fe88d6fab80 1 Processor -- start 2023-11-27T15:26:28.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.016+0000 7fe88d6fab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557554793b60 0x5575547a4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.016+0000 7fe88d6fab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557554794470 0x5575547a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.027 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.016+0000 7fe88d6fab80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575547988b0 0x5575547a3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.028 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.016+0000 7fe88d6fab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575547a3570 con 0x557554794470 2023-11-27T15:26:28.028 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.016+0000 7fe88d6fab80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575547a36b0 con 0x5575547988b0 2023-11-27T15:26:28.028 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.016+0000 7fe88d6fab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575547a3870 con 0x557554793b60 2023-11-27T15:26:28.028 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.016+0000 7fe88d6fab80 10 client.0 _ll_register_callbacks cb 0x7fe86c00fb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:28.028 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.016+0000 7fe87a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557554794470 0x5575547a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.028 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.016+0000 7fe87a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557554794470 0x5575547a2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55450/0 (socket says 172.21.15.52:55450) 2023-11-27T15:26:28.028 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.016+0000 7fe87a7fc700 1 -- 172.21.15.52:0/2677494657 learned_addr learned my addr 172.21.15.52:0/2677494657 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:28.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.016+0000 7fe87b7fe700 1 --2- 172.21.15.52:0/2677494657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575547988b0 0x5575547a3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.016+0000 7fe87affd700 1 --2- 172.21.15.52:0/2677494657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557554793b60 0x5575547a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.016+0000 7fe87a7fc700 1 -- 172.21.15.52:0/2677494657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557554793b60 msgr2=0x5575547a4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.016+0000 7fe87a7fc700 1 --2- 172.21.15.52:0/2677494657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557554793b60 0x5575547a4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.016+0000 7fe87a7fc700 1 -- 172.21.15.52:0/2677494657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575547988b0 msgr2=0x5575547a3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.016+0000 7fe87a7fc700 1 --2- 172.21.15.52:0/2677494657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575547988b0 0x5575547a3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.016+0000 7fe87a7fc700 1 -- 172.21.15.52:0/2677494657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5575546ab190 con 0x557554794470 2023-11-27T15:26:28.029 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.016+0000 7fe87affd700 1 --2- 172.21.15.52:0/2677494657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557554793b60 0x5575547a4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:26:28.030 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.017+0000 7fe87a7fc700 1 --2- 172.21.15.52:0/2677494657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557554794470 0x5575547a2af0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fe870096780 tx=0x7fe8700a22d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=718b0f57fb0e3426 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:28.030 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.017+0000 7fe85eff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:26:28.030 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.017+0000 7fe85eff5700 1 -- 172.21.15.52:0/2677494657 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe870098130 con 0x557554794470 2023-11-27T15:26:28.030 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.017+0000 7fe85eff5700 1 -- 172.21.15.52:0/2677494657 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fe8700b8030 con 0x557554794470 2023-11-27T15:26:28.030 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.017+0000 7fe88d6fab80 1 -- 172.21.15.52:0/2677494657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe870007d70 con 0x557554794470 2023-11-27T15:26:28.030 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.017+0000 7fe85eff5700 1 -- 172.21.15.52:0/2677494657 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe870078be0 con 0x557554794470 2023-11-27T15:26:28.030 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.017+0000 7fe88d6fab80 10 client.16857 fetch_fsmap learned FSMap version 34 2023-11-27T15:26:28.030 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.017+0000 7fe88d6fab80 1 -- 172.21.15.52:0/2677494657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5575544fbc60 con 0x557554794470 2023-11-27T15:26:28.031 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.017+0000 7fe85eff5700 1 -- 172.21.15.52:0/2677494657 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe870007d70 con 0x557554794470 2023-11-27T15:26:28.031 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.017+0000 7fe85eff5700 1 -- 172.21.15.52:0/2677494657 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe870099320 con 0x557554794470 2023-11-27T15:26:28.031 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.017+0000 7fe88d6fab80 10 client.16857 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:26:28.031 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.017+0000 7fe88d6fab80 10 client.16857 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:26:28.031 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.017+0000 7fe88d6fab80 20 client.16857 populate_metadata read hostname 'smithi052' 2023-11-27T15:26:28.031 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.017+0000 7fe88d6fab80 1 --2- 172.21.15.52:0/2677494657 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557554802720 0x557554804b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.031 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.017+0000 7fe88d6fab80 4 client.16857 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:26:28.031 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.017+0000 7fe88d6fab80 1 -- 172.21.15.52:0/2677494657 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5575546b3660 con 0x557554802720 2023-11-27T15:26:28.032 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.018+0000 7fe87affd700 1 --2- 172.21.15.52:0/2677494657 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557554802720 0x557554804b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.032 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.018+0000 7fe85eff5700 10 client.16857 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:26:28.032 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.018+0000 7fe87affd700 1 --2- 172.21.15.52:0/2677494657 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557554802720 0x557554804b30 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f77545b60a961893 server_cookie=3ccfe343f5643f55 in_seq=0 out_seq=0 2023-11-27T15:26:28.032 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.018+0000 7fe85eff5700 1 -- 172.21.15.52:0/2677494657 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5575546b3660 con 0x557554802720 2023-11-27T15:26:28.032 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.018+0000 7fe85eff5700 10 client.16857 handle_command_reply: tid=0 2023-11-27T15:26:28.032 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.018+0000 7fe88d6fab80 1 client.16857 shutdown 2023-11-27T15:26:28.032 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.018+0000 7fe88d6fab80 1 -- 172.21.15.52:0/2677494657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557554794470 msgr2=0x5575547a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.032 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.018+0000 7fe88d6fab80 1 --2- 172.21.15.52:0/2677494657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557554794470 0x5575547a2af0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fe870096780 tx=0x7fe8700a22d0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.033 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.018+0000 7fe88d6fab80 1 -- 172.21.15.52:0/2677494657 shutdown_connections 2023-11-27T15:26:28.033 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.018+0000 7fe88d6fab80 1 --2- 172.21.15.52:0/2677494657 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557554802720 0x557554804b30 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.033 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.018+0000 7fe88d6fab80 1 --2- 172.21.15.52:0/2677494657 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575547988b0 0x5575547a3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.033 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.018+0000 7fe88d6fab80 1 --2- 172.21.15.52:0/2677494657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557554793b60 0x5575547a4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.033 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.018+0000 7fe85eff5700 0 client.16857 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:26:28.033 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.018+0000 7fe88d6fab80 1 --2- 172.21.15.52:0/2677494657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557554794470 0x5575547a2af0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.033 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.018+0000 7fe88d6fab80 1 -- 172.21.15.52:0/2677494657 >> 172.21.15.52:0/2677494657 conn(0x557554789360 msgr2=0x557554792960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:28.033 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.018+0000 7fe88d6fab80 1 -- 172.21.15.52:0/2677494657 shutdown_connections 2023-11-27T15:26:28.034 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.018+0000 7fe88d6fab80 1 -- 172.21.15.52:0/2677494657 wait complete. 2023-11-27T15:26:28.034 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.019+0000 7fe88d6fab80 20 client.16857 trim_cache size 0 max 16384 2023-11-27T15:26:28.034 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:26:28.034 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2023-11-27T15:26:28.034 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2023-11-27T15:26:28.035 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:26:28.036 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.019+0000 7fe88156e700 1 -- 172.21.15.52:0/90638081 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe8640a1340 msgr2=0x7fe8640a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.036 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.019+0000 7fe88156e700 1 --2- 172.21.15.52:0/90638081 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe8640a1340 0x7fe8640a37f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe86c005950 tx=0x7fe86c0058e0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.036 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.019+0000 7fe88156e700 1 -- 172.21.15.52:0/90638081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe87c0ffe70 msgr2=0x7fe87c10cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.036 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.019+0000 7fe88156e700 1 --2- 172.21.15.52:0/90638081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe87c0ffe70 0x7fe87c10cf30 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fe87000c9b0 tx=0x7fe87000cd70 comp rx=0 tx=0).stop 2023-11-27T15:26:28.036 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.019+0000 7fe88156e700 1 -- 172.21.15.52:0/90638081 shutdown_connections 2023-11-27T15:26:28.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.019+0000 7fe88156e700 1 --2- 172.21.15.52:0/90638081 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe8640a1340 0x7fe8640a37f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.019+0000 7fe88156e700 1 --2- 172.21.15.52:0/90638081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe87c0ff4d0 0x7fe87c10c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.019+0000 7fe88156e700 1 --2- 172.21.15.52:0/90638081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe87c104510 0x7fe87c1a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.019+0000 7fe88156e700 1 --2- 172.21.15.52:0/90638081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe87c0ffe70 0x7fe87c10cf30 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.019+0000 7fe88156e700 1 -- 172.21.15.52:0/90638081 >> 172.21.15.52:0/90638081 conn(0x7fe87c076070 msgr2=0x7fe87c0fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:28.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.019+0000 7fe88156e700 1 -- 172.21.15.52:0/90638081 shutdown_connections 2023-11-27T15:26:28.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.019+0000 7fe88156e700 1 -- 172.21.15.52:0/90638081 wait complete. 2023-11-27T15:26:28.038 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-27T15:26:28.038 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:26:28.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.160+0000 7f756e24d700 1 -- 172.21.15.52:0/4031010717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7568104460 msgr2=0x7f756810c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.160+0000 7f756e24d700 1 --2- 172.21.15.52:0/4031010717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7568104460 0x7f756810c030 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f75580099c0 tx=0x7f7558009cd0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.160+0000 7f756e24d700 1 -- 172.21.15.52:0/4031010717 shutdown_connections 2023-11-27T15:26:28.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.160+0000 7f756e24d700 1 --2- 172.21.15.52:0/4031010717 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75680ff420 0x7f75680ff7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.160+0000 7f756e24d700 1 --2- 172.21.15.52:0/4031010717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7568104460 0x7f756810c030 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.160+0000 7f756e24d700 1 --2- 172.21.15.52:0/4031010717 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75680ffdc0 0x7f7568103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.160+0000 7f756e24d700 1 -- 172.21.15.52:0/4031010717 >> 172.21.15.52:0/4031010717 conn(0x7f7568076310 msgr2=0x7f7568076710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:28.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.160+0000 7f756e24d700 1 -- 172.21.15.52:0/4031010717 shutdown_connections 2023-11-27T15:26:28.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.160+0000 7f756e24d700 1 -- 172.21.15.52:0/4031010717 wait complete. 2023-11-27T15:26:28.165 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.161+0000 7f756e24d700 1 Processor -- start 2023-11-27T15:26:28.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.161+0000 7f756e24d700 1 -- start start 2023-11-27T15:26:28.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.161+0000 7f756e24d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75680ff420 0x7f7568196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.161+0000 7f756e24d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75680ffdc0 0x7f75681968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.161+0000 7f756e24d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7568104460 0x7f756819ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.161+0000 7f756e24d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756819b370 con 0x7f75680ff420 2023-11-27T15:26:28.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.161+0000 7f756e24d700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756819b4e0 con 0x7f75680ffdc0 2023-11-27T15:26:28.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.161+0000 7f756e24d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756819b6a0 con 0x7f7568104460 2023-11-27T15:26:28.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.161+0000 7f75677fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75680ff420 0x7f7568196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.161+0000 7f75677fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75680ff420 0x7f7568196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55462/0 (socket says 172.21.15.52:55462) 2023-11-27T15:26:28.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.161+0000 7f75677fe700 1 -- 172.21.15.52:0/3319095591 learned_addr learned my addr 172.21.15.52:0/3319095591 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:28.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.161+0000 7f7566ffd700 1 --2- 172.21.15.52:0/3319095591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75680ffdc0 0x7f75681968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.161+0000 7f7567fff700 1 --2- 172.21.15.52:0/3319095591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7568104460 0x7f756819ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.161+0000 7f75677fe700 1 -- 172.21.15.52:0/3319095591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7568104460 msgr2=0x7f756819ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.161+0000 7f75677fe700 1 --2- 172.21.15.52:0/3319095591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7568104460 0x7f756819ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.161+0000 7f75677fe700 1 -- 172.21.15.52:0/3319095591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75680ffdc0 msgr2=0x7f75681968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.161+0000 7f75677fe700 1 --2- 172.21.15.52:0/3319095591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75680ffdc0 0x7f75681968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.161+0000 7f75677fe700 1 -- 172.21.15.52:0/3319095591 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75580095f0 con 0x7f75680ff420 2023-11-27T15:26:28.169 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.161+0000 7f7567fff700 1 --2- 172.21.15.52:0/3319095591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7568104460 0x7f756819ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:26:28.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.162+0000 7f75677fe700 1 --2- 172.21.15.52:0/3319095591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75680ff420 0x7f7568196390 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f75600060b0 tx=0x7f756000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:28.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.162+0000 7f7564ff9700 1 -- 172.21.15.52:0/3319095591 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756000ddd0 con 0x7f75680ff420 2023-11-27T15:26:28.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.162+0000 7f756e24d700 1 -- 172.21.15.52:0/3319095591 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7568110390 con 0x7f75680ff420 2023-11-27T15:26:28.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.162+0000 7f7564ff9700 1 -- 172.21.15.52:0/3319095591 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f756000f040 con 0x7f75680ff420 2023-11-27T15:26:28.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.162+0000 7f7564ff9700 1 -- 172.21.15.52:0/3319095591 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7560014690 con 0x7f75680ff420 2023-11-27T15:26:28.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.162+0000 7f756e24d700 1 -- 172.21.15.52:0/3319095591 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7568110200 con 0x7f75680ff420 2023-11-27T15:26:28.170 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.163+0000 7f756e24d700 1 -- 172.21.15.52:0/3319095591 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7568110520 con 0x7f75680ff420 2023-11-27T15:26:28.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.164+0000 7f7564ff9700 1 -- 172.21.15.52:0/3319095591 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7560004ad0 con 0x7f75680ff420 2023-11-27T15:26:28.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.164+0000 7f7564ff9700 1 --2- 172.21.15.52:0/3319095591 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f75540a12c0 0x7f75540a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.164+0000 7f7564ff9700 1 -- 172.21.15.52:0/3319095591 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f75600ccb90 con 0x7f75680ff420 2023-11-27T15:26:28.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.164+0000 7f7566ffd700 1 --2- 172.21.15.52:0/3319095591 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f75540a12c0 0x7f75540a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.165+0000 7f7566ffd700 1 --2- 172.21.15.52:0/3319095591 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f75540a12c0 0x7f75540a3770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f75681979a0 tx=0x7f755000b410 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:28.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.168+0000 7f7564ff9700 1 -- 172.21.15.52:0/3319095591 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7560095f70 con 0x7f75680ff420 2023-11-27T15:26:28.417 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.413+0000 7f756e24d700 1 -- 172.21.15.52:0/3319095591 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7568197790 con 0x7f75680ff420 2023-11-27T15:26:28.417 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.414+0000 7f7564ff9700 1 -- 172.21.15.52:0/3319095591 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10804 (secure 0 0 0) 0x7f756001a090 con 0x7f75680ff420 2023-11-27T15:26:28.417 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:26:28.418 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2328881376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2328881376},{"type":"v1","addr":"172.21.15.52:6839","nonce":2328881376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2799551570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2799551570},{"type":"v1","addr":"172.21.15.52:6841","nonce":2799551570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:16:09.319630+0000","modified":"2023-11-27T15:26:15.243744+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24494,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/215725838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":215725838},{"type":"v1","addr":"172.21.15.150:6839","nonce":215725838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3571830559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3571830559},{"type":"v1","addr":"172.21.15.102:6839","nonce":3571830559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:26:28.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.416+0000 7f756e24d700 1 -- 172.21.15.52:0/3319095591 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f75540a12c0 msgr2=0x7f75540a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.416+0000 7f756e24d700 1 --2- 172.21.15.52:0/3319095591 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f75540a12c0 0x7f75540a3770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f75681979a0 tx=0x7f755000b410 comp rx=0 tx=0).stop 2023-11-27T15:26:28.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.416+0000 7f756e24d700 1 -- 172.21.15.52:0/3319095591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75680ff420 msgr2=0x7f7568196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.416+0000 7f756e24d700 1 --2- 172.21.15.52:0/3319095591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75680ff420 0x7f7568196390 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f75600060b0 tx=0x7f756000d3b0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.416+0000 7f756e24d700 1 -- 172.21.15.52:0/3319095591 shutdown_connections 2023-11-27T15:26:28.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.416+0000 7f756e24d700 1 --2- 172.21.15.52:0/3319095591 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f75540a12c0 0x7f75540a3770 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.420 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.416+0000 7f756e24d700 1 --2- 172.21.15.52:0/3319095591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7568104460 0x7f756819ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.416+0000 7f756e24d700 1 --2- 172.21.15.52:0/3319095591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75680ff420 0x7f7568196390 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.416+0000 7f756e24d700 1 --2- 172.21.15.52:0/3319095591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75680ffdc0 0x7f75681968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.416+0000 7f756e24d700 1 -- 172.21.15.52:0/3319095591 >> 172.21.15.52:0/3319095591 conn(0x7f7568076310 msgr2=0x7f75680fca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:28.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.416+0000 7f756e24d700 1 -- 172.21.15.52:0/3319095591 shutdown_connections 2023-11-27T15:26:28.421 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.416+0000 7f756e24d700 1 -- 172.21.15.52:0/3319095591 wait complete. 2023-11-27T15:26:28.421 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 34 2023-11-27T15:26:28.432 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-27T15:26:28.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:28 smithi150 ceph-mon[100559]: pgmap v265: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-11-27T15:26:28.559 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.555+0000 7f4359bb1700 1 -- 172.21.15.52:0/2427185542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43540ffdf0 msgr2=0x7f4354103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.559 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.555+0000 7f4359bb1700 1 --2- 172.21.15.52:0/2427185542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43540ffdf0 0x7f4354103db0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f43480099c0 tx=0x7f4348009cd0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.559 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.555+0000 7f4359bb1700 1 -- 172.21.15.52:0/2427185542 shutdown_connections 2023-11-27T15:26:28.559 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.555+0000 7f4359bb1700 1 --2- 172.21.15.52:0/2427185542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43540ff450 0x7f43540ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.559 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.555+0000 7f4359bb1700 1 --2- 172.21.15.52:0/2427185542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4354104490 0x7f435410c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.560 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.555+0000 7f4359bb1700 1 --2- 172.21.15.52:0/2427185542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43540ffdf0 0x7f4354103db0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.560 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.555+0000 7f4359bb1700 1 -- 172.21.15.52:0/2427185542 >> 172.21.15.52:0/2427185542 conn(0x7f43540770b0 msgr2=0x7f43540774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:28.560 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.555+0000 7f4359bb1700 1 -- 172.21.15.52:0/2427185542 shutdown_connections 2023-11-27T15:26:28.560 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.555+0000 7f4359bb1700 1 -- 172.21.15.52:0/2427185542 wait complete. 2023-11-27T15:26:28.562 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.556+0000 7f4359bb1700 1 Processor -- start 2023-11-27T15:26:28.562 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.556+0000 7f4359bb1700 1 -- start start 2023-11-27T15:26:28.562 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.556+0000 7f4359bb1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43540ff450 0x7f43541964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.562 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.556+0000 7f4359bb1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43540ffdf0 0x7f43541969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.563 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.556+0000 7f4359bb1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4354104490 0x7f435419ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.563 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.556+0000 7f4359bb1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435419b490 con 0x7f43540ffdf0 2023-11-27T15:26:28.563 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.556+0000 7f4359bb1700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435419b600 con 0x7f43540ff450 2023-11-27T15:26:28.563 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.556+0000 7f4359bb1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435419b7c0 con 0x7f4354104490 2023-11-27T15:26:28.563 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.556+0000 7f4352ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43540ffdf0 0x7f43541969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.563 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.556+0000 7f4352ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43540ffdf0 0x7f43541969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55478/0 (socket says 172.21.15.52:55478) 2023-11-27T15:26:28.563 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.556+0000 7f4352ffd700 1 -- 172.21.15.52:0/431212789 learned_addr learned my addr 172.21.15.52:0/431212789 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:28.563 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.556+0000 7f43537fe700 1 --2- 172.21.15.52:0/431212789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43540ff450 0x7f43541964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.564 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.556+0000 7f4353fff700 1 --2- 172.21.15.52:0/431212789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4354104490 0x7f435419ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.564 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.557+0000 7f4352ffd700 1 -- 172.21.15.52:0/431212789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4354104490 msgr2=0x7f435419ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.564 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.557+0000 7f4352ffd700 1 --2- 172.21.15.52:0/431212789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4354104490 0x7f435419ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.564 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.557+0000 7f4352ffd700 1 -- 172.21.15.52:0/431212789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43540ff450 msgr2=0x7f43541964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.564 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.557+0000 7f4352ffd700 1 --2- 172.21.15.52:0/431212789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43540ff450 0x7f43541964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.564 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.557+0000 7f4352ffd700 1 -- 172.21.15.52:0/431212789 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43480095f0 con 0x7f43540ffdf0 2023-11-27T15:26:28.564 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.557+0000 7f4352ffd700 1 --2- 172.21.15.52:0/431212789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43540ffdf0 0x7f43541969f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f4348009990 tx=0x7f4348004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:28.564 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.557+0000 7f4350ff9700 1 -- 172.21.15.52:0/431212789 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434801d070 con 0x7f43540ffdf0 2023-11-27T15:26:28.565 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.557+0000 7f4350ff9700 1 -- 172.21.15.52:0/431212789 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4348003f90 con 0x7f43540ffdf0 2023-11-27T15:26:28.565 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.557+0000 7f4359bb1700 1 -- 172.21.15.52:0/431212789 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f435410f210 con 0x7f43540ffdf0 2023-11-27T15:26:28.565 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.557+0000 7f4350ff9700 1 -- 172.21.15.52:0/431212789 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4348022b70 con 0x7f43540ffdf0 2023-11-27T15:26:28.565 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.557+0000 7f4359bb1700 1 -- 172.21.15.52:0/431212789 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f435410f080 con 0x7f43540ffdf0 2023-11-27T15:26:28.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.559+0000 7f4350ff9700 1 -- 172.21.15.52:0/431212789 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f4348003ad0 con 0x7f43540ffdf0 2023-11-27T15:26:28.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.560+0000 7f4350ff9700 1 --2- 172.21.15.52:0/431212789 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f433c0a1340 0x7f433c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.560+0000 7f43537fe700 1 --2- 172.21.15.52:0/431212789 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f433c0a1340 0x7f433c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.560+0000 7f4350ff9700 1 -- 172.21.15.52:0/431212789 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f43480ce230 con 0x7f43540ffdf0 2023-11-27T15:26:28.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.560+0000 7f43537fe700 1 --2- 172.21.15.52:0/431212789 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f433c0a1340 0x7f433c0a37f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4344005950 tx=0x7f434400b410 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:28.570 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.561+0000 7f4365d3db80 1 Processor -- start 2023-11-27T15:26:28.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.561+0000 7f4365d3db80 1 -- start start 2023-11-27T15:26:28.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.561+0000 7f4365d3db80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caae7a8b60 0x55caae7a8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.561+0000 7f4365d3db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55caae7a9470 0x55caae7ad370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.561+0000 7f4365d3db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55caae7ad8b0 0x55caae7add00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.562+0000 7f4365d3db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caae6ae060 con 0x55caae7ad8b0 2023-11-27T15:26:28.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.562+0000 7f4365d3db80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caae7008a0 con 0x55caae7a8b60 2023-11-27T15:26:28.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.562+0000 7f4365d3db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caae6f9d30 con 0x55caae7a9470 2023-11-27T15:26:28.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.562+0000 7f4353fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55caae7ad8b0 0x55caae7add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.562+0000 7f4353fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55caae7ad8b0 0x55caae7add00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55488/0 (socket says 172.21.15.52:55488) 2023-11-27T15:26:28.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.562+0000 7f4353fff700 1 -- 172.21.15.52:0/95705322 learned_addr learned my addr 172.21.15.52:0/95705322 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:28.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.562+0000 7f43537fe700 1 --2- 172.21.15.52:0/95705322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caae7a8b60 0x55caae7a8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.562+0000 7f4352ffd700 1 --2- 172.21.15.52:0/95705322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55caae7a9470 0x55caae7ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.562+0000 7f4353fff700 1 -- 172.21.15.52:0/95705322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55caae7a9470 msgr2=0x55caae7ad370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.562+0000 7f4353fff700 1 --2- 172.21.15.52:0/95705322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55caae7a9470 0x55caae7ad370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.562+0000 7f4353fff700 1 -- 172.21.15.52:0/95705322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caae7a8b60 msgr2=0x55caae7a8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.562+0000 7f4353fff700 1 --2- 172.21.15.52:0/95705322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caae7a8b60 0x55caae7a8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.562+0000 7f4353fff700 1 -- 172.21.15.52:0/95705322 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55caae57b630 con 0x55caae7ad8b0 2023-11-27T15:26:28.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.562+0000 7f4353fff700 1 --2- 172.21.15.52:0/95705322 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55caae7ad8b0 0x55caae7add00 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f434c009d50 tx=0x7f434c0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffad2311813cec server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:28.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.562+0000 7f4339ffb700 1 -- 172.21.15.52:0/95705322 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c018070 con 0x55caae7ad8b0 2023-11-27T15:26:28.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.562+0000 7f4339ffb700 1 -- 172.21.15.52:0/95705322 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f434c00f050 con 0x55caae7ad8b0 2023-11-27T15:26:28.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.562+0000 7f4339ffb700 1 -- 172.21.15.52:0/95705322 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c004830 con 0x55caae7ad8b0 2023-11-27T15:26:28.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.563+0000 7f4365d3db80 1 -- 172.21.15.52:0/95705322 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55caae7ad8b0 msgr2=0x55caae7add00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.563+0000 7f4365d3db80 1 --2- 172.21.15.52:0/95705322 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55caae7ad8b0 0x55caae7add00 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f434c009d50 tx=0x7f434c0077e0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.563+0000 7f4365d3db80 1 -- 172.21.15.52:0/95705322 shutdown_connections 2023-11-27T15:26:28.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.563+0000 7f4365d3db80 1 --2- 172.21.15.52:0/95705322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caae7a8b60 0x55caae7a8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.563+0000 7f4365d3db80 1 --2- 172.21.15.52:0/95705322 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55caae7ad8b0 0x55caae7add00 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.563+0000 7f4365d3db80 1 --2- 172.21.15.52:0/95705322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55caae7a9470 0x55caae7ad370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.563+0000 7f4365d3db80 1 -- 172.21.15.52:0/95705322 >> 172.21.15.52:0/95705322 conn(0x55caae79e360 msgr2=0x55caae7a3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:28.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.563+0000 7f4365d3db80 1 -- 172.21.15.52:0/95705322 shutdown_connections 2023-11-27T15:26:28.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.563+0000 7f4365d3db80 1 -- 172.21.15.52:0/95705322 wait complete. 2023-11-27T15:26:28.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.564+0000 7f4365d3db80 1 -- start start 2023-11-27T15:26:28.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.564+0000 7f4365d3db80 1 Processor -- start 2023-11-27T15:26:28.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.564+0000 7f4365d3db80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caae7a8b60 0x55caae7b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.564+0000 7f4365d3db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55caae7a9470 0x55caae7b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.564+0000 7f4365d3db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55caae7ad8b0 0x55caae7b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.564+0000 7f4365d3db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caae69b8e0 con 0x55caae7ad8b0 2023-11-27T15:26:28.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.564+0000 7f4365d3db80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caae69b580 con 0x55caae7a8b60 2023-11-27T15:26:28.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.564+0000 7f4365d3db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caae57d920 con 0x55caae7a9470 2023-11-27T15:26:28.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.564+0000 7f4365d3db80 10 client.0 _ll_register_callbacks cb 0x55caae6b9780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:28.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.564+0000 7f4353fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55caae7ad8b0 0x55caae7b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.564+0000 7f4353fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55caae7ad8b0 0x55caae7b2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55494/0 (socket says 172.21.15.52:55494) 2023-11-27T15:26:28.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.564+0000 7f4353fff700 1 -- 172.21.15.52:0/2382710809 learned_addr learned my addr 172.21.15.52:0/2382710809 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:28.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.565+0000 7f4352ffd700 1 --2- 172.21.15.52:0/2382710809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55caae7a9470 0x55caae7b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.565+0000 7f43537fe700 1 --2- 172.21.15.52:0/2382710809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caae7a8b60 0x55caae7b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.565+0000 7f4353fff700 1 -- 172.21.15.52:0/2382710809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55caae7a9470 msgr2=0x55caae7b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.565+0000 7f4353fff700 1 --2- 172.21.15.52:0/2382710809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55caae7a9470 0x55caae7b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.565+0000 7f4353fff700 1 -- 172.21.15.52:0/2382710809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caae7a8b60 msgr2=0x55caae7b4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.565+0000 7f4353fff700 1 --2- 172.21.15.52:0/2382710809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caae7a8b60 0x55caae7b4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.565+0000 7f4353fff700 1 -- 172.21.15.52:0/2382710809 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f434c009980 con 0x55caae7ad8b0 2023-11-27T15:26:28.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.565+0000 7f4353fff700 1 --2- 172.21.15.52:0/2382710809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55caae7ad8b0 0x55caae7b2870 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f434c0043b0 tx=0x7f434c003cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b54bed3c4281028 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:28.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.565+0000 7f4332ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:26:28.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.565+0000 7f4332ffd700 1 -- 172.21.15.52:0/2382710809 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c018070 con 0x55caae7ad8b0 2023-11-27T15:26:28.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.565+0000 7f4332ffd700 1 -- 172.21.15.52:0/2382710809 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f434c003ea0 con 0x55caae7ad8b0 2023-11-27T15:26:28.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.565+0000 7f4365d3db80 1 -- 172.21.15.52:0/2382710809 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55caae79bb20 con 0x55caae7ad8b0 2023-11-27T15:26:28.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.565+0000 7f4332ffd700 1 -- 172.21.15.52:0/2382710809 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f434c0041d0 con 0x55caae7ad8b0 2023-11-27T15:26:28.578 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.565+0000 7f4365d3db80 10 client.16875 fetch_fsmap learned FSMap version 34 2023-11-27T15:26:28.578 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.565+0000 7f4365d3db80 1 -- 172.21.15.52:0/2382710809 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55caae6c0190 con 0x55caae7ad8b0 2023-11-27T15:26:28.578 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.565+0000 7f4332ffd700 1 -- 172.21.15.52:0/2382710809 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55caae79bb20 con 0x55caae7ad8b0 2023-11-27T15:26:28.578 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.566+0000 7f4332ffd700 1 -- 172.21.15.52:0/2382710809 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f434c005050 con 0x55caae7ad8b0 2023-11-27T15:26:28.578 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.566+0000 7f4365d3db80 10 client.16875 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:26:28.578 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.566+0000 7f4365d3db80 10 client.16875 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:26:28.578 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.566+0000 7f4365d3db80 20 client.16875 populate_metadata read hostname 'smithi052' 2023-11-27T15:26:28.578 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.566+0000 7f4365d3db80 1 --2- 172.21.15.52:0/2382710809 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55caae7c2700 0x55caae7c4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.579 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.566+0000 7f4365d3db80 4 client.16875 mds_command: new command op to 24472 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:26:28.579 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.566+0000 7f4365d3db80 1 -- 172.21.15.52:0/2382710809 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55caae6bc0a0 con 0x55caae7c2700 2023-11-27T15:26:28.579 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.566+0000 7f43537fe700 1 --2- 172.21.15.52:0/2382710809 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55caae7c2700 0x55caae7c4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.579 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.566+0000 7f4332ffd700 10 client.16875 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:26:28.579 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.566+0000 7f43537fe700 1 --2- 172.21.15.52:0/2382710809 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55caae7c2700 0x55caae7c4b10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fe8a214642f8891c server_cookie=67d1c1a5d8bb5d13 in_seq=0 out_seq=0 2023-11-27T15:26:28.580 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:28 smithi102 ceph-mon[100296]: pgmap v265: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-11-27T15:26:28.580 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.567+0000 7f4332ffd700 1 -- 172.21.15.52:0/2382710809 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55caae6bc0a0 con 0x55caae7c2700 2023-11-27T15:26:28.580 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.567+0000 7f4332ffd700 10 client.16875 handle_command_reply: tid=0 2023-11-27T15:26:28.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.567+0000 7f4365d3db80 1 client.16875 shutdown 2023-11-27T15:26:28.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.567+0000 7f4365d3db80 1 -- 172.21.15.52:0/2382710809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55caae7ad8b0 msgr2=0x55caae7b2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.567+0000 7f4365d3db80 1 --2- 172.21.15.52:0/2382710809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55caae7ad8b0 0x55caae7b2870 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f434c0043b0 tx=0x7f434c003cb0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.567+0000 7f4365d3db80 1 -- 172.21.15.52:0/2382710809 shutdown_connections 2023-11-27T15:26:28.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.567+0000 7f4365d3db80 1 --2- 172.21.15.52:0/2382710809 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55caae7c2700 0x55caae7c4b10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.567+0000 7f4365d3db80 1 --2- 172.21.15.52:0/2382710809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caae7a8b60 0x55caae7b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.567+0000 7f4365d3db80 1 --2- 172.21.15.52:0/2382710809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55caae7ad8b0 0x55caae7b2870 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.567+0000 7f4365d3db80 1 --2- 172.21.15.52:0/2382710809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55caae7a9470 0x55caae7b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.567+0000 7f4332ffd700 0 client.16875 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:26:28.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.567+0000 7f4365d3db80 1 -- 172.21.15.52:0/2382710809 >> 172.21.15.52:0/2382710809 conn(0x55caae79e360 msgr2=0x55caae7aed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:28.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.567+0000 7f4365d3db80 1 -- 172.21.15.52:0/2382710809 shutdown_connections 2023-11-27T15:26:28.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.567+0000 7f4365d3db80 1 -- 172.21.15.52:0/2382710809 wait complete. 2023-11-27T15:26:28.582 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.568+0000 7f4365d3db80 20 client.16875 trim_cache size 0 max 16384 2023-11-27T15:26:28.585 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.581+0000 7f4365d3db80 1 Processor -- start 2023-11-27T15:26:28.586 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.581+0000 7f4365d3db80 1 -- start start 2023-11-27T15:26:28.586 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.581+0000 7f4365d3db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55caae7a8b60 0x55caae7b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.586 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.581+0000 7f4365d3db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55caae7a9470 0x55caae7a3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.586 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.581+0000 7f4365d3db80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caae7ad8b0 0x55caae7a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.586 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.581+0000 7f4365d3db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caae7ac400 con 0x55caae7a8b60 2023-11-27T15:26:28.586 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.581+0000 7f4365d3db80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caae79b960 con 0x55caae7ad8b0 2023-11-27T15:26:28.586 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.581+0000 7f4365d3db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caae7ac180 con 0x55caae7a9470 2023-11-27T15:26:28.586 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.581+0000 7f43537fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55caae7a8b60 0x55caae7b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.587 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.581+0000 7f43537fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55caae7a8b60 0x55caae7b4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55498/0 (socket says 172.21.15.52:55498) 2023-11-27T15:26:28.587 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.581+0000 7f43537fe700 1 -- 172.21.15.52:0/521400486 learned_addr learned my addr 172.21.15.52:0/521400486 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:28.587 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.581+0000 7f4353fff700 1 --2- 172.21.15.52:0/521400486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caae7ad8b0 0x55caae7a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.587 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.582+0000 7f4352ffd700 1 --2- 172.21.15.52:0/521400486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55caae7a9470 0x55caae7a3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.587 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.582+0000 7f43537fe700 1 -- 172.21.15.52:0/521400486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55caae7a9470 msgr2=0x55caae7a3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.587 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.582+0000 7f43537fe700 1 --2- 172.21.15.52:0/521400486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55caae7a9470 0x55caae7a3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.587 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.582+0000 7f43537fe700 1 -- 172.21.15.52:0/521400486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caae7ad8b0 msgr2=0x55caae7a3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.587 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.582+0000 7f43537fe700 1 --2- 172.21.15.52:0/521400486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caae7ad8b0 0x55caae7a3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.588 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.582+0000 7f43537fe700 1 -- 172.21.15.52:0/521400486 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55caae6bfee0 con 0x55caae7a8b60 2023-11-27T15:26:28.588 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.582+0000 7f4352ffd700 1 --2- 172.21.15.52:0/521400486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55caae7a9470 0x55caae7a3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:26:28.588 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.582+0000 7f43537fe700 1 --2- 172.21.15.52:0/521400486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55caae7a8b60 0x55caae7b4b40 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f4344005d90 tx=0x7f4344013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc894310bd13d7f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:28.588 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.582+0000 7f433a7fc700 1 -- 172.21.15.52:0/521400486 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4344024040 con 0x55caae7a8b60 2023-11-27T15:26:28.588 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.582+0000 7f433a7fc700 1 -- 172.21.15.52:0/521400486 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4344015070 con 0x55caae7a8b60 2023-11-27T15:26:28.588 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.582+0000 7f433a7fc700 1 -- 172.21.15.52:0/521400486 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4344004a90 con 0x55caae7a8b60 2023-11-27T15:26:28.592 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.583+0000 7f4365d3db80 1 -- 172.21.15.52:0/521400486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55caae7a8b60 msgr2=0x55caae7b4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.583+0000 7f4365d3db80 1 --2- 172.21.15.52:0/521400486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55caae7a8b60 0x55caae7b4b40 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f4344005d90 tx=0x7f4344013000 comp rx=0 tx=0).stop 2023-11-27T15:26:28.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.583+0000 7f4365d3db80 1 -- 172.21.15.52:0/521400486 shutdown_connections 2023-11-27T15:26:28.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.583+0000 7f4365d3db80 1 --2- 172.21.15.52:0/521400486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caae7ad8b0 0x55caae7a3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.583+0000 7f4365d3db80 1 --2- 172.21.15.52:0/521400486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55caae7a8b60 0x55caae7b4b40 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.583+0000 7f4365d3db80 1 --2- 172.21.15.52:0/521400486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55caae7a9470 0x55caae7a3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.583+0000 7f4365d3db80 1 -- 172.21.15.52:0/521400486 >> 172.21.15.52:0/521400486 conn(0x55caae79e360 msgr2=0x55caae7beb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:28.593 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.583+0000 7f4365d3db80 1 -- 172.21.15.52:0/521400486 shutdown_connections 2023-11-27T15:26:28.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.583+0000 7f4365d3db80 1 -- 172.21.15.52:0/521400486 wait complete. 2023-11-27T15:26:28.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.583+0000 7f4365d3db80 1 -- start start 2023-11-27T15:26:28.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.583+0000 7f4365d3db80 1 Processor -- start 2023-11-27T15:26:28.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.584+0000 7f4365d3db80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caae7a8b60 0x55caae80a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.584+0000 7f4365d3db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55caae7a9470 0x55caae80ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.584+0000 7f4365d3db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55caae7ad8b0 0x55caae80f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.594 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.584+0000 7f4365d3db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caae80fb50 con 0x55caae7ad8b0 2023-11-27T15:26:28.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.584+0000 7f4365d3db80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caae80fcc0 con 0x55caae7a8b60 2023-11-27T15:26:28.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.584+0000 7f4365d3db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caae80fe80 con 0x55caae7a9470 2023-11-27T15:26:28.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.584+0000 7f4365d3db80 10 client.0 _ll_register_callbacks cb 0x7f434400b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:28.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.584+0000 7f4353fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55caae7ad8b0 0x55caae80f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.584+0000 7f4353fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55caae7ad8b0 0x55caae80f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55508/0 (socket says 172.21.15.52:55508) 2023-11-27T15:26:28.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.584+0000 7f4353fff700 1 -- 172.21.15.52:0/3137627351 learned_addr learned my addr 172.21.15.52:0/3137627351 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:28.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.584+0000 7f43537fe700 1 --2- 172.21.15.52:0/3137627351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caae7a8b60 0x55caae80a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.595 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.584+0000 7f4352ffd700 1 --2- 172.21.15.52:0/3137627351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55caae7a9470 0x55caae80ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.584+0000 7f4353fff700 1 -- 172.21.15.52:0/3137627351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55caae7a9470 msgr2=0x55caae80ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.584+0000 7f4353fff700 1 --2- 172.21.15.52:0/3137627351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55caae7a9470 0x55caae80ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.584+0000 7f4353fff700 1 -- 172.21.15.52:0/3137627351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caae7a8b60 msgr2=0x55caae80a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.584+0000 7f4353fff700 1 --2- 172.21.15.52:0/3137627351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caae7a8b60 0x55caae80a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.584+0000 7f4353fff700 1 -- 172.21.15.52:0/3137627351 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43440114c0 con 0x55caae7ad8b0 2023-11-27T15:26:28.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.584+0000 7f43537fe700 1 --2- 172.21.15.52:0/3137627351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caae7a8b60 0x55caae80a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:26:28.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.584+0000 7f4353fff700 1 --2- 172.21.15.52:0/3137627351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55caae7ad8b0 0x55caae80f610 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f434c0053b0 tx=0x7f434c004940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab4a0f576944f2af server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:28.596 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.584+0000 7f4332ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:26:28.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.584+0000 7f4332ffd700 1 -- 172.21.15.52:0/3137627351 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c018070 con 0x55caae7ad8b0 2023-11-27T15:26:28.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.584+0000 7f4365d3db80 1 -- 172.21.15.52:0/3137627351 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f434c00f050 con 0x55caae7ad8b0 2023-11-27T15:26:28.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.584+0000 7f4332ffd700 1 -- 172.21.15.52:0/3137627351 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f434c005a20 con 0x55caae7ad8b0 2023-11-27T15:26:28.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.585+0000 7f4332ffd700 1 -- 172.21.15.52:0/3137627351 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f434c0057e0 con 0x55caae7ad8b0 2023-11-27T15:26:28.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.585+0000 7f4365d3db80 10 client.16881 fetch_fsmap learned FSMap version 34 2023-11-27T15:26:28.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.585+0000 7f4365d3db80 1 -- 172.21.15.52:0/3137627351 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55caae510c60 con 0x55caae7ad8b0 2023-11-27T15:26:28.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.585+0000 7f4332ffd700 1 -- 172.21.15.52:0/3137627351 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f434c00f050 con 0x55caae7ad8b0 2023-11-27T15:26:28.597 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.585+0000 7f4332ffd700 1 -- 172.21.15.52:0/3137627351 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f434c01e570 con 0x55caae7ad8b0 2023-11-27T15:26:28.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.585+0000 7f4365d3db80 10 client.16881 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:26:28.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.585+0000 7f4365d3db80 10 client.16881 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:26:28.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.585+0000 7f4365d3db80 20 client.16881 populate_metadata read hostname 'smithi052' 2023-11-27T15:26:28.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.585+0000 7f4365d3db80 1 --2- 172.21.15.52:0/3137627351 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55caae8172c0 0x55caae8196d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.585+0000 7f4365d3db80 4 client.16881 mds_command: new command op to 24472 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:26:28.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.585+0000 7f4365d3db80 1 -- 172.21.15.52:0/3137627351 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55caae6c8660 con 0x55caae8172c0 2023-11-27T15:26:28.598 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.585+0000 7f43537fe700 1 --2- 172.21.15.52:0/3137627351 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55caae8172c0 0x55caae8196d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.586+0000 7f4332ffd700 10 client.16881 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:26:28.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.586+0000 7f43537fe700 1 --2- 172.21.15.52:0/3137627351 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55caae8172c0 0x55caae8196d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a501176f4174ad1 server_cookie=ec7452b3dff5c8b6 in_seq=0 out_seq=0 2023-11-27T15:26:28.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.586+0000 7f4332ffd700 1 -- 172.21.15.52:0/3137627351 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55caae6c8660 con 0x55caae8172c0 2023-11-27T15:26:28.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.586+0000 7f4332ffd700 10 client.16881 handle_command_reply: tid=0 2023-11-27T15:26:28.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.586+0000 7f4365d3db80 1 client.16881 shutdown 2023-11-27T15:26:28.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.586+0000 7f4365d3db80 1 -- 172.21.15.52:0/3137627351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55caae7ad8b0 msgr2=0x55caae80f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.586+0000 7f4365d3db80 1 --2- 172.21.15.52:0/3137627351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55caae7ad8b0 0x55caae80f610 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f434c0053b0 tx=0x7f434c004940 comp rx=0 tx=0).stop 2023-11-27T15:26:28.599 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.586+0000 7f4365d3db80 1 -- 172.21.15.52:0/3137627351 shutdown_connections 2023-11-27T15:26:28.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.586+0000 7f4365d3db80 1 --2- 172.21.15.52:0/3137627351 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55caae8172c0 0x55caae8196d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.586+0000 7f4365d3db80 1 --2- 172.21.15.52:0/3137627351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55caae7a8b60 0x55caae80a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.586+0000 7f4332ffd700 0 client.16881 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:26:28.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.586+0000 7f4365d3db80 1 --2- 172.21.15.52:0/3137627351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55caae7ad8b0 0x55caae80f610 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.586+0000 7f4365d3db80 1 --2- 172.21.15.52:0/3137627351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55caae7a9470 0x55caae80ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.586+0000 7f4365d3db80 1 -- 172.21.15.52:0/3137627351 >> 172.21.15.52:0/3137627351 conn(0x55caae79e360 msgr2=0x55caae7a7960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:28.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.586+0000 7f4365d3db80 1 -- 172.21.15.52:0/3137627351 shutdown_connections 2023-11-27T15:26:28.600 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.586+0000 7f4365d3db80 1 -- 172.21.15.52:0/3137627351 wait complete. 2023-11-27T15:26:28.601 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.586+0000 7f4365d3db80 20 client.16881 trim_cache size 0 max 16384 2023-11-27T15:26:28.601 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:26:28.602 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.587+0000 7f4359bb1700 1 -- 172.21.15.52:0/431212789 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f433c0a1340 msgr2=0x7f433c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.602 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.587+0000 7f4359bb1700 1 --2- 172.21.15.52:0/431212789 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f433c0a1340 0x7f433c0a37f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4344005950 tx=0x7f434400b410 comp rx=0 tx=0).stop 2023-11-27T15:26:28.602 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.587+0000 7f4359bb1700 1 -- 172.21.15.52:0/431212789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43540ffdf0 msgr2=0x7f43541969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.602 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.587+0000 7f4359bb1700 1 --2- 172.21.15.52:0/431212789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43540ffdf0 0x7f43541969f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f4348009990 tx=0x7f4348004730 comp rx=0 tx=0).stop 2023-11-27T15:26:28.602 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.587+0000 7f4359bb1700 1 -- 172.21.15.52:0/431212789 shutdown_connections 2023-11-27T15:26:28.603 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.587+0000 7f4359bb1700 1 --2- 172.21.15.52:0/431212789 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f433c0a1340 0x7f433c0a37f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.603 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.587+0000 7f4359bb1700 1 --2- 172.21.15.52:0/431212789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43540ff450 0x7f43541964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.603 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.587+0000 7f4359bb1700 1 --2- 172.21.15.52:0/431212789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4354104490 0x7f435419ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.603 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.587+0000 7f4359bb1700 1 --2- 172.21.15.52:0/431212789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43540ffdf0 0x7f43541969f0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.603 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.587+0000 7f4359bb1700 1 -- 172.21.15.52:0/431212789 >> 172.21.15.52:0/431212789 conn(0x7f43540770b0 msgr2=0x7f4354067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:28.603 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.587+0000 7f4359bb1700 1 -- 172.21.15.52:0/431212789 shutdown_connections 2023-11-27T15:26:28.603 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.587+0000 7f4359bb1700 1 -- 172.21.15.52:0/431212789 wait complete. 2023-11-27T15:26:28.604 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:28 smithi052 ceph-mon[99056]: pgmap v265: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-11-27T15:26:28.605 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-27T15:26:28.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.729+0000 7f6e19416700 1 -- 172.21.15.52:0/2820032254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e14101660 msgr2=0x7f6e14104ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.729+0000 7f6e19416700 1 --2- 172.21.15.52:0/2820032254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e14101660 0x7f6e14104ef0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f6e040099c0 tx=0x7f6e04009cd0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.729+0000 7f6e19416700 1 -- 172.21.15.52:0/2820032254 shutdown_connections 2023-11-27T15:26:28.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.729+0000 7f6e19416700 1 --2- 172.21.15.52:0/2820032254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e14105620 0x7f6e1410d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.729+0000 7f6e19416700 1 --2- 172.21.15.52:0/2820032254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e14100cc0 0x7f6e14101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.729+0000 7f6e19416700 1 --2- 172.21.15.52:0/2820032254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e14101660 0x7f6e14104ef0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.729+0000 7f6e19416700 1 -- 172.21.15.52:0/2820032254 >> 172.21.15.52:0/2820032254 conn(0x7f6e14077560 msgr2=0x7f6e14077960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:28.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.729+0000 7f6e19416700 1 -- 172.21.15.52:0/2820032254 shutdown_connections 2023-11-27T15:26:28.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.729+0000 7f6e19416700 1 -- 172.21.15.52:0/2820032254 wait complete. 2023-11-27T15:26:28.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.730+0000 7f6e19416700 1 Processor -- start 2023-11-27T15:26:28.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.730+0000 7f6e19416700 1 -- start start 2023-11-27T15:26:28.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.730+0000 7f6e19416700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e14100cc0 0x7f6e141986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.730+0000 7f6e19416700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e14101660 0x7f6e14198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.730+0000 7f6e19416700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e14105620 0x7f6e1419bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.736 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.730+0000 7f6e19416700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1419c5d0 con 0x7f6e14100cc0 2023-11-27T15:26:28.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.730+0000 7f6e19416700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1419c740 con 0x7f6e14101660 2023-11-27T15:26:28.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.730+0000 7f6e19416700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1419c900 con 0x7f6e14105620 2023-11-27T15:26:28.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.730+0000 7f6e12ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e14100cc0 0x7f6e141986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.730+0000 7f6e12ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e14100cc0 0x7f6e141986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55518/0 (socket says 172.21.15.52:55518) 2023-11-27T15:26:28.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.730+0000 7f6e12ffd700 1 -- 172.21.15.52:0/1181064360 learned_addr learned my addr 172.21.15.52:0/1181064360 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:28.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.730+0000 7f6e127fc700 1 --2- 172.21.15.52:0/1181064360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e14101660 0x7f6e14198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.730+0000 7f6e137fe700 1 --2- 172.21.15.52:0/1181064360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e14105620 0x7f6e1419bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.737 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.730+0000 7f6e12ffd700 1 -- 172.21.15.52:0/1181064360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e14105620 msgr2=0x7f6e1419bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.738 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.730+0000 7f6e12ffd700 1 --2- 172.21.15.52:0/1181064360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e14105620 0x7f6e1419bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.738 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.730+0000 7f6e12ffd700 1 -- 172.21.15.52:0/1181064360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e14101660 msgr2=0x7f6e14198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.738 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.730+0000 7f6e12ffd700 1 --2- 172.21.15.52:0/1181064360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e14101660 0x7f6e14198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.738 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.730+0000 7f6e12ffd700 1 -- 172.21.15.52:0/1181064360 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e040095f0 con 0x7f6e14100cc0 2023-11-27T15:26:28.738 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.730+0000 7f6e137fe700 1 --2- 172.21.15.52:0/1181064360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e14105620 0x7f6e1419bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:26:28.738 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.731+0000 7f6e12ffd700 1 --2- 172.21.15.52:0/1181064360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e14100cc0 0x7f6e141986e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800c920 tx=0x7f6e0800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:28.738 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.731+0000 7f6dfbfff700 1 -- 172.21.15.52:0/1181064360 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e08007b00 con 0x7f6e14100cc0 2023-11-27T15:26:28.739 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.731+0000 7f6e19416700 1 -- 172.21.15.52:0/1181064360 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e14110320 con 0x7f6e14100cc0 2023-11-27T15:26:28.739 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.731+0000 7f6dfbfff700 1 -- 172.21.15.52:0/1181064360 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e0800ce70 con 0x7f6e14100cc0 2023-11-27T15:26:28.739 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.731+0000 7f6dfbfff700 1 -- 172.21.15.52:0/1181064360 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e08005460 con 0x7f6e14100cc0 2023-11-27T15:26:28.739 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.731+0000 7f6e19416700 1 -- 172.21.15.52:0/1181064360 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e14110190 con 0x7f6e14100cc0 2023-11-27T15:26:28.745 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.734+0000 7f6dfbfff700 1 -- 172.21.15.52:0/1181064360 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f6e08004750 con 0x7f6e14100cc0 2023-11-27T15:26:28.745 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.734+0000 7f6dfbfff700 1 --2- 172.21.15.52:0/1181064360 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f6dfc0a12c0 0x7f6dfc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.745 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.734+0000 7f6dfbfff700 1 -- 172.21.15.52:0/1181064360 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f6e080cc950 con 0x7f6e14100cc0 2023-11-27T15:26:28.745 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.734+0000 7f6e127fc700 1 --2- 172.21.15.52:0/1181064360 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f6dfc0a12c0 0x7f6dfc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.745 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.735+0000 7f6e127fc700 1 --2- 172.21.15.52:0/1181064360 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f6dfc0a12c0 0x7f6dfc0a3770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400b580 tx=0x7f6e04003680 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:28.745 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.735+0000 7f6e255a2b80 1 Processor -- start 2023-11-27T15:26:28.745 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.735+0000 7f6e255a2b80 1 -- start start 2023-11-27T15:26:28.746 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.735+0000 7f6e255a2b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b058fe2b60 0x55b058fe2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.746 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.735+0000 7f6e255a2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b058fe3470 0x55b058fe7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.746 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.735+0000 7f6e255a2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b058fe78b0 0x55b058fe7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.746 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.735+0000 7f6e255a2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b058ee8060 con 0x55b058fe3470 2023-11-27T15:26:28.746 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.735+0000 7f6e255a2b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b058f3a8a0 con 0x55b058fe2b60 2023-11-27T15:26:28.746 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.735+0000 7f6e255a2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b058f33d30 con 0x55b058fe78b0 2023-11-27T15:26:28.746 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.735+0000 7f6e127fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b058fe3470 0x55b058fe7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.746 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.735+0000 7f6e127fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b058fe3470 0x55b058fe7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55528/0 (socket says 172.21.15.52:55528) 2023-11-27T15:26:28.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.735+0000 7f6e127fc700 1 -- 172.21.15.52:0/3642292542 learned_addr learned my addr 172.21.15.52:0/3642292542 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:28.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.735+0000 7f6e12ffd700 1 --2- 172.21.15.52:0/3642292542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b058fe2b60 0x55b058fe2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.735+0000 7f6e137fe700 1 --2- 172.21.15.52:0/3642292542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b058fe78b0 0x55b058fe7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.736+0000 7f6e127fc700 1 -- 172.21.15.52:0/3642292542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b058fe78b0 msgr2=0x55b058fe7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.736+0000 7f6e127fc700 1 --2- 172.21.15.52:0/3642292542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b058fe78b0 0x55b058fe7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.736+0000 7f6e127fc700 1 -- 172.21.15.52:0/3642292542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b058fe2b60 msgr2=0x55b058fe2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.736+0000 7f6e127fc700 1 --2- 172.21.15.52:0/3642292542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b058fe2b60 0x55b058fe2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.747 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.736+0000 7f6e127fc700 1 -- 172.21.15.52:0/3642292542 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b058db5630 con 0x55b058fe3470 2023-11-27T15:26:28.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.736+0000 7f6e127fc700 1 --2- 172.21.15.52:0/3642292542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b058fe3470 0x55b058fe7370 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f6e040098e0 tx=0x7f6e0401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3251ab545675dea server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:28.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.736+0000 7f6df97fa700 1 -- 172.21.15.52:0/3642292542 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0402a070 con 0x55b058fe3470 2023-11-27T15:26:28.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.736+0000 7f6df97fa700 1 -- 172.21.15.52:0/3642292542 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e0401e040 con 0x55b058fe3470 2023-11-27T15:26:28.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.736+0000 7f6df97fa700 1 -- 172.21.15.52:0/3642292542 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0400f6c0 con 0x55b058fe3470 2023-11-27T15:26:28.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.737+0000 7f6e255a2b80 1 -- 172.21.15.52:0/3642292542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b058fe3470 msgr2=0x55b058fe7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.737+0000 7f6e255a2b80 1 --2- 172.21.15.52:0/3642292542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b058fe3470 0x55b058fe7370 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f6e040098e0 tx=0x7f6e0401f000 comp rx=0 tx=0).stop 2023-11-27T15:26:28.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.737+0000 7f6e255a2b80 1 -- 172.21.15.52:0/3642292542 shutdown_connections 2023-11-27T15:26:28.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.737+0000 7f6e255a2b80 1 --2- 172.21.15.52:0/3642292542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b058fe2b60 0x55b058fe2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.748 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.737+0000 7f6e255a2b80 1 --2- 172.21.15.52:0/3642292542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b058fe78b0 0x55b058fe7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.737+0000 7f6e255a2b80 1 --2- 172.21.15.52:0/3642292542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b058fe3470 0x55b058fe7370 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.737+0000 7f6e255a2b80 1 -- 172.21.15.52:0/3642292542 >> 172.21.15.52:0/3642292542 conn(0x55b058fd8360 msgr2=0x55b058fddb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:28.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.737+0000 7f6e255a2b80 1 -- 172.21.15.52:0/3642292542 shutdown_connections 2023-11-27T15:26:28.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.737+0000 7f6e255a2b80 1 -- 172.21.15.52:0/3642292542 wait complete. 2023-11-27T15:26:28.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.737+0000 7f6e255a2b80 1 -- start start 2023-11-27T15:26:28.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.737+0000 7f6e255a2b80 1 Processor -- start 2023-11-27T15:26:28.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.737+0000 7f6e255a2b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b058fe2b60 0x55b058fef0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.749 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.737+0000 7f6e255a2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b058fe3470 0x55b058fec2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.737+0000 7f6e255a2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b058fe78b0 0x55b058fec9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.737+0000 7f6e255a2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b058ed58e0 con 0x55b058fe78b0 2023-11-27T15:26:28.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.737+0000 7f6e255a2b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b058ed5580 con 0x55b058fe2b60 2023-11-27T15:26:28.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.737+0000 7f6e255a2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b058db7920 con 0x55b058fe3470 2023-11-27T15:26:28.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.738+0000 7f6e255a2b80 10 client.0 _ll_register_callbacks cb 0x55b058ef3780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:28.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.738+0000 7f6e137fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b058fe78b0 0x55b058fec9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.738+0000 7f6e12ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b058fe2b60 0x55b058fef0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.750 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.738+0000 7f6e12ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b058fe2b60 0x55b058fef0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:45190/0 (socket says 172.21.15.52:45190) 2023-11-27T15:26:28.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.738+0000 7f6e137fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b058fe78b0 0x55b058fec9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55536/0 (socket says 172.21.15.52:55536) 2023-11-27T15:26:28.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.738+0000 7f6e12ffd700 1 -- 172.21.15.52:0/3134597331 learned_addr learned my addr 172.21.15.52:0/3134597331 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:28.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.738+0000 7f6e127fc700 1 --2- 172.21.15.52:0/3134597331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b058fe3470 0x55b058fec2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.738+0000 7f6e12ffd700 1 -- 172.21.15.52:0/3134597331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b058fe3470 msgr2=0x55b058fec2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.738+0000 7f6e12ffd700 1 --2- 172.21.15.52:0/3134597331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b058fe3470 0x55b058fec2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.738+0000 7f6e12ffd700 1 -- 172.21.15.52:0/3134597331 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b058fe78b0 msgr2=0x55b058fec9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.738+0000 7f6e12ffd700 1 --2- 172.21.15.52:0/3134597331 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b058fe78b0 0x55b058fec9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.751 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.738+0000 7f6e12ffd700 1 -- 172.21.15.52:0/3134597331 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e1410d9b0 con 0x55b058fe2b60 2023-11-27T15:26:28.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.738+0000 7f6e137fe700 1 --2- 172.21.15.52:0/3134597331 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b058fe78b0 0x55b058fec9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:26:28.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.738+0000 7f6e12ffd700 1 --2- 172.21.15.52:0/3134597331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b058fe2b60 0x55b058fef0a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6e080ca670 tx=0x7f6e08099a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cd975239dcfb4d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:28.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.738+0000 7f6df6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2023-11-27T15:26:28.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.738+0000 7f6df6ff5700 1 -- 172.21.15.52:0/3134597331 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e080b1070 con 0x55b058fe2b60 2023-11-27T15:26:28.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.738+0000 7f6e255a2b80 1 -- 172.21.15.52:0/3134597331 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b058fd5b20 con 0x55b058fe2b60 2023-11-27T15:26:28.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.738+0000 7f6df6ff5700 1 -- 172.21.15.52:0/3134597331 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f6e080a1e40 con 0x55b058fe2b60 2023-11-27T15:26:28.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.738+0000 7f6df6ff5700 1 -- 172.21.15.52:0/3134597331 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e08099c50 con 0x55b058fe2b60 2023-11-27T15:26:28.752 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.739+0000 7f6e255a2b80 10 client.26591 fetch_fsmap learned FSMap version 34 2023-11-27T15:26:28.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.739+0000 7f6e255a2b80 1 -- 172.21.15.52:0/3134597331 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55b058efa190 con 0x55b058fe2b60 2023-11-27T15:26:28.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.739+0000 7f6df6ff5700 1 -- 172.21.15.52:0/3134597331 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b058fd5b20 con 0x55b058fe2b60 2023-11-27T15:26:28.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.739+0000 7f6df6ff5700 1 -- 172.21.15.52:0/3134597331 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6e080a19b0 con 0x55b058fe2b60 2023-11-27T15:26:28.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.739+0000 7f6e255a2b80 10 client.26591 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:26:28.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.739+0000 7f6e255a2b80 10 client.26591 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:26:28.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.739+0000 7f6e255a2b80 20 client.26591 populate_metadata read hostname 'smithi052' 2023-11-27T15:26:28.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.739+0000 7f6e255a2b80 1 --2- 172.21.15.52:0/3134597331 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55b058ffc790 0x55b058ffeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.739+0000 7f6e255a2b80 4 client.26591 mds_command: new command op to 24494 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:26:28.753 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.739+0000 7f6e255a2b80 1 -- 172.21.15.52:0/3134597331 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b058ef60a0 con 0x55b058ffc790 2023-11-27T15:26:28.754 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.739+0000 7f6e127fc700 1 --2- 172.21.15.52:0/3134597331 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55b058ffc790 0x55b058ffeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.754 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.740+0000 7f6df6ff5700 10 client.26591 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:26:28.754 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.740+0000 7f6e127fc700 1 --2- 172.21.15.52:0/3134597331 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55b058ffc790 0x55b058ffeba0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1f518db44916b574 server_cookie=dbafb3f869e5a12a in_seq=0 out_seq=0 2023-11-27T15:26:28.754 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.740+0000 7f6df6ff5700 1 -- 172.21.15.52:0/3134597331 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b058ef60a0 con 0x55b058ffc790 2023-11-27T15:26:28.754 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.740+0000 7f6df6ff5700 10 client.26591 handle_command_reply: tid=0 2023-11-27T15:26:28.754 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.740+0000 7f6e255a2b80 1 client.26591 shutdown 2023-11-27T15:26:28.754 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.741+0000 7f6e255a2b80 1 -- 172.21.15.52:0/3134597331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b058fe2b60 msgr2=0x55b058fef0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.754 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.741+0000 7f6e255a2b80 1 --2- 172.21.15.52:0/3134597331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b058fe2b60 0x55b058fef0a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6e080ca670 tx=0x7f6e08099a60 comp rx=0 tx=0).stop 2023-11-27T15:26:28.755 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.741+0000 7f6e255a2b80 1 -- 172.21.15.52:0/3134597331 shutdown_connections 2023-11-27T15:26:28.755 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.741+0000 7f6e255a2b80 1 --2- 172.21.15.52:0/3134597331 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55b058ffc790 0x55b058ffeba0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.755 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.741+0000 7f6e255a2b80 1 --2- 172.21.15.52:0/3134597331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b058fe2b60 0x55b058fef0a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.755 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.741+0000 7f6df6ff5700 0 client.26591 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:26:28.755 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.741+0000 7f6e255a2b80 1 --2- 172.21.15.52:0/3134597331 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b058fe78b0 0x55b058fec9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.755 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.741+0000 7f6e255a2b80 1 --2- 172.21.15.52:0/3134597331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b058fe3470 0x55b058fec2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.755 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.741+0000 7f6e255a2b80 1 -- 172.21.15.52:0/3134597331 >> 172.21.15.52:0/3134597331 conn(0x55b058fd8360 msgr2=0x55b058fdcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:28.755 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.741+0000 7f6e255a2b80 1 -- 172.21.15.52:0/3134597331 shutdown_connections 2023-11-27T15:26:28.755 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.741+0000 7f6e255a2b80 1 -- 172.21.15.52:0/3134597331 wait complete. 2023-11-27T15:26:28.756 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.741+0000 7f6e255a2b80 20 client.26591 trim_cache size 0 max 16384 2023-11-27T15:26:28.762 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.754+0000 7f6e255a2b80 1 Processor -- start 2023-11-27T15:26:28.762 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.754+0000 7f6e255a2b80 1 -- start start 2023-11-27T15:26:28.762 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.754+0000 7f6e255a2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b058fe2b60 0x55b058ff3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.754+0000 7f6e255a2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b058fe3470 0x55b058fe9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.754+0000 7f6e255a2b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b058fe78b0 0x55b058feebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.754+0000 7f6e255a2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b058fe3970 con 0x55b058fe3470 2023-11-27T15:26:28.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.754+0000 7f6e255a2b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b058fd7ee0 con 0x55b058fe78b0 2023-11-27T15:26:28.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.754+0000 7f6e255a2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b058fd59a0 con 0x55b058fe2b60 2023-11-27T15:26:28.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.754+0000 7f6e127fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b058fe3470 0x55b058fe9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.754+0000 7f6e127fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b058fe3470 0x55b058fe9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55542/0 (socket says 172.21.15.52:55542) 2023-11-27T15:26:28.763 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.754+0000 7f6e127fc700 1 -- 172.21.15.52:0/3668577000 learned_addr learned my addr 172.21.15.52:0/3668577000 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:28.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.754+0000 7f6e12ffd700 1 --2- 172.21.15.52:0/3668577000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b058fe2b60 0x55b058ff3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.754+0000 7f6e137fe700 1 --2- 172.21.15.52:0/3668577000 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b058fe78b0 0x55b058feebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.754+0000 7f6e127fc700 1 -- 172.21.15.52:0/3668577000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b058fe2b60 msgr2=0x55b058ff3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.754+0000 7f6e127fc700 1 --2- 172.21.15.52:0/3668577000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b058fe2b60 0x55b058ff3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.754+0000 7f6e127fc700 1 -- 172.21.15.52:0/3668577000 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b058fe78b0 msgr2=0x55b058feebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.754+0000 7f6e127fc700 1 --2- 172.21.15.52:0/3668577000 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b058fe78b0 0x55b058feebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.754+0000 7f6e127fc700 1 -- 172.21.15.52:0/3668577000 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b058fe7f50 con 0x55b058fe3470 2023-11-27T15:26:28.764 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.754+0000 7f6e127fc700 1 --2- 172.21.15.52:0/3668577000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b058fe3470 0x55b058fe9ce0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400c010 tx=0x7f6e04009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f866b6de5f1b41 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:28.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.754+0000 7f6df9ffb700 1 -- 172.21.15.52:0/3668577000 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0402a040 con 0x55b058fe3470 2023-11-27T15:26:28.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.755+0000 7f6df9ffb700 1 -- 172.21.15.52:0/3668577000 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e0403c5b0 con 0x55b058fe3470 2023-11-27T15:26:28.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.755+0000 7f6df9ffb700 1 -- 172.21.15.52:0/3668577000 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0402fb40 con 0x55b058fe3470 2023-11-27T15:26:28.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.755+0000 7f6e255a2b80 1 -- 172.21.15.52:0/3668577000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b058fe3470 msgr2=0x55b058fe9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.755+0000 7f6e255a2b80 1 --2- 172.21.15.52:0/3668577000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b058fe3470 0x55b058fe9ce0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400c010 tx=0x7f6e04009cd0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.755+0000 7f6e255a2b80 1 -- 172.21.15.52:0/3668577000 shutdown_connections 2023-11-27T15:26:28.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.755+0000 7f6e255a2b80 1 --2- 172.21.15.52:0/3668577000 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b058fe78b0 0x55b058feebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.755+0000 7f6e255a2b80 1 --2- 172.21.15.52:0/3668577000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b058fe2b60 0x55b058ff3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.765 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.755+0000 7f6e255a2b80 1 --2- 172.21.15.52:0/3668577000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b058fe3470 0x55b058fe9ce0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.766 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.755+0000 7f6e255a2b80 1 -- 172.21.15.52:0/3668577000 >> 172.21.15.52:0/3668577000 conn(0x55b058fd8360 msgr2=0x55b058ff8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:28.766 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.755+0000 7f6e255a2b80 1 -- 172.21.15.52:0/3668577000 shutdown_connections 2023-11-27T15:26:28.766 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.755+0000 7f6e255a2b80 1 -- 172.21.15.52:0/3668577000 wait complete. 2023-11-27T15:26:28.766 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.755+0000 7f6e255a2b80 1 -- start start 2023-11-27T15:26:28.766 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.755+0000 7f6e255a2b80 1 Processor -- start 2023-11-27T15:26:28.766 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.756+0000 7f6e255a2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b058fe2b60 0x55b058ff3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.766 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.756+0000 7f6e255a2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b058fe3470 0x55b058fef920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.766 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.756+0000 7f6e255a2b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b058fe78b0 0x55b058fefe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.756+0000 7f6e255a2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b058ff0510 con 0x55b058fe2b60 2023-11-27T15:26:28.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.756+0000 7f6e255a2b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b058ff0680 con 0x55b058fe78b0 2023-11-27T15:26:28.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.756+0000 7f6e255a2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05904f640 con 0x55b058fe3470 2023-11-27T15:26:28.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.756+0000 7f6e255a2b80 10 client.0 _ll_register_callbacks cb 0x7f6e080995c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:28.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.756+0000 7f6e12ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b058fe2b60 0x55b058ff3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.756+0000 7f6e12ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b058fe2b60 0x55b058ff3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55546/0 (socket says 172.21.15.52:55546) 2023-11-27T15:26:28.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.756+0000 7f6e12ffd700 1 -- 172.21.15.52:0/3290621337 learned_addr learned my addr 172.21.15.52:0/3290621337 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:28.767 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.756+0000 7f6e137fe700 1 --2- 172.21.15.52:0/3290621337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b058fe78b0 0x55b058fefe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.756+0000 7f6e127fc700 1 --2- 172.21.15.52:0/3290621337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b058fe3470 0x55b058fef920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.756+0000 7f6e12ffd700 1 -- 172.21.15.52:0/3290621337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b058fe3470 msgr2=0x55b058fef920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.756+0000 7f6e12ffd700 1 --2- 172.21.15.52:0/3290621337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b058fe3470 0x55b058fef920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.756+0000 7f6e12ffd700 1 -- 172.21.15.52:0/3290621337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b058fe78b0 msgr2=0x55b058fefe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.756+0000 7f6e12ffd700 1 --2- 172.21.15.52:0/3290621337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b058fe78b0 0x55b058fefe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.756+0000 7f6e12ffd700 1 -- 172.21.15.52:0/3290621337 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b058db5630 con 0x55b058fe2b60 2023-11-27T15:26:28.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.756+0000 7f6e127fc700 1 --2- 172.21.15.52:0/3290621337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b058fe3470 0x55b058fef920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:26:28.768 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.756+0000 7f6e137fe700 1 --2- 172.21.15.52:0/3290621337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b058fe78b0 0x55b058fefe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:26:28.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.756+0000 7f6e12ffd700 1 --2- 172.21.15.52:0/3290621337 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b058fe2b60 0x55b058ff3b50 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6e080a1910 tx=0x7f6e08000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aec21c2bd89ffa4 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:28.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.756+0000 7f6df6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:26:28.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.756+0000 7f6df6ff5700 1 -- 172.21.15.52:0/3290621337 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e080b1070 con 0x55b058fe2b60 2023-11-27T15:26:28.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.756+0000 7f6df6ff5700 1 -- 172.21.15.52:0/3290621337 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f6e08004c60 con 0x55b058fe2b60 2023-11-27T15:26:28.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.756+0000 7f6e255a2b80 1 -- 172.21.15.52:0/3290621337 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e0401e040 con 0x55b058fe2b60 2023-11-27T15:26:28.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.757+0000 7f6df6ff5700 1 -- 172.21.15.52:0/3290621337 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e080a1e40 con 0x55b058fe2b60 2023-11-27T15:26:28.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.757+0000 7f6e255a2b80 10 client.16899 fetch_fsmap learned FSMap version 34 2023-11-27T15:26:28.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.757+0000 7f6e255a2b80 1 -- 172.21.15.52:0/3290621337 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55b058d61de0 con 0x55b058fe2b60 2023-11-27T15:26:28.769 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.757+0000 7f6df6ff5700 1 -- 172.21.15.52:0/3290621337 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e0401e040 con 0x55b058fe2b60 2023-11-27T15:26:28.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.757+0000 7f6df6ff5700 1 -- 172.21.15.52:0/3290621337 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6e08098950 con 0x55b058fe2b60 2023-11-27T15:26:28.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.757+0000 7f6e255a2b80 10 client.16899 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:26:28.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.757+0000 7f6e255a2b80 10 client.16899 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:26:28.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.757+0000 7f6e255a2b80 20 client.16899 populate_metadata read hostname 'smithi052' 2023-11-27T15:26:28.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.757+0000 7f6e255a2b80 1 --2- 172.21.15.52:0/3290621337 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55b059050860 0x55b059052c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.757+0000 7f6e255a2b80 4 client.16899 mds_command: new command op to 24494 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:26:28.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.757+0000 7f6e255a2b80 1 -- 172.21.15.52:0/3290621337 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b058f02660 con 0x55b059050860 2023-11-27T15:26:28.770 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.757+0000 7f6e127fc700 1 --2- 172.21.15.52:0/3290621337 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55b059050860 0x55b059052c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.758+0000 7f6df6ff5700 10 client.16899 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:26:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.758+0000 7f6e127fc700 1 --2- 172.21.15.52:0/3290621337 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55b059050860 0x55b059052c40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a357881b81f61f30 server_cookie=251368bba634e207 in_seq=0 out_seq=0 2023-11-27T15:26:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.758+0000 7f6df6ff5700 1 -- 172.21.15.52:0/3290621337 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b058f02660 con 0x55b059050860 2023-11-27T15:26:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.758+0000 7f6df6ff5700 10 client.16899 handle_command_reply: tid=0 2023-11-27T15:26:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.758+0000 7f6e255a2b80 1 client.16899 shutdown 2023-11-27T15:26:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.758+0000 7f6e255a2b80 1 -- 172.21.15.52:0/3290621337 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b058fe2b60 msgr2=0x55b058ff3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.758+0000 7f6e255a2b80 1 --2- 172.21.15.52:0/3290621337 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b058fe2b60 0x55b058ff3b50 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6e080a1910 tx=0x7f6e08000bc0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.758+0000 7f6e255a2b80 1 -- 172.21.15.52:0/3290621337 shutdown_connections 2023-11-27T15:26:28.771 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.758+0000 7f6e255a2b80 1 --2- 172.21.15.52:0/3290621337 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55b059050860 0x55b059052c40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.758+0000 7f6e255a2b80 1 --2- 172.21.15.52:0/3290621337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b058fe78b0 0x55b058fefe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.758+0000 7f6e255a2b80 1 --2- 172.21.15.52:0/3290621337 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b058fe2b60 0x55b058ff3b50 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.758+0000 7f6e255a2b80 1 --2- 172.21.15.52:0/3290621337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b058fe3470 0x55b058fef920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.758+0000 7f6df6ff5700 0 client.16899 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:26:28.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.758+0000 7f6e255a2b80 1 -- 172.21.15.52:0/3290621337 >> 172.21.15.52:0/3290621337 conn(0x55b058fd8360 msgr2=0x55b058fe6830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:28.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.758+0000 7f6e255a2b80 1 -- 172.21.15.52:0/3290621337 shutdown_connections 2023-11-27T15:26:28.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.758+0000 7f6e255a2b80 1 -- 172.21.15.52:0/3290621337 wait complete. 2023-11-27T15:26:28.772 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.758+0000 7f6e255a2b80 20 client.16899 trim_cache size 0 max 16384 2023-11-27T15:26:28.773 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:26:28.773 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.759+0000 7f6e19416700 1 -- 172.21.15.52:0/1181064360 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f6dfc0a12c0 msgr2=0x7f6dfc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.759+0000 7f6e19416700 1 --2- 172.21.15.52:0/1181064360 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f6dfc0a12c0 0x7f6dfc0a3770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400b580 tx=0x7f6e04003680 comp rx=0 tx=0).stop 2023-11-27T15:26:28.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.759+0000 7f6e19416700 1 -- 172.21.15.52:0/1181064360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e14100cc0 msgr2=0x7f6e141986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.759+0000 7f6e19416700 1 --2- 172.21.15.52:0/1181064360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e14100cc0 0x7f6e141986e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800c920 tx=0x7f6e0800cc30 comp rx=0 tx=0).stop 2023-11-27T15:26:28.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.759+0000 7f6e19416700 1 -- 172.21.15.52:0/1181064360 shutdown_connections 2023-11-27T15:26:28.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.759+0000 7f6e19416700 1 --2- 172.21.15.52:0/1181064360 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f6dfc0a12c0 0x7f6dfc0a3770 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.759+0000 7f6e19416700 1 --2- 172.21.15.52:0/1181064360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e14105620 0x7f6e1419bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.774 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.759+0000 7f6e19416700 1 --2- 172.21.15.52:0/1181064360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e14100cc0 0x7f6e141986e0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.775 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.759+0000 7f6e19416700 1 --2- 172.21.15.52:0/1181064360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6e14101660 0x7f6e14198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.775 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.759+0000 7f6e19416700 1 -- 172.21.15.52:0/1181064360 >> 172.21.15.52:0/1181064360 conn(0x7f6e14077560 msgr2=0x7f6e141001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:28.775 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.759+0000 7f6e19416700 1 -- 172.21.15.52:0/1181064360 shutdown_connections 2023-11-27T15:26:28.775 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.759+0000 7f6e19416700 1 -- 172.21.15.52:0/1181064360 wait complete. 2023-11-27T15:26:28.776 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-27T15:26:28.903 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.899+0000 7f0edeaa5700 1 -- 172.21.15.52:0/1591058214 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ed81005b0 msgr2=0x7f0ed8100980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.903 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.899+0000 7f0edeaa5700 1 --2- 172.21.15.52:0/1591058214 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ed81005b0 0x7f0ed8100980 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc0099c0 tx=0x7f0ecc009cd0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.903 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.900+0000 7f0edeaa5700 1 -- 172.21.15.52:0/1591058214 shutdown_connections 2023-11-27T15:26:28.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.900+0000 7f0edeaa5700 1 --2- 172.21.15.52:0/1591058214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ed81055f0 0x7f0ed810d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.900+0000 7f0edeaa5700 1 --2- 172.21.15.52:0/1591058214 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ed81005b0 0x7f0ed8100980 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.900+0000 7f0edeaa5700 1 --2- 172.21.15.52:0/1591058214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ed8100f50 0x7f0ed8104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.900+0000 7f0edeaa5700 1 -- 172.21.15.52:0/1591058214 >> 172.21.15.52:0/1591058214 conn(0x7f0ed8077170 msgr2=0x7f0ed8077570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:28.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.900+0000 7f0edeaa5700 1 -- 172.21.15.52:0/1591058214 shutdown_connections 2023-11-27T15:26:28.904 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.900+0000 7f0edeaa5700 1 -- 172.21.15.52:0/1591058214 wait complete. 2023-11-27T15:26:28.906 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.900+0000 7f0edeaa5700 1 Processor -- start 2023-11-27T15:26:28.906 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.900+0000 7f0edeaa5700 1 -- start start 2023-11-27T15:26:28.906 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.900+0000 7f0edeaa5700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ed81005b0 0x7f0ed8196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.906 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.900+0000 7f0edeaa5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ed8100f50 0x7f0ed8196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.907 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.900+0000 7f0edeaa5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ed81055f0 0x7f0ed819aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.907 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.900+0000 7f0edeaa5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed819b4e0 con 0x7f0ed81055f0 2023-11-27T15:26:28.907 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.900+0000 7f0edeaa5700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed819b650 con 0x7f0ed81005b0 2023-11-27T15:26:28.907 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.900+0000 7f0edeaa5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed819b810 con 0x7f0ed8100f50 2023-11-27T15:26:28.907 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.901+0000 7f0edd042700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ed81055f0 0x7f0ed819aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.907 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.901+0000 7f0edd042700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ed81055f0 0x7f0ed819aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55562/0 (socket says 172.21.15.52:55562) 2023-11-27T15:26:28.907 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.901+0000 7f0edd042700 1 -- 172.21.15.52:0/3955771437 learned_addr learned my addr 172.21.15.52:0/3955771437 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:28.908 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.901+0000 7f0edc841700 1 --2- 172.21.15.52:0/3955771437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ed81005b0 0x7f0ed8196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.908 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.901+0000 7f0ed7fff700 1 --2- 172.21.15.52:0/3955771437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ed8100f50 0x7f0ed8196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.908 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.901+0000 7f0edd042700 1 -- 172.21.15.52:0/3955771437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ed8100f50 msgr2=0x7f0ed8196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.908 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.901+0000 7f0edd042700 1 --2- 172.21.15.52:0/3955771437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ed8100f50 0x7f0ed8196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.908 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.901+0000 7f0edd042700 1 -- 172.21.15.52:0/3955771437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ed81005b0 msgr2=0x7f0ed8196500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.908 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.901+0000 7f0edd042700 1 --2- 172.21.15.52:0/3955771437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ed81005b0 0x7f0ed8196500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.908 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.901+0000 7f0edd042700 1 -- 172.21.15.52:0/3955771437 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ecc0095f0 con 0x7f0ed81055f0 2023-11-27T15:26:28.909 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.901+0000 7f0edd042700 1 --2- 172.21.15.52:0/3955771437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ed81055f0 0x7f0ed819aea0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000ada0 tx=0x7f0ed000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:28.909 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.901+0000 7f0ed5ffb700 1 -- 172.21.15.52:0/3955771437 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed000f070 con 0x7f0ed81055f0 2023-11-27T15:26:28.909 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.901+0000 7f0ed5ffb700 1 -- 172.21.15.52:0/3955771437 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ed0007b30 con 0x7f0ed81055f0 2023-11-27T15:26:28.909 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.901+0000 7f0ed5ffb700 1 -- 172.21.15.52:0/3955771437 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed00053e0 con 0x7f0ed81055f0 2023-11-27T15:26:28.909 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.901+0000 7f0edeaa5700 1 -- 172.21.15.52:0/3955771437 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed8110400 con 0x7f0ed81055f0 2023-11-27T15:26:28.909 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.902+0000 7f0edeaa5700 1 -- 172.21.15.52:0/3955771437 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed8110270 con 0x7f0ed81055f0 2023-11-27T15:26:28.911 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.904+0000 7f0ed5ffb700 1 -- 172.21.15.52:0/3955771437 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0ed000c980 con 0x7f0ed81055f0 2023-11-27T15:26:28.911 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.905+0000 7f0ed5ffb700 1 --2- 172.21.15.52:0/3955771437 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f0ec80a1340 0x7f0ec80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.905+0000 7f0edc841700 1 --2- 172.21.15.52:0/3955771437 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f0ec80a1340 0x7f0ec80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.905+0000 7f0ed5ffb700 1 -- 172.21.15.52:0/3955771437 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f0ed00cc280 con 0x7f0ed81055f0 2023-11-27T15:26:28.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.906+0000 7f0edc841700 1 --2- 172.21.15.52:0/3955771437 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f0ec80a1340 0x7f0ec80a37f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc006010 tx=0x7f0ecc009f10 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:28.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.906+0000 7f0eeac31b80 1 Processor -- start 2023-11-27T15:26:28.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.906+0000 7f0eeac31b80 1 -- start start 2023-11-27T15:26:28.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.906+0000 7f0eeac31b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e5ed69b60 0x564e5ed69f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.906+0000 7f0eeac31b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e5ed6a470 0x564e5ed6e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.912 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.906+0000 7f0eeac31b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e5ed6e8b0 0x564e5ed6ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.913 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.906+0000 7f0eeac31b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e5ec6f060 con 0x564e5ed69b60 2023-11-27T15:26:28.913 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.906+0000 7f0eeac31b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e5ecc18a0 con 0x564e5ed6a470 2023-11-27T15:26:28.913 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.906+0000 7f0eeac31b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e5ecbad30 con 0x564e5ed6e8b0 2023-11-27T15:26:28.913 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.906+0000 7f0edc841700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e5ed69b60 0x564e5ed69f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.913 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.906+0000 7f0edc841700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e5ed69b60 0x564e5ed69f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55566/0 (socket says 172.21.15.52:55566) 2023-11-27T15:26:28.913 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.906+0000 7f0edc841700 1 -- 172.21.15.52:0/483004137 learned_addr learned my addr 172.21.15.52:0/483004137 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:28.913 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.906+0000 7f0ed7fff700 1 --2- 172.21.15.52:0/483004137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e5ed6a470 0x564e5ed6e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.913 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.906+0000 7f0edd042700 1 --2- 172.21.15.52:0/483004137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e5ed6e8b0 0x564e5ed6ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.914 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.907+0000 7f0edc841700 1 -- 172.21.15.52:0/483004137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e5ed6e8b0 msgr2=0x564e5ed6ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.914 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.907+0000 7f0edc841700 1 --2- 172.21.15.52:0/483004137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e5ed6e8b0 0x564e5ed6ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.914 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.907+0000 7f0edc841700 1 -- 172.21.15.52:0/483004137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e5ed6a470 msgr2=0x564e5ed6e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.914 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.907+0000 7f0edc841700 1 --2- 172.21.15.52:0/483004137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e5ed6a470 0x564e5ed6e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.914 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.907+0000 7f0edc841700 1 -- 172.21.15.52:0/483004137 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e5eb3c630 con 0x564e5ed69b60 2023-11-27T15:26:28.914 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.907+0000 7f0ed7fff700 1 --2- 172.21.15.52:0/483004137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e5ed6a470 0x564e5ed6e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:26:28.914 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.907+0000 7f0edc841700 1 --2- 172.21.15.52:0/483004137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e5ed69b60 0x564e5ed69f30 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc015000 tx=0x7f0ecc020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2caf5e0127cc606c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:28.918 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.908+0000 7f0ec2ffd700 1 -- 172.21.15.52:0/483004137 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc0038c0 con 0x564e5ed69b60 2023-11-27T15:26:28.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.908+0000 7f0ec2ffd700 1 -- 172.21.15.52:0/483004137 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ecc01e040 con 0x564e5ed69b60 2023-11-27T15:26:28.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.908+0000 7f0ec2ffd700 1 -- 172.21.15.52:0/483004137 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc00f8c0 con 0x564e5ed69b60 2023-11-27T15:26:28.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.909+0000 7f0eeac31b80 1 -- 172.21.15.52:0/483004137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e5ed69b60 msgr2=0x564e5ed69f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.909+0000 7f0eeac31b80 1 --2- 172.21.15.52:0/483004137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e5ed69b60 0x564e5ed69f30 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc015000 tx=0x7f0ecc020000 comp rx=0 tx=0).stop 2023-11-27T15:26:28.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.909+0000 7f0eeac31b80 1 -- 172.21.15.52:0/483004137 shutdown_connections 2023-11-27T15:26:28.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.909+0000 7f0eeac31b80 1 --2- 172.21.15.52:0/483004137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e5ed6e8b0 0x564e5ed6ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.909+0000 7f0eeac31b80 1 --2- 172.21.15.52:0/483004137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e5ed69b60 0x564e5ed69f30 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.919 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.909+0000 7f0eeac31b80 1 --2- 172.21.15.52:0/483004137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e5ed6a470 0x564e5ed6e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.909+0000 7f0eeac31b80 1 -- 172.21.15.52:0/483004137 >> 172.21.15.52:0/483004137 conn(0x564e5ed5f360 msgr2=0x564e5ed64b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:28.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.909+0000 7f0eeac31b80 1 -- 172.21.15.52:0/483004137 shutdown_connections 2023-11-27T15:26:28.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.909+0000 7f0eeac31b80 1 -- 172.21.15.52:0/483004137 wait complete. 2023-11-27T15:26:28.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.909+0000 7f0eeac31b80 1 -- start start 2023-11-27T15:26:28.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.910+0000 7f0eeac31b80 1 Processor -- start 2023-11-27T15:26:28.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.910+0000 7f0eeac31b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e5ed69b60 0x564e5ed76090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.910+0000 7f0eeac31b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e5ed6a470 0x564e5ed73380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.920 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.910+0000 7f0eeac31b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e5ed6e8b0 0x564e5ed73ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.910+0000 7f0eeac31b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e5ec5c8e0 con 0x564e5ed6e8b0 2023-11-27T15:26:28.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.910+0000 7f0eeac31b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e5ec5c580 con 0x564e5ed69b60 2023-11-27T15:26:28.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.910+0000 7f0eeac31b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e5eb3e920 con 0x564e5ed6a470 2023-11-27T15:26:28.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.910+0000 7f0eeac31b80 10 client.0 _ll_register_callbacks cb 0x564e5ec7a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:28.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.910+0000 7f0edd042700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e5ed6e8b0 0x564e5ed73ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.910+0000 7f0edd042700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e5ed6e8b0 0x564e5ed73ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55576/0 (socket says 172.21.15.52:55576) 2023-11-27T15:26:28.921 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.910+0000 7f0edd042700 1 -- 172.21.15.52:0/532061471 learned_addr learned my addr 172.21.15.52:0/532061471 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.910+0000 7f0edc841700 1 --2- 172.21.15.52:0/532061471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e5ed69b60 0x564e5ed76090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.910+0000 7f0ed7fff700 1 --2- 172.21.15.52:0/532061471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e5ed6a470 0x564e5ed73380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.911+0000 7f0edd042700 1 -- 172.21.15.52:0/532061471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e5ed6a470 msgr2=0x564e5ed73380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.911+0000 7f0edd042700 1 --2- 172.21.15.52:0/532061471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e5ed6a470 0x564e5ed73380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.911+0000 7f0edd042700 1 -- 172.21.15.52:0/532061471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e5ed69b60 msgr2=0x564e5ed76090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.911+0000 7f0edd042700 1 --2- 172.21.15.52:0/532061471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e5ed69b60 0x564e5ed76090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.911+0000 7f0edd042700 1 -- 172.21.15.52:0/532061471 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed810da00 con 0x564e5ed6e8b0 2023-11-27T15:26:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.911+0000 7f0edd042700 1 --2- 172.21.15.52:0/532061471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e5ed6e8b0 0x564e5ed73ac0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0ed00c9a00 tx=0x7f0ed0098e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1beed6934b0b293 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:28.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.911+0000 7f0eb7fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:26:28.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.911+0000 7f0eb7fff700 1 -- 172.21.15.52:0/532061471 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed00993b0 con 0x564e5ed6e8b0 2023-11-27T15:26:28.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.911+0000 7f0eb7fff700 1 -- 172.21.15.52:0/532061471 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f0ed00cc070 con 0x564e5ed6e8b0 2023-11-27T15:26:28.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.911+0000 7f0eeac31b80 1 -- 172.21.15.52:0/532061471 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e5ed5cb20 con 0x564e5ed6e8b0 2023-11-27T15:26:28.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.911+0000 7f0eb7fff700 1 -- 172.21.15.52:0/532061471 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ed009d3e0 con 0x564e5ed6e8b0 2023-11-27T15:26:28.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.911+0000 7f0eeac31b80 10 client.16911 fetch_fsmap learned FSMap version 34 2023-11-27T15:26:28.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.911+0000 7f0eeac31b80 1 -- 172.21.15.52:0/532061471 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x564e5ec81190 con 0x564e5ed6e8b0 2023-11-27T15:26:28.923 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.911+0000 7f0eb7fff700 1 -- 172.21.15.52:0/532061471 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564e5ed5cb20 con 0x564e5ed6e8b0 2023-11-27T15:26:28.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.911+0000 7f0eb7fff700 1 -- 172.21.15.52:0/532061471 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0ed0099610 con 0x564e5ed6e8b0 2023-11-27T15:26:28.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.912+0000 7f0eeac31b80 10 client.16911 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:26:28.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.912+0000 7f0eeac31b80 10 client.16911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:26:28.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.912+0000 7f0eeac31b80 20 client.16911 populate_metadata read hostname 'smithi052' 2023-11-27T15:26:28.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.912+0000 7f0eeac31b80 1 --2- 172.21.15.52:0/532061471 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x564e5ed83930 0x564e5ed85d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.912+0000 7f0eeac31b80 4 client.16911 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:26:28.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.912+0000 7f0eeac31b80 1 -- 172.21.15.52:0/532061471 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e5ec7d0a0 con 0x564e5ed83930 2023-11-27T15:26:28.924 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.912+0000 7f0edc841700 1 --2- 172.21.15.52:0/532061471 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x564e5ed83930 0x564e5ed85d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.912+0000 7f0eb7fff700 10 client.16911 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:26:28.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.912+0000 7f0edc841700 1 --2- 172.21.15.52:0/532061471 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x564e5ed83930 0x564e5ed85d40 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4e3e6b1e8f192c9 server_cookie=fd302f3ed81ebb0e in_seq=0 out_seq=0 2023-11-27T15:26:28.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.913+0000 7f0eb7fff700 1 -- 172.21.15.52:0/532061471 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564e5ec7d0a0 con 0x564e5ed83930 2023-11-27T15:26:28.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.913+0000 7f0eb7fff700 10 client.16911 handle_command_reply: tid=0 2023-11-27T15:26:28.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.913+0000 7f0eeac31b80 1 client.16911 shutdown 2023-11-27T15:26:28.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.913+0000 7f0eeac31b80 1 -- 172.21.15.52:0/532061471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e5ed6e8b0 msgr2=0x564e5ed73ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.913+0000 7f0eeac31b80 1 --2- 172.21.15.52:0/532061471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e5ed6e8b0 0x564e5ed73ac0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0ed00c9a00 tx=0x7f0ed0098e00 comp rx=0 tx=0).stop 2023-11-27T15:26:28.925 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.913+0000 7f0eeac31b80 1 -- 172.21.15.52:0/532061471 shutdown_connections 2023-11-27T15:26:28.926 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.913+0000 7f0eeac31b80 1 --2- 172.21.15.52:0/532061471 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x564e5ed83930 0x564e5ed85d40 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.926 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.913+0000 7f0eeac31b80 1 --2- 172.21.15.52:0/532061471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e5ed69b60 0x564e5ed76090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.926 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.913+0000 7f0eb7fff700 0 client.16911 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:26:28.926 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.913+0000 7f0eeac31b80 1 --2- 172.21.15.52:0/532061471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e5ed6e8b0 0x564e5ed73ac0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.926 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.913+0000 7f0eeac31b80 1 --2- 172.21.15.52:0/532061471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e5ed6a470 0x564e5ed73380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.926 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.913+0000 7f0eeac31b80 1 -- 172.21.15.52:0/532061471 >> 172.21.15.52:0/532061471 conn(0x564e5ed5f360 msgr2=0x564e5ed63de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:28.926 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.913+0000 7f0eeac31b80 1 -- 172.21.15.52:0/532061471 shutdown_connections 2023-11-27T15:26:28.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.913+0000 7f0eeac31b80 1 -- 172.21.15.52:0/532061471 wait complete. 2023-11-27T15:26:28.927 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.913+0000 7f0eeac31b80 20 client.16911 trim_cache size 0 max 16384 2023-11-27T15:26:28.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.926+0000 7f0eeac31b80 1 Processor -- start 2023-11-27T15:26:28.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.926+0000 7f0eeac31b80 1 -- start start 2023-11-27T15:26:28.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.926+0000 7f0eeac31b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e5ed5f360 0x564e5ed6d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.926+0000 7f0eeac31b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e5ed69b60 0x564e5ed64560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.926+0000 7f0eeac31b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e5ed6a470 0x564e5ed64aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.926+0000 7f0eeac31b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e5ed748f0 con 0x564e5ed5f360 2023-11-27T15:26:28.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.926+0000 7f0eeac31b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e5ed5eee0 con 0x564e5ed69b60 2023-11-27T15:26:28.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.926+0000 7f0eeac31b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e5ed6a900 con 0x564e5ed6a470 2023-11-27T15:26:28.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.927+0000 7f0edc841700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e5ed5f360 0x564e5ed6d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.927+0000 7f0edc841700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e5ed5f360 0x564e5ed6d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55580/0 (socket says 172.21.15.52:55580) 2023-11-27T15:26:28.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.927+0000 7f0edc841700 1 -- 172.21.15.52:0/606252670 learned_addr learned my addr 172.21.15.52:0/606252670 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:28.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.927+0000 7f0ed7fff700 1 --2- 172.21.15.52:0/606252670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e5ed69b60 0x564e5ed64560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.927+0000 7f0edd042700 1 --2- 172.21.15.52:0/606252670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e5ed6a470 0x564e5ed64aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.927+0000 7f0edc841700 1 -- 172.21.15.52:0/606252670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e5ed6a470 msgr2=0x564e5ed64aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.927+0000 7f0edc841700 1 --2- 172.21.15.52:0/606252670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e5ed6a470 0x564e5ed64aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.927+0000 7f0edc841700 1 -- 172.21.15.52:0/606252670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e5ed69b60 msgr2=0x564e5ed64560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.927+0000 7f0edc841700 1 --2- 172.21.15.52:0/606252670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e5ed69b60 0x564e5ed64560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.927+0000 7f0edc841700 1 -- 172.21.15.52:0/606252670 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e5ec80ee0 con 0x564e5ed5f360 2023-11-27T15:26:28.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.927+0000 7f0ed7fff700 1 --2- 172.21.15.52:0/606252670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e5ed69b60 0x564e5ed64560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:26:28.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.927+0000 7f0edc841700 1 --2- 172.21.15.52:0/606252670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e5ed5f360 0x564e5ed6d1a0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc015000 tx=0x7f0ecc01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4707a0f76fca161 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:28.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.927+0000 7f0ec37fe700 1 -- 172.21.15.52:0/606252670 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc005a00 con 0x564e5ed5f360 2023-11-27T15:26:28.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.927+0000 7f0ec37fe700 1 -- 172.21.15.52:0/606252670 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ecc03c0f0 con 0x564e5ed5f360 2023-11-27T15:26:28.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.927+0000 7f0ec37fe700 1 -- 172.21.15.52:0/606252670 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc003cc0 con 0x564e5ed5f360 2023-11-27T15:26:28.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.928+0000 7f0eeac31b80 1 -- 172.21.15.52:0/606252670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e5ed5f360 msgr2=0x564e5ed6d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.928+0000 7f0eeac31b80 1 --2- 172.21.15.52:0/606252670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e5ed5f360 0x564e5ed6d1a0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc015000 tx=0x7f0ecc01a040 comp rx=0 tx=0).stop 2023-11-27T15:26:28.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.928+0000 7f0eeac31b80 1 -- 172.21.15.52:0/606252670 shutdown_connections 2023-11-27T15:26:28.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.928+0000 7f0eeac31b80 1 --2- 172.21.15.52:0/606252670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e5ed6a470 0x564e5ed64aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.928+0000 7f0eeac31b80 1 --2- 172.21.15.52:0/606252670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e5ed5f360 0x564e5ed6d1a0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.938 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.928+0000 7f0eeac31b80 1 --2- 172.21.15.52:0/606252670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e5ed69b60 0x564e5ed64560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.928+0000 7f0eeac31b80 1 -- 172.21.15.52:0/606252670 >> 172.21.15.52:0/606252670 conn(0x564e5ed73c30 msgr2=0x564e5ed6dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:28.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.928+0000 7f0eeac31b80 1 -- 172.21.15.52:0/606252670 shutdown_connections 2023-11-27T15:26:28.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.928+0000 7f0eeac31b80 1 -- 172.21.15.52:0/606252670 wait complete. 2023-11-27T15:26:28.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.928+0000 7f0eeac31b80 1 -- start start 2023-11-27T15:26:28.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.928+0000 7f0eeac31b80 1 Processor -- start 2023-11-27T15:26:28.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.929+0000 7f0eeac31b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e5ed5f360 0x564e5ed76920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.929+0000 7f0eeac31b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e5ed69b60 0x564e5ed76e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.939 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.929+0000 7f0eeac31b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e5ed6a470 0x564e5ed773c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.929+0000 7f0eeac31b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e5edd6950 con 0x564e5ed5f360 2023-11-27T15:26:28.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.929+0000 7f0eeac31b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e5edd6ac0 con 0x564e5ed6a470 2023-11-27T15:26:28.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.929+0000 7f0eeac31b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e5edd6c80 con 0x564e5ed69b60 2023-11-27T15:26:28.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.929+0000 7f0eeac31b80 10 client.0 _ll_register_callbacks cb 0x564e5ec7a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:28.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.929+0000 7f0edc841700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e5ed5f360 0x564e5ed76920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.929+0000 7f0edc841700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e5ed5f360 0x564e5ed76920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55586/0 (socket says 172.21.15.52:55586) 2023-11-27T15:26:28.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.929+0000 7f0edc841700 1 -- 172.21.15.52:0/2811511094 learned_addr learned my addr 172.21.15.52:0/2811511094 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:28.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.929+0000 7f0ed7fff700 1 --2- 172.21.15.52:0/2811511094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e5ed69b60 0x564e5ed76e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.929+0000 7f0edd042700 1 --2- 172.21.15.52:0/2811511094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e5ed6a470 0x564e5ed773c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.929+0000 7f0edc841700 1 -- 172.21.15.52:0/2811511094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e5ed69b60 msgr2=0x564e5ed76e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.929+0000 7f0edc841700 1 --2- 172.21.15.52:0/2811511094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e5ed69b60 0x564e5ed76e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.929+0000 7f0edc841700 1 -- 172.21.15.52:0/2811511094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e5ed6a470 msgr2=0x564e5ed773c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:28.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.929+0000 7f0edc841700 1 --2- 172.21.15.52:0/2811511094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e5ed6a470 0x564e5ed773c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.929+0000 7f0edc841700 1 -- 172.21.15.52:0/2811511094 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e5eb3c630 con 0x564e5ed5f360 2023-11-27T15:26:28.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.929+0000 7f0ed7fff700 1 --2- 172.21.15.52:0/2811511094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e5ed69b60 0x564e5ed76e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:26:28.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.929+0000 7f0edc841700 1 --2- 172.21.15.52:0/2811511094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e5ed5f360 0x564e5ed76920 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc00bd50 tx=0x7f0ecc0055b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3d01970b77e94 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:28.942 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.929+0000 7f0eb7fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:26:28.942 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.929+0000 7f0eb7fff700 1 -- 172.21.15.52:0/2811511094 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc03cd10 con 0x564e5ed5f360 2023-11-27T15:26:28.942 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.929+0000 7f0eeac31b80 1 -- 172.21.15.52:0/2811511094 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ecc01e040 con 0x564e5ed5f360 2023-11-27T15:26:28.942 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.929+0000 7f0eb7fff700 1 -- 172.21.15.52:0/2811511094 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f0ecc003960 con 0x564e5ed5f360 2023-11-27T15:26:28.942 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.929+0000 7f0eb7fff700 1 -- 172.21.15.52:0/2811511094 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ecc03ce70 con 0x564e5ed5f360 2023-11-27T15:26:28.942 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.929+0000 7f0eeac31b80 10 client.16917 fetch_fsmap learned FSMap version 34 2023-11-27T15:26:28.942 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.930+0000 7f0eeac31b80 1 -- 172.21.15.52:0/2811511094 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x564e5ead1c60 con 0x564e5ed5f360 2023-11-27T15:26:28.942 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.930+0000 7f0eb7fff700 1 -- 172.21.15.52:0/2811511094 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ecc054020 con 0x564e5ed5f360 2023-11-27T15:26:28.942 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.930+0000 7f0eb7fff700 1 -- 172.21.15.52:0/2811511094 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0ecc045070 con 0x564e5ed5f360 2023-11-27T15:26:28.943 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.930+0000 7f0eeac31b80 10 client.16917 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:26:28.943 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.930+0000 7f0eeac31b80 10 client.16917 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:26:28.943 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.930+0000 7f0eeac31b80 20 client.16917 populate_metadata read hostname 'smithi052' 2023-11-27T15:26:28.943 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.930+0000 7f0eeac31b80 1 --2- 172.21.15.52:0/2811511094 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x564e5ed6e8b0 0x564e5edd9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:28.943 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.930+0000 7f0eeac31b80 4 client.16917 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:26:28.943 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.930+0000 7f0eeac31b80 1 -- 172.21.15.52:0/2811511094 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564e5ec89660 con 0x564e5ed6e8b0 2023-11-27T15:26:28.943 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.930+0000 7f0ed7fff700 1 --2- 172.21.15.52:0/2811511094 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x564e5ed6e8b0 0x564e5edd9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:28.943 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.931+0000 7f0eb7fff700 10 client.16917 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:26:28.944 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.931+0000 7f0ed7fff700 1 --2- 172.21.15.52:0/2811511094 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x564e5ed6e8b0 0x564e5edd9ee0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e321e6de00f4114a server_cookie=f6e10b4785d6a88a in_seq=0 out_seq=0 2023-11-27T15:26:28.944 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.931+0000 7f0eb7fff700 1 -- 172.21.15.52:0/2811511094 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564e5ec89660 con 0x564e5ed6e8b0 2023-11-27T15:26:28.944 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.931+0000 7f0eb7fff700 10 client.16917 handle_command_reply: tid=0 2023-11-27T15:26:28.944 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.931+0000 7f0eeac31b80 1 client.16917 shutdown 2023-11-27T15:26:28.944 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.931+0000 7f0eeac31b80 1 -- 172.21.15.52:0/2811511094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e5ed5f360 msgr2=0x564e5ed76920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.944 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.931+0000 7f0eeac31b80 1 --2- 172.21.15.52:0/2811511094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e5ed5f360 0x564e5ed76920 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc00bd50 tx=0x7f0ecc0055b0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.944 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.931+0000 7f0eeac31b80 1 -- 172.21.15.52:0/2811511094 shutdown_connections 2023-11-27T15:26:28.944 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.931+0000 7f0eeac31b80 1 --2- 172.21.15.52:0/2811511094 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x564e5ed6e8b0 0x564e5edd9ee0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.945 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.931+0000 7f0eeac31b80 1 --2- 172.21.15.52:0/2811511094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e5ed6a470 0x564e5ed773c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.945 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.931+0000 7f0eb7fff700 0 client.16917 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:26:28.945 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.931+0000 7f0eeac31b80 1 --2- 172.21.15.52:0/2811511094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e5ed5f360 0x564e5ed76920 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.945 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.931+0000 7f0eeac31b80 1 --2- 172.21.15.52:0/2811511094 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x564e5ed69b60 0x564e5ed76e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.945 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.931+0000 7f0eeac31b80 1 -- 172.21.15.52:0/2811511094 >> 172.21.15.52:0/2811511094 conn(0x564e5ed73c30 msgr2=0x564e5ed68960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:28.945 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.931+0000 7f0eeac31b80 1 -- 172.21.15.52:0/2811511094 shutdown_connections 2023-11-27T15:26:28.945 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.931+0000 7f0eeac31b80 1 -- 172.21.15.52:0/2811511094 wait complete. 2023-11-27T15:26:28.945 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.931+0000 7f0eeac31b80 20 client.16917 trim_cache size 0 max 16384 2023-11-27T15:26:28.946 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:26:28.946 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.932+0000 7f0edeaa5700 1 -- 172.21.15.52:0/3955771437 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f0ec80a1340 msgr2=0x7f0ec80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.932+0000 7f0edeaa5700 1 --2- 172.21.15.52:0/3955771437 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f0ec80a1340 0x7f0ec80a37f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc006010 tx=0x7f0ecc009f10 comp rx=0 tx=0).stop 2023-11-27T15:26:28.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.932+0000 7f0edeaa5700 1 -- 172.21.15.52:0/3955771437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ed81055f0 msgr2=0x7f0ed819aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:28.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.932+0000 7f0edeaa5700 1 --2- 172.21.15.52:0/3955771437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ed81055f0 0x7f0ed819aea0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000ada0 tx=0x7f0ed000c330 comp rx=0 tx=0).stop 2023-11-27T15:26:28.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.932+0000 7f0edeaa5700 1 -- 172.21.15.52:0/3955771437 shutdown_connections 2023-11-27T15:26:28.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.932+0000 7f0edeaa5700 1 --2- 172.21.15.52:0/3955771437 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f0ec80a1340 0x7f0ec80a37f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.932+0000 7f0edeaa5700 1 --2- 172.21.15.52:0/3955771437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ed81005b0 0x7f0ed8196500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.947 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.932+0000 7f0edeaa5700 1 --2- 172.21.15.52:0/3955771437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ed81055f0 0x7f0ed819aea0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.948 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.932+0000 7f0edeaa5700 1 --2- 172.21.15.52:0/3955771437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ed8100f50 0x7f0ed8196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:28.948 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.932+0000 7f0edeaa5700 1 -- 172.21.15.52:0/3955771437 >> 172.21.15.52:0/3955771437 conn(0x7f0ed8077170 msgr2=0x7f0ed80fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:28.948 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.932+0000 7f0edeaa5700 1 -- 172.21.15.52:0/3955771437 shutdown_connections 2023-11-27T15:26:28.948 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:28.932+0000 7f0edeaa5700 1 -- 172.21.15.52:0/3955771437 wait complete. 2023-11-27T15:26:29.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:29 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3319095591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:26:29.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:29 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3319095591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:26:29.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:29 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3319095591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:26:29.949 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-27T15:26:29.950 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-27T15:26:30.097 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.091+0000 7f745c266700 1 -- 172.21.15.52:0/879902922 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7454100cc0 msgr2=0x7f7454101090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:30.098 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.091+0000 7f745c266700 1 --2- 172.21.15.52:0/879902922 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7454100cc0 0x7f7454101090 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f74440099c0 tx=0x7f7444009cd0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.098 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.092+0000 7f745c266700 1 -- 172.21.15.52:0/879902922 shutdown_connections 2023-11-27T15:26:30.098 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.092+0000 7f745c266700 1 --2- 172.21.15.52:0/879902922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7454105620 0x7f745410d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.098 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.092+0000 7f745c266700 1 --2- 172.21.15.52:0/879902922 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7454100cc0 0x7f7454101090 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.098 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.092+0000 7f745c266700 1 --2- 172.21.15.52:0/879902922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7454101660 0x7f7454104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.099 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.092+0000 7f745c266700 1 -- 172.21.15.52:0/879902922 >> 172.21.15.52:0/879902922 conn(0x7f74540775c0 msgr2=0x7f74540779c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:30.099 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.092+0000 7f745c266700 1 -- 172.21.15.52:0/879902922 shutdown_connections 2023-11-27T15:26:30.099 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.092+0000 7f745c266700 1 -- 172.21.15.52:0/879902922 wait complete. 2023-11-27T15:26:30.099 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.092+0000 7f745c266700 1 Processor -- start 2023-11-27T15:26:30.099 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.092+0000 7f745c266700 1 -- start start 2023-11-27T15:26:30.099 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.092+0000 7f745c266700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7454100cc0 0x7f7454072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.099 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.093+0000 7f745c266700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7454101660 0x7f7454073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.099 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.093+0000 7f745c266700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7454105620 0x7f74541a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.093+0000 7f745c266700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7454073bd0 con 0x7f7454100cc0 2023-11-27T15:26:30.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.093+0000 7f745c266700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74541a4b50 con 0x7f7454105620 2023-11-27T15:26:30.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.093+0000 7f745c266700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74541a4c90 con 0x7f7454101660 2023-11-27T15:26:30.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.093+0000 7f745a002700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7454100cc0 0x7f7454072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.093+0000 7f745a002700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7454100cc0 0x7f7454072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55600/0 (socket says 172.21.15.52:55600) 2023-11-27T15:26:30.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.093+0000 7f745a002700 1 -- 172.21.15.52:0/263186737 learned_addr learned my addr 172.21.15.52:0/263186737 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:30.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.093+0000 7f745a803700 1 --2- 172.21.15.52:0/263186737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7454105620 0x7f74541a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.093+0000 7f7459801700 1 --2- 172.21.15.52:0/263186737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7454101660 0x7f7454073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.093+0000 7f745a002700 1 -- 172.21.15.52:0/263186737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7454101660 msgr2=0x7f7454073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:30.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.093+0000 7f745a002700 1 --2- 172.21.15.52:0/263186737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7454101660 0x7f7454073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.093+0000 7f745a002700 1 -- 172.21.15.52:0/263186737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7454105620 msgr2=0x7f74541a4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:30.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.093+0000 7f745a002700 1 --2- 172.21.15.52:0/263186737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7454105620 0x7f74541a4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.093+0000 7f745a002700 1 -- 172.21.15.52:0/263186737 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74440095f0 con 0x7f7454100cc0 2023-11-27T15:26:30.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.093+0000 7f745a002700 1 --2- 172.21.15.52:0/263186737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7454100cc0 0x7f7454072df0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f7444004740 tx=0x7f7444004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:30.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.093+0000 7f744affd700 1 -- 172.21.15.52:0/263186737 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744401d070 con 0x7f7454100cc0 2023-11-27T15:26:30.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.093+0000 7f744affd700 1 -- 172.21.15.52:0/263186737 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7444004170 con 0x7f7454100cc0 2023-11-27T15:26:30.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.093+0000 7f744affd700 1 -- 172.21.15.52:0/263186737 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744400f460 con 0x7f7454100cc0 2023-11-27T15:26:30.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.093+0000 7f745c266700 1 -- 172.21.15.52:0/263186737 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74541103e0 con 0x7f7454100cc0 2023-11-27T15:26:30.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.093+0000 7f745c266700 1 -- 172.21.15.52:0/263186737 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7454110250 con 0x7f7454100cc0 2023-11-27T15:26:30.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.096+0000 7f744affd700 1 -- 172.21.15.52:0/263186737 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f74440226e0 con 0x7f7454100cc0 2023-11-27T15:26:30.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.096+0000 7f744affd700 1 --2- 172.21.15.52:0/263186737 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f74400a12c0 0x7f74400a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.096+0000 7f744affd700 1 -- 172.21.15.52:0/263186737 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f74440cdff0 con 0x7f7454100cc0 2023-11-27T15:26:30.109 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.096+0000 7f7459801700 1 --2- 172.21.15.52:0/263186737 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f74400a12c0 0x7f74400a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.097+0000 7f7459801700 1 --2- 172.21.15.52:0/263186737 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f74400a12c0 0x7f74400a3770 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f74540745e0 tx=0x7f7450005d50 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:30.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.097+0000 7f74683f2b80 1 Processor -- start 2023-11-27T15:26:30.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.097+0000 7f74683f2b80 1 -- start start 2023-11-27T15:26:30.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.098+0000 7f74683f2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56244cd11be0 0x56244cd11fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.098+0000 7f74683f2b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56244cd124f0 0x56244cd163f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.098+0000 7f74683f2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56244cd16930 0x56244cd16d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.110 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.098+0000 7f74683f2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244cc170e0 con 0x56244cd11be0 2023-11-27T15:26:30.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.098+0000 7f74683f2b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244cc69920 con 0x56244cd124f0 2023-11-27T15:26:30.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.098+0000 7f74683f2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244cc62db0 con 0x56244cd16930 2023-11-27T15:26:30.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.098+0000 7f745a002700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56244cd11be0 0x56244cd11fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.098+0000 7f745a002700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56244cd11be0 0x56244cd11fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55608/0 (socket says 172.21.15.52:55608) 2023-11-27T15:26:30.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.098+0000 7f745a002700 1 -- 172.21.15.52:0/400456690 learned_addr learned my addr 172.21.15.52:0/400456690 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:30.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.098+0000 7f7459801700 1 --2- 172.21.15.52:0/400456690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56244cd124f0 0x56244cd163f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.098+0000 7f745a803700 1 --2- 172.21.15.52:0/400456690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56244cd16930 0x56244cd16d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.098+0000 7f745a002700 1 -- 172.21.15.52:0/400456690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56244cd16930 msgr2=0x56244cd16d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:30.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.098+0000 7f745a002700 1 --2- 172.21.15.52:0/400456690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56244cd16930 0x56244cd16d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.098+0000 7f745a002700 1 -- 172.21.15.52:0/400456690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56244cd124f0 msgr2=0x56244cd163f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:30.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.098+0000 7f745a002700 1 --2- 172.21.15.52:0/400456690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56244cd124f0 0x56244cd163f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.098+0000 7f745a002700 1 -- 172.21.15.52:0/400456690 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56244cae46b0 con 0x56244cd11be0 2023-11-27T15:26:30.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.098+0000 7f745a803700 1 --2- 172.21.15.52:0/400456690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56244cd16930 0x56244cd16d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:26:30.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.098+0000 7f7459801700 1 --2- 172.21.15.52:0/400456690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56244cd124f0 0x56244cd163f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:26:30.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.098+0000 7f745a002700 1 --2- 172.21.15.52:0/400456690 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56244cd11be0 0x56244cd11fb0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f7444097b90 tx=0x7f7444009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce81050336df602d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:30.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.098+0000 7f743ffff700 1 -- 172.21.15.52:0/400456690 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74440a34e0 con 0x56244cd11be0 2023-11-27T15:26:30.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.098+0000 7f743ffff700 1 -- 172.21.15.52:0/400456690 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74440994c0 con 0x56244cd11be0 2023-11-27T15:26:30.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.098+0000 7f743ffff700 1 -- 172.21.15.52:0/400456690 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74440a3050 con 0x56244cd11be0 2023-11-27T15:26:30.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.099+0000 7f74683f2b80 1 -- 172.21.15.52:0/400456690 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56244cd11be0 msgr2=0x56244cd11fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:30.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.099+0000 7f74683f2b80 1 --2- 172.21.15.52:0/400456690 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56244cd11be0 0x56244cd11fb0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f7444097b90 tx=0x7f7444009310 comp rx=0 tx=0).stop 2023-11-27T15:26:30.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.099+0000 7f74683f2b80 1 -- 172.21.15.52:0/400456690 shutdown_connections 2023-11-27T15:26:30.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.099+0000 7f74683f2b80 1 --2- 172.21.15.52:0/400456690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56244cd16930 0x56244cd16d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.099+0000 7f74683f2b80 1 --2- 172.21.15.52:0/400456690 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56244cd11be0 0x56244cd11fb0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.099+0000 7f74683f2b80 1 --2- 172.21.15.52:0/400456690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56244cd124f0 0x56244cd163f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.099+0000 7f74683f2b80 1 -- 172.21.15.52:0/400456690 >> 172.21.15.52:0/400456690 conn(0x56244cd073e0 msgr2=0x56244cd0cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:30.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.099+0000 7f74683f2b80 1 -- 172.21.15.52:0/400456690 shutdown_connections 2023-11-27T15:26:30.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.099+0000 7f74683f2b80 1 -- 172.21.15.52:0/400456690 wait complete. 2023-11-27T15:26:30.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.100+0000 7f74683f2b80 1 -- start start 2023-11-27T15:26:30.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.100+0000 7f74683f2b80 1 Processor -- start 2023-11-27T15:26:30.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.100+0000 7f74683f2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56244cd11be0 0x56244cd1e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.100+0000 7f74683f2b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56244cd124f0 0x56244cd1b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.100+0000 7f74683f2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56244cd16930 0x56244cd1b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.100+0000 7f74683f2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244cc04960 con 0x56244cd11be0 2023-11-27T15:26:30.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.100+0000 7f74683f2b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244cc04600 con 0x56244cd124f0 2023-11-27T15:26:30.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.100+0000 7f74683f2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244cae69a0 con 0x56244cd16930 2023-11-27T15:26:30.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.100+0000 7f74683f2b80 10 client.0 _ll_register_callbacks cb 0x56244cc22800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:30.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.100+0000 7f745a002700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56244cd11be0 0x56244cd1e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.100+0000 7f745a002700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56244cd11be0 0x56244cd1e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55614/0 (socket says 172.21.15.52:55614) 2023-11-27T15:26:30.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.100+0000 7f745a002700 1 -- 172.21.15.52:0/3686270196 learned_addr learned my addr 172.21.15.52:0/3686270196 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:30.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.101+0000 7f7459801700 1 --2- 172.21.15.52:0/3686270196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56244cd124f0 0x56244cd1b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.101+0000 7f745a803700 1 --2- 172.21.15.52:0/3686270196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56244cd16930 0x56244cd1b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.101+0000 7f745a002700 1 -- 172.21.15.52:0/3686270196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56244cd16930 msgr2=0x56244cd1b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:30.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.101+0000 7f745a002700 1 --2- 172.21.15.52:0/3686270196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56244cd16930 0x56244cd1b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.101+0000 7f745a002700 1 -- 172.21.15.52:0/3686270196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56244cd124f0 msgr2=0x56244cd1b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:30.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.101+0000 7f745a002700 1 --2- 172.21.15.52:0/3686270196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56244cd124f0 0x56244cd1b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.101+0000 7f745a002700 1 -- 172.21.15.52:0/3686270196 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7454110250 con 0x56244cd11be0 2023-11-27T15:26:30.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.101+0000 7f7459801700 1 --2- 172.21.15.52:0/3686270196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56244cd124f0 0x56244cd1b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:26:30.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.101+0000 7f745a002700 1 --2- 172.21.15.52:0/3686270196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56244cd11be0 0x56244cd1e080 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f74440cb7c0 tx=0x7f7444009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7c0657c71b4b8c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:30.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.101+0000 7f743d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:26:30.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.101+0000 7f743d7fa700 1 -- 172.21.15.52:0/3686270196 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74440a3050 con 0x56244cd11be0 2023-11-27T15:26:30.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.101+0000 7f743d7fa700 1 -- 172.21.15.52:0/3686270196 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f74440a31b0 con 0x56244cd11be0 2023-11-27T15:26:30.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.101+0000 7f74683f2b80 1 -- 172.21.15.52:0/3686270196 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56244cd04ba0 con 0x56244cd11be0 2023-11-27T15:26:30.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.101+0000 7f743d7fa700 1 -- 172.21.15.52:0/3686270196 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7444003cb0 con 0x56244cd11be0 2023-11-27T15:26:30.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.101+0000 7f74683f2b80 10 client.16929 fetch_fsmap learned FSMap version 34 2023-11-27T15:26:30.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.101+0000 7f74683f2b80 1 -- 172.21.15.52:0/3686270196 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56244cc29210 con 0x56244cd11be0 2023-11-27T15:26:30.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.101+0000 7f743d7fa700 1 -- 172.21.15.52:0/3686270196 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74440b9020 con 0x56244cd11be0 2023-11-27T15:26:30.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.102+0000 7f743d7fa700 1 -- 172.21.15.52:0/3686270196 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f744409a080 con 0x56244cd11be0 2023-11-27T15:26:30.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.102+0000 7f74683f2b80 10 client.16929 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:26:30.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.102+0000 7f74683f2b80 10 client.16929 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:26:30.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.102+0000 7f74683f2b80 20 client.16929 populate_metadata read hostname 'smithi052' 2023-11-27T15:26:30.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.102+0000 7f74683f2b80 1 --2- 172.21.15.52:0/3686270196 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x56244cd2b0c0 0x56244cd2d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.102+0000 7f74683f2b80 4 client.16929 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:26:30.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.102+0000 7f74683f2b80 1 -- 172.21.15.52:0/3686270196 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56244cc25120 con 0x56244cd2b0c0 2023-11-27T15:26:30.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.102+0000 7f7459801700 1 --2- 172.21.15.52:0/3686270196 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x56244cd2b0c0 0x56244cd2d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.102+0000 7f743d7fa700 10 client.16929 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:26:30.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.102+0000 7f7459801700 1 --2- 172.21.15.52:0/3686270196 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x56244cd2b0c0 0x56244cd2d4d0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a12326d1912619d server_cookie=26f05bde407dd369 in_seq=0 out_seq=0 2023-11-27T15:26:30.120 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.103+0000 7f743d7fa700 1 -- 172.21.15.52:0/3686270196 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56244cc25120 con 0x56244cd2b0c0 2023-11-27T15:26:30.120 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.103+0000 7f743d7fa700 10 client.16929 handle_command_reply: tid=0 2023-11-27T15:26:30.120 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.103+0000 7f74683f2b80 1 client.16929 shutdown 2023-11-27T15:26:30.120 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.103+0000 7f74683f2b80 1 -- 172.21.15.52:0/3686270196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56244cd11be0 msgr2=0x56244cd1e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:30.120 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.103+0000 7f74683f2b80 1 --2- 172.21.15.52:0/3686270196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56244cd11be0 0x56244cd1e080 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f74440cb7c0 tx=0x7f7444009cd0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.120 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.103+0000 7f74683f2b80 1 -- 172.21.15.52:0/3686270196 shutdown_connections 2023-11-27T15:26:30.120 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.103+0000 7f74683f2b80 1 --2- 172.21.15.52:0/3686270196 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x56244cd2b0c0 0x56244cd2d4d0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.120 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.103+0000 7f74683f2b80 1 --2- 172.21.15.52:0/3686270196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56244cd16930 0x56244cd1b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.121 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.103+0000 7f74683f2b80 1 --2- 172.21.15.52:0/3686270196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56244cd11be0 0x56244cd1e080 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.121 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.103+0000 7f743d7fa700 0 client.16929 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:26:30.121 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.103+0000 7f74683f2b80 1 --2- 172.21.15.52:0/3686270196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56244cd124f0 0x56244cd1b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.121 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.103+0000 7f74683f2b80 1 -- 172.21.15.52:0/3686270196 >> 172.21.15.52:0/3686270196 conn(0x56244cd073e0 msgr2=0x56244cd17de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:30.121 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.103+0000 7f74683f2b80 1 -- 172.21.15.52:0/3686270196 shutdown_connections 2023-11-27T15:26:30.121 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.103+0000 7f74683f2b80 1 -- 172.21.15.52:0/3686270196 wait complete. 2023-11-27T15:26:30.121 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.103+0000 7f74683f2b80 20 client.16929 trim_cache size 0 max 16384 2023-11-27T15:26:30.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.120+0000 7f74683f2b80 1 Processor -- start 2023-11-27T15:26:30.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.120+0000 7f74683f2b80 1 -- start start 2023-11-27T15:26:30.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.121+0000 7f74683f2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56244cd11be0 0x56244cd22830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.121+0000 7f74683f2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56244cd124f0 0x56244cd1dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.121+0000 7f74683f2b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56244cd16930 0x56244cd1ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.121+0000 7f74683f2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244cd1c720 con 0x56244cd124f0 2023-11-27T15:26:30.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.121+0000 7f74683f2b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244cd27660 con 0x56244cd16930 2023-11-27T15:26:30.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.121+0000 7f74683f2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244cd15440 con 0x56244cd11be0 2023-11-27T15:26:30.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.121+0000 7f7459801700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56244cd124f0 0x56244cd1dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.121+0000 7f7459801700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56244cd124f0 0x56244cd1dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55624/0 (socket says 172.21.15.52:55624) 2023-11-27T15:26:30.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.121+0000 7f7459801700 1 -- 172.21.15.52:0/1744719771 learned_addr learned my addr 172.21.15.52:0/1744719771 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:30.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.121+0000 7f745a002700 1 --2- 172.21.15.52:0/1744719771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56244cd11be0 0x56244cd22830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.121+0000 7f745a803700 1 --2- 172.21.15.52:0/1744719771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56244cd16930 0x56244cd1ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.121+0000 7f7459801700 1 -- 172.21.15.52:0/1744719771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56244cd11be0 msgr2=0x56244cd22830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:30.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.121+0000 7f7459801700 1 --2- 172.21.15.52:0/1744719771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56244cd11be0 0x56244cd22830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.121+0000 7f7459801700 1 -- 172.21.15.52:0/1744719771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56244cd16930 msgr2=0x56244cd1ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:30.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.121+0000 7f7459801700 1 --2- 172.21.15.52:0/1744719771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56244cd16930 0x56244cd1ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.121+0000 7f7459801700 1 -- 172.21.15.52:0/1744719771 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56244cd16fd0 con 0x56244cd124f0 2023-11-27T15:26:30.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.121+0000 7f745a803700 1 --2- 172.21.15.52:0/1744719771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56244cd16930 0x56244cd1ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:26:30.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.121+0000 7f7459801700 1 --2- 172.21.15.52:0/1744719771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56244cd124f0 0x56244cd1dbb0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f7454101a30 tx=0x7f7450013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6854c01d8707b86 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:30.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.122+0000 7f7448ff9700 1 -- 172.21.15.52:0/1744719771 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745002b070 con 0x56244cd124f0 2023-11-27T15:26:30.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.122+0000 7f7448ff9700 1 -- 172.21.15.52:0/1744719771 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f745000ded0 con 0x56244cd124f0 2023-11-27T15:26:30.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.122+0000 7f7448ff9700 1 -- 172.21.15.52:0/1744719771 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7450023040 con 0x56244cd124f0 2023-11-27T15:26:30.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.122+0000 7f74683f2b80 1 -- 172.21.15.52:0/1744719771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56244cd124f0 msgr2=0x56244cd1dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:30.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.122+0000 7f74683f2b80 1 --2- 172.21.15.52:0/1744719771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56244cd124f0 0x56244cd1dbb0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f7454101a30 tx=0x7f7450013000 comp rx=0 tx=0).stop 2023-11-27T15:26:30.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.122+0000 7f74683f2b80 1 -- 172.21.15.52:0/1744719771 shutdown_connections 2023-11-27T15:26:30.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.122+0000 7f74683f2b80 1 --2- 172.21.15.52:0/1744719771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56244cd16930 0x56244cd1ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.122+0000 7f74683f2b80 1 --2- 172.21.15.52:0/1744719771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56244cd11be0 0x56244cd22830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.122+0000 7f74683f2b80 1 --2- 172.21.15.52:0/1744719771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56244cd124f0 0x56244cd1dbb0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.122+0000 7f74683f2b80 1 -- 172.21.15.52:0/1744719771 >> 172.21.15.52:0/1744719771 conn(0x56244cd073e0 msgr2=0x56244cd0c600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:30.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.122+0000 7f74683f2b80 1 -- 172.21.15.52:0/1744719771 shutdown_connections 2023-11-27T15:26:30.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.122+0000 7f74683f2b80 1 -- 172.21.15.52:0/1744719771 wait complete. 2023-11-27T15:26:30.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.122+0000 7f74683f2b80 1 -- start start 2023-11-27T15:26:30.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.123+0000 7f74683f2b80 1 Processor -- start 2023-11-27T15:26:30.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.123+0000 7f74683f2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56244cd11be0 0x56244cd1f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.123+0000 7f74683f2b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56244cd124f0 0x56244cd1fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.123+0000 7f74683f2b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56244cd16930 0x56244cd7ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.123+0000 7f74683f2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244cd20250 con 0x56244cd11be0 2023-11-27T15:26:30.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.123+0000 7f74683f2b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244cd7f4d0 con 0x56244cd124f0 2023-11-27T15:26:30.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.123+0000 7f74683f2b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56244cd7f690 con 0x56244cd16930 2023-11-27T15:26:30.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.123+0000 7f74683f2b80 10 client.0 _ll_register_callbacks cb 0x7f744409ad50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:30.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.123+0000 7f745a002700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56244cd11be0 0x56244cd1f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.123+0000 7f745a002700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56244cd11be0 0x56244cd1f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55628/0 (socket says 172.21.15.52:55628) 2023-11-27T15:26:30.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.123+0000 7f745a002700 1 -- 172.21.15.52:0/3703747563 learned_addr learned my addr 172.21.15.52:0/3703747563 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:30.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.123+0000 7f7459801700 1 --2- 172.21.15.52:0/3703747563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56244cd124f0 0x56244cd1fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.123+0000 7f745a803700 1 --2- 172.21.15.52:0/3703747563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56244cd16930 0x56244cd7ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.123+0000 7f745a002700 1 -- 172.21.15.52:0/3703747563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56244cd16930 msgr2=0x56244cd7ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:30.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.123+0000 7f745a002700 1 --2- 172.21.15.52:0/3703747563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56244cd16930 0x56244cd7ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.123+0000 7f745a002700 1 -- 172.21.15.52:0/3703747563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56244cd124f0 msgr2=0x56244cd1fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:30.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.123+0000 7f745a002700 1 --2- 172.21.15.52:0/3703747563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56244cd124f0 0x56244cd1fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.123+0000 7f745a002700 1 -- 172.21.15.52:0/3703747563 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f745000d820 con 0x56244cd11be0 2023-11-27T15:26:30.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.124+0000 7f745a002700 1 --2- 172.21.15.52:0/3703747563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56244cd11be0 0x56244cd1f570 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f744409a4e0 tx=0x7f744409b2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e8f03b6646cf7 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:30.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.124+0000 7f743d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:26:30.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.124+0000 7f743d7fa700 1 -- 172.21.15.52:0/3703747563 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74440a34e0 con 0x56244cd11be0 2023-11-27T15:26:30.137 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.124+0000 7f743d7fa700 1 -- 172.21.15.52:0/3703747563 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f74440b91b0 con 0x56244cd11be0 2023-11-27T15:26:30.137 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.124+0000 7f74683f2b80 1 -- 172.21.15.52:0/3703747563 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74440994c0 con 0x56244cd11be0 2023-11-27T15:26:30.137 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.124+0000 7f743d7fa700 1 -- 172.21.15.52:0/3703747563 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74440b9370 con 0x56244cd11be0 2023-11-27T15:26:30.137 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.124+0000 7f74683f2b80 10 client.16935 fetch_fsmap learned FSMap version 34 2023-11-27T15:26:30.137 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.124+0000 7f74683f2b80 1 -- 172.21.15.52:0/3703747563 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56244ca90e60 con 0x56244cd11be0 2023-11-27T15:26:30.137 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.124+0000 7f743d7fa700 1 -- 172.21.15.52:0/3703747563 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74440994c0 con 0x56244cd11be0 2023-11-27T15:26:30.137 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.124+0000 7f743d7fa700 1 -- 172.21.15.52:0/3703747563 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7444096f10 con 0x56244cd11be0 2023-11-27T15:26:30.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.124+0000 7f74683f2b80 10 client.16935 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:26:30.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.124+0000 7f74683f2b80 10 client.16935 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:26:30.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.124+0000 7f74683f2b80 20 client.16935 populate_metadata read hostname 'smithi052' 2023-11-27T15:26:30.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.124+0000 7f74683f2b80 1 --2- 172.21.15.52:0/3703747563 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x56244cd80900 0x56244cd82ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.124+0000 7f74683f2b80 4 client.16935 mds_command: new command op to 24536 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-27T15:26:30.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.124+0000 7f74683f2b80 1 -- 172.21.15.52:0/3703747563 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56244cc316e0 con 0x56244cd80900 2023-11-27T15:26:30.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.125+0000 7f7459801700 1 --2- 172.21.15.52:0/3703747563 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x56244cd80900 0x56244cd82ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.125+0000 7f7459801700 1 --2- 172.21.15.52:0/3703747563 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x56244cd80900 0x56244cd82ce0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0aeb97bdb6e8235 server_cookie=a94cd921c355a308 in_seq=0 out_seq=0 2023-11-27T15:26:30.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.125+0000 7f743d7fa700 10 client.16935 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:26:30.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.126+0000 7f743d7fa700 1 -- 172.21.15.52:0/3703747563 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56244cc316e0 con 0x56244cd80900 2023-11-27T15:26:30.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.126+0000 7f743d7fa700 10 client.16935 handle_command_reply: tid=0 2023-11-27T15:26:30.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.126+0000 7f74683f2b80 1 client.16935 shutdown 2023-11-27T15:26:30.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.126+0000 7f74683f2b80 1 -- 172.21.15.52:0/3703747563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56244cd11be0 msgr2=0x56244cd1f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:30.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.126+0000 7f74683f2b80 1 --2- 172.21.15.52:0/3703747563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56244cd11be0 0x56244cd1f570 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f744409a4e0 tx=0x7f744409b2a0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.126+0000 7f74683f2b80 1 -- 172.21.15.52:0/3703747563 shutdown_connections 2023-11-27T15:26:30.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.126+0000 7f74683f2b80 1 --2- 172.21.15.52:0/3703747563 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x56244cd80900 0x56244cd82ce0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.126+0000 7f74683f2b80 1 --2- 172.21.15.52:0/3703747563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56244cd16930 0x56244cd7ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.126+0000 7f74683f2b80 1 --2- 172.21.15.52:0/3703747563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56244cd11be0 0x56244cd1f570 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.126+0000 7f743d7fa700 0 client.16935 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:26:30.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.126+0000 7f74683f2b80 1 --2- 172.21.15.52:0/3703747563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56244cd124f0 0x56244cd1fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.126+0000 7f74683f2b80 1 -- 172.21.15.52:0/3703747563 >> 172.21.15.52:0/3703747563 conn(0x56244cd073e0 msgr2=0x56244cd1ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:30.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.126+0000 7f74683f2b80 1 -- 172.21.15.52:0/3703747563 shutdown_connections 2023-11-27T15:26:30.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.126+0000 7f74683f2b80 1 -- 172.21.15.52:0/3703747563 wait complete. 2023-11-27T15:26:30.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.127+0000 7f74683f2b80 20 client.16935 trim_cache size 0 max 16384 2023-11-27T15:26:30.142 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:26:30.143 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2023-11-27T15:26:30.143 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "45ba5e98-8a31-4a1a-bacd-7a8ec84a9c00", 2023-11-27T15:26:30.143 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2023-11-27T15:26:30.143 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:26:30.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.127+0000 7f745c266700 1 -- 172.21.15.52:0/263186737 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f74400a12c0 msgr2=0x7f74400a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:30.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.127+0000 7f745c266700 1 --2- 172.21.15.52:0/263186737 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f74400a12c0 0x7f74400a3770 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f74540745e0 tx=0x7f7450005d50 comp rx=0 tx=0).stop 2023-11-27T15:26:30.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.127+0000 7f745c266700 1 -- 172.21.15.52:0/263186737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7454100cc0 msgr2=0x7f7454072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:30.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.127+0000 7f745c266700 1 --2- 172.21.15.52:0/263186737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7454100cc0 0x7f7454072df0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f7444004740 tx=0x7f7444004770 comp rx=0 tx=0).stop 2023-11-27T15:26:30.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.127+0000 7f745c266700 1 -- 172.21.15.52:0/263186737 shutdown_connections 2023-11-27T15:26:30.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.127+0000 7f745c266700 1 --2- 172.21.15.52:0/263186737 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f74400a12c0 0x7f74400a3770 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.127+0000 7f745c266700 1 --2- 172.21.15.52:0/263186737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7454105620 0x7f74541a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.127+0000 7f745c266700 1 --2- 172.21.15.52:0/263186737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7454100cc0 0x7f7454072df0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.145 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.127+0000 7f745c266700 1 --2- 172.21.15.52:0/263186737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7454101660 0x7f7454073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.146 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.127+0000 7f745c266700 1 -- 172.21.15.52:0/263186737 >> 172.21.15.52:0/263186737 conn(0x7f74540775c0 msgr2=0x7f74540fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:30.146 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.127+0000 7f745c266700 1 -- 172.21.15.52:0/263186737 shutdown_connections 2023-11-27T15:26:30.146 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.127+0000 7f745c266700 1 -- 172.21.15.52:0/263186737 wait complete. 2023-11-27T15:26:30.146 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:26:30.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.270+0000 7fc6f7d5c700 1 -- 172.21.15.52:0/4045861692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6f01005d0 msgr2=0x7fc6f0103e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:30.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.270+0000 7fc6f7d5c700 1 --2- 172.21.15.52:0/4045861692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6f01005d0 0x7fc6f0103e60 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fc6e40099c0 tx=0x7fc6e4009cd0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.270+0000 7fc6f7d5c700 1 -- 172.21.15.52:0/4045861692 shutdown_connections 2023-11-27T15:26:30.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.270+0000 7fc6f7d5c700 1 --2- 172.21.15.52:0/4045861692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6f0104590 0x7fc6f010c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.270+0000 7fc6f7d5c700 1 --2- 172.21.15.52:0/4045861692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6f00ffc30 0x7fc6f0100000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.270+0000 7fc6f7d5c700 1 --2- 172.21.15.52:0/4045861692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6f01005d0 0x7fc6f0103e60 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.270+0000 7fc6f7d5c700 1 -- 172.21.15.52:0/4045861692 >> 172.21.15.52:0/4045861692 conn(0x7fc6f0075470 msgr2=0x7fc6f0075870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:30.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.270+0000 7fc6f7d5c700 1 -- 172.21.15.52:0/4045861692 shutdown_connections 2023-11-27T15:26:30.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.270+0000 7fc6f7d5c700 1 -- 172.21.15.52:0/4045861692 wait complete. 2023-11-27T15:26:30.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.271+0000 7fc6f7d5c700 1 Processor -- start 2023-11-27T15:26:30.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.271+0000 7fc6f7d5c700 1 -- start start 2023-11-27T15:26:30.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.271+0000 7fc6f7d5c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6f00ffc30 0x7fc6f010ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.271+0000 7fc6f7d5c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6f01005d0 0x7fc6f010cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.278 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.271+0000 7fc6f7d5c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6f0104590 0x7fc6f01a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.271+0000 7fc6f7d5c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f010d790 con 0x7fc6f0104590 2023-11-27T15:26:30.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.271+0000 7fc6f7d5c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f01a4b40 con 0x7fc6f01005d0 2023-11-27T15:26:30.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.271+0000 7fc6f7d5c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f01a4c80 con 0x7fc6f00ffc30 2023-11-27T15:26:30.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.271+0000 7fc6f62f9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6f0104590 0x7fc6f01a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.271+0000 7fc6f62f9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6f0104590 0x7fc6f01a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55644/0 (socket says 172.21.15.52:55644) 2023-11-27T15:26:30.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.271+0000 7fc6f62f9700 1 -- 172.21.15.52:0/1866579912 learned_addr learned my addr 172.21.15.52:0/1866579912 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:30.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.272+0000 7fc6f5af8700 1 --2- 172.21.15.52:0/1866579912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6f00ffc30 0x7fc6f010ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.272+0000 7fc6f62f9700 1 -- 172.21.15.52:0/1866579912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6f00ffc30 msgr2=0x7fc6f010ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:30.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.272+0000 7fc6f62f9700 1 --2- 172.21.15.52:0/1866579912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6f00ffc30 0x7fc6f010ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.272+0000 7fc6f62f9700 1 -- 172.21.15.52:0/1866579912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6f01005d0 msgr2=0x7fc6f010cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:30.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.272+0000 7fc6f62f9700 1 --2- 172.21.15.52:0/1866579912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6f01005d0 0x7fc6f010cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.272+0000 7fc6f62f9700 1 -- 172.21.15.52:0/1866579912 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e40095f0 con 0x7fc6f0104590 2023-11-27T15:26:30.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.272+0000 7fc6f5af8700 1 --2- 172.21.15.52:0/1866579912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6f00ffc30 0x7fc6f010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:26:30.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.272+0000 7fc6f62f9700 1 --2- 172.21.15.52:0/1866579912 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6f0104590 0x7fc6f01a4600 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fc6e800ef30 tx=0x7fc6e800c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:30.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.272+0000 7fc6e2ffd700 1 -- 172.21.15.52:0/1866579912 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8018070 con 0x7fc6f0104590 2023-11-27T15:26:30.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.272+0000 7fc6e2ffd700 1 -- 172.21.15.52:0/1866579912 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6e800f040 con 0x7fc6f0104590 2023-11-27T15:26:30.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.272+0000 7fc6f7d5c700 1 -- 172.21.15.52:0/1866579912 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f0110470 con 0x7fc6f0104590 2023-11-27T15:26:30.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.272+0000 7fc6e2ffd700 1 -- 172.21.15.52:0/1866579912 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8003dc0 con 0x7fc6f0104590 2023-11-27T15:26:30.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.272+0000 7fc6f7d5c700 1 -- 172.21.15.52:0/1866579912 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f01102e0 con 0x7fc6f0104590 2023-11-27T15:26:30.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.274+0000 7fc6e2ffd700 1 -- 172.21.15.52:0/1866579912 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fc6e800caa0 con 0x7fc6f0104590 2023-11-27T15:26:30.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.275+0000 7fc6e2ffd700 1 --2- 172.21.15.52:0/1866579912 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fc6dc0a1340 0x7fc6dc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.275+0000 7fc6e2ffd700 1 -- 172.21.15.52:0/1866579912 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fc6e80cc920 con 0x7fc6f0104590 2023-11-27T15:26:30.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.275+0000 7fc6f5af8700 1 --2- 172.21.15.52:0/1866579912 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fc6dc0a1340 0x7fc6dc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.275+0000 7fc6f5af8700 1 --2- 172.21.15.52:0/1866579912 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fc6dc0a1340 0x7fc6dc0a37f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec005950 tx=0x7fc6ec005f50 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:30.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.276+0000 7fc703ee8b80 1 Processor -- start 2023-11-27T15:26:30.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.276+0000 7fc703ee8b80 1 -- start start 2023-11-27T15:26:30.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.276+0000 7fc703ee8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf8567cb60 0x55bf8567cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.276+0000 7fc703ee8b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8567d470 0x55bf85681370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.276+0000 7fc703ee8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf856818b0 0x55bf85681d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.276+0000 7fc703ee8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf85582060 con 0x55bf8567cb60 2023-11-27T15:26:30.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.276+0000 7fc703ee8b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf855d48a0 con 0x55bf8567d470 2023-11-27T15:26:30.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.276+0000 7fc703ee8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf855cdd30 con 0x55bf856818b0 2023-11-27T15:26:30.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.276+0000 7fc6f5af8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf8567cb60 0x55bf8567cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.276+0000 7fc6f5af8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf8567cb60 0x55bf8567cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55650/0 (socket says 172.21.15.52:55650) 2023-11-27T15:26:30.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.276+0000 7fc6f5af8700 1 -- 172.21.15.52:0/755074559 learned_addr learned my addr 172.21.15.52:0/755074559 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:30.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.276+0000 7fc6f52f7700 1 --2- 172.21.15.52:0/755074559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8567d470 0x55bf85681370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.277+0000 7fc6f5af8700 1 -- 172.21.15.52:0/755074559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf856818b0 msgr2=0x55bf85681d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:30.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.277+0000 7fc6f5af8700 1 --2- 172.21.15.52:0/755074559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf856818b0 0x55bf85681d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.277+0000 7fc6f5af8700 1 -- 172.21.15.52:0/755074559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8567d470 msgr2=0x55bf85681370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:30.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.277+0000 7fc6f5af8700 1 --2- 172.21.15.52:0/755074559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8567d470 0x55bf85681370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.277+0000 7fc6f5af8700 1 -- 172.21.15.52:0/755074559 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf8544f630 con 0x55bf8567cb60 2023-11-27T15:26:30.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.277+0000 7fc6f5af8700 1 --2- 172.21.15.52:0/755074559 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf8567cb60 0x55bf8567cf30 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec00ed30 tx=0x7fc6ec011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68328d177cd1826b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:30.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.277+0000 7fc6dbfff700 1 -- 172.21.15.52:0/755074559 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec010d80 con 0x55bf8567cb60 2023-11-27T15:26:30.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.277+0000 7fc6dbfff700 1 -- 172.21.15.52:0/755074559 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6ec004cf0 con 0x55bf8567cb60 2023-11-27T15:26:30.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.277+0000 7fc6dbfff700 1 -- 172.21.15.52:0/755074559 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec01d6b0 con 0x55bf8567cb60 2023-11-27T15:26:30.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.278+0000 7fc703ee8b80 1 -- 172.21.15.52:0/755074559 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf8567cb60 msgr2=0x55bf8567cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:30.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.278+0000 7fc703ee8b80 1 --2- 172.21.15.52:0/755074559 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf8567cb60 0x55bf8567cf30 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec00ed30 tx=0x7fc6ec011000 comp rx=0 tx=0).stop 2023-11-27T15:26:30.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.278+0000 7fc703ee8b80 1 -- 172.21.15.52:0/755074559 shutdown_connections 2023-11-27T15:26:30.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.278+0000 7fc703ee8b80 1 --2- 172.21.15.52:0/755074559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf856818b0 0x55bf85681d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.278+0000 7fc703ee8b80 1 --2- 172.21.15.52:0/755074559 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf8567cb60 0x55bf8567cf30 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.278+0000 7fc703ee8b80 1 --2- 172.21.15.52:0/755074559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8567d470 0x55bf85681370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.278+0000 7fc703ee8b80 1 -- 172.21.15.52:0/755074559 >> 172.21.15.52:0/755074559 conn(0x55bf85672360 msgr2=0x55bf85677b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:30.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.278+0000 7fc703ee8b80 1 -- 172.21.15.52:0/755074559 shutdown_connections 2023-11-27T15:26:30.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.278+0000 7fc703ee8b80 1 -- 172.21.15.52:0/755074559 wait complete. 2023-11-27T15:26:30.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.278+0000 7fc703ee8b80 1 -- start start 2023-11-27T15:26:30.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.278+0000 7fc703ee8b80 1 Processor -- start 2023-11-27T15:26:30.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.279+0000 7fc703ee8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf8567cb60 0x55bf856890d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.279+0000 7fc703ee8b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8567d470 0x55bf85686350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.279+0000 7fc703ee8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf856818b0 0x55bf85686a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.279+0000 7fc703ee8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf8556f8e0 con 0x55bf856818b0 2023-11-27T15:26:30.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.279+0000 7fc703ee8b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf8556f580 con 0x55bf8567d470 2023-11-27T15:26:30.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.279+0000 7fc703ee8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf85451920 con 0x55bf8567cb60 2023-11-27T15:26:30.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.279+0000 7fc703ee8b80 10 client.0 _ll_register_callbacks cb 0x55bf8558d780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:30.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.279+0000 7fc6f62f9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf856818b0 0x55bf85686a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.279+0000 7fc6f62f9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf856818b0 0x55bf85686a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55660/0 (socket says 172.21.15.52:55660) 2023-11-27T15:26:30.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.279+0000 7fc6f62f9700 1 -- 172.21.15.52:0/3244782516 learned_addr learned my addr 172.21.15.52:0/3244782516 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:30.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.279+0000 7fc6f52f7700 1 --2- 172.21.15.52:0/3244782516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8567d470 0x55bf85686350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.279+0000 7fc6f5af8700 1 --2- 172.21.15.52:0/3244782516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf8567cb60 0x55bf856890d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.279+0000 7fc6f62f9700 1 -- 172.21.15.52:0/3244782516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf8567cb60 msgr2=0x55bf856890d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:30.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.279+0000 7fc6f62f9700 1 --2- 172.21.15.52:0/3244782516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf8567cb60 0x55bf856890d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.279+0000 7fc6f62f9700 1 -- 172.21.15.52:0/3244782516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8567d470 msgr2=0x55bf85686350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:30.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.279+0000 7fc6f62f9700 1 --2- 172.21.15.52:0/3244782516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8567d470 0x55bf85686350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.279+0000 7fc6f62f9700 1 -- 172.21.15.52:0/3244782516 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6ec00ea10 con 0x55bf856818b0 2023-11-27T15:26:30.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.279+0000 7fc6f5af8700 1 --2- 172.21.15.52:0/3244782516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf8567cb60 0x55bf856890d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:26:30.295 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.279+0000 7fc6f62f9700 1 --2- 172.21.15.52:0/3244782516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf856818b0 0x55bf85686a40 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80964c0 tx=0x7fc6e8099680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a526c51f14168e8e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:30.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.279+0000 7fc6d97fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:26:30.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.280+0000 7fc6d97fa700 1 -- 172.21.15.52:0/3244782516 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e809da80 con 0x55bf856818b0 2023-11-27T15:26:30.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.280+0000 7fc6d97fa700 1 -- 172.21.15.52:0/3244782516 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fc6e80a1e10 con 0x55bf856818b0 2023-11-27T15:26:30.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.280+0000 7fc703ee8b80 1 -- 172.21.15.52:0/3244782516 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bf8566fb20 con 0x55bf856818b0 2023-11-27T15:26:30.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.280+0000 7fc6d97fa700 1 -- 172.21.15.52:0/3244782516 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6e8098f40 con 0x55bf856818b0 2023-11-27T15:26:30.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.280+0000 7fc703ee8b80 10 client.16947 fetch_fsmap learned FSMap version 34 2023-11-27T15:26:30.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.280+0000 7fc703ee8b80 1 -- 172.21.15.52:0/3244782516 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55bf85594190 con 0x55bf856818b0 2023-11-27T15:26:30.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.280+0000 7fc6d97fa700 1 -- 172.21.15.52:0/3244782516 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bf8566fb20 con 0x55bf856818b0 2023-11-27T15:26:30.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.280+0000 7fc6d97fa700 1 -- 172.21.15.52:0/3244782516 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc6e8095840 con 0x55bf856818b0 2023-11-27T15:26:30.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.280+0000 7fc703ee8b80 10 client.16947 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:26:30.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.280+0000 7fc703ee8b80 10 client.16947 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:26:30.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.280+0000 7fc703ee8b80 20 client.16947 populate_metadata read hostname 'smithi052' 2023-11-27T15:26:30.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.280+0000 7fc703ee8b80 1 --2- 172.21.15.52:0/3244782516 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55bf85696790 0x55bf85698ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.280+0000 7fc703ee8b80 4 client.16947 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:26:30.298 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.280+0000 7fc703ee8b80 1 -- 172.21.15.52:0/3244782516 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf855900a0 con 0x55bf85696790 2023-11-27T15:26:30.298 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.281+0000 7fc6f5af8700 1 --2- 172.21.15.52:0/3244782516 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55bf85696790 0x55bf85698ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.298 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.281+0000 7fc6d97fa700 10 client.16947 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:26:30.298 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.281+0000 7fc6f5af8700 1 --2- 172.21.15.52:0/3244782516 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55bf85696790 0x55bf85698ba0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5f881d0c44747db server_cookie=1ffc78c070c5cb9b in_seq=0 out_seq=0 2023-11-27T15:26:30.298 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.282+0000 7fc6d97fa700 1 -- 172.21.15.52:0/3244782516 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bf855900a0 con 0x55bf85696790 2023-11-27T15:26:30.298 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.282+0000 7fc6d97fa700 10 client.16947 handle_command_reply: tid=0 2023-11-27T15:26:30.298 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.282+0000 7fc703ee8b80 1 client.16947 shutdown 2023-11-27T15:26:30.299 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.282+0000 7fc703ee8b80 1 -- 172.21.15.52:0/3244782516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf856818b0 msgr2=0x55bf85686a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:30.299 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.282+0000 7fc703ee8b80 1 --2- 172.21.15.52:0/3244782516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf856818b0 0x55bf85686a40 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80964c0 tx=0x7fc6e8099680 comp rx=0 tx=0).stop 2023-11-27T15:26:30.299 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.282+0000 7fc703ee8b80 1 -- 172.21.15.52:0/3244782516 shutdown_connections 2023-11-27T15:26:30.299 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.282+0000 7fc703ee8b80 1 --2- 172.21.15.52:0/3244782516 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55bf85696790 0x55bf85698ba0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.299 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.282+0000 7fc703ee8b80 1 --2- 172.21.15.52:0/3244782516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf856818b0 0x55bf85686a40 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.299 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.282+0000 7fc703ee8b80 1 --2- 172.21.15.52:0/3244782516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf8567cb60 0x55bf856890d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.299 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.282+0000 7fc6d97fa700 0 client.16947 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:26:30.300 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.282+0000 7fc703ee8b80 1 --2- 172.21.15.52:0/3244782516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8567d470 0x55bf85686350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.300 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.282+0000 7fc703ee8b80 1 -- 172.21.15.52:0/3244782516 >> 172.21.15.52:0/3244782516 conn(0x55bf85672360 msgr2=0x55bf85676de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:30.300 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.282+0000 7fc703ee8b80 1 -- 172.21.15.52:0/3244782516 shutdown_connections 2023-11-27T15:26:30.300 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.282+0000 7fc703ee8b80 1 -- 172.21.15.52:0/3244782516 wait complete. 2023-11-27T15:26:30.300 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.282+0000 7fc703ee8b80 20 client.16947 trim_cache size 0 max 16384 2023-11-27T15:26:30.308 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.299+0000 7fc703ee8b80 1 Processor -- start 2023-11-27T15:26:30.308 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.299+0000 7fc703ee8b80 1 -- start start 2023-11-27T15:26:30.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.299+0000 7fc703ee8b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf85672360 0x55bf856801a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.299+0000 7fc703ee8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf8567cb60 0x55bf85677560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.299+0000 7fc703ee8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf8567d470 0x55bf85677aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.299+0000 7fc703ee8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf8567dbc0 con 0x55bf8567d470 2023-11-27T15:26:30.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.299+0000 7fc703ee8b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf85671ee0 con 0x55bf85672360 2023-11-27T15:26:30.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.299+0000 7fc703ee8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf8566f660 con 0x55bf8567cb60 2023-11-27T15:26:30.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.299+0000 7fc6f62f9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf8567d470 0x55bf85677aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.299+0000 7fc6f62f9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf8567d470 0x55bf85677aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55668/0 (socket says 172.21.15.52:55668) 2023-11-27T15:26:30.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.299+0000 7fc6f62f9700 1 -- 172.21.15.52:0/4126086234 learned_addr learned my addr 172.21.15.52:0/4126086234 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:30.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.299+0000 7fc6f52f7700 1 --2- 172.21.15.52:0/4126086234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf8567cb60 0x55bf85677560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.299+0000 7fc6f5af8700 1 --2- 172.21.15.52:0/4126086234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf85672360 0x55bf856801a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.299+0000 7fc6f62f9700 1 -- 172.21.15.52:0/4126086234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf8567cb60 msgr2=0x55bf85677560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:30.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.299+0000 7fc6f62f9700 1 --2- 172.21.15.52:0/4126086234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf8567cb60 0x55bf85677560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.310 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.299+0000 7fc6f62f9700 1 -- 172.21.15.52:0/4126086234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf85672360 msgr2=0x55bf856801a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:30.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.299+0000 7fc6f62f9700 1 --2- 172.21.15.52:0/4126086234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf85672360 0x55bf856801a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.299+0000 7fc6f62f9700 1 -- 172.21.15.52:0/4126086234 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf85593ee0 con 0x55bf8567d470 2023-11-27T15:26:30.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.299+0000 7fc6f52f7700 1 --2- 172.21.15.52:0/4126086234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf8567cb60 0x55bf85677560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:26:30.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.299+0000 7fc6f5af8700 1 --2- 172.21.15.52:0/4126086234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf85672360 0x55bf856801a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:26:30.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.299+0000 7fc6f62f9700 1 --2- 172.21.15.52:0/4126086234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf8567d470 0x55bf85677aa0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8004de0 tx=0x7fc6e8099bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d2646d44361aac server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:30.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.300+0000 7fc6e0ff9700 1 -- 172.21.15.52:0/4126086234 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e80a1e10 con 0x55bf8567d470 2023-11-27T15:26:30.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.300+0000 7fc6e0ff9700 1 -- 172.21.15.52:0/4126086234 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6e809da80 con 0x55bf8567d470 2023-11-27T15:26:30.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.300+0000 7fc6e0ff9700 1 -- 172.21.15.52:0/4126086234 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e801b970 con 0x55bf8567d470 2023-11-27T15:26:30.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.300+0000 7fc703ee8b80 1 -- 172.21.15.52:0/4126086234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf8567d470 msgr2=0x55bf85677aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:30.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.300+0000 7fc703ee8b80 1 --2- 172.21.15.52:0/4126086234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf8567d470 0x55bf85677aa0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8004de0 tx=0x7fc6e8099bd0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.300+0000 7fc703ee8b80 1 -- 172.21.15.52:0/4126086234 shutdown_connections 2023-11-27T15:26:30.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.300+0000 7fc703ee8b80 1 --2- 172.21.15.52:0/4126086234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf85672360 0x55bf856801a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.300+0000 7fc703ee8b80 1 --2- 172.21.15.52:0/4126086234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf8567d470 0x55bf85677aa0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.300+0000 7fc703ee8b80 1 --2- 172.21.15.52:0/4126086234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf8567cb60 0x55bf85677560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.300+0000 7fc703ee8b80 1 -- 172.21.15.52:0/4126086234 >> 172.21.15.52:0/4126086234 conn(0x55bf85681fb0 msgr2=0x55bf85680a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:30.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.300+0000 7fc703ee8b80 1 -- 172.21.15.52:0/4126086234 shutdown_connections 2023-11-27T15:26:30.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.300+0000 7fc703ee8b80 1 -- 172.21.15.52:0/4126086234 wait complete. 2023-11-27T15:26:30.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.300+0000 7fc703ee8b80 1 -- start start 2023-11-27T15:26:30.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.301+0000 7fc703ee8b80 1 Processor -- start 2023-11-27T15:26:30.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.301+0000 7fc703ee8b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf85672360 0x55bf856e3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.301+0000 7fc703ee8b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf8567cb60 0x55bf856e3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.314 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.301+0000 7fc703ee8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf8567d470 0x55bf85689980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.314 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.301+0000 7fc703ee8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf85689f50 con 0x55bf8567d470 2023-11-27T15:26:30.314 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.301+0000 7fc703ee8b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf8568a0c0 con 0x55bf85672360 2023-11-27T15:26:30.314 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.301+0000 7fc703ee8b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf8568a280 con 0x55bf8567cb60 2023-11-27T15:26:30.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.301+0000 7fc703ee8b80 10 client.0 _ll_register_callbacks cb 0x7fc6e8099890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:30.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.301+0000 7fc6f62f9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf8567d470 0x55bf85689980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.301+0000 7fc6f62f9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf8567d470 0x55bf85689980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55674/0 (socket says 172.21.15.52:55674) 2023-11-27T15:26:30.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.301+0000 7fc6f62f9700 1 -- 172.21.15.52:0/2579597487 learned_addr learned my addr 172.21.15.52:0/2579597487 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:26:30.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.301+0000 7fc6f5af8700 1 --2- 172.21.15.52:0/2579597487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf85672360 0x55bf856e3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.301+0000 7fc6f52f7700 1 --2- 172.21.15.52:0/2579597487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf8567cb60 0x55bf856e3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.302+0000 7fc6f62f9700 1 -- 172.21.15.52:0/2579597487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf8567cb60 msgr2=0x55bf856e3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:30.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.302+0000 7fc6f62f9700 1 --2- 172.21.15.52:0/2579597487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf8567cb60 0x55bf856e3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.302+0000 7fc6f62f9700 1 -- 172.21.15.52:0/2579597487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf85672360 msgr2=0x55bf856e3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:30.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.302+0000 7fc6f62f9700 1 --2- 172.21.15.52:0/2579597487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf85672360 0x55bf856e3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.302+0000 7fc6f62f9700 1 -- 172.21.15.52:0/2579597487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf85594190 con 0x55bf8567d470 2023-11-27T15:26:30.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.302+0000 7fc6f62f9700 1 --2- 172.21.15.52:0/2579597487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf8567d470 0x55bf85689980 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8009c10 tx=0x7fc6e8098000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f358a471b3c9aab4 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:30.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.302+0000 7fc6d97fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:26:30.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.302+0000 7fc6d97fa700 1 -- 172.21.15.52:0/2579597487 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e800ce10 con 0x55bf8567d470 2023-11-27T15:26:30.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.302+0000 7fc6d97fa700 1 -- 172.21.15.52:0/2579597487 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fc6e801bbb0 con 0x55bf8567d470 2023-11-27T15:26:30.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.302+0000 7fc703ee8b80 1 -- 172.21.15.52:0/2579597487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc6ec004cf0 con 0x55bf8567d470 2023-11-27T15:26:30.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.302+0000 7fc6d97fa700 1 -- 172.21.15.52:0/2579597487 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6e801bd70 con 0x55bf8567d470 2023-11-27T15:26:30.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.302+0000 7fc703ee8b80 10 client.16953 fetch_fsmap learned FSMap version 34 2023-11-27T15:26:30.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.302+0000 7fc703ee8b80 1 -- 172.21.15.52:0/2579597487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55bf853e4c60 con 0x55bf8567d470 2023-11-27T15:26:30.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.302+0000 7fc6d97fa700 1 -- 172.21.15.52:0/2579597487 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6ec004cf0 con 0x55bf8567d470 2023-11-27T15:26:30.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.302+0000 7fc6d97fa700 1 -- 172.21.15.52:0/2579597487 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc6e80842c0 con 0x55bf8567d470 2023-11-27T15:26:30.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.302+0000 7fc703ee8b80 10 client.16953 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:26:30.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.302+0000 7fc703ee8b80 10 client.16953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:26:30.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.302+0000 7fc703ee8b80 20 client.16953 populate_metadata read hostname 'smithi052' 2023-11-27T15:26:30.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.302+0000 7fc703ee8b80 1 --2- 172.21.15.52:0/2579597487 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55bf856818b0 0x55bf856ed200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:30.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.302+0000 7fc703ee8b80 4 client.16953 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:26:30.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.302+0000 7fc703ee8b80 1 -- 172.21.15.52:0/2579597487 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bf8559c660 con 0x55bf856818b0 2023-11-27T15:26:30.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.303+0000 7fc6f5af8700 1 --2- 172.21.15.52:0/2579597487 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55bf856818b0 0x55bf856ed200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:30.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.303+0000 7fc6f5af8700 1 --2- 172.21.15.52:0/2579597487 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55bf856818b0 0x55bf856ed200 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f113039dea5caf17 server_cookie=4846484ddb51dfa2 in_seq=0 out_seq=0 2023-11-27T15:26:30.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.303+0000 7fc6d97fa700 10 client.16953 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:26:30.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.303+0000 7fc6d97fa700 1 -- 172.21.15.52:0/2579597487 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55bf8559c660 con 0x55bf856818b0 2023-11-27T15:26:30.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.303+0000 7fc6d97fa700 10 client.16953 handle_command_reply: tid=0 2023-11-27T15:26:30.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.303+0000 7fc703ee8b80 1 client.16953 shutdown 2023-11-27T15:26:30.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.303+0000 7fc703ee8b80 1 -- 172.21.15.52:0/2579597487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf8567d470 msgr2=0x55bf85689980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:30.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.303+0000 7fc703ee8b80 1 --2- 172.21.15.52:0/2579597487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf8567d470 0x55bf85689980 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8009c10 tx=0x7fc6e8098000 comp rx=0 tx=0).stop 2023-11-27T15:26:30.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.303+0000 7fc703ee8b80 1 -- 172.21.15.52:0/2579597487 shutdown_connections 2023-11-27T15:26:30.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.303+0000 7fc703ee8b80 1 --2- 172.21.15.52:0/2579597487 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55bf856818b0 0x55bf856ed200 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.303+0000 7fc703ee8b80 1 --2- 172.21.15.52:0/2579597487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf85672360 0x55bf856e3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.303+0000 7fc6d97fa700 0 client.16953 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:26:30.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.303+0000 7fc703ee8b80 1 --2- 172.21.15.52:0/2579597487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf8567d470 0x55bf85689980 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.303+0000 7fc703ee8b80 1 --2- 172.21.15.52:0/2579597487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55bf8567cb60 0x55bf856e3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.303+0000 7fc703ee8b80 1 -- 172.21.15.52:0/2579597487 >> 172.21.15.52:0/2579597487 conn(0x55bf85681fb0 msgr2=0x55bf8567b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:30.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.304+0000 7fc703ee8b80 1 -- 172.21.15.52:0/2579597487 shutdown_connections 2023-11-27T15:26:30.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.304+0000 7fc703ee8b80 1 -- 172.21.15.52:0/2579597487 wait complete. 2023-11-27T15:26:30.322 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.304+0000 7fc703ee8b80 20 client.16953 trim_cache size 0 max 16384 2023-11-27T15:26:30.322 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:26:30.322 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-27T15:26:30.323 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2023-11-27T15:26:30.323 INFO:teuthology.orchestra.run.smithi052.stdout: "45ba5e98-8a31-4a1a-bacd-7a8ec84a9c00": { 2023-11-27T15:26:30.323 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2023-11-27T15:26:30.323 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "45ba5e98-8a31-4a1a-bacd-7a8ec84a9c00", 2023-11-27T15:26:30.323 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2023-11-27T15:26:30.323 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:26:30.324 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:26:30.324 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:26:30.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.304+0000 7fc6f7d5c700 1 -- 172.21.15.52:0/1866579912 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fc6dc0a1340 msgr2=0x7fc6dc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:30.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.304+0000 7fc6f7d5c700 1 --2- 172.21.15.52:0/1866579912 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fc6dc0a1340 0x7fc6dc0a37f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec005950 tx=0x7fc6ec005f50 comp rx=0 tx=0).stop 2023-11-27T15:26:30.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.304+0000 7fc6f7d5c700 1 -- 172.21.15.52:0/1866579912 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6f0104590 msgr2=0x7fc6f01a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:30.326 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.304+0000 7fc6f7d5c700 1 --2- 172.21.15.52:0/1866579912 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6f0104590 0x7fc6f01a4600 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fc6e800ef30 tx=0x7fc6e800c450 comp rx=0 tx=0).stop 2023-11-27T15:26:30.327 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.304+0000 7fc6f7d5c700 1 -- 172.21.15.52:0/1866579912 shutdown_connections 2023-11-27T15:26:30.327 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.304+0000 7fc6f7d5c700 1 --2- 172.21.15.52:0/1866579912 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fc6dc0a1340 0x7fc6dc0a37f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.327 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.304+0000 7fc6f7d5c700 1 --2- 172.21.15.52:0/1866579912 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6f0104590 0x7fc6f01a4600 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.327 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.304+0000 7fc6f7d5c700 1 --2- 172.21.15.52:0/1866579912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6f00ffc30 0x7fc6f010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.327 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.304+0000 7fc6f7d5c700 1 --2- 172.21.15.52:0/1866579912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6f01005d0 0x7fc6f010cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:30.328 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.304+0000 7fc6f7d5c700 1 -- 172.21.15.52:0/1866579912 >> 172.21.15.52:0/1866579912 conn(0x7fc6f0075470 msgr2=0x7fc6f00fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:30.328 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.304+0000 7fc6f7d5c700 1 -- 172.21.15.52:0/1866579912 shutdown_connections 2023-11-27T15:26:30.328 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:26:30.304+0000 7fc6f7d5c700 1 -- 172.21.15.52:0/1866579912 wait complete. 2023-11-27T15:26:30.328 INFO:tasks.cephfs.filesystem:scrub status for tag:45ba5e98-8a31-4a1a-bacd-7a8ec84a9c00 - {'path': '/', 'tag': '45ba5e98-8a31-4a1a-bacd-7a8ec84a9c00', 'options': 'recursive,force'} 2023-11-27T15:26:30.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:30 smithi150 ceph-mon[100559]: pgmap v266: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:26:30.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:30 smithi102 ceph-mon[100296]: pgmap v266: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:26:30.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:30 smithi052 ceph-mon[99056]: pgmap v266: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:26:31.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:31 smithi150 ceph-mon[100559]: scrub queued for path: / 2023-11-27T15:26:31.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:31 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:26:31.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:31 smithi150 ceph-mon[100559]: scrub summary: active paths [/] 2023-11-27T15:26:31.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:31 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:26:31.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:31 smithi102 ceph-mon[100296]: scrub queued for path: / 2023-11-27T15:26:31.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:31 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:26:31.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:31 smithi102 ceph-mon[100296]: scrub summary: active paths [/] 2023-11-27T15:26:31.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:31 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:26:31.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:31 smithi052 ceph-mon[99056]: scrub queued for path: / 2023-11-27T15:26:31.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:31 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:26:31.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:31 smithi052 ceph-mon[99056]: scrub summary: active paths [/] 2023-11-27T15:26:31.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:31 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:26:32.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:32 smithi150 ceph-mon[100559]: pgmap v267: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:26:32.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:32 smithi102 ceph-mon[100296]: pgmap v267: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:26:32.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:32 smithi052 ceph-mon[99056]: pgmap v267: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:26:34.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:34 smithi150 ceph-mon[100559]: pgmap v268: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:26:34.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:34 smithi102 ceph-mon[100296]: pgmap v268: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:26:34.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:34 smithi052 ceph-mon[99056]: pgmap v268: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:26:35.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:35 smithi150 ceph-mon[100559]: scrub summary: idle 2023-11-27T15:26:35.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:35 smithi102 ceph-mon[100296]: scrub summary: idle 2023-11-27T15:26:35.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:35 smithi052 ceph-mon[99056]: scrub summary: idle 2023-11-27T15:26:36.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:36 smithi150 ceph-mon[100559]: pgmap v269: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:26:36.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:36 smithi102 ceph-mon[100296]: pgmap v269: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:26:36.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:36 smithi052 ceph-mon[99056]: pgmap v269: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:26:38.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:38 smithi150 ceph-mon[100559]: pgmap v270: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:26:38.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:38 smithi102 ceph-mon[100296]: pgmap v270: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:26:38.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:38 smithi052 ceph-mon[99056]: pgmap v270: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:26:40.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:39 smithi150 ceph-mon[100559]: pgmap v271: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:26:40.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:39 smithi102 ceph-mon[100296]: pgmap v271: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:26:40.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:39 smithi052 ceph-mon[99056]: pgmap v271: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:26:42.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:42 smithi150 ceph-mon[100559]: pgmap v272: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:26:46.289 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:44 smithi150 ceph-mon[100559]: pgmap v273: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:26:46.289 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:42 smithi052 ceph-mon[99056]: pgmap v272: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:26:46.290 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:44 smithi052 ceph-mon[99056]: pgmap v273: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:26:46.290 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:42 smithi102 ceph-mon[100296]: pgmap v272: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:26:46.290 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:44 smithi102 ceph-mon[100296]: pgmap v273: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:26:46.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:46 smithi150 ceph-mon[100559]: pgmap v274: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:26:46.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:46 smithi102 ceph-mon[100296]: pgmap v274: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:26:46.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:46 smithi052 ceph-mon[99056]: pgmap v274: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:26:48.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:48 smithi150 ceph-mon[100559]: pgmap v275: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:26:48.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:48 smithi102 ceph-mon[100296]: pgmap v275: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:26:48.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:48 smithi052 ceph-mon[99056]: pgmap v275: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:26:50.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:49 smithi150 ceph-mon[100559]: pgmap v276: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:26:50.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:49 smithi102 ceph-mon[100296]: pgmap v276: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:26:50.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:49 smithi052 ceph-mon[99056]: pgmap v276: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:26:52.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:52 smithi150 ceph-mon[100559]: pgmap v277: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:26:52.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:52 smithi102 ceph-mon[100296]: pgmap v277: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:26:52.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:52 smithi052 ceph-mon[99056]: pgmap v277: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:26:54.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:54 smithi150 ceph-mon[100559]: pgmap v278: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:26:54.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:54 smithi102 ceph-mon[100296]: pgmap v278: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:26:54.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:54 smithi052 ceph-mon[99056]: pgmap v278: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:26:56.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:56 smithi150 ceph-mon[100559]: pgmap v279: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:26:56.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:56 smithi102 ceph-mon[100296]: pgmap v279: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:26:56.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:56 smithi052 ceph-mon[99056]: pgmap v279: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:26:58.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:58 smithi150 ceph-mon[100559]: pgmap v280: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:26:58.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:58 smithi102 ceph-mon[100296]: pgmap v280: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:26:58.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:58 smithi052 ceph-mon[99056]: pgmap v280: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:27:00.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:26:59 smithi150 ceph-mon[100559]: pgmap v281: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:27:00.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:26:59 smithi102 ceph-mon[100296]: pgmap v281: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:27:00.330 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:27:00.360 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:26:59 smithi052 ceph-mon[99056]: pgmap v281: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:27:00.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.468+0000 7f99d25cb700 1 -- 172.21.15.52:0/2894249070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99cc104180 msgr2=0x7f99cc104550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:00.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.468+0000 7f99d25cb700 1 --2- 172.21.15.52:0/2894249070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99cc104180 0x7f99cc104550 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f99bc0099c0 tx=0x7f99bc009cd0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.468+0000 7f99d25cb700 1 -- 172.21.15.52:0/2894249070 shutdown_connections 2023-11-27T15:27:00.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.468+0000 7f99d25cb700 1 --2- 172.21.15.52:0/2894249070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99cc100340 0x7f99cc10c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.468+0000 7f99d25cb700 1 --2- 172.21.15.52:0/2894249070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99cc104180 0x7f99cc104550 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.468+0000 7f99d25cb700 1 --2- 172.21.15.52:0/2894249070 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99cc0ff990 0x7f99cc0ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.468+0000 7f99d25cb700 1 -- 172.21.15.52:0/2894249070 >> 172.21.15.52:0/2894249070 conn(0x7f99cc075920 msgr2=0x7f99cc075d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:00.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.469+0000 7f99d25cb700 1 -- 172.21.15.52:0/2894249070 shutdown_connections 2023-11-27T15:27:00.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.469+0000 7f99d25cb700 1 -- 172.21.15.52:0/2894249070 wait complete. 2023-11-27T15:27:00.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.469+0000 7f99d25cb700 1 Processor -- start 2023-11-27T15:27:00.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.469+0000 7f99d25cb700 1 -- start start 2023-11-27T15:27:00.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.470+0000 7f99d25cb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99cc0ff990 0x7f99cc195350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:00.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.470+0000 7f99d25cb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99cc100340 0x7f99cc195890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:00.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.470+0000 7f99d25cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99cc104180 0x7f99cc199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:00.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.470+0000 7f99d25cb700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc19a330 con 0x7f99cc104180 2023-11-27T15:27:00.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.470+0000 7f99d25cb700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc19a4a0 con 0x7f99cc100340 2023-11-27T15:27:00.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.470+0000 7f99d25cb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc19a660 con 0x7f99cc0ff990 2023-11-27T15:27:00.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.470+0000 7f99d0b68700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99cc104180 0x7f99cc199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:00.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.470+0000 7f99d0b68700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99cc104180 0x7f99cc199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55688/0 (socket says 172.21.15.52:55688) 2023-11-27T15:27:00.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.470+0000 7f99d0b68700 1 -- 172.21.15.52:0/3320440003 learned_addr learned my addr 172.21.15.52:0/3320440003 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:00.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.470+0000 7f99cbfff700 1 --2- 172.21.15.52:0/3320440003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99cc0ff990 0x7f99cc195350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:00.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.470+0000 7f99cb7fe700 1 --2- 172.21.15.52:0/3320440003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99cc100340 0x7f99cc195890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:00.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.470+0000 7f99d0b68700 1 -- 172.21.15.52:0/3320440003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99cc0ff990 msgr2=0x7f99cc195350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:00.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.470+0000 7f99d0b68700 1 --2- 172.21.15.52:0/3320440003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99cc0ff990 0x7f99cc195350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.470+0000 7f99d0b68700 1 -- 172.21.15.52:0/3320440003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99cc100340 msgr2=0x7f99cc195890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:00.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.470+0000 7f99d0b68700 1 --2- 172.21.15.52:0/3320440003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99cc100340 0x7f99cc195890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.470+0000 7f99d0b68700 1 -- 172.21.15.52:0/3320440003 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99bc0095f0 con 0x7f99cc104180 2023-11-27T15:27:00.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.470+0000 7f99d0b68700 1 --2- 172.21.15.52:0/3320440003 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99cc104180 0x7f99cc199cf0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f99c4007ee0 tx=0x7f99c400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:00.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.470+0000 7f99c97fa700 1 -- 172.21.15.52:0/3320440003 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c4004db0 con 0x7f99cc104180 2023-11-27T15:27:00.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.470+0000 7f99d25cb700 1 -- 172.21.15.52:0/3320440003 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99cc10f1f0 con 0x7f99cc104180 2023-11-27T15:27:00.482 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.471+0000 7f99c97fa700 1 -- 172.21.15.52:0/3320440003 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99c400f040 con 0x7f99cc104180 2023-11-27T15:27:00.482 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.471+0000 7f99c97fa700 1 -- 172.21.15.52:0/3320440003 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c40058a0 con 0x7f99cc104180 2023-11-27T15:27:00.482 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.471+0000 7f99d25cb700 1 -- 172.21.15.52:0/3320440003 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99cc10f060 con 0x7f99cc104180 2023-11-27T15:27:00.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.473+0000 7f99c97fa700 1 -- 172.21.15.52:0/3320440003 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f99c400db20 con 0x7f99cc104180 2023-11-27T15:27:00.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.473+0000 7f99c97fa700 1 --2- 172.21.15.52:0/3320440003 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f99b40a12c0 0x7f99b40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:00.490 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.474+0000 7f99c97fa700 1 -- 172.21.15.52:0/3320440003 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f99c40cc610 con 0x7f99cc104180 2023-11-27T15:27:00.490 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.474+0000 7f99cbfff700 1 --2- 172.21.15.52:0/3320440003 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f99b40a12c0 0x7f99b40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:00.490 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.474+0000 7f99cbfff700 1 --2- 172.21.15.52:0/3320440003 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f99b40a12c0 0x7f99b40a3770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f99bc000c00 tx=0x7f99bc003680 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:00.490 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.474+0000 7f99de757b80 1 Processor -- start 2023-11-27T15:27:00.490 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.474+0000 7f99de757b80 1 -- start start 2023-11-27T15:27:00.491 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.475+0000 7f99de757b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557aa0f1ed70 0x557aa0f1f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:00.491 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.475+0000 7f99de757b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557aa0f1f680 0x557aa0f23580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:00.491 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.475+0000 7f99de757b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557aa0f23ac0 0x557aa0f23f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:00.491 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.475+0000 7f99de757b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa0e22060 con 0x557aa0f1f680 2023-11-27T15:27:00.491 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.475+0000 7f99de757b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa0e748a0 con 0x557aa0f1ed70 2023-11-27T15:27:00.491 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.475+0000 7f99de757b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa0e6dd30 con 0x557aa0f23ac0 2023-11-27T15:27:00.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.475+0000 7f99cb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557aa0f1f680 0x557aa0f23580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:00.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.475+0000 7f99cb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557aa0f1f680 0x557aa0f23580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55694/0 (socket says 172.21.15.52:55694) 2023-11-27T15:27:00.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.475+0000 7f99cb7fe700 1 -- 172.21.15.52:0/11995637 learned_addr learned my addr 172.21.15.52:0/11995637 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:00.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.475+0000 7f99cbfff700 1 --2- 172.21.15.52:0/11995637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557aa0f1ed70 0x557aa0f1f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:00.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.475+0000 7f99d0b68700 1 --2- 172.21.15.52:0/11995637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557aa0f23ac0 0x557aa0f23f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:00.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.475+0000 7f99cb7fe700 1 -- 172.21.15.52:0/11995637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557aa0f23ac0 msgr2=0x557aa0f23f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:00.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.475+0000 7f99cb7fe700 1 --2- 172.21.15.52:0/11995637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557aa0f23ac0 0x557aa0f23f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.475+0000 7f99cb7fe700 1 -- 172.21.15.52:0/11995637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557aa0f1ed70 msgr2=0x557aa0f1f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:00.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.475+0000 7f99cb7fe700 1 --2- 172.21.15.52:0/11995637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557aa0f1ed70 0x557aa0f1f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.475+0000 7f99cb7fe700 1 -- 172.21.15.52:0/11995637 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557aa0cef630 con 0x557aa0f1f680 2023-11-27T15:27:00.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.475+0000 7f99cbfff700 1 --2- 172.21.15.52:0/11995637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557aa0f1ed70 0x557aa0f1f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:00.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.475+0000 7f99cb7fe700 1 --2- 172.21.15.52:0/11995637 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557aa0f1f680 0x557aa0f23580 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f99c0007c90 tx=0x7f99c00103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1b4f998da1e596 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:00.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.476+0000 7f99b27fc700 1 -- 172.21.15.52:0/11995637 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c0010d60 con 0x557aa0f1f680 2023-11-27T15:27:00.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.476+0000 7f99b27fc700 1 -- 172.21.15.52:0/11995637 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99c0010ec0 con 0x557aa0f1f680 2023-11-27T15:27:00.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.476+0000 7f99b27fc700 1 -- 172.21.15.52:0/11995637 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c001a470 con 0x557aa0f1f680 2023-11-27T15:27:00.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.476+0000 7f99de757b80 1 -- 172.21.15.52:0/11995637 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557aa0f1f680 msgr2=0x557aa0f23580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:00.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.476+0000 7f99de757b80 1 --2- 172.21.15.52:0/11995637 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557aa0f1f680 0x557aa0f23580 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f99c0007c90 tx=0x7f99c00103d0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.476+0000 7f99de757b80 1 -- 172.21.15.52:0/11995637 shutdown_connections 2023-11-27T15:27:00.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.476+0000 7f99de757b80 1 --2- 172.21.15.52:0/11995637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557aa0f1ed70 0x557aa0f1f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.476+0000 7f99de757b80 1 --2- 172.21.15.52:0/11995637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557aa0f23ac0 0x557aa0f23f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.476+0000 7f99de757b80 1 --2- 172.21.15.52:0/11995637 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557aa0f1f680 0x557aa0f23580 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.476+0000 7f99de757b80 1 -- 172.21.15.52:0/11995637 >> 172.21.15.52:0/11995637 conn(0x557aa0f12360 msgr2=0x557aa0f17b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:00.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.476+0000 7f99de757b80 1 -- 172.21.15.52:0/11995637 shutdown_connections 2023-11-27T15:27:00.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.476+0000 7f99de757b80 1 -- 172.21.15.52:0/11995637 wait complete. 2023-11-27T15:27:00.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.477+0000 7f99de757b80 1 -- start start 2023-11-27T15:27:00.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.477+0000 7f99de757b80 1 Processor -- start 2023-11-27T15:27:00.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.477+0000 7f99de757b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557aa0f1ed70 0x557aa0f2b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:00.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.477+0000 7f99de757b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557aa0f1f680 0x557aa0f284b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:00.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.477+0000 7f99de757b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557aa0f23ac0 0x557aa0f28b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:00.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.477+0000 7f99de757b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa0e0f8e0 con 0x557aa0f1f680 2023-11-27T15:27:00.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.477+0000 7f99de757b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa0e0f580 con 0x557aa0f23ac0 2023-11-27T15:27:00.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.477+0000 7f99de757b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa0cf1920 con 0x557aa0f1ed70 2023-11-27T15:27:00.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.477+0000 7f99de757b80 10 client.0 _ll_register_callbacks cb 0x557aa0e2d780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:00.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.477+0000 7f99cb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557aa0f1f680 0x557aa0f284b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:00.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.477+0000 7f99cb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557aa0f1f680 0x557aa0f284b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55700/0 (socket says 172.21.15.52:55700) 2023-11-27T15:27:00.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.477+0000 7f99cb7fe700 1 -- 172.21.15.52:0/3396986605 learned_addr learned my addr 172.21.15.52:0/3396986605 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:00.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.477+0000 7f99d0b68700 1 --2- 172.21.15.52:0/3396986605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557aa0f23ac0 0x557aa0f28b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:00.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.478+0000 7f99cbfff700 1 --2- 172.21.15.52:0/3396986605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557aa0f1ed70 0x557aa0f2b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:00.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.478+0000 7f99cb7fe700 1 -- 172.21.15.52:0/3396986605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557aa0f1ed70 msgr2=0x557aa0f2b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:00.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.478+0000 7f99cb7fe700 1 --2- 172.21.15.52:0/3396986605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557aa0f1ed70 0x557aa0f2b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.478+0000 7f99cb7fe700 1 -- 172.21.15.52:0/3396986605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557aa0f23ac0 msgr2=0x557aa0f28b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:00.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.478+0000 7f99cb7fe700 1 --2- 172.21.15.52:0/3396986605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557aa0f23ac0 0x557aa0f28b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.478+0000 7f99cb7fe700 1 -- 172.21.15.52:0/3396986605 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c00078c0 con 0x557aa0f1f680 2023-11-27T15:27:00.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.478+0000 7f99cb7fe700 1 --2- 172.21.15.52:0/3396986605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557aa0f1f680 0x557aa0f284b0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f99c00107d0 tx=0x7f99c0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7dfc4a8a8541da7 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:00.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.478+0000 7f99ab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:27:00.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.478+0000 7f99ab7fe700 1 -- 172.21.15.52:0/3396986605 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c001f070 con 0x557aa0f1f680 2023-11-27T15:27:00.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.478+0000 7f99ab7fe700 1 -- 172.21.15.52:0/3396986605 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f99c0010b60 con 0x557aa0f1f680 2023-11-27T15:27:00.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.478+0000 7f99de757b80 1 -- 172.21.15.52:0/3396986605 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557aa0e300a0 con 0x557aa0f1f680 2023-11-27T15:27:00.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.478+0000 7f99ab7fe700 1 -- 172.21.15.52:0/3396986605 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99c0010d40 con 0x557aa0f1f680 2023-11-27T15:27:00.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.478+0000 7f99de757b80 10 client.16965 fetch_fsmap learned FSMap version 34 2023-11-27T15:27:00.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.478+0000 7f99de757b80 1 -- 172.21.15.52:0/3396986605 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x557aa0e34190 con 0x557aa0f1f680 2023-11-27T15:27:00.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.478+0000 7f99ab7fe700 1 -- 172.21.15.52:0/3396986605 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99c0030020 con 0x557aa0f1f680 2023-11-27T15:27:00.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.479+0000 7f99ab7fe700 1 -- 172.21.15.52:0/3396986605 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f99c0004750 con 0x557aa0f1f680 2023-11-27T15:27:00.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.479+0000 7f99de757b80 10 client.16965 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:27:00.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.479+0000 7f99de757b80 10 client.16965 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:00.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.479+0000 7f99de757b80 20 client.16965 populate_metadata read hostname 'smithi052' 2023-11-27T15:27:00.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.479+0000 7f99de757b80 1 --2- 172.21.15.52:0/3396986605 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557aa0f38980 0x557aa0f3ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:00.501 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.479+0000 7f99de757b80 4 client.16965 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:27:00.501 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.479+0000 7f99de757b80 1 -- 172.21.15.52:0/3396986605 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557aa0e3c660 con 0x557aa0f38980 2023-11-27T15:27:00.501 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.479+0000 7f99cbfff700 1 --2- 172.21.15.52:0/3396986605 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557aa0f38980 0x557aa0f3ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:00.501 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.479+0000 7f99cbfff700 1 --2- 172.21.15.52:0/3396986605 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557aa0f38980 0x557aa0f3ad90 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65cf6de613342729 server_cookie=f399a1ac8827cd5f in_seq=0 out_seq=0 2023-11-27T15:27:00.501 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.479+0000 7f99ab7fe700 10 client.16965 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:00.501 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.480+0000 7f99ab7fe700 1 -- 172.21.15.52:0/3396986605 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557aa0e3c660 con 0x557aa0f38980 2023-11-27T15:27:00.502 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.480+0000 7f99ab7fe700 10 client.16965 handle_command_reply: tid=0 2023-11-27T15:27:00.502 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.480+0000 7f99de757b80 1 client.16965 shutdown 2023-11-27T15:27:00.502 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.480+0000 7f99de757b80 1 -- 172.21.15.52:0/3396986605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557aa0f1f680 msgr2=0x557aa0f284b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:00.502 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.480+0000 7f99de757b80 1 --2- 172.21.15.52:0/3396986605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557aa0f1f680 0x557aa0f284b0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f99c00107d0 tx=0x7f99c0003b90 comp rx=0 tx=0).stop 2023-11-27T15:27:00.502 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.480+0000 7f99de757b80 1 -- 172.21.15.52:0/3396986605 shutdown_connections 2023-11-27T15:27:00.502 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.480+0000 7f99de757b80 1 --2- 172.21.15.52:0/3396986605 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557aa0f38980 0x557aa0f3ad90 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.503 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.480+0000 7f99de757b80 1 --2- 172.21.15.52:0/3396986605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557aa0f23ac0 0x557aa0f28b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.503 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.480+0000 7f99ab7fe700 0 client.16965 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:00.503 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.480+0000 7f99de757b80 1 --2- 172.21.15.52:0/3396986605 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557aa0f1ed70 0x557aa0f2b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.503 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.480+0000 7f99de757b80 1 --2- 172.21.15.52:0/3396986605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557aa0f1f680 0x557aa0f284b0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.503 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.480+0000 7f99de757b80 1 -- 172.21.15.52:0/3396986605 >> 172.21.15.52:0/3396986605 conn(0x557aa0f12360 msgr2=0x557aa0f16de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:00.503 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.480+0000 7f99de757b80 1 -- 172.21.15.52:0/3396986605 shutdown_connections 2023-11-27T15:27:00.504 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.481+0000 7f99de757b80 1 -- 172.21.15.52:0/3396986605 wait complete. 2023-11-27T15:27:00.504 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.481+0000 7f99de757b80 20 client.16965 trim_cache size 0 max 16384 2023-11-27T15:27:00.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.497+0000 7f99de757b80 1 Processor -- start 2023-11-27T15:27:00.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.497+0000 7f99de757b80 1 -- start start 2023-11-27T15:27:00.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.497+0000 7f99de757b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557aa0f1ed70 0x557aa0f2fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:00.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.498+0000 7f99de757b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557aa0f1f680 0x557aa0f27c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:00.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.498+0000 7f99de757b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557aa0f23ac0 0x557aa0f25e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:00.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.498+0000 7f99de757b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa0f29450 con 0x557aa0f23ac0 2023-11-27T15:27:00.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.498+0000 7f99de757b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa0f1fe00 con 0x557aa0f1f680 2023-11-27T15:27:00.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.498+0000 7f99de757b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa0f35470 con 0x557aa0f1ed70 2023-11-27T15:27:00.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.498+0000 7f99d0b68700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557aa0f23ac0 0x557aa0f25e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:00.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.498+0000 7f99d0b68700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557aa0f23ac0 0x557aa0f25e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55710/0 (socket says 172.21.15.52:55710) 2023-11-27T15:27:00.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.498+0000 7f99d0b68700 1 -- 172.21.15.52:0/2601792616 learned_addr learned my addr 172.21.15.52:0/2601792616 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:00.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.498+0000 7f99cbfff700 1 --2- 172.21.15.52:0/2601792616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557aa0f1ed70 0x557aa0f2fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:00.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.498+0000 7f99cb7fe700 1 --2- 172.21.15.52:0/2601792616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557aa0f1f680 0x557aa0f27c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:00.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.498+0000 7f99d0b68700 1 -- 172.21.15.52:0/2601792616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557aa0f1ed70 msgr2=0x557aa0f2fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:00.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.498+0000 7f99d0b68700 1 --2- 172.21.15.52:0/2601792616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557aa0f1ed70 0x557aa0f2fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.498+0000 7f99d0b68700 1 -- 172.21.15.52:0/2601792616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557aa0f1f680 msgr2=0x557aa0f27c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:00.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.498+0000 7f99d0b68700 1 --2- 172.21.15.52:0/2601792616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557aa0f1f680 0x557aa0f27c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.498+0000 7f99d0b68700 1 -- 172.21.15.52:0/2601792616 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557aa0e33ee0 con 0x557aa0f23ac0 2023-11-27T15:27:00.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.498+0000 7f99cb7fe700 1 --2- 172.21.15.52:0/2601792616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557aa0f1f680 0x557aa0f27c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:00.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.498+0000 7f99d0b68700 1 --2- 172.21.15.52:0/2601792616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557aa0f23ac0 0x557aa0f25e20 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f99c4099310 tx=0x7f99c40b9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce67c932c5f8a60 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:00.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.498+0000 7f99b2ffd700 1 -- 172.21.15.52:0/2601792616 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c409d640 con 0x557aa0f23ac0 2023-11-27T15:27:00.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.498+0000 7f99b2ffd700 1 -- 172.21.15.52:0/2601792616 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99c40a1a60 con 0x557aa0f23ac0 2023-11-27T15:27:00.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.498+0000 7f99b2ffd700 1 -- 172.21.15.52:0/2601792616 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c40840b0 con 0x557aa0f23ac0 2023-11-27T15:27:00.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.499+0000 7f99de757b80 1 -- 172.21.15.52:0/2601792616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557aa0f23ac0 msgr2=0x557aa0f25e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:00.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.499+0000 7f99de757b80 1 --2- 172.21.15.52:0/2601792616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557aa0f23ac0 0x557aa0f25e20 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f99c4099310 tx=0x7f99c40b9040 comp rx=0 tx=0).stop 2023-11-27T15:27:00.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.499+0000 7f99de757b80 1 -- 172.21.15.52:0/2601792616 shutdown_connections 2023-11-27T15:27:00.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.499+0000 7f99de757b80 1 --2- 172.21.15.52:0/2601792616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557aa0f23ac0 0x557aa0f25e20 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.499+0000 7f99de757b80 1 --2- 172.21.15.52:0/2601792616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557aa0f1ed70 0x557aa0f2fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.499+0000 7f99de757b80 1 --2- 172.21.15.52:0/2601792616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557aa0f1f680 0x557aa0f27c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.499+0000 7f99de757b80 1 -- 172.21.15.52:0/2601792616 >> 172.21.15.52:0/2601792616 conn(0x557aa0f12360 msgr2=0x557aa0f25710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:00.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.499+0000 7f99de757b80 1 -- 172.21.15.52:0/2601792616 shutdown_connections 2023-11-27T15:27:00.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.499+0000 7f99de757b80 1 -- 172.21.15.52:0/2601792616 wait complete. 2023-11-27T15:27:00.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.499+0000 7f99de757b80 1 -- start start 2023-11-27T15:27:00.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.499+0000 7f99de757b80 1 Processor -- start 2023-11-27T15:27:00.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.500+0000 7f99de757b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557aa0f1ed70 0x557aa0f169a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:00.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.500+0000 7f99de757b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557aa0f1f680 0x557aa0f16ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:00.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.500+0000 7f99de757b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557aa0f23ac0 0x557aa0f17420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:00.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.500+0000 7f99de757b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa0f301e0 con 0x557aa0f23ac0 2023-11-27T15:27:00.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.500+0000 7f99de757b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa0f8c800 con 0x557aa0f1ed70 2023-11-27T15:27:00.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.500+0000 7f99de757b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa0f8c970 con 0x557aa0f1f680 2023-11-27T15:27:00.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.500+0000 7f99de757b80 10 client.0 _ll_register_callbacks cb 0x7f99c0007ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:00.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.500+0000 7f99d0b68700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557aa0f23ac0 0x557aa0f17420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:00.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.500+0000 7f99cbfff700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557aa0f1ed70 0x557aa0f169a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:00.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.500+0000 7f99d0b68700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557aa0f23ac0 0x557aa0f17420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55716/0 (socket says 172.21.15.52:55716) 2023-11-27T15:27:00.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.500+0000 7f99d0b68700 1 -- 172.21.15.52:0/732904263 learned_addr learned my addr 172.21.15.52:0/732904263 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:00.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.500+0000 7f99cb7fe700 1 --2- 172.21.15.52:0/732904263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557aa0f1f680 0x557aa0f16ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:00.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.500+0000 7f99d0b68700 1 -- 172.21.15.52:0/732904263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557aa0f1f680 msgr2=0x557aa0f16ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:00.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.500+0000 7f99d0b68700 1 --2- 172.21.15.52:0/732904263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557aa0f1f680 0x557aa0f16ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.500+0000 7f99d0b68700 1 -- 172.21.15.52:0/732904263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557aa0f1ed70 msgr2=0x557aa0f169a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:00.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.500+0000 7f99d0b68700 1 --2- 172.21.15.52:0/732904263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557aa0f1ed70 0x557aa0f169a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.500+0000 7f99d0b68700 1 -- 172.21.15.52:0/732904263 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99cc10f060 con 0x557aa0f23ac0 2023-11-27T15:27:00.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.500+0000 7f99d0b68700 1 --2- 172.21.15.52:0/732904263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557aa0f23ac0 0x557aa0f17420 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f99c40ca2b0 tx=0x7f99c4004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc43ae35ade2eab server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:00.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.500+0000 7f99ab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:27:00.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.500+0000 7f99ab7fe700 1 -- 172.21.15.52:0/732904263 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c4097750 con 0x557aa0f23ac0 2023-11-27T15:27:00.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.500+0000 7f99ab7fe700 1 -- 172.21.15.52:0/732904263 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f99c40c3e40 con 0x557aa0f23ac0 2023-11-27T15:27:00.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.500+0000 7f99de757b80 1 -- 172.21.15.52:0/732904263 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557aa0f0fb20 con 0x557aa0f23ac0 2023-11-27T15:27:00.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.501+0000 7f99ab7fe700 1 -- 172.21.15.52:0/732904263 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99c409d640 con 0x557aa0f23ac0 2023-11-27T15:27:00.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.501+0000 7f99de757b80 10 client.16971 fetch_fsmap learned FSMap version 34 2023-11-27T15:27:00.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.501+0000 7f99de757b80 1 -- 172.21.15.52:0/732904263 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x557aa0c84c60 con 0x557aa0f23ac0 2023-11-27T15:27:00.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.501+0000 7f99ab7fe700 1 -- 172.21.15.52:0/732904263 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557aa0f0fb20 con 0x557aa0f23ac0 2023-11-27T15:27:00.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.501+0000 7f99ab7fe700 1 -- 172.21.15.52:0/732904263 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f99c40a15d0 con 0x557aa0f23ac0 2023-11-27T15:27:00.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.501+0000 7f99de757b80 10 client.16971 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:27:00.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.501+0000 7f99de757b80 10 client.16971 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:00.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.501+0000 7f99de757b80 20 client.16971 populate_metadata read hostname 'smithi052' 2023-11-27T15:27:00.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.501+0000 7f99de757b80 1 --2- 172.21.15.52:0/732904263 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557aa0f8de20 0x557aa0f90230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:00.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.501+0000 7f99de757b80 4 client.16971 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:27:00.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.501+0000 7f99de757b80 1 -- 172.21.15.52:0/732904263 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557aa0cbcf70 con 0x557aa0f8de20 2023-11-27T15:27:00.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.501+0000 7f99cbfff700 1 --2- 172.21.15.52:0/732904263 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557aa0f8de20 0x557aa0f90230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:00.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.502+0000 7f99ab7fe700 10 client.16971 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:00.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.502+0000 7f99cbfff700 1 --2- 172.21.15.52:0/732904263 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557aa0f8de20 0x557aa0f90230 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fc444115d926cdc server_cookie=442c6fbef20856bd in_seq=0 out_seq=0 2023-11-27T15:27:00.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.502+0000 7f99ab7fe700 1 -- 172.21.15.52:0/732904263 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557aa0cbcf70 con 0x557aa0f8de20 2023-11-27T15:27:00.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.502+0000 7f99ab7fe700 10 client.16971 handle_command_reply: tid=0 2023-11-27T15:27:00.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.502+0000 7f99de757b80 1 client.16971 shutdown 2023-11-27T15:27:00.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.502+0000 7f99de757b80 1 -- 172.21.15.52:0/732904263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557aa0f23ac0 msgr2=0x557aa0f17420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:00.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.502+0000 7f99de757b80 1 --2- 172.21.15.52:0/732904263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557aa0f23ac0 0x557aa0f17420 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f99c40ca2b0 tx=0x7f99c4004f10 comp rx=0 tx=0).stop 2023-11-27T15:27:00.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.502+0000 7f99de757b80 1 -- 172.21.15.52:0/732904263 shutdown_connections 2023-11-27T15:27:00.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.502+0000 7f99de757b80 1 --2- 172.21.15.52:0/732904263 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557aa0f8de20 0x557aa0f90230 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.502+0000 7f99de757b80 1 --2- 172.21.15.52:0/732904263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557aa0f1ed70 0x557aa0f169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.502+0000 7f99de757b80 1 --2- 172.21.15.52:0/732904263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557aa0f23ac0 0x557aa0f17420 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.502+0000 7f99de757b80 1 --2- 172.21.15.52:0/732904263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557aa0f1f680 0x557aa0f16ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.523 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.502+0000 7f99ab7fe700 0 client.16971 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:00.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.502+0000 7f99de757b80 1 -- 172.21.15.52:0/732904263 >> 172.21.15.52:0/732904263 conn(0x557aa0f12360 msgr2=0x557aa0f23290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:00.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.502+0000 7f99de757b80 1 -- 172.21.15.52:0/732904263 shutdown_connections 2023-11-27T15:27:00.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.502+0000 7f99de757b80 1 -- 172.21.15.52:0/732904263 wait complete. 2023-11-27T15:27:00.524 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.502+0000 7f99de757b80 20 client.16971 trim_cache size 0 max 16384 2023-11-27T15:27:00.524 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:27:00.525 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2023-11-27T15:27:00.525 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2023-11-27T15:27:00.525 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:27:00.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.503+0000 7f99d25cb700 1 -- 172.21.15.52:0/3320440003 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f99b40a12c0 msgr2=0x7f99b40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:00.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.503+0000 7f99d25cb700 1 --2- 172.21.15.52:0/3320440003 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f99b40a12c0 0x7f99b40a3770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f99bc000c00 tx=0x7f99bc003680 comp rx=0 tx=0).stop 2023-11-27T15:27:00.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.503+0000 7f99d25cb700 1 -- 172.21.15.52:0/3320440003 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99cc104180 msgr2=0x7f99cc199cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:00.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.503+0000 7f99d25cb700 1 --2- 172.21.15.52:0/3320440003 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99cc104180 0x7f99cc199cf0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f99c4007ee0 tx=0x7f99c400d3b0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.503+0000 7f99d25cb700 1 -- 172.21.15.52:0/3320440003 shutdown_connections 2023-11-27T15:27:00.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.503+0000 7f99d25cb700 1 --2- 172.21.15.52:0/3320440003 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f99b40a12c0 0x7f99b40a3770 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.503+0000 7f99d25cb700 1 --2- 172.21.15.52:0/3320440003 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99cc104180 0x7f99cc199cf0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.503+0000 7f99d25cb700 1 --2- 172.21.15.52:0/3320440003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99cc0ff990 0x7f99cc195350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.503+0000 7f99d25cb700 1 --2- 172.21.15.52:0/3320440003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99cc100340 0x7f99cc195890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.503+0000 7f99d25cb700 1 -- 172.21.15.52:0/3320440003 >> 172.21.15.52:0/3320440003 conn(0x7f99cc075920 msgr2=0x7f99cc0fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:00.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.503+0000 7f99d25cb700 1 -- 172.21.15.52:0/3320440003 shutdown_connections 2023-11-27T15:27:00.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.503+0000 7f99d25cb700 1 -- 172.21.15.52:0/3320440003 wait complete. 2023-11-27T15:27:00.528 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-27T15:27:00.529 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:27:00.696 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.689+0000 7ffb3db1c700 1 -- 172.21.15.52:0/1749275779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb380ffcf0 msgr2=0x7ffb381000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:00.696 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.689+0000 7ffb3db1c700 1 --2- 172.21.15.52:0/1749275779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb380ffcf0 0x7ffb381000c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7ffb200099c0 tx=0x7ffb20009cd0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.696 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.689+0000 7ffb3db1c700 1 -- 172.21.15.52:0/1749275779 shutdown_connections 2023-11-27T15:27:00.696 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.689+0000 7ffb3db1c700 1 --2- 172.21.15.52:0/1749275779 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb38103490 0x7ffb3810c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.697 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.689+0000 7ffb3db1c700 1 --2- 172.21.15.52:0/1749275779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb380ffcf0 0x7ffb381000c0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.697 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.689+0000 7ffb3db1c700 1 --2- 172.21.15.52:0/1749275779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb38100690 0x7ffb38102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.697 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.689+0000 7ffb3db1c700 1 -- 172.21.15.52:0/1749275779 >> 172.21.15.52:0/1749275779 conn(0x7ffb380763a0 msgr2=0x7ffb380767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:00.697 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.689+0000 7ffb3db1c700 1 -- 172.21.15.52:0/1749275779 shutdown_connections 2023-11-27T15:27:00.697 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.689+0000 7ffb3db1c700 1 -- 172.21.15.52:0/1749275779 wait complete. 2023-11-27T15:27:00.698 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.689+0000 7ffb3db1c700 1 Processor -- start 2023-11-27T15:27:00.698 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.689+0000 7ffb3db1c700 1 -- start start 2023-11-27T15:27:00.698 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.690+0000 7ffb3db1c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb380ffcf0 0x7ffb38197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:00.698 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.690+0000 7ffb3db1c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb38100690 0x7ffb38197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:00.698 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.690+0000 7ffb3db1c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb38103490 0x7ffb381a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:00.698 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.690+0000 7ffb3db1c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb381a2800 con 0x7ffb38100690 2023-11-27T15:27:00.699 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.690+0000 7ffb3db1c700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb381a2940 con 0x7ffb38103490 2023-11-27T15:27:00.699 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.690+0000 7ffb3db1c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb381a2a80 con 0x7ffb380ffcf0 2023-11-27T15:27:00.699 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.690+0000 7ffb36ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb38100690 0x7ffb38197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:00.699 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.690+0000 7ffb36ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb38100690 0x7ffb38197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55728/0 (socket says 172.21.15.52:55728) 2023-11-27T15:27:00.699 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.690+0000 7ffb36ffd700 1 -- 172.21.15.52:0/44642392 learned_addr learned my addr 172.21.15.52:0/44642392 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:00.699 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.690+0000 7ffb37fff700 1 --2- 172.21.15.52:0/44642392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb38103490 0x7ffb381a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:00.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.690+0000 7ffb36ffd700 1 -- 172.21.15.52:0/44642392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb380ffcf0 msgr2=0x7ffb38197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:00.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.690+0000 7ffb377fe700 1 --2- 172.21.15.52:0/44642392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb380ffcf0 0x7ffb38197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:00.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.690+0000 7ffb36ffd700 1 --2- 172.21.15.52:0/44642392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb380ffcf0 0x7ffb38197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.690+0000 7ffb36ffd700 1 -- 172.21.15.52:0/44642392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb38103490 msgr2=0x7ffb381a22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:00.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.690+0000 7ffb36ffd700 1 --2- 172.21.15.52:0/44642392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb38103490 0x7ffb381a22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.690+0000 7ffb36ffd700 1 -- 172.21.15.52:0/44642392 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb200095f0 con 0x7ffb38100690 2023-11-27T15:27:00.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.690+0000 7ffb37fff700 1 --2- 172.21.15.52:0/44642392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb38103490 0x7ffb381a22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:00.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.690+0000 7ffb36ffd700 1 --2- 172.21.15.52:0/44642392 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb38100690 0x7ffb38197780 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ffb2800e910 tx=0x7ffb2800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:00.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.690+0000 7ffb34ff9700 1 -- 172.21.15.52:0/44642392 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb28009730 con 0x7ffb38100690 2023-11-27T15:27:00.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.690+0000 7ffb34ff9700 1 -- 172.21.15.52:0/44642392 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb28004cf0 con 0x7ffb38100690 2023-11-27T15:27:00.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.690+0000 7ffb3db1c700 1 -- 172.21.15.52:0/44642392 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb3810f2e0 con 0x7ffb38100690 2023-11-27T15:27:00.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.690+0000 7ffb34ff9700 1 -- 172.21.15.52:0/44642392 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb280055c0 con 0x7ffb38100690 2023-11-27T15:27:00.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.690+0000 7ffb3db1c700 1 -- 172.21.15.52:0/44642392 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb3810f150 con 0x7ffb38100690 2023-11-27T15:27:00.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.692+0000 7ffb3db1c700 1 -- 172.21.15.52:0/44642392 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb3810f470 con 0x7ffb38100690 2023-11-27T15:27:00.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.693+0000 7ffb34ff9700 1 -- 172.21.15.52:0/44642392 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7ffb2800cc60 con 0x7ffb38100690 2023-11-27T15:27:00.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.693+0000 7ffb34ff9700 1 --2- 172.21.15.52:0/44642392 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ffb240a1340 0x7ffb240a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:00.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.694+0000 7ffb377fe700 1 --2- 172.21.15.52:0/44642392 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ffb240a1340 0x7ffb240a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:00.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.694+0000 7ffb34ff9700 1 -- 172.21.15.52:0/44642392 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7ffb28014070 con 0x7ffb38100690 2023-11-27T15:27:00.704 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.694+0000 7ffb377fe700 1 --2- 172.21.15.52:0/44642392 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ffb240a1340 0x7ffb240a37f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ffb200098e0 tx=0x7ffb20019040 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:00.704 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.697+0000 7ffb34ff9700 1 -- 172.21.15.52:0/44642392 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ffb28099f30 con 0x7ffb38100690 2023-11-27T15:27:00.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.960+0000 7ffb3db1c700 1 -- 172.21.15.52:0/44642392 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb380671d0 con 0x7ffb38100690 2023-11-27T15:27:00.966 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.961+0000 7ffb34ff9700 1 -- 172.21.15.52:0/44642392 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10804 (secure 0 0 0) 0x7ffb280ade60 con 0x7ffb38100690 2023-11-27T15:27:00.966 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:27:00.967 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2328881376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2328881376},{"type":"v1","addr":"172.21.15.52:6839","nonce":2328881376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2799551570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2799551570},{"type":"v1","addr":"172.21.15.52:6841","nonce":2799551570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:16:09.319630+0000","modified":"2023-11-27T15:26:15.243744+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24494,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/215725838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":215725838},{"type":"v1","addr":"172.21.15.150:6839","nonce":215725838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3571830559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3571830559},{"type":"v1","addr":"172.21.15.102:6839","nonce":3571830559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:27:00.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.963+0000 7ffb3db1c700 1 -- 172.21.15.52:0/44642392 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ffb240a1340 msgr2=0x7ffb240a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:00.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.963+0000 7ffb3db1c700 1 --2- 172.21.15.52:0/44642392 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ffb240a1340 0x7ffb240a37f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ffb200098e0 tx=0x7ffb20019040 comp rx=0 tx=0).stop 2023-11-27T15:27:00.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.963+0000 7ffb3db1c700 1 -- 172.21.15.52:0/44642392 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb38100690 msgr2=0x7ffb38197780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:00.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.963+0000 7ffb3db1c700 1 --2- 172.21.15.52:0/44642392 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb38100690 0x7ffb38197780 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ffb2800e910 tx=0x7ffb2800ecd0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.963+0000 7ffb3db1c700 1 -- 172.21.15.52:0/44642392 shutdown_connections 2023-11-27T15:27:00.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.963+0000 7ffb3db1c700 1 --2- 172.21.15.52:0/44642392 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ffb240a1340 0x7ffb240a37f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.963+0000 7ffb3db1c700 1 --2- 172.21.15.52:0/44642392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb38103490 0x7ffb381a22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.970 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.963+0000 7ffb3db1c700 1 --2- 172.21.15.52:0/44642392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb380ffcf0 0x7ffb38197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.970 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.963+0000 7ffb3db1c700 1 --2- 172.21.15.52:0/44642392 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb38100690 0x7ffb38197780 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:00.970 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.963+0000 7ffb3db1c700 1 -- 172.21.15.52:0/44642392 >> 172.21.15.52:0/44642392 conn(0x7ffb380763a0 msgr2=0x7ffb380fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:00.970 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.963+0000 7ffb3db1c700 1 -- 172.21.15.52:0/44642392 shutdown_connections 2023-11-27T15:27:00.970 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:00.963+0000 7ffb3db1c700 1 -- 172.21.15.52:0/44642392 wait complete. 2023-11-27T15:27:00.971 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 34 2023-11-27T15:27:00.981 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-27T15:27:01.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.105+0000 7f202edd0700 1 -- 172.21.15.52:0/1516711670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2028072e70 msgr2=0x7f2028073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:01.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.105+0000 7f202edd0700 1 --2- 172.21.15.52:0/1516711670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2028072e70 0x7f2028073240 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f201c009990 tx=0x7f201c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.105+0000 7f202edd0700 1 -- 172.21.15.52:0/1516711670 shutdown_connections 2023-11-27T15:27:01.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.105+0000 7f202edd0700 1 --2- 172.21.15.52:0/1516711670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f202810cba0 0x7f202810ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.105+0000 7f202edd0700 1 --2- 172.21.15.52:0/1516711670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2028072e70 0x7f2028073240 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.105+0000 7f202edd0700 1 --2- 172.21.15.52:0/1516711670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2028073780 0x7f202810c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.105+0000 7f202edd0700 1 -- 172.21.15.52:0/1516711670 >> 172.21.15.52:0/1516711670 conn(0x7f2028077170 msgr2=0x7f2028077570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:01.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.105+0000 7f202edd0700 1 -- 172.21.15.52:0/1516711670 shutdown_connections 2023-11-27T15:27:01.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.105+0000 7f202edd0700 1 -- 172.21.15.52:0/1516711670 wait complete. 2023-11-27T15:27:01.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.106+0000 7f202edd0700 1 Processor -- start 2023-11-27T15:27:01.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.106+0000 7f202edd0700 1 -- start start 2023-11-27T15:27:01.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.106+0000 7f202edd0700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2028072e70 0x7f2028198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.106+0000 7f202edd0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2028073780 0x7f2028198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.106+0000 7f202edd0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f202810cba0 0x7f202819cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.106+0000 7f202edd0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202819d620 con 0x7f202810cba0 2023-11-27T15:27:01.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.106+0000 7f202edd0700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202819d790 con 0x7f2028072e70 2023-11-27T15:27:01.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.106+0000 7f202edd0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202819d950 con 0x7f2028073780 2023-11-27T15:27:01.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.106+0000 7f202d36d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f202810cba0 0x7f202819cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.106+0000 7f202d36d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f202810cba0 0x7f202819cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55744/0 (socket says 172.21.15.52:55744) 2023-11-27T15:27:01.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.106+0000 7f202d36d700 1 -- 172.21.15.52:0/605694517 learned_addr learned my addr 172.21.15.52:0/605694517 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:01.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.106+0000 7f202cb6c700 1 --2- 172.21.15.52:0/605694517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2028072e70 0x7f2028198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.107+0000 7f2027fff700 1 --2- 172.21.15.52:0/605694517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2028073780 0x7f2028198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.107+0000 7f202d36d700 1 -- 172.21.15.52:0/605694517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2028073780 msgr2=0x7f2028198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:01.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.107+0000 7f202d36d700 1 --2- 172.21.15.52:0/605694517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2028073780 0x7f2028198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.107+0000 7f202d36d700 1 -- 172.21.15.52:0/605694517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2028072e70 msgr2=0x7f2028198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:01.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.107+0000 7f202d36d700 1 --2- 172.21.15.52:0/605694517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2028072e70 0x7f2028198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.107+0000 7f202d36d700 1 -- 172.21.15.52:0/605694517 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f201c0095f0 con 0x7f202810cba0 2023-11-27T15:27:01.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.107+0000 7f202d36d700 1 --2- 172.21.15.52:0/605694517 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f202810cba0 0x7f202819cfe0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f2020007ee0 tx=0x7f202000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:01.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.107+0000 7f2025ffb700 1 -- 172.21.15.52:0/605694517 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202000ddd0 con 0x7f202810cba0 2023-11-27T15:27:01.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.107+0000 7f2025ffb700 1 -- 172.21.15.52:0/605694517 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f202000f040 con 0x7f202810cba0 2023-11-27T15:27:01.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.107+0000 7f2025ffb700 1 -- 172.21.15.52:0/605694517 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20200048e0 con 0x7f202810cba0 2023-11-27T15:27:01.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.107+0000 7f202edd0700 1 -- 172.21.15.52:0/605694517 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2028112580 con 0x7f202810cba0 2023-11-27T15:27:01.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.107+0000 7f202edd0700 1 -- 172.21.15.52:0/605694517 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20281123f0 con 0x7f202810cba0 2023-11-27T15:27:01.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.109+0000 7f2025ffb700 1 -- 172.21.15.52:0/605694517 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f2020003a00 con 0x7f202810cba0 2023-11-27T15:27:01.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.110+0000 7f2025ffb700 1 --2- 172.21.15.52:0/605694517 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f20180a12c0 0x7f20180a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.110+0000 7f2025ffb700 1 -- 172.21.15.52:0/605694517 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f20200cc610 con 0x7f202810cba0 2023-11-27T15:27:01.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.110+0000 7f202cb6c700 1 --2- 172.21.15.52:0/605694517 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f20180a12c0 0x7f20180a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.127 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.110+0000 7f202cb6c700 1 --2- 172.21.15.52:0/605694517 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f20180a12c0 0x7f20180a3770 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f201c00b5c0 tx=0x7f201c005960 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:01.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.111+0000 7f203af5cb80 1 Processor -- start 2023-11-27T15:27:01.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.111+0000 7f203af5cb80 1 -- start start 2023-11-27T15:27:01.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.111+0000 7f203af5cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c172cab60 0x559c172caf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.111+0000 7f203af5cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c172cb470 0x559c172cf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.128 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.111+0000 7f203af5cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c172cf8b0 0x559c172cfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.111+0000 7f203af5cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c171d0060 con 0x559c172cb470 2023-11-27T15:27:01.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.111+0000 7f203af5cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c172228a0 con 0x559c172cf8b0 2023-11-27T15:27:01.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.111+0000 7f203af5cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c1721bd30 con 0x559c172cab60 2023-11-27T15:27:01.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.111+0000 7f2027fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c172cb470 0x559c172cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.129 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.111+0000 7f2027fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c172cb470 0x559c172cf370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55750/0 (socket says 172.21.15.52:55750) 2023-11-27T15:27:01.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.111+0000 7f2027fff700 1 -- 172.21.15.52:0/172471985 learned_addr learned my addr 172.21.15.52:0/172471985 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:01.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.111+0000 7f202d36d700 1 --2- 172.21.15.52:0/172471985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c172cf8b0 0x559c172cfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.111+0000 7f202cb6c700 1 --2- 172.21.15.52:0/172471985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c172cab60 0x559c172caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.112+0000 7f2027fff700 1 -- 172.21.15.52:0/172471985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c172cab60 msgr2=0x559c172caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:01.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.112+0000 7f2027fff700 1 --2- 172.21.15.52:0/172471985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c172cab60 0x559c172caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.130 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.112+0000 7f2027fff700 1 -- 172.21.15.52:0/172471985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c172cf8b0 msgr2=0x559c172cfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:01.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.112+0000 7f2027fff700 1 --2- 172.21.15.52:0/172471985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c172cf8b0 0x559c172cfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.112+0000 7f2027fff700 1 -- 172.21.15.52:0/172471985 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c1709d630 con 0x559c172cb470 2023-11-27T15:27:01.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.112+0000 7f2027fff700 1 --2- 172.21.15.52:0/172471985 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c172cb470 0x559c172cf370 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f201400dc40 tx=0x7f201400df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f236c9420e5dd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:01.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.112+0000 7f2012ffd700 1 -- 172.21.15.52:0/172471985 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201400fb70 con 0x559c172cb470 2023-11-27T15:27:01.131 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.112+0000 7f2012ffd700 1 -- 172.21.15.52:0/172471985 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2014004cf0 con 0x559c172cb470 2023-11-27T15:27:01.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.112+0000 7f2012ffd700 1 -- 172.21.15.52:0/172471985 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20140056f0 con 0x559c172cb470 2023-11-27T15:27:01.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.113+0000 7f203af5cb80 1 -- 172.21.15.52:0/172471985 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c172cb470 msgr2=0x559c172cf370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:01.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.113+0000 7f203af5cb80 1 --2- 172.21.15.52:0/172471985 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c172cb470 0x559c172cf370 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f201400dc40 tx=0x7f201400df50 comp rx=0 tx=0).stop 2023-11-27T15:27:01.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.113+0000 7f203af5cb80 1 -- 172.21.15.52:0/172471985 shutdown_connections 2023-11-27T15:27:01.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.113+0000 7f203af5cb80 1 --2- 172.21.15.52:0/172471985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c172cf8b0 0x559c172cfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.132 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.113+0000 7f203af5cb80 1 --2- 172.21.15.52:0/172471985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c172cab60 0x559c172caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.113+0000 7f203af5cb80 1 --2- 172.21.15.52:0/172471985 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c172cb470 0x559c172cf370 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.113+0000 7f203af5cb80 1 -- 172.21.15.52:0/172471985 >> 172.21.15.52:0/172471985 conn(0x559c172c0360 msgr2=0x559c172c5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:01.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.113+0000 7f203af5cb80 1 -- 172.21.15.52:0/172471985 shutdown_connections 2023-11-27T15:27:01.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.113+0000 7f203af5cb80 1 -- 172.21.15.52:0/172471985 wait complete. 2023-11-27T15:27:01.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.113+0000 7f203af5cb80 1 -- start start 2023-11-27T15:27:01.133 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.113+0000 7f203af5cb80 1 Processor -- start 2023-11-27T15:27:01.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.114+0000 7f203af5cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c172cab60 0x559c172d7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.114+0000 7f203af5cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c172cb470 0x559c172d4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.114+0000 7f203af5cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c172cf8b0 0x559c172d48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.114+0000 7f203af5cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c171bd8e0 con 0x559c172cb470 2023-11-27T15:27:01.134 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.114+0000 7f203af5cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c171bd580 con 0x559c172cf8b0 2023-11-27T15:27:01.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.114+0000 7f203af5cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c1709f920 con 0x559c172cab60 2023-11-27T15:27:01.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.114+0000 7f203af5cb80 10 client.0 _ll_register_callbacks cb 0x559c171db780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:01.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.114+0000 7f2027fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c172cb470 0x559c172d4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.114+0000 7f2027fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c172cb470 0x559c172d4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55756/0 (socket says 172.21.15.52:55756) 2023-11-27T15:27:01.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.114+0000 7f2027fff700 1 -- 172.21.15.52:0/466148752 learned_addr learned my addr 172.21.15.52:0/466148752 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:01.135 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.114+0000 7f202cb6c700 1 --2- 172.21.15.52:0/466148752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c172cab60 0x559c172d7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.114+0000 7f202d36d700 1 --2- 172.21.15.52:0/466148752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c172cf8b0 0x559c172d48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.114+0000 7f2027fff700 1 -- 172.21.15.52:0/466148752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c172cab60 msgr2=0x559c172d7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:01.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.114+0000 7f2027fff700 1 --2- 172.21.15.52:0/466148752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c172cab60 0x559c172d7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.114+0000 7f2027fff700 1 -- 172.21.15.52:0/466148752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c172cf8b0 msgr2=0x559c172d48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:01.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.114+0000 7f2027fff700 1 --2- 172.21.15.52:0/466148752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c172cf8b0 0x559c172d48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.136 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.114+0000 7f2027fff700 1 -- 172.21.15.52:0/466148752 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f201400d8a0 con 0x559c172cb470 2023-11-27T15:27:01.137 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.114+0000 7f202d36d700 1 --2- 172.21.15.52:0/466148752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c172cf8b0 0x559c172d48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:01.137 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.114+0000 7f2027fff700 1 --2- 172.21.15.52:0/466148752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c172cb470 0x559c172d4270 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f20140049b0 tx=0x7f2014004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a81b54b794431d2b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:01.137 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.114+0000 7f2007fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:27:01.137 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.114+0000 7f2007fff700 1 -- 172.21.15.52:0/466148752 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2014020070 con 0x559c172cb470 2023-11-27T15:27:01.137 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.114+0000 7f2007fff700 1 -- 172.21.15.52:0/466148752 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f20140054e0 con 0x559c172cb470 2023-11-27T15:27:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.114+0000 7f203af5cb80 1 -- 172.21.15.52:0/466148752 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c172bdb20 con 0x559c172cb470 2023-11-27T15:27:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.115+0000 7f2007fff700 1 -- 172.21.15.52:0/466148752 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20140056c0 con 0x559c172cb470 2023-11-27T15:27:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.115+0000 7f203af5cb80 10 client.16989 fetch_fsmap learned FSMap version 34 2023-11-27T15:27:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.115+0000 7f203af5cb80 1 -- 172.21.15.52:0/466148752 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x559c171e2190 con 0x559c172cb470 2023-11-27T15:27:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.115+0000 7f2007fff700 1 -- 172.21.15.52:0/466148752 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559c172bdb20 con 0x559c172cb470 2023-11-27T15:27:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.115+0000 7f2007fff700 1 -- 172.21.15.52:0/466148752 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2014011a40 con 0x559c172cb470 2023-11-27T15:27:01.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.115+0000 7f203af5cb80 10 client.16989 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:27:01.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.115+0000 7f203af5cb80 10 client.16989 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:27:01.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.115+0000 7f203af5cb80 20 client.16989 populate_metadata read hostname 'smithi052' 2023-11-27T15:27:01.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.115+0000 7f203af5cb80 1 --2- 172.21.15.52:0/466148752 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x559c172e46f0 0x559c172e6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.115+0000 7f203af5cb80 4 client.16989 mds_command: new command op to 24472 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:27:01.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.115+0000 7f203af5cb80 1 -- 172.21.15.52:0/466148752 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c171de0a0 con 0x559c172e46f0 2023-11-27T15:27:01.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.115+0000 7f202cb6c700 1 --2- 172.21.15.52:0/466148752 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x559c172e46f0 0x559c172e6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.116+0000 7f2007fff700 10 client.16989 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:27:01.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.116+0000 7f202cb6c700 1 --2- 172.21.15.52:0/466148752 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x559c172e46f0 0x559c172e6b00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eeadf6a303827b14 server_cookie=7b73cce1f152a634 in_seq=0 out_seq=0 2023-11-27T15:27:01.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.116+0000 7f2007fff700 1 -- 172.21.15.52:0/466148752 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559c171de0a0 con 0x559c172e46f0 2023-11-27T15:27:01.140 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.116+0000 7f2007fff700 10 client.16989 handle_command_reply: tid=0 2023-11-27T15:27:01.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.116+0000 7f203af5cb80 1 client.16989 shutdown 2023-11-27T15:27:01.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.116+0000 7f203af5cb80 1 -- 172.21.15.52:0/466148752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c172cb470 msgr2=0x559c172d4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:01.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.116+0000 7f203af5cb80 1 --2- 172.21.15.52:0/466148752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c172cb470 0x559c172d4270 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f20140049b0 tx=0x7f2014004a90 comp rx=0 tx=0).stop 2023-11-27T15:27:01.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.116+0000 7f203af5cb80 1 -- 172.21.15.52:0/466148752 shutdown_connections 2023-11-27T15:27:01.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.116+0000 7f203af5cb80 1 --2- 172.21.15.52:0/466148752 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x559c172e46f0 0x559c172e6b00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.116+0000 7f203af5cb80 1 --2- 172.21.15.52:0/466148752 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c172cf8b0 0x559c172d48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.116+0000 7f203af5cb80 1 --2- 172.21.15.52:0/466148752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c172cab60 0x559c172d7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.116+0000 7f2007fff700 0 client.16989 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:27:01.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.116+0000 7f203af5cb80 1 --2- 172.21.15.52:0/466148752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c172cb470 0x559c172d4270 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.116+0000 7f203af5cb80 1 -- 172.21.15.52:0/466148752 >> 172.21.15.52:0/466148752 conn(0x559c172c0360 msgr2=0x559c172d0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:01.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.116+0000 7f203af5cb80 1 -- 172.21.15.52:0/466148752 shutdown_connections 2023-11-27T15:27:01.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.117+0000 7f203af5cb80 1 -- 172.21.15.52:0/466148752 wait complete. 2023-11-27T15:27:01.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.117+0000 7f203af5cb80 20 client.16989 trim_cache size 0 max 16384 2023-11-27T15:27:01.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.130+0000 7f203af5cb80 1 Processor -- start 2023-11-27T15:27:01.149 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.130+0000 7f203af5cb80 1 -- start start 2023-11-27T15:27:01.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.130+0000 7f203af5cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c172cab60 0x559c172c4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.130+0000 7f203af5cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c172cb470 0x559c172db9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.130+0000 7f203af5cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c172cf8b0 0x559c172d6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.130+0000 7f203af5cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c172cbbc0 con 0x559c172cb470 2023-11-27T15:27:01.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.130+0000 7f203af5cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c172bfee0 con 0x559c172cab60 2023-11-27T15:27:01.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.130+0000 7f203af5cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c172d27d0 con 0x559c172cf8b0 2023-11-27T15:27:01.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.130+0000 7f2027fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c172cb470 0x559c172db9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.130+0000 7f2027fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c172cb470 0x559c172db9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55764/0 (socket says 172.21.15.52:55764) 2023-11-27T15:27:01.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.130+0000 7f2027fff700 1 -- 172.21.15.52:0/3080698428 learned_addr learned my addr 172.21.15.52:0/3080698428 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:01.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.130+0000 7f202d36d700 1 --2- 172.21.15.52:0/3080698428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c172cf8b0 0x559c172d6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.130+0000 7f202cb6c700 1 --2- 172.21.15.52:0/3080698428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c172cab60 0x559c172c4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.130+0000 7f2027fff700 1 -- 172.21.15.52:0/3080698428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c172cf8b0 msgr2=0x559c172d6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:01.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.130+0000 7f2027fff700 1 --2- 172.21.15.52:0/3080698428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c172cf8b0 0x559c172d6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.130+0000 7f2027fff700 1 -- 172.21.15.52:0/3080698428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c172cab60 msgr2=0x559c172c4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:01.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.130+0000 7f2027fff700 1 --2- 172.21.15.52:0/3080698428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c172cab60 0x559c172c4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.130+0000 7f2027fff700 1 -- 172.21.15.52:0/3080698428 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c171e1ee0 con 0x559c172cb470 2023-11-27T15:27:01.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.130+0000 7f202d36d700 1 --2- 172.21.15.52:0/3080698428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c172cf8b0 0x559c172d6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:01.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.130+0000 7f202cb6c700 1 --2- 172.21.15.52:0/3080698428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c172cab60 0x559c172c4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:01.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.131+0000 7f2027fff700 1 --2- 172.21.15.52:0/3080698428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c172cb470 0x559c172db9a0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f201400f640 tx=0x7f201400dfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb5d8a0ae6bc52a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:01.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.131+0000 7f20137fe700 1 -- 172.21.15.52:0/3080698428 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2014020070 con 0x559c172cb470 2023-11-27T15:27:01.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.131+0000 7f20137fe700 1 -- 172.21.15.52:0/3080698428 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2014025470 con 0x559c172cb470 2023-11-27T15:27:01.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.131+0000 7f20137fe700 1 -- 172.21.15.52:0/3080698428 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20140043a0 con 0x559c172cb470 2023-11-27T15:27:01.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.131+0000 7f203af5cb80 1 -- 172.21.15.52:0/3080698428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c172cb470 msgr2=0x559c172db9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:01.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.131+0000 7f203af5cb80 1 --2- 172.21.15.52:0/3080698428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c172cb470 0x559c172db9a0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f201400f640 tx=0x7f201400dfd0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.131+0000 7f203af5cb80 1 -- 172.21.15.52:0/3080698428 shutdown_connections 2023-11-27T15:27:01.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.131+0000 7f203af5cb80 1 --2- 172.21.15.52:0/3080698428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c172cab60 0x559c172c4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.131+0000 7f203af5cb80 1 --2- 172.21.15.52:0/3080698428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c172cf8b0 0x559c172d6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.131+0000 7f203af5cb80 1 --2- 172.21.15.52:0/3080698428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c172cb470 0x559c172db9a0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.131+0000 7f203af5cb80 1 -- 172.21.15.52:0/3080698428 >> 172.21.15.52:0/3080698428 conn(0x559c172c0360 msgr2=0x559c172d1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:01.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.131+0000 7f203af5cb80 1 -- 172.21.15.52:0/3080698428 shutdown_connections 2023-11-27T15:27:01.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.131+0000 7f203af5cb80 1 -- 172.21.15.52:0/3080698428 wait complete. 2023-11-27T15:27:01.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.132+0000 7f203af5cb80 1 -- start start 2023-11-27T15:27:01.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.132+0000 7f203af5cb80 1 Processor -- start 2023-11-27T15:27:01.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.132+0000 7f203af5cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c172cab60 0x559c172d9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.132+0000 7f203af5cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c172cb470 0x559c172da0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.132+0000 7f203af5cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c172cf8b0 0x559c173375b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.132+0000 7f203af5cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c172da840 con 0x559c172cb470 2023-11-27T15:27:01.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.132+0000 7f203af5cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c17337af0 con 0x559c172cf8b0 2023-11-27T15:27:01.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.132+0000 7f203af5cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c17337c60 con 0x559c172cab60 2023-11-27T15:27:01.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.132+0000 7f203af5cb80 10 client.0 _ll_register_callbacks cb 0x7f20200096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:01.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.132+0000 7f2027fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c172cb470 0x559c172da0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.132+0000 7f2027fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c172cb470 0x559c172da0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55770/0 (socket says 172.21.15.52:55770) 2023-11-27T15:27:01.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.132+0000 7f2027fff700 1 -- 172.21.15.52:0/2164355153 learned_addr learned my addr 172.21.15.52:0/2164355153 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:01.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.132+0000 7f202d36d700 1 --2- 172.21.15.52:0/2164355153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c172cf8b0 0x559c173375b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.132+0000 7f202cb6c700 1 --2- 172.21.15.52:0/2164355153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c172cab60 0x559c172d9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.132+0000 7f2027fff700 1 -- 172.21.15.52:0/2164355153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c172cab60 msgr2=0x559c172d9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:01.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.132+0000 7f2027fff700 1 --2- 172.21.15.52:0/2164355153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c172cab60 0x559c172d9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.132+0000 7f2027fff700 1 -- 172.21.15.52:0/2164355153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c172cf8b0 msgr2=0x559c173375b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:01.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.132+0000 7f2027fff700 1 --2- 172.21.15.52:0/2164355153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c172cf8b0 0x559c173375b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.133+0000 7f2027fff700 1 -- 172.21.15.52:0/2164355153 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559c171e2190 con 0x559c172cb470 2023-11-27T15:27:01.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.133+0000 7f202cb6c700 1 --2- 172.21.15.52:0/2164355153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c172cab60 0x559c172d9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:01.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.133+0000 7f202d36d700 1 --2- 172.21.15.52:0/2164355153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c172cf8b0 0x559c173375b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:01.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.133+0000 7f2027fff700 1 --2- 172.21.15.52:0/2164355153 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c172cb470 0x559c172da0c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f2014004ad0 tx=0x7f201400d5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c870680175d6aa0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:01.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.133+0000 7f2007fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:27:01.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.133+0000 7f2007fff700 1 -- 172.21.15.52:0/2164355153 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2014020040 con 0x559c172cb470 2023-11-27T15:27:01.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.133+0000 7f2007fff700 1 -- 172.21.15.52:0/2164355153 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f2014000e40 con 0x559c172cb470 2023-11-27T15:27:01.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.133+0000 7f203af5cb80 1 -- 172.21.15.52:0/2164355153 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2014004cf0 con 0x559c172cb470 2023-11-27T15:27:01.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.133+0000 7f2007fff700 1 -- 172.21.15.52:0/2164355153 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2014004030 con 0x559c172cb470 2023-11-27T15:27:01.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.133+0000 7f203af5cb80 10 client.16995 fetch_fsmap learned FSMap version 34 2023-11-27T15:27:01.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.133+0000 7f203af5cb80 1 -- 172.21.15.52:0/2164355153 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x559c17032c60 con 0x559c172cb470 2023-11-27T15:27:01.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.133+0000 7f2007fff700 1 -- 172.21.15.52:0/2164355153 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2014043020 con 0x559c172cb470 2023-11-27T15:27:01.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.133+0000 7f2007fff700 1 -- 172.21.15.52:0/2164355153 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2014025630 con 0x559c172cb470 2023-11-27T15:27:01.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.133+0000 7f203af5cb80 10 client.16995 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:27:01.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.133+0000 7f203af5cb80 10 client.16995 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:27:01.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.133+0000 7f203af5cb80 20 client.16995 populate_metadata read hostname 'smithi052' 2023-11-27T15:27:01.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.133+0000 7f203af5cb80 1 --2- 172.21.15.52:0/2164355153 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x559c17338f10 0x559c1733b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.133+0000 7f203af5cb80 4 client.16995 mds_command: new command op to 24472 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:27:01.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.133+0000 7f203af5cb80 1 -- 172.21.15.52:0/2164355153 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559c171ea660 con 0x559c17338f10 2023-11-27T15:27:01.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.134+0000 7f202cb6c700 1 --2- 172.21.15.52:0/2164355153 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x559c17338f10 0x559c1733b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.134+0000 7f2007fff700 10 client.16995 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:27:01.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.134+0000 7f202cb6c700 1 --2- 172.21.15.52:0/2164355153 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x559c17338f10 0x559c1733b320 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=471354e07006b9e6 server_cookie=7771b39cb5c90526 in_seq=0 out_seq=0 2023-11-27T15:27:01.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.134+0000 7f2007fff700 1 -- 172.21.15.52:0/2164355153 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559c171ea660 con 0x559c17338f10 2023-11-27T15:27:01.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.134+0000 7f2007fff700 10 client.16995 handle_command_reply: tid=0 2023-11-27T15:27:01.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.134+0000 7f203af5cb80 1 client.16995 shutdown 2023-11-27T15:27:01.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.134+0000 7f203af5cb80 1 -- 172.21.15.52:0/2164355153 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c172cb470 msgr2=0x559c172da0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:01.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.134+0000 7f203af5cb80 1 --2- 172.21.15.52:0/2164355153 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c172cb470 0x559c172da0c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f2014004ad0 tx=0x7f201400d5f0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.134+0000 7f203af5cb80 1 -- 172.21.15.52:0/2164355153 shutdown_connections 2023-11-27T15:27:01.162 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.134+0000 7f203af5cb80 1 --2- 172.21.15.52:0/2164355153 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x559c17338f10 0x559c1733b320 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.134+0000 7f203af5cb80 1 --2- 172.21.15.52:0/2164355153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c172cf8b0 0x559c173375b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.134+0000 7f203af5cb80 1 --2- 172.21.15.52:0/2164355153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559c172cab60 0x559c172d9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.134+0000 7f2007fff700 0 client.16995 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:27:01.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.134+0000 7f203af5cb80 1 --2- 172.21.15.52:0/2164355153 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c172cb470 0x559c172da0c0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.134+0000 7f203af5cb80 1 -- 172.21.15.52:0/2164355153 >> 172.21.15.52:0/2164355153 conn(0x559c172c0360 msgr2=0x559c172d1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:01.163 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.134+0000 7f203af5cb80 1 -- 172.21.15.52:0/2164355153 shutdown_connections 2023-11-27T15:27:01.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.134+0000 7f203af5cb80 1 -- 172.21.15.52:0/2164355153 wait complete. 2023-11-27T15:27:01.164 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.135+0000 7f203af5cb80 20 client.16995 trim_cache size 0 max 16384 2023-11-27T15:27:01.164 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:27:01.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.135+0000 7f202edd0700 1 -- 172.21.15.52:0/605694517 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f20180a12c0 msgr2=0x7f20180a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:01.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.135+0000 7f202edd0700 1 --2- 172.21.15.52:0/605694517 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f20180a12c0 0x7f20180a3770 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f201c00b5c0 tx=0x7f201c005960 comp rx=0 tx=0).stop 2023-11-27T15:27:01.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.135+0000 7f202edd0700 1 -- 172.21.15.52:0/605694517 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f202810cba0 msgr2=0x7f202819cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:01.166 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.135+0000 7f202edd0700 1 --2- 172.21.15.52:0/605694517 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f202810cba0 0x7f202819cfe0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f2020007ee0 tx=0x7f202000d3b0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.135+0000 7f202edd0700 1 -- 172.21.15.52:0/605694517 shutdown_connections 2023-11-27T15:27:01.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.135+0000 7f202edd0700 1 --2- 172.21.15.52:0/605694517 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f20180a12c0 0x7f20180a3770 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.135+0000 7f202edd0700 1 --2- 172.21.15.52:0/605694517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2028072e70 0x7f2028198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.135+0000 7f202edd0700 1 --2- 172.21.15.52:0/605694517 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f202810cba0 0x7f202819cfe0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.135+0000 7f202edd0700 1 --2- 172.21.15.52:0/605694517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2028073780 0x7f2028198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.167 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.135+0000 7f202edd0700 1 -- 172.21.15.52:0/605694517 >> 172.21.15.52:0/605694517 conn(0x7f2028077170 msgr2=0x7f20280fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:01.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.135+0000 7f202edd0700 1 -- 172.21.15.52:0/605694517 shutdown_connections 2023-11-27T15:27:01.168 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.135+0000 7f202edd0700 1 -- 172.21.15.52:0/605694517 wait complete. 2023-11-27T15:27:01.168 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-27T15:27:01.277 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:01 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/44642392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:27:01.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.292+0000 7f637acef700 1 -- 172.21.15.52:0/1886112628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63740ff300 msgr2=0x7f637410b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:01.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.292+0000 7f637acef700 1 --2- 172.21.15.52:0/1886112628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63740ff300 0x7f637410b790 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f636c00b0a0 tx=0x7f636c00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.292+0000 7f637acef700 1 -- 172.21.15.52:0/1886112628 shutdown_connections 2023-11-27T15:27:01.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.292+0000 7f637acef700 1 --2- 172.21.15.52:0/1886112628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63740ff300 0x7f637410b790 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.292+0000 7f637acef700 1 --2- 172.21.15.52:0/1886112628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6374104960 0x7f6374104d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.292+0000 7f637acef700 1 --2- 172.21.15.52:0/1886112628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63740fe950 0x7f63740fedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.292+0000 7f637acef700 1 -- 172.21.15.52:0/1886112628 >> 172.21.15.52:0/1886112628 conn(0x7f6374075920 msgr2=0x7f6374075d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:01.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.292+0000 7f637acef700 1 -- 172.21.15.52:0/1886112628 shutdown_connections 2023-11-27T15:27:01.298 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.292+0000 7f637acef700 1 -- 172.21.15.52:0/1886112628 wait complete. 2023-11-27T15:27:01.300 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.292+0000 7f637acef700 1 Processor -- start 2023-11-27T15:27:01.300 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.293+0000 7f637acef700 1 -- start start 2023-11-27T15:27:01.301 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.293+0000 7f637acef700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63740fe950 0x7f637419a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.301 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.293+0000 7f637acef700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63740ff300 0x7f637419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.301 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.293+0000 7f637acef700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6374104960 0x7f6374195a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.301 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.293+0000 7f637acef700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63741961c0 con 0x7f63740fe950 2023-11-27T15:27:01.301 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.293+0000 7f637acef700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6374196330 con 0x7f6374104960 2023-11-27T15:27:01.301 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.293+0000 7f637acef700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63741964f0 con 0x7f63740ff300 2023-11-27T15:27:01.302 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.293+0000 7f6378a8b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63740fe950 0x7f637419a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.302 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.293+0000 7f6378a8b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63740fe950 0x7f637419a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55782/0 (socket says 172.21.15.52:55782) 2023-11-27T15:27:01.302 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.293+0000 7f6378a8b700 1 -- 172.21.15.52:0/1426793095 learned_addr learned my addr 172.21.15.52:0/1426793095 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:01.302 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.293+0000 7f6373fff700 1 --2- 172.21.15.52:0/1426793095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63740ff300 0x7f637419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.302 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.293+0000 7f637928c700 1 --2- 172.21.15.52:0/1426793095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6374104960 0x7f6374195a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.302 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.293+0000 7f6378a8b700 1 -- 172.21.15.52:0/1426793095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63740ff300 msgr2=0x7f637419a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:01.302 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.293+0000 7f6378a8b700 1 --2- 172.21.15.52:0/1426793095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63740ff300 0x7f637419a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.303 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.293+0000 7f6378a8b700 1 -- 172.21.15.52:0/1426793095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6374104960 msgr2=0x7f6374195a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:01.303 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.293+0000 7f6378a8b700 1 --2- 172.21.15.52:0/1426793095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6374104960 0x7f6374195a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.303 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.293+0000 7f6378a8b700 1 -- 172.21.15.52:0/1426793095 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f636c009d00 con 0x7f63740fe950 2023-11-27T15:27:01.303 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.293+0000 7f6373fff700 1 --2- 172.21.15.52:0/1426793095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63740ff300 0x7f637419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:01.303 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.293+0000 7f6378a8b700 1 --2- 172.21.15.52:0/1426793095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63740fe950 0x7f637419a120 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f636800e970 tx=0x7f636800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:01.303 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.294+0000 7f6371ffb700 1 -- 172.21.15.52:0/1426793095 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6368009790 con 0x7f63740fe950 2023-11-27T15:27:01.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.294+0000 7f6371ffb700 1 -- 172.21.15.52:0/1426793095 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63680044e0 con 0x7f63740fe950 2023-11-27T15:27:01.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.294+0000 7f637acef700 1 -- 172.21.15.52:0/1426793095 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f637410ea40 con 0x7f63740fe950 2023-11-27T15:27:01.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.294+0000 7f6371ffb700 1 -- 172.21.15.52:0/1426793095 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6368003e30 con 0x7f63740fe950 2023-11-27T15:27:01.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.294+0000 7f637acef700 1 -- 172.21.15.52:0/1426793095 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f637410e8b0 con 0x7f63740fe950 2023-11-27T15:27:01.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.296+0000 7f6371ffb700 1 -- 172.21.15.52:0/1426793095 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f6368003f90 con 0x7f63740fe950 2023-11-27T15:27:01.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.296+0000 7f6371ffb700 1 --2- 172.21.15.52:0/1426793095 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f63640a12c0 0x7f63640a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.311 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.297+0000 7f6373fff700 1 --2- 172.21.15.52:0/1426793095 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f63640a12c0 0x7f63640a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.297+0000 7f6371ffb700 1 -- 172.21.15.52:0/1426793095 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f6368014070 con 0x7f63740fe950 2023-11-27T15:27:01.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.297+0000 7f6373fff700 1 --2- 172.21.15.52:0/1426793095 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f63640a12c0 0x7f63640a3770 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6374196bb0 tx=0x7f6360009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:01.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.297+0000 7f6386e7bb80 1 Processor -- start 2023-11-27T15:27:01.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.297+0000 7f6386e7bb80 1 -- start start 2023-11-27T15:27:01.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.297+0000 7f6386e7bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56450cd00970 0x56450cd00d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.312 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.297+0000 7f6386e7bb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56450cd01280 0x56450cd0eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.298+0000 7f6386e7bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56450cd0f530 0x56450cd11910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.298+0000 7f6386e7bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450cc06060 con 0x56450cd0f530 2023-11-27T15:27:01.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.298+0000 7f6386e7bb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450cc588a0 con 0x56450cd01280 2023-11-27T15:27:01.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.298+0000 7f6386e7bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450cc51d30 con 0x56450cd00970 2023-11-27T15:27:01.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.298+0000 7f637928c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56450cd0f530 0x56450cd11910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.298+0000 7f637928c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56450cd0f530 0x56450cd11910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55794/0 (socket says 172.21.15.52:55794) 2023-11-27T15:27:01.313 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.298+0000 7f637928c700 1 -- 172.21.15.52:0/2278953901 learned_addr learned my addr 172.21.15.52:0/2278953901 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:01.314 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.298+0000 7f6373fff700 1 --2- 172.21.15.52:0/2278953901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56450cd01280 0x56450cd0eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.314 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.298+0000 7f6378a8b700 1 --2- 172.21.15.52:0/2278953901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56450cd00970 0x56450cd00d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.314 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.298+0000 7f637928c700 1 -- 172.21.15.52:0/2278953901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56450cd00970 msgr2=0x56450cd00d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:01.314 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.298+0000 7f637928c700 1 --2- 172.21.15.52:0/2278953901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56450cd00970 0x56450cd00d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.314 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.298+0000 7f637928c700 1 -- 172.21.15.52:0/2278953901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56450cd01280 msgr2=0x56450cd0eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:01.314 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.298+0000 7f637928c700 1 --2- 172.21.15.52:0/2278953901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56450cd01280 0x56450cd0eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.298+0000 7f637928c700 1 -- 172.21.15.52:0/2278953901 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56450cad3630 con 0x56450cd0f530 2023-11-27T15:27:01.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.298+0000 7f637928c700 1 --2- 172.21.15.52:0/2278953901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56450cd0f530 0x56450cd11910 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f636c009350 tx=0x7f636c009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f31fc9398dbe69 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:01.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.298+0000 7f635effd700 1 -- 172.21.15.52:0/2278953901 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c00e050 con 0x56450cd0f530 2023-11-27T15:27:01.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.298+0000 7f635effd700 1 -- 172.21.15.52:0/2278953901 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f636c009b80 con 0x56450cd0f530 2023-11-27T15:27:01.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.298+0000 7f635effd700 1 -- 172.21.15.52:0/2278953901 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c0052a0 con 0x56450cd0f530 2023-11-27T15:27:01.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.299+0000 7f6386e7bb80 1 -- 172.21.15.52:0/2278953901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56450cd0f530 msgr2=0x56450cd11910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:01.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.299+0000 7f6386e7bb80 1 --2- 172.21.15.52:0/2278953901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56450cd0f530 0x56450cd11910 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f636c009350 tx=0x7f636c009380 comp rx=0 tx=0).stop 2023-11-27T15:27:01.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.299+0000 7f6386e7bb80 1 -- 172.21.15.52:0/2278953901 shutdown_connections 2023-11-27T15:27:01.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.299+0000 7f6386e7bb80 1 --2- 172.21.15.52:0/2278953901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56450cd0f530 0x56450cd11910 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.299+0000 7f6386e7bb80 1 --2- 172.21.15.52:0/2278953901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56450cd00970 0x56450cd00d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.299+0000 7f6386e7bb80 1 --2- 172.21.15.52:0/2278953901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56450cd01280 0x56450cd0eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.299+0000 7f6386e7bb80 1 -- 172.21.15.52:0/2278953901 >> 172.21.15.52:0/2278953901 conn(0x56450ccf6360 msgr2=0x56450ccfbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:01.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.299+0000 7f6386e7bb80 1 -- 172.21.15.52:0/2278953901 shutdown_connections 2023-11-27T15:27:01.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.299+0000 7f6386e7bb80 1 -- 172.21.15.52:0/2278953901 wait complete. 2023-11-27T15:27:01.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.299+0000 7f6386e7bb80 1 -- start start 2023-11-27T15:27:01.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.300+0000 7f6386e7bb80 1 Processor -- start 2023-11-27T15:27:01.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.300+0000 7f6386e7bb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56450cd00970 0x56450cd0f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.300+0000 7f6386e7bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56450cd01280 0x56450cd0fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.300+0000 7f6386e7bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56450cd0f530 0x56450cd1a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.317 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.300+0000 7f6386e7bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450cbf38e0 con 0x56450cd01280 2023-11-27T15:27:01.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.300+0000 7f6386e7bb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450cbf3580 con 0x56450cd00970 2023-11-27T15:27:01.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.300+0000 7f6386e7bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450cad5920 con 0x56450cd0f530 2023-11-27T15:27:01.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.300+0000 7f6386e7bb80 10 client.0 _ll_register_callbacks cb 0x56450cc11780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:01.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.300+0000 7f6373fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56450cd01280 0x56450cd0fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.300+0000 7f6373fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56450cd01280 0x56450cd0fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55798/0 (socket says 172.21.15.52:55798) 2023-11-27T15:27:01.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.300+0000 7f6373fff700 1 -- 172.21.15.52:0/917307262 learned_addr learned my addr 172.21.15.52:0/917307262 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:01.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.300+0000 7f6378a8b700 1 --2- 172.21.15.52:0/917307262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56450cd00970 0x56450cd0f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.300+0000 7f637928c700 1 --2- 172.21.15.52:0/917307262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56450cd0f530 0x56450cd1a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.300+0000 7f6373fff700 1 -- 172.21.15.52:0/917307262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56450cd0f530 msgr2=0x56450cd1a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:01.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.300+0000 7f6373fff700 1 --2- 172.21.15.52:0/917307262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56450cd0f530 0x56450cd1a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.300+0000 7f6373fff700 1 -- 172.21.15.52:0/917307262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56450cd00970 msgr2=0x56450cd0f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:01.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.300+0000 7f6373fff700 1 --2- 172.21.15.52:0/917307262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56450cd00970 0x56450cd0f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.300+0000 7f6373fff700 1 -- 172.21.15.52:0/917307262 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f637410c090 con 0x56450cd01280 2023-11-27T15:27:01.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.301+0000 7f6373fff700 1 --2- 172.21.15.52:0/917307262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56450cd01280 0x56450cd0fe40 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f63600045a0 tx=0x7f63600047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b58ad7730664e7a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:01.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.301+0000 7f6353fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:27:01.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.301+0000 7f6353fff700 1 -- 172.21.15.52:0/917307262 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6360003ea0 con 0x56450cd01280 2023-11-27T15:27:01.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.301+0000 7f6353fff700 1 -- 172.21.15.52:0/917307262 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f6360018040 con 0x56450cd01280 2023-11-27T15:27:01.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.301+0000 7f6386e7bb80 1 -- 172.21.15.52:0/917307262 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56450cc140a0 con 0x56450cd01280 2023-11-27T15:27:01.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.301+0000 7f6353fff700 1 -- 172.21.15.52:0/917307262 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6360011c60 con 0x56450cd01280 2023-11-27T15:27:01.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.301+0000 7f6386e7bb80 10 client.17007 fetch_fsmap learned FSMap version 34 2023-11-27T15:27:01.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.301+0000 7f6386e7bb80 1 -- 172.21.15.52:0/917307262 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56450cc18190 con 0x56450cd01280 2023-11-27T15:27:01.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.301+0000 7f6353fff700 1 -- 172.21.15.52:0/917307262 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56450cc140a0 con 0x56450cd01280 2023-11-27T15:27:01.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.301+0000 7f6353fff700 1 -- 172.21.15.52:0/917307262 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6360026020 con 0x56450cd01280 2023-11-27T15:27:01.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.301+0000 7f6386e7bb80 10 client.17007 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:27:01.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.301+0000 7f6386e7bb80 10 client.17007 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:27:01.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.301+0000 7f6386e7bb80 20 client.17007 populate_metadata read hostname 'smithi052' 2023-11-27T15:27:01.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.301+0000 7f6386e7bb80 1 --2- 172.21.15.52:0/917307262 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x56450cd1ec20 0x56450cd21030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.301+0000 7f6386e7bb80 4 client.17007 mds_command: new command op to 24494 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:27:01.322 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.301+0000 7f6386e7bb80 1 -- 172.21.15.52:0/917307262 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56450cc20660 con 0x56450cd1ec20 2023-11-27T15:27:01.322 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.302+0000 7f6378a8b700 1 --2- 172.21.15.52:0/917307262 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x56450cd1ec20 0x56450cd21030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.322 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.302+0000 7f6353fff700 10 client.17007 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:27:01.322 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.302+0000 7f6378a8b700 1 --2- 172.21.15.52:0/917307262 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x56450cd1ec20 0x56450cd21030 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b0a3dc0d68fb46c8 server_cookie=e295b01f9868c229 in_seq=0 out_seq=0 2023-11-27T15:27:01.322 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.303+0000 7f6353fff700 1 -- 172.21.15.52:0/917307262 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56450cc20660 con 0x56450cd1ec20 2023-11-27T15:27:01.322 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.303+0000 7f6353fff700 10 client.17007 handle_command_reply: tid=0 2023-11-27T15:27:01.322 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.303+0000 7f6386e7bb80 1 client.17007 shutdown 2023-11-27T15:27:01.323 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.303+0000 7f6386e7bb80 1 -- 172.21.15.52:0/917307262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56450cd01280 msgr2=0x56450cd0fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:01.323 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.303+0000 7f6386e7bb80 1 --2- 172.21.15.52:0/917307262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56450cd01280 0x56450cd0fe40 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f63600045a0 tx=0x7f63600047c0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.323 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.303+0000 7f6386e7bb80 1 -- 172.21.15.52:0/917307262 shutdown_connections 2023-11-27T15:27:01.323 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.303+0000 7f6386e7bb80 1 --2- 172.21.15.52:0/917307262 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x56450cd1ec20 0x56450cd21030 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.323 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.303+0000 7f6386e7bb80 1 --2- 172.21.15.52:0/917307262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56450cd00970 0x56450cd0f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.323 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.303+0000 7f6353fff700 0 client.17007 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:27:01.324 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.303+0000 7f6386e7bb80 1 --2- 172.21.15.52:0/917307262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56450cd0f530 0x56450cd1a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.324 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.303+0000 7f6386e7bb80 1 --2- 172.21.15.52:0/917307262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56450cd01280 0x56450cd0fe40 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.324 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.303+0000 7f6386e7bb80 1 -- 172.21.15.52:0/917307262 >> 172.21.15.52:0/917307262 conn(0x56450ccf6360 msgr2=0x56450ccfade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:01.324 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.303+0000 7f6386e7bb80 1 -- 172.21.15.52:0/917307262 shutdown_connections 2023-11-27T15:27:01.324 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.303+0000 7f6386e7bb80 1 -- 172.21.15.52:0/917307262 wait complete. 2023-11-27T15:27:01.324 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.303+0000 7f6386e7bb80 20 client.17007 trim_cache size 0 max 16384 2023-11-27T15:27:01.330 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:01 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/44642392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:27:01.330 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.316+0000 7f6386e7bb80 1 Processor -- start 2023-11-27T15:27:01.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.316+0000 7f6386e7bb80 1 -- start start 2023-11-27T15:27:01.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.316+0000 7f6386e7bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56450cd00970 0x56450cd14930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.316+0000 7f6386e7bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56450cd01280 0x56450cd08de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.316+0000 7f6386e7bb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56450cd0f530 0x56450cd09920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.316+0000 7f6386e7bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450cd023e0 con 0x56450cd01280 2023-11-27T15:27:01.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.316+0000 7f6386e7bb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450ccf3660 con 0x56450cd0f530 2023-11-27T15:27:01.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.316+0000 7f6386e7bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450cd0cba0 con 0x56450cd00970 2023-11-27T15:27:01.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.316+0000 7f6373fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56450cd01280 0x56450cd08de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.316+0000 7f6373fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56450cd01280 0x56450cd08de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55806/0 (socket says 172.21.15.52:55806) 2023-11-27T15:27:01.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.316+0000 7f6373fff700 1 -- 172.21.15.52:0/1368500042 learned_addr learned my addr 172.21.15.52:0/1368500042 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:01.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.316+0000 7f6378a8b700 1 --2- 172.21.15.52:0/1368500042 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56450cd00970 0x56450cd14930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.317+0000 7f637928c700 1 --2- 172.21.15.52:0/1368500042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56450cd0f530 0x56450cd09920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.317+0000 7f6373fff700 1 -- 172.21.15.52:0/1368500042 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56450cd00970 msgr2=0x56450cd14930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:01.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.317+0000 7f6373fff700 1 --2- 172.21.15.52:0/1368500042 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56450cd00970 0x56450cd14930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.317+0000 7f6373fff700 1 -- 172.21.15.52:0/1368500042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56450cd0f530 msgr2=0x56450cd09920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:01.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.317+0000 7f6373fff700 1 --2- 172.21.15.52:0/1368500042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56450cd0f530 0x56450cd09920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.317+0000 7f6373fff700 1 -- 172.21.15.52:0/1368500042 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56450cc17ee0 con 0x56450cd01280 2023-11-27T15:27:01.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.317+0000 7f6373fff700 1 --2- 172.21.15.52:0/1368500042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56450cd01280 0x56450cd08de0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f6360011ad0 tx=0x7f6360009a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c87dc2e24f175978 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:01.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.317+0000 7f635f7fe700 1 -- 172.21.15.52:0/1368500042 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6360009dc0 con 0x56450cd01280 2023-11-27T15:27:01.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.317+0000 7f635f7fe700 1 -- 172.21.15.52:0/1368500042 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6360031d80 con 0x56450cd01280 2023-11-27T15:27:01.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.317+0000 7f635f7fe700 1 -- 172.21.15.52:0/1368500042 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636001f900 con 0x56450cd01280 2023-11-27T15:27:01.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.318+0000 7f6386e7bb80 1 -- 172.21.15.52:0/1368500042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56450cd01280 msgr2=0x56450cd08de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:01.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.318+0000 7f6386e7bb80 1 --2- 172.21.15.52:0/1368500042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56450cd01280 0x56450cd08de0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f6360011ad0 tx=0x7f6360009a80 comp rx=0 tx=0).stop 2023-11-27T15:27:01.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.318+0000 7f6386e7bb80 1 -- 172.21.15.52:0/1368500042 shutdown_connections 2023-11-27T15:27:01.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.318+0000 7f6386e7bb80 1 --2- 172.21.15.52:0/1368500042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56450cd0f530 0x56450cd09920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.318+0000 7f6386e7bb80 1 --2- 172.21.15.52:0/1368500042 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56450cd00970 0x56450cd14930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.318+0000 7f6386e7bb80 1 --2- 172.21.15.52:0/1368500042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56450cd01280 0x56450cd08de0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.318+0000 7f6386e7bb80 1 -- 172.21.15.52:0/1368500042 >> 172.21.15.52:0/1368500042 conn(0x56450ccf6360 msgr2=0x56450cd1afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:01.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.318+0000 7f6386e7bb80 1 -- 172.21.15.52:0/1368500042 shutdown_connections 2023-11-27T15:27:01.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.318+0000 7f6386e7bb80 1 -- 172.21.15.52:0/1368500042 wait complete. 2023-11-27T15:27:01.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.318+0000 7f6386e7bb80 1 -- start start 2023-11-27T15:27:01.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.318+0000 7f6386e7bb80 1 Processor -- start 2023-11-27T15:27:01.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.318+0000 7f6386e7bb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56450cd00970 0x56450cd10780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.318+0000 7f6386e7bb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56450cd01280 0x56450cd10cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.318+0000 7f6386e7bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56450cd0f530 0x56450cd71ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.318+0000 7f6386e7bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450cd115b0 con 0x56450cd0f530 2023-11-27T15:27:01.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.318+0000 7f6386e7bb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450cd72010 con 0x56450cd01280 2023-11-27T15:27:01.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.318+0000 7f6386e7bb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450cd72180 con 0x56450cd00970 2023-11-27T15:27:01.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.319+0000 7f6386e7bb80 10 client.0 _ll_register_callbacks cb 0x7f63600043d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:01.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.319+0000 7f637928c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56450cd0f530 0x56450cd71ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.319+0000 7f637928c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56450cd0f530 0x56450cd71ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55814/0 (socket says 172.21.15.52:55814) 2023-11-27T15:27:01.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.319+0000 7f637928c700 1 -- 172.21.15.52:0/4258740216 learned_addr learned my addr 172.21.15.52:0/4258740216 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:01.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.319+0000 7f6373fff700 1 --2- 172.21.15.52:0/4258740216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56450cd01280 0x56450cd10cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.319+0000 7f6378a8b700 1 --2- 172.21.15.52:0/4258740216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56450cd00970 0x56450cd10780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.319+0000 7f637928c700 1 -- 172.21.15.52:0/4258740216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56450cd00970 msgr2=0x56450cd10780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:01.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.319+0000 7f637928c700 1 --2- 172.21.15.52:0/4258740216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56450cd00970 0x56450cd10780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.319+0000 7f637928c700 1 -- 172.21.15.52:0/4258740216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56450cd01280 msgr2=0x56450cd10cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:01.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.319+0000 7f637928c700 1 --2- 172.21.15.52:0/4258740216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56450cd01280 0x56450cd10cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.319+0000 7f637928c700 1 -- 172.21.15.52:0/4258740216 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56450cc18190 con 0x56450cd0f530 2023-11-27T15:27:01.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.319+0000 7f637928c700 1 --2- 172.21.15.52:0/4258740216 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56450cd0f530 0x56450cd71ad0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f636c009350 tx=0x7f636c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd49454af5a6dc27 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:01.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.319+0000 7f6353fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:27:01.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.319+0000 7f6353fff700 1 -- 172.21.15.52:0/4258740216 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c0076a0 con 0x56450cd0f530 2023-11-27T15:27:01.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.319+0000 7f6353fff700 1 -- 172.21.15.52:0/4258740216 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f636c007800 con 0x56450cd0f530 2023-11-27T15:27:01.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.319+0000 7f6386e7bb80 1 -- 172.21.15.52:0/4258740216 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56450ccf3b20 con 0x56450cd0f530 2023-11-27T15:27:01.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.319+0000 7f6353fff700 1 -- 172.21.15.52:0/4258740216 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f636c00e020 con 0x56450cd0f530 2023-11-27T15:27:01.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.319+0000 7f6386e7bb80 10 client.17013 fetch_fsmap learned FSMap version 34 2023-11-27T15:27:01.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.319+0000 7f6386e7bb80 1 -- 172.21.15.52:0/4258740216 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56450ca68c60 con 0x56450cd0f530 2023-11-27T15:27:01.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.320+0000 7f6353fff700 1 -- 172.21.15.52:0/4258740216 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56450ccf3b20 con 0x56450cd0f530 2023-11-27T15:27:01.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.320+0000 7f6353fff700 1 -- 172.21.15.52:0/4258740216 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f636c00b0d0 con 0x56450cd0f530 2023-11-27T15:27:01.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.320+0000 7f6386e7bb80 10 client.17013 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:27:01.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.320+0000 7f6386e7bb80 10 client.17013 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:27:01.340 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.320+0000 7f6386e7bb80 20 client.17013 populate_metadata read hostname 'smithi052' 2023-11-27T15:27:01.340 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.320+0000 7f6386e7bb80 1 --2- 172.21.15.52:0/4258740216 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x56450cd73430 0x56450cd75810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.340 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.320+0000 7f6386e7bb80 4 client.17013 mds_command: new command op to 24494 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:27:01.340 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.320+0000 7f6386e7bb80 1 -- 172.21.15.52:0/4258740216 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56450caa0f70 con 0x56450cd73430 2023-11-27T15:27:01.340 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.320+0000 7f6378a8b700 1 --2- 172.21.15.52:0/4258740216 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x56450cd73430 0x56450cd75810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.340 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.321+0000 7f6353fff700 10 client.17013 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:27:01.340 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.321+0000 7f6378a8b700 1 --2- 172.21.15.52:0/4258740216 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x56450cd73430 0x56450cd75810 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f9e2a1a3b43d77dd server_cookie=98e39370263d9864 in_seq=0 out_seq=0 2023-11-27T15:27:01.341 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.321+0000 7f6353fff700 1 -- 172.21.15.52:0/4258740216 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56450caa0f70 con 0x56450cd73430 2023-11-27T15:27:01.341 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.321+0000 7f6353fff700 10 client.17013 handle_command_reply: tid=0 2023-11-27T15:27:01.341 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.321+0000 7f6386e7bb80 1 client.17013 shutdown 2023-11-27T15:27:01.341 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.321+0000 7f6386e7bb80 1 -- 172.21.15.52:0/4258740216 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56450cd0f530 msgr2=0x56450cd71ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:01.341 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.321+0000 7f6386e7bb80 1 --2- 172.21.15.52:0/4258740216 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56450cd0f530 0x56450cd71ad0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f636c009350 tx=0x7f636c004c80 comp rx=0 tx=0).stop 2023-11-27T15:27:01.341 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.321+0000 7f6386e7bb80 1 -- 172.21.15.52:0/4258740216 shutdown_connections 2023-11-27T15:27:01.341 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.321+0000 7f6386e7bb80 1 --2- 172.21.15.52:0/4258740216 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x56450cd73430 0x56450cd75810 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.342 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.321+0000 7f6386e7bb80 1 --2- 172.21.15.52:0/4258740216 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56450cd0f530 0x56450cd71ad0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.342 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.321+0000 7f6386e7bb80 1 --2- 172.21.15.52:0/4258740216 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56450cd00970 0x56450cd10780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.342 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.321+0000 7f6353fff700 0 client.17013 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:27:01.342 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.321+0000 7f6386e7bb80 1 --2- 172.21.15.52:0/4258740216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56450cd01280 0x56450cd10cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.342 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.321+0000 7f6386e7bb80 1 -- 172.21.15.52:0/4258740216 >> 172.21.15.52:0/4258740216 conn(0x56450ccf6360 msgr2=0x56450cd09920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:01.342 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.321+0000 7f6386e7bb80 1 -- 172.21.15.52:0/4258740216 shutdown_connections 2023-11-27T15:27:01.342 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.321+0000 7f6386e7bb80 1 -- 172.21.15.52:0/4258740216 wait complete. 2023-11-27T15:27:01.343 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.321+0000 7f6386e7bb80 20 client.17013 trim_cache size 0 max 16384 2023-11-27T15:27:01.343 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:27:01.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.323+0000 7f637acef700 1 -- 172.21.15.52:0/1426793095 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f63640a12c0 msgr2=0x7f63640a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:01.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.323+0000 7f637acef700 1 --2- 172.21.15.52:0/1426793095 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f63640a12c0 0x7f63640a3770 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6374196bb0 tx=0x7f6360009260 comp rx=0 tx=0).stop 2023-11-27T15:27:01.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.323+0000 7f637acef700 1 -- 172.21.15.52:0/1426793095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63740fe950 msgr2=0x7f637419a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:01.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.323+0000 7f637acef700 1 --2- 172.21.15.52:0/1426793095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63740fe950 0x7f637419a120 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f636800e970 tx=0x7f636800ed30 comp rx=0 tx=0).stop 2023-11-27T15:27:01.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.323+0000 7f637acef700 1 -- 172.21.15.52:0/1426793095 shutdown_connections 2023-11-27T15:27:01.344 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.323+0000 7f637acef700 1 --2- 172.21.15.52:0/1426793095 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f63640a12c0 0x7f63640a3770 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.345 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.323+0000 7f637acef700 1 --2- 172.21.15.52:0/1426793095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6374104960 0x7f6374195a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.345 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.323+0000 7f637acef700 1 --2- 172.21.15.52:0/1426793095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63740fe950 0x7f637419a120 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.345 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.323+0000 7f637acef700 1 --2- 172.21.15.52:0/1426793095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63740ff300 0x7f637419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.345 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.323+0000 7f637acef700 1 -- 172.21.15.52:0/1426793095 >> 172.21.15.52:0/1426793095 conn(0x7f6374075920 msgr2=0x7f63741024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:01.345 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.323+0000 7f637acef700 1 -- 172.21.15.52:0/1426793095 shutdown_connections 2023-11-27T15:27:01.345 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.323+0000 7f637acef700 1 -- 172.21.15.52:0/1426793095 wait complete. 2023-11-27T15:27:01.346 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:01 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/44642392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:27:01.346 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-27T15:27:01.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.470+0000 7f1918851700 1 -- 172.21.15.52:0/1420027594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1914104180 msgr2=0x7f1914104550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:01.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.470+0000 7f1918851700 1 --2- 172.21.15.52:0/1420027594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1914104180 0x7f1914104550 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f19080099c0 tx=0x7f1908009cd0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.470+0000 7f1918851700 1 -- 172.21.15.52:0/1420027594 shutdown_connections 2023-11-27T15:27:01.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.470+0000 7f1918851700 1 --2- 172.21.15.52:0/1420027594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1914100340 0x7f191410c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.470+0000 7f1918851700 1 --2- 172.21.15.52:0/1420027594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1914104180 0x7f1914104550 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.470+0000 7f1918851700 1 --2- 172.21.15.52:0/1420027594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19140ff990 0x7f19140ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.470+0000 7f1918851700 1 -- 172.21.15.52:0/1420027594 >> 172.21.15.52:0/1420027594 conn(0x7f1914075920 msgr2=0x7f1914075d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:01.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.470+0000 7f1918851700 1 -- 172.21.15.52:0/1420027594 shutdown_connections 2023-11-27T15:27:01.476 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.470+0000 7f1918851700 1 -- 172.21.15.52:0/1420027594 wait complete. 2023-11-27T15:27:01.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.470+0000 7f1918851700 1 Processor -- start 2023-11-27T15:27:01.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.471+0000 7f1918851700 1 -- start start 2023-11-27T15:27:01.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.471+0000 7f1918851700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19140ff990 0x7f19141963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.471+0000 7f1918851700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1914100340 0x7f19141968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.471+0000 7f1918851700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1914104180 0x7f191419ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.471+0000 7f1918851700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191419b390 con 0x7f19140ff990 2023-11-27T15:27:01.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.471+0000 7f1918851700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191419b500 con 0x7f1914100340 2023-11-27T15:27:01.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.471+0000 7f1918851700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191419b6c0 con 0x7f1914104180 2023-11-27T15:27:01.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.471+0000 7f191259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19140ff990 0x7f19141963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.471+0000 7f191259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19140ff990 0x7f19141963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55824/0 (socket says 172.21.15.52:55824) 2023-11-27T15:27:01.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.471+0000 7f191259c700 1 -- 172.21.15.52:0/909227118 learned_addr learned my addr 172.21.15.52:0/909227118 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:01.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.471+0000 7f1912d9d700 1 --2- 172.21.15.52:0/909227118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1914104180 0x7f191419ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.471+0000 7f1911d9b700 1 --2- 172.21.15.52:0/909227118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1914100340 0x7f19141968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.471+0000 7f191259c700 1 -- 172.21.15.52:0/909227118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1914104180 msgr2=0x7f191419ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:01.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.471+0000 7f191259c700 1 --2- 172.21.15.52:0/909227118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1914104180 0x7f191419ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.471+0000 7f191259c700 1 -- 172.21.15.52:0/909227118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1914100340 msgr2=0x7f19141968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:01.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.471+0000 7f191259c700 1 --2- 172.21.15.52:0/909227118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1914100340 0x7f19141968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.471+0000 7f191259c700 1 -- 172.21.15.52:0/909227118 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19080095f0 con 0x7f19140ff990 2023-11-27T15:27:01.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.471+0000 7f1912d9d700 1 --2- 172.21.15.52:0/909227118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1914104180 0x7f191419ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:01.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.471+0000 7f1911d9b700 1 --2- 172.21.15.52:0/909227118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1914100340 0x7f19141968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:01.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.471+0000 7f191259c700 1 --2- 172.21.15.52:0/909227118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19140ff990 0x7f19141963b0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f1908009990 tx=0x7f19080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:01.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.471+0000 7f19037fe700 1 -- 172.21.15.52:0/909227118 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190801d070 con 0x7f19140ff990 2023-11-27T15:27:01.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.472+0000 7f19037fe700 1 -- 172.21.15.52:0/909227118 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1908004010 con 0x7f19140ff990 2023-11-27T15:27:01.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.472+0000 7f19037fe700 1 -- 172.21.15.52:0/909227118 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190800fd30 con 0x7f19140ff990 2023-11-27T15:27:01.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.472+0000 7f1918851700 1 -- 172.21.15.52:0/909227118 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f191410f240 con 0x7f19140ff990 2023-11-27T15:27:01.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.472+0000 7f1918851700 1 -- 172.21.15.52:0/909227118 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f191410f0b0 con 0x7f19140ff990 2023-11-27T15:27:01.488 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.474+0000 7f19037fe700 1 -- 172.21.15.52:0/909227118 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f190800f460 con 0x7f19140ff990 2023-11-27T15:27:01.488 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.474+0000 7f19037fe700 1 --2- 172.21.15.52:0/909227118 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f18fc0a12c0 0x7f18fc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.488 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.475+0000 7f19037fe700 1 -- 172.21.15.52:0/909227118 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f19080cdd70 con 0x7f19140ff990 2023-11-27T15:27:01.488 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.475+0000 7f1911d9b700 1 --2- 172.21.15.52:0/909227118 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f18fc0a12c0 0x7f18fc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.488 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.475+0000 7f1911d9b700 1 --2- 172.21.15.52:0/909227118 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f18fc0a12c0 0x7f18fc0a3770 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1904005d90 tx=0x7f1904005d20 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:01.488 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.475+0000 7f19249ddb80 1 Processor -- start 2023-11-27T15:27:01.488 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.475+0000 7f19249ddb80 1 -- start start 2023-11-27T15:27:01.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.476+0000 7f19249ddb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556420d7f70 0x5556420d8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.476+0000 7f19249ddb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556420d8880 0x5556420dc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.476+0000 7f19249ddb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556420dccc0 0x5556420dd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.476+0000 7f19249ddb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x555641fd9060 con 0x5556420d8880 2023-11-27T15:27:01.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.476+0000 7f19249ddb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564202b8a0 con 0x5556420dccc0 2023-11-27T15:27:01.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.476+0000 7f19249ddb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555642024d30 con 0x5556420d7f70 2023-11-27T15:27:01.489 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.476+0000 7f1911d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556420d8880 0x5556420dc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.490 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.476+0000 7f1911d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556420d8880 0x5556420dc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55834/0 (socket says 172.21.15.52:55834) 2023-11-27T15:27:01.490 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.476+0000 7f1911d9b700 1 -- 172.21.15.52:0/1422230655 learned_addr learned my addr 172.21.15.52:0/1422230655 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:01.490 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.476+0000 7f191259c700 1 --2- 172.21.15.52:0/1422230655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556420d7f70 0x5556420d8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.490 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.476+0000 7f1912d9d700 1 --2- 172.21.15.52:0/1422230655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556420dccc0 0x5556420dd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.490 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.476+0000 7f1911d9b700 1 -- 172.21.15.52:0/1422230655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556420d7f70 msgr2=0x5556420d8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:01.490 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.476+0000 7f1911d9b700 1 --2- 172.21.15.52:0/1422230655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556420d7f70 0x5556420d8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.490 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.476+0000 7f1911d9b700 1 -- 172.21.15.52:0/1422230655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556420dccc0 msgr2=0x5556420dd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:01.490 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.476+0000 7f1911d9b700 1 --2- 172.21.15.52:0/1422230655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556420dccc0 0x5556420dd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.491 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.476+0000 7f1911d9b700 1 -- 172.21.15.52:0/1422230655 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555641ea6630 con 0x5556420d8880 2023-11-27T15:27:01.491 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.476+0000 7f1912d9d700 1 --2- 172.21.15.52:0/1422230655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556420dccc0 0x5556420dd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:01.491 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.476+0000 7f191259c700 1 --2- 172.21.15.52:0/1422230655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556420d7f70 0x5556420d8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:27:01.491 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.476+0000 7f1911d9b700 1 --2- 172.21.15.52:0/1422230655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556420d8880 0x5556420dc780 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f190400ee10 tx=0x7f1904011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a68cbbdc2bfbed server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:01.491 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.476+0000 7f1900ff9700 1 -- 172.21.15.52:0/1422230655 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1904011be0 con 0x5556420d8880 2023-11-27T15:27:01.491 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.477+0000 7f1900ff9700 1 -- 172.21.15.52:0/1422230655 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1904011d40 con 0x5556420d8880 2023-11-27T15:27:01.491 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.477+0000 7f1900ff9700 1 -- 172.21.15.52:0/1422230655 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1904010630 con 0x5556420d8880 2023-11-27T15:27:01.491 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.477+0000 7f19249ddb80 1 -- 172.21.15.52:0/1422230655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556420d8880 msgr2=0x5556420dc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:01.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.477+0000 7f19249ddb80 1 --2- 172.21.15.52:0/1422230655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556420d8880 0x5556420dc780 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f190400ee10 tx=0x7f1904011000 comp rx=0 tx=0).stop 2023-11-27T15:27:01.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.477+0000 7f19249ddb80 1 -- 172.21.15.52:0/1422230655 shutdown_connections 2023-11-27T15:27:01.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.477+0000 7f19249ddb80 1 --2- 172.21.15.52:0/1422230655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556420dccc0 0x5556420dd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.477+0000 7f19249ddb80 1 --2- 172.21.15.52:0/1422230655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556420d7f70 0x5556420d8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.477+0000 7f19249ddb80 1 --2- 172.21.15.52:0/1422230655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556420d8880 0x5556420dc780 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.477+0000 7f19249ddb80 1 -- 172.21.15.52:0/1422230655 >> 172.21.15.52:0/1422230655 conn(0x5556420c9360 msgr2=0x5556420ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:01.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.477+0000 7f19249ddb80 1 -- 172.21.15.52:0/1422230655 shutdown_connections 2023-11-27T15:27:01.492 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.477+0000 7f19249ddb80 1 -- 172.21.15.52:0/1422230655 wait complete. 2023-11-27T15:27:01.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.478+0000 7f19249ddb80 1 -- start start 2023-11-27T15:27:01.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.478+0000 7f19249ddb80 1 Processor -- start 2023-11-27T15:27:01.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.478+0000 7f19249ddb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556420d7f70 0x5556420e44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.478+0000 7f19249ddb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556420d8880 0x5556420e17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.478+0000 7f19249ddb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556420dccc0 0x5556420e1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.478+0000 7f19249ddb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x555641fc68e0 con 0x5556420dccc0 2023-11-27T15:27:01.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.478+0000 7f19249ddb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555641fc6580 con 0x5556420d7f70 2023-11-27T15:27:01.493 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.478+0000 7f19249ddb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555641ea8920 con 0x5556420d8880 2023-11-27T15:27:01.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.478+0000 7f19249ddb80 10 client.0 _ll_register_callbacks cb 0x555641fe4780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:01.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.478+0000 7f1912d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556420dccc0 0x5556420e1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.478+0000 7f1912d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556420dccc0 0x5556420e1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55842/0 (socket says 172.21.15.52:55842) 2023-11-27T15:27:01.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.478+0000 7f1912d9d700 1 -- 172.21.15.52:0/1708790329 learned_addr learned my addr 172.21.15.52:0/1708790329 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:01.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.478+0000 7f1911d9b700 1 --2- 172.21.15.52:0/1708790329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556420d8880 0x5556420e17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.478+0000 7f191259c700 1 --2- 172.21.15.52:0/1708790329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556420d7f70 0x5556420e44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.494 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.479+0000 7f1912d9d700 1 -- 172.21.15.52:0/1708790329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556420d8880 msgr2=0x5556420e17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:01.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.479+0000 7f1912d9d700 1 --2- 172.21.15.52:0/1708790329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556420d8880 0x5556420e17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.479+0000 7f1912d9d700 1 -- 172.21.15.52:0/1708790329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556420d7f70 msgr2=0x5556420e44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:01.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.479+0000 7f1912d9d700 1 --2- 172.21.15.52:0/1708790329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556420d7f70 0x5556420e44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.479+0000 7f1912d9d700 1 -- 172.21.15.52:0/1708790329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190400eb40 con 0x5556420dccc0 2023-11-27T15:27:01.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.479+0000 7f1911d9b700 1 --2- 172.21.15.52:0/1708790329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556420d8880 0x5556420e17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:27:01.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.479+0000 7f1912d9d700 1 --2- 172.21.15.52:0/1708790329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556420dccc0 0x5556420e1eb0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f190c007b70 tx=0x7f190c007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857b30191f3b06b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:01.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.479+0000 7f18f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:27:01.495 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.479+0000 7f18f9ffb700 1 -- 172.21.15.52:0/1708790329 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c00f9f0 con 0x5556420dccc0 2023-11-27T15:27:01.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.479+0000 7f18f9ffb700 1 -- 172.21.15.52:0/1708790329 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f190c00fb50 con 0x5556420dccc0 2023-11-27T15:27:01.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.479+0000 7f19249ddb80 1 -- 172.21.15.52:0/1708790329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556420c6b20 con 0x5556420dccc0 2023-11-27T15:27:01.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.479+0000 7f18f9ffb700 1 -- 172.21.15.52:0/1708790329 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f190c00fd10 con 0x5556420dccc0 2023-11-27T15:27:01.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.479+0000 7f19249ddb80 10 client.17025 fetch_fsmap learned FSMap version 34 2023-11-27T15:27:01.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.479+0000 7f19249ddb80 1 -- 172.21.15.52:0/1708790329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x555641feb190 con 0x5556420dccc0 2023-11-27T15:27:01.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.479+0000 7f18f9ffb700 1 -- 172.21.15.52:0/1708790329 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5556420c6b20 con 0x5556420dccc0 2023-11-27T15:27:01.496 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.479+0000 7f18f9ffb700 1 -- 172.21.15.52:0/1708790329 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f190c01c3a0 con 0x5556420dccc0 2023-11-27T15:27:01.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.479+0000 7f19249ddb80 10 client.17025 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:27:01.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.479+0000 7f19249ddb80 10 client.17025 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:01.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.479+0000 7f19249ddb80 20 client.17025 populate_metadata read hostname 'smithi052' 2023-11-27T15:27:01.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.479+0000 7f19249ddb80 1 --2- 172.21.15.52:0/1708790329 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5556420f1d20 0x5556420f4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.479+0000 7f19249ddb80 4 client.17025 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:27:01.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.479+0000 7f19249ddb80 1 -- 172.21.15.52:0/1708790329 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555641fe70a0 con 0x5556420f1d20 2023-11-27T15:27:01.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.480+0000 7f191259c700 1 --2- 172.21.15.52:0/1708790329 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5556420f1d20 0x5556420f4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.497 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.480+0000 7f18f9ffb700 10 client.17025 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:01.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.480+0000 7f191259c700 1 --2- 172.21.15.52:0/1708790329 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5556420f1d20 0x5556420f4130 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1672d49c3ffde36b server_cookie=1b3a8e005597ab3a in_seq=0 out_seq=0 2023-11-27T15:27:01.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.481+0000 7f18f9ffb700 1 -- 172.21.15.52:0/1708790329 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555641fe70a0 con 0x5556420f1d20 2023-11-27T15:27:01.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.481+0000 7f18f9ffb700 10 client.17025 handle_command_reply: tid=0 2023-11-27T15:27:01.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.481+0000 7f19249ddb80 1 client.17025 shutdown 2023-11-27T15:27:01.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.481+0000 7f19249ddb80 1 -- 172.21.15.52:0/1708790329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556420dccc0 msgr2=0x5556420e1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:01.498 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.481+0000 7f19249ddb80 1 --2- 172.21.15.52:0/1708790329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556420dccc0 0x5556420e1eb0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f190c007b70 tx=0x7f190c007e80 comp rx=0 tx=0).stop 2023-11-27T15:27:01.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.481+0000 7f19249ddb80 1 -- 172.21.15.52:0/1708790329 shutdown_connections 2023-11-27T15:27:01.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.481+0000 7f19249ddb80 1 --2- 172.21.15.52:0/1708790329 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5556420f1d20 0x5556420f4130 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.481+0000 7f19249ddb80 1 --2- 172.21.15.52:0/1708790329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556420d7f70 0x5556420e44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.481+0000 7f18f9ffb700 0 client.17025 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:01.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.481+0000 7f19249ddb80 1 --2- 172.21.15.52:0/1708790329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556420dccc0 0x5556420e1eb0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.481+0000 7f19249ddb80 1 --2- 172.21.15.52:0/1708790329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556420d8880 0x5556420e17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.481+0000 7f19249ddb80 1 -- 172.21.15.52:0/1708790329 >> 172.21.15.52:0/1708790329 conn(0x5556420c9360 msgr2=0x5556420cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:01.499 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.481+0000 7f19249ddb80 1 -- 172.21.15.52:0/1708790329 shutdown_connections 2023-11-27T15:27:01.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.481+0000 7f19249ddb80 1 -- 172.21.15.52:0/1708790329 wait complete. 2023-11-27T15:27:01.500 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.481+0000 7f19249ddb80 20 client.17025 trim_cache size 0 max 16384 2023-11-27T15:27:01.506 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.494+0000 7f19249ddb80 1 Processor -- start 2023-11-27T15:27:01.506 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.494+0000 7f19249ddb80 1 -- start start 2023-11-27T15:27:01.506 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.494+0000 7f19249ddb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556420d7f70 0x5556420da5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.506 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.494+0000 7f19249ddb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556420d8880 0x5556420e1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.506 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.494+0000 7f19249ddb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556420dccc0 0x5556420e8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.506 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.494+0000 7f19249ddb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556420d01c0 con 0x5556420d8880 2023-11-27T15:27:01.507 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.494+0000 7f19249ddb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556420c6a10 con 0x5556420d7f70 2023-11-27T15:27:01.507 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.494+0000 7f19249ddb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556420d8fd0 con 0x5556420dccc0 2023-11-27T15:27:01.507 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.494+0000 7f1911d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556420d8880 0x5556420e1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.507 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.494+0000 7f1911d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556420d8880 0x5556420e1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55848/0 (socket says 172.21.15.52:55848) 2023-11-27T15:27:01.507 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.494+0000 7f1911d9b700 1 -- 172.21.15.52:0/3332354763 learned_addr learned my addr 172.21.15.52:0/3332354763 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:01.507 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.494+0000 7f191259c700 1 --2- 172.21.15.52:0/3332354763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556420d7f70 0x5556420da5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.507 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.494+0000 7f1912d9d700 1 --2- 172.21.15.52:0/3332354763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556420dccc0 0x5556420e8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.495+0000 7f1911d9b700 1 -- 172.21.15.52:0/3332354763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556420dccc0 msgr2=0x5556420e8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:01.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.495+0000 7f1911d9b700 1 --2- 172.21.15.52:0/3332354763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556420dccc0 0x5556420e8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.495+0000 7f1911d9b700 1 -- 172.21.15.52:0/3332354763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556420d7f70 msgr2=0x5556420da5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:01.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.495+0000 7f1911d9b700 1 --2- 172.21.15.52:0/3332354763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556420d7f70 0x5556420da5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.495+0000 7f1911d9b700 1 -- 172.21.15.52:0/3332354763 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555641feaee0 con 0x5556420d8880 2023-11-27T15:27:01.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.495+0000 7f191259c700 1 --2- 172.21.15.52:0/3332354763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556420d7f70 0x5556420da5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:01.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.495+0000 7f1911d9b700 1 --2- 172.21.15.52:0/3332354763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556420d8880 0x5556420e1dc0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f1904011a30 tx=0x7f1904011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6b179af00a8726 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:01.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.495+0000 7f19017fa700 1 -- 172.21.15.52:0/3332354763 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1904004d80 con 0x5556420d8880 2023-11-27T15:27:01.509 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.495+0000 7f19017fa700 1 -- 172.21.15.52:0/3332354763 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f190400eec0 con 0x5556420d8880 2023-11-27T15:27:01.509 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.495+0000 7f19017fa700 1 -- 172.21.15.52:0/3332354763 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1904003680 con 0x5556420d8880 2023-11-27T15:27:01.509 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.495+0000 7f19249ddb80 1 -- 172.21.15.52:0/3332354763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556420d8880 msgr2=0x5556420e1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:01.509 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.495+0000 7f19249ddb80 1 --2- 172.21.15.52:0/3332354763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556420d8880 0x5556420e1dc0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f1904011a30 tx=0x7f1904011eb0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.509 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.495+0000 7f19249ddb80 1 -- 172.21.15.52:0/3332354763 shutdown_connections 2023-11-27T15:27:01.509 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.495+0000 7f19249ddb80 1 --2- 172.21.15.52:0/3332354763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556420d7f70 0x5556420da5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.509 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.495+0000 7f19249ddb80 1 --2- 172.21.15.52:0/3332354763 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556420dccc0 0x5556420e8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.509 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.495+0000 7f19249ddb80 1 --2- 172.21.15.52:0/3332354763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556420d8880 0x5556420e1dc0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.495+0000 7f19249ddb80 1 -- 172.21.15.52:0/3332354763 >> 172.21.15.52:0/3332354763 conn(0x5556420c9360 msgr2=0x5556420ed500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:01.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.495+0000 7f19249ddb80 1 -- 172.21.15.52:0/3332354763 shutdown_connections 2023-11-27T15:27:01.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.495+0000 7f19249ddb80 1 -- 172.21.15.52:0/3332354763 wait complete. 2023-11-27T15:27:01.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.496+0000 7f19249ddb80 1 -- start start 2023-11-27T15:27:01.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.496+0000 7f19249ddb80 1 Processor -- start 2023-11-27T15:27:01.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.496+0000 7f19249ddb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556420d7f70 0x5556420cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.510 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.496+0000 7f19249ddb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556420d8880 0x5556420cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.496+0000 7f19249ddb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556420dccc0 0x5556420cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.496+0000 7f19249ddb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556420e9340 con 0x5556420d8880 2023-11-27T15:27:01.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.496+0000 7f19249ddb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556420ce510 con 0x5556420dccc0 2023-11-27T15:27:01.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.496+0000 7f19249ddb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556420ce6a0 con 0x5556420d7f70 2023-11-27T15:27:01.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.496+0000 7f19249ddb80 10 client.0 _ll_register_callbacks cb 0x7f190c007730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:01.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.496+0000 7f1911d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556420d8880 0x5556420cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.496+0000 7f1911d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556420d8880 0x5556420cda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55854/0 (socket says 172.21.15.52:55854) 2023-11-27T15:27:01.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.496+0000 7f1911d9b700 1 -- 172.21.15.52:0/1833001732 learned_addr learned my addr 172.21.15.52:0/1833001732 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:01.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.496+0000 7f191259c700 1 --2- 172.21.15.52:0/1833001732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556420d7f70 0x5556420cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.496+0000 7f1912d9d700 1 --2- 172.21.15.52:0/1833001732 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556420dccc0 0x5556420cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.496+0000 7f1911d9b700 1 -- 172.21.15.52:0/1833001732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556420d7f70 msgr2=0x5556420cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:01.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.496+0000 7f1911d9b700 1 --2- 172.21.15.52:0/1833001732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556420d7f70 0x5556420cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.496+0000 7f1911d9b700 1 -- 172.21.15.52:0/1833001732 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556420dccc0 msgr2=0x5556420cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:01.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.496+0000 7f1911d9b700 1 --2- 172.21.15.52:0/1833001732 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556420dccc0 0x5556420cdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.496+0000 7f1911d9b700 1 -- 172.21.15.52:0/1833001732 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555641ea6630 con 0x5556420d8880 2023-11-27T15:27:01.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.497+0000 7f1912d9d700 1 --2- 172.21.15.52:0/1833001732 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556420dccc0 0x5556420cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:01.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.497+0000 7f1911d9b700 1 --2- 172.21.15.52:0/1833001732 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556420d8880 0x5556420cda90 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f1904013040 tx=0x7f190400e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e8b1469dfcb62a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:01.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.497+0000 7f18f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:27:01.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.497+0000 7f18f9ffb700 1 -- 172.21.15.52:0/1833001732 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1904004e90 con 0x5556420d8880 2023-11-27T15:27:01.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.497+0000 7f18f9ffb700 1 -- 172.21.15.52:0/1833001732 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f1904004ff0 con 0x5556420d8880 2023-11-27T15:27:01.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.497+0000 7f19249ddb80 1 -- 172.21.15.52:0/1833001732 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555641ff3660 con 0x5556420d8880 2023-11-27T15:27:01.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.497+0000 7f18f9ffb700 1 -- 172.21.15.52:0/1833001732 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19040051b0 con 0x5556420d8880 2023-11-27T15:27:01.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.497+0000 7f19249ddb80 10 client.17031 fetch_fsmap learned FSMap version 34 2023-11-27T15:27:01.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.497+0000 7f19249ddb80 1 -- 172.21.15.52:0/1833001732 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x555641e3bc60 con 0x5556420d8880 2023-11-27T15:27:01.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.497+0000 7f18f9ffb700 1 -- 172.21.15.52:0/1833001732 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555641ff3660 con 0x5556420d8880 2023-11-27T15:27:01.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.497+0000 7f18f9ffb700 1 -- 172.21.15.52:0/1833001732 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f190403a7f0 con 0x5556420d8880 2023-11-27T15:27:01.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.497+0000 7f19249ddb80 10 client.17031 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:27:01.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.497+0000 7f19249ddb80 10 client.17031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:01.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.497+0000 7f19249ddb80 20 client.17031 populate_metadata read hostname 'smithi052' 2023-11-27T15:27:01.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.497+0000 7f19249ddb80 1 --2- 172.21.15.52:0/1833001732 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5556421462b0 0x5556421486c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:01.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.497+0000 7f19249ddb80 4 client.17031 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:27:01.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.497+0000 7f19249ddb80 1 -- 172.21.15.52:0/1833001732 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555641e73f70 con 0x5556421462b0 2023-11-27T15:27:01.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.498+0000 7f191259c700 1 --2- 172.21.15.52:0/1833001732 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5556421462b0 0x5556421486c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:01.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.498+0000 7f18f9ffb700 10 client.17031 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:01.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.498+0000 7f191259c700 1 --2- 172.21.15.52:0/1833001732 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5556421462b0 0x5556421486c0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e4eebd0c57c8e91 server_cookie=e35507ceb996a883 in_seq=0 out_seq=0 2023-11-27T15:27:01.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.498+0000 7f18f9ffb700 1 -- 172.21.15.52:0/1833001732 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555641e73f70 con 0x5556421462b0 2023-11-27T15:27:01.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.498+0000 7f18f9ffb700 10 client.17031 handle_command_reply: tid=0 2023-11-27T15:27:01.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.498+0000 7f19249ddb80 1 client.17031 shutdown 2023-11-27T15:27:01.515 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.498+0000 7f19249ddb80 1 -- 172.21.15.52:0/1833001732 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556420d8880 msgr2=0x5556420cda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:01.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.498+0000 7f19249ddb80 1 --2- 172.21.15.52:0/1833001732 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556420d8880 0x5556420cda90 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f1904013040 tx=0x7f190400e9a0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.498+0000 7f19249ddb80 1 -- 172.21.15.52:0/1833001732 shutdown_connections 2023-11-27T15:27:01.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.498+0000 7f19249ddb80 1 --2- 172.21.15.52:0/1833001732 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5556421462b0 0x5556421486c0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.498+0000 7f19249ddb80 1 --2- 172.21.15.52:0/1833001732 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5556420dccc0 0x5556420cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.498+0000 7f19249ddb80 1 --2- 172.21.15.52:0/1833001732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5556420d7f70 0x5556420cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.498+0000 7f19249ddb80 1 --2- 172.21.15.52:0/1833001732 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556420d8880 0x5556420cda90 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.498+0000 7f18f9ffb700 0 client.17031 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:01.516 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.498+0000 7f19249ddb80 1 -- 172.21.15.52:0/1833001732 >> 172.21.15.52:0/1833001732 conn(0x5556420c9360 msgr2=0x5556420dd790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:01.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.499+0000 7f19249ddb80 1 -- 172.21.15.52:0/1833001732 shutdown_connections 2023-11-27T15:27:01.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.499+0000 7f19249ddb80 1 -- 172.21.15.52:0/1833001732 wait complete. 2023-11-27T15:27:01.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.499+0000 7f19249ddb80 20 client.17031 trim_cache size 0 max 16384 2023-11-27T15:27:01.517 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:27:01.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.499+0000 7f1918851700 1 -- 172.21.15.52:0/909227118 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f18fc0a12c0 msgr2=0x7f18fc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:01.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.499+0000 7f1918851700 1 --2- 172.21.15.52:0/909227118 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f18fc0a12c0 0x7f18fc0a3770 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1904005d90 tx=0x7f1904005d20 comp rx=0 tx=0).stop 2023-11-27T15:27:01.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.499+0000 7f1918851700 1 -- 172.21.15.52:0/909227118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19140ff990 msgr2=0x7f19141963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:01.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.499+0000 7f1918851700 1 --2- 172.21.15.52:0/909227118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19140ff990 0x7f19141963b0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f1908009990 tx=0x7f19080057f0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.499+0000 7f1918851700 1 -- 172.21.15.52:0/909227118 shutdown_connections 2023-11-27T15:27:01.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.499+0000 7f1918851700 1 --2- 172.21.15.52:0/909227118 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f18fc0a12c0 0x7f18fc0a3770 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.499+0000 7f1918851700 1 --2- 172.21.15.52:0/909227118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1914104180 0x7f191419ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.499+0000 7f1918851700 1 --2- 172.21.15.52:0/909227118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19140ff990 0x7f19141963b0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.499+0000 7f1918851700 1 --2- 172.21.15.52:0/909227118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1914100340 0x7f19141968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:01.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.499+0000 7f1918851700 1 -- 172.21.15.52:0/909227118 >> 172.21.15.52:0/909227118 conn(0x7f1914075920 msgr2=0x7f19140fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:01.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.499+0000 7f1918851700 1 -- 172.21.15.52:0/909227118 shutdown_connections 2023-11-27T15:27:01.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:01.499+0000 7f1918851700 1 -- 172.21.15.52:0/909227118 wait complete. 2023-11-27T15:27:02.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:02 smithi150 ceph-mon[100559]: pgmap v282: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-11-27T15:27:02.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:02 smithi102 ceph-mon[100296]: pgmap v282: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-11-27T15:27:02.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:02 smithi052 ceph-mon[99056]: pgmap v282: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-11-27T15:27:02.521 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-27T15:27:02.521 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-27T15:27:02.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.662+0000 7f64ed1b1700 1 -- 172.21.15.52:0/860632648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64e81005b0 msgr2=0x7f64e8100980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:02.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.662+0000 7f64ed1b1700 1 --2- 172.21.15.52:0/860632648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64e81005b0 0x7f64e8100980 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f64dc0099c0 tx=0x7f64dc009cd0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.667 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.662+0000 7f64ed1b1700 1 -- 172.21.15.52:0/860632648 shutdown_connections 2023-11-27T15:27:02.668 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.662+0000 7f64ed1b1700 1 --2- 172.21.15.52:0/860632648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64e81055f0 0x7f64e810d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.668 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.662+0000 7f64ed1b1700 1 --2- 172.21.15.52:0/860632648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64e81005b0 0x7f64e8100980 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.668 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.662+0000 7f64ed1b1700 1 --2- 172.21.15.52:0/860632648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64e8100f50 0x7f64e8104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.668 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.662+0000 7f64ed1b1700 1 -- 172.21.15.52:0/860632648 >> 172.21.15.52:0/860632648 conn(0x7f64e8077170 msgr2=0x7f64e8077570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:02.668 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.662+0000 7f64ed1b1700 1 -- 172.21.15.52:0/860632648 shutdown_connections 2023-11-27T15:27:02.668 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.662+0000 7f64ed1b1700 1 -- 172.21.15.52:0/860632648 wait complete. 2023-11-27T15:27:02.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.663+0000 7f64ed1b1700 1 Processor -- start 2023-11-27T15:27:02.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.663+0000 7f64ed1b1700 1 -- start start 2023-11-27T15:27:02.670 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.663+0000 7f64ed1b1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64e81005b0 0x7f64e81963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.663+0000 7f64ed1b1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64e8100f50 0x7f64e8196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.663+0000 7f64ed1b1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64e81055f0 0x7f64e819ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.663+0000 7f64ed1b1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e819b3d0 con 0x7f64e81005b0 2023-11-27T15:27:02.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.663+0000 7f64ed1b1700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e819b540 con 0x7f64e81055f0 2023-11-27T15:27:02.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.663+0000 7f64ed1b1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e819b700 con 0x7f64e8100f50 2023-11-27T15:27:02.671 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.664+0000 7f64e6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64e81005b0 0x7f64e81963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.664+0000 7f64e6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64e81005b0 0x7f64e81963f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55866/0 (socket says 172.21.15.52:55866) 2023-11-27T15:27:02.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.664+0000 7f64e6d9d700 1 -- 172.21.15.52:0/2135387666 learned_addr learned my addr 172.21.15.52:0/2135387666 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:02.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.664+0000 7f64e759e700 1 --2- 172.21.15.52:0/2135387666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64e81055f0 0x7f64e819ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.664+0000 7f64e659c700 1 --2- 172.21.15.52:0/2135387666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64e8100f50 0x7f64e8196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.664+0000 7f64e6d9d700 1 -- 172.21.15.52:0/2135387666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64e8100f50 msgr2=0x7f64e8196930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:02.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.664+0000 7f64e6d9d700 1 --2- 172.21.15.52:0/2135387666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64e8100f50 0x7f64e8196930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.664+0000 7f64e6d9d700 1 -- 172.21.15.52:0/2135387666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64e81055f0 msgr2=0x7f64e819ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:02.672 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.664+0000 7f64e6d9d700 1 --2- 172.21.15.52:0/2135387666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64e81055f0 0x7f64e819ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.664+0000 7f64e6d9d700 1 -- 172.21.15.52:0/2135387666 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64dc0095f0 con 0x7f64e81005b0 2023-11-27T15:27:02.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.664+0000 7f64e759e700 1 --2- 172.21.15.52:0/2135387666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64e81055f0 0x7f64e819ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:02.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.664+0000 7f64e6d9d700 1 --2- 172.21.15.52:0/2135387666 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64e81005b0 0x7f64e81963f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009990 tx=0x7f64dc009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:02.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.664+0000 7f64cffff700 1 -- 172.21.15.52:0/2135387666 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc01d070 con 0x7f64e81005b0 2023-11-27T15:27:02.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.664+0000 7f64ed1b1700 1 -- 172.21.15.52:0/2135387666 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e81103e0 con 0x7f64e81005b0 2023-11-27T15:27:02.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.664+0000 7f64cffff700 1 -- 172.21.15.52:0/2135387666 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64dc004010 con 0x7f64e81005b0 2023-11-27T15:27:02.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.664+0000 7f64cffff700 1 -- 172.21.15.52:0/2135387666 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc022d40 con 0x7f64e81005b0 2023-11-27T15:27:02.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.664+0000 7f64ed1b1700 1 -- 172.21.15.52:0/2135387666 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e8110250 con 0x7f64e81005b0 2023-11-27T15:27:02.680 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.666+0000 7f64cffff700 1 -- 172.21.15.52:0/2135387666 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f64dc022470 con 0x7f64e81005b0 2023-11-27T15:27:02.680 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.667+0000 7f64cffff700 1 --2- 172.21.15.52:0/2135387666 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f64d00a12c0 0x7f64d00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.680 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.667+0000 7f64cffff700 1 -- 172.21.15.52:0/2135387666 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f64dc0ce130 con 0x7f64e81005b0 2023-11-27T15:27:02.680 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.667+0000 7f64e659c700 1 --2- 172.21.15.52:0/2135387666 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f64d00a12c0 0x7f64d00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.680 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.668+0000 7f64e659c700 1 --2- 172.21.15.52:0/2135387666 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f64d00a12c0 0x7f64d00a3770 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f64d8005950 tx=0x7f64d80058e0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:02.680 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.668+0000 7f64f933db80 1 Processor -- start 2023-11-27T15:27:02.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.668+0000 7f64f933db80 1 -- start start 2023-11-27T15:27:02.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.668+0000 7f64f933db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558769943ff0 0x5587699443c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.668+0000 7f64f933db80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558769944900 0x558769948800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.668+0000 7f64f933db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558769948d40 0x558769949190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.668+0000 7f64f933db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587698450e0 con 0x558769948d40 2023-11-27T15:27:02.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.668+0000 7f64f933db80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558769897920 con 0x558769944900 2023-11-27T15:27:02.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.668+0000 7f64f933db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558769890db0 con 0x558769943ff0 2023-11-27T15:27:02.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.668+0000 7f64e759e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558769948d40 0x558769949190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.682 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.668+0000 7f64e759e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558769948d40 0x558769949190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55878/0 (socket says 172.21.15.52:55878) 2023-11-27T15:27:02.682 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.668+0000 7f64e6d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558769943ff0 0x5587699443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.682 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.668+0000 7f64e759e700 1 -- 172.21.15.52:0/139379629 learned_addr learned my addr 172.21.15.52:0/139379629 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:02.682 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.668+0000 7f64e659c700 1 --2- 172.21.15.52:0/139379629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558769944900 0x558769948800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.682 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.669+0000 7f64e6d9d700 1 -- 172.21.15.52:0/139379629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558769944900 msgr2=0x558769948800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:02.682 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.669+0000 7f64e6d9d700 1 --2- 172.21.15.52:0/139379629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558769944900 0x558769948800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.682 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.669+0000 7f64e6d9d700 1 -- 172.21.15.52:0/139379629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558769948d40 msgr2=0x558769949190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:02.682 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.669+0000 7f64e6d9d700 1 --2- 172.21.15.52:0/139379629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558769948d40 0x558769949190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.683 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.669+0000 7f64e6d9d700 1 -- 172.21.15.52:0/139379629 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587697126b0 con 0x558769943ff0 2023-11-27T15:27:02.683 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.669+0000 7f64e759e700 1 --2- 172.21.15.52:0/139379629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558769948d40 0x558769949190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:27:02.683 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.669+0000 7f64e6d9d700 1 --2- 172.21.15.52:0/139379629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558769943ff0 0x5587699443c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f64dc0cb7f0 tx=0x7f64dc005aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e82e1d60b67d585 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:02.683 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.669+0000 7f64cd7fa700 1 -- 172.21.15.52:0/139379629 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc0a10b0 con 0x558769943ff0 2023-11-27T15:27:02.683 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.669+0000 7f64cd7fa700 1 -- 172.21.15.52:0/139379629 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64dc004180 con 0x558769943ff0 2023-11-27T15:27:02.683 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.669+0000 7f64cd7fa700 1 -- 172.21.15.52:0/139379629 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc09a6f0 con 0x558769943ff0 2023-11-27T15:27:02.683 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.670+0000 7f64f933db80 1 -- 172.21.15.52:0/139379629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558769943ff0 msgr2=0x5587699443c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:02.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.670+0000 7f64f933db80 1 --2- 172.21.15.52:0/139379629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558769943ff0 0x5587699443c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f64dc0cb7f0 tx=0x7f64dc005aa0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.670+0000 7f64f933db80 1 -- 172.21.15.52:0/139379629 shutdown_connections 2023-11-27T15:27:02.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.670+0000 7f64f933db80 1 --2- 172.21.15.52:0/139379629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558769948d40 0x558769949190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.670+0000 7f64f933db80 1 --2- 172.21.15.52:0/139379629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558769943ff0 0x5587699443c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.670+0000 7f64f933db80 1 --2- 172.21.15.52:0/139379629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558769944900 0x558769948800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.670+0000 7f64f933db80 1 -- 172.21.15.52:0/139379629 >> 172.21.15.52:0/139379629 conn(0x5587699353e0 msgr2=0x55876993abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:02.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.670+0000 7f64f933db80 1 -- 172.21.15.52:0/139379629 shutdown_connections 2023-11-27T15:27:02.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.670+0000 7f64f933db80 1 -- 172.21.15.52:0/139379629 wait complete. 2023-11-27T15:27:02.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.670+0000 7f64f933db80 1 -- start start 2023-11-27T15:27:02.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.671+0000 7f64f933db80 1 Processor -- start 2023-11-27T15:27:02.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.671+0000 7f64f933db80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558769943ff0 0x5587699503b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.671+0000 7f64f933db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558769944900 0x55876994d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.671+0000 7f64f933db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558769948d40 0x55876994db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.671+0000 7f64f933db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x558769832960 con 0x558769944900 2023-11-27T15:27:02.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.671+0000 7f64f933db80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558769832600 con 0x558769943ff0 2023-11-27T15:27:02.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.671+0000 7f64f933db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587697149a0 con 0x558769948d40 2023-11-27T15:27:02.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.671+0000 7f64f933db80 10 client.0 _ll_register_callbacks cb 0x558769850800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:02.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.671+0000 7f64e659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558769944900 0x55876994d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.671+0000 7f64e659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558769944900 0x55876994d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55882/0 (socket says 172.21.15.52:55882) 2023-11-27T15:27:02.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.671+0000 7f64e659c700 1 -- 172.21.15.52:0/2076098196 learned_addr learned my addr 172.21.15.52:0/2076098196 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:02.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.671+0000 7f64e6d9d700 1 --2- 172.21.15.52:0/2076098196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558769943ff0 0x5587699503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.671+0000 7f64e759e700 1 --2- 172.21.15.52:0/2076098196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558769948d40 0x55876994db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.671+0000 7f64e659c700 1 -- 172.21.15.52:0/2076098196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558769948d40 msgr2=0x55876994db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:02.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.671+0000 7f64e659c700 1 --2- 172.21.15.52:0/2076098196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558769948d40 0x55876994db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.671+0000 7f64e659c700 1 -- 172.21.15.52:0/2076098196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558769943ff0 msgr2=0x5587699503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:02.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.671+0000 7f64e659c700 1 --2- 172.21.15.52:0/2076098196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558769943ff0 0x5587699503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.671+0000 7f64e659c700 1 -- 172.21.15.52:0/2076098196 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e8110250 con 0x558769944900 2023-11-27T15:27:02.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.671+0000 7f64e759e700 1 --2- 172.21.15.52:0/2076098196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558769948d40 0x55876994db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:02.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.671+0000 7f64e659c700 1 --2- 172.21.15.52:0/2076098196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558769944900 0x55876994d5c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f64d8011720 tx=0x7f64d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c5dac7542557038 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:02.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.671+0000 7f64caff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:27:02.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.672+0000 7f64caff5700 1 -- 172.21.15.52:0/2076098196 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d8004a30 con 0x558769944900 2023-11-27T15:27:02.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.672+0000 7f64caff5700 1 -- 172.21.15.52:0/2076098196 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f64d8004b90 con 0x558769944900 2023-11-27T15:27:02.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.672+0000 7f64f933db80 1 -- 172.21.15.52:0/2076098196 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558769932ba0 con 0x558769944900 2023-11-27T15:27:02.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.672+0000 7f64caff5700 1 -- 172.21.15.52:0/2076098196 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64d8004d90 con 0x558769944900 2023-11-27T15:27:02.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.672+0000 7f64f933db80 10 client.17043 fetch_fsmap learned FSMap version 34 2023-11-27T15:27:02.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.672+0000 7f64f933db80 1 -- 172.21.15.52:0/2076098196 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x558769857210 con 0x558769944900 2023-11-27T15:27:02.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.672+0000 7f64caff5700 1 -- 172.21.15.52:0/2076098196 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558769932ba0 con 0x558769944900 2023-11-27T15:27:02.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.672+0000 7f64caff5700 1 -- 172.21.15.52:0/2076098196 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f64d8025030 con 0x558769944900 2023-11-27T15:27:02.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.672+0000 7f64f933db80 10 client.17043 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:27:02.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.672+0000 7f64f933db80 10 client.17043 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:02.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.672+0000 7f64f933db80 20 client.17043 populate_metadata read hostname 'smithi052' 2023-11-27T15:27:02.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.672+0000 7f64f933db80 1 --2- 172.21.15.52:0/2076098196 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55876995da70 0x55876995fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.672+0000 7f64f933db80 4 client.17043 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:27:02.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.672+0000 7f64f933db80 1 -- 172.21.15.52:0/2076098196 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558769853120 con 0x55876995da70 2023-11-27T15:27:02.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.673+0000 7f64e6d9d700 1 --2- 172.21.15.52:0/2076098196 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55876995da70 0x55876995fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.673+0000 7f64caff5700 10 client.17043 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:02.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.673+0000 7f64e6d9d700 1 --2- 172.21.15.52:0/2076098196 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55876995da70 0x55876995fe80 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed5ca34784588ec7 server_cookie=d8a80f1a20525be3 in_seq=0 out_seq=0 2023-11-27T15:27:02.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.674+0000 7f64caff5700 1 -- 172.21.15.52:0/2076098196 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558769853120 con 0x55876995da70 2023-11-27T15:27:02.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.674+0000 7f64caff5700 10 client.17043 handle_command_reply: tid=0 2023-11-27T15:27:02.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.674+0000 7f64f933db80 1 client.17043 shutdown 2023-11-27T15:27:02.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.674+0000 7f64f933db80 1 -- 172.21.15.52:0/2076098196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558769944900 msgr2=0x55876994d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:02.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.674+0000 7f64f933db80 1 --2- 172.21.15.52:0/2076098196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558769944900 0x55876994d5c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f64d8011720 tx=0x7f64d8004750 comp rx=0 tx=0).stop 2023-11-27T15:27:02.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.674+0000 7f64f933db80 1 -- 172.21.15.52:0/2076098196 shutdown_connections 2023-11-27T15:27:02.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.674+0000 7f64f933db80 1 --2- 172.21.15.52:0/2076098196 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55876995da70 0x55876995fe80 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.674+0000 7f64f933db80 1 --2- 172.21.15.52:0/2076098196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558769943ff0 0x5587699503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.674+0000 7f64f933db80 1 --2- 172.21.15.52:0/2076098196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558769948d40 0x55876994db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.674+0000 7f64caff5700 0 client.17043 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:02.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.674+0000 7f64f933db80 1 --2- 172.21.15.52:0/2076098196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558769944900 0x55876994d5c0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.674+0000 7f64f933db80 1 -- 172.21.15.52:0/2076098196 >> 172.21.15.52:0/2076098196 conn(0x5587699353e0 msgr2=0x55876994a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:02.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.674+0000 7f64f933db80 1 -- 172.21.15.52:0/2076098196 shutdown_connections 2023-11-27T15:27:02.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.674+0000 7f64f933db80 1 -- 172.21.15.52:0/2076098196 wait complete. 2023-11-27T15:27:02.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.674+0000 7f64f933db80 20 client.17043 trim_cache size 0 max 16384 2023-11-27T15:27:02.699 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.691+0000 7f64f933db80 1 Processor -- start 2023-11-27T15:27:02.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.691+0000 7f64f933db80 1 -- start start 2023-11-27T15:27:02.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.691+0000 7f64f933db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558769943ff0 0x55876993ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.691+0000 7f64f933db80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558769944900 0x558769954be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.691+0000 7f64f933db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558769948d40 0x55876994fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.691+0000 7f64f933db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55876995b0a0 con 0x558769943ff0 2023-11-27T15:27:02.700 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.691+0000 7f64f933db80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x558769945050 con 0x558769944900 2023-11-27T15:27:02.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.691+0000 7f64f933db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558769934f90 con 0x558769948d40 2023-11-27T15:27:02.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.691+0000 7f64e6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558769943ff0 0x55876993ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.691+0000 7f64e6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558769943ff0 0x55876993ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55888/0 (socket says 172.21.15.52:55888) 2023-11-27T15:27:02.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.691+0000 7f64e6d9d700 1 -- 172.21.15.52:0/3690735702 learned_addr learned my addr 172.21.15.52:0/3690735702 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:02.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.692+0000 7f64e659c700 1 --2- 172.21.15.52:0/3690735702 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558769944900 0x558769954be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.692+0000 7f64e759e700 1 --2- 172.21.15.52:0/3690735702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558769948d40 0x55876994fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.692+0000 7f64e6d9d700 1 -- 172.21.15.52:0/3690735702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558769948d40 msgr2=0x55876994fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:02.701 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.692+0000 7f64e6d9d700 1 --2- 172.21.15.52:0/3690735702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558769948d40 0x55876994fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.692+0000 7f64e6d9d700 1 -- 172.21.15.52:0/3690735702 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558769944900 msgr2=0x558769954be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:02.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.692+0000 7f64e6d9d700 1 --2- 172.21.15.52:0/3690735702 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558769944900 0x558769954be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.692+0000 7f64e6d9d700 1 -- 172.21.15.52:0/3690735702 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587699493e0 con 0x558769943ff0 2023-11-27T15:27:02.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.692+0000 7f64e6d9d700 1 --2- 172.21.15.52:0/3690735702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558769943ff0 0x55876993ac60 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f64dc0cbd70 tx=0x7f64dc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6abb35bea6d5948 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:02.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.692+0000 7f64cdffb700 1 -- 172.21.15.52:0/3690735702 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc0a10b0 con 0x558769943ff0 2023-11-27T15:27:02.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.692+0000 7f64cdffb700 1 -- 172.21.15.52:0/3690735702 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64dc0a35c0 con 0x558769943ff0 2023-11-27T15:27:02.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.692+0000 7f64cdffb700 1 -- 172.21.15.52:0/3690735702 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc0a3130 con 0x558769943ff0 2023-11-27T15:27:02.702 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.692+0000 7f64f933db80 1 -- 172.21.15.52:0/3690735702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558769943ff0 msgr2=0x55876993ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:02.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.693+0000 7f64f933db80 1 --2- 172.21.15.52:0/3690735702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558769943ff0 0x55876993ac60 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f64dc0cbd70 tx=0x7f64dc009310 comp rx=0 tx=0).stop 2023-11-27T15:27:02.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.693+0000 7f64f933db80 1 -- 172.21.15.52:0/3690735702 shutdown_connections 2023-11-27T15:27:02.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.693+0000 7f64f933db80 1 --2- 172.21.15.52:0/3690735702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558769948d40 0x55876994fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.693+0000 7f64f933db80 1 --2- 172.21.15.52:0/3690735702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558769943ff0 0x55876993ac60 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.693+0000 7f64f933db80 1 --2- 172.21.15.52:0/3690735702 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558769944900 0x558769954be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.693+0000 7f64f933db80 1 -- 172.21.15.52:0/3690735702 >> 172.21.15.52:0/3690735702 conn(0x5587699353e0 msgr2=0x55876993f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:02.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.693+0000 7f64f933db80 1 -- 172.21.15.52:0/3690735702 shutdown_connections 2023-11-27T15:27:02.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.693+0000 7f64f933db80 1 -- 172.21.15.52:0/3690735702 wait complete. 2023-11-27T15:27:02.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.693+0000 7f64f933db80 1 -- start start 2023-11-27T15:27:02.704 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.693+0000 7f64f933db80 1 Processor -- start 2023-11-27T15:27:02.704 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.693+0000 7f64f933db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558769943ff0 0x558769954580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.704 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.693+0000 7f64f933db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558769944900 0x558769954ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.704 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.693+0000 7f64f933db80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558769948d40 0x558769950b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.704 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.693+0000 7f64f933db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x558769951250 con 0x558769943ff0 2023-11-27T15:27:02.704 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.693+0000 7f64f933db80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587699513c0 con 0x558769948d40 2023-11-27T15:27:02.704 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.693+0000 7f64f933db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558769951580 con 0x558769944900 2023-11-27T15:27:02.704 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.693+0000 7f64f933db80 10 client.0 _ll_register_callbacks cb 0x7f64d800ed80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:02.705 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.693+0000 7f64e6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558769943ff0 0x558769954580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.705 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.693+0000 7f64e6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558769943ff0 0x558769954580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55894/0 (socket says 172.21.15.52:55894) 2023-11-27T15:27:02.705 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.693+0000 7f64e6d9d700 1 -- 172.21.15.52:0/1104368610 learned_addr learned my addr 172.21.15.52:0/1104368610 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:02.705 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.693+0000 7f64e759e700 1 --2- 172.21.15.52:0/1104368610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558769948d40 0x558769950b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.705 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.694+0000 7f64e659c700 1 --2- 172.21.15.52:0/1104368610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558769944900 0x558769954ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.705 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.694+0000 7f64e6d9d700 1 -- 172.21.15.52:0/1104368610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558769944900 msgr2=0x558769954ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:02.705 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.694+0000 7f64e6d9d700 1 --2- 172.21.15.52:0/1104368610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558769944900 0x558769954ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.705 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.694+0000 7f64e6d9d700 1 -- 172.21.15.52:0/1104368610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558769948d40 msgr2=0x558769950b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:02.706 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.694+0000 7f64e6d9d700 1 --2- 172.21.15.52:0/1104368610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558769948d40 0x558769950b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.706 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.694+0000 7f64e6d9d700 1 -- 172.21.15.52:0/1104368610 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587697126b0 con 0x558769943ff0 2023-11-27T15:27:02.706 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.694+0000 7f64e659c700 1 --2- 172.21.15.52:0/1104368610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558769944900 0x558769954ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:02.706 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.694+0000 7f64e6d9d700 1 --2- 172.21.15.52:0/1104368610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558769943ff0 0x558769954580 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009fd0 tx=0x7f64dc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2bcbfcae0bdd18 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:02.706 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.694+0000 7f64caff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:27:02.706 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.694+0000 7f64caff5700 1 -- 172.21.15.52:0/1104368610 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc09a8f0 con 0x558769943ff0 2023-11-27T15:27:02.706 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.694+0000 7f64caff5700 1 -- 172.21.15.52:0/1104368610 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f64dc09b460 con 0x558769943ff0 2023-11-27T15:27:02.706 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.694+0000 7f64f933db80 1 -- 172.21.15.52:0/1104368610 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f64dc004180 con 0x558769943ff0 2023-11-27T15:27:02.706 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.694+0000 7f64caff5700 1 -- 172.21.15.52:0/1104368610 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64dc09b640 con 0x558769943ff0 2023-11-27T15:27:02.707 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.694+0000 7f64f933db80 10 client.17049 fetch_fsmap learned FSMap version 34 2023-11-27T15:27:02.707 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.694+0000 7f64f933db80 1 -- 172.21.15.52:0/1104368610 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5587696bee60 con 0x558769943ff0 2023-11-27T15:27:02.707 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.694+0000 7f64caff5700 1 -- 172.21.15.52:0/1104368610 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64dc004180 con 0x558769943ff0 2023-11-27T15:27:02.707 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.694+0000 7f64caff5700 1 -- 172.21.15.52:0/1104368610 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f64dc0b3b50 con 0x558769943ff0 2023-11-27T15:27:02.707 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.694+0000 7f64f933db80 10 client.17049 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:27:02.707 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.694+0000 7f64f933db80 10 client.17049 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:02.707 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.694+0000 7f64f933db80 20 client.17049 populate_metadata read hostname 'smithi052' 2023-11-27T15:27:02.707 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.694+0000 7f64f933db80 1 --2- 172.21.15.52:0/1104368610 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5587699b25a0 0x5587699b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.708 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.694+0000 7f64f933db80 4 client.17049 mds_command: new command op to 24536 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-27T15:27:02.708 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.694+0000 7f64f933db80 1 -- 172.21.15.52:0/1104368610 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55876985f6e0 con 0x5587699b25a0 2023-11-27T15:27:02.708 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.695+0000 7f64e659c700 1 --2- 172.21.15.52:0/1104368610 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5587699b25a0 0x5587699b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.708 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.695+0000 7f64caff5700 10 client.17049 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:02.708 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.695+0000 7f64e659c700 1 --2- 172.21.15.52:0/1104368610 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5587699b25a0 0x5587699b4980 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f695bce066f34977 server_cookie=13d76be2787a6cfa in_seq=0 out_seq=0 2023-11-27T15:27:02.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.696+0000 7f64caff5700 1 -- 172.21.15.52:0/1104368610 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55876985f6e0 con 0x5587699b25a0 2023-11-27T15:27:02.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.696+0000 7f64caff5700 10 client.17049 handle_command_reply: tid=0 2023-11-27T15:27:02.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.696+0000 7f64f933db80 1 client.17049 shutdown 2023-11-27T15:27:02.709 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.697+0000 7f64f933db80 1 -- 172.21.15.52:0/1104368610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558769943ff0 msgr2=0x558769954580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:02.710 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.697+0000 7f64f933db80 1 --2- 172.21.15.52:0/1104368610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558769943ff0 0x558769954580 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009fd0 tx=0x7f64dc0042f0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.710 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.697+0000 7f64f933db80 1 -- 172.21.15.52:0/1104368610 shutdown_connections 2023-11-27T15:27:02.710 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.697+0000 7f64f933db80 1 --2- 172.21.15.52:0/1104368610 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5587699b25a0 0x5587699b4980 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.710 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.697+0000 7f64f933db80 1 --2- 172.21.15.52:0/1104368610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558769948d40 0x558769950b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.710 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.697+0000 7f64f933db80 1 --2- 172.21.15.52:0/1104368610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558769943ff0 0x558769954580 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.710 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.697+0000 7f64f933db80 1 --2- 172.21.15.52:0/1104368610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558769944900 0x558769954ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.710 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.697+0000 7f64f933db80 1 -- 172.21.15.52:0/1104368610 >> 172.21.15.52:0/1104368610 conn(0x5587699353e0 msgr2=0x5587699483f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:02.710 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.697+0000 7f64caff5700 0 client.17049 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:02.711 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.697+0000 7f64f933db80 1 -- 172.21.15.52:0/1104368610 shutdown_connections 2023-11-27T15:27:02.711 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.697+0000 7f64f933db80 1 -- 172.21.15.52:0/1104368610 wait complete. 2023-11-27T15:27:02.711 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.697+0000 7f64f933db80 20 client.17049 trim_cache size 0 max 16384 2023-11-27T15:27:02.711 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:27:02.711 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2023-11-27T15:27:02.711 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "f2b993c0-1198-46a5-8fbe-490e9efe0545", 2023-11-27T15:27:02.711 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2023-11-27T15:27:02.712 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:27:02.712 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.697+0000 7f64ed1b1700 1 -- 172.21.15.52:0/2135387666 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f64d00a12c0 msgr2=0x7f64d00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:02.712 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.697+0000 7f64ed1b1700 1 --2- 172.21.15.52:0/2135387666 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f64d00a12c0 0x7f64d00a3770 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f64d8005950 tx=0x7f64d80058e0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.713 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.697+0000 7f64ed1b1700 1 -- 172.21.15.52:0/2135387666 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64e81005b0 msgr2=0x7f64e81963f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:02.713 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.697+0000 7f64ed1b1700 1 --2- 172.21.15.52:0/2135387666 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64e81005b0 0x7f64e81963f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009990 tx=0x7f64dc009580 comp rx=0 tx=0).stop 2023-11-27T15:27:02.713 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.697+0000 7f64ed1b1700 1 -- 172.21.15.52:0/2135387666 shutdown_connections 2023-11-27T15:27:02.713 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.697+0000 7f64ed1b1700 1 --2- 172.21.15.52:0/2135387666 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f64d00a12c0 0x7f64d00a3770 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.713 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.697+0000 7f64ed1b1700 1 --2- 172.21.15.52:0/2135387666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64e81055f0 0x7f64e819ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.713 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.697+0000 7f64ed1b1700 1 --2- 172.21.15.52:0/2135387666 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64e81005b0 0x7f64e81963f0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.713 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.697+0000 7f64ed1b1700 1 --2- 172.21.15.52:0/2135387666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64e8100f50 0x7f64e8196930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.713 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.697+0000 7f64ed1b1700 1 -- 172.21.15.52:0/2135387666 >> 172.21.15.52:0/2135387666 conn(0x7f64e8077170 msgr2=0x7f64e80fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:02.714 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.698+0000 7f64ed1b1700 1 -- 172.21.15.52:0/2135387666 shutdown_connections 2023-11-27T15:27:02.714 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.698+0000 7f64ed1b1700 1 -- 172.21.15.52:0/2135387666 wait complete. 2023-11-27T15:27:02.715 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:27:02.842 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.838+0000 7f79dc44f700 1 -- 172.21.15.52:0/2466672244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79d4100f10 msgr2=0x7f79d4104ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:02.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.838+0000 7f79dc44f700 1 --2- 172.21.15.52:0/2466672244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79d4100f10 0x7f79d4104ed0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f79c40099c0 tx=0x7f79c4009cd0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.838+0000 7f79dc44f700 1 -- 172.21.15.52:0/2466672244 shutdown_connections 2023-11-27T15:27:02.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.838+0000 7f79dc44f700 1 --2- 172.21.15.52:0/2466672244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79d41055b0 0x7f79d410d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.838+0000 7f79dc44f700 1 --2- 172.21.15.52:0/2466672244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79d4100570 0x7f79d4100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.838+0000 7f79dc44f700 1 --2- 172.21.15.52:0/2466672244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79d4100f10 0x7f79d4104ed0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.838+0000 7f79dc44f700 1 -- 172.21.15.52:0/2466672244 >> 172.21.15.52:0/2466672244 conn(0x7f79d40770e0 msgr2=0x7f79d40774e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:02.843 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.838+0000 7f79dc44f700 1 -- 172.21.15.52:0/2466672244 shutdown_connections 2023-11-27T15:27:02.844 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.838+0000 7f79dc44f700 1 -- 172.21.15.52:0/2466672244 wait complete. 2023-11-27T15:27:02.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.839+0000 7f79dc44f700 1 Processor -- start 2023-11-27T15:27:02.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.839+0000 7f79dc44f700 1 -- start start 2023-11-27T15:27:02.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.839+0000 7f79dc44f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79d4100570 0x7f79d4196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.845 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.839+0000 7f79dc44f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79d4100f10 0x7f79d41969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.839+0000 7f79dc44f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79d41055b0 0x7f79d419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.839+0000 7f79dc44f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d419b470 con 0x7f79d4100f10 2023-11-27T15:27:02.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.839+0000 7f79dc44f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d419b5e0 con 0x7f79d4100570 2023-11-27T15:27:02.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.839+0000 7f79dc44f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d419b7a0 con 0x7f79d41055b0 2023-11-27T15:27:02.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.839+0000 7f79d99ea700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79d4100f10 0x7f79d41969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.839+0000 7f79d99ea700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79d4100f10 0x7f79d41969d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55910/0 (socket says 172.21.15.52:55910) 2023-11-27T15:27:02.846 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.839+0000 7f79d99ea700 1 -- 172.21.15.52:0/1523604929 learned_addr learned my addr 172.21.15.52:0/1523604929 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:02.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.839+0000 7f79da9ec700 1 --2- 172.21.15.52:0/1523604929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79d41055b0 0x7f79d419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.839+0000 7f79da1eb700 1 --2- 172.21.15.52:0/1523604929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79d4100570 0x7f79d4196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.839+0000 7f79d99ea700 1 -- 172.21.15.52:0/1523604929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79d41055b0 msgr2=0x7f79d419ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:02.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.839+0000 7f79d99ea700 1 --2- 172.21.15.52:0/1523604929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79d41055b0 0x7f79d419ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.839+0000 7f79d99ea700 1 -- 172.21.15.52:0/1523604929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79d4100570 msgr2=0x7f79d4196490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:02.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.839+0000 7f79d99ea700 1 --2- 172.21.15.52:0/1523604929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79d4100570 0x7f79d4196490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.839+0000 7f79d99ea700 1 -- 172.21.15.52:0/1523604929 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c40095f0 con 0x7f79d4100f10 2023-11-27T15:27:02.847 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.839+0000 7f79d99ea700 1 --2- 172.21.15.52:0/1523604929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79d4100f10 0x7f79d41969d0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f79c40098e0 tx=0x7f79c400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:02.848 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.840+0000 7f79cb7fe700 1 -- 172.21.15.52:0/1523604929 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c401d070 con 0x7f79d4100f10 2023-11-27T15:27:02.848 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.840+0000 7f79cb7fe700 1 -- 172.21.15.52:0/1523604929 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79c4022470 con 0x7f79d4100f10 2023-11-27T15:27:02.848 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.840+0000 7f79dc44f700 1 -- 172.21.15.52:0/1523604929 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d4110330 con 0x7f79d4100f10 2023-11-27T15:27:02.848 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.840+0000 7f79cb7fe700 1 -- 172.21.15.52:0/1523604929 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c4017650 con 0x7f79d4100f10 2023-11-27T15:27:02.848 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.840+0000 7f79dc44f700 1 -- 172.21.15.52:0/1523604929 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d41101a0 con 0x7f79d4100f10 2023-11-27T15:27:02.854 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.842+0000 7f79cb7fe700 1 -- 172.21.15.52:0/1523604929 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f79c4022890 con 0x7f79d4100f10 2023-11-27T15:27:02.854 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.842+0000 7f79cb7fe700 1 --2- 172.21.15.52:0/1523604929 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f79c00a1340 0x7f79c00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.854 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.842+0000 7f79cb7fe700 1 -- 172.21.15.52:0/1523604929 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f79c40cd930 con 0x7f79d4100f10 2023-11-27T15:27:02.854 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.842+0000 7f79da1eb700 1 --2- 172.21.15.52:0/1523604929 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f79c00a1340 0x7f79c00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.855 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.843+0000 7f79da1eb700 1 --2- 172.21.15.52:0/1523604929 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f79c00a1340 0x7f79c00a37f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f79d0009c60 tx=0x7f79d0009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:02.855 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.843+0000 7f79e85dbb80 1 Processor -- start 2023-11-27T15:27:02.855 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.844+0000 7f79e85dbb80 1 -- start start 2023-11-27T15:27:02.855 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.844+0000 7f79e85dbb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec9fe88b60 0x55ec9fe88f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.855 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.844+0000 7f79e85dbb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec9fe89470 0x55ec9fe8d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.855 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.844+0000 7f79e85dbb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec9fe8d8b0 0x55ec9fe8dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.855 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.844+0000 7f79e85dbb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9fd8e060 con 0x55ec9fe8d8b0 2023-11-27T15:27:02.855 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.844+0000 7f79e85dbb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9fde08a0 con 0x55ec9fe89470 2023-11-27T15:27:02.855 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.844+0000 7f79e85dbb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9fdd9d30 con 0x55ec9fe88b60 2023-11-27T15:27:02.856 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.844+0000 7f79da9ec700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec9fe8d8b0 0x55ec9fe8dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.856 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.844+0000 7f79da9ec700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec9fe8d8b0 0x55ec9fe8dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55920/0 (socket says 172.21.15.52:55920) 2023-11-27T15:27:02.856 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.844+0000 7f79da9ec700 1 -- 172.21.15.52:0/3240807954 learned_addr learned my addr 172.21.15.52:0/3240807954 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:02.856 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.844+0000 7f79d99ea700 1 --2- 172.21.15.52:0/3240807954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec9fe89470 0x55ec9fe8d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.856 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.844+0000 7f79da1eb700 1 --2- 172.21.15.52:0/3240807954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec9fe88b60 0x55ec9fe88f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.856 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.844+0000 7f79da9ec700 1 -- 172.21.15.52:0/3240807954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec9fe88b60 msgr2=0x55ec9fe88f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:02.856 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.844+0000 7f79da9ec700 1 --2- 172.21.15.52:0/3240807954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec9fe88b60 0x55ec9fe88f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.856 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.844+0000 7f79da9ec700 1 -- 172.21.15.52:0/3240807954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec9fe89470 msgr2=0x55ec9fe8d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:02.857 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.844+0000 7f79da9ec700 1 --2- 172.21.15.52:0/3240807954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec9fe89470 0x55ec9fe8d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.857 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.844+0000 7f79da9ec700 1 -- 172.21.15.52:0/3240807954 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec9fc5b630 con 0x55ec9fe8d8b0 2023-11-27T15:27:02.857 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.844+0000 7f79da1eb700 1 --2- 172.21.15.52:0/3240807954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec9fe88b60 0x55ec9fe88f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:02.857 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.844+0000 7f79da9ec700 1 --2- 172.21.15.52:0/3240807954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec9fe8d8b0 0x55ec9fe8dd00 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f79cc011680 tx=0x7f79cc009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b270de3b565bb5a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:02.857 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.845+0000 7f79c8dac700 1 -- 172.21.15.52:0/3240807954 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc00e040 con 0x55ec9fe8d8b0 2023-11-27T15:27:02.857 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.845+0000 7f79c8dac700 1 -- 172.21.15.52:0/3240807954 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79cc00f040 con 0x55ec9fe8d8b0 2023-11-27T15:27:02.857 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.845+0000 7f79c8dac700 1 -- 172.21.15.52:0/3240807954 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc004860 con 0x55ec9fe8d8b0 2023-11-27T15:27:02.857 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.845+0000 7f79e85dbb80 1 -- 172.21.15.52:0/3240807954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec9fe8d8b0 msgr2=0x55ec9fe8dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:02.858 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.845+0000 7f79e85dbb80 1 --2- 172.21.15.52:0/3240807954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec9fe8d8b0 0x55ec9fe8dd00 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f79cc011680 tx=0x7f79cc009920 comp rx=0 tx=0).stop 2023-11-27T15:27:02.858 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.845+0000 7f79e85dbb80 1 -- 172.21.15.52:0/3240807954 shutdown_connections 2023-11-27T15:27:02.858 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.845+0000 7f79e85dbb80 1 --2- 172.21.15.52:0/3240807954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec9fe8d8b0 0x55ec9fe8dd00 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.858 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.845+0000 7f79e85dbb80 1 --2- 172.21.15.52:0/3240807954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec9fe88b60 0x55ec9fe88f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.858 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.845+0000 7f79e85dbb80 1 --2- 172.21.15.52:0/3240807954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec9fe89470 0x55ec9fe8d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.858 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.845+0000 7f79e85dbb80 1 -- 172.21.15.52:0/3240807954 >> 172.21.15.52:0/3240807954 conn(0x55ec9fe7e360 msgr2=0x55ec9fe83b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:02.858 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.845+0000 7f79e85dbb80 1 -- 172.21.15.52:0/3240807954 shutdown_connections 2023-11-27T15:27:02.858 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.845+0000 7f79e85dbb80 1 -- 172.21.15.52:0/3240807954 wait complete. 2023-11-27T15:27:02.859 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.846+0000 7f79e85dbb80 1 -- start start 2023-11-27T15:27:02.859 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.846+0000 7f79e85dbb80 1 Processor -- start 2023-11-27T15:27:02.859 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.846+0000 7f79e85dbb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec9fe88b60 0x55ec9fe95080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.859 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.846+0000 7f79e85dbb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec9fe89470 0x55ec9fe92290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.859 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.846+0000 7f79e85dbb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec9fe8d8b0 0x55ec9fe929d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.859 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.846+0000 7f79e85dbb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9fd7b8e0 con 0x55ec9fe88b60 2023-11-27T15:27:02.859 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.846+0000 7f79e85dbb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9fd7b580 con 0x55ec9fe8d8b0 2023-11-27T15:27:02.859 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.846+0000 7f79e85dbb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9fc5d920 con 0x55ec9fe89470 2023-11-27T15:27:02.860 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.846+0000 7f79e85dbb80 10 client.0 _ll_register_callbacks cb 0x55ec9fd99780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:02.860 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.846+0000 7f79da1eb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec9fe88b60 0x55ec9fe95080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.860 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.846+0000 7f79da1eb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec9fe88b60 0x55ec9fe95080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55922/0 (socket says 172.21.15.52:55922) 2023-11-27T15:27:02.860 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.846+0000 7f79da1eb700 1 -- 172.21.15.52:0/4268296529 learned_addr learned my addr 172.21.15.52:0/4268296529 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:02.860 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.846+0000 7f79da9ec700 1 --2- 172.21.15.52:0/4268296529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec9fe8d8b0 0x55ec9fe929d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.860 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.847+0000 7f79d99ea700 1 --2- 172.21.15.52:0/4268296529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec9fe89470 0x55ec9fe92290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.860 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.847+0000 7f79da1eb700 1 -- 172.21.15.52:0/4268296529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec9fe89470 msgr2=0x55ec9fe92290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:02.860 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.847+0000 7f79da1eb700 1 --2- 172.21.15.52:0/4268296529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec9fe89470 0x55ec9fe92290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.861 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.847+0000 7f79da1eb700 1 -- 172.21.15.52:0/4268296529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec9fe8d8b0 msgr2=0x55ec9fe929d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:02.861 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.847+0000 7f79da1eb700 1 --2- 172.21.15.52:0/4268296529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec9fe8d8b0 0x55ec9fe929d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.861 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.847+0000 7f79da1eb700 1 -- 172.21.15.52:0/4268296529 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79cc011330 con 0x55ec9fe88b60 2023-11-27T15:27:02.861 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.847+0000 7f79da9ec700 1 --2- 172.21.15.52:0/4268296529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec9fe8d8b0 0x55ec9fe929d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:02.861 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.847+0000 7f79da1eb700 1 --2- 172.21.15.52:0/4268296529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec9fe88b60 0x55ec9fe95080 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f79d001ec80 tx=0x7f79d00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ec0b2fc1f72e20 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:02.861 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.847+0000 7f79bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:27:02.861 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.847+0000 7f79bdffb700 1 -- 172.21.15.52:0/4268296529 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d000ede0 con 0x55ec9fe88b60 2023-11-27T15:27:02.861 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.847+0000 7f79bdffb700 1 -- 172.21.15.52:0/4268296529 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f79d001c430 con 0x55ec9fe88b60 2023-11-27T15:27:02.861 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.847+0000 7f79e85dbb80 1 -- 172.21.15.52:0/4268296529 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec9fe7bb20 con 0x55ec9fe88b60 2023-11-27T15:27:02.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.847+0000 7f79bdffb700 1 -- 172.21.15.52:0/4268296529 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79d001c5f0 con 0x55ec9fe88b60 2023-11-27T15:27:02.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.847+0000 7f79e85dbb80 10 client.17061 fetch_fsmap learned FSMap version 34 2023-11-27T15:27:02.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.847+0000 7f79e85dbb80 1 -- 172.21.15.52:0/4268296529 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55ec9fda0190 con 0x55ec9fe88b60 2023-11-27T15:27:02.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.847+0000 7f79bdffb700 1 -- 172.21.15.52:0/4268296529 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79d0033080 con 0x55ec9fe88b60 2023-11-27T15:27:02.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.848+0000 7f79bdffb700 1 -- 172.21.15.52:0/4268296529 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f79d0027880 con 0x55ec9fe88b60 2023-11-27T15:27:02.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.848+0000 7f79e85dbb80 10 client.17061 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:27:02.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.848+0000 7f79e85dbb80 10 client.17061 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:02.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.848+0000 7f79e85dbb80 20 client.17061 populate_metadata read hostname 'smithi052' 2023-11-27T15:27:02.862 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.848+0000 7f79e85dbb80 1 --2- 172.21.15.52:0/4268296529 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55ec9fea2900 0x55ec9fea4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.848+0000 7f79e85dbb80 4 client.17061 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:27:02.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.848+0000 7f79e85dbb80 1 -- 172.21.15.52:0/4268296529 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec9fd9c0a0 con 0x55ec9fea2900 2023-11-27T15:27:02.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.848+0000 7f79d99ea700 1 --2- 172.21.15.52:0/4268296529 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55ec9fea2900 0x55ec9fea4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.848+0000 7f79bdffb700 10 client.17061 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:02.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.848+0000 7f79d99ea700 1 --2- 172.21.15.52:0/4268296529 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55ec9fea2900 0x55ec9fea4d10 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd2f3c2d9b7662ab server_cookie=9c5a0117c09604dc in_seq=0 out_seq=0 2023-11-27T15:27:02.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.849+0000 7f79bdffb700 1 -- 172.21.15.52:0/4268296529 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ec9fd9c0a0 con 0x55ec9fea2900 2023-11-27T15:27:02.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.849+0000 7f79bdffb700 10 client.17061 handle_command_reply: tid=0 2023-11-27T15:27:02.863 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.849+0000 7f79e85dbb80 1 client.17061 shutdown 2023-11-27T15:27:02.864 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.849+0000 7f79e85dbb80 1 -- 172.21.15.52:0/4268296529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec9fe88b60 msgr2=0x55ec9fe95080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:02.864 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.849+0000 7f79e85dbb80 1 --2- 172.21.15.52:0/4268296529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec9fe88b60 0x55ec9fe95080 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f79d001ec80 tx=0x7f79d00047c0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.864 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.849+0000 7f79e85dbb80 1 -- 172.21.15.52:0/4268296529 shutdown_connections 2023-11-27T15:27:02.864 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.849+0000 7f79e85dbb80 1 --2- 172.21.15.52:0/4268296529 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55ec9fea2900 0x55ec9fea4d10 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.864 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.849+0000 7f79e85dbb80 1 --2- 172.21.15.52:0/4268296529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec9fe8d8b0 0x55ec9fe929d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.864 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.849+0000 7f79bdffb700 0 client.17061 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:02.864 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.849+0000 7f79e85dbb80 1 --2- 172.21.15.52:0/4268296529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec9fe88b60 0x55ec9fe95080 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.864 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.849+0000 7f79e85dbb80 1 --2- 172.21.15.52:0/4268296529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec9fe89470 0x55ec9fe92290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.849+0000 7f79e85dbb80 1 -- 172.21.15.52:0/4268296529 >> 172.21.15.52:0/4268296529 conn(0x55ec9fe7e360 msgr2=0x55ec9fe82de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:02.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.849+0000 7f79e85dbb80 1 -- 172.21.15.52:0/4268296529 shutdown_connections 2023-11-27T15:27:02.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.849+0000 7f79e85dbb80 1 -- 172.21.15.52:0/4268296529 wait complete. 2023-11-27T15:27:02.865 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.849+0000 7f79e85dbb80 20 client.17061 trim_cache size 0 max 16384 2023-11-27T15:27:02.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.867+0000 7f79e85dbb80 1 Processor -- start 2023-11-27T15:27:02.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.867+0000 7f79e85dbb80 1 -- start start 2023-11-27T15:27:02.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.867+0000 7f79e85dbb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec9fe88b60 0x55ec9fe94bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.867+0000 7f79e85dbb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec9fe89470 0x55ec9fe999a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.867+0000 7f79e85dbb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec9fe8d8b0 0x55ec9fe91b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.867+0000 7f79e85dbb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9fe8c460 con 0x55ec9fe88b60 2023-11-27T15:27:02.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.867+0000 7f79e85dbb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9fe89900 con 0x55ec9fe8d8b0 2023-11-27T15:27:02.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.867+0000 7f79e85dbb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9fe8c140 con 0x55ec9fe89470 2023-11-27T15:27:02.878 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.867+0000 7f79da1eb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec9fe88b60 0x55ec9fe94bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.878 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.867+0000 7f79da1eb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec9fe88b60 0x55ec9fe94bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55930/0 (socket says 172.21.15.52:55930) 2023-11-27T15:27:02.878 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.867+0000 7f79da1eb700 1 -- 172.21.15.52:0/2140108542 learned_addr learned my addr 172.21.15.52:0/2140108542 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:02.878 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.867+0000 7f79d99ea700 1 --2- 172.21.15.52:0/2140108542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec9fe89470 0x55ec9fe999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.878 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.867+0000 7f79da1eb700 1 -- 172.21.15.52:0/2140108542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec9fe89470 msgr2=0x55ec9fe999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:02.878 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.867+0000 7f79da1eb700 1 --2- 172.21.15.52:0/2140108542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec9fe89470 0x55ec9fe999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.878 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.867+0000 7f79da9ec700 1 --2- 172.21.15.52:0/2140108542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec9fe8d8b0 0x55ec9fe91b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.879 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.867+0000 7f79da1eb700 1 -- 172.21.15.52:0/2140108542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec9fe8d8b0 msgr2=0x55ec9fe91b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:02.879 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.867+0000 7f79da1eb700 1 --2- 172.21.15.52:0/2140108542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec9fe8d8b0 0x55ec9fe91b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.879 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.867+0000 7f79da1eb700 1 -- 172.21.15.52:0/2140108542 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec9fd9fee0 con 0x55ec9fe88b60 2023-11-27T15:27:02.879 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.867+0000 7f79d99ea700 1 --2- 172.21.15.52:0/2140108542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec9fe89470 0x55ec9fe999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:02.879 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.867+0000 7f79da9ec700 1 --2- 172.21.15.52:0/2140108542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec9fe8d8b0 0x55ec9fe91b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:02.879 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.867+0000 7f79da1eb700 1 --2- 172.21.15.52:0/2140108542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec9fe88b60 0x55ec9fe94bb0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f79d0009ac0 tx=0x7f79d00096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4866e9e2176f9231 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:02.879 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.868+0000 7f79c95ad700 1 -- 172.21.15.52:0/2140108542 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d0027440 con 0x55ec9fe88b60 2023-11-27T15:27:02.880 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.868+0000 7f79c95ad700 1 -- 172.21.15.52:0/2140108542 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ec9fe7bb20 con 0x55ec9fe88b60 2023-11-27T15:27:02.880 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.868+0000 7f79c95ad700 1 -- 172.21.15.52:0/2140108542 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d002fc80 con 0x55ec9fe88b60 2023-11-27T15:27:02.880 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.868+0000 7f79e85dbb80 1 -- 172.21.15.52:0/2140108542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec9fe88b60 msgr2=0x55ec9fe94bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:02.880 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.868+0000 7f79e85dbb80 1 --2- 172.21.15.52:0/2140108542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec9fe88b60 0x55ec9fe94bb0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f79d0009ac0 tx=0x7f79d00096d0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.880 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.868+0000 7f79e85dbb80 1 -- 172.21.15.52:0/2140108542 shutdown_connections 2023-11-27T15:27:02.880 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.868+0000 7f79e85dbb80 1 --2- 172.21.15.52:0/2140108542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec9fe8d8b0 0x55ec9fe91b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.880 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.868+0000 7f79e85dbb80 1 --2- 172.21.15.52:0/2140108542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec9fe88b60 0x55ec9fe94bb0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.880 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.868+0000 7f79e85dbb80 1 --2- 172.21.15.52:0/2140108542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec9fe89470 0x55ec9fe999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.881 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.868+0000 7f79e85dbb80 1 -- 172.21.15.52:0/2140108542 >> 172.21.15.52:0/2140108542 conn(0x55ec9fe7e360 msgr2=0x55ec9fe8fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:02.881 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.868+0000 7f79e85dbb80 1 -- 172.21.15.52:0/2140108542 shutdown_connections 2023-11-27T15:27:02.881 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.868+0000 7f79e85dbb80 1 -- 172.21.15.52:0/2140108542 wait complete. 2023-11-27T15:27:02.881 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.868+0000 7f79e85dbb80 1 -- start start 2023-11-27T15:27:02.881 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.869+0000 7f79e85dbb80 1 Processor -- start 2023-11-27T15:27:02.881 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.869+0000 7f79e85dbb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec9fe88b60 0x55ec9fe95b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.881 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.869+0000 7f79e85dbb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec9fe89470 0x55ec9fe960f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.881 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.869+0000 7f79e85dbb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec9fe8d8b0 0x55ec9feef7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.882 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.869+0000 7f79e85dbb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9feefce0 con 0x55ec9fe88b60 2023-11-27T15:27:02.882 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.869+0000 7f79e85dbb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9feefe50 con 0x55ec9fe8d8b0 2023-11-27T15:27:02.882 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.869+0000 7f79e85dbb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec9fef0010 con 0x55ec9fe89470 2023-11-27T15:27:02.882 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.869+0000 7f79e85dbb80 10 client.0 _ll_register_callbacks cb 0x7f79d001e7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:02.882 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.869+0000 7f79da1eb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec9fe88b60 0x55ec9fe95b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.882 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.869+0000 7f79da1eb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec9fe88b60 0x55ec9fe95b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55936/0 (socket says 172.21.15.52:55936) 2023-11-27T15:27:02.882 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.869+0000 7f79da1eb700 1 -- 172.21.15.52:0/2049031539 learned_addr learned my addr 172.21.15.52:0/2049031539 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:02.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.869+0000 7f79da9ec700 1 --2- 172.21.15.52:0/2049031539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec9fe8d8b0 0x55ec9feef7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.869+0000 7f79d99ea700 1 --2- 172.21.15.52:0/2049031539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec9fe89470 0x55ec9fe960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.869+0000 7f79da1eb700 1 -- 172.21.15.52:0/2049031539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec9fe89470 msgr2=0x55ec9fe960f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:02.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.869+0000 7f79da1eb700 1 --2- 172.21.15.52:0/2049031539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec9fe89470 0x55ec9fe960f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.869+0000 7f79da1eb700 1 -- 172.21.15.52:0/2049031539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec9fe8d8b0 msgr2=0x55ec9feef7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:02.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.869+0000 7f79da1eb700 1 --2- 172.21.15.52:0/2049031539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec9fe8d8b0 0x55ec9feef7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.869+0000 7f79da1eb700 1 -- 172.21.15.52:0/2049031539 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec9fda0190 con 0x55ec9fe88b60 2023-11-27T15:27:02.883 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.869+0000 7f79d99ea700 1 --2- 172.21.15.52:0/2049031539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec9fe89470 0x55ec9fe960f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:02.884 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.869+0000 7f79da1eb700 1 --2- 172.21.15.52:0/2049031539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec9fe88b60 0x55ec9fe95b70 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f79d0027d80 tx=0x7f79d0027c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec545d9f7c678e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:02.884 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.869+0000 7f79bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:27:02.884 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.870+0000 7f79bdffb700 1 -- 172.21.15.52:0/2049031539 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d002f620 con 0x55ec9fe88b60 2023-11-27T15:27:02.884 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.870+0000 7f79bdffb700 1 -- 172.21.15.52:0/2049031539 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f79d002f430 con 0x55ec9fe88b60 2023-11-27T15:27:02.884 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.870+0000 7f79e85dbb80 1 -- 172.21.15.52:0/2049031539 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f79cc00f040 con 0x55ec9fe88b60 2023-11-27T15:27:02.884 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.870+0000 7f79bdffb700 1 -- 172.21.15.52:0/2049031539 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79d002f900 con 0x55ec9fe88b60 2023-11-27T15:27:02.884 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.870+0000 7f79e85dbb80 10 client.17067 fetch_fsmap learned FSMap version 34 2023-11-27T15:27:02.884 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.870+0000 7f79e85dbb80 1 -- 172.21.15.52:0/2049031539 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55ec9fbf0c60 con 0x55ec9fe88b60 2023-11-27T15:27:02.885 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.870+0000 7f79bdffb700 1 -- 172.21.15.52:0/2049031539 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79cc00f040 con 0x55ec9fe88b60 2023-11-27T15:27:02.885 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.870+0000 7f79bdffb700 1 -- 172.21.15.52:0/2049031539 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f79d0038030 con 0x55ec9fe88b60 2023-11-27T15:27:02.885 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.870+0000 7f79e85dbb80 10 client.17067 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:27:02.885 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.870+0000 7f79e85dbb80 10 client.17067 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:02.885 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.870+0000 7f79e85dbb80 20 client.17067 populate_metadata read hostname 'smithi052' 2023-11-27T15:27:02.885 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.870+0000 7f79e85dbb80 1 --2- 172.21.15.52:0/2049031539 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55ec9fef7430 0x55ec9fef9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:02.885 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.870+0000 7f79e85dbb80 4 client.17067 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:27:02.885 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.870+0000 7f79e85dbb80 1 -- 172.21.15.52:0/2049031539 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ec9fda8660 con 0x55ec9fef7430 2023-11-27T15:27:02.886 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.870+0000 7f79d99ea700 1 --2- 172.21.15.52:0/2049031539 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55ec9fef7430 0x55ec9fef9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:02.886 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.871+0000 7f79d99ea700 1 --2- 172.21.15.52:0/2049031539 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55ec9fef7430 0x55ec9fef9840 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49121c9663de988e server_cookie=51d5c6f20c9ab27e in_seq=0 out_seq=0 2023-11-27T15:27:02.886 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.871+0000 7f79bdffb700 10 client.17067 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:02.886 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.871+0000 7f79bdffb700 1 -- 172.21.15.52:0/2049031539 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ec9fda8660 con 0x55ec9fef7430 2023-11-27T15:27:02.886 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.871+0000 7f79bdffb700 10 client.17067 handle_command_reply: tid=0 2023-11-27T15:27:02.886 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.871+0000 7f79e85dbb80 1 client.17067 shutdown 2023-11-27T15:27:02.886 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.871+0000 7f79e85dbb80 1 -- 172.21.15.52:0/2049031539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec9fe88b60 msgr2=0x55ec9fe95b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:02.887 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.871+0000 7f79e85dbb80 1 --2- 172.21.15.52:0/2049031539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec9fe88b60 0x55ec9fe95b70 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f79d0027d80 tx=0x7f79d0027c50 comp rx=0 tx=0).stop 2023-11-27T15:27:02.887 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.871+0000 7f79e85dbb80 1 -- 172.21.15.52:0/2049031539 shutdown_connections 2023-11-27T15:27:02.887 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.871+0000 7f79e85dbb80 1 --2- 172.21.15.52:0/2049031539 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55ec9fef7430 0x55ec9fef9840 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.887 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.871+0000 7f79e85dbb80 1 --2- 172.21.15.52:0/2049031539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec9fe8d8b0 0x55ec9feef7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.887 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.871+0000 7f79bdffb700 0 client.17067 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:02.887 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.871+0000 7f79e85dbb80 1 --2- 172.21.15.52:0/2049031539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec9fe88b60 0x55ec9fe95b70 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.887 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.871+0000 7f79e85dbb80 1 --2- 172.21.15.52:0/2049031539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ec9fe89470 0x55ec9fe960f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.887 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.871+0000 7f79e85dbb80 1 -- 172.21.15.52:0/2049031539 >> 172.21.15.52:0/2049031539 conn(0x55ec9fe7e360 msgr2=0x55ec9fe8fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:02.888 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.871+0000 7f79e85dbb80 1 -- 172.21.15.52:0/2049031539 shutdown_connections 2023-11-27T15:27:02.888 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.871+0000 7f79e85dbb80 1 -- 172.21.15.52:0/2049031539 wait complete. 2023-11-27T15:27:02.888 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.871+0000 7f79e85dbb80 20 client.17067 trim_cache size 0 max 16384 2023-11-27T15:27:02.888 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:27:02.888 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-27T15:27:02.888 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2023-11-27T15:27:02.889 INFO:teuthology.orchestra.run.smithi052.stdout: "f2b993c0-1198-46a5-8fbe-490e9efe0545": { 2023-11-27T15:27:02.889 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2023-11-27T15:27:02.889 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "f2b993c0-1198-46a5-8fbe-490e9efe0545", 2023-11-27T15:27:02.889 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2023-11-27T15:27:02.889 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:27:02.889 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:27:02.889 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:27:02.890 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.872+0000 7f79dc44f700 1 -- 172.21.15.52:0/1523604929 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f79c00a1340 msgr2=0x7f79c00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:02.891 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.872+0000 7f79dc44f700 1 --2- 172.21.15.52:0/1523604929 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f79c00a1340 0x7f79c00a37f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f79d0009c60 tx=0x7f79d0009260 comp rx=0 tx=0).stop 2023-11-27T15:27:02.891 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.872+0000 7f79dc44f700 1 -- 172.21.15.52:0/1523604929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79d4100f10 msgr2=0x7f79d41969d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:02.891 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.872+0000 7f79dc44f700 1 --2- 172.21.15.52:0/1523604929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79d4100f10 0x7f79d41969d0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f79c40098e0 tx=0x7f79c400f690 comp rx=0 tx=0).stop 2023-11-27T15:27:02.891 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.872+0000 7f79dc44f700 1 -- 172.21.15.52:0/1523604929 shutdown_connections 2023-11-27T15:27:02.891 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.872+0000 7f79dc44f700 1 --2- 172.21.15.52:0/1523604929 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f79c00a1340 0x7f79c00a37f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.891 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.872+0000 7f79dc44f700 1 --2- 172.21.15.52:0/1523604929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f79d4100570 0x7f79d4196490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.891 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.872+0000 7f79dc44f700 1 --2- 172.21.15.52:0/1523604929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79d41055b0 0x7f79d419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.892 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.872+0000 7f79dc44f700 1 --2- 172.21.15.52:0/1523604929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79d4100f10 0x7f79d41969d0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:02.892 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.872+0000 7f79dc44f700 1 -- 172.21.15.52:0/1523604929 >> 172.21.15.52:0/1523604929 conn(0x7f79d40770e0 msgr2=0x7f79d40fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:02.892 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.872+0000 7f79dc44f700 1 -- 172.21.15.52:0/1523604929 shutdown_connections 2023-11-27T15:27:02.892 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:02.872+0000 7f79dc44f700 1 -- 172.21.15.52:0/1523604929 wait complete. 2023-11-27T15:27:02.892 INFO:tasks.cephfs.filesystem:scrub status for tag:f2b993c0-1198-46a5-8fbe-490e9efe0545 - {'path': '/', 'tag': 'f2b993c0-1198-46a5-8fbe-490e9efe0545', 'options': 'recursive,force'} 2023-11-27T15:27:03.277 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:03 smithi150 ceph-mon[100559]: scrub queued for path: / 2023-11-27T15:27:03.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:03 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:03.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:03 smithi150 ceph-mon[100559]: scrub summary: active paths [/] 2023-11-27T15:27:03.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:03 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:03.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:03 smithi102 ceph-mon[100296]: scrub queued for path: / 2023-11-27T15:27:03.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:03 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:03.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:03 smithi102 ceph-mon[100296]: scrub summary: active paths [/] 2023-11-27T15:27:03.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:03 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:03.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:03 smithi052 ceph-mon[99056]: scrub queued for path: / 2023-11-27T15:27:03.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:03 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:03.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:03 smithi052 ceph-mon[99056]: scrub summary: active paths [/] 2023-11-27T15:27:03.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:03 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:04.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:04 smithi102 ceph-mon[100296]: pgmap v283: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-11-27T15:27:04.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:04 smithi052 ceph-mon[99056]: pgmap v283: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-11-27T15:27:04.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:04 smithi150 ceph-mon[100559]: pgmap v283: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-11-27T15:27:05.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:05 smithi102 ceph-mon[100296]: scrub summary: idle 2023-11-27T15:27:05.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:05 smithi052 ceph-mon[99056]: scrub summary: idle 2023-11-27T15:27:05.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:05 smithi150 ceph-mon[100559]: scrub summary: idle 2023-11-27T15:27:06.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:06 smithi102 ceph-mon[100296]: pgmap v284: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-11-27T15:27:06.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:06 smithi052 ceph-mon[99056]: pgmap v284: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-11-27T15:27:06.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:06 smithi150 ceph-mon[100559]: pgmap v284: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-11-27T15:27:08.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:08 smithi150 ceph-mon[100559]: pgmap v285: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-11-27T15:27:08.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:08 smithi102 ceph-mon[100296]: pgmap v285: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-11-27T15:27:08.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:08 smithi052 ceph-mon[99056]: pgmap v285: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-11-27T15:27:10.277 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:09 smithi150 ceph-mon[100559]: pgmap v286: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-11-27T15:27:10.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:09 smithi102 ceph-mon[100296]: pgmap v286: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-11-27T15:27:10.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:09 smithi052 ceph-mon[99056]: pgmap v286: 97 pgs: 97 active+clean; 712 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-11-27T15:27:11.277 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:10 smithi150 ceph-mon[100559]: osdmap e93: 12 total, 12 up, 12 in 2023-11-27T15:27:11.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:10 smithi102 ceph-mon[100296]: osdmap e93: 12 total, 12 up, 12 in 2023-11-27T15:27:11.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:10 smithi052 ceph-mon[99056]: osdmap e93: 12 total, 12 up, 12 in 2023-11-27T15:27:12.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:11 smithi150 ceph-mon[100559]: pgmap v288: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:27:12.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:11 smithi102 ceph-mon[100296]: pgmap v288: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:27:12.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:11 smithi052 ceph-mon[99056]: pgmap v288: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:27:14.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:14 smithi150 ceph-mon[100559]: pgmap v289: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:27:14.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:14 smithi150 ceph-mon[100559]: osdmap e94: 12 total, 12 up, 12 in 2023-11-27T15:27:14.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:14 smithi102 ceph-mon[100296]: pgmap v289: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:27:14.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:14 smithi102 ceph-mon[100296]: osdmap e94: 12 total, 12 up, 12 in 2023-11-27T15:27:14.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:14 smithi052 ceph-mon[99056]: pgmap v289: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:27:14.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:14 smithi052 ceph-mon[99056]: osdmap e94: 12 total, 12 up, 12 in 2023-11-27T15:27:16.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:16 smithi150 ceph-mon[100559]: pgmap v291: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 127 B/s wr, 3 op/s 2023-11-27T15:27:16.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:16 smithi102 ceph-mon[100296]: pgmap v291: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 127 B/s wr, 3 op/s 2023-11-27T15:27:16.615 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:16 smithi052 ceph-mon[99056]: pgmap v291: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 127 B/s wr, 3 op/s 2023-11-27T15:27:17.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:17 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:27:17.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:17 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:27:17.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:17 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:27:18.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:18 smithi150 ceph-mon[100559]: pgmap v292: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-27T15:27:18.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:18 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:27:18.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:18 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:27:18.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:18 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:27:18.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:18 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:27:18.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:18 smithi150 ceph-mon[100559]: Service mds.all has no related daemon already placed. 2023-11-27T15:27:18.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:18 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:27:18.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:18 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:27:18.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:18 smithi102 ceph-mon[100296]: pgmap v292: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-27T15:27:18.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:18 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:27:18.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:18 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:27:18.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:18 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:27:18.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:18 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:27:18.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:18 smithi102 ceph-mon[100296]: Service mds.all has no related daemon already placed. 2023-11-27T15:27:18.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:18 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:27:18.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:18 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:27:18.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:18 smithi052 ceph-mon[99056]: pgmap v292: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-27T15:27:18.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:18 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:27:18.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:18 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:27:18.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:18 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:27:18.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:18 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:27:18.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:18 smithi052 ceph-mon[99056]: Service mds.all has no related daemon already placed. 2023-11-27T15:27:18.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:18 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:27:18.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:18 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:27:20.277 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:19 smithi150 ceph-mon[100559]: pgmap v293: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 221 B/s wr, 6 op/s 2023-11-27T15:27:20.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:19 smithi102 ceph-mon[100296]: pgmap v293: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 221 B/s wr, 6 op/s 2023-11-27T15:27:20.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:19 smithi052 ceph-mon[99056]: pgmap v293: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 221 B/s wr, 6 op/s 2023-11-27T15:27:22.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:22 smithi150 ceph-mon[100559]: pgmap v294: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2023-11-27T15:27:22.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:22 smithi102 ceph-mon[100296]: pgmap v294: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2023-11-27T15:27:22.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:22 smithi052 ceph-mon[99056]: pgmap v294: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2023-11-27T15:27:24.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:24 smithi150 ceph-mon[100559]: pgmap v295: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2023-11-27T15:27:24.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:24 smithi102 ceph-mon[100296]: pgmap v295: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2023-11-27T15:27:24.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:24 smithi052 ceph-mon[99056]: pgmap v295: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2023-11-27T15:27:26.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:26 smithi150 ceph-mon[100559]: pgmap v296: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 90 B/s wr, 5 op/s 2023-11-27T15:27:26.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:26 smithi102 ceph-mon[100296]: pgmap v296: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 90 B/s wr, 5 op/s 2023-11-27T15:27:26.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:26 smithi052 ceph-mon[99056]: pgmap v296: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 90 B/s wr, 5 op/s 2023-11-27T15:27:28.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:28 smithi150 ceph-mon[100559]: pgmap v297: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2023-11-27T15:27:28.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:28 smithi102 ceph-mon[100296]: pgmap v297: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2023-11-27T15:27:28.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:28 smithi052 ceph-mon[99056]: pgmap v297: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2023-11-27T15:27:30.277 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:29 smithi150 ceph-mon[100559]: pgmap v298: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-11-27T15:27:30.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:29 smithi102 ceph-mon[100296]: pgmap v298: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-11-27T15:27:30.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:29 smithi052 ceph-mon[99056]: pgmap v298: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-11-27T15:27:32.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:32 smithi150 ceph-mon[100559]: pgmap v299: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:27:32.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:32 smithi102 ceph-mon[100296]: pgmap v299: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:27:32.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:32 smithi052 ceph-mon[99056]: pgmap v299: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:27:32.893 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:27:33.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.033+0000 7f0897cdb700 1 -- 172.21.15.52:0/971863690 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08900ff6a0 msgr2=0x7f0890104ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.033+0000 7f0897cdb700 1 --2- 172.21.15.52:0/971863690 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08900ff6a0 0x7f0890104ef0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f08800099c0 tx=0x7f0880009cd0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.033+0000 7f0897cdb700 1 -- 172.21.15.52:0/971863690 shutdown_connections 2023-11-27T15:27:33.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.033+0000 7f0897cdb700 1 --2- 172.21.15.52:0/971863690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0890105620 0x7f089010d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.033+0000 7f0897cdb700 1 --2- 172.21.15.52:0/971863690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08900fed00 0x7f08900ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.033+0000 7f0897cdb700 1 --2- 172.21.15.52:0/971863690 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08900ff6a0 0x7f0890104ef0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.038 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.033+0000 7f0897cdb700 1 -- 172.21.15.52:0/971863690 >> 172.21.15.52:0/971863690 conn(0x7f08900748f0 msgr2=0x7f0890074cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:33.038 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.033+0000 7f0897cdb700 1 -- 172.21.15.52:0/971863690 shutdown_connections 2023-11-27T15:27:33.038 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.033+0000 7f0897cdb700 1 -- 172.21.15.52:0/971863690 wait complete. 2023-11-27T15:27:33.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.034+0000 7f0897cdb700 1 Processor -- start 2023-11-27T15:27:33.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.034+0000 7f0897cdb700 1 -- start start 2023-11-27T15:27:33.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.034+0000 7f0897cdb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08900fed00 0x7f08900ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.034+0000 7f0897cdb700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08900ff6a0 0x7f08901001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.034+0000 7f0897cdb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0890105620 0x7f08901a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.034+0000 7f0897cdb700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08901009e0 con 0x7f0890105620 2023-11-27T15:27:33.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.034+0000 7f0897cdb700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08901a4b80 con 0x7f08900ff6a0 2023-11-27T15:27:33.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.034+0000 7f0897cdb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08901a4cc0 con 0x7f08900fed00 2023-11-27T15:27:33.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.034+0000 7f0896278700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0890105620 0x7f08901a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.034+0000 7f0896278700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0890105620 0x7f08901a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55952/0 (socket says 172.21.15.52:55952) 2023-11-27T15:27:33.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.034+0000 7f0896278700 1 -- 172.21.15.52:0/2570457097 learned_addr learned my addr 172.21.15.52:0/2570457097 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:33.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.034+0000 7f0895276700 1 --2- 172.21.15.52:0/2570457097 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08900ff6a0 0x7f08901001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.034+0000 7f0895a77700 1 --2- 172.21.15.52:0/2570457097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08900fed00 0x7f08900ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.034+0000 7f0896278700 1 -- 172.21.15.52:0/2570457097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08900fed00 msgr2=0x7f08900ffc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.034+0000 7f0896278700 1 --2- 172.21.15.52:0/2570457097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08900fed00 0x7f08900ffc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.034+0000 7f0896278700 1 -- 172.21.15.52:0/2570457097 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08900ff6a0 msgr2=0x7f08901001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.034+0000 7f0896278700 1 --2- 172.21.15.52:0/2570457097 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08900ff6a0 0x7f08901001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.034+0000 7f0896278700 1 -- 172.21.15.52:0/2570457097 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08800095f0 con 0x7f0890105620 2023-11-27T15:27:33.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.035+0000 7f0896278700 1 --2- 172.21.15.52:0/2570457097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0890105620 0x7f08901a4640 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f0888007ee0 tx=0x7f088800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:33.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.035+0000 7f0886ffd700 1 -- 172.21.15.52:0/2570457097 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0888017070 con 0x7f0890105620 2023-11-27T15:27:33.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.035+0000 7f0897cdb700 1 -- 172.21.15.52:0/2570457097 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08901103a0 con 0x7f0890105620 2023-11-27T15:27:33.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.035+0000 7f0886ffd700 1 -- 172.21.15.52:0/2570457097 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f088800f040 con 0x7f0890105620 2023-11-27T15:27:33.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.035+0000 7f0886ffd700 1 -- 172.21.15.52:0/2570457097 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0888005580 con 0x7f0890105620 2023-11-27T15:27:33.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.035+0000 7f0897cdb700 1 -- 172.21.15.52:0/2570457097 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0890110210 con 0x7f0890105620 2023-11-27T15:27:33.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.037+0000 7f0886ffd700 1 -- 172.21.15.52:0/2570457097 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0888005780 con 0x7f0890105620 2023-11-27T15:27:33.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.037+0000 7f0886ffd700 1 --2- 172.21.15.52:0/2570457097 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f087c0a12c0 0x7f087c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.038+0000 7f0886ffd700 1 -- 172.21.15.52:0/2570457097 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f08880cc460 con 0x7f0890105620 2023-11-27T15:27:33.049 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.038+0000 7f0895a77700 1 --2- 172.21.15.52:0/2570457097 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f087c0a12c0 0x7f087c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.038+0000 7f0895a77700 1 --2- 172.21.15.52:0/2570457097 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f087c0a12c0 0x7f087c0a3770 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f088c005950 tx=0x7f088c00b410 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:33.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.038+0000 7f08a3e67b80 1 Processor -- start 2023-11-27T15:27:33.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.039+0000 7f08a3e67b80 1 -- start start 2023-11-27T15:27:33.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.039+0000 7f08a3e67b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51b65bb60 0x55f51b65bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.039+0000 7f08a3e67b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f51b65c470 0x55f51b660370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.039+0000 7f08a3e67b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f51b6608b0 0x55f51b660d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.039+0000 7f08a3e67b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f51b561060 con 0x55f51b65c470 2023-11-27T15:27:33.050 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.039+0000 7f08a3e67b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f51b5b38a0 con 0x55f51b6608b0 2023-11-27T15:27:33.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.039+0000 7f08a3e67b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f51b5acd30 con 0x55f51b65bb60 2023-11-27T15:27:33.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.039+0000 7f0895276700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f51b65c470 0x55f51b660370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.039+0000 7f0895276700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f51b65c470 0x55f51b660370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55958/0 (socket says 172.21.15.52:55958) 2023-11-27T15:27:33.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.039+0000 7f0895276700 1 -- 172.21.15.52:0/281511560 learned_addr learned my addr 172.21.15.52:0/281511560 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:33.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.039+0000 7f0895a77700 1 --2- 172.21.15.52:0/281511560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51b65bb60 0x55f51b65bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.039+0000 7f0896278700 1 --2- 172.21.15.52:0/281511560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f51b6608b0 0x55f51b660d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.039+0000 7f0895276700 1 -- 172.21.15.52:0/281511560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51b65bb60 msgr2=0x55f51b65bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.051 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.039+0000 7f0895276700 1 --2- 172.21.15.52:0/281511560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51b65bb60 0x55f51b65bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.052 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.039+0000 7f0895276700 1 -- 172.21.15.52:0/281511560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f51b6608b0 msgr2=0x55f51b660d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.052 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.039+0000 7f0895276700 1 --2- 172.21.15.52:0/281511560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f51b6608b0 0x55f51b660d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.052 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.039+0000 7f0895276700 1 -- 172.21.15.52:0/281511560 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f51b42e630 con 0x55f51b65c470 2023-11-27T15:27:33.052 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.040+0000 7f0895276700 1 --2- 172.21.15.52:0/281511560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f51b65c470 0x55f51b660370 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f0880006010 tx=0x7f0880009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd10f84b75f9cb29 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:33.052 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.040+0000 7f087bfff700 1 -- 172.21.15.52:0/281511560 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088001d070 con 0x55f51b65c470 2023-11-27T15:27:33.052 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.040+0000 7f087bfff700 1 -- 172.21.15.52:0/281511560 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f088001e040 con 0x55f51b65c470 2023-11-27T15:27:33.052 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.040+0000 7f087bfff700 1 -- 172.21.15.52:0/281511560 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08800038c0 con 0x55f51b65c470 2023-11-27T15:27:33.052 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.041+0000 7f08a3e67b80 1 -- 172.21.15.52:0/281511560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f51b65c470 msgr2=0x55f51b660370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.053 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.041+0000 7f08a3e67b80 1 --2- 172.21.15.52:0/281511560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f51b65c470 0x55f51b660370 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f0880006010 tx=0x7f0880009570 comp rx=0 tx=0).stop 2023-11-27T15:27:33.053 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.041+0000 7f08a3e67b80 1 -- 172.21.15.52:0/281511560 shutdown_connections 2023-11-27T15:27:33.053 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.041+0000 7f08a3e67b80 1 --2- 172.21.15.52:0/281511560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f51b6608b0 0x55f51b660d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.053 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.041+0000 7f08a3e67b80 1 --2- 172.21.15.52:0/281511560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51b65bb60 0x55f51b65bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.053 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.041+0000 7f08a3e67b80 1 --2- 172.21.15.52:0/281511560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f51b65c470 0x55f51b660370 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.053 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.041+0000 7f08a3e67b80 1 -- 172.21.15.52:0/281511560 >> 172.21.15.52:0/281511560 conn(0x55f51b651360 msgr2=0x55f51b656b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:33.053 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.041+0000 7f08a3e67b80 1 -- 172.21.15.52:0/281511560 shutdown_connections 2023-11-27T15:27:33.053 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.041+0000 7f08a3e67b80 1 -- 172.21.15.52:0/281511560 wait complete. 2023-11-27T15:27:33.054 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.041+0000 7f08a3e67b80 1 -- start start 2023-11-27T15:27:33.054 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.041+0000 7f08a3e67b80 1 Processor -- start 2023-11-27T15:27:33.054 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.042+0000 7f08a3e67b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f51b65bb60 0x55f51b667fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.054 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.042+0000 7f08a3e67b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51b65c470 0x55f51b665240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.054 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.042+0000 7f08a3e67b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f51b6608b0 0x55f51b665870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.054 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.042+0000 7f08a3e67b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f51b54e8e0 con 0x55f51b65bb60 2023-11-27T15:27:33.054 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.042+0000 7f08a3e67b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f51b54e580 con 0x55f51b6608b0 2023-11-27T15:27:33.054 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.042+0000 7f08a3e67b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f51b430920 con 0x55f51b65c470 2023-11-27T15:27:33.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.042+0000 7f08a3e67b80 10 client.0 _ll_register_callbacks cb 0x55f51b56c780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:33.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.042+0000 7f0895a77700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f51b65bb60 0x55f51b667fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.042+0000 7f0895a77700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f51b65bb60 0x55f51b667fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55962/0 (socket says 172.21.15.52:55962) 2023-11-27T15:27:33.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.042+0000 7f0895a77700 1 -- 172.21.15.52:0/2565847902 learned_addr learned my addr 172.21.15.52:0/2565847902 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:33.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.042+0000 7f0895276700 1 --2- 172.21.15.52:0/2565847902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51b65c470 0x55f51b665240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.042+0000 7f0896278700 1 --2- 172.21.15.52:0/2565847902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f51b6608b0 0x55f51b665870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.042+0000 7f0895a77700 1 -- 172.21.15.52:0/2565847902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51b65c470 msgr2=0x55f51b665240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.055 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.042+0000 7f0895a77700 1 --2- 172.21.15.52:0/2565847902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51b65c470 0x55f51b665240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.042+0000 7f0895a77700 1 -- 172.21.15.52:0/2565847902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f51b6608b0 msgr2=0x55f51b665870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.042+0000 7f0895a77700 1 --2- 172.21.15.52:0/2565847902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f51b6608b0 0x55f51b665870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.042+0000 7f0895a77700 1 -- 172.21.15.52:0/2565847902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f089010d9e0 con 0x55f51b65bb60 2023-11-27T15:27:33.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.042+0000 7f0896278700 1 --2- 172.21.15.52:0/2565847902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f51b6608b0 0x55f51b665870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:33.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.042+0000 7f0895a77700 1 --2- 172.21.15.52:0/2565847902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f51b65bb60 0x55f51b667fc0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f088c0116f0 tx=0x7f088c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af7dc32e155861f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:33.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.042+0000 7f08797fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:27:33.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.042+0000 7f08797fa700 1 -- 172.21.15.52:0/2565847902 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088c028470 con 0x55f51b65bb60 2023-11-27T15:27:33.056 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.043+0000 7f08797fa700 1 -- 172.21.15.52:0/2565847902 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f088c017070 con 0x55f51b65bb60 2023-11-27T15:27:33.057 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.043+0000 7f08a3e67b80 1 -- 172.21.15.52:0/2565847902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f51b64eb20 con 0x55f51b65bb60 2023-11-27T15:27:33.057 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.043+0000 7f08797fa700 1 -- 172.21.15.52:0/2565847902 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f088c0285d0 con 0x55f51b65bb60 2023-11-27T15:27:33.057 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.043+0000 7f08a3e67b80 10 client.17079 fetch_fsmap learned FSMap version 34 2023-11-27T15:27:33.057 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.043+0000 7f08a3e67b80 1 -- 172.21.15.52:0/2565847902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f51b573190 con 0x55f51b65bb60 2023-11-27T15:27:33.057 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.043+0000 7f08797fa700 1 -- 172.21.15.52:0/2565847902 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f51b64eb20 con 0x55f51b65bb60 2023-11-27T15:27:33.057 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.043+0000 7f08797fa700 1 -- 172.21.15.52:0/2565847902 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f088c01f6d0 con 0x55f51b65bb60 2023-11-27T15:27:33.058 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.043+0000 7f08a3e67b80 10 client.17079 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:27:33.058 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.043+0000 7f08a3e67b80 10 client.17079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:33.058 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.043+0000 7f08a3e67b80 20 client.17079 populate_metadata read hostname 'smithi052' 2023-11-27T15:27:33.058 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.043+0000 7f08a3e67b80 1 --2- 172.21.15.52:0/2565847902 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55f51b675800 0x55f51b677c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.058 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.043+0000 7f08a3e67b80 4 client.17079 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:27:33.058 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.043+0000 7f08a3e67b80 1 -- 172.21.15.52:0/2565847902 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f51b56f0a0 con 0x55f51b675800 2023-11-27T15:27:33.058 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.043+0000 7f0895276700 1 --2- 172.21.15.52:0/2565847902 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55f51b675800 0x55f51b677c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.058 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.044+0000 7f0895276700 1 --2- 172.21.15.52:0/2565847902 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55f51b675800 0x55f51b677c10 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5803d7d968bc624 server_cookie=7623582c3c5803ca in_seq=0 out_seq=0 2023-11-27T15:27:33.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.044+0000 7f08797fa700 10 client.17079 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:33.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.044+0000 7f08797fa700 1 -- 172.21.15.52:0/2565847902 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f51b56f0a0 con 0x55f51b675800 2023-11-27T15:27:33.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.044+0000 7f08797fa700 10 client.17079 handle_command_reply: tid=0 2023-11-27T15:27:33.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.044+0000 7f08a3e67b80 1 client.17079 shutdown 2023-11-27T15:27:33.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.045+0000 7f08a3e67b80 1 -- 172.21.15.52:0/2565847902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f51b65bb60 msgr2=0x55f51b667fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.045+0000 7f08a3e67b80 1 --2- 172.21.15.52:0/2565847902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f51b65bb60 0x55f51b667fc0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f088c0116f0 tx=0x7f088c0047c0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.045+0000 7f08a3e67b80 1 -- 172.21.15.52:0/2565847902 shutdown_connections 2023-11-27T15:27:33.059 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.045+0000 7f08a3e67b80 1 --2- 172.21.15.52:0/2565847902 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55f51b675800 0x55f51b677c10 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.060 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.045+0000 7f08a3e67b80 1 --2- 172.21.15.52:0/2565847902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f51b6608b0 0x55f51b665870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.060 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.045+0000 7f08a3e67b80 1 --2- 172.21.15.52:0/2565847902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f51b65bb60 0x55f51b667fc0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.060 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.045+0000 7f08a3e67b80 1 --2- 172.21.15.52:0/2565847902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51b65c470 0x55f51b665240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.060 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.045+0000 7f08797fa700 0 client.17079 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:33.060 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.045+0000 7f08a3e67b80 1 -- 172.21.15.52:0/2565847902 >> 172.21.15.52:0/2565847902 conn(0x55f51b651360 msgr2=0x55f51b661d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:33.060 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.045+0000 7f08a3e67b80 1 -- 172.21.15.52:0/2565847902 shutdown_connections 2023-11-27T15:27:33.060 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.045+0000 7f08a3e67b80 1 -- 172.21.15.52:0/2565847902 wait complete. 2023-11-27T15:27:33.060 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.045+0000 7f08a3e67b80 20 client.17079 trim_cache size 0 max 16384 2023-11-27T15:27:33.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.062+0000 7f08a3e67b80 1 Processor -- start 2023-11-27T15:27:33.070 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.062+0000 7f08a3e67b80 1 -- start start 2023-11-27T15:27:33.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.062+0000 7f08a3e67b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f51b65bb60 0x55f51b66c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.062+0000 7f08a3e67b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51b65c470 0x55f51b671b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.062+0000 7f08a3e67b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f51b6608b0 0x55f51b667b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.062+0000 7f08a3e67b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f51b6581c0 con 0x55f51b6608b0 2023-11-27T15:27:33.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.062+0000 7f08a3e67b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f51b65cbc0 con 0x55f51b65bb60 2023-11-27T15:27:33.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.062+0000 7f08a3e67b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f51b672f10 con 0x55f51b65c470 2023-11-27T15:27:33.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.062+0000 7f0896278700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f51b6608b0 0x55f51b667b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.071 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.062+0000 7f0896278700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f51b6608b0 0x55f51b667b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55974/0 (socket says 172.21.15.52:55974) 2023-11-27T15:27:33.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.062+0000 7f0896278700 1 -- 172.21.15.52:0/3541882292 learned_addr learned my addr 172.21.15.52:0/3541882292 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:33.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.062+0000 7f0895276700 1 --2- 172.21.15.52:0/3541882292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51b65c470 0x55f51b671b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.062+0000 7f0895a77700 1 --2- 172.21.15.52:0/3541882292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f51b65bb60 0x55f51b66c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.062+0000 7f0896278700 1 -- 172.21.15.52:0/3541882292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51b65c470 msgr2=0x55f51b671b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.062+0000 7f0896278700 1 --2- 172.21.15.52:0/3541882292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51b65c470 0x55f51b671b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.062+0000 7f0896278700 1 -- 172.21.15.52:0/3541882292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f51b65bb60 msgr2=0x55f51b66c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.062+0000 7f0896278700 1 --2- 172.21.15.52:0/3541882292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f51b65bb60 0x55f51b66c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.072 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.062+0000 7f0896278700 1 -- 172.21.15.52:0/3541882292 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f51b572ee0 con 0x55f51b6608b0 2023-11-27T15:27:33.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.062+0000 7f0895a77700 1 --2- 172.21.15.52:0/3541882292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f51b65bb60 0x55f51b66c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:33.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.062+0000 7f0896278700 1 --2- 172.21.15.52:0/3541882292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f51b6608b0 0x55f51b667b60 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f08880c9c30 tx=0x7f0888084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb43a3123e3ff3c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:33.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.063+0000 7f0884ff9700 1 -- 172.21.15.52:0/3541882292 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0888099500 con 0x55f51b6608b0 2023-11-27T15:27:33.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.063+0000 7f0884ff9700 1 -- 172.21.15.52:0/3541882292 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f088809d530 con 0x55f51b6608b0 2023-11-27T15:27:33.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.063+0000 7f0884ff9700 1 -- 172.21.15.52:0/3541882292 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0888095c10 con 0x55f51b6608b0 2023-11-27T15:27:33.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.063+0000 7f08a3e67b80 1 -- 172.21.15.52:0/3541882292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f51b6608b0 msgr2=0x55f51b667b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.073 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.063+0000 7f08a3e67b80 1 --2- 172.21.15.52:0/3541882292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f51b6608b0 0x55f51b667b60 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f08880c9c30 tx=0x7f0888084040 comp rx=0 tx=0).stop 2023-11-27T15:27:33.074 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.063+0000 7f08a3e67b80 1 -- 172.21.15.52:0/3541882292 shutdown_connections 2023-11-27T15:27:33.074 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.063+0000 7f08a3e67b80 1 --2- 172.21.15.52:0/3541882292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f51b65bb60 0x55f51b66c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.074 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.063+0000 7f08a3e67b80 1 --2- 172.21.15.52:0/3541882292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f51b6608b0 0x55f51b667b60 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.074 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.063+0000 7f08a3e67b80 1 --2- 172.21.15.52:0/3541882292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51b65c470 0x55f51b671b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.074 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.063+0000 7f08a3e67b80 1 -- 172.21.15.52:0/3541882292 >> 172.21.15.52:0/3541882292 conn(0x55f51b651360 msgr2=0x55f51b655550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:33.074 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.063+0000 7f08a3e67b80 1 -- 172.21.15.52:0/3541882292 shutdown_connections 2023-11-27T15:27:33.074 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.063+0000 7f08a3e67b80 1 -- 172.21.15.52:0/3541882292 wait complete. 2023-11-27T15:27:33.074 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.064+0000 7f08a3e67b80 1 -- start start 2023-11-27T15:27:33.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.064+0000 7f08a3e67b80 1 Processor -- start 2023-11-27T15:27:33.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.064+0000 7f08a3e67b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f51b65bb60 0x55f51b6c1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.064+0000 7f08a3e67b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f51b65c470 0x55f51b6c1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.064+0000 7f08a3e67b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51b6608b0 0x55f51b668920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.064+0000 7f08a3e67b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f51b669030 con 0x55f51b65bb60 2023-11-27T15:27:33.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.064+0000 7f08a3e67b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f51b6691a0 con 0x55f51b65c470 2023-11-27T15:27:33.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.064+0000 7f08a3e67b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f51b669360 con 0x55f51b6608b0 2023-11-27T15:27:33.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.064+0000 7f08a3e67b80 10 client.0 _ll_register_callbacks cb 0x7f088c010e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:33.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.064+0000 7f0895a77700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f51b65bb60 0x55f51b6c1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.064+0000 7f0895a77700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f51b65bb60 0x55f51b6c1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55976/0 (socket says 172.21.15.52:55976) 2023-11-27T15:27:33.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.064+0000 7f0895a77700 1 -- 172.21.15.52:0/2387753168 learned_addr learned my addr 172.21.15.52:0/2387753168 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:33.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.064+0000 7f0896278700 1 --2- 172.21.15.52:0/2387753168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51b6608b0 0x55f51b668920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.064+0000 7f0895276700 1 --2- 172.21.15.52:0/2387753168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f51b65c470 0x55f51b6c1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.064+0000 7f0895a77700 1 -- 172.21.15.52:0/2387753168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51b6608b0 msgr2=0x55f51b668920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.076 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.064+0000 7f0895a77700 1 --2- 172.21.15.52:0/2387753168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51b6608b0 0x55f51b668920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.064+0000 7f0895a77700 1 -- 172.21.15.52:0/2387753168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f51b65c470 msgr2=0x55f51b6c1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.064+0000 7f0895a77700 1 --2- 172.21.15.52:0/2387753168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f51b65c470 0x55f51b6c1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.064+0000 7f0895a77700 1 -- 172.21.15.52:0/2387753168 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0890110210 con 0x55f51b65bb60 2023-11-27T15:27:33.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.064+0000 7f0895276700 1 --2- 172.21.15.52:0/2387753168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f51b65c470 0x55f51b6c1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:33.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.065+0000 7f0895a77700 1 --2- 172.21.15.52:0/2387753168 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f51b65bb60 0x55f51b6c1980 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f088c0116c0 tx=0x7f088c01f940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41006f9083387a8b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:33.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.065+0000 7f08797fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:27:33.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.065+0000 7f08797fa700 1 -- 172.21.15.52:0/2387753168 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088c028880 con 0x55f51b65bb60 2023-11-27T15:27:33.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.065+0000 7f08797fa700 1 -- 172.21.15.52:0/2387753168 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f088c017040 con 0x55f51b65bb60 2023-11-27T15:27:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.065+0000 7f08a3e67b80 1 -- 172.21.15.52:0/2387753168 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f088001e040 con 0x55f51b65bb60 2023-11-27T15:27:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.065+0000 7f08797fa700 1 -- 172.21.15.52:0/2387753168 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f088c010a40 con 0x55f51b65bb60 2023-11-27T15:27:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.065+0000 7f08a3e67b80 10 client.17085 fetch_fsmap learned FSMap version 34 2023-11-27T15:27:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.065+0000 7f08a3e67b80 1 -- 172.21.15.52:0/2387753168 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f51b3c3c60 con 0x55f51b65bb60 2023-11-27T15:27:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.065+0000 7f08797fa700 1 -- 172.21.15.52:0/2387753168 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f088001e040 con 0x55f51b65bb60 2023-11-27T15:27:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.065+0000 7f08797fa700 1 -- 172.21.15.52:0/2387753168 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f088c0315d0 con 0x55f51b65bb60 2023-11-27T15:27:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.065+0000 7f08a3e67b80 10 client.17085 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:27:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.065+0000 7f08a3e67b80 10 client.17085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.065+0000 7f08a3e67b80 20 client.17085 populate_metadata read hostname 'smithi052' 2023-11-27T15:27:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.065+0000 7f08a3e67b80 1 --2- 172.21.15.52:0/2387753168 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55f51b6ca8d0 0x55f51b6ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.065+0000 7f08a3e67b80 4 client.17085 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:27:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.065+0000 7f08a3e67b80 1 -- 172.21.15.52:0/2387753168 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f51b57b660 con 0x55f51b6ca8d0 2023-11-27T15:27:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.066+0000 7f0895276700 1 --2- 172.21.15.52:0/2387753168 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55f51b6ca8d0 0x55f51b6ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.066+0000 7f0895276700 1 --2- 172.21.15.52:0/2387753168 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55f51b6ca8d0 0x55f51b6ccce0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13be4557127be60d server_cookie=e6125c03e66821a4 in_seq=0 out_seq=0 2023-11-27T15:27:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.066+0000 7f08797fa700 10 client.17085 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.066+0000 7f08797fa700 1 -- 172.21.15.52:0/2387753168 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f51b57b660 con 0x55f51b6ca8d0 2023-11-27T15:27:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.066+0000 7f08797fa700 10 client.17085 handle_command_reply: tid=0 2023-11-27T15:27:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.066+0000 7f08a3e67b80 1 client.17085 shutdown 2023-11-27T15:27:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.066+0000 7f08a3e67b80 1 -- 172.21.15.52:0/2387753168 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f51b65bb60 msgr2=0x55f51b6c1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.066+0000 7f08a3e67b80 1 --2- 172.21.15.52:0/2387753168 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f51b65bb60 0x55f51b6c1980 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f088c0116c0 tx=0x7f088c01f940 comp rx=0 tx=0).stop 2023-11-27T15:27:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.066+0000 7f08a3e67b80 1 -- 172.21.15.52:0/2387753168 shutdown_connections 2023-11-27T15:27:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.066+0000 7f08a3e67b80 1 --2- 172.21.15.52:0/2387753168 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55f51b6ca8d0 0x55f51b6ccce0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.066+0000 7f08a3e67b80 1 --2- 172.21.15.52:0/2387753168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f51b6608b0 0x55f51b668920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.066+0000 7f08797fa700 0 client.17085 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:33.081 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.066+0000 7f08a3e67b80 1 --2- 172.21.15.52:0/2387753168 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f51b65bb60 0x55f51b6c1980 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.081 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.066+0000 7f08a3e67b80 1 --2- 172.21.15.52:0/2387753168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f51b65c470 0x55f51b6c1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.081 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.066+0000 7f08a3e67b80 1 -- 172.21.15.52:0/2387753168 >> 172.21.15.52:0/2387753168 conn(0x55f51b651360 msgr2=0x55f51b671b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:33.081 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.066+0000 7f08a3e67b80 1 -- 172.21.15.52:0/2387753168 shutdown_connections 2023-11-27T15:27:33.081 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.066+0000 7f08a3e67b80 1 -- 172.21.15.52:0/2387753168 wait complete. 2023-11-27T15:27:33.081 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.067+0000 7f08a3e67b80 20 client.17085 trim_cache size 0 max 16384 2023-11-27T15:27:33.082 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:27:33.082 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2023-11-27T15:27:33.082 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2023-11-27T15:27:33.082 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:27:33.083 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.067+0000 7f0897cdb700 1 -- 172.21.15.52:0/2570457097 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f087c0a12c0 msgr2=0x7f087c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.083 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.067+0000 7f0897cdb700 1 --2- 172.21.15.52:0/2570457097 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f087c0a12c0 0x7f087c0a3770 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f088c005950 tx=0x7f088c00b410 comp rx=0 tx=0).stop 2023-11-27T15:27:33.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.067+0000 7f0897cdb700 1 -- 172.21.15.52:0/2570457097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0890105620 msgr2=0x7f08901a4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.067+0000 7f0897cdb700 1 --2- 172.21.15.52:0/2570457097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0890105620 0x7f08901a4640 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f0888007ee0 tx=0x7f088800d3b0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.067+0000 7f0897cdb700 1 -- 172.21.15.52:0/2570457097 shutdown_connections 2023-11-27T15:27:33.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.067+0000 7f0897cdb700 1 --2- 172.21.15.52:0/2570457097 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f087c0a12c0 0x7f087c0a3770 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.067+0000 7f0897cdb700 1 --2- 172.21.15.52:0/2570457097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0890105620 0x7f08901a4640 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.067+0000 7f0897cdb700 1 --2- 172.21.15.52:0/2570457097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08900fed00 0x7f08900ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.084 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.067+0000 7f0897cdb700 1 --2- 172.21.15.52:0/2570457097 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f08900ff6a0 0x7f08901001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.067+0000 7f0897cdb700 1 -- 172.21.15.52:0/2570457097 >> 172.21.15.52:0/2570457097 conn(0x7f08900748f0 msgr2=0x7f08900fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:33.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.067+0000 7f0897cdb700 1 -- 172.21.15.52:0/2570457097 shutdown_connections 2023-11-27T15:27:33.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.067+0000 7f0897cdb700 1 -- 172.21.15.52:0/2570457097 wait complete. 2023-11-27T15:27:33.085 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-27T15:27:33.085 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:27:33.213 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.207+0000 7f99afe10700 1 -- 172.21.15.52:0/3390374715 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99a8100690 msgr2=0x7f99a8102d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.213 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.207+0000 7f99afe10700 1 --2- 172.21.15.52:0/3390374715 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99a8100690 0x7f99a8102d60 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f999c0099c0 tx=0x7f999c009cd0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.213 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.208+0000 7f99afe10700 1 -- 172.21.15.52:0/3390374715 shutdown_connections 2023-11-27T15:27:33.213 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.208+0000 7f99afe10700 1 --2- 172.21.15.52:0/3390374715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99a80ffcf0 0x7f99a81000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.213 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.208+0000 7f99afe10700 1 --2- 172.21.15.52:0/3390374715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99a8103490 0x7f99a810c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.214 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.208+0000 7f99afe10700 1 --2- 172.21.15.52:0/3390374715 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99a8100690 0x7f99a8102d60 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.214 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.208+0000 7f99afe10700 1 -- 172.21.15.52:0/3390374715 >> 172.21.15.52:0/3390374715 conn(0x7f99a80763a0 msgr2=0x7f99a80767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:33.214 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.208+0000 7f99afe10700 1 -- 172.21.15.52:0/3390374715 shutdown_connections 2023-11-27T15:27:33.214 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.208+0000 7f99afe10700 1 -- 172.21.15.52:0/3390374715 wait complete. 2023-11-27T15:27:33.214 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.208+0000 7f99afe10700 1 Processor -- start 2023-11-27T15:27:33.214 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.208+0000 7f99afe10700 1 -- start start 2023-11-27T15:27:33.214 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.208+0000 7f99afe10700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99a80ffcf0 0x7f99a8195320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.214 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.208+0000 7f99afe10700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99a8100690 0x7f99a8195860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.215 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.208+0000 7f99afe10700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99a8103490 0x7f99a8199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.215 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.208+0000 7f99afe10700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a819a300 con 0x7f99a8100690 2023-11-27T15:27:33.215 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.208+0000 7f99afe10700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a819a470 con 0x7f99a80ffcf0 2023-11-27T15:27:33.215 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.208+0000 7f99afe10700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a819a630 con 0x7f99a8103490 2023-11-27T15:27:33.215 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.209+0000 7f99ad3ab700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99a8100690 0x7f99a8195860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.215 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.209+0000 7f99adbac700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99a80ffcf0 0x7f99a8195320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.215 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.209+0000 7f99ad3ab700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99a8100690 0x7f99a8195860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55992/0 (socket says 172.21.15.52:55992) 2023-11-27T15:27:33.215 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.209+0000 7f99adbac700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99a80ffcf0 0x7f99a8195320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:45648/0 (socket says 172.21.15.52:45648) 2023-11-27T15:27:33.216 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.209+0000 7f99ad3ab700 1 -- 172.21.15.52:0/1288987755 learned_addr learned my addr 172.21.15.52:0/1288987755 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:33.216 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.209+0000 7f99ae3ad700 1 --2- 172.21.15.52:0/1288987755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99a8103490 0x7f99a8199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.216 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.209+0000 7f99ad3ab700 1 -- 172.21.15.52:0/1288987755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99a8103490 msgr2=0x7f99a8199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.216 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.209+0000 7f99ad3ab700 1 --2- 172.21.15.52:0/1288987755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99a8103490 0x7f99a8199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.216 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.209+0000 7f99ad3ab700 1 -- 172.21.15.52:0/1288987755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99a80ffcf0 msgr2=0x7f99a8195320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.216 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.209+0000 7f99ad3ab700 1 --2- 172.21.15.52:0/1288987755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99a80ffcf0 0x7f99a8195320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.216 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.209+0000 7f99ad3ab700 1 -- 172.21.15.52:0/1288987755 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f999c0095f0 con 0x7f99a8100690 2023-11-27T15:27:33.216 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.209+0000 7f99ad3ab700 1 --2- 172.21.15.52:0/1288987755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99a8100690 0x7f99a8195860 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f999c009fd0 tx=0x7f999c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:33.217 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.209+0000 7f999affd700 1 -- 172.21.15.52:0/1288987755 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c01d070 con 0x7f99a8100690 2023-11-27T15:27:33.217 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.209+0000 7f999affd700 1 -- 172.21.15.52:0/1288987755 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f999c005020 con 0x7f99a8100690 2023-11-27T15:27:33.217 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.209+0000 7f999affd700 1 -- 172.21.15.52:0/1288987755 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c004aa0 con 0x7f99a8100690 2023-11-27T15:27:33.217 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.209+0000 7f99afe10700 1 -- 172.21.15.52:0/1288987755 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a810f280 con 0x7f99a8100690 2023-11-27T15:27:33.217 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.209+0000 7f99afe10700 1 -- 172.21.15.52:0/1288987755 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a810f0f0 con 0x7f99a8100690 2023-11-27T15:27:33.218 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.210+0000 7f99afe10700 1 -- 172.21.15.52:0/1288987755 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a810f410 con 0x7f99a8100690 2023-11-27T15:27:33.218 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.211+0000 7f999affd700 1 -- 172.21.15.52:0/1288987755 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f999c0054b0 con 0x7f99a8100690 2023-11-27T15:27:33.218 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.212+0000 7f999affd700 1 --2- 172.21.15.52:0/1288987755 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f99940a1340 0x7f99940a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.218 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.212+0000 7f99adbac700 1 --2- 172.21.15.52:0/1288987755 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f99940a1340 0x7f99940a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.218 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.212+0000 7f999affd700 1 -- 172.21.15.52:0/1288987755 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f999c0cd830 con 0x7f99a8100690 2023-11-27T15:27:33.219 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.212+0000 7f99adbac700 1 --2- 172.21.15.52:0/1288987755 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f99940a1340 0x7f99940a37f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f99a4005950 tx=0x7f99a400b410 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:33.219 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.216+0000 7f999affd700 1 -- 172.21.15.52:0/1288987755 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f999c096c40 con 0x7f99a8100690 2023-11-27T15:27:33.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.460+0000 7f99afe10700 1 -- 172.21.15.52:0/1288987755 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99a80671d0 con 0x7f99a8100690 2023-11-27T15:27:33.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.461+0000 7f999affd700 1 -- 172.21.15.52:0/1288987755 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10804 (secure 0 0 0) 0x7f999c09a260 con 0x7f99a8100690 2023-11-27T15:27:33.464 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:27:33.464 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2328881376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2328881376},{"type":"v1","addr":"172.21.15.52:6839","nonce":2328881376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2799551570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2799551570},{"type":"v1","addr":"172.21.15.52:6841","nonce":2799551570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:16:09.319630+0000","modified":"2023-11-27T15:26:15.243744+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24494,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/215725838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":215725838},{"type":"v1","addr":"172.21.15.150:6839","nonce":215725838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3571830559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3571830559},{"type":"v1","addr":"172.21.15.102:6839","nonce":3571830559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:27:33.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.463+0000 7f99afe10700 1 -- 172.21.15.52:0/1288987755 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f99940a1340 msgr2=0x7f99940a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.463+0000 7f99afe10700 1 --2- 172.21.15.52:0/1288987755 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f99940a1340 0x7f99940a37f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f99a4005950 tx=0x7f99a400b410 comp rx=0 tx=0).stop 2023-11-27T15:27:33.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.463+0000 7f99afe10700 1 -- 172.21.15.52:0/1288987755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99a8100690 msgr2=0x7f99a8195860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.463+0000 7f99afe10700 1 --2- 172.21.15.52:0/1288987755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99a8100690 0x7f99a8195860 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f999c009fd0 tx=0x7f999c009580 comp rx=0 tx=0).stop 2023-11-27T15:27:33.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.463+0000 7f99afe10700 1 -- 172.21.15.52:0/1288987755 shutdown_connections 2023-11-27T15:27:33.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.463+0000 7f99afe10700 1 --2- 172.21.15.52:0/1288987755 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f99940a1340 0x7f99940a37f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.463+0000 7f99afe10700 1 --2- 172.21.15.52:0/1288987755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99a80ffcf0 0x7f99a8195320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.463+0000 7f99afe10700 1 --2- 172.21.15.52:0/1288987755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99a8103490 0x7f99a8199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.463+0000 7f99afe10700 1 --2- 172.21.15.52:0/1288987755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99a8100690 0x7f99a8195860 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.463+0000 7f99afe10700 1 -- 172.21.15.52:0/1288987755 >> 172.21.15.52:0/1288987755 conn(0x7f99a80763a0 msgr2=0x7f99a80fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:33.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.463+0000 7f99afe10700 1 -- 172.21.15.52:0/1288987755 shutdown_connections 2023-11-27T15:27:33.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.463+0000 7f99afe10700 1 -- 172.21.15.52:0/1288987755 wait complete. 2023-11-27T15:27:33.468 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 34 2023-11-27T15:27:33.478 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-27T15:27:33.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.602+0000 7f7a4d50a700 1 -- 172.21.15.52:0/1060132577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a48105210 msgr2=0x7f7a481055e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.602+0000 7f7a4d50a700 1 --2- 172.21.15.52:0/1060132577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a48105210 0x7f7a481055e0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f7a38009990 tx=0x7f7a38009ca0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.602+0000 7f7a4d50a700 1 -- 172.21.15.52:0/1060132577 shutdown_connections 2023-11-27T15:27:33.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.602+0000 7f7a4d50a700 1 --2- 172.21.15.52:0/1060132577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a48100530 0x7f7a4810c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.602+0000 7f7a4d50a700 1 --2- 172.21.15.52:0/1060132577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a48105210 0x7f7a481055e0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.606 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.602+0000 7f7a4d50a700 1 --2- 172.21.15.52:0/1060132577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a480ffba0 0x7f7a480ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.607 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.602+0000 7f7a4d50a700 1 -- 172.21.15.52:0/1060132577 >> 172.21.15.52:0/1060132577 conn(0x7f7a480748f0 msgr2=0x7f7a48074cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:33.607 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.602+0000 7f7a4d50a700 1 -- 172.21.15.52:0/1060132577 shutdown_connections 2023-11-27T15:27:33.607 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.602+0000 7f7a4d50a700 1 -- 172.21.15.52:0/1060132577 wait complete. 2023-11-27T15:27:33.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.603+0000 7f7a4d50a700 1 Processor -- start 2023-11-27T15:27:33.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.603+0000 7f7a4d50a700 1 -- start start 2023-11-27T15:27:33.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.603+0000 7f7a4d50a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a480ffba0 0x7f7a48195360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.604+0000 7f7a4d50a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a48100530 0x7f7a481958a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.604+0000 7f7a4d50a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a48105210 0x7f7a48199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.604+0000 7f7a4d50a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4819a340 con 0x7f7a480ffba0 2023-11-27T15:27:33.609 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.604+0000 7f7a4d50a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4819a4b0 con 0x7f7a48105210 2023-11-27T15:27:33.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.604+0000 7f7a4d50a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4819a670 con 0x7f7a48100530 2023-11-27T15:27:33.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.604+0000 7f7a46ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a480ffba0 0x7f7a48195360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.604+0000 7f7a46ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a480ffba0 0x7f7a48195360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56002/0 (socket says 172.21.15.52:56002) 2023-11-27T15:27:33.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.604+0000 7f7a46ffd700 1 -- 172.21.15.52:0/3560378751 learned_addr learned my addr 172.21.15.52:0/3560378751 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:33.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.604+0000 7f7a477fe700 1 --2- 172.21.15.52:0/3560378751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a48105210 0x7f7a48199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.604+0000 7f7a467fc700 1 --2- 172.21.15.52:0/3560378751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a48100530 0x7f7a481958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.604+0000 7f7a46ffd700 1 -- 172.21.15.52:0/3560378751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a48100530 msgr2=0x7f7a481958a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.610 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.604+0000 7f7a46ffd700 1 --2- 172.21.15.52:0/3560378751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a48100530 0x7f7a481958a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.604+0000 7f7a46ffd700 1 -- 172.21.15.52:0/3560378751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a48105210 msgr2=0x7f7a48199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.604+0000 7f7a46ffd700 1 --2- 172.21.15.52:0/3560378751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a48105210 0x7f7a48199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.604+0000 7f7a46ffd700 1 -- 172.21.15.52:0/3560378751 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a380095f0 con 0x7f7a480ffba0 2023-11-27T15:27:33.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.604+0000 7f7a46ffd700 1 --2- 172.21.15.52:0/3560378751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a480ffba0 0x7f7a48195360 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f7a38005910 tx=0x7f7a3800b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:33.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.604+0000 7f7a3ffff700 1 -- 172.21.15.52:0/3560378751 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3801d070 con 0x7f7a480ffba0 2023-11-27T15:27:33.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.604+0000 7f7a3ffff700 1 -- 172.21.15.52:0/3560378751 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a3800fb80 con 0x7f7a480ffba0 2023-11-27T15:27:33.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.604+0000 7f7a3ffff700 1 -- 172.21.15.52:0/3560378751 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a38017650 con 0x7f7a480ffba0 2023-11-27T15:27:33.611 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.604+0000 7f7a4d50a700 1 -- 172.21.15.52:0/3560378751 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a4810f250 con 0x7f7a480ffba0 2023-11-27T15:27:33.612 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.604+0000 7f7a4d50a700 1 -- 172.21.15.52:0/3560378751 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a4810f0c0 con 0x7f7a480ffba0 2023-11-27T15:27:33.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.607+0000 7f7a3ffff700 1 -- 172.21.15.52:0/3560378751 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7a38022470 con 0x7f7a480ffba0 2023-11-27T15:27:33.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.607+0000 7f7a3ffff700 1 --2- 172.21.15.52:0/3560378751 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f7a340a1340 0x7f7a340a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.617 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.607+0000 7f7a3ffff700 1 -- 172.21.15.52:0/3560378751 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f7a380ce3e0 con 0x7f7a480ffba0 2023-11-27T15:27:33.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.607+0000 7f7a467fc700 1 --2- 172.21.15.52:0/3560378751 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f7a340a1340 0x7f7a340a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.608+0000 7f7a467fc700 1 --2- 172.21.15.52:0/3560378751 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f7a340a1340 0x7f7a340a37f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7a48196970 tx=0x7f7a300094f0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:33.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.608+0000 7f7a59696b80 1 Processor -- start 2023-11-27T15:27:33.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.608+0000 7f7a59696b80 1 -- start start 2023-11-27T15:27:33.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.609+0000 7f7a59696b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55994d540d70 0x55994d541140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.609+0000 7f7a59696b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55994d541680 0x55994d545580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.609+0000 7f7a59696b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55994d545ac0 0x55994d545f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.618 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.609+0000 7f7a59696b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55994d444060 con 0x55994d545ac0 2023-11-27T15:27:33.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.609+0000 7f7a59696b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55994d4968a0 con 0x55994d541680 2023-11-27T15:27:33.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.609+0000 7f7a59696b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55994d48fd30 con 0x55994d540d70 2023-11-27T15:27:33.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.609+0000 7f7a477fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55994d545ac0 0x55994d545f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.609+0000 7f7a477fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55994d545ac0 0x55994d545f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56014/0 (socket says 172.21.15.52:56014) 2023-11-27T15:27:33.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.609+0000 7f7a477fe700 1 -- 172.21.15.52:0/3219903455 learned_addr learned my addr 172.21.15.52:0/3219903455 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:33.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.609+0000 7f7a467fc700 1 --2- 172.21.15.52:0/3219903455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55994d541680 0x55994d545580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.609+0000 7f7a46ffd700 1 --2- 172.21.15.52:0/3219903455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55994d540d70 0x55994d541140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.619 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.609+0000 7f7a477fe700 1 -- 172.21.15.52:0/3219903455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55994d540d70 msgr2=0x55994d541140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.609+0000 7f7a477fe700 1 --2- 172.21.15.52:0/3219903455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55994d540d70 0x55994d541140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.609+0000 7f7a477fe700 1 -- 172.21.15.52:0/3219903455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55994d541680 msgr2=0x55994d545580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.609+0000 7f7a477fe700 1 --2- 172.21.15.52:0/3219903455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55994d541680 0x55994d545580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.609+0000 7f7a477fe700 1 -- 172.21.15.52:0/3219903455 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55994d311630 con 0x55994d545ac0 2023-11-27T15:27:33.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.609+0000 7f7a477fe700 1 --2- 172.21.15.52:0/3219903455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55994d545ac0 0x55994d545f10 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f7a40009440 tx=0x7f7a40009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30068c6cbbc0d16c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:33.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.609+0000 7f7a3d7fa700 1 -- 172.21.15.52:0/3219903455 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4000e070 con 0x55994d545ac0 2023-11-27T15:27:33.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.609+0000 7f7a3d7fa700 1 -- 172.21.15.52:0/3219903455 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a4000f040 con 0x55994d545ac0 2023-11-27T15:27:33.620 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.610+0000 7f7a3d7fa700 1 -- 172.21.15.52:0/3219903455 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a40015ac0 con 0x55994d545ac0 2023-11-27T15:27:33.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.610+0000 7f7a59696b80 1 -- 172.21.15.52:0/3219903455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55994d545ac0 msgr2=0x55994d545f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.610+0000 7f7a59696b80 1 --2- 172.21.15.52:0/3219903455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55994d545ac0 0x55994d545f10 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f7a40009440 tx=0x7f7a40009920 comp rx=0 tx=0).stop 2023-11-27T15:27:33.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.610+0000 7f7a59696b80 1 -- 172.21.15.52:0/3219903455 shutdown_connections 2023-11-27T15:27:33.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.610+0000 7f7a59696b80 1 --2- 172.21.15.52:0/3219903455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55994d545ac0 0x55994d545f10 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.610+0000 7f7a59696b80 1 --2- 172.21.15.52:0/3219903455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55994d540d70 0x55994d541140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.610+0000 7f7a59696b80 1 --2- 172.21.15.52:0/3219903455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55994d541680 0x55994d545580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.610+0000 7f7a59696b80 1 -- 172.21.15.52:0/3219903455 >> 172.21.15.52:0/3219903455 conn(0x55994d534360 msgr2=0x55994d539b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:33.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.610+0000 7f7a59696b80 1 -- 172.21.15.52:0/3219903455 shutdown_connections 2023-11-27T15:27:33.621 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.610+0000 7f7a59696b80 1 -- 172.21.15.52:0/3219903455 wait complete. 2023-11-27T15:27:33.622 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.611+0000 7f7a59696b80 1 -- start start 2023-11-27T15:27:33.622 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.611+0000 7f7a59696b80 1 Processor -- start 2023-11-27T15:27:33.622 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.611+0000 7f7a59696b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55994d540d70 0x55994d54d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.622 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.611+0000 7f7a59696b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55994d541680 0x55994d54a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.622 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.611+0000 7f7a59696b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55994d545ac0 0x55994d54abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.622 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.611+0000 7f7a59696b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55994d4318e0 con 0x55994d541680 2023-11-27T15:27:33.622 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.611+0000 7f7a59696b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55994d431580 con 0x55994d540d70 2023-11-27T15:27:33.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.611+0000 7f7a59696b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55994d313920 con 0x55994d545ac0 2023-11-27T15:27:33.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.611+0000 7f7a59696b80 10 client.0 _ll_register_callbacks cb 0x55994d44f780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:33.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.611+0000 7f7a467fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55994d541680 0x55994d54a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.611+0000 7f7a467fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55994d541680 0x55994d54a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56018/0 (socket says 172.21.15.52:56018) 2023-11-27T15:27:33.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.611+0000 7f7a46ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55994d540d70 0x55994d54d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.611+0000 7f7a467fc700 1 -- 172.21.15.52:0/468357250 learned_addr learned my addr 172.21.15.52:0/468357250 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:33.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.611+0000 7f7a477fe700 1 --2- 172.21.15.52:0/468357250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55994d545ac0 0x55994d54abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.611+0000 7f7a46ffd700 1 -- 172.21.15.52:0/468357250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55994d545ac0 msgr2=0x55994d54abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.611+0000 7f7a46ffd700 1 --2- 172.21.15.52:0/468357250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55994d545ac0 0x55994d54abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.611+0000 7f7a46ffd700 1 -- 172.21.15.52:0/468357250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55994d541680 msgr2=0x55994d54a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.611+0000 7f7a46ffd700 1 --2- 172.21.15.52:0/468357250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55994d541680 0x55994d54a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.611+0000 7f7a46ffd700 1 -- 172.21.15.52:0/468357250 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a400090f0 con 0x55994d540d70 2023-11-27T15:27:33.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.612+0000 7f7a46ffd700 1 --2- 172.21.15.52:0/468357250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55994d540d70 0x55994d54d290 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7a380cbb00 tx=0x7f7a3809af60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcf33ff6180e6fc3 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:33.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.612+0000 7f7a2e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2023-11-27T15:27:33.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.612+0000 7f7a2e7fc700 1 -- 172.21.15.52:0/468357250 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3809b970 con 0x55994d540d70 2023-11-27T15:27:33.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.612+0000 7f7a59696b80 1 -- 172.21.15.52:0/468357250 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55994d531b20 con 0x55994d540d70 2023-11-27T15:27:33.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.612+0000 7f7a2e7fc700 1 -- 172.21.15.52:0/468357250 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f7a380a38d0 con 0x55994d540d70 2023-11-27T15:27:33.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.612+0000 7f7a2e7fc700 1 -- 172.21.15.52:0/468357250 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a3809aa00 con 0x55994d540d70 2023-11-27T15:27:33.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.612+0000 7f7a59696b80 10 client.26759 fetch_fsmap learned FSMap version 34 2023-11-27T15:27:33.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.612+0000 7f7a59696b80 1 -- 172.21.15.52:0/468357250 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55994d456190 con 0x55994d540d70 2023-11-27T15:27:33.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.612+0000 7f7a2e7fc700 1 -- 172.21.15.52:0/468357250 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55994d531b20 con 0x55994d540d70 2023-11-27T15:27:33.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.613+0000 7f7a2e7fc700 1 -- 172.21.15.52:0/468357250 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7a380ce070 con 0x55994d540d70 2023-11-27T15:27:33.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.613+0000 7f7a59696b80 10 client.26759 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:27:33.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.613+0000 7f7a59696b80 10 client.26759 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:27:33.625 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.613+0000 7f7a59696b80 20 client.26759 populate_metadata read hostname 'smithi052' 2023-11-27T15:27:33.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.613+0000 7f7a59696b80 1 --2- 172.21.15.52:0/468357250 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55994d55a8f0 0x55994d55cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.613+0000 7f7a59696b80 4 client.26759 mds_command: new command op to 24472 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:27:33.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.613+0000 7f7a59696b80 1 -- 172.21.15.52:0/468357250 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55994d4520a0 con 0x55994d55a8f0 2023-11-27T15:27:33.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.613+0000 7f7a467fc700 1 --2- 172.21.15.52:0/468357250 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55994d55a8f0 0x55994d55cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.613+0000 7f7a467fc700 1 --2- 172.21.15.52:0/468357250 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55994d55a8f0 0x55994d55cd00 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d97f09e8e0ce87f0 server_cookie=b55598c037eb7057 in_seq=0 out_seq=0 2023-11-27T15:27:33.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.614+0000 7f7a2e7fc700 10 client.26759 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:27:33.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.614+0000 7f7a2e7fc700 1 -- 172.21.15.52:0/468357250 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55994d4520a0 con 0x55994d55a8f0 2023-11-27T15:27:33.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.614+0000 7f7a2e7fc700 10 client.26759 handle_command_reply: tid=0 2023-11-27T15:27:33.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.614+0000 7f7a59696b80 1 client.26759 shutdown 2023-11-27T15:27:33.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.614+0000 7f7a59696b80 1 -- 172.21.15.52:0/468357250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55994d540d70 msgr2=0x55994d54d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.614+0000 7f7a59696b80 1 --2- 172.21.15.52:0/468357250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55994d540d70 0x55994d54d290 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7a380cbb00 tx=0x7f7a3809af60 comp rx=0 tx=0).stop 2023-11-27T15:27:33.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.614+0000 7f7a59696b80 1 -- 172.21.15.52:0/468357250 shutdown_connections 2023-11-27T15:27:33.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.614+0000 7f7a59696b80 1 --2- 172.21.15.52:0/468357250 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55994d55a8f0 0x55994d55cd00 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.614+0000 7f7a59696b80 1 --2- 172.21.15.52:0/468357250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55994d540d70 0x55994d54d290 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.614+0000 7f7a2e7fc700 0 client.26759 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:27:33.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.614+0000 7f7a59696b80 1 --2- 172.21.15.52:0/468357250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55994d545ac0 0x55994d54abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.614+0000 7f7a59696b80 1 --2- 172.21.15.52:0/468357250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55994d541680 0x55994d54a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.614+0000 7f7a59696b80 1 -- 172.21.15.52:0/468357250 >> 172.21.15.52:0/468357250 conn(0x55994d534360 msgr2=0x55994d538de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:33.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.614+0000 7f7a59696b80 1 -- 172.21.15.52:0/468357250 shutdown_connections 2023-11-27T15:27:33.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.614+0000 7f7a59696b80 1 -- 172.21.15.52:0/468357250 wait complete. 2023-11-27T15:27:33.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.615+0000 7f7a59696b80 20 client.26759 trim_cache size 0 max 16384 2023-11-27T15:27:33.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.627+0000 7f7a59696b80 1 Processor -- start 2023-11-27T15:27:33.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.627+0000 7f7a59696b80 1 -- start start 2023-11-27T15:27:33.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.628+0000 7f7a59696b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55994d540d70 0x55994d54cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.628+0000 7f7a59696b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55994d541680 0x55994d543ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.628+0000 7f7a59696b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55994d545ac0 0x55994d544000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.628+0000 7f7a59696b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55994d54b930 con 0x55994d540d70 2023-11-27T15:27:33.633 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.628+0000 7f7a59696b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55994d543480 con 0x55994d545ac0 2023-11-27T15:27:33.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.628+0000 7f7a59696b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55994d543640 con 0x55994d541680 2023-11-27T15:27:33.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.628+0000 7f7a46ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55994d540d70 0x55994d54cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.628+0000 7f7a46ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55994d540d70 0x55994d54cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56024/0 (socket says 172.21.15.52:56024) 2023-11-27T15:27:33.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.628+0000 7f7a46ffd700 1 -- 172.21.15.52:0/298341791 learned_addr learned my addr 172.21.15.52:0/298341791 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:33.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.628+0000 7f7a467fc700 1 --2- 172.21.15.52:0/298341791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55994d541680 0x55994d543ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.628+0000 7f7a477fe700 1 --2- 172.21.15.52:0/298341791 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55994d545ac0 0x55994d544000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.628+0000 7f7a46ffd700 1 -- 172.21.15.52:0/298341791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55994d541680 msgr2=0x55994d543ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.634 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.628+0000 7f7a46ffd700 1 --2- 172.21.15.52:0/298341791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55994d541680 0x55994d543ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.628+0000 7f7a46ffd700 1 -- 172.21.15.52:0/298341791 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55994d545ac0 msgr2=0x55994d544000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.628+0000 7f7a46ffd700 1 --2- 172.21.15.52:0/298341791 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55994d545ac0 0x55994d544000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.628+0000 7f7a46ffd700 1 -- 172.21.15.52:0/298341791 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55994d455ee0 con 0x55994d540d70 2023-11-27T15:27:33.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.628+0000 7f7a46ffd700 1 --2- 172.21.15.52:0/298341791 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55994d540d70 0x55994d54cdc0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f7a380a33a0 tx=0x7f7a3809f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743dc5dc361cc6b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:33.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.629+0000 7f7a3dffb700 1 -- 172.21.15.52:0/298341791 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a380a3640 con 0x55994d540d70 2023-11-27T15:27:33.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.629+0000 7f7a3dffb700 1 -- 172.21.15.52:0/298341791 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a380a38d0 con 0x55994d540d70 2023-11-27T15:27:33.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.629+0000 7f7a3dffb700 1 -- 172.21.15.52:0/298341791 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3809b9d0 con 0x55994d540d70 2023-11-27T15:27:33.635 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.629+0000 7f7a59696b80 1 -- 172.21.15.52:0/298341791 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55994d540d70 msgr2=0x55994d54cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.629+0000 7f7a59696b80 1 --2- 172.21.15.52:0/298341791 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55994d540d70 0x55994d54cdc0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f7a380a33a0 tx=0x7f7a3809f6c0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.629+0000 7f7a59696b80 1 -- 172.21.15.52:0/298341791 shutdown_connections 2023-11-27T15:27:33.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.629+0000 7f7a59696b80 1 --2- 172.21.15.52:0/298341791 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55994d545ac0 0x55994d544000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.629+0000 7f7a59696b80 1 --2- 172.21.15.52:0/298341791 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55994d540d70 0x55994d54cdc0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.629+0000 7f7a59696b80 1 --2- 172.21.15.52:0/298341791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55994d541680 0x55994d543ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.629+0000 7f7a59696b80 1 -- 172.21.15.52:0/298341791 >> 172.21.15.52:0/298341791 conn(0x55994d534360 msgr2=0x55994d547ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:33.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.629+0000 7f7a59696b80 1 -- 172.21.15.52:0/298341791 shutdown_connections 2023-11-27T15:27:33.636 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.629+0000 7f7a59696b80 1 -- 172.21.15.52:0/298341791 wait complete. 2023-11-27T15:27:33.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.630+0000 7f7a59696b80 1 -- start start 2023-11-27T15:27:33.637 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.630+0000 7f7a59696b80 1 Processor -- start 2023-11-27T15:27:33.640 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.630+0000 7f7a59696b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55994d540d70 0x55994d53dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.640 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.630+0000 7f7a59696b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55994d541680 0x55994d53e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.641 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.630+0000 7f7a59696b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55994d545ac0 0x55994d53e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.641 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.630+0000 7f7a59696b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55994d5af850 con 0x55994d541680 2023-11-27T15:27:33.641 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.630+0000 7f7a59696b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55994d5af9c0 con 0x55994d540d70 2023-11-27T15:27:33.641 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.630+0000 7f7a59696b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55994d5afb80 con 0x55994d545ac0 2023-11-27T15:27:33.641 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.630+0000 7f7a59696b80 10 client.0 _ll_register_callbacks cb 0x7f7a3809b050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:33.641 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.630+0000 7f7a467fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55994d541680 0x55994d53e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.641 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.630+0000 7f7a467fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55994d541680 0x55994d53e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56032/0 (socket says 172.21.15.52:56032) 2023-11-27T15:27:33.641 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.630+0000 7f7a467fc700 1 -- 172.21.15.52:0/3343084524 learned_addr learned my addr 172.21.15.52:0/3343084524 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:33.642 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.630+0000 7f7a477fe700 1 --2- 172.21.15.52:0/3343084524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55994d545ac0 0x55994d53e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.642 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.630+0000 7f7a46ffd700 1 --2- 172.21.15.52:0/3343084524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55994d540d70 0x55994d53dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.642 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.630+0000 7f7a467fc700 1 -- 172.21.15.52:0/3343084524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55994d545ac0 msgr2=0x55994d53e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.642 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.630+0000 7f7a467fc700 1 --2- 172.21.15.52:0/3343084524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55994d545ac0 0x55994d53e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.642 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.630+0000 7f7a467fc700 1 -- 172.21.15.52:0/3343084524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55994d540d70 msgr2=0x55994d53dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.642 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.630+0000 7f7a467fc700 1 --2- 172.21.15.52:0/3343084524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55994d540d70 0x55994d53dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.642 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.630+0000 7f7a467fc700 1 -- 172.21.15.52:0/3343084524 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55994d456190 con 0x55994d541680 2023-11-27T15:27:33.642 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.630+0000 7f7a46ffd700 1 --2- 172.21.15.52:0/3343084524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55994d540d70 0x55994d53dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:33.643 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.631+0000 7f7a467fc700 1 --2- 172.21.15.52:0/3343084524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55994d541680 0x55994d53e000 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x55994d55d610 tx=0x7f7a300047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce7ddce61881ead server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:33.643 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.631+0000 7f7a2e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:27:33.643 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.631+0000 7f7a2e7fc700 1 -- 172.21.15.52:0/3343084524 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a30025050 con 0x55994d541680 2023-11-27T15:27:33.643 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.631+0000 7f7a2e7fc700 1 -- 172.21.15.52:0/3343084524 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f7a30018040 con 0x55994d541680 2023-11-27T15:27:33.643 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.631+0000 7f7a59696b80 1 -- 172.21.15.52:0/3343084524 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a4000f040 con 0x55994d541680 2023-11-27T15:27:33.643 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.631+0000 7f7a2e7fc700 1 -- 172.21.15.52:0/3343084524 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a30027e10 con 0x55994d541680 2023-11-27T15:27:33.643 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.631+0000 7f7a59696b80 10 client.17109 fetch_fsmap learned FSMap version 34 2023-11-27T15:27:33.643 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.631+0000 7f7a59696b80 1 -- 172.21.15.52:0/3343084524 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55994d2a6c60 con 0x55994d541680 2023-11-27T15:27:33.644 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.631+0000 7f7a2e7fc700 1 -- 172.21.15.52:0/3343084524 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a4000f040 con 0x55994d541680 2023-11-27T15:27:33.644 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.631+0000 7f7a2e7fc700 1 -- 172.21.15.52:0/3343084524 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7a30030500 con 0x55994d541680 2023-11-27T15:27:33.644 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.631+0000 7f7a59696b80 10 client.17109 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:27:33.644 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.631+0000 7f7a59696b80 10 client.17109 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:27:33.644 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.631+0000 7f7a59696b80 20 client.17109 populate_metadata read hostname 'smithi052' 2023-11-27T15:27:33.644 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.631+0000 7f7a59696b80 1 --2- 172.21.15.52:0/3343084524 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55994d55a8f0 0x55994d5b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.644 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.631+0000 7f7a59696b80 4 client.17109 mds_command: new command op to 24472 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:27:33.644 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.631+0000 7f7a59696b80 1 -- 172.21.15.52:0/3343084524 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55994d45e660 con 0x55994d55a8f0 2023-11-27T15:27:33.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.632+0000 7f7a46ffd700 1 --2- 172.21.15.52:0/3343084524 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55994d55a8f0 0x55994d5b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.632+0000 7f7a46ffd700 1 --2- 172.21.15.52:0/3343084524 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55994d55a8f0 0x55994d5b3370 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=738cafb46495e672 server_cookie=e5ae6c5ca5e265f8 in_seq=0 out_seq=0 2023-11-27T15:27:33.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.632+0000 7f7a2e7fc700 10 client.17109 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:27:33.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.632+0000 7f7a2e7fc700 1 -- 172.21.15.52:0/3343084524 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55994d45e660 con 0x55994d55a8f0 2023-11-27T15:27:33.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.632+0000 7f7a2e7fc700 10 client.17109 handle_command_reply: tid=0 2023-11-27T15:27:33.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.632+0000 7f7a59696b80 1 client.17109 shutdown 2023-11-27T15:27:33.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.632+0000 7f7a59696b80 1 -- 172.21.15.52:0/3343084524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55994d541680 msgr2=0x55994d53e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.645 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.632+0000 7f7a59696b80 1 --2- 172.21.15.52:0/3343084524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55994d541680 0x55994d53e000 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x55994d55d610 tx=0x7f7a300047c0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.632+0000 7f7a59696b80 1 -- 172.21.15.52:0/3343084524 shutdown_connections 2023-11-27T15:27:33.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.632+0000 7f7a59696b80 1 --2- 172.21.15.52:0/3343084524 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x55994d55a8f0 0x55994d5b3370 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.632+0000 7f7a59696b80 1 --2- 172.21.15.52:0/3343084524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55994d540d70 0x55994d53dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.632+0000 7f7a2e7fc700 0 client.17109 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:27:33.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.632+0000 7f7a59696b80 1 --2- 172.21.15.52:0/3343084524 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55994d545ac0 0x55994d53e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.632+0000 7f7a59696b80 1 --2- 172.21.15.52:0/3343084524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55994d541680 0x55994d53e000 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.632+0000 7f7a59696b80 1 -- 172.21.15.52:0/3343084524 >> 172.21.15.52:0/3343084524 conn(0x55994d534360 msgr2=0x55994d547ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:33.646 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.632+0000 7f7a59696b80 1 -- 172.21.15.52:0/3343084524 shutdown_connections 2023-11-27T15:27:33.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.633+0000 7f7a59696b80 1 -- 172.21.15.52:0/3343084524 wait complete. 2023-11-27T15:27:33.647 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.633+0000 7f7a59696b80 20 client.17109 trim_cache size 0 max 16384 2023-11-27T15:27:33.647 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:27:33.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.633+0000 7f7a4d50a700 1 -- 172.21.15.52:0/3560378751 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f7a340a1340 msgr2=0x7f7a340a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.648 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.633+0000 7f7a4d50a700 1 --2- 172.21.15.52:0/3560378751 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f7a340a1340 0x7f7a340a37f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7a48196970 tx=0x7f7a300094f0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.633+0000 7f7a4d50a700 1 -- 172.21.15.52:0/3560378751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a480ffba0 msgr2=0x7f7a48195360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.633+0000 7f7a4d50a700 1 --2- 172.21.15.52:0/3560378751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a480ffba0 0x7f7a48195360 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f7a38005910 tx=0x7f7a3800b610 comp rx=0 tx=0).stop 2023-11-27T15:27:33.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.633+0000 7f7a4d50a700 1 -- 172.21.15.52:0/3560378751 shutdown_connections 2023-11-27T15:27:33.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.633+0000 7f7a4d50a700 1 --2- 172.21.15.52:0/3560378751 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f7a340a1340 0x7f7a340a37f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.633+0000 7f7a4d50a700 1 --2- 172.21.15.52:0/3560378751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a48105210 0x7f7a48199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.633+0000 7f7a4d50a700 1 --2- 172.21.15.52:0/3560378751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a480ffba0 0x7f7a48195360 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.633+0000 7f7a4d50a700 1 --2- 172.21.15.52:0/3560378751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a48100530 0x7f7a481958a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.649 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.633+0000 7f7a4d50a700 1 -- 172.21.15.52:0/3560378751 >> 172.21.15.52:0/3560378751 conn(0x7f7a480748f0 msgr2=0x7f7a480fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:33.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.633+0000 7f7a4d50a700 1 -- 172.21.15.52:0/3560378751 shutdown_connections 2023-11-27T15:27:33.650 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.633+0000 7f7a4d50a700 1 -- 172.21.15.52:0/3560378751 wait complete. 2023-11-27T15:27:33.650 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-27T15:27:33.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.774+0000 7fde1eb00700 1 -- 172.21.15.52:0/32099772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde181055d0 msgr2=0x7fde1810d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.774+0000 7fde1eb00700 1 --2- 172.21.15.52:0/32099772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde181055d0 0x7fde1810d1e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fde1000b0d0 tx=0x7fde1000b3e0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.774+0000 7fde1eb00700 1 -- 172.21.15.52:0/32099772 shutdown_connections 2023-11-27T15:27:33.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.774+0000 7fde1eb00700 1 --2- 172.21.15.52:0/32099772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde181055d0 0x7fde1810d1e0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.774+0000 7fde1eb00700 1 --2- 172.21.15.52:0/32099772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde180feaf0 0x7fde180feec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.778 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.774+0000 7fde1eb00700 1 --2- 172.21.15.52:0/32099772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde180ff490 0x7fde18104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.774+0000 7fde1eb00700 1 -- 172.21.15.52:0/32099772 >> 172.21.15.52:0/32099772 conn(0x7fde18075920 msgr2=0x7fde18075d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:33.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.774+0000 7fde1eb00700 1 -- 172.21.15.52:0/32099772 shutdown_connections 2023-11-27T15:27:33.779 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.774+0000 7fde1eb00700 1 -- 172.21.15.52:0/32099772 wait complete. 2023-11-27T15:27:33.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.775+0000 7fde1eb00700 1 Processor -- start 2023-11-27T15:27:33.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.775+0000 7fde1eb00700 1 -- start start 2023-11-27T15:27:33.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.775+0000 7fde1eb00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde180feaf0 0x7fde18196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.775+0000 7fde1eb00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde180ff490 0x7fde18196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.775+0000 7fde1eb00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde181055d0 0x7fde1819af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.781 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.775+0000 7fde1eb00700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1819b570 con 0x7fde181055d0 2023-11-27T15:27:33.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.775+0000 7fde1eb00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1819b6e0 con 0x7fde180feaf0 2023-11-27T15:27:33.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.775+0000 7fde1eb00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1819b8a0 con 0x7fde180ff490 2023-11-27T15:27:33.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.776+0000 7fde1d09d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde181055d0 0x7fde1819af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.776+0000 7fde1d09d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde181055d0 0x7fde1819af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56048/0 (socket says 172.21.15.52:56048) 2023-11-27T15:27:33.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.776+0000 7fde1d09d700 1 -- 172.21.15.52:0/118316398 learned_addr learned my addr 172.21.15.52:0/118316398 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:33.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.776+0000 7fde1c89c700 1 --2- 172.21.15.52:0/118316398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde180feaf0 0x7fde18196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.776+0000 7fde17fff700 1 --2- 172.21.15.52:0/118316398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde180ff490 0x7fde18196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.782 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.776+0000 7fde1d09d700 1 -- 172.21.15.52:0/118316398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde180ff490 msgr2=0x7fde18196ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.783 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.776+0000 7fde1d09d700 1 --2- 172.21.15.52:0/118316398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde180ff490 0x7fde18196ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.783 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.776+0000 7fde1d09d700 1 -- 172.21.15.52:0/118316398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde180feaf0 msgr2=0x7fde18196590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.783 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.776+0000 7fde1d09d700 1 --2- 172.21.15.52:0/118316398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde180feaf0 0x7fde18196590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.783 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.776+0000 7fde1d09d700 1 -- 172.21.15.52:0/118316398 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde10009d00 con 0x7fde181055d0 2023-11-27T15:27:33.783 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.776+0000 7fde1d09d700 1 --2- 172.21.15.52:0/118316398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde181055d0 0x7fde1819af30 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fde10007820 tx=0x7fde10007850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:33.783 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.776+0000 7fde15ffb700 1 -- 172.21.15.52:0/118316398 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1000e040 con 0x7fde181055d0 2023-11-27T15:27:33.783 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.776+0000 7fde15ffb700 1 -- 172.21.15.52:0/118316398 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde10003e10 con 0x7fde181055d0 2023-11-27T15:27:33.783 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.776+0000 7fde1eb00700 1 -- 172.21.15.52:0/118316398 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde18110440 con 0x7fde181055d0 2023-11-27T15:27:33.784 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.776+0000 7fde15ffb700 1 -- 172.21.15.52:0/118316398 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1001b890 con 0x7fde181055d0 2023-11-27T15:27:33.784 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.777+0000 7fde1eb00700 1 -- 172.21.15.52:0/118316398 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde181102b0 con 0x7fde181055d0 2023-11-27T15:27:33.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.779+0000 7fde15ffb700 1 -- 172.21.15.52:0/118316398 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fde10024410 con 0x7fde181055d0 2023-11-27T15:27:33.790 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.779+0000 7fde15ffb700 1 --2- 172.21.15.52:0/118316398 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fde080a12c0 0x7fde080a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.780+0000 7fde15ffb700 1 -- 172.21.15.52:0/118316398 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fde100ce250 con 0x7fde181055d0 2023-11-27T15:27:33.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.780+0000 7fde1c89c700 1 --2- 172.21.15.52:0/118316398 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fde080a12c0 0x7fde080a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.780+0000 7fde1c89c700 1 --2- 172.21.15.52:0/118316398 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fde080a12c0 0x7fde080a3770 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fde0c005950 tx=0x7fde0c00b410 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:33.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.780+0000 7fde2ac8cb80 1 Processor -- start 2023-11-27T15:27:33.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.780+0000 7fde2ac8cb80 1 -- start start 2023-11-27T15:27:33.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.780+0000 7fde2ac8cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a184054b60 0x55a184054f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.780+0000 7fde2ac8cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a184055470 0x55a184059370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.791 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.780+0000 7fde2ac8cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a1840598b0 0x55a184059d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.780+0000 7fde2ac8cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a183f5a060 con 0x55a1840598b0 2023-11-27T15:27:33.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.780+0000 7fde2ac8cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a183fac8a0 con 0x55a184054b60 2023-11-27T15:27:33.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.780+0000 7fde2ac8cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a183fa5d30 con 0x55a184055470 2023-11-27T15:27:33.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.781+0000 7fde1d09d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a1840598b0 0x55a184059d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.781+0000 7fde1c89c700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a184054b60 0x55a184054f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.781+0000 7fde1d09d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a1840598b0 0x55a184059d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56056/0 (socket says 172.21.15.52:56056) 2023-11-27T15:27:33.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.781+0000 7fde1d09d700 1 -- 172.21.15.52:0/1907254302 learned_addr learned my addr 172.21.15.52:0/1907254302 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:33.792 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.781+0000 7fde17fff700 1 --2- 172.21.15.52:0/1907254302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a184055470 0x55a184059370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.781+0000 7fde1d09d700 1 -- 172.21.15.52:0/1907254302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a184055470 msgr2=0x55a184059370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.781+0000 7fde1d09d700 1 --2- 172.21.15.52:0/1907254302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a184055470 0x55a184059370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.781+0000 7fde1d09d700 1 -- 172.21.15.52:0/1907254302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a184054b60 msgr2=0x55a184054f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.781+0000 7fde1d09d700 1 --2- 172.21.15.52:0/1907254302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a184054b60 0x55a184054f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.781+0000 7fde1d09d700 1 -- 172.21.15.52:0/1907254302 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a183e27630 con 0x55a1840598b0 2023-11-27T15:27:33.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.781+0000 7fde17fff700 1 --2- 172.21.15.52:0/1907254302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a184055470 0x55a184059370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:33.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.781+0000 7fde1d09d700 1 --2- 172.21.15.52:0/1907254302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a1840598b0 0x55a184059d00 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fde10097820 tx=0x7fde1009b0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852e62edbc0a754e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:33.793 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.781+0000 7fde02ffd700 1 -- 172.21.15.52:0/1907254302 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1009f120 con 0x55a1840598b0 2023-11-27T15:27:33.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.781+0000 7fde02ffd700 1 -- 172.21.15.52:0/1907254302 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde1009b320 con 0x55a1840598b0 2023-11-27T15:27:33.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.781+0000 7fde02ffd700 1 -- 172.21.15.52:0/1907254302 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1009b580 con 0x55a1840598b0 2023-11-27T15:27:33.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.782+0000 7fde2ac8cb80 1 -- 172.21.15.52:0/1907254302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a1840598b0 msgr2=0x55a184059d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.782+0000 7fde2ac8cb80 1 --2- 172.21.15.52:0/1907254302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a1840598b0 0x55a184059d00 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fde10097820 tx=0x7fde1009b0f0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.782+0000 7fde2ac8cb80 1 -- 172.21.15.52:0/1907254302 shutdown_connections 2023-11-27T15:27:33.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.782+0000 7fde2ac8cb80 1 --2- 172.21.15.52:0/1907254302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a184054b60 0x55a184054f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.782+0000 7fde2ac8cb80 1 --2- 172.21.15.52:0/1907254302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a1840598b0 0x55a184059d00 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.782+0000 7fde2ac8cb80 1 --2- 172.21.15.52:0/1907254302 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a184055470 0x55a184059370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.782+0000 7fde2ac8cb80 1 -- 172.21.15.52:0/1907254302 >> 172.21.15.52:0/1907254302 conn(0x55a18404a360 msgr2=0x55a18404fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:33.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.782+0000 7fde2ac8cb80 1 -- 172.21.15.52:0/1907254302 shutdown_connections 2023-11-27T15:27:33.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.782+0000 7fde2ac8cb80 1 -- 172.21.15.52:0/1907254302 wait complete. 2023-11-27T15:27:33.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.782+0000 7fde2ac8cb80 1 -- start start 2023-11-27T15:27:33.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.782+0000 7fde2ac8cb80 1 Processor -- start 2023-11-27T15:27:33.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.783+0000 7fde2ac8cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a184054b60 0x55a184061010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.783+0000 7fde2ac8cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a184055470 0x55a18405e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.783+0000 7fde2ac8cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1840598b0 0x55a18405e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.783+0000 7fde2ac8cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a183f478e0 con 0x55a184055470 2023-11-27T15:27:33.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.783+0000 7fde2ac8cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a183f47580 con 0x55a1840598b0 2023-11-27T15:27:33.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.783+0000 7fde2ac8cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a183e29920 con 0x55a184054b60 2023-11-27T15:27:33.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.783+0000 7fde2ac8cb80 10 client.0 _ll_register_callbacks cb 0x55a183f65780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:33.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.783+0000 7fde17fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a184055470 0x55a18405e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.783+0000 7fde17fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a184055470 0x55a18405e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56060/0 (socket says 172.21.15.52:56060) 2023-11-27T15:27:33.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.783+0000 7fde17fff700 1 -- 172.21.15.52:0/3561763739 learned_addr learned my addr 172.21.15.52:0/3561763739 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:33.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.783+0000 7fde1c89c700 1 --2- 172.21.15.52:0/3561763739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a184054b60 0x55a184061010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.783+0000 7fde1d09d700 1 --2- 172.21.15.52:0/3561763739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1840598b0 0x55a18405e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.783+0000 7fde17fff700 1 -- 172.21.15.52:0/3561763739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a184054b60 msgr2=0x55a184061010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.783+0000 7fde17fff700 1 --2- 172.21.15.52:0/3561763739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a184054b60 0x55a184061010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.783+0000 7fde17fff700 1 -- 172.21.15.52:0/3561763739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1840598b0 msgr2=0x55a18405e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.783+0000 7fde17fff700 1 --2- 172.21.15.52:0/3561763739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1840598b0 0x55a18405e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.783+0000 7fde17fff700 1 -- 172.21.15.52:0/3561763739 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde181102b0 con 0x55a184055470 2023-11-27T15:27:33.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.783+0000 7fde1c89c700 1 --2- 172.21.15.52:0/3561763739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a184054b60 0x55a184061010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:33.797 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.783+0000 7fde17fff700 1 --2- 172.21.15.52:0/3561763739 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a184055470 0x55a18405e290 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fde0400b9c0 tx=0x7fde0400bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876d5a241abf129 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:33.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.783+0000 7fddf7fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:27:33.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.784+0000 7fddf7fff700 1 -- 172.21.15.52:0/3561763739 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde04004560 con 0x55a184055470 2023-11-27T15:27:33.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.784+0000 7fddf7fff700 1 -- 172.21.15.52:0/3561763739 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fde04004830 con 0x55a184055470 2023-11-27T15:27:33.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.784+0000 7fde2ac8cb80 1 -- 172.21.15.52:0/3561763739 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a184047b20 con 0x55a184055470 2023-11-27T15:27:33.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.784+0000 7fddf7fff700 1 -- 172.21.15.52:0/3561763739 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde040049f0 con 0x55a184055470 2023-11-27T15:27:33.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.784+0000 7fde2ac8cb80 10 client.17121 fetch_fsmap learned FSMap version 34 2023-11-27T15:27:33.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.784+0000 7fde2ac8cb80 1 -- 172.21.15.52:0/3561763739 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55a183f6c190 con 0x55a184055470 2023-11-27T15:27:33.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.784+0000 7fddf7fff700 1 -- 172.21.15.52:0/3561763739 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a184047b20 con 0x55a184055470 2023-11-27T15:27:33.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.784+0000 7fddf7fff700 1 -- 172.21.15.52:0/3561763739 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fde0401a680 con 0x55a184055470 2023-11-27T15:27:33.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.784+0000 7fde2ac8cb80 10 client.17121 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:27:33.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.784+0000 7fde2ac8cb80 10 client.17121 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:27:33.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.784+0000 7fde2ac8cb80 20 client.17121 populate_metadata read hostname 'smithi052' 2023-11-27T15:27:33.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.784+0000 7fde2ac8cb80 1 --2- 172.21.15.52:0/3561763739 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55a18406e5e0 0x55a1840709f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.784+0000 7fde2ac8cb80 4 client.17121 mds_command: new command op to 24494 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:27:33.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.784+0000 7fde2ac8cb80 1 -- 172.21.15.52:0/3561763739 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a183f680a0 con 0x55a18406e5e0 2023-11-27T15:27:33.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.784+0000 7fde1c89c700 1 --2- 172.21.15.52:0/3561763739 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55a18406e5e0 0x55a1840709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.785+0000 7fde1c89c700 1 --2- 172.21.15.52:0/3561763739 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55a18406e5e0 0x55a1840709f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b75e57c0c82fea12 server_cookie=5b6bffbab116016e in_seq=0 out_seq=0 2023-11-27T15:27:33.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.785+0000 7fddf7fff700 10 client.17121 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:27:33.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.785+0000 7fddf7fff700 1 -- 172.21.15.52:0/3561763739 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a183f680a0 con 0x55a18406e5e0 2023-11-27T15:27:33.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.785+0000 7fddf7fff700 10 client.17121 handle_command_reply: tid=0 2023-11-27T15:27:33.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.786+0000 7fde2ac8cb80 1 client.17121 shutdown 2023-11-27T15:27:33.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.786+0000 7fde2ac8cb80 1 -- 172.21.15.52:0/3561763739 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a184055470 msgr2=0x55a18405e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.786+0000 7fde2ac8cb80 1 --2- 172.21.15.52:0/3561763739 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a184055470 0x55a18405e290 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fde0400b9c0 tx=0x7fde0400bd80 comp rx=0 tx=0).stop 2023-11-27T15:27:33.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.786+0000 7fde2ac8cb80 1 -- 172.21.15.52:0/3561763739 shutdown_connections 2023-11-27T15:27:33.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.786+0000 7fde2ac8cb80 1 --2- 172.21.15.52:0/3561763739 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55a18406e5e0 0x55a1840709f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.786+0000 7fde2ac8cb80 1 --2- 172.21.15.52:0/3561763739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a1840598b0 0x55a18405e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.786+0000 7fddf7fff700 0 client.17121 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:27:33.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.786+0000 7fde2ac8cb80 1 --2- 172.21.15.52:0/3561763739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a184054b60 0x55a184061010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.786+0000 7fde2ac8cb80 1 --2- 172.21.15.52:0/3561763739 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a184055470 0x55a18405e290 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.786+0000 7fde2ac8cb80 1 -- 172.21.15.52:0/3561763739 >> 172.21.15.52:0/3561763739 conn(0x55a18404a360 msgr2=0x55a18405ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:33.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.786+0000 7fde2ac8cb80 1 -- 172.21.15.52:0/3561763739 shutdown_connections 2023-11-27T15:27:33.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.786+0000 7fde2ac8cb80 1 -- 172.21.15.52:0/3561763739 wait complete. 2023-11-27T15:27:33.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.786+0000 7fde2ac8cb80 20 client.17121 trim_cache size 0 max 16384 2023-11-27T15:27:33.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.799+0000 7fde2ac8cb80 1 Processor -- start 2023-11-27T15:27:33.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.799+0000 7fde2ac8cb80 1 -- start start 2023-11-27T15:27:33.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.799+0000 7fde2ac8cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a184054b60 0x55a18404fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.804 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.799+0000 7fde2ac8cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a184055470 0x55a1840657b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.799+0000 7fde2ac8cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1840598b0 0x55a18405acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.799+0000 7fde2ac8cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1840479d0 con 0x55a184054b60 2023-11-27T15:27:33.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.799+0000 7fde2ac8cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a18405c7f0 con 0x55a184055470 2023-11-27T15:27:33.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.799+0000 7fde2ac8cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1840583c0 con 0x55a1840598b0 2023-11-27T15:27:33.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.799+0000 7fde1c89c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a184054b60 0x55a18404fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.799+0000 7fde1c89c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a184054b60 0x55a18404fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56066/0 (socket says 172.21.15.52:56066) 2023-11-27T15:27:33.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.799+0000 7fde1c89c700 1 -- 172.21.15.52:0/2696383492 learned_addr learned my addr 172.21.15.52:0/2696383492 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:33.805 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.799+0000 7fde17fff700 1 --2- 172.21.15.52:0/2696383492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a184055470 0x55a1840657b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.799+0000 7fde1d09d700 1 --2- 172.21.15.52:0/2696383492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1840598b0 0x55a18405acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.799+0000 7fde1c89c700 1 -- 172.21.15.52:0/2696383492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1840598b0 msgr2=0x55a18405acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.799+0000 7fde1c89c700 1 --2- 172.21.15.52:0/2696383492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1840598b0 0x55a18405acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.799+0000 7fde1c89c700 1 -- 172.21.15.52:0/2696383492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a184055470 msgr2=0x55a1840657b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.799+0000 7fde1c89c700 1 --2- 172.21.15.52:0/2696383492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a184055470 0x55a1840657b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.800+0000 7fde1c89c700 1 -- 172.21.15.52:0/2696383492 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a18404bb00 con 0x55a184054b60 2023-11-27T15:27:33.806 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.800+0000 7fde17fff700 1 --2- 172.21.15.52:0/2696383492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a184055470 0x55a1840657b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:33.807 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.800+0000 7fde1c89c700 1 --2- 172.21.15.52:0/2696383492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a184054b60 0x55a18404fbe0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fde0c005d90 tx=0x7fde0c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b36c8efe6aa4989 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:33.807 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.800+0000 7fde037fe700 1 -- 172.21.15.52:0/2696383492 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c026070 con 0x55a184054b60 2023-11-27T15:27:33.807 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.800+0000 7fde037fe700 1 -- 172.21.15.52:0/2696383492 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde0c015070 con 0x55a184054b60 2023-11-27T15:27:33.807 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.800+0000 7fde037fe700 1 -- 172.21.15.52:0/2696383492 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c004bc0 con 0x55a184054b60 2023-11-27T15:27:33.807 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.800+0000 7fde2ac8cb80 1 -- 172.21.15.52:0/2696383492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a184054b60 msgr2=0x55a18404fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.807 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.800+0000 7fde2ac8cb80 1 --2- 172.21.15.52:0/2696383492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a184054b60 0x55a18404fbe0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fde0c005d90 tx=0x7fde0c013000 comp rx=0 tx=0).stop 2023-11-27T15:27:33.807 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.800+0000 7fde2ac8cb80 1 -- 172.21.15.52:0/2696383492 shutdown_connections 2023-11-27T15:27:33.807 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.800+0000 7fde2ac8cb80 1 --2- 172.21.15.52:0/2696383492 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a1840598b0 0x55a18405acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.808 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.800+0000 7fde2ac8cb80 1 --2- 172.21.15.52:0/2696383492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a184054b60 0x55a18404fbe0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.808 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.800+0000 7fde2ac8cb80 1 --2- 172.21.15.52:0/2696383492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a184055470 0x55a1840657b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.808 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.800+0000 7fde2ac8cb80 1 -- 172.21.15.52:0/2696383492 >> 172.21.15.52:0/2696383492 conn(0x55a18404a360 msgr2=0x55a1840578f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:33.808 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.800+0000 7fde2ac8cb80 1 -- 172.21.15.52:0/2696383492 shutdown_connections 2023-11-27T15:27:33.808 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.800+0000 7fde2ac8cb80 1 -- 172.21.15.52:0/2696383492 wait complete. 2023-11-27T15:27:33.808 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.801+0000 7fde2ac8cb80 1 -- start start 2023-11-27T15:27:33.808 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.801+0000 7fde2ac8cb80 1 Processor -- start 2023-11-27T15:27:33.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.801+0000 7fde2ac8cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a184054b60 0x55a18405dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.811 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.801+0000 7fde2ac8cb80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a184055470 0x55a18405e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.801+0000 7fde2ac8cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a1840598b0 0x55a18405ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.801+0000 7fde2ac8cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a184065dc0 con 0x55a1840598b0 2023-11-27T15:27:33.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.801+0000 7fde2ac8cb80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a18405cf80 con 0x55a184055470 2023-11-27T15:27:33.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.801+0000 7fde2ac8cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a18405d140 con 0x55a184054b60 2023-11-27T15:27:33.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.801+0000 7fde2ac8cb80 10 client.0 _ll_register_callbacks cb 0x55a183f65780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:33.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.801+0000 7fde1d09d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a1840598b0 0x55a18405ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.812 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.801+0000 7fde1d09d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a1840598b0 0x55a18405ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56076/0 (socket says 172.21.15.52:56076) 2023-11-27T15:27:33.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.801+0000 7fde1d09d700 1 -- 172.21.15.52:0/1932625699 learned_addr learned my addr 172.21.15.52:0/1932625699 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:33.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.801+0000 7fde1c89c700 1 --2- 172.21.15.52:0/1932625699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a184054b60 0x55a18405dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.801+0000 7fde17fff700 1 --2- 172.21.15.52:0/1932625699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a184055470 0x55a18405e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.802+0000 7fde1d09d700 1 -- 172.21.15.52:0/1932625699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a184054b60 msgr2=0x55a18405dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.802+0000 7fde1d09d700 1 --2- 172.21.15.52:0/1932625699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a184054b60 0x55a18405dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.802+0000 7fde1d09d700 1 -- 172.21.15.52:0/1932625699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a184055470 msgr2=0x55a18405e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.802+0000 7fde1d09d700 1 --2- 172.21.15.52:0/1932625699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a184055470 0x55a18405e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.802+0000 7fde1d09d700 1 -- 172.21.15.52:0/1932625699 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde0c00beb0 con 0x55a1840598b0 2023-11-27T15:27:33.813 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.802+0000 7fde1d09d700 1 --2- 172.21.15.52:0/1932625699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a1840598b0 0x55a18405ca40 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fde100077f0 tx=0x7fde10097780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5f4e5d2ca8fdef server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:33.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.802+0000 7fddf7fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:27:33.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.802+0000 7fddf7fff700 1 -- 172.21.15.52:0/1932625699 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1009aa10 con 0x55a1840598b0 2023-11-27T15:27:33.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.802+0000 7fddf7fff700 1 -- 172.21.15.52:0/1932625699 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fde1009f350 con 0x55a1840598b0 2023-11-27T15:27:33.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.802+0000 7fde2ac8cb80 1 -- 172.21.15.52:0/1932625699 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde1009b320 con 0x55a1840598b0 2023-11-27T15:27:33.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.802+0000 7fddf7fff700 1 -- 172.21.15.52:0/1932625699 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde1009f120 con 0x55a1840598b0 2023-11-27T15:27:33.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.802+0000 7fde2ac8cb80 10 client.17127 fetch_fsmap learned FSMap version 34 2023-11-27T15:27:33.814 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.802+0000 7fde2ac8cb80 1 -- 172.21.15.52:0/1932625699 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55a183dd3de0 con 0x55a1840598b0 2023-11-27T15:27:33.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.802+0000 7fddf7fff700 1 -- 172.21.15.52:0/1932625699 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde1009b320 con 0x55a1840598b0 2023-11-27T15:27:33.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.802+0000 7fddf7fff700 1 -- 172.21.15.52:0/1932625699 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fde100a3250 con 0x55a1840598b0 2023-11-27T15:27:33.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.802+0000 7fde2ac8cb80 10 client.17127 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:27:33.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.802+0000 7fde2ac8cb80 10 client.17127 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:27:33.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.802+0000 7fde2ac8cb80 20 client.17127 populate_metadata read hostname 'smithi052' 2023-11-27T15:27:33.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.803+0000 7fde2ac8cb80 1 --2- 172.21.15.52:0/1932625699 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55a1840c7210 0x55a1840c9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.803+0000 7fde2ac8cb80 4 client.17127 mds_command: new command op to 24494 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:27:33.815 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.803+0000 7fde2ac8cb80 1 -- 172.21.15.52:0/1932625699 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a183f74660 con 0x55a1840c7210 2023-11-27T15:27:33.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.803+0000 7fde1c89c700 1 --2- 172.21.15.52:0/1932625699 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55a1840c7210 0x55a1840c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.803+0000 7fddf7fff700 10 client.17127 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:27:33.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.803+0000 7fde1c89c700 1 --2- 172.21.15.52:0/1932625699 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55a1840c7210 0x55a1840c9620 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eb8fef773f28e129 server_cookie=c38df2d8aa505529 in_seq=0 out_seq=0 2023-11-27T15:27:33.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.803+0000 7fddf7fff700 1 -- 172.21.15.52:0/1932625699 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a183f74660 con 0x55a1840c7210 2023-11-27T15:27:33.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.803+0000 7fddf7fff700 10 client.17127 handle_command_reply: tid=0 2023-11-27T15:27:33.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.803+0000 7fde2ac8cb80 1 client.17127 shutdown 2023-11-27T15:27:33.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.804+0000 7fde2ac8cb80 1 -- 172.21.15.52:0/1932625699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a1840598b0 msgr2=0x55a18405ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.816 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.804+0000 7fde2ac8cb80 1 --2- 172.21.15.52:0/1932625699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a1840598b0 0x55a18405ca40 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fde100077f0 tx=0x7fde10097780 comp rx=0 tx=0).stop 2023-11-27T15:27:33.817 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.804+0000 7fde2ac8cb80 1 -- 172.21.15.52:0/1932625699 shutdown_connections 2023-11-27T15:27:33.817 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.804+0000 7fde2ac8cb80 1 --2- 172.21.15.52:0/1932625699 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55a1840c7210 0x55a1840c9620 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.817 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.804+0000 7fde2ac8cb80 1 --2- 172.21.15.52:0/1932625699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a1840598b0 0x55a18405ca40 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.817 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.804+0000 7fde2ac8cb80 1 --2- 172.21.15.52:0/1932625699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a184054b60 0x55a18405dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.817 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.804+0000 7fde2ac8cb80 1 --2- 172.21.15.52:0/1932625699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a184055470 0x55a18405e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.817 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.804+0000 7fddf7fff700 0 client.17127 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:27:33.817 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.804+0000 7fde2ac8cb80 1 -- 172.21.15.52:0/1932625699 >> 172.21.15.52:0/1932625699 conn(0x55a18404a360 msgr2=0x55a18405b570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:33.818 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.804+0000 7fde2ac8cb80 1 -- 172.21.15.52:0/1932625699 shutdown_connections 2023-11-27T15:27:33.818 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.804+0000 7fde2ac8cb80 1 -- 172.21.15.52:0/1932625699 wait complete. 2023-11-27T15:27:33.818 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.804+0000 7fde2ac8cb80 20 client.17127 trim_cache size 0 max 16384 2023-11-27T15:27:33.818 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:27:33.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.804+0000 7fde1eb00700 1 -- 172.21.15.52:0/118316398 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fde080a12c0 msgr2=0x7fde080a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.804+0000 7fde1eb00700 1 --2- 172.21.15.52:0/118316398 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fde080a12c0 0x7fde080a3770 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fde0c005950 tx=0x7fde0c00b410 comp rx=0 tx=0).stop 2023-11-27T15:27:33.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.804+0000 7fde1eb00700 1 -- 172.21.15.52:0/118316398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde181055d0 msgr2=0x7fde1819af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.804+0000 7fde1eb00700 1 --2- 172.21.15.52:0/118316398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde181055d0 0x7fde1819af30 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fde10007820 tx=0x7fde10007850 comp rx=0 tx=0).stop 2023-11-27T15:27:33.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.804+0000 7fde1eb00700 1 -- 172.21.15.52:0/118316398 shutdown_connections 2023-11-27T15:27:33.819 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.804+0000 7fde1eb00700 1 --2- 172.21.15.52:0/118316398 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fde080a12c0 0x7fde080a3770 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.804+0000 7fde1eb00700 1 --2- 172.21.15.52:0/118316398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde180feaf0 0x7fde18196590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.804+0000 7fde1eb00700 1 --2- 172.21.15.52:0/118316398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde181055d0 0x7fde1819af30 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.804+0000 7fde1eb00700 1 --2- 172.21.15.52:0/118316398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde180ff490 0x7fde18196ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.804+0000 7fde1eb00700 1 -- 172.21.15.52:0/118316398 >> 172.21.15.52:0/118316398 conn(0x7fde18075920 msgr2=0x7fde180fc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:33.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.805+0000 7fde1eb00700 1 -- 172.21.15.52:0/118316398 shutdown_connections 2023-11-27T15:27:33.820 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.805+0000 7fde1eb00700 1 -- 172.21.15.52:0/118316398 wait complete. 2023-11-27T15:27:33.821 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-27T15:27:33.949 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.946+0000 7f2175419700 1 -- 172.21.15.52:0/601083903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21700ffba0 msgr2=0x7f21700ffff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.949 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.946+0000 7f2175419700 1 --2- 172.21.15.52:0/601083903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21700ffba0 0x7f21700ffff0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f2164009990 tx=0x7f2164009ca0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.946+0000 7f2175419700 1 -- 172.21.15.52:0/601083903 shutdown_connections 2023-11-27T15:27:33.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.946+0000 7f2175419700 1 --2- 172.21.15.52:0/601083903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2170100530 0x7f217010c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.946+0000 7f2175419700 1 --2- 172.21.15.52:0/601083903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2170105210 0x7f21701055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.946+0000 7f2175419700 1 --2- 172.21.15.52:0/601083903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21700ffba0 0x7f21700ffff0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.946+0000 7f2175419700 1 -- 172.21.15.52:0/601083903 >> 172.21.15.52:0/601083903 conn(0x7f21700748f0 msgr2=0x7f2170074cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:33.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.946+0000 7f2175419700 1 -- 172.21.15.52:0/601083903 shutdown_connections 2023-11-27T15:27:33.950 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.946+0000 7f2175419700 1 -- 172.21.15.52:0/601083903 wait complete. 2023-11-27T15:27:33.951 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.946+0000 7f2175419700 1 Processor -- start 2023-11-27T15:27:33.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.946+0000 7f2175419700 1 -- start start 2023-11-27T15:27:33.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.947+0000 7f2175419700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21700ffba0 0x7f21701953e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.952 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.947+0000 7f2175419700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2170100530 0x7f2170195920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.947+0000 7f2175419700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2170105210 0x7f2170199d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.947+0000 7f2175419700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217019a3c0 con 0x7f2170105210 2023-11-27T15:27:33.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.947+0000 7f2175419700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217019a530 con 0x7f21700ffba0 2023-11-27T15:27:33.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.947+0000 7f2175419700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217019a6f0 con 0x7f2170100530 2023-11-27T15:27:33.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.947+0000 7f216f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2170105210 0x7f2170199d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.947+0000 7f216f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2170105210 0x7f2170199d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56090/0 (socket says 172.21.15.52:56090) 2023-11-27T15:27:33.953 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.947+0000 7f216f7fe700 1 -- 172.21.15.52:0/3085843332 learned_addr learned my addr 172.21.15.52:0/3085843332 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:33.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.947+0000 7f216e7fc700 1 --2- 172.21.15.52:0/3085843332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2170100530 0x7f2170195920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.947+0000 7f216effd700 1 --2- 172.21.15.52:0/3085843332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21700ffba0 0x7f21701953e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.947+0000 7f216f7fe700 1 -- 172.21.15.52:0/3085843332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2170100530 msgr2=0x7f2170195920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.947+0000 7f216f7fe700 1 --2- 172.21.15.52:0/3085843332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2170100530 0x7f2170195920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.947+0000 7f216f7fe700 1 -- 172.21.15.52:0/3085843332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21700ffba0 msgr2=0x7f21701953e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.947+0000 7f216f7fe700 1 --2- 172.21.15.52:0/3085843332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21700ffba0 0x7f21701953e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.947+0000 7f216f7fe700 1 -- 172.21.15.52:0/3085843332 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21640095f0 con 0x7f2170105210 2023-11-27T15:27:33.954 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.947+0000 7f216f7fe700 1 --2- 172.21.15.52:0/3085843332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2170105210 0x7f2170199d80 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f2168007ee0 tx=0x7f216800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:33.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.947+0000 7f2157fff700 1 -- 172.21.15.52:0/3085843332 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216800dd20 con 0x7f2170105210 2023-11-27T15:27:33.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.947+0000 7f2157fff700 1 -- 172.21.15.52:0/3085843332 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f216800f040 con 0x7f2170105210 2023-11-27T15:27:33.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.948+0000 7f2175419700 1 -- 172.21.15.52:0/3085843332 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f217010f270 con 0x7f2170105210 2023-11-27T15:27:33.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.948+0000 7f2157fff700 1 -- 172.21.15.52:0/3085843332 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21680048a0 con 0x7f2170105210 2023-11-27T15:27:33.955 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.948+0000 7f2175419700 1 -- 172.21.15.52:0/3085843332 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f217010f0e0 con 0x7f2170105210 2023-11-27T15:27:33.961 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.950+0000 7f2157fff700 1 -- 172.21.15.52:0/3085843332 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f2168003a00 con 0x7f2170105210 2023-11-27T15:27:33.961 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.951+0000 7f2157fff700 1 --2- 172.21.15.52:0/3085843332 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f21580a12c0 0x7f21580a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.962 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.951+0000 7f2157fff700 1 -- 172.21.15.52:0/3085843332 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f21680cc5b0 con 0x7f2170105210 2023-11-27T15:27:33.962 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.951+0000 7f216effd700 1 --2- 172.21.15.52:0/3085843332 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f21580a12c0 0x7f21580a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.962 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.951+0000 7f216effd700 1 --2- 172.21.15.52:0/3085843332 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f21580a12c0 0x7f21580a3770 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2160005950 tx=0x7f216000b410 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:33.962 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.951+0000 7f21815a5b80 1 Processor -- start 2023-11-27T15:27:33.962 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.951+0000 7f21815a5b80 1 -- start start 2023-11-27T15:27:33.962 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.952+0000 7f21815a5b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9253d0970 0x55a9253d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.962 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.952+0000 7f21815a5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9253d1280 0x55a9253deff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.962 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.952+0000 7f21815a5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a9253df530 0x55a9253e1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.963 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.952+0000 7f21815a5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9252d6060 con 0x55a9253df530 2023-11-27T15:27:33.963 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.952+0000 7f21815a5b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9253288a0 con 0x55a9253d0970 2023-11-27T15:27:33.963 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.952+0000 7f21815a5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a925321d30 con 0x55a9253d1280 2023-11-27T15:27:33.963 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.952+0000 7f216f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a9253df530 0x55a9253e1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.963 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.952+0000 7f216f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a9253df530 0x55a9253e1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56098/0 (socket says 172.21.15.52:56098) 2023-11-27T15:27:33.963 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.952+0000 7f216f7fe700 1 -- 172.21.15.52:0/1787783943 learned_addr learned my addr 172.21.15.52:0/1787783943 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:33.963 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.952+0000 7f216effd700 1 --2- 172.21.15.52:0/1787783943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9253d0970 0x55a9253d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.963 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.952+0000 7f216f7fe700 1 -- 172.21.15.52:0/1787783943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9253d1280 msgr2=0x55a9253deff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.964 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.952+0000 7f216e7fc700 1 --2- 172.21.15.52:0/1787783943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9253d1280 0x55a9253deff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.964 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.952+0000 7f216f7fe700 1 --2- 172.21.15.52:0/1787783943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9253d1280 0x55a9253deff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.964 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.952+0000 7f216f7fe700 1 -- 172.21.15.52:0/1787783943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9253d0970 msgr2=0x55a9253d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.964 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.952+0000 7f216f7fe700 1 --2- 172.21.15.52:0/1787783943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9253d0970 0x55a9253d0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.964 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.952+0000 7f216f7fe700 1 -- 172.21.15.52:0/1787783943 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9251a3630 con 0x55a9253df530 2023-11-27T15:27:33.964 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.952+0000 7f216effd700 1 --2- 172.21.15.52:0/1787783943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9253d0970 0x55a9253d0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:33.964 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.952+0000 7f216e7fc700 1 --2- 172.21.15.52:0/1787783943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9253d1280 0x55a9253deff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:33.964 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.952+0000 7f216f7fe700 1 --2- 172.21.15.52:0/1787783943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a9253df530 0x55a9253e1910 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2168095b50 tx=0x7f2168099210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3907fef5969e0c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:33.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.952+0000 7f21557fa700 1 -- 172.21.15.52:0/1787783943 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216809d680 con 0x55a9253df530 2023-11-27T15:27:33.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.952+0000 7f21557fa700 1 -- 172.21.15.52:0/1787783943 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21680a1aa0 con 0x55a9253df530 2023-11-27T15:27:33.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.952+0000 7f21557fa700 1 -- 172.21.15.52:0/1787783943 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2168095d60 con 0x55a9253df530 2023-11-27T15:27:33.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.953+0000 7f21815a5b80 1 -- 172.21.15.52:0/1787783943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a9253df530 msgr2=0x55a9253e1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.953+0000 7f21815a5b80 1 --2- 172.21.15.52:0/1787783943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a9253df530 0x55a9253e1910 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2168095b50 tx=0x7f2168099210 comp rx=0 tx=0).stop 2023-11-27T15:27:33.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.953+0000 7f21815a5b80 1 -- 172.21.15.52:0/1787783943 shutdown_connections 2023-11-27T15:27:33.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.953+0000 7f21815a5b80 1 --2- 172.21.15.52:0/1787783943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9253d0970 0x55a9253d0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.953+0000 7f21815a5b80 1 --2- 172.21.15.52:0/1787783943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a9253df530 0x55a9253e1910 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.965 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.953+0000 7f21815a5b80 1 --2- 172.21.15.52:0/1787783943 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9253d1280 0x55a9253deff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.966 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.953+0000 7f21815a5b80 1 -- 172.21.15.52:0/1787783943 >> 172.21.15.52:0/1787783943 conn(0x55a9253c6360 msgr2=0x55a9253cbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:33.966 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.953+0000 7f21815a5b80 1 -- 172.21.15.52:0/1787783943 shutdown_connections 2023-11-27T15:27:33.966 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.953+0000 7f21815a5b80 1 -- 172.21.15.52:0/1787783943 wait complete. 2023-11-27T15:27:33.966 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.953+0000 7f21815a5b80 1 -- start start 2023-11-27T15:27:33.966 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.954+0000 7f21815a5b80 1 Processor -- start 2023-11-27T15:27:33.966 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.954+0000 7f21815a5b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9253d0970 0x55a9253df900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.966 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.954+0000 7f21815a5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9253d1280 0x55a9253dfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.966 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.954+0000 7f21815a5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a9253df530 0x55a9253ea980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.954+0000 7f21815a5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9252c38e0 con 0x55a9253df530 2023-11-27T15:27:33.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.954+0000 7f21815a5b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9252c3580 con 0x55a9253d0970 2023-11-27T15:27:33.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.954+0000 7f21815a5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9251a5920 con 0x55a9253d1280 2023-11-27T15:27:33.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.954+0000 7f21815a5b80 10 client.0 _ll_register_callbacks cb 0x55a9252e1780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:33.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.954+0000 7f216f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a9253df530 0x55a9253ea980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.954+0000 7f216f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a9253df530 0x55a9253ea980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56104/0 (socket says 172.21.15.52:56104) 2023-11-27T15:27:33.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.954+0000 7f216f7fe700 1 -- 172.21.15.52:0/2096770107 learned_addr learned my addr 172.21.15.52:0/2096770107 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:33.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.954+0000 7f216effd700 1 --2- 172.21.15.52:0/2096770107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9253d0970 0x55a9253df900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.967 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.954+0000 7f216e7fc700 1 --2- 172.21.15.52:0/2096770107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9253d1280 0x55a9253dfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.954+0000 7f216f7fe700 1 -- 172.21.15.52:0/2096770107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9253d1280 msgr2=0x55a9253dfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.954+0000 7f216f7fe700 1 --2- 172.21.15.52:0/2096770107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9253d1280 0x55a9253dfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.954+0000 7f216f7fe700 1 -- 172.21.15.52:0/2096770107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9253d0970 msgr2=0x55a9253df900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.954+0000 7f216f7fe700 1 --2- 172.21.15.52:0/2096770107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9253d0970 0x55a9253df900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.954+0000 7f216f7fe700 1 -- 172.21.15.52:0/2096770107 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f217010f0e0 con 0x55a9253df530 2023-11-27T15:27:33.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.954+0000 7f216effd700 1 --2- 172.21.15.52:0/2096770107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9253d0970 0x55a9253df900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:27:33.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.954+0000 7f216f7fe700 1 --2- 172.21.15.52:0/2096770107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a9253df530 0x55a9253ea980 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f21680ca2d0 tx=0x7f21680c9cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35187986df6aea6 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:33.968 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.954+0000 7f2152ff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:27:33.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.955+0000 7f2152ff5700 1 -- 172.21.15.52:0/2096770107 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21680979f0 con 0x55a9253df530 2023-11-27T15:27:33.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.955+0000 7f2152ff5700 1 -- 172.21.15.52:0/2096770107 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f21680c9f50 con 0x55a9253df530 2023-11-27T15:27:33.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.955+0000 7f21815a5b80 1 -- 172.21.15.52:0/2096770107 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9253c3b20 con 0x55a9253df530 2023-11-27T15:27:33.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.955+0000 7f2152ff5700 1 -- 172.21.15.52:0/2096770107 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21680998b0 con 0x55a9253df530 2023-11-27T15:27:33.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.955+0000 7f21815a5b80 10 client.17139 fetch_fsmap learned FSMap version 34 2023-11-27T15:27:33.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.955+0000 7f21815a5b80 1 -- 172.21.15.52:0/2096770107 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55a9252e8190 con 0x55a9253df530 2023-11-27T15:27:33.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.955+0000 7f2152ff5700 1 -- 172.21.15.52:0/2096770107 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a9253c3b20 con 0x55a9253df530 2023-11-27T15:27:33.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.955+0000 7f2152ff5700 1 -- 172.21.15.52:0/2096770107 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f21680a16a0 con 0x55a9253df530 2023-11-27T15:27:33.969 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.955+0000 7f21815a5b80 10 client.17139 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:27:33.970 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.955+0000 7f21815a5b80 10 client.17139 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:33.970 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.955+0000 7f21815a5b80 20 client.17139 populate_metadata read hostname 'smithi052' 2023-11-27T15:27:33.970 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.955+0000 7f21815a5b80 1 --2- 172.21.15.52:0/2096770107 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55a9253ee750 0x55a9253f0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.970 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.955+0000 7f21815a5b80 4 client.17139 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:27:33.970 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.955+0000 7f21815a5b80 1 -- 172.21.15.52:0/2096770107 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9252e40a0 con 0x55a9253ee750 2023-11-27T15:27:33.970 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.955+0000 7f216effd700 1 --2- 172.21.15.52:0/2096770107 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55a9253ee750 0x55a9253f0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.970 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.956+0000 7f2152ff5700 10 client.17139 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:33.971 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.956+0000 7f216effd700 1 --2- 172.21.15.52:0/2096770107 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55a9253ee750 0x55a9253f0b60 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f794499e72a6d84 server_cookie=b3acf991f3b89fb2 in_seq=0 out_seq=0 2023-11-27T15:27:33.971 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.956+0000 7f2152ff5700 1 -- 172.21.15.52:0/2096770107 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a9252e40a0 con 0x55a9253ee750 2023-11-27T15:27:33.971 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.956+0000 7f2152ff5700 10 client.17139 handle_command_reply: tid=0 2023-11-27T15:27:33.971 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.957+0000 7f21815a5b80 1 client.17139 shutdown 2023-11-27T15:27:33.971 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.957+0000 7f21815a5b80 1 -- 172.21.15.52:0/2096770107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a9253df530 msgr2=0x55a9253ea980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.971 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.957+0000 7f21815a5b80 1 --2- 172.21.15.52:0/2096770107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a9253df530 0x55a9253ea980 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f21680ca2d0 tx=0x7f21680c9cc0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.971 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.957+0000 7f21815a5b80 1 -- 172.21.15.52:0/2096770107 shutdown_connections 2023-11-27T15:27:33.971 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.957+0000 7f21815a5b80 1 --2- 172.21.15.52:0/2096770107 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55a9253ee750 0x55a9253f0b60 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.972 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.957+0000 7f21815a5b80 1 --2- 172.21.15.52:0/2096770107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9253d0970 0x55a9253df900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.972 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.957+0000 7f2152ff5700 0 client.17139 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:33.972 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.957+0000 7f21815a5b80 1 --2- 172.21.15.52:0/2096770107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a9253df530 0x55a9253ea980 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.972 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.957+0000 7f21815a5b80 1 --2- 172.21.15.52:0/2096770107 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9253d1280 0x55a9253dfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.972 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.957+0000 7f21815a5b80 1 -- 172.21.15.52:0/2096770107 >> 172.21.15.52:0/2096770107 conn(0x55a9253c6360 msgr2=0x55a9253cade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:33.972 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.957+0000 7f21815a5b80 1 -- 172.21.15.52:0/2096770107 shutdown_connections 2023-11-27T15:27:33.972 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.957+0000 7f21815a5b80 1 -- 172.21.15.52:0/2096770107 wait complete. 2023-11-27T15:27:33.972 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.957+0000 7f21815a5b80 20 client.17139 trim_cache size 0 max 16384 2023-11-27T15:27:33.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.970+0000 7f21815a5b80 1 Processor -- start 2023-11-27T15:27:33.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.970+0000 7f21815a5b80 1 -- start start 2023-11-27T15:27:33.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.970+0000 7f21815a5b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9253d0970 0x55a9253d8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.970+0000 7f21815a5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a9253d1280 0x55a9253cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.970+0000 7f21815a5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9253df530 0x55a9253cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.970+0000 7f21815a5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9253de780 con 0x55a9253d1280 2023-11-27T15:27:33.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.970+0000 7f21815a5b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9253e04a0 con 0x55a9253d0970 2023-11-27T15:27:33.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.970+0000 7f21815a5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9253deca0 con 0x55a9253df530 2023-11-27T15:27:33.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.970+0000 7f216e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a9253d1280 0x55a9253cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.980 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.970+0000 7f216e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a9253d1280 0x55a9253cb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56110/0 (socket says 172.21.15.52:56110) 2023-11-27T15:27:33.980 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.970+0000 7f216e7fc700 1 -- 172.21.15.52:0/2055121998 learned_addr learned my addr 172.21.15.52:0/2055121998 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:33.980 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.970+0000 7f216f7fe700 1 --2- 172.21.15.52:0/2055121998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9253df530 0x55a9253cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.980 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.970+0000 7f216effd700 1 --2- 172.21.15.52:0/2055121998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9253d0970 0x55a9253d8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.980 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.970+0000 7f216e7fc700 1 -- 172.21.15.52:0/2055121998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9253df530 msgr2=0x55a9253cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.980 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.970+0000 7f216e7fc700 1 --2- 172.21.15.52:0/2055121998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9253df530 0x55a9253cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.980 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.970+0000 7f216e7fc700 1 -- 172.21.15.52:0/2055121998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9253d0970 msgr2=0x55a9253d8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.980 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.970+0000 7f216e7fc700 1 --2- 172.21.15.52:0/2055121998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9253d0970 0x55a9253d8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.980 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.970+0000 7f216e7fc700 1 -- 172.21.15.52:0/2055121998 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9253de190 con 0x55a9253d1280 2023-11-27T15:27:33.981 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.971+0000 7f216e7fc700 1 --2- 172.21.15.52:0/2055121998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a9253d1280 0x55a9253cb560 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f2164009990 tx=0x7f2164005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d1c0848426c204 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:33.981 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.971+0000 7f2155ffb700 1 -- 172.21.15.52:0/2055121998 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216400b9d0 con 0x55a9253d1280 2023-11-27T15:27:33.981 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.971+0000 7f2155ffb700 1 -- 172.21.15.52:0/2055121998 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f216401e070 con 0x55a9253d1280 2023-11-27T15:27:33.981 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.971+0000 7f2155ffb700 1 -- 172.21.15.52:0/2055121998 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216400f460 con 0x55a9253d1280 2023-11-27T15:27:33.981 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.971+0000 7f21815a5b80 1 -- 172.21.15.52:0/2055121998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a9253d1280 msgr2=0x55a9253cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.981 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.971+0000 7f21815a5b80 1 --2- 172.21.15.52:0/2055121998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a9253d1280 0x55a9253cb560 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f2164009990 tx=0x7f2164005fb0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.981 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.971+0000 7f21815a5b80 1 -- 172.21.15.52:0/2055121998 shutdown_connections 2023-11-27T15:27:33.981 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.971+0000 7f21815a5b80 1 --2- 172.21.15.52:0/2055121998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9253d0970 0x55a9253d8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.982 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.971+0000 7f21815a5b80 1 --2- 172.21.15.52:0/2055121998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9253df530 0x55a9253cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.982 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.971+0000 7f21815a5b80 1 --2- 172.21.15.52:0/2055121998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a9253d1280 0x55a9253cb560 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.982 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.971+0000 7f21815a5b80 1 -- 172.21.15.52:0/2055121998 >> 172.21.15.52:0/2055121998 conn(0x55a9253c6360 msgr2=0x55a9253d9920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:33.982 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.971+0000 7f21815a5b80 1 -- 172.21.15.52:0/2055121998 shutdown_connections 2023-11-27T15:27:33.982 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.971+0000 7f21815a5b80 1 -- 172.21.15.52:0/2055121998 wait complete. 2023-11-27T15:27:33.982 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.972+0000 7f21815a5b80 1 -- start start 2023-11-27T15:27:33.982 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.972+0000 7f21815a5b80 1 Processor -- start 2023-11-27T15:27:33.982 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.972+0000 7f21815a5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a9253d0970 0x55a9253e0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.983 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.972+0000 7f21815a5b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9253d1280 0x55a9253e0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.983 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.972+0000 7f21815a5b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9253df530 0x55a9253e1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.983 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.972+0000 7f21815a5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a925441630 con 0x55a9253d0970 2023-11-27T15:27:33.983 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.972+0000 7f21815a5b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9254417a0 con 0x55a9253df530 2023-11-27T15:27:33.983 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.972+0000 7f21815a5b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a925441960 con 0x55a9253d1280 2023-11-27T15:27:33.983 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.972+0000 7f21815a5b80 10 client.0 _ll_register_callbacks cb 0x7f2168099440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:33.983 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.972+0000 7f216effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a9253d0970 0x55a9253e0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.983 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.972+0000 7f216effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a9253d0970 0x55a9253e0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56114/0 (socket says 172.21.15.52:56114) 2023-11-27T15:27:33.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.972+0000 7f216effd700 1 -- 172.21.15.52:0/294703244 learned_addr learned my addr 172.21.15.52:0/294703244 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:33.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.972+0000 7f216f7fe700 1 --2- 172.21.15.52:0/294703244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9253df530 0x55a9253e1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.972+0000 7f216e7fc700 1 --2- 172.21.15.52:0/294703244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9253d1280 0x55a9253e0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.972+0000 7f216effd700 1 -- 172.21.15.52:0/294703244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9253d1280 msgr2=0x55a9253e0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.972+0000 7f216effd700 1 --2- 172.21.15.52:0/294703244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9253d1280 0x55a9253e0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.972+0000 7f216effd700 1 -- 172.21.15.52:0/294703244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9253df530 msgr2=0x55a9253e1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:33.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.972+0000 7f216effd700 1 --2- 172.21.15.52:0/294703244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9253df530 0x55a9253e1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.972+0000 7f216effd700 1 -- 172.21.15.52:0/294703244 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f217010c8b0 con 0x55a9253d0970 2023-11-27T15:27:33.985 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.972+0000 7f216e7fc700 1 --2- 172.21.15.52:0/294703244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9253d1280 0x55a9253e0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:33.985 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.973+0000 7f216effd700 1 --2- 172.21.15.52:0/294703244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a9253d0970 0x55a9253e0780 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x55a9253f1400 tx=0x7f21600047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0bf39d1f1dfef31 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:33.985 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.973+0000 7f2152ff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:27:33.985 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.973+0000 7f2152ff5700 1 -- 172.21.15.52:0/294703244 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216002a070 con 0x55a9253d0970 2023-11-27T15:27:33.985 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.973+0000 7f2152ff5700 1 -- 172.21.15.52:0/294703244 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f2160017070 con 0x55a9253d0970 2023-11-27T15:27:33.985 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.973+0000 7f21815a5b80 1 -- 172.21.15.52:0/294703244 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f21680a1aa0 con 0x55a9253d0970 2023-11-27T15:27:33.985 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.973+0000 7f2152ff5700 1 -- 172.21.15.52:0/294703244 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2160025070 con 0x55a9253d0970 2023-11-27T15:27:33.985 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.973+0000 7f21815a5b80 10 client.17145 fetch_fsmap learned FSMap version 34 2023-11-27T15:27:33.985 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.973+0000 7f21815a5b80 1 -- 172.21.15.52:0/294703244 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55a92514fde0 con 0x55a9253d0970 2023-11-27T15:27:33.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.973+0000 7f2152ff5700 1 -- 172.21.15.52:0/294703244 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21680a1aa0 con 0x55a9253d0970 2023-11-27T15:27:33.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.973+0000 7f2152ff5700 1 -- 172.21.15.52:0/294703244 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f216002d020 con 0x55a9253d0970 2023-11-27T15:27:33.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.973+0000 7f21815a5b80 10 client.17145 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:27:33.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.973+0000 7f21815a5b80 10 client.17145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:33.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.973+0000 7f21815a5b80 20 client.17145 populate_metadata read hostname 'smithi052' 2023-11-27T15:27:33.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.973+0000 7f21815a5b80 1 --2- 172.21.15.52:0/294703244 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55a9254430d0 0x55a9254454e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:33.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.973+0000 7f21815a5b80 4 client.17145 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:27:33.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.973+0000 7f21815a5b80 1 -- 172.21.15.52:0/294703244 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a9252f0660 con 0x55a9254430d0 2023-11-27T15:27:33.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.974+0000 7f216e7fc700 1 --2- 172.21.15.52:0/294703244 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55a9254430d0 0x55a9254454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:33.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.974+0000 7f2152ff5700 10 client.17145 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:33.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.974+0000 7f216e7fc700 1 --2- 172.21.15.52:0/294703244 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55a9254430d0 0x55a9254454e0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a10d908bfee61edc server_cookie=f74de73dd9dd61bb in_seq=0 out_seq=0 2023-11-27T15:27:33.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.974+0000 7f2152ff5700 1 -- 172.21.15.52:0/294703244 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a9252f0660 con 0x55a9254430d0 2023-11-27T15:27:33.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.974+0000 7f2152ff5700 10 client.17145 handle_command_reply: tid=0 2023-11-27T15:27:33.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.974+0000 7f21815a5b80 1 client.17145 shutdown 2023-11-27T15:27:33.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.974+0000 7f21815a5b80 1 -- 172.21.15.52:0/294703244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a9253d0970 msgr2=0x55a9253e0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.974+0000 7f21815a5b80 1 --2- 172.21.15.52:0/294703244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a9253d0970 0x55a9253e0780 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x55a9253f1400 tx=0x7f21600047c0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.987 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.974+0000 7f21815a5b80 1 -- 172.21.15.52:0/294703244 shutdown_connections 2023-11-27T15:27:33.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.974+0000 7f21815a5b80 1 --2- 172.21.15.52:0/294703244 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55a9254430d0 0x55a9254454e0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.974+0000 7f21815a5b80 1 --2- 172.21.15.52:0/294703244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a9253df530 0x55a9253e1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.974+0000 7f2152ff5700 0 client.17145 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:33.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.974+0000 7f21815a5b80 1 --2- 172.21.15.52:0/294703244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a9253d0970 0x55a9253e0780 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.974+0000 7f21815a5b80 1 --2- 172.21.15.52:0/294703244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9253d1280 0x55a9253e0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.974+0000 7f21815a5b80 1 -- 172.21.15.52:0/294703244 >> 172.21.15.52:0/294703244 conn(0x55a9253c6360 msgr2=0x55a9253cf960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:33.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.974+0000 7f21815a5b80 1 -- 172.21.15.52:0/294703244 shutdown_connections 2023-11-27T15:27:33.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.974+0000 7f21815a5b80 1 -- 172.21.15.52:0/294703244 wait complete. 2023-11-27T15:27:33.988 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.975+0000 7f21815a5b80 20 client.17145 trim_cache size 0 max 16384 2023-11-27T15:27:33.989 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:27:33.989 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.975+0000 7f2175419700 1 -- 172.21.15.52:0/3085843332 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f21580a12c0 msgr2=0x7f21580a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.990 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.975+0000 7f2175419700 1 --2- 172.21.15.52:0/3085843332 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f21580a12c0 0x7f21580a3770 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2160005950 tx=0x7f216000b410 comp rx=0 tx=0).stop 2023-11-27T15:27:33.990 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.975+0000 7f2175419700 1 -- 172.21.15.52:0/3085843332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2170105210 msgr2=0x7f2170199d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:33.990 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.975+0000 7f2175419700 1 --2- 172.21.15.52:0/3085843332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2170105210 0x7f2170199d80 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f2168007ee0 tx=0x7f216800d3b0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.990 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.975+0000 7f2175419700 1 -- 172.21.15.52:0/3085843332 shutdown_connections 2023-11-27T15:27:33.990 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.975+0000 7f2175419700 1 --2- 172.21.15.52:0/3085843332 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f21580a12c0 0x7f21580a3770 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.990 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.975+0000 7f2175419700 1 --2- 172.21.15.52:0/3085843332 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f21700ffba0 0x7f21701953e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.990 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.975+0000 7f2175419700 1 --2- 172.21.15.52:0/3085843332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2170105210 0x7f2170199d80 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.991 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.975+0000 7f2175419700 1 --2- 172.21.15.52:0/3085843332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2170100530 0x7f2170195920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:33.991 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.975+0000 7f2175419700 1 -- 172.21.15.52:0/3085843332 >> 172.21.15.52:0/3085843332 conn(0x7f21700748f0 msgr2=0x7f21700fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:33.991 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.975+0000 7f2175419700 1 -- 172.21.15.52:0/3085843332 shutdown_connections 2023-11-27T15:27:33.991 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:33.975+0000 7f2175419700 1 -- 172.21.15.52:0/3085843332 wait complete. 2023-11-27T15:27:34.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:34 smithi150 ceph-mon[100559]: pgmap v300: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:27:34.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:34 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1288987755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:27:34.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:34 smithi102 ceph-mon[100296]: pgmap v300: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:27:34.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:34 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1288987755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:27:34.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:34 smithi052 ceph-mon[99056]: pgmap v300: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:27:34.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:34 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1288987755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:27:34.992 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-27T15:27:34.993 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-27T15:27:35.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.134+0000 7f0be82ad700 1 -- 172.21.15.52:0/372075368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0be00ff450 msgr2=0x7f0be00ff820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:35.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.134+0000 7f0be82ad700 1 --2- 172.21.15.52:0/372075368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0be00ff450 0x7f0be00ff820 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc0099c0 tx=0x7f0bdc009cd0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.134+0000 7f0be82ad700 1 -- 172.21.15.52:0/372075368 shutdown_connections 2023-11-27T15:27:35.138 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.134+0000 7f0be82ad700 1 --2- 172.21.15.52:0/372075368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0be0104490 0x7f0be010c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.134+0000 7f0be82ad700 1 --2- 172.21.15.52:0/372075368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0be00ff450 0x7f0be00ff820 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.134+0000 7f0be82ad700 1 --2- 172.21.15.52:0/372075368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0be00ffdf0 0x7f0be0103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.134+0000 7f0be82ad700 1 -- 172.21.15.52:0/372075368 >> 172.21.15.52:0/372075368 conn(0x7f0be00770b0 msgr2=0x7f0be00774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:35.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.134+0000 7f0be82ad700 1 -- 172.21.15.52:0/372075368 shutdown_connections 2023-11-27T15:27:35.139 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.134+0000 7f0be82ad700 1 -- 172.21.15.52:0/372075368 wait complete. 2023-11-27T15:27:35.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.135+0000 7f0be82ad700 1 Processor -- start 2023-11-27T15:27:35.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.135+0000 7f0be82ad700 1 -- start start 2023-11-27T15:27:35.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.135+0000 7f0be82ad700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0be00ff450 0x7f0be01963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.141 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.135+0000 7f0be82ad700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0be00ffdf0 0x7f0be0196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.135+0000 7f0be82ad700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0be0104490 0x7f0be019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.135+0000 7f0be82ad700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be019b3b0 con 0x7f0be00ffdf0 2023-11-27T15:27:35.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.135+0000 7f0be82ad700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be019b520 con 0x7f0be0104490 2023-11-27T15:27:35.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.135+0000 7f0be82ad700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be019b6e0 con 0x7f0be00ff450 2023-11-27T15:27:35.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.135+0000 7f0be5848700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0be00ffdf0 0x7f0be0196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.136+0000 7f0be5848700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0be00ffdf0 0x7f0be0196910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56130/0 (socket says 172.21.15.52:56130) 2023-11-27T15:27:35.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.136+0000 7f0be5848700 1 -- 172.21.15.52:0/3736623793 learned_addr learned my addr 172.21.15.52:0/3736623793 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:35.142 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.136+0000 7f0be684a700 1 --2- 172.21.15.52:0/3736623793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0be0104490 0x7f0be019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.136+0000 7f0be6049700 1 --2- 172.21.15.52:0/3736623793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0be00ff450 0x7f0be01963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.136+0000 7f0be5848700 1 -- 172.21.15.52:0/3736623793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0be00ff450 msgr2=0x7f0be01963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:35.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.136+0000 7f0be5848700 1 --2- 172.21.15.52:0/3736623793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0be00ff450 0x7f0be01963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.136+0000 7f0be5848700 1 -- 172.21.15.52:0/3736623793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0be0104490 msgr2=0x7f0be019ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:35.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.136+0000 7f0be5848700 1 --2- 172.21.15.52:0/3736623793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0be0104490 0x7f0be019ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.136+0000 7f0be5848700 1 -- 172.21.15.52:0/3736623793 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bdc0095f0 con 0x7f0be00ffdf0 2023-11-27T15:27:35.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.136+0000 7f0be684a700 1 --2- 172.21.15.52:0/3736623793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0be0104490 0x7f0be019ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:35.143 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.136+0000 7f0be5848700 1 --2- 172.21.15.52:0/3736623793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0be00ffdf0 0x7f0be0196910 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400d950 tx=0x7f0bd400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:35.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.136+0000 7f0bd37fe700 1 -- 172.21.15.52:0/3736623793 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4009b40 con 0x7f0be00ffdf0 2023-11-27T15:27:35.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.136+0000 7f0bd37fe700 1 -- 172.21.15.52:0/3736623793 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bd400de30 con 0x7f0be00ffdf0 2023-11-27T15:27:35.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.136+0000 7f0bd37fe700 1 -- 172.21.15.52:0/3736623793 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd400f620 con 0x7f0be00ffdf0 2023-11-27T15:27:35.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.136+0000 7f0be82ad700 1 -- 172.21.15.52:0/3736623793 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be010f250 con 0x7f0be00ffdf0 2023-11-27T15:27:35.144 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.136+0000 7f0be82ad700 1 -- 172.21.15.52:0/3736623793 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be010f0c0 con 0x7f0be00ffdf0 2023-11-27T15:27:35.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.139+0000 7f0bd37fe700 1 -- 172.21.15.52:0/3736623793 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0bd4010460 con 0x7f0be00ffdf0 2023-11-27T15:27:35.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.139+0000 7f0bd37fe700 1 --2- 172.21.15.52:0/3736623793 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f0bcc0a12c0 0x7f0bcc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.150 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.139+0000 7f0be6049700 1 --2- 172.21.15.52:0/3736623793 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f0bcc0a12c0 0x7f0bcc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.139+0000 7f0bd37fe700 1 -- 172.21.15.52:0/3736623793 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f0bd40cc300 con 0x7f0be00ffdf0 2023-11-27T15:27:35.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.140+0000 7f0be6049700 1 --2- 172.21.15.52:0/3736623793 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f0bcc0a12c0 0x7f0bcc0a3770 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc009990 tx=0x7f0bdc01a040 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:35.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.140+0000 7f0bf4439b80 1 Processor -- start 2023-11-27T15:27:35.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.140+0000 7f0bf4439b80 1 -- start start 2023-11-27T15:27:35.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.141+0000 7f0bf4439b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f196ddbe0 0x562f196ddfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.141+0000 7f0bf4439b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f196de4f0 0x562f196e23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.141+0000 7f0bf4439b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562f196e2930 0x562f196e2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.151 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.141+0000 7f0bf4439b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f195e30e0 con 0x562f196e2930 2023-11-27T15:27:35.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.141+0000 7f0bf4439b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f19635920 con 0x562f196de4f0 2023-11-27T15:27:35.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.141+0000 7f0bf4439b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f1962edb0 con 0x562f196ddbe0 2023-11-27T15:27:35.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.141+0000 7f0be684a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562f196e2930 0x562f196e2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.141+0000 7f0be684a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562f196e2930 0x562f196e2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56140/0 (socket says 172.21.15.52:56140) 2023-11-27T15:27:35.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.141+0000 7f0be684a700 1 -- 172.21.15.52:0/2886571967 learned_addr learned my addr 172.21.15.52:0/2886571967 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:35.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.141+0000 7f0be6049700 1 --2- 172.21.15.52:0/2886571967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f196ddbe0 0x562f196ddfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.141+0000 7f0be5848700 1 --2- 172.21.15.52:0/2886571967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f196de4f0 0x562f196e23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.152 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.141+0000 7f0be684a700 1 -- 172.21.15.52:0/2886571967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f196ddbe0 msgr2=0x562f196ddfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:35.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.141+0000 7f0be684a700 1 --2- 172.21.15.52:0/2886571967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f196ddbe0 0x562f196ddfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.141+0000 7f0be684a700 1 -- 172.21.15.52:0/2886571967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f196de4f0 msgr2=0x562f196e23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:35.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.141+0000 7f0be684a700 1 --2- 172.21.15.52:0/2886571967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f196de4f0 0x562f196e23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.141+0000 7f0be684a700 1 -- 172.21.15.52:0/2886571967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f194b06b0 con 0x562f196e2930 2023-11-27T15:27:35.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.141+0000 7f0be684a700 1 --2- 172.21.15.52:0/2886571967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562f196e2930 0x562f196e2d80 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8009440 tx=0x7f0bd8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f54403cab3c379c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:35.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.141+0000 7f0bd0dac700 1 -- 172.21.15.52:0/2886571967 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd800e070 con 0x562f196e2930 2023-11-27T15:27:35.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.142+0000 7f0bd0dac700 1 -- 172.21.15.52:0/2886571967 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bd800f040 con 0x562f196e2930 2023-11-27T15:27:35.153 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.142+0000 7f0bd0dac700 1 -- 172.21.15.52:0/2886571967 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8014a70 con 0x562f196e2930 2023-11-27T15:27:35.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.142+0000 7f0bf4439b80 1 -- 172.21.15.52:0/2886571967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562f196e2930 msgr2=0x562f196e2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:35.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.142+0000 7f0bf4439b80 1 --2- 172.21.15.52:0/2886571967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562f196e2930 0x562f196e2d80 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8009440 tx=0x7f0bd8009920 comp rx=0 tx=0).stop 2023-11-27T15:27:35.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.142+0000 7f0bf4439b80 1 -- 172.21.15.52:0/2886571967 shutdown_connections 2023-11-27T15:27:35.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.142+0000 7f0bf4439b80 1 --2- 172.21.15.52:0/2886571967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562f196e2930 0x562f196e2d80 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.142+0000 7f0bf4439b80 1 --2- 172.21.15.52:0/2886571967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f196ddbe0 0x562f196ddfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.142+0000 7f0bf4439b80 1 --2- 172.21.15.52:0/2886571967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f196de4f0 0x562f196e23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.142+0000 7f0bf4439b80 1 -- 172.21.15.52:0/2886571967 >> 172.21.15.52:0/2886571967 conn(0x562f196d33e0 msgr2=0x562f196d8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:35.154 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.142+0000 7f0bf4439b80 1 -- 172.21.15.52:0/2886571967 shutdown_connections 2023-11-27T15:27:35.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.142+0000 7f0bf4439b80 1 -- 172.21.15.52:0/2886571967 wait complete. 2023-11-27T15:27:35.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.143+0000 7f0bf4439b80 1 -- start start 2023-11-27T15:27:35.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.143+0000 7f0bf4439b80 1 Processor -- start 2023-11-27T15:27:35.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.143+0000 7f0bf4439b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562f196ddbe0 0x562f196ea0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.143+0000 7f0bf4439b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f196de4f0 0x562f196e72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.143+0000 7f0bf4439b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f196e2930 0x562f196e78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.143+0000 7f0bf4439b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f195d0960 con 0x562f196ddbe0 2023-11-27T15:27:35.155 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.143+0000 7f0bf4439b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f195d0600 con 0x562f196de4f0 2023-11-27T15:27:35.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.143+0000 7f0bf4439b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f194b29a0 con 0x562f196e2930 2023-11-27T15:27:35.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.143+0000 7f0bf4439b80 10 client.0 _ll_register_callbacks cb 0x562f195ee800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:35.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.143+0000 7f0be6049700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562f196ddbe0 0x562f196ea0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.143+0000 7f0be6049700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562f196ddbe0 0x562f196ea0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56142/0 (socket says 172.21.15.52:56142) 2023-11-27T15:27:35.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.143+0000 7f0be6049700 1 -- 172.21.15.52:0/4026421389 learned_addr learned my addr 172.21.15.52:0/4026421389 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:35.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.143+0000 7f0be5848700 1 --2- 172.21.15.52:0/4026421389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f196de4f0 0x562f196e72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.143+0000 7f0be684a700 1 --2- 172.21.15.52:0/4026421389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f196e2930 0x562f196e78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.156 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.144+0000 7f0be6049700 1 -- 172.21.15.52:0/4026421389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f196e2930 msgr2=0x562f196e78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:35.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.144+0000 7f0be6049700 1 --2- 172.21.15.52:0/4026421389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f196e2930 0x562f196e78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.144+0000 7f0be6049700 1 -- 172.21.15.52:0/4026421389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f196de4f0 msgr2=0x562f196e72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:35.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.144+0000 7f0be6049700 1 --2- 172.21.15.52:0/4026421389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f196de4f0 0x562f196e72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.144+0000 7f0be6049700 1 -- 172.21.15.52:0/4026421389 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd80090f0 con 0x562f196ddbe0 2023-11-27T15:27:35.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.144+0000 7f0be5848700 1 --2- 172.21.15.52:0/4026421389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f196de4f0 0x562f196e72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:35.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.144+0000 7f0be6049700 1 --2- 172.21.15.52:0/4026421389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562f196ddbe0 0x562f196ea0c0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc0058e0 tx=0x7f0bdc009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50292e683ea7fae2 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:35.157 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.144+0000 7f0bc9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:27:35.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.144+0000 7f0bc9ffb700 1 -- 172.21.15.52:0/4026421389 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc01d040 con 0x562f196ddbe0 2023-11-27T15:27:35.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.144+0000 7f0bc9ffb700 1 -- 172.21.15.52:0/4026421389 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f0bdc023820 con 0x562f196ddbe0 2023-11-27T15:27:35.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.144+0000 7f0bf4439b80 1 -- 172.21.15.52:0/4026421389 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f196d0ba0 con 0x562f196ddbe0 2023-11-27T15:27:35.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.144+0000 7f0bc9ffb700 1 -- 172.21.15.52:0/4026421389 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bdc0239e0 con 0x562f196ddbe0 2023-11-27T15:27:35.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.144+0000 7f0bf4439b80 10 client.17157 fetch_fsmap learned FSMap version 34 2023-11-27T15:27:35.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.144+0000 7f0bf4439b80 1 -- 172.21.15.52:0/4026421389 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x562f195f5210 con 0x562f196ddbe0 2023-11-27T15:27:35.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.144+0000 7f0bc9ffb700 1 -- 172.21.15.52:0/4026421389 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bdc03c020 con 0x562f196ddbe0 2023-11-27T15:27:35.158 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.144+0000 7f0bc9ffb700 1 -- 172.21.15.52:0/4026421389 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0bdc02fbe0 con 0x562f196ddbe0 2023-11-27T15:27:35.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.144+0000 7f0bf4439b80 10 client.17157 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:27:35.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.144+0000 7f0bf4439b80 10 client.17157 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:35.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.145+0000 7f0bf4439b80 20 client.17157 populate_metadata read hostname 'smithi052' 2023-11-27T15:27:35.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.145+0000 7f0bf4439b80 1 --2- 172.21.15.52:0/4026421389 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x562f196f78a0 0x562f196f9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.145+0000 7f0bf4439b80 4 client.17157 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:27:35.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.145+0000 7f0bf4439b80 1 -- 172.21.15.52:0/4026421389 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f195f1120 con 0x562f196f78a0 2023-11-27T15:27:35.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.145+0000 7f0be5848700 1 --2- 172.21.15.52:0/4026421389 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x562f196f78a0 0x562f196f9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.159 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.145+0000 7f0bc9ffb700 10 client.17157 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:35.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.145+0000 7f0be5848700 1 --2- 172.21.15.52:0/4026421389 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x562f196f78a0 0x562f196f9c80 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba3ceda7033bf37f server_cookie=94882efb9f6aab19 in_seq=0 out_seq=0 2023-11-27T15:27:35.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.146+0000 7f0bc9ffb700 1 -- 172.21.15.52:0/4026421389 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562f195f1120 con 0x562f196f78a0 2023-11-27T15:27:35.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.146+0000 7f0bc9ffb700 10 client.17157 handle_command_reply: tid=0 2023-11-27T15:27:35.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.146+0000 7f0bf4439b80 1 client.17157 shutdown 2023-11-27T15:27:35.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.146+0000 7f0bf4439b80 1 -- 172.21.15.52:0/4026421389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562f196ddbe0 msgr2=0x562f196ea0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:35.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.146+0000 7f0bf4439b80 1 --2- 172.21.15.52:0/4026421389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562f196ddbe0 0x562f196ea0c0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc0058e0 tx=0x7f0bdc009cd0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.146+0000 7f0bf4439b80 1 -- 172.21.15.52:0/4026421389 shutdown_connections 2023-11-27T15:27:35.160 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.146+0000 7f0bf4439b80 1 --2- 172.21.15.52:0/4026421389 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x562f196f78a0 0x562f196f9c80 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.146+0000 7f0bf4439b80 1 --2- 172.21.15.52:0/4026421389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f196e2930 0x562f196e78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.146+0000 7f0bc9ffb700 0 client.17157 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:35.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.146+0000 7f0bf4439b80 1 --2- 172.21.15.52:0/4026421389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562f196ddbe0 0x562f196ea0c0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.146+0000 7f0bf4439b80 1 --2- 172.21.15.52:0/4026421389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f196de4f0 0x562f196e72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.146+0000 7f0bf4439b80 1 -- 172.21.15.52:0/4026421389 >> 172.21.15.52:0/4026421389 conn(0x562f196d33e0 msgr2=0x562f196e3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:35.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.146+0000 7f0bf4439b80 1 -- 172.21.15.52:0/4026421389 shutdown_connections 2023-11-27T15:27:35.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.146+0000 7f0bf4439b80 1 -- 172.21.15.52:0/4026421389 wait complete. 2023-11-27T15:27:35.161 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.146+0000 7f0bf4439b80 20 client.17157 trim_cache size 0 max 16384 2023-11-27T15:27:35.171 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.163+0000 7f0bf4439b80 1 Processor -- start 2023-11-27T15:27:35.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.164+0000 7f0bf4439b80 1 -- start start 2023-11-27T15:27:35.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.164+0000 7f0bf4439b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562f196ddbe0 0x562f196e7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.164+0000 7f0bf4439b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f196de4f0 0x562f196eea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.164+0000 7f0bf4439b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f196e2930 0x562f196e9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.164+0000 7f0bf4439b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f196e7ec0 con 0x562f196ddbe0 2023-11-27T15:27:35.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.164+0000 7f0bf4439b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f196e5850 con 0x562f196de4f0 2023-11-27T15:27:35.172 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.164+0000 7f0bf4439b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f196e1440 con 0x562f196e2930 2023-11-27T15:27:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.164+0000 7f0be6049700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562f196ddbe0 0x562f196e7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.164+0000 7f0be6049700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562f196ddbe0 0x562f196e7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56150/0 (socket says 172.21.15.52:56150) 2023-11-27T15:27:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.164+0000 7f0be6049700 1 -- 172.21.15.52:0/3519899565 learned_addr learned my addr 172.21.15.52:0/3519899565 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.164+0000 7f0be5848700 1 --2- 172.21.15.52:0/3519899565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f196de4f0 0x562f196eea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.164+0000 7f0be684a700 1 --2- 172.21.15.52:0/3519899565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f196e2930 0x562f196e9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.164+0000 7f0be6049700 1 -- 172.21.15.52:0/3519899565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f196e2930 msgr2=0x562f196e9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.164+0000 7f0be6049700 1 --2- 172.21.15.52:0/3519899565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f196e2930 0x562f196e9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.164+0000 7f0be6049700 1 -- 172.21.15.52:0/3519899565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f196de4f0 msgr2=0x562f196eea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.164+0000 7f0be6049700 1 --2- 172.21.15.52:0/3519899565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f196de4f0 0x562f196eea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.164+0000 7f0be6049700 1 -- 172.21.15.52:0/3519899565 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f195f4f60 con 0x562f196ddbe0 2023-11-27T15:27:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.164+0000 7f0be684a700 1 --2- 172.21.15.52:0/3519899565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f196e2930 0x562f196e9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.164+0000 7f0be5848700 1 --2- 172.21.15.52:0/3519899565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f196de4f0 0x562f196eea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.164+0000 7f0be6049700 1 --2- 172.21.15.52:0/3519899565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562f196ddbe0 0x562f196e7280 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc00a010 tx=0x7f0bdc0092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1bbd3e37dbb6dc3 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.165+0000 7f0bd15ad700 1 -- 172.21.15.52:0/3519899565 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc01d070 con 0x562f196ddbe0 2023-11-27T15:27:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.165+0000 7f0bd15ad700 1 -- 172.21.15.52:0/3519899565 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bdc03c030 con 0x562f196ddbe0 2023-11-27T15:27:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.165+0000 7f0bd15ad700 1 -- 172.21.15.52:0/3519899565 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc023a60 con 0x562f196ddbe0 2023-11-27T15:27:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.165+0000 7f0bf4439b80 1 -- 172.21.15.52:0/3519899565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562f196ddbe0 msgr2=0x562f196e7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.165+0000 7f0bf4439b80 1 --2- 172.21.15.52:0/3519899565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562f196ddbe0 0x562f196e7280 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc00a010 tx=0x7f0bdc0092f0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.165+0000 7f0bf4439b80 1 -- 172.21.15.52:0/3519899565 shutdown_connections 2023-11-27T15:27:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.165+0000 7f0bf4439b80 1 --2- 172.21.15.52:0/3519899565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f196e2930 0x562f196e9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.165+0000 7f0bf4439b80 1 --2- 172.21.15.52:0/3519899565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562f196ddbe0 0x562f196e7280 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.165+0000 7f0bf4439b80 1 --2- 172.21.15.52:0/3519899565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f196de4f0 0x562f196eea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.165+0000 7f0bf4439b80 1 -- 172.21.15.52:0/3519899565 >> 172.21.15.52:0/3519899565 conn(0x562f196d33e0 msgr2=0x562f196e6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:35.176 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.165+0000 7f0bf4439b80 1 -- 172.21.15.52:0/3519899565 shutdown_connections 2023-11-27T15:27:35.176 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.165+0000 7f0bf4439b80 1 -- 172.21.15.52:0/3519899565 wait complete. 2023-11-27T15:27:35.176 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.165+0000 7f0bf4439b80 1 -- start start 2023-11-27T15:27:35.176 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.166+0000 7f0bf4439b80 1 Processor -- start 2023-11-27T15:27:35.176 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.166+0000 7f0bf4439b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f196ddbe0 0x562f196ed2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.176 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.166+0000 7f0bf4439b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f196de4f0 0x562f196ed820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.176 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.166+0000 7f0bf4439b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562f196e2930 0x562f196edd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.176 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.166+0000 7f0bf4439b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f1974b870 con 0x562f196e2930 2023-11-27T15:27:35.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.166+0000 7f0bf4439b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f1974b9b0 con 0x562f196de4f0 2023-11-27T15:27:35.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.166+0000 7f0bf4439b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f1974bb70 con 0x562f196ddbe0 2023-11-27T15:27:35.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.166+0000 7f0bf4439b80 10 client.0 _ll_register_callbacks cb 0x7f0bdc00bf90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:35.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.166+0000 7f0be684a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562f196e2930 0x562f196edd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.166+0000 7f0be684a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562f196e2930 0x562f196edd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56160/0 (socket says 172.21.15.52:56160) 2023-11-27T15:27:35.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.166+0000 7f0be684a700 1 -- 172.21.15.52:0/1324449903 learned_addr learned my addr 172.21.15.52:0/1324449903 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:35.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.166+0000 7f0be5848700 1 --2- 172.21.15.52:0/1324449903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f196de4f0 0x562f196ed820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.166+0000 7f0be6049700 1 --2- 172.21.15.52:0/1324449903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f196ddbe0 0x562f196ed2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.166+0000 7f0be684a700 1 -- 172.21.15.52:0/1324449903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f196ddbe0 msgr2=0x562f196ed2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:35.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.166+0000 7f0be684a700 1 --2- 172.21.15.52:0/1324449903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f196ddbe0 0x562f196ed2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.166+0000 7f0be684a700 1 -- 172.21.15.52:0/1324449903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f196de4f0 msgr2=0x562f196ed820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:35.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.166+0000 7f0be684a700 1 --2- 172.21.15.52:0/1324449903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f196de4f0 0x562f196ed820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.166+0000 7f0be684a700 1 -- 172.21.15.52:0/1324449903 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f195f5210 con 0x562f196e2930 2023-11-27T15:27:35.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.166+0000 7f0be684a700 1 --2- 172.21.15.52:0/1324449903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562f196e2930 0x562f196edd60 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8007ac0 tx=0x7f0bd8014830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4d784e70f7c39c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:35.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.166+0000 7f0bc9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:27:35.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.167+0000 7f0bc9ffb700 1 -- 172.21.15.52:0/1324449903 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd800e040 con 0x562f196e2930 2023-11-27T15:27:35.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.167+0000 7f0bc9ffb700 1 -- 172.21.15.52:0/1324449903 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f0bd80143e0 con 0x562f196e2930 2023-11-27T15:27:35.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.167+0000 7f0bf4439b80 1 -- 172.21.15.52:0/1324449903 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0bd800f040 con 0x562f196e2930 2023-11-27T15:27:35.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.167+0000 7f0bc9ffb700 1 -- 172.21.15.52:0/1324449903 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bd8014db0 con 0x562f196e2930 2023-11-27T15:27:35.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.167+0000 7f0bf4439b80 10 client.17163 fetch_fsmap learned FSMap version 34 2023-11-27T15:27:35.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.167+0000 7f0bf4439b80 1 -- 172.21.15.52:0/1324449903 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x562f19445ce0 con 0x562f196e2930 2023-11-27T15:27:35.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.167+0000 7f0bc9ffb700 1 -- 172.21.15.52:0/1324449903 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bd800f040 con 0x562f196e2930 2023-11-27T15:27:35.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.167+0000 7f0bc9ffb700 1 -- 172.21.15.52:0/1324449903 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0bd801c070 con 0x562f196e2930 2023-11-27T15:27:35.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.167+0000 7f0bf4439b80 10 client.17163 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:27:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.167+0000 7f0bf4439b80 10 client.17163 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.167+0000 7f0bf4439b80 20 client.17163 populate_metadata read hostname 'smithi052' 2023-11-27T15:27:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.167+0000 7f0bf4439b80 1 --2- 172.21.15.52:0/1324449903 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x562f1974d210 0x562f1974f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.167+0000 7f0bf4439b80 4 client.17163 mds_command: new command op to 24536 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-27T15:27:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.167+0000 7f0bf4439b80 1 -- 172.21.15.52:0/1324449903 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562f195fd6e0 con 0x562f1974d210 2023-11-27T15:27:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.167+0000 7f0be6049700 1 --2- 172.21.15.52:0/1324449903 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x562f1974d210 0x562f1974f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.168+0000 7f0bc9ffb700 10 client.17163 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.168+0000 7f0be6049700 1 --2- 172.21.15.52:0/1324449903 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x562f1974d210 0x562f1974f5f0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd884ac707ad48b8 server_cookie=10b13088ff0e7560 in_seq=0 out_seq=0 2023-11-27T15:27:35.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.169+0000 7f0bc9ffb700 1 -- 172.21.15.52:0/1324449903 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562f195fd6e0 con 0x562f1974d210 2023-11-27T15:27:35.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.169+0000 7f0bc9ffb700 10 client.17163 handle_command_reply: tid=0 2023-11-27T15:27:35.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.169+0000 7f0bf4439b80 1 client.17163 shutdown 2023-11-27T15:27:35.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.169+0000 7f0bf4439b80 1 -- 172.21.15.52:0/1324449903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562f196e2930 msgr2=0x562f196edd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:35.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.169+0000 7f0bf4439b80 1 --2- 172.21.15.52:0/1324449903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562f196e2930 0x562f196edd60 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8007ac0 tx=0x7f0bd8014830 comp rx=0 tx=0).stop 2023-11-27T15:27:35.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.169+0000 7f0bf4439b80 1 -- 172.21.15.52:0/1324449903 shutdown_connections 2023-11-27T15:27:35.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.169+0000 7f0bf4439b80 1 --2- 172.21.15.52:0/1324449903 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x562f1974d210 0x562f1974f5f0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.169+0000 7f0bf4439b80 1 --2- 172.21.15.52:0/1324449903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562f196e2930 0x562f196edd60 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.169+0000 7f0bf4439b80 1 --2- 172.21.15.52:0/1324449903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562f196ddbe0 0x562f196ed2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.169+0000 7f0bf4439b80 1 --2- 172.21.15.52:0/1324449903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f196de4f0 0x562f196ed820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.169+0000 7f0bf4439b80 1 -- 172.21.15.52:0/1324449903 >> 172.21.15.52:0/1324449903 conn(0x562f196d33e0 msgr2=0x562f196e20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:35.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.169+0000 7f0bc9ffb700 0 client.17163 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:35.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.169+0000 7f0bf4439b80 1 -- 172.21.15.52:0/1324449903 shutdown_connections 2023-11-27T15:27:35.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.169+0000 7f0bf4439b80 1 -- 172.21.15.52:0/1324449903 wait complete. 2023-11-27T15:27:35.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.169+0000 7f0bf4439b80 20 client.17163 trim_cache size 0 max 16384 2023-11-27T15:27:35.184 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:27:35.184 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2023-11-27T15:27:35.184 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "2a81c721-a4cb-4be7-94ee-864fe03c385d", 2023-11-27T15:27:35.184 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2023-11-27T15:27:35.184 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:27:35.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.170+0000 7f0be82ad700 1 -- 172.21.15.52:0/3736623793 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f0bcc0a12c0 msgr2=0x7f0bcc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:35.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.170+0000 7f0be82ad700 1 --2- 172.21.15.52:0/3736623793 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f0bcc0a12c0 0x7f0bcc0a3770 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc009990 tx=0x7f0bdc01a040 comp rx=0 tx=0).stop 2023-11-27T15:27:35.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.170+0000 7f0be82ad700 1 -- 172.21.15.52:0/3736623793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0be00ffdf0 msgr2=0x7f0be0196910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:35.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.170+0000 7f0be82ad700 1 --2- 172.21.15.52:0/3736623793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0be00ffdf0 0x7f0be0196910 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400d950 tx=0x7f0bd400dd10 comp rx=0 tx=0).stop 2023-11-27T15:27:35.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.170+0000 7f0be82ad700 1 -- 172.21.15.52:0/3736623793 shutdown_connections 2023-11-27T15:27:35.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.170+0000 7f0be82ad700 1 --2- 172.21.15.52:0/3736623793 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f0bcc0a12c0 0x7f0bcc0a3770 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.170+0000 7f0be82ad700 1 --2- 172.21.15.52:0/3736623793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0be0104490 0x7f0be019ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.170+0000 7f0be82ad700 1 --2- 172.21.15.52:0/3736623793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0be00ff450 0x7f0be01963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.170+0000 7f0be82ad700 1 --2- 172.21.15.52:0/3736623793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0be00ffdf0 0x7f0be0196910 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.170+0000 7f0be82ad700 1 -- 172.21.15.52:0/3736623793 >> 172.21.15.52:0/3736623793 conn(0x7f0be00770b0 msgr2=0x7f0be0102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:35.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.170+0000 7f0be82ad700 1 -- 172.21.15.52:0/3736623793 shutdown_connections 2023-11-27T15:27:35.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.170+0000 7f0be82ad700 1 -- 172.21.15.52:0/3736623793 wait complete. 2023-11-27T15:27:35.187 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:27:35.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.311+0000 7f1217c6d700 1 -- 172.21.15.52:0/3512546566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12100ffdf0 msgr2=0x7f1210103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:35.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.311+0000 7f1217c6d700 1 --2- 172.21.15.52:0/3512546566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12100ffdf0 0x7f1210103db0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f12040099c0 tx=0x7f1204009cd0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.311+0000 7f1217c6d700 1 -- 172.21.15.52:0/3512546566 shutdown_connections 2023-11-27T15:27:35.315 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.311+0000 7f1217c6d700 1 --2- 172.21.15.52:0/3512546566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12100ff450 0x7f12100ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.311+0000 7f1217c6d700 1 --2- 172.21.15.52:0/3512546566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1210104490 0x7f121010c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.311+0000 7f1217c6d700 1 --2- 172.21.15.52:0/3512546566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12100ffdf0 0x7f1210103db0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.311+0000 7f1217c6d700 1 -- 172.21.15.52:0/3512546566 >> 172.21.15.52:0/3512546566 conn(0x7f12100770b0 msgr2=0x7f12100774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:35.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.312+0000 7f1217c6d700 1 -- 172.21.15.52:0/3512546566 shutdown_connections 2023-11-27T15:27:35.316 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.312+0000 7f1217c6d700 1 -- 172.21.15.52:0/3512546566 wait complete. 2023-11-27T15:27:35.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.312+0000 7f1217c6d700 1 Processor -- start 2023-11-27T15:27:35.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.312+0000 7f1217c6d700 1 -- start start 2023-11-27T15:27:35.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.312+0000 7f1217c6d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12100ff450 0x7f1210100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.318 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.312+0000 7f1217c6d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12100ffdf0 0x7f1210101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.312+0000 7f1217c6d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1210104490 0x7f12101a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.312+0000 7f1217c6d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1210101bc0 con 0x7f12100ffdf0 2023-11-27T15:27:35.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.312+0000 7f1217c6d700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12101a4b20 con 0x7f12100ff450 2023-11-27T15:27:35.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.312+0000 7f1217c6d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12101a4c60 con 0x7f1210104490 2023-11-27T15:27:35.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.313+0000 7f1215208700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12100ffdf0 0x7f1210101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.313+0000 7f1215208700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12100ffdf0 0x7f1210101390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56172/0 (socket says 172.21.15.52:56172) 2023-11-27T15:27:35.319 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.313+0000 7f1215208700 1 -- 172.21.15.52:0/1979441666 learned_addr learned my addr 172.21.15.52:0/1979441666 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:35.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.313+0000 7f121620a700 1 --2- 172.21.15.52:0/1979441666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1210104490 0x7f12101a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.313+0000 7f1215a09700 1 --2- 172.21.15.52:0/1979441666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12100ff450 0x7f1210100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.313+0000 7f1215208700 1 -- 172.21.15.52:0/1979441666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1210104490 msgr2=0x7f12101a45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:35.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.313+0000 7f1215208700 1 --2- 172.21.15.52:0/1979441666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1210104490 0x7f12101a45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.313+0000 7f1215208700 1 -- 172.21.15.52:0/1979441666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12100ff450 msgr2=0x7f1210100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:35.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.313+0000 7f1215208700 1 --2- 172.21.15.52:0/1979441666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12100ff450 0x7f1210100e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.320 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.313+0000 7f1215208700 1 -- 172.21.15.52:0/1979441666 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12040095f0 con 0x7f12100ffdf0 2023-11-27T15:27:35.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.313+0000 7f121620a700 1 --2- 172.21.15.52:0/1979441666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1210104490 0x7f12101a45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:35.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.313+0000 7f1215208700 1 --2- 172.21.15.52:0/1979441666 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12100ffdf0 0x7f1210101390 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f12040098e0 tx=0x7f120400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:35.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.313+0000 7f1202ffd700 1 -- 172.21.15.52:0/1979441666 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120401d070 con 0x7f12100ffdf0 2023-11-27T15:27:35.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.313+0000 7f1217c6d700 1 -- 172.21.15.52:0/1979441666 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f121010f210 con 0x7f12100ffdf0 2023-11-27T15:27:35.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.313+0000 7f1202ffd700 1 -- 172.21.15.52:0/1979441666 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f120400fc30 con 0x7f12100ffdf0 2023-11-27T15:27:35.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.313+0000 7f1202ffd700 1 -- 172.21.15.52:0/1979441666 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1204022ea0 con 0x7f12100ffdf0 2023-11-27T15:27:35.321 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.313+0000 7f1217c6d700 1 -- 172.21.15.52:0/1979441666 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f121010f080 con 0x7f12100ffdf0 2023-11-27T15:27:35.328 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.316+0000 7f1202ffd700 1 -- 172.21.15.52:0/1979441666 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f1204022470 con 0x7f12100ffdf0 2023-11-27T15:27:35.328 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.316+0000 7f1202ffd700 1 --2- 172.21.15.52:0/1979441666 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f11fc0a12c0 0x7f11fc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.328 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.317+0000 7f1202ffd700 1 -- 172.21.15.52:0/1979441666 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f12040cda50 con 0x7f12100ffdf0 2023-11-27T15:27:35.328 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.317+0000 7f1215a09700 1 --2- 172.21.15.52:0/1979441666 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f11fc0a12c0 0x7f11fc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.328 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.317+0000 7f1215a09700 1 --2- 172.21.15.52:0/1979441666 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f11fc0a12c0 0x7f11fc0a3770 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f120c009bf0 tx=0x7f120c009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:35.328 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.317+0000 7f1223df9b80 1 Processor -- start 2023-11-27T15:27:35.329 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.318+0000 7f1223df9b80 1 -- start start 2023-11-27T15:27:35.329 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.318+0000 7f1223df9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c77024b60 0x557c77024f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.329 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.318+0000 7f1223df9b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c77025470 0x557c77029370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.329 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.318+0000 7f1223df9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c770298b0 0x557c77029d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.329 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.318+0000 7f1223df9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c76f2a060 con 0x557c77024b60 2023-11-27T15:27:35.329 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.318+0000 7f1223df9b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c76f7c8a0 con 0x557c77025470 2023-11-27T15:27:35.329 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.318+0000 7f1223df9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c76f75d30 con 0x557c770298b0 2023-11-27T15:27:35.329 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.318+0000 7f1215a09700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c77024b60 0x557c77024f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.330 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.318+0000 7f1215a09700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c77024b60 0x557c77024f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56178/0 (socket says 172.21.15.52:56178) 2023-11-27T15:27:35.330 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.318+0000 7f1215a09700 1 -- 172.21.15.52:0/2757502115 learned_addr learned my addr 172.21.15.52:0/2757502115 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:35.330 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.318+0000 7f1215208700 1 --2- 172.21.15.52:0/2757502115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c77025470 0x557c77029370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.330 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.318+0000 7f121620a700 1 --2- 172.21.15.52:0/2757502115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c770298b0 0x557c77029d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.330 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.318+0000 7f1215a09700 1 -- 172.21.15.52:0/2757502115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c770298b0 msgr2=0x557c77029d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:35.330 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.318+0000 7f1215a09700 1 --2- 172.21.15.52:0/2757502115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c770298b0 0x557c77029d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.330 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.318+0000 7f1215a09700 1 -- 172.21.15.52:0/2757502115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c77025470 msgr2=0x557c77029370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:35.330 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.318+0000 7f1215a09700 1 --2- 172.21.15.52:0/2757502115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c77025470 0x557c77029370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.318+0000 7f1215a09700 1 -- 172.21.15.52:0/2757502115 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c76df7630 con 0x557c77024b60 2023-11-27T15:27:35.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.318+0000 7f121620a700 1 --2- 172.21.15.52:0/2757502115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c770298b0 0x557c77029d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:35.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.319+0000 7f1215a09700 1 --2- 172.21.15.52:0/2757502115 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c77024b60 0x557c77024f30 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f120c011e40 tx=0x7f120c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d206cd0c30e7d164 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:35.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.319+0000 7f11fbfff700 1 -- 172.21.15.52:0/2757502115 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c012dc0 con 0x557c77024b60 2023-11-27T15:27:35.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.319+0000 7f11fbfff700 1 -- 172.21.15.52:0/2757502115 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f120c016070 con 0x557c77024b60 2023-11-27T15:27:35.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.319+0000 7f11fbfff700 1 -- 172.21.15.52:0/2757502115 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c003db0 con 0x557c77024b60 2023-11-27T15:27:35.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.319+0000 7f1223df9b80 1 -- 172.21.15.52:0/2757502115 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c77024b60 msgr2=0x557c77024f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:35.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.319+0000 7f1223df9b80 1 --2- 172.21.15.52:0/2757502115 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c77024b60 0x557c77024f30 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f120c011e40 tx=0x7f120c014000 comp rx=0 tx=0).stop 2023-11-27T15:27:35.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.320+0000 7f1223df9b80 1 -- 172.21.15.52:0/2757502115 shutdown_connections 2023-11-27T15:27:35.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.320+0000 7f1223df9b80 1 --2- 172.21.15.52:0/2757502115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c770298b0 0x557c77029d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.320+0000 7f1223df9b80 1 --2- 172.21.15.52:0/2757502115 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c77024b60 0x557c77024f30 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.320+0000 7f1223df9b80 1 --2- 172.21.15.52:0/2757502115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c77025470 0x557c77029370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.320+0000 7f1223df9b80 1 -- 172.21.15.52:0/2757502115 >> 172.21.15.52:0/2757502115 conn(0x557c7701a360 msgr2=0x557c7701fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:35.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.320+0000 7f1223df9b80 1 -- 172.21.15.52:0/2757502115 shutdown_connections 2023-11-27T15:27:35.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.320+0000 7f1223df9b80 1 -- 172.21.15.52:0/2757502115 wait complete. 2023-11-27T15:27:35.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.320+0000 7f1223df9b80 1 -- start start 2023-11-27T15:27:35.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.320+0000 7f1223df9b80 1 Processor -- start 2023-11-27T15:27:35.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.321+0000 7f1223df9b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c77024b60 0x557c77031050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.321+0000 7f1223df9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c77025470 0x557c7702e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.321+0000 7f1223df9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c770298b0 0x557c7702e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.321+0000 7f1223df9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c76f178e0 con 0x557c770298b0 2023-11-27T15:27:35.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.321+0000 7f1223df9b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c76f17580 con 0x557c77024b60 2023-11-27T15:27:35.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.321+0000 7f1223df9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c76df9920 con 0x557c77025470 2023-11-27T15:27:35.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.321+0000 7f1223df9b80 10 client.0 _ll_register_callbacks cb 0x557c76f35780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:35.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.321+0000 7f121620a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c770298b0 0x557c7702e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.321+0000 7f121620a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c770298b0 0x557c7702e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56188/0 (socket says 172.21.15.52:56188) 2023-11-27T15:27:35.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.321+0000 7f121620a700 1 -- 172.21.15.52:0/4082858798 learned_addr learned my addr 172.21.15.52:0/4082858798 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:35.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.321+0000 7f1215208700 1 --2- 172.21.15.52:0/4082858798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c77025470 0x557c7702e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.321+0000 7f1215a09700 1 --2- 172.21.15.52:0/4082858798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c77024b60 0x557c77031050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.321+0000 7f121620a700 1 -- 172.21.15.52:0/4082858798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c77025470 msgr2=0x557c7702e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:35.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.321+0000 7f121620a700 1 --2- 172.21.15.52:0/4082858798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c77025470 0x557c7702e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.321+0000 7f121620a700 1 -- 172.21.15.52:0/4082858798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c77024b60 msgr2=0x557c77031050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:35.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.321+0000 7f121620a700 1 --2- 172.21.15.52:0/4082858798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c77024b60 0x557c77031050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.321+0000 7f121620a700 1 -- 172.21.15.52:0/4082858798 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f120c011b20 con 0x557c770298b0 2023-11-27T15:27:35.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.321+0000 7f1215208700 1 --2- 172.21.15.52:0/4082858798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c77025470 0x557c7702e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:35.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.321+0000 7f121620a700 1 --2- 172.21.15.52:0/4082858798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c770298b0 0x557c7702e950 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f1208007b30 tx=0x7f1208007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434fb98540fc7e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:35.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.321+0000 7f11f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:27:35.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.322+0000 7f11f97fa700 1 -- 172.21.15.52:0/4082858798 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120800e030 con 0x557c770298b0 2023-11-27T15:27:35.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.322+0000 7f11f97fa700 1 -- 172.21.15.52:0/4082858798 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f120800f070 con 0x557c770298b0 2023-11-27T15:27:35.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.322+0000 7f1223df9b80 1 -- 172.21.15.52:0/4082858798 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c77017b20 con 0x557c770298b0 2023-11-27T15:27:35.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.322+0000 7f11f97fa700 1 -- 172.21.15.52:0/4082858798 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12080119c0 con 0x557c770298b0 2023-11-27T15:27:35.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.322+0000 7f1223df9b80 10 client.17175 fetch_fsmap learned FSMap version 34 2023-11-27T15:27:35.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.322+0000 7f1223df9b80 1 -- 172.21.15.52:0/4082858798 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x557c76f3c190 con 0x557c770298b0 2023-11-27T15:27:35.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.322+0000 7f11f97fa700 1 -- 172.21.15.52:0/4082858798 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557c77017b20 con 0x557c770298b0 2023-11-27T15:27:35.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.322+0000 7f11f97fa700 1 -- 172.21.15.52:0/4082858798 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1208014070 con 0x557c770298b0 2023-11-27T15:27:35.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.322+0000 7f1223df9b80 10 client.17175 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:27:35.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.322+0000 7f1223df9b80 10 client.17175 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:35.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.322+0000 7f1223df9b80 20 client.17175 populate_metadata read hostname 'smithi052' 2023-11-27T15:27:35.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.322+0000 7f1223df9b80 1 --2- 172.21.15.52:0/4082858798 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557c7703e8c0 0x557c77040cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.322+0000 7f1223df9b80 4 client.17175 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:27:35.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.322+0000 7f1223df9b80 1 -- 172.21.15.52:0/4082858798 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c76f380a0 con 0x557c7703e8c0 2023-11-27T15:27:35.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.322+0000 7f1215a09700 1 --2- 172.21.15.52:0/4082858798 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557c7703e8c0 0x557c77040cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.323+0000 7f11f97fa700 10 client.17175 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:35.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.323+0000 7f1215a09700 1 --2- 172.21.15.52:0/4082858798 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557c7703e8c0 0x557c77040cd0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af64ba883e944469 server_cookie=1b5ded20f98221c in_seq=0 out_seq=0 2023-11-27T15:27:35.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.323+0000 7f11f97fa700 1 -- 172.21.15.52:0/4082858798 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557c76f380a0 con 0x557c7703e8c0 2023-11-27T15:27:35.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.323+0000 7f11f97fa700 10 client.17175 handle_command_reply: tid=0 2023-11-27T15:27:35.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.323+0000 7f1223df9b80 1 client.17175 shutdown 2023-11-27T15:27:35.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.324+0000 7f1223df9b80 1 -- 172.21.15.52:0/4082858798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c770298b0 msgr2=0x557c7702e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:35.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.324+0000 7f1223df9b80 1 --2- 172.21.15.52:0/4082858798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c770298b0 0x557c7702e950 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f1208007b30 tx=0x7f1208007ef0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.324+0000 7f1223df9b80 1 -- 172.21.15.52:0/4082858798 shutdown_connections 2023-11-27T15:27:35.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.324+0000 7f1223df9b80 1 --2- 172.21.15.52:0/4082858798 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557c7703e8c0 0x557c77040cd0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.324+0000 7f1223df9b80 1 --2- 172.21.15.52:0/4082858798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c77024b60 0x557c77031050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.324+0000 7f1223df9b80 1 --2- 172.21.15.52:0/4082858798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c770298b0 0x557c7702e950 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.324+0000 7f1223df9b80 1 --2- 172.21.15.52:0/4082858798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c77025470 0x557c7702e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.324+0000 7f11f97fa700 0 client.17175 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:35.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.324+0000 7f1223df9b80 1 -- 172.21.15.52:0/4082858798 >> 172.21.15.52:0/4082858798 conn(0x557c7701a360 msgr2=0x557c7701ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:35.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.324+0000 7f1223df9b80 1 -- 172.21.15.52:0/4082858798 shutdown_connections 2023-11-27T15:27:35.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.324+0000 7f1223df9b80 1 -- 172.21.15.52:0/4082858798 wait complete. 2023-11-27T15:27:35.339 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.324+0000 7f1223df9b80 20 client.17175 trim_cache size 0 max 16384 2023-11-27T15:27:35.349 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.341+0000 7f1223df9b80 1 Processor -- start 2023-11-27T15:27:35.349 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.341+0000 7f1223df9b80 1 -- start start 2023-11-27T15:27:35.349 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.341+0000 7f1223df9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c77024b60 0x557c770281a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.349 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.341+0000 7f1223df9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c77025470 0x557c770359b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.350 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.341+0000 7f1223df9b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c770298b0 0x557c77028a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.350 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.341+0000 7f1223df9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c770179f0 con 0x557c77024b60 2023-11-27T15:27:35.350 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.341+0000 7f1223df9b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c7701a170 con 0x557c770298b0 2023-11-27T15:27:35.350 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.341+0000 7f1223df9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c77020950 con 0x557c77025470 2023-11-27T15:27:35.350 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.341+0000 7f1215a09700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c77024b60 0x557c770281a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.350 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.341+0000 7f1215a09700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c77024b60 0x557c770281a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56192/0 (socket says 172.21.15.52:56192) 2023-11-27T15:27:35.350 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.341+0000 7f1215a09700 1 -- 172.21.15.52:0/2588869398 learned_addr learned my addr 172.21.15.52:0/2588869398 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:35.350 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.341+0000 7f1215208700 1 --2- 172.21.15.52:0/2588869398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c77025470 0x557c770359b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.351 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.341+0000 7f1215a09700 1 -- 172.21.15.52:0/2588869398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c77025470 msgr2=0x557c770359b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:35.351 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.341+0000 7f1215a09700 1 --2- 172.21.15.52:0/2588869398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c77025470 0x557c770359b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.351 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.341+0000 7f121620a700 1 --2- 172.21.15.52:0/2588869398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c770298b0 0x557c77028a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.351 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.341+0000 7f1215a09700 1 -- 172.21.15.52:0/2588869398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c770298b0 msgr2=0x557c77028a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:35.351 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.341+0000 7f1215a09700 1 --2- 172.21.15.52:0/2588869398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c770298b0 0x557c77028a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.351 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.341+0000 7f1215a09700 1 -- 172.21.15.52:0/2588869398 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c7701bb00 con 0x557c77024b60 2023-11-27T15:27:35.351 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.341+0000 7f1215208700 1 --2- 172.21.15.52:0/2588869398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c77025470 0x557c770359b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:35.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.341+0000 7f121620a700 1 --2- 172.21.15.52:0/2588869398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c770298b0 0x557c77028a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:35.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.341+0000 7f1215a09700 1 --2- 172.21.15.52:0/2588869398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c77024b60 0x557c770281a0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f120c011e10 tx=0x7f120c005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ecd8822b414bb5f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:35.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.342+0000 7f1200dac700 1 -- 172.21.15.52:0/2588869398 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c0335e0 con 0x557c77024b60 2023-11-27T15:27:35.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.342+0000 7f1200dac700 1 -- 172.21.15.52:0/2588869398 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f120c02e040 con 0x557c77024b60 2023-11-27T15:27:35.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.342+0000 7f1200dac700 1 -- 172.21.15.52:0/2588869398 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c005280 con 0x557c77024b60 2023-11-27T15:27:35.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.342+0000 7f1223df9b80 1 -- 172.21.15.52:0/2588869398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c77024b60 msgr2=0x557c770281a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:35.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.342+0000 7f1223df9b80 1 --2- 172.21.15.52:0/2588869398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c77024b60 0x557c770281a0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f120c011e10 tx=0x7f120c005760 comp rx=0 tx=0).stop 2023-11-27T15:27:35.352 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.342+0000 7f1223df9b80 1 -- 172.21.15.52:0/2588869398 shutdown_connections 2023-11-27T15:27:35.353 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.342+0000 7f1223df9b80 1 --2- 172.21.15.52:0/2588869398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c770298b0 0x557c77028a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.353 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.342+0000 7f1223df9b80 1 --2- 172.21.15.52:0/2588869398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c77024b60 0x557c770281a0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.353 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.342+0000 7f1223df9b80 1 --2- 172.21.15.52:0/2588869398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c77025470 0x557c770359b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.353 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.342+0000 7f1223df9b80 1 -- 172.21.15.52:0/2588869398 >> 172.21.15.52:0/2588869398 conn(0x557c7701a360 msgr2=0x557c7703abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:35.353 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.342+0000 7f1223df9b80 1 -- 172.21.15.52:0/2588869398 shutdown_connections 2023-11-27T15:27:35.353 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.342+0000 7f1223df9b80 1 -- 172.21.15.52:0/2588869398 wait complete. 2023-11-27T15:27:35.353 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.343+0000 7f1223df9b80 1 -- start start 2023-11-27T15:27:35.353 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.343+0000 7f1223df9b80 1 Processor -- start 2023-11-27T15:27:35.354 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.343+0000 7f1223df9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c77024b60 0x557c770336c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.354 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.343+0000 7f1223df9b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c77025470 0x557c77033c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.354 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.343+0000 7f1223df9b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c770298b0 0x557c77091740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.354 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.343+0000 7f1223df9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c77034360 con 0x557c77024b60 2023-11-27T15:27:35.354 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.343+0000 7f1223df9b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c77091c80 con 0x557c77025470 2023-11-27T15:27:35.354 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.343+0000 7f1223df9b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c77091e40 con 0x557c770298b0 2023-11-27T15:27:35.354 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.343+0000 7f1223df9b80 10 client.0 _ll_register_callbacks cb 0x557c76f35780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:35.354 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.343+0000 7f1215a09700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c77024b60 0x557c770336c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.355 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.343+0000 7f1215a09700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c77024b60 0x557c770336c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56198/0 (socket says 172.21.15.52:56198) 2023-11-27T15:27:35.355 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.343+0000 7f1215a09700 1 -- 172.21.15.52:0/39035239 learned_addr learned my addr 172.21.15.52:0/39035239 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:35.355 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.343+0000 7f121620a700 1 --2- 172.21.15.52:0/39035239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c770298b0 0x557c77091740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.355 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.343+0000 7f1215208700 1 --2- 172.21.15.52:0/39035239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c77025470 0x557c77033c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.355 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.343+0000 7f1215a09700 1 -- 172.21.15.52:0/39035239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c770298b0 msgr2=0x557c77091740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:35.355 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.343+0000 7f1215a09700 1 --2- 172.21.15.52:0/39035239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c770298b0 0x557c77091740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.355 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.343+0000 7f1215a09700 1 -- 172.21.15.52:0/39035239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c77025470 msgr2=0x557c77033c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:35.355 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.343+0000 7f1215a09700 1 --2- 172.21.15.52:0/39035239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c77025470 0x557c77033c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.356 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.343+0000 7f1215a09700 1 -- 172.21.15.52:0/39035239 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c76df7630 con 0x557c77024b60 2023-11-27T15:27:35.356 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.344+0000 7f1215a09700 1 --2- 172.21.15.52:0/39035239 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c77024b60 0x557c770336c0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f120c012e60 tx=0x7f120c012ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba2c3b267329fc0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:35.356 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.344+0000 7f11f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:27:35.356 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.344+0000 7f11f97fa700 1 -- 172.21.15.52:0/39035239 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c033b60 con 0x557c77024b60 2023-11-27T15:27:35.356 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.344+0000 7f11f97fa700 1 -- 172.21.15.52:0/39035239 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f120c018070 con 0x557c77024b60 2023-11-27T15:27:35.356 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.344+0000 7f1223df9b80 1 -- 172.21.15.52:0/39035239 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f120c016070 con 0x557c77024b60 2023-11-27T15:27:35.356 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.344+0000 7f11f97fa700 1 -- 172.21.15.52:0/39035239 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f120c0335f0 con 0x557c77024b60 2023-11-27T15:27:35.357 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.344+0000 7f1223df9b80 10 client.17181 fetch_fsmap learned FSMap version 34 2023-11-27T15:27:35.357 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.344+0000 7f1223df9b80 1 -- 172.21.15.52:0/39035239 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x557c76da3de0 con 0x557c77024b60 2023-11-27T15:27:35.357 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.344+0000 7f11f97fa700 1 -- 172.21.15.52:0/39035239 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f120c04d020 con 0x557c77024b60 2023-11-27T15:27:35.357 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.344+0000 7f11f97fa700 1 -- 172.21.15.52:0/39035239 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f120c025070 con 0x557c77024b60 2023-11-27T15:27:35.357 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.344+0000 7f1223df9b80 10 client.17181 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:27:35.357 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.344+0000 7f1223df9b80 10 client.17181 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:35.357 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.344+0000 7f1223df9b80 20 client.17181 populate_metadata read hostname 'smithi052' 2023-11-27T15:27:35.357 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.344+0000 7f1223df9b80 1 --2- 172.21.15.52:0/39035239 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557c77093040 0x557c77095420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:35.358 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.344+0000 7f1223df9b80 4 client.17181 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:27:35.358 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.344+0000 7f1223df9b80 1 -- 172.21.15.52:0/39035239 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557c76f44660 con 0x557c77093040 2023-11-27T15:27:35.358 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.345+0000 7f1215208700 1 --2- 172.21.15.52:0/39035239 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557c77093040 0x557c77095420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:35.358 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.345+0000 7f1215208700 1 --2- 172.21.15.52:0/39035239 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557c77093040 0x557c77095420 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ac31065adc75d00 server_cookie=31d768d6a2eac7ed in_seq=0 out_seq=0 2023-11-27T15:27:35.358 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.345+0000 7f11f97fa700 10 client.17181 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:35.358 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.345+0000 7f11f97fa700 1 -- 172.21.15.52:0/39035239 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557c76f44660 con 0x557c77093040 2023-11-27T15:27:35.358 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.345+0000 7f11f97fa700 10 client.17181 handle_command_reply: tid=0 2023-11-27T15:27:35.358 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.345+0000 7f1223df9b80 1 client.17181 shutdown 2023-11-27T15:27:35.359 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.345+0000 7f1223df9b80 1 -- 172.21.15.52:0/39035239 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c77024b60 msgr2=0x557c770336c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:35.359 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.345+0000 7f1223df9b80 1 --2- 172.21.15.52:0/39035239 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c77024b60 0x557c770336c0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f120c012e60 tx=0x7f120c012ec0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.359 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.345+0000 7f1223df9b80 1 -- 172.21.15.52:0/39035239 shutdown_connections 2023-11-27T15:27:35.359 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.345+0000 7f1223df9b80 1 --2- 172.21.15.52:0/39035239 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x557c77093040 0x557c77095420 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.359 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.345+0000 7f1223df9b80 1 --2- 172.21.15.52:0/39035239 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c770298b0 0x557c77091740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.359 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.345+0000 7f11f97fa700 0 client.17181 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:27:35.359 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.345+0000 7f1223df9b80 1 --2- 172.21.15.52:0/39035239 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c77024b60 0x557c770336c0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.359 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.345+0000 7f1223df9b80 1 --2- 172.21.15.52:0/39035239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c77025470 0x557c77033c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.345+0000 7f1223df9b80 1 -- 172.21.15.52:0/39035239 >> 172.21.15.52:0/39035239 conn(0x557c7701a360 msgr2=0x557c77028a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:35.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.346+0000 7f1223df9b80 1 -- 172.21.15.52:0/39035239 shutdown_connections 2023-11-27T15:27:35.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.346+0000 7f1223df9b80 1 -- 172.21.15.52:0/39035239 wait complete. 2023-11-27T15:27:35.360 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.346+0000 7f1223df9b80 20 client.17181 trim_cache size 0 max 16384 2023-11-27T15:27:35.360 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:27:35.360 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-27T15:27:35.361 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2023-11-27T15:27:35.361 INFO:teuthology.orchestra.run.smithi052.stdout: "2a81c721-a4cb-4be7-94ee-864fe03c385d": { 2023-11-27T15:27:35.361 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2023-11-27T15:27:35.361 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "2a81c721-a4cb-4be7-94ee-864fe03c385d", 2023-11-27T15:27:35.361 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2023-11-27T15:27:35.361 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:27:35.361 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:27:35.361 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:27:35.363 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.346+0000 7f1217c6d700 1 -- 172.21.15.52:0/1979441666 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f11fc0a12c0 msgr2=0x7f11fc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:35.363 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.346+0000 7f1217c6d700 1 --2- 172.21.15.52:0/1979441666 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f11fc0a12c0 0x7f11fc0a3770 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f120c009bf0 tx=0x7f120c009260 comp rx=0 tx=0).stop 2023-11-27T15:27:35.363 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.346+0000 7f1217c6d700 1 -- 172.21.15.52:0/1979441666 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12100ffdf0 msgr2=0x7f1210101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:35.363 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.346+0000 7f1217c6d700 1 --2- 172.21.15.52:0/1979441666 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12100ffdf0 0x7f1210101390 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f12040098e0 tx=0x7f120400f740 comp rx=0 tx=0).stop 2023-11-27T15:27:35.363 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.346+0000 7f1217c6d700 1 -- 172.21.15.52:0/1979441666 shutdown_connections 2023-11-27T15:27:35.363 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.346+0000 7f1217c6d700 1 --2- 172.21.15.52:0/1979441666 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f11fc0a12c0 0x7f11fc0a3770 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.364 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.346+0000 7f1217c6d700 1 --2- 172.21.15.52:0/1979441666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12100ff450 0x7f1210100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.364 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.346+0000 7f1217c6d700 1 --2- 172.21.15.52:0/1979441666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1210104490 0x7f12101a45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.364 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.346+0000 7f1217c6d700 1 --2- 172.21.15.52:0/1979441666 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12100ffdf0 0x7f1210101390 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:35.364 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.346+0000 7f1217c6d700 1 -- 172.21.15.52:0/1979441666 >> 172.21.15.52:0/1979441666 conn(0x7f12100770b0 msgr2=0x7f1210067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:35.364 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.346+0000 7f1217c6d700 1 -- 172.21.15.52:0/1979441666 shutdown_connections 2023-11-27T15:27:35.364 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:35.346+0000 7f1217c6d700 1 -- 172.21.15.52:0/1979441666 wait complete. 2023-11-27T15:27:35.364 INFO:tasks.cephfs.filesystem:scrub status for tag:2a81c721-a4cb-4be7-94ee-864fe03c385d - {'path': '/', 'tag': '2a81c721-a4cb-4be7-94ee-864fe03c385d', 'options': 'recursive,force'} 2023-11-27T15:27:36.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:36 smithi150 ceph-mon[100559]: scrub queued for path: / 2023-11-27T15:27:36.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:36 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:36.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:36 smithi150 ceph-mon[100559]: scrub summary: active paths [/] 2023-11-27T15:27:36.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:36 smithi150 ceph-mon[100559]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:36.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:36 smithi150 ceph-mon[100559]: pgmap v301: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:27:36.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:36 smithi102 ceph-mon[100296]: scrub queued for path: / 2023-11-27T15:27:36.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:36 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:36.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:36 smithi102 ceph-mon[100296]: scrub summary: active paths [/] 2023-11-27T15:27:36.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:36 smithi102 ceph-mon[100296]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:36.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:36 smithi102 ceph-mon[100296]: pgmap v301: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:27:36.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:36 smithi052 ceph-mon[99056]: scrub queued for path: / 2023-11-27T15:27:36.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:36 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:36.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:36 smithi052 ceph-mon[99056]: scrub summary: active paths [/] 2023-11-27T15:27:36.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:36 smithi052 ceph-mon[99056]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:36.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:36 smithi052 ceph-mon[99056]: pgmap v301: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:27:38.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:38 smithi150 ceph-mon[100559]: pgmap v302: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:27:38.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:38 smithi102 ceph-mon[100296]: pgmap v302: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:27:38.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:38 smithi052 ceph-mon[99056]: pgmap v302: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:27:40.277 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:39 smithi150 ceph-mon[100559]: pgmap v303: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:27:40.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:39 smithi150 ceph-mon[100559]: scrub summary: idle 2023-11-27T15:27:40.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:39 smithi102 ceph-mon[100296]: pgmap v303: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:27:40.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:39 smithi102 ceph-mon[100296]: scrub summary: idle 2023-11-27T15:27:40.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:39 smithi052 ceph-mon[99056]: pgmap v303: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:27:40.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:39 smithi052 ceph-mon[99056]: scrub summary: idle 2023-11-27T15:27:42.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:42 smithi150 ceph-mon[100559]: pgmap v304: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:27:42.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:42 smithi102 ceph-mon[100296]: pgmap v304: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:27:42.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:42 smithi052 ceph-mon[99056]: pgmap v304: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:27:44.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:44 smithi150 ceph-mon[100559]: pgmap v305: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:27:44.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:44 smithi102 ceph-mon[100296]: pgmap v305: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:27:44.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:44 smithi052 ceph-mon[99056]: pgmap v305: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:27:46.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:46 smithi150 ceph-mon[100559]: pgmap v306: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:27:46.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:46 smithi102 ceph-mon[100296]: pgmap v306: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:27:46.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:46 smithi052 ceph-mon[99056]: pgmap v306: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-27T15:27:48.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:48 smithi150 ceph-mon[100559]: pgmap v307: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:27:48.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:48 smithi102 ceph-mon[100296]: pgmap v307: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:27:48.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:48 smithi052 ceph-mon[99056]: pgmap v307: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-27T15:27:50.278 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:49 smithi150 ceph-mon[100559]: pgmap v308: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:27:50.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:49 smithi102 ceph-mon[100296]: pgmap v308: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:27:50.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:49 smithi052 ceph-mon[99056]: pgmap v308: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:27:52.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:52 smithi150 ceph-mon[100559]: pgmap v309: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:27:52.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:52 smithi102 ceph-mon[100296]: pgmap v309: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:27:52.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:52 smithi052 ceph-mon[99056]: pgmap v309: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:27:54.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:54 smithi150 ceph-mon[100559]: pgmap v310: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:27:54.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:54 smithi102 ceph-mon[100296]: pgmap v310: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:27:54.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:54 smithi052 ceph-mon[99056]: pgmap v310: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:27:56.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:56 smithi150 ceph-mon[100559]: pgmap v311: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:27:56.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:56 smithi102 ceph-mon[100296]: pgmap v311: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:27:56.615 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:56 smithi052 ceph-mon[99056]: pgmap v311: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:27:57.653 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-11-27T15:27:57.794 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.791+0000 7f2ce02c8700 1 -- 172.21.15.52:0/108828035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cd80ff3c0 msgr2=0x7f2cd80ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:57.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.791+0000 7f2ce02c8700 1 --2- 172.21.15.52:0/108828035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cd80ff3c0 0x7f2cd80ff790 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f2cd40099c0 tx=0x7f2cd4009cd0 comp rx=0 tx=0).stop 2023-11-27T15:27:57.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.791+0000 7f2ce02c8700 1 -- 172.21.15.52:0/108828035 shutdown_connections 2023-11-27T15:27:57.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.791+0000 7f2ce02c8700 1 --2- 172.21.15.52:0/108828035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2cd8104400 0x7f2cd810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:57.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.791+0000 7f2ce02c8700 1 --2- 172.21.15.52:0/108828035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cd80ff3c0 0x7f2cd80ff790 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:57.795 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.791+0000 7f2ce02c8700 1 --2- 172.21.15.52:0/108828035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cd80ffd60 0x7f2cd8103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:57.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.791+0000 7f2ce02c8700 1 -- 172.21.15.52:0/108828035 >> 172.21.15.52:0/108828035 conn(0x7f2cd80763a0 msgr2=0x7f2cd80767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:57.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.791+0000 7f2ce02c8700 1 -- 172.21.15.52:0/108828035 shutdown_connections 2023-11-27T15:27:57.796 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.791+0000 7f2ce02c8700 1 -- 172.21.15.52:0/108828035 wait complete. 2023-11-27T15:27:57.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.791+0000 7f2ce02c8700 1 Processor -- start 2023-11-27T15:27:57.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.792+0000 7f2ce02c8700 1 -- start start 2023-11-27T15:27:57.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.792+0000 7f2ce02c8700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2cd80ff3c0 0x7f2cd8197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:57.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.792+0000 7f2ce02c8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cd80ffd60 0x7f2cd81976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:57.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.792+0000 7f2ce02c8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cd8104400 0x7f2cd81a21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:57.798 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.792+0000 7f2ce02c8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd81a2720 con 0x7f2cd8104400 2023-11-27T15:27:57.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.792+0000 7f2ce02c8700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd81a2860 con 0x7f2cd80ff3c0 2023-11-27T15:27:57.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.792+0000 7f2ce02c8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd81a29a0 con 0x7f2cd80ffd60 2023-11-27T15:27:57.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.792+0000 7f2cde865700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cd8104400 0x7f2cd81a21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:57.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.792+0000 7f2cde865700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cd8104400 0x7f2cd81a21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56214/0 (socket says 172.21.15.52:56214) 2023-11-27T15:27:57.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.792+0000 7f2cde865700 1 -- 172.21.15.52:0/3638743609 learned_addr learned my addr 172.21.15.52:0/3638743609 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:57.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.792+0000 7f2cde064700 1 --2- 172.21.15.52:0/3638743609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2cd80ff3c0 0x7f2cd8197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:57.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.792+0000 7f2cdd863700 1 --2- 172.21.15.52:0/3638743609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cd80ffd60 0x7f2cd81976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:57.799 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.792+0000 7f2cde865700 1 -- 172.21.15.52:0/3638743609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cd80ffd60 msgr2=0x7f2cd81976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:57.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.792+0000 7f2cde865700 1 --2- 172.21.15.52:0/3638743609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cd80ffd60 0x7f2cd81976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:57.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.792+0000 7f2cde865700 1 -- 172.21.15.52:0/3638743609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2cd80ff3c0 msgr2=0x7f2cd8197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:57.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.792+0000 7f2cde865700 1 --2- 172.21.15.52:0/3638743609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2cd80ff3c0 0x7f2cd8197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:57.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.792+0000 7f2cde865700 1 -- 172.21.15.52:0/3638743609 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd40095f0 con 0x7f2cd8104400 2023-11-27T15:27:57.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.792+0000 7f2cde865700 1 --2- 172.21.15.52:0/3638743609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cd8104400 0x7f2cd81a21e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0007ee0 tx=0x7f2cd000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:57.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.793+0000 7f2ccf7fe700 1 -- 172.21.15.52:0/3638743609 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd0017070 con 0x7f2cd8104400 2023-11-27T15:27:57.800 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.793+0000 7f2ce02c8700 1 -- 172.21.15.52:0/3638743609 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd810f1c0 con 0x7f2cd8104400 2023-11-27T15:27:57.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.793+0000 7f2ccf7fe700 1 -- 172.21.15.52:0/3638743609 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cd000f040 con 0x7f2cd8104400 2023-11-27T15:27:57.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.793+0000 7f2ccf7fe700 1 -- 172.21.15.52:0/3638743609 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd0005740 con 0x7f2cd8104400 2023-11-27T15:27:57.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.793+0000 7f2ce02c8700 1 -- 172.21.15.52:0/3638743609 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd810f030 con 0x7f2cd8104400 2023-11-27T15:27:57.801 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.794+0000 7f2ce02c8700 1 -- 172.21.15.52:0/3638743609 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cd810f350 con 0x7f2cd8104400 2023-11-27T15:27:57.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.795+0000 7f2ccf7fe700 1 -- 172.21.15.52:0/3638743609 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f2cd000dc60 con 0x7f2cd8104400 2023-11-27T15:27:57.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.795+0000 7f2ccf7fe700 1 --2- 172.21.15.52:0/3638743609 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f2cc40a12c0 0x7f2cc40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:57.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.795+0000 7f2ccf7fe700 1 -- 172.21.15.52:0/3638743609 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f2cd00cc4d0 con 0x7f2cd8104400 2023-11-27T15:27:57.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.795+0000 7f2cde064700 1 --2- 172.21.15.52:0/3638743609 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f2cc40a12c0 0x7f2cc40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:57.802 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.796+0000 7f2cde064700 1 --2- 172.21.15.52:0/3638743609 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f2cc40a12c0 0x7f2cc40a3770 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009990 tx=0x7f2cd4003680 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:57.803 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:57.800+0000 7f2ccf7fe700 1 -- 172.21.15.52:0/3638743609 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2cd00958b0 con 0x7f2cd8104400 2023-11-27T15:27:58.028 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.025+0000 7f2ce02c8700 1 -- 172.21.15.52:0/3638743609 --> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f2cd8103410 con 0x7f2cc40a12c0 2023-11-27T15:27:58.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.034+0000 7f2ccf7fe700 1 -- 172.21.15.52:0/3638743609 <== mgr.15387 v2:172.21.15.52:6800/1148545855 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+335 (secure 0 0 0) 0x7f2cd8103410 con 0x7f2cc40a12c0 2023-11-27T15:27:58.037 INFO:teuthology.orchestra.run.smithi052.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-11-27T00:00:00", "created": "2023-11-27T15:17:55", "first": "2023-11-27T15:18:00", "last": "2023-11-27T15:27:00", "last_pruned": "2023-11-27T15:27:00", "created_count": 10, "pruned_count": 4, "active": true} 2023-11-27T15:27:58.039 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.036+0000 7f2ce02c8700 1 -- 172.21.15.52:0/3638743609 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f2cc40a12c0 msgr2=0x7f2cc40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:58.039 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.036+0000 7f2ce02c8700 1 --2- 172.21.15.52:0/3638743609 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f2cc40a12c0 0x7f2cc40a3770 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009990 tx=0x7f2cd4003680 comp rx=0 tx=0).stop 2023-11-27T15:27:58.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.036+0000 7f2ce02c8700 1 -- 172.21.15.52:0/3638743609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cd8104400 msgr2=0x7f2cd81a21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:58.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.036+0000 7f2ce02c8700 1 --2- 172.21.15.52:0/3638743609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cd8104400 0x7f2cd81a21e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0007ee0 tx=0x7f2cd000d3b0 comp rx=0 tx=0).stop 2023-11-27T15:27:58.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.036+0000 7f2ce02c8700 1 -- 172.21.15.52:0/3638743609 shutdown_connections 2023-11-27T15:27:58.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.036+0000 7f2ce02c8700 1 --2- 172.21.15.52:0/3638743609 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f2cc40a12c0 0x7f2cc40a3770 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:58.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.036+0000 7f2ce02c8700 1 --2- 172.21.15.52:0/3638743609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2cd80ff3c0 0x7f2cd8197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:58.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.036+0000 7f2ce02c8700 1 --2- 172.21.15.52:0/3638743609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cd8104400 0x7f2cd81a21e0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:58.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.036+0000 7f2ce02c8700 1 --2- 172.21.15.52:0/3638743609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cd80ffd60 0x7f2cd81976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:58.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.036+0000 7f2ce02c8700 1 -- 172.21.15.52:0/3638743609 >> 172.21.15.52:0/3638743609 conn(0x7f2cd80763a0 msgr2=0x7f2cd8102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:58.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.036+0000 7f2ce02c8700 1 -- 172.21.15.52:0/3638743609 shutdown_connections 2023-11-27T15:27:58.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.036+0000 7f2ce02c8700 1 -- 172.21.15.52:0/3638743609 wait complete. 2023-11-27T15:27:58.051 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-11-27T15:27:58.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.176+0000 7f7f7a6a3700 1 -- 172.21.15.52:0/3864922897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f74100c50 msgr2=0x7f7f74101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:58.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.176+0000 7f7f7a6a3700 1 --2- 172.21.15.52:0/3864922897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f74100c50 0x7f7f74101020 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f7f640099c0 tx=0x7f7f64009cd0 comp rx=0 tx=0).stop 2023-11-27T15:27:58.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.176+0000 7f7f7a6a3700 1 -- 172.21.15.52:0/3864922897 shutdown_connections 2023-11-27T15:27:58.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.176+0000 7f7f7a6a3700 1 --2- 172.21.15.52:0/3864922897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f74105560 0x7f7f7410d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:58.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.176+0000 7f7f7a6a3700 1 --2- 172.21.15.52:0/3864922897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f74100c50 0x7f7f74101020 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:58.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.176+0000 7f7f7a6a3700 1 --2- 172.21.15.52:0/3864922897 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f741015f0 0x7f7f74104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:58.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.176+0000 7f7f7a6a3700 1 -- 172.21.15.52:0/3864922897 >> 172.21.15.52:0/3864922897 conn(0x7f7f74077550 msgr2=0x7f7f74077950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:58.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.176+0000 7f7f7a6a3700 1 -- 172.21.15.52:0/3864922897 shutdown_connections 2023-11-27T15:27:58.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.176+0000 7f7f7a6a3700 1 -- 172.21.15.52:0/3864922897 wait complete. 2023-11-27T15:27:58.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.176+0000 7f7f7a6a3700 1 Processor -- start 2023-11-27T15:27:58.181 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.176+0000 7f7f7a6a3700 1 -- start start 2023-11-27T15:27:58.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.177+0000 7f7f7a6a3700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f74100c50 0x7f7f74072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:58.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.177+0000 7f7f7a6a3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f741015f0 0x7f7f74073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:58.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.177+0000 7f7f7a6a3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f74105560 0x7f7f741a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:58.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.177+0000 7f7f7a6a3700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f74073b60 con 0x7f7f74105560 2023-11-27T15:27:58.183 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.177+0000 7f7f7a6a3700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f741a4ac0 con 0x7f7f74100c50 2023-11-27T15:27:58.184 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.177+0000 7f7f7a6a3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f741a4c00 con 0x7f7f741015f0 2023-11-27T15:27:58.184 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.177+0000 7f7f78c40700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f74105560 0x7f7f741a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:58.184 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.177+0000 7f7f78c40700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f74105560 0x7f7f741a4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56228/0 (socket says 172.21.15.52:56228) 2023-11-27T15:27:58.184 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.177+0000 7f7f78c40700 1 -- 172.21.15.52:0/1721860219 learned_addr learned my addr 172.21.15.52:0/1721860219 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:27:58.184 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.177+0000 7f7f737fe700 1 --2- 172.21.15.52:0/1721860219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f741015f0 0x7f7f74073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:58.184 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.177+0000 7f7f73fff700 1 --2- 172.21.15.52:0/1721860219 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f74100c50 0x7f7f74072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:58.184 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.177+0000 7f7f78c40700 1 -- 172.21.15.52:0/1721860219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f741015f0 msgr2=0x7f7f74073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:58.184 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.177+0000 7f7f78c40700 1 --2- 172.21.15.52:0/1721860219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f741015f0 0x7f7f74073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:58.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.177+0000 7f7f78c40700 1 -- 172.21.15.52:0/1721860219 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f74100c50 msgr2=0x7f7f74072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:58.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.177+0000 7f7f78c40700 1 --2- 172.21.15.52:0/1721860219 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f74100c50 0x7f7f74072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:58.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.177+0000 7f7f78c40700 1 -- 172.21.15.52:0/1721860219 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f640095f0 con 0x7f7f74105560 2023-11-27T15:27:58.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.177+0000 7f7f78c40700 1 --2- 172.21.15.52:0/1721860219 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f74105560 0x7f7f741a4580 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00ef30 tx=0x7f7f6c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:58.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.177+0000 7f7f717fa700 1 -- 172.21.15.52:0/1721860219 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c018070 con 0x7f7f74105560 2023-11-27T15:27:58.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.178+0000 7f7f717fa700 1 -- 172.21.15.52:0/1721860219 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f6c00f040 con 0x7f7f74105560 2023-11-27T15:27:58.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.178+0000 7f7f7a6a3700 1 -- 172.21.15.52:0/1721860219 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f741103b0 con 0x7f7f74105560 2023-11-27T15:27:58.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.178+0000 7f7f717fa700 1 -- 172.21.15.52:0/1721860219 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c00cd00 con 0x7f7f74105560 2023-11-27T15:27:58.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.178+0000 7f7f7a6a3700 1 -- 172.21.15.52:0/1721860219 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f74110220 con 0x7f7f74105560 2023-11-27T15:27:58.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.178+0000 7f7f7a6a3700 1 -- 172.21.15.52:0/1721860219 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f74110540 con 0x7f7f74105560 2023-11-27T15:27:58.186 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.180+0000 7f7f717fa700 1 -- 172.21.15.52:0/1721860219 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7f6c003a00 con 0x7f7f74105560 2023-11-27T15:27:58.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.180+0000 7f7f717fa700 1 --2- 172.21.15.52:0/1721860219 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f7f5c0a12c0 0x7f7f5c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:58.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.180+0000 7f7f73fff700 1 --2- 172.21.15.52:0/1721860219 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f7f5c0a12c0 0x7f7f5c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:58.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.180+0000 7f7f717fa700 1 -- 172.21.15.52:0/1721860219 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f7f6c0cd110 con 0x7f7f74105560 2023-11-27T15:27:58.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.181+0000 7f7f73fff700 1 --2- 172.21.15.52:0/1721860219 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f7f5c0a12c0 0x7f7f5c0a3770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7f64009990 tx=0x7f7f64005fb0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:58.187 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.184+0000 7f7f717fa700 1 -- 172.21.15.52:0/1721860219 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7f6c096620 con 0x7f7f74105560 2023-11-27T15:27:58.365 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:58 smithi052 ceph-mon[99056]: pgmap v312: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:27:58.415 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.412+0000 7f7f7a6a3700 1 -- 172.21.15.52:0/1721860219 --> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f7f740ff0f0 con 0x7f7f5c0a12c0 2023-11-27T15:27:58.423 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.419+0000 7f7f717fa700 1 -- 172.21.15.52:0/1721860219 <== mgr.15387 v2:172.21.15.52:6800/1148545855 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f7f740ff0f0 con 0x7f7f5c0a12c0 2023-11-27T15:27:58.423 INFO:teuthology.orchestra.run.smithi052.stdout:/ 1M 6M3h 2023-11-27T15:27:58.425 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.422+0000 7f7f7a6a3700 1 -- 172.21.15.52:0/1721860219 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f7f5c0a12c0 msgr2=0x7f7f5c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:58.425 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.422+0000 7f7f7a6a3700 1 --2- 172.21.15.52:0/1721860219 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f7f5c0a12c0 0x7f7f5c0a3770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7f64009990 tx=0x7f7f64005fb0 comp rx=0 tx=0).stop 2023-11-27T15:27:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.422+0000 7f7f7a6a3700 1 -- 172.21.15.52:0/1721860219 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f74105560 msgr2=0x7f7f741a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.422+0000 7f7f7a6a3700 1 --2- 172.21.15.52:0/1721860219 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f74105560 0x7f7f741a4580 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00ef30 tx=0x7f7f6c00c3a0 comp rx=0 tx=0).stop 2023-11-27T15:27:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.422+0000 7f7f7a6a3700 1 -- 172.21.15.52:0/1721860219 shutdown_connections 2023-11-27T15:27:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.422+0000 7f7f7a6a3700 1 --2- 172.21.15.52:0/1721860219 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f7f5c0a12c0 0x7f7f5c0a3770 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.422+0000 7f7f7a6a3700 1 --2- 172.21.15.52:0/1721860219 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f74100c50 0x7f7f74072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.422+0000 7f7f7a6a3700 1 --2- 172.21.15.52:0/1721860219 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f74105560 0x7f7f741a4580 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.422+0000 7f7f7a6a3700 1 --2- 172.21.15.52:0/1721860219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f741015f0 0x7f7f74073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:58.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.422+0000 7f7f7a6a3700 1 -- 172.21.15.52:0/1721860219 >> 172.21.15.52:0/1721860219 conn(0x7f7f74077550 msgr2=0x7f7f740fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:58.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.422+0000 7f7f7a6a3700 1 -- 172.21.15.52:0/1721860219 shutdown_connections 2023-11-27T15:27:58.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:27:58.422+0000 7f7f7a6a3700 1 -- 172.21.15.52:0/1721860219 wait complete. 2023-11-27T15:27:58.437 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-11-27T15:27:58.448 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-11-27T15:27:58.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:58 smithi150 ceph-mon[100559]: pgmap v312: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:27:58.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:58 smithi102 ceph-mon[100296]: pgmap v312: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:27:59.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:59 smithi150 ceph-mon[100559]: from='client.17187 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:27:59.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:27:59 smithi150 ceph-mon[100559]: from='client.17193 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:27:59.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:59 smithi102 ceph-mon[100296]: from='client.17187 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:27:59.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:27:59 smithi102 ceph-mon[100296]: from='client.17193 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:27:59.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:59 smithi052 ceph-mon[99056]: from='client.17187 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:27:59.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:27:59 smithi052 ceph-mon[99056]: from='client.17193 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:28:00.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:00 smithi150 ceph-mon[100559]: pgmap v313: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:28:00.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:00 smithi102 ceph-mon[100296]: pgmap v313: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:28:00.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:00 smithi052 ceph-mon[99056]: pgmap v313: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:28:02.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:02 smithi150 ceph-mon[100559]: pgmap v314: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2023-11-27T15:28:02.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:02 smithi102 ceph-mon[100296]: pgmap v314: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2023-11-27T15:28:02.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:02 smithi052 ceph-mon[99056]: pgmap v314: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2023-11-27T15:28:04.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:04 smithi150 ceph-mon[100559]: pgmap v315: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-11-27T15:28:04.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:04 smithi102 ceph-mon[100296]: pgmap v315: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-11-27T15:28:04.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:04 smithi052 ceph-mon[99056]: pgmap v315: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-11-27T15:28:05.365 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:28:05.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.504+0000 7f9717f3b700 1 -- 172.21.15.52:0/3235172745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9710104500 msgr2=0x7f971010c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:05.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.504+0000 7f9717f3b700 1 --2- 172.21.15.52:0/3235172745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9710104500 0x7f971010c0d0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f970800b0d0 tx=0x7f970800b3e0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.508 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.504+0000 7f9717f3b700 1 -- 172.21.15.52:0/3235172745 shutdown_connections 2023-11-27T15:28:05.509 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.504+0000 7f9717f3b700 1 --2- 172.21.15.52:0/3235172745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9710104500 0x7f971010c0d0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.509 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.504+0000 7f9717f3b700 1 --2- 172.21.15.52:0/3235172745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97100ff4c0 0x7f97100ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.509 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.504+0000 7f9717f3b700 1 --2- 172.21.15.52:0/3235172745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f97100ffe60 0x7f9710103e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.509 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.504+0000 7f9717f3b700 1 -- 172.21.15.52:0/3235172745 >> 172.21.15.52:0/3235172745 conn(0x7f9710076200 msgr2=0x7f9710076600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:05.509 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.505+0000 7f9717f3b700 1 -- 172.21.15.52:0/3235172745 shutdown_connections 2023-11-27T15:28:05.509 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.505+0000 7f9717f3b700 1 -- 172.21.15.52:0/3235172745 wait complete. 2023-11-27T15:28:05.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.505+0000 7f9717f3b700 1 Processor -- start 2023-11-27T15:28:05.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.505+0000 7f9717f3b700 1 -- start start 2023-11-27T15:28:05.511 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.506+0000 7f9717f3b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97100ff4c0 0x7f97101953d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:05.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.506+0000 7f9717f3b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97100ffe60 0x7f9710195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:05.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.506+0000 7f9717f3b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9710104500 0x7f9710199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:05.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.506+0000 7f9717f3b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971019a3b0 con 0x7f97100ff4c0 2023-11-27T15:28:05.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.506+0000 7f9717f3b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971019a520 con 0x7f9710104500 2023-11-27T15:28:05.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.506+0000 7f9717f3b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971019a6e0 con 0x7f97100ffe60 2023-11-27T15:28:05.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.506+0000 7f9715cd7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97100ff4c0 0x7f97101953d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:05.512 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.506+0000 7f9715cd7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97100ff4c0 0x7f97101953d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56238/0 (socket says 172.21.15.52:56238) 2023-11-27T15:28:05.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.506+0000 7f9715cd7700 1 -- 172.21.15.52:0/65834400 learned_addr learned my addr 172.21.15.52:0/65834400 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:05.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.506+0000 7f97164d8700 1 --2- 172.21.15.52:0/65834400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9710104500 0x7f9710199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:05.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.506+0000 7f97154d6700 1 --2- 172.21.15.52:0/65834400 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97100ffe60 0x7f9710195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:05.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.506+0000 7f9715cd7700 1 -- 172.21.15.52:0/65834400 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97100ffe60 msgr2=0x7f9710195910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:05.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.506+0000 7f9715cd7700 1 --2- 172.21.15.52:0/65834400 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97100ffe60 0x7f9710195910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.506+0000 7f9715cd7700 1 -- 172.21.15.52:0/65834400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9710104500 msgr2=0x7f9710199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:05.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.506+0000 7f9715cd7700 1 --2- 172.21.15.52:0/65834400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9710104500 0x7f9710199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.513 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.506+0000 7f9715cd7700 1 -- 172.21.15.52:0/65834400 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9708009d00 con 0x7f97100ff4c0 2023-11-27T15:28:05.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.507+0000 7f9715cd7700 1 --2- 172.21.15.52:0/65834400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97100ff4c0 0x7f97101953d0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f970000d950 tx=0x7f970000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:05.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.507+0000 7f9706ffd700 1 -- 172.21.15.52:0/65834400 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970000f890 con 0x7f97100ff4c0 2023-11-27T15:28:05.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.507+0000 7f9706ffd700 1 -- 172.21.15.52:0/65834400 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9700010460 con 0x7f97100ff4c0 2023-11-27T15:28:05.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.507+0000 7f9717f3b700 1 -- 172.21.15.52:0/65834400 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f971010f2e0 con 0x7f97100ff4c0 2023-11-27T15:28:05.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.507+0000 7f9706ffd700 1 -- 172.21.15.52:0/65834400 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970000e610 con 0x7f97100ff4c0 2023-11-27T15:28:05.514 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.507+0000 7f9717f3b700 1 -- 172.21.15.52:0/65834400 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f971010f150 con 0x7f97100ff4c0 2023-11-27T15:28:05.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.509+0000 7f9706ffd700 1 -- 172.21.15.52:0/65834400 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f97000108d0 con 0x7f97100ff4c0 2023-11-27T15:28:05.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.510+0000 7f9706ffd700 1 --2- 172.21.15.52:0/65834400 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f96fc0a12c0 0x7f96fc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:05.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.510+0000 7f97154d6700 1 --2- 172.21.15.52:0/65834400 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f96fc0a12c0 0x7f96fc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:05.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.510+0000 7f9706ffd700 1 -- 172.21.15.52:0/65834400 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f97000ccb00 con 0x7f97100ff4c0 2023-11-27T15:28:05.517 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.510+0000 7f97154d6700 1 --2- 172.21.15.52:0/65834400 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f96fc0a12c0 0x7f96fc0a3770 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f97101969e0 tx=0x7f970c0073d0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:05.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.511+0000 7f97240c7b80 1 Processor -- start 2023-11-27T15:28:05.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.511+0000 7f97240c7b80 1 -- start start 2023-11-27T15:28:05.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.511+0000 7f97240c7b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618357dcd70 0x5618357dd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:05.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.511+0000 7f97240c7b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618357dd680 0x5618357e1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:05.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.511+0000 7f97240c7b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618357e1ac0 0x5618357e1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:05.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.511+0000 7f97240c7b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618356e0060 con 0x5618357dcd70 2023-11-27T15:28:05.518 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.511+0000 7f97240c7b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618357328a0 con 0x5618357e1ac0 2023-11-27T15:28:05.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.511+0000 7f97240c7b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56183572bd30 con 0x5618357dd680 2023-11-27T15:28:05.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.512+0000 7f9715cd7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618357dcd70 0x5618357dd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:05.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.512+0000 7f9715cd7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618357dcd70 0x5618357dd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56246/0 (socket says 172.21.15.52:56246) 2023-11-27T15:28:05.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.512+0000 7f9715cd7700 1 -- 172.21.15.52:0/1384660095 learned_addr learned my addr 172.21.15.52:0/1384660095 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:05.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.512+0000 7f97164d8700 1 --2- 172.21.15.52:0/1384660095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618357e1ac0 0x5618357e1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:05.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.512+0000 7f97154d6700 1 --2- 172.21.15.52:0/1384660095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618357dd680 0x5618357e1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:05.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.512+0000 7f9715cd7700 1 -- 172.21.15.52:0/1384660095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618357dd680 msgr2=0x5618357e1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:05.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.512+0000 7f9715cd7700 1 --2- 172.21.15.52:0/1384660095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618357dd680 0x5618357e1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.519 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.512+0000 7f9715cd7700 1 -- 172.21.15.52:0/1384660095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618357e1ac0 msgr2=0x5618357e1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:05.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.512+0000 7f9715cd7700 1 --2- 172.21.15.52:0/1384660095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618357e1ac0 0x5618357e1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.512+0000 7f9715cd7700 1 -- 172.21.15.52:0/1384660095 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618355ad630 con 0x5618357dcd70 2023-11-27T15:28:05.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.512+0000 7f9715cd7700 1 --2- 172.21.15.52:0/1384660095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618357dcd70 0x5618357dd140 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f97000960a0 tx=0x7f9700099ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1254865c8da3e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:05.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.512+0000 7f96fbfff700 1 -- 172.21.15.52:0/1384660095 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97000a1ff0 con 0x5618357dcd70 2023-11-27T15:28:05.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.512+0000 7f96fbfff700 1 -- 172.21.15.52:0/1384660095 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9700097f40 con 0x5618357dcd70 2023-11-27T15:28:05.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.512+0000 7f96fbfff700 1 -- 172.21.15.52:0/1384660095 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97000992f0 con 0x5618357dcd70 2023-11-27T15:28:05.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.513+0000 7f97240c7b80 1 -- 172.21.15.52:0/1384660095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618357dcd70 msgr2=0x5618357dd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:05.520 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.513+0000 7f97240c7b80 1 --2- 172.21.15.52:0/1384660095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618357dcd70 0x5618357dd140 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f97000960a0 tx=0x7f9700099ba0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.513+0000 7f97240c7b80 1 -- 172.21.15.52:0/1384660095 shutdown_connections 2023-11-27T15:28:05.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.513+0000 7f97240c7b80 1 --2- 172.21.15.52:0/1384660095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618357e1ac0 0x5618357e1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.513+0000 7f97240c7b80 1 --2- 172.21.15.52:0/1384660095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618357dcd70 0x5618357dd140 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.513+0000 7f97240c7b80 1 --2- 172.21.15.52:0/1384660095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618357dd680 0x5618357e1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.513+0000 7f97240c7b80 1 -- 172.21.15.52:0/1384660095 >> 172.21.15.52:0/1384660095 conn(0x5618357d0360 msgr2=0x5618357d5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:05.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.513+0000 7f97240c7b80 1 -- 172.21.15.52:0/1384660095 shutdown_connections 2023-11-27T15:28:05.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.513+0000 7f97240c7b80 1 -- 172.21.15.52:0/1384660095 wait complete. 2023-11-27T15:28:05.521 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.514+0000 7f97240c7b80 1 -- start start 2023-11-27T15:28:05.522 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.514+0000 7f97240c7b80 1 Processor -- start 2023-11-27T15:28:05.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.514+0000 7f97240c7b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618357dcd70 0x5618357e92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:05.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.514+0000 7f97240c7b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618357dd680 0x5618357e6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:05.525 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.514+0000 7f97240c7b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618357e1ac0 0x5618357e6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:05.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.514+0000 7f97240c7b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618356cd8e0 con 0x5618357e1ac0 2023-11-27T15:28:05.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.514+0000 7f97240c7b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618356cd580 con 0x5618357dd680 2023-11-27T15:28:05.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.514+0000 7f97240c7b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618355af920 con 0x5618357dcd70 2023-11-27T15:28:05.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.514+0000 7f97240c7b80 10 client.0 _ll_register_callbacks cb 0x5618356eb780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:05.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.514+0000 7f97164d8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618357e1ac0 0x5618357e6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:05.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.514+0000 7f97164d8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618357e1ac0 0x5618357e6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56256/0 (socket says 172.21.15.52:56256) 2023-11-27T15:28:05.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.514+0000 7f97164d8700 1 -- 172.21.15.52:0/1410954948 learned_addr learned my addr 172.21.15.52:0/1410954948 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:05.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.514+0000 7f9715cd7700 1 --2- 172.21.15.52:0/1410954948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618357dcd70 0x5618357e92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:05.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.514+0000 7f97154d6700 1 --2- 172.21.15.52:0/1410954948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618357dd680 0x5618357e6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:05.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.514+0000 7f97164d8700 1 -- 172.21.15.52:0/1410954948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618357dcd70 msgr2=0x5618357e92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:05.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.514+0000 7f97164d8700 1 --2- 172.21.15.52:0/1410954948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618357dcd70 0x5618357e92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.515+0000 7f97164d8700 1 -- 172.21.15.52:0/1410954948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618357dd680 msgr2=0x5618357e6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:05.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.515+0000 7f97164d8700 1 --2- 172.21.15.52:0/1410954948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618357dd680 0x5618357e6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.515+0000 7f97164d8700 1 -- 172.21.15.52:0/1410954948 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f971010f150 con 0x5618357e1ac0 2023-11-27T15:28:05.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.515+0000 7f97154d6700 1 --2- 172.21.15.52:0/1410954948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618357dd680 0x5618357e6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:28:05.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.515+0000 7f9715cd7700 1 --2- 172.21.15.52:0/1410954948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618357dcd70 0x5618357e92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:28:05.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.515+0000 7f97164d8700 1 --2- 172.21.15.52:0/1410954948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618357e1ac0 0x5618357e6c80 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f9708009cd0 tx=0x7f9708003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20dba0fa62dd1b44 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:05.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.515+0000 7f96f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:28:05.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.515+0000 7f96f97fa700 1 -- 172.21.15.52:0/1410954948 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970800e030 con 0x5618357e1ac0 2023-11-27T15:28:05.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.515+0000 7f97240c7b80 1 -- 172.21.15.52:0/1410954948 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5618357cdb20 con 0x5618357e1ac0 2023-11-27T15:28:05.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.515+0000 7f96f97fa700 1 -- 172.21.15.52:0/1410954948 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f9708007970 con 0x5618357e1ac0 2023-11-27T15:28:05.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.515+0000 7f96f97fa700 1 -- 172.21.15.52:0/1410954948 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9708026470 con 0x5618357e1ac0 2023-11-27T15:28:05.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.515+0000 7f97240c7b80 10 client.17205 fetch_fsmap learned FSMap version 34 2023-11-27T15:28:05.528 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.515+0000 7f97240c7b80 1 -- 172.21.15.52:0/1410954948 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5618356f2190 con 0x5618357e1ac0 2023-11-27T15:28:05.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.515+0000 7f96f97fa700 1 -- 172.21.15.52:0/1410954948 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5618357cdb20 con 0x5618357e1ac0 2023-11-27T15:28:05.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.515+0000 7f96f97fa700 1 -- 172.21.15.52:0/1410954948 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9708019070 con 0x5618357e1ac0 2023-11-27T15:28:05.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.515+0000 7f97240c7b80 10 client.17205 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:28:05.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.515+0000 7f97240c7b80 10 client.17205 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:28:05.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.515+0000 7f97240c7b80 20 client.17205 populate_metadata read hostname 'smithi052' 2023-11-27T15:28:05.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.516+0000 7f97240c7b80 1 --2- 172.21.15.52:0/1410954948 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5618357f6970 0x5618357f8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:05.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.516+0000 7f97240c7b80 4 client.17205 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:28:05.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.516+0000 7f97240c7b80 1 -- 172.21.15.52:0/1410954948 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5618356ee0a0 con 0x5618357f6970 2023-11-27T15:28:05.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.516+0000 7f9715cd7700 1 --2- 172.21.15.52:0/1410954948 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5618357f6970 0x5618357f8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:05.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.516+0000 7f9715cd7700 1 --2- 172.21.15.52:0/1410954948 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5618357f6970 0x5618357f8d80 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4faffe0c1826ad server_cookie=a1be3c8e19fcde25 in_seq=0 out_seq=0 2023-11-27T15:28:05.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.516+0000 7f96f97fa700 10 client.17205 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:28:05.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.517+0000 7f96f97fa700 1 -- 172.21.15.52:0/1410954948 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5618356ee0a0 con 0x5618357f6970 2023-11-27T15:28:05.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.517+0000 7f96f97fa700 10 client.17205 handle_command_reply: tid=0 2023-11-27T15:28:05.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.517+0000 7f97240c7b80 1 client.17205 shutdown 2023-11-27T15:28:05.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.517+0000 7f97240c7b80 1 -- 172.21.15.52:0/1410954948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618357e1ac0 msgr2=0x5618357e6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:05.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.517+0000 7f97240c7b80 1 --2- 172.21.15.52:0/1410954948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618357e1ac0 0x5618357e6c80 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f9708009cd0 tx=0x7f9708003d50 comp rx=0 tx=0).stop 2023-11-27T15:28:05.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.517+0000 7f97240c7b80 1 -- 172.21.15.52:0/1410954948 shutdown_connections 2023-11-27T15:28:05.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.517+0000 7f97240c7b80 1 --2- 172.21.15.52:0/1410954948 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x5618357f6970 0x5618357f8d80 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.517+0000 7f97240c7b80 1 --2- 172.21.15.52:0/1410954948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618357e1ac0 0x5618357e6c80 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.517+0000 7f96f97fa700 0 client.17205 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:28:05.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.517+0000 7f97240c7b80 1 --2- 172.21.15.52:0/1410954948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618357dcd70 0x5618357e92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.517+0000 7f97240c7b80 1 --2- 172.21.15.52:0/1410954948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618357dd680 0x5618357e6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.517+0000 7f97240c7b80 1 -- 172.21.15.52:0/1410954948 >> 172.21.15.52:0/1410954948 conn(0x5618357d0360 msgr2=0x5618357e2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:05.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.517+0000 7f97240c7b80 1 -- 172.21.15.52:0/1410954948 shutdown_connections 2023-11-27T15:28:05.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.517+0000 7f97240c7b80 1 -- 172.21.15.52:0/1410954948 wait complete. 2023-11-27T15:28:05.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.517+0000 7f97240c7b80 20 client.17205 trim_cache size 0 max 16384 2023-11-27T15:28:05.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.534+0000 7f97240c7b80 1 Processor -- start 2023-11-27T15:28:05.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.534+0000 7f97240c7b80 1 -- start start 2023-11-27T15:28:05.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.534+0000 7f97240c7b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618357dcd70 0x5618357df3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:05.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.534+0000 7f97240c7b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618357dd680 0x5618357e6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:05.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.534+0000 7f97240c7b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618357e1ac0 0x5618357eda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:05.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.534+0000 7f97240c7b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618357e35e0 con 0x5618357dcd70 2023-11-27T15:28:05.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.534+0000 7f97240c7b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618357e0720 con 0x5618357dd680 2023-11-27T15:28:05.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.534+0000 7f97240c7b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618357e7700 con 0x5618357e1ac0 2023-11-27T15:28:05.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.535+0000 7f9715cd7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618357dcd70 0x5618357df3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:05.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.535+0000 7f9715cd7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618357dcd70 0x5618357df3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56260/0 (socket says 172.21.15.52:56260) 2023-11-27T15:28:05.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.535+0000 7f9715cd7700 1 -- 172.21.15.52:0/3334149663 learned_addr learned my addr 172.21.15.52:0/3334149663 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:05.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.535+0000 7f97154d6700 1 --2- 172.21.15.52:0/3334149663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618357dd680 0x5618357e6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:05.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.535+0000 7f9715cd7700 1 -- 172.21.15.52:0/3334149663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618357e1ac0 msgr2=0x5618357eda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:05.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.535+0000 7f9715cd7700 1 --2- 172.21.15.52:0/3334149663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618357e1ac0 0x5618357eda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.535+0000 7f9715cd7700 1 -- 172.21.15.52:0/3334149663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618357dd680 msgr2=0x5618357e6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:05.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.535+0000 7f9715cd7700 1 --2- 172.21.15.52:0/3334149663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618357dd680 0x5618357e6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.535+0000 7f9715cd7700 1 -- 172.21.15.52:0/3334149663 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618357d1b00 con 0x5618357dcd70 2023-11-27T15:28:05.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.535+0000 7f97154d6700 1 --2- 172.21.15.52:0/3334149663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618357dd680 0x5618357e6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:28:05.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.535+0000 7f9715cd7700 1 --2- 172.21.15.52:0/3334149663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618357dcd70 0x5618357df3b0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f97000998a0 tx=0x7f9700096010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd47d9e51b840cb server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:05.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.535+0000 7f9704ff9700 1 -- 172.21.15.52:0/3334149663 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9700096330 con 0x5618357dcd70 2023-11-27T15:28:05.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.535+0000 7f9704ff9700 1 -- 172.21.15.52:0/3334149663 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97000b2180 con 0x5618357dcd70 2023-11-27T15:28:05.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.535+0000 7f9704ff9700 1 -- 172.21.15.52:0/3334149663 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9700099e00 con 0x5618357dcd70 2023-11-27T15:28:05.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.536+0000 7f97240c7b80 1 -- 172.21.15.52:0/3334149663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618357dcd70 msgr2=0x5618357df3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:05.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.536+0000 7f97240c7b80 1 --2- 172.21.15.52:0/3334149663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618357dcd70 0x5618357df3b0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f97000998a0 tx=0x7f9700096010 comp rx=0 tx=0).stop 2023-11-27T15:28:05.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.536+0000 7f97240c7b80 1 -- 172.21.15.52:0/3334149663 shutdown_connections 2023-11-27T15:28:05.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.536+0000 7f97240c7b80 1 --2- 172.21.15.52:0/3334149663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618357e1ac0 0x5618357eda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.536+0000 7f97240c7b80 1 --2- 172.21.15.52:0/3334149663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618357dcd70 0x5618357df3b0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.536+0000 7f97240c7b80 1 --2- 172.21.15.52:0/3334149663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618357dd680 0x5618357e6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.536+0000 7f97240c7b80 1 -- 172.21.15.52:0/3334149663 >> 172.21.15.52:0/3334149663 conn(0x5618357d0360 msgr2=0x5618357dfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:05.545 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.536+0000 7f97240c7b80 1 -- 172.21.15.52:0/3334149663 shutdown_connections 2023-11-27T15:28:05.546 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.536+0000 7f97240c7b80 1 -- 172.21.15.52:0/3334149663 wait complete. 2023-11-27T15:28:05.546 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.536+0000 7f97240c7b80 1 -- start start 2023-11-27T15:28:05.546 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.536+0000 7f97240c7b80 1 Processor -- start 2023-11-27T15:28:05.546 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.537+0000 7f97240c7b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618357dcd70 0x5618357d5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:05.546 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.537+0000 7f97240c7b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618357dd680 0x5618357d5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:05.546 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.537+0000 7f97240c7b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618357e1ac0 0x5618357d4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:05.546 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.537+0000 7f97240c7b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618357d4ac0 con 0x5618357dd680 2023-11-27T15:28:05.546 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.537+0000 7f97240c7b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618357d4c30 con 0x5618357e1ac0 2023-11-27T15:28:05.547 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.537+0000 7f97240c7b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618357d4df0 con 0x5618357dcd70 2023-11-27T15:28:05.547 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.537+0000 7f97240c7b80 10 client.0 _ll_register_callbacks cb 0x7f9708009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:05.547 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.537+0000 7f97154d6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618357dd680 0x5618357d5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:05.547 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.537+0000 7f97154d6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618357dd680 0x5618357d5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56268/0 (socket says 172.21.15.52:56268) 2023-11-27T15:28:05.547 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.537+0000 7f97154d6700 1 -- 172.21.15.52:0/98607964 learned_addr learned my addr 172.21.15.52:0/98607964 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:05.547 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.537+0000 7f9715cd7700 1 --2- 172.21.15.52:0/98607964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618357dcd70 0x5618357d5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:05.547 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.537+0000 7f97164d8700 1 --2- 172.21.15.52:0/98607964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618357e1ac0 0x5618357d4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:05.547 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.537+0000 7f97154d6700 1 -- 172.21.15.52:0/98607964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618357dcd70 msgr2=0x5618357d5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:05.548 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.537+0000 7f97154d6700 1 --2- 172.21.15.52:0/98607964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618357dcd70 0x5618357d5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.548 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.537+0000 7f97154d6700 1 -- 172.21.15.52:0/98607964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618357e1ac0 msgr2=0x5618357d4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:05.548 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.537+0000 7f97154d6700 1 --2- 172.21.15.52:0/98607964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618357e1ac0 0x5618357d4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.548 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.537+0000 7f97154d6700 1 -- 172.21.15.52:0/98607964 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618355ad630 con 0x5618357dd680 2023-11-27T15:28:05.548 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.537+0000 7f97154d6700 1 --2- 172.21.15.52:0/98607964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618357dd680 0x5618357d5c60 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f970c011490 tx=0x7f970c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5585b62a2519fc5 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:05.548 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.537+0000 7f96f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:28:05.548 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.537+0000 7f96f97fa700 1 -- 172.21.15.52:0/98607964 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c024070 con 0x5618357dd680 2023-11-27T15:28:05.548 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.537+0000 7f96f97fa700 1 -- 172.21.15.52:0/98607964 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f970c017070 con 0x5618357dd680 2023-11-27T15:28:05.549 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.537+0000 7f97240c7b80 1 -- 172.21.15.52:0/98607964 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9700097f40 con 0x5618357dd680 2023-11-27T15:28:05.549 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.538+0000 7f96f97fa700 1 -- 172.21.15.52:0/98607964 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f970c011ae0 con 0x5618357dd680 2023-11-27T15:28:05.549 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.538+0000 7f97240c7b80 10 client.17211 fetch_fsmap learned FSMap version 34 2023-11-27T15:28:05.549 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.538+0000 7f97240c7b80 1 -- 172.21.15.52:0/98607964 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x561835559de0 con 0x5618357dd680 2023-11-27T15:28:05.549 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.538+0000 7f96f97fa700 1 -- 172.21.15.52:0/98607964 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9700097f40 con 0x5618357dd680 2023-11-27T15:28:05.549 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.538+0000 7f96f97fa700 1 -- 172.21.15.52:0/98607964 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f970c003a50 con 0x5618357dd680 2023-11-27T15:28:05.549 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.538+0000 7f97240c7b80 10 client.17211 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:28:05.549 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.538+0000 7f97240c7b80 10 client.17211 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:28:05.550 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.538+0000 7f97240c7b80 20 client.17211 populate_metadata read hostname 'smithi052' 2023-11-27T15:28:05.550 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.538+0000 7f97240c7b80 1 --2- 172.21.15.52:0/98607964 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x56183584b880 0x56183584dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:05.550 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.538+0000 7f97240c7b80 4 client.17211 mds_command: new command op to 24536 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:28:05.550 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.538+0000 7f97240c7b80 1 -- 172.21.15.52:0/98607964 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5618356fa660 con 0x56183584b880 2023-11-27T15:28:05.550 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.538+0000 7f9715cd7700 1 --2- 172.21.15.52:0/98607964 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x56183584b880 0x56183584dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:05.550 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.539+0000 7f96f97fa700 10 client.17211 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:28:05.550 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.539+0000 7f9715cd7700 1 --2- 172.21.15.52:0/98607964 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x56183584b880 0x56183584dc90 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d233d5b049587ea0 server_cookie=96050302edffcd4b in_seq=0 out_seq=0 2023-11-27T15:28:05.551 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.539+0000 7f96f97fa700 1 -- 172.21.15.52:0/98607964 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5618356fa660 con 0x56183584b880 2023-11-27T15:28:05.552 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.539+0000 7f96f97fa700 10 client.17211 handle_command_reply: tid=0 2023-11-27T15:28:05.552 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.539+0000 7f97240c7b80 1 client.17211 shutdown 2023-11-27T15:28:05.552 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.539+0000 7f97240c7b80 1 -- 172.21.15.52:0/98607964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618357dd680 msgr2=0x5618357d5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:05.552 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.539+0000 7f97240c7b80 1 --2- 172.21.15.52:0/98607964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618357dd680 0x5618357d5c60 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f970c011490 tx=0x7f970c0047c0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.552 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.539+0000 7f97240c7b80 1 -- 172.21.15.52:0/98607964 shutdown_connections 2023-11-27T15:28:05.552 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.539+0000 7f97240c7b80 1 --2- 172.21.15.52:0/98607964 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x56183584b880 0x56183584dc90 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.552 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.539+0000 7f97240c7b80 1 --2- 172.21.15.52:0/98607964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618357e1ac0 0x5618357d4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.539+0000 7f97240c7b80 1 --2- 172.21.15.52:0/98607964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5618357dcd70 0x5618357d5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.539+0000 7f96f97fa700 0 client.17211 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:28:05.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.539+0000 7f97240c7b80 1 --2- 172.21.15.52:0/98607964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618357dd680 0x5618357d5c60 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.539+0000 7f97240c7b80 1 -- 172.21.15.52:0/98607964 >> 172.21.15.52:0/98607964 conn(0x5618357d0360 msgr2=0x5618357e0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:05.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.539+0000 7f97240c7b80 1 -- 172.21.15.52:0/98607964 shutdown_connections 2023-11-27T15:28:05.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.539+0000 7f97240c7b80 1 -- 172.21.15.52:0/98607964 wait complete. 2023-11-27T15:28:05.553 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.539+0000 7f97240c7b80 20 client.17211 trim_cache size 0 max 16384 2023-11-27T15:28:05.554 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:28:05.554 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2023-11-27T15:28:05.554 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2023-11-27T15:28:05.554 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:28:05.555 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.540+0000 7f9717f3b700 1 -- 172.21.15.52:0/65834400 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f96fc0a12c0 msgr2=0x7f96fc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:05.555 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.540+0000 7f9717f3b700 1 --2- 172.21.15.52:0/65834400 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f96fc0a12c0 0x7f96fc0a3770 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f97101969e0 tx=0x7f970c0073d0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.556 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.540+0000 7f9717f3b700 1 -- 172.21.15.52:0/65834400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97100ff4c0 msgr2=0x7f97101953d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:05.556 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.540+0000 7f9717f3b700 1 --2- 172.21.15.52:0/65834400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97100ff4c0 0x7f97101953d0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f970000d950 tx=0x7f970000dc60 comp rx=0 tx=0).stop 2023-11-27T15:28:05.556 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.540+0000 7f9717f3b700 1 -- 172.21.15.52:0/65834400 shutdown_connections 2023-11-27T15:28:05.556 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.540+0000 7f9717f3b700 1 --2- 172.21.15.52:0/65834400 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f96fc0a12c0 0x7f96fc0a3770 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.556 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.540+0000 7f9717f3b700 1 --2- 172.21.15.52:0/65834400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9710104500 0x7f9710199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.556 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.540+0000 7f9717f3b700 1 --2- 172.21.15.52:0/65834400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97100ff4c0 0x7f97101953d0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.556 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.540+0000 7f9717f3b700 1 --2- 172.21.15.52:0/65834400 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97100ffe60 0x7f9710195910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.557 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.540+0000 7f9717f3b700 1 -- 172.21.15.52:0/65834400 >> 172.21.15.52:0/65834400 conn(0x7f9710076200 msgr2=0x7f97100fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:05.557 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.540+0000 7f9717f3b700 1 -- 172.21.15.52:0/65834400 shutdown_connections 2023-11-27T15:28:05.557 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.540+0000 7f9717f3b700 1 -- 172.21.15.52:0/65834400 wait complete. 2023-11-27T15:28:05.557 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-27T15:28:05.557 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:28:05.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.680+0000 7fcd5ed60700 1 -- 172.21.15.52:0/3825233458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd58100a50 msgr2=0x7fcd58100ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:05.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.680+0000 7fcd5ed60700 1 --2- 172.21.15.52:0/3825233458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd58100a50 0x7fcd58100ea0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fcd44009990 tx=0x7fcd44009ca0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.680+0000 7fcd5ed60700 1 -- 172.21.15.52:0/3825233458 shutdown_connections 2023-11-27T15:28:05.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.680+0000 7fcd5ed60700 1 --2- 172.21.15.52:0/3825233458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd581013e0 0x7fcd5810d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.680+0000 7fcd5ed60700 1 --2- 172.21.15.52:0/3825233458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd581062d0 0x7fcd581066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.680+0000 7fcd5ed60700 1 --2- 172.21.15.52:0/3825233458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd58100a50 0x7fcd58100ea0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.680+0000 7fcd5ed60700 1 -- 172.21.15.52:0/3825233458 >> 172.21.15.52:0/3825233458 conn(0x7fcd580774a0 msgr2=0x7fcd580778a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:05.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.680+0000 7fcd5ed60700 1 -- 172.21.15.52:0/3825233458 shutdown_connections 2023-11-27T15:28:05.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.680+0000 7fcd5ed60700 1 -- 172.21.15.52:0/3825233458 wait complete. 2023-11-27T15:28:05.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.680+0000 7fcd5ed60700 1 Processor -- start 2023-11-27T15:28:05.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.681+0000 7fcd5ed60700 1 -- start start 2023-11-27T15:28:05.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.681+0000 7fcd5ed60700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd58100a50 0x7fcd58196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:05.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.681+0000 7fcd5ed60700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd581013e0 0x7fcd581968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:05.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.681+0000 7fcd5ed60700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd581062d0 0x7fcd5819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:05.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.681+0000 7fcd5ed60700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5819b350 con 0x7fcd581013e0 2023-11-27T15:28:05.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.681+0000 7fcd5ed60700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5819b4c0 con 0x7fcd58100a50 2023-11-27T15:28:05.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.681+0000 7fcd5ed60700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5819b680 con 0x7fcd581062d0 2023-11-27T15:28:05.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.681+0000 7fcd57fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd581013e0 0x7fcd581968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:05.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.681+0000 7fcd5cafc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd58100a50 0x7fcd58196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:05.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.681+0000 7fcd5cafc700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd58100a50 0x7fcd58196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.52:45938/0 (socket says 172.21.15.52:45938) 2023-11-27T15:28:05.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.681+0000 7fcd5cafc700 1 -- 172.21.15.52:0/3180186663 learned_addr learned my addr 172.21.15.52:0/3180186663 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:05.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.681+0000 7fcd57fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd581013e0 0x7fcd581968b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56282/0 (socket says 172.21.15.52:56282) 2023-11-27T15:28:05.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.681+0000 7fcd5d2fd700 1 --2- 172.21.15.52:0/3180186663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd581062d0 0x7fcd5819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:05.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.681+0000 7fcd57fff700 1 -- 172.21.15.52:0/3180186663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd581062d0 msgr2=0x7fcd5819ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:05.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.681+0000 7fcd57fff700 1 --2- 172.21.15.52:0/3180186663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd581062d0 0x7fcd5819ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.681+0000 7fcd57fff700 1 -- 172.21.15.52:0/3180186663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd58100a50 msgr2=0x7fcd58196370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:05.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.681+0000 7fcd57fff700 1 --2- 172.21.15.52:0/3180186663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd58100a50 0x7fcd58196370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.681+0000 7fcd57fff700 1 -- 172.21.15.52:0/3180186663 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd440095f0 con 0x7fcd581013e0 2023-11-27T15:28:05.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.681+0000 7fcd57fff700 1 --2- 172.21.15.52:0/3180186663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd581013e0 0x7fcd581968b0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fcd44009990 tx=0x7fcd44009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:05.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.682+0000 7fcd55ffb700 1 -- 172.21.15.52:0/3180186663 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4401d070 con 0x7fcd581013e0 2023-11-27T15:28:05.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.682+0000 7fcd55ffb700 1 -- 172.21.15.52:0/3180186663 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd44021410 con 0x7fcd581013e0 2023-11-27T15:28:05.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.682+0000 7fcd55ffb700 1 -- 172.21.15.52:0/3180186663 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd44017650 con 0x7fcd581013e0 2023-11-27T15:28:05.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.682+0000 7fcd5ed60700 1 -- 172.21.15.52:0/3180186663 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd58110330 con 0x7fcd581013e0 2023-11-27T15:28:05.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.682+0000 7fcd5ed60700 1 -- 172.21.15.52:0/3180186663 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd581101a0 con 0x7fcd581013e0 2023-11-27T15:28:05.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.683+0000 7fcd5ed60700 1 -- 172.21.15.52:0/3180186663 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd581104c0 con 0x7fcd581013e0 2023-11-27T15:28:05.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.684+0000 7fcd55ffb700 1 -- 172.21.15.52:0/3180186663 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fcd44021810 con 0x7fcd581013e0 2023-11-27T15:28:05.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.684+0000 7fcd55ffb700 1 --2- 172.21.15.52:0/3180186663 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fcd480a12c0 0x7fcd480a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:05.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.685+0000 7fcd55ffb700 1 -- 172.21.15.52:0/3180186663 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fcd440cdff0 con 0x7fcd581013e0 2023-11-27T15:28:05.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.685+0000 7fcd5cafc700 1 --2- 172.21.15.52:0/3180186663 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fcd480a12c0 0x7fcd480a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:05.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.685+0000 7fcd5cafc700 1 --2- 172.21.15.52:0/3180186663 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fcd480a12c0 0x7fcd480a3770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c009ca0 tx=0x7fcd4c009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:05.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.689+0000 7fcd55ffb700 1 -- 172.21.15.52:0/3180186663 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcd44097460 con 0x7fcd581013e0 2023-11-27T15:28:05.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.933+0000 7fcd5ed60700 1 -- 172.21.15.52:0/3180186663 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd581a33e0 con 0x7fcd581013e0 2023-11-27T15:28:05.937 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.934+0000 7fcd55ffb700 1 -- 172.21.15.52:0/3180186663 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10804 (secure 0 0 0) 0x7fcd4409aa80 con 0x7fcd581013e0 2023-11-27T15:28:05.938 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:28:05.938 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2328881376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2328881376},{"type":"v1","addr":"172.21.15.52:6839","nonce":2328881376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2799551570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2799551570},{"type":"v1","addr":"172.21.15.52:6841","nonce":2799551570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:16:09.319630+0000","modified":"2023-11-27T15:26:15.243744+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24494,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/215725838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":215725838},{"type":"v1","addr":"172.21.15.150:6839","nonce":215725838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3571830559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3571830559},{"type":"v1","addr":"172.21.15.102:6839","nonce":3571830559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:28:05.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.936+0000 7fcd5ed60700 1 -- 172.21.15.52:0/3180186663 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fcd480a12c0 msgr2=0x7fcd480a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:05.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.936+0000 7fcd5ed60700 1 --2- 172.21.15.52:0/3180186663 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fcd480a12c0 0x7fcd480a3770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c009ca0 tx=0x7fcd4c009260 comp rx=0 tx=0).stop 2023-11-27T15:28:05.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.936+0000 7fcd5ed60700 1 -- 172.21.15.52:0/3180186663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd581013e0 msgr2=0x7fcd581968b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:05.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.936+0000 7fcd5ed60700 1 --2- 172.21.15.52:0/3180186663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd581013e0 0x7fcd581968b0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fcd44009990 tx=0x7fcd44009580 comp rx=0 tx=0).stop 2023-11-27T15:28:05.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.936+0000 7fcd5ed60700 1 -- 172.21.15.52:0/3180186663 shutdown_connections 2023-11-27T15:28:05.940 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.936+0000 7fcd5ed60700 1 --2- 172.21.15.52:0/3180186663 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fcd480a12c0 0x7fcd480a3770 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.936+0000 7fcd5ed60700 1 --2- 172.21.15.52:0/3180186663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd58100a50 0x7fcd58196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.936+0000 7fcd5ed60700 1 --2- 172.21.15.52:0/3180186663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd581062d0 0x7fcd5819ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.936+0000 7fcd5ed60700 1 --2- 172.21.15.52:0/3180186663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd581013e0 0x7fcd581968b0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:05.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.936+0000 7fcd5ed60700 1 -- 172.21.15.52:0/3180186663 >> 172.21.15.52:0/3180186663 conn(0x7fcd580774a0 msgr2=0x7fcd580ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:05.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.936+0000 7fcd5ed60700 1 -- 172.21.15.52:0/3180186663 shutdown_connections 2023-11-27T15:28:05.941 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:05.936+0000 7fcd5ed60700 1 -- 172.21.15.52:0/3180186663 wait complete. 2023-11-27T15:28:05.942 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 34 2023-11-27T15:28:05.952 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-27T15:28:06.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.078+0000 7f147930b700 1 -- 172.21.15.52:0/2638037023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14740ff450 msgr2=0x7f14740ff820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:06.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.078+0000 7f147930b700 1 --2- 172.21.15.52:0/2638037023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14740ff450 0x7f14740ff820 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f14680099c0 tx=0x7f1468009cd0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.078+0000 7f147930b700 1 -- 172.21.15.52:0/2638037023 shutdown_connections 2023-11-27T15:28:06.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.078+0000 7f147930b700 1 --2- 172.21.15.52:0/2638037023 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1474104490 0x7f147410c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.078+0000 7f147930b700 1 --2- 172.21.15.52:0/2638037023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14740ff450 0x7f14740ff820 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.078+0000 7f147930b700 1 --2- 172.21.15.52:0/2638037023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14740ffdf0 0x7f1474103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.078+0000 7f147930b700 1 -- 172.21.15.52:0/2638037023 >> 172.21.15.52:0/2638037023 conn(0x7f14740770b0 msgr2=0x7f14740774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:06.083 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.078+0000 7f147930b700 1 -- 172.21.15.52:0/2638037023 shutdown_connections 2023-11-27T15:28:06.083 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.078+0000 7f147930b700 1 -- 172.21.15.52:0/2638037023 wait complete. 2023-11-27T15:28:06.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.079+0000 7f147930b700 1 Processor -- start 2023-11-27T15:28:06.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.079+0000 7f147930b700 1 -- start start 2023-11-27T15:28:06.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.079+0000 7f147930b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14740ff450 0x7f14741953d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.079+0000 7f147930b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14740ffdf0 0x7f1474195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.079+0000 7f147930b700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1474104490 0x7f1474199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.085 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.080+0000 7f147930b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147419a3b0 con 0x7f14740ffdf0 2023-11-27T15:28:06.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.080+0000 7f147930b700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147419a520 con 0x7f1474104490 2023-11-27T15:28:06.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.080+0000 7f147930b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147419a6e0 con 0x7f14740ff450 2023-11-27T15:28:06.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.080+0000 7f14727fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14740ffdf0 0x7f1474195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.080+0000 7f14727fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14740ffdf0 0x7f1474195910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56296/0 (socket says 172.21.15.52:56296) 2023-11-27T15:28:06.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.080+0000 7f14727fc700 1 -- 172.21.15.52:0/1463507330 learned_addr learned my addr 172.21.15.52:0/1463507330 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:06.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.080+0000 7f1472ffd700 1 --2- 172.21.15.52:0/1463507330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14740ff450 0x7f14741953d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.080+0000 7f14737fe700 1 --2- 172.21.15.52:0/1463507330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1474104490 0x7f1474199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.086 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.080+0000 7f14727fc700 1 -- 172.21.15.52:0/1463507330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14740ff450 msgr2=0x7f14741953d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:06.087 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.080+0000 7f14727fc700 1 --2- 172.21.15.52:0/1463507330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14740ff450 0x7f14741953d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.087 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.080+0000 7f14727fc700 1 -- 172.21.15.52:0/1463507330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1474104490 msgr2=0x7f1474199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:06.087 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.080+0000 7f14727fc700 1 --2- 172.21.15.52:0/1463507330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1474104490 0x7f1474199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.087 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.080+0000 7f14727fc700 1 -- 172.21.15.52:0/1463507330 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14680095f0 con 0x7f14740ffdf0 2023-11-27T15:28:06.087 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.080+0000 7f14727fc700 1 --2- 172.21.15.52:0/1463507330 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14740ffdf0 0x7f1474195910 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f146400d980 tx=0x7f146400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:06.087 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.080+0000 7f145bfff700 1 -- 172.21.15.52:0/1463507330 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1464009b40 con 0x7f14740ffdf0 2023-11-27T15:28:06.087 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.080+0000 7f145bfff700 1 -- 172.21.15.52:0/1463507330 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f146400be20 con 0x7f14740ffdf0 2023-11-27T15:28:06.087 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.080+0000 7f147930b700 1 -- 172.21.15.52:0/1463507330 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f147410f2c0 con 0x7f14740ffdf0 2023-11-27T15:28:06.087 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.080+0000 7f145bfff700 1 -- 172.21.15.52:0/1463507330 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14640054e0 con 0x7f14740ffdf0 2023-11-27T15:28:06.088 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.080+0000 7f147930b700 1 -- 172.21.15.52:0/1463507330 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f147410f130 con 0x7f14740ffdf0 2023-11-27T15:28:06.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.082+0000 7f145bfff700 1 -- 172.21.15.52:0/1463507330 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f1464004750 con 0x7f14740ffdf0 2023-11-27T15:28:06.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.083+0000 7f145bfff700 1 --2- 172.21.15.52:0/1463507330 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f145c0a12c0 0x7f145c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.083+0000 7f145bfff700 1 -- 172.21.15.52:0/1463507330 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f14640cc690 con 0x7f14740ffdf0 2023-11-27T15:28:06.094 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.083+0000 7f1472ffd700 1 --2- 172.21.15.52:0/1463507330 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f145c0a12c0 0x7f145c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.084+0000 7f1472ffd700 1 --2- 172.21.15.52:0/1463507330 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f145c0a12c0 0x7f145c0a3770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1468009fd0 tx=0x7f1468005fb0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:06.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.084+0000 7f1485497b80 1 Processor -- start 2023-11-27T15:28:06.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.084+0000 7f1485497b80 1 -- start start 2023-11-27T15:28:06.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.084+0000 7f1485497b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555633789b60 0x555633789f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.084+0000 7f1485497b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55563378a470 0x55563378e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.084+0000 7f1485497b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55563378e8b0 0x55563378ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.084+0000 7f1485497b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55563368f060 con 0x55563378e8b0 2023-11-27T15:28:06.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.084+0000 7f1485497b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556336e18a0 con 0x555633789b60 2023-11-27T15:28:06.095 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.084+0000 7f1485497b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556336dad30 con 0x55563378a470 2023-11-27T15:28:06.096 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.084+0000 7f14737fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55563378e8b0 0x55563378ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.096 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.085+0000 7f14737fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55563378e8b0 0x55563378ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56306/0 (socket says 172.21.15.52:56306) 2023-11-27T15:28:06.096 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.085+0000 7f1472ffd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555633789b60 0x555633789f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.096 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.085+0000 7f14737fe700 1 -- 172.21.15.52:0/364524246 learned_addr learned my addr 172.21.15.52:0/364524246 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:06.096 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.085+0000 7f14727fc700 1 --2- 172.21.15.52:0/364524246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55563378a470 0x55563378e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.096 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.085+0000 7f14737fe700 1 -- 172.21.15.52:0/364524246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55563378a470 msgr2=0x55563378e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:06.096 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.085+0000 7f14737fe700 1 --2- 172.21.15.52:0/364524246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55563378a470 0x55563378e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.096 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.085+0000 7f14737fe700 1 -- 172.21.15.52:0/364524246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555633789b60 msgr2=0x555633789f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:06.097 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.085+0000 7f14737fe700 1 --2- 172.21.15.52:0/364524246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555633789b60 0x555633789f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.097 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.085+0000 7f14737fe700 1 -- 172.21.15.52:0/364524246 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55563355c630 con 0x55563378e8b0 2023-11-27T15:28:06.097 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.085+0000 7f14737fe700 1 --2- 172.21.15.52:0/364524246 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55563378e8b0 0x55563378ed00 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f146c009c40 tx=0x7f146c009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2fa43684ce0ef4 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:06.097 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.085+0000 7f14597fa700 1 -- 172.21.15.52:0/364524246 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c014400 con 0x55563378e8b0 2023-11-27T15:28:06.097 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.085+0000 7f14597fa700 1 -- 172.21.15.52:0/364524246 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f146c014a20 con 0x55563378e8b0 2023-11-27T15:28:06.097 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.085+0000 7f14597fa700 1 -- 172.21.15.52:0/364524246 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c013970 con 0x55563378e8b0 2023-11-27T15:28:06.097 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.086+0000 7f1485497b80 1 -- 172.21.15.52:0/364524246 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55563378e8b0 msgr2=0x55563378ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:06.097 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.086+0000 7f1485497b80 1 --2- 172.21.15.52:0/364524246 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55563378e8b0 0x55563378ed00 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f146c009c40 tx=0x7f146c009f50 comp rx=0 tx=0).stop 2023-11-27T15:28:06.098 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.086+0000 7f1485497b80 1 -- 172.21.15.52:0/364524246 shutdown_connections 2023-11-27T15:28:06.098 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.086+0000 7f1485497b80 1 --2- 172.21.15.52:0/364524246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555633789b60 0x555633789f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.098 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.086+0000 7f1485497b80 1 --2- 172.21.15.52:0/364524246 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55563378e8b0 0x55563378ed00 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.098 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.086+0000 7f1485497b80 1 --2- 172.21.15.52:0/364524246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55563378a470 0x55563378e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.098 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.086+0000 7f1485497b80 1 -- 172.21.15.52:0/364524246 >> 172.21.15.52:0/364524246 conn(0x55563377f360 msgr2=0x555633784b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:06.098 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.086+0000 7f1485497b80 1 -- 172.21.15.52:0/364524246 shutdown_connections 2023-11-27T15:28:06.098 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.086+0000 7f1485497b80 1 -- 172.21.15.52:0/364524246 wait complete. 2023-11-27T15:28:06.098 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.086+0000 7f1485497b80 1 -- start start 2023-11-27T15:28:06.099 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.086+0000 7f1485497b80 1 Processor -- start 2023-11-27T15:28:06.099 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.087+0000 7f1485497b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555633789b60 0x555633793360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.099 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.087+0000 7f1485497b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55563378a470 0x5556337938a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.099 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.087+0000 7f1485497b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55563378e8b0 0x555633793de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.099 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.087+0000 7f1485497b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55563367c8e0 con 0x55563378a470 2023-11-27T15:28:06.099 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.087+0000 7f1485497b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55563367c580 con 0x555633789b60 2023-11-27T15:28:06.099 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.087+0000 7f1485497b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55563355e920 con 0x55563378e8b0 2023-11-27T15:28:06.099 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.087+0000 7f1485497b80 10 client.0 _ll_register_callbacks cb 0x55563369a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:06.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.087+0000 7f14727fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55563378a470 0x5556337938a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.087+0000 7f14727fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55563378a470 0x5556337938a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56310/0 (socket says 172.21.15.52:56310) 2023-11-27T15:28:06.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.087+0000 7f14727fc700 1 -- 172.21.15.52:0/1649819447 learned_addr learned my addr 172.21.15.52:0/1649819447 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:06.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.087+0000 7f1472ffd700 1 --2- 172.21.15.52:0/1649819447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555633789b60 0x555633793360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.087+0000 7f14737fe700 1 --2- 172.21.15.52:0/1649819447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55563378e8b0 0x555633793de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.087+0000 7f14727fc700 1 -- 172.21.15.52:0/1649819447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55563378e8b0 msgr2=0x555633793de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:06.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.087+0000 7f14727fc700 1 --2- 172.21.15.52:0/1649819447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55563378e8b0 0x555633793de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.087+0000 7f14727fc700 1 -- 172.21.15.52:0/1649819447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555633789b60 msgr2=0x555633793360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:06.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.087+0000 7f14727fc700 1 --2- 172.21.15.52:0/1649819447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555633789b60 0x555633793360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.087+0000 7f14727fc700 1 -- 172.21.15.52:0/1649819447 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146c0098a0 con 0x55563378a470 2023-11-27T15:28:06.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.087+0000 7f1472ffd700 1 --2- 172.21.15.52:0/1649819447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555633789b60 0x555633793360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:28:06.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.087+0000 7f14727fc700 1 --2- 172.21.15.52:0/1649819447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55563378a470 0x5556337938a0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f1464095c30 tx=0x7f14640991a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373c6f08e44a6f8f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:06.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.087+0000 7f1456ff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:28:06.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.087+0000 7f1456ff5700 1 -- 172.21.15.52:0/1649819447 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14640a1b80 con 0x55563378a470 2023-11-27T15:28:06.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.087+0000 7f1456ff5700 1 -- 172.21.15.52:0/1649819447 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f1464099990 con 0x55563378a470 2023-11-27T15:28:06.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.087+0000 7f1485497b80 1 -- 172.21.15.52:0/1649819447 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55563377cb20 con 0x55563378a470 2023-11-27T15:28:06.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.088+0000 7f1456ff5700 1 -- 172.21.15.52:0/1649819447 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1464099b50 con 0x55563378a470 2023-11-27T15:28:06.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.088+0000 7f1485497b80 10 client.17229 fetch_fsmap learned FSMap version 34 2023-11-27T15:28:06.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.088+0000 7f1485497b80 1 -- 172.21.15.52:0/1649819447 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5556336a1190 con 0x55563378a470 2023-11-27T15:28:06.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.088+0000 7f1456ff5700 1 -- 172.21.15.52:0/1649819447 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14640c7070 con 0x55563378a470 2023-11-27T15:28:06.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.088+0000 7f1456ff5700 1 -- 172.21.15.52:0/1649819447 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f14640a1780 con 0x55563378a470 2023-11-27T15:28:06.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.088+0000 7f1485497b80 10 client.17229 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:28:06.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.088+0000 7f1485497b80 10 client.17229 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:28:06.102 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.088+0000 7f1485497b80 20 client.17229 populate_metadata read hostname 'smithi052' 2023-11-27T15:28:06.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.088+0000 7f1485497b80 1 --2- 172.21.15.52:0/1649819447 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5556337a3930 0x5556337a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.088+0000 7f1485497b80 4 client.17229 mds_command: new command op to 24472 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:28:06.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.088+0000 7f1485497b80 1 -- 172.21.15.52:0/1649819447 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55563369d0a0 con 0x5556337a3930 2023-11-27T15:28:06.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.088+0000 7f1472ffd700 1 --2- 172.21.15.52:0/1649819447 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5556337a3930 0x5556337a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.089+0000 7f1456ff5700 10 client.17229 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:28:06.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.089+0000 7f1472ffd700 1 --2- 172.21.15.52:0/1649819447 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5556337a3930 0x5556337a5d40 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9515d47d440e093f server_cookie=49461de10f9017cc in_seq=0 out_seq=0 2023-11-27T15:28:06.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.089+0000 7f1456ff5700 1 -- 172.21.15.52:0/1649819447 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55563369d0a0 con 0x5556337a3930 2023-11-27T15:28:06.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.089+0000 7f1456ff5700 10 client.17229 handle_command_reply: tid=0 2023-11-27T15:28:06.103 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.089+0000 7f1485497b80 1 client.17229 shutdown 2023-11-27T15:28:06.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.089+0000 7f1485497b80 1 -- 172.21.15.52:0/1649819447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55563378a470 msgr2=0x5556337938a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:06.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.089+0000 7f1485497b80 1 --2- 172.21.15.52:0/1649819447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55563378a470 0x5556337938a0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f1464095c30 tx=0x7f14640991a0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.090+0000 7f1485497b80 1 -- 172.21.15.52:0/1649819447 shutdown_connections 2023-11-27T15:28:06.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.090+0000 7f1485497b80 1 --2- 172.21.15.52:0/1649819447 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5556337a3930 0x5556337a5d40 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.090+0000 7f1485497b80 1 --2- 172.21.15.52:0/1649819447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555633789b60 0x555633793360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.090+0000 7f1456ff5700 0 client.17229 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:28:06.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.090+0000 7f1485497b80 1 --2- 172.21.15.52:0/1649819447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55563378e8b0 0x555633793de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.090+0000 7f1485497b80 1 --2- 172.21.15.52:0/1649819447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55563378a470 0x5556337938a0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.090+0000 7f1485497b80 1 -- 172.21.15.52:0/1649819447 >> 172.21.15.52:0/1649819447 conn(0x55563377f360 msgr2=0x55563378fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:06.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.090+0000 7f1485497b80 1 -- 172.21.15.52:0/1649819447 shutdown_connections 2023-11-27T15:28:06.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.090+0000 7f1485497b80 1 -- 172.21.15.52:0/1649819447 wait complete. 2023-11-27T15:28:06.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.090+0000 7f1485497b80 20 client.17229 trim_cache size 0 max 16384 2023-11-27T15:28:06.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.103+0000 7f1485497b80 1 Processor -- start 2023-11-27T15:28:06.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.103+0000 7f1485497b80 1 -- start start 2023-11-27T15:28:06.111 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.103+0000 7f1485497b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555633789b60 0x555633791580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.103+0000 7f1485497b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55563378a470 0x55563379a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.103+0000 7f1485497b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55563378e8b0 0x55563378dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.103+0000 7f1485497b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55563378a9a0 con 0x55563378a470 2023-11-27T15:28:06.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.103+0000 7f1485497b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55563379aee0 con 0x555633789b60 2023-11-27T15:28:06.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.103+0000 7f1485497b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55563378d1c0 con 0x55563378e8b0 2023-11-27T15:28:06.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.103+0000 7f14727fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55563378a470 0x55563379a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.112 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.103+0000 7f14727fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55563378a470 0x55563379a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56318/0 (socket says 172.21.15.52:56318) 2023-11-27T15:28:06.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.103+0000 7f14727fc700 1 -- 172.21.15.52:0/2653413700 learned_addr learned my addr 172.21.15.52:0/2653413700 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:06.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.103+0000 7f1472ffd700 1 --2- 172.21.15.52:0/2653413700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555633789b60 0x555633791580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.103+0000 7f14737fe700 1 --2- 172.21.15.52:0/2653413700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55563378e8b0 0x55563378dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.103+0000 7f14727fc700 1 -- 172.21.15.52:0/2653413700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55563378e8b0 msgr2=0x55563378dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:06.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.103+0000 7f14727fc700 1 --2- 172.21.15.52:0/2653413700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55563378e8b0 0x55563378dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.103+0000 7f14727fc700 1 -- 172.21.15.52:0/2653413700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555633789b60 msgr2=0x555633791580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:06.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.103+0000 7f14727fc700 1 --2- 172.21.15.52:0/2653413700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555633789b60 0x555633791580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.113 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.103+0000 7f14727fc700 1 -- 172.21.15.52:0/2653413700 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556336a0ee0 con 0x55563378a470 2023-11-27T15:28:06.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.103+0000 7f14737fe700 1 --2- 172.21.15.52:0/2653413700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55563378e8b0 0x55563378dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:28:06.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.103+0000 7f1472ffd700 1 --2- 172.21.15.52:0/2653413700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555633789b60 0x555633791580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:28:06.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.104+0000 7f14727fc700 1 --2- 172.21.15.52:0/2653413700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55563378a470 0x55563379a9a0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f1464099430 tx=0x7f1464004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c197157a45b29ea server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:06.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.104+0000 7f1459ffb700 1 -- 172.21.15.52:0/2653413700 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1464004b50 con 0x55563378a470 2023-11-27T15:28:06.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.104+0000 7f1459ffb700 1 -- 172.21.15.52:0/2653413700 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14640c7040 con 0x55563378a470 2023-11-27T15:28:06.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.104+0000 7f1459ffb700 1 -- 172.21.15.52:0/2653413700 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1464098c20 con 0x55563378a470 2023-11-27T15:28:06.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.104+0000 7f1485497b80 1 -- 172.21.15.52:0/2653413700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55563378a470 msgr2=0x55563379a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:06.114 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.104+0000 7f1485497b80 1 --2- 172.21.15.52:0/2653413700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55563378a470 0x55563379a9a0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f1464099430 tx=0x7f1464004dd0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.104+0000 7f1485497b80 1 -- 172.21.15.52:0/2653413700 shutdown_connections 2023-11-27T15:28:06.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.104+0000 7f1485497b80 1 --2- 172.21.15.52:0/2653413700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555633789b60 0x555633791580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.104+0000 7f1485497b80 1 --2- 172.21.15.52:0/2653413700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55563378e8b0 0x55563378dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.104+0000 7f1485497b80 1 --2- 172.21.15.52:0/2653413700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55563378a470 0x55563379a9a0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.104+0000 7f1485497b80 1 -- 172.21.15.52:0/2653413700 >> 172.21.15.52:0/2653413700 conn(0x55563377f360 msgr2=0x55563378d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:06.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.105+0000 7f1485497b80 1 -- 172.21.15.52:0/2653413700 shutdown_connections 2023-11-27T15:28:06.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.105+0000 7f1485497b80 1 -- 172.21.15.52:0/2653413700 wait complete. 2023-11-27T15:28:06.115 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.105+0000 7f1485497b80 1 -- start start 2023-11-27T15:28:06.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.105+0000 7f1485497b80 1 Processor -- start 2023-11-27T15:28:06.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.105+0000 7f1485497b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555633789b60 0x55563378f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.105+0000 7f1485497b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55563378a470 0x55563378fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.105+0000 7f1485497b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55563378e8b0 0x555633792c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.105+0000 7f1485497b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556337900e0 con 0x555633789b60 2023-11-27T15:28:06.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.105+0000 7f1485497b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x555633790250 con 0x55563378a470 2023-11-27T15:28:06.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.105+0000 7f1485497b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555633793140 con 0x55563378e8b0 2023-11-27T15:28:06.116 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.105+0000 7f1485497b80 10 client.0 _ll_register_callbacks cb 0x7f1464099290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:06.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.105+0000 7f1472ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555633789b60 0x55563378f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.105+0000 7f1472ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555633789b60 0x55563378f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56322/0 (socket says 172.21.15.52:56322) 2023-11-27T15:28:06.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.105+0000 7f1472ffd700 1 -- 172.21.15.52:0/1369990149 learned_addr learned my addr 172.21.15.52:0/1369990149 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:06.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.105+0000 7f14737fe700 1 --2- 172.21.15.52:0/1369990149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55563378e8b0 0x555633792c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.105+0000 7f14727fc700 1 --2- 172.21.15.52:0/1369990149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55563378a470 0x55563378fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.105+0000 7f1472ffd700 1 -- 172.21.15.52:0/1369990149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55563378e8b0 msgr2=0x555633792c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:06.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.106+0000 7f1472ffd700 1 --2- 172.21.15.52:0/1369990149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55563378e8b0 0x555633792c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.117 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.106+0000 7f1472ffd700 1 -- 172.21.15.52:0/1369990149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55563378a470 msgr2=0x55563378fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:06.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.106+0000 7f1472ffd700 1 --2- 172.21.15.52:0/1369990149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55563378a470 0x55563378fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.106+0000 7f1472ffd700 1 -- 172.21.15.52:0/1369990149 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5556336a1190 con 0x555633789b60 2023-11-27T15:28:06.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.106+0000 7f14737fe700 1 --2- 172.21.15.52:0/1369990149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55563378e8b0 0x555633792c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:28:06.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.106+0000 7f14727fc700 1 --2- 172.21.15.52:0/1369990149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55563378a470 0x55563378fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:28:06.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.106+0000 7f1472ffd700 1 --2- 172.21.15.52:0/1369990149 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555633789b60 0x55563378f5d0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f1468005f50 tx=0x7f1468009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47aea11a14ec2fa4 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:06.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.106+0000 7f1456ff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:28:06.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.106+0000 7f1456ff5700 1 -- 172.21.15.52:0/1369990149 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146802b040 con 0x555633789b60 2023-11-27T15:28:06.118 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.106+0000 7f1456ff5700 1 -- 172.21.15.52:0/1369990149 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f1468034480 con 0x555633789b60 2023-11-27T15:28:06.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.106+0000 7f1485497b80 1 -- 172.21.15.52:0/1369990149 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f146c014a20 con 0x555633789b60 2023-11-27T15:28:06.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.106+0000 7f1456ff5700 1 -- 172.21.15.52:0/1369990149 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f146800b790 con 0x555633789b60 2023-11-27T15:28:06.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.106+0000 7f1485497b80 10 client.17235 fetch_fsmap learned FSMap version 34 2023-11-27T15:28:06.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.106+0000 7f1485497b80 1 -- 172.21.15.52:0/1369990149 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5556334f1c60 con 0x555633789b60 2023-11-27T15:28:06.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.106+0000 7f1456ff5700 1 -- 172.21.15.52:0/1369990149 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f146c014a20 con 0x555633789b60 2023-11-27T15:28:06.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.106+0000 7f1456ff5700 1 -- 172.21.15.52:0/1369990149 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1468003db0 con 0x555633789b60 2023-11-27T15:28:06.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.106+0000 7f1485497b80 10 client.17235 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:28:06.119 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.106+0000 7f1485497b80 10 client.17235 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-11-27T15:28:06.120 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.106+0000 7f1485497b80 20 client.17235 populate_metadata read hostname 'smithi052' 2023-11-27T15:28:06.120 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.106+0000 7f1485497b80 1 --2- 172.21.15.52:0/1369990149 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5556337f8280 0x5556337fa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.120 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.106+0000 7f1485497b80 4 client.17235 mds_command: new command op to 24472 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:28:06.120 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.106+0000 7f1485497b80 1 -- 172.21.15.52:0/1369990149 --> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5556336a9660 con 0x5556337f8280 2023-11-27T15:28:06.120 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.107+0000 7f14727fc700 1 --2- 172.21.15.52:0/1369990149 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5556337f8280 0x5556337fa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.120 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.107+0000 7f1456ff5700 10 client.17235 ms_handle_connect on v2:172.21.15.150:6836/805360541 2023-11-27T15:28:06.120 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.107+0000 7f14727fc700 1 --2- 172.21.15.52:0/1369990149 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5556337f8280 0x5556337fa690 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=35377157410743ae server_cookie=6d2695034b6995b8 in_seq=0 out_seq=0 2023-11-27T15:28:06.120 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.107+0000 7f1456ff5700 1 -- 172.21.15.52:0/1369990149 <== mds.2 v2:172.21.15.150:6836/805360541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5556336a9660 con 0x5556337f8280 2023-11-27T15:28:06.121 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.107+0000 7f1456ff5700 10 client.17235 handle_command_reply: tid=0 2023-11-27T15:28:06.121 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.107+0000 7f1485497b80 1 client.17235 shutdown 2023-11-27T15:28:06.121 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.107+0000 7f1485497b80 1 -- 172.21.15.52:0/1369990149 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555633789b60 msgr2=0x55563378f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:06.121 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.107+0000 7f1485497b80 1 --2- 172.21.15.52:0/1369990149 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555633789b60 0x55563378f5d0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f1468005f50 tx=0x7f1468009cd0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.121 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.107+0000 7f1485497b80 1 -- 172.21.15.52:0/1369990149 shutdown_connections 2023-11-27T15:28:06.121 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.107+0000 7f1485497b80 1 --2- 172.21.15.52:0/1369990149 >> [v2:172.21.15.150:6836/805360541,v1:172.21.15.150:6837/805360541] conn(0x5556337f8280 0x5556337fa690 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.121 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.107+0000 7f1485497b80 1 --2- 172.21.15.52:0/1369990149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55563378e8b0 0x555633792c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.121 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.107+0000 7f1456ff5700 0 client.17235 ms_handle_reset on v2:172.21.15.150:6836/805360541 2023-11-27T15:28:06.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.107+0000 7f1485497b80 1 --2- 172.21.15.52:0/1369990149 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555633789b60 0x55563378f5d0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.107+0000 7f1485497b80 1 --2- 172.21.15.52:0/1369990149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55563378a470 0x55563378fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.107+0000 7f1485497b80 1 -- 172.21.15.52:0/1369990149 >> 172.21.15.52:0/1369990149 conn(0x55563377f360 msgr2=0x5556337891f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:06.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.107+0000 7f1485497b80 1 -- 172.21.15.52:0/1369990149 shutdown_connections 2023-11-27T15:28:06.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.107+0000 7f1485497b80 1 -- 172.21.15.52:0/1369990149 wait complete. 2023-11-27T15:28:06.122 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.108+0000 7f1485497b80 20 client.17235 trim_cache size 0 max 16384 2023-11-27T15:28:06.122 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:28:06.124 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.108+0000 7f147930b700 1 -- 172.21.15.52:0/1463507330 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f145c0a12c0 msgr2=0x7f145c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:06.124 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.108+0000 7f147930b700 1 --2- 172.21.15.52:0/1463507330 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f145c0a12c0 0x7f145c0a3770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1468009fd0 tx=0x7f1468005fb0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.124 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.108+0000 7f147930b700 1 -- 172.21.15.52:0/1463507330 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14740ffdf0 msgr2=0x7f1474195910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:06.124 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.108+0000 7f147930b700 1 --2- 172.21.15.52:0/1463507330 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14740ffdf0 0x7f1474195910 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f146400d980 tx=0x7f146400dd40 comp rx=0 tx=0).stop 2023-11-27T15:28:06.124 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.108+0000 7f147930b700 1 -- 172.21.15.52:0/1463507330 shutdown_connections 2023-11-27T15:28:06.124 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.108+0000 7f147930b700 1 --2- 172.21.15.52:0/1463507330 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f145c0a12c0 0x7f145c0a3770 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.124 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.108+0000 7f147930b700 1 --2- 172.21.15.52:0/1463507330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1474104490 0x7f1474199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.125 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.108+0000 7f147930b700 1 --2- 172.21.15.52:0/1463507330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14740ff450 0x7f14741953d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.125 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.108+0000 7f147930b700 1 --2- 172.21.15.52:0/1463507330 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14740ffdf0 0x7f1474195910 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.125 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.108+0000 7f147930b700 1 -- 172.21.15.52:0/1463507330 >> 172.21.15.52:0/1463507330 conn(0x7f14740770b0 msgr2=0x7f14740fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:06.125 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.108+0000 7f147930b700 1 -- 172.21.15.52:0/1463507330 shutdown_connections 2023-11-27T15:28:06.125 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.108+0000 7f147930b700 1 -- 172.21.15.52:0/1463507330 wait complete. 2023-11-27T15:28:06.125 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-27T15:28:06.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.249+0000 7fa7bdce4700 1 -- 172.21.15.52:0/3967388136 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7b8104960 msgr2=0x7fa7b8104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:06.253 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.249+0000 7fa7bdce4700 1 --2- 172.21.15.52:0/3967388136 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7b8104960 0x7fa7b8104d30 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fa7a800aaa0 tx=0x7fa7a800adb0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.249+0000 7fa7bdce4700 1 -- 172.21.15.52:0/3967388136 shutdown_connections 2023-11-27T15:28:06.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.249+0000 7fa7bdce4700 1 --2- 172.21.15.52:0/3967388136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7b80ff300 0x7fa7b810b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.249+0000 7fa7bdce4700 1 --2- 172.21.15.52:0/3967388136 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7b8104960 0x7fa7b8104d30 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.249+0000 7fa7bdce4700 1 --2- 172.21.15.52:0/3967388136 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7b80fe950 0x7fa7b80fedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.249+0000 7fa7bdce4700 1 -- 172.21.15.52:0/3967388136 >> 172.21.15.52:0/3967388136 conn(0x7fa7b8075920 msgr2=0x7fa7b8075d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:06.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.249+0000 7fa7bdce4700 1 -- 172.21.15.52:0/3967388136 shutdown_connections 2023-11-27T15:28:06.254 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.250+0000 7fa7bdce4700 1 -- 172.21.15.52:0/3967388136 wait complete. 2023-11-27T15:28:06.256 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.250+0000 7fa7bdce4700 1 Processor -- start 2023-11-27T15:28:06.256 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.251+0000 7fa7bdce4700 1 -- start start 2023-11-27T15:28:06.256 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.251+0000 7fa7bdce4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7b80fe950 0x7fa7b81963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.257 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.251+0000 7fa7bdce4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7b80ff300 0x7fa7b8196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.257 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.251+0000 7fa7bdce4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7b8104960 0x7fa7b819ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.257 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.251+0000 7fa7bdce4700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b819b3c0 con 0x7fa7b80fe950 2023-11-27T15:28:06.257 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.251+0000 7fa7bdce4700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b819b530 con 0x7fa7b80ff300 2023-11-27T15:28:06.257 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.251+0000 7fa7bdce4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b819b6f0 con 0x7fa7b8104960 2023-11-27T15:28:06.257 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.251+0000 7fa7b77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7b80fe950 0x7fa7b81963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.257 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.251+0000 7fa7b77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7b80fe950 0x7fa7b81963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56336/0 (socket says 172.21.15.52:56336) 2023-11-27T15:28:06.257 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.251+0000 7fa7b77fe700 1 -- 172.21.15.52:0/946938234 learned_addr learned my addr 172.21.15.52:0/946938234 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:06.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.251+0000 7fa7b6ffd700 1 --2- 172.21.15.52:0/946938234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7b80ff300 0x7fa7b8196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.251+0000 7fa7b7fff700 1 --2- 172.21.15.52:0/946938234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7b8104960 0x7fa7b819ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.251+0000 7fa7b77fe700 1 -- 172.21.15.52:0/946938234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7b8104960 msgr2=0x7fa7b819ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:06.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.251+0000 7fa7b77fe700 1 --2- 172.21.15.52:0/946938234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7b8104960 0x7fa7b819ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.251+0000 7fa7b77fe700 1 -- 172.21.15.52:0/946938234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7b80ff300 msgr2=0x7fa7b8196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:06.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.251+0000 7fa7b77fe700 1 --2- 172.21.15.52:0/946938234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7b80ff300 0x7fa7b8196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.258 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.251+0000 7fa7b77fe700 1 -- 172.21.15.52:0/946938234 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a800a700 con 0x7fa7b80fe950 2023-11-27T15:28:06.259 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.252+0000 7fa7b77fe700 1 --2- 172.21.15.52:0/946938234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7b80fe950 0x7fa7b81963e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8005f50 tx=0x7fa7a800cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:06.259 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.252+0000 7fa7b4ff9700 1 -- 172.21.15.52:0/946938234 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8004230 con 0x7fa7b80fe950 2023-11-27T15:28:06.259 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.252+0000 7fa7bdce4700 1 -- 172.21.15.52:0/946938234 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b810e160 con 0x7fa7b80fe950 2023-11-27T15:28:06.259 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.252+0000 7fa7b4ff9700 1 -- 172.21.15.52:0/946938234 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7a8004390 con 0x7fa7b80fe950 2023-11-27T15:28:06.259 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.252+0000 7fa7b4ff9700 1 -- 172.21.15.52:0/946938234 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8016680 con 0x7fa7b80fe950 2023-11-27T15:28:06.259 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.252+0000 7fa7bdce4700 1 -- 172.21.15.52:0/946938234 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b810dfd0 con 0x7fa7b80fe950 2023-11-27T15:28:06.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.254+0000 7fa7b4ff9700 1 -- 172.21.15.52:0/946938234 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fa7a8004500 con 0x7fa7b80fe950 2023-11-27T15:28:06.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.255+0000 7fa7b4ff9700 1 --2- 172.21.15.52:0/946938234 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fa7a00a12c0 0x7fa7a00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.255+0000 7fa7b6ffd700 1 --2- 172.21.15.52:0/946938234 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fa7a00a12c0 0x7fa7a00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.265 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.255+0000 7fa7b4ff9700 1 -- 172.21.15.52:0/946938234 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fa7a8012070 con 0x7fa7b80fe950 2023-11-27T15:28:06.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.255+0000 7fa7b6ffd700 1 --2- 172.21.15.52:0/946938234 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fa7a00a12c0 0x7fa7a00a3770 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa7b81979f0 tx=0x7fa7ac00b410 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:06.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.256+0000 7fa7c9e70b80 1 Processor -- start 2023-11-27T15:28:06.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.256+0000 7fa7c9e70b80 1 -- start start 2023-11-27T15:28:06.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.256+0000 7fa7c9e70b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b12054fb60 0x55b12054ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.256+0000 7fa7c9e70b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b120550470 0x55b120554370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.256+0000 7fa7c9e70b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1205548b0 0x55b120554d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.266 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.256+0000 7fa7c9e70b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b120455060 con 0x55b1205548b0 2023-11-27T15:28:06.267 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.256+0000 7fa7c9e70b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1204a78a0 con 0x55b12054fb60 2023-11-27T15:28:06.267 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.256+0000 7fa7c9e70b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1204a0d30 con 0x55b120550470 2023-11-27T15:28:06.267 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.256+0000 7fa7b7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1205548b0 0x55b120554d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.267 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.256+0000 7fa7b7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1205548b0 0x55b120554d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56348/0 (socket says 172.21.15.52:56348) 2023-11-27T15:28:06.267 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.256+0000 7fa7b7fff700 1 -- 172.21.15.52:0/1357430223 learned_addr learned my addr 172.21.15.52:0/1357430223 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:06.267 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.256+0000 7fa7b77fe700 1 --2- 172.21.15.52:0/1357430223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b12054fb60 0x55b12054ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.267 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.256+0000 7fa7b6ffd700 1 --2- 172.21.15.52:0/1357430223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b120550470 0x55b120554370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.267 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.256+0000 7fa7b7fff700 1 -- 172.21.15.52:0/1357430223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b120550470 msgr2=0x55b120554370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:06.268 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.256+0000 7fa7b7fff700 1 --2- 172.21.15.52:0/1357430223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b120550470 0x55b120554370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.268 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.256+0000 7fa7b7fff700 1 -- 172.21.15.52:0/1357430223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b12054fb60 msgr2=0x55b12054ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:06.268 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.256+0000 7fa7b7fff700 1 --2- 172.21.15.52:0/1357430223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b12054fb60 0x55b12054ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.268 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.256+0000 7fa7b7fff700 1 -- 172.21.15.52:0/1357430223 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b120322630 con 0x55b1205548b0 2023-11-27T15:28:06.268 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.257+0000 7fa7b7fff700 1 --2- 172.21.15.52:0/1357430223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1205548b0 0x55b120554d00 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009d00 tx=0x7fa7b00077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bbda53a90611db4 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:06.268 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.257+0000 7fa79dffb700 1 -- 172.21.15.52:0/1357430223 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0019070 con 0x55b1205548b0 2023-11-27T15:28:06.268 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.257+0000 7fa79dffb700 1 -- 172.21.15.52:0/1357430223 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7b000f050 con 0x55b1205548b0 2023-11-27T15:28:06.268 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.257+0000 7fa79dffb700 1 -- 172.21.15.52:0/1357430223 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0004870 con 0x55b1205548b0 2023-11-27T15:28:06.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.257+0000 7fa7c9e70b80 1 -- 172.21.15.52:0/1357430223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1205548b0 msgr2=0x55b120554d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:06.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.257+0000 7fa7c9e70b80 1 --2- 172.21.15.52:0/1357430223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1205548b0 0x55b120554d00 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009d00 tx=0x7fa7b00077e0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.257+0000 7fa7c9e70b80 1 -- 172.21.15.52:0/1357430223 shutdown_connections 2023-11-27T15:28:06.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.257+0000 7fa7c9e70b80 1 --2- 172.21.15.52:0/1357430223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b12054fb60 0x55b12054ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.257+0000 7fa7c9e70b80 1 --2- 172.21.15.52:0/1357430223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1205548b0 0x55b120554d00 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.257+0000 7fa7c9e70b80 1 --2- 172.21.15.52:0/1357430223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b120550470 0x55b120554370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.257+0000 7fa7c9e70b80 1 -- 172.21.15.52:0/1357430223 >> 172.21.15.52:0/1357430223 conn(0x55b120545360 msgr2=0x55b12054ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:06.269 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.257+0000 7fa7c9e70b80 1 -- 172.21.15.52:0/1357430223 shutdown_connections 2023-11-27T15:28:06.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.257+0000 7fa7c9e70b80 1 -- 172.21.15.52:0/1357430223 wait complete. 2023-11-27T15:28:06.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.258+0000 7fa7c9e70b80 1 -- start start 2023-11-27T15:28:06.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.258+0000 7fa7c9e70b80 1 Processor -- start 2023-11-27T15:28:06.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.258+0000 7fa7c9e70b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b12054fb60 0x55b120559360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.258+0000 7fa7c9e70b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b120550470 0x55b1205598a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.258+0000 7fa7c9e70b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1205548b0 0x55b120559de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.258+0000 7fa7c9e70b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1204428e0 con 0x55b120550470 2023-11-27T15:28:06.270 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.258+0000 7fa7c9e70b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b120442580 con 0x55b1205548b0 2023-11-27T15:28:06.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.258+0000 7fa7c9e70b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b120324920 con 0x55b12054fb60 2023-11-27T15:28:06.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.258+0000 7fa7c9e70b80 10 client.0 _ll_register_callbacks cb 0x55b120460780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:06.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.258+0000 7fa7b6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b120550470 0x55b1205598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.258+0000 7fa7b6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b120550470 0x55b1205598a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56352/0 (socket says 172.21.15.52:56352) 2023-11-27T15:28:06.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.258+0000 7fa7b6ffd700 1 -- 172.21.15.52:0/1016959939 learned_addr learned my addr 172.21.15.52:0/1016959939 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:06.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.259+0000 7fa7b7fff700 1 --2- 172.21.15.52:0/1016959939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1205548b0 0x55b120559de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.259+0000 7fa7b77fe700 1 --2- 172.21.15.52:0/1016959939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b12054fb60 0x55b120559360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.271 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.259+0000 7fa7b6ffd700 1 -- 172.21.15.52:0/1016959939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b12054fb60 msgr2=0x55b120559360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:06.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.259+0000 7fa7b6ffd700 1 --2- 172.21.15.52:0/1016959939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b12054fb60 0x55b120559360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.259+0000 7fa7b6ffd700 1 -- 172.21.15.52:0/1016959939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1205548b0 msgr2=0x55b120559de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:06.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.259+0000 7fa7b6ffd700 1 --2- 172.21.15.52:0/1016959939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1205548b0 0x55b120559de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.259+0000 7fa7b6ffd700 1 -- 172.21.15.52:0/1016959939 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b0009960 con 0x55b120550470 2023-11-27T15:28:06.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.259+0000 7fa7b6ffd700 1 --2- 172.21.15.52:0/1016959939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b120550470 0x55b1205598a0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac0116f0 tx=0x7fa7ac0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fedba67f91b5788f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:06.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.259+0000 7fa796ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:28:06.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.259+0000 7fa796ffd700 1 -- 172.21.15.52:0/1016959939 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac028670 con 0x55b120550470 2023-11-27T15:28:06.272 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.259+0000 7fa796ffd700 1 -- 172.21.15.52:0/1016959939 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fa7ac017070 con 0x55b120550470 2023-11-27T15:28:06.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.259+0000 7fa7c9e70b80 1 -- 172.21.15.52:0/1016959939 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b120542b20 con 0x55b120550470 2023-11-27T15:28:06.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.259+0000 7fa796ffd700 1 -- 172.21.15.52:0/1016959939 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7ac011e70 con 0x55b120550470 2023-11-27T15:28:06.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.259+0000 7fa7c9e70b80 10 client.17247 fetch_fsmap learned FSMap version 34 2023-11-27T15:28:06.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.259+0000 7fa7c9e70b80 1 -- 172.21.15.52:0/1016959939 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55b120467190 con 0x55b120550470 2023-11-27T15:28:06.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.259+0000 7fa796ffd700 1 -- 172.21.15.52:0/1016959939 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b120542b20 con 0x55b120550470 2023-11-27T15:28:06.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.260+0000 7fa796ffd700 1 -- 172.21.15.52:0/1016959939 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa7ac01f8e0 con 0x55b120550470 2023-11-27T15:28:06.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.260+0000 7fa7c9e70b80 10 client.17247 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:28:06.273 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.260+0000 7fa7c9e70b80 10 client.17247 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:28:06.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.260+0000 7fa7c9e70b80 20 client.17247 populate_metadata read hostname 'smithi052' 2023-11-27T15:28:06.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.260+0000 7fa7c9e70b80 1 --2- 172.21.15.52:0/1016959939 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55b120569a50 0x55b12056be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.260+0000 7fa7c9e70b80 4 client.17247 mds_command: new command op to 24494 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:28:06.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.260+0000 7fa7c9e70b80 1 -- 172.21.15.52:0/1016959939 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b1204630a0 con 0x55b120569a50 2023-11-27T15:28:06.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.260+0000 7fa7b77fe700 1 --2- 172.21.15.52:0/1016959939 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55b120569a50 0x55b12056be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.260+0000 7fa796ffd700 10 client.17247 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:28:06.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.260+0000 7fa7b77fe700 1 --2- 172.21.15.52:0/1016959939 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55b120569a50 0x55b12056be60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4e0a2842418abfb9 server_cookie=f3d4a14e4a059870 in_seq=0 out_seq=0 2023-11-27T15:28:06.274 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.261+0000 7fa796ffd700 1 -- 172.21.15.52:0/1016959939 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b1204630a0 con 0x55b120569a50 2023-11-27T15:28:06.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.261+0000 7fa796ffd700 10 client.17247 handle_command_reply: tid=0 2023-11-27T15:28:06.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.261+0000 7fa7c9e70b80 1 client.17247 shutdown 2023-11-27T15:28:06.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.261+0000 7fa7c9e70b80 1 -- 172.21.15.52:0/1016959939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b120550470 msgr2=0x55b1205598a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:06.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.261+0000 7fa7c9e70b80 1 --2- 172.21.15.52:0/1016959939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b120550470 0x55b1205598a0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac0116f0 tx=0x7fa7ac0047c0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.261+0000 7fa7c9e70b80 1 -- 172.21.15.52:0/1016959939 shutdown_connections 2023-11-27T15:28:06.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.261+0000 7fa7c9e70b80 1 --2- 172.21.15.52:0/1016959939 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55b120569a50 0x55b12056be60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.261+0000 7fa7c9e70b80 1 --2- 172.21.15.52:0/1016959939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1205548b0 0x55b120559de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.275 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.261+0000 7fa7c9e70b80 1 --2- 172.21.15.52:0/1016959939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b12054fb60 0x55b120559360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.261+0000 7fa796ffd700 0 client.17247 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:28:06.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.261+0000 7fa7c9e70b80 1 --2- 172.21.15.52:0/1016959939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b120550470 0x55b1205598a0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.261+0000 7fa7c9e70b80 1 -- 172.21.15.52:0/1016959939 >> 172.21.15.52:0/1016959939 conn(0x55b120545360 msgr2=0x55b120555d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:06.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.262+0000 7fa7c9e70b80 1 -- 172.21.15.52:0/1016959939 shutdown_connections 2023-11-27T15:28:06.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.262+0000 7fa7c9e70b80 1 -- 172.21.15.52:0/1016959939 wait complete. 2023-11-27T15:28:06.276 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.262+0000 7fa7c9e70b80 20 client.17247 trim_cache size 0 max 16384 2023-11-27T15:28:06.279 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.275+0000 7fa7c9e70b80 1 Processor -- start 2023-11-27T15:28:06.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.275+0000 7fa7c9e70b80 1 -- start start 2023-11-27T15:28:06.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.275+0000 7fa7c9e70b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b12054fb60 0x55b120557580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.275+0000 7fa7c9e70b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b120550470 0x55b1205609a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.275+0000 7fa7c9e70b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1205548b0 0x55b1205538a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.275+0000 7fa7c9e70b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b120550bc0 con 0x55b120550470 2023-11-27T15:28:06.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.275+0000 7fa7c9e70b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b120553490 con 0x55b1205548b0 2023-11-27T15:28:06.280 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.275+0000 7fa7c9e70b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b120560ee0 con 0x55b12054fb60 2023-11-27T15:28:06.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.275+0000 7fa7b6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b120550470 0x55b1205609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.275+0000 7fa7b6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b120550470 0x55b1205609a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56360/0 (socket says 172.21.15.52:56360) 2023-11-27T15:28:06.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.275+0000 7fa7b6ffd700 1 -- 172.21.15.52:0/3890130380 learned_addr learned my addr 172.21.15.52:0/3890130380 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:06.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.276+0000 7fa7b77fe700 1 --2- 172.21.15.52:0/3890130380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b12054fb60 0x55b120557580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.276+0000 7fa7b7fff700 1 --2- 172.21.15.52:0/3890130380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1205548b0 0x55b1205538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.276+0000 7fa7b6ffd700 1 -- 172.21.15.52:0/3890130380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b12054fb60 msgr2=0x55b120557580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:06.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.276+0000 7fa7b6ffd700 1 --2- 172.21.15.52:0/3890130380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b12054fb60 0x55b120557580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.281 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.276+0000 7fa7b6ffd700 1 -- 172.21.15.52:0/3890130380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1205548b0 msgr2=0x55b1205538a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:06.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.276+0000 7fa7b6ffd700 1 --2- 172.21.15.52:0/3890130380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1205548b0 0x55b1205538a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.276+0000 7fa7b6ffd700 1 -- 172.21.15.52:0/3890130380 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b120466ee0 con 0x55b120550470 2023-11-27T15:28:06.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.276+0000 7fa7b6ffd700 1 --2- 172.21.15.52:0/3890130380 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b120550470 0x55b1205609a0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac01fce0 tx=0x7fa7ac004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6056799e993cee61 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:06.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.276+0000 7fa79e7fc700 1 -- 172.21.15.52:0/3890130380 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac00bdb0 con 0x55b120550470 2023-11-27T15:28:06.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.276+0000 7fa79e7fc700 1 -- 172.21.15.52:0/3890130380 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7ac015070 con 0x55b120550470 2023-11-27T15:28:06.282 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.276+0000 7fa79e7fc700 1 -- 172.21.15.52:0/3890130380 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac004e80 con 0x55b120550470 2023-11-27T15:28:06.286 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.277+0000 7fa7c9e70b80 1 -- 172.21.15.52:0/3890130380 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b120550470 msgr2=0x55b1205609a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:06.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.277+0000 7fa7c9e70b80 1 --2- 172.21.15.52:0/3890130380 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b120550470 0x55b1205609a0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac01fce0 tx=0x7fa7ac004710 comp rx=0 tx=0).stop 2023-11-27T15:28:06.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.277+0000 7fa7c9e70b80 1 -- 172.21.15.52:0/3890130380 shutdown_connections 2023-11-27T15:28:06.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.277+0000 7fa7c9e70b80 1 --2- 172.21.15.52:0/3890130380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b1205548b0 0x55b1205538a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.277+0000 7fa7c9e70b80 1 --2- 172.21.15.52:0/3890130380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b12054fb60 0x55b120557580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.277+0000 7fa7c9e70b80 1 --2- 172.21.15.52:0/3890130380 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b120550470 0x55b1205609a0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.277+0000 7fa7c9e70b80 1 -- 172.21.15.52:0/3890130380 >> 172.21.15.52:0/3890130380 conn(0x55b120545360 msgr2=0x55b120552930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:06.287 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.277+0000 7fa7c9e70b80 1 -- 172.21.15.52:0/3890130380 shutdown_connections 2023-11-27T15:28:06.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.277+0000 7fa7c9e70b80 1 -- 172.21.15.52:0/3890130380 wait complete. 2023-11-27T15:28:06.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.277+0000 7fa7c9e70b80 1 -- start start 2023-11-27T15:28:06.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.278+0000 7fa7c9e70b80 1 Processor -- start 2023-11-27T15:28:06.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.278+0000 7fa7c9e70b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b12054fb60 0x55b120555750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.278+0000 7fa7c9e70b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b120550470 0x55b120555c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.278+0000 7fa7c9e70b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1205548b0 0x55b120549550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.278+0000 7fa7c9e70b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1205561d0 con 0x55b120550470 2023-11-27T15:28:06.288 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.278+0000 7fa7c9e70b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b120556340 con 0x55b12054fb60 2023-11-27T15:28:06.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.278+0000 7fa7c9e70b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b120549a90 con 0x55b1205548b0 2023-11-27T15:28:06.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.278+0000 7fa7c9e70b80 10 client.0 _ll_register_callbacks cb 0x7fa7ac010d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:06.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.278+0000 7fa7b6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b120550470 0x55b120555c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.278+0000 7fa7b6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b120550470 0x55b120555c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56366/0 (socket says 172.21.15.52:56366) 2023-11-27T15:28:06.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.278+0000 7fa7b6ffd700 1 -- 172.21.15.52:0/1250077148 learned_addr learned my addr 172.21.15.52:0/1250077148 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:06.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.278+0000 7fa7b77fe700 1 --2- 172.21.15.52:0/1250077148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b12054fb60 0x55b120555750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.278+0000 7fa7b7fff700 1 --2- 172.21.15.52:0/1250077148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1205548b0 0x55b120549550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.289 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.278+0000 7fa7b6ffd700 1 -- 172.21.15.52:0/1250077148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1205548b0 msgr2=0x55b120549550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:06.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.278+0000 7fa7b6ffd700 1 --2- 172.21.15.52:0/1250077148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1205548b0 0x55b120549550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.278+0000 7fa7b6ffd700 1 -- 172.21.15.52:0/1250077148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b12054fb60 msgr2=0x55b120555750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:06.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.278+0000 7fa7b6ffd700 1 --2- 172.21.15.52:0/1250077148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b12054fb60 0x55b120555750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.278+0000 7fa7b6ffd700 1 -- 172.21.15.52:0/1250077148 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7ac011e70 con 0x55b120550470 2023-11-27T15:28:06.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.278+0000 7fa7b7fff700 1 --2- 172.21.15.52:0/1250077148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1205548b0 0x55b120549550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:28:06.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.278+0000 7fa7b77fe700 1 --2- 172.21.15.52:0/1250077148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b12054fb60 0x55b120555750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:28:06.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.279+0000 7fa7b6ffd700 1 --2- 172.21.15.52:0/1250077148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b120550470 0x55b120555c90 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac01fc60 tx=0x7fa7ac004de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47ed2ab3279c079 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:06.290 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.279+0000 7fa796ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:28:06.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.279+0000 7fa796ffd700 1 -- 172.21.15.52:0/1250077148 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac011ab0 con 0x55b120550470 2023-11-27T15:28:06.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.279+0000 7fa796ffd700 1 -- 172.21.15.52:0/1250077148 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fa7ac017070 con 0x55b120550470 2023-11-27T15:28:06.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.279+0000 7fa7c9e70b80 1 -- 172.21.15.52:0/1250077148 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa7b000f050 con 0x55b120550470 2023-11-27T15:28:06.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.279+0000 7fa796ffd700 1 -- 172.21.15.52:0/1250077148 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7ac01f640 con 0x55b120550470 2023-11-27T15:28:06.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.279+0000 7fa7c9e70b80 10 client.17253 fetch_fsmap learned FSMap version 34 2023-11-27T15:28:06.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.279+0000 7fa7c9e70b80 1 -- 172.21.15.52:0/1250077148 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55b1202b7c60 con 0x55b120550470 2023-11-27T15:28:06.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.279+0000 7fa796ffd700 1 -- 172.21.15.52:0/1250077148 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7ac04d020 con 0x55b120550470 2023-11-27T15:28:06.291 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.279+0000 7fa796ffd700 1 -- 172.21.15.52:0/1250077148 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa7ac036030 con 0x55b120550470 2023-11-27T15:28:06.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.279+0000 7fa7c9e70b80 10 client.17253 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:28:06.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.279+0000 7fa7c9e70b80 10 client.17253 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:28:06.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.279+0000 7fa7c9e70b80 20 client.17253 populate_metadata read hostname 'smithi052' 2023-11-27T15:28:06.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.279+0000 7fa7c9e70b80 1 --2- 172.21.15.52:0/1250077148 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55b1205be150 0x55b1205c0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.279+0000 7fa7c9e70b80 4 client.17253 mds_command: new command op to 24494 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:28:06.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.279+0000 7fa7c9e70b80 1 -- 172.21.15.52:0/1250077148 --> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b12046f660 con 0x55b1205be150 2023-11-27T15:28:06.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.280+0000 7fa7b77fe700 1 --2- 172.21.15.52:0/1250077148 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55b1205be150 0x55b1205c0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.292 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.280+0000 7fa796ffd700 10 client.17253 ms_handle_connect on v2:172.21.15.102:6836/2141793919 2023-11-27T15:28:06.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.280+0000 7fa7b77fe700 1 --2- 172.21.15.52:0/1250077148 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55b1205be150 0x55b1205c0530 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1288c3b0d7ac78d4 server_cookie=448a354e893b9f8 in_seq=0 out_seq=0 2023-11-27T15:28:06.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.280+0000 7fa796ffd700 1 -- 172.21.15.52:0/1250077148 <== mds.1 v2:172.21.15.102:6836/2141793919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b12046f660 con 0x55b1205be150 2023-11-27T15:28:06.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.280+0000 7fa796ffd700 10 client.17253 handle_command_reply: tid=0 2023-11-27T15:28:06.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.280+0000 7fa7c9e70b80 1 client.17253 shutdown 2023-11-27T15:28:06.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.280+0000 7fa7c9e70b80 1 -- 172.21.15.52:0/1250077148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b120550470 msgr2=0x55b120555c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:06.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.280+0000 7fa7c9e70b80 1 --2- 172.21.15.52:0/1250077148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b120550470 0x55b120555c90 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac01fc60 tx=0x7fa7ac004de0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.280+0000 7fa7c9e70b80 1 -- 172.21.15.52:0/1250077148 shutdown_connections 2023-11-27T15:28:06.293 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.280+0000 7fa7c9e70b80 1 --2- 172.21.15.52:0/1250077148 >> [v2:172.21.15.102:6836/2141793919,v1:172.21.15.102:6837/2141793919] conn(0x55b1205be150 0x55b1205c0530 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.280+0000 7fa7c9e70b80 1 --2- 172.21.15.52:0/1250077148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b12054fb60 0x55b120555750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.280+0000 7fa7c9e70b80 1 --2- 172.21.15.52:0/1250077148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b1205548b0 0x55b120549550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.280+0000 7fa7c9e70b80 1 --2- 172.21.15.52:0/1250077148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b120550470 0x55b120555c90 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.280+0000 7fa796ffd700 0 client.17253 ms_handle_reset on v2:172.21.15.102:6836/2141793919 2023-11-27T15:28:06.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.280+0000 7fa7c9e70b80 1 -- 172.21.15.52:0/1250077148 >> 172.21.15.52:0/1250077148 conn(0x55b120545360 msgr2=0x55b120559300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:06.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.280+0000 7fa7c9e70b80 1 -- 172.21.15.52:0/1250077148 shutdown_connections 2023-11-27T15:28:06.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.281+0000 7fa7c9e70b80 1 -- 172.21.15.52:0/1250077148 wait complete. 2023-11-27T15:28:06.294 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.281+0000 7fa7c9e70b80 20 client.17253 trim_cache size 0 max 16384 2023-11-27T15:28:06.295 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:28:06.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.281+0000 7fa7bdce4700 1 -- 172.21.15.52:0/946938234 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fa7a00a12c0 msgr2=0x7fa7a00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:06.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.281+0000 7fa7bdce4700 1 --2- 172.21.15.52:0/946938234 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fa7a00a12c0 0x7fa7a00a3770 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa7b81979f0 tx=0x7fa7ac00b410 comp rx=0 tx=0).stop 2023-11-27T15:28:06.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.281+0000 7fa7bdce4700 1 -- 172.21.15.52:0/946938234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7b80fe950 msgr2=0x7fa7b81963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:06.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.281+0000 7fa7bdce4700 1 --2- 172.21.15.52:0/946938234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7b80fe950 0x7fa7b81963e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8005f50 tx=0x7fa7a800cda0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.281+0000 7fa7bdce4700 1 -- 172.21.15.52:0/946938234 shutdown_connections 2023-11-27T15:28:06.296 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.281+0000 7fa7bdce4700 1 --2- 172.21.15.52:0/946938234 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fa7a00a12c0 0x7fa7a00a3770 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.281+0000 7fa7bdce4700 1 --2- 172.21.15.52:0/946938234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7b8104960 0x7fa7b819ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.281+0000 7fa7bdce4700 1 --2- 172.21.15.52:0/946938234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7b80fe950 0x7fa7b81963e0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.281+0000 7fa7bdce4700 1 --2- 172.21.15.52:0/946938234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7b80ff300 0x7fa7b8196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.281+0000 7fa7bdce4700 1 -- 172.21.15.52:0/946938234 >> 172.21.15.52:0/946938234 conn(0x7fa7b8075920 msgr2=0x7fa7b80fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:06.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.281+0000 7fa7bdce4700 1 -- 172.21.15.52:0/946938234 shutdown_connections 2023-11-27T15:28:06.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.281+0000 7fa7bdce4700 1 -- 172.21.15.52:0/946938234 wait complete. 2023-11-27T15:28:06.298 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-27T15:28:06.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.422+0000 7f1f184b9700 1 -- 172.21.15.52:0/2763772230 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f100ffdf0 msgr2=0x7f1f10103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:06.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.422+0000 7f1f184b9700 1 --2- 172.21.15.52:0/2763772230 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f100ffdf0 0x7f1f10103db0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f1f000099c0 tx=0x7f1f00009cd0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.423+0000 7f1f184b9700 1 -- 172.21.15.52:0/2763772230 shutdown_connections 2023-11-27T15:28:06.426 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.423+0000 7f1f184b9700 1 --2- 172.21.15.52:0/2763772230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f10104490 0x7f1f1010c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.423+0000 7f1f184b9700 1 --2- 172.21.15.52:0/2763772230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f100ff450 0x7f1f100ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.423+0000 7f1f184b9700 1 --2- 172.21.15.52:0/2763772230 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f100ffdf0 0x7f1f10103db0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.423+0000 7f1f184b9700 1 -- 172.21.15.52:0/2763772230 >> 172.21.15.52:0/2763772230 conn(0x7f1f10077080 msgr2=0x7f1f10077480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:06.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.423+0000 7f1f184b9700 1 -- 172.21.15.52:0/2763772230 shutdown_connections 2023-11-27T15:28:06.427 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.423+0000 7f1f184b9700 1 -- 172.21.15.52:0/2763772230 wait complete. 2023-11-27T15:28:06.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.423+0000 7f1f184b9700 1 Processor -- start 2023-11-27T15:28:06.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.423+0000 7f1f184b9700 1 -- start start 2023-11-27T15:28:06.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.423+0000 7f1f184b9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f100ff450 0x7f1f10197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.423+0000 7f1f184b9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f100ffdf0 0x7f1f10197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.429 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.423+0000 7f1f184b9700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f10104490 0x7f1f101a2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.423+0000 7f1f184b9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f10198050 con 0x7f1f100ffdf0 2023-11-27T15:28:06.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.423+0000 7f1f184b9700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f101a27c0 con 0x7f1f10104490 2023-11-27T15:28:06.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.423+0000 7f1f184b9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f101a2930 con 0x7f1f100ff450 2023-11-27T15:28:06.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.424+0000 7f1f15a54700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f100ffdf0 0x7f1f10197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.424+0000 7f1f15a54700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f100ffdf0 0x7f1f10197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56380/0 (socket says 172.21.15.52:56380) 2023-11-27T15:28:06.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.424+0000 7f1f15a54700 1 -- 172.21.15.52:0/3745986658 learned_addr learned my addr 172.21.15.52:0/3745986658 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:06.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.424+0000 7f1f16a56700 1 --2- 172.21.15.52:0/3745986658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f10104490 0x7f1f101a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.430 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.424+0000 7f1f16255700 1 --2- 172.21.15.52:0/3745986658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f100ff450 0x7f1f10197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.424+0000 7f1f15a54700 1 -- 172.21.15.52:0/3745986658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f100ff450 msgr2=0x7f1f10197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:06.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.424+0000 7f1f15a54700 1 --2- 172.21.15.52:0/3745986658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f100ff450 0x7f1f10197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.424+0000 7f1f15a54700 1 -- 172.21.15.52:0/3745986658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f10104490 msgr2=0x7f1f101a2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:06.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.424+0000 7f1f15a54700 1 --2- 172.21.15.52:0/3745986658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f10104490 0x7f1f101a2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.424+0000 7f1f15a54700 1 -- 172.21.15.52:0/3745986658 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f000095f0 con 0x7f1f100ffdf0 2023-11-27T15:28:06.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.424+0000 7f1f15a54700 1 --2- 172.21.15.52:0/3745986658 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f100ffdf0 0x7f1f10197740 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f1f00009fd0 tx=0x7f1f00009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:06.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.424+0000 7f1f077fe700 1 -- 172.21.15.52:0/3745986658 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0001d070 con 0x7f1f100ffdf0 2023-11-27T15:28:06.431 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.424+0000 7f1f184b9700 1 -- 172.21.15.52:0/3745986658 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f1010f210 con 0x7f1f100ffdf0 2023-11-27T15:28:06.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.424+0000 7f1f077fe700 1 -- 172.21.15.52:0/3745986658 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f000054e0 con 0x7f1f100ffdf0 2023-11-27T15:28:06.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.424+0000 7f1f077fe700 1 -- 172.21.15.52:0/3745986658 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f00004b10 con 0x7f1f100ffdf0 2023-11-27T15:28:06.432 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.424+0000 7f1f184b9700 1 -- 172.21.15.52:0/3745986658 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f1010f080 con 0x7f1f100ffdf0 2023-11-27T15:28:06.438 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.426+0000 7f1f077fe700 1 -- 172.21.15.52:0/3745986658 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f1f0000fbc0 con 0x7f1f100ffdf0 2023-11-27T15:28:06.438 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.427+0000 7f1f077fe700 1 --2- 172.21.15.52:0/3745986658 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f1efc0a12c0 0x7f1efc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.438 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.427+0000 7f1f16255700 1 --2- 172.21.15.52:0/3745986658 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f1efc0a12c0 0x7f1efc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.438 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.427+0000 7f1f077fe700 1 -- 172.21.15.52:0/3745986658 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f1f000cd9d0 con 0x7f1f100ffdf0 2023-11-27T15:28:06.438 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.427+0000 7f1f16255700 1 --2- 172.21.15.52:0/3745986658 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f1efc0a12c0 0x7f1efc0a3770 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c005950 tx=0x7f1f0c0058e0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:06.439 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.428+0000 7f1f24645b80 1 Processor -- start 2023-11-27T15:28:06.439 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.428+0000 7f1f24645b80 1 -- start start 2023-11-27T15:28:06.439 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.429+0000 7f1f24645b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c32a9ccf70 0x55c32a9cd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.439 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.429+0000 7f1f24645b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c32a9cd880 0x55c32a9d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.439 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.429+0000 7f1f24645b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c32a9d1cc0 0x55c32a9d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.439 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.429+0000 7f1f24645b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c32a8ce060 con 0x55c32a9cd880 2023-11-27T15:28:06.439 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.429+0000 7f1f24645b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c32a9208a0 con 0x55c32a9d1cc0 2023-11-27T15:28:06.439 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.429+0000 7f1f24645b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c32a919d30 con 0x55c32a9ccf70 2023-11-27T15:28:06.440 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.429+0000 7f1f15a54700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c32a9cd880 0x55c32a9d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.440 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.429+0000 7f1f15a54700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c32a9cd880 0x55c32a9d1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56388/0 (socket says 172.21.15.52:56388) 2023-11-27T15:28:06.440 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.429+0000 7f1f15a54700 1 -- 172.21.15.52:0/2224950550 learned_addr learned my addr 172.21.15.52:0/2224950550 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:06.440 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.429+0000 7f1f16a56700 1 --2- 172.21.15.52:0/2224950550 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c32a9d1cc0 0x55c32a9d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.440 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.429+0000 7f1f16255700 1 --2- 172.21.15.52:0/2224950550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c32a9ccf70 0x55c32a9cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.440 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.429+0000 7f1f15a54700 1 -- 172.21.15.52:0/2224950550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c32a9ccf70 msgr2=0x55c32a9cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:06.440 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.429+0000 7f1f15a54700 1 --2- 172.21.15.52:0/2224950550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c32a9ccf70 0x55c32a9cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.440 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.429+0000 7f1f15a54700 1 -- 172.21.15.52:0/2224950550 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c32a9d1cc0 msgr2=0x55c32a9d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:06.440 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.429+0000 7f1f15a54700 1 --2- 172.21.15.52:0/2224950550 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c32a9d1cc0 0x55c32a9d2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.429+0000 7f1f15a54700 1 -- 172.21.15.52:0/2224950550 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c32a79b630 con 0x55c32a9cd880 2023-11-27T15:28:06.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.429+0000 7f1f15a54700 1 --2- 172.21.15.52:0/2224950550 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c32a9cd880 0x55c32a9d1780 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f1f00097510 tx=0x7f1f0009a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b8c6e1a930e661 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:06.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.429+0000 7f1f04ff9700 1 -- 172.21.15.52:0/2224950550 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0009ead0 con 0x55c32a9cd880 2023-11-27T15:28:06.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.429+0000 7f1f04ff9700 1 -- 172.21.15.52:0/2224950550 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f000a2e60 con 0x55c32a9cd880 2023-11-27T15:28:06.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.429+0000 7f1f04ff9700 1 -- 172.21.15.52:0/2224950550 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f000a29d0 con 0x55c32a9cd880 2023-11-27T15:28:06.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.430+0000 7f1f24645b80 1 -- 172.21.15.52:0/2224950550 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c32a9cd880 msgr2=0x55c32a9d1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:06.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.430+0000 7f1f24645b80 1 --2- 172.21.15.52:0/2224950550 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c32a9cd880 0x55c32a9d1780 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f1f00097510 tx=0x7f1f0009a5e0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.441 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.430+0000 7f1f24645b80 1 -- 172.21.15.52:0/2224950550 shutdown_connections 2023-11-27T15:28:06.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.430+0000 7f1f24645b80 1 --2- 172.21.15.52:0/2224950550 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c32a9d1cc0 0x55c32a9d2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.430+0000 7f1f24645b80 1 --2- 172.21.15.52:0/2224950550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c32a9ccf70 0x55c32a9cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.430+0000 7f1f24645b80 1 --2- 172.21.15.52:0/2224950550 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c32a9cd880 0x55c32a9d1780 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.430+0000 7f1f24645b80 1 -- 172.21.15.52:0/2224950550 >> 172.21.15.52:0/2224950550 conn(0x55c32a9be360 msgr2=0x55c32a9c3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:06.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.430+0000 7f1f24645b80 1 -- 172.21.15.52:0/2224950550 shutdown_connections 2023-11-27T15:28:06.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.430+0000 7f1f24645b80 1 -- 172.21.15.52:0/2224950550 wait complete. 2023-11-27T15:28:06.442 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.431+0000 7f1f24645b80 1 -- start start 2023-11-27T15:28:06.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.431+0000 7f1f24645b80 1 Processor -- start 2023-11-27T15:28:06.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.431+0000 7f1f24645b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c32a9ccf70 0x55c32a9d9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.431+0000 7f1f24645b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c32a9cd880 0x55c32a9d65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.431+0000 7f1f24645b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c32a9d1cc0 0x55c32a9d6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.431+0000 7f1f24645b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c32a8bb8e0 con 0x55c32a9d1cc0 2023-11-27T15:28:06.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.431+0000 7f1f24645b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c32a8bb580 con 0x55c32a9ccf70 2023-11-27T15:28:06.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.431+0000 7f1f24645b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c32a79d920 con 0x55c32a9cd880 2023-11-27T15:28:06.443 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.431+0000 7f1f24645b80 10 client.0 _ll_register_callbacks cb 0x55c32a8d9780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:06.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.431+0000 7f1f16a56700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c32a9d1cc0 0x55c32a9d6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.431+0000 7f1f16a56700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c32a9d1cc0 0x55c32a9d6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56396/0 (socket says 172.21.15.52:56396) 2023-11-27T15:28:06.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.431+0000 7f1f16a56700 1 -- 172.21.15.52:0/4104977557 learned_addr learned my addr 172.21.15.52:0/4104977557 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:06.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.431+0000 7f1f16255700 1 --2- 172.21.15.52:0/4104977557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c32a9ccf70 0x55c32a9d9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.431+0000 7f1f15a54700 1 --2- 172.21.15.52:0/4104977557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c32a9cd880 0x55c32a9d65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.432+0000 7f1f16a56700 1 -- 172.21.15.52:0/4104977557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c32a9cd880 msgr2=0x55c32a9d65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:06.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.432+0000 7f1f16a56700 1 --2- 172.21.15.52:0/4104977557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c32a9cd880 0x55c32a9d65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.444 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.432+0000 7f1f16a56700 1 -- 172.21.15.52:0/4104977557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c32a9ccf70 msgr2=0x55c32a9d9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:06.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.432+0000 7f1f16a56700 1 --2- 172.21.15.52:0/4104977557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c32a9ccf70 0x55c32a9d9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.432+0000 7f1f16a56700 1 -- 172.21.15.52:0/4104977557 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f1010f080 con 0x55c32a9d1cc0 2023-11-27T15:28:06.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.432+0000 7f1f16255700 1 --2- 172.21.15.52:0/4104977557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c32a9ccf70 0x55c32a9d9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:28:06.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.432+0000 7f1f16a56700 1 --2- 172.21.15.52:0/4104977557 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c32a9d1cc0 0x55c32a9d6b00 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f1f08007bf0 tx=0x7f1f08007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7647ea0b7d8f4b65 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:06.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.432+0000 7f1ef9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:28:06.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.432+0000 7f1ef9ffb700 1 -- 172.21.15.52:0/4104977557 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0800e070 con 0x55c32a9d1cc0 2023-11-27T15:28:06.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.432+0000 7f1ef9ffb700 1 -- 172.21.15.52:0/4104977557 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f1f08012b20 con 0x55c32a9d1cc0 2023-11-27T15:28:06.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.432+0000 7f1f24645b80 1 -- 172.21.15.52:0/4104977557 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c32a9bbb20 con 0x55c32a9d1cc0 2023-11-27T15:28:06.445 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.432+0000 7f1ef9ffb700 1 -- 172.21.15.52:0/4104977557 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f080090b0 con 0x55c32a9d1cc0 2023-11-27T15:28:06.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.432+0000 7f1f24645b80 10 client.17265 fetch_fsmap learned FSMap version 34 2023-11-27T15:28:06.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.432+0000 7f1f24645b80 1 -- 172.21.15.52:0/4104977557 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c32a8e0190 con 0x55c32a9d1cc0 2023-11-27T15:28:06.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.432+0000 7f1ef9ffb700 1 -- 172.21.15.52:0/4104977557 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c32a9bbb20 con 0x55c32a9d1cc0 2023-11-27T15:28:06.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.432+0000 7f1ef9ffb700 1 -- 172.21.15.52:0/4104977557 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1f08014040 con 0x55c32a9d1cc0 2023-11-27T15:28:06.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.432+0000 7f1f24645b80 10 client.17265 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:28:06.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.432+0000 7f1f24645b80 10 client.17265 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:28:06.446 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.433+0000 7f1f24645b80 20 client.17265 populate_metadata read hostname 'smithi052' 2023-11-27T15:28:06.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.433+0000 7f1f24645b80 1 --2- 172.21.15.52:0/4104977557 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c32a9e6960 0x55c32a9e8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.433+0000 7f1f24645b80 4 client.17265 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:28:06.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.433+0000 7f1f24645b80 1 -- 172.21.15.52:0/4104977557 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c32a8dc0a0 con 0x55c32a9e6960 2023-11-27T15:28:06.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.433+0000 7f1f16255700 1 --2- 172.21.15.52:0/4104977557 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c32a9e6960 0x55c32a9e8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.433+0000 7f1ef9ffb700 10 client.17265 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:28:06.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.433+0000 7f1f16255700 1 --2- 172.21.15.52:0/4104977557 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c32a9e6960 0x55c32a9e8d70 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ce1afe2d63f667d server_cookie=934876832799ad7c in_seq=0 out_seq=0 2023-11-27T15:28:06.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.434+0000 7f1ef9ffb700 1 -- 172.21.15.52:0/4104977557 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c32a8dc0a0 con 0x55c32a9e6960 2023-11-27T15:28:06.447 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.434+0000 7f1ef9ffb700 10 client.17265 handle_command_reply: tid=0 2023-11-27T15:28:06.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.434+0000 7f1f24645b80 1 client.17265 shutdown 2023-11-27T15:28:06.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.434+0000 7f1f24645b80 1 -- 172.21.15.52:0/4104977557 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c32a9d1cc0 msgr2=0x55c32a9d6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:06.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.434+0000 7f1f24645b80 1 --2- 172.21.15.52:0/4104977557 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c32a9d1cc0 0x55c32a9d6b00 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f1f08007bf0 tx=0x7f1f08007f00 comp rx=0 tx=0).stop 2023-11-27T15:28:06.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.434+0000 7f1f24645b80 1 -- 172.21.15.52:0/4104977557 shutdown_connections 2023-11-27T15:28:06.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.434+0000 7f1f24645b80 1 --2- 172.21.15.52:0/4104977557 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c32a9e6960 0x55c32a9e8d70 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.434+0000 7f1f24645b80 1 --2- 172.21.15.52:0/4104977557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c32a9ccf70 0x55c32a9d9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.434+0000 7f1f24645b80 1 --2- 172.21.15.52:0/4104977557 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c32a9d1cc0 0x55c32a9d6b00 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.448 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.434+0000 7f1f24645b80 1 --2- 172.21.15.52:0/4104977557 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c32a9cd880 0x55c32a9d65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.434+0000 7f1ef9ffb700 0 client.17265 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:28:06.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.434+0000 7f1f24645b80 1 -- 172.21.15.52:0/4104977557 >> 172.21.15.52:0/4104977557 conn(0x55c32a9be360 msgr2=0x55c32a9d3170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:06.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.434+0000 7f1f24645b80 1 -- 172.21.15.52:0/4104977557 shutdown_connections 2023-11-27T15:28:06.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.434+0000 7f1f24645b80 1 -- 172.21.15.52:0/4104977557 wait complete. 2023-11-27T15:28:06.449 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.434+0000 7f1f24645b80 20 client.17265 trim_cache size 0 max 16384 2023-11-27T15:28:06.455 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.447+0000 7f1f24645b80 1 Processor -- start 2023-11-27T15:28:06.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.447+0000 7f1f24645b80 1 -- start start 2023-11-27T15:28:06.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.447+0000 7f1f24645b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c32a9ccf70 0x55c32a9ddb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.447+0000 7f1f24645b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c32a9cd880 0x55c32a9d4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.447+0000 7f1f24645b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c32a9d1cc0 0x55c32a9d52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.447+0000 7f1f24645b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c32a9e3320 con 0x55c32a9d1cc0 2023-11-27T15:28:06.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.447+0000 7f1f24645b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c32a9c5370 con 0x55c32a9ccf70 2023-11-27T15:28:06.456 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.447+0000 7f1f24645b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c32a9bdee0 con 0x55c32a9cd880 2023-11-27T15:28:06.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.447+0000 7f1f16a56700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c32a9d1cc0 0x55c32a9d52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.447+0000 7f1f16a56700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c32a9d1cc0 0x55c32a9d52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56404/0 (socket says 172.21.15.52:56404) 2023-11-27T15:28:06.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.447+0000 7f1f16a56700 1 -- 172.21.15.52:0/3254560434 learned_addr learned my addr 172.21.15.52:0/3254560434 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:06.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.447+0000 7f1f15a54700 1 --2- 172.21.15.52:0/3254560434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c32a9cd880 0x55c32a9d4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.447+0000 7f1f16255700 1 --2- 172.21.15.52:0/3254560434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c32a9ccf70 0x55c32a9ddb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.447+0000 7f1f16a56700 1 -- 172.21.15.52:0/3254560434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c32a9cd880 msgr2=0x55c32a9d4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:06.457 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.448+0000 7f1f16a56700 1 --2- 172.21.15.52:0/3254560434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c32a9cd880 0x55c32a9d4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.448+0000 7f1f16a56700 1 -- 172.21.15.52:0/3254560434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c32a9ccf70 msgr2=0x55c32a9ddb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:06.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.448+0000 7f1f16a56700 1 --2- 172.21.15.52:0/3254560434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c32a9ccf70 0x55c32a9ddb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.448+0000 7f1f16a56700 1 -- 172.21.15.52:0/3254560434 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c32a8dfee0 con 0x55c32a9d1cc0 2023-11-27T15:28:06.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.448+0000 7f1f16255700 1 --2- 172.21.15.52:0/3254560434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c32a9ccf70 0x55c32a9ddb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:28:06.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.448+0000 7f1f15a54700 1 --2- 172.21.15.52:0/3254560434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c32a9cd880 0x55c32a9d4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:28:06.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.448+0000 7f1f16a56700 1 --2- 172.21.15.52:0/3254560434 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c32a9d1cc0 0x55c32a9d52b0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800b010 tx=0x7f1f080125a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8566af061b40fe1d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:06.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.448+0000 7f1f057fa700 1 -- 172.21.15.52:0/3254560434 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0800e040 con 0x55c32a9d1cc0 2023-11-27T15:28:06.458 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.448+0000 7f1f057fa700 1 -- 172.21.15.52:0/3254560434 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f0800f040 con 0x55c32a9d1cc0 2023-11-27T15:28:06.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.448+0000 7f1f057fa700 1 -- 172.21.15.52:0/3254560434 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f08015c40 con 0x55c32a9d1cc0 2023-11-27T15:28:06.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.448+0000 7f1f24645b80 1 -- 172.21.15.52:0/3254560434 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c32a9d1cc0 msgr2=0x55c32a9d52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:06.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.448+0000 7f1f24645b80 1 --2- 172.21.15.52:0/3254560434 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c32a9d1cc0 0x55c32a9d52b0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800b010 tx=0x7f1f080125a0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.448+0000 7f1f24645b80 1 -- 172.21.15.52:0/3254560434 shutdown_connections 2023-11-27T15:28:06.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.448+0000 7f1f24645b80 1 --2- 172.21.15.52:0/3254560434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c32a9ccf70 0x55c32a9ddb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.448+0000 7f1f24645b80 1 --2- 172.21.15.52:0/3254560434 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c32a9d1cc0 0x55c32a9d52b0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.448+0000 7f1f24645b80 1 --2- 172.21.15.52:0/3254560434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c32a9cd880 0x55c32a9d4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.459 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.448+0000 7f1f24645b80 1 -- 172.21.15.52:0/3254560434 >> 172.21.15.52:0/3254560434 conn(0x55c32a9be360 msgr2=0x55c32a9c3640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:06.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.448+0000 7f1f24645b80 1 -- 172.21.15.52:0/3254560434 shutdown_connections 2023-11-27T15:28:06.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.449+0000 7f1f24645b80 1 -- 172.21.15.52:0/3254560434 wait complete. 2023-11-27T15:28:06.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.449+0000 7f1f24645b80 1 -- start start 2023-11-27T15:28:06.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.449+0000 7f1f24645b80 1 Processor -- start 2023-11-27T15:28:06.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.449+0000 7f1f24645b80 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c32a9ccf70 0x55c32a9d9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.449+0000 7f1f24645b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c32a9cd880 0x55c32a9da080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.449+0000 7f1f24645b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c32a9d1cc0 0x55c32aa39810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.449+0000 7f1f24645b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c32a9da990 con 0x55c32a9d1cc0 2023-11-27T15:28:06.460 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.449+0000 7f1f24645b80 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c32aa39d50 con 0x55c32a9ccf70 2023-11-27T15:28:06.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.449+0000 7f1f24645b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c32aa39e90 con 0x55c32a9cd880 2023-11-27T15:28:06.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.449+0000 7f1f24645b80 10 client.0 _ll_register_callbacks cb 0x7f1f08007590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:06.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.449+0000 7f1f16a56700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c32a9d1cc0 0x55c32aa39810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.449+0000 7f1f16a56700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c32a9d1cc0 0x55c32aa39810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56410/0 (socket says 172.21.15.52:56410) 2023-11-27T15:28:06.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.449+0000 7f1f16a56700 1 -- 172.21.15.52:0/51794369 learned_addr learned my addr 172.21.15.52:0/51794369 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:06.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.449+0000 7f1f16255700 1 --2- 172.21.15.52:0/51794369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c32a9ccf70 0x55c32a9d9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.450+0000 7f1f15a54700 1 --2- 172.21.15.52:0/51794369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c32a9cd880 0x55c32a9da080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.461 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.450+0000 7f1f16a56700 1 -- 172.21.15.52:0/51794369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c32a9cd880 msgr2=0x55c32a9da080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:06.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.450+0000 7f1f16a56700 1 --2- 172.21.15.52:0/51794369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c32a9cd880 0x55c32a9da080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.450+0000 7f1f16a56700 1 -- 172.21.15.52:0/51794369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c32a9ccf70 msgr2=0x55c32a9d9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:06.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.450+0000 7f1f16a56700 1 --2- 172.21.15.52:0/51794369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c32a9ccf70 0x55c32a9d9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.450+0000 7f1f16a56700 1 -- 172.21.15.52:0/51794369 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c32a8e0190 con 0x55c32a9d1cc0 2023-11-27T15:28:06.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.450+0000 7f1f16a56700 1 --2- 172.21.15.52:0/51794369 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c32a9d1cc0 0x55c32aa39810 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009ce0 tx=0x7f1f08009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab57182b18c5a1c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:06.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.450+0000 7f1ef9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2023-11-27T15:28:06.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.450+0000 7f1ef9ffb700 1 -- 172.21.15.52:0/51794369 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0800e040 con 0x55c32a9d1cc0 2023-11-27T15:28:06.462 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.450+0000 7f1ef9ffb700 1 -- 172.21.15.52:0/51794369 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f1f08012b20 con 0x55c32a9d1cc0 2023-11-27T15:28:06.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.450+0000 7f1f24645b80 1 -- 172.21.15.52:0/51794369 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f000a2e60 con 0x55c32a9d1cc0 2023-11-27T15:28:06.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.450+0000 7f1ef9ffb700 1 -- 172.21.15.52:0/51794369 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f08012ce0 con 0x55c32a9d1cc0 2023-11-27T15:28:06.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.450+0000 7f1f24645b80 10 client.17271 fetch_fsmap learned FSMap version 34 2023-11-27T15:28:06.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.450+0000 7f1f24645b80 1 -- 172.21.15.52:0/51794369 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c32a730c60 con 0x55c32a9d1cc0 2023-11-27T15:28:06.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.450+0000 7f1ef9ffb700 1 -- 172.21.15.52:0/51794369 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f0803d080 con 0x55c32a9d1cc0 2023-11-27T15:28:06.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.450+0000 7f1ef9ffb700 1 -- 172.21.15.52:0/51794369 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1f08014040 con 0x55c32a9d1cc0 2023-11-27T15:28:06.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.450+0000 7f1f24645b80 10 client.17271 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:28:06.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.450+0000 7f1f24645b80 10 client.17271 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:28:06.463 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.450+0000 7f1f24645b80 20 client.17271 populate_metadata read hostname 'smithi052' 2023-11-27T15:28:06.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.450+0000 7f1f24645b80 1 --2- 172.21.15.52:0/51794369 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c32aa3afb0 0x55c32aa3d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:06.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.450+0000 7f1f24645b80 4 client.17271 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:28:06.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.450+0000 7f1f24645b80 1 -- 172.21.15.52:0/51794369 --> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c32a8e8660 con 0x55c32aa3afb0 2023-11-27T15:28:06.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.451+0000 7f1f16255700 1 --2- 172.21.15.52:0/51794369 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c32aa3afb0 0x55c32aa3d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:06.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.451+0000 7f1f16255700 1 --2- 172.21.15.52:0/51794369 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c32aa3afb0 0x55c32aa3d390 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1d32cd7d6ffeff9 server_cookie=a17e075fac36b2f2 in_seq=0 out_seq=0 2023-11-27T15:28:06.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.451+0000 7f1ef9ffb700 10 client.17271 ms_handle_connect on v2:172.21.15.102:6838/3571830559 2023-11-27T15:28:06.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.451+0000 7f1ef9ffb700 1 -- 172.21.15.52:0/51794369 <== mds.0 v2:172.21.15.102:6838/3571830559 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c32a8e8660 con 0x55c32aa3afb0 2023-11-27T15:28:06.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.451+0000 7f1ef9ffb700 10 client.17271 handle_command_reply: tid=0 2023-11-27T15:28:06.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.451+0000 7f1f24645b80 1 client.17271 shutdown 2023-11-27T15:28:06.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.451+0000 7f1f24645b80 1 -- 172.21.15.52:0/51794369 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c32a9d1cc0 msgr2=0x55c32aa39810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:06.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.451+0000 7f1f24645b80 1 --2- 172.21.15.52:0/51794369 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c32a9d1cc0 0x55c32aa39810 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009ce0 tx=0x7f1f08009950 comp rx=0 tx=0).stop 2023-11-27T15:28:06.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.451+0000 7f1f24645b80 1 -- 172.21.15.52:0/51794369 shutdown_connections 2023-11-27T15:28:06.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.451+0000 7f1f24645b80 1 --2- 172.21.15.52:0/51794369 >> [v2:172.21.15.102:6838/3571830559,v1:172.21.15.102:6839/3571830559] conn(0x55c32aa3afb0 0x55c32aa3d390 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.451+0000 7f1f24645b80 1 --2- 172.21.15.52:0/51794369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c32a9ccf70 0x55c32a9d9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.451+0000 7f1f24645b80 1 --2- 172.21.15.52:0/51794369 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c32a9d1cc0 0x55c32aa39810 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.451+0000 7f1ef9ffb700 0 client.17271 ms_handle_reset on v2:172.21.15.102:6838/3571830559 2023-11-27T15:28:06.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.451+0000 7f1f24645b80 1 --2- 172.21.15.52:0/51794369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c32a9cd880 0x55c32a9da080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.451+0000 7f1f24645b80 1 -- 172.21.15.52:0/51794369 >> 172.21.15.52:0/51794369 conn(0x55c32a9be360 msgr2=0x55c32a9d4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:06.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.452+0000 7f1f24645b80 1 -- 172.21.15.52:0/51794369 shutdown_connections 2023-11-27T15:28:06.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.452+0000 7f1f24645b80 1 -- 172.21.15.52:0/51794369 wait complete. 2023-11-27T15:28:06.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.452+0000 7f1f24645b80 20 client.17271 trim_cache size 0 max 16384 2023-11-27T15:28:06.466 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2023-11-27T15:28:06.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.452+0000 7f1f184b9700 1 -- 172.21.15.52:0/3745986658 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f1efc0a12c0 msgr2=0x7f1efc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:06.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.452+0000 7f1f184b9700 1 --2- 172.21.15.52:0/3745986658 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f1efc0a12c0 0x7f1efc0a3770 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c005950 tx=0x7f1f0c0058e0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.452+0000 7f1f184b9700 1 -- 172.21.15.52:0/3745986658 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f100ffdf0 msgr2=0x7f1f10197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:06.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.452+0000 7f1f184b9700 1 --2- 172.21.15.52:0/3745986658 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f100ffdf0 0x7f1f10197740 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f1f00009fd0 tx=0x7f1f00009580 comp rx=0 tx=0).stop 2023-11-27T15:28:06.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.452+0000 7f1f184b9700 1 -- 172.21.15.52:0/3745986658 shutdown_connections 2023-11-27T15:28:06.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.452+0000 7f1f184b9700 1 --2- 172.21.15.52:0/3745986658 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f1efc0a12c0 0x7f1efc0a3770 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.452+0000 7f1f184b9700 1 --2- 172.21.15.52:0/3745986658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f10104490 0x7f1f101a2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.452+0000 7f1f184b9700 1 --2- 172.21.15.52:0/3745986658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f100ff450 0x7f1f10197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.452+0000 7f1f184b9700 1 --2- 172.21.15.52:0/3745986658 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f100ffdf0 0x7f1f10197740 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:06.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.452+0000 7f1f184b9700 1 -- 172.21.15.52:0/3745986658 >> 172.21.15.52:0/3745986658 conn(0x7f1f10077080 msgr2=0x7f1f100fcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:06.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.452+0000 7f1f184b9700 1 -- 172.21.15.52:0/3745986658 shutdown_connections 2023-11-27T15:28:06.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:06.452+0000 7f1f184b9700 1 -- 172.21.15.52:0/3745986658 wait complete. 2023-11-27T15:28:06.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:06 smithi150 ceph-mon[100559]: pgmap v316: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-11-27T15:28:06.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:06 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3180186663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:28:06.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:06 smithi102 ceph-mon[100296]: pgmap v316: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-11-27T15:28:06.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:06 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3180186663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:28:06.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:06 smithi052 ceph-mon[99056]: pgmap v316: 97 pgs: 97 active+clean; 712 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-11-27T15:28:06.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:06 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3180186663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:28:07.470 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2023-11-27T15:28:07.471 INFO:tasks.fwd_scrub:done joining 2023-11-27T15:28:07.471 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-11-27T15:28:07.482 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:28:07.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.619+0000 7fc111c9a700 1 -- 172.21.15.52:0/1592000157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc10c1044f0 msgr2=0x7fc10c10c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:07.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.619+0000 7fc111c9a700 1 --2- 172.21.15.52:0/1592000157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc10c1044f0 0x7fc10c10c0c0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fc10400b0d0 tx=0x7fc10400b3e0 comp rx=0 tx=0).stop 2023-11-27T15:28:07.623 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.619+0000 7fc111c9a700 1 -- 172.21.15.52:0/1592000157 shutdown_connections 2023-11-27T15:28:07.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.619+0000 7fc111c9a700 1 --2- 172.21.15.52:0/1592000157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc10c0ff4b0 0x7fc10c0ff880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:07.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.619+0000 7fc111c9a700 1 --2- 172.21.15.52:0/1592000157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc10c1044f0 0x7fc10c10c0c0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:07.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.619+0000 7fc111c9a700 1 --2- 172.21.15.52:0/1592000157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc10c0ffe50 0x7fc10c103e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:07.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.619+0000 7fc111c9a700 1 -- 172.21.15.52:0/1592000157 >> 172.21.15.52:0/1592000157 conn(0x7fc10c0760d0 msgr2=0x7fc10c0764d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:07.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.619+0000 7fc111c9a700 1 -- 172.21.15.52:0/1592000157 shutdown_connections 2023-11-27T15:28:07.624 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.619+0000 7fc111c9a700 1 -- 172.21.15.52:0/1592000157 wait complete. 2023-11-27T15:28:07.626 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.620+0000 7fc111c9a700 1 Processor -- start 2023-11-27T15:28:07.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.620+0000 7fc111c9a700 1 -- start start 2023-11-27T15:28:07.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.620+0000 7fc111c9a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc10c0ff4b0 0x7fc10c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:07.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.620+0000 7fc111c9a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc10c0ffe50 0x7fc10c198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:07.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.620+0000 7fc111c9a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc10c1044f0 0x7fc10c19bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:07.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.620+0000 7fc111c9a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10c19c5b0 con 0x7fc10c0ffe50 2023-11-27T15:28:07.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.620+0000 7fc111c9a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10c19c720 con 0x7fc10c0ff4b0 2023-11-27T15:28:07.627 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.620+0000 7fc111c9a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10c19c8e0 con 0x7fc10c1044f0 2023-11-27T15:28:07.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.621+0000 7fc10affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc10c0ffe50 0x7fc10c198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:07.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.621+0000 7fc10affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc10c0ffe50 0x7fc10c198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56422/0 (socket says 172.21.15.52:56422) 2023-11-27T15:28:07.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.621+0000 7fc10affd700 1 -- 172.21.15.52:0/4265018433 learned_addr learned my addr 172.21.15.52:0/4265018433 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:07.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.621+0000 7fc10b7fe700 1 --2- 172.21.15.52:0/4265018433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc10c0ff4b0 0x7fc10c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:07.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.621+0000 7fc10bfff700 1 --2- 172.21.15.52:0/4265018433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc10c1044f0 0x7fc10c19bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:07.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.621+0000 7fc10affd700 1 -- 172.21.15.52:0/4265018433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc10c1044f0 msgr2=0x7fc10c19bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:07.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.621+0000 7fc10affd700 1 --2- 172.21.15.52:0/4265018433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc10c1044f0 0x7fc10c19bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:07.628 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.621+0000 7fc10affd700 1 -- 172.21.15.52:0/4265018433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc10c0ff4b0 msgr2=0x7fc10c1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:07.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.621+0000 7fc10affd700 1 --2- 172.21.15.52:0/4265018433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc10c0ff4b0 0x7fc10c1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:07.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.621+0000 7fc10affd700 1 -- 172.21.15.52:0/4265018433 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc104009d00 con 0x7fc10c0ffe50 2023-11-27T15:28:07.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.621+0000 7fc10b7fe700 1 --2- 172.21.15.52:0/4265018433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc10c0ff4b0 0x7fc10c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:28:07.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.621+0000 7fc10affd700 1 --2- 172.21.15.52:0/4265018433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc10c0ffe50 0x7fc10c198c00 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400c9b0 tx=0x7fc0f400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:07.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.621+0000 7fc108ff9700 1 -- 172.21.15.52:0/4265018433 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4007aa0 con 0x7fc10c0ffe50 2023-11-27T15:28:07.629 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.621+0000 7fc108ff9700 1 -- 172.21.15.52:0/4265018433 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0f4007c00 con 0x7fc10c0ffe50 2023-11-27T15:28:07.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.621+0000 7fc111c9a700 1 -- 172.21.15.52:0/4265018433 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc10c10f310 con 0x7fc10c0ffe50 2023-11-27T15:28:07.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.621+0000 7fc108ff9700 1 -- 172.21.15.52:0/4265018433 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4018660 con 0x7fc10c0ffe50 2023-11-27T15:28:07.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.621+0000 7fc111c9a700 1 -- 172.21.15.52:0/4265018433 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc10c10f180 con 0x7fc10c0ffe50 2023-11-27T15:28:07.630 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.622+0000 7fc111c9a700 1 -- 172.21.15.52:0/4265018433 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc10c10f4a0 con 0x7fc10c0ffe50 2023-11-27T15:28:07.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.624+0000 7fc108ff9700 1 -- 172.21.15.52:0/4265018433 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fc0f400f450 con 0x7fc10c0ffe50 2023-11-27T15:28:07.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.624+0000 7fc108ff9700 1 --2- 172.21.15.52:0/4265018433 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fc0f80a1340 0x7fc0f80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:07.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.624+0000 7fc108ff9700 1 -- 172.21.15.52:0/4265018433 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fc0f40cd060 con 0x7fc10c0ffe50 2023-11-27T15:28:07.631 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.624+0000 7fc10b7fe700 1 --2- 172.21.15.52:0/4265018433 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fc0f80a1340 0x7fc0f80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:07.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.625+0000 7fc10b7fe700 1 --2- 172.21.15.52:0/4265018433 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fc0f80a1340 0x7fc0f80a37f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc009cb0 tx=0x7fc0fc009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:07.632 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.628+0000 7fc108ff9700 1 -- 172.21.15.52:0/4265018433 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc0f40964c0 con 0x7fc10c0ffe50 2023-11-27T15:28:07.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.830+0000 7fc111c9a700 1 -- 172.21.15.52:0/4265018433 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc10c067eb0 con 0x7fc10c0ffe50 2023-11-27T15:28:07.835 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.831+0000 7fc108ff9700 1 -- 172.21.15.52:0/4265018433 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+23125 (secure 0 0 0) 0x7fc0f4099ae0 con 0x7fc10c0ffe50 2023-11-27T15:28:07.835 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:28:07.835 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":94,"fsid":"c6c7df88-8d36-11ee-95a2-87774f69a715","created":"2023-11-27T15:09:03.333345+0000","modified":"2023-11-27T15:27:13.916052+0000","last_up_change":"2023-11-27T15:15:27.656134+0000","last_in_change":"2023-11-27T15:15:10.671153+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:12:14.177523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:16:07.842251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:16:08.728520+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":93,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58d69ede-c293-49f5-abbe-8187793f2b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6803","nonce":608216415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6805","nonce":608216415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6809","nonce":608216415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6807","nonce":608216415}]},"public_addr":"172.21.15.52:6803/608216415","cluster_addr":"172.21.15.52:6805/608216415","heartbeat_back_addr":"172.21.15.52:6809/608216415","heartbeat_front_addr":"172.21.15.52:6807/608216415","state":["exists","up"]},{"osd":1,"uuid":"d6a46b91-adc8-45ef-80f5-566ae400911f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6801","nonce":2538384087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6803","nonce":2538384087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6807","nonce":2538384087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6805","nonce":2538384087}]},"public_addr":"172.21.15.102:6801/2538384087","cluster_addr":"172.21.15.102:6803/2538384087","heartbeat_back_addr":"172.21.15.102:6807/2538384087","heartbeat_front_addr":"172.21.15.102:6805/2538384087","state":["exists","up"]},{"osd":2,"uuid":"846ba125-11f8-46b2-8941-bd5e1947cc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6801","nonce":4258038221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6803","nonce":4258038221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6807","nonce":4258038221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6805","nonce":4258038221}]},"public_addr":"172.21.15.150:6801/4258038221","cluster_addr":"172.21.15.150:6803/4258038221","heartbeat_back_addr":"172.21.15.150:6807/4258038221","heartbeat_front_addr":"172.21.15.150:6805/4258038221","state":["exists","up"]},{"osd":3,"uuid":"950fcd67-1057-4288-a389-714f1ec94261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6811","nonce":2622100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6813","nonce":2622100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6817","nonce":2622100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6815","nonce":2622100752}]},"public_addr":"172.21.15.52:6811/2622100752","cluster_addr":"172.21.15.52:6813/2622100752","heartbeat_back_addr":"172.21.15.52:6817/2622100752","heartbeat_front_addr":"172.21.15.52:6815/2622100752","state":["exists","up"]},{"osd":4,"uuid":"1c96a676-cea9-4deb-bad1-f7fd240c3d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6809","nonce":1546794763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6811","nonce":1546794763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6815","nonce":1546794763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6813","nonce":1546794763}]},"public_addr":"172.21.15.102:6809/1546794763","cluster_addr":"172.21.15.102:6811/1546794763","heartbeat_back_addr":"172.21.15.102:6815/1546794763","heartbeat_front_addr":"172.21.15.102:6813/1546794763","state":["exists","up"]},{"osd":5,"uuid":"38a75af1-fd4d-433e-b579-0609109c851c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6809","nonce":461511375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6811","nonce":461511375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6815","nonce":461511375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6813","nonce":461511375}]},"public_addr":"172.21.15.150:6809/461511375","cluster_addr":"172.21.15.150:6811/461511375","heartbeat_back_addr":"172.21.15.150:6815/461511375","heartbeat_front_addr":"172.21.15.150:6813/461511375","state":["exists","up"]},{"osd":6,"uuid":"2f0ede4a-d175-4b4a-b876-c0f442048901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6819","nonce":1002457464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6821","nonce":1002457464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6825","nonce":1002457464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6823","nonce":1002457464}]},"public_addr":"172.21.15.52:6819/1002457464","cluster_addr":"172.21.15.52:6821/1002457464","heartbeat_back_addr":"172.21.15.52:6825/1002457464","heartbeat_front_addr":"172.21.15.52:6823/1002457464","state":["exists","up"]},{"osd":7,"uuid":"6b635192-abcf-4bbf-bf78-a019b351bca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6817","nonce":3249840064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6819","nonce":3249840064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6823","nonce":3249840064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6821","nonce":3249840064}]},"public_addr":"172.21.15.102:6817/3249840064","cluster_addr":"172.21.15.102:6819/3249840064","heartbeat_back_addr":"172.21.15.102:6823/3249840064","heartbeat_front_addr":"172.21.15.102:6821/3249840064","state":["exists","up"]},{"osd":8,"uuid":"c7ca8007-7170-47bf-956c-3f00edf27e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6817","nonce":3672583574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6819","nonce":3672583574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6823","nonce":3672583574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6821","nonce":3672583574}]},"public_addr":"172.21.15.150:6817/3672583574","cluster_addr":"172.21.15.150:6819/3672583574","heartbeat_back_addr":"172.21.15.150:6823/3672583574","heartbeat_front_addr":"172.21.15.150:6821/3672583574","state":["exists","up"]},{"osd":9,"uuid":"0e252c2c-01a4-48ba-832d-067ed940fefb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6827","nonce":4287934320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6829","nonce":4287934320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6833","nonce":4287934320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6831","nonce":4287934320}]},"public_addr":"172.21.15.52:6827/4287934320","cluster_addr":"172.21.15.52:6829/4287934320","heartbeat_back_addr":"172.21.15.52:6833/4287934320","heartbeat_front_addr":"172.21.15.52:6831/4287934320","state":["exists","up"]},{"osd":10,"uuid":"bb540142-7563-44a4-b7a3-cb662714c128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6825","nonce":3056034201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6827","nonce":3056034201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6831","nonce":3056034201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6829","nonce":3056034201}]},"public_addr":"172.21.15.102:6825/3056034201","cluster_addr":"172.21.15.102:6827/3056034201","heartbeat_back_addr":"172.21.15.102:6831/3056034201","heartbeat_front_addr":"172.21.15.102:6829/3056034201","state":["exists","up"]},{"osd":11,"uuid":"3156528e-e0f8-4d44-9946-c8b6339c2bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6825","nonce":3207559739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6827","nonce":3207559739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6831","nonce":3207559739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6829","nonce":3207559739}]},"public_addr":"172.21.15.150:6825/3207559739","cluster_addr":"172.21.15.150:6827/3207559739","heartbeat_back_addr":"172.21.15.150:6831/3207559739","heartbeat_front_addr":"172.21.15.150:6829/3207559739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:26.826099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:48.331371+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:10.459738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:32.356472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:54.043170+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:15.648582+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:37.166463+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:59.240819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:20.710097+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:43.364563+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:05.279433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:26.344573+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":5,"to":7}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/4025191157":"2023-11-28T15:17:54.913525+0000","172.21.15.52:6801/2640036585":"2023-11-28T15:17:54.913525+0000","172.21.15.52:6800/2640036585":"2023-11-28T15:17:54.913525+0000","172.21.15.52:0/1392282110":"2023-11-28T15:10:03.644870+0000","172.21.15.52:0/4081249946":"2023-11-28T15:17:54.913525+0000","172.21.15.52:0/1479993479":"2023-11-28T15:10:03.644870+0000","172.21.15.52:0/2690561203":"2023-11-28T15:17:54.913525+0000","172.21.15.52:0/2454989397":"2023-11-28T15:17:54.913525+0000","172.21.15.52:6801/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6800/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/961562652":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/1290665390":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2410529742":"2023-11-28T15:09:37.118690+0000","172.21.15.52:6800/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2130037044":"2023-11-28T15:10:03.644870+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:28:07.837 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.833+0000 7fc111c9a700 1 -- 172.21.15.52:0/4265018433 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fc0f80a1340 msgr2=0x7fc0f80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:07.837 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.833+0000 7fc111c9a700 1 --2- 172.21.15.52:0/4265018433 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fc0f80a1340 0x7fc0f80a37f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc009cb0 tx=0x7fc0fc009260 comp rx=0 tx=0).stop 2023-11-27T15:28:07.837 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.833+0000 7fc111c9a700 1 -- 172.21.15.52:0/4265018433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc10c0ffe50 msgr2=0x7fc10c198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:07.837 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.833+0000 7fc111c9a700 1 --2- 172.21.15.52:0/4265018433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc10c0ffe50 0x7fc10c198c00 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400c9b0 tx=0x7fc0f400cd70 comp rx=0 tx=0).stop 2023-11-27T15:28:07.838 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.834+0000 7fc111c9a700 1 -- 172.21.15.52:0/4265018433 shutdown_connections 2023-11-27T15:28:07.838 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.834+0000 7fc111c9a700 1 --2- 172.21.15.52:0/4265018433 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fc0f80a1340 0x7fc0f80a37f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:07.838 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.834+0000 7fc111c9a700 1 --2- 172.21.15.52:0/4265018433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc10c0ff4b0 0x7fc10c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:07.838 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.834+0000 7fc111c9a700 1 --2- 172.21.15.52:0/4265018433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc10c1044f0 0x7fc10c19bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:07.838 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.834+0000 7fc111c9a700 1 --2- 172.21.15.52:0/4265018433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc10c0ffe50 0x7fc10c198c00 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:07.838 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.834+0000 7fc111c9a700 1 -- 172.21.15.52:0/4265018433 >> 172.21.15.52:0/4265018433 conn(0x7fc10c0760d0 msgr2=0x7fc10c0fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:07.838 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.834+0000 7fc111c9a700 1 -- 172.21.15.52:0/4265018433 shutdown_connections 2023-11-27T15:28:07.839 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.834+0000 7fc111c9a700 1 -- 172.21.15.52:0/4265018433 wait complete. 2023-11-27T15:28:07.849 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:12:14.177523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-27T15:16:07.842251+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-27T15:16:08.728520+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 93, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-27T15:28:07.850 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:28:07.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.974+0000 7f6a379ba700 1 -- 172.21.15.52:0/3346933549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a300ff990 msgr2=0x7f6a300ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:07.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.974+0000 7f6a379ba700 1 --2- 172.21.15.52:0/3346933549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a300ff990 0x7f6a300ffe00 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c0099c0 tx=0x7f6a2c009cd0 comp rx=0 tx=0).stop 2023-11-27T15:28:07.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.974+0000 7f6a379ba700 1 -- 172.21.15.52:0/3346933549 shutdown_connections 2023-11-27T15:28:07.978 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.974+0000 7f6a379ba700 1 --2- 172.21.15.52:0/3346933549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a30100340 0x7f6a3010c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:07.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.974+0000 7f6a379ba700 1 --2- 172.21.15.52:0/3346933549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a30104180 0x7f6a30104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:07.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.974+0000 7f6a379ba700 1 --2- 172.21.15.52:0/3346933549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a300ff990 0x7f6a300ffe00 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:07.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.974+0000 7f6a379ba700 1 -- 172.21.15.52:0/3346933549 >> 172.21.15.52:0/3346933549 conn(0x7f6a30075920 msgr2=0x7f6a30075d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:07.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.974+0000 7f6a379ba700 1 -- 172.21.15.52:0/3346933549 shutdown_connections 2023-11-27T15:28:07.979 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.974+0000 7f6a379ba700 1 -- 172.21.15.52:0/3346933549 wait complete. 2023-11-27T15:28:07.981 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.975+0000 7f6a379ba700 1 Processor -- start 2023-11-27T15:28:07.981 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.975+0000 7f6a379ba700 1 -- start start 2023-11-27T15:28:07.981 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.975+0000 7f6a379ba700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a300ff990 0x7f6a30196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:07.981 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.975+0000 7f6a379ba700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a30100340 0x7f6a301968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:07.981 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.975+0000 7f6a379ba700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a30104180 0x7f6a3019ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:07.982 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.975+0000 7f6a379ba700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3019b340 con 0x7f6a30104180 2023-11-27T15:28:07.982 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.975+0000 7f6a379ba700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3019b4b0 con 0x7f6a30100340 2023-11-27T15:28:07.982 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.975+0000 7f6a379ba700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3019b670 con 0x7f6a300ff990 2023-11-27T15:28:07.982 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.975+0000 7f6a35f57700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a30104180 0x7f6a3019ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:07.982 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.975+0000 7f6a35f57700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a30104180 0x7f6a3019ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56438/0 (socket says 172.21.15.52:56438) 2023-11-27T15:28:07.982 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.975+0000 7f6a35756700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a300ff990 0x7f6a30196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:07.982 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.975+0000 7f6a35f57700 1 -- 172.21.15.52:0/790861914 learned_addr learned my addr 172.21.15.52:0/790861914 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:07.983 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.975+0000 7f6a34f55700 1 --2- 172.21.15.52:0/790861914 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a30100340 0x7f6a301968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:07.983 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.976+0000 7f6a35f57700 1 -- 172.21.15.52:0/790861914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a300ff990 msgr2=0x7f6a30196360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:07.983 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.976+0000 7f6a35f57700 1 --2- 172.21.15.52:0/790861914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a300ff990 0x7f6a30196360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:07.983 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.976+0000 7f6a35f57700 1 -- 172.21.15.52:0/790861914 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a30100340 msgr2=0x7f6a301968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:07.983 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.976+0000 7f6a35f57700 1 --2- 172.21.15.52:0/790861914 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a30100340 0x7f6a301968a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:07.983 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.976+0000 7f6a35f57700 1 -- 172.21.15.52:0/790861914 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a2c0095f0 con 0x7f6a30104180 2023-11-27T15:28:07.983 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.976+0000 7f6a35f57700 1 --2- 172.21.15.52:0/790861914 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a30104180 0x7f6a3019ad00 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f6a2800ab40 tx=0x7f6a2800ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:07.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.976+0000 7f6a267fc700 1 -- 172.21.15.52:0/790861914 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2800f070 con 0x7f6a30104180 2023-11-27T15:28:07.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.976+0000 7f6a267fc700 1 -- 172.21.15.52:0/790861914 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a28007a70 con 0x7f6a30104180 2023-11-27T15:28:07.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.976+0000 7f6a379ba700 1 -- 172.21.15.52:0/790861914 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a3010f210 con 0x7f6a30104180 2023-11-27T15:28:07.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.976+0000 7f6a267fc700 1 -- 172.21.15.52:0/790861914 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2800c580 con 0x7f6a30104180 2023-11-27T15:28:07.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.976+0000 7f6a379ba700 1 -- 172.21.15.52:0/790861914 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a3010f080 con 0x7f6a30104180 2023-11-27T15:28:07.984 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.977+0000 7f6a379ba700 1 -- 172.21.15.52:0/790861914 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a3010f3a0 con 0x7f6a30104180 2023-11-27T15:28:07.985 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.978+0000 7f6a267fc700 1 -- 172.21.15.52:0/790861914 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f6a28003a20 con 0x7f6a30104180 2023-11-27T15:28:07.985 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.979+0000 7f6a267fc700 1 --2- 172.21.15.52:0/790861914 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f6a1c0a12c0 0x7f6a1c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:07.985 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.979+0000 7f6a35756700 1 --2- 172.21.15.52:0/790861914 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f6a1c0a12c0 0x7f6a1c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:07.985 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.979+0000 7f6a267fc700 1 -- 172.21.15.52:0/790861914 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f6a280cbec0 con 0x7f6a30104180 2023-11-27T15:28:07.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.979+0000 7f6a35756700 1 --2- 172.21.15.52:0/790861914 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f6a1c0a12c0 0x7f6a1c0a3770 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6a20005950 tx=0x7f6a2000b410 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:07.986 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:07.983+0000 7f6a267fc700 1 -- 172.21.15.52:0/790861914 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6a28095200 con 0x7f6a30104180 2023-11-27T15:28:08.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.181+0000 7f6a379ba700 1 -- 172.21.15.52:0/790861914 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6a301a33d0 con 0x7f6a30104180 2023-11-27T15:28:08.185 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.182+0000 7f6a267fc700 1 -- 172.21.15.52:0/790861914 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6a28098820 con 0x7f6a30104180 2023-11-27T15:28:08.186 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2023-11-27T15:28:08.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.184+0000 7f6a379ba700 1 -- 172.21.15.52:0/790861914 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f6a1c0a12c0 msgr2=0x7f6a1c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:08.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.184+0000 7f6a379ba700 1 --2- 172.21.15.52:0/790861914 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f6a1c0a12c0 0x7f6a1c0a3770 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6a20005950 tx=0x7f6a2000b410 comp rx=0 tx=0).stop 2023-11-27T15:28:08.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.184+0000 7f6a379ba700 1 -- 172.21.15.52:0/790861914 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a30104180 msgr2=0x7f6a3019ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:08.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.184+0000 7f6a379ba700 1 --2- 172.21.15.52:0/790861914 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a30104180 0x7f6a3019ad00 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f6a2800ab40 tx=0x7f6a2800ae50 comp rx=0 tx=0).stop 2023-11-27T15:28:08.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.184+0000 7f6a379ba700 1 -- 172.21.15.52:0/790861914 shutdown_connections 2023-11-27T15:28:08.188 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.184+0000 7f6a379ba700 1 --2- 172.21.15.52:0/790861914 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f6a1c0a12c0 0x7f6a1c0a3770 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:08.189 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.184+0000 7f6a379ba700 1 --2- 172.21.15.52:0/790861914 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a30104180 0x7f6a3019ad00 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:08.189 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.184+0000 7f6a379ba700 1 --2- 172.21.15.52:0/790861914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a300ff990 0x7f6a30196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:08.189 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.184+0000 7f6a379ba700 1 --2- 172.21.15.52:0/790861914 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a30100340 0x7f6a301968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:08.189 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.184+0000 7f6a379ba700 1 -- 172.21.15.52:0/790861914 >> 172.21.15.52:0/790861914 conn(0x7f6a30075920 msgr2=0x7f6a300fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:08.189 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.184+0000 7f6a379ba700 1 -- 172.21.15.52:0/790861914 shutdown_connections 2023-11-27T15:28:08.189 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.184+0000 7f6a379ba700 1 -- 172.21.15.52:0/790861914 wait complete. 2023-11-27T15:28:08.200 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-27T15:28:08.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.325+0000 7fb4f3335700 1 -- 172.21.15.52:0/1559692843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4ec0feaf0 msgr2=0x7fb4ec0feec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:08.331 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.325+0000 7fb4f3335700 1 --2- 172.21.15.52:0/1559692843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4ec0feaf0 0x7fb4ec0feec0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80099c0 tx=0x7fb4e8009cd0 comp rx=0 tx=0).stop 2023-11-27T15:28:08.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.325+0000 7fb4f3335700 1 -- 172.21.15.52:0/1559692843 shutdown_connections 2023-11-27T15:28:08.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.325+0000 7fb4f3335700 1 --2- 172.21.15.52:0/1559692843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4ec1055d0 0x7fb4ec10d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:08.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.325+0000 7fb4f3335700 1 --2- 172.21.15.52:0/1559692843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4ec0feaf0 0x7fb4ec0feec0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:08.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.325+0000 7fb4f3335700 1 --2- 172.21.15.52:0/1559692843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4ec0ff490 0x7fb4ec104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:08.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.326+0000 7fb4f3335700 1 -- 172.21.15.52:0/1559692843 >> 172.21.15.52:0/1559692843 conn(0x7fb4ec075920 msgr2=0x7fb4ec075d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:08.332 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.326+0000 7fb4f3335700 1 -- 172.21.15.52:0/1559692843 shutdown_connections 2023-11-27T15:28:08.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.326+0000 7fb4f3335700 1 -- 172.21.15.52:0/1559692843 wait complete. 2023-11-27T15:28:08.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.326+0000 7fb4f3335700 1 Processor -- start 2023-11-27T15:28:08.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.326+0000 7fb4f3335700 1 -- start start 2023-11-27T15:28:08.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.326+0000 7fb4f3335700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4ec0feaf0 0x7fb4ec1963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:08.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.326+0000 7fb4f3335700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4ec0ff490 0x7fb4ec1968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:08.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.326+0000 7fb4f3335700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4ec1055d0 0x7fb4ec19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:08.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.326+0000 7fb4f3335700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ec19b390 con 0x7fb4ec0ff490 2023-11-27T15:28:08.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.326+0000 7fb4f3335700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ec19b500 con 0x7fb4ec1055d0 2023-11-27T15:28:08.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.326+0000 7fb4f3335700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ec19b6c0 con 0x7fb4ec0feaf0 2023-11-27T15:28:08.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.327+0000 7fb4f08d0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4ec0ff490 0x7fb4ec1968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:08.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.327+0000 7fb4f08d0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4ec0ff490 0x7fb4ec1968f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56450/0 (socket says 172.21.15.52:56450) 2023-11-27T15:28:08.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.327+0000 7fb4f08d0700 1 -- 172.21.15.52:0/772781610 learned_addr learned my addr 172.21.15.52:0/772781610 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:08.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.327+0000 7fb4f18d2700 1 --2- 172.21.15.52:0/772781610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4ec1055d0 0x7fb4ec19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:08.334 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.327+0000 7fb4f10d1700 1 --2- 172.21.15.52:0/772781610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4ec0feaf0 0x7fb4ec1963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:08.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.327+0000 7fb4f08d0700 1 -- 172.21.15.52:0/772781610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4ec0feaf0 msgr2=0x7fb4ec1963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:08.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.327+0000 7fb4f08d0700 1 --2- 172.21.15.52:0/772781610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4ec0feaf0 0x7fb4ec1963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:08.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.327+0000 7fb4f08d0700 1 -- 172.21.15.52:0/772781610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4ec1055d0 msgr2=0x7fb4ec19ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:08.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.327+0000 7fb4f08d0700 1 --2- 172.21.15.52:0/772781610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4ec1055d0 0x7fb4ec19ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:08.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.327+0000 7fb4f08d0700 1 -- 172.21.15.52:0/772781610 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e80095f0 con 0x7fb4ec0ff490 2023-11-27T15:28:08.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.327+0000 7fb4f18d2700 1 --2- 172.21.15.52:0/772781610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4ec1055d0 0x7fb4ec19ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:28:08.335 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.327+0000 7fb4f08d0700 1 --2- 172.21.15.52:0/772781610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4ec0ff490 0x7fb4ec1968f0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc00e940 tx=0x7fb4dc00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:08.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.327+0000 7fb4e27fc700 1 -- 172.21.15.52:0/772781610 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc009730 con 0x7fb4ec0ff490 2023-11-27T15:28:08.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.327+0000 7fb4e27fc700 1 -- 172.21.15.52:0/772781610 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4dc00cbd0 con 0x7fb4ec0ff490 2023-11-27T15:28:08.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.327+0000 7fb4f3335700 1 -- 172.21.15.52:0/772781610 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4ec1103d0 con 0x7fb4ec0ff490 2023-11-27T15:28:08.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.327+0000 7fb4e27fc700 1 -- 172.21.15.52:0/772781610 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc018680 con 0x7fb4ec0ff490 2023-11-27T15:28:08.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.327+0000 7fb4f3335700 1 -- 172.21.15.52:0/772781610 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4ec110240 con 0x7fb4ec0ff490 2023-11-27T15:28:08.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.328+0000 7fb4f3335700 1 -- 172.21.15.52:0/772781610 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4ec110560 con 0x7fb4ec0ff490 2023-11-27T15:28:08.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.329+0000 7fb4e27fc700 1 -- 172.21.15.52:0/772781610 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb4dc00cd40 con 0x7fb4ec0ff490 2023-11-27T15:28:08.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.330+0000 7fb4e27fc700 1 --2- 172.21.15.52:0/772781610 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb4d80a12c0 0x7fb4d80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:08.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.330+0000 7fb4e27fc700 1 -- 172.21.15.52:0/772781610 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fb4dc014070 con 0x7fb4ec0ff490 2023-11-27T15:28:08.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.330+0000 7fb4f10d1700 1 --2- 172.21.15.52:0/772781610 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb4d80a12c0 0x7fb4d80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:08.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.331+0000 7fb4f10d1700 1 --2- 172.21.15.52:0/772781610 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb4d80a12c0 0x7fb4d80a3770 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb4e800b580 tx=0x7fb4e8003680 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:08.338 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.334+0000 7fb4e27fc700 1 -- 172.21.15.52:0/772781610 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb4dc0965b0 con 0x7fb4ec0ff490 2023-11-27T15:28:08.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:08 smithi150 ceph-mon[100559]: pgmap v317: 97 pgs: 97 active+clean; 712 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2023-11-27T15:28:08.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:08 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/4265018433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:28:08.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:08 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/790861914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:28:08.539 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.536+0000 7fb4f3335700 1 -- 172.21.15.52:0/772781610 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb4ec1a3480 con 0x7fb4ec0ff490 2023-11-27T15:28:08.540 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.537+0000 7fb4e27fc700 1 -- 172.21.15.52:0/772781610 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb4dc026020 con 0x7fb4ec0ff490 2023-11-27T15:28:08.540 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2023-11-27T15:28:08.542 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.538+0000 7fb4f3335700 1 -- 172.21.15.52:0/772781610 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb4d80a12c0 msgr2=0x7fb4d80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:08.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.538+0000 7fb4f3335700 1 --2- 172.21.15.52:0/772781610 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb4d80a12c0 0x7fb4d80a3770 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb4e800b580 tx=0x7fb4e8003680 comp rx=0 tx=0).stop 2023-11-27T15:28:08.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.539+0000 7fb4f3335700 1 -- 172.21.15.52:0/772781610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4ec0ff490 msgr2=0x7fb4ec1968f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:08.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.539+0000 7fb4f3335700 1 --2- 172.21.15.52:0/772781610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4ec0ff490 0x7fb4ec1968f0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc00e940 tx=0x7fb4dc00ed00 comp rx=0 tx=0).stop 2023-11-27T15:28:08.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.539+0000 7fb4f3335700 1 -- 172.21.15.52:0/772781610 shutdown_connections 2023-11-27T15:28:08.543 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.539+0000 7fb4f3335700 1 --2- 172.21.15.52:0/772781610 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb4d80a12c0 0x7fb4d80a3770 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:08.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.539+0000 7fb4f3335700 1 --2- 172.21.15.52:0/772781610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4ec1055d0 0x7fb4ec19ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:08.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.539+0000 7fb4f3335700 1 --2- 172.21.15.52:0/772781610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb4ec0feaf0 0x7fb4ec1963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:08.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.539+0000 7fb4f3335700 1 --2- 172.21.15.52:0/772781610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4ec0ff490 0x7fb4ec1968f0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:08.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.539+0000 7fb4f3335700 1 -- 172.21.15.52:0/772781610 >> 172.21.15.52:0/772781610 conn(0x7fb4ec075920 msgr2=0x7fb4ec0fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:08.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.539+0000 7fb4f3335700 1 -- 172.21.15.52:0/772781610 shutdown_connections 2023-11-27T15:28:08.544 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.539+0000 7fb4f3335700 1 -- 172.21.15.52:0/772781610 wait complete. 2023-11-27T15:28:08.553 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:08 smithi052 ceph-mon[99056]: pgmap v317: 97 pgs: 97 active+clean; 712 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2023-11-27T15:28:08.553 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:08 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/4265018433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:28:08.554 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:08 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/790861914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:28:08.555 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-27T15:28:08.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:08 smithi102 ceph-mon[100296]: pgmap v317: 97 pgs: 97 active+clean; 712 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2023-11-27T15:28:08.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:08 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/4265018433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:28:08.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:08 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/790861914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:28:08.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.680+0000 7fe65322f700 1 -- 172.21.15.52:0/2975409101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe64c0ff450 msgr2=0x7fe64c0ff820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:08.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.680+0000 7fe65322f700 1 --2- 172.21.15.52:0/2975409101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe64c0ff450 0x7fe64c0ff820 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe6400099c0 tx=0x7fe640009cd0 comp rx=0 tx=0).stop 2023-11-27T15:28:08.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.680+0000 7fe65322f700 1 -- 172.21.15.52:0/2975409101 shutdown_connections 2023-11-27T15:28:08.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.680+0000 7fe65322f700 1 --2- 172.21.15.52:0/2975409101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe64c104490 0x7fe64c10c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:08.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.680+0000 7fe65322f700 1 --2- 172.21.15.52:0/2975409101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe64c0ff450 0x7fe64c0ff820 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:08.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.680+0000 7fe65322f700 1 --2- 172.21.15.52:0/2975409101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe64c0ffdf0 0x7fe64c103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:08.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.680+0000 7fe65322f700 1 -- 172.21.15.52:0/2975409101 >> 172.21.15.52:0/2975409101 conn(0x7fe64c0770b0 msgr2=0x7fe64c0774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:08.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.680+0000 7fe65322f700 1 -- 172.21.15.52:0/2975409101 shutdown_connections 2023-11-27T15:28:08.688 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.680+0000 7fe65322f700 1 -- 172.21.15.52:0/2975409101 wait complete. 2023-11-27T15:28:08.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.680+0000 7fe65322f700 1 Processor -- start 2023-11-27T15:28:08.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.680+0000 7fe65322f700 1 -- start start 2023-11-27T15:28:08.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.681+0000 7fe65322f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe64c0ff450 0x7fe64c100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:08.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.681+0000 7fe65322f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe64c0ffdf0 0x7fe64c101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:08.689 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.681+0000 7fe65322f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe64c104490 0x7fe64c1a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:08.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.681+0000 7fe65322f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64c101bc0 con 0x7fe64c0ff450 2023-11-27T15:28:08.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.681+0000 7fe65322f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64c1a4b20 con 0x7fe64c104490 2023-11-27T15:28:08.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.681+0000 7fe65322f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64c1a4c60 con 0x7fe64c0ffdf0 2023-11-27T15:28:08.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.681+0000 7fe650fcb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe64c0ff450 0x7fe64c100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:08.690 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.681+0000 7fe650fcb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe64c0ff450 0x7fe64c100e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56462/0 (socket says 172.21.15.52:56462) 2023-11-27T15:28:08.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.681+0000 7fe650fcb700 1 -- 172.21.15.52:0/2483185157 learned_addr learned my addr 172.21.15.52:0/2483185157 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:08.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.681+0000 7fe6517cc700 1 --2- 172.21.15.52:0/2483185157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe64c104490 0x7fe64c1a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:08.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.681+0000 7fe64bfff700 1 --2- 172.21.15.52:0/2483185157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe64c0ffdf0 0x7fe64c101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:08.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.681+0000 7fe650fcb700 1 -- 172.21.15.52:0/2483185157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe64c0ffdf0 msgr2=0x7fe64c101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:08.691 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.681+0000 7fe650fcb700 1 --2- 172.21.15.52:0/2483185157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe64c0ffdf0 0x7fe64c101390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:08.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.681+0000 7fe650fcb700 1 -- 172.21.15.52:0/2483185157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe64c104490 msgr2=0x7fe64c1a45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:08.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.681+0000 7fe650fcb700 1 --2- 172.21.15.52:0/2483185157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe64c104490 0x7fe64c1a45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:08.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.681+0000 7fe650fcb700 1 -- 172.21.15.52:0/2483185157 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6400095f0 con 0x7fe64c0ff450 2023-11-27T15:28:08.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.681+0000 7fe6517cc700 1 --2- 172.21.15.52:0/2483185157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe64c104490 0x7fe64c1a45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:28:08.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.681+0000 7fe650fcb700 1 --2- 172.21.15.52:0/2483185157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe64c0ff450 0x7fe64c100e50 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe64000b580 tx=0x7fe64000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:08.692 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.681+0000 7fe649ffb700 1 -- 172.21.15.52:0/2483185157 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64001d070 con 0x7fe64c0ff450 2023-11-27T15:28:08.693 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.681+0000 7fe65322f700 1 -- 172.21.15.52:0/2483185157 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe64c10f250 con 0x7fe64c0ff450 2023-11-27T15:28:08.693 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.681+0000 7fe649ffb700 1 -- 172.21.15.52:0/2483185157 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6400054f0 con 0x7fe64c0ff450 2023-11-27T15:28:08.693 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.681+0000 7fe649ffb700 1 -- 172.21.15.52:0/2483185157 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe640004bd0 con 0x7fe64c0ff450 2023-11-27T15:28:08.693 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.682+0000 7fe65322f700 1 -- 172.21.15.52:0/2483185157 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe64c10f0c0 con 0x7fe64c0ff450 2023-11-27T15:28:08.693 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.683+0000 7fe65322f700 1 -- 172.21.15.52:0/2483185157 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe64c10f3e0 con 0x7fe64c0ff450 2023-11-27T15:28:08.695 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.684+0000 7fe649ffb700 1 -- 172.21.15.52:0/2483185157 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe64000fc30 con 0x7fe64c0ff450 2023-11-27T15:28:08.695 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.684+0000 7fe649ffb700 1 --2- 172.21.15.52:0/2483185157 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe63c0a12c0 0x7fe63c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:08.695 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.684+0000 7fe649ffb700 1 -- 172.21.15.52:0/2483185157 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fe6400cdbc0 con 0x7fe64c0ff450 2023-11-27T15:28:08.695 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.684+0000 7fe64bfff700 1 --2- 172.21.15.52:0/2483185157 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe63c0a12c0 0x7fe63c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:08.695 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.685+0000 7fe64bfff700 1 --2- 172.21.15.52:0/2483185157 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe63c0a12c0 0x7fe63c0a3770 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe638005950 tx=0x7fe6380058e0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:08.696 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.688+0000 7fe649ffb700 1 -- 172.21.15.52:0/2483185157 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe640097030 con 0x7fe64c0ff450 2023-11-27T15:28:08.892 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.888+0000 7fe65322f700 1 -- 172.21.15.52:0/2483185157 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe64c1a5520 con 0x7fe64c0ff450 2023-11-27T15:28:08.892 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.888+0000 7fe649ffb700 1 -- 172.21.15.52:0/2483185157 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe64009a650 con 0x7fe64c0ff450 2023-11-27T15:28:08.892 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2023-11-27T15:28:08.894 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.891+0000 7fe65322f700 1 -- 172.21.15.52:0/2483185157 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe63c0a12c0 msgr2=0x7fe63c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:08.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.891+0000 7fe65322f700 1 --2- 172.21.15.52:0/2483185157 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe63c0a12c0 0x7fe63c0a3770 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe638005950 tx=0x7fe6380058e0 comp rx=0 tx=0).stop 2023-11-27T15:28:08.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.891+0000 7fe65322f700 1 -- 172.21.15.52:0/2483185157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe64c0ff450 msgr2=0x7fe64c100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:08.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.891+0000 7fe65322f700 1 --2- 172.21.15.52:0/2483185157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe64c0ff450 0x7fe64c100e50 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe64000b580 tx=0x7fe64000f740 comp rx=0 tx=0).stop 2023-11-27T15:28:08.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.891+0000 7fe65322f700 1 -- 172.21.15.52:0/2483185157 shutdown_connections 2023-11-27T15:28:08.895 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.891+0000 7fe65322f700 1 --2- 172.21.15.52:0/2483185157 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fe63c0a12c0 0x7fe63c0a3770 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:08.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.891+0000 7fe65322f700 1 --2- 172.21.15.52:0/2483185157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe64c104490 0x7fe64c1a45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:08.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.891+0000 7fe65322f700 1 --2- 172.21.15.52:0/2483185157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe64c0ff450 0x7fe64c100e50 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:08.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.891+0000 7fe65322f700 1 --2- 172.21.15.52:0/2483185157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe64c0ffdf0 0x7fe64c101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:08.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.891+0000 7fe65322f700 1 -- 172.21.15.52:0/2483185157 >> 172.21.15.52:0/2483185157 conn(0x7fe64c0770b0 msgr2=0x7fe64c103030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:08.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.891+0000 7fe65322f700 1 -- 172.21.15.52:0/2483185157 shutdown_connections 2023-11-27T15:28:08.896 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:08.891+0000 7fe65322f700 1 -- 172.21.15.52:0/2483185157 wait complete. 2023-11-27T15:28:08.906 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-11-27T15:28:09.036 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.030+0000 7f62dbc1f700 1 -- 172.21.15.52:0/3492910974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62d40ff3a0 msgr2=0x7f62d40ff770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:09.036 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.030+0000 7f62dbc1f700 1 --2- 172.21.15.52:0/3492910974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62d40ff3a0 0x7f62d40ff770 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f62d00099c0 tx=0x7f62d0009cd0 comp rx=0 tx=0).stop 2023-11-27T15:28:09.036 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.030+0000 7f62dbc1f700 1 -- 172.21.15.52:0/3492910974 shutdown_connections 2023-11-27T15:28:09.036 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.030+0000 7f62dbc1f700 1 --2- 172.21.15.52:0/3492910974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62d4104390 0x7f62d410bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:09.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.030+0000 7f62dbc1f700 1 --2- 172.21.15.52:0/3492910974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62d40ff3a0 0x7f62d40ff770 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:09.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.030+0000 7f62dbc1f700 1 --2- 172.21.15.52:0/3492910974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62d40ffd40 0x7f62d4103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:09.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.030+0000 7f62dbc1f700 1 -- 172.21.15.52:0/3492910974 >> 172.21.15.52:0/3492910974 conn(0x7f62d4076230 msgr2=0x7f62d4076630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:09.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.030+0000 7f62dbc1f700 1 -- 172.21.15.52:0/3492910974 shutdown_connections 2023-11-27T15:28:09.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.030+0000 7f62dbc1f700 1 -- 172.21.15.52:0/3492910974 wait complete. 2023-11-27T15:28:09.037 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.030+0000 7f62dbc1f700 1 Processor -- start 2023-11-27T15:28:09.038 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.031+0000 7f62dbc1f700 1 -- start start 2023-11-27T15:28:09.038 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.031+0000 7f62dbc1f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62d40ff3a0 0x7f62d4100d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:09.038 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.031+0000 7f62dbc1f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62d40ffd40 0x7f62d4101290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:09.038 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.031+0000 7f62dbc1f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62d4104390 0x7f62d41a4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:09.038 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.031+0000 7f62dbc1f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d4101ac0 con 0x7f62d4104390 2023-11-27T15:28:09.038 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.031+0000 7f62dbc1f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d41a4a70 con 0x7f62d40ffd40 2023-11-27T15:28:09.039 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.031+0000 7f62dbc1f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d41a4bb0 con 0x7f62d40ff3a0 2023-11-27T15:28:09.039 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.031+0000 7f62da1bc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62d4104390 0x7f62d41a4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:09.039 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.031+0000 7f62da1bc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62d4104390 0x7f62d41a4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56480/0 (socket says 172.21.15.52:56480) 2023-11-27T15:28:09.039 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.031+0000 7f62da1bc700 1 -- 172.21.15.52:0/4042619355 learned_addr learned my addr 172.21.15.52:0/4042619355 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:09.039 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.031+0000 7f62d99bb700 1 --2- 172.21.15.52:0/4042619355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62d40ff3a0 0x7f62d4100d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:09.039 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.031+0000 7f62d91ba700 1 --2- 172.21.15.52:0/4042619355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62d40ffd40 0x7f62d4101290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:09.039 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.031+0000 7f62da1bc700 1 -- 172.21.15.52:0/4042619355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62d40ff3a0 msgr2=0x7f62d4100d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:09.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.031+0000 7f62da1bc700 1 --2- 172.21.15.52:0/4042619355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62d40ff3a0 0x7f62d4100d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:09.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.031+0000 7f62da1bc700 1 -- 172.21.15.52:0/4042619355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62d40ffd40 msgr2=0x7f62d4101290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:09.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.031+0000 7f62da1bc700 1 --2- 172.21.15.52:0/4042619355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62d40ffd40 0x7f62d4101290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:09.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.031+0000 7f62da1bc700 1 -- 172.21.15.52:0/4042619355 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d00095f0 con 0x7f62d4104390 2023-11-27T15:28:09.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.031+0000 7f62da1bc700 1 --2- 172.21.15.52:0/4042619355 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62d4104390 0x7f62d41a4530 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00ab50 tx=0x7f62cc00af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:09.040 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.031+0000 7f62c6ffd700 1 -- 172.21.15.52:0/4042619355 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc00f070 con 0x7f62d4104390 2023-11-27T15:28:09.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.032+0000 7f62c6ffd700 1 -- 172.21.15.52:0/4042619355 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62cc007a80 con 0x7f62d4104390 2023-11-27T15:28:09.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.032+0000 7f62c6ffd700 1 -- 172.21.15.52:0/4042619355 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc005310 con 0x7f62d4104390 2023-11-27T15:28:09.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.032+0000 7f62dbc1f700 1 -- 172.21.15.52:0/4042619355 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d410f150 con 0x7f62d4104390 2023-11-27T15:28:09.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.032+0000 7f62dbc1f700 1 -- 172.21.15.52:0/4042619355 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d410efc0 con 0x7f62d4104390 2023-11-27T15:28:09.041 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.032+0000 7f62dbc1f700 1 -- 172.21.15.52:0/4042619355 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d410f2e0 con 0x7f62d4104390 2023-11-27T15:28:09.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.034+0000 7f62c6ffd700 1 -- 172.21.15.52:0/4042619355 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f62cc004ad0 con 0x7f62d4104390 2023-11-27T15:28:09.042 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.034+0000 7f62c6ffd700 1 --2- 172.21.15.52:0/4042619355 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f62c00a1340 0x7f62c00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:09.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.034+0000 7f62d99bb700 1 --2- 172.21.15.52:0/4042619355 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f62c00a1340 0x7f62c00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:09.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.034+0000 7f62c6ffd700 1 -- 172.21.15.52:0/4042619355 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f62cc0cc7e0 con 0x7f62d4104390 2023-11-27T15:28:09.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.035+0000 7f62d99bb700 1 --2- 172.21.15.52:0/4042619355 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f62c00a1340 0x7f62c00a37f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f62d0000c00 tx=0x7f62d0005fb0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:09.043 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.038+0000 7f62c6ffd700 1 -- 172.21.15.52:0/4042619355 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f62cc095c50 con 0x7f62d4104390 2023-11-27T15:28:09.297 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.294+0000 7f62dbc1f700 1 -- 172.21.15.52:0/4042619355 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f62d4067160 con 0x7f62d4104390 2023-11-27T15:28:09.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.302+0000 7f62c6ffd700 1 -- 172.21.15.52:0/4042619355 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) v1 ==== 81+0+733775 (secure 0 0 0) 0x7f62cc099270 con 0x7f62d4104390 2023-11-27T15:28:09.315 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:28:09.315 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:28:09.316 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 26, 2023-11-27T15:28:09.316 INFO:teuthology.orchestra.run.smithi052.stdout: "active_gid": 15387, 2023-11-27T15:28:09.316 INFO:teuthology.orchestra.run.smithi052.stdout: "active_name": "x", 2023-11-27T15:28:09.316 INFO:teuthology.orchestra.run.smithi052.stdout: "active_addrs": { 2023-11-27T15:28:09.316 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2023-11-27T15:28:09.316 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.317 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2023-11-27T15:28:09.317 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:6800", 2023-11-27T15:28:09.317 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 1148545855 2023-11-27T15:28:09.317 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.317 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.317 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v1", 2023-11-27T15:28:09.318 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:6801", 2023-11-27T15:28:09.318 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 1148545855 2023-11-27T15:28:09.318 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.318 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2023-11-27T15:28:09.318 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.318 INFO:teuthology.orchestra.run.smithi052.stdout: "active_addr": "172.21.15.52:6801/1148545855", 2023-11-27T15:28:09.319 INFO:teuthology.orchestra.run.smithi052.stdout: "active_change": "2023-11-27T15:17:54.913761+0000", 2023-11-27T15:28:09.319 INFO:teuthology.orchestra.run.smithi052.stdout: "active_mgr_features": 4540138322906710015, 2023-11-27T15:28:09.319 INFO:teuthology.orchestra.run.smithi052.stdout: "available": true, 2023-11-27T15:28:09.319 INFO:teuthology.orchestra.run.smithi052.stdout: "standbys": [ 2023-11-27T15:28:09.319 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.319 INFO:teuthology.orchestra.run.smithi052.stdout: "gid": 15378, 2023-11-27T15:28:09.319 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "y", 2023-11-27T15:28:09.320 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr_features": 4540138322906710015, 2023-11-27T15:28:09.320 INFO:teuthology.orchestra.run.smithi052.stdout: "available_modules": [ 2023-11-27T15:28:09.320 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.320 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "alerts", 2023-11-27T15:28:09.320 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:09.320 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:09.320 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:09.321 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2023-11-27T15:28:09.321 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2023-11-27T15:28:09.321 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:09.321 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.321 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.321 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2023-11-27T15:28:09.322 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.322 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.322 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.322 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How frequently to reexamine health status", 2023-11-27T15:28:09.322 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.322 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.323 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.323 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.323 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:09.323 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:09.323 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.324 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.324 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.324 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.324 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.324 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.324 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.325 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.325 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.325 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.325 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.325 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.326 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.326 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.326 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.326 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.326 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.327 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.327 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.327 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:09.327 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:09.327 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.328 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.328 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.328 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.328 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.328 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.328 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.329 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.329 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.329 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.329 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.329 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.329 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:09.330 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:09.330 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.330 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.330 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.330 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:09.330 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.331 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.331 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.331 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.331 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.331 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.331 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.332 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.332 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.332 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.332 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.332 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.332 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.333 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.333 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.333 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:09.333 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:09.333 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.333 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.333 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.334 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.334 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.334 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.334 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.334 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.334 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.335 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.335 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.335 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.335 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_destination": { 2023-11-27T15:28:09.335 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_destination", 2023-11-27T15:28:09.335 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.336 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.336 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.336 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.336 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.336 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.336 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.337 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Email address to send alerts to", 2023-11-27T15:28:09.337 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.337 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.337 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.337 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.337 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_from_name": { 2023-11-27T15:28:09.337 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_from_name", 2023-11-27T15:28:09.338 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.338 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.338 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.338 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "Ceph", 2023-11-27T15:28:09.338 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.338 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.339 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.339 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Email From: name", 2023-11-27T15:28:09.339 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.339 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.339 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.339 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.340 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_host": { 2023-11-27T15:28:09.340 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_host", 2023-11-27T15:28:09.340 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.340 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.340 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.340 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.340 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.341 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.341 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.341 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP server", 2023-11-27T15:28:09.341 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.341 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.341 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.341 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.342 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_password": { 2023-11-27T15:28:09.342 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_password", 2023-11-27T15:28:09.342 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.342 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.342 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.342 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.343 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.343 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.343 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.343 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Password to authenticate with", 2023-11-27T15:28:09.343 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.343 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.343 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.344 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.344 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_port": { 2023-11-27T15:28:09.344 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_port", 2023-11-27T15:28:09.344 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.344 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.344 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.345 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "465", 2023-11-27T15:28:09.345 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.345 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.345 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.345 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP port", 2023-11-27T15:28:09.345 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.345 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.346 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.346 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.346 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_sender": { 2023-11-27T15:28:09.346 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_sender", 2023-11-27T15:28:09.346 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.346 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.347 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.347 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.347 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.347 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.347 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.347 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP envelope sender", 2023-11-27T15:28:09.347 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.348 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.348 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.348 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.348 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_ssl": { 2023-11-27T15:28:09.348 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_ssl", 2023-11-27T15:28:09.349 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.349 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.349 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.349 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.349 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.349 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.349 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.350 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-27T15:28:09.350 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.350 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.350 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.350 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.350 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_user": { 2023-11-27T15:28:09.350 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_user", 2023-11-27T15:28:09.351 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.351 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.351 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.351 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.351 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.351 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.351 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.352 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "User to authenticate as", 2023-11-27T15:28:09.352 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.352 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.352 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.352 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.352 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.352 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.353 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.353 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "balancer", 2023-11-27T15:28:09.353 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:09.353 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:09.353 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:09.353 INFO:teuthology.orchestra.run.smithi052.stdout: "active": { 2023-11-27T15:28:09.353 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "active", 2023-11-27T15:28:09.354 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.354 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.354 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.354 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.354 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.354 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.355 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.355 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "automatically balance PGs across cluster", 2023-11-27T15:28:09.355 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.355 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.355 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.355 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.355 INFO:teuthology.orchestra.run.smithi052.stdout: "begin_time": { 2023-11-27T15:28:09.356 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "begin_time", 2023-11-27T15:28:09.356 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.356 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.356 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.356 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0000", 2023-11-27T15:28:09.356 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.356 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.357 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.357 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "beginning time of day to automatically balance", 2023-11-27T15:28:09.357 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-27T15:28:09.357 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.357 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.357 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.357 INFO:teuthology.orchestra.run.smithi052.stdout: "begin_weekday": { 2023-11-27T15:28:09.358 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "begin_weekday", 2023-11-27T15:28:09.358 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2023-11-27T15:28:09.358 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.358 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.358 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2023-11-27T15:28:09.358 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0", 2023-11-27T15:28:09.359 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "6", 2023-11-27T15:28:09.359 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.359 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-27T15:28:09.359 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-27T15:28:09.359 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.359 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.359 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.360 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_max_iterations": { 2023-11-27T15:28:09.360 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_max_iterations", 2023-11-27T15:28:09.360 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2023-11-27T15:28:09.360 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.360 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.360 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "25", 2023-11-27T15:28:09.360 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2023-11-27T15:28:09.360 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "250", 2023-11-27T15:28:09.361 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.361 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-27T15:28:09.361 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.361 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.361 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.361 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.361 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_metrics": { 2023-11-27T15:28:09.362 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_metrics", 2023-11-27T15:28:09.362 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.362 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.362 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.362 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "pgs,objects,bytes", 2023-11-27T15:28:09.362 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.363 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.363 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.363 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-27T15:28:09.363 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-27T15:28:09.363 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.363 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.363 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.364 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_step": { 2023-11-27T15:28:09.364 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_step", 2023-11-27T15:28:09.364 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2023-11-27T15:28:09.364 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.364 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.364 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0.5", 2023-11-27T15:28:09.364 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0.001", 2023-11-27T15:28:09.365 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "0.999", 2023-11-27T15:28:09.365 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.365 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "aggressiveness of optimization", 2023-11-27T15:28:09.365 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-27T15:28:09.365 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.365 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.365 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.366 INFO:teuthology.orchestra.run.smithi052.stdout: "end_time": { 2023-11-27T15:28:09.366 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "end_time", 2023-11-27T15:28:09.366 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.366 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.366 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.366 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "2359", 2023-11-27T15:28:09.367 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.367 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.367 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.367 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "ending time of day to automatically balance", 2023-11-27T15:28:09.367 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-27T15:28:09.367 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.367 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.368 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.368 INFO:teuthology.orchestra.run.smithi052.stdout: "end_weekday": { 2023-11-27T15:28:09.368 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "end_weekday", 2023-11-27T15:28:09.368 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2023-11-27T15:28:09.368 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.368 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.368 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2023-11-27T15:28:09.369 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0", 2023-11-27T15:28:09.369 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "6", 2023-11-27T15:28:09.369 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.369 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-27T15:28:09.369 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-27T15:28:09.369 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.369 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.369 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.370 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:09.370 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:09.370 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.370 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.370 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.370 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.371 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.371 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.371 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.371 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.371 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.371 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.371 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.371 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.372 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.372 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.372 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.372 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.372 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.372 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.372 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.373 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:09.373 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:09.373 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.373 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.373 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.373 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.374 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.374 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.374 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.374 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.374 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.374 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.374 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.375 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.375 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:09.375 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:09.375 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.375 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.375 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.375 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:09.375 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.376 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.376 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.376 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.376 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.376 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.376 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.376 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.376 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.377 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.377 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.377 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.377 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.377 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.377 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.378 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:09.378 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:09.378 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.378 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.378 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.378 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.378 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.379 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.379 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.379 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.379 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.379 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.379 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.379 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.380 INFO:teuthology.orchestra.run.smithi052.stdout: "min_score": { 2023-11-27T15:28:09.380 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "min_score", 2023-11-27T15:28:09.380 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2023-11-27T15:28:09.380 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.380 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.380 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2023-11-27T15:28:09.380 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.380 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.381 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.381 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-27T15:28:09.381 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.381 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.381 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.381 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.381 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": { 2023-11-27T15:28:09.381 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mode", 2023-11-27T15:28:09.382 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.382 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.382 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.382 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "upmap", 2023-11-27T15:28:09.382 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.382 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.383 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.383 INFO:teuthology.orchestra.run.smithi052.stdout: "crush-compat", 2023-11-27T15:28:09.383 INFO:teuthology.orchestra.run.smithi052.stdout: "none", 2023-11-27T15:28:09.383 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap" 2023-11-27T15:28:09.383 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.383 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Balancer mode", 2023-11-27T15:28:09.383 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.384 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.384 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.384 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.384 INFO:teuthology.orchestra.run.smithi052.stdout: "pool_ids": { 2023-11-27T15:28:09.384 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pool_ids", 2023-11-27T15:28:09.384 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.384 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.384 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.385 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.385 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.385 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.385 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.385 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-27T15:28:09.385 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.385 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.386 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.386 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.386 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2023-11-27T15:28:09.386 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2023-11-27T15:28:09.386 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:09.386 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.386 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.387 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2023-11-27T15:28:09.387 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.387 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.387 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.387 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-27T15:28:09.387 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.387 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.388 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.388 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.388 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap_max_deviation": { 2023-11-27T15:28:09.388 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "upmap_max_deviation", 2023-11-27T15:28:09.388 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.388 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.388 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.388 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2023-11-27T15:28:09.389 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2023-11-27T15:28:09.389 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.389 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.389 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-27T15:28:09.389 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-27T15:28:09.389 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.389 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.390 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.390 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap_max_optimizations": { 2023-11-27T15:28:09.390 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "upmap_max_optimizations", 2023-11-27T15:28:09.390 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2023-11-27T15:28:09.390 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.390 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.390 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2023-11-27T15:28:09.391 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.391 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.391 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.391 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-27T15:28:09.391 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.391 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.391 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.391 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.392 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.392 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.392 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.392 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cephadm", 2023-11-27T15:28:09.392 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:09.392 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:09.392 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:09.393 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_down_multiplier": { 2023-11-27T15:28:09.393 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_down_multiplier", 2023-11-27T15:28:09.393 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2023-11-27T15:28:09.393 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.393 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.393 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3.0", 2023-11-27T15:28:09.393 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.394 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.394 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.394 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-27T15:28:09.394 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.394 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.394 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.394 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.395 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_refresh_rate": { 2023-11-27T15:28:09.395 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_refresh_rate", 2023-11-27T15:28:09.395 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:09.395 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.395 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.395 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "20", 2023-11-27T15:28:09.395 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.395 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.396 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.396 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-27T15:28:09.396 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.396 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.396 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.396 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.396 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_starting_port": { 2023-11-27T15:28:09.396 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_starting_port", 2023-11-27T15:28:09.397 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.397 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.397 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.397 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "4721", 2023-11-27T15:28:09.397 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.397 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.397 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.398 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-27T15:28:09.398 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.398 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.398 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.398 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.398 INFO:teuthology.orchestra.run.smithi052.stdout: "alertmanager_web_password": { 2023-11-27T15:28:09.399 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "alertmanager_web_password", 2023-11-27T15:28:09.399 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.399 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.399 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.399 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2023-11-27T15:28:09.399 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.399 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.399 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.399 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Alertmanager web password", 2023-11-27T15:28:09.400 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.400 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.400 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.400 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.400 INFO:teuthology.orchestra.run.smithi052.stdout: "alertmanager_web_user": { 2023-11-27T15:28:09.400 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "alertmanager_web_user", 2023-11-27T15:28:09.400 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.400 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.401 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.401 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2023-11-27T15:28:09.401 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.401 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.401 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.401 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Alertmanager web user", 2023-11-27T15:28:09.401 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.401 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.402 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.402 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.402 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_ptrace": { 2023-11-27T15:28:09.402 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_ptrace", 2023-11-27T15:28:09.402 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.402 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.402 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.403 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.403 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.403 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.403 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.403 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-27T15:28:09.403 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-27T15:28:09.403 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.403 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.404 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.404 INFO:teuthology.orchestra.run.smithi052.stdout: "autotune_interval": { 2023-11-27T15:28:09.404 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "autotune_interval", 2023-11-27T15:28:09.404 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:09.404 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.404 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.404 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2023-11-27T15:28:09.404 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.405 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.405 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.405 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-27T15:28:09.405 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.405 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.405 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.405 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.405 INFO:teuthology.orchestra.run.smithi052.stdout: "autotune_memory_target_ratio": { 2023-11-27T15:28:09.406 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "autotune_memory_target_ratio", 2023-11-27T15:28:09.406 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2023-11-27T15:28:09.406 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.406 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.406 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0.7", 2023-11-27T15:28:09.406 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.406 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.407 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.407 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-27T15:28:09.407 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.407 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.407 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.407 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.407 INFO:teuthology.orchestra.run.smithi052.stdout: "cgroups_split": { 2023-11-27T15:28:09.407 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cgroups_split", 2023-11-27T15:28:09.408 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.408 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.408 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.408 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.408 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.408 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.408 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.409 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-27T15:28:09.409 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.409 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.409 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.409 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.409 INFO:teuthology.orchestra.run.smithi052.stdout: "config_checks_enabled": { 2023-11-27T15:28:09.409 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_checks_enabled", 2023-11-27T15:28:09.409 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.410 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.410 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.410 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.410 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.410 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.410 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.410 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-27T15:28:09.411 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.411 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.411 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.411 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.411 INFO:teuthology.orchestra.run.smithi052.stdout: "config_dashboard": { 2023-11-27T15:28:09.411 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_dashboard", 2023-11-27T15:28:09.411 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.411 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.412 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.412 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.412 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.412 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.412 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.412 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-27T15:28:09.412 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.412 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.413 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.413 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.413 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_alertmanager": { 2023-11-27T15:28:09.413 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_alertmanager", 2023-11-27T15:28:09.413 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.413 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.413 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.413 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-27T15:28:09.414 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.414 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.414 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.414 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2023-11-27T15:28:09.414 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.414 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.414 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.415 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.415 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_base": { 2023-11-27T15:28:09.415 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_base", 2023-11-27T15:28:09.415 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.415 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.415 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.415 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-27T15:28:09.415 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.416 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.416 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.416 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Container image name, without the tag", 2023-11-27T15:28:09.416 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.416 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.416 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.416 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.416 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_elasticsearch": { 2023-11-27T15:28:09.417 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_elasticsearch", 2023-11-27T15:28:09.417 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.417 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.417 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.417 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-27T15:28:09.417 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.417 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.417 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.418 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "elasticsearch container image", 2023-11-27T15:28:09.418 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.418 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.418 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.418 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.418 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_grafana": { 2023-11-27T15:28:09.418 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_grafana", 2023-11-27T15:28:09.419 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.419 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.419 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.419 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-27T15:28:09.419 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.419 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.419 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.419 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2023-11-27T15:28:09.419 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.420 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.420 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.420 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.420 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_haproxy": { 2023-11-27T15:28:09.420 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_haproxy", 2023-11-27T15:28:09.420 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.420 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.420 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.421 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-27T15:28:09.421 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.421 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.421 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.421 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "HAproxy container image", 2023-11-27T15:28:09.421 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.421 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.421 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.422 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.422 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_agent": { 2023-11-27T15:28:09.422 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_agent", 2023-11-27T15:28:09.422 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.422 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.422 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.422 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-27T15:28:09.423 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.423 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.423 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.423 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger agent container image", 2023-11-27T15:28:09.423 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.423 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.423 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.424 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.424 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_collector": { 2023-11-27T15:28:09.424 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_collector", 2023-11-27T15:28:09.424 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.424 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.424 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.424 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-27T15:28:09.424 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.425 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.425 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.425 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger collector container image", 2023-11-27T15:28:09.425 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.425 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.425 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.425 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.426 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_query": { 2023-11-27T15:28:09.426 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_query", 2023-11-27T15:28:09.426 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.426 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.426 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.426 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-27T15:28:09.426 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.427 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.427 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.427 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger query container image", 2023-11-27T15:28:09.427 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.427 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.427 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.427 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.428 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_keepalived": { 2023-11-27T15:28:09.428 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_keepalived", 2023-11-27T15:28:09.428 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.428 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.428 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.428 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-27T15:28:09.428 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.428 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.429 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.429 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Keepalived container image", 2023-11-27T15:28:09.429 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.429 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.429 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.429 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.429 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_loki": { 2023-11-27T15:28:09.429 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_loki", 2023-11-27T15:28:09.430 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.430 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.430 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.430 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-27T15:28:09.430 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.430 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.430 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.430 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Loki container image", 2023-11-27T15:28:09.431 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.431 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.431 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.431 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.431 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_node_exporter": { 2023-11-27T15:28:09.431 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_node_exporter", 2023-11-27T15:28:09.431 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.432 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.432 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.432 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-27T15:28:09.432 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.432 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.432 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.432 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2023-11-27T15:28:09.432 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.433 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.433 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.433 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.433 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_prometheus": { 2023-11-27T15:28:09.433 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_prometheus", 2023-11-27T15:28:09.433 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.434 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.434 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.434 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-27T15:28:09.434 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.434 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.434 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.434 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2023-11-27T15:28:09.435 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.435 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.435 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.435 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.435 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_promtail": { 2023-11-27T15:28:09.435 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_promtail", 2023-11-27T15:28:09.435 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.435 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.436 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.436 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-27T15:28:09.436 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.436 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.436 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.436 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Promtail container image", 2023-11-27T15:28:09.436 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.437 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.437 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.437 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.437 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_snmp_gateway": { 2023-11-27T15:28:09.437 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_snmp_gateway", 2023-11-27T15:28:09.437 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.437 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.437 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.438 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-27T15:28:09.438 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.438 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.438 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.438 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SNMP Gateway container image", 2023-11-27T15:28:09.438 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.438 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.439 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.439 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.439 INFO:teuthology.orchestra.run.smithi052.stdout: "container_init": { 2023-11-27T15:28:09.439 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_init", 2023-11-27T15:28:09.439 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.439 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.439 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.439 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.440 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.440 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.440 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.440 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Run podman/docker with `--init`", 2023-11-27T15:28:09.440 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.440 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.440 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.440 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.440 INFO:teuthology.orchestra.run.smithi052.stdout: "daemon_cache_timeout": { 2023-11-27T15:28:09.441 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "daemon_cache_timeout", 2023-11-27T15:28:09.441 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:09.441 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.441 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.441 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2023-11-27T15:28:09.441 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.441 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.441 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.442 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-27T15:28:09.442 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.442 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.442 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.442 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.442 INFO:teuthology.orchestra.run.smithi052.stdout: "default_cephadm_command_timeout": { 2023-11-27T15:28:09.442 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "default_cephadm_command_timeout", 2023-11-27T15:28:09.443 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:09.443 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.443 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.443 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "900", 2023-11-27T15:28:09.443 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.443 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.443 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.443 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-27T15:28:09.444 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.444 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.444 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.444 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.444 INFO:teuthology.orchestra.run.smithi052.stdout: "default_registry": { 2023-11-27T15:28:09.444 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "default_registry", 2023-11-27T15:28:09.444 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.444 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.445 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.445 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io", 2023-11-27T15:28:09.445 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.445 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.445 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.445 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-27T15:28:09.445 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.445 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.446 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.446 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.446 INFO:teuthology.orchestra.run.smithi052.stdout: "device_cache_timeout": { 2023-11-27T15:28:09.446 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_cache_timeout", 2023-11-27T15:28:09.446 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:09.446 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.446 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.447 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1800", 2023-11-27T15:28:09.447 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.447 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.447 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.447 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache device inventory", 2023-11-27T15:28:09.447 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.447 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.447 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.448 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.448 INFO:teuthology.orchestra.run.smithi052.stdout: "device_enhanced_scan": { 2023-11-27T15:28:09.448 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_enhanced_scan", 2023-11-27T15:28:09.448 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.448 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.448 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.448 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.448 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.449 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.449 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.449 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-27T15:28:09.449 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.449 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.449 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.449 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.449 INFO:teuthology.orchestra.run.smithi052.stdout: "facts_cache_timeout": { 2023-11-27T15:28:09.450 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "facts_cache_timeout", 2023-11-27T15:28:09.450 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:09.450 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.450 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.450 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2023-11-27T15:28:09.450 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.450 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.450 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.451 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache host facts data", 2023-11-27T15:28:09.451 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.451 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.451 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.451 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.451 INFO:teuthology.orchestra.run.smithi052.stdout: "host_check_interval": { 2023-11-27T15:28:09.451 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "host_check_interval", 2023-11-27T15:28:09.451 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:09.452 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.452 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.452 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2023-11-27T15:28:09.452 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.452 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.452 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.452 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to perform a host check", 2023-11-27T15:28:09.452 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.453 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.453 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.453 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.453 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:09.453 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:09.453 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.453 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.453 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.454 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.454 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.454 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.454 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.454 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.454 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.454 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.454 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.455 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.455 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.455 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.455 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.455 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.455 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.455 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.455 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.456 INFO:teuthology.orchestra.run.smithi052.stdout: "log_refresh_metadata": { 2023-11-27T15:28:09.456 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_refresh_metadata", 2023-11-27T15:28:09.456 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.456 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.456 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.456 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.456 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.456 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.457 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.457 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-27T15:28:09.457 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.457 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.457 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.457 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.457 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:09.458 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:09.458 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.458 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.458 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.458 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.458 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.458 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.458 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.459 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-27T15:28:09.459 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.459 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.459 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.459 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.459 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:09.459 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:09.460 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.460 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.460 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.460 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:09.460 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.460 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.460 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.460 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.460 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.461 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.461 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.461 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.461 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.461 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.461 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.461 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.461 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.462 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.462 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.462 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:09.462 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:09.462 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.462 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.463 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.463 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.463 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.463 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.463 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.463 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.463 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.463 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.464 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.464 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.464 INFO:teuthology.orchestra.run.smithi052.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-27T15:28:09.464 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-27T15:28:09.464 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.464 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.464 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.464 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.465 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.465 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.465 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.465 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-27T15:28:09.465 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.465 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.465 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.466 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.466 INFO:teuthology.orchestra.run.smithi052.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-27T15:28:09.466 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-27T15:28:09.466 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.466 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.466 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.466 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "*", 2023-11-27T15:28:09.467 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.467 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.467 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.467 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-27T15:28:09.467 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.467 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.467 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.467 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.468 INFO:teuthology.orchestra.run.smithi052.stdout: "max_count_per_host": { 2023-11-27T15:28:09.468 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_count_per_host", 2023-11-27T15:28:09.468 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.468 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.468 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.468 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2023-11-27T15:28:09.468 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.468 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.469 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.469 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "max number of daemons per service per host", 2023-11-27T15:28:09.469 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.469 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.469 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.469 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.469 INFO:teuthology.orchestra.run.smithi052.stdout: "max_osd_draining_count": { 2023-11-27T15:28:09.469 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_osd_draining_count", 2023-11-27T15:28:09.470 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.470 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.470 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.470 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2023-11-27T15:28:09.470 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.470 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.470 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.471 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-27T15:28:09.471 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.471 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.471 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.471 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.471 INFO:teuthology.orchestra.run.smithi052.stdout: "migration_current": { 2023-11-27T15:28:09.471 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "migration_current", 2023-11-27T15:28:09.471 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.471 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.472 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.472 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.472 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.472 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.472 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.472 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "internal - do not modify", 2023-11-27T15:28:09.472 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.472 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.473 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.473 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.473 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": { 2023-11-27T15:28:09.473 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mode", 2023-11-27T15:28:09.473 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.473 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.473 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.474 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "root", 2023-11-27T15:28:09.474 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.474 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.474 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.474 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm-package", 2023-11-27T15:28:09.474 INFO:teuthology.orchestra.run.smithi052.stdout: "root" 2023-11-27T15:28:09.474 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.474 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "mode for remote execution of cephadm", 2023-11-27T15:28:09.475 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.475 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.475 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.475 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.475 INFO:teuthology.orchestra.run.smithi052.stdout: "prometheus_alerts_path": { 2023-11-27T15:28:09.475 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus_alerts_path", 2023-11-27T15:28:09.475 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.475 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.476 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.476 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-27T15:28:09.476 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.476 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.476 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.476 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-27T15:28:09.476 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.476 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.477 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.477 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.477 INFO:teuthology.orchestra.run.smithi052.stdout: "prometheus_web_password": { 2023-11-27T15:28:09.477 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus_web_password", 2023-11-27T15:28:09.477 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.477 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.477 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.477 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2023-11-27T15:28:09.478 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.478 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.478 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.478 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus web password", 2023-11-27T15:28:09.478 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.478 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.478 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.479 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.479 INFO:teuthology.orchestra.run.smithi052.stdout: "prometheus_web_user": { 2023-11-27T15:28:09.479 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus_web_user", 2023-11-27T15:28:09.479 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.479 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.479 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.479 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2023-11-27T15:28:09.479 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.479 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.480 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.480 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus web user", 2023-11-27T15:28:09.480 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.480 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.480 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.480 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.480 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_insecure": { 2023-11-27T15:28:09.480 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_insecure", 2023-11-27T15:28:09.481 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.481 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.481 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.481 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.481 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.481 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.481 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.481 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-27T15:28:09.482 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.482 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.482 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.482 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.482 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_password": { 2023-11-27T15:28:09.482 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_password", 2023-11-27T15:28:09.482 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.482 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.483 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.483 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.483 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.483 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.483 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.483 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-27T15:28:09.483 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.484 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.484 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.484 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.484 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_url": { 2023-11-27T15:28:09.484 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_url", 2023-11-27T15:28:09.484 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.484 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.484 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.484 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.485 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.485 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.485 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.485 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-27T15:28:09.485 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.485 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.485 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.486 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.486 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_username": { 2023-11-27T15:28:09.486 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_username", 2023-11-27T15:28:09.486 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.486 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.486 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.486 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.487 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.487 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.487 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.487 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-27T15:28:09.487 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.487 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.487 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.487 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.488 INFO:teuthology.orchestra.run.smithi052.stdout: "secure_monitoring_stack": { 2023-11-27T15:28:09.488 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "secure_monitoring_stack", 2023-11-27T15:28:09.488 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.488 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.488 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.488 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.488 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.488 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.489 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.489 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-27T15:28:09.489 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.489 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.489 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.489 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.489 INFO:teuthology.orchestra.run.smithi052.stdout: "service_discovery_port": { 2023-11-27T15:28:09.489 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "service_discovery_port", 2023-11-27T15:28:09.490 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.490 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.490 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.490 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8765", 2023-11-27T15:28:09.490 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.490 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.490 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.491 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "cephadm service discovery port", 2023-11-27T15:28:09.491 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.491 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.491 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.491 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.491 INFO:teuthology.orchestra.run.smithi052.stdout: "ssh_config_file": { 2023-11-27T15:28:09.491 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssh_config_file", 2023-11-27T15:28:09.491 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.492 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.492 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.492 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.492 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.492 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.492 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.492 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-27T15:28:09.492 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.493 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.493 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.493 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.493 INFO:teuthology.orchestra.run.smithi052.stdout: "use_agent": { 2023-11-27T15:28:09.493 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "use_agent", 2023-11-27T15:28:09.493 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.493 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.494 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.494 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.494 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.494 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.494 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.494 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-27T15:28:09.494 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.494 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.495 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.495 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.495 INFO:teuthology.orchestra.run.smithi052.stdout: "use_repo_digest": { 2023-11-27T15:28:09.495 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "use_repo_digest", 2023-11-27T15:28:09.495 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.495 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.495 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.495 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.496 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.496 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.496 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.496 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-27T15:28:09.496 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.496 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.496 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.496 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.497 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_failed_host_check": { 2023-11-27T15:28:09.497 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_failed_host_check", 2023-11-27T15:28:09.497 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.497 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.497 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.497 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.497 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.498 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.498 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.498 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if the host check fails", 2023-11-27T15:28:09.498 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.498 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.498 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.498 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.498 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_stray_daemons": { 2023-11-27T15:28:09.499 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_stray_daemons", 2023-11-27T15:28:09.499 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.499 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.499 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.499 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.499 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.499 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.499 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.500 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-27T15:28:09.500 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.500 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.500 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.500 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.500 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_stray_hosts": { 2023-11-27T15:28:09.500 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_stray_hosts", 2023-11-27T15:28:09.500 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.501 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.501 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.501 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.501 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.501 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.501 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.501 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-27T15:28:09.501 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.502 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.502 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.502 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.502 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.502 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.502 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.502 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crash", 2023-11-27T15:28:09.503 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:09.503 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:09.503 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:09.503 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:09.503 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:09.503 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.503 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.503 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.503 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.504 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.504 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.504 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.504 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.504 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.504 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.504 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.504 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.505 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.505 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.505 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.505 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.505 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.505 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.505 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.505 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:09.506 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:09.506 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.506 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.506 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.506 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.506 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.506 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.507 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.507 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.507 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.507 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.507 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.507 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.507 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:09.507 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:09.508 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.508 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.508 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.508 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:09.508 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.508 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.508 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.508 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.509 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.509 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.509 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.509 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.509 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.509 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.509 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.509 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.510 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.510 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.510 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.510 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:09.510 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:09.510 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.510 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.511 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.511 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.511 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.511 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.511 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.511 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.511 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.511 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.512 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.512 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.512 INFO:teuthology.orchestra.run.smithi052.stdout: "retain_interval": { 2023-11-27T15:28:09.512 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "retain_interval", 2023-11-27T15:28:09.512 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:09.512 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.512 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.512 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "31536000", 2023-11-27T15:28:09.513 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.513 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.513 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.513 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-27T15:28:09.513 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.513 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.513 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.513 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.514 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_recent_interval": { 2023-11-27T15:28:09.514 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_recent_interval", 2023-11-27T15:28:09.514 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:09.514 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.514 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.514 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1209600", 2023-11-27T15:28:09.514 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.514 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.515 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.515 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-27T15:28:09.515 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.515 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.515 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.515 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.515 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.515 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.516 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.516 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "dashboard", 2023-11-27T15:28:09.516 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:09.516 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:09.516 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:09.516 INFO:teuthology.orchestra.run.smithi052.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-27T15:28:09.516 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-27T15:28:09.516 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.517 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.517 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.517 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2023-11-27T15:28:09.517 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.517 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.517 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.517 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.517 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.518 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.518 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.518 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.518 INFO:teuthology.orchestra.run.smithi052.stdout: "ALERTMANAGER_API_HOST": { 2023-11-27T15:28:09.518 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-27T15:28:09.518 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.518 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.518 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.519 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.519 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.519 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.519 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.519 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.519 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.519 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.519 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.520 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.520 INFO:teuthology.orchestra.run.smithi052.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-27T15:28:09.520 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-27T15:28:09.520 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.520 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.520 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.520 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.520 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.521 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.521 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.521 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.521 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.521 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.521 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.521 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.521 INFO:teuthology.orchestra.run.smithi052.stdout: "AUDIT_API_ENABLED": { 2023-11-27T15:28:09.522 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "AUDIT_API_ENABLED", 2023-11-27T15:28:09.522 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.522 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.522 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.522 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.522 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.522 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.523 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.523 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.523 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.523 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.523 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.523 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.523 INFO:teuthology.orchestra.run.smithi052.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-27T15:28:09.523 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-27T15:28:09.524 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.524 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.524 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.524 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.524 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.524 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.524 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.524 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.525 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.525 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.525 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.525 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.525 INFO:teuthology.orchestra.run.smithi052.stdout: "ENABLE_BROWSABLE_API": { 2023-11-27T15:28:09.525 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-27T15:28:09.525 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.525 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.526 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.526 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.526 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.526 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.526 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.526 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.526 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.527 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.527 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.527 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.527 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-27T15:28:09.527 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-27T15:28:09.527 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.527 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.528 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.528 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.528 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.528 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.528 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.528 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.528 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.528 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.529 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.529 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.529 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-27T15:28:09.529 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-27T15:28:09.529 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.529 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.529 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.529 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.530 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.530 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.530 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.530 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.530 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.530 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.530 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.530 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.531 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-27T15:28:09.531 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-27T15:28:09.531 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.531 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.531 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.531 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.531 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.531 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.532 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.532 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.532 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.532 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.532 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.532 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.532 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-27T15:28:09.532 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-27T15:28:09.533 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.533 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.533 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.533 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.533 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.533 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.533 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.533 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.534 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.534 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.534 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.534 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.534 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-27T15:28:09.534 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-27T15:28:09.534 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.535 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.535 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.535 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.535 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.535 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.535 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.535 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.536 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.536 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.536 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.536 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.536 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-27T15:28:09.536 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-27T15:28:09.536 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.537 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.537 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.537 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.537 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.537 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.537 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.537 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.537 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.538 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.538 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.538 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.538 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-27T15:28:09.538 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-27T15:28:09.538 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.538 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.538 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.539 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.539 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.539 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.539 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.539 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.539 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.539 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.540 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.540 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.540 INFO:teuthology.orchestra.run.smithi052.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-27T15:28:09.540 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-27T15:28:09.540 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.540 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.540 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.540 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.541 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.541 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.541 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.541 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.541 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.541 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.541 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.541 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.542 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_PASSWORD": { 2023-11-27T15:28:09.542 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-27T15:28:09.542 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.542 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.542 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.542 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2023-11-27T15:28:09.542 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.543 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.543 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.543 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.543 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.543 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.543 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.543 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.543 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-27T15:28:09.544 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-27T15:28:09.544 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.544 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.544 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.544 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.544 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.544 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.544 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.545 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.545 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.545 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.545 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.545 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.545 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_URL": { 2023-11-27T15:28:09.545 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_URL", 2023-11-27T15:28:09.545 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.546 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.546 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.546 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.546 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.546 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.546 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.546 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.547 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.547 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.547 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.547 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.547 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_USERNAME": { 2023-11-27T15:28:09.547 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-27T15:28:09.547 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.547 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.548 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.548 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2023-11-27T15:28:09.548 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.548 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.548 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.548 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.548 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.548 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.549 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.549 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.549 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-27T15:28:09.549 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-27T15:28:09.549 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.549 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.549 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.550 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.550 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.550 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.550 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.550 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.550 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.550 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.550 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.551 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.551 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-27T15:28:09.551 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-27T15:28:09.551 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.551 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.551 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.551 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.551 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.552 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.552 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.552 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.552 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.552 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.552 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.552 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.552 INFO:teuthology.orchestra.run.smithi052.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-27T15:28:09.553 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-27T15:28:09.553 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.553 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.553 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.553 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.553 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.553 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.553 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.554 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.554 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.554 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.554 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.554 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.554 INFO:teuthology.orchestra.run.smithi052.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-27T15:28:09.554 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-27T15:28:09.554 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.555 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.555 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.555 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.555 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.555 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.555 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.555 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.555 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.556 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.556 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.556 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.556 INFO:teuthology.orchestra.run.smithi052.stdout: "PROMETHEUS_API_HOST": { 2023-11-27T15:28:09.556 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-27T15:28:09.556 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.556 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.556 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.557 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.557 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.557 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.557 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.557 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.557 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.557 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.557 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.558 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.558 INFO:teuthology.orchestra.run.smithi052.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-27T15:28:09.558 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-27T15:28:09.558 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.558 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.558 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.558 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.559 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.559 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.559 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.559 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.559 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.559 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.559 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.559 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.560 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-27T15:28:09.560 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-27T15:28:09.560 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.560 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.560 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.560 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.560 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.560 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.561 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.561 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.561 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.561 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.561 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.561 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.561 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-27T15:28:09.561 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-27T15:28:09.562 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.562 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.562 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.562 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.562 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.562 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.562 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.563 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.563 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.563 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.563 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.563 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.563 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-27T15:28:09.563 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-27T15:28:09.564 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.564 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.564 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.564 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.564 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.564 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.564 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.565 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.565 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.565 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.565 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.565 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.565 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-27T15:28:09.565 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-27T15:28:09.565 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.566 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.566 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.566 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.566 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.566 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.566 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.566 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.566 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.567 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.567 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.567 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.567 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-27T15:28:09.567 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-27T15:28:09.567 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.567 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.568 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.568 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.568 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.568 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.568 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.568 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.568 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.568 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.569 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.569 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.569 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-27T15:28:09.569 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-27T15:28:09.569 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.569 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.569 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.569 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.570 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.570 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.570 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.570 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.570 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.570 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.570 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.570 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.570 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-27T15:28:09.571 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-27T15:28:09.571 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.571 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.571 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.571 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.571 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.571 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.571 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.572 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.572 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.572 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.572 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.572 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.572 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_ENABLED": { 2023-11-27T15:28:09.572 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-27T15:28:09.572 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.573 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.573 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.573 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.573 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.573 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.573 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.573 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.573 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.574 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.574 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.574 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.574 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-27T15:28:09.574 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-27T15:28:09.574 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.574 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.574 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.575 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-27T15:28:09.575 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.575 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.575 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.575 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.575 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.575 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.575 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.576 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.576 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-27T15:28:09.576 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-27T15:28:09.576 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.576 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.576 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.576 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2023-11-27T15:28:09.576 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.577 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.577 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.577 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.577 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.577 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.577 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.577 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.578 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-27T15:28:09.578 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-27T15:28:09.578 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.578 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.578 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.578 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8", 2023-11-27T15:28:09.578 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.578 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.579 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.579 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.579 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.579 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.579 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.579 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.579 INFO:teuthology.orchestra.run.smithi052.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-27T15:28:09.579 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-27T15:28:09.580 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.580 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.580 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.580 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "45", 2023-11-27T15:28:09.580 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.580 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.580 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.580 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.581 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.581 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.581 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.581 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.581 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_ACCESS_KEY": { 2023-11-27T15:28:09.581 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-27T15:28:09.581 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.581 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.582 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.582 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.582 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.582 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.582 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.582 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.582 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.582 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.583 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.583 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.583 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-27T15:28:09.583 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-27T15:28:09.583 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.583 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.583 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.584 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2023-11-27T15:28:09.584 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.584 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.584 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.584 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.584 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.584 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.585 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.585 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.585 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_SECRET_KEY": { 2023-11-27T15:28:09.585 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-27T15:28:09.585 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.585 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.585 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.585 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.586 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.586 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.586 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.586 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.586 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.586 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.586 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.587 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.587 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_SSL_VERIFY": { 2023-11-27T15:28:09.587 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-27T15:28:09.587 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.587 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.587 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.587 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.587 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.588 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.588 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.588 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.588 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.588 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.588 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.588 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.589 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-27T15:28:09.589 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-27T15:28:09.589 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.589 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.589 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.589 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2023-11-27T15:28:09.589 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.589 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.590 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.590 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.590 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.590 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.590 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.590 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.590 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-27T15:28:09.590 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-27T15:28:09.591 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.591 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.591 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.591 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2023-11-27T15:28:09.591 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.591 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.591 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.591 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.592 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.592 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.592 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.592 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.592 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-27T15:28:09.592 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-27T15:28:09.592 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.592 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.593 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.593 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2023-11-27T15:28:09.593 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.593 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.593 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.593 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.593 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.594 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.594 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.594 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.594 INFO:teuthology.orchestra.run.smithi052.stdout: "cross_origin_url": { 2023-11-27T15:28:09.594 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cross_origin_url", 2023-11-27T15:28:09.594 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.594 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.594 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.595 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.595 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.595 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.595 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.595 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.595 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.595 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.595 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.595 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.596 INFO:teuthology.orchestra.run.smithi052.stdout: "crt_file": { 2023-11-27T15:28:09.596 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crt_file", 2023-11-27T15:28:09.596 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.596 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.596 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.596 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.596 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.597 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.597 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.597 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.597 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.597 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.597 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.597 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.597 INFO:teuthology.orchestra.run.smithi052.stdout: "debug": { 2023-11-27T15:28:09.598 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "debug", 2023-11-27T15:28:09.598 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.598 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.598 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.598 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.598 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.598 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.599 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.599 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable/disable debug options", 2023-11-27T15:28:09.599 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.599 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.599 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.599 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.599 INFO:teuthology.orchestra.run.smithi052.stdout: "jwt_token_ttl": { 2023-11-27T15:28:09.599 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "jwt_token_ttl", 2023-11-27T15:28:09.599 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.600 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.600 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.600 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "28800", 2023-11-27T15:28:09.600 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.600 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.600 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.600 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.600 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.601 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.601 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.601 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.601 INFO:teuthology.orchestra.run.smithi052.stdout: "key_file": { 2023-11-27T15:28:09.601 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "key_file", 2023-11-27T15:28:09.601 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.601 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.601 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.602 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.602 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.602 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.602 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.602 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.602 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.602 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.602 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.603 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.603 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:09.603 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:09.603 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.603 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.603 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.603 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.603 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.604 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.604 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.604 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.604 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.604 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.604 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.604 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.604 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.605 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.605 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.605 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.605 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.605 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.605 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.605 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:09.605 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:09.606 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.606 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.606 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.606 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.606 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.606 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.606 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.607 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.607 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.607 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.607 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.607 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.607 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:09.607 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:09.607 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.608 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.608 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.608 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:09.608 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.608 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.608 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.608 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.608 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.609 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.609 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.609 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.609 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.609 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.609 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.609 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.609 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.610 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.610 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.610 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:09.610 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:09.610 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.610 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.610 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.610 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.611 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.611 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.611 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.611 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.611 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.611 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.611 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.611 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.612 INFO:teuthology.orchestra.run.smithi052.stdout: "motd": { 2023-11-27T15:28:09.612 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "motd", 2023-11-27T15:28:09.612 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.612 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.612 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.612 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.612 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.612 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.613 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.613 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "The message of the day", 2023-11-27T15:28:09.613 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.613 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.613 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.613 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.613 INFO:teuthology.orchestra.run.smithi052.stdout: "redirect_resolve_ip_addr": { 2023-11-27T15:28:09.613 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "redirect_resolve_ip_addr", 2023-11-27T15:28:09.614 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.614 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.614 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.614 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.614 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.614 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.614 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.615 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.615 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.615 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.615 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.615 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.615 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2023-11-27T15:28:09.615 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2023-11-27T15:28:09.615 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.616 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.616 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.616 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "::", 2023-11-27T15:28:09.616 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.616 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.616 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.616 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.616 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.617 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.617 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.617 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.617 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2023-11-27T15:28:09.617 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2023-11-27T15:28:09.617 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.617 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.618 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.618 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8080", 2023-11-27T15:28:09.618 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.618 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.618 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.618 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.618 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.618 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.619 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.619 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.619 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl": { 2023-11-27T15:28:09.619 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl", 2023-11-27T15:28:09.619 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.619 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.619 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.619 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.620 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.620 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.620 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.620 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.620 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.620 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.620 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.620 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.621 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl_server_port": { 2023-11-27T15:28:09.621 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl_server_port", 2023-11-27T15:28:09.621 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.621 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.621 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.621 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8443", 2023-11-27T15:28:09.621 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.621 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.622 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.622 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.622 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.622 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.622 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.622 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.622 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_behaviour": { 2023-11-27T15:28:09.623 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_behaviour", 2023-11-27T15:28:09.623 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.623 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.623 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.623 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "redirect", 2023-11-27T15:28:09.623 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.623 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.623 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.624 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.624 INFO:teuthology.orchestra.run.smithi052.stdout: "redirect" 2023-11-27T15:28:09.624 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.624 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.624 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.624 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.624 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.624 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.624 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_error_status_code": { 2023-11-27T15:28:09.625 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_error_status_code", 2023-11-27T15:28:09.625 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.625 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.625 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.625 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "500", 2023-11-27T15:28:09.625 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "400", 2023-11-27T15:28:09.625 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "599", 2023-11-27T15:28:09.626 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.626 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.626 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.626 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.626 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.626 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.626 INFO:teuthology.orchestra.run.smithi052.stdout: "url_prefix": { 2023-11-27T15:28:09.627 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "url_prefix", 2023-11-27T15:28:09.627 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.627 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.627 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.627 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.627 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.627 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.627 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.628 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.628 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.628 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.628 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.628 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.628 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.628 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.628 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.629 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "devicehealth", 2023-11-27T15:28:09.629 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:09.629 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:09.629 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:09.629 INFO:teuthology.orchestra.run.smithi052.stdout: "enable_monitoring": { 2023-11-27T15:28:09.629 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enable_monitoring", 2023-11-27T15:28:09.629 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.629 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.630 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.630 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.630 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.630 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.630 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.630 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "monitor device health metrics", 2023-11-27T15:28:09.630 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.631 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.631 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.631 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.631 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:09.631 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:09.631 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.631 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.631 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.632 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.632 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.632 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.632 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.632 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.632 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.632 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.633 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.633 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.633 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.633 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.633 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.633 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.634 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.634 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.634 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.634 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:09.634 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:09.634 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.634 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.634 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.635 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.635 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.635 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.635 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.635 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.635 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.635 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.635 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.636 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.636 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:09.636 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:09.636 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.636 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.636 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.636 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:09.636 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.637 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.637 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.637 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.637 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.637 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.637 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.637 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.638 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.638 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.638 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.638 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.638 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.638 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.638 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.638 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:09.639 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:09.639 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.639 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.639 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.639 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.639 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.639 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.639 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.640 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.640 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.640 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.640 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.640 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.640 INFO:teuthology.orchestra.run.smithi052.stdout: "mark_out_threshold": { 2023-11-27T15:28:09.640 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mark_out_threshold", 2023-11-27T15:28:09.640 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:09.641 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.641 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.641 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "2419200", 2023-11-27T15:28:09.641 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.641 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.641 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.641 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-27T15:28:09.641 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.642 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.642 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.642 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.642 INFO:teuthology.orchestra.run.smithi052.stdout: "pool_name": { 2023-11-27T15:28:09.642 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pool_name", 2023-11-27T15:28:09.642 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.642 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.642 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.643 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "device_health_metrics", 2023-11-27T15:28:09.643 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.643 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.643 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.643 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-27T15:28:09.643 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.643 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.643 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.644 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.644 INFO:teuthology.orchestra.run.smithi052.stdout: "retention_period": { 2023-11-27T15:28:09.644 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "retention_period", 2023-11-27T15:28:09.644 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:09.644 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.644 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.644 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15552000", 2023-11-27T15:28:09.644 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.645 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.645 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.645 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long to retain device health metrics", 2023-11-27T15:28:09.645 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.645 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.645 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.645 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.645 INFO:teuthology.orchestra.run.smithi052.stdout: "scrape_frequency": { 2023-11-27T15:28:09.646 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "scrape_frequency", 2023-11-27T15:28:09.646 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:09.646 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.646 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.646 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "86400", 2023-11-27T15:28:09.646 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.646 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.647 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.647 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-27T15:28:09.647 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.647 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.647 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.647 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.647 INFO:teuthology.orchestra.run.smithi052.stdout: "self_heal": { 2023-11-27T15:28:09.647 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "self_heal", 2023-11-27T15:28:09.648 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.648 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.648 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.648 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.648 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.648 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.648 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.648 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-27T15:28:09.649 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.649 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.649 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.649 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.649 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2023-11-27T15:28:09.649 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2023-11-27T15:28:09.649 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:09.649 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.650 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.650 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2023-11-27T15:28:09.650 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.650 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.650 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.650 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to wake up and check device health", 2023-11-27T15:28:09.650 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.650 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.651 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.651 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.651 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_threshold": { 2023-11-27T15:28:09.651 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_threshold", 2023-11-27T15:28:09.651 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:09.651 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.651 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.651 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "7257600", 2023-11-27T15:28:09.652 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.652 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.652 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.652 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-27T15:28:09.652 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.652 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.652 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.652 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.653 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.653 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.653 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.653 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "diskprediction_local", 2023-11-27T15:28:09.653 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:09.653 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:09.653 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:09.653 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:09.654 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:09.654 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.654 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.654 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.654 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.654 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.654 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.655 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.655 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.655 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.655 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.655 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.655 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.655 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.655 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.656 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.656 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.656 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.656 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.656 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.656 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:09.656 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:09.656 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.657 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.657 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.657 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.657 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.657 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.657 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.657 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.658 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.658 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.658 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.658 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.658 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:09.658 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:09.658 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.658 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.659 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.659 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:09.659 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.659 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.659 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.659 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.659 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.659 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.660 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.660 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.660 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.660 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.660 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.660 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.660 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.660 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.661 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.661 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:09.661 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:09.661 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.661 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.661 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.661 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.661 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.662 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.662 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.662 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.662 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.662 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.662 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.662 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.663 INFO:teuthology.orchestra.run.smithi052.stdout: "predict_interval": { 2023-11-27T15:28:09.663 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "predict_interval", 2023-11-27T15:28:09.663 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.663 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.663 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.663 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "86400", 2023-11-27T15:28:09.663 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.663 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.664 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.664 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.664 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.664 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.664 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.664 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.664 INFO:teuthology.orchestra.run.smithi052.stdout: "predictor_model": { 2023-11-27T15:28:09.664 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "predictor_model", 2023-11-27T15:28:09.665 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.665 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.665 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.665 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "prophetstor", 2023-11-27T15:28:09.665 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.665 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.665 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.665 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.666 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.666 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.666 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.666 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.666 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2023-11-27T15:28:09.666 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2023-11-27T15:28:09.666 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.666 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.667 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.667 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2023-11-27T15:28:09.667 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.667 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.667 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.667 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.667 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.667 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.668 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.668 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.668 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.668 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.668 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.668 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "influx", 2023-11-27T15:28:09.668 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": false, 2023-11-27T15:28:09.669 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "influxdb python module not found", 2023-11-27T15:28:09.669 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:09.669 INFO:teuthology.orchestra.run.smithi052.stdout: "batch_size": { 2023-11-27T15:28:09.669 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "batch_size", 2023-11-27T15:28:09.669 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.669 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.669 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.669 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5000", 2023-11-27T15:28:09.670 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.670 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.670 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.670 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-27T15:28:09.670 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.670 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.670 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.670 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.671 INFO:teuthology.orchestra.run.smithi052.stdout: "database": { 2023-11-27T15:28:09.671 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "database", 2023-11-27T15:28:09.671 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.671 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.671 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.671 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "ceph", 2023-11-27T15:28:09.671 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.671 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.672 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.672 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-27T15:28:09.672 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.672 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.672 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.672 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.672 INFO:teuthology.orchestra.run.smithi052.stdout: "hostname": { 2023-11-27T15:28:09.672 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "hostname", 2023-11-27T15:28:09.673 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.673 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.673 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.673 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.673 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.673 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.673 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.673 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB server hostname", 2023-11-27T15:28:09.674 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.674 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.674 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.674 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.674 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2023-11-27T15:28:09.674 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2023-11-27T15:28:09.674 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:09.674 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.675 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.675 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "30", 2023-11-27T15:28:09.675 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "5", 2023-11-27T15:28:09.675 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.675 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.675 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-27T15:28:09.675 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.675 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.675 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.676 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.676 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:09.676 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:09.676 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.676 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.676 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.676 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.676 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.677 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.677 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.677 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.677 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.677 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.677 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.677 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.678 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.678 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.678 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.678 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.678 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.678 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.678 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.678 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:09.679 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:09.679 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.679 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.679 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.679 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.679 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.679 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.679 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.680 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.680 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.680 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.680 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.680 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.680 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:09.680 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:09.681 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.681 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.681 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.681 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:09.681 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.681 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.681 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.681 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.682 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.682 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.682 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.682 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.682 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.682 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.682 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.682 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.683 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.683 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.683 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.683 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:09.683 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:09.683 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.683 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.683 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.683 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.684 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.684 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.684 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.684 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.684 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.684 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.684 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.685 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.685 INFO:teuthology.orchestra.run.smithi052.stdout: "password": { 2023-11-27T15:28:09.685 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "password", 2023-11-27T15:28:09.685 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.685 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.685 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.685 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.686 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.686 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.686 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.686 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "password of InfluxDB server user", 2023-11-27T15:28:09.686 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.686 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.686 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.686 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.687 INFO:teuthology.orchestra.run.smithi052.stdout: "port": { 2023-11-27T15:28:09.687 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "port", 2023-11-27T15:28:09.687 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.687 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.687 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.687 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8086", 2023-11-27T15:28:09.687 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.687 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.688 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.688 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB server port", 2023-11-27T15:28:09.688 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.688 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.688 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.688 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.688 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl": { 2023-11-27T15:28:09.688 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl", 2023-11-27T15:28:09.689 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.689 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.689 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.689 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "false", 2023-11-27T15:28:09.689 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.689 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.689 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.689 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-27T15:28:09.690 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.690 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.690 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.690 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.690 INFO:teuthology.orchestra.run.smithi052.stdout: "threads": { 2023-11-27T15:28:09.690 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "threads", 2023-11-27T15:28:09.690 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.691 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.691 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.691 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2023-11-27T15:28:09.691 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2023-11-27T15:28:09.691 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "32", 2023-11-27T15:28:09.691 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.691 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-27T15:28:09.691 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.692 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.692 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.692 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.692 INFO:teuthology.orchestra.run.smithi052.stdout: "username": { 2023-11-27T15:28:09.692 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "username", 2023-11-27T15:28:09.692 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.692 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.692 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.693 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.693 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.693 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.693 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.693 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "username of InfluxDB server user", 2023-11-27T15:28:09.693 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.693 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.694 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.694 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.694 INFO:teuthology.orchestra.run.smithi052.stdout: "verify_ssl": { 2023-11-27T15:28:09.694 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "verify_ssl", 2023-11-27T15:28:09.694 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.694 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.694 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.695 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "true", 2023-11-27T15:28:09.695 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.695 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.695 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.695 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-27T15:28:09.695 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.695 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.695 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.696 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.696 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.696 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.696 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.696 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "insights", 2023-11-27T15:28:09.696 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:09.696 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:09.696 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:09.697 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:09.697 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:09.697 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.697 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.697 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.697 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.697 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.697 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.698 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.698 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.698 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.698 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.698 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.698 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.698 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.699 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.699 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.699 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.699 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.699 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.699 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.699 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:09.699 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:09.700 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.700 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.700 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.700 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.700 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.700 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.700 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.700 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.701 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.701 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.701 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.701 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.701 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:09.701 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:09.701 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.702 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.702 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.702 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:09.702 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.702 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.702 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.702 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.702 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.703 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.703 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.703 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.703 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.703 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.703 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.703 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.704 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.704 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.704 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.704 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:09.704 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:09.704 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.704 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.704 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.705 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.705 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.705 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.705 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.705 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.705 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.705 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.705 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.706 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.706 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.706 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.706 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.706 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "iostat", 2023-11-27T15:28:09.706 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:09.706 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:09.707 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:09.707 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:09.707 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:09.707 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.707 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.707 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.707 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.707 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.708 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.708 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.708 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.708 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.708 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.708 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.708 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.708 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.709 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.709 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.709 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.709 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.709 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.709 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.709 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:09.709 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:09.710 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.710 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.710 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.710 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.710 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.710 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.710 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.710 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.711 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.711 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.711 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.711 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.711 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:09.711 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:09.711 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.711 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.712 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.712 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:09.712 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.712 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.712 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.712 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.712 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.712 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.713 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.713 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.713 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.713 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.713 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.713 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.713 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.713 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.714 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.714 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:09.714 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:09.714 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.714 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.714 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.714 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.715 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.715 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.715 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.715 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.715 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.715 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.715 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.715 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.716 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.716 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.716 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.716 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "k8sevents", 2023-11-27T15:28:09.716 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:09.716 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:09.716 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:09.716 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_event_retention_days": { 2023-11-27T15:28:09.717 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ceph_event_retention_days", 2023-11-27T15:28:09.717 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.717 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.717 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.717 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "7", 2023-11-27T15:28:09.717 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.717 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.717 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.718 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Days to hold ceph event information within local cache", 2023-11-27T15:28:09.718 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.718 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.718 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.718 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.718 INFO:teuthology.orchestra.run.smithi052.stdout: "config_check_secs": { 2023-11-27T15:28:09.718 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_check_secs", 2023-11-27T15:28:09.718 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.719 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.719 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.719 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2023-11-27T15:28:09.719 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "10", 2023-11-27T15:28:09.719 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.719 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.719 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-11-27T15:28:09.719 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.720 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.720 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.720 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.720 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:09.720 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:09.720 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.720 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.720 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.721 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.721 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.721 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.721 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.721 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.721 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.721 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.721 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.722 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.722 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.722 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.722 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.722 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.722 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.722 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.722 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.723 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:09.723 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:09.723 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.723 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.723 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.723 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.723 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.723 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.724 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.724 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.724 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.724 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.724 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.724 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.724 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:09.724 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:09.725 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.725 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.725 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.725 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:09.725 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.725 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.725 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.725 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.726 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.726 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.726 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.726 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.726 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.726 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.726 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.727 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.727 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.727 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.727 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.727 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:09.727 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:09.727 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.727 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.728 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.728 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.728 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.728 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.728 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.728 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.728 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.728 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.729 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.729 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.729 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.729 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.729 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.729 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "localpool", 2023-11-27T15:28:09.729 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:09.730 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:09.730 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:09.730 INFO:teuthology.orchestra.run.smithi052.stdout: "failure_domain": { 2023-11-27T15:28:09.730 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "failure_domain", 2023-11-27T15:28:09.730 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.730 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.730 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.730 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "host", 2023-11-27T15:28:09.731 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.731 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.731 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.731 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "failure domain for any created local pool", 2023-11-27T15:28:09.731 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-27T15:28:09.731 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.731 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.732 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.732 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:09.732 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:09.732 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.732 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.732 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.732 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.732 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.733 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.733 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.733 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.733 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.733 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.733 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.733 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.733 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.734 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.734 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.734 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.734 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.734 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.734 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.734 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:09.734 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:09.735 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.735 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.735 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.735 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.735 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.735 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.735 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.735 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.736 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.736 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.736 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.736 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.736 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:09.736 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:09.736 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.736 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.737 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.737 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:09.737 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.737 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.737 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.737 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.737 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.738 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.738 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.738 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.738 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.738 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.738 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.738 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.738 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.739 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.739 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.739 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:09.739 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:09.739 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.739 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.739 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.739 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.740 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.740 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.740 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.740 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.740 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.740 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.740 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.740 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.741 INFO:teuthology.orchestra.run.smithi052.stdout: "min_size": { 2023-11-27T15:28:09.741 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "min_size", 2023-11-27T15:28:09.741 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.741 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.741 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.741 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.741 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.741 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.742 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.742 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default min_size for any created local pool", 2023-11-27T15:28:09.742 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-27T15:28:09.742 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.742 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.742 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.742 INFO:teuthology.orchestra.run.smithi052.stdout: "num_rep": { 2023-11-27T15:28:09.742 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "num_rep", 2023-11-27T15:28:09.743 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.743 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.743 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.743 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3", 2023-11-27T15:28:09.743 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.743 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.743 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.743 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default replica count for any created local pool", 2023-11-27T15:28:09.744 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.744 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.744 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.744 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.744 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_num": { 2023-11-27T15:28:09.744 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pg_num", 2023-11-27T15:28:09.744 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.744 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.745 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.745 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "128", 2023-11-27T15:28:09.745 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.745 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.745 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.745 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default pg_num for any created local pool", 2023-11-27T15:28:09.745 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.745 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.746 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.746 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.746 INFO:teuthology.orchestra.run.smithi052.stdout: "prefix": { 2023-11-27T15:28:09.746 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prefix", 2023-11-27T15:28:09.746 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.746 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.746 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.746 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.747 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.747 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.747 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.747 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "name prefix for any created local pool", 2023-11-27T15:28:09.747 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.747 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.747 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.747 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.748 INFO:teuthology.orchestra.run.smithi052.stdout: "subtree": { 2023-11-27T15:28:09.748 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "subtree", 2023-11-27T15:28:09.748 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.748 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.748 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.748 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "rack", 2023-11-27T15:28:09.748 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.748 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.749 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.749 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-27T15:28:09.749 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-27T15:28:09.749 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.749 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.749 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.749 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.749 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.750 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.750 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mds_autoscaler", 2023-11-27T15:28:09.750 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:09.750 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:09.750 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:09.750 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:09.750 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:09.751 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.751 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.751 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.751 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.751 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.751 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.751 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.751 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.752 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.752 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.752 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.752 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.752 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.752 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.752 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.752 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.753 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.753 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.753 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.753 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:09.753 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:09.753 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.753 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.753 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.754 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.754 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.754 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.754 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.754 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.754 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.754 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.754 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.755 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.755 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:09.755 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:09.755 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.755 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.755 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.755 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:09.756 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.756 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.756 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.756 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.756 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.756 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.756 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.756 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.757 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.757 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.757 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.757 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.757 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.757 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.757 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.757 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:09.758 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:09.758 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.758 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.758 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.758 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.758 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.758 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.758 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.759 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.759 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.759 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.759 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.759 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.759 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.759 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.759 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.760 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mirroring", 2023-11-27T15:28:09.760 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:09.760 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:09.760 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:09.760 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:09.760 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:09.760 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.761 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.761 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.761 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.761 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.761 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.761 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.761 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.761 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.762 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.762 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.762 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.762 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.762 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.762 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.762 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.762 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.763 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.763 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.763 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:09.763 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:09.763 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.763 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.763 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.763 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.764 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.764 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.764 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.764 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.764 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.764 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.764 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.764 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.765 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:09.765 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:09.765 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.765 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.765 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.765 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:09.765 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.765 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.766 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.766 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.766 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.766 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.766 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.766 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.766 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.767 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.767 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.767 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.767 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.767 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.767 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.767 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:09.767 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:09.768 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.768 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.768 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.768 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.768 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.768 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.768 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.768 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.769 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.769 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.769 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.769 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.769 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.769 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.769 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.770 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "nfs", 2023-11-27T15:28:09.770 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:09.770 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:09.770 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:09.770 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:09.770 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:09.770 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.770 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.771 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.771 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.771 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.771 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.771 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.771 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.771 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.771 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.771 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.772 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.772 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.772 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.772 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.772 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.772 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.772 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.772 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.773 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:09.773 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:09.773 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.773 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.773 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.773 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.773 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.773 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.774 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.774 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.774 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.774 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.774 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.774 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.774 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:09.774 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:09.775 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.775 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.775 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.775 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:09.775 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.775 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.775 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.776 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.776 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.776 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.776 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.776 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.776 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.776 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.776 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.777 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.777 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.777 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.777 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.777 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:09.777 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:09.777 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.777 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.778 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.778 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.778 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.778 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.778 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.778 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.778 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.779 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.779 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.779 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.779 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.779 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.779 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.779 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "orchestrator", 2023-11-27T15:28:09.779 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:09.780 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:09.780 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:09.780 INFO:teuthology.orchestra.run.smithi052.stdout: "fail_fs": { 2023-11-27T15:28:09.780 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "fail_fs", 2023-11-27T15:28:09.780 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.780 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.780 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.780 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.780 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.781 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.781 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.781 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-27T15:28:09.781 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.781 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.781 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.781 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.781 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:09.782 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:09.782 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.782 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.782 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.782 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.782 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.782 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.782 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.783 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.783 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.783 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.783 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.783 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.783 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.783 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.783 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.784 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.784 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.784 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.784 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.784 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:09.784 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:09.784 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.784 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.785 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.785 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.785 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.785 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.785 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.785 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.785 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.786 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.786 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.786 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.786 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:09.786 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:09.786 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.786 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.786 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.787 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:09.787 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.787 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.787 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.787 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.787 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.787 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.787 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.788 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.788 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.788 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.788 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.788 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.788 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.788 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.789 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.789 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:09.789 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:09.789 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.789 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.789 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.789 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.789 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.790 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.790 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.790 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.790 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.790 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.790 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.790 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.790 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator": { 2023-11-27T15:28:09.791 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "orchestrator", 2023-11-27T15:28:09.791 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.791 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.791 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.791 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.791 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.791 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.791 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.792 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm", 2023-11-27T15:28:09.792 INFO:teuthology.orchestra.run.smithi052.stdout: "rook", 2023-11-27T15:28:09.792 INFO:teuthology.orchestra.run.smithi052.stdout: "test_orchestrator" 2023-11-27T15:28:09.792 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.792 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Orchestrator backend", 2023-11-27T15:28:09.792 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.792 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.792 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.793 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.793 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.793 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.793 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.793 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "osd_perf_query", 2023-11-27T15:28:09.793 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:09.793 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:09.794 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:09.794 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:09.794 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:09.794 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.794 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.794 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.794 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.795 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.795 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.795 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.795 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.795 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.795 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.795 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.795 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.796 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.796 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.796 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.796 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.796 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.796 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.796 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.796 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:09.797 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:09.797 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.797 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.797 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.797 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.797 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.797 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.797 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.798 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.798 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.798 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.798 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.798 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.798 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:09.798 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:09.798 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.799 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.799 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.799 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:09.799 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.799 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.799 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.799 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.799 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.799 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.800 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.800 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.800 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.800 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.800 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.800 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.800 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.800 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.801 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.801 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:09.801 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:09.801 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.801 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.801 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.801 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.801 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.802 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.802 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.802 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.802 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.802 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.802 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.802 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.802 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.803 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.803 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.803 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "osd_support", 2023-11-27T15:28:09.803 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:09.803 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:09.803 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:09.803 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:09.803 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:09.804 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.804 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.804 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.804 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.804 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.804 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.804 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.804 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.805 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.805 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.805 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.805 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.805 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.805 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.805 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.805 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.806 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.806 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.806 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.806 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:09.806 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:09.806 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.806 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.806 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.807 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.807 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.807 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.807 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.807 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.807 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.807 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.807 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.808 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.808 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:09.808 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:09.808 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.808 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.808 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.808 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:09.808 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.809 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.809 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.809 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.809 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.809 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.809 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.809 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.809 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.810 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.810 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.810 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.810 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.810 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.810 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.810 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:09.810 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:09.811 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.811 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.811 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.811 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.811 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.811 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.811 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.811 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.812 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.812 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.812 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.812 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.812 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.812 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.812 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.812 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pg_autoscaler", 2023-11-27T15:28:09.813 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:09.813 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:09.813 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:09.813 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:09.813 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:09.813 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.813 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.813 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.814 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.814 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.814 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.814 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.814 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.814 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.814 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.814 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.815 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.815 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.815 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.815 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.815 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.815 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.815 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.815 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.816 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:09.816 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:09.816 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.816 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.816 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.816 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.816 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.816 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.816 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.817 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.817 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.817 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.817 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.817 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.817 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:09.817 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:09.818 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.818 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.818 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.818 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:09.818 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.818 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.818 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.818 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.819 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.819 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.819 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.819 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.819 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.819 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.819 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.819 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.819 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.820 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.820 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.820 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:09.820 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:09.820 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.820 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.820 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.820 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.821 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.821 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.821 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.821 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.821 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.821 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.821 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.821 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.822 INFO:teuthology.orchestra.run.smithi052.stdout: "noautoscale": { 2023-11-27T15:28:09.822 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "noautoscale", 2023-11-27T15:28:09.822 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.822 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.822 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.822 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.822 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.822 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.822 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.823 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "global autoscale flag", 2023-11-27T15:28:09.823 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-11-27T15:28:09.823 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.823 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.823 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.823 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2023-11-27T15:28:09.823 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2023-11-27T15:28:09.824 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:09.824 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.824 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.824 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2023-11-27T15:28:09.824 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.824 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.824 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.824 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.825 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.825 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.825 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.825 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.825 INFO:teuthology.orchestra.run.smithi052.stdout: "threshold": { 2023-11-27T15:28:09.825 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "threshold", 2023-11-27T15:28:09.825 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2023-11-27T15:28:09.825 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.826 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.826 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3.0", 2023-11-27T15:28:09.826 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1.0", 2023-11-27T15:28:09.826 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.826 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.826 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "scaling threshold", 2023-11-27T15:28:09.827 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-27T15:28:09.827 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.827 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.827 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.827 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.827 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.827 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.827 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "progress", 2023-11-27T15:28:09.828 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:09.828 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:09.828 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:09.828 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_pg_recovery_event": { 2023-11-27T15:28:09.828 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_pg_recovery_event", 2023-11-27T15:28:09.828 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.828 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.828 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.829 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.829 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.829 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.829 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.829 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-27T15:28:09.829 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.829 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.829 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.830 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.830 INFO:teuthology.orchestra.run.smithi052.stdout: "enabled": { 2023-11-27T15:28:09.830 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enabled", 2023-11-27T15:28:09.830 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.830 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.830 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.830 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.830 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.831 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.831 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.831 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.831 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.831 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.831 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.831 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.831 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:09.832 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:09.832 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.832 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.832 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.832 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.832 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.832 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.832 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.833 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.833 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.833 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.833 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.833 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.834 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.834 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.834 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.834 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.834 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.834 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.834 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.834 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:09.835 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:09.835 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.835 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.835 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.835 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.835 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.835 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.836 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.836 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.836 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.836 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.836 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.836 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.836 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:09.836 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:09.836 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.837 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.837 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.837 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:09.837 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.837 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.837 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.837 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.838 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.838 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.838 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.838 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.838 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.838 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.838 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.838 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.839 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.839 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.839 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.839 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:09.839 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:09.839 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.839 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.839 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.840 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.840 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.840 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.840 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.840 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.840 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.840 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.840 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.841 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.841 INFO:teuthology.orchestra.run.smithi052.stdout: "max_completed_events": { 2023-11-27T15:28:09.841 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_completed_events", 2023-11-27T15:28:09.841 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.841 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.841 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.841 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "50", 2023-11-27T15:28:09.841 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.842 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.842 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.842 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "number of past completed events to remember", 2023-11-27T15:28:09.842 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.842 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.842 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.842 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.843 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2023-11-27T15:28:09.843 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2023-11-27T15:28:09.843 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:09.843 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.843 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.843 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2023-11-27T15:28:09.843 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.843 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.844 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.844 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long the module is going to sleep", 2023-11-27T15:28:09.844 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.844 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.844 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.844 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.844 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.844 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.845 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.845 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus", 2023-11-27T15:28:09.845 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:09.845 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:09.845 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:09.845 INFO:teuthology.orchestra.run.smithi052.stdout: "cache": { 2023-11-27T15:28:09.845 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cache", 2023-11-27T15:28:09.845 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.846 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.846 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.846 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.846 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.846 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.846 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.846 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.846 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.847 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.847 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.847 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.847 INFO:teuthology.orchestra.run.smithi052.stdout: "exclude_perf_counters": { 2023-11-27T15:28:09.847 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "exclude_perf_counters", 2023-11-27T15:28:09.847 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.847 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.847 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.848 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.848 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.848 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.848 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.848 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-27T15:28:09.848 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-27T15:28:09.848 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.849 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.849 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.849 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:09.849 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:09.849 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.849 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.849 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.849 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.850 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.850 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.850 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.850 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.850 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.850 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.850 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.850 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.851 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.851 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.851 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.851 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.851 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.851 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.851 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.851 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:09.852 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:09.852 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.852 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.852 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.852 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.852 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.852 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.852 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.853 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.853 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.853 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.853 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.853 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.853 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:09.853 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:09.853 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.854 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.854 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.854 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:09.854 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.854 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.855 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.855 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.855 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.855 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.855 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.855 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.855 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.856 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.856 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.856 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.856 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.856 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.856 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.856 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:09.857 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:09.857 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.857 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.857 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.857 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.857 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.857 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.858 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.858 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.858 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.858 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.858 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.858 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.859 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_stats_pools": { 2023-11-27T15:28:09.859 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_stats_pools", 2023-11-27T15:28:09.859 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.859 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.859 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.859 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.859 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.859 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.860 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.860 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.860 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.860 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.860 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.860 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.860 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-27T15:28:09.861 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-27T15:28:09.861 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.861 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.861 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.861 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "300", 2023-11-27T15:28:09.861 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.861 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.862 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.862 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.862 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.862 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.862 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.862 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.862 INFO:teuthology.orchestra.run.smithi052.stdout: "scrape_interval": { 2023-11-27T15:28:09.863 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "scrape_interval", 2023-11-27T15:28:09.863 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2023-11-27T15:28:09.863 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.863 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.863 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15.0", 2023-11-27T15:28:09.863 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.863 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.864 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.864 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.864 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.864 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.864 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.864 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.864 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2023-11-27T15:28:09.865 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2023-11-27T15:28:09.865 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.865 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.865 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.865 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "::", 2023-11-27T15:28:09.865 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.865 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.866 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.866 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-27T15:28:09.866 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.866 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.866 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.866 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.866 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2023-11-27T15:28:09.867 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2023-11-27T15:28:09.867 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.867 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.867 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.867 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "9283", 2023-11-27T15:28:09.867 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.867 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.867 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.868 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-27T15:28:09.868 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.868 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.868 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.868 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.868 INFO:teuthology.orchestra.run.smithi052.stdout: "stale_cache_strategy": { 2023-11-27T15:28:09.869 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "stale_cache_strategy", 2023-11-27T15:28:09.869 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.869 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.869 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.869 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "log", 2023-11-27T15:28:09.869 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.869 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.870 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.870 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.870 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.870 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.870 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.870 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.870 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_behaviour": { 2023-11-27T15:28:09.871 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_behaviour", 2023-11-27T15:28:09.871 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.871 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.871 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.871 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "default", 2023-11-27T15:28:09.871 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.871 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.871 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.872 INFO:teuthology.orchestra.run.smithi052.stdout: "default", 2023-11-27T15:28:09.872 INFO:teuthology.orchestra.run.smithi052.stdout: "error" 2023-11-27T15:28:09.872 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.872 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.872 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.872 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.872 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.873 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.873 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_error_status_code": { 2023-11-27T15:28:09.873 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_error_status_code", 2023-11-27T15:28:09.873 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.873 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.873 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.873 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "500", 2023-11-27T15:28:09.873 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "400", 2023-11-27T15:28:09.874 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "599", 2023-11-27T15:28:09.874 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.874 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.874 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.874 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.874 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.875 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.875 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.875 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.875 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.875 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_support", 2023-11-27T15:28:09.875 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:09.875 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:09.876 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:09.876 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:09.876 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:09.876 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.876 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.876 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.876 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.876 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.877 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.877 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.877 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.877 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.877 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.877 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.877 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.878 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.878 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.878 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.878 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.878 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.878 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.879 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.879 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:09.879 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:09.879 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.879 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.879 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.879 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.880 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.880 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.880 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.880 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.880 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.880 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.881 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.881 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.881 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:09.881 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:09.881 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.881 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.881 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.882 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:09.882 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.882 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.882 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.882 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.882 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.882 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.883 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.883 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.883 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.883 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.883 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.883 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.884 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.884 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.884 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.884 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:09.884 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:09.884 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.884 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.885 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.885 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.885 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.885 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.885 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.885 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.885 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.886 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.886 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.886 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.886 INFO:teuthology.orchestra.run.smithi052.stdout: "max_concurrent_snap_create": { 2023-11-27T15:28:09.886 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_concurrent_snap_create", 2023-11-27T15:28:09.886 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.886 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.887 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.887 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2023-11-27T15:28:09.887 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.887 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.887 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.887 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.887 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.888 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.888 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.888 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.888 INFO:teuthology.orchestra.run.smithi052.stdout: "mirror_snapshot_schedule": { 2023-11-27T15:28:09.888 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mirror_snapshot_schedule", 2023-11-27T15:28:09.888 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.888 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.889 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.889 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.889 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.889 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.889 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.889 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.890 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.890 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.890 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.890 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.890 INFO:teuthology.orchestra.run.smithi052.stdout: "trash_purge_schedule": { 2023-11-27T15:28:09.890 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "trash_purge_schedule", 2023-11-27T15:28:09.890 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.891 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.891 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.891 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.891 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.891 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.891 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.891 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.892 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.892 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.892 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.892 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.892 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.892 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.892 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.893 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "restful", 2023-11-27T15:28:09.893 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:09.893 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:09.893 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:09.893 INFO:teuthology.orchestra.run.smithi052.stdout: "enable_auth": { 2023-11-27T15:28:09.893 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enable_auth", 2023-11-27T15:28:09.893 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.894 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.894 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.894 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.894 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.894 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.894 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.894 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.895 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.895 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.895 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.895 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.895 INFO:teuthology.orchestra.run.smithi052.stdout: "key_file": { 2023-11-27T15:28:09.895 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "key_file", 2023-11-27T15:28:09.895 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.896 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.896 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.896 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.896 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.896 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.896 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.896 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.897 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.897 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.897 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.897 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.897 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:09.897 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:09.897 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.897 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.898 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.898 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.898 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.898 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.898 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.898 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.898 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.899 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.899 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.899 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.899 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.899 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.899 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.900 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.900 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.900 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.900 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.900 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:09.900 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:09.901 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.901 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.901 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.901 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.901 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.901 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.901 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.902 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.902 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.902 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.902 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.902 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.902 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:09.902 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:09.903 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.903 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.903 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.903 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:09.903 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.904 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.904 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.904 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.904 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.904 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.904 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.905 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.905 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.905 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.905 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.905 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.905 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.905 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.906 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.906 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:09.906 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:09.906 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.906 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.906 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.906 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.907 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.907 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.907 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.907 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.907 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.907 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.908 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.908 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.908 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2023-11-27T15:28:09.908 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2023-11-27T15:28:09.908 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.908 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.908 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.909 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.909 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.909 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.909 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.909 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.909 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.909 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.910 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.910 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.910 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2023-11-27T15:28:09.910 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2023-11-27T15:28:09.910 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.910 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.910 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.911 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.911 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.911 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.911 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.911 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.911 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.911 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.912 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.912 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.912 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.912 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.912 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.912 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rgw", 2023-11-27T15:28:09.912 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:09.913 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:09.913 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:09.913 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:09.913 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:09.913 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.913 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.913 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.913 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.914 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.914 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.914 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.914 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.914 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.914 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.914 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.915 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.915 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.915 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.915 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.915 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.915 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.915 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.916 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.916 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:09.916 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:09.916 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.916 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.916 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.916 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.917 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.917 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.917 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.917 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.917 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.917 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.917 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.918 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.918 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:09.918 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:09.918 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.918 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.918 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.918 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:09.919 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.919 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.919 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.919 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.919 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.919 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.919 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.920 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.920 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.920 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.920 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.920 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.920 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.920 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.921 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.921 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:09.921 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:09.921 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.921 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.921 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.921 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.921 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.922 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.922 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.922 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.922 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.922 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.922 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.923 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.923 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.923 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.923 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.923 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rook", 2023-11-27T15:28:09.923 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:09.923 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:09.923 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:09.924 INFO:teuthology.orchestra.run.smithi052.stdout: "drive_group_interval": { 2023-11-27T15:28:09.924 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "drive_group_interval", 2023-11-27T15:28:09.924 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2023-11-27T15:28:09.924 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.924 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.924 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "300.0", 2023-11-27T15:28:09.924 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.925 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.925 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.925 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-11-27T15:28:09.925 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.925 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.925 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.925 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.926 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:09.926 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:09.926 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.926 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.926 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.926 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.926 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.927 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.927 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.927 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.927 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.927 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.927 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.927 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.928 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.928 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.928 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.928 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.928 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.928 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.928 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.929 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:09.929 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:09.929 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.929 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.929 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.929 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.929 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.929 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.930 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.930 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.930 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.930 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.930 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.930 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.930 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:09.931 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:09.931 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.931 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.931 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.931 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:09.931 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.931 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.931 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.932 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.932 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.932 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.932 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.932 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.932 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.933 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.933 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.933 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.933 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.933 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.933 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.933 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:09.934 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:09.934 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.934 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.934 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.934 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.934 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.934 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.935 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.935 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.935 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.935 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.935 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.935 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.936 INFO:teuthology.orchestra.run.smithi052.stdout: "storage_class": { 2023-11-27T15:28:09.936 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "storage_class", 2023-11-27T15:28:09.936 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.936 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.936 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.936 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "local", 2023-11-27T15:28:09.936 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.937 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.937 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.937 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-11-27T15:28:09.937 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.937 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.937 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.937 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.938 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.938 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.938 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.938 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "selftest", 2023-11-27T15:28:09.938 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:09.938 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:09.939 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:09.939 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:09.939 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:09.939 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.939 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.939 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.939 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.940 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.940 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.940 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.940 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.940 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.940 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.940 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.941 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.941 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.941 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.941 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.941 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.941 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.941 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.942 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.942 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:09.942 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:09.942 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.942 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.942 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.942 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.943 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.943 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.943 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.943 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.943 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.943 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.943 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.944 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.944 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:09.944 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:09.944 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.944 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.944 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.945 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:09.945 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.945 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.945 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.945 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.945 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.945 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.946 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.946 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.946 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.946 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.946 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.946 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.947 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.947 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.947 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.947 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:09.947 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:09.947 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.947 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.948 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.948 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.948 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.948 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.948 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.948 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.949 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.949 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.949 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.949 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.949 INFO:teuthology.orchestra.run.smithi052.stdout: "roption1": { 2023-11-27T15:28:09.949 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "roption1", 2023-11-27T15:28:09.949 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.950 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.950 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.950 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.950 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.950 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.950 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.950 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.951 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.951 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.951 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.951 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.951 INFO:teuthology.orchestra.run.smithi052.stdout: "roption2": { 2023-11-27T15:28:09.951 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "roption2", 2023-11-27T15:28:09.951 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.952 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.952 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.952 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "xyz", 2023-11-27T15:28:09.952 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.952 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.952 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.952 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.953 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.953 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.953 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.953 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.953 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption1": { 2023-11-27T15:28:09.953 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption1", 2023-11-27T15:28:09.953 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.954 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.954 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.954 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.954 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.954 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.954 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.954 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.955 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.955 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.955 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.955 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.955 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption2": { 2023-11-27T15:28:09.955 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption2", 2023-11-27T15:28:09.955 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.956 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.956 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.956 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.956 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.956 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.956 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.956 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.956 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.957 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.957 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.957 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.957 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption3": { 2023-11-27T15:28:09.957 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption3", 2023-11-27T15:28:09.957 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2023-11-27T15:28:09.957 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.958 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.958 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.958 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.958 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.958 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.958 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.958 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.959 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.959 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.959 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.959 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption4": { 2023-11-27T15:28:09.959 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption4", 2023-11-27T15:28:09.959 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.959 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.960 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.960 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.960 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.960 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.960 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.960 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.960 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.961 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.961 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.961 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.961 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption5": { 2023-11-27T15:28:09.961 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption5", 2023-11-27T15:28:09.961 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.961 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.961 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.962 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.962 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.962 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.962 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.962 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.962 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.962 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.963 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.963 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.963 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption6": { 2023-11-27T15:28:09.963 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption6", 2023-11-27T15:28:09.963 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.963 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.963 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.964 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:09.964 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.964 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.964 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.964 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.964 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.964 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.965 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.965 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.965 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption7": { 2023-11-27T15:28:09.965 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption7", 2023-11-27T15:28:09.965 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:09.965 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.965 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.965 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.966 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2023-11-27T15:28:09.966 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "42", 2023-11-27T15:28:09.966 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.966 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.966 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.966 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.967 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.967 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.967 INFO:teuthology.orchestra.run.smithi052.stdout: "testkey": { 2023-11-27T15:28:09.967 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testkey", 2023-11-27T15:28:09.967 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.967 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.967 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.968 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.968 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.968 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.968 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.968 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.968 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.968 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.969 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.969 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.969 INFO:teuthology.orchestra.run.smithi052.stdout: "testlkey": { 2023-11-27T15:28:09.969 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testlkey", 2023-11-27T15:28:09.969 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.969 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.969 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.970 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.970 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.970 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.970 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.970 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.970 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.970 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.971 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.971 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.971 INFO:teuthology.orchestra.run.smithi052.stdout: "testnewline": { 2023-11-27T15:28:09.971 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testnewline", 2023-11-27T15:28:09.971 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.971 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.971 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:09.972 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.972 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.972 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.972 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.972 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.972 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.972 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.972 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.973 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.973 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.973 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.973 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.973 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snap_schedule", 2023-11-27T15:28:09.973 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:09.973 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:09.974 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:09.974 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_m_granularity": { 2023-11-27T15:28:09.974 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_m_granularity", 2023-11-27T15:28:09.974 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.974 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.974 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.974 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.975 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.975 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.975 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.975 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow minute scheduled snapshots", 2023-11-27T15:28:09.975 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.975 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.975 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.975 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.976 INFO:teuthology.orchestra.run.smithi052.stdout: "dump_on_update": { 2023-11-27T15:28:09.976 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "dump_on_update", 2023-11-27T15:28:09.976 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.976 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.976 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.976 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.976 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.977 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.977 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.977 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "dump database to debug log on update", 2023-11-27T15:28:09.977 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.977 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.977 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.977 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.978 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:09.978 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:09.978 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.978 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.978 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.978 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.978 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.979 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.979 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.979 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.979 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.979 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.979 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.979 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.980 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.980 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.980 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.980 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.980 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.980 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.980 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.981 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:09.981 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:09.981 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.981 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.981 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.981 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.981 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.981 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.982 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.982 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.982 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.982 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.982 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.982 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.982 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:09.983 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:09.983 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.983 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.983 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.983 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:09.983 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.983 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.984 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.984 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.984 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.984 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.984 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.984 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.984 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.984 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.985 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.985 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.985 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.985 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.985 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.985 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:09.985 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:09.986 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.986 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.986 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.986 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.986 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.986 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.986 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.987 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.987 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.987 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.987 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.987 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.987 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.987 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.988 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.988 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "stats", 2023-11-27T15:28:09.988 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:09.988 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:09.988 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:09.988 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:09.989 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:09.989 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.989 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.989 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.989 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:09.989 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.989 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.990 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.990 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.990 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.990 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.990 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.990 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.990 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.991 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.991 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.991 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.991 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.991 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.991 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.991 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:09.992 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:09.992 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.992 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.992 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.992 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.992 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.992 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.993 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.993 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.993 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.993 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.993 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.993 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.993 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:09.994 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:09.994 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:09.994 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.994 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.994 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:09.994 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.994 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.995 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:09.995 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:09.995 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:09.995 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:09.995 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:09.995 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:09.996 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:09.996 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:09.996 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.996 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.996 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.996 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.996 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.997 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:09.997 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:09.997 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:09.997 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:09.997 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:09.997 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:09.997 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:09.998 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:09.998 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:09.998 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:09.998 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:09.998 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:09.998 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:09.998 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.999 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:09.999 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:09.999 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:09.999 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "status", 2023-11-27T15:28:09.999 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:09.999 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.000 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.000 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.000 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.000 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.000 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.000 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.000 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.000 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.001 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.001 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.001 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.001 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.001 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.001 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.001 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.002 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.002 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.002 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.002 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.002 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.002 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.002 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.003 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.003 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.003 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.003 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.003 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.003 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.003 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.004 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.004 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.004 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.004 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.004 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.004 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.004 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.005 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.005 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.005 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.005 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.005 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.005 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.005 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.006 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.006 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.006 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.006 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.006 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.006 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.006 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.007 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.007 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.007 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.007 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.007 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.007 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.007 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.008 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.008 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.008 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.008 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.008 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.008 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.009 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.009 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.009 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.009 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.009 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.009 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.009 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.010 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.010 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.010 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.010 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.010 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "telegraf", 2023-11-27T15:28:10.010 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.010 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.011 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.011 INFO:teuthology.orchestra.run.smithi052.stdout: "address": { 2023-11-27T15:28:10.011 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "address", 2023-11-27T15:28:10.011 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.011 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.011 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.011 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-27T15:28:10.012 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.012 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.012 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.012 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.012 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.012 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.012 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.013 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.013 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2023-11-27T15:28:10.013 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2023-11-27T15:28:10.013 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:10.013 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.013 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.013 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15", 2023-11-27T15:28:10.014 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.014 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.014 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.014 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.014 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.014 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.014 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.015 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.015 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.015 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.015 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.015 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.015 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.015 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.016 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.016 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.016 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.016 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.016 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.016 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.016 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.017 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.017 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.017 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.017 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.017 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.017 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.017 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.018 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.018 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.018 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.018 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.018 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.018 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.019 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.019 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.019 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.019 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.019 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.019 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.019 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.019 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.020 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.020 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.020 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.020 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.020 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.020 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.020 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.021 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.021 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.021 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.021 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.021 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.021 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.021 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.022 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.022 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.022 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.022 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.022 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.022 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.022 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.023 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.023 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.023 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.023 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.023 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.023 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.023 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.024 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.024 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.024 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.024 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.024 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.024 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.024 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.025 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.025 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.025 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.025 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.025 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "telemetry", 2023-11-27T15:28:10.025 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.025 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.026 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.026 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_basic": { 2023-11-27T15:28:10.026 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_basic", 2023-11-27T15:28:10.026 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.026 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.026 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.026 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.027 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.027 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.027 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.027 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-27T15:28:10.027 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.027 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.027 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.028 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.028 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_crash": { 2023-11-27T15:28:10.028 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_crash", 2023-11-27T15:28:10.028 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.028 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.028 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.028 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.029 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.029 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.029 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.029 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-27T15:28:10.029 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.029 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.029 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.030 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.030 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_device": { 2023-11-27T15:28:10.030 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_device", 2023-11-27T15:28:10.030 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.030 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.030 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.031 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.031 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.031 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.031 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.031 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-27T15:28:10.031 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.032 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.032 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.032 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.032 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_ident": { 2023-11-27T15:28:10.032 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_ident", 2023-11-27T15:28:10.032 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.033 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.033 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.033 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.033 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.033 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.033 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.034 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-27T15:28:10.034 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.034 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.034 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.034 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.034 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_perf": { 2023-11-27T15:28:10.035 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_perf", 2023-11-27T15:28:10.035 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.035 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.035 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.035 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.035 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.035 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.036 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.036 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-27T15:28:10.036 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.036 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.036 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.036 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.036 INFO:teuthology.orchestra.run.smithi052.stdout: "contact": { 2023-11-27T15:28:10.037 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "contact", 2023-11-27T15:28:10.037 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.037 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.037 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.037 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.037 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.037 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.038 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.038 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.038 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.038 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.038 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.038 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.039 INFO:teuthology.orchestra.run.smithi052.stdout: "description": { 2023-11-27T15:28:10.039 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "description", 2023-11-27T15:28:10.039 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.039 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.039 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.039 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.039 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.040 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.040 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.040 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.040 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.040 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.040 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.040 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.041 INFO:teuthology.orchestra.run.smithi052.stdout: "device_url": { 2023-11-27T15:28:10.041 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_url", 2023-11-27T15:28:10.041 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.041 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.041 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.041 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-27T15:28:10.041 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.042 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.042 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.042 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.042 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.042 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.042 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.042 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.043 INFO:teuthology.orchestra.run.smithi052.stdout: "enabled": { 2023-11-27T15:28:10.043 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enabled", 2023-11-27T15:28:10.043 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.043 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.043 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.043 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.044 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.044 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.044 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.044 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.044 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.044 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.044 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.045 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.045 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2023-11-27T15:28:10.045 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2023-11-27T15:28:10.045 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.045 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.045 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.045 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "24", 2023-11-27T15:28:10.046 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "8", 2023-11-27T15:28:10.046 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.046 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.046 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.046 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.046 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.047 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.047 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.047 INFO:teuthology.orchestra.run.smithi052.stdout: "last_opt_revision": { 2023-11-27T15:28:10.047 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "last_opt_revision", 2023-11-27T15:28:10.047 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.047 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.047 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.047 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1", 2023-11-27T15:28:10.048 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.048 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.048 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.048 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.048 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.048 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.048 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.049 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.049 INFO:teuthology.orchestra.run.smithi052.stdout: "leaderboard": { 2023-11-27T15:28:10.049 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "leaderboard", 2023-11-27T15:28:10.049 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.049 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.049 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.049 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.050 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.050 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.050 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.050 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.050 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.050 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.051 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.051 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.051 INFO:teuthology.orchestra.run.smithi052.stdout: "leaderboard_description": { 2023-11-27T15:28:10.051 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "leaderboard_description", 2023-11-27T15:28:10.051 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.051 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.051 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.052 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.052 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.052 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.052 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.052 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.052 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.053 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.053 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.053 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.053 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.053 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.053 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.053 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.054 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.054 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.054 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.054 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.054 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.054 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.054 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.055 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.055 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.055 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.055 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.055 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.055 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.055 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.056 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.056 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.056 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.056 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.056 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.056 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.056 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.057 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.057 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.057 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.057 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.057 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.057 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.057 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.058 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.058 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.058 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.058 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.058 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.058 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.059 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.059 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.059 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.059 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.059 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.059 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.059 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.060 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.060 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.060 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.060 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.060 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.060 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.060 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.060 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.061 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.061 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.061 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.061 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.061 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.061 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.062 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.062 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.062 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.062 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.062 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.062 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.063 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.063 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.063 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.063 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.063 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.063 INFO:teuthology.orchestra.run.smithi052.stdout: "organization": { 2023-11-27T15:28:10.064 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "organization", 2023-11-27T15:28:10.064 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.064 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.064 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.064 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.064 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.064 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.065 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.065 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.065 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.065 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.065 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.065 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.066 INFO:teuthology.orchestra.run.smithi052.stdout: "proxy": { 2023-11-27T15:28:10.066 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "proxy", 2023-11-27T15:28:10.066 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.066 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.066 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.066 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.066 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.067 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.067 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.067 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.067 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.067 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.067 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.067 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.068 INFO:teuthology.orchestra.run.smithi052.stdout: "url": { 2023-11-27T15:28:10.068 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "url", 2023-11-27T15:28:10.068 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.068 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.068 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.068 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-27T15:28:10.068 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.069 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.069 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.069 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.069 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.069 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.069 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.069 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.070 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.070 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.070 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.070 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "test_orchestrator", 2023-11-27T15:28:10.070 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.070 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.070 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.071 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.071 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.071 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.071 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.071 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.071 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.071 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.072 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.072 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.072 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.072 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.072 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.072 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.072 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.073 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.073 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.073 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.073 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.073 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.073 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.074 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.074 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.074 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.074 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.074 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.074 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.074 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.075 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.075 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.075 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.075 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.075 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.075 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.075 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.076 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.076 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.076 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.076 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.076 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.076 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.076 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.077 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.077 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.077 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.077 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.077 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.077 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.077 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.078 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.078 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.078 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.078 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.078 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.078 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.079 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.079 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.079 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.079 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.079 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.079 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.079 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.079 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.080 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.080 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.080 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.080 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.080 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.080 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.080 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.081 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.081 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.081 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.081 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.081 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "volumes", 2023-11-27T15:28:10.081 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.082 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.082 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.082 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.082 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.082 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.083 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.083 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.083 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.083 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.083 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.083 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.083 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.084 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.084 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.084 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.084 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.084 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.084 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.084 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.085 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.085 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.085 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.085 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.085 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.085 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.086 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.086 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.086 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.086 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.086 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.087 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.087 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.087 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.087 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.087 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.087 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.087 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.088 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.088 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.088 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.088 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.088 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.088 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.088 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.089 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.089 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.089 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.089 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.089 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.089 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.089 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.090 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.090 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.090 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.090 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.090 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.090 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.090 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.091 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.091 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.091 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.091 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.091 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.091 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.091 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.091 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.092 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.092 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.092 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.092 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.092 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.092 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.092 INFO:teuthology.orchestra.run.smithi052.stdout: "max_concurrent_clones": { 2023-11-27T15:28:10.093 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_concurrent_clones", 2023-11-27T15:28:10.093 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.093 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.093 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.093 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "4", 2023-11-27T15:28:10.093 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.093 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.094 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.094 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-27T15:28:10.094 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.094 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.094 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.094 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.094 INFO:teuthology.orchestra.run.smithi052.stdout: "snapshot_clone_delay": { 2023-11-27T15:28:10.095 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snapshot_clone_delay", 2023-11-27T15:28:10.095 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.095 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.095 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.095 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2023-11-27T15:28:10.095 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.096 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.096 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.096 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-27T15:28:10.096 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.096 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.096 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.096 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.097 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.097 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.097 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.097 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix", 2023-11-27T15:28:10.097 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.097 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.097 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.098 INFO:teuthology.orchestra.run.smithi052.stdout: "discovery_interval": { 2023-11-27T15:28:10.098 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "discovery_interval", 2023-11-27T15:28:10.098 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2023-11-27T15:28:10.098 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.098 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.098 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "100", 2023-11-27T15:28:10.098 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.099 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.099 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.099 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.099 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.099 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.099 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.099 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.100 INFO:teuthology.orchestra.run.smithi052.stdout: "identifier": { 2023-11-27T15:28:10.100 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "identifier", 2023-11-27T15:28:10.100 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.100 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.100 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.100 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.100 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.101 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.101 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.101 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.101 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.101 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.101 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.101 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.102 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2023-11-27T15:28:10.102 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2023-11-27T15:28:10.102 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:10.102 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.102 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.102 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2023-11-27T15:28:10.102 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.103 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.103 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.103 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.103 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.103 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.103 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.103 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.104 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.104 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.104 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.104 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.104 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.104 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.104 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.105 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.105 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.105 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.105 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.105 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.105 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.105 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.106 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.106 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.106 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.106 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.106 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.106 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.106 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.107 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.107 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.107 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.107 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.107 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.107 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.107 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.108 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.108 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.108 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.108 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.108 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.108 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.108 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.109 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.109 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.109 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.109 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.109 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.109 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.109 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.110 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.110 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.110 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.110 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.110 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.110 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.110 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.111 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.111 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.111 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.111 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.111 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.111 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.111 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.112 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.112 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.112 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.112 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.112 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.112 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.112 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.113 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.113 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.113 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.113 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.113 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.113 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.113 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.113 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_host": { 2023-11-27T15:28:10.114 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_host", 2023-11-27T15:28:10.114 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.114 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.114 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.114 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.114 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.115 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.115 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.115 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.115 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.115 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.115 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.115 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.115 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_port": { 2023-11-27T15:28:10.116 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_port", 2023-11-27T15:28:10.116 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.116 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.116 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.116 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10051", 2023-11-27T15:28:10.116 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.116 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.117 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.117 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.117 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.117 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.117 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.117 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.117 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_sender": { 2023-11-27T15:28:10.118 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_sender", 2023-11-27T15:28:10.118 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.118 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.118 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.118 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-27T15:28:10.118 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.119 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.119 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.119 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.119 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.119 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.119 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.119 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.120 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.120 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.120 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2023-11-27T15:28:10.120 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.120 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.120 INFO:teuthology.orchestra.run.smithi052.stdout: "gid": 15381, 2023-11-27T15:28:10.120 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "z", 2023-11-27T15:28:10.120 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr_features": 4540138322906710015, 2023-11-27T15:28:10.121 INFO:teuthology.orchestra.run.smithi052.stdout: "available_modules": [ 2023-11-27T15:28:10.121 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.121 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "alerts", 2023-11-27T15:28:10.121 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.121 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.121 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.121 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2023-11-27T15:28:10.122 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2023-11-27T15:28:10.122 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:10.122 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.122 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.122 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2023-11-27T15:28:10.122 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.122 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.123 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.123 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How frequently to reexamine health status", 2023-11-27T15:28:10.123 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.123 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.123 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.123 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.123 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.124 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.124 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.124 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.124 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.124 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.124 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.124 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.125 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.125 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.125 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.125 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.125 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.125 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.125 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.125 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.126 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.126 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.126 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.126 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.126 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.126 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.126 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.127 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.127 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.127 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.127 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.127 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.127 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.127 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.128 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.128 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.128 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.128 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.128 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.128 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.129 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.129 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.129 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.129 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.129 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.129 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.129 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.130 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.130 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.130 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.130 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.130 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.130 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.130 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.131 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.131 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.131 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.131 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.131 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.131 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.131 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.132 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.132 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.132 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.132 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.132 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.132 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.132 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.133 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.133 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.133 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.133 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.133 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.133 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.133 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_destination": { 2023-11-27T15:28:10.134 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_destination", 2023-11-27T15:28:10.134 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.134 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.134 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.134 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.134 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.134 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.135 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.135 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Email address to send alerts to", 2023-11-27T15:28:10.135 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.135 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.135 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.135 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.135 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_from_name": { 2023-11-27T15:28:10.136 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_from_name", 2023-11-27T15:28:10.136 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.136 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.136 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.136 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "Ceph", 2023-11-27T15:28:10.136 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.136 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.136 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.137 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Email From: name", 2023-11-27T15:28:10.137 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.137 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.137 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.137 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.137 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_host": { 2023-11-27T15:28:10.138 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_host", 2023-11-27T15:28:10.138 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.138 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.138 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.138 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.138 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.138 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.139 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.139 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP server", 2023-11-27T15:28:10.139 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.139 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.139 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.139 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.139 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_password": { 2023-11-27T15:28:10.140 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_password", 2023-11-27T15:28:10.140 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.140 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.140 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.140 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.140 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.140 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.141 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.141 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Password to authenticate with", 2023-11-27T15:28:10.141 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.141 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.141 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.141 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.141 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_port": { 2023-11-27T15:28:10.141 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_port", 2023-11-27T15:28:10.142 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.142 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.142 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.142 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "465", 2023-11-27T15:28:10.142 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.142 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.143 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.143 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP port", 2023-11-27T15:28:10.143 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.143 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.143 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.143 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.143 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_sender": { 2023-11-27T15:28:10.143 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_sender", 2023-11-27T15:28:10.144 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.144 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.144 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.144 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.144 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.144 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.144 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.145 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP envelope sender", 2023-11-27T15:28:10.145 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.145 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.145 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.145 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.145 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_ssl": { 2023-11-27T15:28:10.145 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_ssl", 2023-11-27T15:28:10.146 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.146 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.146 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.146 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.146 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.146 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.147 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.147 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-27T15:28:10.147 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.147 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.147 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.147 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.147 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_user": { 2023-11-27T15:28:10.148 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_user", 2023-11-27T15:28:10.148 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.148 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.148 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.148 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.148 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.148 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.149 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.149 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "User to authenticate as", 2023-11-27T15:28:10.149 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.149 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.149 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.149 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.149 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.150 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.150 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.150 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "balancer", 2023-11-27T15:28:10.150 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.150 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.150 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.150 INFO:teuthology.orchestra.run.smithi052.stdout: "active": { 2023-11-27T15:28:10.151 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "active", 2023-11-27T15:28:10.151 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.151 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.151 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.151 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.151 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.151 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.152 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.152 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "automatically balance PGs across cluster", 2023-11-27T15:28:10.152 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.152 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.152 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.152 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.152 INFO:teuthology.orchestra.run.smithi052.stdout: "begin_time": { 2023-11-27T15:28:10.153 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "begin_time", 2023-11-27T15:28:10.153 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.153 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.153 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.153 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0000", 2023-11-27T15:28:10.153 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.153 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.154 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.154 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "beginning time of day to automatically balance", 2023-11-27T15:28:10.154 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-27T15:28:10.154 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.154 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.154 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.154 INFO:teuthology.orchestra.run.smithi052.stdout: "begin_weekday": { 2023-11-27T15:28:10.155 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "begin_weekday", 2023-11-27T15:28:10.155 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2023-11-27T15:28:10.155 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.155 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.155 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2023-11-27T15:28:10.155 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0", 2023-11-27T15:28:10.156 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "6", 2023-11-27T15:28:10.156 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.156 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-27T15:28:10.156 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-27T15:28:10.156 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.156 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.156 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.157 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_max_iterations": { 2023-11-27T15:28:10.157 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_max_iterations", 2023-11-27T15:28:10.157 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2023-11-27T15:28:10.157 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.157 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.157 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "25", 2023-11-27T15:28:10.157 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2023-11-27T15:28:10.158 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "250", 2023-11-27T15:28:10.158 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.158 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-27T15:28:10.158 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.158 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.158 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.158 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.159 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_metrics": { 2023-11-27T15:28:10.159 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_metrics", 2023-11-27T15:28:10.159 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.159 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.159 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.159 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "pgs,objects,bytes", 2023-11-27T15:28:10.159 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.160 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.160 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.160 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-27T15:28:10.160 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-27T15:28:10.160 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.160 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.160 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.161 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_step": { 2023-11-27T15:28:10.161 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_step", 2023-11-27T15:28:10.161 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2023-11-27T15:28:10.161 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.161 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.161 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0.5", 2023-11-27T15:28:10.161 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0.001", 2023-11-27T15:28:10.162 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "0.999", 2023-11-27T15:28:10.162 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.162 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "aggressiveness of optimization", 2023-11-27T15:28:10.162 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-27T15:28:10.162 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.162 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.163 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.163 INFO:teuthology.orchestra.run.smithi052.stdout: "end_time": { 2023-11-27T15:28:10.163 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "end_time", 2023-11-27T15:28:10.163 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.163 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.163 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.163 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "2359", 2023-11-27T15:28:10.164 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.164 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.164 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.164 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "ending time of day to automatically balance", 2023-11-27T15:28:10.164 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-27T15:28:10.164 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.164 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.164 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.165 INFO:teuthology.orchestra.run.smithi052.stdout: "end_weekday": { 2023-11-27T15:28:10.165 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "end_weekday", 2023-11-27T15:28:10.165 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2023-11-27T15:28:10.165 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.165 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.165 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2023-11-27T15:28:10.165 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0", 2023-11-27T15:28:10.166 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "6", 2023-11-27T15:28:10.166 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.166 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-27T15:28:10.166 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-27T15:28:10.166 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.166 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.167 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.167 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.167 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.167 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.167 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.167 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.167 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.167 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.168 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.168 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.168 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.168 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.168 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.168 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.169 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.169 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.169 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.169 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.169 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.169 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.169 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.169 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.170 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.170 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.170 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.170 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.170 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.170 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.171 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.171 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.171 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.171 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.171 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.171 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.171 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.172 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.172 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.172 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.172 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.172 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.172 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.172 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.172 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.173 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.173 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.173 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.173 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.173 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.173 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.173 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.174 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.174 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.174 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.174 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.174 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.174 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.175 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.175 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.175 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.175 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.175 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.175 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.175 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.176 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.176 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.176 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.176 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.176 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.176 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.176 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.177 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.177 INFO:teuthology.orchestra.run.smithi052.stdout: "min_score": { 2023-11-27T15:28:10.177 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "min_score", 2023-11-27T15:28:10.177 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2023-11-27T15:28:10.177 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.177 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.177 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2023-11-27T15:28:10.178 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.178 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.178 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.178 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-27T15:28:10.178 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.178 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.178 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.179 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.179 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": { 2023-11-27T15:28:10.179 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mode", 2023-11-27T15:28:10.179 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.179 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.179 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.179 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "upmap", 2023-11-27T15:28:10.180 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.180 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.180 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.180 INFO:teuthology.orchestra.run.smithi052.stdout: "crush-compat", 2023-11-27T15:28:10.180 INFO:teuthology.orchestra.run.smithi052.stdout: "none", 2023-11-27T15:28:10.180 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap" 2023-11-27T15:28:10.180 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.181 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Balancer mode", 2023-11-27T15:28:10.181 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.181 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.181 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.181 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.181 INFO:teuthology.orchestra.run.smithi052.stdout: "pool_ids": { 2023-11-27T15:28:10.182 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pool_ids", 2023-11-27T15:28:10.182 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.182 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.182 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.182 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.182 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.182 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.183 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.183 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-27T15:28:10.183 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.183 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.183 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.183 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.183 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2023-11-27T15:28:10.184 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2023-11-27T15:28:10.184 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:10.184 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.184 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.184 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2023-11-27T15:28:10.184 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.184 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.185 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.185 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-27T15:28:10.185 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.185 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.185 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.185 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.185 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap_max_deviation": { 2023-11-27T15:28:10.186 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "upmap_max_deviation", 2023-11-27T15:28:10.186 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.186 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.186 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.186 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2023-11-27T15:28:10.186 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2023-11-27T15:28:10.186 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.187 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.187 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-27T15:28:10.187 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-27T15:28:10.187 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.187 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.187 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.187 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap_max_optimizations": { 2023-11-27T15:28:10.188 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "upmap_max_optimizations", 2023-11-27T15:28:10.188 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2023-11-27T15:28:10.188 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.188 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.188 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2023-11-27T15:28:10.188 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.188 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.189 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.189 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-27T15:28:10.189 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.189 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.189 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.189 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.189 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.190 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.190 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.190 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cephadm", 2023-11-27T15:28:10.190 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.190 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.190 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.191 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_down_multiplier": { 2023-11-27T15:28:10.191 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_down_multiplier", 2023-11-27T15:28:10.191 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2023-11-27T15:28:10.191 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.191 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.191 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3.0", 2023-11-27T15:28:10.191 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.192 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.192 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.192 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-27T15:28:10.192 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.192 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.192 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.192 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.193 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_refresh_rate": { 2023-11-27T15:28:10.193 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_refresh_rate", 2023-11-27T15:28:10.193 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:10.193 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.193 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.193 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "20", 2023-11-27T15:28:10.193 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.194 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.194 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.194 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-27T15:28:10.194 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.194 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.194 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.195 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.195 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_starting_port": { 2023-11-27T15:28:10.195 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_starting_port", 2023-11-27T15:28:10.195 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.195 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.195 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.195 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "4721", 2023-11-27T15:28:10.196 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.196 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.196 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.196 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-27T15:28:10.196 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.196 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.196 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.197 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.197 INFO:teuthology.orchestra.run.smithi052.stdout: "alertmanager_web_password": { 2023-11-27T15:28:10.197 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "alertmanager_web_password", 2023-11-27T15:28:10.197 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.197 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.197 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.197 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2023-11-27T15:28:10.198 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.198 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.198 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.198 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Alertmanager web password", 2023-11-27T15:28:10.198 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.198 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.198 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.199 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.199 INFO:teuthology.orchestra.run.smithi052.stdout: "alertmanager_web_user": { 2023-11-27T15:28:10.199 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "alertmanager_web_user", 2023-11-27T15:28:10.199 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.199 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.199 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.199 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2023-11-27T15:28:10.200 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.200 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.200 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.200 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Alertmanager web user", 2023-11-27T15:28:10.200 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.200 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.200 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.201 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.201 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_ptrace": { 2023-11-27T15:28:10.201 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_ptrace", 2023-11-27T15:28:10.201 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.201 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.201 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.201 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.202 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.202 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.202 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.202 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-27T15:28:10.202 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-27T15:28:10.202 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.202 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.203 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.203 INFO:teuthology.orchestra.run.smithi052.stdout: "autotune_interval": { 2023-11-27T15:28:10.203 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "autotune_interval", 2023-11-27T15:28:10.203 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:10.203 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.203 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.203 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2023-11-27T15:28:10.204 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.204 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.204 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.204 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-27T15:28:10.204 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.204 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.205 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.205 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.205 INFO:teuthology.orchestra.run.smithi052.stdout: "autotune_memory_target_ratio": { 2023-11-27T15:28:10.205 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "autotune_memory_target_ratio", 2023-11-27T15:28:10.205 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2023-11-27T15:28:10.205 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.206 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.206 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0.7", 2023-11-27T15:28:10.206 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.206 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.206 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.206 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-27T15:28:10.206 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.207 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.207 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.207 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.207 INFO:teuthology.orchestra.run.smithi052.stdout: "cgroups_split": { 2023-11-27T15:28:10.207 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cgroups_split", 2023-11-27T15:28:10.207 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.207 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.208 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.208 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.208 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.208 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.208 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.208 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-27T15:28:10.209 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.209 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.209 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.209 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.209 INFO:teuthology.orchestra.run.smithi052.stdout: "config_checks_enabled": { 2023-11-27T15:28:10.209 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_checks_enabled", 2023-11-27T15:28:10.209 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.210 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.210 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.210 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.210 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.210 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.210 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.210 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-27T15:28:10.211 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.211 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.211 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.211 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.211 INFO:teuthology.orchestra.run.smithi052.stdout: "config_dashboard": { 2023-11-27T15:28:10.211 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_dashboard", 2023-11-27T15:28:10.211 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.212 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.212 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.212 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.212 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.212 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.212 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.212 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-27T15:28:10.213 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.213 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.213 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.213 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.213 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_alertmanager": { 2023-11-27T15:28:10.213 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_alertmanager", 2023-11-27T15:28:10.213 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.214 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.214 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.214 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-27T15:28:10.214 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.214 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.214 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.214 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2023-11-27T15:28:10.215 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.215 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.215 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.215 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.215 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_base": { 2023-11-27T15:28:10.215 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_base", 2023-11-27T15:28:10.215 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.216 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.216 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.216 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-27T15:28:10.216 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.216 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.216 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.216 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Container image name, without the tag", 2023-11-27T15:28:10.217 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.217 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.217 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.217 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.217 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_elasticsearch": { 2023-11-27T15:28:10.217 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_elasticsearch", 2023-11-27T15:28:10.217 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.218 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.218 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.218 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-27T15:28:10.218 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.218 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.218 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.219 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "elasticsearch container image", 2023-11-27T15:28:10.219 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.219 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.219 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.219 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.219 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_grafana": { 2023-11-27T15:28:10.219 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_grafana", 2023-11-27T15:28:10.220 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.220 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.220 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.220 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-27T15:28:10.220 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.220 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.220 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.221 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2023-11-27T15:28:10.221 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.221 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.221 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.221 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.221 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_haproxy": { 2023-11-27T15:28:10.221 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_haproxy", 2023-11-27T15:28:10.222 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.222 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.222 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.222 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-27T15:28:10.222 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.222 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.222 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.223 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "HAproxy container image", 2023-11-27T15:28:10.223 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.223 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.223 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.223 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.223 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_agent": { 2023-11-27T15:28:10.223 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_agent", 2023-11-27T15:28:10.224 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.224 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.224 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.224 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-27T15:28:10.224 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.224 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.224 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.225 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger agent container image", 2023-11-27T15:28:10.225 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.225 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.225 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.225 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.225 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_collector": { 2023-11-27T15:28:10.225 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_collector", 2023-11-27T15:28:10.226 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.226 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.226 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.226 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-27T15:28:10.226 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.226 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.226 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.227 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger collector container image", 2023-11-27T15:28:10.227 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.227 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.227 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.227 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.227 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_query": { 2023-11-27T15:28:10.227 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_query", 2023-11-27T15:28:10.228 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.228 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.228 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.228 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-27T15:28:10.228 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.228 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.229 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.229 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger query container image", 2023-11-27T15:28:10.229 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.229 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.229 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.229 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.230 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_keepalived": { 2023-11-27T15:28:10.230 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_keepalived", 2023-11-27T15:28:10.230 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.230 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.230 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.230 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-27T15:28:10.230 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.231 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.231 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.231 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Keepalived container image", 2023-11-27T15:28:10.231 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.231 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.231 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.232 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.232 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_loki": { 2023-11-27T15:28:10.232 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_loki", 2023-11-27T15:28:10.232 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.232 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.232 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.232 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-27T15:28:10.233 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.233 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.233 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.233 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Loki container image", 2023-11-27T15:28:10.233 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.233 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.234 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.234 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.234 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_node_exporter": { 2023-11-27T15:28:10.234 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_node_exporter", 2023-11-27T15:28:10.234 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.234 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.234 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.235 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-27T15:28:10.235 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.235 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.235 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.235 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2023-11-27T15:28:10.235 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.235 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.236 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.236 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.236 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_prometheus": { 2023-11-27T15:28:10.236 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_prometheus", 2023-11-27T15:28:10.236 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.236 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.236 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.237 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-27T15:28:10.237 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.237 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.237 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.237 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2023-11-27T15:28:10.237 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.237 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.238 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.238 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.238 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_promtail": { 2023-11-27T15:28:10.238 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_promtail", 2023-11-27T15:28:10.238 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.238 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.239 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.239 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-27T15:28:10.239 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.239 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.239 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.239 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Promtail container image", 2023-11-27T15:28:10.239 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.239 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.240 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.240 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.240 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_snmp_gateway": { 2023-11-27T15:28:10.240 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_snmp_gateway", 2023-11-27T15:28:10.240 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.240 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.240 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.241 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-27T15:28:10.241 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.241 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.241 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.241 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SNMP Gateway container image", 2023-11-27T15:28:10.241 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.241 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.242 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.242 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.242 INFO:teuthology.orchestra.run.smithi052.stdout: "container_init": { 2023-11-27T15:28:10.242 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_init", 2023-11-27T15:28:10.242 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.242 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.242 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.243 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.243 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.243 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.243 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.243 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Run podman/docker with `--init`", 2023-11-27T15:28:10.243 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.243 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.244 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.244 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.244 INFO:teuthology.orchestra.run.smithi052.stdout: "daemon_cache_timeout": { 2023-11-27T15:28:10.244 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "daemon_cache_timeout", 2023-11-27T15:28:10.244 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:10.244 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.244 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.244 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2023-11-27T15:28:10.245 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.245 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.245 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.245 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-27T15:28:10.245 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.245 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.245 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.246 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.246 INFO:teuthology.orchestra.run.smithi052.stdout: "default_cephadm_command_timeout": { 2023-11-27T15:28:10.246 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "default_cephadm_command_timeout", 2023-11-27T15:28:10.246 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:10.246 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.246 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.246 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "900", 2023-11-27T15:28:10.247 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.247 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.247 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.247 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-27T15:28:10.247 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.247 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.247 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.247 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.248 INFO:teuthology.orchestra.run.smithi052.stdout: "default_registry": { 2023-11-27T15:28:10.248 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "default_registry", 2023-11-27T15:28:10.248 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.248 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.248 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.248 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io", 2023-11-27T15:28:10.248 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.249 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.249 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.249 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-27T15:28:10.249 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.249 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.249 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.249 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.249 INFO:teuthology.orchestra.run.smithi052.stdout: "device_cache_timeout": { 2023-11-27T15:28:10.250 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_cache_timeout", 2023-11-27T15:28:10.250 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:10.250 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.250 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.250 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1800", 2023-11-27T15:28:10.250 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.250 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.251 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.251 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache device inventory", 2023-11-27T15:28:10.251 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.251 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.251 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.251 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.251 INFO:teuthology.orchestra.run.smithi052.stdout: "device_enhanced_scan": { 2023-11-27T15:28:10.252 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_enhanced_scan", 2023-11-27T15:28:10.252 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.252 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.252 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.252 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.252 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.252 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.252 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.253 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-27T15:28:10.253 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.253 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.253 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.253 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.253 INFO:teuthology.orchestra.run.smithi052.stdout: "facts_cache_timeout": { 2023-11-27T15:28:10.253 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "facts_cache_timeout", 2023-11-27T15:28:10.253 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:10.254 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.254 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.254 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2023-11-27T15:28:10.254 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.254 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.254 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.254 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache host facts data", 2023-11-27T15:28:10.255 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.255 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.255 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.255 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.255 INFO:teuthology.orchestra.run.smithi052.stdout: "host_check_interval": { 2023-11-27T15:28:10.255 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "host_check_interval", 2023-11-27T15:28:10.255 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:10.256 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.256 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.256 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2023-11-27T15:28:10.256 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.256 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.256 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.256 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to perform a host check", 2023-11-27T15:28:10.256 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.257 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.257 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.257 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.257 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.257 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.257 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.257 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.257 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.258 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.258 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.258 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.258 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.258 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.258 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.259 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.259 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.259 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.259 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.259 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.259 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.259 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.259 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.260 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.260 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.260 INFO:teuthology.orchestra.run.smithi052.stdout: "log_refresh_metadata": { 2023-11-27T15:28:10.260 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_refresh_metadata", 2023-11-27T15:28:10.260 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.260 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.260 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.261 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.261 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.261 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.261 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.261 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-27T15:28:10.261 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.261 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.261 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.262 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.262 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.262 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.262 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.262 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.262 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.262 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.263 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.263 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.263 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.263 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-27T15:28:10.263 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.263 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.263 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.263 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.264 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.264 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.264 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.264 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.264 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.264 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.264 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.265 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.265 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.265 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.265 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.265 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.265 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.265 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.265 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.266 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.266 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.266 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.266 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.266 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.266 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.266 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.267 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.267 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.267 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.267 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.267 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.267 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.267 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.267 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.268 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.268 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.268 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.268 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.268 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.268 INFO:teuthology.orchestra.run.smithi052.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-27T15:28:10.268 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-27T15:28:10.269 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.269 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.269 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.269 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.269 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.269 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.269 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.270 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-27T15:28:10.270 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.270 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.270 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.270 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.270 INFO:teuthology.orchestra.run.smithi052.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-27T15:28:10.270 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-27T15:28:10.270 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.271 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.271 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.271 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "*", 2023-11-27T15:28:10.271 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.271 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.271 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.271 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-27T15:28:10.272 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.272 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.272 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.272 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.272 INFO:teuthology.orchestra.run.smithi052.stdout: "max_count_per_host": { 2023-11-27T15:28:10.272 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_count_per_host", 2023-11-27T15:28:10.272 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.272 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.273 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.273 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2023-11-27T15:28:10.273 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.273 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.273 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.273 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "max number of daemons per service per host", 2023-11-27T15:28:10.273 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.274 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.274 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.274 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.274 INFO:teuthology.orchestra.run.smithi052.stdout: "max_osd_draining_count": { 2023-11-27T15:28:10.274 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_osd_draining_count", 2023-11-27T15:28:10.274 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.274 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.274 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.275 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2023-11-27T15:28:10.275 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.275 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.275 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.275 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-27T15:28:10.275 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.275 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.276 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.276 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.276 INFO:teuthology.orchestra.run.smithi052.stdout: "migration_current": { 2023-11-27T15:28:10.276 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "migration_current", 2023-11-27T15:28:10.276 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.276 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.276 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.276 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.277 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.277 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.277 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.277 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "internal - do not modify", 2023-11-27T15:28:10.277 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.277 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.277 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.278 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.278 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": { 2023-11-27T15:28:10.278 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mode", 2023-11-27T15:28:10.278 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.278 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.278 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.278 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "root", 2023-11-27T15:28:10.279 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.279 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.279 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.279 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm-package", 2023-11-27T15:28:10.279 INFO:teuthology.orchestra.run.smithi052.stdout: "root" 2023-11-27T15:28:10.279 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.280 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "mode for remote execution of cephadm", 2023-11-27T15:28:10.280 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.280 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.280 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.280 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.280 INFO:teuthology.orchestra.run.smithi052.stdout: "prometheus_alerts_path": { 2023-11-27T15:28:10.280 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus_alerts_path", 2023-11-27T15:28:10.280 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.281 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.281 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.281 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-27T15:28:10.281 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.281 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.281 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.281 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-27T15:28:10.282 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.282 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.282 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.282 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.282 INFO:teuthology.orchestra.run.smithi052.stdout: "prometheus_web_password": { 2023-11-27T15:28:10.282 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus_web_password", 2023-11-27T15:28:10.282 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.282 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.283 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.283 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2023-11-27T15:28:10.283 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.283 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.283 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.283 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus web password", 2023-11-27T15:28:10.283 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.284 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.284 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.284 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.284 INFO:teuthology.orchestra.run.smithi052.stdout: "prometheus_web_user": { 2023-11-27T15:28:10.284 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus_web_user", 2023-11-27T15:28:10.284 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.284 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.285 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.285 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2023-11-27T15:28:10.285 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.285 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.285 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.285 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus web user", 2023-11-27T15:28:10.285 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.285 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.286 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.286 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.286 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_insecure": { 2023-11-27T15:28:10.286 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_insecure", 2023-11-27T15:28:10.286 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.286 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.286 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.287 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.287 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.287 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.287 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.287 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-27T15:28:10.287 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.287 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.288 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.288 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.288 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_password": { 2023-11-27T15:28:10.288 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_password", 2023-11-27T15:28:10.288 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.288 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.288 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.289 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.289 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.289 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.289 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.289 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-27T15:28:10.289 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.289 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.290 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.290 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.290 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_url": { 2023-11-27T15:28:10.290 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_url", 2023-11-27T15:28:10.290 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.290 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.290 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.291 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.291 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.291 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.291 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.291 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-27T15:28:10.291 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.291 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.292 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.292 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.292 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_username": { 2023-11-27T15:28:10.292 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_username", 2023-11-27T15:28:10.292 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.292 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.292 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.293 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.293 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.293 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.293 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.293 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-27T15:28:10.293 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.293 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.294 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.294 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.294 INFO:teuthology.orchestra.run.smithi052.stdout: "secure_monitoring_stack": { 2023-11-27T15:28:10.294 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "secure_monitoring_stack", 2023-11-27T15:28:10.294 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.294 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.294 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.295 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.295 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.295 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.295 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.295 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-27T15:28:10.295 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.295 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.296 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.296 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.296 INFO:teuthology.orchestra.run.smithi052.stdout: "service_discovery_port": { 2023-11-27T15:28:10.296 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "service_discovery_port", 2023-11-27T15:28:10.296 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.296 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.296 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.296 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8765", 2023-11-27T15:28:10.297 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.297 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.297 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.297 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "cephadm service discovery port", 2023-11-27T15:28:10.297 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.297 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.297 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.298 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.298 INFO:teuthology.orchestra.run.smithi052.stdout: "ssh_config_file": { 2023-11-27T15:28:10.298 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssh_config_file", 2023-11-27T15:28:10.298 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.298 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.298 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.298 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.299 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.299 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.299 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.299 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-27T15:28:10.299 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.299 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.299 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.299 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.300 INFO:teuthology.orchestra.run.smithi052.stdout: "use_agent": { 2023-11-27T15:28:10.300 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "use_agent", 2023-11-27T15:28:10.300 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.300 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.300 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.300 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.300 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.300 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.301 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.301 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-27T15:28:10.301 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.301 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.301 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.301 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.302 INFO:teuthology.orchestra.run.smithi052.stdout: "use_repo_digest": { 2023-11-27T15:28:10.302 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "use_repo_digest", 2023-11-27T15:28:10.302 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.302 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.302 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.302 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.302 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.303 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.303 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.303 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-27T15:28:10.303 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.303 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.303 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.303 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.303 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_failed_host_check": { 2023-11-27T15:28:10.304 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_failed_host_check", 2023-11-27T15:28:10.304 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.304 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.304 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.304 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.304 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.304 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.305 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.305 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if the host check fails", 2023-11-27T15:28:10.305 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.305 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.305 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.305 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.305 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_stray_daemons": { 2023-11-27T15:28:10.306 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_stray_daemons", 2023-11-27T15:28:10.306 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.306 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.306 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.306 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.306 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.306 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.306 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.307 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-27T15:28:10.307 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.307 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.307 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.307 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.307 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_stray_hosts": { 2023-11-27T15:28:10.307 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_stray_hosts", 2023-11-27T15:28:10.308 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.308 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.308 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.308 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.308 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.308 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.308 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.309 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-27T15:28:10.309 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.309 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.309 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.309 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.309 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.309 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.309 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.310 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crash", 2023-11-27T15:28:10.310 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.310 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.310 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.310 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.310 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.310 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.311 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.311 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.311 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.311 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.311 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.311 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.311 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.311 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.312 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.312 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.312 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.312 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.312 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.312 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.312 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.313 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.313 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.313 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.313 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.313 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.313 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.313 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.313 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.314 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.314 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.314 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.314 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.314 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.314 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.314 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.315 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.315 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.315 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.315 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.315 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.315 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.315 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.315 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.316 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.316 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.316 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.316 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.316 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.316 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.316 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.317 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.317 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.317 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.317 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.317 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.317 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.317 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.318 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.318 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.318 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.318 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.318 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.318 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.318 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.319 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.319 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.319 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.319 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.319 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.319 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.319 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.319 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.320 INFO:teuthology.orchestra.run.smithi052.stdout: "retain_interval": { 2023-11-27T15:28:10.320 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "retain_interval", 2023-11-27T15:28:10.320 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:10.320 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.320 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.320 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "31536000", 2023-11-27T15:28:10.320 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.321 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.321 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.321 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-27T15:28:10.321 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.321 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.321 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.321 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.321 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_recent_interval": { 2023-11-27T15:28:10.322 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_recent_interval", 2023-11-27T15:28:10.322 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:10.322 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.322 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.322 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1209600", 2023-11-27T15:28:10.322 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.322 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.323 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.323 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-27T15:28:10.323 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.323 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.323 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.323 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.323 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.323 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.324 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.324 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "dashboard", 2023-11-27T15:28:10.324 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.324 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.324 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.324 INFO:teuthology.orchestra.run.smithi052.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-27T15:28:10.324 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-27T15:28:10.324 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.325 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.325 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.325 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2023-11-27T15:28:10.325 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.325 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.325 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.325 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.326 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.326 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.326 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.326 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.326 INFO:teuthology.orchestra.run.smithi052.stdout: "ALERTMANAGER_API_HOST": { 2023-11-27T15:28:10.326 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-27T15:28:10.326 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.327 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.327 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.327 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.327 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.327 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.327 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.327 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.328 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.328 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.328 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.328 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.328 INFO:teuthology.orchestra.run.smithi052.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-27T15:28:10.328 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-27T15:28:10.329 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.329 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.329 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.329 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.329 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.329 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.329 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.330 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.330 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.330 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.330 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.330 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.330 INFO:teuthology.orchestra.run.smithi052.stdout: "AUDIT_API_ENABLED": { 2023-11-27T15:28:10.330 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "AUDIT_API_ENABLED", 2023-11-27T15:28:10.330 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.331 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.331 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.331 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.331 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.331 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.331 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.331 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.332 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.332 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.332 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.332 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.332 INFO:teuthology.orchestra.run.smithi052.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-27T15:28:10.332 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-27T15:28:10.332 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.332 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.333 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.333 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.333 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.333 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.333 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.333 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.333 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.333 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.334 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.334 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.334 INFO:teuthology.orchestra.run.smithi052.stdout: "ENABLE_BROWSABLE_API": { 2023-11-27T15:28:10.334 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-27T15:28:10.334 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.334 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.334 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.335 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.335 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.335 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.335 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.335 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.335 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.335 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.335 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.336 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.336 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-27T15:28:10.336 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-27T15:28:10.336 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.336 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.336 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.336 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.337 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.337 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.337 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.337 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.337 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.337 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.337 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.338 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.338 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-27T15:28:10.338 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-27T15:28:10.338 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.338 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.338 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.339 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.339 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.339 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.339 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.339 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.339 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.339 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.339 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.340 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.340 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-27T15:28:10.340 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-27T15:28:10.340 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.340 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.340 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.340 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.341 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.341 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.341 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.341 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.341 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.341 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.341 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.341 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.342 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-27T15:28:10.342 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-27T15:28:10.342 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.342 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.342 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.342 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.342 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.343 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.343 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.343 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.343 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.343 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.343 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.343 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.344 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-27T15:28:10.344 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-27T15:28:10.344 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.344 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.344 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.344 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.344 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.345 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.345 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.345 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.345 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.345 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.345 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.345 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.346 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-27T15:28:10.346 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-27T15:28:10.346 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.346 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.346 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.346 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.346 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.347 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.347 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.347 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.347 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.347 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.347 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.347 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.348 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-27T15:28:10.348 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-27T15:28:10.348 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.348 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.348 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.348 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.348 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.349 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.349 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.349 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.349 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.349 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.349 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.349 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.349 INFO:teuthology.orchestra.run.smithi052.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-27T15:28:10.350 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-27T15:28:10.350 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.350 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.350 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.350 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.350 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.350 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.351 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.351 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.351 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.351 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.351 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.351 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.351 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_PASSWORD": { 2023-11-27T15:28:10.352 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-27T15:28:10.352 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.352 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.352 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.352 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2023-11-27T15:28:10.352 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.352 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.352 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.353 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.353 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.353 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.353 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.353 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.353 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-27T15:28:10.353 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-27T15:28:10.354 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.354 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.354 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.354 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.354 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.354 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.354 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.354 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.355 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.355 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.355 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.355 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.355 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_URL": { 2023-11-27T15:28:10.355 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_URL", 2023-11-27T15:28:10.355 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.356 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.356 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.356 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.356 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.356 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.356 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.356 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.356 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.357 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.357 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.357 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.357 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_USERNAME": { 2023-11-27T15:28:10.357 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-27T15:28:10.357 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.357 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.358 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.358 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2023-11-27T15:28:10.358 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.358 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.358 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.358 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.359 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.359 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.359 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.359 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.359 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-27T15:28:10.359 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-27T15:28:10.359 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.359 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.360 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.360 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.360 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.360 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.360 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.360 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.360 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.360 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.361 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.361 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.361 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-27T15:28:10.361 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-27T15:28:10.361 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.361 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.361 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.362 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.362 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.362 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.362 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.362 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.362 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.362 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.363 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.363 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.363 INFO:teuthology.orchestra.run.smithi052.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-27T15:28:10.363 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-27T15:28:10.363 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.363 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.363 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.363 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.364 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.364 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.364 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.364 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.364 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.364 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.364 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.364 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.365 INFO:teuthology.orchestra.run.smithi052.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-27T15:28:10.365 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-27T15:28:10.365 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.365 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.365 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.365 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.365 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.366 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.366 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.366 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.366 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.366 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.366 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.366 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.367 INFO:teuthology.orchestra.run.smithi052.stdout: "PROMETHEUS_API_HOST": { 2023-11-27T15:28:10.367 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-27T15:28:10.367 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.367 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.367 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.367 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.367 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.368 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.368 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.368 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.368 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.368 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.368 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.368 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.369 INFO:teuthology.orchestra.run.smithi052.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-27T15:28:10.369 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-27T15:28:10.369 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.369 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.369 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.369 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.369 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.369 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.370 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.370 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.370 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.370 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.370 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.370 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.370 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-27T15:28:10.371 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-27T15:28:10.371 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.371 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.371 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.371 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.371 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.371 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.372 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.372 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.372 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.372 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.372 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.372 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.372 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-27T15:28:10.372 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-27T15:28:10.373 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.373 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.373 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.373 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.373 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.373 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.373 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.374 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.374 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.374 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.374 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.374 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.374 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-27T15:28:10.374 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-27T15:28:10.374 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.375 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.375 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.375 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.375 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.375 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.375 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.375 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.376 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.376 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.376 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.376 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.376 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-27T15:28:10.376 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-27T15:28:10.376 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.376 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.377 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.377 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.377 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.377 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.377 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.377 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.377 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.378 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.378 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.378 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.378 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-27T15:28:10.378 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-27T15:28:10.378 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.378 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.379 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.379 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.379 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.379 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.379 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.379 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.379 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.380 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.380 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.380 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.380 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-27T15:28:10.380 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-27T15:28:10.380 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.380 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.381 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.381 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.381 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.381 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.381 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.381 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.381 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.381 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.382 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.382 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.382 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-27T15:28:10.382 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-27T15:28:10.382 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.382 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.382 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.383 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.383 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.383 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.383 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.383 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.383 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.383 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.384 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.384 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.384 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_ENABLED": { 2023-11-27T15:28:10.384 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-27T15:28:10.384 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.384 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.384 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.384 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.385 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.385 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.385 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.385 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.385 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.385 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.385 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.386 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.386 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-27T15:28:10.386 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-27T15:28:10.386 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.386 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.386 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.386 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-27T15:28:10.387 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.387 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.387 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.387 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.387 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.387 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.387 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.388 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.388 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-27T15:28:10.388 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-27T15:28:10.388 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.388 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.388 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.388 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2023-11-27T15:28:10.389 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.389 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.389 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.389 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.389 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.389 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.389 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.390 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.390 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-27T15:28:10.390 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-27T15:28:10.390 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.390 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.390 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.391 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8", 2023-11-27T15:28:10.391 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.391 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.391 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.391 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.391 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.391 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.392 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.392 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.392 INFO:teuthology.orchestra.run.smithi052.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-27T15:28:10.392 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-27T15:28:10.392 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.392 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.392 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.392 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "45", 2023-11-27T15:28:10.393 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.393 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.393 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.393 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.393 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.393 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.393 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.394 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.394 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_ACCESS_KEY": { 2023-11-27T15:28:10.394 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-27T15:28:10.394 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.394 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.394 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.394 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.395 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.395 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.395 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.395 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.395 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.395 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.395 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.396 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.396 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-27T15:28:10.396 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-27T15:28:10.396 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.396 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.396 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.396 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2023-11-27T15:28:10.397 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.397 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.397 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.397 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.397 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.397 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.397 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.397 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.398 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_SECRET_KEY": { 2023-11-27T15:28:10.398 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-27T15:28:10.398 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.398 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.398 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.398 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.399 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.399 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.399 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.399 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.399 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.399 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.399 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.399 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.400 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_SSL_VERIFY": { 2023-11-27T15:28:10.400 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-27T15:28:10.400 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.400 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.400 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.400 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.400 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.401 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.401 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.401 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.401 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.401 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.401 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.401 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.401 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-27T15:28:10.402 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-27T15:28:10.402 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.402 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.402 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.402 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2023-11-27T15:28:10.402 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.402 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.403 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.403 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.403 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.403 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.403 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.403 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.403 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-27T15:28:10.404 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-27T15:28:10.404 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.404 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.404 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.404 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2023-11-27T15:28:10.404 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.404 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.404 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.405 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.405 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.405 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.405 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.405 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.405 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-27T15:28:10.405 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-27T15:28:10.406 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.406 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.406 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.406 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2023-11-27T15:28:10.406 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.406 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.406 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.406 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.407 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.407 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.407 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.407 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.407 INFO:teuthology.orchestra.run.smithi052.stdout: "cross_origin_url": { 2023-11-27T15:28:10.407 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cross_origin_url", 2023-11-27T15:28:10.407 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.408 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.408 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.408 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.408 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.408 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.408 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.408 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.409 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.409 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.409 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.409 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.409 INFO:teuthology.orchestra.run.smithi052.stdout: "crt_file": { 2023-11-27T15:28:10.409 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crt_file", 2023-11-27T15:28:10.409 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.410 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.410 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.410 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.410 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.410 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.410 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.410 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.410 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.411 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.411 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.411 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.411 INFO:teuthology.orchestra.run.smithi052.stdout: "debug": { 2023-11-27T15:28:10.411 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "debug", 2023-11-27T15:28:10.411 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.411 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.412 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.412 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.412 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.412 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.412 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.412 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable/disable debug options", 2023-11-27T15:28:10.412 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.412 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.413 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.413 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.413 INFO:teuthology.orchestra.run.smithi052.stdout: "jwt_token_ttl": { 2023-11-27T15:28:10.413 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "jwt_token_ttl", 2023-11-27T15:28:10.413 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.413 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.413 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.413 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "28800", 2023-11-27T15:28:10.414 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.414 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.414 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.414 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.414 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.414 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.414 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.415 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.415 INFO:teuthology.orchestra.run.smithi052.stdout: "key_file": { 2023-11-27T15:28:10.415 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "key_file", 2023-11-27T15:28:10.415 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.415 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.415 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.415 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.415 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.416 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.416 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.416 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.416 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.416 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.416 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.416 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.417 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.417 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.417 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.417 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.417 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.417 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.417 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.417 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.418 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.418 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.418 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.418 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.418 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.418 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.418 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.419 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.419 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.419 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.419 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.419 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.419 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.419 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.420 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.420 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.420 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.420 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.420 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.420 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.420 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.420 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.421 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.421 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.421 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.421 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.421 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.421 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.421 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.422 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.422 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.422 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.422 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.422 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.422 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.422 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.423 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.423 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.423 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.423 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.423 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.423 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.423 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.423 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.424 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.424 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.424 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.424 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.424 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.424 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.424 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.425 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.425 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.425 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.425 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.425 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.425 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.425 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.425 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.426 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.426 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.426 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.426 INFO:teuthology.orchestra.run.smithi052.stdout: "motd": { 2023-11-27T15:28:10.426 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "motd", 2023-11-27T15:28:10.426 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.426 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.427 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.427 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.427 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.427 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.427 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.427 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "The message of the day", 2023-11-27T15:28:10.428 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.428 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.428 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.428 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.428 INFO:teuthology.orchestra.run.smithi052.stdout: "redirect_resolve_ip_addr": { 2023-11-27T15:28:10.428 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "redirect_resolve_ip_addr", 2023-11-27T15:28:10.429 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.429 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.429 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.429 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.429 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.429 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.429 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.430 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.430 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.430 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.430 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.430 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.430 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2023-11-27T15:28:10.431 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2023-11-27T15:28:10.431 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.431 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.431 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.431 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "::", 2023-11-27T15:28:10.431 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.431 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.432 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.432 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.432 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.432 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.432 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.432 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.433 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2023-11-27T15:28:10.433 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2023-11-27T15:28:10.433 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.433 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.433 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.433 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8080", 2023-11-27T15:28:10.433 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.433 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.434 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.434 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.434 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.434 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.434 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.434 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.434 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl": { 2023-11-27T15:28:10.435 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl", 2023-11-27T15:28:10.435 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.435 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.435 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.435 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.435 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.435 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.436 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.436 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.436 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.436 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.436 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.436 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.436 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl_server_port": { 2023-11-27T15:28:10.437 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl_server_port", 2023-11-27T15:28:10.437 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.437 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.437 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.437 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8443", 2023-11-27T15:28:10.437 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.438 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.438 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.438 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.438 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.438 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.438 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.438 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.439 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_behaviour": { 2023-11-27T15:28:10.439 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_behaviour", 2023-11-27T15:28:10.439 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.439 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.439 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.439 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "redirect", 2023-11-27T15:28:10.439 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.440 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.440 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.440 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.440 INFO:teuthology.orchestra.run.smithi052.stdout: "redirect" 2023-11-27T15:28:10.440 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.440 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.440 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.441 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.441 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.441 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.441 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_error_status_code": { 2023-11-27T15:28:10.441 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_error_status_code", 2023-11-27T15:28:10.441 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.441 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.441 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.442 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "500", 2023-11-27T15:28:10.442 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "400", 2023-11-27T15:28:10.442 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "599", 2023-11-27T15:28:10.442 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.442 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.442 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.442 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.443 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.443 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.443 INFO:teuthology.orchestra.run.smithi052.stdout: "url_prefix": { 2023-11-27T15:28:10.443 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "url_prefix", 2023-11-27T15:28:10.443 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.443 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.443 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.443 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.444 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.444 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.444 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.444 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.444 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.444 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.444 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.445 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.445 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.445 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.445 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.445 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "devicehealth", 2023-11-27T15:28:10.445 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.445 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.446 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.446 INFO:teuthology.orchestra.run.smithi052.stdout: "enable_monitoring": { 2023-11-27T15:28:10.446 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enable_monitoring", 2023-11-27T15:28:10.446 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.446 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.446 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.446 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.447 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.447 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.447 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.447 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "monitor device health metrics", 2023-11-27T15:28:10.447 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.447 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.447 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.448 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.448 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.448 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.448 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.448 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.448 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.448 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.449 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.449 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.449 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.449 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.449 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.449 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.449 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.450 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.450 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.450 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.450 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.450 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.450 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.450 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.451 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.451 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.451 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.451 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.451 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.451 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.451 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.451 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.452 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.452 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.452 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.452 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.452 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.452 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.452 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.453 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.453 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.453 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.453 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.453 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.453 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.453 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.453 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.454 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.454 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.454 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.454 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.454 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.454 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.454 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.455 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.455 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.455 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.455 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.455 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.455 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.455 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.456 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.456 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.456 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.456 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.456 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.456 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.456 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.456 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.457 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.457 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.457 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.457 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.457 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.457 INFO:teuthology.orchestra.run.smithi052.stdout: "mark_out_threshold": { 2023-11-27T15:28:10.457 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mark_out_threshold", 2023-11-27T15:28:10.458 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:10.458 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.458 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.458 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "2419200", 2023-11-27T15:28:10.458 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.458 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.458 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.459 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-27T15:28:10.459 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.459 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.459 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.459 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.459 INFO:teuthology.orchestra.run.smithi052.stdout: "pool_name": { 2023-11-27T15:28:10.459 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pool_name", 2023-11-27T15:28:10.460 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.460 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.460 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.460 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "device_health_metrics", 2023-11-27T15:28:10.460 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.460 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.460 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.460 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-27T15:28:10.461 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.461 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.461 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.461 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.461 INFO:teuthology.orchestra.run.smithi052.stdout: "retention_period": { 2023-11-27T15:28:10.461 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "retention_period", 2023-11-27T15:28:10.461 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:10.461 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.462 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.462 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15552000", 2023-11-27T15:28:10.462 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.462 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.462 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.462 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long to retain device health metrics", 2023-11-27T15:28:10.462 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.463 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.463 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.463 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.463 INFO:teuthology.orchestra.run.smithi052.stdout: "scrape_frequency": { 2023-11-27T15:28:10.463 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "scrape_frequency", 2023-11-27T15:28:10.463 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:10.463 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.464 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.464 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "86400", 2023-11-27T15:28:10.464 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.464 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.464 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.464 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-27T15:28:10.464 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.464 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.465 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.465 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.465 INFO:teuthology.orchestra.run.smithi052.stdout: "self_heal": { 2023-11-27T15:28:10.465 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "self_heal", 2023-11-27T15:28:10.465 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.465 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.465 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.465 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.466 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.466 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.466 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.466 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-27T15:28:10.466 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.466 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.466 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.467 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.467 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2023-11-27T15:28:10.467 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2023-11-27T15:28:10.467 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:10.467 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.467 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.467 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2023-11-27T15:28:10.468 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.468 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.468 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.468 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to wake up and check device health", 2023-11-27T15:28:10.468 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.468 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.468 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.469 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.469 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_threshold": { 2023-11-27T15:28:10.469 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_threshold", 2023-11-27T15:28:10.469 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:10.469 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.469 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.469 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "7257600", 2023-11-27T15:28:10.469 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.470 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.470 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.470 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-27T15:28:10.470 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.470 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.470 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.470 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.471 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.471 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.471 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.471 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "diskprediction_local", 2023-11-27T15:28:10.471 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.471 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.471 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.471 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.472 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.472 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.472 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.472 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.472 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.472 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.472 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.473 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.473 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.473 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.473 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.473 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.473 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.473 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.473 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.474 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.474 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.474 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.474 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.474 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.474 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.474 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.475 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.475 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.475 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.475 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.475 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.475 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.475 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.475 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.476 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.476 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.476 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.476 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.476 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.476 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.476 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.476 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.477 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.477 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.477 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.477 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.477 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.477 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.477 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.478 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.478 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.478 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.478 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.478 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.478 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.478 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.479 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.479 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.479 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.479 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.479 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.479 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.479 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.480 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.480 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.480 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.480 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.480 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.480 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.480 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.480 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.481 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.481 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.481 INFO:teuthology.orchestra.run.smithi052.stdout: "predict_interval": { 2023-11-27T15:28:10.481 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "predict_interval", 2023-11-27T15:28:10.481 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.481 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.481 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.482 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "86400", 2023-11-27T15:28:10.482 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.482 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.482 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.482 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.482 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.482 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.483 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.483 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.483 INFO:teuthology.orchestra.run.smithi052.stdout: "predictor_model": { 2023-11-27T15:28:10.483 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "predictor_model", 2023-11-27T15:28:10.483 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.483 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.483 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.484 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "prophetstor", 2023-11-27T15:28:10.484 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.484 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.484 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.484 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.484 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.484 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.484 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.485 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.485 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2023-11-27T15:28:10.485 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2023-11-27T15:28:10.485 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.485 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.485 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.485 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2023-11-27T15:28:10.485 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.486 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.486 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.486 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.486 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.486 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.486 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.486 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.487 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.487 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.487 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.487 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "influx", 2023-11-27T15:28:10.487 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": false, 2023-11-27T15:28:10.487 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "influxdb python module not found", 2023-11-27T15:28:10.487 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.488 INFO:teuthology.orchestra.run.smithi052.stdout: "batch_size": { 2023-11-27T15:28:10.488 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "batch_size", 2023-11-27T15:28:10.488 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.488 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.488 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.488 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5000", 2023-11-27T15:28:10.488 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.489 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.489 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.489 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-27T15:28:10.489 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.489 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.489 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.489 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.490 INFO:teuthology.orchestra.run.smithi052.stdout: "database": { 2023-11-27T15:28:10.490 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "database", 2023-11-27T15:28:10.490 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.490 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.490 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.490 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "ceph", 2023-11-27T15:28:10.490 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.491 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.491 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.491 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-27T15:28:10.491 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.491 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.491 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.491 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.492 INFO:teuthology.orchestra.run.smithi052.stdout: "hostname": { 2023-11-27T15:28:10.492 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "hostname", 2023-11-27T15:28:10.492 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.492 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.492 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.492 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.492 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.493 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.493 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.493 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB server hostname", 2023-11-27T15:28:10.493 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.493 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.493 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.493 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.494 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2023-11-27T15:28:10.494 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2023-11-27T15:28:10.494 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:10.494 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.494 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.494 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "30", 2023-11-27T15:28:10.494 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "5", 2023-11-27T15:28:10.495 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.495 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.495 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-27T15:28:10.495 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.495 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.495 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.495 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.495 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.496 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.496 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.496 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.496 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.496 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.496 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.496 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.497 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.497 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.497 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.497 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.497 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.497 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.497 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.498 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.498 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.498 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.498 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.498 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.498 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.498 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.499 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.499 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.499 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.499 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.499 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.499 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.499 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.500 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.500 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.500 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.500 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.500 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.500 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.500 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.501 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.501 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.501 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.501 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.501 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.501 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.501 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.501 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.502 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.502 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.502 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.502 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.502 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.502 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.502 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.503 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.503 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.503 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.503 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.503 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.503 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.503 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.504 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.504 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.504 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.504 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.504 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.504 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.504 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.505 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.505 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.505 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.505 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.505 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.505 INFO:teuthology.orchestra.run.smithi052.stdout: "password": { 2023-11-27T15:28:10.505 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "password", 2023-11-27T15:28:10.506 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.506 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.506 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.506 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.506 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.506 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.506 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.507 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "password of InfluxDB server user", 2023-11-27T15:28:10.507 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.507 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.507 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.507 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.507 INFO:teuthology.orchestra.run.smithi052.stdout: "port": { 2023-11-27T15:28:10.507 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "port", 2023-11-27T15:28:10.508 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.508 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.508 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.508 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8086", 2023-11-27T15:28:10.508 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.508 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.508 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.509 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB server port", 2023-11-27T15:28:10.509 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.509 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.509 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.509 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.509 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl": { 2023-11-27T15:28:10.509 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl", 2023-11-27T15:28:10.510 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.510 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.510 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.510 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "false", 2023-11-27T15:28:10.510 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.510 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.510 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.511 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-27T15:28:10.511 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.511 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.511 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.511 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.511 INFO:teuthology.orchestra.run.smithi052.stdout: "threads": { 2023-11-27T15:28:10.511 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "threads", 2023-11-27T15:28:10.511 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.512 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.512 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.512 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2023-11-27T15:28:10.512 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2023-11-27T15:28:10.512 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "32", 2023-11-27T15:28:10.512 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.512 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-27T15:28:10.513 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.513 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.513 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.513 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.513 INFO:teuthology.orchestra.run.smithi052.stdout: "username": { 2023-11-27T15:28:10.513 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "username", 2023-11-27T15:28:10.513 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.514 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.514 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.514 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.514 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.514 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.514 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.514 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "username of InfluxDB server user", 2023-11-27T15:28:10.515 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.515 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.515 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.515 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.515 INFO:teuthology.orchestra.run.smithi052.stdout: "verify_ssl": { 2023-11-27T15:28:10.515 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "verify_ssl", 2023-11-27T15:28:10.515 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.516 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.516 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.516 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "true", 2023-11-27T15:28:10.516 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.516 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.516 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.516 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-27T15:28:10.517 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.517 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.517 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.517 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.517 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.517 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.517 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.518 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "insights", 2023-11-27T15:28:10.518 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.518 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.518 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.518 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.518 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.518 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.519 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.519 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.519 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.519 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.519 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.519 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.519 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.520 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.520 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.520 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.520 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.520 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.520 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.520 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.521 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.521 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.521 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.521 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.521 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.521 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.521 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.521 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.522 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.522 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.522 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.522 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.522 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.522 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.522 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.523 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.523 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.523 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.523 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.523 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.523 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.523 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.524 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.524 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.524 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.524 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.524 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.524 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.524 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.524 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.525 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.525 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.525 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.525 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.525 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.525 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.525 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.526 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.526 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.526 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.526 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.526 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.526 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.526 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.527 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.527 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.527 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.527 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.527 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.527 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.527 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.528 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.528 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.528 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.528 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.528 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.528 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "iostat", 2023-11-27T15:28:10.529 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.529 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.529 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.529 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.529 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.529 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.529 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.530 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.530 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.530 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.530 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.530 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.530 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.530 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.531 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.531 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.531 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.531 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.531 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.531 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.531 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.532 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.532 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.532 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.532 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.532 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.532 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.532 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.533 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.533 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.533 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.533 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.533 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.533 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.533 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.534 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.534 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.534 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.534 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.534 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.534 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.535 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.535 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.535 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.535 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.535 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.535 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.535 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.536 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.536 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.536 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.536 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.536 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.536 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.536 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.537 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.537 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.537 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.537 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.537 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.537 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.538 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.538 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.538 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.538 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.538 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.538 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.538 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.539 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.539 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.539 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.539 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.539 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.539 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.539 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.540 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.540 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "k8sevents", 2023-11-27T15:28:10.540 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.540 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.540 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.540 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_event_retention_days": { 2023-11-27T15:28:10.541 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ceph_event_retention_days", 2023-11-27T15:28:10.541 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.541 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.541 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.541 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "7", 2023-11-27T15:28:10.541 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.541 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.542 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.542 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Days to hold ceph event information within local cache", 2023-11-27T15:28:10.542 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.542 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.542 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.542 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.543 INFO:teuthology.orchestra.run.smithi052.stdout: "config_check_secs": { 2023-11-27T15:28:10.543 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_check_secs", 2023-11-27T15:28:10.543 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.543 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.543 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.543 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2023-11-27T15:28:10.543 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "10", 2023-11-27T15:28:10.544 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.544 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.544 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-11-27T15:28:10.544 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.544 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.544 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.544 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.545 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.545 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.545 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.545 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.545 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.545 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.546 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.546 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.546 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.546 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.546 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.546 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.546 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.547 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.547 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.547 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.547 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.547 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.547 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.547 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.548 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.548 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.548 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.548 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.548 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.548 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.549 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.549 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.549 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.549 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.549 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.549 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.549 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.549 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.550 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.550 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.550 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.550 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.550 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.550 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.551 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.551 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.551 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.551 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.551 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.551 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.551 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.552 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.552 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.552 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.552 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.552 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.552 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.552 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.553 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.553 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.553 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.553 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.553 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.553 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.553 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.554 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.554 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.554 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.554 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.554 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.554 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.554 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.555 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.555 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.555 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.555 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.555 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.555 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "localpool", 2023-11-27T15:28:10.555 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.556 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.556 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.556 INFO:teuthology.orchestra.run.smithi052.stdout: "failure_domain": { 2023-11-27T15:28:10.556 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "failure_domain", 2023-11-27T15:28:10.556 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.556 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.556 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.557 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "host", 2023-11-27T15:28:10.557 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.557 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.557 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.557 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "failure domain for any created local pool", 2023-11-27T15:28:10.557 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-27T15:28:10.557 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.558 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.558 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.558 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.558 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.558 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.558 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.558 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.559 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.559 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.559 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.559 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.559 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.559 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.559 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.560 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.560 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.560 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.560 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.560 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.560 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.560 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.560 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.561 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.561 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.561 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.561 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.561 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.561 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.561 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.562 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.562 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.562 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.562 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.562 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.562 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.562 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.563 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.563 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.563 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.563 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.563 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.563 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.563 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.564 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.564 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.564 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.564 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.564 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.564 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.564 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.564 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.565 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.565 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.565 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.565 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.565 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.565 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.565 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.566 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.566 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.566 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.566 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.566 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.566 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.566 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.567 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.567 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.567 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.567 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.567 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.567 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.567 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.568 INFO:teuthology.orchestra.run.smithi052.stdout: "min_size": { 2023-11-27T15:28:10.568 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "min_size", 2023-11-27T15:28:10.568 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.568 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.568 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.568 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.568 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.569 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.569 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.569 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default min_size for any created local pool", 2023-11-27T15:28:10.569 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-27T15:28:10.569 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.569 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.569 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.570 INFO:teuthology.orchestra.run.smithi052.stdout: "num_rep": { 2023-11-27T15:28:10.570 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "num_rep", 2023-11-27T15:28:10.570 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.570 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.570 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.570 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3", 2023-11-27T15:28:10.570 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.571 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.571 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.571 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default replica count for any created local pool", 2023-11-27T15:28:10.571 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.571 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.571 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.571 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.571 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_num": { 2023-11-27T15:28:10.572 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pg_num", 2023-11-27T15:28:10.572 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.572 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.572 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.572 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "128", 2023-11-27T15:28:10.572 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.572 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.573 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.573 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default pg_num for any created local pool", 2023-11-27T15:28:10.573 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.573 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.573 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.573 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.573 INFO:teuthology.orchestra.run.smithi052.stdout: "prefix": { 2023-11-27T15:28:10.574 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prefix", 2023-11-27T15:28:10.574 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.574 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.574 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.574 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.574 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.574 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.575 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.575 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "name prefix for any created local pool", 2023-11-27T15:28:10.575 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.575 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.575 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.575 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.575 INFO:teuthology.orchestra.run.smithi052.stdout: "subtree": { 2023-11-27T15:28:10.575 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "subtree", 2023-11-27T15:28:10.576 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.576 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.576 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.576 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "rack", 2023-11-27T15:28:10.576 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.576 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.576 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.577 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-27T15:28:10.577 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-27T15:28:10.577 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.577 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.577 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.577 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.577 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.578 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.578 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mds_autoscaler", 2023-11-27T15:28:10.578 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.578 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.578 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.578 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.579 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.579 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.579 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.579 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.579 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.579 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.579 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.579 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.580 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.580 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.580 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.580 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.580 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.580 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.580 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.581 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.581 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.581 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.581 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.581 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.581 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.581 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.582 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.582 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.582 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.582 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.582 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.582 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.582 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.583 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.583 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.583 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.583 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.583 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.583 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.583 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.584 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.584 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.584 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.584 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.584 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.584 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.584 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.584 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.585 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.585 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.585 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.585 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.585 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.585 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.585 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.586 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.586 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.586 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.586 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.586 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.586 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.586 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.587 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.587 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.587 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.587 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.587 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.587 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.587 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.588 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.588 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.588 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.588 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.588 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.589 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.589 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.589 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mirroring", 2023-11-27T15:28:10.589 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.589 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.589 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.589 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.590 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.590 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.590 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.590 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.590 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.590 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.591 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.591 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.591 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.591 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.591 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.591 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.591 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.592 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.592 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.592 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.592 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.592 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.592 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.592 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.593 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.593 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.593 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.593 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.593 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.593 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.593 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.594 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.594 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.594 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.594 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.594 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.594 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.595 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.595 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.595 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.595 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.595 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.595 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.595 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.596 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.596 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.596 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.596 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.596 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.596 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.596 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.597 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.597 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.597 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.597 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.597 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.597 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.597 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.598 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.598 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.598 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.598 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.598 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.598 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.598 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.599 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.599 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.599 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.599 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.599 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.599 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.599 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.600 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.600 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.600 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.600 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.600 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "nfs", 2023-11-27T15:28:10.600 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.600 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.601 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.601 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.601 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.601 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.601 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.601 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.601 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.602 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.602 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.602 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.602 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.602 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.602 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.602 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.603 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.603 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.603 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.603 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.603 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.603 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.603 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.603 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.604 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.604 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.604 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.604 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.604 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.604 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.604 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.605 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.605 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.605 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.605 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.605 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.605 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.605 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.606 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.606 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.606 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.606 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.606 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.606 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.606 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.607 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.607 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.607 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.607 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.607 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.607 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.607 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.608 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.608 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.608 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.608 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.608 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.608 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.608 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.609 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.609 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.609 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.609 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.609 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.609 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.609 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.610 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.610 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.610 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.610 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.610 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.610 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.610 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.611 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.611 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.611 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.611 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "orchestrator", 2023-11-27T15:28:10.611 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.611 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.611 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.611 INFO:teuthology.orchestra.run.smithi052.stdout: "fail_fs": { 2023-11-27T15:28:10.612 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "fail_fs", 2023-11-27T15:28:10.612 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.612 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.612 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.612 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.612 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.612 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.613 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.613 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-27T15:28:10.613 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.613 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.613 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.613 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.613 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.614 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.614 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.614 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.614 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.614 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.614 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.614 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.615 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.615 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.615 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.615 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.615 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.615 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.615 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.616 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.616 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.616 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.616 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.616 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.616 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.616 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.617 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.617 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.617 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.617 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.617 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.618 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.618 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.618 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.618 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.618 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.618 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.618 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.618 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.619 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.619 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.619 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.619 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.619 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.619 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.619 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.619 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.620 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.620 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.620 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.620 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.620 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.620 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.620 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.620 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.620 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.621 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.621 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.621 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.621 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.621 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.621 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.621 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.621 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.622 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.622 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.622 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.622 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.622 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.622 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.622 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.622 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.623 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.623 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.623 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator": { 2023-11-27T15:28:10.623 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "orchestrator", 2023-11-27T15:28:10.623 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.623 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.623 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.623 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.624 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.624 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.624 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.624 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm", 2023-11-27T15:28:10.624 INFO:teuthology.orchestra.run.smithi052.stdout: "rook", 2023-11-27T15:28:10.624 INFO:teuthology.orchestra.run.smithi052.stdout: "test_orchestrator" 2023-11-27T15:28:10.624 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.624 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Orchestrator backend", 2023-11-27T15:28:10.624 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.625 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.625 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.625 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.625 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.625 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.625 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.625 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "osd_perf_query", 2023-11-27T15:28:10.625 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.626 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.626 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.626 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.626 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.626 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.626 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.626 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.626 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.627 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.627 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.627 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.627 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.627 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.627 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.627 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.628 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.628 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.628 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.628 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.628 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.628 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.628 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.628 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.629 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.629 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.629 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.629 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.629 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.629 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.629 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.629 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.630 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.630 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.630 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.630 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.630 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.630 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.630 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.630 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.631 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.631 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.631 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.631 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.631 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.631 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.631 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.631 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.632 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.632 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.632 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.632 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.632 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.632 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.632 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.632 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.633 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.633 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.633 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.633 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.633 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.633 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.633 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.634 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.634 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.634 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.634 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.634 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.634 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.634 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.634 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.635 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.635 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.635 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.635 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.635 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.635 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "osd_support", 2023-11-27T15:28:10.635 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.635 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.636 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.636 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.636 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.636 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.636 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.636 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.636 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.637 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.637 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.637 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.637 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.637 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.637 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.637 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.637 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.638 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.638 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.638 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.638 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.638 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.638 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.638 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.638 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.639 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.639 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.639 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.639 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.639 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.639 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.639 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.639 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.640 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.640 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.640 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.640 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.640 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.640 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.640 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.641 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.641 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.641 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.641 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.641 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.641 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.641 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.641 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.642 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.642 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.642 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.642 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.642 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.642 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.642 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.643 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.643 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.643 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.643 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.643 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.643 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.643 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.643 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.644 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.644 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.644 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.644 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.644 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.644 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.644 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.644 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.644 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.645 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.645 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.645 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.645 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.645 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pg_autoscaler", 2023-11-27T15:28:10.645 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.645 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.645 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.646 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.646 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.646 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.646 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.646 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.646 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.646 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.647 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.647 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.647 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.647 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.647 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.647 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.647 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.647 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.648 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.648 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.648 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.648 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.648 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.648 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.648 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.648 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.649 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.649 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.649 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.649 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.649 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.649 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.649 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.649 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.650 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.650 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.650 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.650 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.650 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.650 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.650 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.650 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.651 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.651 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.651 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.651 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.651 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.651 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.651 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.651 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.652 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.652 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.652 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.652 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.652 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.652 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.652 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.652 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.653 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.653 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.653 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.653 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.653 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.653 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.653 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.653 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.653 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.654 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.654 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.654 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.654 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.654 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.654 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.654 INFO:teuthology.orchestra.run.smithi052.stdout: "noautoscale": { 2023-11-27T15:28:10.655 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "noautoscale", 2023-11-27T15:28:10.655 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.655 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.655 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.655 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.655 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.655 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.655 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.656 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "global autoscale flag", 2023-11-27T15:28:10.656 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-11-27T15:28:10.656 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.656 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.656 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.656 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2023-11-27T15:28:10.656 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2023-11-27T15:28:10.656 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:10.657 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.657 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.657 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2023-11-27T15:28:10.657 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.657 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.657 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.657 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.657 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.658 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.658 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.658 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.658 INFO:teuthology.orchestra.run.smithi052.stdout: "threshold": { 2023-11-27T15:28:10.658 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "threshold", 2023-11-27T15:28:10.658 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2023-11-27T15:28:10.658 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.658 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.659 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3.0", 2023-11-27T15:28:10.659 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1.0", 2023-11-27T15:28:10.659 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.659 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.659 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "scaling threshold", 2023-11-27T15:28:10.659 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-27T15:28:10.659 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.659 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.660 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.660 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.660 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.660 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.660 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "progress", 2023-11-27T15:28:10.660 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.660 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.660 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.661 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_pg_recovery_event": { 2023-11-27T15:28:10.661 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_pg_recovery_event", 2023-11-27T15:28:10.661 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.661 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.661 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.661 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.661 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.661 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.662 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.662 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-27T15:28:10.662 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.662 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.662 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.662 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.662 INFO:teuthology.orchestra.run.smithi052.stdout: "enabled": { 2023-11-27T15:28:10.663 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enabled", 2023-11-27T15:28:10.663 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.663 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.663 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.663 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.663 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.663 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.663 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.664 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.664 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.664 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.664 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.664 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.664 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.664 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.664 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.664 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.665 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.665 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.665 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.665 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.665 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.665 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.665 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.666 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.666 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.666 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.666 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.666 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.666 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.666 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.667 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.667 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.667 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.667 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.667 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.667 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.667 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.667 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.668 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.668 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.668 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.668 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.668 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.668 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.668 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.668 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.669 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.669 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.669 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.669 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.669 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.669 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.669 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.669 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.670 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.670 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.670 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.670 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.670 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.670 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.670 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.671 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.671 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.671 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.671 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.671 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.671 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.671 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.671 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.672 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.672 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.672 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.672 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.672 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.672 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.672 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.672 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.673 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.673 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.673 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.673 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.673 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.673 INFO:teuthology.orchestra.run.smithi052.stdout: "max_completed_events": { 2023-11-27T15:28:10.673 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_completed_events", 2023-11-27T15:28:10.673 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.674 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.674 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.674 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "50", 2023-11-27T15:28:10.674 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.674 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.674 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.674 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "number of past completed events to remember", 2023-11-27T15:28:10.674 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.675 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.675 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.675 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.675 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2023-11-27T15:28:10.675 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2023-11-27T15:28:10.675 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:10.675 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.675 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.676 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2023-11-27T15:28:10.676 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.676 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.676 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.676 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long the module is going to sleep", 2023-11-27T15:28:10.676 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.676 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.676 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.677 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.677 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.677 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.677 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.677 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus", 2023-11-27T15:28:10.677 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.677 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.677 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.678 INFO:teuthology.orchestra.run.smithi052.stdout: "cache": { 2023-11-27T15:28:10.678 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cache", 2023-11-27T15:28:10.678 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.678 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.678 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.678 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.678 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.679 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.679 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.679 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.679 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.679 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.679 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.679 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.679 INFO:teuthology.orchestra.run.smithi052.stdout: "exclude_perf_counters": { 2023-11-27T15:28:10.680 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "exclude_perf_counters", 2023-11-27T15:28:10.680 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.680 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.680 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.680 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.680 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.680 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.680 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.680 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-27T15:28:10.681 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-27T15:28:10.681 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.681 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.681 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.681 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.681 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.681 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.682 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.682 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.682 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.682 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.682 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.682 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.682 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.682 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.683 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.683 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.683 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.683 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.683 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.683 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.683 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.684 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.684 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.684 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.684 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.684 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.684 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.684 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.684 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.685 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.685 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.685 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.685 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.685 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.685 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.685 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.685 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.686 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.686 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.686 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.686 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.686 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.686 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.686 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.686 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.687 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.687 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.687 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.687 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.687 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.687 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.687 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.687 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.688 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.688 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.688 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.688 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.688 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.688 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.688 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.688 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.689 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.689 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.689 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.689 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.689 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.689 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.689 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.689 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.690 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.690 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.690 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.690 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.690 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_stats_pools": { 2023-11-27T15:28:10.690 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_stats_pools", 2023-11-27T15:28:10.690 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.690 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.691 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.691 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.691 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.691 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.691 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.691 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.691 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.691 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.692 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.692 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.692 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-27T15:28:10.692 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-27T15:28:10.692 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.692 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.692 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.693 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "300", 2023-11-27T15:28:10.693 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.693 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.693 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.693 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.693 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.693 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.693 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.694 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.694 INFO:teuthology.orchestra.run.smithi052.stdout: "scrape_interval": { 2023-11-27T15:28:10.694 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "scrape_interval", 2023-11-27T15:28:10.694 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2023-11-27T15:28:10.694 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.694 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.694 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15.0", 2023-11-27T15:28:10.695 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.695 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.695 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.695 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.695 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.695 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.695 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.695 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.696 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2023-11-27T15:28:10.696 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2023-11-27T15:28:10.696 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.696 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.696 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.696 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "::", 2023-11-27T15:28:10.696 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.696 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.697 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.697 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-27T15:28:10.697 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.697 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.697 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.697 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.697 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2023-11-27T15:28:10.698 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2023-11-27T15:28:10.698 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.698 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.698 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.698 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "9283", 2023-11-27T15:28:10.698 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.698 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.698 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.699 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-27T15:28:10.699 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.699 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.699 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.699 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.699 INFO:teuthology.orchestra.run.smithi052.stdout: "stale_cache_strategy": { 2023-11-27T15:28:10.699 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "stale_cache_strategy", 2023-11-27T15:28:10.699 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.700 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.700 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.700 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "log", 2023-11-27T15:28:10.700 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.700 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.700 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.700 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.700 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.701 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.701 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.701 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.701 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_behaviour": { 2023-11-27T15:28:10.701 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_behaviour", 2023-11-27T15:28:10.701 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.701 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.701 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.702 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "default", 2023-11-27T15:28:10.702 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.702 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.702 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.702 INFO:teuthology.orchestra.run.smithi052.stdout: "default", 2023-11-27T15:28:10.702 INFO:teuthology.orchestra.run.smithi052.stdout: "error" 2023-11-27T15:28:10.702 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.702 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.703 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.703 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.703 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.703 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.703 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_error_status_code": { 2023-11-27T15:28:10.703 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_error_status_code", 2023-11-27T15:28:10.703 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.703 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.704 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.704 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "500", 2023-11-27T15:28:10.704 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "400", 2023-11-27T15:28:10.704 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "599", 2023-11-27T15:28:10.704 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.704 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.704 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.704 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.705 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.705 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.705 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.705 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.705 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.705 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_support", 2023-11-27T15:28:10.705 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.705 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.705 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.706 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.706 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.706 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.706 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.706 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.706 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.706 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.707 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.707 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.707 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.707 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.707 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.707 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.707 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.707 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.708 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.708 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.708 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.708 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.708 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.708 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.708 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.708 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.709 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.709 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.709 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.709 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.709 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.709 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.709 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.709 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.710 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.710 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.710 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.710 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.710 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.710 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.710 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.710 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.711 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.711 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.711 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.711 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.711 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.711 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.711 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.711 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.712 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.712 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.712 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.712 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.712 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.712 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.712 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.712 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.713 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.713 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.713 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.713 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.713 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.713 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.713 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.714 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.714 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.714 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.714 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.714 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.714 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.714 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.714 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.715 INFO:teuthology.orchestra.run.smithi052.stdout: "max_concurrent_snap_create": { 2023-11-27T15:28:10.715 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_concurrent_snap_create", 2023-11-27T15:28:10.715 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.715 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.715 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.715 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2023-11-27T15:28:10.715 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.715 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.715 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.716 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.716 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.716 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.716 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.716 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.716 INFO:teuthology.orchestra.run.smithi052.stdout: "mirror_snapshot_schedule": { 2023-11-27T15:28:10.716 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mirror_snapshot_schedule", 2023-11-27T15:28:10.717 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.717 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.717 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.717 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.717 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.717 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.717 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.717 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.718 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.718 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.718 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.718 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.718 INFO:teuthology.orchestra.run.smithi052.stdout: "trash_purge_schedule": { 2023-11-27T15:28:10.718 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "trash_purge_schedule", 2023-11-27T15:28:10.718 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.719 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.719 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.719 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.719 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.719 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.719 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.719 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.719 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.720 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.720 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.720 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.720 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.720 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.720 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.720 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "restful", 2023-11-27T15:28:10.720 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.721 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.721 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.721 INFO:teuthology.orchestra.run.smithi052.stdout: "enable_auth": { 2023-11-27T15:28:10.721 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enable_auth", 2023-11-27T15:28:10.721 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.721 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.721 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.721 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.721 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.722 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.722 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.722 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.722 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.722 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.722 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.722 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.723 INFO:teuthology.orchestra.run.smithi052.stdout: "key_file": { 2023-11-27T15:28:10.723 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "key_file", 2023-11-27T15:28:10.723 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.723 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.723 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.723 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.723 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.723 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.723 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.724 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.724 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.724 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.724 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.724 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.724 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.724 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.725 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.725 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.725 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.725 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.725 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.725 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.725 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.725 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.725 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.726 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.726 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.726 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.726 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.726 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.726 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.726 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.727 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.727 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.727 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.727 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.727 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.727 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.727 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.728 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.728 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.728 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.728 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.728 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.728 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.728 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.729 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.729 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.729 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.729 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.729 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.729 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.729 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.729 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.730 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.730 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.730 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.730 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.730 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.730 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.730 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.730 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.731 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.731 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.731 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.731 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.731 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.731 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.731 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.731 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.732 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.732 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.732 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.732 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.732 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.732 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.732 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.732 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.733 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.733 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.733 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.733 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.733 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.733 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.733 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2023-11-27T15:28:10.733 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2023-11-27T15:28:10.734 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.734 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.734 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.734 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.734 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.734 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.734 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.734 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.735 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.735 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.735 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.735 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.735 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2023-11-27T15:28:10.735 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2023-11-27T15:28:10.735 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.735 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.736 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.736 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.736 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.736 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.736 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.736 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.736 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.737 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.737 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.737 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.737 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.737 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.737 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.737 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rgw", 2023-11-27T15:28:10.737 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.737 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.738 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.738 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.738 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.738 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.738 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.738 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.738 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.739 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.739 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.739 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.739 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.739 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.739 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.739 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.739 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.740 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.740 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.740 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.740 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.740 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.740 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.740 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.740 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.740 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.741 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.741 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.741 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.741 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.741 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.741 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.741 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.742 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.742 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.742 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.742 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.742 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.742 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.742 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.742 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.743 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.743 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.743 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.743 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.743 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.743 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.743 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.743 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.744 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.744 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.744 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.744 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.744 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.744 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.744 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.744 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.745 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.745 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.745 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.745 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.745 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.745 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.745 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.745 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.745 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.746 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.746 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.746 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.746 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.746 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.746 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.746 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.746 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.747 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.747 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.747 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rook", 2023-11-27T15:28:10.747 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.747 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.747 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.747 INFO:teuthology.orchestra.run.smithi052.stdout: "drive_group_interval": { 2023-11-27T15:28:10.747 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "drive_group_interval", 2023-11-27T15:28:10.748 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2023-11-27T15:28:10.748 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.748 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.748 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "300.0", 2023-11-27T15:28:10.748 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.748 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.748 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.748 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-11-27T15:28:10.749 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.749 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.749 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.749 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.749 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.749 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.749 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.749 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.750 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.750 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.750 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.750 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.750 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.750 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.750 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.751 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.751 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.751 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.751 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.751 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.751 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.751 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.751 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.752 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.752 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.752 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.752 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.752 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.752 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.752 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.752 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.752 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.753 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.753 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.753 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.753 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.753 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.753 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.753 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.753 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.754 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.754 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.754 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.754 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.754 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.754 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.754 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.755 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.755 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.755 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.755 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.755 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.755 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.755 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.755 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.756 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.756 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.756 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.756 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.756 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.756 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.756 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.756 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.756 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.757 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.757 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.757 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.757 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.757 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.757 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.757 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.758 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.758 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.758 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.758 INFO:teuthology.orchestra.run.smithi052.stdout: "storage_class": { 2023-11-27T15:28:10.758 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "storage_class", 2023-11-27T15:28:10.758 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.758 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.758 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.759 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "local", 2023-11-27T15:28:10.759 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.759 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.759 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.759 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-11-27T15:28:10.759 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.759 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.759 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.760 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.760 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.760 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.760 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.760 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "selftest", 2023-11-27T15:28:10.760 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.760 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.760 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.761 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.761 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.761 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.761 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.761 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.761 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.761 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.761 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.762 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.762 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.762 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.762 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.762 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.762 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.762 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.762 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.763 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.763 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.763 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.763 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.763 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.763 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.763 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.764 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.764 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.764 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.764 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.764 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.764 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.764 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.764 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.765 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.765 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.765 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.765 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.765 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.765 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.765 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.765 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.766 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.766 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.766 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.766 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.766 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.766 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.766 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.767 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.767 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.767 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.767 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.767 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.767 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.767 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.767 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.768 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.768 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.768 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.768 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.768 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.768 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.768 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.768 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.769 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.769 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.769 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.769 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.769 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.769 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.769 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.769 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.770 INFO:teuthology.orchestra.run.smithi052.stdout: "roption1": { 2023-11-27T15:28:10.770 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "roption1", 2023-11-27T15:28:10.770 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.770 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.770 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.770 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.770 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.770 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.771 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.771 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.771 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.771 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.771 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.771 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.771 INFO:teuthology.orchestra.run.smithi052.stdout: "roption2": { 2023-11-27T15:28:10.771 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "roption2", 2023-11-27T15:28:10.772 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.772 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.772 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.772 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "xyz", 2023-11-27T15:28:10.772 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.772 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.772 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.772 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.773 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.773 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.773 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.773 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.773 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption1": { 2023-11-27T15:28:10.773 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption1", 2023-11-27T15:28:10.773 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.773 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.774 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.774 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.774 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.774 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.774 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.774 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.774 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.774 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.775 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.775 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.775 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption2": { 2023-11-27T15:28:10.775 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption2", 2023-11-27T15:28:10.775 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.775 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.775 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.775 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.776 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.776 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.776 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.776 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.776 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.776 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.776 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.776 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.776 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption3": { 2023-11-27T15:28:10.777 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption3", 2023-11-27T15:28:10.777 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2023-11-27T15:28:10.777 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.777 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.777 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.777 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.777 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.778 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.778 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.778 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.778 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.778 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.778 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.778 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption4": { 2023-11-27T15:28:10.778 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption4", 2023-11-27T15:28:10.779 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.779 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.779 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.779 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.779 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.779 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.779 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.779 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.780 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.780 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.780 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.780 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.780 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption5": { 2023-11-27T15:28:10.780 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption5", 2023-11-27T15:28:10.780 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.781 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.781 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.781 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.781 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.781 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.781 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.781 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.781 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.781 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.782 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.782 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.782 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption6": { 2023-11-27T15:28:10.782 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption6", 2023-11-27T15:28:10.782 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.782 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.782 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.782 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.783 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.783 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.783 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.783 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.783 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.783 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.783 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.783 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.784 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption7": { 2023-11-27T15:28:10.784 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption7", 2023-11-27T15:28:10.784 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.784 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.784 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.784 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.784 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2023-11-27T15:28:10.784 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "42", 2023-11-27T15:28:10.785 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.785 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.785 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.785 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.785 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.785 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.785 INFO:teuthology.orchestra.run.smithi052.stdout: "testkey": { 2023-11-27T15:28:10.785 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testkey", 2023-11-27T15:28:10.786 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.786 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.786 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.786 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.786 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.786 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.786 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.787 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.787 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.787 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.787 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.787 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.787 INFO:teuthology.orchestra.run.smithi052.stdout: "testlkey": { 2023-11-27T15:28:10.787 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testlkey", 2023-11-27T15:28:10.787 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.788 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.788 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.788 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.788 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.788 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.788 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.788 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.788 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.788 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.789 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.789 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.789 INFO:teuthology.orchestra.run.smithi052.stdout: "testnewline": { 2023-11-27T15:28:10.789 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testnewline", 2023-11-27T15:28:10.789 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.789 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.789 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.789 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.790 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.790 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.790 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.790 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.790 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.790 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.790 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.791 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.791 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.791 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.791 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.791 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snap_schedule", 2023-11-27T15:28:10.791 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.791 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.791 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.792 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_m_granularity": { 2023-11-27T15:28:10.792 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_m_granularity", 2023-11-27T15:28:10.792 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.792 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.792 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.792 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.792 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.792 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.793 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.793 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow minute scheduled snapshots", 2023-11-27T15:28:10.793 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.793 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.793 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.793 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.793 INFO:teuthology.orchestra.run.smithi052.stdout: "dump_on_update": { 2023-11-27T15:28:10.794 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "dump_on_update", 2023-11-27T15:28:10.794 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.794 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.794 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.794 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.794 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.794 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.794 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.795 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "dump database to debug log on update", 2023-11-27T15:28:10.795 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.795 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.795 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.795 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.795 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.795 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.795 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.796 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.796 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.796 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.796 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.796 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.796 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.796 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.796 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.797 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.797 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.797 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.797 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.797 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.797 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.797 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.797 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.798 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.798 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.798 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.798 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.798 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.798 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.798 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.799 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.799 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.799 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.799 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.799 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.799 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.799 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.799 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.800 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.800 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.800 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.800 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.800 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.800 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.800 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.800 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.801 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.801 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.801 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.801 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.801 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.801 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.801 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.801 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.802 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.802 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.802 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.802 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.802 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.802 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.802 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.802 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.803 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.803 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.803 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.803 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.803 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.803 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.803 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.803 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.804 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.804 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.804 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.804 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.804 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.804 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.804 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.804 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "stats", 2023-11-27T15:28:10.805 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.805 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.805 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.805 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.805 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.805 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.805 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.805 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.806 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.806 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.806 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.806 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.806 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.806 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.806 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.806 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.807 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.807 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.807 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.807 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.807 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.807 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.807 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.807 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.807 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.808 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.808 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.808 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.808 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.808 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.808 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.808 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.808 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.809 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.809 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.809 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.809 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.809 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.809 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.809 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.809 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.810 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.810 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.810 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.810 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.810 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.810 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.810 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.810 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.811 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.811 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.811 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.811 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.811 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.811 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.811 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.811 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.812 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.812 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.812 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.812 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.812 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.812 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.812 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.812 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.813 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.813 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.813 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.813 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.813 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.813 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.813 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.813 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.814 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.814 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.814 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.814 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "status", 2023-11-27T15:28:10.814 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.814 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.814 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.814 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.815 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.815 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.815 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.815 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.815 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.815 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.815 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.815 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.816 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.816 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.816 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.816 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.816 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.816 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.816 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.816 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.816 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.817 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.817 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.817 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.817 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.817 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.817 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.817 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.817 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.818 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.818 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.818 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.818 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.818 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.818 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.818 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.818 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.819 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.819 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.819 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.819 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.819 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.819 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.819 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.819 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.820 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.820 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.820 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.820 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.820 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.820 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.820 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.820 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.820 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.821 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.821 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.821 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.821 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.821 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.821 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.821 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.821 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.822 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.822 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.822 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.822 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.822 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.822 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.822 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.822 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.822 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.823 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.823 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.823 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.823 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.823 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.823 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "telegraf", 2023-11-27T15:28:10.823 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.823 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.824 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.824 INFO:teuthology.orchestra.run.smithi052.stdout: "address": { 2023-11-27T15:28:10.824 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "address", 2023-11-27T15:28:10.824 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.824 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.824 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.824 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-27T15:28:10.824 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.824 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.825 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.825 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.825 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.825 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.825 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.825 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.825 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2023-11-27T15:28:10.825 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2023-11-27T15:28:10.826 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:10.826 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.826 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.826 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15", 2023-11-27T15:28:10.826 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.826 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.826 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.826 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.827 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.827 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.827 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.827 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.827 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.827 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.827 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.827 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.828 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.828 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.828 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.828 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.828 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.828 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.828 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.828 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.829 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.829 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.829 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.829 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.829 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.829 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.829 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.829 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.830 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.830 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.830 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.830 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.830 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.830 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.830 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.830 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.831 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.831 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.831 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.831 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.831 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.831 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.831 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.831 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.832 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.832 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.832 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.832 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.832 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.832 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.832 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.832 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.833 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.833 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.833 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.833 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.833 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.833 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.833 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.834 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.834 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.834 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.834 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.834 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.834 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.834 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.834 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.835 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.835 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.835 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.835 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.835 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.835 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.835 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.835 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.836 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.836 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.836 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.836 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.836 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.836 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.836 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "telemetry", 2023-11-27T15:28:10.836 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.837 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.837 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.837 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_basic": { 2023-11-27T15:28:10.837 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_basic", 2023-11-27T15:28:10.837 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.837 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.837 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.837 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.838 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.838 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.838 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.838 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-27T15:28:10.838 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.838 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.838 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.838 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.839 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_crash": { 2023-11-27T15:28:10.839 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_crash", 2023-11-27T15:28:10.839 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.839 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.839 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.839 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.839 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.840 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.840 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.840 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-27T15:28:10.840 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.840 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.840 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.840 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.840 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_device": { 2023-11-27T15:28:10.841 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_device", 2023-11-27T15:28:10.841 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.841 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.841 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.841 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.841 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.841 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.842 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.842 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-27T15:28:10.842 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.842 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.842 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.842 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.842 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_ident": { 2023-11-27T15:28:10.842 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_ident", 2023-11-27T15:28:10.843 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.843 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.843 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.843 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.843 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.843 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.843 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.844 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-27T15:28:10.844 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.844 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.844 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.844 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.844 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_perf": { 2023-11-27T15:28:10.844 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_perf", 2023-11-27T15:28:10.844 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.845 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.845 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.845 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.845 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.845 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.845 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.845 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-27T15:28:10.845 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.846 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.846 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.846 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.846 INFO:teuthology.orchestra.run.smithi052.stdout: "contact": { 2023-11-27T15:28:10.846 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "contact", 2023-11-27T15:28:10.846 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.846 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.846 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.847 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.847 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.847 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.847 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.847 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.847 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.847 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.847 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.848 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.848 INFO:teuthology.orchestra.run.smithi052.stdout: "description": { 2023-11-27T15:28:10.848 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "description", 2023-11-27T15:28:10.848 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.848 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.848 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.848 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.848 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.849 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.849 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.849 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.849 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.849 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.849 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.849 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.849 INFO:teuthology.orchestra.run.smithi052.stdout: "device_url": { 2023-11-27T15:28:10.850 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_url", 2023-11-27T15:28:10.850 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.850 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.850 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.850 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-27T15:28:10.850 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.850 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.850 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.851 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.851 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.851 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.851 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.851 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.851 INFO:teuthology.orchestra.run.smithi052.stdout: "enabled": { 2023-11-27T15:28:10.851 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enabled", 2023-11-27T15:28:10.851 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.852 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.852 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.852 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.852 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.852 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.852 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.852 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.852 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.853 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.853 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.853 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.853 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2023-11-27T15:28:10.853 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2023-11-27T15:28:10.853 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.853 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.853 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.854 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "24", 2023-11-27T15:28:10.854 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "8", 2023-11-27T15:28:10.854 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.854 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.854 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.854 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.854 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.854 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.854 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.855 INFO:teuthology.orchestra.run.smithi052.stdout: "last_opt_revision": { 2023-11-27T15:28:10.855 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "last_opt_revision", 2023-11-27T15:28:10.855 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.855 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.855 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.855 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1", 2023-11-27T15:28:10.855 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.855 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.856 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.856 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.856 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.856 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.856 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.856 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.856 INFO:teuthology.orchestra.run.smithi052.stdout: "leaderboard": { 2023-11-27T15:28:10.856 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "leaderboard", 2023-11-27T15:28:10.857 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.857 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.857 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.857 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.857 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.857 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.857 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.857 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.858 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.858 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.858 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.858 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.858 INFO:teuthology.orchestra.run.smithi052.stdout: "leaderboard_description": { 2023-11-27T15:28:10.858 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "leaderboard_description", 2023-11-27T15:28:10.858 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.859 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.859 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.859 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.859 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.859 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.859 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.859 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.859 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.860 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.860 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.860 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.860 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.860 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.860 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.860 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.860 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.861 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.861 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.861 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.861 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.861 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.861 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.861 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.861 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.861 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.862 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.862 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.862 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.862 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.862 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.862 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.862 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.863 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.863 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.863 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.863 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.863 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.863 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.863 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.863 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.864 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.864 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.864 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.864 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.864 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.864 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.864 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.864 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.865 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.865 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.865 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.865 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.865 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.865 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.865 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.865 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.866 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.866 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.866 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.866 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.866 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.866 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.866 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.866 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.867 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.867 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.867 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.867 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.867 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.867 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.868 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.868 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.868 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.868 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.868 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.868 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.868 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.868 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.868 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.869 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.869 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.869 INFO:teuthology.orchestra.run.smithi052.stdout: "organization": { 2023-11-27T15:28:10.869 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "organization", 2023-11-27T15:28:10.869 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.869 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.869 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.869 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.870 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.870 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.870 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.870 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.870 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.870 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.870 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.870 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.871 INFO:teuthology.orchestra.run.smithi052.stdout: "proxy": { 2023-11-27T15:28:10.871 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "proxy", 2023-11-27T15:28:10.871 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.871 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.871 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.871 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.871 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.871 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.872 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.872 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.872 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.872 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.872 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.872 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.872 INFO:teuthology.orchestra.run.smithi052.stdout: "url": { 2023-11-27T15:28:10.872 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "url", 2023-11-27T15:28:10.873 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.873 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.873 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.873 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-27T15:28:10.873 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.873 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.873 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.873 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.873 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.874 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.874 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.874 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.874 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.874 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.874 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.874 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "test_orchestrator", 2023-11-27T15:28:10.874 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.875 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.875 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.875 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.875 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.875 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.875 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.875 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.875 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.876 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.876 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.876 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.876 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.876 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.876 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.876 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.877 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.877 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.877 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.877 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.877 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.877 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.877 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.877 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.878 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.878 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.878 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.878 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.878 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.878 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.878 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.878 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.879 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.879 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.879 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.879 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.879 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.879 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.879 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.879 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.880 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.880 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.880 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.880 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.880 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.880 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.880 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.880 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.881 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.881 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.881 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.881 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.881 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.881 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.881 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.881 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.882 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.882 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.882 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.882 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.882 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.882 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.882 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.882 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.883 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.883 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.883 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.883 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.883 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.883 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.883 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.884 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.884 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.884 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.884 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.884 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.884 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "volumes", 2023-11-27T15:28:10.884 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.884 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.885 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.885 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.885 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.885 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.885 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.885 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.885 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.885 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.886 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.886 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.886 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.886 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.886 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.886 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.886 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.886 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.887 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.887 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.887 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.887 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.887 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.887 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.887 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.887 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.888 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.888 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.888 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.888 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.888 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.888 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.888 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.888 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.889 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.889 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.889 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.889 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.889 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.889 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.889 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.889 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.890 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.890 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.890 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.890 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.890 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.890 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.890 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.890 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.891 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.891 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.891 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.891 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.891 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.891 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.891 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.892 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.892 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.892 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.892 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.892 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.892 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.892 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.892 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.893 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.893 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.893 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.893 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.893 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.893 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.893 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.893 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.894 INFO:teuthology.orchestra.run.smithi052.stdout: "max_concurrent_clones": { 2023-11-27T15:28:10.894 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_concurrent_clones", 2023-11-27T15:28:10.894 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.894 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.894 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.894 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "4", 2023-11-27T15:28:10.894 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.895 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.895 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.895 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-27T15:28:10.895 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.895 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.895 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.895 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.896 INFO:teuthology.orchestra.run.smithi052.stdout: "snapshot_clone_delay": { 2023-11-27T15:28:10.896 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snapshot_clone_delay", 2023-11-27T15:28:10.896 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.896 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.896 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.896 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2023-11-27T15:28:10.896 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.896 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.897 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.897 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-27T15:28:10.897 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.897 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.897 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.897 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.897 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.897 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.898 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.898 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix", 2023-11-27T15:28:10.898 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.898 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.898 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.898 INFO:teuthology.orchestra.run.smithi052.stdout: "discovery_interval": { 2023-11-27T15:28:10.899 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "discovery_interval", 2023-11-27T15:28:10.899 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2023-11-27T15:28:10.899 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.899 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.899 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "100", 2023-11-27T15:28:10.899 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.899 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.899 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.900 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.900 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.900 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.900 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.900 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.900 INFO:teuthology.orchestra.run.smithi052.stdout: "identifier": { 2023-11-27T15:28:10.900 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "identifier", 2023-11-27T15:28:10.900 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.901 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.901 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.901 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.901 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.901 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.901 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.901 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.901 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.902 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.902 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.902 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.902 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2023-11-27T15:28:10.902 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2023-11-27T15:28:10.902 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:10.902 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.902 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.903 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2023-11-27T15:28:10.903 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.903 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.903 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.903 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.903 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.903 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.903 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.904 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.904 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.904 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.904 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.904 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.904 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.904 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.904 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.905 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.905 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.905 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.905 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.905 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.905 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.905 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.905 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.906 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.906 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.906 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.906 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.906 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.906 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.906 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.906 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.907 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.907 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.907 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.907 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.907 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.907 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.907 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.907 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.908 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.908 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.908 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.908 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.908 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.908 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.908 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.908 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.909 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.909 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.909 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.909 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.909 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.909 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.909 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.909 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.910 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.910 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.910 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.910 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.910 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.910 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.910 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.911 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.911 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.911 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.911 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.911 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.911 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.911 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.911 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.911 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.912 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.912 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.912 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.912 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.912 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.912 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.912 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.912 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_host": { 2023-11-27T15:28:10.913 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_host", 2023-11-27T15:28:10.913 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.913 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.913 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.913 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.913 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.913 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.914 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.914 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.914 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.914 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.914 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.914 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.914 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_port": { 2023-11-27T15:28:10.914 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_port", 2023-11-27T15:28:10.915 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.915 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.915 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.915 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10051", 2023-11-27T15:28:10.915 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.915 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.915 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.915 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.916 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.916 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.916 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.916 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.916 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_sender": { 2023-11-27T15:28:10.916 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_sender", 2023-11-27T15:28:10.916 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.916 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.917 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.917 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-27T15:28:10.917 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.917 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.917 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.917 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.917 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.917 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.918 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.918 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.918 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.918 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.918 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2023-11-27T15:28:10.918 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.918 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.919 INFO:teuthology.orchestra.run.smithi052.stdout: "modules": [ 2023-11-27T15:28:10.919 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm", 2023-11-27T15:28:10.919 INFO:teuthology.orchestra.run.smithi052.stdout: "dashboard", 2023-11-27T15:28:10.919 INFO:teuthology.orchestra.run.smithi052.stdout: "iostat", 2023-11-27T15:28:10.919 INFO:teuthology.orchestra.run.smithi052.stdout: "nfs", 2023-11-27T15:28:10.919 INFO:teuthology.orchestra.run.smithi052.stdout: "restful", 2023-11-27T15:28:10.919 INFO:teuthology.orchestra.run.smithi052.stdout: "snap_schedule" 2023-11-27T15:28:10.919 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.920 INFO:teuthology.orchestra.run.smithi052.stdout: "available_modules": [ 2023-11-27T15:28:10.920 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.920 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "alerts", 2023-11-27T15:28:10.920 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.920 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.920 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.920 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2023-11-27T15:28:10.920 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2023-11-27T15:28:10.921 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:10.921 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.921 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.921 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2023-11-27T15:28:10.921 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.921 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.921 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.921 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How frequently to reexamine health status", 2023-11-27T15:28:10.922 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.922 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.922 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.922 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.922 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.922 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.922 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.922 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.922 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.923 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.923 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.923 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.923 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.923 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.923 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.923 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.924 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.924 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.924 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.924 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.924 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.924 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.924 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.924 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.925 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.925 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.925 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.925 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.925 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.925 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.925 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.925 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.926 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.926 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.926 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.926 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.926 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.926 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.926 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.927 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.927 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.927 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.927 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.927 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.927 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.927 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.927 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.928 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.928 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.928 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.928 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.928 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.928 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.928 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.929 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.929 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.929 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.929 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.929 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.929 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.929 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.929 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.929 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.930 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.930 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.930 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.930 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.930 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.930 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.930 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.930 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.931 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.931 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.931 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.931 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_destination": { 2023-11-27T15:28:10.931 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_destination", 2023-11-27T15:28:10.931 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.931 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.931 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.932 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.932 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.932 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.932 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.932 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Email address to send alerts to", 2023-11-27T15:28:10.932 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.932 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.932 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.933 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.933 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_from_name": { 2023-11-27T15:28:10.933 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_from_name", 2023-11-27T15:28:10.933 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.933 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.933 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.933 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "Ceph", 2023-11-27T15:28:10.933 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.934 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.934 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.934 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Email From: name", 2023-11-27T15:28:10.934 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.934 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.934 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.934 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.934 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_host": { 2023-11-27T15:28:10.935 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_host", 2023-11-27T15:28:10.935 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.935 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.935 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.935 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.935 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.935 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.935 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.936 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP server", 2023-11-27T15:28:10.936 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.936 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.936 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.936 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.936 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_password": { 2023-11-27T15:28:10.936 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_password", 2023-11-27T15:28:10.936 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.937 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.937 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.937 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.937 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.937 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.937 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.937 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Password to authenticate with", 2023-11-27T15:28:10.937 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.937 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.938 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.938 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.938 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_port": { 2023-11-27T15:28:10.938 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_port", 2023-11-27T15:28:10.938 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.938 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.938 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.938 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "465", 2023-11-27T15:28:10.939 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.939 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.939 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.939 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP port", 2023-11-27T15:28:10.939 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.939 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.939 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.939 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.940 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_sender": { 2023-11-27T15:28:10.940 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_sender", 2023-11-27T15:28:10.940 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.940 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.940 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.940 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.940 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.940 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.941 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.941 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP envelope sender", 2023-11-27T15:28:10.941 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.941 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.941 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.941 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.941 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_ssl": { 2023-11-27T15:28:10.941 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_ssl", 2023-11-27T15:28:10.942 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.942 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.942 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.942 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.942 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.942 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.942 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.943 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-27T15:28:10.943 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.943 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.943 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.943 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.943 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_user": { 2023-11-27T15:28:10.943 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_user", 2023-11-27T15:28:10.943 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.944 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.944 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.944 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.944 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.944 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.944 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.944 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "User to authenticate as", 2023-11-27T15:28:10.944 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.945 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.945 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.945 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.945 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.945 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.945 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.945 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "balancer", 2023-11-27T15:28:10.945 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.945 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.946 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.946 INFO:teuthology.orchestra.run.smithi052.stdout: "active": { 2023-11-27T15:28:10.946 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "active", 2023-11-27T15:28:10.946 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.946 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.946 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.946 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.946 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.947 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.947 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.947 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "automatically balance PGs across cluster", 2023-11-27T15:28:10.947 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.947 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.947 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.947 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.947 INFO:teuthology.orchestra.run.smithi052.stdout: "begin_time": { 2023-11-27T15:28:10.948 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "begin_time", 2023-11-27T15:28:10.948 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.948 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.948 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.948 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0000", 2023-11-27T15:28:10.948 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.948 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.948 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.948 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "beginning time of day to automatically balance", 2023-11-27T15:28:10.949 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-27T15:28:10.949 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.949 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.949 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.949 INFO:teuthology.orchestra.run.smithi052.stdout: "begin_weekday": { 2023-11-27T15:28:10.949 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "begin_weekday", 2023-11-27T15:28:10.949 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2023-11-27T15:28:10.949 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.950 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.950 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2023-11-27T15:28:10.950 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0", 2023-11-27T15:28:10.950 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "6", 2023-11-27T15:28:10.950 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.950 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-27T15:28:10.950 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-27T15:28:10.950 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.951 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.951 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.951 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_max_iterations": { 2023-11-27T15:28:10.951 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_max_iterations", 2023-11-27T15:28:10.951 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2023-11-27T15:28:10.951 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.951 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.951 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "25", 2023-11-27T15:28:10.952 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2023-11-27T15:28:10.952 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "250", 2023-11-27T15:28:10.952 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.952 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-27T15:28:10.952 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.952 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.952 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.952 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.952 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_metrics": { 2023-11-27T15:28:10.953 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_metrics", 2023-11-27T15:28:10.953 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.953 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.953 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.953 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "pgs,objects,bytes", 2023-11-27T15:28:10.953 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.953 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.953 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.954 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-27T15:28:10.954 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-27T15:28:10.954 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.954 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.954 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.954 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_step": { 2023-11-27T15:28:10.954 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_step", 2023-11-27T15:28:10.955 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2023-11-27T15:28:10.955 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.955 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.955 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0.5", 2023-11-27T15:28:10.955 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0.001", 2023-11-27T15:28:10.955 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "0.999", 2023-11-27T15:28:10.955 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.955 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "aggressiveness of optimization", 2023-11-27T15:28:10.955 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-27T15:28:10.956 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.956 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.956 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.956 INFO:teuthology.orchestra.run.smithi052.stdout: "end_time": { 2023-11-27T15:28:10.956 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "end_time", 2023-11-27T15:28:10.956 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.956 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.956 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.957 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "2359", 2023-11-27T15:28:10.957 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.957 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.957 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.957 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "ending time of day to automatically balance", 2023-11-27T15:28:10.957 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-27T15:28:10.957 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.957 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.958 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.958 INFO:teuthology.orchestra.run.smithi052.stdout: "end_weekday": { 2023-11-27T15:28:10.958 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "end_weekday", 2023-11-27T15:28:10.958 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2023-11-27T15:28:10.958 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.958 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.958 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2023-11-27T15:28:10.959 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0", 2023-11-27T15:28:10.959 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "6", 2023-11-27T15:28:10.959 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.959 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-27T15:28:10.959 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-27T15:28:10.959 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.959 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.959 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.959 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:10.960 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:10.960 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.960 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.960 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.960 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.960 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.960 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.960 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.961 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.961 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.961 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.961 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.961 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.961 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.961 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.961 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.962 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.962 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.962 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.962 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.962 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:10.962 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:10.962 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.962 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.963 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.963 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.963 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.963 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.963 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.963 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.963 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.963 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.964 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.964 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.964 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:10.964 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:10.964 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.964 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.964 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.965 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:10.965 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.965 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.965 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.965 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:10.965 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:10.965 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:10.965 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:10.966 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:10.966 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:10.966 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.966 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.966 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.966 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.966 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.966 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.967 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:10.967 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:10.967 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.967 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.967 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.967 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.967 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.968 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.968 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.968 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:10.968 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.968 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.968 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.968 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.969 INFO:teuthology.orchestra.run.smithi052.stdout: "min_score": { 2023-11-27T15:28:10.969 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "min_score", 2023-11-27T15:28:10.969 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2023-11-27T15:28:10.969 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.969 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.969 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2023-11-27T15:28:10.969 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.969 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.970 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.970 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-27T15:28:10.970 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.970 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.970 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.970 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.970 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": { 2023-11-27T15:28:10.970 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mode", 2023-11-27T15:28:10.971 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.971 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.971 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.971 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "upmap", 2023-11-27T15:28:10.971 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.971 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.971 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:10.971 INFO:teuthology.orchestra.run.smithi052.stdout: "crush-compat", 2023-11-27T15:28:10.972 INFO:teuthology.orchestra.run.smithi052.stdout: "none", 2023-11-27T15:28:10.972 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap" 2023-11-27T15:28:10.972 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:10.972 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Balancer mode", 2023-11-27T15:28:10.972 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.972 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.972 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.973 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.973 INFO:teuthology.orchestra.run.smithi052.stdout: "pool_ids": { 2023-11-27T15:28:10.973 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pool_ids", 2023-11-27T15:28:10.973 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.973 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.973 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.973 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:10.973 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.974 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.974 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.974 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-27T15:28:10.974 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.974 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.974 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.974 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.974 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2023-11-27T15:28:10.975 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2023-11-27T15:28:10.975 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:10.975 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.975 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.975 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2023-11-27T15:28:10.975 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.975 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.975 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.976 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-27T15:28:10.976 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.976 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.976 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.976 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.976 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap_max_deviation": { 2023-11-27T15:28:10.976 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "upmap_max_deviation", 2023-11-27T15:28:10.976 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.977 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.977 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.977 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2023-11-27T15:28:10.977 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2023-11-27T15:28:10.977 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.977 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.977 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-27T15:28:10.978 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-27T15:28:10.978 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.978 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.978 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.978 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap_max_optimizations": { 2023-11-27T15:28:10.978 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "upmap_max_optimizations", 2023-11-27T15:28:10.978 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2023-11-27T15:28:10.978 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.978 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:10.979 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2023-11-27T15:28:10.979 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.979 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.979 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.979 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-27T15:28:10.979 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.979 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.979 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.980 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.980 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:10.980 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.980 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:10.980 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cephadm", 2023-11-27T15:28:10.980 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:10.980 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:10.980 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:10.981 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_down_multiplier": { 2023-11-27T15:28:10.981 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_down_multiplier", 2023-11-27T15:28:10.981 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2023-11-27T15:28:10.981 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.981 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.981 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3.0", 2023-11-27T15:28:10.981 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.981 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.982 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.982 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-27T15:28:10.982 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.982 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.982 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.982 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.982 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_refresh_rate": { 2023-11-27T15:28:10.982 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_refresh_rate", 2023-11-27T15:28:10.983 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:10.983 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.983 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.983 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "20", 2023-11-27T15:28:10.983 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.983 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.983 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.983 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-27T15:28:10.983 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.984 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.984 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.984 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.984 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_starting_port": { 2023-11-27T15:28:10.984 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_starting_port", 2023-11-27T15:28:10.984 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:10.984 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.985 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.985 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "4721", 2023-11-27T15:28:10.985 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.985 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.985 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.985 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-27T15:28:10.985 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.985 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.986 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.986 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.986 INFO:teuthology.orchestra.run.smithi052.stdout: "alertmanager_web_password": { 2023-11-27T15:28:10.986 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "alertmanager_web_password", 2023-11-27T15:28:10.986 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.986 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.986 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.986 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2023-11-27T15:28:10.987 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.987 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.987 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.987 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Alertmanager web password", 2023-11-27T15:28:10.987 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.987 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.987 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.987 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.988 INFO:teuthology.orchestra.run.smithi052.stdout: "alertmanager_web_user": { 2023-11-27T15:28:10.988 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "alertmanager_web_user", 2023-11-27T15:28:10.988 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:10.988 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.988 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.988 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2023-11-27T15:28:10.988 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.988 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.989 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.989 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Alertmanager web user", 2023-11-27T15:28:10.989 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.989 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.989 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.989 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.989 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_ptrace": { 2023-11-27T15:28:10.989 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_ptrace", 2023-11-27T15:28:10.990 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.990 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.990 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.990 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.990 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.990 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.990 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.990 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-27T15:28:10.991 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-27T15:28:10.991 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.991 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.991 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.991 INFO:teuthology.orchestra.run.smithi052.stdout: "autotune_interval": { 2023-11-27T15:28:10.991 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "autotune_interval", 2023-11-27T15:28:10.991 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:10.991 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.992 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.992 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2023-11-27T15:28:10.992 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.992 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.992 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.992 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-27T15:28:10.992 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.992 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.993 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.993 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.993 INFO:teuthology.orchestra.run.smithi052.stdout: "autotune_memory_target_ratio": { 2023-11-27T15:28:10.993 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "autotune_memory_target_ratio", 2023-11-27T15:28:10.993 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2023-11-27T15:28:10.993 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.993 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.994 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0.7", 2023-11-27T15:28:10.994 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.994 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.994 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.994 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-27T15:28:10.994 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.994 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.994 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.995 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.995 INFO:teuthology.orchestra.run.smithi052.stdout: "cgroups_split": { 2023-11-27T15:28:10.995 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cgroups_split", 2023-11-27T15:28:10.995 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.995 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.995 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.995 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.995 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.996 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.996 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.996 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-27T15:28:10.996 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.996 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.996 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.996 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.997 INFO:teuthology.orchestra.run.smithi052.stdout: "config_checks_enabled": { 2023-11-27T15:28:10.997 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_checks_enabled", 2023-11-27T15:28:10.997 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.997 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.997 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.997 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:10.997 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.997 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:10.998 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:10.998 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-27T15:28:10.998 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:10.998 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:10.998 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:10.998 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:10.998 INFO:teuthology.orchestra.run.smithi052.stdout: "config_dashboard": { 2023-11-27T15:28:10.999 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_dashboard", 2023-11-27T15:28:10.999 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:10.999 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:10.999 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:10.999 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:10.999 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:10.999 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.000 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.000 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-27T15:28:11.000 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.000 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.000 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.000 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.000 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_alertmanager": { 2023-11-27T15:28:11.000 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_alertmanager", 2023-11-27T15:28:11.000 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.001 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.001 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.001 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-27T15:28:11.001 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.001 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.001 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.001 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2023-11-27T15:28:11.001 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.002 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.002 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.002 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.002 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_base": { 2023-11-27T15:28:11.002 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_base", 2023-11-27T15:28:11.002 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.002 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.002 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.003 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-27T15:28:11.003 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.003 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.003 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.003 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Container image name, without the tag", 2023-11-27T15:28:11.003 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.003 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.003 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.004 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.004 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_elasticsearch": { 2023-11-27T15:28:11.004 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_elasticsearch", 2023-11-27T15:28:11.004 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.004 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.004 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.004 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-27T15:28:11.004 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.005 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.005 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.005 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "elasticsearch container image", 2023-11-27T15:28:11.005 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.005 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.005 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.005 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.005 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_grafana": { 2023-11-27T15:28:11.006 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_grafana", 2023-11-27T15:28:11.006 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.006 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.006 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.006 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-27T15:28:11.006 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.006 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.006 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.007 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2023-11-27T15:28:11.007 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.007 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.007 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.007 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.007 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_haproxy": { 2023-11-27T15:28:11.007 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_haproxy", 2023-11-27T15:28:11.007 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.008 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.008 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.008 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-27T15:28:11.008 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.008 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.008 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.008 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "HAproxy container image", 2023-11-27T15:28:11.008 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.009 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.009 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.009 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.009 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_agent": { 2023-11-27T15:28:11.009 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_agent", 2023-11-27T15:28:11.009 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.009 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.009 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.010 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-27T15:28:11.010 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.010 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.010 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.010 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger agent container image", 2023-11-27T15:28:11.010 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.010 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.010 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.011 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.011 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_collector": { 2023-11-27T15:28:11.011 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_collector", 2023-11-27T15:28:11.011 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.011 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.011 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.011 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-27T15:28:11.011 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.012 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.012 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.012 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger collector container image", 2023-11-27T15:28:11.012 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.012 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.012 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.012 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.012 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_query": { 2023-11-27T15:28:11.013 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_query", 2023-11-27T15:28:11.013 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.013 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.013 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.013 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-27T15:28:11.013 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.013 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.013 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.013 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger query container image", 2023-11-27T15:28:11.014 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.014 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.014 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.014 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.014 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_keepalived": { 2023-11-27T15:28:11.014 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_keepalived", 2023-11-27T15:28:11.014 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.014 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.015 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.015 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-27T15:28:11.015 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.015 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.015 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.015 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Keepalived container image", 2023-11-27T15:28:11.015 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.016 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.016 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.016 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.016 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_loki": { 2023-11-27T15:28:11.016 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_loki", 2023-11-27T15:28:11.016 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.016 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.016 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.017 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-27T15:28:11.017 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.017 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.017 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.017 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Loki container image", 2023-11-27T15:28:11.017 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.017 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.018 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.018 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.018 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_node_exporter": { 2023-11-27T15:28:11.018 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_node_exporter", 2023-11-27T15:28:11.018 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.018 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.018 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.018 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-27T15:28:11.019 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.019 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.019 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.019 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2023-11-27T15:28:11.019 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.019 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.019 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.019 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.020 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_prometheus": { 2023-11-27T15:28:11.020 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_prometheus", 2023-11-27T15:28:11.020 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.020 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.020 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.020 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-27T15:28:11.020 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.020 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.021 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.021 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2023-11-27T15:28:11.021 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.021 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.021 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.021 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.021 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_promtail": { 2023-11-27T15:28:11.021 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_promtail", 2023-11-27T15:28:11.021 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.022 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.022 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.022 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-27T15:28:11.022 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.022 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.022 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.022 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Promtail container image", 2023-11-27T15:28:11.022 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.023 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.023 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.023 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.023 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_snmp_gateway": { 2023-11-27T15:28:11.023 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_snmp_gateway", 2023-11-27T15:28:11.023 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.023 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.023 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.024 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-27T15:28:11.024 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.024 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.024 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.024 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SNMP Gateway container image", 2023-11-27T15:28:11.024 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.024 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.024 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.025 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.025 INFO:teuthology.orchestra.run.smithi052.stdout: "container_init": { 2023-11-27T15:28:11.025 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_init", 2023-11-27T15:28:11.025 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.025 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.025 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.025 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.025 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.026 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.026 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.026 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Run podman/docker with `--init`", 2023-11-27T15:28:11.026 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.026 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.026 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.026 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.027 INFO:teuthology.orchestra.run.smithi052.stdout: "daemon_cache_timeout": { 2023-11-27T15:28:11.027 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "daemon_cache_timeout", 2023-11-27T15:28:11.027 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:11.027 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.027 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.027 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2023-11-27T15:28:11.027 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.027 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.028 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.028 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-27T15:28:11.028 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.028 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.028 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.028 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.028 INFO:teuthology.orchestra.run.smithi052.stdout: "default_cephadm_command_timeout": { 2023-11-27T15:28:11.029 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "default_cephadm_command_timeout", 2023-11-27T15:28:11.029 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:11.029 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.029 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.029 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "900", 2023-11-27T15:28:11.029 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.029 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.029 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.030 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-27T15:28:11.030 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.030 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.030 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.030 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.030 INFO:teuthology.orchestra.run.smithi052.stdout: "default_registry": { 2023-11-27T15:28:11.030 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "default_registry", 2023-11-27T15:28:11.030 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.031 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.031 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.031 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io", 2023-11-27T15:28:11.031 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.031 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.031 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.031 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-27T15:28:11.031 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.032 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.032 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.032 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.032 INFO:teuthology.orchestra.run.smithi052.stdout: "device_cache_timeout": { 2023-11-27T15:28:11.032 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_cache_timeout", 2023-11-27T15:28:11.032 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:11.032 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.032 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.033 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1800", 2023-11-27T15:28:11.033 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.033 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.033 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.033 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache device inventory", 2023-11-27T15:28:11.033 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.033 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.033 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.034 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.034 INFO:teuthology.orchestra.run.smithi052.stdout: "device_enhanced_scan": { 2023-11-27T15:28:11.034 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_enhanced_scan", 2023-11-27T15:28:11.034 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.034 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.034 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.034 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.034 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.035 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.035 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.035 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-27T15:28:11.035 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.035 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.035 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.035 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.035 INFO:teuthology.orchestra.run.smithi052.stdout: "facts_cache_timeout": { 2023-11-27T15:28:11.036 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "facts_cache_timeout", 2023-11-27T15:28:11.036 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:11.036 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.036 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.036 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2023-11-27T15:28:11.036 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.036 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.037 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.037 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache host facts data", 2023-11-27T15:28:11.037 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.037 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.037 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.037 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.037 INFO:teuthology.orchestra.run.smithi052.stdout: "host_check_interval": { 2023-11-27T15:28:11.037 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "host_check_interval", 2023-11-27T15:28:11.038 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:11.038 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.038 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.038 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2023-11-27T15:28:11.038 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.038 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.038 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.039 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to perform a host check", 2023-11-27T15:28:11.039 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.039 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.039 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.039 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.039 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.039 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.039 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.040 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.040 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.040 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.040 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.040 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.040 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.040 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.040 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.041 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.041 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.041 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.041 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.041 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.041 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.041 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.041 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.041 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.042 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.042 INFO:teuthology.orchestra.run.smithi052.stdout: "log_refresh_metadata": { 2023-11-27T15:28:11.042 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_refresh_metadata", 2023-11-27T15:28:11.042 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.042 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.042 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.043 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.043 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.043 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.043 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.043 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-27T15:28:11.043 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.043 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.043 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.044 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.044 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.044 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.044 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.044 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.044 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.044 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.045 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.045 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.045 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.045 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-27T15:28:11.045 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.046 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.046 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.046 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.046 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.047 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.047 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.047 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.047 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.047 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.047 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.047 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.047 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.048 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.048 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.048 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.048 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.048 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.048 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.048 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.049 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.049 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.049 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.049 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.049 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.049 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.049 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.049 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.049 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.050 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.050 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.050 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.050 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.050 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.050 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.050 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.051 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.051 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.051 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.051 INFO:teuthology.orchestra.run.smithi052.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-27T15:28:11.051 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-27T15:28:11.051 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.051 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.052 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.052 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.052 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.052 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.052 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.052 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-27T15:28:11.052 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.052 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.053 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.053 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.053 INFO:teuthology.orchestra.run.smithi052.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-27T15:28:11.053 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-27T15:28:11.053 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.053 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.053 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.053 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "*", 2023-11-27T15:28:11.054 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.054 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.054 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.054 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-27T15:28:11.054 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.054 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.054 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.054 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.055 INFO:teuthology.orchestra.run.smithi052.stdout: "max_count_per_host": { 2023-11-27T15:28:11.055 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_count_per_host", 2023-11-27T15:28:11.055 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.055 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.055 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.055 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2023-11-27T15:28:11.055 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.055 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.056 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.056 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "max number of daemons per service per host", 2023-11-27T15:28:11.056 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.056 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.056 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.056 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.056 INFO:teuthology.orchestra.run.smithi052.stdout: "max_osd_draining_count": { 2023-11-27T15:28:11.057 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_osd_draining_count", 2023-11-27T15:28:11.057 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.057 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.057 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.057 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2023-11-27T15:28:11.057 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.057 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.057 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.058 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-27T15:28:11.058 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.058 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.058 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.058 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.058 INFO:teuthology.orchestra.run.smithi052.stdout: "migration_current": { 2023-11-27T15:28:11.058 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "migration_current", 2023-11-27T15:28:11.059 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.059 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.059 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.059 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.059 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.059 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.059 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.059 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "internal - do not modify", 2023-11-27T15:28:11.060 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.060 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.060 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.060 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.060 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": { 2023-11-27T15:28:11.060 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mode", 2023-11-27T15:28:11.060 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.060 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.061 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.061 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "root", 2023-11-27T15:28:11.061 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.061 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.061 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.061 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm-package", 2023-11-27T15:28:11.061 INFO:teuthology.orchestra.run.smithi052.stdout: "root" 2023-11-27T15:28:11.061 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.062 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "mode for remote execution of cephadm", 2023-11-27T15:28:11.062 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.062 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.062 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.062 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.062 INFO:teuthology.orchestra.run.smithi052.stdout: "prometheus_alerts_path": { 2023-11-27T15:28:11.062 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus_alerts_path", 2023-11-27T15:28:11.062 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.063 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.063 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.063 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-27T15:28:11.063 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.063 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.063 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.063 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-27T15:28:11.064 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.064 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.064 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.064 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.064 INFO:teuthology.orchestra.run.smithi052.stdout: "prometheus_web_password": { 2023-11-27T15:28:11.064 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus_web_password", 2023-11-27T15:28:11.064 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.064 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.065 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.065 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2023-11-27T15:28:11.065 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.065 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.065 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.065 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus web password", 2023-11-27T15:28:11.065 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.065 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.066 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.066 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.066 INFO:teuthology.orchestra.run.smithi052.stdout: "prometheus_web_user": { 2023-11-27T15:28:11.066 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus_web_user", 2023-11-27T15:28:11.066 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.066 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.066 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.066 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2023-11-27T15:28:11.067 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.067 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.067 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.067 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus web user", 2023-11-27T15:28:11.067 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.067 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.067 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.068 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.068 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_insecure": { 2023-11-27T15:28:11.068 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_insecure", 2023-11-27T15:28:11.068 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.068 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.068 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.068 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.069 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.069 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.069 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.069 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-27T15:28:11.069 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.069 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.069 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.069 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.070 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_password": { 2023-11-27T15:28:11.070 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_password", 2023-11-27T15:28:11.070 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.070 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.070 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.070 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.070 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.071 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.071 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.071 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-27T15:28:11.071 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.071 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.071 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.071 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.071 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_url": { 2023-11-27T15:28:11.072 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_url", 2023-11-27T15:28:11.072 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.072 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.072 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.072 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.072 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.072 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.073 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.073 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-27T15:28:11.073 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.073 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.073 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.073 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.073 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_username": { 2023-11-27T15:28:11.074 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_username", 2023-11-27T15:28:11.074 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.074 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.074 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.074 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.074 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.074 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.074 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.075 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-27T15:28:11.075 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.075 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.075 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.075 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.075 INFO:teuthology.orchestra.run.smithi052.stdout: "secure_monitoring_stack": { 2023-11-27T15:28:11.075 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "secure_monitoring_stack", 2023-11-27T15:28:11.075 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.076 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.076 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.076 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.076 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.076 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.076 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.076 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-27T15:28:11.076 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.077 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.077 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.077 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.077 INFO:teuthology.orchestra.run.smithi052.stdout: "service_discovery_port": { 2023-11-27T15:28:11.077 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "service_discovery_port", 2023-11-27T15:28:11.077 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.077 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.078 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.078 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8765", 2023-11-27T15:28:11.078 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.078 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.078 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.078 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "cephadm service discovery port", 2023-11-27T15:28:11.078 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.078 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.079 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.079 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.079 INFO:teuthology.orchestra.run.smithi052.stdout: "ssh_config_file": { 2023-11-27T15:28:11.079 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssh_config_file", 2023-11-27T15:28:11.079 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.079 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.079 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.079 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.080 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.080 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.080 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.080 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-27T15:28:11.080 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.080 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.080 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.080 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.081 INFO:teuthology.orchestra.run.smithi052.stdout: "use_agent": { 2023-11-27T15:28:11.081 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "use_agent", 2023-11-27T15:28:11.081 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.081 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.081 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.081 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.081 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.081 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.082 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.082 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-27T15:28:11.082 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.082 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.082 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.082 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.082 INFO:teuthology.orchestra.run.smithi052.stdout: "use_repo_digest": { 2023-11-27T15:28:11.083 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "use_repo_digest", 2023-11-27T15:28:11.083 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.083 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.083 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.083 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.083 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.083 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.083 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.084 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-27T15:28:11.084 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.084 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.084 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.084 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.084 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_failed_host_check": { 2023-11-27T15:28:11.084 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_failed_host_check", 2023-11-27T15:28:11.084 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.085 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.085 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.085 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.085 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.085 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.085 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.085 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if the host check fails", 2023-11-27T15:28:11.086 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.086 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.086 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.086 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.086 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_stray_daemons": { 2023-11-27T15:28:11.086 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_stray_daemons", 2023-11-27T15:28:11.086 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.086 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.087 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.087 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.087 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.087 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.087 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.087 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-27T15:28:11.087 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.087 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.088 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.088 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.088 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_stray_hosts": { 2023-11-27T15:28:11.088 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_stray_hosts", 2023-11-27T15:28:11.088 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.088 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.088 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.088 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.089 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.089 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.089 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.089 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-27T15:28:11.089 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.089 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.089 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.090 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.090 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.090 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.090 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.090 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crash", 2023-11-27T15:28:11.090 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:11.090 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:11.090 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.091 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.091 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.091 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.091 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.091 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.091 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.091 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.091 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.092 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.092 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.092 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.092 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.092 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.092 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.092 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.093 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.093 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.093 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.093 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.093 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.093 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.093 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.093 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.094 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.094 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.094 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.094 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.094 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.094 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.094 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.094 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.095 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.095 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.095 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.095 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.095 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.095 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.095 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.096 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.096 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.096 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.096 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.096 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.096 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.096 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.097 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.097 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.097 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.097 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.097 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.097 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.097 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.097 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.098 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.098 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.098 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.098 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.098 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.098 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.098 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.099 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.099 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.099 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.099 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.099 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.099 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.099 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.100 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.100 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.100 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.100 INFO:teuthology.orchestra.run.smithi052.stdout: "retain_interval": { 2023-11-27T15:28:11.100 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "retain_interval", 2023-11-27T15:28:11.100 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:11.100 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.100 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.101 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "31536000", 2023-11-27T15:28:11.101 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.101 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.101 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.101 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-27T15:28:11.101 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.101 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.101 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.102 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.102 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_recent_interval": { 2023-11-27T15:28:11.102 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_recent_interval", 2023-11-27T15:28:11.102 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:11.102 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.102 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.102 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1209600", 2023-11-27T15:28:11.102 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.103 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.103 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.103 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-27T15:28:11.103 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.103 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.103 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.103 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.103 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.104 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.104 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.104 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "dashboard", 2023-11-27T15:28:11.104 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:11.104 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:11.104 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.104 INFO:teuthology.orchestra.run.smithi052.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-27T15:28:11.104 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-27T15:28:11.105 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.105 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.105 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.105 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2023-11-27T15:28:11.105 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.105 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.105 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.105 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.105 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.106 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.106 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.106 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.106 INFO:teuthology.orchestra.run.smithi052.stdout: "ALERTMANAGER_API_HOST": { 2023-11-27T15:28:11.106 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-27T15:28:11.106 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.106 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.106 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.107 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.107 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.107 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.107 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.107 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.107 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.107 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.108 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.108 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.108 INFO:teuthology.orchestra.run.smithi052.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-27T15:28:11.108 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-27T15:28:11.108 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.108 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.108 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.108 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.109 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.109 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.109 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.109 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.109 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.109 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.109 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.109 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.110 INFO:teuthology.orchestra.run.smithi052.stdout: "AUDIT_API_ENABLED": { 2023-11-27T15:28:11.110 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "AUDIT_API_ENABLED", 2023-11-27T15:28:11.110 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.110 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.110 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.110 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.110 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.110 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.111 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.111 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.111 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.111 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.111 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.111 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.111 INFO:teuthology.orchestra.run.smithi052.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-27T15:28:11.111 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-27T15:28:11.112 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.112 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.112 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.112 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.112 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.112 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.112 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.112 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.113 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.113 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.113 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.113 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.113 INFO:teuthology.orchestra.run.smithi052.stdout: "ENABLE_BROWSABLE_API": { 2023-11-27T15:28:11.113 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-27T15:28:11.113 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.113 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.113 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.114 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.114 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.114 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.114 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.114 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.114 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.114 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.115 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.115 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.115 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-27T15:28:11.115 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-27T15:28:11.115 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.115 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.115 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.115 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.116 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.116 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.116 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.116 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.116 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.116 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.116 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.116 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.116 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-27T15:28:11.117 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-27T15:28:11.117 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.117 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.117 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.117 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.117 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.117 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.118 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.118 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.118 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.118 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.118 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.118 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.118 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-27T15:28:11.119 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-27T15:28:11.119 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.119 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.119 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.119 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.119 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.119 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.119 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.120 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.120 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.120 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.120 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.120 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.120 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-27T15:28:11.120 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-27T15:28:11.120 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.121 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.121 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.121 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.121 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.121 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.121 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.121 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.121 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.122 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.122 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.122 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.122 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-27T15:28:11.122 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-27T15:28:11.122 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.122 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.122 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.123 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.123 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.123 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.123 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.123 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.123 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.123 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.123 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.123 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.124 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-27T15:28:11.124 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-27T15:28:11.124 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.124 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.124 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.124 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.124 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.124 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.125 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.125 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.125 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.125 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.125 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.125 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.125 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-27T15:28:11.125 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-27T15:28:11.126 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.126 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.126 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.126 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.126 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.126 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.126 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.126 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.127 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.127 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.127 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.127 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.127 INFO:teuthology.orchestra.run.smithi052.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-27T15:28:11.127 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-27T15:28:11.127 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.128 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.128 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.128 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.128 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.128 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.128 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.128 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.128 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.129 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.129 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.129 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.129 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_PASSWORD": { 2023-11-27T15:28:11.129 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-27T15:28:11.129 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.129 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.130 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.130 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2023-11-27T15:28:11.130 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.130 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.130 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.130 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.130 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.131 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.131 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.131 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.131 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-27T15:28:11.131 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-27T15:28:11.131 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.131 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.131 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.132 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.132 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.132 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.132 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.132 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.132 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.132 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.132 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.133 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.133 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_URL": { 2023-11-27T15:28:11.133 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_URL", 2023-11-27T15:28:11.133 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.133 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.133 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.133 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.133 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.134 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.134 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.134 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.134 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.134 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.134 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.134 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.134 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_USERNAME": { 2023-11-27T15:28:11.135 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-27T15:28:11.135 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.135 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.135 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.135 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2023-11-27T15:28:11.135 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.135 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.136 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.136 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.136 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.136 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.136 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.136 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.136 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-27T15:28:11.136 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-27T15:28:11.137 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.137 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.137 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.137 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.137 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.137 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.137 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.137 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.138 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.138 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.138 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.138 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.138 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-27T15:28:11.138 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-27T15:28:11.138 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.139 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.139 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.139 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.139 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.139 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.139 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.139 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.139 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.140 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.140 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.140 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.140 INFO:teuthology.orchestra.run.smithi052.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-27T15:28:11.140 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-27T15:28:11.140 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.140 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.140 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.141 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.141 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.141 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.141 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.141 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.141 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.141 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.141 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.142 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.142 INFO:teuthology.orchestra.run.smithi052.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-27T15:28:11.142 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-27T15:28:11.142 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.142 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.142 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.143 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.143 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.143 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.143 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.143 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.143 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.143 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.143 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.144 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.144 INFO:teuthology.orchestra.run.smithi052.stdout: "PROMETHEUS_API_HOST": { 2023-11-27T15:28:11.144 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-27T15:28:11.144 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.144 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.144 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.144 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.144 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.145 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.145 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.145 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.145 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.145 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.145 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.145 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.146 INFO:teuthology.orchestra.run.smithi052.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-27T15:28:11.146 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-27T15:28:11.146 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.146 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.146 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.146 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.146 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.146 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.147 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.147 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.147 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.147 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.147 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.147 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.147 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-27T15:28:11.148 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-27T15:28:11.148 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.148 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.148 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.148 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.148 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.148 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.148 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.149 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.149 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.149 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.149 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.149 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.149 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-27T15:28:11.149 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-27T15:28:11.149 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.150 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.150 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.150 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.150 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.150 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.150 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.150 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.150 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.151 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.151 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.151 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.151 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-27T15:28:11.151 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-27T15:28:11.151 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.151 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.151 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.152 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.152 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.152 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.152 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.152 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.152 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.152 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.152 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.153 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.153 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-27T15:28:11.153 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-27T15:28:11.153 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.153 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.153 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.153 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.153 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.154 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.154 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.154 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.154 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.154 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.154 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.154 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.155 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-27T15:28:11.155 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-27T15:28:11.155 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.155 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.155 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.155 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.155 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.155 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.156 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.156 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.156 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.156 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.156 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.156 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.156 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-27T15:28:11.156 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-27T15:28:11.157 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.157 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.157 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.157 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.157 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.157 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.157 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.157 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.158 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.158 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.158 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.158 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.158 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-27T15:28:11.158 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-27T15:28:11.158 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.158 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.159 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.159 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.159 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.159 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.159 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.159 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.159 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.159 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.160 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.160 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.160 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_ENABLED": { 2023-11-27T15:28:11.160 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-27T15:28:11.160 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.160 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.160 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.160 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.161 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.161 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.161 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.161 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.161 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.161 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.161 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.161 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.162 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-27T15:28:11.162 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-27T15:28:11.162 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.162 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.162 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.162 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-27T15:28:11.162 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.162 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.163 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.163 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.163 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.163 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.163 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.163 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.163 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-27T15:28:11.163 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-27T15:28:11.164 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.164 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.164 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.164 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2023-11-27T15:28:11.164 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.164 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.164 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.164 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.165 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.165 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.165 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.165 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.165 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-27T15:28:11.165 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-27T15:28:11.165 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.165 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.166 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.166 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8", 2023-11-27T15:28:11.166 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.166 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.166 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.166 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.166 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.167 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.167 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.167 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.167 INFO:teuthology.orchestra.run.smithi052.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-27T15:28:11.167 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-27T15:28:11.167 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.167 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.167 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.168 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "45", 2023-11-27T15:28:11.168 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.168 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.168 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.168 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.168 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.168 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.168 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.169 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.169 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_ACCESS_KEY": { 2023-11-27T15:28:11.169 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-27T15:28:11.169 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.169 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.169 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.170 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.170 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.170 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.170 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.170 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.170 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.170 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.170 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.171 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.171 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-27T15:28:11.171 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-27T15:28:11.171 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.171 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.171 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.171 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2023-11-27T15:28:11.171 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.172 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.172 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.172 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.172 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.172 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.172 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.172 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.172 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_SECRET_KEY": { 2023-11-27T15:28:11.173 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-27T15:28:11.173 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.173 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.173 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.173 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.173 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.173 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.173 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.174 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.174 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.174 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.174 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.174 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.174 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_SSL_VERIFY": { 2023-11-27T15:28:11.174 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-27T15:28:11.174 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.175 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.175 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.175 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.175 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.175 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.175 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.175 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.175 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.176 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.176 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.176 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.176 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-27T15:28:11.176 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-27T15:28:11.176 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.176 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.176 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.177 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2023-11-27T15:28:11.177 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.177 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.177 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.177 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.177 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.177 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.178 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.178 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.178 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-27T15:28:11.178 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-27T15:28:11.178 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.178 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.178 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.178 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2023-11-27T15:28:11.179 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.179 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.179 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.179 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.179 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.179 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.179 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.179 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.180 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-27T15:28:11.180 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-27T15:28:11.180 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.180 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.180 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.180 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2023-11-27T15:28:11.180 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.180 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.180 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.181 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.181 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.181 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.181 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.181 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.181 INFO:teuthology.orchestra.run.smithi052.stdout: "cross_origin_url": { 2023-11-27T15:28:11.181 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cross_origin_url", 2023-11-27T15:28:11.181 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.182 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.182 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.182 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.182 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.182 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.182 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.182 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.183 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.183 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.183 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.183 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.183 INFO:teuthology.orchestra.run.smithi052.stdout: "crt_file": { 2023-11-27T15:28:11.183 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crt_file", 2023-11-27T15:28:11.183 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.183 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.184 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.184 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.184 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.184 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.184 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.184 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.184 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.184 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.185 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.185 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.185 INFO:teuthology.orchestra.run.smithi052.stdout: "debug": { 2023-11-27T15:28:11.185 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "debug", 2023-11-27T15:28:11.185 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.185 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.185 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.185 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.186 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.186 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.186 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.186 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable/disable debug options", 2023-11-27T15:28:11.186 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.186 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.186 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.187 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.187 INFO:teuthology.orchestra.run.smithi052.stdout: "jwt_token_ttl": { 2023-11-27T15:28:11.187 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "jwt_token_ttl", 2023-11-27T15:28:11.187 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.187 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.187 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.187 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "28800", 2023-11-27T15:28:11.187 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.188 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.188 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.188 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.188 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.188 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.188 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.188 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.188 INFO:teuthology.orchestra.run.smithi052.stdout: "key_file": { 2023-11-27T15:28:11.189 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "key_file", 2023-11-27T15:28:11.189 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.189 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.189 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.189 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.189 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.189 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.190 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.190 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.190 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.190 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.190 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.190 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.190 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.190 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.191 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.191 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.191 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.191 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.191 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.191 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.191 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.191 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.192 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.192 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.192 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.192 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.192 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.192 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.192 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.193 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.193 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.193 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.193 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.193 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.193 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.193 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.193 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.194 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.194 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.194 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.194 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.194 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.194 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.194 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.194 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.195 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.195 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.195 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.195 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.195 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.195 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.195 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.195 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.196 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.196 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.196 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.196 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.196 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.196 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.197 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.197 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.197 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.197 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.197 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.197 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.197 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.197 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.198 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.198 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.198 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.198 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.198 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.198 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.198 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.198 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.199 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.199 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.199 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.199 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.199 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.199 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.199 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.199 INFO:teuthology.orchestra.run.smithi052.stdout: "motd": { 2023-11-27T15:28:11.200 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "motd", 2023-11-27T15:28:11.200 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.200 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.200 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.200 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.200 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.201 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.201 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.201 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "The message of the day", 2023-11-27T15:28:11.201 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.201 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.201 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.201 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.201 INFO:teuthology.orchestra.run.smithi052.stdout: "redirect_resolve_ip_addr": { 2023-11-27T15:28:11.201 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "redirect_resolve_ip_addr", 2023-11-27T15:28:11.202 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.202 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.202 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.202 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.202 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.202 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.202 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.203 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.203 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.203 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.203 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.203 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.203 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2023-11-27T15:28:11.203 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2023-11-27T15:28:11.203 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.203 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.204 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.204 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "::", 2023-11-27T15:28:11.204 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.204 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.204 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.204 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.204 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.204 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.205 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.205 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.205 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2023-11-27T15:28:11.205 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2023-11-27T15:28:11.205 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.205 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.205 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.205 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8080", 2023-11-27T15:28:11.206 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.206 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.206 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.206 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.206 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.206 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.206 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.206 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.207 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl": { 2023-11-27T15:28:11.207 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl", 2023-11-27T15:28:11.207 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.207 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.207 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.207 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.207 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.207 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.208 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.208 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.208 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.208 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.208 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.208 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.208 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl_server_port": { 2023-11-27T15:28:11.208 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl_server_port", 2023-11-27T15:28:11.209 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.209 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.209 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.209 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8443", 2023-11-27T15:28:11.209 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.209 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.209 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.209 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.210 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.210 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.210 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.210 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.210 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_behaviour": { 2023-11-27T15:28:11.210 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_behaviour", 2023-11-27T15:28:11.210 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.210 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.211 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.211 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "redirect", 2023-11-27T15:28:11.211 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.211 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.211 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.211 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.211 INFO:teuthology.orchestra.run.smithi052.stdout: "redirect" 2023-11-27T15:28:11.211 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.212 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.212 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.212 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.212 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.212 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.212 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_error_status_code": { 2023-11-27T15:28:11.212 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_error_status_code", 2023-11-27T15:28:11.212 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.213 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.213 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.213 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "500", 2023-11-27T15:28:11.213 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "400", 2023-11-27T15:28:11.213 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "599", 2023-11-27T15:28:11.213 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.213 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.213 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.214 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.214 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.214 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.214 INFO:teuthology.orchestra.run.smithi052.stdout: "url_prefix": { 2023-11-27T15:28:11.214 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "url_prefix", 2023-11-27T15:28:11.214 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.214 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.214 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.215 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.215 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.215 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.215 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.215 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.215 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.215 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.215 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.216 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.216 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.216 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.216 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.216 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "devicehealth", 2023-11-27T15:28:11.216 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:11.216 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:11.216 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.217 INFO:teuthology.orchestra.run.smithi052.stdout: "enable_monitoring": { 2023-11-27T15:28:11.217 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enable_monitoring", 2023-11-27T15:28:11.217 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.217 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.217 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.217 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.217 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.218 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.218 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.218 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "monitor device health metrics", 2023-11-27T15:28:11.218 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.218 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.218 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.218 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.218 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.219 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.219 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.219 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.219 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.219 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.219 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.219 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.219 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.220 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.220 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.220 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.220 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.220 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.220 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.220 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.220 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.221 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.221 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.221 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.221 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.221 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.221 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.221 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.221 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.222 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.222 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.222 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.222 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.222 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.222 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.222 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.222 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.223 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.223 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.223 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.223 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.223 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.223 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.223 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.223 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.224 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.224 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.224 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.224 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.224 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.224 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.224 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.224 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.225 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.225 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.225 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.225 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.225 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.225 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.225 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.225 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.226 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.226 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.226 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.226 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.226 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.226 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.226 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.226 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.227 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.227 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.227 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.227 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.227 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.227 INFO:teuthology.orchestra.run.smithi052.stdout: "mark_out_threshold": { 2023-11-27T15:28:11.227 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mark_out_threshold", 2023-11-27T15:28:11.227 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:11.228 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.228 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.228 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "2419200", 2023-11-27T15:28:11.228 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.228 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.228 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.228 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-27T15:28:11.229 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.229 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.229 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.229 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.229 INFO:teuthology.orchestra.run.smithi052.stdout: "pool_name": { 2023-11-27T15:28:11.229 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pool_name", 2023-11-27T15:28:11.229 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.229 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.230 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.230 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "device_health_metrics", 2023-11-27T15:28:11.230 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.230 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.230 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.230 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-27T15:28:11.230 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.230 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.231 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.231 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.231 INFO:teuthology.orchestra.run.smithi052.stdout: "retention_period": { 2023-11-27T15:28:11.231 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "retention_period", 2023-11-27T15:28:11.231 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:11.231 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.231 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.231 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15552000", 2023-11-27T15:28:11.232 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.232 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.232 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.232 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long to retain device health metrics", 2023-11-27T15:28:11.232 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.232 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.232 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.232 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.233 INFO:teuthology.orchestra.run.smithi052.stdout: "scrape_frequency": { 2023-11-27T15:28:11.233 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "scrape_frequency", 2023-11-27T15:28:11.233 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:11.233 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.233 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.233 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "86400", 2023-11-27T15:28:11.233 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.233 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.234 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.234 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-27T15:28:11.234 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.234 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.234 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.234 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.234 INFO:teuthology.orchestra.run.smithi052.stdout: "self_heal": { 2023-11-27T15:28:11.235 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "self_heal", 2023-11-27T15:28:11.235 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.235 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.235 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.235 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.235 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.235 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.236 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.236 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-27T15:28:11.236 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.236 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.236 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.236 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.236 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2023-11-27T15:28:11.236 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2023-11-27T15:28:11.237 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:11.237 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.237 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.237 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2023-11-27T15:28:11.237 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.237 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.237 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.237 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to wake up and check device health", 2023-11-27T15:28:11.238 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.238 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.238 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.238 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.238 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_threshold": { 2023-11-27T15:28:11.238 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_threshold", 2023-11-27T15:28:11.238 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:11.239 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.239 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.239 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "7257600", 2023-11-27T15:28:11.239 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.239 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.239 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.239 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-27T15:28:11.239 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.240 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.240 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.240 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.240 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.240 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.240 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.240 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "diskprediction_local", 2023-11-27T15:28:11.240 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:11.241 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:11.241 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.241 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.241 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.241 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.241 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.241 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.242 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.242 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.242 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.242 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.242 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.242 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.242 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.242 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.243 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.243 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.243 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.243 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.243 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.243 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.244 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.244 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.244 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.244 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.244 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.244 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.244 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.244 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.245 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.245 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.245 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.245 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.245 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.245 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.245 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.245 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.246 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.246 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.246 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.246 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.246 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.246 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.246 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.247 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.247 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.247 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.247 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.247 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.247 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.247 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.247 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.248 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.248 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.248 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.248 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.248 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.248 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.248 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.248 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.249 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.249 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.249 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.249 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.249 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.249 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.249 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.249 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.250 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.250 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.250 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.250 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.250 INFO:teuthology.orchestra.run.smithi052.stdout: "predict_interval": { 2023-11-27T15:28:11.250 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "predict_interval", 2023-11-27T15:28:11.250 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.251 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.251 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.251 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "86400", 2023-11-27T15:28:11.251 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.251 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.251 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.251 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.251 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.252 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.252 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.252 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.252 INFO:teuthology.orchestra.run.smithi052.stdout: "predictor_model": { 2023-11-27T15:28:11.252 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "predictor_model", 2023-11-27T15:28:11.252 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.252 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.252 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.253 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "prophetstor", 2023-11-27T15:28:11.253 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.253 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.253 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.253 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.253 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.253 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.254 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.254 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.254 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2023-11-27T15:28:11.254 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2023-11-27T15:28:11.254 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.254 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.254 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.254 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2023-11-27T15:28:11.255 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.255 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.255 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.255 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.255 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.255 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.255 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.255 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.256 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.256 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.256 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.256 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "influx", 2023-11-27T15:28:11.256 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": false, 2023-11-27T15:28:11.256 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "influxdb python module not found", 2023-11-27T15:28:11.256 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.256 INFO:teuthology.orchestra.run.smithi052.stdout: "batch_size": { 2023-11-27T15:28:11.257 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "batch_size", 2023-11-27T15:28:11.257 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.257 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.257 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.257 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5000", 2023-11-27T15:28:11.257 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.257 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.257 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.258 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-27T15:28:11.258 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.258 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.258 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.258 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.258 INFO:teuthology.orchestra.run.smithi052.stdout: "database": { 2023-11-27T15:28:11.258 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "database", 2023-11-27T15:28:11.259 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.259 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.259 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.259 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "ceph", 2023-11-27T15:28:11.259 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.259 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.259 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.259 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-27T15:28:11.259 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.260 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.260 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.260 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.260 INFO:teuthology.orchestra.run.smithi052.stdout: "hostname": { 2023-11-27T15:28:11.260 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "hostname", 2023-11-27T15:28:11.260 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.260 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.260 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.261 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.261 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.261 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.261 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.261 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB server hostname", 2023-11-27T15:28:11.261 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.261 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.261 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.262 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.262 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2023-11-27T15:28:11.262 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2023-11-27T15:28:11.262 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:11.262 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.262 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.262 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "30", 2023-11-27T15:28:11.262 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "5", 2023-11-27T15:28:11.263 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.263 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.263 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-27T15:28:11.263 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.263 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.263 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.263 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.263 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.264 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.264 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.264 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.264 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.264 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.264 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.264 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.264 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.265 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.265 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.265 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.265 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.265 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.265 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.265 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.265 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.266 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.266 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.266 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.266 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.266 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.266 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.266 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.266 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.267 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.267 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.267 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.267 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.267 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.267 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.267 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.268 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.268 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.268 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.268 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.268 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.268 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.268 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.268 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.269 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.269 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.269 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.269 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.269 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.269 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.269 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.270 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.270 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.270 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.270 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.270 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.270 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.270 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.270 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.271 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.271 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.271 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.271 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.271 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.271 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.271 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.271 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.271 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.272 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.272 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.272 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.272 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.272 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.272 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.272 INFO:teuthology.orchestra.run.smithi052.stdout: "password": { 2023-11-27T15:28:11.272 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "password", 2023-11-27T15:28:11.273 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.273 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.273 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.273 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.273 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.273 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.273 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.274 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "password of InfluxDB server user", 2023-11-27T15:28:11.274 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.274 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.274 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.274 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.274 INFO:teuthology.orchestra.run.smithi052.stdout: "port": { 2023-11-27T15:28:11.274 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "port", 2023-11-27T15:28:11.274 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.275 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.275 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.275 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8086", 2023-11-27T15:28:11.275 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.275 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.275 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.276 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB server port", 2023-11-27T15:28:11.276 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.276 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.276 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.276 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.276 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl": { 2023-11-27T15:28:11.276 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl", 2023-11-27T15:28:11.277 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.277 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.277 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.277 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "false", 2023-11-27T15:28:11.277 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.278 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.278 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.278 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-27T15:28:11.278 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.278 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.278 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.278 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.279 INFO:teuthology.orchestra.run.smithi052.stdout: "threads": { 2023-11-27T15:28:11.279 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "threads", 2023-11-27T15:28:11.279 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.279 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.279 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.279 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2023-11-27T15:28:11.279 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2023-11-27T15:28:11.279 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "32", 2023-11-27T15:28:11.280 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.280 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-27T15:28:11.280 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.280 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.280 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.280 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.280 INFO:teuthology.orchestra.run.smithi052.stdout: "username": { 2023-11-27T15:28:11.280 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "username", 2023-11-27T15:28:11.281 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.281 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.281 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.281 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.281 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.281 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.281 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.281 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "username of InfluxDB server user", 2023-11-27T15:28:11.282 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.282 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.282 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.282 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.282 INFO:teuthology.orchestra.run.smithi052.stdout: "verify_ssl": { 2023-11-27T15:28:11.282 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "verify_ssl", 2023-11-27T15:28:11.282 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.283 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.283 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.283 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "true", 2023-11-27T15:28:11.283 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.283 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.283 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.283 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-27T15:28:11.283 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.284 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.284 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.284 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.284 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.284 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.284 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.284 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "insights", 2023-11-27T15:28:11.285 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:11.285 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:11.285 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.285 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.285 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.285 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.285 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.286 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.286 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.286 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.286 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.286 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.286 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.286 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.286 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.287 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.287 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.287 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.287 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.287 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.287 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.287 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.288 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.288 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.288 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.288 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.288 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.288 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.288 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.288 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.289 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.289 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.289 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.289 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.289 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.289 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.289 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.289 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.290 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.290 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.290 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.290 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.290 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.290 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.290 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.291 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.291 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.291 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.291 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.291 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.291 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.291 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.291 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.292 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.292 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.292 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.292 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.292 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.292 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.292 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.293 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.293 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.293 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.293 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.293 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.293 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.293 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.294 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.294 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.294 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.294 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.294 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.294 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.294 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.294 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.295 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.295 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "iostat", 2023-11-27T15:28:11.295 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:11.295 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:11.295 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.295 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.295 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.296 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.296 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.296 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.296 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.296 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.296 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.296 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.296 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.297 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.297 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.297 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.297 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.297 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.297 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.297 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.297 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.298 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.298 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.298 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.298 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.298 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.298 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.298 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.299 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.299 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.299 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.299 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.299 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.299 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.299 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.299 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.300 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.300 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.300 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.300 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.300 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.300 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.300 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.301 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.301 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.301 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.301 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.301 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.301 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.301 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.301 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.302 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.302 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.302 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.302 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.302 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.302 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.302 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.303 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.303 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.303 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.303 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.303 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.303 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.303 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.303 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.304 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.304 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.304 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.304 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.304 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.304 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.304 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.304 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.305 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.305 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.305 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "k8sevents", 2023-11-27T15:28:11.305 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:11.305 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:11.305 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.305 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_event_retention_days": { 2023-11-27T15:28:11.305 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ceph_event_retention_days", 2023-11-27T15:28:11.306 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.306 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.306 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.306 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "7", 2023-11-27T15:28:11.306 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.306 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.306 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.306 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Days to hold ceph event information within local cache", 2023-11-27T15:28:11.307 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.307 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.307 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.307 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.307 INFO:teuthology.orchestra.run.smithi052.stdout: "config_check_secs": { 2023-11-27T15:28:11.307 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_check_secs", 2023-11-27T15:28:11.307 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.308 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.308 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.308 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2023-11-27T15:28:11.308 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "10", 2023-11-27T15:28:11.308 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.308 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.308 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-11-27T15:28:11.308 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.309 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.309 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.309 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.309 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.309 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.309 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.309 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.309 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.310 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.310 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.310 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.310 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.310 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.310 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.310 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.311 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.311 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.311 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.311 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.311 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.311 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.311 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.311 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.312 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.312 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.312 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.312 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.312 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.312 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.312 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.313 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.313 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.313 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.313 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.313 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.313 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.313 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.313 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.314 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.314 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.314 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.314 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.314 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.314 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.314 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.314 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.315 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.315 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.315 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.315 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.315 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.315 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.315 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.315 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.316 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.316 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.316 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.316 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.316 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.316 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.316 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.316 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.317 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.317 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.317 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.317 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.317 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.317 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.317 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.318 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.318 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.318 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.318 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.318 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.318 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.318 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.318 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "localpool", 2023-11-27T15:28:11.319 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:11.319 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:11.319 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.319 INFO:teuthology.orchestra.run.smithi052.stdout: "failure_domain": { 2023-11-27T15:28:11.319 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "failure_domain", 2023-11-27T15:28:11.319 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.319 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.319 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.320 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "host", 2023-11-27T15:28:11.320 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.320 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.320 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.320 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "failure domain for any created local pool", 2023-11-27T15:28:11.320 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-27T15:28:11.320 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.320 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.321 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.321 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.321 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.321 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.321 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.321 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.321 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.321 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.322 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.322 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.322 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.322 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.322 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.322 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.322 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.322 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.323 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.323 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.323 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.323 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.323 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.323 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.323 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.323 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.324 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.324 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.324 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.324 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.324 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.324 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.324 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.324 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.324 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.325 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.325 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.325 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.325 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.325 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.325 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.325 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.325 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.326 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.326 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.326 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.326 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.326 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.326 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.326 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.327 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.327 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.327 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.327 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.327 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.327 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.327 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.327 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.328 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.328 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.328 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.328 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.328 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.328 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.328 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.328 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.329 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.329 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.329 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.329 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.329 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.329 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.329 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.330 INFO:teuthology.orchestra.run.smithi052.stdout: "min_size": { 2023-11-27T15:28:11.330 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "min_size", 2023-11-27T15:28:11.330 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.330 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.330 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.330 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.330 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.330 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.331 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.331 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default min_size for any created local pool", 2023-11-27T15:28:11.331 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-27T15:28:11.331 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.331 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.331 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.331 INFO:teuthology.orchestra.run.smithi052.stdout: "num_rep": { 2023-11-27T15:28:11.332 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "num_rep", 2023-11-27T15:28:11.332 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.332 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.332 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.332 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3", 2023-11-27T15:28:11.332 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.332 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.332 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.333 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default replica count for any created local pool", 2023-11-27T15:28:11.333 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.333 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.333 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.333 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.333 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_num": { 2023-11-27T15:28:11.333 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pg_num", 2023-11-27T15:28:11.333 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.334 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.334 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.334 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "128", 2023-11-27T15:28:11.334 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.334 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.334 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.334 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default pg_num for any created local pool", 2023-11-27T15:28:11.334 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.335 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.335 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.335 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.335 INFO:teuthology.orchestra.run.smithi052.stdout: "prefix": { 2023-11-27T15:28:11.335 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prefix", 2023-11-27T15:28:11.335 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.335 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.336 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.336 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.336 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.336 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.336 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.336 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "name prefix for any created local pool", 2023-11-27T15:28:11.336 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.336 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.337 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.337 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.337 INFO:teuthology.orchestra.run.smithi052.stdout: "subtree": { 2023-11-27T15:28:11.337 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "subtree", 2023-11-27T15:28:11.337 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.337 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.337 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.337 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "rack", 2023-11-27T15:28:11.338 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.338 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.338 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.338 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-27T15:28:11.339 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-27T15:28:11.339 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.339 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.339 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.339 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.339 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.339 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.339 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mds_autoscaler", 2023-11-27T15:28:11.340 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:11.340 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:11.340 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.340 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.340 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.340 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.340 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.340 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.341 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.341 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.341 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.341 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.341 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.341 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.341 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.342 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.342 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.342 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.342 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.342 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.342 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.342 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.342 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.343 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.343 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.343 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.343 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.343 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.343 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.343 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.344 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.344 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.344 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.344 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.344 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.344 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.344 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.344 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.345 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.345 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.345 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.345 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.345 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.345 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.345 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.345 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.346 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.346 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.346 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.346 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.346 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.346 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.346 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.347 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.347 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.347 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.347 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.347 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.347 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.347 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.347 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.348 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.348 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.348 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.348 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.348 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.348 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.348 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.348 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.349 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.349 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.349 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.349 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.349 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.349 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.349 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.349 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mirroring", 2023-11-27T15:28:11.350 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:11.350 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:11.350 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.350 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.350 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.350 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.350 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.351 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.351 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.351 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.351 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.352 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.352 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.352 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.352 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.353 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.353 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.353 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.353 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.353 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.353 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.353 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.353 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.353 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.354 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.354 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.354 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.354 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.354 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.354 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.354 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.355 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.355 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.355 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.355 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.355 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.355 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.355 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.356 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.356 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.356 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.356 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.356 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.356 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.356 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.356 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.357 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.357 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.357 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.357 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.357 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.357 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.357 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.357 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.358 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.358 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.358 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.358 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.358 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.358 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.358 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.359 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.359 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.359 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.359 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.359 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.359 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.359 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.359 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.359 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.360 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.360 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.360 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.360 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.360 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.360 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.360 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "nfs", 2023-11-27T15:28:11.360 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:11.361 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:11.361 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.361 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.361 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.361 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.361 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.361 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.361 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.362 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.362 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.362 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.362 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.362 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.362 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.362 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.363 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.363 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.363 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.363 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.363 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.363 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.363 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.364 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.364 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.364 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.364 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.364 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.364 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.364 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.364 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.364 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.365 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.365 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.365 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.365 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.365 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.365 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.365 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.366 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.366 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.366 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.366 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.366 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.366 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.366 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.366 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.367 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.367 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.367 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.367 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.367 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.367 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.367 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.368 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.368 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.368 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.368 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.368 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.368 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.368 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.368 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.369 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.369 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.369 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.369 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.369 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.369 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.369 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.370 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.370 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.370 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.370 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.370 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.370 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.370 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.371 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "orchestrator", 2023-11-27T15:28:11.371 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:11.371 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:11.371 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.371 INFO:teuthology.orchestra.run.smithi052.stdout: "fail_fs": { 2023-11-27T15:28:11.371 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "fail_fs", 2023-11-27T15:28:11.371 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.371 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.372 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.372 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.372 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.372 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.372 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.372 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-27T15:28:11.372 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.372 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.373 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.373 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.373 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.373 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.373 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.373 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.373 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.373 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.374 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.374 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.374 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.374 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.374 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.374 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.374 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.375 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.375 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.375 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.375 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.375 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.375 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.375 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.375 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.376 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.376 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.376 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.376 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.376 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.376 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.376 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.377 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.377 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.377 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.377 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.377 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.377 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.377 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.378 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.378 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.378 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.378 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.378 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.378 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.378 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.379 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.379 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.379 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.379 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.379 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.379 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.379 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.379 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.380 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.380 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.380 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.380 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.380 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.380 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.380 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.380 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.381 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.381 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.381 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.381 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.381 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.381 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.381 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.381 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.382 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.382 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.382 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.382 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.382 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator": { 2023-11-27T15:28:11.382 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "orchestrator", 2023-11-27T15:28:11.382 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.383 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.383 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.383 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.383 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.383 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.383 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.383 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm", 2023-11-27T15:28:11.383 INFO:teuthology.orchestra.run.smithi052.stdout: "rook", 2023-11-27T15:28:11.384 INFO:teuthology.orchestra.run.smithi052.stdout: "test_orchestrator" 2023-11-27T15:28:11.384 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.384 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Orchestrator backend", 2023-11-27T15:28:11.384 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.384 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.384 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.384 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.384 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.385 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.385 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.385 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "osd_perf_query", 2023-11-27T15:28:11.385 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:11.385 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:11.385 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.385 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.385 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.386 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.386 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.386 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.386 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.386 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.386 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.386 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.387 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.387 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.387 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.387 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.387 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.387 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.387 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.387 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.388 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.388 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.388 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.388 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.388 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.388 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.388 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.389 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.389 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.389 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.389 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.389 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.389 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.389 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.389 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.390 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.390 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.390 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.390 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.390 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.390 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.390 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.390 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.391 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.391 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.391 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.391 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.391 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.391 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.391 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.391 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.392 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.392 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.392 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.392 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.392 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.392 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.392 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.393 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.393 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.393 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.393 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.393 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.393 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.393 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.393 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.393 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.394 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.394 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.394 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.394 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.394 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.394 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.394 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.395 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.395 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.395 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "osd_support", 2023-11-27T15:28:11.395 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:11.395 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:11.395 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.395 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.395 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.396 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.396 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.396 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.396 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.396 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.396 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.396 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.397 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.397 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.397 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.397 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.397 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.397 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.397 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.397 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.398 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.398 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.398 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.398 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.398 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.398 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.398 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.399 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.399 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.399 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.399 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.399 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.399 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.399 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.401 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.401 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.401 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.401 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.401 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.401 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.402 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.402 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.402 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.402 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.403 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.403 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.403 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.403 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.403 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.403 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.403 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.403 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.404 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.404 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.404 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.404 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.404 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.404 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.404 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.404 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.405 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.405 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.405 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.405 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.405 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.405 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.405 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.405 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.406 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.406 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.406 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.406 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.406 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.406 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.406 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.407 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.407 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pg_autoscaler", 2023-11-27T15:28:11.407 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:11.407 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:11.407 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.407 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.407 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.407 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.408 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.408 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.408 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.408 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.408 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.408 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.408 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.408 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.409 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.409 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.409 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.409 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.409 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.409 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.409 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.409 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.410 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.410 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.410 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.410 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.410 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.410 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.410 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.410 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.411 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.411 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.411 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.411 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.411 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.411 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.411 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.411 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.412 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.412 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.412 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.412 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.412 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.412 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.412 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.412 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.413 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.413 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.413 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.413 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.413 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.413 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.413 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.413 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.414 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.414 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.414 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.414 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.414 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.414 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.414 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.414 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.415 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.415 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.415 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.415 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.415 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.415 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.415 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.416 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.416 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.416 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.416 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.416 INFO:teuthology.orchestra.run.smithi052.stdout: "noautoscale": { 2023-11-27T15:28:11.416 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "noautoscale", 2023-11-27T15:28:11.416 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.416 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.417 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.417 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.417 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.417 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.417 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.417 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "global autoscale flag", 2023-11-27T15:28:11.417 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-11-27T15:28:11.417 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.418 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.418 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.418 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2023-11-27T15:28:11.418 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2023-11-27T15:28:11.418 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:11.418 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.418 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.419 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2023-11-27T15:28:11.419 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.419 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.419 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.419 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.419 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.419 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.419 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.420 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.420 INFO:teuthology.orchestra.run.smithi052.stdout: "threshold": { 2023-11-27T15:28:11.420 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "threshold", 2023-11-27T15:28:11.420 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2023-11-27T15:28:11.420 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.420 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.420 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3.0", 2023-11-27T15:28:11.420 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1.0", 2023-11-27T15:28:11.421 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.421 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.421 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "scaling threshold", 2023-11-27T15:28:11.421 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-27T15:28:11.421 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.421 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.421 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.421 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.422 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.422 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.422 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "progress", 2023-11-27T15:28:11.422 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:11.422 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:11.422 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.422 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_pg_recovery_event": { 2023-11-27T15:28:11.423 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_pg_recovery_event", 2023-11-27T15:28:11.423 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.423 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.423 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.423 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.423 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.423 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.423 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.424 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-27T15:28:11.424 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.424 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.424 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.424 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.424 INFO:teuthology.orchestra.run.smithi052.stdout: "enabled": { 2023-11-27T15:28:11.424 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enabled", 2023-11-27T15:28:11.424 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.425 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.425 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.425 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.425 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.425 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.425 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.425 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.426 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.426 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.426 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.426 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.426 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.426 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.426 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.427 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.427 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.427 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.427 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.427 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.427 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.427 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.427 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.428 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.428 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.428 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.428 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.428 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.428 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.428 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.428 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.429 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.429 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.429 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.429 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.429 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.429 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.429 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.430 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.430 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.430 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.430 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.430 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.430 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.430 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.431 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.431 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.431 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.431 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.431 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.431 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.431 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.431 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.432 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.432 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.432 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.432 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.432 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.432 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.432 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.433 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.433 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.433 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.433 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.433 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.433 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.433 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.433 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.434 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.434 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.434 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.434 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.434 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.434 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.434 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.435 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.435 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.435 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.435 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.435 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.435 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.435 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.435 INFO:teuthology.orchestra.run.smithi052.stdout: "max_completed_events": { 2023-11-27T15:28:11.436 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_completed_events", 2023-11-27T15:28:11.436 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.436 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.436 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.436 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "50", 2023-11-27T15:28:11.436 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.436 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.437 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.437 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "number of past completed events to remember", 2023-11-27T15:28:11.437 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.437 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.437 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.437 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.437 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2023-11-27T15:28:11.438 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2023-11-27T15:28:11.438 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:11.438 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.438 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.438 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2023-11-27T15:28:11.438 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.438 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.438 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.439 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long the module is going to sleep", 2023-11-27T15:28:11.439 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.439 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.439 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.439 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.439 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.439 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.440 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.440 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus", 2023-11-27T15:28:11.440 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:11.440 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:11.440 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.440 INFO:teuthology.orchestra.run.smithi052.stdout: "cache": { 2023-11-27T15:28:11.440 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cache", 2023-11-27T15:28:11.440 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.441 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.441 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.441 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.441 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.441 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.441 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.441 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.441 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.442 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.442 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.442 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.442 INFO:teuthology.orchestra.run.smithi052.stdout: "exclude_perf_counters": { 2023-11-27T15:28:11.442 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "exclude_perf_counters", 2023-11-27T15:28:11.442 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.442 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.442 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.443 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.443 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.443 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.443 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.443 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-27T15:28:11.443 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-27T15:28:11.443 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.444 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.444 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.444 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.444 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.444 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.444 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.444 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.444 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.445 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.445 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.445 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.445 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.445 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.445 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.445 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.445 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.446 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.446 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.446 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.446 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.446 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.446 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.446 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.447 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.447 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.447 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.447 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.447 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.447 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.447 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.447 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.448 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.448 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.448 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.448 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.448 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.448 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.448 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.448 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.449 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.449 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.449 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.449 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.449 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.449 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.449 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.450 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.450 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.450 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.450 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.450 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.450 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.450 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.450 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.451 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.451 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.451 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.451 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.451 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.451 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.451 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.451 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.452 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.452 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.452 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.452 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.452 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.452 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.452 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.453 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.453 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.453 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.453 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_stats_pools": { 2023-11-27T15:28:11.453 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_stats_pools", 2023-11-27T15:28:11.453 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.453 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.453 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.454 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.454 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.454 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.454 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.454 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.454 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.454 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.455 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.455 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.455 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-27T15:28:11.455 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-27T15:28:11.455 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.455 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.455 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.455 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "300", 2023-11-27T15:28:11.456 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.456 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.456 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.456 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.456 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.456 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.456 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.456 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.457 INFO:teuthology.orchestra.run.smithi052.stdout: "scrape_interval": { 2023-11-27T15:28:11.457 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "scrape_interval", 2023-11-27T15:28:11.457 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2023-11-27T15:28:11.457 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.457 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.457 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15.0", 2023-11-27T15:28:11.457 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.457 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.458 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.458 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.458 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.458 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.458 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.458 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.458 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2023-11-27T15:28:11.459 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2023-11-27T15:28:11.459 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.459 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.459 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.459 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "::", 2023-11-27T15:28:11.459 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.459 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.459 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.460 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-27T15:28:11.460 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.460 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.460 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.460 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.460 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2023-11-27T15:28:11.460 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2023-11-27T15:28:11.460 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.461 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.461 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.461 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "9283", 2023-11-27T15:28:11.461 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.461 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.461 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.461 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-27T15:28:11.461 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.462 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.462 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.462 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.462 INFO:teuthology.orchestra.run.smithi052.stdout: "stale_cache_strategy": { 2023-11-27T15:28:11.462 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "stale_cache_strategy", 2023-11-27T15:28:11.462 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.462 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.462 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.463 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "log", 2023-11-27T15:28:11.463 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.463 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.463 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.463 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.463 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.463 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.463 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.464 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.464 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_behaviour": { 2023-11-27T15:28:11.464 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_behaviour", 2023-11-27T15:28:11.464 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.464 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.464 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.464 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "default", 2023-11-27T15:28:11.464 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.465 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.465 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.465 INFO:teuthology.orchestra.run.smithi052.stdout: "default", 2023-11-27T15:28:11.465 INFO:teuthology.orchestra.run.smithi052.stdout: "error" 2023-11-27T15:28:11.465 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.465 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.465 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.465 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.466 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.466 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.466 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_error_status_code": { 2023-11-27T15:28:11.466 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_error_status_code", 2023-11-27T15:28:11.466 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.466 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.466 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.466 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "500", 2023-11-27T15:28:11.467 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "400", 2023-11-27T15:28:11.467 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "599", 2023-11-27T15:28:11.467 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.467 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.467 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.467 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.467 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.467 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.468 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.468 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.468 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.468 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_support", 2023-11-27T15:28:11.468 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:11.468 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:11.468 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.469 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.469 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.469 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.469 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.469 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.469 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.469 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.469 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.470 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.470 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.470 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.470 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.470 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.470 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.470 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.470 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.471 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.471 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.471 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.471 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.471 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.471 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.471 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.471 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.472 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.472 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.472 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.472 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.472 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.472 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.472 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.472 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.472 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.473 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.473 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.473 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.473 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.473 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.473 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.473 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.473 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.474 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.474 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.474 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.474 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.474 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.474 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.474 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.475 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.475 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.475 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.475 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.475 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.475 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.475 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.475 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.476 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.476 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.476 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.476 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.476 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.476 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.476 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.476 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.477 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.477 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.477 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.477 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.477 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.477 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.477 INFO:teuthology.orchestra.run.smithi052.stdout: "max_concurrent_snap_create": { 2023-11-27T15:28:11.477 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_concurrent_snap_create", 2023-11-27T15:28:11.478 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.478 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.478 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.478 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2023-11-27T15:28:11.478 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.478 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.479 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.479 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.479 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.479 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.479 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.479 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.479 INFO:teuthology.orchestra.run.smithi052.stdout: "mirror_snapshot_schedule": { 2023-11-27T15:28:11.479 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mirror_snapshot_schedule", 2023-11-27T15:28:11.479 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.480 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.480 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.480 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.480 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.480 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.480 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.480 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.480 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.481 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.481 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.481 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.481 INFO:teuthology.orchestra.run.smithi052.stdout: "trash_purge_schedule": { 2023-11-27T15:28:11.481 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "trash_purge_schedule", 2023-11-27T15:28:11.481 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.481 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.481 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.482 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.482 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.482 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.482 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.482 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.482 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.482 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.482 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.483 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.483 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.483 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.483 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.483 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "restful", 2023-11-27T15:28:11.483 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:11.483 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:11.484 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.484 INFO:teuthology.orchestra.run.smithi052.stdout: "enable_auth": { 2023-11-27T15:28:11.484 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enable_auth", 2023-11-27T15:28:11.484 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.484 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.484 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.484 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.484 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.484 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.485 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.485 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.485 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.485 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.485 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.485 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.485 INFO:teuthology.orchestra.run.smithi052.stdout: "key_file": { 2023-11-27T15:28:11.486 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "key_file", 2023-11-27T15:28:11.486 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.486 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.486 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.486 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.486 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.486 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.486 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.486 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.487 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.487 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.487 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.487 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.487 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.487 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.487 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.488 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.488 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.488 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.488 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.488 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.488 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.488 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.488 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.489 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.489 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.489 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.489 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.489 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.489 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.489 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.489 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.490 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.490 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.490 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.490 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.490 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.490 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.490 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.491 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.491 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.491 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.491 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.491 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.491 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.491 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.491 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.492 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.492 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.492 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.492 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.492 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.492 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.492 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.492 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.493 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.493 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.493 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.493 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.493 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.493 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.493 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.493 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.494 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.494 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.494 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.494 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.494 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.494 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.494 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.495 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.495 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.495 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.495 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.495 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.495 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.495 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.495 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.495 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.496 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.496 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.496 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.496 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.496 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2023-11-27T15:28:11.496 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2023-11-27T15:28:11.496 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.497 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.497 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.497 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.497 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.497 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.497 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.497 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.497 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.498 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.498 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.498 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.498 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2023-11-27T15:28:11.498 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2023-11-27T15:28:11.498 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.498 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.498 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.499 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.499 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.499 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.499 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.499 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.499 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.499 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.499 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.499 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.500 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.500 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.500 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.500 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rgw", 2023-11-27T15:28:11.500 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:11.500 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:11.500 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.500 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.501 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.501 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.501 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.501 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.501 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.501 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.501 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.501 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.502 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.502 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.502 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.502 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.502 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.502 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.502 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.502 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.502 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.503 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.503 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.503 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.503 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.503 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.503 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.503 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.503 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.504 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.504 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.504 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.504 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.504 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.504 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.504 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.504 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.505 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.505 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.505 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.505 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.505 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.505 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.505 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.505 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.505 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.506 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.506 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.506 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.506 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.506 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.506 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.506 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.506 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.507 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.507 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.507 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.507 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.507 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.507 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.507 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.507 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.507 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.508 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.508 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.508 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.508 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.508 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.508 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.508 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.508 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.508 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.509 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.509 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.509 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.509 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.509 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rook", 2023-11-27T15:28:11.509 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:11.509 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:11.509 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.510 INFO:teuthology.orchestra.run.smithi052.stdout: "drive_group_interval": { 2023-11-27T15:28:11.510 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "drive_group_interval", 2023-11-27T15:28:11.510 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2023-11-27T15:28:11.510 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.510 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.510 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "300.0", 2023-11-27T15:28:11.510 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.511 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.511 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.511 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-11-27T15:28:11.511 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.511 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.511 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.511 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.511 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.511 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.512 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.512 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.512 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.512 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.512 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.512 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.512 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.512 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.513 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.513 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.513 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.513 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.513 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.513 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.513 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.513 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.514 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.514 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.514 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.514 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.514 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.514 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.514 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.514 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.515 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.515 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.515 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.515 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.515 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.515 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.515 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.515 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.516 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.516 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.516 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.516 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.516 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.516 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.516 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.516 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.517 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.517 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.517 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.517 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.517 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.517 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.517 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.518 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.518 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.518 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.518 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.518 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.518 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.518 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.519 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.519 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.519 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.519 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.519 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.519 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.519 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.519 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.520 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.520 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.520 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.520 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.520 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.520 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.520 INFO:teuthology.orchestra.run.smithi052.stdout: "storage_class": { 2023-11-27T15:28:11.520 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "storage_class", 2023-11-27T15:28:11.521 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.521 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.521 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.521 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "local", 2023-11-27T15:28:11.521 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.521 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.521 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.521 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-11-27T15:28:11.522 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.522 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.522 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.522 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.522 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.522 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.522 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.523 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "selftest", 2023-11-27T15:28:11.523 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:11.523 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:11.523 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.523 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.523 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.523 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.523 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.524 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.524 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.524 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.524 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.524 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.524 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.524 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.524 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.525 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.525 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.525 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.525 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.525 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.525 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.525 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.525 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.526 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.526 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.526 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.526 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.526 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.526 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.526 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.526 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.527 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.527 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.527 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.527 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.527 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.527 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.527 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.527 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.528 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.528 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.528 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.528 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.528 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.528 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.528 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.528 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.528 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.529 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.529 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.529 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.529 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.529 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.529 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.529 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.529 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.530 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.530 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.530 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.530 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.530 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.530 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.530 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.530 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.531 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.531 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.531 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.531 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.531 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.531 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.531 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.531 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.532 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.532 INFO:teuthology.orchestra.run.smithi052.stdout: "roption1": { 2023-11-27T15:28:11.532 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "roption1", 2023-11-27T15:28:11.532 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.532 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.532 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.532 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.532 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.533 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.533 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.533 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.533 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.533 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.533 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.533 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.533 INFO:teuthology.orchestra.run.smithi052.stdout: "roption2": { 2023-11-27T15:28:11.533 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "roption2", 2023-11-27T15:28:11.534 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.534 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.534 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.534 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "xyz", 2023-11-27T15:28:11.534 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.534 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.534 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.534 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.535 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.535 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.535 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.535 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.535 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption1": { 2023-11-27T15:28:11.535 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption1", 2023-11-27T15:28:11.535 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.535 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.535 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.536 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.536 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.536 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.536 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.536 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.536 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.536 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.536 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.537 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.537 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption2": { 2023-11-27T15:28:11.537 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption2", 2023-11-27T15:28:11.537 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.537 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.537 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.537 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.537 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.538 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.538 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.538 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.538 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.538 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.538 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.538 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.538 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption3": { 2023-11-27T15:28:11.539 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption3", 2023-11-27T15:28:11.539 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2023-11-27T15:28:11.539 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.539 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.539 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.539 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.539 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.539 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.540 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.540 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.540 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.540 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.540 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.540 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption4": { 2023-11-27T15:28:11.540 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption4", 2023-11-27T15:28:11.540 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.541 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.541 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.541 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.541 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.541 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.541 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.541 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.541 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.542 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.542 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.542 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.542 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption5": { 2023-11-27T15:28:11.542 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption5", 2023-11-27T15:28:11.542 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.542 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.542 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.543 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.543 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.543 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.543 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.543 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.543 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.543 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.543 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.544 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.544 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption6": { 2023-11-27T15:28:11.544 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption6", 2023-11-27T15:28:11.544 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.544 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.544 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.544 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.545 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.545 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.545 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.545 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.545 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.545 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.545 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.545 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.546 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption7": { 2023-11-27T15:28:11.546 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption7", 2023-11-27T15:28:11.546 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.546 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.546 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.546 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.546 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2023-11-27T15:28:11.546 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "42", 2023-11-27T15:28:11.547 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.547 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.547 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.547 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.547 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.547 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.547 INFO:teuthology.orchestra.run.smithi052.stdout: "testkey": { 2023-11-27T15:28:11.548 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testkey", 2023-11-27T15:28:11.548 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.548 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.548 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.548 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.548 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.548 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.548 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.549 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.549 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.549 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.549 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.549 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.549 INFO:teuthology.orchestra.run.smithi052.stdout: "testlkey": { 2023-11-27T15:28:11.549 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testlkey", 2023-11-27T15:28:11.549 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.550 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.550 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.550 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.550 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.550 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.550 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.550 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.551 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.551 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.551 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.551 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.551 INFO:teuthology.orchestra.run.smithi052.stdout: "testnewline": { 2023-11-27T15:28:11.551 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testnewline", 2023-11-27T15:28:11.551 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.551 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.552 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.552 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.552 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.552 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.552 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.552 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.552 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.552 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.553 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.553 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.553 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.553 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.553 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.553 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snap_schedule", 2023-11-27T15:28:11.553 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:11.554 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:11.554 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.554 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_m_granularity": { 2023-11-27T15:28:11.554 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_m_granularity", 2023-11-27T15:28:11.554 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.554 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.554 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.554 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.555 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.555 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.555 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.555 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow minute scheduled snapshots", 2023-11-27T15:28:11.555 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.555 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.555 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.555 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.557 INFO:teuthology.orchestra.run.smithi052.stdout: "dump_on_update": { 2023-11-27T15:28:11.557 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "dump_on_update", 2023-11-27T15:28:11.557 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.557 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.558 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.558 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.558 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.558 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.558 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.558 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "dump database to debug log on update", 2023-11-27T15:28:11.558 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.559 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.559 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.559 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.559 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.559 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.559 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.559 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.559 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.560 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.560 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.560 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.560 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.560 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.560 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.560 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.560 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.561 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.561 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.561 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.561 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.561 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.561 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.561 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.561 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.562 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.562 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.562 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.562 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.562 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.562 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.562 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.563 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.563 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.563 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.563 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.563 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.563 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.563 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.563 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.564 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.564 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.564 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.564 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.564 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.564 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.564 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.564 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.565 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.565 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.565 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.565 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.565 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.565 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.565 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.565 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.566 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.566 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.566 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.566 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.566 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.566 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.566 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.567 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.567 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.567 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.567 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.567 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.567 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.567 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.567 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.568 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.568 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.568 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.568 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.568 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.568 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.568 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "stats", 2023-11-27T15:28:11.569 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:11.569 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:11.569 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.569 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.569 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.569 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.569 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.569 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.570 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.570 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.570 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.570 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.570 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.570 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.570 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.570 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.571 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.571 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.571 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.571 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.571 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.571 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.571 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.571 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.572 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.572 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.572 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.572 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.572 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.572 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.572 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.572 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.573 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.573 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.573 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.573 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.573 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.573 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.573 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.573 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.574 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.574 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.574 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.574 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.574 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.574 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.574 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.575 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.575 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.575 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.575 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.575 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.575 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.575 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.575 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.576 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.576 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.576 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.576 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.576 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.576 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.576 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.576 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.577 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.577 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.577 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.577 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.577 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.577 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.577 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.578 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.578 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.578 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.578 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.578 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.578 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.578 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "status", 2023-11-27T15:28:11.579 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:11.579 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:11.579 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.579 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.579 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.579 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.579 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.579 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.580 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.580 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.580 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.580 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.580 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.580 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.580 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.580 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.581 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.581 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.581 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.581 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.581 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.581 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.581 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.581 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.582 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.582 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.582 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.582 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.582 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.582 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.582 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.582 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.582 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.583 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.583 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.583 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.583 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.583 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.583 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.583 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.584 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.584 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.584 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.584 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.584 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.584 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.584 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.584 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.585 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.585 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.585 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.585 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.585 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.585 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.585 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.585 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.586 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.586 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.586 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.586 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.586 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.586 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.586 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.587 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.587 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.587 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.587 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.587 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.587 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.587 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.587 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.588 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.588 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.588 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.588 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.588 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.588 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "telegraf", 2023-11-27T15:28:11.588 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:11.589 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:11.589 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.589 INFO:teuthology.orchestra.run.smithi052.stdout: "address": { 2023-11-27T15:28:11.589 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "address", 2023-11-27T15:28:11.589 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.589 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.589 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.589 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-27T15:28:11.590 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.590 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.590 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.590 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.590 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.590 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.590 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.590 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.591 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2023-11-27T15:28:11.591 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2023-11-27T15:28:11.591 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:11.591 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.591 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.591 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15", 2023-11-27T15:28:11.591 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.591 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.592 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.592 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.592 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.592 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.592 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.592 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.592 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.592 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.593 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.593 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.593 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.593 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.593 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.593 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.593 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.593 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.594 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.594 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.594 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.594 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.594 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.594 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.594 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.595 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.595 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.595 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.595 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.595 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.595 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.595 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.595 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.596 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.596 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.596 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.596 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.596 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.596 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.596 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.596 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.597 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.597 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.597 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.597 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.597 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.597 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.597 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.597 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.598 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.598 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.598 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.598 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.598 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.598 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.598 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.599 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.599 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.599 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.599 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.599 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.599 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.599 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.599 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.600 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.600 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.600 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.600 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.600 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.600 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.600 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.600 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.601 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.601 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.601 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.601 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.601 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.601 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.601 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.601 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.602 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.602 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "telemetry", 2023-11-27T15:28:11.602 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:11.602 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:11.602 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.602 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_basic": { 2023-11-27T15:28:11.602 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_basic", 2023-11-27T15:28:11.603 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.603 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.603 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.603 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.603 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.603 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.603 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.603 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-27T15:28:11.604 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.604 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.604 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.604 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.604 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_crash": { 2023-11-27T15:28:11.604 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_crash", 2023-11-27T15:28:11.604 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.605 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.605 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.605 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.605 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.605 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.605 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.605 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-27T15:28:11.605 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.606 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.606 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.606 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.606 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_device": { 2023-11-27T15:28:11.606 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_device", 2023-11-27T15:28:11.606 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.606 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.606 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.607 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2023-11-27T15:28:11.607 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.607 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.607 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.607 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-27T15:28:11.607 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.607 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.608 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.608 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.608 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_ident": { 2023-11-27T15:28:11.608 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_ident", 2023-11-27T15:28:11.608 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.608 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.608 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.608 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.609 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.609 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.609 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.609 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-27T15:28:11.609 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.609 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.609 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.609 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.609 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_perf": { 2023-11-27T15:28:11.610 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_perf", 2023-11-27T15:28:11.610 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.610 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.610 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.610 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.610 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.610 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.610 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.611 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-27T15:28:11.611 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.611 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.611 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.611 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.611 INFO:teuthology.orchestra.run.smithi052.stdout: "contact": { 2023-11-27T15:28:11.611 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "contact", 2023-11-27T15:28:11.611 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.612 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.612 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.612 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.612 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.612 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.612 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.612 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.612 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.613 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.613 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.613 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.613 INFO:teuthology.orchestra.run.smithi052.stdout: "description": { 2023-11-27T15:28:11.613 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "description", 2023-11-27T15:28:11.613 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.613 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.613 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.614 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.614 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.614 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.614 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.614 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.614 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.614 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.614 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.614 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.615 INFO:teuthology.orchestra.run.smithi052.stdout: "device_url": { 2023-11-27T15:28:11.615 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_url", 2023-11-27T15:28:11.615 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.615 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.615 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.615 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-27T15:28:11.615 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.616 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.616 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.616 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.616 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.616 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.616 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.616 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.616 INFO:teuthology.orchestra.run.smithi052.stdout: "enabled": { 2023-11-27T15:28:11.617 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enabled", 2023-11-27T15:28:11.617 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.617 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.617 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.617 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.617 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.617 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.618 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.618 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.618 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.618 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.618 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.618 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.618 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2023-11-27T15:28:11.618 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2023-11-27T15:28:11.619 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.619 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.619 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.619 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "24", 2023-11-27T15:28:11.619 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "8", 2023-11-27T15:28:11.619 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.619 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.620 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.620 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.620 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.620 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.620 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.620 INFO:teuthology.orchestra.run.smithi052.stdout: "last_opt_revision": { 2023-11-27T15:28:11.620 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "last_opt_revision", 2023-11-27T15:28:11.620 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.621 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.621 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.621 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1", 2023-11-27T15:28:11.621 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.621 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.621 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.621 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.621 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.622 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.622 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.622 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.622 INFO:teuthology.orchestra.run.smithi052.stdout: "leaderboard": { 2023-11-27T15:28:11.622 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "leaderboard", 2023-11-27T15:28:11.622 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.622 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.622 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.623 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.623 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.623 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.623 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.623 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.623 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.623 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.623 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.624 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.624 INFO:teuthology.orchestra.run.smithi052.stdout: "leaderboard_description": { 2023-11-27T15:28:11.624 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "leaderboard_description", 2023-11-27T15:28:11.624 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.624 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.624 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.624 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.625 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.625 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.625 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.625 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.625 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.625 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.625 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.625 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.626 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.626 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.626 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.626 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.626 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.626 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.626 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.626 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.627 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.627 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.627 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.627 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.627 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.627 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.627 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.628 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.628 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.628 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.628 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.628 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.628 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.628 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.628 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.629 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.629 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.629 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.629 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.629 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.629 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.629 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.629 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.630 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.630 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.630 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.630 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.630 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.630 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.630 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.631 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.631 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.631 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.631 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.631 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.631 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.631 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.631 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.632 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.632 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.632 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.632 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.632 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.632 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.632 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.632 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.633 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.633 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.633 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.633 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.633 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.633 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.633 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.634 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.634 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.634 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.634 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.634 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.634 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.634 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.634 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.635 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.635 INFO:teuthology.orchestra.run.smithi052.stdout: "organization": { 2023-11-27T15:28:11.635 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "organization", 2023-11-27T15:28:11.635 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.635 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.635 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.635 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.636 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.636 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.636 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.636 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.636 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.636 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.636 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.636 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.637 INFO:teuthology.orchestra.run.smithi052.stdout: "proxy": { 2023-11-27T15:28:11.637 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "proxy", 2023-11-27T15:28:11.637 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.637 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.637 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.637 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.637 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.637 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.638 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.638 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.638 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.638 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.638 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.638 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.638 INFO:teuthology.orchestra.run.smithi052.stdout: "url": { 2023-11-27T15:28:11.639 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "url", 2023-11-27T15:28:11.639 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.639 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.639 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.639 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-27T15:28:11.639 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.639 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.639 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.640 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.640 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.640 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.640 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.640 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.640 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.640 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.641 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.641 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "test_orchestrator", 2023-11-27T15:28:11.641 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:11.641 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:11.641 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.641 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.641 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.641 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.642 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.642 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.642 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.642 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.642 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.642 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.642 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.642 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.643 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.643 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.643 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.643 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.643 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.643 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.643 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.643 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.644 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.644 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.644 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.644 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.644 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.644 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.644 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.644 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.645 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.645 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.645 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.645 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.645 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.645 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.645 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.645 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.646 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.646 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.646 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.646 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.646 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.646 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.646 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.646 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.647 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.647 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.647 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.647 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.647 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.647 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.647 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.648 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.648 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.648 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.648 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.648 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.648 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.648 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.648 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.649 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.649 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.649 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.649 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.649 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.649 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.649 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.649 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.650 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.650 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.650 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.650 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.650 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.650 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.650 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.650 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "volumes", 2023-11-27T15:28:11.651 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:11.651 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:11.651 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.651 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.651 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.651 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.651 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.651 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.652 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.652 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.652 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.652 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.652 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.652 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.652 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.652 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.653 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.653 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.653 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.653 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.653 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.653 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.653 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.653 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.654 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.654 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.654 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.654 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.654 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.654 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.654 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.654 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.655 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.655 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.655 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.655 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.655 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.655 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.655 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.656 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.656 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.656 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.656 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.656 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.656 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.656 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.656 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.657 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.657 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.657 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.657 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.657 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.657 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.657 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.657 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.658 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.658 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.658 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.658 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.658 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.658 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.658 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.659 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.659 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.659 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.659 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.659 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.659 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.659 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.659 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.660 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.660 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.660 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.660 INFO:teuthology.orchestra.run.smithi052.stdout: "max_concurrent_clones": { 2023-11-27T15:28:11.660 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_concurrent_clones", 2023-11-27T15:28:11.660 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.660 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.660 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.661 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "4", 2023-11-27T15:28:11.661 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.661 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.661 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.661 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-27T15:28:11.661 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.661 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.661 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.661 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.662 INFO:teuthology.orchestra.run.smithi052.stdout: "snapshot_clone_delay": { 2023-11-27T15:28:11.662 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snapshot_clone_delay", 2023-11-27T15:28:11.662 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.662 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.662 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.662 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2023-11-27T15:28:11.662 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.663 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.663 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.663 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-27T15:28:11.663 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.663 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.663 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.663 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.663 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.663 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.664 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.664 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix", 2023-11-27T15:28:11.664 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2023-11-27T15:28:11.664 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2023-11-27T15:28:11.664 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2023-11-27T15:28:11.664 INFO:teuthology.orchestra.run.smithi052.stdout: "discovery_interval": { 2023-11-27T15:28:11.664 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "discovery_interval", 2023-11-27T15:28:11.664 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2023-11-27T15:28:11.665 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.665 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.665 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "100", 2023-11-27T15:28:11.665 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.665 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.665 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.665 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.665 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.666 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.666 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.666 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.666 INFO:teuthology.orchestra.run.smithi052.stdout: "identifier": { 2023-11-27T15:28:11.666 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "identifier", 2023-11-27T15:28:11.666 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.666 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.666 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.667 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.667 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.667 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.667 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.667 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.667 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.667 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.668 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.668 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.668 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2023-11-27T15:28:11.668 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2023-11-27T15:28:11.668 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2023-11-27T15:28:11.668 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.668 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.668 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2023-11-27T15:28:11.669 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.669 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.669 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.669 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.669 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.669 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.669 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.669 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.670 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2023-11-27T15:28:11.670 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2023-11-27T15:28:11.670 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.670 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.670 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.670 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.670 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.671 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.671 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.671 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.671 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.671 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.671 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.671 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.671 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.672 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.672 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.672 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.672 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.672 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.672 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.672 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2023-11-27T15:28:11.672 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2023-11-27T15:28:11.673 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.673 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.673 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.673 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.673 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.673 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.673 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.673 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.674 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.674 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.674 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.674 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.674 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2023-11-27T15:28:11.674 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:11.674 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.675 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.675 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.675 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2023-11-27T15:28:11.675 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.675 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.675 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2023-11-27T15:28:11.675 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2023-11-27T15:28:11.675 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2023-11-27T15:28:11.676 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2023-11-27T15:28:11.676 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2023-11-27T15:28:11.676 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2023-11-27T15:28:11.676 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2023-11-27T15:28:11.676 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.676 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.676 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.677 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.677 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.677 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.677 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2023-11-27T15:28:11.677 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2023-11-27T15:28:11.677 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2023-11-27T15:28:11.677 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.677 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2023-11-27T15:28:11.678 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2023-11-27T15:28:11.678 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.678 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.678 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.678 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.678 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.678 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.679 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.679 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.679 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_host": { 2023-11-27T15:28:11.679 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_host", 2023-11-27T15:28:11.679 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.679 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.679 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.680 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2023-11-27T15:28:11.680 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.680 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.680 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.680 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.680 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.680 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.680 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.681 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.681 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_port": { 2023-11-27T15:28:11.681 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_port", 2023-11-27T15:28:11.681 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2023-11-27T15:28:11.681 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.681 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.681 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10051", 2023-11-27T15:28:11.681 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.682 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.682 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.682 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.682 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.682 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.682 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.682 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.682 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_sender": { 2023-11-27T15:28:11.683 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_sender", 2023-11-27T15:28:11.683 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2023-11-27T15:28:11.683 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2023-11-27T15:28:11.683 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2023-11-27T15:28:11.683 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-27T15:28:11.683 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2023-11-27T15:28:11.683 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2023-11-27T15:28:11.684 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2023-11-27T15:28:11.684 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2023-11-27T15:28:11.684 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2023-11-27T15:28:11.684 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2023-11-27T15:28:11.684 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2023-11-27T15:28:11.684 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.684 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.684 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.685 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.685 INFO:teuthology.orchestra.run.smithi052.stdout: "services": { 2023-11-27T15:28:11.685 INFO:teuthology.orchestra.run.smithi052.stdout: "dashboard": "https://172.21.15.52:8443/" 2023-11-27T15:28:11.685 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.685 INFO:teuthology.orchestra.run.smithi052.stdout: "always_on_modules": { 2023-11-27T15:28:11.685 INFO:teuthology.orchestra.run.smithi052.stdout: "octopus": [ 2023-11-27T15:28:11.685 INFO:teuthology.orchestra.run.smithi052.stdout: "balancer", 2023-11-27T15:28:11.685 INFO:teuthology.orchestra.run.smithi052.stdout: "crash", 2023-11-27T15:28:11.686 INFO:teuthology.orchestra.run.smithi052.stdout: "devicehealth", 2023-11-27T15:28:11.686 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator", 2023-11-27T15:28:11.686 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_autoscaler", 2023-11-27T15:28:11.686 INFO:teuthology.orchestra.run.smithi052.stdout: "progress", 2023-11-27T15:28:11.686 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_support", 2023-11-27T15:28:11.686 INFO:teuthology.orchestra.run.smithi052.stdout: "status", 2023-11-27T15:28:11.686 INFO:teuthology.orchestra.run.smithi052.stdout: "telemetry", 2023-11-27T15:28:11.687 INFO:teuthology.orchestra.run.smithi052.stdout: "volumes" 2023-11-27T15:28:11.687 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.687 INFO:teuthology.orchestra.run.smithi052.stdout: "pacific": [ 2023-11-27T15:28:11.687 INFO:teuthology.orchestra.run.smithi052.stdout: "balancer", 2023-11-27T15:28:11.687 INFO:teuthology.orchestra.run.smithi052.stdout: "crash", 2023-11-27T15:28:11.687 INFO:teuthology.orchestra.run.smithi052.stdout: "devicehealth", 2023-11-27T15:28:11.687 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator", 2023-11-27T15:28:11.687 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_autoscaler", 2023-11-27T15:28:11.688 INFO:teuthology.orchestra.run.smithi052.stdout: "progress", 2023-11-27T15:28:11.688 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_support", 2023-11-27T15:28:11.688 INFO:teuthology.orchestra.run.smithi052.stdout: "status", 2023-11-27T15:28:11.688 INFO:teuthology.orchestra.run.smithi052.stdout: "telemetry", 2023-11-27T15:28:11.688 INFO:teuthology.orchestra.run.smithi052.stdout: "volumes" 2023-11-27T15:28:11.688 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.688 INFO:teuthology.orchestra.run.smithi052.stdout: "quincy": [ 2023-11-27T15:28:11.689 INFO:teuthology.orchestra.run.smithi052.stdout: "balancer", 2023-11-27T15:28:11.689 INFO:teuthology.orchestra.run.smithi052.stdout: "crash", 2023-11-27T15:28:11.689 INFO:teuthology.orchestra.run.smithi052.stdout: "devicehealth", 2023-11-27T15:28:11.689 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator", 2023-11-27T15:28:11.689 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_autoscaler", 2023-11-27T15:28:11.689 INFO:teuthology.orchestra.run.smithi052.stdout: "progress", 2023-11-27T15:28:11.689 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_support", 2023-11-27T15:28:11.689 INFO:teuthology.orchestra.run.smithi052.stdout: "status", 2023-11-27T15:28:11.690 INFO:teuthology.orchestra.run.smithi052.stdout: "telemetry", 2023-11-27T15:28:11.690 INFO:teuthology.orchestra.run.smithi052.stdout: "volumes" 2023-11-27T15:28:11.690 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2023-11-27T15:28:11.690 INFO:teuthology.orchestra.run.smithi052.stdout: "reef": [ 2023-11-27T15:28:11.690 INFO:teuthology.orchestra.run.smithi052.stdout: "balancer", 2023-11-27T15:28:11.690 INFO:teuthology.orchestra.run.smithi052.stdout: "crash", 2023-11-27T15:28:11.690 INFO:teuthology.orchestra.run.smithi052.stdout: "devicehealth", 2023-11-27T15:28:11.691 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator", 2023-11-27T15:28:11.691 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_autoscaler", 2023-11-27T15:28:11.691 INFO:teuthology.orchestra.run.smithi052.stdout: "progress", 2023-11-27T15:28:11.691 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_support", 2023-11-27T15:28:11.691 INFO:teuthology.orchestra.run.smithi052.stdout: "status", 2023-11-27T15:28:11.691 INFO:teuthology.orchestra.run.smithi052.stdout: "telemetry", 2023-11-27T15:28:11.691 INFO:teuthology.orchestra.run.smithi052.stdout: "volumes" 2023-11-27T15:28:11.691 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2023-11-27T15:28:11.692 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.692 INFO:teuthology.orchestra.run.smithi052.stdout: "last_failure_osd_epoch": 86, 2023-11-27T15:28:11.692 INFO:teuthology.orchestra.run.smithi052.stdout: "active_clients": [ 2023-11-27T15:28:11.692 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.692 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "devicehealth", 2023-11-27T15:28:11.692 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2023-11-27T15:28:11.692 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.692 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2023-11-27T15:28:11.693 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:0", 2023-11-27T15:28:11.693 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 4187871282 2023-11-27T15:28:11.693 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.693 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2023-11-27T15:28:11.693 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.693 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.693 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "libcephsqlite", 2023-11-27T15:28:11.693 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2023-11-27T15:28:11.694 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.694 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2023-11-27T15:28:11.694 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:0", 2023-11-27T15:28:11.694 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 1287497792 2023-11-27T15:28:11.694 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.694 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2023-11-27T15:28:11.694 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.695 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.695 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_support", 2023-11-27T15:28:11.695 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2023-11-27T15:28:11.695 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.695 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2023-11-27T15:28:11.695 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:0", 2023-11-27T15:28:11.695 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 3369530954 2023-11-27T15:28:11.695 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.696 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2023-11-27T15:28:11.696 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.696 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.696 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snap_schedule", 2023-11-27T15:28:11.696 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2023-11-27T15:28:11.696 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.696 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2023-11-27T15:28:11.696 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:0", 2023-11-27T15:28:11.697 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 4044304202 2023-11-27T15:28:11.697 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.697 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2023-11-27T15:28:11.697 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.697 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.697 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snap_schedule", 2023-11-27T15:28:11.697 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2023-11-27T15:28:11.697 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.698 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2023-11-27T15:28:11.698 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:0", 2023-11-27T15:28:11.698 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 2359439356 2023-11-27T15:28:11.698 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.698 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2023-11-27T15:28:11.698 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:11.698 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.699 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "volumes", 2023-11-27T15:28:11.699 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2023-11-27T15:28:11.699 INFO:teuthology.orchestra.run.smithi052.stdout: { 2023-11-27T15:28:11.699 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2023-11-27T15:28:11.699 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:0", 2023-11-27T15:28:11.699 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 104816431 2023-11-27T15:28:11.699 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.699 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2023-11-27T15:28:11.700 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:11.700 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2023-11-27T15:28:11.700 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:28:11.701 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:09 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/772781610' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:28:11.702 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:09 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2483185157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:28:11.702 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:10 smithi150 ceph-mon[100559]: pgmap v318: 97 pgs: 97 active+clean; 712 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-11-27T15:28:11.702 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:10 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/4042619355' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-11-27T15:28:11.702 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:09 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/772781610' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:28:11.702 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:09 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2483185157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:28:11.702 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:10 smithi102 ceph-mon[100296]: pgmap v318: 97 pgs: 97 active+clean; 712 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-11-27T15:28:11.703 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:10 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/4042619355' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-11-27T15:28:11.703 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:11 smithi102 ceph-mon[100296]: osdmap e95: 12 total, 12 up, 12 in 2023-11-27T15:28:11.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.307+0000 7f62dbc1f700 1 -- 172.21.15.52:0/4042619355 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f62c00a1340 msgr2=0x7f62c00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:11.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.307+0000 7f62dbc1f700 1 --2- 172.21.15.52:0/4042619355 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f62c00a1340 0x7f62c00a37f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f62d0000c00 tx=0x7f62d0005fb0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.703 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.307+0000 7f62dbc1f700 1 -- 172.21.15.52:0/4042619355 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62d4104390 msgr2=0x7f62d41a4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:11.704 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.307+0000 7f62dbc1f700 1 --2- 172.21.15.52:0/4042619355 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62d4104390 0x7f62d41a4530 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00ab50 tx=0x7f62cc00af10 comp rx=0 tx=0).stop 2023-11-27T15:28:11.704 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.307+0000 7f62dbc1f700 1 -- 172.21.15.52:0/4042619355 shutdown_connections 2023-11-27T15:28:11.704 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.307+0000 7f62dbc1f700 1 --2- 172.21.15.52:0/4042619355 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f62c00a1340 0x7f62c00a37f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.704 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.307+0000 7f62dbc1f700 1 --2- 172.21.15.52:0/4042619355 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62d4104390 0x7f62d41a4530 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.704 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.307+0000 7f62dbc1f700 1 --2- 172.21.15.52:0/4042619355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62d40ff3a0 0x7f62d4100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.704 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.307+0000 7f62dbc1f700 1 --2- 172.21.15.52:0/4042619355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62d40ffd40 0x7f62d4101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.704 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.307+0000 7f62dbc1f700 1 -- 172.21.15.52:0/4042619355 >> 172.21.15.52:0/4042619355 conn(0x7f62d4076230 msgr2=0x7f62d4068ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:11.705 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.308+0000 7f62dbc1f700 1 -- 172.21.15.52:0/4042619355 shutdown_connections 2023-11-27T15:28:11.705 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:09.308+0000 7f62dbc1f700 1 -- 172.21.15.52:0/4042619355 wait complete. 2023-11-27T15:28:11.712 DEBUG:tasks.check_counter:Getting stats from a 2023-11-27T15:28:11.712 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph daemon mds.a perf dump 2023-11-27T15:28:11.713 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:09 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/772781610' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:28:11.713 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:09 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2483185157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:28:11.714 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:10 smithi052 ceph-mon[99056]: pgmap v318: 97 pgs: 97 active+clean; 712 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-11-27T15:28:11.714 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:10 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/4042619355' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-11-27T15:28:11.714 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:11 smithi052 ceph-mon[99056]: osdmap e95: 12 total, 12 up, 12 in 2023-11-27T15:28:11.777 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:11 smithi150 ceph-mon[100559]: osdmap e95: 12 total, 12 up, 12 in 2023-11-27T15:28:12.387 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:28:12.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:12 smithi102 ceph-mon[100296]: pgmap v320: 97 pgs: 97 active+clean; 712 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:28:12.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:12 smithi052 ceph-mon[99056]: pgmap v320: 97 pgs: 97 active+clean; 712 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:28:12.777 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:12 smithi150 ceph-mon[100559]: pgmap v320: 97 pgs: 97 active+clean; 712 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:28:14.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:14 smithi052 ceph-mon[99056]: pgmap v321: 97 pgs: 97 active+clean; 712 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:28:14.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:14 smithi052 ceph-mon[99056]: osdmap e96: 12 total, 12 up, 12 in 2023-11-27T15:28:14.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:14 smithi102 ceph-mon[100296]: pgmap v321: 97 pgs: 97 active+clean; 712 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:28:14.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:14 smithi102 ceph-mon[100296]: osdmap e96: 12 total, 12 up, 12 in 2023-11-27T15:28:14.777 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:14 smithi150 ceph-mon[100559]: pgmap v321: 97 pgs: 97 active+clean; 712 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-27T15:28:14.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:14 smithi150 ceph-mon[100559]: osdmap e96: 12 total, 12 up, 12 in 2023-11-27T15:28:15.102 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:28:15.103 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:28:15.103 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 0, 2023-11-27T15:28:15.103 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 2, 2023-11-27T15:28:15.103 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 0, 2023-11-27T15:28:15.103 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 0, 2023-11-27T15:28:15.104 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 4, 2023-11-27T15:28:15.104 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 0, 2023-11-27T15:28:15.104 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.001549038, 2023-11-27T15:28:15.104 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.000000000, 2023-11-27T15:28:15.105 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.000313513, 2023-11-27T15:28:15.105 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:28:15.105 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:15.105 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2023-11-27T15:28:15.105 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2023-11-27T15:28:15.106 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2023-11-27T15:28:15.106 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:15.106 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:15.106 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2023-11-27T15:28:15.106 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2023-11-27T15:28:15.107 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2023-11-27T15:28:15.107 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:15.107 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-27T15:28:15.107 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 0 2023-11-27T15:28:15.107 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:15.108 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:28:15.108 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 209, 2023-11-27T15:28:15.108 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 198, 2023-11-27T15:28:15.108 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 1553776, 2023-11-27T15:28:15.109 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 112640, 2023-11-27T15:28:15.109 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 2, 2023-11-27T15:28:15.109 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 1, 2023-11-27T15:28:15.109 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.040840751, 2023-11-27T15:28:15.109 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.012615717, 2023-11-27T15:28:15.110 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.032990687, 2023-11-27T15:28:15.110 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.004775776, 2023-11-27T15:28:15.110 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:15.110 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 197, 2023-11-27T15:28:15.110 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.009029928, 2023-11-27T15:28:15.111 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000045837 2023-11-27T15:28:15.111 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:15.111 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:15.111 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2023-11-27T15:28:15.111 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2023-11-27T15:28:15.112 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2023-11-27T15:28:15.112 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:15.112 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 1553776, 2023-11-27T15:28:15.112 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 112640 2023-11-27T15:28:15.113 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:15.113 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:28:15.113 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 6, 2023-11-27T15:28:15.113 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 152, 2023-11-27T15:28:15.113 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 3312, 2023-11-27T15:28:15.114 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 178576, 2023-11-27T15:28:15.114 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 5, 2023-11-27T15:28:15.114 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 3, 2023-11-27T15:28:15.114 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.012733009, 2023-11-27T15:28:15.114 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.008205259, 2023-11-27T15:28:15.115 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.002303047, 2023-11-27T15:28:15.115 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:28:15.115 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:15.115 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 150, 2023-11-27T15:28:15.115 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.009778349, 2023-11-27T15:28:15.116 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000065188 2023-11-27T15:28:15.116 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:15.116 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:15.116 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2023-11-27T15:28:15.116 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2023-11-27T15:28:15.117 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2023-11-27T15:28:15.117 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:15.117 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-11-27T15:28:15.117 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 178576 2023-11-27T15:28:15.117 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:15.118 INFO:teuthology.orchestra.run.smithi052.stdout: "cct": { 2023-11-27T15:28:15.118 INFO:teuthology.orchestra.run.smithi052.stdout: "total_workers": 0, 2023-11-27T15:28:15.118 INFO:teuthology.orchestra.run.smithi052.stdout: "unhealthy_workers": 0 2023-11-27T15:28:15.118 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:15.119 INFO:teuthology.orchestra.run.smithi052.stdout: "mempool": { 2023-11-27T15:28:15.119 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:28:15.119 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_items": 0, 2023-11-27T15:28:15.119 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:28:15.119 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:28:15.120 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:28:15.120 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:28:15.120 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:28:15.120 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:28:15.120 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:28:15.121 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:28:15.121 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:28:15.121 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:28:15.121 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:28:15.121 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:28:15.122 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:28:15.122 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_items": 0, 2023-11-27T15:28:15.122 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:28:15.122 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_items": 0, 2023-11-27T15:28:15.122 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:28:15.123 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:28:15.123 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:28:15.123 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:28:15.123 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:28:15.123 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:28:15.124 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:28:15.124 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_items": 0, 2023-11-27T15:28:15.124 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:28:15.124 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:28:15.125 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:28:15.125 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_items": 0, 2023-11-27T15:28:15.125 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_bytes": 0, 2023-11-27T15:28:15.125 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_items": 0, 2023-11-27T15:28:15.125 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:28:15.126 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:28:15.126 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:28:15.126 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:28:15.126 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_bytes": 148994, 2023-11-27T15:28:15.126 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_items": 31, 2023-11-27T15:28:15.127 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_bytes": 0, 2023-11-27T15:28:15.127 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_items": 0, 2023-11-27T15:28:15.127 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_bytes": 0, 2023-11-27T15:28:15.127 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_items": 0, 2023-11-27T15:28:15.127 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:28:15.128 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_items": 0, 2023-11-27T15:28:15.128 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:28:15.128 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_items": 0, 2023-11-27T15:28:15.128 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_bytes": 0, 2023-11-27T15:28:15.128 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_items": 0, 2023-11-27T15:28:15.129 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:28:15.129 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:28:15.129 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_bytes": 0, 2023-11-27T15:28:15.129 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_items": 0, 2023-11-27T15:28:15.129 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_bytes": 928, 2023-11-27T15:28:15.130 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_items": 2, 2023-11-27T15:28:15.130 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_bytes": 0, 2023-11-27T15:28:15.130 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_items": 0, 2023-11-27T15:28:15.130 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_bytes": 0, 2023-11-27T15:28:15.130 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_items": 0 2023-11-27T15:28:15.131 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:15.131 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:28:15.131 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2023-11-27T15:28:15.131 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2023-11-27T15:28:15.131 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2023-11-27T15:28:15.132 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 211, 2023-11-27T15:28:15.132 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 1524289, 2023-11-27T15:28:15.132 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:15.132 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 211, 2023-11-27T15:28:15.132 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2023-11-27T15:28:15.133 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2023-11-27T15:28:15.133 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 211, 2023-11-27T15:28:15.133 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 1524289, 2023-11-27T15:28:15.133 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2023-11-27T15:28:15.134 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2023-11-27T15:28:15.134 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2023-11-27T15:28:15.134 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2023-11-27T15:28:15.134 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:15.134 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:15.135 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:28:15.546 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2023-11-27T15:28:15.546 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2023-11-27T15:28:15.546 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2023-11-27T15:28:15.546 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2023-11-27T15:28:15.547 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2023-11-27T15:28:15.547 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2023-11-27T15:28:15.547 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2023-11-27T15:28:15.547 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2023-11-27T15:28:15.548 DEBUG:tasks.check_counter:Getting stats from d 2023-11-27T15:28:15.548 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph daemon mds.d perf dump 2023-11-27T15:28:16.198 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:28:16.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:16 smithi102 ceph-mon[100296]: pgmap v323: 97 pgs: 97 active+clean; 712 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2023-11-27T15:28:16.607 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:16 smithi052 ceph-mon[99056]: pgmap v323: 97 pgs: 97 active+clean; 712 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2023-11-27T15:28:16.777 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:16 smithi150 ceph-mon[100559]: pgmap v323: 97 pgs: 97 active+clean; 712 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2023-11-27T15:28:17.528 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:28:17.528 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:28:17.529 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 0, 2023-11-27T15:28:17.529 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 2, 2023-11-27T15:28:17.529 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 0, 2023-11-27T15:28:17.529 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 0, 2023-11-27T15:28:17.529 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 4, 2023-11-27T15:28:17.529 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 0, 2023-11-27T15:28:17.530 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.001979490, 2023-11-27T15:28:17.530 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.000000000, 2023-11-27T15:28:17.530 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.000325562, 2023-11-27T15:28:17.530 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:28:17.530 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:17.531 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2023-11-27T15:28:17.531 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2023-11-27T15:28:17.531 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2023-11-27T15:28:17.531 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:17.531 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:17.532 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2023-11-27T15:28:17.532 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2023-11-27T15:28:17.532 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2023-11-27T15:28:17.532 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:17.532 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-27T15:28:17.533 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 0 2023-11-27T15:28:17.533 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:17.533 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:28:17.533 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 6, 2023-11-27T15:28:17.533 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 157, 2023-11-27T15:28:17.533 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 3312, 2023-11-27T15:28:17.534 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 179456, 2023-11-27T15:28:17.534 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 5, 2023-11-27T15:28:17.534 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 3, 2023-11-27T15:28:17.534 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.012898128, 2023-11-27T15:28:17.534 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.008349206, 2023-11-27T15:28:17.535 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.002357946, 2023-11-27T15:28:17.535 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:28:17.535 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:17.535 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 155, 2023-11-27T15:28:17.535 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.013039636, 2023-11-27T15:28:17.535 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000084126 2023-11-27T15:28:17.536 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:17.536 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:17.536 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2023-11-27T15:28:17.536 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2023-11-27T15:28:17.536 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2023-11-27T15:28:17.537 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:17.537 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-11-27T15:28:17.537 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 179456 2023-11-27T15:28:17.537 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:17.537 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:28:17.537 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 213, 2023-11-27T15:28:17.538 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 202, 2023-11-27T15:28:17.538 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 1554864, 2023-11-27T15:28:17.538 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 115008, 2023-11-27T15:28:17.538 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 2, 2023-11-27T15:28:17.538 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 1, 2023-11-27T15:28:17.539 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.041411962, 2023-11-27T15:28:17.539 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.012668718, 2023-11-27T15:28:17.539 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.033974052, 2023-11-27T15:28:17.539 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.004806590, 2023-11-27T15:28:17.539 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:17.540 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 201, 2023-11-27T15:28:17.540 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.011163500, 2023-11-27T15:28:17.540 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000055539 2023-11-27T15:28:17.540 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:17.540 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:17.541 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2023-11-27T15:28:17.541 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2023-11-27T15:28:17.541 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2023-11-27T15:28:17.541 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:17.541 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 1554864, 2023-11-27T15:28:17.541 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 115008 2023-11-27T15:28:17.542 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:17.542 INFO:teuthology.orchestra.run.smithi052.stdout: "cct": { 2023-11-27T15:28:17.542 INFO:teuthology.orchestra.run.smithi052.stdout: "total_workers": 0, 2023-11-27T15:28:17.542 INFO:teuthology.orchestra.run.smithi052.stdout: "unhealthy_workers": 0 2023-11-27T15:28:17.542 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:17.543 INFO:teuthology.orchestra.run.smithi052.stdout: "mempool": { 2023-11-27T15:28:17.543 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:28:17.543 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_items": 0, 2023-11-27T15:28:17.543 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:28:17.543 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:28:17.543 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:28:17.544 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:28:17.544 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:28:17.544 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:28:17.544 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:28:17.544 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:28:17.544 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:28:17.545 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:28:17.545 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:28:17.545 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:28:17.545 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:28:17.545 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_items": 0, 2023-11-27T15:28:17.546 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:28:17.546 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_items": 0, 2023-11-27T15:28:17.546 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:28:17.546 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:28:17.546 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:28:17.547 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:28:17.547 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:28:17.547 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:28:17.547 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:28:17.547 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_items": 0, 2023-11-27T15:28:17.547 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:28:17.548 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:28:17.548 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:28:17.548 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_items": 0, 2023-11-27T15:28:17.548 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_bytes": 0, 2023-11-27T15:28:17.548 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_items": 0, 2023-11-27T15:28:17.548 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:28:17.549 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:28:17.549 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:28:17.549 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:28:17.549 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_bytes": 148977, 2023-11-27T15:28:17.549 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_items": 30, 2023-11-27T15:28:17.549 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_bytes": 0, 2023-11-27T15:28:17.550 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_items": 0, 2023-11-27T15:28:17.550 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_bytes": 0, 2023-11-27T15:28:17.550 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_items": 0, 2023-11-27T15:28:17.550 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:28:17.550 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_items": 0, 2023-11-27T15:28:17.551 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:28:17.551 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_items": 0, 2023-11-27T15:28:17.551 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_bytes": 0, 2023-11-27T15:28:17.551 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_items": 0, 2023-11-27T15:28:17.551 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:28:17.551 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:28:17.552 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_bytes": 0, 2023-11-27T15:28:17.552 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_items": 0, 2023-11-27T15:28:17.552 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_bytes": 928, 2023-11-27T15:28:17.552 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_items": 2, 2023-11-27T15:28:17.552 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_bytes": 0, 2023-11-27T15:28:17.552 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_items": 0, 2023-11-27T15:28:17.553 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_bytes": 0, 2023-11-27T15:28:17.553 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_items": 0 2023-11-27T15:28:17.553 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:17.553 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:28:17.553 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2023-11-27T15:28:17.553 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2023-11-27T15:28:17.554 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2023-11-27T15:28:17.554 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 215, 2023-11-27T15:28:17.554 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 1524809, 2023-11-27T15:28:17.554 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:17.554 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 215, 2023-11-27T15:28:17.554 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2023-11-27T15:28:17.555 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2023-11-27T15:28:17.555 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 215, 2023-11-27T15:28:17.555 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 1524809, 2023-11-27T15:28:17.555 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2023-11-27T15:28:17.555 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2023-11-27T15:28:17.555 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2023-11-27T15:28:17.555 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2023-11-27T15:28:17.556 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:17.556 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:17.556 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:28:17.967 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-11-27T15:28:17.967 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-11-27T15:28:17.967 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2023-11-27T15:28:17.967 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2023-11-27T15:28:17.967 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2023-11-27T15:28:17.967 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2023-11-27T15:28:17.967 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2023-11-27T15:28:17.968 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2023-11-27T15:28:17.968 DEBUG:tasks.check_counter:Getting stats from g 2023-11-27T15:28:17.968 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph daemon mds.g perf dump 2023-11-27T15:28:18.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:18 smithi102 ceph-mon[100296]: pgmap v324: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-27T15:28:18.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:18 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:28:18.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:18 smithi052 ceph-mon[99056]: pgmap v324: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-27T15:28:18.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:18 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:28:18.627 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:28:18.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:18 smithi150 ceph-mon[100559]: pgmap v324: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-27T15:28:18.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:18 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:28:20.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:19 smithi052 ceph-mon[99056]: pgmap v325: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 229 B/s wr, 7 op/s 2023-11-27T15:28:20.277 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:19 smithi150 ceph-mon[100559]: pgmap v325: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 229 B/s wr, 7 op/s 2023-11-27T15:28:20.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:19 smithi102 ceph-mon[100296]: pgmap v325: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 229 B/s wr, 7 op/s 2023-11-27T15:28:20.390 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:28:20.390 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:28:20.390 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 0, 2023-11-27T15:28:20.391 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 2, 2023-11-27T15:28:20.391 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 0, 2023-11-27T15:28:20.391 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 0, 2023-11-27T15:28:20.391 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 4, 2023-11-27T15:28:20.391 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 0, 2023-11-27T15:28:20.391 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.001711052, 2023-11-27T15:28:20.391 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.000000000, 2023-11-27T15:28:20.391 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.000484708, 2023-11-27T15:28:20.392 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:28:20.392 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:20.392 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2023-11-27T15:28:20.392 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2023-11-27T15:28:20.392 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2023-11-27T15:28:20.392 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:20.392 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:20.392 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2023-11-27T15:28:20.393 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2023-11-27T15:28:20.393 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2023-11-27T15:28:20.393 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:20.393 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-27T15:28:20.393 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 0 2023-11-27T15:28:20.393 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:20.393 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:28:20.393 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 215, 2023-11-27T15:28:20.394 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 202, 2023-11-27T15:28:20.394 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 1557312, 2023-11-27T15:28:20.394 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 114272, 2023-11-27T15:28:20.394 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 2, 2023-11-27T15:28:20.394 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 1, 2023-11-27T15:28:20.394 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.041826253, 2023-11-27T15:28:20.394 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.012919801, 2023-11-27T15:28:20.395 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.033703218, 2023-11-27T15:28:20.395 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.004946888, 2023-11-27T15:28:20.395 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:20.395 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 202, 2023-11-27T15:28:20.395 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.010762955, 2023-11-27T15:28:20.395 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000053281 2023-11-27T15:28:20.395 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:20.395 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:20.396 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2023-11-27T15:28:20.396 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2023-11-27T15:28:20.396 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2023-11-27T15:28:20.396 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:20.396 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 1557312, 2023-11-27T15:28:20.396 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 114272 2023-11-27T15:28:20.396 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:20.397 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:28:20.397 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 2, 2023-11-27T15:28:20.397 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 152, 2023-11-27T15:28:20.397 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 320, 2023-11-27T15:28:20.397 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 178480, 2023-11-27T15:28:20.397 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 5, 2023-11-27T15:28:20.397 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 3, 2023-11-27T15:28:20.397 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.012382529, 2023-11-27T15:28:20.398 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.007986912, 2023-11-27T15:28:20.398 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.002129659, 2023-11-27T15:28:20.398 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:28:20.398 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:20.398 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 149, 2023-11-27T15:28:20.398 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.014566705, 2023-11-27T15:28:20.398 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000097763 2023-11-27T15:28:20.399 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:20.399 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:20.399 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2023-11-27T15:28:20.399 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2023-11-27T15:28:20.399 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2023-11-27T15:28:20.399 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:20.399 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 320, 2023-11-27T15:28:20.399 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 178480 2023-11-27T15:28:20.400 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:20.400 INFO:teuthology.orchestra.run.smithi052.stdout: "cct": { 2023-11-27T15:28:20.400 INFO:teuthology.orchestra.run.smithi052.stdout: "total_workers": 0, 2023-11-27T15:28:20.400 INFO:teuthology.orchestra.run.smithi052.stdout: "unhealthy_workers": 0 2023-11-27T15:28:20.400 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:20.400 INFO:teuthology.orchestra.run.smithi052.stdout: "mempool": { 2023-11-27T15:28:20.400 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:28:20.400 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_items": 0, 2023-11-27T15:28:20.401 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:28:20.401 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:28:20.401 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:28:20.401 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:28:20.401 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:28:20.401 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:28:20.401 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:28:20.401 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:28:20.402 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:28:20.402 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:28:20.402 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:28:20.402 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:28:20.402 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:28:20.402 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_items": 0, 2023-11-27T15:28:20.402 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:28:20.403 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_items": 0, 2023-11-27T15:28:20.403 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:28:20.403 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:28:20.403 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:28:20.403 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:28:20.403 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:28:20.403 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:28:20.403 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:28:20.404 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_items": 0, 2023-11-27T15:28:20.404 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:28:20.404 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:28:20.404 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:28:20.404 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_items": 0, 2023-11-27T15:28:20.404 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_bytes": 0, 2023-11-27T15:28:20.404 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_items": 0, 2023-11-27T15:28:20.404 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:28:20.405 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:28:20.405 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:28:20.405 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:28:20.405 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_bytes": 148977, 2023-11-27T15:28:20.406 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_items": 30, 2023-11-27T15:28:20.406 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_bytes": 0, 2023-11-27T15:28:20.406 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_items": 0, 2023-11-27T15:28:20.406 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_bytes": 0, 2023-11-27T15:28:20.406 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_items": 0, 2023-11-27T15:28:20.406 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:28:20.406 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_items": 0, 2023-11-27T15:28:20.407 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:28:20.407 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_items": 0, 2023-11-27T15:28:20.407 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_bytes": 0, 2023-11-27T15:28:20.407 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_items": 0, 2023-11-27T15:28:20.407 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:28:20.407 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:28:20.407 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_bytes": 0, 2023-11-27T15:28:20.407 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_items": 0, 2023-11-27T15:28:20.408 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_bytes": 928, 2023-11-27T15:28:20.408 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_items": 2, 2023-11-27T15:28:20.408 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_bytes": 0, 2023-11-27T15:28:20.408 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_items": 0, 2023-11-27T15:28:20.408 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_bytes": 0, 2023-11-27T15:28:20.408 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_items": 0 2023-11-27T15:28:20.408 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:20.408 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:28:20.409 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2023-11-27T15:28:20.409 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2023-11-27T15:28:20.409 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2023-11-27T15:28:20.409 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 213, 2023-11-27T15:28:20.409 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 1524549, 2023-11-27T15:28:20.409 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:20.409 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 213, 2023-11-27T15:28:20.410 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2023-11-27T15:28:20.410 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2023-11-27T15:28:20.410 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 213, 2023-11-27T15:28:20.410 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 1524549, 2023-11-27T15:28:20.410 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2023-11-27T15:28:20.410 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2023-11-27T15:28:20.410 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2023-11-27T15:28:20.411 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2023-11-27T15:28:20.411 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:20.411 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:20.411 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:28:20.835 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2023-11-27T15:28:20.836 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2023-11-27T15:28:20.836 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2023-11-27T15:28:20.836 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2023-11-27T15:28:20.836 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2023-11-27T15:28:20.836 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2023-11-27T15:28:20.836 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2023-11-27T15:28:20.836 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2023-11-27T15:28:20.837 DEBUG:tasks.check_counter:Getting stats from j 2023-11-27T15:28:20.837 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph daemon mds.j perf dump 2023-11-27T15:28:21.513 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:28:21.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:21 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:28:21.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:21 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:28:21.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:21 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:28:21.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:21 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:28:21.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:21 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:21.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:21 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:21.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:21 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:21.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:21 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:21.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:21 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:21.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:21 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:21.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:21 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:21.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:21 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:21.530 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:21 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:28:21.530 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:21 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:28:21.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:21 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:28:21.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:21 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:28:21.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:21 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:28:21.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:21 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:28:21.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:21 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:21.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:21 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:21.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:21 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:21.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:21 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:21.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:21 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:21.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:21 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:21.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:21 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:21.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:21 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:21.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:21 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:28:21.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:21 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:28:21.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:21 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:28:21.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:21 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:28:21.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:21 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:28:21.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:21 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:28:21.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:21 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:21.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:21 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:21.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:21 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:21.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:21 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:21.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:21 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:21.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:21 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:21.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:21 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:21.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:21 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:21.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:21 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:28:21.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:21 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:28:22.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:22 smithi150 ceph-mon[100559]: Adjusting osd_memory_target on smithi102 to 196.2M 2023-11-27T15:28:22.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:22 smithi150 ceph-mon[100559]: Unable to set osd_memory_target on smithi102 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-11-27T15:28:22.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:22 smithi150 ceph-mon[100559]: Adjusting osd_memory_target on smithi150 to 196.2M 2023-11-27T15:28:22.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:22 smithi150 ceph-mon[100559]: Unable to set osd_memory_target on smithi150 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-11-27T15:28:22.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:22 smithi150 ceph-mon[100559]: pgmap v326: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-11-27T15:28:22.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:22 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:22.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:22 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:22.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:22 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:22.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:22 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:22.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:22 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:28:22.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:22 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:28:22.529 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:22 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:28:22.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:22 smithi102 ceph-mon[100296]: Adjusting osd_memory_target on smithi102 to 196.2M 2023-11-27T15:28:22.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:22 smithi102 ceph-mon[100296]: Unable to set osd_memory_target on smithi102 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-11-27T15:28:22.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:22 smithi102 ceph-mon[100296]: Adjusting osd_memory_target on smithi150 to 196.2M 2023-11-27T15:28:22.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:22 smithi102 ceph-mon[100296]: Unable to set osd_memory_target on smithi150 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-11-27T15:28:22.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:22 smithi102 ceph-mon[100296]: pgmap v326: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-11-27T15:28:22.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:22 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:22.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:22 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:22.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:22 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:22.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:22 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:22.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:22 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:28:22.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:22 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:28:22.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:22 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:28:22.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:22 smithi052 ceph-mon[99056]: Adjusting osd_memory_target on smithi102 to 196.2M 2023-11-27T15:28:22.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:22 smithi052 ceph-mon[99056]: Unable to set osd_memory_target on smithi102 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-11-27T15:28:22.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:22 smithi052 ceph-mon[99056]: Adjusting osd_memory_target on smithi150 to 196.2M 2023-11-27T15:28:22.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:22 smithi052 ceph-mon[99056]: Unable to set osd_memory_target on smithi150 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-11-27T15:28:22.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:22 smithi052 ceph-mon[99056]: pgmap v326: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-11-27T15:28:22.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:22 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:22.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:22 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:22.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:22 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:22.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:22 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:28:22.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:22 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:28:22.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:22 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:28:22.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:22 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' 2023-11-27T15:28:22.971 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2023-11-27T15:28:22.971 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:28:22.972 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 0, 2023-11-27T15:28:22.972 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 2, 2023-11-27T15:28:22.972 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 0, 2023-11-27T15:28:22.972 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 0, 2023-11-27T15:28:22.973 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 4, 2023-11-27T15:28:22.973 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 0, 2023-11-27T15:28:22.973 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.001583712, 2023-11-27T15:28:22.973 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.000000000, 2023-11-27T15:28:22.973 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.000447983, 2023-11-27T15:28:22.973 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:28:22.974 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:22.974 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2023-11-27T15:28:22.974 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2023-11-27T15:28:22.974 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2023-11-27T15:28:22.974 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:22.975 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:22.975 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2023-11-27T15:28:22.975 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2023-11-27T15:28:22.975 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2023-11-27T15:28:22.975 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:22.975 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-27T15:28:22.976 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 0 2023-11-27T15:28:22.976 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:22.976 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:28:22.976 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 2, 2023-11-27T15:28:22.976 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 156, 2023-11-27T15:28:22.976 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 320, 2023-11-27T15:28:22.977 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 179184, 2023-11-27T15:28:22.977 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 5, 2023-11-27T15:28:22.977 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 3, 2023-11-27T15:28:22.977 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.012405028, 2023-11-27T15:28:22.977 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.007964046, 2023-11-27T15:28:22.978 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.002097624, 2023-11-27T15:28:22.978 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:28:22.978 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:22.978 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 153, 2023-11-27T15:28:22.978 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.024200947, 2023-11-27T15:28:22.978 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000158176 2023-11-27T15:28:22.979 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:22.979 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:22.979 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2023-11-27T15:28:22.979 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2023-11-27T15:28:22.979 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2023-11-27T15:28:22.980 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:22.980 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 320, 2023-11-27T15:28:22.980 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 179184 2023-11-27T15:28:22.980 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:22.980 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:28:22.981 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 219, 2023-11-27T15:28:22.981 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 206, 2023-11-27T15:28:22.981 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 1558400, 2023-11-27T15:28:22.981 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 116640, 2023-11-27T15:28:22.981 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 2, 2023-11-27T15:28:22.981 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 1, 2023-11-27T15:28:22.982 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.043827901, 2023-11-27T15:28:22.982 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.013660326, 2023-11-27T15:28:22.982 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.034880188, 2023-11-27T15:28:22.982 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.004985329, 2023-11-27T15:28:22.982 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:22.983 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 206, 2023-11-27T15:28:22.983 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.011202373, 2023-11-27T15:28:22.983 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000054380 2023-11-27T15:28:22.983 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:22.983 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:22.983 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2023-11-27T15:28:22.984 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2023-11-27T15:28:22.984 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2023-11-27T15:28:22.984 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:22.984 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 1558400, 2023-11-27T15:28:22.984 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 116640 2023-11-27T15:28:22.984 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:22.985 INFO:teuthology.orchestra.run.smithi052.stdout: "cct": { 2023-11-27T15:28:22.985 INFO:teuthology.orchestra.run.smithi052.stdout: "total_workers": 0, 2023-11-27T15:28:22.985 INFO:teuthology.orchestra.run.smithi052.stdout: "unhealthy_workers": 0 2023-11-27T15:28:22.985 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:22.985 INFO:teuthology.orchestra.run.smithi052.stdout: "mempool": { 2023-11-27T15:28:22.986 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:28:22.986 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_items": 0, 2023-11-27T15:28:22.986 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:28:22.986 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:28:22.986 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:28:22.986 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:28:22.987 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:28:22.987 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:28:22.987 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:28:22.987 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:28:22.987 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:28:22.987 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:28:22.988 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:28:22.988 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:28:22.988 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:28:22.988 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_items": 0, 2023-11-27T15:28:22.988 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:28:22.989 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_items": 0, 2023-11-27T15:28:22.989 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:28:22.989 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:28:22.989 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:28:22.989 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:28:22.989 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:28:22.990 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:28:22.990 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:28:22.990 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_items": 0, 2023-11-27T15:28:22.990 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:28:22.990 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:28:22.990 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:28:22.991 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_items": 0, 2023-11-27T15:28:22.991 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_bytes": 0, 2023-11-27T15:28:22.991 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_items": 0, 2023-11-27T15:28:22.991 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:28:22.991 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:28:22.992 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:28:22.992 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:28:22.992 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_bytes": 148977, 2023-11-27T15:28:22.992 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_items": 30, 2023-11-27T15:28:22.992 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_bytes": 0, 2023-11-27T15:28:22.992 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_items": 0, 2023-11-27T15:28:22.993 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_bytes": 0, 2023-11-27T15:28:22.993 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_items": 0, 2023-11-27T15:28:22.993 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:28:22.993 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_items": 0, 2023-11-27T15:28:22.993 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:28:22.994 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_items": 0, 2023-11-27T15:28:22.994 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_bytes": 0, 2023-11-27T15:28:22.994 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_items": 0, 2023-11-27T15:28:22.994 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:28:22.994 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:28:22.994 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_bytes": 0, 2023-11-27T15:28:22.995 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_items": 0, 2023-11-27T15:28:22.995 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_bytes": 928, 2023-11-27T15:28:22.995 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_items": 2, 2023-11-27T15:28:22.995 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_bytes": 0, 2023-11-27T15:28:22.995 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_items": 0, 2023-11-27T15:28:22.996 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_bytes": 0, 2023-11-27T15:28:22.996 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_items": 0 2023-11-27T15:28:22.996 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2023-11-27T15:28:22.996 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:28:22.996 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2023-11-27T15:28:22.996 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2023-11-27T15:28:22.997 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2023-11-27T15:28:22.997 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 217, 2023-11-27T15:28:22.997 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 1525069, 2023-11-27T15:28:22.997 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:22.997 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 217, 2023-11-27T15:28:22.997 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2023-11-27T15:28:22.998 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2023-11-27T15:28:22.998 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 217, 2023-11-27T15:28:22.998 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 1525069, 2023-11-27T15:28:22.998 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2023-11-27T15:28:22.998 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2023-11-27T15:28:22.998 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2023-11-27T15:28:22.999 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2023-11-27T15:28:22.999 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:22.999 INFO:teuthology.orchestra.run.smithi052.stdout: } 2023-11-27T15:28:22.999 INFO:teuthology.orchestra.run.smithi052.stdout:} 2023-11-27T15:28:23.374 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-11-27T15:28:23.374 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-11-27T15:28:23.374 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2023-11-27T15:28:23.374 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2023-11-27T15:28:23.374 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2023-11-27T15:28:23.374 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2023-11-27T15:28:23.375 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2023-11-27T15:28:23.375 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2023-11-27T15:28:23.375 DEBUG:tasks.check_counter:Getting stats from b 2023-11-27T15:28:23.375 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph daemon mds.b perf dump 2023-11-27T15:28:23.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:23 smithi102 ceph-mon[100296]: Adjusting osd_memory_target on smithi052 to 197.2M 2023-11-27T15:28:23.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:23 smithi102 ceph-mon[100296]: Unable to set osd_memory_target on smithi052 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-11-27T15:28:23.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:23 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:28:23.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:23 smithi102 ceph-mon[100296]: Service mds.all has no related daemon already placed. 2023-11-27T15:28:23.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:23 smithi102 ceph-mon[100296]: Service mgr has no related daemon already placed. 2023-11-27T15:28:23.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:23 smithi102 ceph-mon[100296]: Service mon has no related daemon already placed. 2023-11-27T15:28:23.615 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:23 smithi052 ceph-mon[99056]: Adjusting osd_memory_target on smithi052 to 197.2M 2023-11-27T15:28:23.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:23 smithi052 ceph-mon[99056]: Unable to set osd_memory_target on smithi052 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-11-27T15:28:23.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:23 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:28:23.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:23 smithi052 ceph-mon[99056]: Service mds.all has no related daemon already placed. 2023-11-27T15:28:23.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:23 smithi052 ceph-mon[99056]: Service mgr has no related daemon already placed. 2023-11-27T15:28:23.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:23 smithi052 ceph-mon[99056]: Service mon has no related daemon already placed. 2023-11-27T15:28:23.777 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:23 smithi150 ceph-mon[100559]: Adjusting osd_memory_target on smithi052 to 197.2M 2023-11-27T15:28:23.777 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:23 smithi150 ceph-mon[100559]: Unable to set osd_memory_target on smithi052 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-11-27T15:28:23.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:23 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:28:23.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:23 smithi150 ceph-mon[100559]: Service mds.all has no related daemon already placed. 2023-11-27T15:28:23.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:23 smithi150 ceph-mon[100559]: Service mgr has no related daemon already placed. 2023-11-27T15:28:23.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:23 smithi150 ceph-mon[100559]: Service mon has no related daemon already placed. 2023-11-27T15:28:24.029 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:28:24.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:24 smithi102 ceph-mon[100296]: pgmap v327: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-11-27T15:28:24.615 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:24 smithi052 ceph-mon[99056]: pgmap v327: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-11-27T15:28:24.777 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:24 smithi150 ceph-mon[100559]: pgmap v327: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-11-27T15:28:25.503 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2023-11-27T15:28:25.504 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:28:25.504 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 100049, 2023-11-27T15:28:25.504 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 209356, 2023-11-27T15:28:25.504 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 30086839, 2023-11-27T15:28:25.504 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 240166029, 2023-11-27T15:28:25.504 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 145, 2023-11-27T15:28:25.504 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 4, 2023-11-27T15:28:25.505 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 5.448285759, 2023-11-27T15:28:25.505 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 2.769275600, 2023-11-27T15:28:25.505 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 6.793809771, 2023-11-27T15:28:25.505 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.437003993, 2023-11-27T15:28:25.505 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:25.505 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 209354, 2023-11-27T15:28:25.505 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 6.394132733, 2023-11-27T15:28:25.505 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000030542 2023-11-27T15:28:25.506 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.506 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:25.506 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 178722, 2023-11-27T15:28:25.506 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.033553887, 2023-11-27T15:28:25.506 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000187 2023-11-27T15:28:25.506 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.506 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 94352, 2023-11-27T15:28:25.507 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 30512 2023-11-27T15:28:25.507 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.507 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:28:25.507 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 13524, 2023-11-27T15:28:25.507 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 13626, 2023-11-27T15:28:25.507 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 6607140, 2023-11-27T15:28:25.507 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 80762286, 2023-11-27T15:28:25.507 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 10, 2023-11-27T15:28:25.508 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 7, 2023-11-27T15:28:25.508 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 1.105364612, 2023-11-27T15:28:25.508 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.381969549, 2023-11-27T15:28:25.508 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.203311263, 2023-11-27T15:28:25.508 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.481073614, 2023-11-27T15:28:25.508 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:25.508 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 13626, 2023-11-27T15:28:25.508 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 2.159031684, 2023-11-27T15:28:25.509 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000158449 2023-11-27T15:28:25.509 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.509 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:25.509 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 2, 2023-11-27T15:28:25.509 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000246, 2023-11-27T15:28:25.509 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000123 2023-11-27T15:28:25.509 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.509 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 1654528, 2023-11-27T15:28:25.509 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 453552 2023-11-27T15:28:25.510 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.510 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:28:25.510 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 58347, 2023-11-27T15:28:25.510 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 42444, 2023-11-27T15:28:25.510 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 100300394, 2023-11-27T15:28:25.510 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 58942729, 2023-11-27T15:28:25.510 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 11, 2023-11-27T15:28:25.511 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 6, 2023-11-27T15:28:25.511 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 2.761368442, 2023-11-27T15:28:25.511 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 1.041988759, 2023-11-27T15:28:25.511 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 1.801274280, 2023-11-27T15:28:25.511 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.300851433, 2023-11-27T15:28:25.511 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:25.511 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 42441, 2023-11-27T15:28:25.511 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 2.214686525, 2023-11-27T15:28:25.511 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000052182 2023-11-27T15:28:25.512 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.512 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:25.512 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 90442, 2023-11-27T15:28:25.512 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.015565640, 2023-11-27T15:28:25.512 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000172 2023-11-27T15:28:25.512 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.512 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 371776, 2023-11-27T15:28:25.513 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 222960 2023-11-27T15:28:25.513 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.513 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2023-11-27T15:28:25.513 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 1, 2023-11-27T15:28:25.513 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2023-11-27T15:28:25.513 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.513 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-MDSRank": { 2023-11-27T15:28:25.513 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2023-11-27T15:28:25.514 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2023-11-27T15:28:25.514 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 6495, 2023-11-27T15:28:25.514 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 64.059768396, 2023-11-27T15:28:25.514 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.009862935 2023-11-27T15:28:25.514 INFO:teuthology.orchestra.run.smithi102.stdout: } 2023-11-27T15:28:25.514 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.514 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-PurgeQueue": { 2023-11-27T15:28:25.514 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2023-11-27T15:28:25.515 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2023-11-27T15:28:25.515 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 19943, 2023-11-27T15:28:25.515 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 8.612590997, 2023-11-27T15:28:25.515 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000431860 2023-11-27T15:28:25.515 INFO:teuthology.orchestra.run.smithi102.stdout: } 2023-11-27T15:28:25.515 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.515 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2023-11-27T15:28:25.515 INFO:teuthology.orchestra.run.smithi102.stdout: "request": 68314, 2023-11-27T15:28:25.515 INFO:teuthology.orchestra.run.smithi102.stdout: "reply": 59653, 2023-11-27T15:28:25.516 INFO:teuthology.orchestra.run.smithi102.stdout: "reply_latency": { 2023-11-27T15:28:25.516 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 59653, 2023-11-27T15:28:25.516 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 186.364188488, 2023-11-27T15:28:25.516 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.003124137 2023-11-27T15:28:25.516 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.516 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_reply": 0, 2023-11-27T15:28:25.516 INFO:teuthology.orchestra.run.smithi102.stdout: "forward": 8661, 2023-11-27T15:28:25.516 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_complete": 0, 2023-11-27T15:28:25.517 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_keys": 0, 2023-11-27T15:28:25.517 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_commit": 1533, 2023-11-27T15:28:25.517 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_split": 189, 2023-11-27T15:28:25.517 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_merge": 4, 2023-11-27T15:28:25.517 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes": 19981, 2023-11-27T15:28:25.517 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_top": 0, 2023-11-27T15:28:25.517 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_bottom": 53, 2023-11-27T15:28:25.517 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pin_tail": 19928, 2023-11-27T15:28:25.518 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pinned": 19980, 2023-11-27T15:28:25.518 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_expired": 40359, 2023-11-27T15:28:25.518 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_with_caps": 5, 2023-11-27T15:28:25.518 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": 5, 2023-11-27T15:28:25.518 INFO:teuthology.orchestra.run.smithi102.stdout: "subtrees": 3, 2023-11-27T15:28:25.518 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse": 84090, 2023-11-27T15:28:25.518 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_hit": 75118, 2023-11-27T15:28:25.518 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_forward": 6875, 2023-11-27T15:28:25.519 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_discover": 3, 2023-11-27T15:28:25.519 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_dir_fetch": 0, 2023-11-27T15:28:25.519 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_remote_ino": 0, 2023-11-27T15:28:25.519 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_lock": 0, 2023-11-27T15:28:25.519 INFO:teuthology.orchestra.run.smithi102.stdout: "load_cent": 0, 2023-11-27T15:28:25.519 INFO:teuthology.orchestra.run.smithi102.stdout: "q": 0, 2023-11-27T15:28:25.519 INFO:teuthology.orchestra.run.smithi102.stdout: "exported": 431, 2023-11-27T15:28:25.519 INFO:teuthology.orchestra.run.smithi102.stdout: "exported_inodes": 39425, 2023-11-27T15:28:25.520 INFO:teuthology.orchestra.run.smithi102.stdout: "imported": 382, 2023-11-27T15:28:25.520 INFO:teuthology.orchestra.run.smithi102.stdout: "imported_inodes": 0, 2023-11-27T15:28:25.520 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_dir_fetch": 0, 2023-11-27T15:28:25.520 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_backtrace_fetch": 8697, 2023-11-27T15:28:25.520 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_peer_discover": 0, 2023-11-27T15:28:25.520 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rfiles": 4, 2023-11-27T15:28:25.520 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rbytes": 246, 2023-11-27T15:28:25.520 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rsnaps": 6, 2023-11-27T15:28:25.521 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_fetch": 8697, 2023-11-27T15:28:25.521 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_set_tag": 0, 2023-11-27T15:28:25.521 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_repaired": 0, 2023-11-27T15:28:25.521 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_inotable_repaired": 0, 2023-11-27T15:28:25.521 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_inodes": 158, 2023-11-27T15:28:25.521 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_base_inodes": 16, 2023-11-27T15:28:25.521 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dirfrag_rstats": 142, 2023-11-27T15:28:25.521 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_file_inodes": 8539, 2023-11-27T15:28:25.522 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_inode_file_caps": 4, 2023-11-27T15:28:25.522 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_revoke": 994, 2023-11-27T15:28:25.522 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_grant": 44897, 2023-11-27T15:28:25.522 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_trunc": 1, 2023-11-27T15:28:25.522 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flushsnap_ack": 57, 2023-11-27T15:28:25.522 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-27T15:28:25.522 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps": 19744, 2023-11-27T15:28:25.522 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps_dirty": 18740, 2023-11-27T15:28:25.523 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_cap_release": 966, 2023-11-27T15:28:25.523 INFO:teuthology.orchestra.run.smithi102.stdout: "process_request_cap_release": 29461 2023-11-27T15:28:25.523 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.523 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_cache": { 2023-11-27T15:28:25.523 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update": 1026, 2023-11-27T15:28:25.523 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update_receipt": 2051, 2023-11-27T15:28:25.523 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_try_discover": 1027, 2023-11-27T15:28:25.523 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_send_discover": 1027, 2023-11-27T15:28:25.523 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_handle_discover": 1051, 2023-11-27T15:28:25.524 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays": 7, 2023-11-27T15:28:25.524 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_delayed": 0, 2023-11-27T15:28:25.524 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_enqueuing": 0, 2023-11-27T15:28:25.524 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_created": 14182, 2023-11-27T15:28:25.524 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_enqueued": 14175, 2023-11-27T15:28:25.524 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_reintegrated": 0, 2023-11-27T15:28:25.524 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_migrated": 0, 2023-11-27T15:28:25.524 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_processing": 0, 2023-11-27T15:28:25.525 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_enqueued": 0, 2023-11-27T15:28:25.525 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_prioritized": 0, 2023-11-27T15:28:25.525 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_started": 0, 2023-11-27T15:28:25.525 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_completed": 0, 2023-11-27T15:28:25.525 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_enqueue_scrub": 16, 2023-11-27T15:28:25.525 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_exportdir": 451, 2023-11-27T15:28:25.525 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_flush": 0, 2023-11-27T15:28:25.525 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragmentdir": 200, 2023-11-27T15:28:25.526 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragstats": 0, 2023-11-27T15:28:25.526 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_inodestats": 0 2023-11-27T15:28:25.526 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.526 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_log": { 2023-11-27T15:28:25.526 INFO:teuthology.orchestra.run.smithi102.stdout: "evadd": 81776, 2023-11-27T15:28:25.526 INFO:teuthology.orchestra.run.smithi102.stdout: "evex": 0, 2023-11-27T15:28:25.526 INFO:teuthology.orchestra.run.smithi102.stdout: "evtrm": 0, 2023-11-27T15:28:25.526 INFO:teuthology.orchestra.run.smithi102.stdout: "ev": 81776, 2023-11-27T15:28:25.527 INFO:teuthology.orchestra.run.smithi102.stdout: "evexg": 0, 2023-11-27T15:28:25.527 INFO:teuthology.orchestra.run.smithi102.stdout: "evexd": 0, 2023-11-27T15:28:25.527 INFO:teuthology.orchestra.run.smithi102.stdout: "segadd": 108, 2023-11-27T15:28:25.527 INFO:teuthology.orchestra.run.smithi102.stdout: "segex": 0, 2023-11-27T15:28:25.527 INFO:teuthology.orchestra.run.smithi102.stdout: "segtrm": 0, 2023-11-27T15:28:25.527 INFO:teuthology.orchestra.run.smithi102.stdout: "seg": 108, 2023-11-27T15:28:25.527 INFO:teuthology.orchestra.run.smithi102.stdout: "segexg": 0, 2023-11-27T15:28:25.528 INFO:teuthology.orchestra.run.smithi102.stdout: "segexd": 0, 2023-11-27T15:28:25.528 INFO:teuthology.orchestra.run.smithi102.stdout: "expos": 4194304, 2023-11-27T15:28:25.528 INFO:teuthology.orchestra.run.smithi102.stdout: "wrpos": 244659762, 2023-11-27T15:28:25.528 INFO:teuthology.orchestra.run.smithi102.stdout: "rdpos": 0, 2023-11-27T15:28:25.528 INFO:teuthology.orchestra.run.smithi102.stdout: "jlat": { 2023-11-27T15:28:25.528 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 1926, 2023-11-27T15:28:25.528 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 84.663238049, 2023-11-27T15:28:25.528 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.043958067 2023-11-27T15:28:25.529 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.529 INFO:teuthology.orchestra.run.smithi102.stdout: "replayed": 0 2023-11-27T15:28:25.529 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.529 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_mem": { 2023-11-27T15:28:25.529 INFO:teuthology.orchestra.run.smithi102.stdout: "ino": 10798, 2023-11-27T15:28:25.529 INFO:teuthology.orchestra.run.smithi102.stdout: "ino+": 55358, 2023-11-27T15:28:25.529 INFO:teuthology.orchestra.run.smithi102.stdout: "ino-": 44560, 2023-11-27T15:28:25.529 INFO:teuthology.orchestra.run.smithi102.stdout: "dir": 502, 2023-11-27T15:28:25.530 INFO:teuthology.orchestra.run.smithi102.stdout: "dir+": 3601, 2023-11-27T15:28:25.530 INFO:teuthology.orchestra.run.smithi102.stdout: "dir-": 3099, 2023-11-27T15:28:25.530 INFO:teuthology.orchestra.run.smithi102.stdout: "dn": 19981, 2023-11-27T15:28:25.530 INFO:teuthology.orchestra.run.smithi102.stdout: "dn+": 74367, 2023-11-27T15:28:25.530 INFO:teuthology.orchestra.run.smithi102.stdout: "dn-": 54386, 2023-11-27T15:28:25.530 INFO:teuthology.orchestra.run.smithi102.stdout: "cap": 5, 2023-11-27T15:28:25.530 INFO:teuthology.orchestra.run.smithi102.stdout: "cap+": 43582, 2023-11-27T15:28:25.530 INFO:teuthology.orchestra.run.smithi102.stdout: "cap-": 43577, 2023-11-27T15:28:25.531 INFO:teuthology.orchestra.run.smithi102.stdout: "rss": 212512, 2023-11-27T15:28:25.531 INFO:teuthology.orchestra.run.smithi102.stdout: "heap": 207156 2023-11-27T15:28:25.531 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.531 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_server": { 2023-11-27T15:28:25.531 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_client_request": 68746, 2023-11-27T15:28:25.531 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_server_request": 13, 2023-11-27T15:28:25.531 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_request": 68314, 2023-11-27T15:28:25.531 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_session": 82, 2023-11-27T15:28:25.532 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_peer_request": 99, 2023-11-27T15:28:25.532 INFO:teuthology.orchestra.run.smithi102.stdout: "req_create_latency": { 2023-11-27T15:28:25.532 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 29745, 2023-11-27T15:28:25.532 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 95.193781539, 2023-11-27T15:28:25.532 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.003200328 2023-11-27T15:28:25.532 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.532 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getattr_latency": { 2023-11-27T15:28:25.532 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 411, 2023-11-27T15:28:25.533 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 1.053074182, 2023-11-27T15:28:25.533 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.002562224 2023-11-27T15:28:25.533 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.533 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getfilelock_latency": { 2023-11-27T15:28:25.533 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:25.533 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:25.533 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:25.534 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.534 INFO:teuthology.orchestra.run.smithi102.stdout: "req_link_latency": { 2023-11-27T15:28:25.534 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:25.534 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:25.534 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:25.534 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.534 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookup_latency": { 2023-11-27T15:28:25.534 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 14120, 2023-11-27T15:28:25.535 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 23.702403973, 2023-11-27T15:28:25.535 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.001678640 2023-11-27T15:28:25.535 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.535 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookuphash_latency": { 2023-11-27T15:28:25.535 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:25.535 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:25.535 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:25.535 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.535 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupino_latency": { 2023-11-27T15:28:25.536 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:25.536 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:25.536 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:25.536 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.536 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupname_latency": { 2023-11-27T15:28:25.536 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:25.536 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:25.537 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:25.537 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.537 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupparent_latency": { 2023-11-27T15:28:25.537 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:25.537 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:25.537 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:25.537 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.537 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupsnap_latency": { 2023-11-27T15:28:25.538 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:25.538 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:25.538 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:25.538 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.538 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lssnap_latency": { 2023-11-27T15:28:25.538 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 11, 2023-11-27T15:28:25.538 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.002532457, 2023-11-27T15:28:25.538 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000230223 2023-11-27T15:28:25.539 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.539 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mkdir_latency": { 2023-11-27T15:28:25.539 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 125, 2023-11-27T15:28:25.539 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.130615591, 2023-11-27T15:28:25.539 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.001044924 2023-11-27T15:28:25.539 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.539 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mknod_latency": { 2023-11-27T15:28:25.539 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:25.540 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:25.540 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:25.540 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.540 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mksnap_latency": { 2023-11-27T15:28:25.540 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 11, 2023-11-27T15:28:25.540 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 1.529221988, 2023-11-27T15:28:25.540 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.139020180 2023-11-27T15:28:25.540 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.541 INFO:teuthology.orchestra.run.smithi102.stdout: "req_open_latency": { 2023-11-27T15:28:25.541 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:25.541 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:25.541 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:25.541 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.541 INFO:teuthology.orchestra.run.smithi102.stdout: "req_readdir_latency": { 2023-11-27T15:28:25.541 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 998, 2023-11-27T15:28:25.541 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 1.001696554, 2023-11-27T15:28:25.542 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.001003703 2023-11-27T15:28:25.542 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.542 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rename_latency": { 2023-11-27T15:28:25.542 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 2, 2023-11-27T15:28:25.542 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.002611700, 2023-11-27T15:28:25.542 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.001305850 2023-11-27T15:28:25.542 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.542 INFO:teuthology.orchestra.run.smithi102.stdout: "req_renamesnap_latency": { 2023-11-27T15:28:25.543 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:25.543 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:25.543 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:25.543 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.543 INFO:teuthology.orchestra.run.smithi102.stdout: "req_snapdiff_latency": { 2023-11-27T15:28:25.543 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:25.543 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:25.543 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:25.544 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.544 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmdir_latency": { 2023-11-27T15:28:25.544 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 112, 2023-11-27T15:28:25.544 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.360394516, 2023-11-27T15:28:25.544 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.003217808 2023-11-27T15:28:25.544 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.544 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmsnap_latency": { 2023-11-27T15:28:25.544 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 5, 2023-11-27T15:28:25.544 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.024030029, 2023-11-27T15:28:25.545 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.004806005 2023-11-27T15:28:25.545 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.545 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmxattr_latency": { 2023-11-27T15:28:25.545 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 3, 2023-11-27T15:28:25.545 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.002061641, 2023-11-27T15:28:25.545 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000687213 2023-11-27T15:28:25.545 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.545 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setattr_latency": { 2023-11-27T15:28:25.546 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 3, 2023-11-27T15:28:25.546 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.012111178, 2023-11-27T15:28:25.546 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.004037059 2023-11-27T15:28:25.546 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.546 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setdirlayout_latency": { 2023-11-27T15:28:25.546 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:25.546 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:25.547 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:25.547 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.547 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setfilelock_latency": { 2023-11-27T15:28:25.547 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 22, 2023-11-27T15:28:25.547 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 10.024736576, 2023-11-27T15:28:25.547 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.455669844 2023-11-27T15:28:25.547 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.547 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setlayout_latency": { 2023-11-27T15:28:25.548 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:25.548 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:25.548 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:25.548 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.548 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setxattr_latency": { 2023-11-27T15:28:25.548 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 15, 2023-11-27T15:28:25.548 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.028745275, 2023-11-27T15:28:25.548 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.001916351 2023-11-27T15:28:25.549 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.549 INFO:teuthology.orchestra.run.smithi102.stdout: "req_symlink_latency": { 2023-11-27T15:28:25.549 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:25.549 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:25.549 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:25.549 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.549 INFO:teuthology.orchestra.run.smithi102.stdout: "req_unlink_latency": { 2023-11-27T15:28:25.549 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 14070, 2023-11-27T15:28:25.549 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 53.296171289, 2023-11-27T15:28:25.550 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.003787929 2023-11-27T15:28:25.550 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.550 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_revoke_eviction": 0, 2023-11-27T15:28:25.550 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_acquisition_throttle": 0, 2023-11-27T15:28:25.550 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getvxattr_latency": { 2023-11-27T15:28:25.550 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:25.550 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:25.551 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:25.551 INFO:teuthology.orchestra.run.smithi102.stdout: } 2023-11-27T15:28:25.551 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.551 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_sessions": { 2023-11-27T15:28:25.551 INFO:teuthology.orchestra.run.smithi102.stdout: "session_count": 2, 2023-11-27T15:28:25.551 INFO:teuthology.orchestra.run.smithi102.stdout: "session_add": 5, 2023-11-27T15:28:25.551 INFO:teuthology.orchestra.run.smithi102.stdout: "session_remove": 3, 2023-11-27T15:28:25.551 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_open": 2, 2023-11-27T15:28:25.551 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_stale": 0, 2023-11-27T15:28:25.552 INFO:teuthology.orchestra.run.smithi102.stdout: "total_load": 626, 2023-11-27T15:28:25.552 INFO:teuthology.orchestra.run.smithi102.stdout: "average_load": 313, 2023-11-27T15:28:25.552 INFO:teuthology.orchestra.run.smithi102.stdout: "avg_session_uptime": 635 2023-11-27T15:28:25.552 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.552 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2023-11-27T15:28:25.552 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:28:25.552 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2023-11-27T15:28:25.552 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:28:25.553 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:28:25.553 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:28:25.553 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:28:25.553 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:28:25.553 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:28:25.553 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:28:25.553 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:28:25.553 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:28:25.554 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:28:25.554 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:28:25.554 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:28:25.554 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:28:25.554 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2023-11-27T15:28:25.554 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:28:25.554 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2023-11-27T15:28:25.554 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:28:25.555 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:28:25.555 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:28:25.555 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:28:25.555 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:28:25.555 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:28:25.555 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:28:25.555 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2023-11-27T15:28:25.555 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:28:25.556 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:28:25.556 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:28:25.556 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2023-11-27T15:28:25.556 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2023-11-27T15:28:25.556 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2023-11-27T15:28:25.556 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:28:25.556 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:28:25.556 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:28:25.557 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:28:25.557 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 683820, 2023-11-27T15:28:25.557 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 9314, 2023-11-27T15:28:25.557 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 616, 2023-11-27T15:28:25.557 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 7, 2023-11-27T15:28:25.557 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2023-11-27T15:28:25.557 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2023-11-27T15:28:25.557 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:28:25.558 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2023-11-27T15:28:25.558 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:28:25.558 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2023-11-27T15:28:25.558 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 8224, 2023-11-27T15:28:25.558 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 171, 2023-11-27T15:28:25.558 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:28:25.558 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:28:25.559 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2023-11-27T15:28:25.559 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2023-11-27T15:28:25.559 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 39017948, 2023-11-27T15:28:25.559 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 372099, 2023-11-27T15:28:25.559 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2023-11-27T15:28:25.559 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2023-11-27T15:28:25.559 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2023-11-27T15:28:25.559 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2023-11-27T15:28:25.560 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.560 INFO:teuthology.orchestra.run.smithi102.stdout: "objecter": { 2023-11-27T15:28:25.560 INFO:teuthology.orchestra.run.smithi102.stdout: "op_active": 0, 2023-11-27T15:28:25.560 INFO:teuthology.orchestra.run.smithi102.stdout: "op_laggy": 0, 2023-11-27T15:28:25.560 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send": 27891, 2023-11-27T15:28:25.560 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send_bytes": 260031353, 2023-11-27T15:28:25.560 INFO:teuthology.orchestra.run.smithi102.stdout: "op_resend": 37, 2023-11-27T15:28:25.560 INFO:teuthology.orchestra.run.smithi102.stdout: "op_reply": 27891, 2023-11-27T15:28:25.561 INFO:teuthology.orchestra.run.smithi102.stdout: "op_latency": { 2023-11-27T15:28:25.561 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 27891, 2023-11-27T15:28:25.561 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 84.773179151, 2023-11-27T15:28:25.561 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.003039445 2023-11-27T15:28:25.561 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.561 INFO:teuthology.orchestra.run.smithi102.stdout: "op_inflight": 0, 2023-11-27T15:28:25.561 INFO:teuthology.orchestra.run.smithi102.stdout: "oplen_avg": { 2023-11-27T15:28:25.561 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 27891, 2023-11-27T15:28:25.562 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 29529 2023-11-27T15:28:25.562 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.562 INFO:teuthology.orchestra.run.smithi102.stdout: "op": 27891, 2023-11-27T15:28:25.562 INFO:teuthology.orchestra.run.smithi102.stdout: "op_r": 8983, 2023-11-27T15:28:25.562 INFO:teuthology.orchestra.run.smithi102.stdout: "op_w": 18908, 2023-11-27T15:28:25.562 INFO:teuthology.orchestra.run.smithi102.stdout: "op_rmw": 0, 2023-11-27T15:28:25.562 INFO:teuthology.orchestra.run.smithi102.stdout: "op_pg": 0, 2023-11-27T15:28:25.562 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_stat": 0, 2023-11-27T15:28:25.563 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_create": 11, 2023-11-27T15:28:25.563 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_read": 270, 2023-11-27T15:28:25.563 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_write": 2180, 2023-11-27T15:28:25.563 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writefull": 135, 2023-11-27T15:28:25.563 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writesame": 0, 2023-11-27T15:28:25.563 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_append": 0, 2023-11-27T15:28:25.563 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_zero": 0, 2023-11-27T15:28:25.563 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_truncate": 6, 2023-11-27T15:28:25.564 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_delete": 14977, 2023-11-27T15:28:25.564 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_mapext": 0, 2023-11-27T15:28:25.564 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_sparse_read": 0, 2023-11-27T15:28:25.564 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_clonerange": 0, 2023-11-27T15:28:25.564 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_getxattr": 8713, 2023-11-27T15:28:25.564 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_setxattr": 22, 2023-11-27T15:28:25.564 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_cmpxattr": 0, 2023-11-27T15:28:25.564 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_rmxattr": 0, 2023-11-27T15:28:25.564 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_resetxattrs": 0, 2023-11-27T15:28:25.565 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_call": 0, 2023-11-27T15:28:25.565 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_watch": 0, 2023-11-27T15:28:25.565 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_notify": 0, 2023-11-27T15:28:25.565 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_src_cmpxattr": 0, 2023-11-27T15:28:25.565 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls": 0, 2023-11-27T15:28:25.565 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls_filter": 0, 2023-11-27T15:28:25.565 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_other": 1, 2023-11-27T15:28:25.566 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_active": 0, 2023-11-27T15:28:25.566 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_send": 0, 2023-11-27T15:28:25.566 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_resend": 0, 2023-11-27T15:28:25.566 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_ping": 0, 2023-11-27T15:28:25.566 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_active": 0, 2023-11-27T15:28:25.566 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_send": 0, 2023-11-27T15:28:25.566 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_resend": 0, 2023-11-27T15:28:25.566 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_active": 0, 2023-11-27T15:28:25.567 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_send": 0, 2023-11-27T15:28:25.567 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_resend": 0, 2023-11-27T15:28:25.567 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_active": 0, 2023-11-27T15:28:25.567 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_send": 0, 2023-11-27T15:28:25.567 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_resend": 0, 2023-11-27T15:28:25.567 INFO:teuthology.orchestra.run.smithi102.stdout: "command_active": 0, 2023-11-27T15:28:25.567 INFO:teuthology.orchestra.run.smithi102.stdout: "command_send": 0, 2023-11-27T15:28:25.567 INFO:teuthology.orchestra.run.smithi102.stdout: "command_resend": 0, 2023-11-27T15:28:25.568 INFO:teuthology.orchestra.run.smithi102.stdout: "map_epoch": 96, 2023-11-27T15:28:25.568 INFO:teuthology.orchestra.run.smithi102.stdout: "map_full": 0, 2023-11-27T15:28:25.568 INFO:teuthology.orchestra.run.smithi102.stdout: "map_inc": 14, 2023-11-27T15:28:25.568 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_sessions": 12, 2023-11-27T15:28:25.568 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_open": 12, 2023-11-27T15:28:25.568 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_close": 0, 2023-11-27T15:28:25.568 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_laggy": 0, 2023-11-27T15:28:25.568 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_wr": 3170, 2023-11-27T15:28:25.569 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_rd": 0, 2023-11-27T15:28:25.569 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_del": 44 2023-11-27T15:28:25.569 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.569 INFO:teuthology.orchestra.run.smithi102.stdout: "oft": { 2023-11-27T15:28:25.569 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_objs": 1, 2023-11-27T15:28:25.569 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_kv_pairs": 0, 2023-11-27T15:28:25.569 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_updates": 0, 2023-11-27T15:28:25.569 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_removes": 0 2023-11-27T15:28:25.570 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.570 INFO:teuthology.orchestra.run.smithi102.stdout: "osd-slow-ops": { 2023-11-27T15:28:25.570 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_ops_count": 69012 2023-11-27T15:28:25.570 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.570 INFO:teuthology.orchestra.run.smithi102.stdout: "purge_queue": { 2023-11-27T15:28:25.570 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops": 0, 2023-11-27T15:28:25.570 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops_high_water": 10, 2023-11-27T15:28:25.571 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing": 0, 2023-11-27T15:28:25.571 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_high_water": 10, 2023-11-27T15:28:25.571 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed": 14175, 2023-11-27T15:28:25.571 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_item_in_journal": 0 2023-11-27T15:28:25.571 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.571 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:28:25.571 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2023-11-27T15:28:25.571 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2023-11-27T15:28:25.572 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2023-11-27T15:28:25.572 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 171916, 2023-11-27T15:28:25.572 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 122889595, 2023-11-27T15:28:25.572 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:25.572 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 171916, 2023-11-27T15:28:25.572 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2023-11-27T15:28:25.572 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2023-11-27T15:28:25.572 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 171916, 2023-11-27T15:28:25.573 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 122889595, 2023-11-27T15:28:25.573 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2023-11-27T15:28:25.573 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:25.573 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:25.573 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:25.573 INFO:teuthology.orchestra.run.smithi102.stdout: } 2023-11-27T15:28:25.573 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.573 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_bytes": { 2023-11-27T15:28:25.574 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2023-11-27T15:28:25.574 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2023-11-27T15:28:25.574 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2023-11-27T15:28:25.574 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2023-11-27T15:28:25.574 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2023-11-27T15:28:25.574 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:25.574 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2023-11-27T15:28:25.574 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 27891, 2023-11-27T15:28:25.575 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 261616596, 2023-11-27T15:28:25.575 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 27891, 2023-11-27T15:28:25.575 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 261616596, 2023-11-27T15:28:25.575 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2023-11-27T15:28:25.575 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:25.575 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:25.575 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:25.575 INFO:teuthology.orchestra.run.smithi102.stdout: } 2023-11-27T15:28:25.576 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.576 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_ops": { 2023-11-27T15:28:25.576 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2023-11-27T15:28:25.576 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 1024, 2023-11-27T15:28:25.576 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2023-11-27T15:28:25.576 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2023-11-27T15:28:25.576 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2023-11-27T15:28:25.576 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:25.577 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2023-11-27T15:28:25.577 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 27891, 2023-11-27T15:28:25.577 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 27891, 2023-11-27T15:28:25.577 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 27891, 2023-11-27T15:28:25.577 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 27891, 2023-11-27T15:28:25.577 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2023-11-27T15:28:25.577 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:25.577 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:25.578 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:25.578 INFO:teuthology.orchestra.run.smithi102.stdout: } 2023-11-27T15:28:25.578 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.578 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle": { 2023-11-27T15:28:25.578 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2023-11-27T15:28:25.578 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2023-11-27T15:28:25.578 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2023-11-27T15:28:25.578 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 14175, 2023-11-27T15:28:25.579 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 1585243, 2023-11-27T15:28:25.579 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:25.579 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 14175, 2023-11-27T15:28:25.579 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2023-11-27T15:28:25.579 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2023-11-27T15:28:25.579 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 254, 2023-11-27T15:28:25.579 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 1585243, 2023-11-27T15:28:25.579 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2023-11-27T15:28:25.580 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:25.580 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:25.580 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:25.580 INFO:teuthology.orchestra.run.smithi102.stdout: } 2023-11-27T15:28:25.580 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:25.580 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle-0x56020abc80a0": { 2023-11-27T15:28:25.580 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2023-11-27T15:28:25.580 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2023-11-27T15:28:25.581 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2023-11-27T15:28:25.581 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 81776, 2023-11-27T15:28:25.581 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 240465458, 2023-11-27T15:28:25.581 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:25.581 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 81776, 2023-11-27T15:28:25.581 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2023-11-27T15:28:25.581 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2023-11-27T15:28:25.581 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 1926, 2023-11-27T15:28:25.582 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 240465458, 2023-11-27T15:28:25.582 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2023-11-27T15:28:25.582 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:25.582 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:25.582 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:25.582 INFO:teuthology.orchestra.run.smithi102.stdout: } 2023-11-27T15:28:25.582 INFO:teuthology.orchestra.run.smithi102.stdout: } 2023-11-27T15:28:25.582 INFO:teuthology.orchestra.run.smithi102.stdout:} 2023-11-27T15:28:25.884 INFO:tasks.check_counter:Daemon mds.b mds.exported=431 2023-11-27T15:28:25.884 INFO:tasks.check_counter:Daemon mds.b mds.imported=382 2023-11-27T15:28:25.884 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=1026 2023-11-27T15:28:25.885 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=2051 2023-11-27T15:28:25.885 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2023-11-27T15:28:25.885 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2023-11-27T15:28:25.885 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2023-11-27T15:28:25.885 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=189 2023-11-27T15:28:25.885 DEBUG:tasks.check_counter:Getting stats from e 2023-11-27T15:28:25.885 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph daemon mds.e perf dump 2023-11-27T15:28:26.525 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:28:26.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:26 smithi102 ceph-mon[100296]: pgmap v328: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 90 B/s wr, 5 op/s 2023-11-27T15:28:26.615 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:26 smithi052 ceph-mon[99056]: pgmap v328: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 90 B/s wr, 5 op/s 2023-11-27T15:28:26.777 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:26 smithi150 ceph-mon[100559]: pgmap v328: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 90 B/s wr, 5 op/s 2023-11-27T15:28:27.771 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2023-11-27T15:28:27.772 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:28:27.772 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 1762, 2023-11-27T15:28:27.772 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 1763, 2023-11-27T15:28:27.772 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 84439578, 2023-11-27T15:28:27.772 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 547678, 2023-11-27T15:28:27.772 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 8, 2023-11-27T15:28:27.772 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 4, 2023-11-27T15:28:27.772 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.189725861, 2023-11-27T15:28:27.773 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.048468594, 2023-11-27T15:28:27.773 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.074453728, 2023-11-27T15:28:27.773 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.053207304, 2023-11-27T15:28:27.773 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:27.773 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 1761, 2023-11-27T15:28:27.773 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.059474293, 2023-11-27T15:28:27.773 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000033773 2023-11-27T15:28:27.773 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.774 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:27.774 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.774 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.774 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.774 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.774 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-27T15:28:27.775 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 0 2023-11-27T15:28:27.775 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.775 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:28:27.775 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 540, 2023-11-27T15:28:27.775 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 513, 2023-11-27T15:28:27.775 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 81436578, 2023-11-27T15:28:27.775 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 209701, 2023-11-27T15:28:27.775 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 6, 2023-11-27T15:28:27.776 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 5, 2023-11-27T15:28:27.776 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.097321029, 2023-11-27T15:28:27.776 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.018394787, 2023-11-27T15:28:27.776 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.074617859, 2023-11-27T15:28:27.776 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.012367484, 2023-11-27T15:28:27.776 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:27.776 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 513, 2023-11-27T15:28:27.776 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.021216611, 2023-11-27T15:28:27.777 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000041357 2023-11-27T15:28:27.777 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.777 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:27.777 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.777 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.777 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.777 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.778 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 1635344, 2023-11-27T15:28:27.778 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 119760 2023-11-27T15:28:27.778 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.778 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:28:27.778 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 614, 2023-11-27T15:28:27.778 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 768, 2023-11-27T15:28:27.778 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 94070980, 2023-11-27T15:28:27.778 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 613708, 2023-11-27T15:28:27.779 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 8, 2023-11-27T15:28:27.779 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 6, 2023-11-27T15:28:27.779 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.093971692, 2023-11-27T15:28:27.779 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.018222151, 2023-11-27T15:28:27.779 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.053940468, 2023-11-27T15:28:27.779 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.015620319, 2023-11-27T15:28:27.779 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:27.779 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 765, 2023-11-27T15:28:27.780 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.027330093, 2023-11-27T15:28:27.780 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000035725 2023-11-27T15:28:27.780 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.780 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:27.780 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.780 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.780 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.780 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.781 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 320, 2023-11-27T15:28:27.781 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 423376 2023-11-27T15:28:27.781 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.781 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2023-11-27T15:28:27.781 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 1, 2023-11-27T15:28:27.781 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2023-11-27T15:28:27.781 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.781 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-MDSRank": { 2023-11-27T15:28:27.782 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2023-11-27T15:28:27.782 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2023-11-27T15:28:27.782 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 4213, 2023-11-27T15:28:27.782 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.112397272, 2023-11-27T15:28:27.782 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000026678 2023-11-27T15:28:27.782 INFO:teuthology.orchestra.run.smithi102.stdout: } 2023-11-27T15:28:27.782 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.782 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-PurgeQueue": { 2023-11-27T15:28:27.783 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2023-11-27T15:28:27.783 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2023-11-27T15:28:27.783 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.783 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.783 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.783 INFO:teuthology.orchestra.run.smithi102.stdout: } 2023-11-27T15:28:27.783 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.783 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2023-11-27T15:28:27.784 INFO:teuthology.orchestra.run.smithi102.stdout: "request": 0, 2023-11-27T15:28:27.784 INFO:teuthology.orchestra.run.smithi102.stdout: "reply": 0, 2023-11-27T15:28:27.784 INFO:teuthology.orchestra.run.smithi102.stdout: "reply_latency": { 2023-11-27T15:28:27.784 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.784 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.784 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.784 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.784 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_reply": 0, 2023-11-27T15:28:27.785 INFO:teuthology.orchestra.run.smithi102.stdout: "forward": 0, 2023-11-27T15:28:27.785 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_complete": 0, 2023-11-27T15:28:27.785 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_keys": 0, 2023-11-27T15:28:27.785 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_commit": 0, 2023-11-27T15:28:27.785 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_split": 0, 2023-11-27T15:28:27.785 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_merge": 0, 2023-11-27T15:28:27.785 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes": 63326, 2023-11-27T15:28:27.785 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_top": 0, 2023-11-27T15:28:27.785 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_bottom": 0, 2023-11-27T15:28:27.786 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pin_tail": 63326, 2023-11-27T15:28:27.786 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pinned": 63326, 2023-11-27T15:28:27.786 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_expired": 0, 2023-11-27T15:28:27.786 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_with_caps": 0, 2023-11-27T15:28:27.786 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": 0, 2023-11-27T15:28:27.787 INFO:teuthology.orchestra.run.smithi102.stdout: "subtrees": 1, 2023-11-27T15:28:27.787 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse": 0, 2023-11-27T15:28:27.787 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_hit": 0, 2023-11-27T15:28:27.787 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_forward": 0, 2023-11-27T15:28:27.787 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_discover": 0, 2023-11-27T15:28:27.787 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_dir_fetch": 0, 2023-11-27T15:28:27.787 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_remote_ino": 0, 2023-11-27T15:28:27.787 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_lock": 0, 2023-11-27T15:28:27.788 INFO:teuthology.orchestra.run.smithi102.stdout: "load_cent": 0, 2023-11-27T15:28:27.788 INFO:teuthology.orchestra.run.smithi102.stdout: "q": 0, 2023-11-27T15:28:27.788 INFO:teuthology.orchestra.run.smithi102.stdout: "exported": 0, 2023-11-27T15:28:27.788 INFO:teuthology.orchestra.run.smithi102.stdout: "exported_inodes": 0, 2023-11-27T15:28:27.788 INFO:teuthology.orchestra.run.smithi102.stdout: "imported": 0, 2023-11-27T15:28:27.788 INFO:teuthology.orchestra.run.smithi102.stdout: "imported_inodes": 0, 2023-11-27T15:28:27.788 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_dir_fetch": 0, 2023-11-27T15:28:27.788 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_backtrace_fetch": 0, 2023-11-27T15:28:27.789 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_peer_discover": 0, 2023-11-27T15:28:27.789 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rfiles": 0, 2023-11-27T15:28:27.789 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rbytes": 0, 2023-11-27T15:28:27.789 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rsnaps": 0, 2023-11-27T15:28:27.789 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_fetch": 0, 2023-11-27T15:28:27.789 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_set_tag": 0, 2023-11-27T15:28:27.789 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_repaired": 0, 2023-11-27T15:28:27.789 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_inotable_repaired": 0, 2023-11-27T15:28:27.790 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_inodes": 0, 2023-11-27T15:28:27.790 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_base_inodes": 0, 2023-11-27T15:28:27.790 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dirfrag_rstats": 0, 2023-11-27T15:28:27.790 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_file_inodes": 0, 2023-11-27T15:28:27.790 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_inode_file_caps": 0, 2023-11-27T15:28:27.790 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_revoke": 0, 2023-11-27T15:28:27.790 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_grant": 0, 2023-11-27T15:28:27.791 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_trunc": 0, 2023-11-27T15:28:27.791 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-27T15:28:27.791 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-27T15:28:27.791 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps": 0, 2023-11-27T15:28:27.791 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps_dirty": 0, 2023-11-27T15:28:27.791 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_cap_release": 0, 2023-11-27T15:28:27.791 INFO:teuthology.orchestra.run.smithi102.stdout: "process_request_cap_release": 0 2023-11-27T15:28:27.791 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.791 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_cache": { 2023-11-27T15:28:27.792 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update": 0, 2023-11-27T15:28:27.792 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update_receipt": 0, 2023-11-27T15:28:27.792 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_try_discover": 0, 2023-11-27T15:28:27.792 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_send_discover": 0, 2023-11-27T15:28:27.792 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_handle_discover": 0, 2023-11-27T15:28:27.792 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays": 0, 2023-11-27T15:28:27.792 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_delayed": 0, 2023-11-27T15:28:27.792 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_enqueuing": 0, 2023-11-27T15:28:27.793 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_created": 0, 2023-11-27T15:28:27.793 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_enqueued": 0, 2023-11-27T15:28:27.793 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_reintegrated": 0, 2023-11-27T15:28:27.793 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_migrated": 0, 2023-11-27T15:28:27.793 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_processing": 0, 2023-11-27T15:28:27.793 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_enqueued": 0, 2023-11-27T15:28:27.793 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_prioritized": 0, 2023-11-27T15:28:27.793 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_started": 0, 2023-11-27T15:28:27.794 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_completed": 0, 2023-11-27T15:28:27.794 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_enqueue_scrub": 0, 2023-11-27T15:28:27.794 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_exportdir": 0, 2023-11-27T15:28:27.794 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_flush": 0, 2023-11-27T15:28:27.794 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragmentdir": 0, 2023-11-27T15:28:27.794 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragstats": 0, 2023-11-27T15:28:27.794 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_inodestats": 0 2023-11-27T15:28:27.794 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.795 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_log": { 2023-11-27T15:28:27.795 INFO:teuthology.orchestra.run.smithi102.stdout: "evadd": 0, 2023-11-27T15:28:27.795 INFO:teuthology.orchestra.run.smithi102.stdout: "evex": 0, 2023-11-27T15:28:27.795 INFO:teuthology.orchestra.run.smithi102.stdout: "evtrm": 0, 2023-11-27T15:28:27.795 INFO:teuthology.orchestra.run.smithi102.stdout: "ev": 0, 2023-11-27T15:28:27.795 INFO:teuthology.orchestra.run.smithi102.stdout: "evexg": 0, 2023-11-27T15:28:27.795 INFO:teuthology.orchestra.run.smithi102.stdout: "evexd": 0, 2023-11-27T15:28:27.795 INFO:teuthology.orchestra.run.smithi102.stdout: "segadd": 0, 2023-11-27T15:28:27.796 INFO:teuthology.orchestra.run.smithi102.stdout: "segex": 0, 2023-11-27T15:28:27.796 INFO:teuthology.orchestra.run.smithi102.stdout: "segtrm": 0, 2023-11-27T15:28:27.796 INFO:teuthology.orchestra.run.smithi102.stdout: "seg": 116, 2023-11-27T15:28:27.796 INFO:teuthology.orchestra.run.smithi102.stdout: "segexg": 0, 2023-11-27T15:28:27.796 INFO:teuthology.orchestra.run.smithi102.stdout: "segexd": 0, 2023-11-27T15:28:27.796 INFO:teuthology.orchestra.run.smithi102.stdout: "expos": 4194304, 2023-11-27T15:28:27.796 INFO:teuthology.orchestra.run.smithi102.stdout: "wrpos": 0, 2023-11-27T15:28:27.796 INFO:teuthology.orchestra.run.smithi102.stdout: "rdpos": 261650792, 2023-11-27T15:28:27.797 INFO:teuthology.orchestra.run.smithi102.stdout: "jlat": { 2023-11-27T15:28:27.797 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.797 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.797 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.797 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.797 INFO:teuthology.orchestra.run.smithi102.stdout: "replayed": 88194 2023-11-27T15:28:27.797 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.798 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_mem": { 2023-11-27T15:28:27.798 INFO:teuthology.orchestra.run.smithi102.stdout: "ino": 23276, 2023-11-27T15:28:27.798 INFO:teuthology.orchestra.run.smithi102.stdout: "ino+": 44363, 2023-11-27T15:28:27.798 INFO:teuthology.orchestra.run.smithi102.stdout: "ino-": 21087, 2023-11-27T15:28:27.798 INFO:teuthology.orchestra.run.smithi102.stdout: "dir": 1024, 2023-11-27T15:28:27.798 INFO:teuthology.orchestra.run.smithi102.stdout: "dir+": 1203, 2023-11-27T15:28:27.798 INFO:teuthology.orchestra.run.smithi102.stdout: "dir-": 179, 2023-11-27T15:28:27.798 INFO:teuthology.orchestra.run.smithi102.stdout: "dn": 63326, 2023-11-27T15:28:27.799 INFO:teuthology.orchestra.run.smithi102.stdout: "dn+": 65229, 2023-11-27T15:28:27.799 INFO:teuthology.orchestra.run.smithi102.stdout: "dn-": 1903, 2023-11-27T15:28:27.799 INFO:teuthology.orchestra.run.smithi102.stdout: "cap": 0, 2023-11-27T15:28:27.799 INFO:teuthology.orchestra.run.smithi102.stdout: "cap+": 0, 2023-11-27T15:28:27.799 INFO:teuthology.orchestra.run.smithi102.stdout: "cap-": 0, 2023-11-27T15:28:27.799 INFO:teuthology.orchestra.run.smithi102.stdout: "rss": 190700, 2023-11-27T15:28:27.799 INFO:teuthology.orchestra.run.smithi102.stdout: "heap": 207156 2023-11-27T15:28:27.799 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.800 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_server": { 2023-11-27T15:28:27.800 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_client_request": 0, 2023-11-27T15:28:27.800 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_server_request": 0, 2023-11-27T15:28:27.800 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_request": 0, 2023-11-27T15:28:27.800 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_session": 0, 2023-11-27T15:28:27.800 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_peer_request": 0, 2023-11-27T15:28:27.800 INFO:teuthology.orchestra.run.smithi102.stdout: "req_create_latency": { 2023-11-27T15:28:27.800 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.800 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.801 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.801 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.801 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getattr_latency": { 2023-11-27T15:28:27.801 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.801 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.801 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.801 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.801 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getfilelock_latency": { 2023-11-27T15:28:27.802 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.802 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.802 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.802 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.802 INFO:teuthology.orchestra.run.smithi102.stdout: "req_link_latency": { 2023-11-27T15:28:27.802 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.802 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.802 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.803 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.803 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookup_latency": { 2023-11-27T15:28:27.803 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.803 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.803 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.803 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.803 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookuphash_latency": { 2023-11-27T15:28:27.803 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.804 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.804 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.804 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.804 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupino_latency": { 2023-11-27T15:28:27.804 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.804 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.804 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.804 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.805 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupname_latency": { 2023-11-27T15:28:27.805 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.805 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.805 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.805 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.805 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupparent_latency": { 2023-11-27T15:28:27.805 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.805 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.806 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.806 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.806 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupsnap_latency": { 2023-11-27T15:28:27.806 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.806 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.806 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.806 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.806 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lssnap_latency": { 2023-11-27T15:28:27.807 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.807 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.807 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.807 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.807 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mkdir_latency": { 2023-11-27T15:28:27.807 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.807 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.808 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.808 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.808 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mknod_latency": { 2023-11-27T15:28:27.808 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.808 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.808 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.808 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.808 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mksnap_latency": { 2023-11-27T15:28:27.809 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.809 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.809 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.809 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.809 INFO:teuthology.orchestra.run.smithi102.stdout: "req_open_latency": { 2023-11-27T15:28:27.809 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.809 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.809 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.810 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.810 INFO:teuthology.orchestra.run.smithi102.stdout: "req_readdir_latency": { 2023-11-27T15:28:27.810 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.810 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.810 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.810 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.810 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rename_latency": { 2023-11-27T15:28:27.810 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.811 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.811 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.811 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.811 INFO:teuthology.orchestra.run.smithi102.stdout: "req_renamesnap_latency": { 2023-11-27T15:28:27.811 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.811 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.811 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.811 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.812 INFO:teuthology.orchestra.run.smithi102.stdout: "req_snapdiff_latency": { 2023-11-27T15:28:27.812 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.812 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.812 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.812 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.812 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmdir_latency": { 2023-11-27T15:28:27.812 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.812 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.813 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.813 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.813 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmsnap_latency": { 2023-11-27T15:28:27.813 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.813 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.813 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.813 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.813 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmxattr_latency": { 2023-11-27T15:28:27.814 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.814 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.814 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.814 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.814 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setattr_latency": { 2023-11-27T15:28:27.814 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.814 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.815 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.815 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.815 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setdirlayout_latency": { 2023-11-27T15:28:27.815 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.815 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.815 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.815 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.815 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setfilelock_latency": { 2023-11-27T15:28:27.816 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.816 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.816 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.816 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.816 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setlayout_latency": { 2023-11-27T15:28:27.816 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.816 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.816 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.817 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.817 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setxattr_latency": { 2023-11-27T15:28:27.817 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.817 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.817 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.817 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.817 INFO:teuthology.orchestra.run.smithi102.stdout: "req_symlink_latency": { 2023-11-27T15:28:27.817 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.818 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.818 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.818 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.818 INFO:teuthology.orchestra.run.smithi102.stdout: "req_unlink_latency": { 2023-11-27T15:28:27.818 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.818 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.818 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.819 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.819 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_revoke_eviction": 0, 2023-11-27T15:28:27.819 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_acquisition_throttle": 0, 2023-11-27T15:28:27.819 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getvxattr_latency": { 2023-11-27T15:28:27.819 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.819 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.819 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.819 INFO:teuthology.orchestra.run.smithi102.stdout: } 2023-11-27T15:28:27.820 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.820 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_sessions": { 2023-11-27T15:28:27.820 INFO:teuthology.orchestra.run.smithi102.stdout: "session_count": 2, 2023-11-27T15:28:27.820 INFO:teuthology.orchestra.run.smithi102.stdout: "session_add": 5, 2023-11-27T15:28:27.820 INFO:teuthology.orchestra.run.smithi102.stdout: "session_remove": 3, 2023-11-27T15:28:27.820 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_open": 4, 2023-11-27T15:28:27.820 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_stale": 0, 2023-11-27T15:28:27.820 INFO:teuthology.orchestra.run.smithi102.stdout: "total_load": 0, 2023-11-27T15:28:27.821 INFO:teuthology.orchestra.run.smithi102.stdout: "average_load": 0, 2023-11-27T15:28:27.821 INFO:teuthology.orchestra.run.smithi102.stdout: "avg_session_uptime": 3637 2023-11-27T15:28:27.821 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.821 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2023-11-27T15:28:27.821 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:28:27.821 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2023-11-27T15:28:27.821 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:28:27.821 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:28:27.822 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:28:27.822 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:28:27.822 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:28:27.822 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:28:27.822 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:28:27.822 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:28:27.822 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:28:27.822 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:28:27.823 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:28:27.823 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:28:27.823 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:28:27.823 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2023-11-27T15:28:27.823 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:28:27.823 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2023-11-27T15:28:27.823 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:28:27.823 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:28:27.824 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:28:27.824 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:28:27.824 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:28:27.824 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:28:27.824 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:28:27.824 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2023-11-27T15:28:27.824 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:28:27.824 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:28:27.825 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:28:27.825 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2023-11-27T15:28:27.825 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2023-11-27T15:28:27.825 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2023-11-27T15:28:27.825 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:28:27.825 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:28:27.825 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:28:27.825 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:28:27.826 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 1029691, 2023-11-27T15:28:27.826 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 23344, 2023-11-27T15:28:27.826 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 0, 2023-11-27T15:28:27.826 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 0, 2023-11-27T15:28:27.826 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2023-11-27T15:28:27.826 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2023-11-27T15:28:27.826 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:28:27.826 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2023-11-27T15:28:27.827 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:28:27.827 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2023-11-27T15:28:27.827 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 8224, 2023-11-27T15:28:27.827 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 171, 2023-11-27T15:28:27.827 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:28:27.827 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:28:27.827 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2023-11-27T15:28:27.828 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2023-11-27T15:28:27.828 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 97464946, 2023-11-27T15:28:27.828 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 769837, 2023-11-27T15:28:27.828 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2023-11-27T15:28:27.828 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2023-11-27T15:28:27.828 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2023-11-27T15:28:27.828 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2023-11-27T15:28:27.828 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.829 INFO:teuthology.orchestra.run.smithi102.stdout: "objecter": { 2023-11-27T15:28:27.829 INFO:teuthology.orchestra.run.smithi102.stdout: "op_active": 0, 2023-11-27T15:28:27.829 INFO:teuthology.orchestra.run.smithi102.stdout: "op_laggy": 0, 2023-11-27T15:28:27.829 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send": 2662, 2023-11-27T15:28:27.829 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send_bytes": 21, 2023-11-27T15:28:27.829 INFO:teuthology.orchestra.run.smithi102.stdout: "op_resend": 3, 2023-11-27T15:28:27.829 INFO:teuthology.orchestra.run.smithi102.stdout: "op_reply": 2662, 2023-11-27T15:28:27.829 INFO:teuthology.orchestra.run.smithi102.stdout: "op_latency": { 2023-11-27T15:28:27.830 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 2662, 2023-11-27T15:28:27.830 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 2.699942218, 2023-11-27T15:28:27.830 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.001014253 2023-11-27T15:28:27.830 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.830 INFO:teuthology.orchestra.run.smithi102.stdout: "op_inflight": 0, 2023-11-27T15:28:27.830 INFO:teuthology.orchestra.run.smithi102.stdout: "oplen_avg": { 2023-11-27T15:28:27.830 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 2662, 2023-11-27T15:28:27.830 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 2663 2023-11-27T15:28:27.831 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.831 INFO:teuthology.orchestra.run.smithi102.stdout: "op": 2662, 2023-11-27T15:28:27.831 INFO:teuthology.orchestra.run.smithi102.stdout: "op_r": 2662, 2023-11-27T15:28:27.831 INFO:teuthology.orchestra.run.smithi102.stdout: "op_w": 0, 2023-11-27T15:28:27.831 INFO:teuthology.orchestra.run.smithi102.stdout: "op_rmw": 0, 2023-11-27T15:28:27.831 INFO:teuthology.orchestra.run.smithi102.stdout: "op_pg": 0, 2023-11-27T15:28:27.831 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_stat": 1751, 2023-11-27T15:28:27.831 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_create": 0, 2023-11-27T15:28:27.832 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_read": 909, 2023-11-27T15:28:27.832 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_write": 0, 2023-11-27T15:28:27.832 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writefull": 0, 2023-11-27T15:28:27.832 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writesame": 0, 2023-11-27T15:28:27.832 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_append": 0, 2023-11-27T15:28:27.832 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_zero": 0, 2023-11-27T15:28:27.832 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_truncate": 0, 2023-11-27T15:28:27.832 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_delete": 0, 2023-11-27T15:28:27.833 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_mapext": 0, 2023-11-27T15:28:27.833 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_sparse_read": 0, 2023-11-27T15:28:27.833 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_clonerange": 0, 2023-11-27T15:28:27.833 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_getxattr": 1, 2023-11-27T15:28:27.833 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_setxattr": 0, 2023-11-27T15:28:27.833 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_cmpxattr": 0, 2023-11-27T15:28:27.833 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_rmxattr": 0, 2023-11-27T15:28:27.833 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_resetxattrs": 0, 2023-11-27T15:28:27.834 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_call": 0, 2023-11-27T15:28:27.834 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_watch": 0, 2023-11-27T15:28:27.834 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_notify": 0, 2023-11-27T15:28:27.834 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_src_cmpxattr": 0, 2023-11-27T15:28:27.834 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls": 0, 2023-11-27T15:28:27.834 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls_filter": 0, 2023-11-27T15:28:27.834 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_other": 0, 2023-11-27T15:28:27.835 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_active": 0, 2023-11-27T15:28:27.835 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_send": 0, 2023-11-27T15:28:27.835 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_resend": 0, 2023-11-27T15:28:27.835 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_ping": 0, 2023-11-27T15:28:27.835 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_active": 0, 2023-11-27T15:28:27.835 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_send": 0, 2023-11-27T15:28:27.835 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_resend": 0, 2023-11-27T15:28:27.835 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_active": 0, 2023-11-27T15:28:27.836 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_send": 0, 2023-11-27T15:28:27.836 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_resend": 0, 2023-11-27T15:28:27.836 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_active": 0, 2023-11-27T15:28:27.836 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_send": 0, 2023-11-27T15:28:27.836 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_resend": 0, 2023-11-27T15:28:27.836 INFO:teuthology.orchestra.run.smithi102.stdout: "command_active": 0, 2023-11-27T15:28:27.836 INFO:teuthology.orchestra.run.smithi102.stdout: "command_send": 0, 2023-11-27T15:28:27.836 INFO:teuthology.orchestra.run.smithi102.stdout: "command_resend": 0, 2023-11-27T15:28:27.837 INFO:teuthology.orchestra.run.smithi102.stdout: "map_epoch": 96, 2023-11-27T15:28:27.837 INFO:teuthology.orchestra.run.smithi102.stdout: "map_full": 0, 2023-11-27T15:28:27.837 INFO:teuthology.orchestra.run.smithi102.stdout: "map_inc": 13, 2023-11-27T15:28:27.837 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_sessions": 11, 2023-11-27T15:28:27.837 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_open": 11, 2023-11-27T15:28:27.837 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_close": 0, 2023-11-27T15:28:27.837 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_laggy": 0, 2023-11-27T15:28:27.837 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_wr": 0, 2023-11-27T15:28:27.838 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_rd": 2, 2023-11-27T15:28:27.838 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_del": 0 2023-11-27T15:28:27.838 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.838 INFO:teuthology.orchestra.run.smithi102.stdout: "oft": { 2023-11-27T15:28:27.838 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_objs": 0, 2023-11-27T15:28:27.838 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_kv_pairs": 0, 2023-11-27T15:28:27.838 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_updates": 0, 2023-11-27T15:28:27.838 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_removes": 0 2023-11-27T15:28:27.839 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.839 INFO:teuthology.orchestra.run.smithi102.stdout: "osd-slow-ops": { 2023-11-27T15:28:27.839 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_ops_count": 0 2023-11-27T15:28:27.839 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.839 INFO:teuthology.orchestra.run.smithi102.stdout: "purge_queue": { 2023-11-27T15:28:27.839 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops": 0, 2023-11-27T15:28:27.839 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops_high_water": 0, 2023-11-27T15:28:27.839 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing": 0, 2023-11-27T15:28:27.840 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_high_water": 0, 2023-11-27T15:28:27.840 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed": 0, 2023-11-27T15:28:27.840 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_item_in_journal": 0 2023-11-27T15:28:27.840 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.840 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:28:27.840 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2023-11-27T15:28:27.840 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2023-11-27T15:28:27.840 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2023-11-27T15:28:27.841 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 2912, 2023-11-27T15:28:27.841 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 259669480, 2023-11-27T15:28:27.841 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:27.841 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 2912, 2023-11-27T15:28:27.841 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2023-11-27T15:28:27.841 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2023-11-27T15:28:27.841 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 2912, 2023-11-27T15:28:27.841 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 259669480, 2023-11-27T15:28:27.842 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2023-11-27T15:28:27.842 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.842 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.842 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.842 INFO:teuthology.orchestra.run.smithi102.stdout: } 2023-11-27T15:28:27.842 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.842 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_bytes": { 2023-11-27T15:28:27.842 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2023-11-27T15:28:27.843 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2023-11-27T15:28:27.843 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2023-11-27T15:28:27.843 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2023-11-27T15:28:27.843 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2023-11-27T15:28:27.843 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:27.843 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2023-11-27T15:28:27.843 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 2662, 2023-11-27T15:28:27.843 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 257571380, 2023-11-27T15:28:27.844 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 2662, 2023-11-27T15:28:27.844 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 257571380, 2023-11-27T15:28:27.844 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2023-11-27T15:28:27.844 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.844 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.844 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.844 INFO:teuthology.orchestra.run.smithi102.stdout: } 2023-11-27T15:28:27.844 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.845 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_ops": { 2023-11-27T15:28:27.845 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2023-11-27T15:28:27.845 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 1024, 2023-11-27T15:28:27.845 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2023-11-27T15:28:27.845 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2023-11-27T15:28:27.845 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2023-11-27T15:28:27.845 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:27.845 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2023-11-27T15:28:27.846 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 2662, 2023-11-27T15:28:27.846 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 2662, 2023-11-27T15:28:27.846 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 2662, 2023-11-27T15:28:27.846 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 2662, 2023-11-27T15:28:27.846 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2023-11-27T15:28:27.846 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.846 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.846 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.847 INFO:teuthology.orchestra.run.smithi102.stdout: } 2023-11-27T15:28:27.847 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.847 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle": { 2023-11-27T15:28:27.847 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2023-11-27T15:28:27.847 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2023-11-27T15:28:27.847 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2023-11-27T15:28:27.847 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2023-11-27T15:28:27.847 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2023-11-27T15:28:27.848 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:27.848 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2023-11-27T15:28:27.848 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2023-11-27T15:28:27.848 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2023-11-27T15:28:27.848 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 0, 2023-11-27T15:28:27.848 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 0, 2023-11-27T15:28:27.848 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2023-11-27T15:28:27.848 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.848 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.849 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.849 INFO:teuthology.orchestra.run.smithi102.stdout: } 2023-11-27T15:28:27.849 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:27.849 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle-0x55a06f18e460": { 2023-11-27T15:28:27.849 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2023-11-27T15:28:27.849 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2023-11-27T15:28:27.849 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2023-11-27T15:28:27.849 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2023-11-27T15:28:27.850 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2023-11-27T15:28:27.850 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:27.850 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2023-11-27T15:28:27.850 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2023-11-27T15:28:27.850 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2023-11-27T15:28:27.850 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 0, 2023-11-27T15:28:27.850 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 0, 2023-11-27T15:28:27.851 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2023-11-27T15:28:27.851 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:27.851 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:27.851 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:27.851 INFO:teuthology.orchestra.run.smithi102.stdout: } 2023-11-27T15:28:27.851 INFO:teuthology.orchestra.run.smithi102.stdout: } 2023-11-27T15:28:27.851 INFO:teuthology.orchestra.run.smithi102.stdout:} 2023-11-27T15:28:28.065 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-11-27T15:28:28.065 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-11-27T15:28:28.065 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2023-11-27T15:28:28.065 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2023-11-27T15:28:28.065 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2023-11-27T15:28:28.065 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2023-11-27T15:28:28.066 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2023-11-27T15:28:28.066 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2023-11-27T15:28:28.066 DEBUG:tasks.check_counter:Getting stats from h 2023-11-27T15:28:28.066 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph daemon mds.h perf dump 2023-11-27T15:28:28.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:28 smithi102 ceph-mon[100296]: pgmap v329: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-11-27T15:28:28.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:28 smithi052 ceph-mon[99056]: pgmap v329: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-11-27T15:28:28.701 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:28:28.777 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:28 smithi150 ceph-mon[100559]: pgmap v329: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-11-27T15:28:30.077 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:29 smithi102 ceph-mon[100296]: pgmap v330: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:28:30.097 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2023-11-27T15:28:30.098 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:28:30.098 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 40114, 2023-11-27T15:28:30.098 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 40574, 2023-11-27T15:28:30.098 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 36231746, 2023-11-27T15:28:30.098 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 117229840, 2023-11-27T15:28:30.098 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 19, 2023-11-27T15:28:30.099 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 6, 2023-11-27T15:28:30.099 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 2.229449219, 2023-11-27T15:28:30.099 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.849645138, 2023-11-27T15:28:30.099 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 1.217274814, 2023-11-27T15:28:30.099 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.335253307, 2023-11-27T15:28:30.099 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:30.099 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 40570, 2023-11-27T15:28:30.099 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 2.146990802, 2023-11-27T15:28:30.099 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000052920 2023-11-27T15:28:30.100 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.100 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:30.100 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 64026, 2023-11-27T15:28:30.100 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.009014488, 2023-11-27T15:28:30.100 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000140 2023-11-27T15:28:30.100 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.100 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-27T15:28:30.100 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 305232 2023-11-27T15:28:30.101 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.101 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:28:30.101 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 97987, 2023-11-27T15:28:30.101 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 173940, 2023-11-27T15:28:30.101 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 39355507, 2023-11-27T15:28:30.101 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 197543005, 2023-11-27T15:28:30.101 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 27, 2023-11-27T15:28:30.101 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 8, 2023-11-27T15:28:30.101 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 4.583158785, 2023-11-27T15:28:30.102 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 2.376470447, 2023-11-27T15:28:30.102 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 420.867904692, 2023-11-27T15:28:30.102 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.263651631, 2023-11-27T15:28:30.102 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:30.102 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 173942, 2023-11-27T15:28:30.102 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 6.279107085, 2023-11-27T15:28:30.102 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000036098 2023-11-27T15:28:30.102 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.103 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:30.103 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 146784, 2023-11-27T15:28:30.103 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.029597882, 2023-11-27T15:28:30.103 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000201 2023-11-27T15:28:30.103 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.103 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 1642720, 2023-11-27T15:28:30.103 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 126912 2023-11-27T15:28:30.103 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.103 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:28:30.104 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 8278, 2023-11-27T15:28:30.104 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 8326, 2023-11-27T15:28:30.104 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 4370455, 2023-11-27T15:28:30.104 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 97426560, 2023-11-27T15:28:30.104 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 21, 2023-11-27T15:28:30.104 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 6, 2023-11-27T15:28:30.104 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.961073760, 2023-11-27T15:28:30.104 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.426656913, 2023-11-27T15:28:30.104 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.125682265, 2023-11-27T15:28:30.105 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.373530831, 2023-11-27T15:28:30.105 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:30.105 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 8323, 2023-11-27T15:28:30.105 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 1.522125055, 2023-11-27T15:28:30.105 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000182881 2023-11-27T15:28:30.105 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.105 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:30.105 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 81, 2023-11-27T15:28:30.106 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000013428, 2023-11-27T15:28:30.106 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000165 2023-11-27T15:28:30.106 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.106 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 6432, 2023-11-27T15:28:30.106 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 130480 2023-11-27T15:28:30.106 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.106 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2023-11-27T15:28:30.106 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 1, 2023-11-27T15:28:30.107 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2023-11-27T15:28:30.107 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.107 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-MDSRank": { 2023-11-27T15:28:30.107 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2023-11-27T15:28:30.107 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2023-11-27T15:28:30.107 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 671, 2023-11-27T15:28:30.107 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 84.074888683, 2023-11-27T15:28:30.108 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.125297896 2023-11-27T15:28:30.108 INFO:teuthology.orchestra.run.smithi102.stdout: } 2023-11-27T15:28:30.108 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.108 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-PurgeQueue": { 2023-11-27T15:28:30.108 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2023-11-27T15:28:30.108 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2023-11-27T15:28:30.108 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 27681, 2023-11-27T15:28:30.108 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 19.493596248, 2023-11-27T15:28:30.108 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000704222 2023-11-27T15:28:30.109 INFO:teuthology.orchestra.run.smithi102.stdout: } 2023-11-27T15:28:30.109 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.109 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2023-11-27T15:28:30.109 INFO:teuthology.orchestra.run.smithi102.stdout: "request": 53410, 2023-11-27T15:28:30.109 INFO:teuthology.orchestra.run.smithi102.stdout: "reply": 46360, 2023-11-27T15:28:30.109 INFO:teuthology.orchestra.run.smithi102.stdout: "reply_latency": { 2023-11-27T15:28:30.109 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 46360, 2023-11-27T15:28:30.109 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 179.888033655, 2023-11-27T15:28:30.109 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.003880242 2023-11-27T15:28:30.110 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.110 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_reply": 0, 2023-11-27T15:28:30.110 INFO:teuthology.orchestra.run.smithi102.stdout: "forward": 7050, 2023-11-27T15:28:30.110 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_complete": 0, 2023-11-27T15:28:30.110 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_keys": 0, 2023-11-27T15:28:30.110 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_commit": 1046, 2023-11-27T15:28:30.110 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_split": 129, 2023-11-27T15:28:30.110 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_merge": 3, 2023-11-27T15:28:30.111 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes": 43001, 2023-11-27T15:28:30.111 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_top": 2, 2023-11-27T15:28:30.111 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_bottom": 6, 2023-11-27T15:28:30.111 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pin_tail": 42993, 2023-11-27T15:28:30.111 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pinned": 42998, 2023-11-27T15:28:30.111 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_expired": 1410, 2023-11-27T15:28:30.111 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_with_caps": 1, 2023-11-27T15:28:30.111 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": 1, 2023-11-27T15:28:30.112 INFO:teuthology.orchestra.run.smithi102.stdout: "subtrees": 4, 2023-11-27T15:28:30.112 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse": 65097, 2023-11-27T15:28:30.112 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_hit": 57947, 2023-11-27T15:28:30.112 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_forward": 7045, 2023-11-27T15:28:30.112 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_discover": 8, 2023-11-27T15:28:30.112 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_dir_fetch": 0, 2023-11-27T15:28:30.112 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_remote_ino": 0, 2023-11-27T15:28:30.112 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_lock": 0, 2023-11-27T15:28:30.112 INFO:teuthology.orchestra.run.smithi102.stdout: "load_cent": 0, 2023-11-27T15:28:30.113 INFO:teuthology.orchestra.run.smithi102.stdout: "q": 0, 2023-11-27T15:28:30.113 INFO:teuthology.orchestra.run.smithi102.stdout: "exported": 46, 2023-11-27T15:28:30.113 INFO:teuthology.orchestra.run.smithi102.stdout: "exported_inodes": 1891, 2023-11-27T15:28:30.113 INFO:teuthology.orchestra.run.smithi102.stdout: "imported": 39, 2023-11-27T15:28:30.113 INFO:teuthology.orchestra.run.smithi102.stdout: "imported_inodes": 0, 2023-11-27T15:28:30.113 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_dir_fetch": 0, 2023-11-27T15:28:30.113 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_backtrace_fetch": 39, 2023-11-27T15:28:30.113 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_peer_discover": 0, 2023-11-27T15:28:30.114 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rfiles": 4, 2023-11-27T15:28:30.114 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rbytes": 246, 2023-11-27T15:28:30.114 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rsnaps": 6, 2023-11-27T15:28:30.114 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_fetch": 39, 2023-11-27T15:28:30.114 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_set_tag": 0, 2023-11-27T15:28:30.114 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_repaired": 0, 2023-11-27T15:28:30.114 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_inotable_repaired": 0, 2023-11-27T15:28:30.114 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_inodes": 2, 2023-11-27T15:28:30.115 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_base_inodes": 0, 2023-11-27T15:28:30.115 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dirfrag_rstats": 2, 2023-11-27T15:28:30.115 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_file_inodes": 37, 2023-11-27T15:28:30.115 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_inode_file_caps": 4, 2023-11-27T15:28:30.115 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_revoke": 27, 2023-11-27T15:28:30.115 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_grant": 34977, 2023-11-27T15:28:30.115 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_trunc": 0, 2023-11-27T15:28:30.115 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-27T15:28:30.115 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-27T15:28:30.116 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps": 32606, 2023-11-27T15:28:30.116 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps_dirty": 32577, 2023-11-27T15:28:30.116 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_cap_release": 214, 2023-11-27T15:28:30.116 INFO:teuthology.orchestra.run.smithi102.stdout: "process_request_cap_release": 31645 2023-11-27T15:28:30.116 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.116 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_cache": { 2023-11-27T15:28:30.116 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update": 2046, 2023-11-27T15:28:30.116 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update_receipt": 3046, 2023-11-27T15:28:30.117 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_try_discover": 1525, 2023-11-27T15:28:30.117 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_send_discover": 1530, 2023-11-27T15:28:30.117 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_handle_discover": 2061, 2023-11-27T15:28:30.117 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays": 2, 2023-11-27T15:28:30.117 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_delayed": 0, 2023-11-27T15:28:30.117 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_enqueuing": 0, 2023-11-27T15:28:30.118 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_created": 20053, 2023-11-27T15:28:30.118 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_enqueued": 20051, 2023-11-27T15:28:30.118 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_reintegrated": 0, 2023-11-27T15:28:30.118 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_migrated": 0, 2023-11-27T15:28:30.118 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_processing": 0, 2023-11-27T15:28:30.118 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_enqueued": 0, 2023-11-27T15:28:30.118 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_prioritized": 0, 2023-11-27T15:28:30.118 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_started": 0, 2023-11-27T15:28:30.119 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_completed": 0, 2023-11-27T15:28:30.119 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_enqueue_scrub": 0, 2023-11-27T15:28:30.119 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_exportdir": 48, 2023-11-27T15:28:30.119 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_flush": 0, 2023-11-27T15:28:30.119 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragmentdir": 137, 2023-11-27T15:28:30.119 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragstats": 0, 2023-11-27T15:28:30.119 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_inodestats": 0 2023-11-27T15:28:30.119 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.119 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_log": { 2023-11-27T15:28:30.120 INFO:teuthology.orchestra.run.smithi102.stdout: "evadd": 88194, 2023-11-27T15:28:30.120 INFO:teuthology.orchestra.run.smithi102.stdout: "evex": 0, 2023-11-27T15:28:30.120 INFO:teuthology.orchestra.run.smithi102.stdout: "evtrm": 0, 2023-11-27T15:28:30.120 INFO:teuthology.orchestra.run.smithi102.stdout: "ev": 88194, 2023-11-27T15:28:30.120 INFO:teuthology.orchestra.run.smithi102.stdout: "evexg": 0, 2023-11-27T15:28:30.120 INFO:teuthology.orchestra.run.smithi102.stdout: "evexd": 0, 2023-11-27T15:28:30.120 INFO:teuthology.orchestra.run.smithi102.stdout: "segadd": 116, 2023-11-27T15:28:30.120 INFO:teuthology.orchestra.run.smithi102.stdout: "segex": 0, 2023-11-27T15:28:30.120 INFO:teuthology.orchestra.run.smithi102.stdout: "segtrm": 0, 2023-11-27T15:28:30.121 INFO:teuthology.orchestra.run.smithi102.stdout: "seg": 116, 2023-11-27T15:28:30.121 INFO:teuthology.orchestra.run.smithi102.stdout: "segexg": 0, 2023-11-27T15:28:30.121 INFO:teuthology.orchestra.run.smithi102.stdout: "segexd": 0, 2023-11-27T15:28:30.121 INFO:teuthology.orchestra.run.smithi102.stdout: "expos": 4194304, 2023-11-27T15:28:30.121 INFO:teuthology.orchestra.run.smithi102.stdout: "wrpos": 261653353, 2023-11-27T15:28:30.121 INFO:teuthology.orchestra.run.smithi102.stdout: "rdpos": 0, 2023-11-27T15:28:30.121 INFO:teuthology.orchestra.run.smithi102.stdout: "jlat": { 2023-11-27T15:28:30.121 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 552, 2023-11-27T15:28:30.122 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 115.501713390, 2023-11-27T15:28:30.122 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.209242234 2023-11-27T15:28:30.122 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.122 INFO:teuthology.orchestra.run.smithi102.stdout: "replayed": 0 2023-11-27T15:28:30.122 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.122 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_mem": { 2023-11-27T15:28:30.122 INFO:teuthology.orchestra.run.smithi102.stdout: "ino": 23286, 2023-11-27T15:28:30.122 INFO:teuthology.orchestra.run.smithi102.stdout: "ino+": 44377, 2023-11-27T15:28:30.122 INFO:teuthology.orchestra.run.smithi102.stdout: "ino-": 21091, 2023-11-27T15:28:30.123 INFO:teuthology.orchestra.run.smithi102.stdout: "dir": 1034, 2023-11-27T15:28:30.123 INFO:teuthology.orchestra.run.smithi102.stdout: "dir+": 2736, 2023-11-27T15:28:30.123 INFO:teuthology.orchestra.run.smithi102.stdout: "dir-": 1702, 2023-11-27T15:28:30.123 INFO:teuthology.orchestra.run.smithi102.stdout: "dn": 43001, 2023-11-27T15:28:30.123 INFO:teuthology.orchestra.run.smithi102.stdout: "dn+": 64462, 2023-11-27T15:28:30.123 INFO:teuthology.orchestra.run.smithi102.stdout: "dn-": 21461, 2023-11-27T15:28:30.123 INFO:teuthology.orchestra.run.smithi102.stdout: "cap": 1, 2023-11-27T15:28:30.123 INFO:teuthology.orchestra.run.smithi102.stdout: "cap+": 29321, 2023-11-27T15:28:30.123 INFO:teuthology.orchestra.run.smithi102.stdout: "cap-": 29320, 2023-11-27T15:28:30.124 INFO:teuthology.orchestra.run.smithi102.stdout: "rss": 238044, 2023-11-27T15:28:30.124 INFO:teuthology.orchestra.run.smithi102.stdout: "heap": 198964 2023-11-27T15:28:30.124 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.124 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_server": { 2023-11-27T15:28:30.124 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_client_request": 54337, 2023-11-27T15:28:30.124 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_server_request": 30, 2023-11-27T15:28:30.124 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_request": 53410, 2023-11-27T15:28:30.124 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_session": 35, 2023-11-27T15:28:30.125 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_peer_request": 68, 2023-11-27T15:28:30.125 INFO:teuthology.orchestra.run.smithi102.stdout: "req_create_latency": { 2023-11-27T15:28:30.125 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 14708, 2023-11-27T15:28:30.125 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 27.272727111, 2023-11-27T15:28:30.125 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.001854278 2023-11-27T15:28:30.125 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.125 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getattr_latency": { 2023-11-27T15:28:30.125 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 7, 2023-11-27T15:28:30.125 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.129946891, 2023-11-27T15:28:30.126 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.018563841 2023-11-27T15:28:30.126 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.126 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getfilelock_latency": { 2023-11-27T15:28:30.126 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:30.126 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:30.126 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:30.126 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.126 INFO:teuthology.orchestra.run.smithi102.stdout: "req_link_latency": { 2023-11-27T15:28:30.127 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:30.127 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:30.127 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:30.127 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.127 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookup_latency": { 2023-11-27T15:28:30.127 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 10057, 2023-11-27T15:28:30.127 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 24.572989614, 2023-11-27T15:28:30.127 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.002443371 2023-11-27T15:28:30.127 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.128 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookuphash_latency": { 2023-11-27T15:28:30.128 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:30.128 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:30.128 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:30.128 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.128 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupino_latency": { 2023-11-27T15:28:30.128 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 1, 2023-11-27T15:28:30.128 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000105279, 2023-11-27T15:28:30.129 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000105279 2023-11-27T15:28:30.129 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.129 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupname_latency": { 2023-11-27T15:28:30.129 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:30.129 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:30.129 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:30.129 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.129 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupparent_latency": { 2023-11-27T15:28:30.129 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:30.130 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:30.130 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:30.130 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.130 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupsnap_latency": { 2023-11-27T15:28:30.130 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:30.130 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:30.130 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:30.130 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.130 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lssnap_latency": { 2023-11-27T15:28:30.131 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:30.131 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:30.131 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:30.131 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.131 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mkdir_latency": { 2023-11-27T15:28:30.131 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:30.131 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:30.131 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:30.132 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.132 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mknod_latency": { 2023-11-27T15:28:30.132 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:30.132 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:30.132 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:30.132 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.132 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mksnap_latency": { 2023-11-27T15:28:30.132 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:30.132 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:30.133 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:30.133 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.133 INFO:teuthology.orchestra.run.smithi102.stdout: "req_open_latency": { 2023-11-27T15:28:30.133 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:30.133 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:30.133 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:30.133 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.133 INFO:teuthology.orchestra.run.smithi102.stdout: "req_readdir_latency": { 2023-11-27T15:28:30.133 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 1534, 2023-11-27T15:28:30.133 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 36.276912357, 2023-11-27T15:28:30.134 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.023648573 2023-11-27T15:28:30.134 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.134 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rename_latency": { 2023-11-27T15:28:30.134 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:30.134 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:30.134 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:30.134 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.134 INFO:teuthology.orchestra.run.smithi102.stdout: "req_renamesnap_latency": { 2023-11-27T15:28:30.135 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:30.135 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:30.135 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:30.135 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.135 INFO:teuthology.orchestra.run.smithi102.stdout: "req_snapdiff_latency": { 2023-11-27T15:28:30.135 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:30.135 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:30.135 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:30.135 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.136 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmdir_latency": { 2023-11-27T15:28:30.136 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 2, 2023-11-27T15:28:30.136 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.033340217, 2023-11-27T15:28:30.136 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.016670108 2023-11-27T15:28:30.136 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.136 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmsnap_latency": { 2023-11-27T15:28:30.136 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:30.136 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:30.136 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:30.137 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.137 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmxattr_latency": { 2023-11-27T15:28:30.137 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:30.137 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:30.137 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:30.137 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.137 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setattr_latency": { 2023-11-27T15:28:30.137 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:30.138 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:30.138 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:30.138 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.138 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setdirlayout_latency": { 2023-11-27T15:28:30.138 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:30.138 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:30.138 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:30.138 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.138 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setfilelock_latency": { 2023-11-27T15:28:30.139 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:30.139 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:30.139 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:30.139 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.139 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setlayout_latency": { 2023-11-27T15:28:30.139 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:30.139 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:30.139 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:30.139 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.140 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setxattr_latency": { 2023-11-27T15:28:30.140 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:30.140 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:30.140 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:30.140 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.140 INFO:teuthology.orchestra.run.smithi102.stdout: "req_symlink_latency": { 2023-11-27T15:28:30.140 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:30.140 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:30.140 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:30.141 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.141 INFO:teuthology.orchestra.run.smithi102.stdout: "req_unlink_latency": { 2023-11-27T15:28:30.141 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 20051, 2023-11-27T15:28:30.141 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 91.602012186, 2023-11-27T15:28:30.141 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.004568451 2023-11-27T15:28:30.141 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.141 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_revoke_eviction": 0, 2023-11-27T15:28:30.141 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_acquisition_throttle": 0, 2023-11-27T15:28:30.142 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getvxattr_latency": { 2023-11-27T15:28:30.142 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:30.142 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:30.142 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:30.142 INFO:teuthology.orchestra.run.smithi102.stdout: } 2023-11-27T15:28:30.142 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.142 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_sessions": { 2023-11-27T15:28:30.142 INFO:teuthology.orchestra.run.smithi102.stdout: "session_count": 2, 2023-11-27T15:28:30.142 INFO:teuthology.orchestra.run.smithi102.stdout: "session_add": 5, 2023-11-27T15:28:30.143 INFO:teuthology.orchestra.run.smithi102.stdout: "session_remove": 3, 2023-11-27T15:28:30.143 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_open": 2, 2023-11-27T15:28:30.143 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_stale": 0, 2023-11-27T15:28:30.143 INFO:teuthology.orchestra.run.smithi102.stdout: "total_load": 21401, 2023-11-27T15:28:30.143 INFO:teuthology.orchestra.run.smithi102.stdout: "average_load": 10700, 2023-11-27T15:28:30.143 INFO:teuthology.orchestra.run.smithi102.stdout: "avg_session_uptime": 626 2023-11-27T15:28:30.143 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.143 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2023-11-27T15:28:30.143 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:28:30.144 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2023-11-27T15:28:30.144 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:28:30.144 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:28:30.144 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:28:30.144 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:28:30.144 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:28:30.144 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:28:30.144 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:28:30.145 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:28:30.145 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:28:30.145 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:28:30.145 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:28:30.145 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:28:30.145 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:28:30.145 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2023-11-27T15:28:30.145 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:28:30.145 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2023-11-27T15:28:30.146 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:28:30.146 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:28:30.146 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:28:30.146 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:28:30.146 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:28:30.146 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:28:30.146 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:28:30.146 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2023-11-27T15:28:30.147 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:28:30.147 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:28:30.147 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:28:30.147 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2023-11-27T15:28:30.147 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2023-11-27T15:28:30.147 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2023-11-27T15:28:30.147 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:28:30.147 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:28:30.147 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:28:30.148 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:28:30.148 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 986354, 2023-11-27T15:28:30.148 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 19831, 2023-11-27T15:28:30.148 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 0, 2023-11-27T15:28:30.148 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 0, 2023-11-27T15:28:30.148 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2023-11-27T15:28:30.148 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2023-11-27T15:28:30.148 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:28:30.149 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2023-11-27T15:28:30.149 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:28:30.149 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2023-11-27T15:28:30.149 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 8224, 2023-11-27T15:28:30.149 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 171, 2023-11-27T15:28:30.149 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:28:30.149 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:28:30.149 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2023-11-27T15:28:30.149 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2023-11-27T15:28:30.150 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 83821119, 2023-11-27T15:28:30.150 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 785203, 2023-11-27T15:28:30.150 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2023-11-27T15:28:30.150 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2023-11-27T15:28:30.150 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2023-11-27T15:28:30.150 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2023-11-27T15:28:30.150 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.151 INFO:teuthology.orchestra.run.smithi102.stdout: "objecter": { 2023-11-27T15:28:30.151 INFO:teuthology.orchestra.run.smithi102.stdout: "op_active": 0, 2023-11-27T15:28:30.151 INFO:teuthology.orchestra.run.smithi102.stdout: "op_laggy": 0, 2023-11-27T15:28:30.151 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send": 22455, 2023-11-27T15:28:30.151 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send_bytes": 271898160, 2023-11-27T15:28:30.151 INFO:teuthology.orchestra.run.smithi102.stdout: "op_resend": 33, 2023-11-27T15:28:30.151 INFO:teuthology.orchestra.run.smithi102.stdout: "op_reply": 22455, 2023-11-27T15:28:30.151 INFO:teuthology.orchestra.run.smithi102.stdout: "op_latency": { 2023-11-27T15:28:30.151 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 22455, 2023-11-27T15:28:30.152 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 75.194380587, 2023-11-27T15:28:30.152 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.003348669 2023-11-27T15:28:30.152 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.152 INFO:teuthology.orchestra.run.smithi102.stdout: "op_inflight": 0, 2023-11-27T15:28:30.152 INFO:teuthology.orchestra.run.smithi102.stdout: "oplen_avg": { 2023-11-27T15:28:30.152 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 22455, 2023-11-27T15:28:30.152 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 23566 2023-11-27T15:28:30.152 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.152 INFO:teuthology.orchestra.run.smithi102.stdout: "op": 22455, 2023-11-27T15:28:30.153 INFO:teuthology.orchestra.run.smithi102.stdout: "op_r": 247, 2023-11-27T15:28:30.153 INFO:teuthology.orchestra.run.smithi102.stdout: "op_w": 22208, 2023-11-27T15:28:30.153 INFO:teuthology.orchestra.run.smithi102.stdout: "op_rmw": 0, 2023-11-27T15:28:30.153 INFO:teuthology.orchestra.run.smithi102.stdout: "op_pg": 0, 2023-11-27T15:28:30.153 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_stat": 0, 2023-11-27T15:28:30.153 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_create": 10, 2023-11-27T15:28:30.153 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_read": 208, 2023-11-27T15:28:30.153 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_write": 760, 2023-11-27T15:28:30.153 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writefull": 89, 2023-11-27T15:28:30.154 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writesame": 0, 2023-11-27T15:28:30.154 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_append": 0, 2023-11-27T15:28:30.154 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_zero": 0, 2023-11-27T15:28:30.154 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_truncate": 0, 2023-11-27T15:28:30.154 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_delete": 20267, 2023-11-27T15:28:30.154 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_mapext": 0, 2023-11-27T15:28:30.154 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_sparse_read": 0, 2023-11-27T15:28:30.154 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_clonerange": 0, 2023-11-27T15:28:30.155 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_getxattr": 39, 2023-11-27T15:28:30.155 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_setxattr": 20, 2023-11-27T15:28:30.155 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_cmpxattr": 0, 2023-11-27T15:28:30.155 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_rmxattr": 0, 2023-11-27T15:28:30.155 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_resetxattrs": 0, 2023-11-27T15:28:30.155 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_call": 0, 2023-11-27T15:28:30.155 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_watch": 0, 2023-11-27T15:28:30.155 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_notify": 0, 2023-11-27T15:28:30.155 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_src_cmpxattr": 0, 2023-11-27T15:28:30.156 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls": 0, 2023-11-27T15:28:30.156 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls_filter": 0, 2023-11-27T15:28:30.156 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_other": 0, 2023-11-27T15:28:30.156 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_active": 0, 2023-11-27T15:28:30.156 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_send": 0, 2023-11-27T15:28:30.156 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_resend": 0, 2023-11-27T15:28:30.156 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_ping": 0, 2023-11-27T15:28:30.156 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_active": 0, 2023-11-27T15:28:30.157 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_send": 0, 2023-11-27T15:28:30.157 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_resend": 0, 2023-11-27T15:28:30.157 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_active": 0, 2023-11-27T15:28:30.157 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_send": 0, 2023-11-27T15:28:30.157 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_resend": 0, 2023-11-27T15:28:30.157 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_active": 0, 2023-11-27T15:28:30.157 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_send": 0, 2023-11-27T15:28:30.157 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_resend": 0, 2023-11-27T15:28:30.157 INFO:teuthology.orchestra.run.smithi102.stdout: "command_active": 0, 2023-11-27T15:28:30.158 INFO:teuthology.orchestra.run.smithi102.stdout: "command_send": 0, 2023-11-27T15:28:30.158 INFO:teuthology.orchestra.run.smithi102.stdout: "command_resend": 0, 2023-11-27T15:28:30.158 INFO:teuthology.orchestra.run.smithi102.stdout: "map_epoch": 96, 2023-11-27T15:28:30.158 INFO:teuthology.orchestra.run.smithi102.stdout: "map_full": 0, 2023-11-27T15:28:30.158 INFO:teuthology.orchestra.run.smithi102.stdout: "map_inc": 13, 2023-11-27T15:28:30.158 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_sessions": 12, 2023-11-27T15:28:30.158 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_open": 12, 2023-11-27T15:28:30.158 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_close": 0, 2023-11-27T15:28:30.159 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_laggy": 0, 2023-11-27T15:28:30.159 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_wr": 2146, 2023-11-27T15:28:30.159 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_rd": 0, 2023-11-27T15:28:30.159 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_del": 27 2023-11-27T15:28:30.159 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.159 INFO:teuthology.orchestra.run.smithi102.stdout: "oft": { 2023-11-27T15:28:30.159 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_objs": 1, 2023-11-27T15:28:30.159 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_kv_pairs": 0, 2023-11-27T15:28:30.160 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_updates": 0, 2023-11-27T15:28:30.160 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_removes": 0 2023-11-27T15:28:30.160 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.160 INFO:teuthology.orchestra.run.smithi102.stdout: "osd-slow-ops": { 2023-11-27T15:28:30.160 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_ops_count": 53625 2023-11-27T15:28:30.160 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.160 INFO:teuthology.orchestra.run.smithi102.stdout: "purge_queue": { 2023-11-27T15:28:30.160 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops": 0, 2023-11-27T15:28:30.160 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops_high_water": 10, 2023-11-27T15:28:30.161 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing": 0, 2023-11-27T15:28:30.161 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_high_water": 10, 2023-11-27T15:28:30.161 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed": 20051, 2023-11-27T15:28:30.161 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_item_in_journal": 0 2023-11-27T15:28:30.161 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.161 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:28:30.161 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2023-11-27T15:28:30.161 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2023-11-27T15:28:30.161 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2023-11-27T15:28:30.162 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 146375, 2023-11-27T15:28:30.162 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 68144889, 2023-11-27T15:28:30.162 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:30.162 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 146375, 2023-11-27T15:28:30.162 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2023-11-27T15:28:30.162 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2023-11-27T15:28:30.162 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 146375, 2023-11-27T15:28:30.162 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 68144889, 2023-11-27T15:28:30.163 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2023-11-27T15:28:30.163 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:30.163 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:30.163 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:30.163 INFO:teuthology.orchestra.run.smithi102.stdout: } 2023-11-27T15:28:30.163 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.163 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_bytes": { 2023-11-27T15:28:30.163 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2023-11-27T15:28:30.163 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2023-11-27T15:28:30.164 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2023-11-27T15:28:30.164 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2023-11-27T15:28:30.164 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2023-11-27T15:28:30.164 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:30.164 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2023-11-27T15:28:30.164 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 22455, 2023-11-27T15:28:30.164 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 274153607, 2023-11-27T15:28:30.164 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 22455, 2023-11-27T15:28:30.164 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 274153607, 2023-11-27T15:28:30.165 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2023-11-27T15:28:30.165 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:30.165 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:30.165 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:30.165 INFO:teuthology.orchestra.run.smithi102.stdout: } 2023-11-27T15:28:30.165 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.165 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_ops": { 2023-11-27T15:28:30.165 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2023-11-27T15:28:30.166 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 1024, 2023-11-27T15:28:30.166 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2023-11-27T15:28:30.166 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2023-11-27T15:28:30.166 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2023-11-27T15:28:30.166 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:30.166 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2023-11-27T15:28:30.166 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 22455, 2023-11-27T15:28:30.166 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 22455, 2023-11-27T15:28:30.167 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 22455, 2023-11-27T15:28:30.167 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 22455, 2023-11-27T15:28:30.167 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2023-11-27T15:28:30.167 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:30.167 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:30.167 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:30.167 INFO:teuthology.orchestra.run.smithi102.stdout: } 2023-11-27T15:28:30.167 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.167 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle": { 2023-11-27T15:28:30.168 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2023-11-27T15:28:30.168 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2023-11-27T15:28:30.168 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2023-11-27T15:28:30.168 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 20051, 2023-11-27T15:28:30.168 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 2255447, 2023-11-27T15:28:30.168 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:30.168 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 20051, 2023-11-27T15:28:30.168 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2023-11-27T15:28:30.169 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2023-11-27T15:28:30.169 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 208, 2023-11-27T15:28:30.169 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 2255447, 2023-11-27T15:28:30.169 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2023-11-27T15:28:30.169 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:30.169 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:30.169 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:30.169 INFO:teuthology.orchestra.run.smithi102.stdout: } 2023-11-27T15:28:30.170 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:30.170 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle-0x562c7966a0a0": { 2023-11-27T15:28:30.170 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2023-11-27T15:28:30.170 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2023-11-27T15:28:30.170 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2023-11-27T15:28:30.170 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 88194, 2023-11-27T15:28:30.170 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 257459049, 2023-11-27T15:28:30.170 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:30.171 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 88194, 2023-11-27T15:28:30.171 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2023-11-27T15:28:30.171 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2023-11-27T15:28:30.171 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 552, 2023-11-27T15:28:30.171 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 257459049, 2023-11-27T15:28:30.171 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2023-11-27T15:28:30.171 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:30.172 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:30.172 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:30.172 INFO:teuthology.orchestra.run.smithi102.stdout: } 2023-11-27T15:28:30.172 INFO:teuthology.orchestra.run.smithi102.stdout: } 2023-11-27T15:28:30.172 INFO:teuthology.orchestra.run.smithi102.stdout:} 2023-11-27T15:28:30.277 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:29 smithi150 ceph-mon[100559]: pgmap v330: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:28:30.365 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:29 smithi052 ceph-mon[99056]: pgmap v330: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:28:30.553 INFO:tasks.check_counter:Daemon mds.h mds.exported=46 2023-11-27T15:28:30.554 INFO:tasks.check_counter:Daemon mds.h mds.imported=39 2023-11-27T15:28:30.554 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=2046 2023-11-27T15:28:30.554 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=3046 2023-11-27T15:28:30.554 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2023-11-27T15:28:30.554 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2023-11-27T15:28:30.554 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2023-11-27T15:28:30.554 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=129 2023-11-27T15:28:30.555 DEBUG:tasks.check_counter:Getting stats from k 2023-11-27T15:28:30.555 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph daemon mds.k perf dump 2023-11-27T15:28:31.193 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:28:32.453 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2023-11-27T15:28:32.454 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:28:32.454 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 0, 2023-11-27T15:28:32.454 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 2, 2023-11-27T15:28:32.454 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 0, 2023-11-27T15:28:32.454 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 0, 2023-11-27T15:28:32.454 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 4, 2023-11-27T15:28:32.454 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 0, 2023-11-27T15:28:32.454 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.001043444, 2023-11-27T15:28:32.455 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.000000000, 2023-11-27T15:28:32.455 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.000239132, 2023-11-27T15:28:32.455 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:28:32.455 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:32.455 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:32.455 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:32.455 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:32.455 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:32.456 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:32.456 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:32.456 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:32.456 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:32.456 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:32.456 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-27T15:28:32.456 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 0 2023-11-27T15:28:32.456 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:32.456 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:28:32.457 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 2, 2023-11-27T15:28:32.457 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 158, 2023-11-27T15:28:32.457 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 320, 2023-11-27T15:28:32.457 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 179536, 2023-11-27T15:28:32.457 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 5, 2023-11-27T15:28:32.457 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 3, 2023-11-27T15:28:32.457 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.010334052, 2023-11-27T15:28:32.457 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.006450716, 2023-11-27T15:28:32.458 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.001797111, 2023-11-27T15:28:32.458 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:28:32.458 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:32.458 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 155, 2023-11-27T15:28:32.458 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.012808477, 2023-11-27T15:28:32.458 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000082635 2023-11-27T15:28:32.458 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:32.458 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:32.459 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:32.459 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:32.459 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:32.459 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:32.459 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 320, 2023-11-27T15:28:32.459 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 179536 2023-11-27T15:28:32.459 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:32.459 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:28:32.460 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 222, 2023-11-27T15:28:32.460 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 209, 2023-11-27T15:28:32.460 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 1559216, 2023-11-27T15:28:32.460 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 118416, 2023-11-27T15:28:32.460 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 2, 2023-11-27T15:28:32.460 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 1, 2023-11-27T15:28:32.460 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.045048893, 2023-11-27T15:28:32.460 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.013605853, 2023-11-27T15:28:32.461 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.034753424, 2023-11-27T15:28:32.461 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.006075203, 2023-11-27T15:28:32.461 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:32.461 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 209, 2023-11-27T15:28:32.461 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.010223289, 2023-11-27T15:28:32.461 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000048915 2023-11-27T15:28:32.461 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:32.461 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:32.461 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:32.462 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:32.462 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:32.462 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:32.462 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 1559216, 2023-11-27T15:28:32.462 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 118416 2023-11-27T15:28:32.462 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:32.462 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2023-11-27T15:28:32.462 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 0, 2023-11-27T15:28:32.463 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2023-11-27T15:28:32.463 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:32.463 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2023-11-27T15:28:32.463 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:28:32.463 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2023-11-27T15:28:32.463 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:28:32.463 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:28:32.463 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:28:32.463 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:28:32.464 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:28:32.464 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:28:32.464 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:28:32.464 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:28:32.464 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:28:32.464 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:28:32.464 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:28:32.464 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:28:32.465 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:28:32.465 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2023-11-27T15:28:32.465 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:28:32.465 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2023-11-27T15:28:32.465 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:28:32.465 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:28:32.465 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:28:32.465 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:28:32.465 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:28:32.466 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:28:32.466 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:28:32.466 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2023-11-27T15:28:32.466 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:28:32.466 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:28:32.466 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:28:32.466 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2023-11-27T15:28:32.466 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2023-11-27T15:28:32.467 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2023-11-27T15:28:32.467 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:28:32.467 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:28:32.467 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:28:32.467 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:28:32.467 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 150008, 2023-11-27T15:28:32.467 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 31, 2023-11-27T15:28:32.467 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 0, 2023-11-27T15:28:32.468 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 0, 2023-11-27T15:28:32.468 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2023-11-27T15:28:32.468 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2023-11-27T15:28:32.468 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:28:32.468 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2023-11-27T15:28:32.468 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:28:32.468 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2023-11-27T15:28:32.468 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 0, 2023-11-27T15:28:32.468 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 0, 2023-11-27T15:28:32.469 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:28:32.469 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:28:32.469 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2023-11-27T15:28:32.469 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2023-11-27T15:28:32.469 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 928, 2023-11-27T15:28:32.469 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 2, 2023-11-27T15:28:32.469 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2023-11-27T15:28:32.469 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2023-11-27T15:28:32.470 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2023-11-27T15:28:32.470 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2023-11-27T15:28:32.470 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2023-11-27T15:28:32.470 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:28:32.470 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2023-11-27T15:28:32.470 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2023-11-27T15:28:32.470 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2023-11-27T15:28:32.470 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 220, 2023-11-27T15:28:32.470 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 1525459, 2023-11-27T15:28:32.471 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:32.471 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 220, 2023-11-27T15:28:32.471 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2023-11-27T15:28:32.471 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2023-11-27T15:28:32.471 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 220, 2023-11-27T15:28:32.471 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 1525459, 2023-11-27T15:28:32.471 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2023-11-27T15:28:32.471 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2023-11-27T15:28:32.471 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2023-11-27T15:28:32.472 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2023-11-27T15:28:32.472 INFO:teuthology.orchestra.run.smithi102.stdout: } 2023-11-27T15:28:32.472 INFO:teuthology.orchestra.run.smithi102.stdout: } 2023-11-27T15:28:32.472 INFO:teuthology.orchestra.run.smithi102.stdout:} 2023-11-27T15:28:32.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:32 smithi150 ceph-mon[100559]: pgmap v331: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:28:32.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:32 smithi102 ceph-mon[100296]: pgmap v331: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:28:32.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:32 smithi052 ceph-mon[99056]: pgmap v331: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:28:32.834 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2023-11-27T15:28:32.834 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2023-11-27T15:28:32.834 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2023-11-27T15:28:32.834 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2023-11-27T15:28:32.834 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2023-11-27T15:28:32.834 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2023-11-27T15:28:32.835 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2023-11-27T15:28:32.835 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2023-11-27T15:28:32.835 DEBUG:tasks.check_counter:Getting stats from c 2023-11-27T15:28:32.835 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph daemon mds.c perf dump 2023-11-27T15:28:33.499 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:28:34.481 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:34 smithi150 ceph-mon[100559]: pgmap v332: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:28:34.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:34 smithi102 ceph-mon[100296]: pgmap v332: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:28:34.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:34 smithi052 ceph-mon[99056]: pgmap v332: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:28:34.698 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2023-11-27T15:28:34.699 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:28:34.699 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 39391, 2023-11-27T15:28:34.699 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 53662, 2023-11-27T15:28:34.699 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 36364222, 2023-11-27T15:28:34.699 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 102239505, 2023-11-27T15:28:34.699 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 28, 2023-11-27T15:28:34.699 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 4, 2023-11-27T15:28:34.699 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 2.341687172, 2023-11-27T15:28:34.700 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.995975406, 2023-11-27T15:28:34.700 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 1.279182763, 2023-11-27T15:28:34.700 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.186297480, 2023-11-27T15:28:34.700 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:34.700 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 53663, 2023-11-27T15:28:34.700 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 1.708389740, 2023-11-27T15:28:34.700 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000031835 2023-11-27T15:28:34.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.700 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:34.701 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 76825, 2023-11-27T15:28:34.701 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.011239904, 2023-11-27T15:28:34.701 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000146 2023-11-27T15:28:34.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.701 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 1633440, 2023-11-27T15:28:34.701 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 125056 2023-11-27T15:28:34.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.701 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:28:34.702 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 44470, 2023-11-27T15:28:34.702 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 44294, 2023-11-27T15:28:34.702 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 53120668, 2023-11-27T15:28:34.702 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 103099607, 2023-11-27T15:28:34.702 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 11, 2023-11-27T15:28:34.702 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 6, 2023-11-27T15:28:34.702 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 2.291831773, 2023-11-27T15:28:34.703 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.927082814, 2023-11-27T15:28:34.703 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 1.224067563, 2023-11-27T15:28:34.703 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.287239701, 2023-11-27T15:28:34.703 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:34.703 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 44291, 2023-11-27T15:28:34.703 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 1.597740839, 2023-11-27T15:28:34.703 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000036073 2023-11-27T15:28:34.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.704 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:34.704 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 64010, 2023-11-27T15:28:34.704 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.008368048, 2023-11-27T15:28:34.704 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000130 2023-11-27T15:28:34.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.704 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 3472, 2023-11-27T15:28:34.704 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 502928 2023-11-27T15:28:34.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.704 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:28:34.705 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 97117, 2023-11-27T15:28:34.705 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 155933, 2023-11-27T15:28:34.705 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 30631748, 2023-11-27T15:28:34.705 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 185226139, 2023-11-27T15:28:34.705 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 26, 2023-11-27T15:28:34.705 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 7, 2023-11-27T15:28:34.705 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 4.381869305, 2023-11-27T15:28:34.705 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 2.222848561, 2023-11-27T15:28:34.706 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 5.256577115, 2023-11-27T15:28:34.706 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.437470758, 2023-11-27T15:28:34.706 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:34.706 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 155929, 2023-11-27T15:28:34.706 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 4.115780213, 2023-11-27T15:28:34.706 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000026395 2023-11-27T15:28:34.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.707 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:34.707 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 114555, 2023-11-27T15:28:34.707 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.023281928, 2023-11-27T15:28:34.707 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000203 2023-11-27T15:28:34.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.707 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 912, 2023-11-27T15:28:34.707 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 688 2023-11-27T15:28:34.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.708 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2023-11-27T15:28:34.708 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 1, 2023-11-27T15:28:34.708 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2023-11-27T15:28:34.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.708 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-MDSRank": { 2023-11-27T15:28:34.708 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2023-11-27T15:28:34.708 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2023-11-27T15:28:34.708 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 10970, 2023-11-27T15:28:34.709 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 59.094440243, 2023-11-27T15:28:34.709 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.005386913 2023-11-27T15:28:34.709 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-27T15:28:34.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.709 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-PurgeQueue": { 2023-11-27T15:28:34.709 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2023-11-27T15:28:34.709 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2023-11-27T15:28:34.709 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 25312, 2023-11-27T15:28:34.709 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 7.209428313, 2023-11-27T15:28:34.710 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000284822 2023-11-27T15:28:34.710 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-27T15:28:34.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.710 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": { 2023-11-27T15:28:34.710 INFO:teuthology.orchestra.run.smithi150.stdout: "request": 46441, 2023-11-27T15:28:34.710 INFO:teuthology.orchestra.run.smithi150.stdout: "reply": 39895, 2023-11-27T15:28:34.710 INFO:teuthology.orchestra.run.smithi150.stdout: "reply_latency": { 2023-11-27T15:28:34.711 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 39895, 2023-11-27T15:28:34.711 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 147.260431609, 2023-11-27T15:28:34.711 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.003691200 2023-11-27T15:28:34.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.711 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_reply": 0, 2023-11-27T15:28:34.711 INFO:teuthology.orchestra.run.smithi150.stdout: "forward": 6546, 2023-11-27T15:28:34.711 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_complete": 0, 2023-11-27T15:28:34.711 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_keys": 0, 2023-11-27T15:28:34.711 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_commit": 397, 2023-11-27T15:28:34.712 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_split": 48, 2023-11-27T15:28:34.712 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_merge": 2, 2023-11-27T15:28:34.712 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes": 44774, 2023-11-27T15:28:34.712 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_top": 1, 2023-11-27T15:28:34.712 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_bottom": 69, 2023-11-27T15:28:34.712 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pin_tail": 44704, 2023-11-27T15:28:34.712 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pinned": 44772, 2023-11-27T15:28:34.712 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_expired": 10401, 2023-11-27T15:28:34.713 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_with_caps": 1, 2023-11-27T15:28:34.713 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": 1, 2023-11-27T15:28:34.713 INFO:teuthology.orchestra.run.smithi150.stdout: "subtrees": 3, 2023-11-27T15:28:34.713 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse": 64678, 2023-11-27T15:28:34.713 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_hit": 57338, 2023-11-27T15:28:34.713 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_forward": 6546, 2023-11-27T15:28:34.713 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_discover": 13, 2023-11-27T15:28:34.713 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_dir_fetch": 0, 2023-11-27T15:28:34.713 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_remote_ino": 0, 2023-11-27T15:28:34.714 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_lock": 0, 2023-11-27T15:28:34.714 INFO:teuthology.orchestra.run.smithi150.stdout: "load_cent": 1000, 2023-11-27T15:28:34.714 INFO:teuthology.orchestra.run.smithi150.stdout: "q": 1, 2023-11-27T15:28:34.714 INFO:teuthology.orchestra.run.smithi150.stdout: "exported": 341, 2023-11-27T15:28:34.714 INFO:teuthology.orchestra.run.smithi150.stdout: "exported_inodes": 13630, 2023-11-27T15:28:34.714 INFO:teuthology.orchestra.run.smithi150.stdout: "imported": 397, 2023-11-27T15:28:34.714 INFO:teuthology.orchestra.run.smithi150.stdout: "imported_inodes": 0, 2023-11-27T15:28:34.714 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_dir_fetch": 0, 2023-11-27T15:28:34.715 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_backtrace_fetch": 19561, 2023-11-27T15:28:34.715 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_peer_discover": 0, 2023-11-27T15:28:34.715 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rfiles": 4, 2023-11-27T15:28:34.715 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rbytes": 246, 2023-11-27T15:28:34.715 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rsnaps": 6, 2023-11-27T15:28:34.715 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_fetch": 19561, 2023-11-27T15:28:34.715 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_set_tag": 0, 2023-11-27T15:28:34.715 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_repaired": 0, 2023-11-27T15:28:34.716 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_inotable_repaired": 0, 2023-11-27T15:28:34.716 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_inodes": 2, 2023-11-27T15:28:34.716 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_base_inodes": 0, 2023-11-27T15:28:34.716 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dirfrag_rstats": 2, 2023-11-27T15:28:34.716 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_file_inodes": 19559, 2023-11-27T15:28:34.716 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_inode_file_caps": 6, 2023-11-27T15:28:34.716 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_revoke": 16, 2023-11-27T15:28:34.716 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_grant": 21734, 2023-11-27T15:28:34.717 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_trunc": 0, 2023-11-27T15:28:34.717 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-27T15:28:34.717 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-27T15:28:34.717 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps": 32426, 2023-11-27T15:28:34.717 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps_dirty": 32413, 2023-11-27T15:28:34.717 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_cap_release": 291, 2023-11-27T15:28:34.717 INFO:teuthology.orchestra.run.smithi150.stdout: "process_request_cap_release": 34169 2023-11-27T15:28:34.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.718 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache": { 2023-11-27T15:28:34.718 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update": 1009, 2023-11-27T15:28:34.718 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update_receipt": 3081, 2023-11-27T15:28:34.718 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_try_discover": 1545, 2023-11-27T15:28:34.718 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_send_discover": 1564, 2023-11-27T15:28:34.718 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_handle_discover": 1009, 2023-11-27T15:28:34.718 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays": 2, 2023-11-27T15:28:34.719 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_delayed": 0, 2023-11-27T15:28:34.719 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_enqueuing": 0, 2023-11-27T15:28:34.719 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_created": 15889, 2023-11-27T15:28:34.719 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_enqueued": 15887, 2023-11-27T15:28:34.719 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_reintegrated": 0, 2023-11-27T15:28:34.719 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_migrated": 0, 2023-11-27T15:28:34.719 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_processing": 0, 2023-11-27T15:28:34.719 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_enqueued": 0, 2023-11-27T15:28:34.719 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_prioritized": 0, 2023-11-27T15:28:34.720 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_started": 0, 2023-11-27T15:28:34.720 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_completed": 0, 2023-11-27T15:28:34.720 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_enqueue_scrub": 0, 2023-11-27T15:28:34.720 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_exportdir": 341, 2023-11-27T15:28:34.720 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_flush": 0, 2023-11-27T15:28:34.720 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragmentdir": 50, 2023-11-27T15:28:34.720 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragstats": 0, 2023-11-27T15:28:34.720 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_inodestats": 0 2023-11-27T15:28:34.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.721 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log": { 2023-11-27T15:28:34.721 INFO:teuthology.orchestra.run.smithi150.stdout: "evadd": 70885, 2023-11-27T15:28:34.721 INFO:teuthology.orchestra.run.smithi150.stdout: "evex": 0, 2023-11-27T15:28:34.721 INFO:teuthology.orchestra.run.smithi150.stdout: "evtrm": 0, 2023-11-27T15:28:34.721 INFO:teuthology.orchestra.run.smithi150.stdout: "ev": 70885, 2023-11-27T15:28:34.721 INFO:teuthology.orchestra.run.smithi150.stdout: "evexg": 0, 2023-11-27T15:28:34.721 INFO:teuthology.orchestra.run.smithi150.stdout: "evexd": 0, 2023-11-27T15:28:34.722 INFO:teuthology.orchestra.run.smithi150.stdout: "segadd": 107, 2023-11-27T15:28:34.722 INFO:teuthology.orchestra.run.smithi150.stdout: "segex": 0, 2023-11-27T15:28:34.722 INFO:teuthology.orchestra.run.smithi150.stdout: "segtrm": 0, 2023-11-27T15:28:34.722 INFO:teuthology.orchestra.run.smithi150.stdout: "seg": 107, 2023-11-27T15:28:34.722 INFO:teuthology.orchestra.run.smithi150.stdout: "segexg": 0, 2023-11-27T15:28:34.722 INFO:teuthology.orchestra.run.smithi150.stdout: "segexd": 0, 2023-11-27T15:28:34.722 INFO:teuthology.orchestra.run.smithi150.stdout: "expos": 4194304, 2023-11-27T15:28:34.722 INFO:teuthology.orchestra.run.smithi150.stdout: "wrpos": 246202356, 2023-11-27T15:28:34.723 INFO:teuthology.orchestra.run.smithi150.stdout: "rdpos": 0, 2023-11-27T15:28:34.723 INFO:teuthology.orchestra.run.smithi150.stdout: "jlat": { 2023-11-27T15:28:34.723 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1096, 2023-11-27T15:28:34.723 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 86.101029195, 2023-11-27T15:28:34.723 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.078559333 2023-11-27T15:28:34.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.723 INFO:teuthology.orchestra.run.smithi150.stdout: "replayed": 0 2023-11-27T15:28:34.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.724 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_mem": { 2023-11-27T15:28:34.724 INFO:teuthology.orchestra.run.smithi150.stdout: "ino": 24915, 2023-11-27T15:28:34.724 INFO:teuthology.orchestra.run.smithi150.stdout: "ino+": 50936, 2023-11-27T15:28:34.724 INFO:teuthology.orchestra.run.smithi150.stdout: "ino-": 26021, 2023-11-27T15:28:34.724 INFO:teuthology.orchestra.run.smithi150.stdout: "dir": 1039, 2023-11-27T15:28:34.724 INFO:teuthology.orchestra.run.smithi150.stdout: "dir+": 2662, 2023-11-27T15:28:34.724 INFO:teuthology.orchestra.run.smithi150.stdout: "dir-": 1623, 2023-11-27T15:28:34.724 INFO:teuthology.orchestra.run.smithi150.stdout: "dn": 44774, 2023-11-27T15:28:34.725 INFO:teuthology.orchestra.run.smithi150.stdout: "dn+": 71062, 2023-11-27T15:28:34.725 INFO:teuthology.orchestra.run.smithi150.stdout: "dn-": 26288, 2023-11-27T15:28:34.725 INFO:teuthology.orchestra.run.smithi150.stdout: "cap": 1, 2023-11-27T15:28:34.725 INFO:teuthology.orchestra.run.smithi150.stdout: "cap+": 31187, 2023-11-27T15:28:34.725 INFO:teuthology.orchestra.run.smithi150.stdout: "cap-": 31186, 2023-11-27T15:28:34.725 INFO:teuthology.orchestra.run.smithi150.stdout: "rss": 241032, 2023-11-27T15:28:34.725 INFO:teuthology.orchestra.run.smithi150.stdout: "heap": 198964 2023-11-27T15:28:34.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.726 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_server": { 2023-11-27T15:28:34.726 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_client_request": 49923, 2023-11-27T15:28:34.726 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_server_request": 49, 2023-11-27T15:28:34.726 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_request": 46441, 2023-11-27T15:28:34.726 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_session": 185, 2023-11-27T15:28:34.726 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_peer_request": 113, 2023-11-27T15:28:34.726 INFO:teuthology.orchestra.run.smithi150.stdout: "req_create_latency": { 2023-11-27T15:28:34.726 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 5559, 2023-11-27T15:28:34.727 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 9.854317468, 2023-11-27T15:28:34.727 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001772678 2023-11-27T15:28:34.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.727 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getattr_latency": { 2023-11-27T15:28:34.727 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 136, 2023-11-27T15:28:34.727 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.216254780, 2023-11-27T15:28:34.727 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001590108 2023-11-27T15:28:34.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.728 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getfilelock_latency": { 2023-11-27T15:28:34.728 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:34.728 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:34.728 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:34.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.728 INFO:teuthology.orchestra.run.smithi150.stdout: "req_link_latency": { 2023-11-27T15:28:34.728 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:34.728 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:34.728 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:34.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.729 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookup_latency": { 2023-11-27T15:28:34.729 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 15771, 2023-11-27T15:28:34.729 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 28.797836072, 2023-11-27T15:28:34.729 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001825999 2023-11-27T15:28:34.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.729 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookuphash_latency": { 2023-11-27T15:28:34.729 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:34.730 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:34.730 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:34.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.730 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupino_latency": { 2023-11-27T15:28:34.730 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1, 2023-11-27T15:28:34.730 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000212922, 2023-11-27T15:28:34.730 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000212922 2023-11-27T15:28:34.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.731 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupname_latency": { 2023-11-27T15:28:34.731 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:34.731 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:34.731 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:34.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.731 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupparent_latency": { 2023-11-27T15:28:34.731 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:34.731 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:34.732 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:34.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.732 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupsnap_latency": { 2023-11-27T15:28:34.732 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:34.732 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:34.732 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:34.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.732 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lssnap_latency": { 2023-11-27T15:28:34.733 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:34.733 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:34.733 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:34.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.733 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mkdir_latency": { 2023-11-27T15:28:34.733 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:34.733 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:34.733 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:34.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.734 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mknod_latency": { 2023-11-27T15:28:34.734 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:34.734 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:34.734 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:34.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.734 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mksnap_latency": { 2023-11-27T15:28:34.734 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:34.735 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:34.735 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:34.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.735 INFO:teuthology.orchestra.run.smithi150.stdout: "req_open_latency": { 2023-11-27T15:28:34.735 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:34.735 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:34.735 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:34.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.736 INFO:teuthology.orchestra.run.smithi150.stdout: "req_readdir_latency": { 2023-11-27T15:28:34.736 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2539, 2023-11-27T15:28:34.736 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 31.507645926, 2023-11-27T15:28:34.736 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.012409470 2023-11-27T15:28:34.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.736 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rename_latency": { 2023-11-27T15:28:34.736 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:34.736 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:34.737 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:34.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.737 INFO:teuthology.orchestra.run.smithi150.stdout: "req_renamesnap_latency": { 2023-11-27T15:28:34.737 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:34.737 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:34.737 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:34.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.737 INFO:teuthology.orchestra.run.smithi150.stdout: "req_snapdiff_latency": { 2023-11-27T15:28:34.738 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:34.738 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:34.738 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:34.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.738 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmdir_latency": { 2023-11-27T15:28:34.738 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2, 2023-11-27T15:28:34.738 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.111922426, 2023-11-27T15:28:34.738 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.055961213 2023-11-27T15:28:34.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.739 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmsnap_latency": { 2023-11-27T15:28:34.739 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:34.739 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:34.739 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:34.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.739 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmxattr_latency": { 2023-11-27T15:28:34.739 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:34.739 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:34.740 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:34.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.740 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setattr_latency": { 2023-11-27T15:28:34.740 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:34.740 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:34.740 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:34.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.740 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setdirlayout_latency": { 2023-11-27T15:28:34.740 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:34.741 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:34.741 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:34.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.741 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setfilelock_latency": { 2023-11-27T15:28:34.741 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:34.741 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:34.741 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:34.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.741 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setlayout_latency": { 2023-11-27T15:28:34.741 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:34.742 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:34.742 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:34.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.742 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setxattr_latency": { 2023-11-27T15:28:34.742 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:34.742 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:34.742 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:34.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.743 INFO:teuthology.orchestra.run.smithi150.stdout: "req_symlink_latency": { 2023-11-27T15:28:34.743 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:34.743 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:34.743 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:34.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.743 INFO:teuthology.orchestra.run.smithi150.stdout: "req_unlink_latency": { 2023-11-27T15:28:34.743 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 15887, 2023-11-27T15:28:34.743 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 76.772242015, 2023-11-27T15:28:34.744 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.004832393 2023-11-27T15:28:34.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.744 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_revoke_eviction": 0, 2023-11-27T15:28:34.744 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_acquisition_throttle": 0, 2023-11-27T15:28:34.744 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getvxattr_latency": { 2023-11-27T15:28:34.744 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:34.744 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:34.744 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:34.744 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-27T15:28:34.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.745 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_sessions": { 2023-11-27T15:28:34.745 INFO:teuthology.orchestra.run.smithi150.stdout: "session_count": 1, 2023-11-27T15:28:34.745 INFO:teuthology.orchestra.run.smithi150.stdout: "session_add": 2, 2023-11-27T15:28:34.745 INFO:teuthology.orchestra.run.smithi150.stdout: "session_remove": 1, 2023-11-27T15:28:34.745 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_open": 1, 2023-11-27T15:28:34.745 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_stale": 0, 2023-11-27T15:28:34.745 INFO:teuthology.orchestra.run.smithi150.stdout: "total_load": 21452, 2023-11-27T15:28:34.746 INFO:teuthology.orchestra.run.smithi150.stdout: "average_load": 21452, 2023-11-27T15:28:34.746 INFO:teuthology.orchestra.run.smithi150.stdout: "avg_session_uptime": 654 2023-11-27T15:28:34.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.746 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2023-11-27T15:28:34.746 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:28:34.746 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2023-11-27T15:28:34.746 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:28:34.746 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:28:34.747 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:28:34.747 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:28:34.747 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:28:34.747 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:28:34.747 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:28:34.747 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:28:34.747 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:28:34.747 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:28:34.748 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:28:34.748 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:28:34.748 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:28:34.748 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2023-11-27T15:28:34.748 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:28:34.748 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2023-11-27T15:28:34.748 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:28:34.748 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:28:34.748 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:28:34.749 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:28:34.749 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:28:34.749 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:28:34.749 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:28:34.749 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2023-11-27T15:28:34.749 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:28:34.749 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:28:34.749 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:28:34.750 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2023-11-27T15:28:34.750 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2023-11-27T15:28:34.750 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2023-11-27T15:28:34.750 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:28:34.750 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:28:34.750 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:28:34.750 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:28:34.750 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 1051882, 2023-11-27T15:28:34.751 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 22487, 2023-11-27T15:28:34.751 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 88, 2023-11-27T15:28:34.751 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 1, 2023-11-27T15:28:34.751 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2023-11-27T15:28:34.751 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2023-11-27T15:28:34.751 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:28:34.751 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2023-11-27T15:28:34.751 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:28:34.752 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2023-11-27T15:28:34.752 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 8224, 2023-11-27T15:28:34.752 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 171, 2023-11-27T15:28:34.752 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:28:34.752 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:28:34.752 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2023-11-27T15:28:34.752 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2023-11-27T15:28:34.752 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 88866098, 2023-11-27T15:28:34.752 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 848774, 2023-11-27T15:28:34.753 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2023-11-27T15:28:34.753 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2023-11-27T15:28:34.753 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2023-11-27T15:28:34.753 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2023-11-27T15:28:34.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.753 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter": { 2023-11-27T15:28:34.753 INFO:teuthology.orchestra.run.smithi150.stdout: "op_active": 0, 2023-11-27T15:28:34.753 INFO:teuthology.orchestra.run.smithi150.stdout: "op_laggy": 0, 2023-11-27T15:28:34.754 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send": 37816, 2023-11-27T15:28:34.754 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send_bytes": 249144743, 2023-11-27T15:28:34.754 INFO:teuthology.orchestra.run.smithi150.stdout: "op_resend": 21, 2023-11-27T15:28:34.754 INFO:teuthology.orchestra.run.smithi150.stdout: "op_reply": 37816, 2023-11-27T15:28:34.754 INFO:teuthology.orchestra.run.smithi150.stdout: "op_latency": { 2023-11-27T15:28:34.754 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 37816, 2023-11-27T15:28:34.754 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 63.052742946, 2023-11-27T15:28:34.754 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001667356 2023-11-27T15:28:34.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.755 INFO:teuthology.orchestra.run.smithi150.stdout: "op_inflight": 0, 2023-11-27T15:28:34.755 INFO:teuthology.orchestra.run.smithi150.stdout: "oplen_avg": { 2023-11-27T15:28:34.755 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 37816, 2023-11-27T15:28:34.755 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 38276 2023-11-27T15:28:34.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.755 INFO:teuthology.orchestra.run.smithi150.stdout: "op": 37816, 2023-11-27T15:28:34.755 INFO:teuthology.orchestra.run.smithi150.stdout: "op_r": 19845, 2023-11-27T15:28:34.756 INFO:teuthology.orchestra.run.smithi150.stdout: "op_w": 17971, 2023-11-27T15:28:34.756 INFO:teuthology.orchestra.run.smithi150.stdout: "op_rmw": 0, 2023-11-27T15:28:34.756 INFO:teuthology.orchestra.run.smithi150.stdout: "op_pg": 0, 2023-11-27T15:28:34.756 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_stat": 0, 2023-11-27T15:28:34.756 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_create": 10, 2023-11-27T15:28:34.756 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_read": 284, 2023-11-27T15:28:34.756 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_write": 1381, 2023-11-27T15:28:34.756 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writefull": 140, 2023-11-27T15:28:34.757 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writesame": 0, 2023-11-27T15:28:34.757 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_append": 0, 2023-11-27T15:28:34.757 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_zero": 0, 2023-11-27T15:28:34.757 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_truncate": 0, 2023-11-27T15:28:34.757 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_delete": 16010, 2023-11-27T15:28:34.757 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_mapext": 0, 2023-11-27T15:28:34.757 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_sparse_read": 0, 2023-11-27T15:28:34.757 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_clonerange": 0, 2023-11-27T15:28:34.758 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_getxattr": 19561, 2023-11-27T15:28:34.758 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_setxattr": 20, 2023-11-27T15:28:34.758 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_cmpxattr": 0, 2023-11-27T15:28:34.758 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_rmxattr": 0, 2023-11-27T15:28:34.758 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_resetxattrs": 0, 2023-11-27T15:28:34.758 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_call": 0, 2023-11-27T15:28:34.758 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_watch": 0, 2023-11-27T15:28:34.758 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_notify": 0, 2023-11-27T15:28:34.759 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_src_cmpxattr": 0, 2023-11-27T15:28:34.759 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls": 0, 2023-11-27T15:28:34.759 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls_filter": 0, 2023-11-27T15:28:34.759 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_other": 0, 2023-11-27T15:28:34.759 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_active": 0, 2023-11-27T15:28:34.759 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_send": 0, 2023-11-27T15:28:34.759 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_resend": 0, 2023-11-27T15:28:34.759 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_ping": 0, 2023-11-27T15:28:34.760 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_active": 0, 2023-11-27T15:28:34.760 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_send": 0, 2023-11-27T15:28:34.760 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_resend": 0, 2023-11-27T15:28:34.760 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_active": 0, 2023-11-27T15:28:34.760 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_send": 0, 2023-11-27T15:28:34.760 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_resend": 0, 2023-11-27T15:28:34.760 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_active": 0, 2023-11-27T15:28:34.760 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_send": 0, 2023-11-27T15:28:34.761 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_resend": 0, 2023-11-27T15:28:34.761 INFO:teuthology.orchestra.run.smithi150.stdout: "command_active": 0, 2023-11-27T15:28:34.761 INFO:teuthology.orchestra.run.smithi150.stdout: "command_send": 0, 2023-11-27T15:28:34.761 INFO:teuthology.orchestra.run.smithi150.stdout: "command_resend": 0, 2023-11-27T15:28:34.761 INFO:teuthology.orchestra.run.smithi150.stdout: "map_epoch": 96, 2023-11-27T15:28:34.761 INFO:teuthology.orchestra.run.smithi150.stdout: "map_full": 0, 2023-11-27T15:28:34.761 INFO:teuthology.orchestra.run.smithi150.stdout: "map_inc": 13, 2023-11-27T15:28:34.761 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_sessions": 12, 2023-11-27T15:28:34.762 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_open": 12, 2023-11-27T15:28:34.762 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_close": 0, 2023-11-27T15:28:34.762 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_laggy": 0, 2023-11-27T15:28:34.762 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_wr": 844, 2023-11-27T15:28:34.762 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_rd": 0, 2023-11-27T15:28:34.762 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_del": 26 2023-11-27T15:28:34.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.762 INFO:teuthology.orchestra.run.smithi150.stdout: "oft": { 2023-11-27T15:28:34.763 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_objs": 1, 2023-11-27T15:28:34.763 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_kv_pairs": 0, 2023-11-27T15:28:34.763 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_updates": 0, 2023-11-27T15:28:34.763 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_removes": 0 2023-11-27T15:28:34.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.763 INFO:teuthology.orchestra.run.smithi150.stdout: "osd-slow-ops": { 2023-11-27T15:28:34.763 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_ops_count": 46882 2023-11-27T15:28:34.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.764 INFO:teuthology.orchestra.run.smithi150.stdout: "purge_queue": { 2023-11-27T15:28:34.764 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops": 0, 2023-11-27T15:28:34.764 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops_high_water": 10, 2023-11-27T15:28:34.764 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing": 0, 2023-11-27T15:28:34.764 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_high_water": 10, 2023-11-27T15:28:34.764 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed": 15887, 2023-11-27T15:28:34.764 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_item_in_journal": 0 2023-11-27T15:28:34.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.765 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:28:34.765 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2023-11-27T15:28:34.765 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2023-11-27T15:28:34.765 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2023-11-27T15:28:34.765 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 180974, 2023-11-27T15:28:34.765 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 105067531, 2023-11-27T15:28:34.765 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:34.765 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 180974, 2023-11-27T15:28:34.766 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2023-11-27T15:28:34.766 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2023-11-27T15:28:34.766 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 180974, 2023-11-27T15:28:34.766 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 105067531, 2023-11-27T15:28:34.766 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2023-11-27T15:28:34.766 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:34.766 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:34.766 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:34.766 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-27T15:28:34.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.767 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_bytes": { 2023-11-27T15:28:34.767 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2023-11-27T15:28:34.767 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2023-11-27T15:28:34.767 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2023-11-27T15:28:34.767 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2023-11-27T15:28:34.767 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2023-11-27T15:28:34.767 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:34.768 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2023-11-27T15:28:34.768 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 37816, 2023-11-27T15:28:34.768 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 250940794, 2023-11-27T15:28:34.768 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 37816, 2023-11-27T15:28:34.768 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 250940794, 2023-11-27T15:28:34.768 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2023-11-27T15:28:34.768 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:34.768 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:34.769 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:34.769 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-27T15:28:34.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.769 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_ops": { 2023-11-27T15:28:34.769 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2023-11-27T15:28:34.769 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 1024, 2023-11-27T15:28:34.769 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2023-11-27T15:28:34.769 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2023-11-27T15:28:34.770 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2023-11-27T15:28:34.770 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:34.770 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2023-11-27T15:28:34.770 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 37816, 2023-11-27T15:28:34.770 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 37816, 2023-11-27T15:28:34.770 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 37816, 2023-11-27T15:28:34.770 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 37816, 2023-11-27T15:28:34.770 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2023-11-27T15:28:34.771 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:34.771 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:34.771 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:34.771 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-27T15:28:34.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.771 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle": { 2023-11-27T15:28:34.771 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2023-11-27T15:28:34.771 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2023-11-27T15:28:34.771 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2023-11-27T15:28:34.772 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 15887, 2023-11-27T15:28:34.772 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 1796051, 2023-11-27T15:28:34.772 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:34.772 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 15887, 2023-11-27T15:28:34.772 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2023-11-27T15:28:34.772 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2023-11-27T15:28:34.772 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 284, 2023-11-27T15:28:34.772 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 1796051, 2023-11-27T15:28:34.773 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2023-11-27T15:28:34.773 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:34.773 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:34.773 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:34.773 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-27T15:28:34.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:34.773 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle-0x560599ee20a0": { 2023-11-27T15:28:34.773 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2023-11-27T15:28:34.774 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2023-11-27T15:28:34.774 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2023-11-27T15:28:34.774 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 70885, 2023-11-27T15:28:34.774 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 242008052, 2023-11-27T15:28:34.774 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:34.774 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 70885, 2023-11-27T15:28:34.774 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2023-11-27T15:28:34.774 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2023-11-27T15:28:34.774 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 1096, 2023-11-27T15:28:34.775 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 242008052, 2023-11-27T15:28:34.775 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2023-11-27T15:28:34.775 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:34.775 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:34.775 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:34.775 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-27T15:28:34.775 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-27T15:28:34.775 INFO:teuthology.orchestra.run.smithi150.stdout:} 2023-11-27T15:28:35.142 INFO:tasks.check_counter:Daemon mds.c mds.exported=341 2023-11-27T15:28:35.143 INFO:tasks.check_counter:Daemon mds.c mds.imported=397 2023-11-27T15:28:35.143 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=1009 2023-11-27T15:28:35.143 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=3081 2023-11-27T15:28:35.143 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2023-11-27T15:28:35.143 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2023-11-27T15:28:35.143 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2023-11-27T15:28:35.143 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=48 2023-11-27T15:28:35.143 DEBUG:tasks.check_counter:Getting stats from f 2023-11-27T15:28:35.144 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph daemon mds.f perf dump 2023-11-27T15:28:35.783 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:28:36.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:36 smithi150 ceph-mon[100559]: pgmap v333: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:28:36.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:36 smithi102 ceph-mon[100296]: pgmap v333: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:28:36.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:36 smithi052 ceph-mon[99056]: pgmap v333: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:28:37.001 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2023-11-27T15:28:37.001 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:28:37.002 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 542, 2023-11-27T15:28:37.002 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 542, 2023-11-27T15:28:37.002 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 88606041, 2023-11-27T15:28:37.002 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 167830, 2023-11-27T15:28:37.002 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 8, 2023-11-27T15:28:37.002 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 4, 2023-11-27T15:28:37.002 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.084334548, 2023-11-27T15:28:37.003 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.011374551, 2023-11-27T15:28:37.003 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.053131476, 2023-11-27T15:28:37.003 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.014897086, 2023-11-27T15:28:37.003 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:37.003 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 541, 2023-11-27T15:28:37.003 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.018560520, 2023-11-27T15:28:37.003 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000034307 2023-11-27T15:28:37.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.004 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:37.004 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.004 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.004 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.004 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-11-27T15:28:37.004 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 448 2023-11-27T15:28:37.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.005 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:28:37.005 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 728, 2023-11-27T15:28:37.005 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 706, 2023-11-27T15:28:37.005 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 94034608, 2023-11-27T15:28:37.005 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 270403, 2023-11-27T15:28:37.005 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 5, 2023-11-27T15:28:37.005 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 4, 2023-11-27T15:28:37.006 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.119901969, 2023-11-27T15:28:37.006 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.023173309, 2023-11-27T15:28:37.006 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.082541474, 2023-11-27T15:28:37.006 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.020704311, 2023-11-27T15:28:37.006 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:37.006 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 705, 2023-11-27T15:28:37.006 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.023201837, 2023-11-27T15:28:37.007 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000032910 2023-11-27T15:28:37.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.007 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:37.007 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.007 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.007 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.007 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 1623360, 2023-11-27T15:28:37.008 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 120496 2023-11-27T15:28:37.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.008 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:28:37.008 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 1620, 2023-11-27T15:28:37.008 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 1776, 2023-11-27T15:28:37.008 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 62504675, 2023-11-27T15:28:37.008 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 930030, 2023-11-27T15:28:37.009 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 8, 2023-11-27T15:28:37.009 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 6, 2023-11-27T15:28:37.009 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.181996271, 2023-11-27T15:28:37.009 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.049390110, 2023-11-27T15:28:37.009 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.062413971, 2023-11-27T15:28:37.009 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.054691381, 2023-11-27T15:28:37.009 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:37.009 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1773, 2023-11-27T15:28:37.010 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.076662041, 2023-11-27T15:28:37.010 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000043238 2023-11-27T15:28:37.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.010 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:37.010 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.010 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.010 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.010 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 320, 2023-11-27T15:28:37.011 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 426832 2023-11-27T15:28:37.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.011 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2023-11-27T15:28:37.011 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 1, 2023-11-27T15:28:37.011 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2023-11-27T15:28:37.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.011 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-MDSRank": { 2023-11-27T15:28:37.011 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2023-11-27T15:28:37.012 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2023-11-27T15:28:37.012 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 4170, 2023-11-27T15:28:37.012 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.123641447, 2023-11-27T15:28:37.012 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000029650 2023-11-27T15:28:37.012 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-27T15:28:37.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.012 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-PurgeQueue": { 2023-11-27T15:28:37.012 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2023-11-27T15:28:37.013 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2023-11-27T15:28:37.013 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.013 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.013 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.013 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-27T15:28:37.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.013 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": { 2023-11-27T15:28:37.013 INFO:teuthology.orchestra.run.smithi150.stdout: "request": 0, 2023-11-27T15:28:37.014 INFO:teuthology.orchestra.run.smithi150.stdout: "reply": 0, 2023-11-27T15:28:37.014 INFO:teuthology.orchestra.run.smithi150.stdout: "reply_latency": { 2023-11-27T15:28:37.014 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.014 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.014 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.014 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_reply": 0, 2023-11-27T15:28:37.014 INFO:teuthology.orchestra.run.smithi150.stdout: "forward": 0, 2023-11-27T15:28:37.015 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_complete": 0, 2023-11-27T15:28:37.015 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_keys": 0, 2023-11-27T15:28:37.015 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_commit": 0, 2023-11-27T15:28:37.015 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_split": 0, 2023-11-27T15:28:37.015 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_merge": 0, 2023-11-27T15:28:37.015 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes": 60794, 2023-11-27T15:28:37.015 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_top": 0, 2023-11-27T15:28:37.015 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_bottom": 0, 2023-11-27T15:28:37.015 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pin_tail": 60794, 2023-11-27T15:28:37.016 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pinned": 60794, 2023-11-27T15:28:37.016 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_expired": 0, 2023-11-27T15:28:37.016 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_with_caps": 0, 2023-11-27T15:28:37.016 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": 0, 2023-11-27T15:28:37.016 INFO:teuthology.orchestra.run.smithi150.stdout: "subtrees": 1, 2023-11-27T15:28:37.016 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse": 0, 2023-11-27T15:28:37.016 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_hit": 0, 2023-11-27T15:28:37.016 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_forward": 0, 2023-11-27T15:28:37.017 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_discover": 0, 2023-11-27T15:28:37.017 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_dir_fetch": 0, 2023-11-27T15:28:37.017 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_remote_ino": 0, 2023-11-27T15:28:37.017 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_lock": 0, 2023-11-27T15:28:37.017 INFO:teuthology.orchestra.run.smithi150.stdout: "load_cent": 0, 2023-11-27T15:28:37.017 INFO:teuthology.orchestra.run.smithi150.stdout: "q": 0, 2023-11-27T15:28:37.017 INFO:teuthology.orchestra.run.smithi150.stdout: "exported": 0, 2023-11-27T15:28:37.017 INFO:teuthology.orchestra.run.smithi150.stdout: "exported_inodes": 0, 2023-11-27T15:28:37.018 INFO:teuthology.orchestra.run.smithi150.stdout: "imported": 0, 2023-11-27T15:28:37.018 INFO:teuthology.orchestra.run.smithi150.stdout: "imported_inodes": 0, 2023-11-27T15:28:37.018 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_dir_fetch": 0, 2023-11-27T15:28:37.018 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_backtrace_fetch": 0, 2023-11-27T15:28:37.018 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_peer_discover": 0, 2023-11-27T15:28:37.018 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rfiles": 0, 2023-11-27T15:28:37.018 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rbytes": 0, 2023-11-27T15:28:37.018 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rsnaps": 0, 2023-11-27T15:28:37.019 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_fetch": 0, 2023-11-27T15:28:37.019 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_set_tag": 0, 2023-11-27T15:28:37.019 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_repaired": 0, 2023-11-27T15:28:37.019 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_inotable_repaired": 0, 2023-11-27T15:28:37.019 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_inodes": 0, 2023-11-27T15:28:37.019 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_base_inodes": 0, 2023-11-27T15:28:37.019 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dirfrag_rstats": 0, 2023-11-27T15:28:37.019 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_file_inodes": 0, 2023-11-27T15:28:37.020 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_inode_file_caps": 0, 2023-11-27T15:28:37.020 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_revoke": 0, 2023-11-27T15:28:37.020 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_grant": 0, 2023-11-27T15:28:37.020 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_trunc": 0, 2023-11-27T15:28:37.020 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-27T15:28:37.020 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-27T15:28:37.020 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps": 0, 2023-11-27T15:28:37.020 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps_dirty": 0, 2023-11-27T15:28:37.020 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_cap_release": 0, 2023-11-27T15:28:37.021 INFO:teuthology.orchestra.run.smithi150.stdout: "process_request_cap_release": 0 2023-11-27T15:28:37.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.021 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache": { 2023-11-27T15:28:37.021 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update": 0, 2023-11-27T15:28:37.021 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update_receipt": 0, 2023-11-27T15:28:37.021 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_try_discover": 0, 2023-11-27T15:28:37.021 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_send_discover": 0, 2023-11-27T15:28:37.021 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_handle_discover": 0, 2023-11-27T15:28:37.022 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays": 0, 2023-11-27T15:28:37.022 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_delayed": 0, 2023-11-27T15:28:37.022 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_enqueuing": 0, 2023-11-27T15:28:37.022 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_created": 0, 2023-11-27T15:28:37.022 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_enqueued": 0, 2023-11-27T15:28:37.022 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_reintegrated": 0, 2023-11-27T15:28:37.022 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_migrated": 0, 2023-11-27T15:28:37.022 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_processing": 0, 2023-11-27T15:28:37.023 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_enqueued": 0, 2023-11-27T15:28:37.023 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_prioritized": 0, 2023-11-27T15:28:37.023 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_started": 0, 2023-11-27T15:28:37.023 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_completed": 0, 2023-11-27T15:28:37.023 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_enqueue_scrub": 0, 2023-11-27T15:28:37.023 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_exportdir": 0, 2023-11-27T15:28:37.023 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_flush": 0, 2023-11-27T15:28:37.023 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragmentdir": 0, 2023-11-27T15:28:37.024 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragstats": 0, 2023-11-27T15:28:37.024 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_inodestats": 0 2023-11-27T15:28:37.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.024 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log": { 2023-11-27T15:28:37.024 INFO:teuthology.orchestra.run.smithi150.stdout: "evadd": 0, 2023-11-27T15:28:37.024 INFO:teuthology.orchestra.run.smithi150.stdout: "evex": 0, 2023-11-27T15:28:37.024 INFO:teuthology.orchestra.run.smithi150.stdout: "evtrm": 0, 2023-11-27T15:28:37.024 INFO:teuthology.orchestra.run.smithi150.stdout: "ev": 0, 2023-11-27T15:28:37.025 INFO:teuthology.orchestra.run.smithi150.stdout: "evexg": 0, 2023-11-27T15:28:37.025 INFO:teuthology.orchestra.run.smithi150.stdout: "evexd": 0, 2023-11-27T15:28:37.025 INFO:teuthology.orchestra.run.smithi150.stdout: "segadd": 0, 2023-11-27T15:28:37.025 INFO:teuthology.orchestra.run.smithi150.stdout: "segex": 0, 2023-11-27T15:28:37.025 INFO:teuthology.orchestra.run.smithi150.stdout: "segtrm": 0, 2023-11-27T15:28:37.025 INFO:teuthology.orchestra.run.smithi150.stdout: "seg": 107, 2023-11-27T15:28:37.025 INFO:teuthology.orchestra.run.smithi150.stdout: "segexg": 0, 2023-11-27T15:28:37.025 INFO:teuthology.orchestra.run.smithi150.stdout: "segexd": 0, 2023-11-27T15:28:37.026 INFO:teuthology.orchestra.run.smithi150.stdout: "expos": 4194304, 2023-11-27T15:28:37.026 INFO:teuthology.orchestra.run.smithi150.stdout: "wrpos": 0, 2023-11-27T15:28:37.026 INFO:teuthology.orchestra.run.smithi150.stdout: "rdpos": 246023307, 2023-11-27T15:28:37.026 INFO:teuthology.orchestra.run.smithi150.stdout: "jlat": { 2023-11-27T15:28:37.026 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.026 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.026 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.027 INFO:teuthology.orchestra.run.smithi150.stdout: "replayed": 70885 2023-11-27T15:28:37.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.027 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_mem": { 2023-11-27T15:28:37.027 INFO:teuthology.orchestra.run.smithi150.stdout: "ino": 24906, 2023-11-27T15:28:37.027 INFO:teuthology.orchestra.run.smithi150.stdout: "ino+": 50913, 2023-11-27T15:28:37.027 INFO:teuthology.orchestra.run.smithi150.stdout: "ino-": 26007, 2023-11-27T15:28:37.027 INFO:teuthology.orchestra.run.smithi150.stdout: "dir": 1031, 2023-11-27T15:28:37.027 INFO:teuthology.orchestra.run.smithi150.stdout: "dir+": 1113, 2023-11-27T15:28:37.028 INFO:teuthology.orchestra.run.smithi150.stdout: "dir-": 82, 2023-11-27T15:28:37.028 INFO:teuthology.orchestra.run.smithi150.stdout: "dn": 60794, 2023-11-27T15:28:37.028 INFO:teuthology.orchestra.run.smithi150.stdout: "dn+": 74435, 2023-11-27T15:28:37.028 INFO:teuthology.orchestra.run.smithi150.stdout: "dn-": 13641, 2023-11-27T15:28:37.028 INFO:teuthology.orchestra.run.smithi150.stdout: "cap": 0, 2023-11-27T15:28:37.028 INFO:teuthology.orchestra.run.smithi150.stdout: "cap+": 0, 2023-11-27T15:28:37.028 INFO:teuthology.orchestra.run.smithi150.stdout: "cap-": 0, 2023-11-27T15:28:37.028 INFO:teuthology.orchestra.run.smithi150.stdout: "rss": 192756, 2023-11-27T15:28:37.028 INFO:teuthology.orchestra.run.smithi150.stdout: "heap": 207156 2023-11-27T15:28:37.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.029 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_server": { 2023-11-27T15:28:37.029 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_client_request": 0, 2023-11-27T15:28:37.029 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_server_request": 0, 2023-11-27T15:28:37.029 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_request": 0, 2023-11-27T15:28:37.029 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_session": 0, 2023-11-27T15:28:37.029 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_peer_request": 0, 2023-11-27T15:28:37.029 INFO:teuthology.orchestra.run.smithi150.stdout: "req_create_latency": { 2023-11-27T15:28:37.030 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.030 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.030 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.030 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getattr_latency": { 2023-11-27T15:28:37.030 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.030 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.030 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.031 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getfilelock_latency": { 2023-11-27T15:28:37.031 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.031 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.031 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.031 INFO:teuthology.orchestra.run.smithi150.stdout: "req_link_latency": { 2023-11-27T15:28:37.031 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.031 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.032 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.032 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookup_latency": { 2023-11-27T15:28:37.032 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.032 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.032 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.033 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookuphash_latency": { 2023-11-27T15:28:37.033 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.033 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.033 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.033 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupino_latency": { 2023-11-27T15:28:37.033 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.033 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.034 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.034 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupname_latency": { 2023-11-27T15:28:37.034 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.034 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.034 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.034 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupparent_latency": { 2023-11-27T15:28:37.035 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.035 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.035 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.035 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupsnap_latency": { 2023-11-27T15:28:37.035 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.035 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.035 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.036 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lssnap_latency": { 2023-11-27T15:28:37.036 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.036 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.036 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.036 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mkdir_latency": { 2023-11-27T15:28:37.036 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.037 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.037 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.037 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mknod_latency": { 2023-11-27T15:28:37.037 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.037 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.037 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.038 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mksnap_latency": { 2023-11-27T15:28:37.038 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.038 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.038 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.038 INFO:teuthology.orchestra.run.smithi150.stdout: "req_open_latency": { 2023-11-27T15:28:37.038 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.039 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.039 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.039 INFO:teuthology.orchestra.run.smithi150.stdout: "req_readdir_latency": { 2023-11-27T15:28:37.039 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.039 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.039 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.039 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rename_latency": { 2023-11-27T15:28:37.040 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.040 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.040 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.040 INFO:teuthology.orchestra.run.smithi150.stdout: "req_renamesnap_latency": { 2023-11-27T15:28:37.040 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.040 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.040 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.041 INFO:teuthology.orchestra.run.smithi150.stdout: "req_snapdiff_latency": { 2023-11-27T15:28:37.041 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.041 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.041 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.041 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmdir_latency": { 2023-11-27T15:28:37.041 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.042 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.042 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.042 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmsnap_latency": { 2023-11-27T15:28:37.042 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.042 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.042 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.043 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmxattr_latency": { 2023-11-27T15:28:37.043 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.043 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.043 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.043 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setattr_latency": { 2023-11-27T15:28:37.043 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.043 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.043 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.044 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setdirlayout_latency": { 2023-11-27T15:28:37.044 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.044 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.044 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.044 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setfilelock_latency": { 2023-11-27T15:28:37.044 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.045 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.045 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.045 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setlayout_latency": { 2023-11-27T15:28:37.045 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.045 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.045 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.046 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setxattr_latency": { 2023-11-27T15:28:37.046 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.046 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.046 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.046 INFO:teuthology.orchestra.run.smithi150.stdout: "req_symlink_latency": { 2023-11-27T15:28:37.046 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.046 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.047 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.047 INFO:teuthology.orchestra.run.smithi150.stdout: "req_unlink_latency": { 2023-11-27T15:28:37.047 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.047 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.047 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.047 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_revoke_eviction": 0, 2023-11-27T15:28:37.048 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_acquisition_throttle": 0, 2023-11-27T15:28:37.048 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getvxattr_latency": { 2023-11-27T15:28:37.048 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.048 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.048 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.048 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-27T15:28:37.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.048 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_sessions": { 2023-11-27T15:28:37.049 INFO:teuthology.orchestra.run.smithi150.stdout: "session_count": 1, 2023-11-27T15:28:37.049 INFO:teuthology.orchestra.run.smithi150.stdout: "session_add": 2, 2023-11-27T15:28:37.049 INFO:teuthology.orchestra.run.smithi150.stdout: "session_remove": 1, 2023-11-27T15:28:37.049 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_open": 1, 2023-11-27T15:28:37.049 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_stale": 0, 2023-11-27T15:28:37.049 INFO:teuthology.orchestra.run.smithi150.stdout: "total_load": 0, 2023-11-27T15:28:37.049 INFO:teuthology.orchestra.run.smithi150.stdout: "average_load": 0, 2023-11-27T15:28:37.049 INFO:teuthology.orchestra.run.smithi150.stdout: "avg_session_uptime": 2127 2023-11-27T15:28:37.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.050 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2023-11-27T15:28:37.050 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:28:37.050 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2023-11-27T15:28:37.050 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:28:37.050 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:28:37.050 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:28:37.050 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:28:37.051 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:28:37.051 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:28:37.051 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:28:37.051 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:28:37.051 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:28:37.051 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:28:37.051 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:28:37.051 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:28:37.052 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:28:37.052 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2023-11-27T15:28:37.052 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:28:37.052 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2023-11-27T15:28:37.052 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:28:37.052 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:28:37.052 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:28:37.052 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:28:37.052 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:28:37.053 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:28:37.053 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:28:37.053 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2023-11-27T15:28:37.053 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:28:37.053 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:28:37.053 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:28:37.053 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2023-11-27T15:28:37.053 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2023-11-27T15:28:37.054 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2023-11-27T15:28:37.054 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:28:37.054 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:28:37.054 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:28:37.054 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:28:37.054 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 1077939, 2023-11-27T15:28:37.054 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 24974, 2023-11-27T15:28:37.054 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2023-11-27T15:28:37.055 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2023-11-27T15:28:37.055 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2023-11-27T15:28:37.055 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2023-11-27T15:28:37.055 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:28:37.055 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2023-11-27T15:28:37.055 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:28:37.055 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2023-11-27T15:28:37.055 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 8224, 2023-11-27T15:28:37.056 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 171, 2023-11-27T15:28:37.056 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:28:37.056 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:28:37.056 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2023-11-27T15:28:37.056 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2023-11-27T15:28:37.056 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 99248860, 2023-11-27T15:28:37.056 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 790260, 2023-11-27T15:28:37.056 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2023-11-27T15:28:37.057 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2023-11-27T15:28:37.057 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2023-11-27T15:28:37.057 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2023-11-27T15:28:37.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.057 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter": { 2023-11-27T15:28:37.057 INFO:teuthology.orchestra.run.smithi150.stdout: "op_active": 0, 2023-11-27T15:28:37.057 INFO:teuthology.orchestra.run.smithi150.stdout: "op_laggy": 0, 2023-11-27T15:28:37.057 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send": 2638, 2023-11-27T15:28:37.058 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send_bytes": 21, 2023-11-27T15:28:37.058 INFO:teuthology.orchestra.run.smithi150.stdout: "op_resend": 3, 2023-11-27T15:28:37.058 INFO:teuthology.orchestra.run.smithi150.stdout: "op_reply": 2638, 2023-11-27T15:28:37.058 INFO:teuthology.orchestra.run.smithi150.stdout: "op_latency": { 2023-11-27T15:28:37.058 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2638, 2023-11-27T15:28:37.058 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 1.688966561, 2023-11-27T15:28:37.058 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000640245 2023-11-27T15:28:37.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.059 INFO:teuthology.orchestra.run.smithi150.stdout: "op_inflight": 0, 2023-11-27T15:28:37.059 INFO:teuthology.orchestra.run.smithi150.stdout: "oplen_avg": { 2023-11-27T15:28:37.059 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2638, 2023-11-27T15:28:37.059 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 2639 2023-11-27T15:28:37.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.059 INFO:teuthology.orchestra.run.smithi150.stdout: "op": 2638, 2023-11-27T15:28:37.059 INFO:teuthology.orchestra.run.smithi150.stdout: "op_r": 2638, 2023-11-27T15:28:37.059 INFO:teuthology.orchestra.run.smithi150.stdout: "op_w": 0, 2023-11-27T15:28:37.060 INFO:teuthology.orchestra.run.smithi150.stdout: "op_rmw": 0, 2023-11-27T15:28:37.060 INFO:teuthology.orchestra.run.smithi150.stdout: "op_pg": 0, 2023-11-27T15:28:37.060 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_stat": 1723, 2023-11-27T15:28:37.060 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_create": 0, 2023-11-27T15:28:37.060 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_read": 913, 2023-11-27T15:28:37.060 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_write": 0, 2023-11-27T15:28:37.060 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writefull": 0, 2023-11-27T15:28:37.060 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writesame": 0, 2023-11-27T15:28:37.061 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_append": 0, 2023-11-27T15:28:37.061 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_zero": 0, 2023-11-27T15:28:37.061 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_truncate": 0, 2023-11-27T15:28:37.061 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_delete": 0, 2023-11-27T15:28:37.061 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_mapext": 0, 2023-11-27T15:28:37.061 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_sparse_read": 0, 2023-11-27T15:28:37.061 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_clonerange": 0, 2023-11-27T15:28:37.061 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_getxattr": 1, 2023-11-27T15:28:37.061 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_setxattr": 0, 2023-11-27T15:28:37.062 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_cmpxattr": 0, 2023-11-27T15:28:37.062 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_rmxattr": 0, 2023-11-27T15:28:37.062 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_resetxattrs": 0, 2023-11-27T15:28:37.062 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_call": 0, 2023-11-27T15:28:37.062 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_watch": 0, 2023-11-27T15:28:37.062 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_notify": 0, 2023-11-27T15:28:37.062 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_src_cmpxattr": 0, 2023-11-27T15:28:37.062 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls": 0, 2023-11-27T15:28:37.063 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls_filter": 0, 2023-11-27T15:28:37.063 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_other": 0, 2023-11-27T15:28:37.063 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_active": 0, 2023-11-27T15:28:37.063 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_send": 0, 2023-11-27T15:28:37.063 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_resend": 0, 2023-11-27T15:28:37.063 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_ping": 0, 2023-11-27T15:28:37.063 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_active": 0, 2023-11-27T15:28:37.063 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_send": 0, 2023-11-27T15:28:37.064 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_resend": 0, 2023-11-27T15:28:37.064 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_active": 0, 2023-11-27T15:28:37.064 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_send": 0, 2023-11-27T15:28:37.064 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_resend": 0, 2023-11-27T15:28:37.064 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_active": 0, 2023-11-27T15:28:37.064 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_send": 0, 2023-11-27T15:28:37.064 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_resend": 0, 2023-11-27T15:28:37.064 INFO:teuthology.orchestra.run.smithi150.stdout: "command_active": 0, 2023-11-27T15:28:37.065 INFO:teuthology.orchestra.run.smithi150.stdout: "command_send": 0, 2023-11-27T15:28:37.065 INFO:teuthology.orchestra.run.smithi150.stdout: "command_resend": 0, 2023-11-27T15:28:37.065 INFO:teuthology.orchestra.run.smithi150.stdout: "map_epoch": 96, 2023-11-27T15:28:37.065 INFO:teuthology.orchestra.run.smithi150.stdout: "map_full": 0, 2023-11-27T15:28:37.065 INFO:teuthology.orchestra.run.smithi150.stdout: "map_inc": 13, 2023-11-27T15:28:37.065 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_sessions": 10, 2023-11-27T15:28:37.065 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_open": 10, 2023-11-27T15:28:37.065 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_close": 0, 2023-11-27T15:28:37.065 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_laggy": 0, 2023-11-27T15:28:37.066 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_wr": 0, 2023-11-27T15:28:37.066 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_rd": 2, 2023-11-27T15:28:37.066 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_del": 0 2023-11-27T15:28:37.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.066 INFO:teuthology.orchestra.run.smithi150.stdout: "oft": { 2023-11-27T15:28:37.066 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_objs": 0, 2023-11-27T15:28:37.066 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_kv_pairs": 0, 2023-11-27T15:28:37.066 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_updates": 0, 2023-11-27T15:28:37.067 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_removes": 0 2023-11-27T15:28:37.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.067 INFO:teuthology.orchestra.run.smithi150.stdout: "osd-slow-ops": { 2023-11-27T15:28:37.067 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_ops_count": 0 2023-11-27T15:28:37.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.067 INFO:teuthology.orchestra.run.smithi150.stdout: "purge_queue": { 2023-11-27T15:28:37.067 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops": 0, 2023-11-27T15:28:37.068 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops_high_water": 0, 2023-11-27T15:28:37.068 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing": 0, 2023-11-27T15:28:37.068 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_high_water": 0, 2023-11-27T15:28:37.068 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed": 0, 2023-11-27T15:28:37.068 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_item_in_journal": 0 2023-11-27T15:28:37.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.068 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:28:37.068 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2023-11-27T15:28:37.069 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2023-11-27T15:28:37.069 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2023-11-27T15:28:37.069 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 2886, 2023-11-27T15:28:37.069 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 244870019, 2023-11-27T15:28:37.069 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:37.069 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 2886, 2023-11-27T15:28:37.069 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2023-11-27T15:28:37.069 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2023-11-27T15:28:37.069 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2886, 2023-11-27T15:28:37.070 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 244870019, 2023-11-27T15:28:37.070 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2023-11-27T15:28:37.070 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.070 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.070 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.070 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-27T15:28:37.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.071 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_bytes": { 2023-11-27T15:28:37.071 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2023-11-27T15:28:37.071 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2023-11-27T15:28:37.071 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2023-11-27T15:28:37.071 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2023-11-27T15:28:37.071 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2023-11-27T15:28:37.071 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:37.071 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2023-11-27T15:28:37.072 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 2638, 2023-11-27T15:28:37.072 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 242784724, 2023-11-27T15:28:37.072 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2638, 2023-11-27T15:28:37.072 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 242784724, 2023-11-27T15:28:37.072 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2023-11-27T15:28:37.072 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.072 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.072 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.072 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-27T15:28:37.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.073 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_ops": { 2023-11-27T15:28:37.073 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2023-11-27T15:28:37.073 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 1024, 2023-11-27T15:28:37.073 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2023-11-27T15:28:37.073 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2023-11-27T15:28:37.073 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2023-11-27T15:28:37.073 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:37.074 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2023-11-27T15:28:37.074 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 2638, 2023-11-27T15:28:37.074 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 2638, 2023-11-27T15:28:37.074 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2638, 2023-11-27T15:28:37.074 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 2638, 2023-11-27T15:28:37.074 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2023-11-27T15:28:37.074 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.074 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.075 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.075 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-27T15:28:37.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.075 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle": { 2023-11-27T15:28:37.075 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2023-11-27T15:28:37.075 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2023-11-27T15:28:37.075 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2023-11-27T15:28:37.075 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2023-11-27T15:28:37.076 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2023-11-27T15:28:37.076 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:37.076 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2023-11-27T15:28:37.076 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2023-11-27T15:28:37.076 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2023-11-27T15:28:37.076 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 0, 2023-11-27T15:28:37.076 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 0, 2023-11-27T15:28:37.076 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2023-11-27T15:28:37.076 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.077 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.077 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.077 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-27T15:28:37.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:37.077 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle-0x562c00688460": { 2023-11-27T15:28:37.077 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2023-11-27T15:28:37.077 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2023-11-27T15:28:37.077 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2023-11-27T15:28:37.078 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2023-11-27T15:28:37.078 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2023-11-27T15:28:37.078 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:37.078 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2023-11-27T15:28:37.078 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2023-11-27T15:28:37.078 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2023-11-27T15:28:37.078 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 0, 2023-11-27T15:28:37.079 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 0, 2023-11-27T15:28:37.079 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2023-11-27T15:28:37.079 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:37.079 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:37.079 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:37.079 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-27T15:28:37.079 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-27T15:28:37.079 INFO:teuthology.orchestra.run.smithi150.stdout:} 2023-11-27T15:28:37.436 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-11-27T15:28:37.436 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2023-11-27T15:28:37.436 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2023-11-27T15:28:37.437 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2023-11-27T15:28:37.437 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2023-11-27T15:28:37.437 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2023-11-27T15:28:37.437 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2023-11-27T15:28:37.437 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2023-11-27T15:28:37.437 DEBUG:tasks.check_counter:Getting stats from i 2023-11-27T15:28:37.437 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph daemon mds.i perf dump 2023-11-27T15:28:38.074 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:28:38.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:38 smithi150 ceph-mon[100559]: pgmap v334: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:28:38.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:38 smithi102 ceph-mon[100296]: pgmap v334: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:28:38.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:38 smithi052 ceph-mon[99056]: pgmap v334: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:28:39.398 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2023-11-27T15:28:39.398 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:28:39.398 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 2208, 2023-11-27T15:28:39.399 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 2181, 2023-11-27T15:28:39.399 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 83304772, 2023-11-27T15:28:39.399 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 728521, 2023-11-27T15:28:39.399 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 7, 2023-11-27T15:28:39.399 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 3, 2023-11-27T15:28:39.399 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.268078021, 2023-11-27T15:28:39.399 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.061262053, 2023-11-27T15:28:39.399 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.119597438, 2023-11-27T15:28:39.400 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.078727942, 2023-11-27T15:28:39.400 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:39.400 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2181, 2023-11-27T15:28:39.400 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.079296200, 2023-11-27T15:28:39.400 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000036357 2023-11-27T15:28:39.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.400 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:39.401 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.401 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.401 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.401 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 1635776, 2023-11-27T15:28:39.401 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 120512 2023-11-27T15:28:39.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.401 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:28:39.402 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 310, 2023-11-27T15:28:39.402 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 464, 2023-11-27T15:28:39.402 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 83988734, 2023-11-27T15:28:39.402 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 520782, 2023-11-27T15:28:39.402 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 9, 2023-11-27T15:28:39.402 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 7, 2023-11-27T15:28:39.403 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.062036061, 2023-11-27T15:28:39.403 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.011156164, 2023-11-27T15:28:39.403 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.040680545, 2023-11-27T15:28:39.403 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.006792843, 2023-11-27T15:28:39.403 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:39.403 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 461, 2023-11-27T15:28:39.403 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.019977648, 2023-11-27T15:28:39.404 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000043335 2023-11-27T15:28:39.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.404 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:39.404 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.404 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.404 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.405 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 320, 2023-11-27T15:28:39.405 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 424928 2023-11-27T15:28:39.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.405 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:28:39.405 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 483, 2023-11-27T15:28:39.406 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 484, 2023-11-27T15:28:39.406 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 75918246, 2023-11-27T15:28:39.406 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 149909, 2023-11-27T15:28:39.406 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 6, 2023-11-27T15:28:39.406 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 5, 2023-11-27T15:28:39.406 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.076049159, 2023-11-27T15:28:39.406 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.013349172, 2023-11-27T15:28:39.407 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.044839077, 2023-11-27T15:28:39.407 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.014154411, 2023-11-27T15:28:39.407 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:39.407 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 482, 2023-11-27T15:28:39.407 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.016880703, 2023-11-27T15:28:39.407 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000035022 2023-11-27T15:28:39.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.407 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:39.408 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.408 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.408 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.408 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-27T15:28:39.408 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 0 2023-11-27T15:28:39.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.409 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2023-11-27T15:28:39.409 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 1, 2023-11-27T15:28:39.409 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2023-11-27T15:28:39.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.409 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-MDSRank": { 2023-11-27T15:28:39.409 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2023-11-27T15:28:39.409 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2023-11-27T15:28:39.409 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 4376, 2023-11-27T15:28:39.410 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.130683159, 2023-11-27T15:28:39.410 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000029863 2023-11-27T15:28:39.410 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-27T15:28:39.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.410 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-PurgeQueue": { 2023-11-27T15:28:39.410 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2023-11-27T15:28:39.410 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2023-11-27T15:28:39.410 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.411 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.411 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.411 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-27T15:28:39.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.411 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": { 2023-11-27T15:28:39.411 INFO:teuthology.orchestra.run.smithi150.stdout: "request": 0, 2023-11-27T15:28:39.411 INFO:teuthology.orchestra.run.smithi150.stdout: "reply": 0, 2023-11-27T15:28:39.411 INFO:teuthology.orchestra.run.smithi150.stdout: "reply_latency": { 2023-11-27T15:28:39.412 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.412 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.412 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.412 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_reply": 0, 2023-11-27T15:28:39.412 INFO:teuthology.orchestra.run.smithi150.stdout: "forward": 0, 2023-11-27T15:28:39.412 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_complete": 0, 2023-11-27T15:28:39.412 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_keys": 0, 2023-11-27T15:28:39.413 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_commit": 0, 2023-11-27T15:28:39.413 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_split": 0, 2023-11-27T15:28:39.413 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_merge": 0, 2023-11-27T15:28:39.413 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes": 34985, 2023-11-27T15:28:39.413 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_top": 0, 2023-11-27T15:28:39.413 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_bottom": 0, 2023-11-27T15:28:39.413 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pin_tail": 34985, 2023-11-27T15:28:39.413 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pinned": 34985, 2023-11-27T15:28:39.413 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_expired": 0, 2023-11-27T15:28:39.414 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_with_caps": 0, 2023-11-27T15:28:39.414 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": 0, 2023-11-27T15:28:39.414 INFO:teuthology.orchestra.run.smithi150.stdout: "subtrees": 2, 2023-11-27T15:28:39.414 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse": 0, 2023-11-27T15:28:39.414 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_hit": 0, 2023-11-27T15:28:39.414 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_forward": 0, 2023-11-27T15:28:39.414 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_discover": 0, 2023-11-27T15:28:39.415 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_dir_fetch": 0, 2023-11-27T15:28:39.415 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_remote_ino": 0, 2023-11-27T15:28:39.415 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_lock": 0, 2023-11-27T15:28:39.415 INFO:teuthology.orchestra.run.smithi150.stdout: "load_cent": 0, 2023-11-27T15:28:39.415 INFO:teuthology.orchestra.run.smithi150.stdout: "q": 0, 2023-11-27T15:28:39.415 INFO:teuthology.orchestra.run.smithi150.stdout: "exported": 0, 2023-11-27T15:28:39.415 INFO:teuthology.orchestra.run.smithi150.stdout: "exported_inodes": 0, 2023-11-27T15:28:39.415 INFO:teuthology.orchestra.run.smithi150.stdout: "imported": 0, 2023-11-27T15:28:39.416 INFO:teuthology.orchestra.run.smithi150.stdout: "imported_inodes": 0, 2023-11-27T15:28:39.416 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_dir_fetch": 0, 2023-11-27T15:28:39.416 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_backtrace_fetch": 0, 2023-11-27T15:28:39.416 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_peer_discover": 0, 2023-11-27T15:28:39.416 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rfiles": 4, 2023-11-27T15:28:39.416 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rbytes": 246, 2023-11-27T15:28:39.416 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rsnaps": 6, 2023-11-27T15:28:39.416 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_fetch": 0, 2023-11-27T15:28:39.417 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_set_tag": 0, 2023-11-27T15:28:39.417 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_repaired": 0, 2023-11-27T15:28:39.417 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_inotable_repaired": 0, 2023-11-27T15:28:39.417 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_inodes": 0, 2023-11-27T15:28:39.417 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_base_inodes": 0, 2023-11-27T15:28:39.417 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dirfrag_rstats": 0, 2023-11-27T15:28:39.417 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_file_inodes": 0, 2023-11-27T15:28:39.417 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_inode_file_caps": 0, 2023-11-27T15:28:39.418 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_revoke": 0, 2023-11-27T15:28:39.418 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_grant": 0, 2023-11-27T15:28:39.418 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_trunc": 0, 2023-11-27T15:28:39.418 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-27T15:28:39.418 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-27T15:28:39.418 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps": 0, 2023-11-27T15:28:39.418 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps_dirty": 0, 2023-11-27T15:28:39.418 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_cap_release": 0, 2023-11-27T15:28:39.419 INFO:teuthology.orchestra.run.smithi150.stdout: "process_request_cap_release": 0 2023-11-27T15:28:39.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.419 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache": { 2023-11-27T15:28:39.419 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update": 0, 2023-11-27T15:28:39.419 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update_receipt": 0, 2023-11-27T15:28:39.419 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_try_discover": 0, 2023-11-27T15:28:39.419 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_send_discover": 0, 2023-11-27T15:28:39.419 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_handle_discover": 0, 2023-11-27T15:28:39.420 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays": 0, 2023-11-27T15:28:39.420 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_delayed": 0, 2023-11-27T15:28:39.420 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_enqueuing": 0, 2023-11-27T15:28:39.420 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_created": 0, 2023-11-27T15:28:39.420 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_enqueued": 0, 2023-11-27T15:28:39.420 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_reintegrated": 0, 2023-11-27T15:28:39.420 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_migrated": 0, 2023-11-27T15:28:39.420 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_processing": 0, 2023-11-27T15:28:39.421 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_enqueued": 0, 2023-11-27T15:28:39.421 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_prioritized": 0, 2023-11-27T15:28:39.421 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_started": 0, 2023-11-27T15:28:39.421 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_completed": 0, 2023-11-27T15:28:39.421 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_enqueue_scrub": 0, 2023-11-27T15:28:39.421 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_exportdir": 0, 2023-11-27T15:28:39.421 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_flush": 0, 2023-11-27T15:28:39.421 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragmentdir": 0, 2023-11-27T15:28:39.422 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragstats": 0, 2023-11-27T15:28:39.422 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_inodestats": 0 2023-11-27T15:28:39.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.422 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log": { 2023-11-27T15:28:39.422 INFO:teuthology.orchestra.run.smithi150.stdout: "evadd": 0, 2023-11-27T15:28:39.422 INFO:teuthology.orchestra.run.smithi150.stdout: "evex": 0, 2023-11-27T15:28:39.422 INFO:teuthology.orchestra.run.smithi150.stdout: "evtrm": 0, 2023-11-27T15:28:39.423 INFO:teuthology.orchestra.run.smithi150.stdout: "ev": 0, 2023-11-27T15:28:39.423 INFO:teuthology.orchestra.run.smithi150.stdout: "evexg": 0, 2023-11-27T15:28:39.423 INFO:teuthology.orchestra.run.smithi150.stdout: "evexd": 0, 2023-11-27T15:28:39.423 INFO:teuthology.orchestra.run.smithi150.stdout: "segadd": 0, 2023-11-27T15:28:39.423 INFO:teuthology.orchestra.run.smithi150.stdout: "segex": 0, 2023-11-27T15:28:39.423 INFO:teuthology.orchestra.run.smithi150.stdout: "segtrm": 0, 2023-11-27T15:28:39.423 INFO:teuthology.orchestra.run.smithi150.stdout: "seg": 108, 2023-11-27T15:28:39.423 INFO:teuthology.orchestra.run.smithi150.stdout: "segexg": 0, 2023-11-27T15:28:39.424 INFO:teuthology.orchestra.run.smithi150.stdout: "segexd": 0, 2023-11-27T15:28:39.424 INFO:teuthology.orchestra.run.smithi150.stdout: "expos": 4194304, 2023-11-27T15:28:39.424 INFO:teuthology.orchestra.run.smithi150.stdout: "wrpos": 0, 2023-11-27T15:28:39.424 INFO:teuthology.orchestra.run.smithi150.stdout: "rdpos": 244659650, 2023-11-27T15:28:39.424 INFO:teuthology.orchestra.run.smithi150.stdout: "jlat": { 2023-11-27T15:28:39.424 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.424 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.424 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.425 INFO:teuthology.orchestra.run.smithi150.stdout: "replayed": 81776 2023-11-27T15:28:39.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.425 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_mem": { 2023-11-27T15:28:39.425 INFO:teuthology.orchestra.run.smithi150.stdout: "ino": 10795, 2023-11-27T15:28:39.425 INFO:teuthology.orchestra.run.smithi150.stdout: "ino+": 56031, 2023-11-27T15:28:39.425 INFO:teuthology.orchestra.run.smithi150.stdout: "ino-": 45236, 2023-11-27T15:28:39.425 INFO:teuthology.orchestra.run.smithi150.stdout: "dir": 500, 2023-11-27T15:28:39.426 INFO:teuthology.orchestra.run.smithi150.stdout: "dir+": 2430, 2023-11-27T15:28:39.426 INFO:teuthology.orchestra.run.smithi150.stdout: "dir-": 1930, 2023-11-27T15:28:39.426 INFO:teuthology.orchestra.run.smithi150.stdout: "dn": 34985, 2023-11-27T15:28:39.426 INFO:teuthology.orchestra.run.smithi150.stdout: "dn+": 74419, 2023-11-27T15:28:39.426 INFO:teuthology.orchestra.run.smithi150.stdout: "dn-": 39434, 2023-11-27T15:28:39.426 INFO:teuthology.orchestra.run.smithi150.stdout: "cap": 0, 2023-11-27T15:28:39.426 INFO:teuthology.orchestra.run.smithi150.stdout: "cap+": 0, 2023-11-27T15:28:39.426 INFO:teuthology.orchestra.run.smithi150.stdout: "cap-": 0, 2023-11-27T15:28:39.427 INFO:teuthology.orchestra.run.smithi150.stdout: "rss": 148660, 2023-11-27T15:28:39.427 INFO:teuthology.orchestra.run.smithi150.stdout: "heap": 207156 2023-11-27T15:28:39.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.427 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_server": { 2023-11-27T15:28:39.427 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_client_request": 0, 2023-11-27T15:28:39.427 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_server_request": 0, 2023-11-27T15:28:39.427 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_request": 0, 2023-11-27T15:28:39.427 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_session": 0, 2023-11-27T15:28:39.428 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_peer_request": 0, 2023-11-27T15:28:39.428 INFO:teuthology.orchestra.run.smithi150.stdout: "req_create_latency": { 2023-11-27T15:28:39.428 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.428 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.428 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.428 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getattr_latency": { 2023-11-27T15:28:39.429 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.429 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.429 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.429 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getfilelock_latency": { 2023-11-27T15:28:39.429 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.429 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.429 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.430 INFO:teuthology.orchestra.run.smithi150.stdout: "req_link_latency": { 2023-11-27T15:28:39.430 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.430 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.430 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.430 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookup_latency": { 2023-11-27T15:28:39.430 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.431 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.431 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.431 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookuphash_latency": { 2023-11-27T15:28:39.431 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.431 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.431 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.432 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupino_latency": { 2023-11-27T15:28:39.432 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.432 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.432 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.432 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupname_latency": { 2023-11-27T15:28:39.432 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.432 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.433 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.433 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupparent_latency": { 2023-11-27T15:28:39.433 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.433 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.433 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.433 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupsnap_latency": { 2023-11-27T15:28:39.434 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.434 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.434 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.434 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lssnap_latency": { 2023-11-27T15:28:39.434 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.434 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.434 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.435 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mkdir_latency": { 2023-11-27T15:28:39.435 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.435 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.435 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.435 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mknod_latency": { 2023-11-27T15:28:39.435 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.436 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.436 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.436 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mksnap_latency": { 2023-11-27T15:28:39.436 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.436 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.436 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.437 INFO:teuthology.orchestra.run.smithi150.stdout: "req_open_latency": { 2023-11-27T15:28:39.437 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.437 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.437 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.437 INFO:teuthology.orchestra.run.smithi150.stdout: "req_readdir_latency": { 2023-11-27T15:28:39.437 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.438 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.438 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.438 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rename_latency": { 2023-11-27T15:28:39.438 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.438 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.438 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.439 INFO:teuthology.orchestra.run.smithi150.stdout: "req_renamesnap_latency": { 2023-11-27T15:28:39.439 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.439 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.439 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.439 INFO:teuthology.orchestra.run.smithi150.stdout: "req_snapdiff_latency": { 2023-11-27T15:28:39.439 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.439 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.440 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.440 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmdir_latency": { 2023-11-27T15:28:39.440 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.440 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.440 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.440 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmsnap_latency": { 2023-11-27T15:28:39.441 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.441 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.441 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.441 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmxattr_latency": { 2023-11-27T15:28:39.441 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.441 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.442 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.442 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setattr_latency": { 2023-11-27T15:28:39.442 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.442 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.442 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.442 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setdirlayout_latency": { 2023-11-27T15:28:39.443 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.443 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.443 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.443 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setfilelock_latency": { 2023-11-27T15:28:39.443 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.443 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.444 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.444 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setlayout_latency": { 2023-11-27T15:28:39.444 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.444 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.444 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.444 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setxattr_latency": { 2023-11-27T15:28:39.445 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.445 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.445 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.445 INFO:teuthology.orchestra.run.smithi150.stdout: "req_symlink_latency": { 2023-11-27T15:28:39.445 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.445 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.445 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.446 INFO:teuthology.orchestra.run.smithi150.stdout: "req_unlink_latency": { 2023-11-27T15:28:39.446 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.446 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.446 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.446 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_revoke_eviction": 0, 2023-11-27T15:28:39.446 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_acquisition_throttle": 0, 2023-11-27T15:28:39.447 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getvxattr_latency": { 2023-11-27T15:28:39.447 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.447 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.447 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.447 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-27T15:28:39.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.447 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_sessions": { 2023-11-27T15:28:39.447 INFO:teuthology.orchestra.run.smithi150.stdout: "session_count": 2, 2023-11-27T15:28:39.448 INFO:teuthology.orchestra.run.smithi150.stdout: "session_add": 5, 2023-11-27T15:28:39.448 INFO:teuthology.orchestra.run.smithi150.stdout: "session_remove": 3, 2023-11-27T15:28:39.448 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_open": 4, 2023-11-27T15:28:39.448 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_stale": 0, 2023-11-27T15:28:39.448 INFO:teuthology.orchestra.run.smithi150.stdout: "total_load": 0, 2023-11-27T15:28:39.448 INFO:teuthology.orchestra.run.smithi150.stdout: "average_load": 0, 2023-11-27T15:28:39.448 INFO:teuthology.orchestra.run.smithi150.stdout: "avg_session_uptime": 3616 2023-11-27T15:28:39.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.449 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2023-11-27T15:28:39.449 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:28:39.449 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2023-11-27T15:28:39.449 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:28:39.449 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:28:39.449 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:28:39.449 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:28:39.449 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:28:39.450 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:28:39.450 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:28:39.450 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:28:39.450 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:28:39.450 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:28:39.450 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:28:39.450 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:28:39.450 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:28:39.451 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2023-11-27T15:28:39.451 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:28:39.451 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2023-11-27T15:28:39.451 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:28:39.451 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:28:39.451 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:28:39.451 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:28:39.451 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:28:39.452 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:28:39.452 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:28:39.452 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2023-11-27T15:28:39.452 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:28:39.452 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:28:39.452 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:28:39.452 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2023-11-27T15:28:39.452 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2023-11-27T15:28:39.453 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2023-11-27T15:28:39.453 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:28:39.453 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:28:39.453 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:28:39.453 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:28:39.453 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 605659, 2023-11-27T15:28:39.453 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 10870, 2023-11-27T15:28:39.453 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2023-11-27T15:28:39.454 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2023-11-27T15:28:39.454 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2023-11-27T15:28:39.454 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2023-11-27T15:28:39.454 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:28:39.454 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2023-11-27T15:28:39.454 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:28:39.454 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2023-11-27T15:28:39.455 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 8224, 2023-11-27T15:28:39.455 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 171, 2023-11-27T15:28:39.455 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:28:39.455 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:28:39.455 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2023-11-27T15:28:39.455 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2023-11-27T15:28:39.455 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 49241913, 2023-11-27T15:28:39.455 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 380680, 2023-11-27T15:28:39.456 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2023-11-27T15:28:39.456 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2023-11-27T15:28:39.456 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2023-11-27T15:28:39.456 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2023-11-27T15:28:39.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.456 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter": { 2023-11-27T15:28:39.456 INFO:teuthology.orchestra.run.smithi150.stdout: "op_active": 0, 2023-11-27T15:28:39.456 INFO:teuthology.orchestra.run.smithi150.stdout: "op_laggy": 0, 2023-11-27T15:28:39.457 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send": 2745, 2023-11-27T15:28:39.457 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send_bytes": 26, 2023-11-27T15:28:39.457 INFO:teuthology.orchestra.run.smithi150.stdout: "op_resend": 4, 2023-11-27T15:28:39.457 INFO:teuthology.orchestra.run.smithi150.stdout: "op_reply": 2745, 2023-11-27T15:28:39.457 INFO:teuthology.orchestra.run.smithi150.stdout: "op_latency": { 2023-11-27T15:28:39.457 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2745, 2023-11-27T15:28:39.457 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 2.873943242, 2023-11-27T15:28:39.457 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001046973 2023-11-27T15:28:39.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.458 INFO:teuthology.orchestra.run.smithi150.stdout: "op_inflight": 0, 2023-11-27T15:28:39.458 INFO:teuthology.orchestra.run.smithi150.stdout: "oplen_avg": { 2023-11-27T15:28:39.458 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2745, 2023-11-27T15:28:39.458 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 2746 2023-11-27T15:28:39.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.458 INFO:teuthology.orchestra.run.smithi150.stdout: "op": 2745, 2023-11-27T15:28:39.458 INFO:teuthology.orchestra.run.smithi150.stdout: "op_r": 2745, 2023-11-27T15:28:39.459 INFO:teuthology.orchestra.run.smithi150.stdout: "op_w": 0, 2023-11-27T15:28:39.459 INFO:teuthology.orchestra.run.smithi150.stdout: "op_rmw": 0, 2023-11-27T15:28:39.459 INFO:teuthology.orchestra.run.smithi150.stdout: "op_pg": 0, 2023-11-27T15:28:39.459 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_stat": 1748, 2023-11-27T15:28:39.459 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_create": 0, 2023-11-27T15:28:39.459 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_read": 994, 2023-11-27T15:28:39.459 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_write": 0, 2023-11-27T15:28:39.460 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writefull": 0, 2023-11-27T15:28:39.460 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writesame": 0, 2023-11-27T15:28:39.460 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_append": 0, 2023-11-27T15:28:39.460 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_zero": 0, 2023-11-27T15:28:39.460 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_truncate": 0, 2023-11-27T15:28:39.460 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_delete": 0, 2023-11-27T15:28:39.460 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_mapext": 0, 2023-11-27T15:28:39.460 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_sparse_read": 0, 2023-11-27T15:28:39.461 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_clonerange": 0, 2023-11-27T15:28:39.461 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_getxattr": 2, 2023-11-27T15:28:39.461 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_setxattr": 0, 2023-11-27T15:28:39.461 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_cmpxattr": 0, 2023-11-27T15:28:39.461 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_rmxattr": 0, 2023-11-27T15:28:39.461 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_resetxattrs": 0, 2023-11-27T15:28:39.461 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_call": 0, 2023-11-27T15:28:39.461 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_watch": 0, 2023-11-27T15:28:39.462 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_notify": 0, 2023-11-27T15:28:39.462 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_src_cmpxattr": 0, 2023-11-27T15:28:39.462 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls": 0, 2023-11-27T15:28:39.462 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls_filter": 0, 2023-11-27T15:28:39.462 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_other": 0, 2023-11-27T15:28:39.462 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_active": 0, 2023-11-27T15:28:39.462 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_send": 0, 2023-11-27T15:28:39.462 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_resend": 0, 2023-11-27T15:28:39.463 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_ping": 0, 2023-11-27T15:28:39.463 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_active": 0, 2023-11-27T15:28:39.463 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_send": 0, 2023-11-27T15:28:39.463 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_resend": 0, 2023-11-27T15:28:39.463 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_active": 0, 2023-11-27T15:28:39.463 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_send": 0, 2023-11-27T15:28:39.463 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_resend": 0, 2023-11-27T15:28:39.463 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_active": 0, 2023-11-27T15:28:39.464 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_send": 0, 2023-11-27T15:28:39.464 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_resend": 0, 2023-11-27T15:28:39.464 INFO:teuthology.orchestra.run.smithi150.stdout: "command_active": 0, 2023-11-27T15:28:39.464 INFO:teuthology.orchestra.run.smithi150.stdout: "command_send": 0, 2023-11-27T15:28:39.464 INFO:teuthology.orchestra.run.smithi150.stdout: "command_resend": 0, 2023-11-27T15:28:39.464 INFO:teuthology.orchestra.run.smithi150.stdout: "map_epoch": 96, 2023-11-27T15:28:39.464 INFO:teuthology.orchestra.run.smithi150.stdout: "map_full": 0, 2023-11-27T15:28:39.464 INFO:teuthology.orchestra.run.smithi150.stdout: "map_inc": 13, 2023-11-27T15:28:39.465 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_sessions": 11, 2023-11-27T15:28:39.465 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_open": 11, 2023-11-27T15:28:39.465 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_close": 0, 2023-11-27T15:28:39.465 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_laggy": 0, 2023-11-27T15:28:39.465 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_wr": 0, 2023-11-27T15:28:39.465 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_rd": 2, 2023-11-27T15:28:39.465 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_del": 0 2023-11-27T15:28:39.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.466 INFO:teuthology.orchestra.run.smithi150.stdout: "oft": { 2023-11-27T15:28:39.466 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_objs": 0, 2023-11-27T15:28:39.466 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_kv_pairs": 0, 2023-11-27T15:28:39.466 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_updates": 0, 2023-11-27T15:28:39.466 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_removes": 0 2023-11-27T15:28:39.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.466 INFO:teuthology.orchestra.run.smithi150.stdout: "osd-slow-ops": { 2023-11-27T15:28:39.467 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_ops_count": 0 2023-11-27T15:28:39.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.467 INFO:teuthology.orchestra.run.smithi150.stdout: "purge_queue": { 2023-11-27T15:28:39.467 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops": 0, 2023-11-27T15:28:39.467 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops_high_water": 0, 2023-11-27T15:28:39.467 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing": 0, 2023-11-27T15:28:39.467 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_high_water": 0, 2023-11-27T15:28:39.467 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed": 0, 2023-11-27T15:28:39.468 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_item_in_journal": 0 2023-11-27T15:28:39.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.468 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:28:39.468 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2023-11-27T15:28:39.468 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2023-11-27T15:28:39.468 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2023-11-27T15:28:39.468 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 2997, 2023-11-27T15:28:39.468 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 242926348, 2023-11-27T15:28:39.469 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:39.469 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 2997, 2023-11-27T15:28:39.469 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2023-11-27T15:28:39.469 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2023-11-27T15:28:39.469 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2997, 2023-11-27T15:28:39.469 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 242926348, 2023-11-27T15:28:39.469 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2023-11-27T15:28:39.470 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.470 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.470 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.470 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-27T15:28:39.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.470 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_bytes": { 2023-11-27T15:28:39.470 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2023-11-27T15:28:39.470 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2023-11-27T15:28:39.471 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2023-11-27T15:28:39.471 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2023-11-27T15:28:39.471 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2023-11-27T15:28:39.471 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:39.471 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2023-11-27T15:28:39.471 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 2745, 2023-11-27T15:28:39.471 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 240814340, 2023-11-27T15:28:39.471 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2745, 2023-11-27T15:28:39.472 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 240814340, 2023-11-27T15:28:39.472 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2023-11-27T15:28:39.472 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.472 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.472 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.472 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-27T15:28:39.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.472 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_ops": { 2023-11-27T15:28:39.473 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2023-11-27T15:28:39.473 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 1024, 2023-11-27T15:28:39.473 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2023-11-27T15:28:39.473 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2023-11-27T15:28:39.473 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2023-11-27T15:28:39.473 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:39.473 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2023-11-27T15:28:39.473 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 2745, 2023-11-27T15:28:39.474 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 2745, 2023-11-27T15:28:39.474 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2745, 2023-11-27T15:28:39.474 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 2745, 2023-11-27T15:28:39.474 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2023-11-27T15:28:39.474 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.474 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.474 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.474 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-27T15:28:39.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.475 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle": { 2023-11-27T15:28:39.475 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2023-11-27T15:28:39.475 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2023-11-27T15:28:39.475 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2023-11-27T15:28:39.475 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2023-11-27T15:28:39.475 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2023-11-27T15:28:39.476 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:39.476 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2023-11-27T15:28:39.476 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2023-11-27T15:28:39.476 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2023-11-27T15:28:39.476 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 0, 2023-11-27T15:28:39.476 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 0, 2023-11-27T15:28:39.476 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2023-11-27T15:28:39.476 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.477 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.477 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.477 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-27T15:28:39.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:39.477 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle-0x561f628e2460": { 2023-11-27T15:28:39.477 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2023-11-27T15:28:39.477 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2023-11-27T15:28:39.477 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2023-11-27T15:28:39.478 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2023-11-27T15:28:39.478 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2023-11-27T15:28:39.478 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:39.478 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2023-11-27T15:28:39.478 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2023-11-27T15:28:39.478 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2023-11-27T15:28:39.478 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 0, 2023-11-27T15:28:39.478 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 0, 2023-11-27T15:28:39.479 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2023-11-27T15:28:39.479 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:39.479 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:39.479 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:39.479 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-27T15:28:39.479 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-27T15:28:39.479 INFO:teuthology.orchestra.run.smithi150.stdout:} 2023-11-27T15:28:39.827 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2023-11-27T15:28:39.827 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2023-11-27T15:28:39.828 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2023-11-27T15:28:39.828 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2023-11-27T15:28:39.828 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2023-11-27T15:28:39.828 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2023-11-27T15:28:39.828 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2023-11-27T15:28:39.828 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2023-11-27T15:28:39.828 DEBUG:tasks.check_counter:Getting stats from l 2023-11-27T15:28:39.829 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 -- ceph daemon mds.l perf dump 2023-11-27T15:28:40.277 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:39 smithi150 ceph-mon[100559]: pgmap v335: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:28:40.327 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:39 smithi102 ceph-mon[100296]: pgmap v335: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:28:40.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:39 smithi052 ceph-mon[99056]: pgmap v335: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:28:40.470 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:28:41.697 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2023-11-27T15:28:41.697 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:28:41.697 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 0, 2023-11-27T15:28:41.697 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 2, 2023-11-27T15:28:41.697 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 0, 2023-11-27T15:28:41.697 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 0, 2023-11-27T15:28:41.697 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 4, 2023-11-27T15:28:41.698 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 0, 2023-11-27T15:28:41.698 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.001229705, 2023-11-27T15:28:41.698 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.000000000, 2023-11-27T15:28:41.698 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.000339529, 2023-11-27T15:28:41.698 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:28:41.698 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:41.698 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:41.698 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:41.698 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:41.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:41.699 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:41.699 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:41.699 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:41.699 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:41.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:41.699 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-27T15:28:41.699 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 0 2023-11-27T15:28:41.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:41.700 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:28:41.700 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 6, 2023-11-27T15:28:41.700 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 163, 2023-11-27T15:28:41.700 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 3312, 2023-11-27T15:28:41.700 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 180512, 2023-11-27T15:28:41.700 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 5, 2023-11-27T15:28:41.700 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 3, 2023-11-27T15:28:41.700 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.010952091, 2023-11-27T15:28:41.701 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.006702766, 2023-11-27T15:28:41.701 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.001964685, 2023-11-27T15:28:41.701 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:28:41.701 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:41.701 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 161, 2023-11-27T15:28:41.701 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.015488798, 2023-11-27T15:28:41.701 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000096203 2023-11-27T15:28:41.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:41.702 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:41.702 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:41.702 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:41.702 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:41.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:41.702 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-11-27T15:28:41.702 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 180512 2023-11-27T15:28:41.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:41.703 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:28:41.703 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 221, 2023-11-27T15:28:41.703 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 210, 2023-11-27T15:28:41.703 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 1557040, 2023-11-27T15:28:41.703 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 119744, 2023-11-27T15:28:41.703 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 2, 2023-11-27T15:28:41.703 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 1, 2023-11-27T15:28:41.703 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.044396491, 2023-11-27T15:28:41.703 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.012938389, 2023-11-27T15:28:41.704 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.034308149, 2023-11-27T15:28:41.704 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.006264220, 2023-11-27T15:28:41.704 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:28:41.704 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 209, 2023-11-27T15:28:41.704 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.011284573, 2023-11-27T15:28:41.704 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000053993 2023-11-27T15:28:41.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:41.704 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:28:41.705 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:41.705 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:41.705 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:41.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:41.705 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 1557040, 2023-11-27T15:28:41.705 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 119744 2023-11-27T15:28:41.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:41.705 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2023-11-27T15:28:41.706 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 0, 2023-11-27T15:28:41.706 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2023-11-27T15:28:41.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:41.706 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2023-11-27T15:28:41.706 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:28:41.706 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2023-11-27T15:28:41.706 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:28:41.706 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:28:41.706 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:28:41.707 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:28:41.707 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:28:41.707 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:28:41.707 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:28:41.707 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:28:41.707 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:28:41.707 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:28:41.707 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:28:41.708 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:28:41.708 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:28:41.708 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2023-11-27T15:28:41.708 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:28:41.708 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2023-11-27T15:28:41.708 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:28:41.708 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:28:41.708 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:28:41.708 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:28:41.709 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:28:41.709 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:28:41.709 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:28:41.709 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2023-11-27T15:28:41.709 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:28:41.709 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:28:41.709 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:28:41.709 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2023-11-27T15:28:41.710 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2023-11-27T15:28:41.710 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2023-11-27T15:28:41.710 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:28:41.710 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:28:41.710 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:28:41.710 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:28:41.710 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 148994, 2023-11-27T15:28:41.710 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 31, 2023-11-27T15:28:41.710 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2023-11-27T15:28:41.711 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2023-11-27T15:28:41.711 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2023-11-27T15:28:41.711 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2023-11-27T15:28:41.711 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:28:41.711 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2023-11-27T15:28:41.711 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:28:41.711 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2023-11-27T15:28:41.711 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 0, 2023-11-27T15:28:41.712 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 0, 2023-11-27T15:28:41.712 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:28:41.712 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:28:41.712 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2023-11-27T15:28:41.712 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2023-11-27T15:28:41.712 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 928, 2023-11-27T15:28:41.712 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 2, 2023-11-27T15:28:41.712 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2023-11-27T15:28:41.712 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2023-11-27T15:28:41.713 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2023-11-27T15:28:41.713 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2023-11-27T15:28:41.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2023-11-27T15:28:41.713 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:28:41.713 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2023-11-27T15:28:41.713 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2023-11-27T15:28:41.713 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2023-11-27T15:28:41.713 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 223, 2023-11-27T15:28:41.714 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 1525849, 2023-11-27T15:28:41.714 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2023-11-27T15:28:41.714 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 223, 2023-11-27T15:28:41.714 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2023-11-27T15:28:41.714 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2023-11-27T15:28:41.714 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 223, 2023-11-27T15:28:41.714 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 1525849, 2023-11-27T15:28:41.714 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2023-11-27T15:28:41.714 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2023-11-27T15:28:41.715 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2023-11-27T15:28:41.715 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2023-11-27T15:28:41.715 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-27T15:28:41.715 INFO:teuthology.orchestra.run.smithi150.stdout: } 2023-11-27T15:28:41.715 INFO:teuthology.orchestra.run.smithi150.stdout:} 2023-11-27T15:28:42.124 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2023-11-27T15:28:42.128 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2023-11-27T15:28:42.128 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2023-11-27T15:28:42.128 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2023-11-27T15:28:42.128 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2023-11-27T15:28:42.128 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2023-11-27T15:28:42.128 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2023-11-27T15:28:42.128 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2023-11-27T15:28:42.129 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-11-27T15:28:42.161 INFO:tasks.kclient:Unmounting kernel clients... 2023-11-27T15:28:42.161 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2023-11-27T15:28:42.161 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-27T15:28:42.208 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2023-11-27T15:28:42.208 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-27T15:28:42.277 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:28:42.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.462+0000 7fd96ac1a700 1 -- 172.21.15.52:0/2400347560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9641013e0 msgr2=0x7fd96410d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:42.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.462+0000 7fd96ac1a700 1 --2- 172.21.15.52:0/2400347560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9641013e0 0x7fd96410d100 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00b0a0 tx=0x7fd95c00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:28:42.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.462+0000 7fd96ac1a700 1 -- 172.21.15.52:0/2400347560 shutdown_connections 2023-11-27T15:28:42.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.462+0000 7fd96ac1a700 1 --2- 172.21.15.52:0/2400347560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9641013e0 0x7fd96410d100 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:42.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.462+0000 7fd96ac1a700 1 --2- 172.21.15.52:0/2400347560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9641062d0 0x7fd9641066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:42.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.462+0000 7fd96ac1a700 1 --2- 172.21.15.52:0/2400347560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd964100a50 0x7fd964100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:42.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.462+0000 7fd96ac1a700 1 -- 172.21.15.52:0/2400347560 >> 172.21.15.52:0/2400347560 conn(0x7fd9640774a0 msgr2=0x7fd9640778a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:42.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.462+0000 7fd96ac1a700 1 -- 172.21.15.52:0/2400347560 shutdown_connections 2023-11-27T15:28:42.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.462+0000 7fd96ac1a700 1 -- 172.21.15.52:0/2400347560 wait complete. 2023-11-27T15:28:42.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.463+0000 7fd96ac1a700 1 Processor -- start 2023-11-27T15:28:42.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.463+0000 7fd96ac1a700 1 -- start start 2023-11-27T15:28:42.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.463+0000 7fd96ac1a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd964100a50 0x7fd964196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:42.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.463+0000 7fd96ac1a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9641013e0 0x7fd964196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:42.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.463+0000 7fd96ac1a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9641062d0 0x7fd96419ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:42.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.463+0000 7fd96ac1a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96419b420 con 0x7fd9641062d0 2023-11-27T15:28:42.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.463+0000 7fd96ac1a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96419b590 con 0x7fd9641013e0 2023-11-27T15:28:42.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.463+0000 7fd96ac1a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96419b750 con 0x7fd964100a50 2023-11-27T15:28:42.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.463+0000 7fd9691b7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9641062d0 0x7fd96419ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:42.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.463+0000 7fd9691b7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9641062d0 0x7fd96419ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56492/0 (socket says 172.21.15.52:56492) 2023-11-27T15:28:42.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.463+0000 7fd9691b7700 1 -- 172.21.15.52:0/2452862938 learned_addr learned my addr 172.21.15.52:0/2452862938 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:42.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.463+0000 7fd963fff700 1 --2- 172.21.15.52:0/2452862938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9641013e0 0x7fd964196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:42.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.464+0000 7fd9689b6700 1 --2- 172.21.15.52:0/2452862938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd964100a50 0x7fd964196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:42.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.464+0000 7fd9691b7700 1 -- 172.21.15.52:0/2452862938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd964100a50 msgr2=0x7fd964196440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:42.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.464+0000 7fd9691b7700 1 --2- 172.21.15.52:0/2452862938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd964100a50 0x7fd964196440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:42.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.464+0000 7fd9691b7700 1 -- 172.21.15.52:0/2452862938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9641013e0 msgr2=0x7fd964196980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:42.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.464+0000 7fd9691b7700 1 --2- 172.21.15.52:0/2452862938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9641013e0 0x7fd964196980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:42.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.464+0000 7fd9691b7700 1 -- 172.21.15.52:0/2452862938 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd95c009d00 con 0x7fd9641062d0 2023-11-27T15:28:42.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.464+0000 7fd9689b6700 1 --2- 172.21.15.52:0/2452862938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd964100a50 0x7fd964196440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:28:42.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.464+0000 7fd9691b7700 1 --2- 172.21.15.52:0/2452862938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9641062d0 0x7fd96419ade0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00b780 tx=0x7fd95c00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:42.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.464+0000 7fd961ffb700 1 -- 172.21.15.52:0/2452862938 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c00e040 con 0x7fd9641062d0 2023-11-27T15:28:42.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.464+0000 7fd961ffb700 1 -- 172.21.15.52:0/2452862938 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd95c004e60 con 0x7fd9641062d0 2023-11-27T15:28:42.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.464+0000 7fd961ffb700 1 -- 172.21.15.52:0/2452862938 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c0052a0 con 0x7fd9641062d0 2023-11-27T15:28:42.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.464+0000 7fd96ac1a700 1 -- 172.21.15.52:0/2452862938 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9641102d0 con 0x7fd9641062d0 2023-11-27T15:28:42.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.464+0000 7fd96ac1a700 1 -- 172.21.15.52:0/2452862938 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd964110140 con 0x7fd9641062d0 2023-11-27T15:28:42.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.465+0000 7fd96ac1a700 1 -- 172.21.15.52:0/2452862938 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd964110460 con 0x7fd9641062d0 2023-11-27T15:28:42.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.466+0000 7fd961ffb700 1 -- 172.21.15.52:0/2452862938 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fd95c005400 con 0x7fd9641062d0 2023-11-27T15:28:42.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.467+0000 7fd961ffb700 1 --2- 172.21.15.52:0/2452862938 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd9540a12c0 0x7fd9540a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:42.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.467+0000 7fd961ffb700 1 -- 172.21.15.52:0/2452862938 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fd95c0ce9f0 con 0x7fd9641062d0 2023-11-27T15:28:42.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.467+0000 7fd9689b6700 1 --2- 172.21.15.52:0/2452862938 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd9540a12c0 0x7fd9540a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:42.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.468+0000 7fd9689b6700 1 --2- 172.21.15.52:0/2452862938 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd9540a12c0 0x7fd9540a3770 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd958009b50 tx=0x7fd958009550 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:42.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.471+0000 7fd961ffb700 1 -- 172.21.15.52:0/2452862938 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd95c097e60 con 0x7fd9641062d0 2023-11-27T15:28:42.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:42 smithi150 ceph-mon[100559]: pgmap v336: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:28:42.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:42 smithi102 ceph-mon[100296]: pgmap v336: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:28:42.615 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:42 smithi052 ceph-mon[99056]: pgmap v336: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:28:42.673 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.669+0000 7fd96ac1a700 1 -- 172.21.15.52:0/2452862938 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9640671d0 con 0x7fd9641062d0 2023-11-27T15:28:42.674 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.671+0000 7fd961ffb700 1 -- 172.21.15.52:0/2452862938 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+23125 (secure 0 0 0) 0x7fd95c09b480 con 0x7fd9641062d0 2023-11-27T15:28:42.675 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:28:42.675 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":96,"fsid":"c6c7df88-8d36-11ee-95a2-87774f69a715","created":"2023-11-27T15:09:03.333345+0000","modified":"2023-11-27T15:28:13.934915+0000","last_up_change":"2023-11-27T15:15:27.656134+0000","last_in_change":"2023-11-27T15:15:10.671153+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:12:14.177523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:16:07.842251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:16:08.728520+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":95,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58d69ede-c293-49f5-abbe-8187793f2b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6803","nonce":608216415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6805","nonce":608216415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6809","nonce":608216415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6807","nonce":608216415}]},"public_addr":"172.21.15.52:6803/608216415","cluster_addr":"172.21.15.52:6805/608216415","heartbeat_back_addr":"172.21.15.52:6809/608216415","heartbeat_front_addr":"172.21.15.52:6807/608216415","state":["exists","up"]},{"osd":1,"uuid":"d6a46b91-adc8-45ef-80f5-566ae400911f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6801","nonce":2538384087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6803","nonce":2538384087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6807","nonce":2538384087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6805","nonce":2538384087}]},"public_addr":"172.21.15.102:6801/2538384087","cluster_addr":"172.21.15.102:6803/2538384087","heartbeat_back_addr":"172.21.15.102:6807/2538384087","heartbeat_front_addr":"172.21.15.102:6805/2538384087","state":["exists","up"]},{"osd":2,"uuid":"846ba125-11f8-46b2-8941-bd5e1947cc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6801","nonce":4258038221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6803","nonce":4258038221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6807","nonce":4258038221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6805","nonce":4258038221}]},"public_addr":"172.21.15.150:6801/4258038221","cluster_addr":"172.21.15.150:6803/4258038221","heartbeat_back_addr":"172.21.15.150:6807/4258038221","heartbeat_front_addr":"172.21.15.150:6805/4258038221","state":["exists","up"]},{"osd":3,"uuid":"950fcd67-1057-4288-a389-714f1ec94261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6811","nonce":2622100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6813","nonce":2622100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6817","nonce":2622100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6815","nonce":2622100752}]},"public_addr":"172.21.15.52:6811/2622100752","cluster_addr":"172.21.15.52:6813/2622100752","heartbeat_back_addr":"172.21.15.52:6817/2622100752","heartbeat_front_addr":"172.21.15.52:6815/2622100752","state":["exists","up"]},{"osd":4,"uuid":"1c96a676-cea9-4deb-bad1-f7fd240c3d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6809","nonce":1546794763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6811","nonce":1546794763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6815","nonce":1546794763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6813","nonce":1546794763}]},"public_addr":"172.21.15.102:6809/1546794763","cluster_addr":"172.21.15.102:6811/1546794763","heartbeat_back_addr":"172.21.15.102:6815/1546794763","heartbeat_front_addr":"172.21.15.102:6813/1546794763","state":["exists","up"]},{"osd":5,"uuid":"38a75af1-fd4d-433e-b579-0609109c851c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6809","nonce":461511375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6811","nonce":461511375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6815","nonce":461511375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6813","nonce":461511375}]},"public_addr":"172.21.15.150:6809/461511375","cluster_addr":"172.21.15.150:6811/461511375","heartbeat_back_addr":"172.21.15.150:6815/461511375","heartbeat_front_addr":"172.21.15.150:6813/461511375","state":["exists","up"]},{"osd":6,"uuid":"2f0ede4a-d175-4b4a-b876-c0f442048901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6819","nonce":1002457464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6821","nonce":1002457464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6825","nonce":1002457464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6823","nonce":1002457464}]},"public_addr":"172.21.15.52:6819/1002457464","cluster_addr":"172.21.15.52:6821/1002457464","heartbeat_back_addr":"172.21.15.52:6825/1002457464","heartbeat_front_addr":"172.21.15.52:6823/1002457464","state":["exists","up"]},{"osd":7,"uuid":"6b635192-abcf-4bbf-bf78-a019b351bca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6817","nonce":3249840064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6819","nonce":3249840064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6823","nonce":3249840064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6821","nonce":3249840064}]},"public_addr":"172.21.15.102:6817/3249840064","cluster_addr":"172.21.15.102:6819/3249840064","heartbeat_back_addr":"172.21.15.102:6823/3249840064","heartbeat_front_addr":"172.21.15.102:6821/3249840064","state":["exists","up"]},{"osd":8,"uuid":"c7ca8007-7170-47bf-956c-3f00edf27e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6817","nonce":3672583574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6819","nonce":3672583574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6823","nonce":3672583574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6821","nonce":3672583574}]},"public_addr":"172.21.15.150:6817/3672583574","cluster_addr":"172.21.15.150:6819/3672583574","heartbeat_back_addr":"172.21.15.150:6823/3672583574","heartbeat_front_addr":"172.21.15.150:6821/3672583574","state":["exists","up"]},{"osd":9,"uuid":"0e252c2c-01a4-48ba-832d-067ed940fefb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6827","nonce":4287934320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6829","nonce":4287934320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6833","nonce":4287934320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6831","nonce":4287934320}]},"public_addr":"172.21.15.52:6827/4287934320","cluster_addr":"172.21.15.52:6829/4287934320","heartbeat_back_addr":"172.21.15.52:6833/4287934320","heartbeat_front_addr":"172.21.15.52:6831/4287934320","state":["exists","up"]},{"osd":10,"uuid":"bb540142-7563-44a4-b7a3-cb662714c128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6825","nonce":3056034201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6827","nonce":3056034201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6831","nonce":3056034201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6829","nonce":3056034201}]},"public_addr":"172.21.15.102:6825/3056034201","cluster_addr":"172.21.15.102:6827/3056034201","heartbeat_back_addr":"172.21.15.102:6831/3056034201","heartbeat_front_addr":"172.21.15.102:6829/3056034201","state":["exists","up"]},{"osd":11,"uuid":"3156528e-e0f8-4d44-9946-c8b6339c2bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6825","nonce":3207559739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6827","nonce":3207559739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6831","nonce":3207559739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6829","nonce":3207559739}]},"public_addr":"172.21.15.150:6825/3207559739","cluster_addr":"172.21.15.150:6827/3207559739","heartbeat_back_addr":"172.21.15.150:6831/3207559739","heartbeat_front_addr":"172.21.15.150:6829/3207559739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:26.826099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:48.331371+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:10.459738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:32.356472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:54.043170+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:15.648582+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:37.166463+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:59.240819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:20.710097+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:43.364563+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:05.279433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:26.344573+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":5,"to":7}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/4025191157":"2023-11-28T15:17:54.913525+0000","172.21.15.52:6801/2640036585":"2023-11-28T15:17:54.913525+0000","172.21.15.52:6800/2640036585":"2023-11-28T15:17:54.913525+0000","172.21.15.52:0/1392282110":"2023-11-28T15:10:03.644870+0000","172.21.15.52:0/4081249946":"2023-11-28T15:17:54.913525+0000","172.21.15.52:0/1479993479":"2023-11-28T15:10:03.644870+0000","172.21.15.52:0/2690561203":"2023-11-28T15:17:54.913525+0000","172.21.15.52:0/2454989397":"2023-11-28T15:17:54.913525+0000","172.21.15.52:6801/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6800/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/961562652":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/1290665390":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2410529742":"2023-11-28T15:09:37.118690+0000","172.21.15.52:6800/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2130037044":"2023-11-28T15:10:03.644870+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:28:42.676 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.673+0000 7fd96ac1a700 1 -- 172.21.15.52:0/2452862938 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd9540a12c0 msgr2=0x7fd9540a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:42.677 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.673+0000 7fd96ac1a700 1 --2- 172.21.15.52:0/2452862938 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd9540a12c0 0x7fd9540a3770 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd958009b50 tx=0x7fd958009550 comp rx=0 tx=0).stop 2023-11-27T15:28:42.677 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.673+0000 7fd96ac1a700 1 -- 172.21.15.52:0/2452862938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9641062d0 msgr2=0x7fd96419ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:42.677 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.673+0000 7fd96ac1a700 1 --2- 172.21.15.52:0/2452862938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9641062d0 0x7fd96419ade0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00b780 tx=0x7fd95c00bd60 comp rx=0 tx=0).stop 2023-11-27T15:28:42.677 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.673+0000 7fd96ac1a700 1 -- 172.21.15.52:0/2452862938 shutdown_connections 2023-11-27T15:28:42.677 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.673+0000 7fd96ac1a700 1 --2- 172.21.15.52:0/2452862938 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fd9540a12c0 0x7fd9540a3770 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:42.677 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.673+0000 7fd96ac1a700 1 --2- 172.21.15.52:0/2452862938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9641062d0 0x7fd96419ade0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:42.677 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.673+0000 7fd96ac1a700 1 --2- 172.21.15.52:0/2452862938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd964100a50 0x7fd964196440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:42.678 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.673+0000 7fd96ac1a700 1 --2- 172.21.15.52:0/2452862938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9641013e0 0x7fd964196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:42.678 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.673+0000 7fd96ac1a700 1 -- 172.21.15.52:0/2452862938 >> 172.21.15.52:0/2452862938 conn(0x7fd9640774a0 msgr2=0x7fd9640ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:42.678 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.673+0000 7fd96ac1a700 1 -- 172.21.15.52:0/2452862938 shutdown_connections 2023-11-27T15:28:42.678 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.673+0000 7fd96ac1a700 1 -- 172.21.15.52:0/2452862938 wait complete. 2023-11-27T15:28:42.689 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:12:14.177523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-27T15:16:07.842251+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-27T15:16:08.728520+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 95, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-27T15:28:42.690 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:28:42.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.818+0000 7f3763ecd700 1 -- 172.21.15.52:0/903435460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f375c0ffdf0 msgr2=0x7f375c103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:42.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.818+0000 7f3763ecd700 1 --2- 172.21.15.52:0/903435460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f375c0ffdf0 0x7f375c103db0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f374c0099c0 tx=0x7f374c009cd0 comp rx=0 tx=0).stop 2023-11-27T15:28:42.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.818+0000 7f3763ecd700 1 -- 172.21.15.52:0/903435460 shutdown_connections 2023-11-27T15:28:42.821 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.818+0000 7f3763ecd700 1 --2- 172.21.15.52:0/903435460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f375c104490 0x7f375c10c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:42.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.818+0000 7f3763ecd700 1 --2- 172.21.15.52:0/903435460 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f375c0ff450 0x7f375c0ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:42.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.818+0000 7f3763ecd700 1 --2- 172.21.15.52:0/903435460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f375c0ffdf0 0x7f375c103db0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:42.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.818+0000 7f3763ecd700 1 -- 172.21.15.52:0/903435460 >> 172.21.15.52:0/903435460 conn(0x7f375c0770b0 msgr2=0x7f375c0774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:42.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.818+0000 7f3763ecd700 1 -- 172.21.15.52:0/903435460 shutdown_connections 2023-11-27T15:28:42.822 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.818+0000 7f3763ecd700 1 -- 172.21.15.52:0/903435460 wait complete. 2023-11-27T15:28:42.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.818+0000 7f3763ecd700 1 Processor -- start 2023-11-27T15:28:42.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.818+0000 7f3763ecd700 1 -- start start 2023-11-27T15:28:42.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.819+0000 7f3763ecd700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f375c0ff450 0x7f375c100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:42.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.819+0000 7f3763ecd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f375c0ffdf0 0x7f375c101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:42.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.819+0000 7f3763ecd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f375c104490 0x7f375c1a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:42.824 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.819+0000 7f3763ecd700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c101bc0 con 0x7f375c0ffdf0 2023-11-27T15:28:42.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.819+0000 7f3763ecd700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c1a4b20 con 0x7f375c0ff450 2023-11-27T15:28:42.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.819+0000 7f3763ecd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c1a4c60 con 0x7f375c104490 2023-11-27T15:28:42.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.819+0000 7f3761468700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f375c0ffdf0 0x7f375c101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:42.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.819+0000 7f3761468700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f375c0ffdf0 0x7f375c101390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56504/0 (socket says 172.21.15.52:56504) 2023-11-27T15:28:42.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.819+0000 7f3761468700 1 -- 172.21.15.52:0/1077259876 learned_addr learned my addr 172.21.15.52:0/1077259876 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:42.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.819+0000 7f3761c69700 1 --2- 172.21.15.52:0/1077259876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f375c0ff450 0x7f375c100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:42.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.819+0000 7f376246a700 1 --2- 172.21.15.52:0/1077259876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f375c104490 0x7f375c1a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:42.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.819+0000 7f3761468700 1 -- 172.21.15.52:0/1077259876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f375c104490 msgr2=0x7f375c1a45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:42.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.819+0000 7f3761468700 1 --2- 172.21.15.52:0/1077259876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f375c104490 0x7f375c1a45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:42.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.819+0000 7f3761468700 1 -- 172.21.15.52:0/1077259876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f375c0ff450 msgr2=0x7f375c100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:42.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.819+0000 7f3761468700 1 --2- 172.21.15.52:0/1077259876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f375c0ff450 0x7f375c100e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:42.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.819+0000 7f3761468700 1 -- 172.21.15.52:0/1077259876 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374c0095f0 con 0x7f375c0ffdf0 2023-11-27T15:28:42.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.819+0000 7f376246a700 1 --2- 172.21.15.52:0/1077259876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f375c104490 0x7f375c1a45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:28:42.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.819+0000 7f3761468700 1 --2- 172.21.15.52:0/1077259876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f375c0ffdf0 0x7f375c101390 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f374c0098e0 tx=0x7f374c00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:42.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.819+0000 7f3752ffd700 1 -- 172.21.15.52:0/1077259876 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c01d070 con 0x7f375c0ffdf0 2023-11-27T15:28:42.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.819+0000 7f3752ffd700 1 -- 172.21.15.52:0/1077259876 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f374c00fc30 con 0x7f375c0ffdf0 2023-11-27T15:28:42.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.819+0000 7f3763ecd700 1 -- 172.21.15.52:0/1077259876 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f375c10f210 con 0x7f375c0ffdf0 2023-11-27T15:28:42.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.819+0000 7f3752ffd700 1 -- 172.21.15.52:0/1077259876 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c022ea0 con 0x7f375c0ffdf0 2023-11-27T15:28:42.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.819+0000 7f3763ecd700 1 -- 172.21.15.52:0/1077259876 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375c10f080 con 0x7f375c0ffdf0 2023-11-27T15:28:42.827 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.820+0000 7f3763ecd700 1 -- 172.21.15.52:0/1077259876 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f375c10f3a0 con 0x7f375c0ffdf0 2023-11-27T15:28:42.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.822+0000 7f3752ffd700 1 -- 172.21.15.52:0/1077259876 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f374c0050c0 con 0x7f375c0ffdf0 2023-11-27T15:28:42.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.823+0000 7f3752ffd700 1 --2- 172.21.15.52:0/1077259876 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f37480a12c0 0x7f37480a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:42.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.823+0000 7f3761c69700 1 --2- 172.21.15.52:0/1077259876 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f37480a12c0 0x7f37480a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:42.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.823+0000 7f3752ffd700 1 -- 172.21.15.52:0/1077259876 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f374c0cdae0 con 0x7f375c0ffdf0 2023-11-27T15:28:42.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.823+0000 7f3761c69700 1 --2- 172.21.15.52:0/1077259876 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f37480a12c0 0x7f37480a3770 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3758009bf0 tx=0x7f3758009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:42.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:42.826+0000 7f3752ffd700 1 -- 172.21.15.52:0/1077259876 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f374c096fe0 con 0x7f375c0ffdf0 2023-11-27T15:28:43.030 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.027+0000 7f3763ecd700 1 -- 172.21.15.52:0/1077259876 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f375c1a5510 con 0x7f375c0ffdf0 2023-11-27T15:28:43.031 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.027+0000 7f3752ffd700 1 -- 172.21.15.52:0/1077259876 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7f374c09a600 con 0x7f375c0ffdf0 2023-11-27T15:28:43.031 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2023-11-27T15:28:43.033 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.030+0000 7f3763ecd700 1 -- 172.21.15.52:0/1077259876 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f37480a12c0 msgr2=0x7f37480a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:43.033 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.030+0000 7f3763ecd700 1 --2- 172.21.15.52:0/1077259876 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f37480a12c0 0x7f37480a3770 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3758009bf0 tx=0x7f3758009260 comp rx=0 tx=0).stop 2023-11-27T15:28:43.033 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.030+0000 7f3763ecd700 1 -- 172.21.15.52:0/1077259876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f375c0ffdf0 msgr2=0x7f375c101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:43.034 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.030+0000 7f3763ecd700 1 --2- 172.21.15.52:0/1077259876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f375c0ffdf0 0x7f375c101390 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f374c0098e0 tx=0x7f374c00f740 comp rx=0 tx=0).stop 2023-11-27T15:28:43.034 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.030+0000 7f3763ecd700 1 -- 172.21.15.52:0/1077259876 shutdown_connections 2023-11-27T15:28:43.034 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.030+0000 7f3763ecd700 1 --2- 172.21.15.52:0/1077259876 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f37480a12c0 0x7f37480a3770 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.034 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.030+0000 7f3763ecd700 1 --2- 172.21.15.52:0/1077259876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f375c0ff450 0x7f375c100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.034 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.030+0000 7f3763ecd700 1 --2- 172.21.15.52:0/1077259876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f375c104490 0x7f375c1a45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.034 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.030+0000 7f3763ecd700 1 --2- 172.21.15.52:0/1077259876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f375c0ffdf0 0x7f375c101390 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.034 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.030+0000 7f3763ecd700 1 -- 172.21.15.52:0/1077259876 >> 172.21.15.52:0/1077259876 conn(0x7f375c0770b0 msgr2=0x7f375c067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:43.034 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.030+0000 7f3763ecd700 1 -- 172.21.15.52:0/1077259876 shutdown_connections 2023-11-27T15:28:43.035 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.030+0000 7f3763ecd700 1 -- 172.21.15.52:0/1077259876 wait complete. 2023-11-27T15:28:43.045 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-27T15:28:43.173 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.170+0000 7f712c44e700 1 -- 172.21.15.52:0/1050837899 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7124105260 msgr2=0x7f7124105630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:43.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.170+0000 7f712c44e700 1 --2- 172.21.15.52:0/1050837899 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7124105260 0x7f7124105630 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7120009990 tx=0x7f7120009ca0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.170+0000 7f712c44e700 1 -- 172.21.15.52:0/1050837899 shutdown_connections 2023-11-27T15:28:43.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.170+0000 7f712c44e700 1 --2- 172.21.15.52:0/1050837899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71240ff1a0 0x7f712410d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.170+0000 7f712c44e700 1 --2- 172.21.15.52:0/1050837899 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7124105260 0x7f7124105630 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.170+0000 7f712c44e700 1 --2- 172.21.15.52:0/1050837899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71240fe7f0 0x7f71240fec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.170+0000 7f712c44e700 1 -- 172.21.15.52:0/1050837899 >> 172.21.15.52:0/1050837899 conn(0x7f7124075890 msgr2=0x7f7124075c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:43.174 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.170+0000 7f712c44e700 1 -- 172.21.15.52:0/1050837899 shutdown_connections 2023-11-27T15:28:43.175 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.170+0000 7f712c44e700 1 -- 172.21.15.52:0/1050837899 wait complete. 2023-11-27T15:28:43.176 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.171+0000 7f712c44e700 1 Processor -- start 2023-11-27T15:28:43.176 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.171+0000 7f712c44e700 1 -- start start 2023-11-27T15:28:43.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.171+0000 7f712c44e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71240fe7f0 0x7f7124196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:43.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.171+0000 7f712c44e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71240ff1a0 0x7f71241968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:43.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.171+0000 7f712c44e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7124105260 0x7f712419ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:43.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.171+0000 7f712c44e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712419b360 con 0x7f71240ff1a0 2023-11-27T15:28:43.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.171+0000 7f712c44e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712419b4d0 con 0x7f7124105260 2023-11-27T15:28:43.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.171+0000 7f712c44e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712419b690 con 0x7f71240fe7f0 2023-11-27T15:28:43.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.171+0000 7f71299e9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71240ff1a0 0x7f71241968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:43.177 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.171+0000 7f71299e9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71240ff1a0 0x7f71241968c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56518/0 (socket says 172.21.15.52:56518) 2023-11-27T15:28:43.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.171+0000 7f71299e9700 1 -- 172.21.15.52:0/3634817008 learned_addr learned my addr 172.21.15.52:0/3634817008 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:43.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.171+0000 7f712a9eb700 1 --2- 172.21.15.52:0/3634817008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7124105260 0x7f712419ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:43.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.171+0000 7f712a1ea700 1 --2- 172.21.15.52:0/3634817008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71240fe7f0 0x7f7124196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:43.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.172+0000 7f71299e9700 1 -- 172.21.15.52:0/3634817008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71240fe7f0 msgr2=0x7f7124196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:43.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.172+0000 7f71299e9700 1 --2- 172.21.15.52:0/3634817008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71240fe7f0 0x7f7124196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.172+0000 7f71299e9700 1 -- 172.21.15.52:0/3634817008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7124105260 msgr2=0x7f712419ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:43.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.172+0000 7f71299e9700 1 --2- 172.21.15.52:0/3634817008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7124105260 0x7f712419ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.178 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.172+0000 7f71299e9700 1 -- 172.21.15.52:0/3634817008 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71200095f0 con 0x7f71240ff1a0 2023-11-27T15:28:43.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.172+0000 7f71299e9700 1 --2- 172.21.15.52:0/3634817008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71240ff1a0 0x7f71241968c0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f711800c9b0 tx=0x7f711800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:43.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.172+0000 7f71177fe700 1 -- 172.21.15.52:0/3634817008 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7118007b00 con 0x7f71240ff1a0 2023-11-27T15:28:43.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.172+0000 7f71177fe700 1 -- 172.21.15.52:0/3634817008 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7118007c60 con 0x7f71240ff1a0 2023-11-27T15:28:43.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.172+0000 7f712c44e700 1 -- 172.21.15.52:0/3634817008 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7124110420 con 0x7f71240ff1a0 2023-11-27T15:28:43.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.172+0000 7f71177fe700 1 -- 172.21.15.52:0/3634817008 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7118018660 con 0x7f71240ff1a0 2023-11-27T15:28:43.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.172+0000 7f712c44e700 1 -- 172.21.15.52:0/3634817008 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7124110290 con 0x7f71240ff1a0 2023-11-27T15:28:43.179 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.173+0000 7f712c44e700 1 -- 172.21.15.52:0/3634817008 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71241105b0 con 0x7f71240ff1a0 2023-11-27T15:28:43.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.175+0000 7f71177fe700 1 -- 172.21.15.52:0/3634817008 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f711800f450 con 0x7f71240ff1a0 2023-11-27T15:28:43.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.175+0000 7f71177fe700 1 --2- 172.21.15.52:0/3634817008 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f71100a12c0 0x7f71100a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:43.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.175+0000 7f71177fe700 1 -- 172.21.15.52:0/3634817008 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f71180ccf70 con 0x7f71240ff1a0 2023-11-27T15:28:43.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.175+0000 7f712a1ea700 1 --2- 172.21.15.52:0/3634817008 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f71100a12c0 0x7f71100a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:43.180 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.176+0000 7f712a1ea700 1 --2- 172.21.15.52:0/3634817008 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f71100a12c0 0x7f71100a3770 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f712000b5c0 tx=0x7f7120005fb0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:43.182 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.179+0000 7f71177fe700 1 -- 172.21.15.52:0/3634817008 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f71180963e0 con 0x7f71240ff1a0 2023-11-27T15:28:43.365 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:43 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2452862938' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:28:43.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:43 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1077259876' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:28:43.382 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.379+0000 7f712c44e700 1 -- 172.21.15.52:0/3634817008 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f71241a3450 con 0x7f71240ff1a0 2023-11-27T15:28:43.383 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.380+0000 7f71177fe700 1 -- 172.21.15.52:0/3634817008 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7118099a00 con 0x7f71240ff1a0 2023-11-27T15:28:43.383 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2023-11-27T15:28:43.385 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.382+0000 7f712c44e700 1 -- 172.21.15.52:0/3634817008 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f71100a12c0 msgr2=0x7f71100a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:43.385 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.382+0000 7f712c44e700 1 --2- 172.21.15.52:0/3634817008 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f71100a12c0 0x7f71100a3770 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f712000b5c0 tx=0x7f7120005fb0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.386 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.382+0000 7f712c44e700 1 -- 172.21.15.52:0/3634817008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71240ff1a0 msgr2=0x7f71241968c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:43.386 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.382+0000 7f712c44e700 1 --2- 172.21.15.52:0/3634817008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71240ff1a0 0x7f71241968c0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f711800c9b0 tx=0x7f711800ccc0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.386 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.382+0000 7f712c44e700 1 -- 172.21.15.52:0/3634817008 shutdown_connections 2023-11-27T15:28:43.386 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.382+0000 7f712c44e700 1 --2- 172.21.15.52:0/3634817008 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f71100a12c0 0x7f71100a3770 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.386 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.382+0000 7f712c44e700 1 --2- 172.21.15.52:0/3634817008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7124105260 0x7f712419ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.386 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.382+0000 7f712c44e700 1 --2- 172.21.15.52:0/3634817008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71240fe7f0 0x7f7124196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.386 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.382+0000 7f712c44e700 1 --2- 172.21.15.52:0/3634817008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71240ff1a0 0x7f71241968c0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.386 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.382+0000 7f712c44e700 1 -- 172.21.15.52:0/3634817008 >> 172.21.15.52:0/3634817008 conn(0x7f7124075890 msgr2=0x7f71240fc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:43.387 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.382+0000 7f712c44e700 1 -- 172.21.15.52:0/3634817008 shutdown_connections 2023-11-27T15:28:43.387 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.382+0000 7f712c44e700 1 -- 172.21.15.52:0/3634817008 wait complete. 2023-11-27T15:28:43.397 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-27T15:28:43.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.522+0000 7f0588b17700 1 -- 172.21.15.52:0/2815941777 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05840ffdf0 msgr2=0x7f0584103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:43.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.522+0000 7f0588b17700 1 --2- 172.21.15.52:0/2815941777 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05840ffdf0 0x7f0584103db0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f056c0099c0 tx=0x7f056c009cd0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.522+0000 7f0588b17700 1 -- 172.21.15.52:0/2815941777 shutdown_connections 2023-11-27T15:28:43.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.522+0000 7f0588b17700 1 --2- 172.21.15.52:0/2815941777 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05840ff450 0x7f05840ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.522+0000 7f0588b17700 1 --2- 172.21.15.52:0/2815941777 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0584104490 0x7f058410c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.526 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.522+0000 7f0588b17700 1 --2- 172.21.15.52:0/2815941777 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05840ffdf0 0x7f0584103db0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.522+0000 7f0588b17700 1 -- 172.21.15.52:0/2815941777 >> 172.21.15.52:0/2815941777 conn(0x7f05840770b0 msgr2=0x7f05840774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:43.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.522+0000 7f0588b17700 1 -- 172.21.15.52:0/2815941777 shutdown_connections 2023-11-27T15:28:43.527 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.522+0000 7f0588b17700 1 -- 172.21.15.52:0/2815941777 wait complete. 2023-11-27T15:28:43.527 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:43 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2452862938' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:28:43.528 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:43 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1077259876' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:28:43.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.523+0000 7f0588b17700 1 Processor -- start 2023-11-27T15:28:43.529 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.523+0000 7f0588b17700 1 -- start start 2023-11-27T15:28:43.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.523+0000 7f0588b17700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05840ff450 0x7f0584197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:43.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.523+0000 7f0588b17700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05840ffdf0 0x7f0584197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:43.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.523+0000 7f0588b17700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0584104490 0x7f05841a2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:43.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.523+0000 7f0588b17700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05841a28d0 con 0x7f05840ffdf0 2023-11-27T15:28:43.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.523+0000 7f0588b17700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05841a2a10 con 0x7f05840ff450 2023-11-27T15:28:43.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.523+0000 7f0588b17700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05841a2b50 con 0x7f0584104490 2023-11-27T15:28:43.530 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.523+0000 7f0581d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05840ffdf0 0x7f0584197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:43.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.523+0000 7f0581d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05840ffdf0 0x7f0584197850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56532/0 (socket says 172.21.15.52:56532) 2023-11-27T15:28:43.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.523+0000 7f0581d9b700 1 -- 172.21.15.52:0/2488734854 learned_addr learned my addr 172.21.15.52:0/2488734854 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:43.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.523+0000 7f058259c700 1 --2- 172.21.15.52:0/2488734854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05840ff450 0x7f0584197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:43.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.523+0000 7f0582d9d700 1 --2- 172.21.15.52:0/2488734854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0584104490 0x7f05841a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:43.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.524+0000 7f0581d9b700 1 -- 172.21.15.52:0/2488734854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0584104490 msgr2=0x7f05841a2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:43.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.524+0000 7f0581d9b700 1 --2- 172.21.15.52:0/2488734854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0584104490 0x7f05841a2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.524+0000 7f0581d9b700 1 -- 172.21.15.52:0/2488734854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05840ff450 msgr2=0x7f0584197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:43.531 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.524+0000 7f0581d9b700 1 --2- 172.21.15.52:0/2488734854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05840ff450 0x7f0584197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.524+0000 7f0581d9b700 1 -- 172.21.15.52:0/2488734854 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056c0095f0 con 0x7f05840ffdf0 2023-11-27T15:28:43.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.524+0000 7f0582d9d700 1 --2- 172.21.15.52:0/2488734854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0584104490 0x7f05841a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:28:43.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.524+0000 7f0581d9b700 1 --2- 172.21.15.52:0/2488734854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05840ffdf0 0x7f0584197850 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f056c009990 tx=0x7f056c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:43.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.524+0000 7f057b7fe700 1 -- 172.21.15.52:0/2488734854 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c01d070 con 0x7f05840ffdf0 2023-11-27T15:28:43.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.524+0000 7f057b7fe700 1 -- 172.21.15.52:0/2488734854 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f056c0054e0 con 0x7f05840ffdf0 2023-11-27T15:28:43.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.524+0000 7f0588b17700 1 -- 172.21.15.52:0/2488734854 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f058410f260 con 0x7f05840ffdf0 2023-11-27T15:28:43.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.524+0000 7f057b7fe700 1 -- 172.21.15.52:0/2488734854 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c004a70 con 0x7f05840ffdf0 2023-11-27T15:28:43.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.524+0000 7f0588b17700 1 -- 172.21.15.52:0/2488734854 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f058410f0d0 con 0x7f05840ffdf0 2023-11-27T15:28:43.532 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.525+0000 7f0588b17700 1 -- 172.21.15.52:0/2488734854 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f058410f3f0 con 0x7f05840ffdf0 2023-11-27T15:28:43.533 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.526+0000 7f057b7fe700 1 -- 172.21.15.52:0/2488734854 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f056c00fb80 con 0x7f05840ffdf0 2023-11-27T15:28:43.533 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.527+0000 7f057b7fe700 1 --2- 172.21.15.52:0/2488734854 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f05700a1340 0x7f05700a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:43.533 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.527+0000 7f058259c700 1 --2- 172.21.15.52:0/2488734854 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f05700a1340 0x7f05700a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:43.533 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.527+0000 7f057b7fe700 1 -- 172.21.15.52:0/2488734854 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f056c0cd960 con 0x7f05840ffdf0 2023-11-27T15:28:43.534 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.527+0000 7f058259c700 1 --2- 172.21.15.52:0/2488734854 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f05700a1340 0x7f05700a37f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0574009c60 tx=0x7f0574009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:43.534 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.530+0000 7f057b7fe700 1 -- 172.21.15.52:0/2488734854 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f056c096d70 con 0x7f05840ffdf0 2023-11-27T15:28:43.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:43 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2452862938' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:28:43.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:43 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1077259876' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:28:43.730 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.727+0000 7f0588b17700 1 -- 172.21.15.52:0/2488734854 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f05840684b0 con 0x7f05840ffdf0 2023-11-27T15:28:43.731 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.727+0000 7f057b7fe700 1 -- 172.21.15.52:0/2488734854 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+11 (secure 0 0 0) 0x7f056c09a390 con 0x7f05840ffdf0 2023-11-27T15:28:43.731 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2023-11-27T15:28:43.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.729+0000 7f0588b17700 1 -- 172.21.15.52:0/2488734854 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f05700a1340 msgr2=0x7f05700a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:43.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.729+0000 7f0588b17700 1 --2- 172.21.15.52:0/2488734854 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f05700a1340 0x7f05700a37f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0574009c60 tx=0x7f0574009260 comp rx=0 tx=0).stop 2023-11-27T15:28:43.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.729+0000 7f0588b17700 1 -- 172.21.15.52:0/2488734854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05840ffdf0 msgr2=0x7f0584197850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:43.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.729+0000 7f0588b17700 1 --2- 172.21.15.52:0/2488734854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05840ffdf0 0x7f0584197850 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f056c009990 tx=0x7f056c00f690 comp rx=0 tx=0).stop 2023-11-27T15:28:43.733 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.729+0000 7f0588b17700 1 -- 172.21.15.52:0/2488734854 shutdown_connections 2023-11-27T15:28:43.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.729+0000 7f0588b17700 1 --2- 172.21.15.52:0/2488734854 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f05700a1340 0x7f05700a37f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.729+0000 7f0588b17700 1 --2- 172.21.15.52:0/2488734854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f05840ff450 0x7f0584197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.729+0000 7f0588b17700 1 --2- 172.21.15.52:0/2488734854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0584104490 0x7f05841a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.729+0000 7f0588b17700 1 --2- 172.21.15.52:0/2488734854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05840ffdf0 0x7f0584197850 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.729+0000 7f0588b17700 1 -- 172.21.15.52:0/2488734854 >> 172.21.15.52:0/2488734854 conn(0x7f05840770b0 msgr2=0x7f05840fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:43.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.730+0000 7f0588b17700 1 -- 172.21.15.52:0/2488734854 shutdown_connections 2023-11-27T15:28:43.734 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.730+0000 7f0588b17700 1 -- 172.21.15.52:0/2488734854 wait complete. 2023-11-27T15:28:43.745 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-27T15:28:43.872 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.868+0000 7fef4b58f700 1 -- 172.21.15.52:0/3278484208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef44101c70 msgr2=0x7fef4410a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:43.872 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.868+0000 7fef4b58f700 1 --2- 172.21.15.52:0/3278484208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef44101c70 0x7fef4410a340 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fef40009990 tx=0x7fef40009ca0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.873 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.868+0000 7fef4b58f700 1 -- 172.21.15.52:0/3278484208 shutdown_connections 2023-11-27T15:28:43.873 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.868+0000 7fef4b58f700 1 --2- 172.21.15.52:0/3278484208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef4410a880 0x7fef4410cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.873 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.868+0000 7fef4b58f700 1 --2- 172.21.15.52:0/3278484208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef44101360 0x7fef44101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.873 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.868+0000 7fef4b58f700 1 --2- 172.21.15.52:0/3278484208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef44101c70 0x7fef4410a340 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.873 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.868+0000 7fef4b58f700 1 -- 172.21.15.52:0/3278484208 >> 172.21.15.52:0/3278484208 conn(0x7fef44075fd0 msgr2=0x7fef440763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:43.873 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.869+0000 7fef4b58f700 1 -- 172.21.15.52:0/3278484208 shutdown_connections 2023-11-27T15:28:43.873 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.869+0000 7fef4b58f700 1 -- 172.21.15.52:0/3278484208 wait complete. 2023-11-27T15:28:43.875 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.869+0000 7fef4b58f700 1 Processor -- start 2023-11-27T15:28:43.875 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.869+0000 7fef4b58f700 1 -- start start 2023-11-27T15:28:43.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.870+0000 7fef4b58f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef44101360 0x7fef44196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:43.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.870+0000 7fef4b58f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef44101c70 0x7fef441968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:43.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.870+0000 7fef4b58f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef4410a880 0x7fef4419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:43.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.870+0000 7fef4b58f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4419b350 con 0x7fef44101360 2023-11-27T15:28:43.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.870+0000 7fef4b58f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4419b4c0 con 0x7fef4410a880 2023-11-27T15:28:43.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.870+0000 7fef4b58f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4419b680 con 0x7fef44101c70 2023-11-27T15:28:43.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.870+0000 7fef4932b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef44101360 0x7fef44196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:43.876 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.870+0000 7fef4932b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef44101360 0x7fef44196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56542/0 (socket says 172.21.15.52:56542) 2023-11-27T15:28:43.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.870+0000 7fef4932b700 1 -- 172.21.15.52:0/65181213 learned_addr learned my addr 172.21.15.52:0/65181213 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:43.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.870+0000 7fef48b2a700 1 --2- 172.21.15.52:0/65181213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef44101c70 0x7fef441968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:43.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.870+0000 7fef49b2c700 1 --2- 172.21.15.52:0/65181213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef4410a880 0x7fef4419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:43.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.870+0000 7fef4932b700 1 -- 172.21.15.52:0/65181213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef44101c70 msgr2=0x7fef441968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:43.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.870+0000 7fef4932b700 1 --2- 172.21.15.52:0/65181213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef44101c70 0x7fef441968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.870+0000 7fef4932b700 1 -- 172.21.15.52:0/65181213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef4410a880 msgr2=0x7fef4419ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:43.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.870+0000 7fef4932b700 1 --2- 172.21.15.52:0/65181213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef4410a880 0x7fef4419ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:43.877 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.870+0000 7fef4932b700 1 -- 172.21.15.52:0/65181213 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef400095f0 con 0x7fef44101360 2023-11-27T15:28:43.878 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.870+0000 7fef49b2c700 1 --2- 172.21.15.52:0/65181213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef4410a880 0x7fef4419ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:28:43.878 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.870+0000 7fef4932b700 1 --2- 172.21.15.52:0/65181213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef44101360 0x7fef44196370 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fef3400e9a0 tx=0x7fef3400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:43.878 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.871+0000 7fef3a7fc700 1 -- 172.21.15.52:0/65181213 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef34009790 con 0x7fef44101360 2023-11-27T15:28:43.878 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.871+0000 7fef3a7fc700 1 -- 172.21.15.52:0/65181213 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef3400cbd0 con 0x7fef44101360 2023-11-27T15:28:43.878 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.871+0000 7fef4b58f700 1 -- 172.21.15.52:0/65181213 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef44110300 con 0x7fef44101360 2023-11-27T15:28:43.878 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.871+0000 7fef3a7fc700 1 -- 172.21.15.52:0/65181213 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef34018680 con 0x7fef44101360 2023-11-27T15:28:43.878 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.871+0000 7fef4b58f700 1 -- 172.21.15.52:0/65181213 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef44110170 con 0x7fef44101360 2023-11-27T15:28:43.878 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.872+0000 7fef4b58f700 1 -- 172.21.15.52:0/65181213 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef44110490 con 0x7fef44101360 2023-11-27T15:28:43.879 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.873+0000 7fef3a7fc700 1 -- 172.21.15.52:0/65181213 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fef3400cd40 con 0x7fef44101360 2023-11-27T15:28:43.879 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.873+0000 7fef3a7fc700 1 --2- 172.21.15.52:0/65181213 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fef300a1340 0x7fef300a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:43.880 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.874+0000 7fef48b2a700 1 --2- 172.21.15.52:0/65181213 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fef300a1340 0x7fef300a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:43.880 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.874+0000 7fef3a7fc700 1 -- 172.21.15.52:0/65181213 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fef34014070 con 0x7fef44101360 2023-11-27T15:28:43.880 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.874+0000 7fef48b2a700 1 --2- 172.21.15.52:0/65181213 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fef300a1340 0x7fef300a37f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fef44197980 tx=0x7fef40005fd0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:43.880 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:43.877+0000 7fef3a7fc700 1 -- 172.21.15.52:0/65181213 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fef340965d0 con 0x7fef44101360 2023-11-27T15:28:44.100 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.097+0000 7fef4b58f700 1 -- 172.21.15.52:0/65181213 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fef4419bb50 con 0x7fef44101360 2023-11-27T15:28:44.101 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.097+0000 7fef3a7fc700 1 -- 172.21.15.52:0/65181213 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v96) v1 ==== 81+0+943 (secure 0 0 0) 0x7fef34026080 con 0x7fef44101360 2023-11-27T15:28:44.101 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/4025191157 2023-11-28T15:17:54.913525+0000 2023-11-27T15:28:44.101 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6801/2640036585 2023-11-28T15:17:54.913525+0000 2023-11-27T15:28:44.101 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6800/2640036585 2023-11-28T15:17:54.913525+0000 2023-11-27T15:28:44.101 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/1392282110 2023-11-28T15:10:03.644870+0000 2023-11-27T15:28:44.101 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/4081249946 2023-11-28T15:17:54.913525+0000 2023-11-27T15:28:44.102 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/1479993479 2023-11-28T15:10:03.644870+0000 2023-11-27T15:28:44.102 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/2690561203 2023-11-28T15:17:54.913525+0000 2023-11-27T15:28:44.102 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/2454989397 2023-11-28T15:17:54.913525+0000 2023-11-27T15:28:44.102 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6801/3470359117 2023-11-28T15:10:03.644870+0000 2023-11-27T15:28:44.102 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6800/3470359117 2023-11-28T15:10:03.644870+0000 2023-11-27T15:28:44.102 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6801/960704660 2023-11-28T15:09:37.118690+0000 2023-11-27T15:28:44.103 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/961562652 2023-11-28T15:09:37.118690+0000 2023-11-27T15:28:44.103 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/1290665390 2023-11-28T15:09:37.118690+0000 2023-11-27T15:28:44.103 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/2410529742 2023-11-28T15:09:37.118690+0000 2023-11-27T15:28:44.103 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6800/960704660 2023-11-28T15:09:37.118690+0000 2023-11-27T15:28:44.103 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/2130037044 2023-11-28T15:10:03.644870+0000 2023-11-27T15:28:44.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.099+0000 7fef4b58f700 1 -- 172.21.15.52:0/65181213 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fef300a1340 msgr2=0x7fef300a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:44.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.099+0000 7fef4b58f700 1 --2- 172.21.15.52:0/65181213 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fef300a1340 0x7fef300a37f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fef44197980 tx=0x7fef40005fd0 comp rx=0 tx=0).stop 2023-11-27T15:28:44.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.099+0000 7fef4b58f700 1 -- 172.21.15.52:0/65181213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef44101360 msgr2=0x7fef44196370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:44.104 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.099+0000 7fef4b58f700 1 --2- 172.21.15.52:0/65181213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef44101360 0x7fef44196370 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fef3400e9a0 tx=0x7fef3400ecb0 comp rx=0 tx=0).stop 2023-11-27T15:28:44.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.099+0000 7fef4b58f700 1 -- 172.21.15.52:0/65181213 shutdown_connections 2023-11-27T15:28:44.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.100+0000 7fef4b58f700 1 --2- 172.21.15.52:0/65181213 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fef300a1340 0x7fef300a37f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:44.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.100+0000 7fef4b58f700 1 --2- 172.21.15.52:0/65181213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef4410a880 0x7fef4419ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:44.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.100+0000 7fef4b58f700 1 --2- 172.21.15.52:0/65181213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef44101360 0x7fef44196370 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:44.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.100+0000 7fef4b58f700 1 --2- 172.21.15.52:0/65181213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef44101c70 0x7fef441968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:44.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.100+0000 7fef4b58f700 1 -- 172.21.15.52:0/65181213 >> 172.21.15.52:0/65181213 conn(0x7fef44075fd0 msgr2=0x7fef44104390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:44.105 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.100+0000 7fef4b58f700 1 -- 172.21.15.52:0/65181213 shutdown_connections 2023-11-27T15:28:44.106 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.100+0000 7fef4b58f700 1 -- 172.21.15.52:0/65181213 wait complete. 2023-11-27T15:28:44.106 INFO:teuthology.orchestra.run.smithi052.stderr:listed 16 entries 2023-11-27T15:28:44.115 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-27T15:28:44.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:28:44.116 DEBUG:teuthology.orchestra.run.smithi052:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-27T15:28:44.188 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2023-11-27T15:28:44.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:28:44.188 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-27T15:28:44.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:28:44.297 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2023-11-27T15:28:44.365 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-27T15:28:44.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:28:44.366 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-27T15:28:44.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:44 smithi052 ceph-mon[99056]: pgmap v337: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:28:44.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:44 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3634817008' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:28:44.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:44 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2488734854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:28:44.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:44 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/65181213' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-27T15:28:44.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:28:44.412 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link delete ceph-brx 2023-11-27T15:28:44.520 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-11-27T15:28:44.530 INFO:tasks.cephadm:Teardown begin 2023-11-27T15:28:44.530 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-27T15:28:44.530 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:28:44.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:44 smithi102 ceph-mon[100296]: pgmap v337: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:28:44.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:44 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3634817008' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:28:44.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:44 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2488734854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:28:44.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:44 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/65181213' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-27T15:28:44.677 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.673+0000 7fc46069a700 1 -- 172.21.15.52:0/934058787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc458068ed0 msgr2=0x7fc4580692a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:44.677 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.673+0000 7fc46069a700 1 --2- 172.21.15.52:0/934058787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc458068ed0 0x7fc4580692a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc448009990 tx=0x7fc448009ca0 comp rx=0 tx=0).stop 2023-11-27T15:28:44.677 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.673+0000 7fc46069a700 1 -- 172.21.15.52:0/934058787 shutdown_connections 2023-11-27T15:28:44.677 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.673+0000 7fc46069a700 1 --2- 172.21.15.52:0/934058787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc458068ed0 0x7fc4580692a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:44.677 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.673+0000 7fc46069a700 1 --2- 172.21.15.52:0/934058787 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc45810cce0 0x7fc45810f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:44.677 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.673+0000 7fc46069a700 1 --2- 172.21.15.52:0/934058787 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4580697e0 0x7fc45810c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:44.677 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.673+0000 7fc46069a700 1 -- 172.21.15.52:0/934058787 >> 172.21.15.52:0/934058787 conn(0x7fc458076100 msgr2=0x7fc458076500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:44.678 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.673+0000 7fc46069a700 1 -- 172.21.15.52:0/934058787 shutdown_connections 2023-11-27T15:28:44.678 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.673+0000 7fc46069a700 1 -- 172.21.15.52:0/934058787 wait complete. 2023-11-27T15:28:44.680 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.674+0000 7fc46069a700 1 Processor -- start 2023-11-27T15:28:44.680 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.674+0000 7fc46069a700 1 -- start start 2023-11-27T15:28:44.680 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.674+0000 7fc46069a700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc458068ed0 0x7fc458198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:44.680 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.675+0000 7fc46069a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4580697e0 0x7fc458198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:44.680 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.675+0000 7fc46069a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc45810cce0 0x7fc45819cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:44.680 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.675+0000 7fc46069a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45819d500 con 0x7fc45810cce0 2023-11-27T15:28:44.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.675+0000 7fc46069a700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45819d670 con 0x7fc458068ed0 2023-11-27T15:28:44.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.675+0000 7fc46069a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45819d830 con 0x7fc4580697e0 2023-11-27T15:28:44.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.675+0000 7fc45ec37700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc45810cce0 0x7fc45819cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:44.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.675+0000 7fc45e436700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc458068ed0 0x7fc458198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:44.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.675+0000 7fc45ec37700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc45810cce0 0x7fc45819cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56560/0 (socket says 172.21.15.52:56560) 2023-11-27T15:28:44.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.675+0000 7fc45ec37700 1 -- 172.21.15.52:0/1932282196 learned_addr learned my addr 172.21.15.52:0/1932282196 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:44.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.675+0000 7fc45dc35700 1 --2- 172.21.15.52:0/1932282196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4580697e0 0x7fc458198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:44.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.675+0000 7fc45ec37700 1 -- 172.21.15.52:0/1932282196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4580697e0 msgr2=0x7fc458198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:44.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.675+0000 7fc45ec37700 1 --2- 172.21.15.52:0/1932282196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4580697e0 0x7fc458198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:44.682 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.675+0000 7fc45ec37700 1 -- 172.21.15.52:0/1932282196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc458068ed0 msgr2=0x7fc458198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:44.682 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.675+0000 7fc45ec37700 1 --2- 172.21.15.52:0/1932282196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc458068ed0 0x7fc458198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:44.682 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.675+0000 7fc45ec37700 1 -- 172.21.15.52:0/1932282196 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4480095f0 con 0x7fc45810cce0 2023-11-27T15:28:44.682 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.675+0000 7fc45dc35700 1 --2- 172.21.15.52:0/1932282196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4580697e0 0x7fc458198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:28:44.682 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.675+0000 7fc45e436700 1 --2- 172.21.15.52:0/1932282196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc458068ed0 0x7fc458198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:28:44.682 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.675+0000 7fc45ec37700 1 --2- 172.21.15.52:0/1932282196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc45810cce0 0x7fc45819cec0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fc45000ef30 tx=0x7fc45000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:44.682 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.675+0000 7fc44f7fe700 1 -- 172.21.15.52:0/1932282196 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc450018070 con 0x7fc45810cce0 2023-11-27T15:28:44.683 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.675+0000 7fc46069a700 1 -- 172.21.15.52:0/1932282196 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4581124b0 con 0x7fc45810cce0 2023-11-27T15:28:44.683 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.675+0000 7fc44f7fe700 1 -- 172.21.15.52:0/1932282196 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc45000f040 con 0x7fc45810cce0 2023-11-27T15:28:44.683 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.675+0000 7fc44f7fe700 1 -- 172.21.15.52:0/1932282196 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45000cdb0 con 0x7fc45810cce0 2023-11-27T15:28:44.683 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.675+0000 7fc46069a700 1 -- 172.21.15.52:0/1932282196 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc458112320 con 0x7fc45810cce0 2023-11-27T15:28:44.683 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.676+0000 7fc46069a700 1 -- 172.21.15.52:0/1932282196 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc458112640 con 0x7fc45810cce0 2023-11-27T15:28:44.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.678+0000 7fc44f7fe700 1 -- 172.21.15.52:0/1932282196 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fc450003a00 con 0x7fc45810cce0 2023-11-27T15:28:44.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.678+0000 7fc44f7fe700 1 --2- 172.21.15.52:0/1932282196 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fc4440a12c0 0x7fc4440a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:44.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.678+0000 7fc44f7fe700 1 -- 172.21.15.52:0/1932282196 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8571+0+0 (secure 0 0 0) 0x7fc4500cd280 con 0x7fc45810cce0 2023-11-27T15:28:44.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.678+0000 7fc45e436700 1 --2- 172.21.15.52:0/1932282196 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fc4440a12c0 0x7fc4440a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:44.684 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.679+0000 7fc45e436700 1 --2- 172.21.15.52:0/1932282196 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fc4440a12c0 0x7fc4440a3770 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc44800b2e0 tx=0x7fc448005fd0 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:44.685 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.682+0000 7fc44f7fe700 1 -- 172.21.15.52:0/1932282196 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc4500966f0 con 0x7fc45810cce0 2023-11-27T15:28:44.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:44 smithi150 ceph-mon[100559]: pgmap v337: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:28:44.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:44 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3634817008' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:28:44.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:44 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2488734854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:28:44.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:44 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/65181213' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-27T15:28:44.931 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.928+0000 7fc46069a700 1 -- 172.21.15.52:0/1932282196 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4580671d0 con 0x7fc45810cce0 2023-11-27T15:28:44.933 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.929+0000 7fc44f7fe700 1 -- 172.21.15.52:0/1932282196 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10804 (secure 0 0 0) 0x7fc450099d10 con 0x7fc45810cce0 2023-11-27T15:28:44.933 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:28:44.933 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2328881376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2328881376},{"type":"v1","addr":"172.21.15.52:6839","nonce":2328881376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2799551570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2799551570},{"type":"v1","addr":"172.21.15.52:6841","nonce":2799551570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:16:09.319630+0000","modified":"2023-11-27T15:26:15.243744+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24494,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/215725838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":215725838},{"type":"v1","addr":"172.21.15.150:6839","nonce":215725838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3571830559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3571830559},{"type":"v1","addr":"172.21.15.102:6839","nonce":3571830559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:28:44.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.931+0000 7fc46069a700 1 -- 172.21.15.52:0/1932282196 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fc4440a12c0 msgr2=0x7fc4440a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:44.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.931+0000 7fc46069a700 1 --2- 172.21.15.52:0/1932282196 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fc4440a12c0 0x7fc4440a3770 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc44800b2e0 tx=0x7fc448005fd0 comp rx=0 tx=0).stop 2023-11-27T15:28:44.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.931+0000 7fc46069a700 1 -- 172.21.15.52:0/1932282196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc45810cce0 msgr2=0x7fc45819cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:44.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.931+0000 7fc46069a700 1 --2- 172.21.15.52:0/1932282196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc45810cce0 0x7fc45819cec0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fc45000ef30 tx=0x7fc45000c3a0 comp rx=0 tx=0).stop 2023-11-27T15:28:44.935 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.931+0000 7fc46069a700 1 -- 172.21.15.52:0/1932282196 shutdown_connections 2023-11-27T15:28:44.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.931+0000 7fc46069a700 1 --2- 172.21.15.52:0/1932282196 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fc4440a12c0 0x7fc4440a3770 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:44.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.931+0000 7fc46069a700 1 --2- 172.21.15.52:0/1932282196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc458068ed0 0x7fc458198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:44.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.931+0000 7fc46069a700 1 --2- 172.21.15.52:0/1932282196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc45810cce0 0x7fc45819cec0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:44.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.931+0000 7fc46069a700 1 --2- 172.21.15.52:0/1932282196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4580697e0 0x7fc458198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:44.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.931+0000 7fc46069a700 1 -- 172.21.15.52:0/1932282196 >> 172.21.15.52:0/1932282196 conn(0x7fc458076100 msgr2=0x7fc4580fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:44.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.932+0000 7fc46069a700 1 -- 172.21.15.52:0/1932282196 shutdown_connections 2023-11-27T15:28:44.936 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:44.932+0000 7fc46069a700 1 -- 172.21.15.52:0/1932282196 wait complete. 2023-11-27T15:28:44.937 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 34 2023-11-27T15:28:44.947 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:28:45.074 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.071+0000 7f61c2d9f700 1 -- 172.21.15.52:0/2505696651 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61bc100320 msgr2=0x7f61bc1006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:45.074 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.071+0000 7f61c2d9f700 1 --2- 172.21.15.52:0/2505696651 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61bc100320 0x7f61bc1006f0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f61b80099c0 tx=0x7f61b8009cd0 comp rx=0 tx=0).stop 2023-11-27T15:28:45.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.071+0000 7f61c2d9f700 1 -- 172.21.15.52:0/2505696651 shutdown_connections 2023-11-27T15:28:45.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.071+0000 7f61c2d9f700 1 --2- 172.21.15.52:0/2505696651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61bc105360 0x7f61bc10cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:45.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.071+0000 7f61c2d9f700 1 --2- 172.21.15.52:0/2505696651 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61bc100320 0x7f61bc1006f0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:45.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.071+0000 7f61c2d9f700 1 --2- 172.21.15.52:0/2505696651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61bc100cc0 0x7f61bc104c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:45.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.071+0000 7f61c2d9f700 1 -- 172.21.15.52:0/2505696651 >> 172.21.15.52:0/2505696651 conn(0x7f61bc0fbc10 msgr2=0x7f61bc0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:45.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.071+0000 7f61c2d9f700 1 -- 172.21.15.52:0/2505696651 shutdown_connections 2023-11-27T15:28:45.075 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.071+0000 7f61c2d9f700 1 -- 172.21.15.52:0/2505696651 wait complete. 2023-11-27T15:28:45.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.072+0000 7f61c2d9f700 1 Processor -- start 2023-11-27T15:28:45.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.072+0000 7f61c2d9f700 1 -- start start 2023-11-27T15:28:45.077 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.072+0000 7f61c2d9f700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61bc100320 0x7f61bc072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:45.078 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.072+0000 7f61c2d9f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61bc100cc0 0x7f61bc073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:45.078 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.072+0000 7f61c2d9f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61bc105360 0x7f61bc1a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:45.078 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.072+0000 7f61c2d9f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc073b60 con 0x7f61bc100cc0 2023-11-27T15:28:45.078 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.072+0000 7f61c2d9f700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc1a47f0 con 0x7f61bc100320 2023-11-27T15:28:45.078 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.072+0000 7f61c2d9f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc1a4930 con 0x7f61bc105360 2023-11-27T15:28:45.078 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.072+0000 7f61c159c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61bc100cc0 0x7f61bc073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:45.078 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.072+0000 7f61c1d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61bc100320 0x7f61bc072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:45.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.072+0000 7f61c159c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61bc100cc0 0x7f61bc073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56572/0 (socket says 172.21.15.52:56572) 2023-11-27T15:28:45.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.072+0000 7f61c159c700 1 -- 172.21.15.52:0/2222763184 learned_addr learned my addr 172.21.15.52:0/2222763184 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:45.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.072+0000 7f61c259e700 1 --2- 172.21.15.52:0/2222763184 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61bc105360 0x7f61bc1a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:45.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.072+0000 7f61c159c700 1 -- 172.21.15.52:0/2222763184 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61bc105360 msgr2=0x7f61bc1a42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:45.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.072+0000 7f61c159c700 1 --2- 172.21.15.52:0/2222763184 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61bc105360 0x7f61bc1a42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:45.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.072+0000 7f61c159c700 1 -- 172.21.15.52:0/2222763184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61bc100320 msgr2=0x7f61bc072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:45.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.072+0000 7f61c159c700 1 --2- 172.21.15.52:0/2222763184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61bc100320 0x7f61bc072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:45.079 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.072+0000 7f61c159c700 1 -- 172.21.15.52:0/2222763184 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b80095f0 con 0x7f61bc100cc0 2023-11-27T15:28:45.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.072+0000 7f61c1d9d700 1 --2- 172.21.15.52:0/2222763184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61bc100320 0x7f61bc072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:28:45.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.073+0000 7f61c159c700 1 --2- 172.21.15.52:0/2222763184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61bc100cc0 0x7f61bc073330 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f61ac00c9b0 tx=0x7f61ac00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:45.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.073+0000 7f61b2ffd700 1 -- 172.21.15.52:0/2222763184 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac007aa0 con 0x7f61bc100cc0 2023-11-27T15:28:45.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.073+0000 7f61b2ffd700 1 -- 172.21.15.52:0/2222763184 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61ac00ce50 con 0x7f61bc100cc0 2023-11-27T15:28:45.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.073+0000 7f61c2d9f700 1 -- 172.21.15.52:0/2222763184 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61bc110120 con 0x7f61bc100cc0 2023-11-27T15:28:45.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.073+0000 7f61b2ffd700 1 -- 172.21.15.52:0/2222763184 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac018460 con 0x7f61bc100cc0 2023-11-27T15:28:45.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.073+0000 7f61c2d9f700 1 -- 172.21.15.52:0/2222763184 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61bc10ff90 con 0x7f61bc100cc0 2023-11-27T15:28:45.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.074+0000 7f61c2d9f700 1 -- 172.21.15.52:0/2222763184 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61bc1102b0 con 0x7f61bc100cc0 2023-11-27T15:28:45.081 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.076+0000 7f61b2ffd700 1 -- 172.21.15.52:0/2222763184 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f61ac00f450 con 0x7f61bc100cc0 2023-11-27T15:28:45.081 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.076+0000 7f61b2ffd700 1 --2- 172.21.15.52:0/2222763184 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f61a80a12c0 0x7f61a80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:45.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.076+0000 7f61c1d9d700 1 --2- 172.21.15.52:0/2222763184 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f61a80a12c0 0x7f61a80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:45.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.076+0000 7f61b2ffd700 1 -- 172.21.15.52:0/2222763184 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f61ac0ccd50 con 0x7f61bc100cc0 2023-11-27T15:28:45.082 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.077+0000 7f61c1d9d700 1 --2- 172.21.15.52:0/2222763184 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f61a80a12c0 0x7f61a80a3770 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f61b8000c00 tx=0x7f61b8003680 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:45.083 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.080+0000 7f61b2ffd700 1 -- 172.21.15.52:0/2222763184 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f61ac096130 con 0x7f61bc100cc0 2023-11-27T15:28:45.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.330+0000 7f61c2d9f700 1 -- 172.21.15.52:0/2222763184 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61bc0670f0 con 0x7f61bc100cc0 2023-11-27T15:28:45.333 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.330+0000 7f61b2ffd700 1 -- 172.21.15.52:0/2222763184 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10804 (secure 0 0 0) 0x7f61ac099750 con 0x7f61bc100cc0 2023-11-27T15:28:45.334 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:28:45.334 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3561136724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3561136724},{"type":"v1","addr":"172.21.15.52:6835","nonce":3561136724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3583014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3583014325},{"type":"v1","addr":"172.21.15.52:6837","nonce":3583014325}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14586,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2328881376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2328881376},{"type":"v1","addr":"172.21.15.52:6839","nonce":2328881376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2799551570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2799551570},{"type":"v1","addr":"172.21.15.52:6841","nonce":2799551570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/1857157307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1857157307},{"type":"v1","addr":"172.21.15.150:6833","nonce":1857157307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/883247933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":883247933},{"type":"v1","addr":"172.21.15.102:6833","nonce":883247933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:16:09.319630+0000","modified":"2023-11-27T15:26:15.243744+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24494,"mds_2":24472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/617413375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":617413375},{"type":"v1","addr":"172.21.15.150:6835","nonce":617413375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.150:6837/805360541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":805360541},{"type":"v1","addr":"172.21.15.150:6837","nonce":805360541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/2398692013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":2398692013},{"type":"v1","addr":"172.21.15.102:6835","nonce":2398692013}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6839/215725838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":215725838},{"type":"v1","addr":"172.21.15.150:6839","nonce":215725838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.102:6837/2141793919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2141793919},{"type":"v1","addr":"172.21.15.102:6837","nonce":2141793919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3571830559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3571830559},{"type":"v1","addr":"172.21.15.102:6839","nonce":3571830559}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:28:45.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.332+0000 7f61c2d9f700 1 -- 172.21.15.52:0/2222763184 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f61a80a12c0 msgr2=0x7f61a80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:45.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.332+0000 7f61c2d9f700 1 --2- 172.21.15.52:0/2222763184 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f61a80a12c0 0x7f61a80a3770 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f61b8000c00 tx=0x7f61b8003680 comp rx=0 tx=0).stop 2023-11-27T15:28:45.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.332+0000 7f61c2d9f700 1 -- 172.21.15.52:0/2222763184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61bc100cc0 msgr2=0x7f61bc073330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:45.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.332+0000 7f61c2d9f700 1 --2- 172.21.15.52:0/2222763184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61bc100cc0 0x7f61bc073330 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f61ac00c9b0 tx=0x7f61ac00ccc0 comp rx=0 tx=0).stop 2023-11-27T15:28:45.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.332+0000 7f61c2d9f700 1 -- 172.21.15.52:0/2222763184 shutdown_connections 2023-11-27T15:28:45.336 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.332+0000 7f61c2d9f700 1 --2- 172.21.15.52:0/2222763184 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f61a80a12c0 0x7f61a80a3770 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:45.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.332+0000 7f61c2d9f700 1 --2- 172.21.15.52:0/2222763184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61bc100320 0x7f61bc072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:45.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.332+0000 7f61c2d9f700 1 --2- 172.21.15.52:0/2222763184 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61bc105360 0x7f61bc1a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:45.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.332+0000 7f61c2d9f700 1 --2- 172.21.15.52:0/2222763184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61bc100cc0 0x7f61bc073330 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:45.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.332+0000 7f61c2d9f700 1 -- 172.21.15.52:0/2222763184 >> 172.21.15.52:0/2222763184 conn(0x7f61bc0fbc10 msgr2=0x7f61bc0fc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:45.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.332+0000 7f61c2d9f700 1 -- 172.21.15.52:0/2222763184 shutdown_connections 2023-11-27T15:28:45.337 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.333+0000 7f61c2d9f700 1 -- 172.21.15.52:0/2222763184 wait complete. 2023-11-27T15:28:45.338 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 34 2023-11-27T15:28:45.348 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:28:45.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.470+0000 7ff0a3797700 1 -- 172.21.15.52:0/2574455125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff09c104460 msgr2=0x7ff09c10c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:45.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.470+0000 7ff0a3797700 1 --2- 172.21.15.52:0/2574455125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff09c104460 0x7ff09c10c030 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7ff09400b0d0 tx=0x7ff09400b3e0 comp rx=0 tx=0).stop 2023-11-27T15:28:45.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.470+0000 7ff0a3797700 1 -- 172.21.15.52:0/2574455125 shutdown_connections 2023-11-27T15:28:45.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.470+0000 7ff0a3797700 1 --2- 172.21.15.52:0/2574455125 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff09c0ff420 0x7ff09c0ff7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:45.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.470+0000 7ff0a3797700 1 --2- 172.21.15.52:0/2574455125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff09c104460 0x7ff09c10c030 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:45.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.470+0000 7ff0a3797700 1 --2- 172.21.15.52:0/2574455125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff09c0ffdc0 0x7ff09c103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:45.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.470+0000 7ff0a3797700 1 -- 172.21.15.52:0/2574455125 >> 172.21.15.52:0/2574455125 conn(0x7ff09c076310 msgr2=0x7ff09c076710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:45.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.471+0000 7ff0a3797700 1 -- 172.21.15.52:0/2574455125 shutdown_connections 2023-11-27T15:28:45.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.471+0000 7ff0a3797700 1 -- 172.21.15.52:0/2574455125 wait complete. 2023-11-27T15:28:45.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.471+0000 7ff0a3797700 1 Processor -- start 2023-11-27T15:28:45.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.471+0000 7ff0a3797700 1 -- start start 2023-11-27T15:28:45.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.472+0000 7ff0a3797700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff09c0ff420 0x7ff09c195430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:45.477 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.472+0000 7ff0a3797700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff09c0ffdc0 0x7ff09c195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:45.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.472+0000 7ff0a3797700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff09c104460 0x7ff09c199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:45.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.472+0000 7ff0a3797700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c19a410 con 0x7ff09c0ff420 2023-11-27T15:28:45.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.472+0000 7ff0a3797700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c19a580 con 0x7ff09c0ffdc0 2023-11-27T15:28:45.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.472+0000 7ff0a3797700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c19a740 con 0x7ff09c104460 2023-11-27T15:28:45.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.472+0000 7ff0a1533700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff09c0ff420 0x7ff09c195430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:45.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.472+0000 7ff0a1533700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff09c0ff420 0x7ff09c195430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56584/0 (socket says 172.21.15.52:56584) 2023-11-27T15:28:45.478 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.472+0000 7ff0a1533700 1 -- 172.21.15.52:0/3763479082 learned_addr learned my addr 172.21.15.52:0/3763479082 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:45.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.472+0000 7ff0a0d32700 1 --2- 172.21.15.52:0/3763479082 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff09c0ffdc0 0x7ff09c195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:45.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.472+0000 7ff0a1d34700 1 --2- 172.21.15.52:0/3763479082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff09c104460 0x7ff09c199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:45.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.472+0000 7ff0a1533700 1 -- 172.21.15.52:0/3763479082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff09c104460 msgr2=0x7ff09c199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:45.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.472+0000 7ff0a1533700 1 --2- 172.21.15.52:0/3763479082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff09c104460 0x7ff09c199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:45.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.472+0000 7ff0a1533700 1 -- 172.21.15.52:0/3763479082 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff09c0ffdc0 msgr2=0x7ff09c195970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:45.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.472+0000 7ff0a1533700 1 --2- 172.21.15.52:0/3763479082 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff09c0ffdc0 0x7ff09c195970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:45.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.472+0000 7ff0a1533700 1 -- 172.21.15.52:0/3763479082 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff098009520 con 0x7ff09c0ff420 2023-11-27T15:28:45.479 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.472+0000 7ff0a1533700 1 --2- 172.21.15.52:0/3763479082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff09c0ff420 0x7ff09c195430 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7ff09800eb30 tx=0x7ff09800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:45.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.473+0000 7ff0927fc700 1 -- 172.21.15.52:0/3763479082 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff098009880 con 0x7ff09c0ff420 2023-11-27T15:28:45.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.473+0000 7ff0a3797700 1 -- 172.21.15.52:0/3763479082 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff094009d00 con 0x7ff09c0ff420 2023-11-27T15:28:45.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.473+0000 7ff0927fc700 1 -- 172.21.15.52:0/3763479082 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0980044e0 con 0x7ff09c0ff420 2023-11-27T15:28:45.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.473+0000 7ff0a3797700 1 -- 172.21.15.52:0/3763479082 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff09c10f230 con 0x7ff09c0ff420 2023-11-27T15:28:45.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.473+0000 7ff0927fc700 1 -- 172.21.15.52:0/3763479082 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff098003e30 con 0x7ff09c0ff420 2023-11-27T15:28:45.480 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.474+0000 7ff0a3797700 1 -- 172.21.15.52:0/3763479082 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff09c10f0a0 con 0x7ff09c0ff420 2023-11-27T15:28:45.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.475+0000 7ff0927fc700 1 -- 172.21.15.52:0/3763479082 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7ff098003f90 con 0x7ff09c0ff420 2023-11-27T15:28:45.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.475+0000 7ff0927fc700 1 --2- 172.21.15.52:0/3763479082 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ff0880a12c0 0x7ff0880a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:45.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.475+0000 7ff0927fc700 1 -- 172.21.15.52:0/3763479082 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8571+0+0 (secure 0 0 0) 0x7ff098014070 con 0x7ff09c0ff420 2023-11-27T15:28:45.481 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.475+0000 7ff0a0d32700 1 --2- 172.21.15.52:0/3763479082 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ff0880a12c0 0x7ff0880a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:45.482 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.476+0000 7ff0a0d32700 1 --2- 172.21.15.52:0/3763479082 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ff0880a12c0 0x7ff0880a3770 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff09c196a40 tx=0x7ff08c009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:45.483 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.480+0000 7ff0927fc700 1 -- 172.21.15.52:0/3763479082 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff098096500 con 0x7ff09c0ff420 2023-11-27T15:28:45.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:45 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1932282196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:28:45.615 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:45 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1932282196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:28:45.681 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.678+0000 7ff0a3797700 1 -- 172.21.15.52:0/3763479082 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff09c10f3c0 con 0x7ff09c0ff420 2023-11-27T15:28:45.683 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.680+0000 7ff0927fc700 1 -- 172.21.15.52:0/3763479082 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+23125 (secure 0 0 0) 0x7ff098099b20 con 0x7ff09c0ff420 2023-11-27T15:28:45.683 INFO:teuthology.orchestra.run.smithi052.stdout: 2023-11-27T15:28:45.684 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":96,"fsid":"c6c7df88-8d36-11ee-95a2-87774f69a715","created":"2023-11-27T15:09:03.333345+0000","modified":"2023-11-27T15:28:13.934915+0000","last_up_change":"2023-11-27T15:15:27.656134+0000","last_in_change":"2023-11-27T15:15:10.671153+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:12:14.177523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:16:07.842251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:16:08.728520+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":95,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58d69ede-c293-49f5-abbe-8187793f2b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6803","nonce":608216415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6805","nonce":608216415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6809","nonce":608216415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":608216415},{"type":"v1","addr":"172.21.15.52:6807","nonce":608216415}]},"public_addr":"172.21.15.52:6803/608216415","cluster_addr":"172.21.15.52:6805/608216415","heartbeat_back_addr":"172.21.15.52:6809/608216415","heartbeat_front_addr":"172.21.15.52:6807/608216415","state":["exists","up"]},{"osd":1,"uuid":"d6a46b91-adc8-45ef-80f5-566ae400911f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6801","nonce":2538384087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6803","nonce":2538384087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6807","nonce":2538384087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":2538384087},{"type":"v1","addr":"172.21.15.102:6805","nonce":2538384087}]},"public_addr":"172.21.15.102:6801/2538384087","cluster_addr":"172.21.15.102:6803/2538384087","heartbeat_back_addr":"172.21.15.102:6807/2538384087","heartbeat_front_addr":"172.21.15.102:6805/2538384087","state":["exists","up"]},{"osd":2,"uuid":"846ba125-11f8-46b2-8941-bd5e1947cc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6801","nonce":4258038221}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6803","nonce":4258038221}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6807","nonce":4258038221}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":4258038221},{"type":"v1","addr":"172.21.15.150:6805","nonce":4258038221}]},"public_addr":"172.21.15.150:6801/4258038221","cluster_addr":"172.21.15.150:6803/4258038221","heartbeat_back_addr":"172.21.15.150:6807/4258038221","heartbeat_front_addr":"172.21.15.150:6805/4258038221","state":["exists","up"]},{"osd":3,"uuid":"950fcd67-1057-4288-a389-714f1ec94261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6811","nonce":2622100752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6813","nonce":2622100752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6817","nonce":2622100752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2622100752},{"type":"v1","addr":"172.21.15.52:6815","nonce":2622100752}]},"public_addr":"172.21.15.52:6811/2622100752","cluster_addr":"172.21.15.52:6813/2622100752","heartbeat_back_addr":"172.21.15.52:6817/2622100752","heartbeat_front_addr":"172.21.15.52:6815/2622100752","state":["exists","up"]},{"osd":4,"uuid":"1c96a676-cea9-4deb-bad1-f7fd240c3d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6809","nonce":1546794763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6811","nonce":1546794763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6815","nonce":1546794763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1546794763},{"type":"v1","addr":"172.21.15.102:6813","nonce":1546794763}]},"public_addr":"172.21.15.102:6809/1546794763","cluster_addr":"172.21.15.102:6811/1546794763","heartbeat_back_addr":"172.21.15.102:6815/1546794763","heartbeat_front_addr":"172.21.15.102:6813/1546794763","state":["exists","up"]},{"osd":5,"uuid":"38a75af1-fd4d-433e-b579-0609109c851c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6809","nonce":461511375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6811","nonce":461511375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6815","nonce":461511375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":461511375},{"type":"v1","addr":"172.21.15.150:6813","nonce":461511375}]},"public_addr":"172.21.15.150:6809/461511375","cluster_addr":"172.21.15.150:6811/461511375","heartbeat_back_addr":"172.21.15.150:6815/461511375","heartbeat_front_addr":"172.21.15.150:6813/461511375","state":["exists","up"]},{"osd":6,"uuid":"2f0ede4a-d175-4b4a-b876-c0f442048901","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6819","nonce":1002457464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6821","nonce":1002457464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6825","nonce":1002457464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1002457464},{"type":"v1","addr":"172.21.15.52:6823","nonce":1002457464}]},"public_addr":"172.21.15.52:6819/1002457464","cluster_addr":"172.21.15.52:6821/1002457464","heartbeat_back_addr":"172.21.15.52:6825/1002457464","heartbeat_front_addr":"172.21.15.52:6823/1002457464","state":["exists","up"]},{"osd":7,"uuid":"6b635192-abcf-4bbf-bf78-a019b351bca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6817","nonce":3249840064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6819","nonce":3249840064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6823","nonce":3249840064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":3249840064},{"type":"v1","addr":"172.21.15.102:6821","nonce":3249840064}]},"public_addr":"172.21.15.102:6817/3249840064","cluster_addr":"172.21.15.102:6819/3249840064","heartbeat_back_addr":"172.21.15.102:6823/3249840064","heartbeat_front_addr":"172.21.15.102:6821/3249840064","state":["exists","up"]},{"osd":8,"uuid":"c7ca8007-7170-47bf-956c-3f00edf27e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6817","nonce":3672583574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6819","nonce":3672583574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6823","nonce":3672583574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":3672583574},{"type":"v1","addr":"172.21.15.150:6821","nonce":3672583574}]},"public_addr":"172.21.15.150:6817/3672583574","cluster_addr":"172.21.15.150:6819/3672583574","heartbeat_back_addr":"172.21.15.150:6823/3672583574","heartbeat_front_addr":"172.21.15.150:6821/3672583574","state":["exists","up"]},{"osd":9,"uuid":"0e252c2c-01a4-48ba-832d-067ed940fefb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6827","nonce":4287934320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6829","nonce":4287934320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6833","nonce":4287934320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":4287934320},{"type":"v1","addr":"172.21.15.52:6831","nonce":4287934320}]},"public_addr":"172.21.15.52:6827/4287934320","cluster_addr":"172.21.15.52:6829/4287934320","heartbeat_back_addr":"172.21.15.52:6833/4287934320","heartbeat_front_addr":"172.21.15.52:6831/4287934320","state":["exists","up"]},{"osd":10,"uuid":"bb540142-7563-44a4-b7a3-cb662714c128","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6825","nonce":3056034201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6827","nonce":3056034201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6831","nonce":3056034201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":3056034201},{"type":"v1","addr":"172.21.15.102:6829","nonce":3056034201}]},"public_addr":"172.21.15.102:6825/3056034201","cluster_addr":"172.21.15.102:6827/3056034201","heartbeat_back_addr":"172.21.15.102:6831/3056034201","heartbeat_front_addr":"172.21.15.102:6829/3056034201","state":["exists","up"]},{"osd":11,"uuid":"3156528e-e0f8-4d44-9946-c8b6339c2bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6825","nonce":3207559739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6827","nonce":3207559739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6831","nonce":3207559739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3207559739},{"type":"v1","addr":"172.21.15.150:6829","nonce":3207559739}]},"public_addr":"172.21.15.150:6825/3207559739","cluster_addr":"172.21.15.150:6827/3207559739","heartbeat_back_addr":"172.21.15.150:6831/3207559739","heartbeat_front_addr":"172.21.15.150:6829/3207559739","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:26.826099+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:11:48.331371+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:10.459738+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:32.356472+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:12:54.043170+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:15.648582+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:37.166463+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:13:59.240819+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:20.710097+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:14:43.364563+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:05.279433+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:26.344573+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":5,"to":7}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/4025191157":"2023-11-28T15:17:54.913525+0000","172.21.15.52:6801/2640036585":"2023-11-28T15:17:54.913525+0000","172.21.15.52:6800/2640036585":"2023-11-28T15:17:54.913525+0000","172.21.15.52:0/1392282110":"2023-11-28T15:10:03.644870+0000","172.21.15.52:0/4081249946":"2023-11-28T15:17:54.913525+0000","172.21.15.52:0/1479993479":"2023-11-28T15:10:03.644870+0000","172.21.15.52:0/2690561203":"2023-11-28T15:17:54.913525+0000","172.21.15.52:0/2454989397":"2023-11-28T15:17:54.913525+0000","172.21.15.52:6801/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6800/3470359117":"2023-11-28T15:10:03.644870+0000","172.21.15.52:6801/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/961562652":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/1290665390":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2410529742":"2023-11-28T15:09:37.118690+0000","172.21.15.52:6800/960704660":"2023-11-28T15:09:37.118690+0000","172.21.15.52:0/2130037044":"2023-11-28T15:10:03.644870+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:28:45.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.682+0000 7ff0a3797700 1 -- 172.21.15.52:0/3763479082 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ff0880a12c0 msgr2=0x7ff0880a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:45.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.682+0000 7ff0a3797700 1 --2- 172.21.15.52:0/3763479082 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ff0880a12c0 0x7ff0880a3770 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff09c196a40 tx=0x7ff08c009260 comp rx=0 tx=0).stop 2023-11-27T15:28:45.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.682+0000 7ff0a3797700 1 -- 172.21.15.52:0/3763479082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff09c0ff420 msgr2=0x7ff09c195430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:45.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.682+0000 7ff0a3797700 1 --2- 172.21.15.52:0/3763479082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff09c0ff420 0x7ff09c195430 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7ff09800eb30 tx=0x7ff09800ee40 comp rx=0 tx=0).stop 2023-11-27T15:28:45.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.682+0000 7ff0a3797700 1 -- 172.21.15.52:0/3763479082 shutdown_connections 2023-11-27T15:28:45.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.682+0000 7ff0a3797700 1 --2- 172.21.15.52:0/3763479082 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7ff0880a12c0 0x7ff0880a3770 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:45.686 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.682+0000 7ff0a3797700 1 --2- 172.21.15.52:0/3763479082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff09c104460 0x7ff09c199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:45.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.682+0000 7ff0a3797700 1 --2- 172.21.15.52:0/3763479082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff09c0ff420 0x7ff09c195430 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:45.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.682+0000 7ff0a3797700 1 --2- 172.21.15.52:0/3763479082 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff09c0ffdc0 0x7ff09c195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:45.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.682+0000 7ff0a3797700 1 -- 172.21.15.52:0/3763479082 >> 172.21.15.52:0/3763479082 conn(0x7ff09c076310 msgr2=0x7ff09c0fca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:45.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.682+0000 7ff0a3797700 1 -- 172.21.15.52:0/3763479082 shutdown_connections 2023-11-27T15:28:45.687 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.682+0000 7ff0a3797700 1 -- 172.21.15.52:0/3763479082 wait complete. 2023-11-27T15:28:45.698 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-27T15:28:45.777 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:45 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1932282196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:28:45.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.821+0000 7f90d5a47700 1 -- 172.21.15.52:0/2915523818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90d00ff390 msgr2=0x7f90d00ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:45.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.821+0000 7f90d5a47700 1 --2- 172.21.15.52:0/2915523818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90d00ff390 0x7f90d00ff760 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f90c40099c0 tx=0x7f90c4009cd0 comp rx=0 tx=0).stop 2023-11-27T15:28:45.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.821+0000 7f90d5a47700 1 -- 172.21.15.52:0/2915523818 shutdown_connections 2023-11-27T15:28:45.825 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.821+0000 7f90d5a47700 1 --2- 172.21.15.52:0/2915523818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90d01043d0 0x7f90d010bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:45.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.821+0000 7f90d5a47700 1 --2- 172.21.15.52:0/2915523818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90d00ff390 0x7f90d00ff760 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:45.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.821+0000 7f90d5a47700 1 --2- 172.21.15.52:0/2915523818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90d00ffd30 0x7f90d0103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:45.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.821+0000 7f90d5a47700 1 -- 172.21.15.52:0/2915523818 >> 172.21.15.52:0/2915523818 conn(0x7f90d0076370 msgr2=0x7f90d0076770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:45.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.821+0000 7f90d5a47700 1 -- 172.21.15.52:0/2915523818 shutdown_connections 2023-11-27T15:28:45.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.821+0000 7f90d5a47700 1 -- 172.21.15.52:0/2915523818 wait complete. 2023-11-27T15:28:45.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.822+0000 7f90d5a47700 1 Processor -- start 2023-11-27T15:28:45.826 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.822+0000 7f90d5a47700 1 -- start start 2023-11-27T15:28:45.828 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.822+0000 7f90d5a47700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90d00ff390 0x7f90d01952f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:45.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.822+0000 7f90d5a47700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90d00ffd30 0x7f90d0195830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:45.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.822+0000 7f90d5a47700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90d01043d0 0x7f90d0199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:45.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.822+0000 7f90d5a47700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d019a2d0 con 0x7f90d00ffd30 2023-11-27T15:28:45.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.822+0000 7f90d5a47700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d019a440 con 0x7f90d01043d0 2023-11-27T15:28:45.829 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.822+0000 7f90d5a47700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d019a600 con 0x7f90d00ff390 2023-11-27T15:28:45.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.822+0000 7f90ce7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90d00ffd30 0x7f90d0195830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:45.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.823+0000 7f90ce7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90d00ffd30 0x7f90d0195830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56600/0 (socket says 172.21.15.52:56600) 2023-11-27T15:28:45.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.823+0000 7f90ce7fc700 1 -- 172.21.15.52:0/1192980582 learned_addr learned my addr 172.21.15.52:0/1192980582 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:45.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.823+0000 7f90cf7fe700 1 --2- 172.21.15.52:0/1192980582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90d01043d0 0x7f90d0199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:45.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.823+0000 7f90ceffd700 1 --2- 172.21.15.52:0/1192980582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90d00ff390 0x7f90d01952f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:45.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.823+0000 7f90ce7fc700 1 -- 172.21.15.52:0/1192980582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90d00ff390 msgr2=0x7f90d01952f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:45.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.823+0000 7f90ce7fc700 1 --2- 172.21.15.52:0/1192980582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90d00ff390 0x7f90d01952f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:45.830 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.823+0000 7f90ce7fc700 1 -- 172.21.15.52:0/1192980582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90d01043d0 msgr2=0x7f90d0199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:45.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.823+0000 7f90ce7fc700 1 --2- 172.21.15.52:0/1192980582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90d01043d0 0x7f90d0199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:45.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.823+0000 7f90ce7fc700 1 -- 172.21.15.52:0/1192980582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c40095f0 con 0x7f90d00ffd30 2023-11-27T15:28:45.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.823+0000 7f90ce7fc700 1 --2- 172.21.15.52:0/1192980582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90d00ffd30 0x7f90d0195830 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f90c000ea50 tx=0x7f90c000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:45.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.823+0000 7f90d4a45700 1 -- 172.21.15.52:0/1192980582 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c0013070 con 0x7f90d00ffd30 2023-11-27T15:28:45.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.823+0000 7f90d5a47700 1 -- 172.21.15.52:0/1192980582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d010f1e0 con 0x7f90d00ffd30 2023-11-27T15:28:45.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.823+0000 7f90d4a45700 1 -- 172.21.15.52:0/1192980582 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90c000be80 con 0x7f90d00ffd30 2023-11-27T15:28:45.831 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.823+0000 7f90d4a45700 1 -- 172.21.15.52:0/1192980582 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c00163b0 con 0x7f90d00ffd30 2023-11-27T15:28:45.832 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.823+0000 7f90d5a47700 1 -- 172.21.15.52:0/1192980582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d010f050 con 0x7f90d00ffd30 2023-11-27T15:28:45.832 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.824+0000 7f90d5a47700 1 -- 172.21.15.52:0/1192980582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90d010f370 con 0x7f90d00ffd30 2023-11-27T15:28:45.832 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.825+0000 7f90d4a45700 1 -- 172.21.15.52:0/1192980582 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f90c0017580 con 0x7f90d00ffd30 2023-11-27T15:28:45.832 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.826+0000 7f90d4a45700 1 --2- 172.21.15.52:0/1192980582 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f90b80a12c0 0x7f90b80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:45.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.826+0000 7f90ceffd700 1 --2- 172.21.15.52:0/1192980582 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f90b80a12c0 0x7f90b80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:45.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.826+0000 7f90d4a45700 1 -- 172.21.15.52:0/1192980582 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8571+0+0 (secure 0 0 0) 0x7f90c00ce360 con 0x7f90d00ffd30 2023-11-27T15:28:45.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.826+0000 7f90ceffd700 1 --2- 172.21.15.52:0/1192980582 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f90b80a12c0 0x7f90b80a3770 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009990 tx=0x7f90c401a040 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:45.833 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:45.830+0000 7f90d4a45700 1 -- 172.21.15.52:0/1192980582 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f90c0097740 con 0x7f90d00ffd30 2023-11-27T15:28:46.080 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.077+0000 7f90d5a47700 1 -- 172.21.15.52:0/1192980582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f90d00679f0 con 0x7f90d00ffd30 2023-11-27T15:28:46.301 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.298+0000 7f90d4a45700 1 -- 172.21.15.52:0/1192980582 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v35) v1 ==== 161+0+0 (secure 0 0 0) 0x7f90c009ad60 con 0x7f90d00ffd30 2023-11-27T15:28:46.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.300+0000 7f90d5a47700 1 -- 172.21.15.52:0/1192980582 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f90b80a12c0 msgr2=0x7f90b80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:46.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.300+0000 7f90d5a47700 1 --2- 172.21.15.52:0/1192980582 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f90b80a12c0 0x7f90b80a3770 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009990 tx=0x7f90c401a040 comp rx=0 tx=0).stop 2023-11-27T15:28:46.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.300+0000 7f90d5a47700 1 -- 172.21.15.52:0/1192980582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90d00ffd30 msgr2=0x7f90d0195830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:46.304 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.300+0000 7f90d5a47700 1 --2- 172.21.15.52:0/1192980582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90d00ffd30 0x7f90d0195830 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f90c000ea50 tx=0x7f90c000ed60 comp rx=0 tx=0).stop 2023-11-27T15:28:46.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.301+0000 7f90d5a47700 1 -- 172.21.15.52:0/1192980582 shutdown_connections 2023-11-27T15:28:46.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.301+0000 7f90d5a47700 1 --2- 172.21.15.52:0/1192980582 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7f90b80a12c0 0x7f90b80a3770 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.301+0000 7f90d5a47700 1 --2- 172.21.15.52:0/1192980582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90d01043d0 0x7f90d0199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.301+0000 7f90d5a47700 1 --2- 172.21.15.52:0/1192980582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90d00ff390 0x7f90d01952f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.301+0000 7f90d5a47700 1 --2- 172.21.15.52:0/1192980582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90d00ffd30 0x7f90d0195830 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.301+0000 7f90d5a47700 1 -- 172.21.15.52:0/1192980582 >> 172.21.15.52:0/1192980582 conn(0x7f90d0076370 msgr2=0x7f90d00fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:46.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.301+0000 7f90d5a47700 1 -- 172.21.15.52:0/1192980582 shutdown_connections 2023-11-27T15:28:46.305 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.301+0000 7f90d5a47700 1 -- 172.21.15.52:0/1192980582 wait complete. 2023-11-27T15:28:46.306 INFO:teuthology.orchestra.run.smithi052.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-27T15:28:46.317 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-27T15:28:46.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.458+0000 7fcf64340700 1 -- 172.21.15.52:0/3836608363 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf5c104560 msgr2=0x7fcf5c10c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:46.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.458+0000 7fcf64340700 1 --2- 172.21.15.52:0/3836608363 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf5c104560 0x7fcf5c10c130 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fcf5400b0d0 tx=0x7fcf5400b3e0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.458+0000 7fcf64340700 1 -- 172.21.15.52:0/3836608363 shutdown_connections 2023-11-27T15:28:46.464 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.458+0000 7fcf64340700 1 --2- 172.21.15.52:0/3836608363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf5c0ff520 0x7fcf5c0ff8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.458+0000 7fcf64340700 1 --2- 172.21.15.52:0/3836608363 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf5c104560 0x7fcf5c10c130 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.458+0000 7fcf64340700 1 --2- 172.21.15.52:0/3836608363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf5c0ffec0 0x7fcf5c103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.458+0000 7fcf64340700 1 -- 172.21.15.52:0/3836608363 >> 172.21.15.52:0/3836608363 conn(0x7fcf5c0760d0 msgr2=0x7fcf5c0764d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:46.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.458+0000 7fcf64340700 1 -- 172.21.15.52:0/3836608363 shutdown_connections 2023-11-27T15:28:46.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.459+0000 7fcf64340700 1 -- 172.21.15.52:0/3836608363 wait complete. 2023-11-27T15:28:46.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.459+0000 7fcf64340700 1 Processor -- start 2023-11-27T15:28:46.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.459+0000 7fcf64340700 1 -- start start 2023-11-27T15:28:46.465 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.459+0000 7fcf64340700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf5c0ff520 0x7fcf5c1973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:46.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.459+0000 7fcf64340700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf5c0ffec0 0x7fcf5c1978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:46.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.459+0000 7fcf64340700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf5c104560 0x7fcf5c1a2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:46.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.459+0000 7fcf64340700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c1981f0 con 0x7fcf5c0ff520 2023-11-27T15:28:46.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.459+0000 7fcf64340700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c1a2960 con 0x7fcf5c104560 2023-11-27T15:28:46.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.459+0000 7fcf64340700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c1a2ad0 con 0x7fcf5c0ffec0 2023-11-27T15:28:46.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.459+0000 7fcf620dc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf5c0ff520 0x7fcf5c1973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:46.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.459+0000 7fcf620dc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf5c0ff520 0x7fcf5c1973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56612/0 (socket says 172.21.15.52:56612) 2023-11-27T15:28:46.466 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.459+0000 7fcf620dc700 1 -- 172.21.15.52:0/2134508621 learned_addr learned my addr 172.21.15.52:0/2134508621 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:46.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.460+0000 7fcf628dd700 1 --2- 172.21.15.52:0/2134508621 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf5c104560 0x7fcf5c1a2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:46.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.460+0000 7fcf618db700 1 --2- 172.21.15.52:0/2134508621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf5c0ffec0 0x7fcf5c1978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:46.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.460+0000 7fcf620dc700 1 -- 172.21.15.52:0/2134508621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf5c0ffec0 msgr2=0x7fcf5c1978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:46.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.460+0000 7fcf620dc700 1 --2- 172.21.15.52:0/2134508621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf5c0ffec0 0x7fcf5c1978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.460+0000 7fcf620dc700 1 -- 172.21.15.52:0/2134508621 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf5c104560 msgr2=0x7fcf5c1a2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:46.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.460+0000 7fcf620dc700 1 --2- 172.21.15.52:0/2134508621 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf5c104560 0x7fcf5c1a2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.460+0000 7fcf620dc700 1 -- 172.21.15.52:0/2134508621 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf54009d00 con 0x7fcf5c0ff520 2023-11-27T15:28:46.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.460+0000 7fcf618db700 1 --2- 172.21.15.52:0/2134508621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf5c0ffec0 0x7fcf5c1978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:28:46.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.460+0000 7fcf620dc700 1 --2- 172.21.15.52:0/2134508621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf5c0ff520 0x7fcf5c1973a0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000d980 tx=0x7fcf5000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:46.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.460+0000 7fcf4f7fe700 1 -- 172.21.15.52:0/2134508621 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf50004220 con 0x7fcf5c0ff520 2023-11-27T15:28:46.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.460+0000 7fcf4f7fe700 1 -- 172.21.15.52:0/2134508621 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf50007640 con 0x7fcf5c0ff520 2023-11-27T15:28:46.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.460+0000 7fcf64340700 1 -- 172.21.15.52:0/2134508621 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf5c10f390 con 0x7fcf5c0ff520 2023-11-27T15:28:46.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.460+0000 7fcf4f7fe700 1 -- 172.21.15.52:0/2134508621 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf50005610 con 0x7fcf5c0ff520 2023-11-27T15:28:46.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.460+0000 7fcf64340700 1 -- 172.21.15.52:0/2134508621 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf5c10f200 con 0x7fcf5c0ff520 2023-11-27T15:28:46.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.461+0000 7fcf64340700 1 -- 172.21.15.52:0/2134508621 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf5c10f520 con 0x7fcf5c0ff520 2023-11-27T15:28:46.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.462+0000 7fcf4f7fe700 1 -- 172.21.15.52:0/2134508621 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fcf50004750 con 0x7fcf5c0ff520 2023-11-27T15:28:46.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.463+0000 7fcf4f7fe700 1 --2- 172.21.15.52:0/2134508621 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fcf480a12c0 0x7fcf480a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:46.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.463+0000 7fcf4f7fe700 1 -- 172.21.15.52:0/2134508621 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8785+0+0 (secure 0 0 0) 0x7fcf500cd150 con 0x7fcf5c0ff520 2023-11-27T15:28:46.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.463+0000 7fcf618db700 1 --2- 172.21.15.52:0/2134508621 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fcf480a12c0 0x7fcf480a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:46.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.463+0000 7fcf618db700 1 --2- 172.21.15.52:0/2134508621 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fcf480a12c0 0x7fcf480a3770 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcf58005e10 tx=0x7fcf58005d60 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:46.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.466+0000 7fcf4f7fe700 1 -- 172.21.15.52:0/2134508621 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcf50096450 con 0x7fcf5c0ff520 2023-11-27T15:28:46.577 INFO:journalctl@ceph.mds.e.smithi102.stdout:Nov 27 15:28:46 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-e[124816]: ignoring --setuser ceph since I am not root 2023-11-27T15:28:46.577 INFO:journalctl@ceph.mds.e.smithi102.stdout:Nov 27 15:28:46 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-e[124816]: ignoring --setgroup ceph since I am not root 2023-11-27T15:28:46.577 INFO:journalctl@ceph.mds.e.smithi102.stdout:Nov 27 15:28:46 smithi102 ceph-mds[124840]: -- 172.21.15.102:0/3052126599 <== mon.0 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564a60663520 con 0x564a60638c00 2023-11-27T15:28:46.578 INFO:journalctl@ceph.mds.e.smithi102.stdout:Nov 27 15:28:46 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-e[124816]: starting mds.e at 2023-11-27T15:28:46.578 INFO:journalctl@ceph.mds.h.smithi102.stdout:Nov 27 15:28:46 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-h[125157]: ignoring --setuser ceph since I am not root 2023-11-27T15:28:46.578 INFO:journalctl@ceph.mds.h.smithi102.stdout:Nov 27 15:28:46 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-h[125157]: ignoring --setgroup ceph since I am not root 2023-11-27T15:28:46.578 INFO:journalctl@ceph.mds.h.smithi102.stdout:Nov 27 15:28:46 smithi102 ceph-mds[125181]: -- 172.21.15.102:0/1836522757 <== mon.2 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5594416cb520 con 0x5594416a0c00 2023-11-27T15:28:46.578 INFO:journalctl@ceph.mds.h.smithi102.stdout:Nov 27 15:28:46 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-h[125157]: starting mds.h at 2023-11-27T15:28:46.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:46 smithi102 ceph-mon[100296]: pgmap v338: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:28:46.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:46 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2222763184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:28:46.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:46 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/3763479082' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:28:46.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:46 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1192980582' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-27T15:28:46.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:46 smithi102 ceph-mon[100296]: osdmap e97: 12 total, 12 up, 12 in 2023-11-27T15:28:46.579 INFO:journalctl@ceph.mds.b.smithi102.stdout:Nov 27 15:28:46 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-b[125504]: ignoring --setuser ceph since I am not root 2023-11-27T15:28:46.580 INFO:journalctl@ceph.mds.b.smithi102.stdout:Nov 27 15:28:46 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-b[125504]: ignoring --setgroup ceph since I am not root 2023-11-27T15:28:46.580 INFO:journalctl@ceph.mds.b.smithi102.stdout:Nov 27 15:28:46 smithi102 ceph-mds[125528]: -- 172.21.15.102:0/2555053659 <== mon.2 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a5bdbd7520 con 0x55a5bdbacc00 2023-11-27T15:28:46.580 INFO:journalctl@ceph.mds.b.smithi102.stdout:Nov 27 15:28:46 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-b[125504]: starting mds.b at 2023-11-27T15:28:46.615 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:46 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mon-a[99032]: 2023-11-27T15:28:46.287+0000 7f438c327700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-27T15:28:46.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:46 smithi052 ceph-mon[99056]: pgmap v338: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:28:46.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:46 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2222763184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:28:46.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:46 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/3763479082' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:28:46.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:46 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1192980582' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-27T15:28:46.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:46 smithi052 ceph-mon[99056]: osdmap e97: 12 total, 12 up, 12 in 2023-11-27T15:28:46.717 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:46.714+0000 7fcf64340700 1 -- 172.21.15.52:0/2134508621 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcf5c1a3550 con 0x7fcf5c0ff520 2023-11-27T15:28:46.777 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:46 smithi150 ceph-mon[100559]: pgmap v338: 97 pgs: 97 active+clean; 712 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:28:46.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:46 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2222763184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:28:46.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:46 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/3763479082' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:28:46.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:46 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1192980582' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-27T15:28:46.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:46 smithi150 ceph-mon[100559]: osdmap e97: 12 total, 12 up, 12 in 2023-11-27T15:28:46.778 INFO:journalctl@ceph.mds.c.smithi150.stdout:Nov 27 15:28:46 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-c[126251]: ignoring --setuser ceph since I am not root 2023-11-27T15:28:46.778 INFO:journalctl@ceph.mds.c.smithi150.stdout:Nov 27 15:28:46 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-c[126251]: ignoring --setgroup ceph since I am not root 2023-11-27T15:28:46.779 INFO:journalctl@ceph.mds.c.smithi150.stdout:Nov 27 15:28:46 smithi150 ceph-mds[126275]: -- 172.21.15.150:0/3858351692 <== mon.0 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cdf924f520 con 0x55cdf9224800 2023-11-27T15:28:46.779 INFO:journalctl@ceph.mds.c.smithi150.stdout:Nov 27 15:28:46 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-c[126251]: starting mds.c at 2023-11-27T15:28:46.779 INFO:journalctl@ceph.mds.f.smithi150.stdout:Nov 27 15:28:46 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-f[125908]: ignoring --setuser ceph since I am not root 2023-11-27T15:28:46.779 INFO:journalctl@ceph.mds.f.smithi150.stdout:Nov 27 15:28:46 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-f[125908]: ignoring --setgroup ceph since I am not root 2023-11-27T15:28:46.779 INFO:journalctl@ceph.mds.f.smithi150.stdout:Nov 27 15:28:46 smithi150 ceph-mds[125932]: -- 172.21.15.150:0/379108660 <== mon.1 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557071565520 con 0x55707153a800 2023-11-27T15:28:46.779 INFO:journalctl@ceph.mds.f.smithi150.stdout:Nov 27 15:28:46 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-f[125908]: starting mds.f at 2023-11-27T15:28:46.780 INFO:journalctl@ceph.mds.i.smithi150.stdout:Nov 27 15:28:46 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-i[126596]: ignoring --setuser ceph since I am not root 2023-11-27T15:28:46.780 INFO:journalctl@ceph.mds.i.smithi150.stdout:Nov 27 15:28:46 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-i[126596]: ignoring --setgroup ceph since I am not root 2023-11-27T15:28:46.780 INFO:journalctl@ceph.mds.i.smithi150.stdout:Nov 27 15:28:46 smithi150 ceph-mds[126620]: -- 172.21.15.150:0/3196637059 <== mon.0 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556622375520 con 0x55662234ac00 2023-11-27T15:28:46.780 INFO:journalctl@ceph.mds.i.smithi150.stdout:Nov 27 15:28:46 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-i[126596]: starting mds.i at 2023-11-27T15:28:47.307 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.301+0000 7fcf4f7fe700 1 -- 172.21.15.52:0/2134508621 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v36) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcf50099a70 con 0x7fcf5c0ff520 2023-11-27T15:28:47.308 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.304+0000 7fcf64340700 1 -- 172.21.15.52:0/2134508621 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fcf480a12c0 msgr2=0x7fcf480a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:47.308 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.304+0000 7fcf64340700 1 --2- 172.21.15.52:0/2134508621 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fcf480a12c0 0x7fcf480a3770 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcf58005e10 tx=0x7fcf58005d60 comp rx=0 tx=0).stop 2023-11-27T15:28:47.308 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.304+0000 7fcf64340700 1 -- 172.21.15.52:0/2134508621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf5c0ff520 msgr2=0x7fcf5c1973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:47.308 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.304+0000 7fcf64340700 1 --2- 172.21.15.52:0/2134508621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf5c0ff520 0x7fcf5c1973a0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000d980 tx=0x7fcf5000dc90 comp rx=0 tx=0).stop 2023-11-27T15:28:47.308 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.304+0000 7fcf64340700 1 -- 172.21.15.52:0/2134508621 shutdown_connections 2023-11-27T15:28:47.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.304+0000 7fcf64340700 1 --2- 172.21.15.52:0/2134508621 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fcf480a12c0 0x7fcf480a3770 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:47.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.304+0000 7fcf64340700 1 --2- 172.21.15.52:0/2134508621 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf5c104560 0x7fcf5c1a2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:47.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.304+0000 7fcf64340700 1 --2- 172.21.15.52:0/2134508621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf5c0ff520 0x7fcf5c1973a0 secure :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000d980 tx=0x7fcf5000dc90 comp rx=0 tx=0).stop 2023-11-27T15:28:47.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.304+0000 7fcf64340700 1 --2- 172.21.15.52:0/2134508621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf5c0ffec0 0x7fcf5c1978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:47.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.304+0000 7fcf64340700 1 -- 172.21.15.52:0/2134508621 >> 172.21.15.52:0/2134508621 conn(0x7fcf5c0760d0 msgr2=0x7fcf5c0fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:47.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.304+0000 7fcf64340700 1 -- 172.21.15.52:0/2134508621 shutdown_connections 2023-11-27T15:28:47.309 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.304+0000 7fcf64340700 1 -- 172.21.15.52:0/2134508621 wait complete. 2023-11-27T15:28:47.324 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-27T15:28:47.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.463+0000 7fb8a5e10700 1 -- 172.21.15.52:0/338953093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8a0105600 msgr2=0x7fb8a010d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:47.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.463+0000 7fb8a5e10700 1 --2- 172.21.15.52:0/338953093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8a0105600 0x7fb8a010d1d0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fb89800b0d0 tx=0x7fb89800b3e0 comp rx=0 tx=0).stop 2023-11-27T15:28:47.467 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.463+0000 7fb8a5e10700 1 -- 172.21.15.52:0/338953093 shutdown_connections 2023-11-27T15:28:47.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.463+0000 7fb8a5e10700 1 --2- 172.21.15.52:0/338953093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8a00690e0 0x7fb8a00694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:47.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.463+0000 7fb8a5e10700 1 --2- 172.21.15.52:0/338953093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8a0105600 0x7fb8a010d1d0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:47.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.463+0000 7fb8a5e10700 1 --2- 172.21.15.52:0/338953093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8a0069a80 0x7fb8a0104f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:47.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.463+0000 7fb8a5e10700 1 -- 172.21.15.52:0/338953093 >> 172.21.15.52:0/338953093 conn(0x7fb8a0075470 msgr2=0x7fb8a0075870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:47.468 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.463+0000 7fb8a5e10700 1 -- 172.21.15.52:0/338953093 shutdown_connections 2023-11-27T15:28:47.469 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.463+0000 7fb8a5e10700 1 -- 172.21.15.52:0/338953093 wait complete. 2023-11-27T15:28:47.470 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.464+0000 7fb8a5e10700 1 Processor -- start 2023-11-27T15:28:47.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.464+0000 7fb8a5e10700 1 -- start start 2023-11-27T15:28:47.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.464+0000 7fb8a5e10700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8a00690e0 0x7fb8a00ffa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:47.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.464+0000 7fb8a5e10700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8a0069a80 0x7fb8a00fffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:47.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.464+0000 7fb8a5e10700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8a0105600 0x7fb8a01a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:47.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.464+0000 7fb8a5e10700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a0100800 con 0x7fb8a0069a80 2023-11-27T15:28:47.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.464+0000 7fb8a5e10700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a01a4c20 con 0x7fb8a0105600 2023-11-27T15:28:47.471 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.464+0000 7fb8a5e10700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a01a4d60 con 0x7fb8a00690e0 2023-11-27T15:28:47.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.464+0000 7fb89effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8a0069a80 0x7fb8a00fffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:47.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.464+0000 7fb89effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8a0069a80 0x7fb8a00fffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56628/0 (socket says 172.21.15.52:56628) 2023-11-27T15:28:47.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.464+0000 7fb89effd700 1 -- 172.21.15.52:0/1002431527 learned_addr learned my addr 172.21.15.52:0/1002431527 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:47.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.464+0000 7fb89f7fe700 1 --2- 172.21.15.52:0/1002431527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8a00690e0 0x7fb8a00ffa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:47.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.464+0000 7fb89ffff700 1 --2- 172.21.15.52:0/1002431527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8a0105600 0x7fb8a01a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:47.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.464+0000 7fb89effd700 1 -- 172.21.15.52:0/1002431527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8a00690e0 msgr2=0x7fb8a00ffa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:47.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.464+0000 7fb89effd700 1 --2- 172.21.15.52:0/1002431527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8a00690e0 0x7fb8a00ffa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:47.472 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.464+0000 7fb89effd700 1 -- 172.21.15.52:0/1002431527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8a0105600 msgr2=0x7fb8a01a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:47.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.464+0000 7fb89effd700 1 --2- 172.21.15.52:0/1002431527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8a0105600 0x7fb8a01a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:47.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.465+0000 7fb89effd700 1 -- 172.21.15.52:0/1002431527 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb898009d00 con 0x7fb8a0069a80 2023-11-27T15:28:47.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.465+0000 7fb89effd700 1 --2- 172.21.15.52:0/1002431527 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8a0069a80 0x7fb8a00fffd0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fb89000e940 tx=0x7fb89000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:47.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.465+0000 7fb89cff9700 1 -- 172.21.15.52:0/1002431527 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb890009730 con 0x7fb8a0069a80 2023-11-27T15:28:47.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.465+0000 7fb89cff9700 1 -- 172.21.15.52:0/1002431527 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb890010430 con 0x7fb8a0069a80 2023-11-27T15:28:47.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.465+0000 7fb8a5e10700 1 -- 172.21.15.52:0/1002431527 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a0110430 con 0x7fb8a0069a80 2023-11-27T15:28:47.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.465+0000 7fb89cff9700 1 -- 172.21.15.52:0/1002431527 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89000cea0 con 0x7fb8a0069a80 2023-11-27T15:28:47.473 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.465+0000 7fb8a5e10700 1 -- 172.21.15.52:0/1002431527 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a01102a0 con 0x7fb8a0069a80 2023-11-27T15:28:47.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.466+0000 7fb8a5e10700 1 -- 172.21.15.52:0/1002431527 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8a01105c0 con 0x7fb8a0069a80 2023-11-27T15:28:47.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.467+0000 7fb89cff9700 1 -- 172.21.15.52:0/1002431527 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb890010810 con 0x7fb8a0069a80 2023-11-27T15:28:47.474 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.467+0000 7fb89cff9700 1 --2- 172.21.15.52:0/1002431527 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb8880a1340 0x7fb8880a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:47.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.468+0000 7fb89f7fe700 1 --2- 172.21.15.52:0/1002431527 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb8880a1340 0x7fb8880a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:47.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.468+0000 7fb89cff9700 1 -- 172.21.15.52:0/1002431527 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8785+0+0 (secure 0 0 0) 0x7fb890014070 con 0x7fb8a0069a80 2023-11-27T15:28:47.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.468+0000 7fb89f7fe700 1 --2- 172.21.15.52:0/1002431527 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb8880a1340 0x7fb8880a37f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb894009c60 tx=0x7fb894009260 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:47.475 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.472+0000 7fb89cff9700 1 -- 172.21.15.52:0/1002431527 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb890051730 con 0x7fb8a0069a80 2023-11-27T15:28:47.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:47 smithi102 ceph-mon[100296]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-11-27T15:28:47.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:47 smithi102 ceph-mon[100296]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-27T15:28:47.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:47 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1192980582' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-11-27T15:28:47.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:47 smithi102 ceph-mon[100296]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-11-27T15:28:47.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:47 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2134508621' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-11-27T15:28:47.615 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:47 smithi052 ceph-mon[99056]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-11-27T15:28:47.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:47 smithi052 ceph-mon[99056]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-27T15:28:47.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:47 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1192980582' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-11-27T15:28:47.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:47 smithi052 ceph-mon[99056]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-11-27T15:28:47.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:47 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2134508621' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-11-27T15:28:47.675 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:47.672+0000 7fb8a5e10700 1 -- 172.21.15.52:0/1002431527 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb8a0067270 con 0x7fb8a0069a80 2023-11-27T15:28:47.777 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:47 smithi150 ceph-mon[100559]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-11-27T15:28:47.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:47 smithi150 ceph-mon[100559]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-27T15:28:47.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:47 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1192980582' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-11-27T15:28:47.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:47 smithi150 ceph-mon[100559]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-11-27T15:28:47.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:47 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2134508621' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-11-27T15:28:48.400 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.396+0000 7fb89cff9700 1 -- 172.21.15.52:0/1002431527 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v98) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb890099c30 con 0x7fb8a0069a80 2023-11-27T15:28:48.404 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.400+0000 7fb8a5e10700 1 -- 172.21.15.52:0/1002431527 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb8880a1340 msgr2=0x7fb8880a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:48.404 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.400+0000 7fb8a5e10700 1 --2- 172.21.15.52:0/1002431527 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb8880a1340 0x7fb8880a37f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb894009c60 tx=0x7fb894009260 comp rx=0 tx=0).stop 2023-11-27T15:28:48.404 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.400+0000 7fb8a5e10700 1 -- 172.21.15.52:0/1002431527 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8a0069a80 msgr2=0x7fb8a00fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:48.404 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.400+0000 7fb8a5e10700 1 --2- 172.21.15.52:0/1002431527 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8a0069a80 0x7fb8a00fffd0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fb89000e940 tx=0x7fb89000ec50 comp rx=0 tx=0).stop 2023-11-27T15:28:48.405 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.401+0000 7fb8a5e10700 1 -- 172.21.15.52:0/1002431527 shutdown_connections 2023-11-27T15:28:48.405 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.401+0000 7fb8a5e10700 1 --2- 172.21.15.52:0/1002431527 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fb8880a1340 0x7fb8880a37f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:48.405 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.401+0000 7fb8a5e10700 1 --2- 172.21.15.52:0/1002431527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb8a0105600 0x7fb8a01a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:48.405 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.401+0000 7fb8a5e10700 1 --2- 172.21.15.52:0/1002431527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8a00690e0 0x7fb8a00ffa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:48.405 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.401+0000 7fb8a5e10700 1 --2- 172.21.15.52:0/1002431527 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8a0069a80 0x7fb8a00fffd0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:48.405 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.401+0000 7fb8a5e10700 1 -- 172.21.15.52:0/1002431527 >> 172.21.15.52:0/1002431527 conn(0x7fb8a0075470 msgr2=0x7fb8a00fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:48.406 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.401+0000 7fb8a5e10700 1 -- 172.21.15.52:0/1002431527 shutdown_connections 2023-11-27T15:28:48.406 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.401+0000 7fb8a5e10700 1 -- 172.21.15.52:0/1002431527 wait complete. 2023-11-27T15:28:48.406 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_metadata' removed 2023-11-27T15:28:48.425 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-27T15:28:48.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.567+0000 7fbab7389700 1 -- 172.21.15.52:0/2462080249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbab0100a50 msgr2=0x7fbab0100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:48.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.567+0000 7fbab7389700 1 --2- 172.21.15.52:0/2462080249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbab0100a50 0x7fbab0100ec0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fbaa00131e0 tx=0x7fbaa0011f90 comp rx=0 tx=0).stop 2023-11-27T15:28:48.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.567+0000 7fbab7389700 1 -- 172.21.15.52:0/2462080249 shutdown_connections 2023-11-27T15:28:48.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.567+0000 7fbab7389700 1 --2- 172.21.15.52:0/2462080249 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbab0101400 0x7fbab010d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:48.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.567+0000 7fbab7389700 1 --2- 172.21.15.52:0/2462080249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbab01022e0 0x7fbab01026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:48.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.567+0000 7fbab7389700 1 --2- 172.21.15.52:0/2462080249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbab0100a50 0x7fbab0100ec0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:48.571 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.567+0000 7fbab7389700 1 -- 172.21.15.52:0/2462080249 >> 172.21.15.52:0/2462080249 conn(0x7fbab0077590 msgr2=0x7fbab0077990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:48.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.567+0000 7fbab7389700 1 -- 172.21.15.52:0/2462080249 shutdown_connections 2023-11-27T15:28:48.572 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.567+0000 7fbab7389700 1 -- 172.21.15.52:0/2462080249 wait complete. 2023-11-27T15:28:48.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.568+0000 7fbab7389700 1 Processor -- start 2023-11-27T15:28:48.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.568+0000 7fbab7389700 1 -- start start 2023-11-27T15:28:48.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.568+0000 7fbab7389700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbab0100a50 0x7fbab0194450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:48.573 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.568+0000 7fbab7389700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbab0101400 0x7fbab0194990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:48.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.568+0000 7fbab7389700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbab01022e0 0x7fbab0198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:48.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.568+0000 7fbab7389700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab0199430 con 0x7fbab0101400 2023-11-27T15:28:48.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.568+0000 7fbab7389700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab01995a0 con 0x7fbab01022e0 2023-11-27T15:28:48.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.568+0000 7fbab7389700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab0199760 con 0x7fbab0100a50 2023-11-27T15:28:48.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.568+0000 7fbab4924700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbab0101400 0x7fbab0194990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:48.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.568+0000 7fbab4924700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbab0101400 0x7fbab0194990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56642/0 (socket says 172.21.15.52:56642) 2023-11-27T15:28:48.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.568+0000 7fbab4924700 1 -- 172.21.15.52:0/1082362022 learned_addr learned my addr 172.21.15.52:0/1082362022 (peer_addr_for_me v2:172.21.15.52:0/0) 2023-11-27T15:28:48.574 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.569+0000 7fbab4924700 1 -- 172.21.15.52:0/1082362022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbab0100a50 msgr2=0x7fbab0194450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:48.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.569+0000 7fbab4924700 1 --2- 172.21.15.52:0/1082362022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbab0100a50 0x7fbab0194450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:48.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.569+0000 7fbab4924700 1 -- 172.21.15.52:0/1082362022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbab01022e0 msgr2=0x7fbab0198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:48.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.569+0000 7fbab4924700 1 --2- 172.21.15.52:0/1082362022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbab01022e0 0x7fbab0198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:48.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.569+0000 7fbab4924700 1 -- 172.21.15.52:0/1082362022 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa0011e00 con 0x7fbab0101400 2023-11-27T15:28:48.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.569+0000 7fbab4924700 1 --2- 172.21.15.52:0/1082362022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbab0101400 0x7fbab0194990 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fbaa00119c0 tx=0x7fbaa00114a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:48.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.569+0000 7fbaa67fc700 1 -- 172.21.15.52:0/1082362022 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0016040 con 0x7fbab0101400 2023-11-27T15:28:48.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.569+0000 7fbab7389700 1 -- 172.21.15.52:0/1082362022 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab010e380 con 0x7fbab0101400 2023-11-27T15:28:48.575 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.569+0000 7fbaa67fc700 1 -- 172.21.15.52:0/1082362022 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbaa0025480 con 0x7fbab0101400 2023-11-27T15:28:48.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.569+0000 7fbab7389700 1 -- 172.21.15.52:0/1082362022 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab010e1f0 con 0x7fbab0101400 2023-11-27T15:28:48.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.569+0000 7fbaa67fc700 1 -- 172.21.15.52:0/1082362022 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0024640 con 0x7fbab0101400 2023-11-27T15:28:48.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.571+0000 7fbab7389700 1 -- 172.21.15.52:0/1082362022 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab010e510 con 0x7fbab0101400 2023-11-27T15:28:48.576 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.573+0000 7fbaa67fc700 1 -- 172.21.15.52:0/1082362022 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fbaa00255f0 con 0x7fbab0101400 2023-11-27T15:28:48.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.574+0000 7fbaa67fc700 1 --2- 172.21.15.52:0/1082362022 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fba9c0a12c0 0x7fba9c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:48.577 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.574+0000 7fbaa67fc700 1 -- 172.21.15.52:0/1082362022 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8231+0+0 (secure 0 0 0) 0x7fbaa0013e40 con 0x7fbab0101400 2023-11-27T15:28:48.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:48 smithi102 ceph-mon[100296]: pgmap v340: 97 pgs: 97 active+clean; 712 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2023-11-27T15:28:48.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:48 smithi102 ceph-mon[100296]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-11-27T15:28:48.578 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:48 smithi102 ceph-mon[100296]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-11-27T15:28:48.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:48 smithi102 ceph-mon[100296]: Cluster is now healthy 2023-11-27T15:28:48.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:48 smithi102 ceph-mon[100296]: mds.? [v2:172.21.15.150:6838/2236242969,v1:172.21.15.150:6839/2236242969] up:boot 2023-11-27T15:28:48.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:48 smithi102 ceph-mon[100296]: mds.? [v2:172.21.15.150:6835/3520718866,v1:172.21.15.150:6837/3520718866] up:boot 2023-11-27T15:28:48.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:48 smithi102 ceph-mon[100296]: mds.? [v2:172.21.15.150:6834/3597070899,v1:172.21.15.150:6836/3597070899] up:boot 2023-11-27T15:28:48.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:48 smithi102 ceph-mon[100296]: mds.? [v2:172.21.15.102:6834/1945120768,v1:172.21.15.102:6835/1945120768] up:boot 2023-11-27T15:28:48.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:48 smithi102 ceph-mon[100296]: mds.? [v2:172.21.15.102:6837/4176976612,v1:172.21.15.102:6839/4176976612] up:boot 2023-11-27T15:28:48.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:48 smithi102 ceph-mon[100296]: mds.? [v2:172.21.15.102:6836/200078583,v1:172.21.15.102:6838/200078583] up:boot 2023-11-27T15:28:48.579 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:48 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/2134508621' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-11-27T15:28:48.580 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:48 smithi102 ceph-mon[100296]: fsmap 12 up:standby 2023-11-27T15:28:48.580 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:48 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-27T15:28:48.580 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:48 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-27T15:28:48.580 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:48 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-27T15:28:48.580 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:48 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-27T15:28:48.580 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:48 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-27T15:28:48.580 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:48 smithi102 ceph-mon[100296]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-27T15:28:48.580 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:48 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1002431527' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-27T15:28:48.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.575+0000 7fbab5125700 1 --2- 172.21.15.52:0/1082362022 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fba9c0a12c0 0x7fba9c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:48.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.575+0000 7fbab5125700 1 --2- 172.21.15.52:0/1082362022 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fba9c0a12c0 0x7fba9c0a3770 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbaac0097b0 tx=0x7fbaac006d20 comp rx=0 tx=0).ready entity=mgr.15387 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:48.581 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.578+0000 7fbaa67fc700 1 -- 172.21.15.52:0/1082362022 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbaa00a02c0 con 0x7fbab0101400 2023-11-27T15:28:48.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:48 smithi052 ceph-mon[99056]: pgmap v340: 97 pgs: 97 active+clean; 712 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2023-11-27T15:28:48.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:48 smithi052 ceph-mon[99056]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-11-27T15:28:48.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:48 smithi052 ceph-mon[99056]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-11-27T15:28:48.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:48 smithi052 ceph-mon[99056]: Cluster is now healthy 2023-11-27T15:28:48.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:48 smithi052 ceph-mon[99056]: mds.? [v2:172.21.15.150:6838/2236242969,v1:172.21.15.150:6839/2236242969] up:boot 2023-11-27T15:28:48.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:48 smithi052 ceph-mon[99056]: mds.? [v2:172.21.15.150:6835/3520718866,v1:172.21.15.150:6837/3520718866] up:boot 2023-11-27T15:28:48.616 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:48 smithi052 ceph-mon[99056]: mds.? [v2:172.21.15.150:6834/3597070899,v1:172.21.15.150:6836/3597070899] up:boot 2023-11-27T15:28:48.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:48 smithi052 ceph-mon[99056]: mds.? [v2:172.21.15.102:6834/1945120768,v1:172.21.15.102:6835/1945120768] up:boot 2023-11-27T15:28:48.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:48 smithi052 ceph-mon[99056]: mds.? [v2:172.21.15.102:6837/4176976612,v1:172.21.15.102:6839/4176976612] up:boot 2023-11-27T15:28:48.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:48 smithi052 ceph-mon[99056]: mds.? [v2:172.21.15.102:6836/200078583,v1:172.21.15.102:6838/200078583] up:boot 2023-11-27T15:28:48.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:48 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/2134508621' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-11-27T15:28:48.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:48 smithi052 ceph-mon[99056]: fsmap 12 up:standby 2023-11-27T15:28:48.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:48 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-27T15:28:48.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:48 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-27T15:28:48.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:48 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-27T15:28:48.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:48 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-27T15:28:48.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:48 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-27T15:28:48.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:48 smithi052 ceph-mon[99056]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-27T15:28:48.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:48 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1002431527' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-27T15:28:48.777 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:48 smithi150 ceph-mon[100559]: pgmap v340: 97 pgs: 97 active+clean; 712 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2023-11-27T15:28:48.777 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:48 smithi150 ceph-mon[100559]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-11-27T15:28:48.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:48 smithi150 ceph-mon[100559]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-11-27T15:28:48.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:48 smithi150 ceph-mon[100559]: Cluster is now healthy 2023-11-27T15:28:48.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:48 smithi150 ceph-mon[100559]: mds.? [v2:172.21.15.150:6838/2236242969,v1:172.21.15.150:6839/2236242969] up:boot 2023-11-27T15:28:48.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:48 smithi150 ceph-mon[100559]: mds.? [v2:172.21.15.150:6835/3520718866,v1:172.21.15.150:6837/3520718866] up:boot 2023-11-27T15:28:48.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:48 smithi150 ceph-mon[100559]: mds.? [v2:172.21.15.150:6834/3597070899,v1:172.21.15.150:6836/3597070899] up:boot 2023-11-27T15:28:48.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:48 smithi150 ceph-mon[100559]: mds.? [v2:172.21.15.102:6834/1945120768,v1:172.21.15.102:6835/1945120768] up:boot 2023-11-27T15:28:48.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:48 smithi150 ceph-mon[100559]: mds.? [v2:172.21.15.102:6837/4176976612,v1:172.21.15.102:6839/4176976612] up:boot 2023-11-27T15:28:48.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:48 smithi150 ceph-mon[100559]: mds.? [v2:172.21.15.102:6836/200078583,v1:172.21.15.102:6838/200078583] up:boot 2023-11-27T15:28:48.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:48 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/2134508621' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-11-27T15:28:48.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:48 smithi150 ceph-mon[100559]: fsmap 12 up:standby 2023-11-27T15:28:48.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:48 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-27T15:28:48.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:48 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-27T15:28:48.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:48 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-27T15:28:48.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:48 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-27T15:28:48.779 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:48 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-27T15:28:48.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:48 smithi150 ceph-mon[100559]: from='mgr.15387 172.21.15.52:0/2617532687' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-27T15:28:48.780 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:48 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1002431527' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-27T15:28:48.786 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:48.782+0000 7fbab7389700 1 -- 172.21.15.52:0/1082362022 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbab01957a0 con 0x7fbab0101400 2023-11-27T15:28:49.402 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:49.398+0000 7fbaa67fc700 1 -- 172.21.15.52:0/1082362022 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v99) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbaa001f0d0 con 0x7fbab0101400 2023-11-27T15:28:49.409 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:49.405+0000 7fbab7389700 1 -- 172.21.15.52:0/1082362022 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fba9c0a12c0 msgr2=0x7fba9c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:49.410 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:49.405+0000 7fbab7389700 1 --2- 172.21.15.52:0/1082362022 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fba9c0a12c0 0x7fba9c0a3770 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbaac0097b0 tx=0x7fbaac006d20 comp rx=0 tx=0).stop 2023-11-27T15:28:49.410 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:49.405+0000 7fbab7389700 1 -- 172.21.15.52:0/1082362022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbab0101400 msgr2=0x7fbab0194990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:49.410 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:49.405+0000 7fbab7389700 1 --2- 172.21.15.52:0/1082362022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbab0101400 0x7fbab0194990 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fbaa00119c0 tx=0x7fbaa00114a0 comp rx=0 tx=0).stop 2023-11-27T15:28:49.410 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:49.405+0000 7fbab7389700 1 -- 172.21.15.52:0/1082362022 shutdown_connections 2023-11-27T15:28:49.410 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:49.405+0000 7fbab7389700 1 --2- 172.21.15.52:0/1082362022 >> [v2:172.21.15.52:6800/1148545855,v1:172.21.15.52:6801/1148545855] conn(0x7fba9c0a12c0 0x7fba9c0a3770 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:49.410 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:49.405+0000 7fbab7389700 1 --2- 172.21.15.52:0/1082362022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbab01022e0 0x7fbab0198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:49.411 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:49.405+0000 7fbab7389700 1 --2- 172.21.15.52:0/1082362022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbab0100a50 0x7fbab0194450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:49.411 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:49.405+0000 7fbab7389700 1 --2- 172.21.15.52:0/1082362022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbab0101400 0x7fbab0194990 secure :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fbaa00119c0 tx=0x7fbaa00114a0 comp rx=0 tx=0).stop 2023-11-27T15:28:49.411 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:49.405+0000 7fbab7389700 1 -- 172.21.15.52:0/1082362022 >> 172.21.15.52:0/1082362022 conn(0x7fbab0077590 msgr2=0x7fbab00ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:49.411 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:49.406+0000 7fbab7389700 1 -- 172.21.15.52:0/1082362022 shutdown_connections 2023-11-27T15:28:49.411 INFO:teuthology.orchestra.run.smithi052.stderr:2023-11-27T15:28:49.406+0000 7fbab7389700 1 -- 172.21.15.52:0/1082362022 wait complete. 2023-11-27T15:28:49.411 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data' removed 2023-11-27T15:28:49.429 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-27T15:28:49.460 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-27T15:28:49.510 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-27T15:28:49.542 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-11-27T15:28:49.543 DEBUG:teuthology.orchestra.run.smithi052:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-11-27T15:28:49.561 DEBUG:teuthology.orchestra.run.smithi102:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-11-27T15:28:49.578 DEBUG:teuthology.orchestra.run.smithi150:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-11-27T15:28:49.605 INFO:tasks.cephadm:Stopping all daemons... 2023-11-27T15:28:49.605 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-11-27T15:28:49.606 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mon.a 2023-11-27T15:28:49.777 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:49 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1002431527' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-11-27T15:28:49.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:49 smithi150 ceph-mon[100559]: osdmap e98: 12 total, 12 up, 12 in 2023-11-27T15:28:49.778 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:49 smithi150 ceph-mon[100559]: from='client.? 172.21.15.52:0/1082362022' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-27T15:28:49.827 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:49 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1002431527' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-11-27T15:28:49.827 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:49 smithi102 ceph-mon[100296]: osdmap e98: 12 total, 12 up, 12 in 2023-11-27T15:28:49.827 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:49 smithi102 ceph-mon[100296]: from='client.? 172.21.15.52:0/1082362022' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-27T15:28:49.866 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:49 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1002431527' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-11-27T15:28:49.866 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:49 smithi052 ceph-mon[99056]: osdmap e98: 12 total, 12 up, 12 in 2023-11-27T15:28:49.866 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:49 smithi052 ceph-mon[99056]: from='client.? 172.21.15.52:0/1082362022' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-27T15:28:49.866 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:49 smithi052 systemd[1]: Stopping Ceph mon.a for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:28:50.365 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:50 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mon-a[99032]: 2023-11-27T15:28:50.079+0000 7f438f32d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:28:50.366 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:50 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mon-a[99032]: 2023-11-27T15:28:50.079+0000 7f438f32d700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-11-27T15:28:50.866 INFO:journalctl@ceph.mon.a.smithi052.stdout:Nov 27 15:28:50 smithi052 bash[262747]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mon-a 2023-11-27T15:28:50.876 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mon.a.service' 2023-11-27T15:28:50.921 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:28:50.921 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-11-27T15:28:50.921 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-11-27T15:28:50.922 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mon.b 2023-11-27T15:28:51.313 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:50 smithi102 systemd[1]: Stopping Ceph mon.b for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:28:51.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:51 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mon-b[100272]: 2023-11-27T15:28:51.308+0000 7f2d5520d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:28:51.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Nov 27 15:28:51 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mon-b[100272]: 2023-11-27T15:28:51.308+0000 7f2d5520d700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-11-27T15:28:51.912 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mon.b.service' 2023-11-27T15:28:51.968 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:28:51.969 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-11-27T15:28:51.969 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-11-27T15:28:51.969 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mon.c 2023-11-27T15:28:52.279 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:52 smithi150 systemd[1]: Stopping Ceph mon.c for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:28:52.777 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:52 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mon-c[100520]: 2023-11-27T15:28:52.442+0000 7fa841226700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:28:52.777 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:52 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mon-c[100520]: 2023-11-27T15:28:52.442+0000 7fa841226700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-11-27T15:28:53.077 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mon.c.service' 2023-11-27T15:28:53.103 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:52 smithi150 bash[132649]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mon-c 2023-11-27T15:28:53.103 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:53 smithi150 systemd[1]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mon.c.service: Succeeded. 2023-11-27T15:28:53.103 INFO:journalctl@ceph.mon.c.smithi150.stdout:Nov 27 15:28:53 smithi150 systemd[1]: Stopped Ceph mon.c for c6c7df88-8d36-11ee-95a2-87774f69a715. 2023-11-27T15:28:53.124 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:28:53.124 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-11-27T15:28:53.124 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-11-27T15:28:53.125 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mgr.x 2023-11-27T15:28:53.490 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:28:53 smithi052 systemd[1]: Stopping Ceph mgr.x for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:28:53.796 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:28:53 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:28:53.486+0000 7f403d121700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:28:53.797 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:28:53 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x[99303]: 2023-11-27T15:28:53.486+0000 7f403d121700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-11-27T15:28:54.033 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mgr.x.service' 2023-11-27T15:28:54.058 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:28:53 smithi052 bash[262898]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-x 2023-11-27T15:28:54.059 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:28:54 smithi052 systemd[1]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mgr.x.service: Succeeded. 2023-11-27T15:28:54.059 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Nov 27 15:28:54 smithi052 systemd[1]: Stopped Ceph mgr.x for c6c7df88-8d36-11ee-95a2-87774f69a715. 2023-11-27T15:28:54.073 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:28:54.073 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-11-27T15:28:54.073 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-11-27T15:28:54.073 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mgr.y 2023-11-27T15:28:54.493 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:28:54 smithi102 systemd[1]: Stopping Ceph mgr.y for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:28:54.776 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:28:54 smithi102 bash[131512]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-y 2023-11-27T15:28:54.958 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Nov 27 15:28:54 smithi102 systemd[1]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-11-27T15:28:54.962 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mgr.y.service' 2023-11-27T15:28:54.998 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:28:54.998 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-11-27T15:28:54.999 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-11-27T15:28:54.999 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mgr.z 2023-11-27T15:28:55.418 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:28:55 smithi150 systemd[1]: Stopping Ceph mgr.z for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:28:55.709 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:28:55 smithi150 bash[132793]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mgr-z 2023-11-27T15:28:55.890 INFO:journalctl@ceph.mgr.z.smithi150.stdout:Nov 27 15:28:55 smithi150 systemd[1]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-11-27T15:28:55.894 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mgr.z.service' 2023-11-27T15:28:55.931 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:28:55.931 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-11-27T15:28:55.931 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-11-27T15:28:55.931 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.0 2023-11-27T15:28:56.329 INFO:journalctl@ceph.osd.0.smithi052.stdout:Nov 27 15:28:55 smithi052 systemd[1]: Stopping Ceph osd.0 for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:28:56.615 INFO:journalctl@ceph.osd.0.smithi052.stdout:Nov 27 15:28:56 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-0[111956]: 2023-11-27T15:28:56.325+0000 7fb7cf94e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:28:56.616 INFO:journalctl@ceph.osd.0.smithi052.stdout:Nov 27 15:28:56 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-0[111956]: 2023-11-27T15:28:56.325+0000 7fb7cf94e700 -1 osd.0 99 *** Got signal Terminated *** 2023-11-27T15:28:56.616 INFO:journalctl@ceph.osd.0.smithi052.stdout:Nov 27 15:28:56 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-0[111956]: 2023-11-27T15:28:56.325+0000 7fb7cf94e700 -1 osd.0 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:29:02.367 INFO:journalctl@ceph.osd.0.smithi052.stdout:Nov 27 15:29:02 smithi052 bash[263037]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-0 2023-11-27T15:29:04.864 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.0.service' 2023-11-27T15:29:04.909 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:29:04.910 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-11-27T15:29:04.910 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-11-27T15:29:04.910 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.1 2023-11-27T15:29:05.273 INFO:journalctl@ceph.osd.1.smithi102.stdout:Nov 27 15:29:04 smithi102 systemd[1]: Stopping Ceph osd.1 for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:29:05.577 INFO:journalctl@ceph.osd.1.smithi102.stdout:Nov 27 15:29:05 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-1[105297]: 2023-11-27T15:29:05.268+0000 7fd743069700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:29:05.577 INFO:journalctl@ceph.osd.1.smithi102.stdout:Nov 27 15:29:05 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-1[105297]: 2023-11-27T15:29:05.268+0000 7fd743069700 -1 osd.1 99 *** Got signal Terminated *** 2023-11-27T15:29:05.577 INFO:journalctl@ceph.osd.1.smithi102.stdout:Nov 27 15:29:05 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-1[105297]: 2023-11-27T15:29:05.268+0000 7fd743069700 -1 osd.1 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:29:11.077 INFO:journalctl@ceph.osd.1.smithi102.stdout:Nov 27 15:29:10 smithi102 bash[131666]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-1 2023-11-27T15:29:12.112 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.1.service' 2023-11-27T15:29:12.152 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:29:12.152 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-11-27T15:29:12.153 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-11-27T15:29:12.153 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.2 2023-11-27T15:29:12.512 INFO:journalctl@ceph.osd.2.smithi150.stdout:Nov 27 15:29:12 smithi150 systemd[1]: Stopping Ceph osd.2 for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:29:12.777 INFO:journalctl@ceph.osd.2.smithi150.stdout:Nov 27 15:29:12 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-2[106369]: 2023-11-27T15:29:12.507+0000 7fa2f7e13700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:29:12.777 INFO:journalctl@ceph.osd.2.smithi150.stdout:Nov 27 15:29:12 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-2[106369]: 2023-11-27T15:29:12.507+0000 7fa2f7e13700 -1 osd.2 99 *** Got signal Terminated *** 2023-11-27T15:29:12.778 INFO:journalctl@ceph.osd.2.smithi150.stdout:Nov 27 15:29:12 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-2[106369]: 2023-11-27T15:29:12.507+0000 7fa2f7e13700 -1 osd.2 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:29:18.527 INFO:journalctl@ceph.osd.2.smithi150.stdout:Nov 27 15:29:18 smithi150 bash[132941]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-2 2023-11-27T15:29:19.994 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.2.service' 2023-11-27T15:29:20.037 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:29:20.037 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-11-27T15:29:20.037 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-11-27T15:29:20.038 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.3 2023-11-27T15:29:20.366 INFO:journalctl@ceph.osd.3.smithi052.stdout:Nov 27 15:29:20 smithi052 systemd[1]: Stopping Ceph osd.3 for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:29:20.866 INFO:journalctl@ceph.osd.3.smithi052.stdout:Nov 27 15:29:20 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-3[117304]: 2023-11-27T15:29:20.409+0000 7f8eff2af700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:29:20.866 INFO:journalctl@ceph.osd.3.smithi052.stdout:Nov 27 15:29:20 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-3[117304]: 2023-11-27T15:29:20.409+0000 7f8eff2af700 -1 osd.3 99 *** Got signal Terminated *** 2023-11-27T15:29:20.867 INFO:journalctl@ceph.osd.3.smithi052.stdout:Nov 27 15:29:20 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-3[117304]: 2023-11-27T15:29:20.409+0000 7f8eff2af700 -1 osd.3 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:29:23.177 INFO:journalctl@ceph.osd.3.smithi052.stdout:Nov 27 15:29:22 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-3[117304]: 2023-11-27T15:29:22.866+0000 7f8ef98a1700 -1 osd.3 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:28:57.593304+0000 front 2023-11-27T15:28:57.593438+0000 (oldest deadline 2023-11-27T15:29:22.293442+0000) 2023-11-27T15:29:23.616 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:23 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:23.174+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:24.116 INFO:journalctl@ceph.osd.3.smithi052.stdout:Nov 27 15:29:23 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-3[117304]: 2023-11-27T15:29:23.844+0000 7f8ef98a1700 -1 osd.3 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:28:57.593304+0000 front 2023-11-27T15:28:57.593438+0000 (oldest deadline 2023-11-27T15:29:22.293442+0000) 2023-11-27T15:29:24.277 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:23 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:29:23.804+0000 7f0ceb06f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:28:58.725336+0000 front 2023-11-27T15:28:58.725173+0000 (oldest deadline 2023-11-27T15:29:23.425277+0000) 2023-11-27T15:29:24.615 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:24 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:24.168+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:25.027 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:24 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:24.814+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:25.028 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:24 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:29:24.755+0000 7f0ceb06f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:28:58.725336+0000 front 2023-11-27T15:28:58.725173+0000 (oldest deadline 2023-11-27T15:29:23.425277+0000) 2023-11-27T15:29:25.116 INFO:journalctl@ceph.osd.3.smithi052.stdout:Nov 27 15:29:24 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-3[117304]: 2023-11-27T15:29:24.832+0000 7f8ef98a1700 -1 osd.3 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:28:57.593304+0000 front 2023-11-27T15:28:57.593438+0000 (oldest deadline 2023-11-27T15:29:22.293442+0000) 2023-11-27T15:29:25.616 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:25 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:25.164+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:25.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:25 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:25.470+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:26.027 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:25 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:25.794+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:26.028 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:25 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:29:25.722+0000 7f0ceb06f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:28:58.725336+0000 front 2023-11-27T15:28:58.725173+0000 (oldest deadline 2023-11-27T15:29:23.425277+0000) 2023-11-27T15:29:26.549 INFO:journalctl@ceph.osd.3.smithi052.stdout:Nov 27 15:29:26 smithi052 bash[263303]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-3 2023-11-27T15:29:26.549 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:26 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:26.164+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:26.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:26 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:26.441+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:26.865 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:26 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:26.546+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:01.183006+0000 front 2023-11-27T15:29:01.183057+0000 (oldest deadline 2023-11-27T15:29:25.883242+0000) 2023-11-27T15:29:27.027 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:26 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:26.800+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:27.028 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:26 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:29:26.742+0000 7f0ceb06f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:28:58.725336+0000 front 2023-11-27T15:28:58.725173+0000 (oldest deadline 2023-11-27T15:29:23.425277+0000) 2023-11-27T15:29:27.555 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:27 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:27.189+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:27.665 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.3.service' 2023-11-27T15:29:27.708 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:29:27.709 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-11-27T15:29:27.709 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-11-27T15:29:27.709 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.4 2023-11-27T15:29:27.737 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:27 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:27.432+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:27.865 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:27 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:27.551+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:01.183006+0000 front 2023-11-27T15:29:01.183057+0000 (oldest deadline 2023-11-27T15:29:25.883242+0000) 2023-11-27T15:29:28.027 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:27 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:29:27.741+0000 7f0ceb06f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:28:58.725336+0000 front 2023-11-27T15:28:58.725173+0000 (oldest deadline 2023-11-27T15:29:23.425277+0000) 2023-11-27T15:29:28.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:27 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:27.824+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:28.061 INFO:journalctl@ceph.osd.4.smithi102.stdout:Nov 27 15:29:27 smithi102 systemd[1]: Stopping Ceph osd.4 for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:29:28.327 INFO:journalctl@ceph.osd.4.smithi102.stdout:Nov 27 15:29:28 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-4[110815]: 2023-11-27T15:29:28.058+0000 7f9581de0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:29:28.327 INFO:journalctl@ceph.osd.4.smithi102.stdout:Nov 27 15:29:28 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-4[110815]: 2023-11-27T15:29:28.058+0000 7f9581de0700 -1 osd.4 99 *** Got signal Terminated *** 2023-11-27T15:29:28.327 INFO:journalctl@ceph.osd.4.smithi102.stdout:Nov 27 15:29:28 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-4[110815]: 2023-11-27T15:29:28.058+0000 7f9581de0700 -1 osd.4 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:29:28.559 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:28 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:28.216+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:28.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:28 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:28.482+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:28.865 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:28 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:28.556+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:01.183006+0000 front 2023-11-27T15:29:01.183057+0000 (oldest deadline 2023-11-27T15:29:25.883242+0000) 2023-11-27T15:29:29.027 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:28 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:28.842+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:29.028 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:28 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:29:28.714+0000 7f0ceb06f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:28:58.725336+0000 front 2023-11-27T15:28:58.725173+0000 (oldest deadline 2023-11-27T15:29:23.425277+0000) 2023-11-27T15:29:29.597 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:29 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:29.189+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:29.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:29 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:29.444+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:29.865 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:29 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:29.593+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:01.183006+0000 front 2023-11-27T15:29:01.183057+0000 (oldest deadline 2023-11-27T15:29:25.883242+0000) 2023-11-27T15:29:30.027 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:29 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:29.812+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:30.028 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:29 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:29:29.697+0000 7f0ceb06f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:28:58.725336+0000 front 2023-11-27T15:28:58.725173+0000 (oldest deadline 2023-11-27T15:29:23.425277+0000) 2023-11-27T15:29:30.558 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:30 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:30.144+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:30.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:30 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:30.437+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:30.865 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:30 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:30.554+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:01.183006+0000 front 2023-11-27T15:29:01.183057+0000 (oldest deadline 2023-11-27T15:29:25.883242+0000) 2023-11-27T15:29:31.027 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:30 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:29:30.653+0000 7f0ceb06f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:28:58.725336+0000 front 2023-11-27T15:28:58.725173+0000 (oldest deadline 2023-11-27T15:29:23.425277+0000) 2023-11-27T15:29:31.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:30 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:30.782+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:31.448 INFO:journalctl@ceph.osd.4.smithi102.stdout:Nov 27 15:29:31 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-4[110815]: 2023-11-27T15:29:31.131+0000 7f957c3d2700 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.834675+0000 front 2023-11-27T15:29:09.834697+0000 (oldest deadline 2023-11-27T15:29:30.934916+0000) 2023-11-27T15:29:31.588 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:31 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:31.168+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:31.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:31 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:31.446+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:31.865 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:31 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:31.584+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:01.183006+0000 front 2023-11-27T15:29:01.183057+0000 (oldest deadline 2023-11-27T15:29:25.883242+0000) 2023-11-27T15:29:32.027 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:31 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:31.773+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:32.027 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:31 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:29:31.693+0000 7f0ceb06f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:28:58.725336+0000 front 2023-11-27T15:28:58.725173+0000 (oldest deadline 2023-11-27T15:29:23.425277+0000) 2023-11-27T15:29:32.028 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:31 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:29:31.693+0000 7f0ceb06f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:07.526458+0000 front 2023-11-27T15:29:07.526447+0000 (oldest deadline 2023-11-27T15:29:31.026534+0000) 2023-11-27T15:29:32.414 INFO:journalctl@ceph.osd.4.smithi102.stdout:Nov 27 15:29:32 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-4[110815]: 2023-11-27T15:29:32.136+0000 7f957c3d2700 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.834675+0000 front 2023-11-27T15:29:09.834697+0000 (oldest deadline 2023-11-27T15:29:30.934916+0000) 2023-11-27T15:29:32.563 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:32 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:32.167+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:32.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:32 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:32.411+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:32.866 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:32 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:32.559+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:01.183006+0000 front 2023-11-27T15:29:01.183057+0000 (oldest deadline 2023-11-27T15:29:25.883242+0000) 2023-11-27T15:29:32.866 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:32 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:32.559+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.384667+0000 front 2023-11-27T15:29:09.384525+0000 (oldest deadline 2023-11-27T15:29:32.284640+0000) 2023-11-27T15:29:33.027 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:32 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:32.764+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:33.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:32 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:32.764+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:29:33.028 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:32 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:29:32.679+0000 7f0ceb06f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:28:58.725336+0000 front 2023-11-27T15:28:58.725173+0000 (oldest deadline 2023-11-27T15:29:23.425277+0000) 2023-11-27T15:29:33.028 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:32 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:29:32.679+0000 7f0ceb06f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:07.526458+0000 front 2023-11-27T15:29:07.526447+0000 (oldest deadline 2023-11-27T15:29:31.026534+0000) 2023-11-27T15:29:33.529 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:33 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:33.122+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:33.529 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:33 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:33.122+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:29:33.692 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:33 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:33.409+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:33.692 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:33 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:33.409+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:29:33.866 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:33 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:33.525+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:01.183006+0000 front 2023-11-27T15:29:01.183057+0000 (oldest deadline 2023-11-27T15:29:25.883242+0000) 2023-11-27T15:29:33.866 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:33 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:33.525+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.384667+0000 front 2023-11-27T15:29:09.384525+0000 (oldest deadline 2023-11-27T15:29:32.284640+0000) 2023-11-27T15:29:34.027 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:33 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:33.729+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:34.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:33 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:33.729+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:29:34.028 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:33 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:29:33.698+0000 7f0ceb06f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:28:58.725336+0000 front 2023-11-27T15:28:58.725173+0000 (oldest deadline 2023-11-27T15:29:23.425277+0000) 2023-11-27T15:29:34.029 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:33 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:29:33.698+0000 7f0ceb06f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:07.526458+0000 front 2023-11-27T15:29:07.526447+0000 (oldest deadline 2023-11-27T15:29:31.026534+0000) 2023-11-27T15:29:34.077 INFO:journalctl@ceph.osd.4.smithi102.stdout:Nov 27 15:29:33 smithi102 bash[131929]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-4 2023-11-27T15:29:34.510 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:34 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:34.142+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:34.511 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:34 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:34.142+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:29:34.771 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:34 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:34.363+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:34.771 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:34 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:34.363+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:29:34.866 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:34 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:34.506+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:01.183006+0000 front 2023-11-27T15:29:01.183057+0000 (oldest deadline 2023-11-27T15:29:25.883242+0000) 2023-11-27T15:29:34.866 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:34 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:34.506+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.384667+0000 front 2023-11-27T15:29:09.384525+0000 (oldest deadline 2023-11-27T15:29:32.284640+0000) 2023-11-27T15:29:35.027 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:34 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:34.681+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:35.027 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:34 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:34.681+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:29:35.028 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:34 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:29:34.696+0000 7f0ceb06f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:28:58.725336+0000 front 2023-11-27T15:28:58.725173+0000 (oldest deadline 2023-11-27T15:29:23.425277+0000) 2023-11-27T15:29:35.028 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:34 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:29:34.696+0000 7f0ceb06f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:07.526458+0000 front 2023-11-27T15:29:07.526447+0000 (oldest deadline 2023-11-27T15:29:31.026534+0000) 2023-11-27T15:29:35.193 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.4.service' 2023-11-27T15:29:35.272 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:29:35.273 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-11-27T15:29:35.273 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-11-27T15:29:35.273 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.5 2023-11-27T15:29:35.515 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:35 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:35.364+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:35.515 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:35 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:35.364+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:29:35.531 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:35 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:35.125+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:35.531 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:35 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:35.125+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:29:35.632 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:35 smithi150 systemd[1]: Stopping Ceph osd.5 for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:29:35.827 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:35 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:35.511+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.265726+0000 front 2023-11-27T15:29:09.265704+0000 (oldest deadline 2023-11-27T15:29:34.565865+0000) 2023-11-27T15:29:35.866 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:35 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:35.527+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:01.183006+0000 front 2023-11-27T15:29:01.183057+0000 (oldest deadline 2023-11-27T15:29:25.883242+0000) 2023-11-27T15:29:35.866 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:35 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:35.527+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.384667+0000 front 2023-11-27T15:29:09.384525+0000 (oldest deadline 2023-11-27T15:29:32.284640+0000) 2023-11-27T15:29:36.027 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:35 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:29:35.628+0000 7f0ceea79700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:29:36.028 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:35 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:29:35.628+0000 7f0ceea79700 -1 osd.5 99 *** Got signal Terminated *** 2023-11-27T15:29:36.028 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:35 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:29:35.628+0000 7f0ceea79700 -1 osd.5 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:29:36.028 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:35 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:29:35.689+0000 7f0ceb06f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:28:58.725336+0000 front 2023-11-27T15:28:58.725173+0000 (oldest deadline 2023-11-27T15:29:23.425277+0000) 2023-11-27T15:29:36.028 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:35 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:29:35.689+0000 7f0ceb06f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:07.526458+0000 front 2023-11-27T15:29:07.526447+0000 (oldest deadline 2023-11-27T15:29:31.026534+0000) 2023-11-27T15:29:36.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:35 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:35.719+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:36.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:35 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:35.719+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:29:36.540 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:36 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:36.167+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:36.540 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:36 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:36.167+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:29:36.827 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:36 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:36.507+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.265726+0000 front 2023-11-27T15:29:09.265704+0000 (oldest deadline 2023-11-27T15:29:34.565865+0000) 2023-11-27T15:29:36.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:36 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:36.397+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:36.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:36 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:36.397+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:29:36.865 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:36 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:36.535+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:01.183006+0000 front 2023-11-27T15:29:01.183057+0000 (oldest deadline 2023-11-27T15:29:25.883242+0000) 2023-11-27T15:29:36.866 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:36 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:36.535+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.384667+0000 front 2023-11-27T15:29:09.384525+0000 (oldest deadline 2023-11-27T15:29:32.284640+0000) 2023-11-27T15:29:37.027 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:36 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:36.751+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.869099+0000 front 2023-11-27T15:29:09.869149+0000 (oldest deadline 2023-11-27T15:29:35.769236+0000) 2023-11-27T15:29:37.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:36 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:36.748+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:37.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:36 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:36.749+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:29:37.028 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:36 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:29:36.723+0000 7f0ceb06f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:28:58.725336+0000 front 2023-11-27T15:28:58.725173+0000 (oldest deadline 2023-11-27T15:29:23.425277+0000) 2023-11-27T15:29:37.028 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:36 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:29:36.723+0000 7f0ceb06f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:07.526458+0000 front 2023-11-27T15:29:07.526447+0000 (oldest deadline 2023-11-27T15:29:31.026534+0000) 2023-11-27T15:29:37.579 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:37 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:37.141+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:37.579 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:37 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:37.141+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:29:37.827 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:37 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:37.468+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.265726+0000 front 2023-11-27T15:29:09.265704+0000 (oldest deadline 2023-11-27T15:29:34.565865+0000) 2023-11-27T15:29:37.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:37 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:37.350+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:37.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:37 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:37.350+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:29:37.865 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:37 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:37.575+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:01.183006+0000 front 2023-11-27T15:29:01.183057+0000 (oldest deadline 2023-11-27T15:29:25.883242+0000) 2023-11-27T15:29:37.866 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:37 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:37.575+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.384667+0000 front 2023-11-27T15:29:09.384525+0000 (oldest deadline 2023-11-27T15:29:32.284640+0000) 2023-11-27T15:29:38.027 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:37 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:37.716+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.869099+0000 front 2023-11-27T15:29:09.869149+0000 (oldest deadline 2023-11-27T15:29:35.769236+0000) 2023-11-27T15:29:38.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:37 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:37.730+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:38.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:37 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:37.730+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:29:38.028 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:37 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:29:37.718+0000 7f0ceb06f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:28:58.725336+0000 front 2023-11-27T15:28:58.725173+0000 (oldest deadline 2023-11-27T15:29:23.425277+0000) 2023-11-27T15:29:38.028 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:37 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:29:37.718+0000 7f0ceb06f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:07.526458+0000 front 2023-11-27T15:29:07.526447+0000 (oldest deadline 2023-11-27T15:29:31.026534+0000) 2023-11-27T15:29:38.365 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:38 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:38.102+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:38.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:38 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:38.102+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:29:38.827 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:38 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:38.460+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.265726+0000 front 2023-11-27T15:29:09.265704+0000 (oldest deadline 2023-11-27T15:29:34.565865+0000) 2023-11-27T15:29:38.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:38 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:38.346+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:38.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:38 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:38.346+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:29:38.866 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:38 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:38.537+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:01.183006+0000 front 2023-11-27T15:29:01.183057+0000 (oldest deadline 2023-11-27T15:29:25.883242+0000) 2023-11-27T15:29:38.866 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:38 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:38.537+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.384667+0000 front 2023-11-27T15:29:09.384525+0000 (oldest deadline 2023-11-27T15:29:32.284640+0000) 2023-11-27T15:29:39.029 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:38 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:38.740+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.869099+0000 front 2023-11-27T15:29:09.869149+0000 (oldest deadline 2023-11-27T15:29:35.769236+0000) 2023-11-27T15:29:39.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:38 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:38.697+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:39.030 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:38 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:38.697+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:29:39.030 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:38 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:29:38.695+0000 7f0ceb06f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:28:58.725336+0000 front 2023-11-27T15:28:58.725173+0000 (oldest deadline 2023-11-27T15:29:23.425277+0000) 2023-11-27T15:29:39.030 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:38 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:29:38.695+0000 7f0ceb06f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:07.526458+0000 front 2023-11-27T15:29:07.526447+0000 (oldest deadline 2023-11-27T15:29:31.026534+0000) 2023-11-27T15:29:39.539 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:39 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:39.117+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:39.539 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:39 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:39.117+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:29:39.577 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:39 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:39.507+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.265726+0000 front 2023-11-27T15:29:09.265704+0000 (oldest deadline 2023-11-27T15:29:34.565865+0000) 2023-11-27T15:29:39.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:39 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:39.297+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:39.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:39 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:39.297+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:29:39.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:39 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:39.297+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:29:39.865 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:39 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:39.535+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:01.183006+0000 front 2023-11-27T15:29:01.183057+0000 (oldest deadline 2023-11-27T15:29:25.883242+0000) 2023-11-27T15:29:39.866 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:39 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:39.535+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.384667+0000 front 2023-11-27T15:29:09.384525+0000 (oldest deadline 2023-11-27T15:29:32.284640+0000) 2023-11-27T15:29:40.027 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:39 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:39.669+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:40.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:39 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:39.669+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:29:40.028 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:39 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:29:39.739+0000 7f0ceb06f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:28:58.725336+0000 front 2023-11-27T15:28:58.725173+0000 (oldest deadline 2023-11-27T15:29:23.425277+0000) 2023-11-27T15:29:40.028 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:39 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5[111833]: 2023-11-27T15:29:39.739+0000 7f0ceb06f700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:07.526458+0000 front 2023-11-27T15:29:07.526447+0000 (oldest deadline 2023-11-27T15:29:31.026534+0000) 2023-11-27T15:29:40.028 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:39 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:39.779+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.869099+0000 front 2023-11-27T15:29:09.869149+0000 (oldest deadline 2023-11-27T15:29:35.769236+0000) 2023-11-27T15:29:40.499 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:40 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:40.160+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:40.499 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:40 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:40.160+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:29:40.546 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:40 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:40.250+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:40.546 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:40 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:40.250+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:29:40.547 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:40 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:40.250+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:29:40.827 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:40 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:40.543+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.265726+0000 front 2023-11-27T15:29:09.265704+0000 (oldest deadline 2023-11-27T15:29:34.565865+0000) 2023-11-27T15:29:40.827 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:40 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:40.543+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:14.566446+0000 front 2023-11-27T15:29:14.566367+0000 (oldest deadline 2023-11-27T15:29:40.466383+0000) 2023-11-27T15:29:40.865 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:40 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:40.495+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:01.183006+0000 front 2023-11-27T15:29:01.183057+0000 (oldest deadline 2023-11-27T15:29:25.883242+0000) 2023-11-27T15:29:40.866 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:40 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:40.495+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.384667+0000 front 2023-11-27T15:29:09.384525+0000 (oldest deadline 2023-11-27T15:29:32.284640+0000) 2023-11-27T15:29:41.027 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:40 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:40.758+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.869099+0000 front 2023-11-27T15:29:09.869149+0000 (oldest deadline 2023-11-27T15:29:35.769236+0000) 2023-11-27T15:29:41.027 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:40 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:40.758+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:15.769572+0000 front 2023-11-27T15:29:15.769595+0000 (oldest deadline 2023-11-27T15:29:39.869711+0000) 2023-11-27T15:29:41.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:40 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:40.622+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:41.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:40 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:40.622+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:29:41.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:40 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:40.622+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:29:41.511 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:41 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:41.176+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:41.511 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:41 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:41.176+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:29:41.527 INFO:journalctl@ceph.osd.5.smithi150.stdout:Nov 27 15:29:41 smithi150 bash[133222]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-5 2023-11-27T15:29:41.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:41 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:41.223+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:41.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:41 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:41.224+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:29:41.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:41 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:41.224+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:29:41.866 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:41 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:41.507+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:01.183006+0000 front 2023-11-27T15:29:01.183057+0000 (oldest deadline 2023-11-27T15:29:25.883242+0000) 2023-11-27T15:29:41.866 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:41 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:41.507+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.384667+0000 front 2023-11-27T15:29:09.384525+0000 (oldest deadline 2023-11-27T15:29:32.284640+0000) 2023-11-27T15:29:42.028 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:41 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:41.755+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.869099+0000 front 2023-11-27T15:29:09.869149+0000 (oldest deadline 2023-11-27T15:29:35.769236+0000) 2023-11-27T15:29:42.028 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:41 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:41.755+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:15.769572+0000 front 2023-11-27T15:29:15.769595+0000 (oldest deadline 2023-11-27T15:29:39.869711+0000) 2023-11-27T15:29:42.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:41 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:41.625+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:42.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:41 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:41.625+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:29:42.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:41 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:41.625+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:29:42.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:41 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:41.593+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.265726+0000 front 2023-11-27T15:29:09.265704+0000 (oldest deadline 2023-11-27T15:29:34.565865+0000) 2023-11-27T15:29:42.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:41 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:41.593+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:14.566446+0000 front 2023-11-27T15:29:14.566367+0000 (oldest deadline 2023-11-27T15:29:40.466383+0000) 2023-11-27T15:29:42.531 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:42 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:42.156+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:42.532 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:42 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:42.156+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:29:42.532 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:42 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:42.156+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.136916+0000 front 2023-11-27T15:29:17.137063+0000 (oldest deadline 2023-11-27T15:29:41.837092+0000) 2023-11-27T15:29:42.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:42 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:42.236+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:42.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:42 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:42.236+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:29:42.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:42 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:42.236+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:29:42.742 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:42 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:42.626+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:42.743 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:42 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:42.626+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:29:42.743 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:42 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:42.626+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:29:42.749 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.5.service' 2023-11-27T15:29:42.788 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:29:42.789 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-11-27T15:29:42.789 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-11-27T15:29:42.789 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.6 2023-11-27T15:29:42.820 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:42 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:42.528+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:01.183006+0000 front 2023-11-27T15:29:01.183057+0000 (oldest deadline 2023-11-27T15:29:25.883242+0000) 2023-11-27T15:29:42.820 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:42 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:42.528+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.384667+0000 front 2023-11-27T15:29:09.384525+0000 (oldest deadline 2023-11-27T15:29:32.284640+0000) 2023-11-27T15:29:42.820 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:42 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:42.528+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:15.786220+0000 front 2023-11-27T15:29:15.786272+0000 (oldest deadline 2023-11-27T15:29:41.685968+0000) 2023-11-27T15:29:43.027 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:42 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:42.786+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.869099+0000 front 2023-11-27T15:29:09.869149+0000 (oldest deadline 2023-11-27T15:29:35.769236+0000) 2023-11-27T15:29:43.027 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:42 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:42.786+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:15.769572+0000 front 2023-11-27T15:29:15.769595+0000 (oldest deadline 2023-11-27T15:29:39.869711+0000) 2023-11-27T15:29:43.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:42 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:42.633+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.265726+0000 front 2023-11-27T15:29:09.265704+0000 (oldest deadline 2023-11-27T15:29:34.565865+0000) 2023-11-27T15:29:43.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:42 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:42.633+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:14.566446+0000 front 2023-11-27T15:29:14.566367+0000 (oldest deadline 2023-11-27T15:29:40.466383+0000) 2023-11-27T15:29:43.115 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:42 smithi052 systemd[1]: Stopping Ceph osd.6 for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:29:43.538 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:43 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:43.162+0000 7fa2c0470700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:29:43.538 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:43 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:43.162+0000 7fa2c0470700 -1 osd.6 99 *** Got signal Terminated *** 2023-11-27T15:29:43.538 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:43 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:43.162+0000 7fa2c0470700 -1 osd.6 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:29:43.539 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:43 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:43.153+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:43.539 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:43 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:43.153+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:29:43.539 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:43 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:43.153+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.136916+0000 front 2023-11-27T15:29:17.137063+0000 (oldest deadline 2023-11-27T15:29:41.837092+0000) 2023-11-27T15:29:43.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:43 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:43.216+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:43.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:43 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:43.216+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:29:43.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:43 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:43.216+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:29:43.865 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:43 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:43.534+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:01.183006+0000 front 2023-11-27T15:29:01.183057+0000 (oldest deadline 2023-11-27T15:29:25.883242+0000) 2023-11-27T15:29:43.866 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:43 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:43.534+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.384667+0000 front 2023-11-27T15:29:09.384525+0000 (oldest deadline 2023-11-27T15:29:32.284640+0000) 2023-11-27T15:29:43.866 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:43 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:43.534+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:15.786220+0000 front 2023-11-27T15:29:15.786272+0000 (oldest deadline 2023-11-27T15:29:41.685968+0000) 2023-11-27T15:29:44.027 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:43 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:43.820+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.869099+0000 front 2023-11-27T15:29:09.869149+0000 (oldest deadline 2023-11-27T15:29:35.769236+0000) 2023-11-27T15:29:44.028 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:43 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:43.820+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:15.769572+0000 front 2023-11-27T15:29:15.769595+0000 (oldest deadline 2023-11-27T15:29:39.869711+0000) 2023-11-27T15:29:44.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:43 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:43.608+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:44.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:43 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:43.608+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:29:44.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:43 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:43.608+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:29:44.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:43 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:43.596+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.265726+0000 front 2023-11-27T15:29:09.265704+0000 (oldest deadline 2023-11-27T15:29:34.565865+0000) 2023-11-27T15:29:44.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:43 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:43.596+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:14.566446+0000 front 2023-11-27T15:29:14.566367+0000 (oldest deadline 2023-11-27T15:29:40.466383+0000) 2023-11-27T15:29:44.502 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:44 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:44.191+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:44.502 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:44 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:44.191+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:29:44.502 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:44 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:44.191+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.136916+0000 front 2023-11-27T15:29:17.137063+0000 (oldest deadline 2023-11-27T15:29:41.837092+0000) 2023-11-27T15:29:44.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:44 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:44.266+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:44.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:44 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:44.266+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:29:44.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:44 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:44.266+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:29:44.859 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:44 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:44.600+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:44.859 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:44 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:44.600+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:29:44.859 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:44 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:44.600+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:29:44.866 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:44 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:44.498+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:01.183006+0000 front 2023-11-27T15:29:01.183057+0000 (oldest deadline 2023-11-27T15:29:25.883242+0000) 2023-11-27T15:29:44.866 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:44 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:44.498+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.384667+0000 front 2023-11-27T15:29:09.384525+0000 (oldest deadline 2023-11-27T15:29:32.284640+0000) 2023-11-27T15:29:44.866 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:44 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:44.498+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:15.786220+0000 front 2023-11-27T15:29:15.786272+0000 (oldest deadline 2023-11-27T15:29:41.685968+0000) 2023-11-27T15:29:45.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:44 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:44.633+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.265726+0000 front 2023-11-27T15:29:09.265704+0000 (oldest deadline 2023-11-27T15:29:34.565865+0000) 2023-11-27T15:29:45.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:44 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:44.633+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:14.566446+0000 front 2023-11-27T15:29:14.566367+0000 (oldest deadline 2023-11-27T15:29:40.466383+0000) 2023-11-27T15:29:45.277 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:44 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:44.855+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.869099+0000 front 2023-11-27T15:29:09.869149+0000 (oldest deadline 2023-11-27T15:29:35.769236+0000) 2023-11-27T15:29:45.277 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:44 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:44.855+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:15.769572+0000 front 2023-11-27T15:29:15.769595+0000 (oldest deadline 2023-11-27T15:29:39.869711+0000) 2023-11-27T15:29:45.541 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:45 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:45.173+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:45.541 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:45 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:45.173+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:29:45.541 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:45 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:45.173+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.136916+0000 front 2023-11-27T15:29:17.137063+0000 (oldest deadline 2023-11-27T15:29:41.837092+0000) 2023-11-27T15:29:45.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:45 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:45.218+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:45.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:45 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:45.218+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:29:45.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:45 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:45.218+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:29:45.866 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:45 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:45.536+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:01.183006+0000 front 2023-11-27T15:29:01.183057+0000 (oldest deadline 2023-11-27T15:29:25.883242+0000) 2023-11-27T15:29:45.866 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:45 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:45.536+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.384667+0000 front 2023-11-27T15:29:09.384525+0000 (oldest deadline 2023-11-27T15:29:32.284640+0000) 2023-11-27T15:29:45.866 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:45 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:45.536+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:15.786220+0000 front 2023-11-27T15:29:15.786272+0000 (oldest deadline 2023-11-27T15:29:41.685968+0000) 2023-11-27T15:29:45.904 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:45 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:45.642+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:45.905 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:45 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:45.642+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:29:45.905 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:45 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:45.642+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:29:46.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:45 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:45.679+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.265726+0000 front 2023-11-27T15:29:09.265704+0000 (oldest deadline 2023-11-27T15:29:34.565865+0000) 2023-11-27T15:29:46.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:45 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:45.679+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:14.566446+0000 front 2023-11-27T15:29:14.566367+0000 (oldest deadline 2023-11-27T15:29:40.466383+0000) 2023-11-27T15:29:46.277 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:45 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:45.902+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.869099+0000 front 2023-11-27T15:29:09.869149+0000 (oldest deadline 2023-11-27T15:29:35.769236+0000) 2023-11-27T15:29:46.278 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:45 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:45.902+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:15.769572+0000 front 2023-11-27T15:29:15.769595+0000 (oldest deadline 2023-11-27T15:29:39.869711+0000) 2023-11-27T15:29:46.278 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:45 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:45.902+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:19.870581+0000 front 2023-11-27T15:29:19.870486+0000 (oldest deadline 2023-11-27T15:29:45.770238+0000) 2023-11-27T15:29:46.526 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:46 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:46.162+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:46.527 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:46 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:46.162+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:29:46.527 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:46 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:46.162+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.136916+0000 front 2023-11-27T15:29:17.137063+0000 (oldest deadline 2023-11-27T15:29:41.837092+0000) 2023-11-27T15:29:46.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:46 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:46.242+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:46.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:46 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:46.242+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:29:46.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:46 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:46.242+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:29:46.578 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:46 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:46.242+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:29:46.866 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:46 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:46.523+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:01.183006+0000 front 2023-11-27T15:29:01.183057+0000 (oldest deadline 2023-11-27T15:29:25.883242+0000) 2023-11-27T15:29:46.866 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:46 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:46.523+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.384667+0000 front 2023-11-27T15:29:09.384525+0000 (oldest deadline 2023-11-27T15:29:32.284640+0000) 2023-11-27T15:29:46.866 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:46 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:46.523+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:15.786220+0000 front 2023-11-27T15:29:15.786272+0000 (oldest deadline 2023-11-27T15:29:41.685968+0000) 2023-11-27T15:29:47.027 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:46 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:46.673+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:47.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:46 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:46.673+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:29:47.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:46 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:46.673+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:29:47.028 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:46 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:46.917+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.869099+0000 front 2023-11-27T15:29:09.869149+0000 (oldest deadline 2023-11-27T15:29:35.769236+0000) 2023-11-27T15:29:47.028 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:46 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:46.917+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:15.769572+0000 front 2023-11-27T15:29:15.769595+0000 (oldest deadline 2023-11-27T15:29:39.869711+0000) 2023-11-27T15:29:47.028 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:46 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:46.917+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:19.870581+0000 front 2023-11-27T15:29:19.870486+0000 (oldest deadline 2023-11-27T15:29:45.770238+0000) 2023-11-27T15:29:47.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:46 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:46.708+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.265726+0000 front 2023-11-27T15:29:09.265704+0000 (oldest deadline 2023-11-27T15:29:34.565865+0000) 2023-11-27T15:29:47.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:46 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:46.708+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:14.566446+0000 front 2023-11-27T15:29:14.566367+0000 (oldest deadline 2023-11-27T15:29:40.466383+0000) 2023-11-27T15:29:47.574 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:47 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:47.171+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:47.575 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:47 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:47.171+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:29:47.575 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:47 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:47.171+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.136916+0000 front 2023-11-27T15:29:17.137063+0000 (oldest deadline 2023-11-27T15:29:41.837092+0000) 2023-11-27T15:29:47.575 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:47 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:47.171+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.138543+0000 front 2023-11-27T15:29:24.138614+0000 (oldest deadline 2023-11-27T15:29:46.438604+0000) 2023-11-27T15:29:47.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:47 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:47.289+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:47.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:47 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:47.289+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:29:47.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:47 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:47.289+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:29:47.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:47 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:47.289+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:29:47.866 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:47 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:47.571+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:01.183006+0000 front 2023-11-27T15:29:01.183057+0000 (oldest deadline 2023-11-27T15:29:25.883242+0000) 2023-11-27T15:29:47.866 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:47 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:47.571+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.384667+0000 front 2023-11-27T15:29:09.384525+0000 (oldest deadline 2023-11-27T15:29:32.284640+0000) 2023-11-27T15:29:47.866 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:47 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6[122835]: 2023-11-27T15:29:47.571+0000 7fa2bca66700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:15.786220+0000 front 2023-11-27T15:29:15.786272+0000 (oldest deadline 2023-11-27T15:29:41.685968+0000) 2023-11-27T15:29:48.027 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:47 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:47.896+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.869099+0000 front 2023-11-27T15:29:09.869149+0000 (oldest deadline 2023-11-27T15:29:35.769236+0000) 2023-11-27T15:29:48.028 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:47 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:47.896+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:15.769572+0000 front 2023-11-27T15:29:15.769595+0000 (oldest deadline 2023-11-27T15:29:39.869711+0000) 2023-11-27T15:29:48.028 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:47 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:47.896+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:19.870581+0000 front 2023-11-27T15:29:19.870486+0000 (oldest deadline 2023-11-27T15:29:45.770238+0000) 2023-11-27T15:29:48.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:47 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:47.714+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:48.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:47 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:47.714+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:29:48.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:47 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:47.714+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:29:48.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:47 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:47.665+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.265726+0000 front 2023-11-27T15:29:09.265704+0000 (oldest deadline 2023-11-27T15:29:34.565865+0000) 2023-11-27T15:29:48.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:47 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:47.665+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:14.566446+0000 front 2023-11-27T15:29:14.566367+0000 (oldest deadline 2023-11-27T15:29:40.466383+0000) 2023-11-27T15:29:48.572 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:48 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:48.169+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:48.573 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:48 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:48.169+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:29:48.573 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:48 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:48.169+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.136916+0000 front 2023-11-27T15:29:17.137063+0000 (oldest deadline 2023-11-27T15:29:41.837092+0000) 2023-11-27T15:29:48.573 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:48 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:48.169+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.138543+0000 front 2023-11-27T15:29:24.138614+0000 (oldest deadline 2023-11-27T15:29:46.438604+0000) 2023-11-27T15:29:48.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:48 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:48.321+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:48.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:48 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:48.321+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:29:48.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:48 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:48.321+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:29:48.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:48 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:48.321+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:29:49.027 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:48 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:48.929+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.869099+0000 front 2023-11-27T15:29:09.869149+0000 (oldest deadline 2023-11-27T15:29:35.769236+0000) 2023-11-27T15:29:49.028 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:48 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:48.929+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:15.769572+0000 front 2023-11-27T15:29:15.769595+0000 (oldest deadline 2023-11-27T15:29:39.869711+0000) 2023-11-27T15:29:49.028 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:48 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:48.929+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:19.870581+0000 front 2023-11-27T15:29:19.870486+0000 (oldest deadline 2023-11-27T15:29:45.770238+0000) 2023-11-27T15:29:49.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:48 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:48.683+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:49.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:48 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:48.683+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:29:49.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:48 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:48.683+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:29:49.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:48 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:48.708+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.265726+0000 front 2023-11-27T15:29:09.265704+0000 (oldest deadline 2023-11-27T15:29:34.565865+0000) 2023-11-27T15:29:49.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:48 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:48.708+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:14.566446+0000 front 2023-11-27T15:29:14.566367+0000 (oldest deadline 2023-11-27T15:29:40.466383+0000) 2023-11-27T15:29:49.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:48 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:48.708+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.767485+0000 front 2023-11-27T15:29:22.767421+0000 (oldest deadline 2023-11-27T15:29:48.667529+0000) 2023-11-27T15:29:49.132 INFO:journalctl@ceph.osd.6.smithi052.stdout:Nov 27 15:29:48 smithi052 bash[263570]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-6 2023-11-27T15:29:49.529 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:49 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:49.127+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:49.529 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:49 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:49.127+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:29:49.529 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:49 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:49.127+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.136916+0000 front 2023-11-27T15:29:17.137063+0000 (oldest deadline 2023-11-27T15:29:41.837092+0000) 2023-11-27T15:29:49.529 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:49 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:49.127+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.138543+0000 front 2023-11-27T15:29:24.138614+0000 (oldest deadline 2023-11-27T15:29:46.438604+0000) 2023-11-27T15:29:49.696 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:49 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:49.336+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:49.697 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:49 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:49.336+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:29:49.697 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:49 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:49.336+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:29:49.697 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:49 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:49.336+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:29:50.028 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:49 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:49.891+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.869099+0000 front 2023-11-27T15:29:09.869149+0000 (oldest deadline 2023-11-27T15:29:35.769236+0000) 2023-11-27T15:29:50.028 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:49 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:49.891+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:15.769572+0000 front 2023-11-27T15:29:15.769595+0000 (oldest deadline 2023-11-27T15:29:39.869711+0000) 2023-11-27T15:29:50.028 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:49 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:49.891+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:19.870581+0000 front 2023-11-27T15:29:19.870486+0000 (oldest deadline 2023-11-27T15:29:45.770238+0000) 2023-11-27T15:29:50.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:49 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:49.719+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:50.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:49 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:49.719+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:29:50.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:49 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:49.719+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:29:50.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:49 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:49.719+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:29:50.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:49 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:49.693+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.265726+0000 front 2023-11-27T15:29:09.265704+0000 (oldest deadline 2023-11-27T15:29:34.565865+0000) 2023-11-27T15:29:50.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:49 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:49.693+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:14.566446+0000 front 2023-11-27T15:29:14.566367+0000 (oldest deadline 2023-11-27T15:29:40.466383+0000) 2023-11-27T15:29:50.078 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:49 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:49.693+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.767485+0000 front 2023-11-27T15:29:22.767421+0000 (oldest deadline 2023-11-27T15:29:48.667529+0000) 2023-11-27T15:29:50.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:50 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:50.101+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:50.367 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:50 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:50.101+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:29:50.367 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:50 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:50.101+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.136916+0000 front 2023-11-27T15:29:17.137063+0000 (oldest deadline 2023-11-27T15:29:41.837092+0000) 2023-11-27T15:29:50.367 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:50 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:50.101+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.138543+0000 front 2023-11-27T15:29:24.138614+0000 (oldest deadline 2023-11-27T15:29:46.438604+0000) 2023-11-27T15:29:50.518 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.6.service' 2023-11-27T15:29:50.602 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:29:50.603 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-11-27T15:29:50.603 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-11-27T15:29:50.603 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.7 2023-11-27T15:29:50.634 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:50 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:50.366+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:50.634 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:50 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:50.366+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:29:50.634 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:50 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:50.366+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:29:50.635 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:50 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:50.366+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:29:50.966 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:50 smithi102 systemd[1]: Stopping Ceph osd.7 for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:29:50.967 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:50 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:50.645+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.265726+0000 front 2023-11-27T15:29:09.265704+0000 (oldest deadline 2023-11-27T15:29:34.565865+0000) 2023-11-27T15:29:50.967 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:50 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:50.645+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:14.566446+0000 front 2023-11-27T15:29:14.566367+0000 (oldest deadline 2023-11-27T15:29:40.466383+0000) 2023-11-27T15:29:50.967 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:50 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:50.645+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.767485+0000 front 2023-11-27T15:29:22.767421+0000 (oldest deadline 2023-11-27T15:29:48.667529+0000) 2023-11-27T15:29:50.967 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:50 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:50.962+0000 7f773632d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:29:50.968 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:50 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:50.962+0000 7f773632d700 -1 osd.7 99 *** Got signal Terminated *** 2023-11-27T15:29:51.027 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:50 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:50.879+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.869099+0000 front 2023-11-27T15:29:09.869149+0000 (oldest deadline 2023-11-27T15:29:35.769236+0000) 2023-11-27T15:29:51.028 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:50 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:50.879+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:15.769572+0000 front 2023-11-27T15:29:15.769595+0000 (oldest deadline 2023-11-27T15:29:39.869711+0000) 2023-11-27T15:29:51.028 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:50 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:50.879+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:19.870581+0000 front 2023-11-27T15:29:19.870486+0000 (oldest deadline 2023-11-27T15:29:45.770238+0000) 2023-11-27T15:29:51.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:50 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:50.755+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:51.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:50 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:50.755+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:29:51.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:50 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:50.755+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:29:51.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:50 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:50.755+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:29:51.327 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:50 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:50.962+0000 7f773632d700 -1 osd.7 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:29:51.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:51 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:51.093+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:51.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:51 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:51.093+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:29:51.367 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:51 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:51.093+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.136916+0000 front 2023-11-27T15:29:17.137063+0000 (oldest deadline 2023-11-27T15:29:41.837092+0000) 2023-11-27T15:29:51.367 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:51 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:51.093+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.138543+0000 front 2023-11-27T15:29:24.138614+0000 (oldest deadline 2023-11-27T15:29:46.438604+0000) 2023-11-27T15:29:51.648 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:51 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:51.327+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:51.649 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:51 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:51.327+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:29:51.649 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:51 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:51.327+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:29:51.649 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:51 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:51.327+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:29:52.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:51 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:51.645+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.265726+0000 front 2023-11-27T15:29:09.265704+0000 (oldest deadline 2023-11-27T15:29:34.565865+0000) 2023-11-27T15:29:52.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:51 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:51.645+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:14.566446+0000 front 2023-11-27T15:29:14.566367+0000 (oldest deadline 2023-11-27T15:29:40.466383+0000) 2023-11-27T15:29:52.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:51 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:51.645+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.767485+0000 front 2023-11-27T15:29:22.767421+0000 (oldest deadline 2023-11-27T15:29:48.667529+0000) 2023-11-27T15:29:52.277 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:51 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:51.850+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.869099+0000 front 2023-11-27T15:29:09.869149+0000 (oldest deadline 2023-11-27T15:29:35.769236+0000) 2023-11-27T15:29:52.278 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:51 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:51.850+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:15.769572+0000 front 2023-11-27T15:29:15.769595+0000 (oldest deadline 2023-11-27T15:29:39.869711+0000) 2023-11-27T15:29:52.278 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:51 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:51.850+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:19.870581+0000 front 2023-11-27T15:29:19.870486+0000 (oldest deadline 2023-11-27T15:29:45.770238+0000) 2023-11-27T15:29:52.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:51 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:51.774+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:52.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:51 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:51.774+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:29:52.279 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:51 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:51.774+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:29:52.279 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:51 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:51.774+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:29:52.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:52 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:52.098+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:52.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:52 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:52.098+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:29:52.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:52 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:52.098+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.136916+0000 front 2023-11-27T15:29:17.137063+0000 (oldest deadline 2023-11-27T15:29:41.837092+0000) 2023-11-27T15:29:52.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:52 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:52.098+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.138543+0000 front 2023-11-27T15:29:24.138614+0000 (oldest deadline 2023-11-27T15:29:46.438604+0000) 2023-11-27T15:29:52.682 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:52 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:52.352+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:52.682 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:52 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:52.352+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:29:52.682 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:52 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:52.352+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:29:52.682 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:52 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:52.352+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:29:53.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:52 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:52.678+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.265726+0000 front 2023-11-27T15:29:09.265704+0000 (oldest deadline 2023-11-27T15:29:34.565865+0000) 2023-11-27T15:29:53.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:52 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:52.678+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:14.566446+0000 front 2023-11-27T15:29:14.566367+0000 (oldest deadline 2023-11-27T15:29:40.466383+0000) 2023-11-27T15:29:53.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:52 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:52.678+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.767485+0000 front 2023-11-27T15:29:22.767421+0000 (oldest deadline 2023-11-27T15:29:48.667529+0000) 2023-11-27T15:29:53.278 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:52 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:52.817+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.869099+0000 front 2023-11-27T15:29:09.869149+0000 (oldest deadline 2023-11-27T15:29:35.769236+0000) 2023-11-27T15:29:53.278 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:52 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:52.817+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:15.769572+0000 front 2023-11-27T15:29:15.769595+0000 (oldest deadline 2023-11-27T15:29:39.869711+0000) 2023-11-27T15:29:53.278 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:52 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:52.817+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:19.870581+0000 front 2023-11-27T15:29:19.870486+0000 (oldest deadline 2023-11-27T15:29:45.770238+0000) 2023-11-27T15:29:53.279 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:52 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:52.817+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:53.279 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:52 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:52.817+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:29:53.279 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:52 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:52.817+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:29:53.279 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:52 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:52.817+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:29:53.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:53 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:53.092+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:53.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:53 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:53.092+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:29:53.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:53 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:53.092+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.136916+0000 front 2023-11-27T15:29:17.137063+0000 (oldest deadline 2023-11-27T15:29:41.837092+0000) 2023-11-27T15:29:53.367 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:53 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:53.092+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.138543+0000 front 2023-11-27T15:29:24.138614+0000 (oldest deadline 2023-11-27T15:29:46.438604+0000) 2023-11-27T15:29:53.651 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:53 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:53.347+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:53.651 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:53 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:53.347+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:29:53.651 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:53 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:53.347+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:29:53.652 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:53 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:53.347+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:29:54.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:53 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:53.646+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.265726+0000 front 2023-11-27T15:29:09.265704+0000 (oldest deadline 2023-11-27T15:29:34.565865+0000) 2023-11-27T15:29:54.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:53 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:53.646+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:14.566446+0000 front 2023-11-27T15:29:14.566367+0000 (oldest deadline 2023-11-27T15:29:40.466383+0000) 2023-11-27T15:29:54.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:53 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:53.646+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.767485+0000 front 2023-11-27T15:29:22.767421+0000 (oldest deadline 2023-11-27T15:29:48.667529+0000) 2023-11-27T15:29:54.278 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:53 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:53.844+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.869099+0000 front 2023-11-27T15:29:09.869149+0000 (oldest deadline 2023-11-27T15:29:35.769236+0000) 2023-11-27T15:29:54.278 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:53 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:53.844+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:15.769572+0000 front 2023-11-27T15:29:15.769595+0000 (oldest deadline 2023-11-27T15:29:39.869711+0000) 2023-11-27T15:29:54.278 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:53 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:53.844+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:19.870581+0000 front 2023-11-27T15:29:19.870486+0000 (oldest deadline 2023-11-27T15:29:45.770238+0000) 2023-11-27T15:29:54.279 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:53 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:53.829+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:54.279 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:53 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:53.829+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:29:54.279 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:53 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:53.829+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:29:54.279 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:53 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:53.829+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:29:54.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:54 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:54.055+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:54.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:54 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:54.055+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:29:54.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:54 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:54.055+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.136916+0000 front 2023-11-27T15:29:17.137063+0000 (oldest deadline 2023-11-27T15:29:41.837092+0000) 2023-11-27T15:29:54.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:54 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:54.055+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.138543+0000 front 2023-11-27T15:29:24.138614+0000 (oldest deadline 2023-11-27T15:29:46.438604+0000) 2023-11-27T15:29:54.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:54 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:54.302+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:54.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:54 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:54.302+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:29:54.578 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:54 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:54.302+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:29:54.578 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:54 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:54.302+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:29:55.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:54 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:54.654+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.265726+0000 front 2023-11-27T15:29:09.265704+0000 (oldest deadline 2023-11-27T15:29:34.565865+0000) 2023-11-27T15:29:55.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:54 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:54.654+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:14.566446+0000 front 2023-11-27T15:29:14.566367+0000 (oldest deadline 2023-11-27T15:29:40.466383+0000) 2023-11-27T15:29:55.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:54 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:54.654+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.767485+0000 front 2023-11-27T15:29:22.767421+0000 (oldest deadline 2023-11-27T15:29:48.667529+0000) 2023-11-27T15:29:55.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:54 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:54.654+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.368626+0000 front 2023-11-27T15:29:30.368641+0000 (oldest deadline 2023-11-27T15:29:54.468780+0000) 2023-11-27T15:29:55.277 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:54 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:54.786+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:55.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:54 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:54.786+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:29:55.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:54 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:54.786+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:29:55.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:54 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:54.786+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:29:55.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:54 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:54.786+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:29:55.278 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:54 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:54.867+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.869099+0000 front 2023-11-27T15:29:09.869149+0000 (oldest deadline 2023-11-27T15:29:35.769236+0000) 2023-11-27T15:29:55.278 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:54 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:54.867+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:15.769572+0000 front 2023-11-27T15:29:15.769595+0000 (oldest deadline 2023-11-27T15:29:39.869711+0000) 2023-11-27T15:29:55.279 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:54 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:54.867+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:19.870581+0000 front 2023-11-27T15:29:19.870486+0000 (oldest deadline 2023-11-27T15:29:45.770238+0000) 2023-11-27T15:29:55.279 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:54 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:54.867+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:28.671510+0000 front 2023-11-27T15:29:28.671404+0000 (oldest deadline 2023-11-27T15:29:54.571359+0000) 2023-11-27T15:29:55.365 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:55 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:55.084+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:55.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:55 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:55.084+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:29:55.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:55 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:55.084+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.136916+0000 front 2023-11-27T15:29:17.137063+0000 (oldest deadline 2023-11-27T15:29:41.837092+0000) 2023-11-27T15:29:55.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:55 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:55.084+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.138543+0000 front 2023-11-27T15:29:24.138614+0000 (oldest deadline 2023-11-27T15:29:46.438604+0000) 2023-11-27T15:29:55.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:55 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:55.084+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.539608+0000 front 2023-11-27T15:29:30.539640+0000 (oldest deadline 2023-11-27T15:29:54.639653+0000) 2023-11-27T15:29:55.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:55 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:55.319+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:55.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:55 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:55.319+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:29:55.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:55 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:55.320+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:29:55.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:55 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:55.320+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:29:56.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:55 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:55.666+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.265726+0000 front 2023-11-27T15:29:09.265704+0000 (oldest deadline 2023-11-27T15:29:34.565865+0000) 2023-11-27T15:29:56.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:55 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:55.666+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:14.566446+0000 front 2023-11-27T15:29:14.566367+0000 (oldest deadline 2023-11-27T15:29:40.466383+0000) 2023-11-27T15:29:56.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:55 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:55.666+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.767485+0000 front 2023-11-27T15:29:22.767421+0000 (oldest deadline 2023-11-27T15:29:48.667529+0000) 2023-11-27T15:29:56.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:55 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7[116260]: 2023-11-27T15:29:55.666+0000 7f773011e700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.368626+0000 front 2023-11-27T15:29:30.368641+0000 (oldest deadline 2023-11-27T15:29:54.468780+0000) 2023-11-27T15:29:56.277 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:55 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:55.860+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.869099+0000 front 2023-11-27T15:29:09.869149+0000 (oldest deadline 2023-11-27T15:29:35.769236+0000) 2023-11-27T15:29:56.277 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:55 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:55.860+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:15.769572+0000 front 2023-11-27T15:29:15.769595+0000 (oldest deadline 2023-11-27T15:29:39.869711+0000) 2023-11-27T15:29:56.278 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:55 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:55.860+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:19.870581+0000 front 2023-11-27T15:29:19.870486+0000 (oldest deadline 2023-11-27T15:29:45.770238+0000) 2023-11-27T15:29:56.278 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:55 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:55.860+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:28.671510+0000 front 2023-11-27T15:29:28.671404+0000 (oldest deadline 2023-11-27T15:29:54.571359+0000) 2023-11-27T15:29:56.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:55 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:55.821+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:56.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:55 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:55.821+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:29:56.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:55 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:55.821+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:29:56.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:55 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:55.821+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:29:56.279 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:55 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:55.821+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:29:56.365 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:56 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:56.063+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:56.365 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:56 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:56.063+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:29:56.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:56 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:56.063+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.136916+0000 front 2023-11-27T15:29:17.137063+0000 (oldest deadline 2023-11-27T15:29:41.837092+0000) 2023-11-27T15:29:56.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:56 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:56.063+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.138543+0000 front 2023-11-27T15:29:24.138614+0000 (oldest deadline 2023-11-27T15:29:46.438604+0000) 2023-11-27T15:29:56.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:56 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:56.063+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.539608+0000 front 2023-11-27T15:29:30.539640+0000 (oldest deadline 2023-11-27T15:29:54.639653+0000) 2023-11-27T15:29:56.823 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:56 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:56.343+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:56.824 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:56 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:56.343+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:29:56.824 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:56 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:56.343+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:29:56.824 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:56 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:56.343+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:29:56.824 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:56 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:56.343+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.524503+0000 front 2023-11-27T15:29:30.524489+0000 (oldest deadline 2023-11-27T15:29:55.824650+0000) 2023-11-27T15:29:57.077 INFO:journalctl@ceph.osd.7.smithi102.stdout:Nov 27 15:29:56 smithi102 bash[132194]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-7 2023-11-27T15:29:57.277 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:56 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:56.879+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.869099+0000 front 2023-11-27T15:29:09.869149+0000 (oldest deadline 2023-11-27T15:29:35.769236+0000) 2023-11-27T15:29:57.277 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:56 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:56.879+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:15.769572+0000 front 2023-11-27T15:29:15.769595+0000 (oldest deadline 2023-11-27T15:29:39.869711+0000) 2023-11-27T15:29:57.277 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:56 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:56.879+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:19.870581+0000 front 2023-11-27T15:29:19.870486+0000 (oldest deadline 2023-11-27T15:29:45.770238+0000) 2023-11-27T15:29:57.277 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:56 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:56.879+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:28.671510+0000 front 2023-11-27T15:29:28.671404+0000 (oldest deadline 2023-11-27T15:29:54.571359+0000) 2023-11-27T15:29:57.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:56 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:56.828+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:57.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:56 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:56.828+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:29:57.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:56 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:56.828+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:29:57.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:56 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:56.828+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:29:57.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:56 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:56.828+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:29:57.365 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:57 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:57.039+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:57.365 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:57 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:57.039+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:29:57.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:57 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:57.039+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.136916+0000 front 2023-11-27T15:29:17.137063+0000 (oldest deadline 2023-11-27T15:29:41.837092+0000) 2023-11-27T15:29:57.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:57 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:57.039+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.138543+0000 front 2023-11-27T15:29:24.138614+0000 (oldest deadline 2023-11-27T15:29:46.438604+0000) 2023-11-27T15:29:57.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:57 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:57.039+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.539608+0000 front 2023-11-27T15:29:30.539640+0000 (oldest deadline 2023-11-27T15:29:54.639653+0000) 2023-11-27T15:29:57.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:57 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:57.319+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:57.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:57 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:57.319+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:29:57.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:57 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:57.319+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:29:57.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:57 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:57.319+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:29:57.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:57 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:57.319+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.524503+0000 front 2023-11-27T15:29:30.524489+0000 (oldest deadline 2023-11-27T15:29:55.824650+0000) 2023-11-27T15:29:58.107 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.7.service' 2023-11-27T15:29:58.188 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:29:58.188 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-11-27T15:29:58.189 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-11-27T15:29:58.189 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.8 2023-11-27T15:29:58.219 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:57 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:57.861+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.869099+0000 front 2023-11-27T15:29:09.869149+0000 (oldest deadline 2023-11-27T15:29:35.769236+0000) 2023-11-27T15:29:58.219 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:57 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:57.861+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:15.769572+0000 front 2023-11-27T15:29:15.769595+0000 (oldest deadline 2023-11-27T15:29:39.869711+0000) 2023-11-27T15:29:58.219 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:57 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:57.861+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:19.870581+0000 front 2023-11-27T15:29:19.870486+0000 (oldest deadline 2023-11-27T15:29:45.770238+0000) 2023-11-27T15:29:58.219 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:57 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:57.861+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:28.671510+0000 front 2023-11-27T15:29:28.671404+0000 (oldest deadline 2023-11-27T15:29:54.571359+0000) 2023-11-27T15:29:58.220 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:57 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:57.819+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:58.220 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:57 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:57.819+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:29:58.220 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:57 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:57.819+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:29:58.220 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:57 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:57.819+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:29:58.220 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:57 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:57.819+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:29:58.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:58 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:58.009+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:58.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:58 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:58.009+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:29:58.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:58 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:58.009+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.136916+0000 front 2023-11-27T15:29:17.137063+0000 (oldest deadline 2023-11-27T15:29:41.837092+0000) 2023-11-27T15:29:58.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:58 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:58.009+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.138543+0000 front 2023-11-27T15:29:24.138614+0000 (oldest deadline 2023-11-27T15:29:46.438604+0000) 2023-11-27T15:29:58.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:58 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:58.009+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.539608+0000 front 2023-11-27T15:29:30.539640+0000 (oldest deadline 2023-11-27T15:29:54.639653+0000) 2023-11-27T15:29:58.527 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:58 smithi150 systemd[1]: Stopping Ceph osd.8 for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:29:58.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:58 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:58.326+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:58.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:58 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:58.326+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:29:58.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:58 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:58.326+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:29:58.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:58 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:58.326+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:29:58.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:58 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:58.326+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.524503+0000 front 2023-11-27T15:29:30.524489+0000 (oldest deadline 2023-11-27T15:29:55.824650+0000) 2023-11-27T15:29:58.856 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:58 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:58.550+0000 7faf53190700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:29:58.856 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:58 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:58.550+0000 7faf53190700 -1 osd.8 99 *** Got signal Terminated *** 2023-11-27T15:29:58.856 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:58 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:58.550+0000 7faf53190700 -1 osd.8 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:29:58.857 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:58 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:58.852+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.869099+0000 front 2023-11-27T15:29:09.869149+0000 (oldest deadline 2023-11-27T15:29:35.769236+0000) 2023-11-27T15:29:58.857 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:58 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:58.795+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:29:58.857 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:58 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:58.795+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:29:58.857 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:58 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:58.795+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:29:58.857 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:58 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:58.795+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:29:58.857 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:58 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:58.795+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:29:59.278 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:58 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:58.852+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:15.769572+0000 front 2023-11-27T15:29:15.769595+0000 (oldest deadline 2023-11-27T15:29:39.869711+0000) 2023-11-27T15:29:59.278 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:58 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:58.852+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:19.870581+0000 front 2023-11-27T15:29:19.870486+0000 (oldest deadline 2023-11-27T15:29:45.770238+0000) 2023-11-27T15:29:59.278 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:58 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:58.852+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:28.671510+0000 front 2023-11-27T15:29:28.671404+0000 (oldest deadline 2023-11-27T15:29:54.571359+0000) 2023-11-27T15:29:59.365 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:59 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:59.057+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:29:59.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:59 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:59.057+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:29:59.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:59 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:59.057+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.136916+0000 front 2023-11-27T15:29:17.137063+0000 (oldest deadline 2023-11-27T15:29:41.837092+0000) 2023-11-27T15:29:59.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:59 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:59.057+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.138543+0000 front 2023-11-27T15:29:24.138614+0000 (oldest deadline 2023-11-27T15:29:46.438604+0000) 2023-11-27T15:29:59.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:29:59 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:29:59.057+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.539608+0000 front 2023-11-27T15:29:30.539640+0000 (oldest deadline 2023-11-27T15:29:54.639653+0000) 2023-11-27T15:29:59.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:59 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:59.312+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:29:59.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:59 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:59.312+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:29:59.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:59 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:59.312+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:29:59.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:59 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:59.312+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:29:59.578 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:29:59 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:29:59.312+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.524503+0000 front 2023-11-27T15:29:30.524489+0000 (oldest deadline 2023-11-27T15:29:55.824650+0000) 2023-11-27T15:30:00.277 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:59 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:59.890+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.869099+0000 front 2023-11-27T15:29:09.869149+0000 (oldest deadline 2023-11-27T15:29:35.769236+0000) 2023-11-27T15:30:00.278 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:59 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:59.890+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:15.769572+0000 front 2023-11-27T15:29:15.769595+0000 (oldest deadline 2023-11-27T15:29:39.869711+0000) 2023-11-27T15:30:00.278 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:59 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:59.890+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:19.870581+0000 front 2023-11-27T15:29:19.870486+0000 (oldest deadline 2023-11-27T15:29:45.770238+0000) 2023-11-27T15:30:00.278 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:29:59 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:29:59.890+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:28.671510+0000 front 2023-11-27T15:29:28.671404+0000 (oldest deadline 2023-11-27T15:29:54.571359+0000) 2023-11-27T15:30:00.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:59 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:59.781+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:30:00.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:59 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:59.781+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:30:00.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:59 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:59.781+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:30:00.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:59 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:59.781+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:30:00.279 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:29:59 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:29:59.781+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:30:00.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:00 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:00.100+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:30:00.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:00 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:00.100+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:30:00.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:00 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:00.100+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.136916+0000 front 2023-11-27T15:29:17.137063+0000 (oldest deadline 2023-11-27T15:29:41.837092+0000) 2023-11-27T15:30:00.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:00 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:00.100+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.138543+0000 front 2023-11-27T15:29:24.138614+0000 (oldest deadline 2023-11-27T15:29:46.438604+0000) 2023-11-27T15:30:00.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:00 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:00.100+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.539608+0000 front 2023-11-27T15:29:30.539640+0000 (oldest deadline 2023-11-27T15:29:54.639653+0000) 2023-11-27T15:30:00.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:00 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:00.307+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:30:00.578 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:00 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:00.307+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:30:00.578 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:00 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:00.307+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:30:00.578 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:00 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:00.307+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:30:00.578 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:00 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:00.307+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.524503+0000 front 2023-11-27T15:29:30.524489+0000 (oldest deadline 2023-11-27T15:29:55.824650+0000) 2023-11-27T15:30:01.027 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:30:00 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:30:00.938+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.869099+0000 front 2023-11-27T15:29:09.869149+0000 (oldest deadline 2023-11-27T15:29:35.769236+0000) 2023-11-27T15:30:01.028 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:30:00 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:30:00.938+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:15.769572+0000 front 2023-11-27T15:29:15.769595+0000 (oldest deadline 2023-11-27T15:29:39.869711+0000) 2023-11-27T15:30:01.028 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:30:00 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:30:00.938+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:19.870581+0000 front 2023-11-27T15:29:19.870486+0000 (oldest deadline 2023-11-27T15:29:45.770238+0000) 2023-11-27T15:30:01.028 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:30:00 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:30:00.938+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:28.671510+0000 front 2023-11-27T15:29:28.671404+0000 (oldest deadline 2023-11-27T15:29:54.571359+0000) 2023-11-27T15:30:01.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:00 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:00.745+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:30:01.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:00 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:00.745+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:30:01.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:00 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:00.745+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:30:01.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:00 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:00.745+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:30:01.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:00 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:00.745+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:30:01.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:01 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:01.073+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:30:01.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:01 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:01.073+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:30:01.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:01 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:01.073+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.136916+0000 front 2023-11-27T15:29:17.137063+0000 (oldest deadline 2023-11-27T15:29:41.837092+0000) 2023-11-27T15:30:01.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:01 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:01.073+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.138543+0000 front 2023-11-27T15:29:24.138614+0000 (oldest deadline 2023-11-27T15:29:46.438604+0000) 2023-11-27T15:30:01.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:01 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:01.073+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.539608+0000 front 2023-11-27T15:29:30.539640+0000 (oldest deadline 2023-11-27T15:29:54.639653+0000) 2023-11-27T15:30:01.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:01 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:01.348+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:30:01.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:01 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:01.348+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:30:01.829 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:01 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:01.348+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:30:01.829 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:01 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:01.348+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:30:01.829 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:01 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:01.348+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.524503+0000 front 2023-11-27T15:29:30.524489+0000 (oldest deadline 2023-11-27T15:29:55.824650+0000) 2023-11-27T15:30:02.278 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:30:01 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:30:01.896+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.869099+0000 front 2023-11-27T15:29:09.869149+0000 (oldest deadline 2023-11-27T15:29:35.769236+0000) 2023-11-27T15:30:02.278 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:30:01 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:30:01.896+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:15.769572+0000 front 2023-11-27T15:29:15.769595+0000 (oldest deadline 2023-11-27T15:29:39.869711+0000) 2023-11-27T15:30:02.278 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:30:01 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:30:01.896+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:19.870581+0000 front 2023-11-27T15:29:19.870486+0000 (oldest deadline 2023-11-27T15:29:45.770238+0000) 2023-11-27T15:30:02.278 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:30:01 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:30:01.896+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:28.671510+0000 front 2023-11-27T15:29:28.671404+0000 (oldest deadline 2023-11-27T15:29:54.571359+0000) 2023-11-27T15:30:02.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:01 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:01.790+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:30:02.279 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:01 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:01.790+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:30:02.279 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:01 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:01.790+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:30:02.279 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:01 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:01.790+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:30:02.279 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:01 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:01.790+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:30:02.616 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:02 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:02.115+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:30:02.616 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:02 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:02.115+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:30:02.616 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:02 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:02.115+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.136916+0000 front 2023-11-27T15:29:17.137063+0000 (oldest deadline 2023-11-27T15:29:41.837092+0000) 2023-11-27T15:30:02.616 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:02 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:02.115+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.138543+0000 front 2023-11-27T15:29:24.138614+0000 (oldest deadline 2023-11-27T15:29:46.438604+0000) 2023-11-27T15:30:02.616 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:02 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:02.115+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.539608+0000 front 2023-11-27T15:29:30.539640+0000 (oldest deadline 2023-11-27T15:29:54.639653+0000) 2023-11-27T15:30:02.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:02 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:02.336+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:30:02.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:02 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:02.336+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:30:02.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:02 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:02.336+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:30:02.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:02 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:02.336+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:30:02.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:02 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:02.336+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.524503+0000 front 2023-11-27T15:29:30.524489+0000 (oldest deadline 2023-11-27T15:29:55.824650+0000) 2023-11-27T15:30:03.027 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:02 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:02.746+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:30:03.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:02 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:02.746+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:30:03.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:02 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:02.746+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:30:03.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:02 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:02.746+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:30:03.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:02 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:02.746+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:30:03.029 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:30:02 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:30:02.846+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.869099+0000 front 2023-11-27T15:29:09.869149+0000 (oldest deadline 2023-11-27T15:29:35.769236+0000) 2023-11-27T15:30:03.029 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:30:02 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:30:02.846+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:15.769572+0000 front 2023-11-27T15:29:15.769595+0000 (oldest deadline 2023-11-27T15:29:39.869711+0000) 2023-11-27T15:30:03.029 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:30:02 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:30:02.846+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:19.870581+0000 front 2023-11-27T15:29:19.870486+0000 (oldest deadline 2023-11-27T15:29:45.770238+0000) 2023-11-27T15:30:03.029 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:30:02 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8[117212]: 2023-11-27T15:30:02.846+0000 7faf4b77e700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:28.671510+0000 front 2023-11-27T15:29:28.671404+0000 (oldest deadline 2023-11-27T15:29:54.571359+0000) 2023-11-27T15:30:03.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:03 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:03.112+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:30:03.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:03 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:03.112+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:30:03.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:03 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:03.112+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.136916+0000 front 2023-11-27T15:29:17.137063+0000 (oldest deadline 2023-11-27T15:29:41.837092+0000) 2023-11-27T15:30:03.367 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:03 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:03.112+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.138543+0000 front 2023-11-27T15:29:24.138614+0000 (oldest deadline 2023-11-27T15:29:46.438604+0000) 2023-11-27T15:30:03.367 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:03 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:03.112+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.539608+0000 front 2023-11-27T15:29:30.539640+0000 (oldest deadline 2023-11-27T15:29:54.639653+0000) 2023-11-27T15:30:03.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:03 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:03.317+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:30:03.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:03 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:03.317+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:30:03.578 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:03 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:03.317+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:30:03.578 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:03 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:03.317+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:30:03.578 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:03 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:03.317+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.524503+0000 front 2023-11-27T15:29:30.524489+0000 (oldest deadline 2023-11-27T15:29:55.824650+0000) 2023-11-27T15:30:04.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:03 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:03.741+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:30:04.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:03 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:03.741+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:30:04.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:03 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:03.741+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:30:04.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:03 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:03.741+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:30:04.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:03 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:03.741+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:30:04.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:03 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:03.741+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.094339+0000 front 2023-11-27T15:29:38.094297+0000 (oldest deadline 2023-11-27T15:30:03.394439+0000) 2023-11-27T15:30:04.527 INFO:journalctl@ceph.osd.8.smithi150.stdout:Nov 27 15:30:04 smithi150 bash[133488]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-8 2023-11-27T15:30:04.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:04 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:04.290+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:30:04.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:04 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:04.290+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:30:04.577 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:04 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:04.290+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:30:04.578 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:04 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:04.290+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:30:04.578 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:04 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:04.290+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.524503+0000 front 2023-11-27T15:29:30.524489+0000 (oldest deadline 2023-11-27T15:29:55.824650+0000) 2023-11-27T15:30:04.616 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:04 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:04.135+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:30:04.616 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:04 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:04.136+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:30:04.616 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:04 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:04.136+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.136916+0000 front 2023-11-27T15:29:17.137063+0000 (oldest deadline 2023-11-27T15:29:41.837092+0000) 2023-11-27T15:30:04.617 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:04 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:04.136+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.138543+0000 front 2023-11-27T15:29:24.138614+0000 (oldest deadline 2023-11-27T15:29:46.438604+0000) 2023-11-27T15:30:04.617 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:04 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:04.136+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.539608+0000 front 2023-11-27T15:29:30.539640+0000 (oldest deadline 2023-11-27T15:29:54.639653+0000) 2023-11-27T15:30:04.617 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:04 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:04.136+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:39.340907+0000 front 2023-11-27T15:29:39.340922+0000 (oldest deadline 2023-11-27T15:30:03.440782+0000) 2023-11-27T15:30:05.027 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:04 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:04.742+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:30:05.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:04 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:04.742+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:30:05.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:04 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:04.742+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:30:05.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:04 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:04.742+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:30:05.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:04 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:04.742+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:30:05.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:04 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:04.742+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.094339+0000 front 2023-11-27T15:29:38.094297+0000 (oldest deadline 2023-11-27T15:30:03.394439+0000) 2023-11-27T15:30:05.590 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.8.service' 2023-11-27T15:30:05.616 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:05 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:05.175+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:30:05.616 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:05 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:05.175+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:30:05.616 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:05 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:05.175+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.136916+0000 front 2023-11-27T15:29:17.137063+0000 (oldest deadline 2023-11-27T15:29:41.837092+0000) 2023-11-27T15:30:05.617 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:05 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:05.175+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.138543+0000 front 2023-11-27T15:29:24.138614+0000 (oldest deadline 2023-11-27T15:29:46.438604+0000) 2023-11-27T15:30:05.617 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:05 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:05.175+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.539608+0000 front 2023-11-27T15:29:30.539640+0000 (oldest deadline 2023-11-27T15:29:54.639653+0000) 2023-11-27T15:30:05.617 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:05 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:05.175+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:39.340907+0000 front 2023-11-27T15:29:39.340922+0000 (oldest deadline 2023-11-27T15:30:03.440782+0000) 2023-11-27T15:30:05.671 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:30:05.673 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-11-27T15:30:05.673 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-11-27T15:30:05.673 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.9 2023-11-27T15:30:05.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:05 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:05.324+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:30:05.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:05 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:05.325+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:30:05.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:05 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:05.325+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:30:05.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:05 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:05.325+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:30:05.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:05 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:05.325+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.524503+0000 front 2023-11-27T15:29:30.524489+0000 (oldest deadline 2023-11-27T15:29:55.824650+0000) 2023-11-27T15:30:05.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:05 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:05.325+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.026534+0000 front 2023-11-27T15:29:38.026665+0000 (oldest deadline 2023-11-27T15:30:05.027009+0000) 2023-11-27T15:30:06.027 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:05 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:05.774+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:30:06.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:05 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:05.774+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:30:06.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:05 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:05.774+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:30:06.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:05 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:05.774+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:30:06.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:05 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:05.774+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:30:06.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:05 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:05.774+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.094339+0000 front 2023-11-27T15:29:38.094297+0000 (oldest deadline 2023-11-27T15:30:03.394439+0000) 2023-11-27T15:30:06.056 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:05 smithi052 systemd[1]: Stopping Ceph osd.9 for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:30:06.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:06 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:06.052+0000 7fb685e9a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:30:06.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:06 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:06.052+0000 7fb685e9a700 -1 osd.9 99 *** Got signal Terminated *** 2023-11-27T15:30:06.366 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:06 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:06.052+0000 7fb685e9a700 -1 osd.9 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:30:06.367 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:06 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:06.155+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:30:06.367 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:06 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:06.155+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:30:06.367 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:06 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:06.155+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.136916+0000 front 2023-11-27T15:29:17.137063+0000 (oldest deadline 2023-11-27T15:29:41.837092+0000) 2023-11-27T15:30:06.367 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:06 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:06.155+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.138543+0000 front 2023-11-27T15:29:24.138614+0000 (oldest deadline 2023-11-27T15:29:46.438604+0000) 2023-11-27T15:30:06.368 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:06 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:06.155+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.539608+0000 front 2023-11-27T15:29:30.539640+0000 (oldest deadline 2023-11-27T15:29:54.639653+0000) 2023-11-27T15:30:06.368 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:06 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:06.155+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:39.340907+0000 front 2023-11-27T15:29:39.340922+0000 (oldest deadline 2023-11-27T15:30:03.440782+0000) 2023-11-27T15:30:06.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:06 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:06.374+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:30:06.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:06 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:06.374+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:30:06.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:06 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:06.374+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:30:06.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:06 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:06.374+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:30:06.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:06 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:06.374+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.524503+0000 front 2023-11-27T15:29:30.524489+0000 (oldest deadline 2023-11-27T15:29:55.824650+0000) 2023-11-27T15:30:06.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:06 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:06.374+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.026534+0000 front 2023-11-27T15:29:38.026665+0000 (oldest deadline 2023-11-27T15:30:05.027009+0000) 2023-11-27T15:30:07.277 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:06 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:06.811+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:30:07.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:06 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:06.811+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:30:07.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:06 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:06.811+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:30:07.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:06 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:06.811+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:30:07.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:06 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:06.811+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:30:07.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:06 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:06.811+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.094339+0000 front 2023-11-27T15:29:38.094297+0000 (oldest deadline 2023-11-27T15:30:03.394439+0000) 2023-11-27T15:30:07.617 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:07 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:07.143+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:30:07.617 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:07 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:07.143+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:30:07.617 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:07 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:07.143+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.136916+0000 front 2023-11-27T15:29:17.137063+0000 (oldest deadline 2023-11-27T15:29:41.837092+0000) 2023-11-27T15:30:07.617 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:07 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:07.143+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.138543+0000 front 2023-11-27T15:29:24.138614+0000 (oldest deadline 2023-11-27T15:29:46.438604+0000) 2023-11-27T15:30:07.617 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:07 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:07.143+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.539608+0000 front 2023-11-27T15:29:30.539640+0000 (oldest deadline 2023-11-27T15:29:54.639653+0000) 2023-11-27T15:30:07.618 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:07 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:07.143+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:39.340907+0000 front 2023-11-27T15:29:39.340922+0000 (oldest deadline 2023-11-27T15:30:03.440782+0000) 2023-11-27T15:30:07.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:07 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:07.402+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:30:07.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:07 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:07.402+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:30:07.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:07 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:07.402+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:30:07.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:07 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:07.402+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:30:07.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:07 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:07.402+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.524503+0000 front 2023-11-27T15:29:30.524489+0000 (oldest deadline 2023-11-27T15:29:55.824650+0000) 2023-11-27T15:30:07.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:07 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:07.402+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.026534+0000 front 2023-11-27T15:29:38.026665+0000 (oldest deadline 2023-11-27T15:30:05.027009+0000) 2023-11-27T15:30:08.277 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:07 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:07.797+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:30:08.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:07 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:07.797+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:30:08.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:07 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:07.797+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:30:08.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:07 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:07.797+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:30:08.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:07 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:07.797+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:30:08.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:07 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:07.797+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.094339+0000 front 2023-11-27T15:29:38.094297+0000 (oldest deadline 2023-11-27T15:30:03.394439+0000) 2023-11-27T15:30:08.616 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:08 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:08.132+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:30:08.616 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:08 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:08.132+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:30:08.616 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:08 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:08.132+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.136916+0000 front 2023-11-27T15:29:17.137063+0000 (oldest deadline 2023-11-27T15:29:41.837092+0000) 2023-11-27T15:30:08.616 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:08 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:08.132+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.138543+0000 front 2023-11-27T15:29:24.138614+0000 (oldest deadline 2023-11-27T15:29:46.438604+0000) 2023-11-27T15:30:08.616 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:08 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:08.132+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.539608+0000 front 2023-11-27T15:29:30.539640+0000 (oldest deadline 2023-11-27T15:29:54.639653+0000) 2023-11-27T15:30:08.616 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:08 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:08.132+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:39.340907+0000 front 2023-11-27T15:29:39.340922+0000 (oldest deadline 2023-11-27T15:30:03.440782+0000) 2023-11-27T15:30:08.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:08 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:08.416+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:30:08.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:08 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:08.416+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:30:08.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:08 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:08.416+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:30:08.829 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:08 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:08.416+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:30:08.829 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:08 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:08.416+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.524503+0000 front 2023-11-27T15:29:30.524489+0000 (oldest deadline 2023-11-27T15:29:55.824650+0000) 2023-11-27T15:30:08.829 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:08 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:08.416+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.026534+0000 front 2023-11-27T15:29:38.026665+0000 (oldest deadline 2023-11-27T15:30:05.027009+0000) 2023-11-27T15:30:09.027 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:08 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:08.768+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:30:09.027 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:08 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:08.768+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:30:09.027 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:08 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:08.768+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:30:09.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:08 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:08.768+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:30:09.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:08 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:08.768+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:30:09.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:08 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:08.768+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.094339+0000 front 2023-11-27T15:29:38.094297+0000 (oldest deadline 2023-11-27T15:30:03.394439+0000) 2023-11-27T15:30:09.616 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:09 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:09.139+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:30:09.616 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:09 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:09.139+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:30:09.616 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:09 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:09.139+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.136916+0000 front 2023-11-27T15:29:17.137063+0000 (oldest deadline 2023-11-27T15:29:41.837092+0000) 2023-11-27T15:30:09.616 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:09 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:09.139+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.138543+0000 front 2023-11-27T15:29:24.138614+0000 (oldest deadline 2023-11-27T15:29:46.438604+0000) 2023-11-27T15:30:09.616 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:09 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:09.139+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.539608+0000 front 2023-11-27T15:29:30.539640+0000 (oldest deadline 2023-11-27T15:29:54.639653+0000) 2023-11-27T15:30:09.617 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:09 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:09.139+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:39.340907+0000 front 2023-11-27T15:29:39.340922+0000 (oldest deadline 2023-11-27T15:30:03.440782+0000) 2023-11-27T15:30:09.617 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:09 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:09.139+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2023-11-27T15:29:45.042068+0000 front 2023-11-27T15:29:45.042095+0000 (oldest deadline 2023-11-27T15:30:08.542076+0000) 2023-11-27T15:30:09.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:09 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:09.425+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:30:09.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:09 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:09.425+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:30:09.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:09 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:09.425+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:30:09.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:09 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:09.425+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:30:09.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:09 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:09.425+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.524503+0000 front 2023-11-27T15:29:30.524489+0000 (oldest deadline 2023-11-27T15:29:55.824650+0000) 2023-11-27T15:30:09.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:09 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:09.425+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.026534+0000 front 2023-11-27T15:29:38.026665+0000 (oldest deadline 2023-11-27T15:30:05.027009+0000) 2023-11-27T15:30:10.027 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:09 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:09.728+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:30:10.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:09 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:09.728+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:30:10.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:09 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:09.728+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:30:10.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:09 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:09.728+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:30:10.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:09 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:09.728+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:30:10.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:09 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:09.728+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.094339+0000 front 2023-11-27T15:29:38.094297+0000 (oldest deadline 2023-11-27T15:30:03.394439+0000) 2023-11-27T15:30:10.616 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:10 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:10.133+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.233295+0000 front 2023-11-27T15:29:00.233345+0000 (oldest deadline 2023-11-27T15:29:22.533497+0000) 2023-11-27T15:30:10.616 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:10 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:10.133+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:10.135624+0000 front 2023-11-27T15:29:10.135696+0000 (oldest deadline 2023-11-27T15:29:32.435742+0000) 2023-11-27T15:30:10.616 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:10 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:10.133+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.136916+0000 front 2023-11-27T15:29:17.137063+0000 (oldest deadline 2023-11-27T15:29:41.837092+0000) 2023-11-27T15:30:10.617 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:10 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:10.133+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.138543+0000 front 2023-11-27T15:29:24.138614+0000 (oldest deadline 2023-11-27T15:29:46.438604+0000) 2023-11-27T15:30:10.617 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:10 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:10.133+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.539608+0000 front 2023-11-27T15:29:30.539640+0000 (oldest deadline 2023-11-27T15:29:54.639653+0000) 2023-11-27T15:30:10.617 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:10 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:10.133+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:39.340907+0000 front 2023-11-27T15:29:39.340922+0000 (oldest deadline 2023-11-27T15:30:03.440782+0000) 2023-11-27T15:30:10.617 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:10 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9[128273]: 2023-11-27T15:30:10.133+0000 7fb67fc8b700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2023-11-27T15:29:45.042068+0000 front 2023-11-27T15:29:45.042095+0000 (oldest deadline 2023-11-27T15:30:08.542076+0000) 2023-11-27T15:30:10.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:10 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:10.464+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:30:10.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:10 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:10.464+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:30:10.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:10 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:10.464+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:30:10.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:10 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:10.464+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:30:10.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:10 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:10.464+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.524503+0000 front 2023-11-27T15:29:30.524489+0000 (oldest deadline 2023-11-27T15:29:55.824650+0000) 2023-11-27T15:30:10.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:10 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:10.464+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.026534+0000 front 2023-11-27T15:29:38.026665+0000 (oldest deadline 2023-11-27T15:30:05.027009+0000) 2023-11-27T15:30:10.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:10 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:10.464+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2023-11-27T15:29:45.027452+0000 front 2023-11-27T15:29:45.027637+0000 (oldest deadline 2023-11-27T15:30:10.327430+0000) 2023-11-27T15:30:11.027 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:10 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:10.753+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:30:11.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:10 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:10.753+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:30:11.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:10 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:10.753+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:30:11.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:10 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:10.753+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:30:11.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:10 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:10.753+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:30:11.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:10 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:10.753+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.094339+0000 front 2023-11-27T15:29:38.094297+0000 (oldest deadline 2023-11-27T15:30:03.394439+0000) 2023-11-27T15:30:11.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:11 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:11.496+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:30:11.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:11 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:11.496+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:30:11.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:11 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:11.496+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:30:11.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:11 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:11.496+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:30:11.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:11 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:11.496+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.524503+0000 front 2023-11-27T15:29:30.524489+0000 (oldest deadline 2023-11-27T15:29:55.824650+0000) 2023-11-27T15:30:11.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:11 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:11.496+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.026534+0000 front 2023-11-27T15:29:38.026665+0000 (oldest deadline 2023-11-27T15:30:05.027009+0000) 2023-11-27T15:30:11.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:11 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:11.496+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2023-11-27T15:29:45.027452+0000 front 2023-11-27T15:29:45.027637+0000 (oldest deadline 2023-11-27T15:30:10.327430+0000) 2023-11-27T15:30:12.116 INFO:journalctl@ceph.osd.9.smithi052.stdout:Nov 27 15:30:11 smithi052 bash[263845]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-9 2023-11-27T15:30:12.277 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:11 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:11.794+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:30:12.277 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:11 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:11.794+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:30:12.277 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:11 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:11.794+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:30:12.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:11 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:11.794+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:30:12.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:11 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:11.794+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:30:12.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:11 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:11.794+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.094339+0000 front 2023-11-27T15:29:38.094297+0000 (oldest deadline 2023-11-27T15:30:03.394439+0000) 2023-11-27T15:30:12.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:12 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:12.475+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:30:12.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:12 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:12.475+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:30:12.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:12 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:12.475+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:30:12.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:12 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:12.475+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:30:12.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:12 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:12.475+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.524503+0000 front 2023-11-27T15:29:30.524489+0000 (oldest deadline 2023-11-27T15:29:55.824650+0000) 2023-11-27T15:30:12.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:12 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:12.475+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.026534+0000 front 2023-11-27T15:29:38.026665+0000 (oldest deadline 2023-11-27T15:30:05.027009+0000) 2023-11-27T15:30:12.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:12 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:12.475+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2023-11-27T15:29:45.027452+0000 front 2023-11-27T15:29:45.027637+0000 (oldest deadline 2023-11-27T15:30:10.327430+0000) 2023-11-27T15:30:13.277 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:12 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:12.827+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:30:13.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:12 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:12.827+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:30:13.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:12 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:12.827+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:30:13.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:12 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:12.827+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:30:13.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:12 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:12.827+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:30:13.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:12 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:12.827+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.094339+0000 front 2023-11-27T15:29:38.094297+0000 (oldest deadline 2023-11-27T15:30:03.394439+0000) 2023-11-27T15:30:13.418 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.9.service' 2023-11-27T15:30:13.460 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:30:13.460 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-11-27T15:30:13.461 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-11-27T15:30:13.461 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.10 2023-11-27T15:30:13.814 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:13 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:13.486+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:30:13.814 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:13 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:13.486+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:30:13.814 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:13 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:13.486+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:30:13.814 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:13 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:13.486+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:30:13.814 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:13 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:13.486+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.524503+0000 front 2023-11-27T15:29:30.524489+0000 (oldest deadline 2023-11-27T15:29:55.824650+0000) 2023-11-27T15:30:13.815 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:13 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:13.486+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.026534+0000 front 2023-11-27T15:29:38.026665+0000 (oldest deadline 2023-11-27T15:30:05.027009+0000) 2023-11-27T15:30:13.815 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:13 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:13.486+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2023-11-27T15:29:45.027452+0000 front 2023-11-27T15:29:45.027637+0000 (oldest deadline 2023-11-27T15:30:10.327430+0000) 2023-11-27T15:30:13.815 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:13 smithi102 systemd[1]: Stopping Ceph osd.10 for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:30:13.815 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:13 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:13.810+0000 7f92ae11e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:30:13.815 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:13 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:13.810+0000 7f92ae11e700 -1 osd.10 99 *** Got signal Terminated *** 2023-11-27T15:30:13.815 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:13 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:13.810+0000 7f92ae11e700 -1 osd.10 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:30:14.277 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:13 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:13.789+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:30:14.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:13 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:13.789+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:30:14.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:13 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:13.789+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:30:14.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:13 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:13.789+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:30:14.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:13 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:13.789+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:30:14.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:13 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:13.789+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.094339+0000 front 2023-11-27T15:29:38.094297+0000 (oldest deadline 2023-11-27T15:30:03.394439+0000) 2023-11-27T15:30:14.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:14 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:14.524+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:30:14.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:14 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:14.524+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:30:14.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:14 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:14.524+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:30:14.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:14 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:14.524+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:30:14.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:14 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:14.524+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.524503+0000 front 2023-11-27T15:29:30.524489+0000 (oldest deadline 2023-11-27T15:29:55.824650+0000) 2023-11-27T15:30:14.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:14 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:14.524+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.026534+0000 front 2023-11-27T15:29:38.026665+0000 (oldest deadline 2023-11-27T15:30:05.027009+0000) 2023-11-27T15:30:14.829 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:14 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:14.524+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2023-11-27T15:29:45.027452+0000 front 2023-11-27T15:29:45.027637+0000 (oldest deadline 2023-11-27T15:30:10.327430+0000) 2023-11-27T15:30:15.277 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:14 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:14.774+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:30:15.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:14 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:14.774+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:30:15.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:14 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:14.774+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:30:15.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:14 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:14.774+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:30:15.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:14 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:14.774+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:30:15.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:14 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:14.774+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.094339+0000 front 2023-11-27T15:29:38.094297+0000 (oldest deadline 2023-11-27T15:30:03.394439+0000) 2023-11-27T15:30:15.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:15 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:15.501+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:30:15.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:15 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:15.501+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:30:15.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:15 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:15.501+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:30:15.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:15 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:15.501+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:30:15.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:15 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:15.501+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.524503+0000 front 2023-11-27T15:29:30.524489+0000 (oldest deadline 2023-11-27T15:29:55.824650+0000) 2023-11-27T15:30:15.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:15 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:15.501+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.026534+0000 front 2023-11-27T15:29:38.026665+0000 (oldest deadline 2023-11-27T15:30:05.027009+0000) 2023-11-27T15:30:15.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:15 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:15.501+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2023-11-27T15:29:45.027452+0000 front 2023-11-27T15:29:45.027637+0000 (oldest deadline 2023-11-27T15:30:10.327430+0000) 2023-11-27T15:30:16.277 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:15 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:15.817+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:30:16.277 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:15 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:15.817+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:30:16.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:15 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:15.817+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:30:16.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:15 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:15.817+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:30:16.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:15 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:15.817+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:30:16.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:15 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:15.817+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.094339+0000 front 2023-11-27T15:29:38.094297+0000 (oldest deadline 2023-11-27T15:30:03.394439+0000) 2023-11-27T15:30:16.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:16 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:16.501+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:30:16.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:16 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:16.501+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:30:16.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:16 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:16.501+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:30:16.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:16 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:16.501+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:30:16.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:16 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:16.501+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.524503+0000 front 2023-11-27T15:29:30.524489+0000 (oldest deadline 2023-11-27T15:29:55.824650+0000) 2023-11-27T15:30:16.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:16 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:16.501+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.026534+0000 front 2023-11-27T15:29:38.026665+0000 (oldest deadline 2023-11-27T15:30:05.027009+0000) 2023-11-27T15:30:16.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:16 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:16.501+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2023-11-27T15:29:45.027452+0000 front 2023-11-27T15:29:45.027637+0000 (oldest deadline 2023-11-27T15:30:10.327430+0000) 2023-11-27T15:30:17.277 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:16 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:16.777+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:30:17.277 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:16 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:16.777+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:30:17.277 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:16 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:16.777+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:30:17.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:16 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:16.777+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:30:17.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:16 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:16.777+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:30:17.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:16 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:16.777+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.094339+0000 front 2023-11-27T15:29:38.094297+0000 (oldest deadline 2023-11-27T15:30:03.394439+0000) 2023-11-27T15:30:17.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:17 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:17.504+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:30:17.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:17 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:17.504+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:30:17.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:17 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:17.504+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:30:17.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:17 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:17.504+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:30:17.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:17 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:17.504+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.524503+0000 front 2023-11-27T15:29:30.524489+0000 (oldest deadline 2023-11-27T15:29:55.824650+0000) 2023-11-27T15:30:17.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:17 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:17.504+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.026534+0000 front 2023-11-27T15:29:38.026665+0000 (oldest deadline 2023-11-27T15:30:05.027009+0000) 2023-11-27T15:30:17.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:17 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:17.504+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2023-11-27T15:29:45.027452+0000 front 2023-11-27T15:29:45.027637+0000 (oldest deadline 2023-11-27T15:30:10.327430+0000) 2023-11-27T15:30:18.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:17 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:17.811+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:30:18.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:17 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:17.811+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:30:18.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:17 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:17.811+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:30:18.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:17 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:17.811+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:30:18.279 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:17 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:17.811+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:30:18.279 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:17 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:17.811+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.094339+0000 front 2023-11-27T15:29:38.094297+0000 (oldest deadline 2023-11-27T15:30:03.394439+0000) 2023-11-27T15:30:18.279 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:17 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:17.811+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6820 osd.7 since back 2023-11-27T15:29:53.296348+0000 front 2023-11-27T15:29:53.296385+0000 (oldest deadline 2023-11-27T15:30:17.396303+0000) 2023-11-27T15:30:18.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:18 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:18.481+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.319548+0000 front 2023-11-27T15:29:00.319401+0000 (oldest deadline 2023-11-27T15:29:25.019441+0000) 2023-11-27T15:30:18.827 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:18 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:18.481+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:09.620766+0000 front 2023-11-27T15:29:09.620702+0000 (oldest deadline 2023-11-27T15:29:33.120975+0000) 2023-11-27T15:30:18.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:18 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:18.481+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:13.622442+0000 front 2023-11-27T15:29:13.622282+0000 (oldest deadline 2023-11-27T15:29:38.922176+0000) 2023-11-27T15:30:18.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:18 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:18.481+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:22.924045+0000 front 2023-11-27T15:29:22.923902+0000 (oldest deadline 2023-11-27T15:29:45.823831+0000) 2023-11-27T15:30:18.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:18 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:18.481+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:30.524503+0000 front 2023-11-27T15:29:30.524489+0000 (oldest deadline 2023-11-27T15:29:55.824650+0000) 2023-11-27T15:30:18.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:18 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:18.481+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.026534+0000 front 2023-11-27T15:29:38.026665+0000 (oldest deadline 2023-11-27T15:30:05.027009+0000) 2023-11-27T15:30:18.828 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:18 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10[121720]: 2023-11-27T15:30:18.481+0000 7f92a670c700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2023-11-27T15:29:45.027452+0000 front 2023-11-27T15:29:45.027637+0000 (oldest deadline 2023-11-27T15:30:10.327430+0000) 2023-11-27T15:30:19.277 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:18 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:18.782+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:30:19.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:18 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:18.782+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:30:19.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:18 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:18.782+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:30:19.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:18 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:18.782+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:30:19.279 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:18 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:18.782+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:30:19.279 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:18 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:18.782+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.094339+0000 front 2023-11-27T15:29:38.094297+0000 (oldest deadline 2023-11-27T15:30:03.394439+0000) 2023-11-27T15:30:19.279 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:18 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:18.782+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6820 osd.7 since back 2023-11-27T15:29:53.296348+0000 front 2023-11-27T15:29:53.296385+0000 (oldest deadline 2023-11-27T15:30:17.396303+0000) 2023-11-27T15:30:19.500 INFO:journalctl@ceph.osd.10.smithi102.stdout:Nov 27 15:30:19 smithi102 bash[132463]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-10 2023-11-27T15:30:20.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:19 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:19.744+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:30:20.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:19 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:19.744+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:30:20.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:19 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:19.744+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:30:20.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:19 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:19.744+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:30:20.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:19 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:19.744+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:30:20.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:19 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:19.744+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.094339+0000 front 2023-11-27T15:29:38.094297+0000 (oldest deadline 2023-11-27T15:30:03.394439+0000) 2023-11-27T15:30:20.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:19 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:19.744+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6820 osd.7 since back 2023-11-27T15:29:53.296348+0000 front 2023-11-27T15:29:53.296385+0000 (oldest deadline 2023-11-27T15:30:17.396303+0000) 2023-11-27T15:30:20.862 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.10.service' 2023-11-27T15:30:20.902 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:30:20.903 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-11-27T15:30:20.903 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-11-27T15:30:20.903 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.11 2023-11-27T15:30:21.027 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:20 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:20.754+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:30:21.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:20 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:20.754+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:30:21.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:20 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:20.754+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:30:21.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:20 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:20.754+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:30:21.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:20 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:20.754+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:30:21.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:20 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:20.754+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.094339+0000 front 2023-11-27T15:29:38.094297+0000 (oldest deadline 2023-11-27T15:30:03.394439+0000) 2023-11-27T15:30:21.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:20 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:20.754+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6820 osd.7 since back 2023-11-27T15:29:53.296348+0000 front 2023-11-27T15:29:53.296385+0000 (oldest deadline 2023-11-27T15:30:17.396303+0000) 2023-11-27T15:30:21.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:20 smithi150 systemd[1]: Stopping Ceph osd.11 for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:30:21.527 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:21 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:21.264+0000 7fa0b8012700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:30:21.528 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:21 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:21.264+0000 7fa0b8012700 -1 osd.11 99 *** Got signal Terminated *** 2023-11-27T15:30:21.528 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:21 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:21.264+0000 7fa0b8012700 -1 osd.11 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:30:22.027 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:21 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:21.735+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:30:22.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:21 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:21.735+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:30:22.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:21 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:21.735+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:30:22.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:21 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:21.735+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:30:22.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:21 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:21.735+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:30:22.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:21 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:21.735+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.094339+0000 front 2023-11-27T15:29:38.094297+0000 (oldest deadline 2023-11-27T15:30:03.394439+0000) 2023-11-27T15:30:22.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:21 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:21.735+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6820 osd.7 since back 2023-11-27T15:29:53.296348+0000 front 2023-11-27T15:29:53.296385+0000 (oldest deadline 2023-11-27T15:30:17.396303+0000) 2023-11-27T15:30:23.277 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:22 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:22.783+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:30:23.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:22 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:22.783+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:30:23.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:22 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:22.783+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:30:23.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:22 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:22.783+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:30:23.278 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:22 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:22.783+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:30:23.279 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:22 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:22.783+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.094339+0000 front 2023-11-27T15:29:38.094297+0000 (oldest deadline 2023-11-27T15:30:03.394439+0000) 2023-11-27T15:30:23.279 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:22 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:22.783+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6820 osd.7 since back 2023-11-27T15:29:53.296348+0000 front 2023-11-27T15:29:53.296385+0000 (oldest deadline 2023-11-27T15:30:17.396303+0000) 2023-11-27T15:30:24.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:23 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:23.744+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:30:24.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:23 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:23.744+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:30:24.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:23 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:23.744+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:30:24.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:23 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:23.744+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:30:24.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:23 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:23.744+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:30:24.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:23 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:23.744+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.094339+0000 front 2023-11-27T15:29:38.094297+0000 (oldest deadline 2023-11-27T15:30:03.394439+0000) 2023-11-27T15:30:24.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:23 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:23.744+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6820 osd.7 since back 2023-11-27T15:29:53.296348+0000 front 2023-11-27T15:29:53.296385+0000 (oldest deadline 2023-11-27T15:30:17.396303+0000) 2023-11-27T15:30:25.027 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:24 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:24.747+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:30:25.027 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:24 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:24.747+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:30:25.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:24 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:24.747+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:30:25.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:24 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:24.747+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:30:25.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:24 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:24.747+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:30:25.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:24 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:24.747+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.094339+0000 front 2023-11-27T15:29:38.094297+0000 (oldest deadline 2023-11-27T15:30:03.394439+0000) 2023-11-27T15:30:25.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:24 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:24.747+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6820 osd.7 since back 2023-11-27T15:29:53.296348+0000 front 2023-11-27T15:29:53.296385+0000 (oldest deadline 2023-11-27T15:30:17.396303+0000) 2023-11-27T15:30:26.027 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:25 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:25.739+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2023-11-27T15:29:00.088106+0000 front 2023-11-27T15:29:00.087959+0000 (oldest deadline 2023-11-27T15:29:24.188016+0000) 2023-11-27T15:30:26.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:25 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:25.739+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2023-11-27T15:29:06.489385+0000 front 2023-11-27T15:29:06.489085+0000 (oldest deadline 2023-11-27T15:29:32.389187+0000) 2023-11-27T15:30:26.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:25 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:25.739+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6804 osd.2 since back 2023-11-27T15:29:17.090269+0000 front 2023-11-27T15:29:17.090236+0000 (oldest deadline 2023-11-27T15:29:40.590587+0000) 2023-11-27T15:30:26.028 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:25 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:25.739+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2023-11-27T15:29:24.691719+0000 front 2023-11-27T15:29:24.691808+0000 (oldest deadline 2023-11-27T15:29:48.791900+0000) 2023-11-27T15:30:26.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:25 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:25.739+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2023-11-27T15:29:29.293345+0000 front 2023-11-27T15:29:29.293222+0000 (oldest deadline 2023-11-27T15:29:54.593147+0000) 2023-11-27T15:30:26.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:25 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:25.739+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.150:6812 osd.5 since back 2023-11-27T15:29:38.094339+0000 front 2023-11-27T15:29:38.094297+0000 (oldest deadline 2023-11-27T15:30:03.394439+0000) 2023-11-27T15:30:26.029 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:25 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11[122658]: 2023-11-27T15:30:25.739+0000 7fa0b4608700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.102:6820 osd.7 since back 2023-11-27T15:29:53.296348+0000 front 2023-11-27T15:29:53.296385+0000 (oldest deadline 2023-11-27T15:30:17.396303+0000) 2023-11-27T15:30:26.944 INFO:journalctl@ceph.osd.11.smithi150.stdout:Nov 27 15:30:26 smithi150 bash[133764]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-osd-11 2023-11-27T15:30:28.413 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@osd.11.service' 2023-11-27T15:30:28.453 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:30:28.454 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-11-27T15:30:28.454 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-11-27T15:30:28.454 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.a 2023-11-27T15:30:28.828 INFO:journalctl@ceph.mds.a.smithi052.stdout:Nov 27 15:30:28 smithi052 systemd[1]: Stopping Ceph mds.a for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:30:29.115 INFO:journalctl@ceph.mds.a.smithi052.stdout:Nov 27 15:30:28 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-a[133646]: 2023-11-27T15:30:28.824+0000 7f65ccd1b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:30:29.116 INFO:journalctl@ceph.mds.a.smithi052.stdout:Nov 27 15:30:28 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-a[133646]: 2023-11-27T15:30:28.824+0000 7f65ccd1b700 -1 mds.a *** got signal Terminated *** 2023-11-27T15:30:30.878 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.a.service' 2023-11-27T15:30:30.914 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:30:30.915 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-11-27T15:30:30.915 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-11-27T15:30:30.916 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.d 2023-11-27T15:30:31.302 INFO:journalctl@ceph.mds.d.smithi052.stdout:Nov 27 15:30:30 smithi052 systemd[1]: Stopping Ceph mds.d for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:30:31.616 INFO:journalctl@ceph.mds.d.smithi052.stdout:Nov 27 15:30:31 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-d[132640]: 2023-11-27T15:30:31.298+0000 7f9d97b50700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:30:31.616 INFO:journalctl@ceph.mds.d.smithi052.stdout:Nov 27 15:30:31 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-d[132640]: 2023-11-27T15:30:31.298+0000 7f9d97b50700 -1 mds.d *** got signal Terminated *** 2023-11-27T15:30:34.866 INFO:journalctl@ceph.mds.d.smithi052.stdout:Nov 27 15:30:34 smithi052 bash[264255]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-d 2023-11-27T15:30:35.179 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.d.service' 2023-11-27T15:30:35.214 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:30:35.214 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-11-27T15:30:35.215 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-11-27T15:30:35.215 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.g 2023-11-27T15:30:35.609 INFO:journalctl@ceph.mds.g.smithi052.stdout:Nov 27 15:30:35 smithi052 systemd[1]: Stopping Ceph mds.g for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:30:35.866 INFO:journalctl@ceph.mds.g.smithi052.stdout:Nov 27 15:30:35 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-g[133259]: 2023-11-27T15:30:35.605+0000 7f393669e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:30:35.866 INFO:journalctl@ceph.mds.g.smithi052.stdout:Nov 27 15:30:35 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-g[133259]: 2023-11-27T15:30:35.605+0000 7f393669e700 -1 mds.g *** got signal Terminated *** 2023-11-27T15:30:40.116 INFO:journalctl@ceph.mds.g.smithi052.stdout:Nov 27 15:30:39 smithi052 bash[264400]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-g 2023-11-27T15:30:40.220 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.g.service' 2023-11-27T15:30:40.255 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:30:40.256 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-11-27T15:30:40.256 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-11-27T15:30:40.256 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.j 2023-11-27T15:30:40.615 INFO:journalctl@ceph.mds.j.smithi052.stdout:Nov 27 15:30:40 smithi052 systemd[1]: Stopping Ceph mds.j for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:30:41.116 INFO:journalctl@ceph.mds.j.smithi052.stdout:Nov 27 15:30:40 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-j[132277]: 2023-11-27T15:30:40.681+0000 7f2e341cc700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:30:41.116 INFO:journalctl@ceph.mds.j.smithi052.stdout:Nov 27 15:30:40 smithi052 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-j[132277]: 2023-11-27T15:30:40.681+0000 7f2e341cc700 -1 mds.j *** got signal Terminated *** 2023-11-27T15:30:44.722 INFO:journalctl@ceph.mds.j.smithi052.stdout:Nov 27 15:30:44 smithi052 bash[264543]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-j 2023-11-27T15:30:44.728 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.j.service' 2023-11-27T15:30:44.763 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:30:44.764 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-11-27T15:30:44.764 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-11-27T15:30:44.764 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.b 2023-11-27T15:30:45.077 INFO:journalctl@ceph.mds.b.smithi102.stdout:Nov 27 15:30:44 smithi102 systemd[1]: Stopping Ceph mds.b for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:30:45.577 INFO:journalctl@ceph.mds.b.smithi102.stdout:Nov 27 15:30:45 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-b[125504]: 2023-11-27T15:30:45.151+0000 7fee80e10700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:30:45.577 INFO:journalctl@ceph.mds.b.smithi102.stdout:Nov 27 15:30:45 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-b[125504]: 2023-11-27T15:30:45.151+0000 7fee80e10700 -1 mds.b *** got signal Terminated *** 2023-11-27T15:30:46.779 INFO:journalctl@ceph.mds.b.smithi102.stdout:Nov 27 15:30:46 smithi102 bash[132724]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-b 2023-11-27T15:30:46.967 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.b.service' 2023-11-27T15:30:47.002 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:30:47.002 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-11-27T15:30:47.002 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-11-27T15:30:47.002 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.e 2023-11-27T15:30:47.327 INFO:journalctl@ceph.mds.e.smithi102.stdout:Nov 27 15:30:47 smithi102 systemd[1]: Stopping Ceph mds.e for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:30:47.827 INFO:journalctl@ceph.mds.e.smithi102.stdout:Nov 27 15:30:47 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-e[124816]: 2023-11-27T15:30:47.380+0000 7f654a42b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:30:47.827 INFO:journalctl@ceph.mds.e.smithi102.stdout:Nov 27 15:30:47 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-e[124816]: 2023-11-27T15:30:47.380+0000 7f654a42b700 -1 mds.e *** got signal Terminated *** 2023-11-27T15:30:50.827 INFO:journalctl@ceph.mds.e.smithi102.stdout:Nov 27 15:30:50 smithi102 bash[132864]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-e 2023-11-27T15:30:50.968 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.e.service' 2023-11-27T15:30:51.003 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:30:51.004 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-11-27T15:30:51.004 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-11-27T15:30:51.004 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.h 2023-11-27T15:30:51.327 INFO:journalctl@ceph.mds.h.smithi102.stdout:Nov 27 15:30:51 smithi102 systemd[1]: Stopping Ceph mds.h for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:30:51.827 INFO:journalctl@ceph.mds.h.smithi102.stdout:Nov 27 15:30:51 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-h[125157]: 2023-11-27T15:30:51.370+0000 7f6d76665700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:30:51.827 INFO:journalctl@ceph.mds.h.smithi102.stdout:Nov 27 15:30:51 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-h[125157]: 2023-11-27T15:30:51.370+0000 7f6d76665700 -1 mds.h *** got signal Terminated *** 2023-11-27T15:30:54.750 INFO:journalctl@ceph.mds.h.smithi102.stdout:Nov 27 15:30:54 smithi102 bash[133010]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-h 2023-11-27T15:30:54.936 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.h.service' 2023-11-27T15:30:54.969 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:30:54.969 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-11-27T15:30:54.970 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-11-27T15:30:54.970 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.k 2023-11-27T15:30:55.327 INFO:journalctl@ceph.mds.k.smithi102.stdout:Nov 27 15:30:55 smithi102 systemd[1]: Stopping Ceph mds.k for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:30:55.827 INFO:journalctl@ceph.mds.k.smithi102.stdout:Nov 27 15:30:55 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-k[124472]: 2023-11-27T15:30:55.346+0000 7fb318c5c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:30:55.827 INFO:journalctl@ceph.mds.k.smithi102.stdout:Nov 27 15:30:55 smithi102 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-k[124472]: 2023-11-27T15:30:55.346+0000 7fb318c5c700 -1 mds.k *** got signal Terminated *** 2023-11-27T15:30:58.308 INFO:journalctl@ceph.mds.k.smithi102.stdout:Nov 27 15:30:58 smithi102 bash[133147]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-k 2023-11-27T15:30:58.513 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.k.service' 2023-11-27T15:30:58.546 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:30:58.547 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-11-27T15:30:58.547 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-11-27T15:30:58.547 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.c 2023-11-27T15:30:58.980 INFO:journalctl@ceph.mds.c.smithi150.stdout:Nov 27 15:30:58 smithi150 systemd[1]: Stopping Ceph mds.c for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:30:59.277 INFO:journalctl@ceph.mds.c.smithi150.stdout:Nov 27 15:30:58 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-c[126251]: 2023-11-27T15:30:58.976+0000 7f6fca67b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:30:59.278 INFO:journalctl@ceph.mds.c.smithi150.stdout:Nov 27 15:30:58 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-c[126251]: 2023-11-27T15:30:58.976+0000 7f6fca67b700 -1 mds.c *** got signal Terminated *** 2023-11-27T15:31:03.019 INFO:journalctl@ceph.mds.c.smithi150.stdout:Nov 27 15:31:02 smithi150 bash[134027]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-c 2023-11-27T15:31:03.025 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.c.service' 2023-11-27T15:31:03.062 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:31:03.063 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-11-27T15:31:03.063 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-11-27T15:31:03.063 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.f 2023-11-27T15:31:03.277 INFO:journalctl@ceph.mds.f.smithi150.stdout:Nov 27 15:31:03 smithi150 systemd[1]: Stopping Ceph mds.f for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:31:03.777 INFO:journalctl@ceph.mds.f.smithi150.stdout:Nov 27 15:31:03 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-f[125908]: 2023-11-27T15:31:03.441+0000 7f6263b71700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:31:03.777 INFO:journalctl@ceph.mds.f.smithi150.stdout:Nov 27 15:31:03 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-f[125908]: 2023-11-27T15:31:03.441+0000 7f6263b71700 -1 mds.f *** got signal Terminated *** 2023-11-27T15:31:06.706 INFO:journalctl@ceph.mds.f.smithi150.stdout:Nov 27 15:31:06 smithi150 bash[134170]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-f 2023-11-27T15:31:06.894 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.f.service' 2023-11-27T15:31:06.928 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:31:06.928 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-11-27T15:31:06.929 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-11-27T15:31:06.929 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.i 2023-11-27T15:31:07.277 INFO:journalctl@ceph.mds.i.smithi150.stdout:Nov 27 15:31:06 smithi150 systemd[1]: Stopping Ceph mds.i for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:31:07.777 INFO:journalctl@ceph.mds.i.smithi150.stdout:Nov 27 15:31:07 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-i[126596]: 2023-11-27T15:31:07.313+0000 7f9abe2ef700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:31:07.777 INFO:journalctl@ceph.mds.i.smithi150.stdout:Nov 27 15:31:07 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-i[126596]: 2023-11-27T15:31:07.313+0000 7f9abe2ef700 -1 mds.i *** got signal Terminated *** 2023-11-27T15:31:10.675 INFO:journalctl@ceph.mds.i.smithi150.stdout:Nov 27 15:31:10 smithi150 bash[134308]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-i 2023-11-27T15:31:10.863 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.i.service' 2023-11-27T15:31:10.896 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:31:10.897 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-11-27T15:31:10.897 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-11-27T15:31:10.898 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.l 2023-11-27T15:31:11.277 INFO:journalctl@ceph.mds.l.smithi150.stdout:Nov 27 15:31:10 smithi150 systemd[1]: Stopping Ceph mds.l for c6c7df88-8d36-11ee-95a2-87774f69a715... 2023-11-27T15:31:11.777 INFO:journalctl@ceph.mds.l.smithi150.stdout:Nov 27 15:31:11 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-l[125569]: 2023-11-27T15:31:11.274+0000 7ff1910fa700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:31:11.777 INFO:journalctl@ceph.mds.l.smithi150.stdout:Nov 27 15:31:11 smithi150 ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-l[125569]: 2023-11-27T15:31:11.274+0000 7ff1910fa700 -1 mds.l *** got signal Terminated *** 2023-11-27T15:31:15.777 INFO:journalctl@ceph.mds.l.smithi150.stdout:Nov 27 15:31:15 smithi150 bash[134446]: ceph-c6c7df88-8d36-11ee-95a2-87774f69a715-mds-l 2023-11-27T15:31:15.812 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c6c7df88-8d36-11ee-95a2-87774f69a715@mds.l.service' 2023-11-27T15:31:15.849 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:31:15.849 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-11-27T15:31:15.849 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 --force --keep-logs 2023-11-27T15:31:18.499 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 --force --keep-logs 2023-11-27T15:31:20.454 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 --force --keep-logs 2023-11-27T15:31:22.411 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-27T15:31:22.448 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-27T15:31:22.483 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-27T15:31:22.513 INFO:tasks.cephadm:Archiving crash dumps... 2023-11-27T15:31:22.515 DEBUG:teuthology.misc:Transferring archived files from smithi052:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/crash to /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467810/remote/smithi052/crash 2023-11-27T15:31:22.515 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/crash -- . 2023-11-27T15:31:22.560 INFO:teuthology.orchestra.run.smithi052.stderr:tar: /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/crash: Cannot open: No such file or directory 2023-11-27T15:31:22.561 INFO:teuthology.orchestra.run.smithi052.stderr:tar: Error is not recoverable: exiting now 2023-11-27T15:31:22.563 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/crash to /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467810/remote/smithi102/crash 2023-11-27T15:31:22.563 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/crash -- . 2023-11-27T15:31:22.626 INFO:teuthology.orchestra.run.smithi102.stderr:tar: /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/crash: Cannot open: No such file or directory 2023-11-27T15:31:22.626 INFO:teuthology.orchestra.run.smithi102.stderr:tar: Error is not recoverable: exiting now 2023-11-27T15:31:22.628 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/crash to /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467810/remote/smithi150/crash 2023-11-27T15:31:22.629 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/crash -- . 2023-11-27T15:31:22.663 INFO:teuthology.orchestra.run.smithi150.stderr:tar: /var/lib/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/crash: Cannot open: No such file or directory 2023-11-27T15:31:22.663 INFO:teuthology.orchestra.run.smithi150.stderr:tar: Error is not recoverable: exiting now 2023-11-27T15:31:22.664 INFO:tasks.cephadm:Checking cluster log for badness... 2023-11-27T15:31:22.664 DEBUG:teuthology.orchestra.run.smithi052:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-11-27T15:31:22.699 INFO:teuthology.orchestra.run.smithi052.stderr:grep: /var/log/ceph/c6c7df88-8d36-11ee-95a2-87774f69a715/ceph.log: No such file or directory 2023-11-27T15:31:22.705 INFO:tasks.cephadm:Compressing logs... 2023-11-27T15:31:22.706 DEBUG:teuthology.orchestra.run.smithi052:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-27T15:31:22.748 DEBUG:teuthology.orchestra.run.smithi102:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-27T15:31:22.750 DEBUG:teuthology.orchestra.run.smithi150:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-27T15:31:22.777 INFO:teuthology.orchestra.run.smithi150.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-11-27T15:31:22.778 INFO:teuthology.orchestra.run.smithi102.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-11-27T15:31:22.792 INFO:teuthology.orchestra.run.smithi052.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-11-27T15:34:30.478 INFO:tasks.cephadm:Archiving logs... 2023-11-27T15:34:30.479 DEBUG:teuthology.misc:Transferring archived files from smithi052:/var/log/ceph to /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467810/remote/smithi052/log 2023-11-27T15:34:30.480 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-27T15:34:33.032 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/log/ceph to /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467810/remote/smithi102/log 2023-11-27T15:34:33.033 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-27T15:34:52.067 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/log/ceph to /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467810/remote/smithi150/log 2023-11-27T15:34:52.068 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-27T15:35:03.146 INFO:tasks.cephadm:Removing cluster... 2023-11-27T15:35:03.146 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 --force 2023-11-27T15:35:04.053 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 --force 2023-11-27T15:35:05.068 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c6c7df88-8d36-11ee-95a2-87774f69a715 --force 2023-11-27T15:35:06.015 INFO:tasks.cephadm:Removing cephadm ... 2023-11-27T15:35:06.015 DEBUG:teuthology.orchestra.run.smithi052:> rm -rf /home/ubuntu/cephtest/cephadm 2023-11-27T15:35:06.038 DEBUG:teuthology.orchestra.run.smithi102:> rm -rf /home/ubuntu/cephtest/cephadm 2023-11-27T15:35:06.060 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf /home/ubuntu/cephtest/cephadm 2023-11-27T15:35:06.078 INFO:tasks.cephadm:Teardown complete 2023-11-27T15:35:06.079 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-27T15:35:06.091 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-27T15:35:06.091 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-27T15:35:06.094 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-27T15:35:06.103 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-27T15:35:06.220 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-27T15:35:06.230 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-27T15:35:06.231 DEBUG:teuthology.orchestra.run.smithi052:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-27T15:35:06.234 DEBUG:teuthology.orchestra.run.smithi102:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-27T15:35:06.237 DEBUG:teuthology.orchestra.run.smithi150:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-27T15:35:06.250 INFO:teuthology.orchestra.run.smithi052.stderr:bash: ntpq: command not found 2023-11-27T15:35:06.253 INFO:teuthology.orchestra.run.smithi150.stderr:bash: ntpq: command not found 2023-11-27T15:35:06.253 INFO:teuthology.orchestra.run.smithi102.stderr:bash: ntpq: command not found 2023-11-27T15:35:06.258 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-27T15:35:06.258 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2023-11-27T15:35:06.258 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 53 -455us[ -478us] +/- 91ms 2023-11-27T15:35:06.259 INFO:teuthology.orchestra.run.smithi150.stdout:^* hv02.front.sepia.ceph.com 3 6 377 52 -2859us[-2881us] +/- 60ms 2023-11-27T15:35:06.259 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 51 +5337us[+5337us] +/- 96ms 2023-11-27T15:35:06.259 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:35:06.260 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-27T15:35:06.261 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2023-11-27T15:35:06.261 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 49 -473us[ -473us] +/- 91ms 2023-11-27T15:35:06.261 INFO:teuthology.orchestra.run.smithi102.stdout:^* hv02.front.sepia.ceph.com 3 6 377 50 -2885us[-2892us] +/- 60ms 2023-11-27T15:35:06.261 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 49 +5342us[+5342us] +/- 96ms 2023-11-27T15:35:06.261 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:35:06.266 INFO:teuthology.orchestra.run.smithi052.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-27T15:35:06.266 INFO:teuthology.orchestra.run.smithi052.stdout:=============================================================================== 2023-11-27T15:35:06.266 INFO:teuthology.orchestra.run.smithi052.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 48 -403us[ -403us] +/- 91ms 2023-11-27T15:35:06.266 INFO:teuthology.orchestra.run.smithi052.stdout:^* hv02.front.sepia.ceph.com 3 6 377 49 -2819us[-2826us] +/- 60ms 2023-11-27T15:35:06.267 INFO:teuthology.orchestra.run.smithi052.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 51 +5388us[+5380us] +/- 96ms 2023-11-27T15:35:06.267 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:35:06.268 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-27T15:35:06.278 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-27T15:35:06.279 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-27T15:35:06.289 DEBUG:teuthology.orchestra.run.smithi052:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-27T15:35:06.533 DEBUG:teuthology.orchestra.run.smithi102:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-27T15:35:06.639 DEBUG:teuthology.orchestra.run.smithi150:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-27T15:35:06.737 DEBUG:teuthology.orchestra.run.smithi052:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-27T15:35:06.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T15:35:06.781 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-27T15:35:06.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T15:35:06.819 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-27T15:35:06.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T15:35:06.857 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-27T15:35:06.871 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-27T15:35:06.884 INFO:teuthology.task.internal:Duration was 2456.336975 seconds 2023-11-27T15:35:06.884 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-27T15:35:06.912 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-27T15:35:06.912 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-27T15:35:06.917 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-27T15:35:06.921 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-27T15:35:06.970 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-27T15:35:06.970 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-27T15:35:07.010 INFO:teuthology.orchestra.run.smithi052.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-27T15:35:07.289 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-27T15:35:07.289 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi052.front.sepia.ceph.com 2023-11-27T15:35:07.289 DEBUG:teuthology.orchestra.run.smithi052:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-27T15:35:07.340 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi102.front.sepia.ceph.com 2023-11-27T15:35:07.340 DEBUG:teuthology.orchestra.run.smithi102:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-27T15:35:07.363 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi150.front.sepia.ceph.com 2023-11-27T15:35:07.364 DEBUG:teuthology.orchestra.run.smithi150:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-27T15:35:07.385 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-27T15:35:07.385 DEBUG:teuthology.orchestra.run.smithi052:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-27T15:35:07.388 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-27T15:35:07.406 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-27T15:35:07.476 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-27T15:35:07.476 DEBUG:teuthology.orchestra.run.smithi052:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-27T15:35:07.479 DEBUG:teuthology.orchestra.run.smithi102:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-27T15:35:07.481 DEBUG:teuthology.orchestra.run.smithi150:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-27T15:35:07.613 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-27T15:35:07.631 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-27T15:35:07.631 DEBUG:teuthology.orchestra.run.smithi052:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-27T15:35:07.673 DEBUG:teuthology.orchestra.run.smithi102:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-27T15:35:07.700 DEBUG:teuthology.orchestra.run.smithi150:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-27T15:35:07.727 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-27T15:35:07.737 DEBUG:teuthology.orchestra.run.smithi052:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-27T15:35:07.739 DEBUG:teuthology.orchestra.run.smithi102:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-27T15:35:07.743 DEBUG:teuthology.orchestra.run.smithi150:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-27T15:35:07.766 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = core 2023-11-27T15:35:07.772 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern = core 2023-11-27T15:35:07.793 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = core 2023-11-27T15:35:07.815 DEBUG:teuthology.orchestra.run.smithi052:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-27T15:35:07.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T15:35:07.849 DEBUG:teuthology.orchestra.run.smithi102:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-27T15:35:07.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T15:35:07.865 DEBUG:teuthology.orchestra.run.smithi150:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-27T15:35:07.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T15:35:07.881 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-27T15:35:07.892 INFO:teuthology.task.internal:Transferring archived files... 2023-11-27T15:35:07.893 DEBUG:teuthology.misc:Transferring archived files from smithi052:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467810/remote/smithi052 2023-11-27T15:35:07.893 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-27T15:35:07.934 DEBUG:teuthology.misc:Transferring archived files from smithi102:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467810/remote/smithi102 2023-11-27T15:35:07.935 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-27T15:35:07.973 DEBUG:teuthology.misc:Transferring archived files from smithi150:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467810/remote/smithi150 2023-11-27T15:35:07.973 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-27T15:35:08.011 INFO:teuthology.task.internal:Removing archive directory... 2023-11-27T15:35:08.012 DEBUG:teuthology.orchestra.run.smithi052:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-27T15:35:08.014 DEBUG:teuthology.orchestra.run.smithi102:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-27T15:35:08.016 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-27T15:35:08.063 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-27T15:35:08.076 INFO:teuthology.task.internal:Not uploading archives. 2023-11-27T15:35:08.076 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-27T15:35:08.086 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-27T15:35:08.087 DEBUG:teuthology.orchestra.run.smithi052:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-27T15:35:08.089 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-27T15:35:08.091 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-27T15:35:08.103 INFO:teuthology.orchestra.run.smithi052.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 27 15:35 /home/ubuntu/cephtest 2023-11-27T15:35:08.105 INFO:teuthology.orchestra.run.smithi102.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 27 15:35 /home/ubuntu/cephtest 2023-11-27T15:35:08.119 INFO:teuthology.orchestra.run.smithi150.stdout: 262181 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 27 15:35 /home/ubuntu/cephtest 2023-11-27T15:35:08.121 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-27T15:35:08.131 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-27T15:35:08.154 INFO:teuthology.nuke:Checking targets against current locks 2023-11-27T15:35:08.179 DEBUG:teuthology.nuke:shortname: smithi052 2023-11-27T15:35:08.179 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-27T15:35:08.202 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi052.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467810', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-27 14:48:52.493518', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-27T15:35:08.206 DEBUG:teuthology.nuke:shortname: smithi102 2023-11-27T15:35:08.206 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-27T15:35:08.230 DEBUG:teuthology.nuke:shortname: smithi150 2023-11-27T15:35:08.230 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-27T15:35:08.235 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi102.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467810', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-27 14:48:52.495442', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-27T15:35:08.257 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467810', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-27 14:48:52.497152', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-27T15:35:08.266 INFO:teuthology.orchestra.console.smithi052:Power off 2023-11-27T15:35:08.267 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-27T15:35:08.291 DEBUG:teuthology.orchestra.console.smithi052:power off output: Chassis Power Control: Down/Off 2023-11-27T15:35:08.291 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-27T15:35:08.305 INFO:teuthology.orchestra.console.smithi102:Power off 2023-11-27T15:35:08.306 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-27T15:35:08.316 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is on 2023-11-27T15:35:08.326 DEBUG:teuthology.orchestra.console.smithi102:power off output: Chassis Power Control: Down/Off 2023-11-27T15:35:08.326 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-27T15:35:08.343 INFO:teuthology.orchestra.console.smithi150:Power off 2023-11-27T15:35:08.344 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-27T15:35:08.353 DEBUG:teuthology.orchestra.console.smithi102:check power output: Chassis Power is on 2023-11-27T15:35:08.365 DEBUG:teuthology.orchestra.console.smithi150:power off output: Chassis Power Control: Down/Off 2023-11-27T15:35:08.365 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-27T15:35:08.386 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2023-11-27T15:35:12.317 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-27T15:35:12.354 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-27T15:35:12.387 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-27T15:35:12.428 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is on 2023-11-27T15:35:12.463 DEBUG:teuthology.orchestra.console.smithi102:check power output: Chassis Power is on 2023-11-27T15:35:12.497 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2023-11-27T15:35:16.433 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-27T15:35:16.464 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-27T15:35:16.498 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-27T15:35:16.544 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is off 2023-11-27T15:35:16.574 DEBUG:teuthology.orchestra.console.smithi102:check power output: Chassis Power is off 2023-11-27T15:35:16.608 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is off 2023-11-27T15:35:16.646 INFO:teuthology.orchestra.console.smithi052:Power off completed 2023-11-27T15:35:16.675 INFO:teuthology.orchestra.console.smithi102:Power off completed 2023-11-27T15:35:16.709 INFO:teuthology.orchestra.console.smithi150:Power off completed 2023-11-27T15:35:16.816 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/misc}} duration: 2456.3369748592377 failure_reason: 'Command failed (workunit test fs/misc/general_vxattrs.sh) on smithi052 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh''' flavor: default owner: scheduled_choffman@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=8b59c8712d754e1481fcddae110def33 status: fail success: false 2023-11-27T15:35:16.816 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-27T15:35:16.902 INFO:teuthology.run:FAIL